commonmark-0.9.1/0000755000175000017500000000000013545663333013263 5ustar niknik00000000000000commonmark-0.9.1/.gitignore0000755000175000017500000000202113252005147015235 0ustar niknik00000000000000# Byte-compiled / optimized / DLL files __pycache__/ *.py[cod] *$py.class # C extensions *.so # Distribution / packaging .Python env/ build/ develop-eggs/ dist/ downloads/ eggs/ .eggs/ lib/ lib64/ parts/ sdist/ var/ *.egg-info/ .installed.cfg *.egg # PyInstaller # Usually these files are written by a python script from a template # before PyInstaller builds the exe, so as to inject date/other infos into it. *.manifest *.spec # Installer logs pip-log.txt pip-delete-this-directory.txt # Unit test / coverage reports htmlcov/ .tox/ .coverage .coverage.* .cache nosetests.xml coverage.xml *,cover .hypothesis/ # Translations *.mo *.pot # Django stuff: *.log local_settings.py # Flask instance folder instance/ # Scrapy stuff: .scrapy # Sphinx documentation docs/_build/ # PyBuilder target/ # IPython Notebook .ipynb_checkpoints # pyenv .python-version # celery beat schedule file celerybeat-schedule # dotenv .env # virtualenv ve/ venv/ ENV/ # Spyder project settings .spyderproject # Rope project settings .ropeproject commonmark-0.9.1/CHANGELOG.md0000644000175000017500000000524413545663315015101 0ustar niknik00000000000000## x.y.z (unreleased) ## 0.9.1 (2019-10-04) - commonmark.py now requires `future >= 0.14.0` on Python 2, for uniform `builtins` imports in Python 2/3 - Added CHANGELOG.md to the manifest file. ## 0.9.0 (2019-05-02) - The CommonMark spec has been updated to 0.29. Completed by @iamahuman. ## 0.8.1 (2018-09-06) - Removed `CommonMark` symlink. So, as of this version, you need to replace all instances of `CommonMark` with `commonmark` in your code. ## 0.8.0 (2018-09-03) - Added the `Node.normalize()` method. (from @lez) - Renamed package name to `commonmark` for PEP8 compliance. You can now do `import commonmark` as well as `import CommonMark`. Closes [#60](https://github.com/rtfd/commonmark.py/issues/60) - Added testing on Python 3.7. ## 0.7.5 (2018-03-13) - Fixed smart dashes bug in Python 3. (from @alvra) ## 0.7.4 (2017-08-05) - The CommonMark spec has been updated to 0.28. ## 0.7.3 (2017-01-05) - The CommonMark spec has been updated to 0.27. ## 0.7.2 (2016-08-10) - Removed outdated files from distributed packages, reported by @hyperknot ## 0.7.1 (2016-07-20) - Updated CommonMark spec to 0.26. - Fixed JSON output invoked by `cmark -aj somefile.md`, reported by @rplCloud - Fixed a bug with `dumpAST()` on ordered lists, from @rflynn ## 0.7.0 (2016-06-07) - The `cmark.py` script is now just called `cmark`. - Fixed a bug where the `cmark.py` script contained a bogus path, reported by @yuhui-lin. - Fixed a bug where a TypeError could occur, reported by @civalin. ## 0.6.4 (2016-05-09) - The CommonMark spec has been updated to 0.25. - The HtmlRenderer has been refactored based on upstream changes in commonmark.js. - Python 2 fixes from @eric-wieser - Contributions from @funkybob and @hyperknot ## 0.6.3 (2016-01-19) - commonmark.py now supports Python 2.6. - The CommonMark spec has been updated to 0.24. ## 0.6.2 (2016-01-08) - Fixed a UnicodeEncodeError when parsing unicode entities on Python 2. As a result, commonmark.py now relies on the "future" module in Python 2, as documented in setup.py. This can be found on pypi: https://pypi.python.org/pypi/future ## 0.6.1 (2016-01-05) - Fixed an IndexError exception that occurred when input string was empty. ## 0.6.0 (2016-01-04) - commonmark.py now complies to the 0.23 CommonMark spec http://spec.commonmark.org/0.23/ - The ExtensionBlock has been removed in this release, since the parser has been rewritten. - Added a compatibility fix for Python 2.6, but this version of Python still isn't really supported. - `HTMLRenderer` has been renamed to `HtmlRenderer`. - `DocParser` has been renamed to `Parser`. ## 0.5.5 (2015-12-18) - Random bug fixes - Internal code structure changes - Compatibility fixes for Python 3 commonmark-0.9.1/LICENSE0000644000175000017500000000315513252005147014260 0ustar niknik00000000000000Copyright (c) 2014, Bibek Kafle and Roland Shoemaker Based on stmd.js: Copyright (c) 2014, John MacFarlane All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the names of Bibek Kafle, Roland Shoemaker nor the names of other contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.commonmark-0.9.1/MANIFEST.in0000644000175000017500000000106513545662472015026 0ustar niknik00000000000000include README.rst include CHANGELOG.md include LICENSE include .gitignore include spec.txt include commonmark/__init__.py include commonmark/blocks.py include commonmark/common.py include commonmark/dump.py include commonmark/entitytrans.py include commonmark/inlines.py include commonmark/main.py include commonmark/node.py include commonmark/utils.py include commonmark/render/__init__.py include commonmark/render/renderer.py include commonmark/render/html.py include commonmark/tests/run_spec_tests.py include commonmark/tests/unit_tests.py commonmark-0.9.1/PKG-INFO0000644000175000017500000001502313545663333014361 0ustar niknik00000000000000Metadata-Version: 2.1 Name: commonmark Version: 0.9.1 Summary: Python parser for the CommonMark Markdown spec Home-page: https://github.com/rtfd/commonmark.py Author: Bibek Kafle , Roland Shoemaker Author-email: rolandshoemaker@gmail.com Maintainer: Nikolas Nyby Maintainer-email: nikolas@gnu.org License: BSD-3-Clause Description: commonmark.py ============= commonmark.py is a pure Python port of `jgm `__'s `commonmark.js `__, a Markdown parser and renderer for the `CommonMark `__ specification, using only native modules. Once both this project and the CommonMark specification are stable we will release the first ``1.0`` version and attempt to keep up to date with changes in ``commonmark.js``. commonmark.py is tested against the CommonMark spec with Python versions 2.7, 3.4, 3.5, 3.6, and 3.7. **Current version:** 0.9.1 |Pypi Link| |Build Status| |Doc Link| Installation ------------ :: $ pip install commonmark Usage ----- :: >>> import commonmark >>> commonmark.commonmark('*hello!*') '

hello!

\n' Or, without the syntactic sugar: .. code:: python import commonmark parser = commonmark.Parser() ast = parser.parse("Hello *World*") renderer = commonmark.HtmlRenderer() html = renderer.render(ast) print(html) #

Hello World

# inspecting the abstract syntax tree json = commonmark.dumpJSON(ast) commonmark.dumpAST(ast) # pretty print generated AST structure There is also a CLI: :: $ cmark README.md -o README.html $ cmark README.md -o README.json -aj # output AST as JSON $ cmark README.md -a # pretty print generated AST structure $ cmark -h usage: cmark [-h] [-o [O]] [-a] [-aj] [infile] Process Markdown according to the CommonMark specification. positional arguments: infile Input Markdown file to parse, defaults to stdin optional arguments: -h, --help show this help message and exit -o [O] Output HTML/JSON file, defaults to stdout -a Print formatted AST -aj Output JSON AST Contributing ------------ If you would like to offer suggestions/optimizations/bugfixes through pull requests please do! Also if you find an error in the parser/renderer that isn't caught by the current test suite please open a new issue and I would also suggest you send the `commonmark.js `__ project a pull request adding your test to the existing test suite. Tests ----- To work on commonmark.py, you will need to be able to run the test suite to make sure your changes don't break anything. To run the tests, you can do something like this: :: $ pyvenv venv $ ./venv/bin/python setup.py develop test The tests script, ``commonmark/tests/run_spec_tests.py``, is pretty much a devtool. As well as running all the tests embedded in ``spec.txt`` it also allows you to run specific tests using the ``-t`` argument, provide information about passed tests with ``-p``, percentage passed by category of test with ``-s``, and enter markdown interactively with ``-i`` (In interactive mode end a block by inputting a line with just ``end``, to quit do the same but with ``quit``). ``-d`` can be used to print call tracing. :: $ ./venv/bin/python commonmark/tests/run_spec_tests.py -h usage: run_spec_tests.py [-h] [-t T] [-p] [-f] [-i] [-d] [-np] [-s] script to run the CommonMark specification tests against the commonmark.py parser. optional arguments: -h, --help show this help message and exit -t T Single test to run or comma separated list of tests (-t 10 or -t 10,11,12,13) -p Print passed test information -f Print failed tests (during -np...) -i Interactive Markdown input mode -d Debug, trace calls -np Only print section header, tick, or cross -s Print percent of tests passed by category Authors ------- - `Bibek Kafle `__ - `Roland Shoemaker `__ - `Nikolas Nyby `__ .. |Pypi Link| image:: https://img.shields.io/pypi/v/commonmark.svg :target: https://pypi.org/project/commonmark/ .. |Build Status| image:: https://travis-ci.org/rtfd/commonmark.py.svg?branch=master :target: https://travis-ci.org/rtfd/commonmark.py .. |Doc Link| image:: https://readthedocs.org/projects/commonmarkpy/badge/?version=latest :target: https://commonmarkpy.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status Keywords: markup,markdown,commonmark Platform: UNKNOWN Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 3 Classifier: Development Status :: 5 - Production/Stable Classifier: Environment :: Other Environment Classifier: Environment :: Console Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Topic :: Documentation Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content Classifier: Topic :: Software Development :: Documentation Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: Topic :: Text Processing :: Markup Classifier: Topic :: Utilities Description-Content-Type: text/x-rst Provides-Extra: test commonmark-0.9.1/README.rst0000644000175000017500000001041113545663315014747 0ustar niknik00000000000000commonmark.py ============= commonmark.py is a pure Python port of `jgm `__'s `commonmark.js `__, a Markdown parser and renderer for the `CommonMark `__ specification, using only native modules. Once both this project and the CommonMark specification are stable we will release the first ``1.0`` version and attempt to keep up to date with changes in ``commonmark.js``. commonmark.py is tested against the CommonMark spec with Python versions 2.7, 3.4, 3.5, 3.6, and 3.7. **Current version:** 0.9.1 |Pypi Link| |Build Status| |Doc Link| Installation ------------ :: $ pip install commonmark Usage ----- :: >>> import commonmark >>> commonmark.commonmark('*hello!*') '

hello!

\n' Or, without the syntactic sugar: .. code:: python import commonmark parser = commonmark.Parser() ast = parser.parse("Hello *World*") renderer = commonmark.HtmlRenderer() html = renderer.render(ast) print(html) #

Hello World

# inspecting the abstract syntax tree json = commonmark.dumpJSON(ast) commonmark.dumpAST(ast) # pretty print generated AST structure There is also a CLI: :: $ cmark README.md -o README.html $ cmark README.md -o README.json -aj # output AST as JSON $ cmark README.md -a # pretty print generated AST structure $ cmark -h usage: cmark [-h] [-o [O]] [-a] [-aj] [infile] Process Markdown according to the CommonMark specification. positional arguments: infile Input Markdown file to parse, defaults to stdin optional arguments: -h, --help show this help message and exit -o [O] Output HTML/JSON file, defaults to stdout -a Print formatted AST -aj Output JSON AST Contributing ------------ If you would like to offer suggestions/optimizations/bugfixes through pull requests please do! Also if you find an error in the parser/renderer that isn't caught by the current test suite please open a new issue and I would also suggest you send the `commonmark.js `__ project a pull request adding your test to the existing test suite. Tests ----- To work on commonmark.py, you will need to be able to run the test suite to make sure your changes don't break anything. To run the tests, you can do something like this: :: $ pyvenv venv $ ./venv/bin/python setup.py develop test The tests script, ``commonmark/tests/run_spec_tests.py``, is pretty much a devtool. As well as running all the tests embedded in ``spec.txt`` it also allows you to run specific tests using the ``-t`` argument, provide information about passed tests with ``-p``, percentage passed by category of test with ``-s``, and enter markdown interactively with ``-i`` (In interactive mode end a block by inputting a line with just ``end``, to quit do the same but with ``quit``). ``-d`` can be used to print call tracing. :: $ ./venv/bin/python commonmark/tests/run_spec_tests.py -h usage: run_spec_tests.py [-h] [-t T] [-p] [-f] [-i] [-d] [-np] [-s] script to run the CommonMark specification tests against the commonmark.py parser. optional arguments: -h, --help show this help message and exit -t T Single test to run or comma separated list of tests (-t 10 or -t 10,11,12,13) -p Print passed test information -f Print failed tests (during -np...) -i Interactive Markdown input mode -d Debug, trace calls -np Only print section header, tick, or cross -s Print percent of tests passed by category Authors ------- - `Bibek Kafle `__ - `Roland Shoemaker `__ - `Nikolas Nyby `__ .. |Pypi Link| image:: https://img.shields.io/pypi/v/commonmark.svg :target: https://pypi.org/project/commonmark/ .. |Build Status| image:: https://travis-ci.org/rtfd/commonmark.py.svg?branch=master :target: https://travis-ci.org/rtfd/commonmark.py .. |Doc Link| image:: https://readthedocs.org/projects/commonmarkpy/badge/?version=latest :target: https://commonmarkpy.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status commonmark-0.9.1/commonmark/0000755000175000017500000000000013545663333015426 5ustar niknik00000000000000commonmark-0.9.1/commonmark/__init__.py0000644000175000017500000000045613343503214017527 0ustar niknik00000000000000# flake8: noqa from __future__ import unicode_literals, absolute_import from commonmark.main import commonmark from commonmark.dump import dumpAST, dumpJSON from commonmark.blocks import Parser from commonmark.render.html import HtmlRenderer from commonmark.render.rst import ReStructuredTextRenderer commonmark-0.9.1/commonmark/blocks.py0000644000175000017500000007331313462635760017265 0ustar niknik00000000000000from __future__ import absolute_import, unicode_literals import re from commonmark import common from commonmark.common import unescape_string from commonmark.inlines import InlineParser from commonmark.node import Node CODE_INDENT = 4 reHtmlBlockOpen = [ re.compile(r'.'), # dummy for 0 re.compile(r'^<(?:script|pre|style)(?:\s|>|$)', re.IGNORECASE), re.compile(r'^'), re.compile(r'\?>'), re.compile(r'>'), re.compile(r'\]\]>'), ] reThematicBreak = re.compile( r'^(?:(?:\*[ \t]*){3,}|(?:_[ \t]*){3,}|(?:-[ \t]*){3,})[ \t]*$') reMaybeSpecial = re.compile(r'^[#`~*+_=<>0-9-]') reNonSpace = re.compile(r'[^ \t\f\v\r\n]') reBulletListMarker = re.compile(r'^[*+-]') reOrderedListMarker = re.compile(r'^(\d{1,9})([.)])') reATXHeadingMarker = re.compile(r'^#{1,6}(?:[ \t]+|$)') reCodeFence = re.compile(r'^`{3,}(?!.*`)|^~{3,}') reClosingCodeFence = re.compile(r'^(?:`{3,}|~{3,})(?= *$)') reSetextHeadingLine = re.compile(r'^(?:=+|-+)[ \t]*$') reLineEnding = re.compile(r'\r\n|\n|\r') def is_blank(s): """Returns True if string contains only space characters.""" return re.search(reNonSpace, s) is None def is_space_or_tab(s): return s in (' ', '\t') def peek(ln, pos): if pos < len(ln): return ln[pos] else: return None def ends_with_blank_line(block): """ Returns true if block ends with a blank line, descending if needed into lists and sublists.""" while block: if block.last_line_blank: return True if not block.last_line_checked and \ block.t in ('list', 'item'): block.last_line_checked = True block = block.last_child else: block.last_line_checked = True break return False def parse_list_marker(parser, container): """ Parse a list marker and return data on the marker (type, start, delimiter, bullet character, padding) or None.""" rest = parser.current_line[parser.next_nonspace:] data = { 'type': None, 'tight': True, # lists are tight by default 'bullet_char': None, 'start': None, 'delimiter': None, 'padding': None, 'marker_offset': parser.indent, } if parser.indent >= 4: return None m = re.search(reBulletListMarker, rest) m2 = re.search(reOrderedListMarker, rest) if m: data['type'] = 'bullet' data['bullet_char'] = m.group()[0] elif m2 and (container.t != 'paragraph' or m2.group(1) == '1'): m = m2 data['type'] = 'ordered' data['start'] = int(m.group(1)) data['delimiter'] = m.group(2) else: return None # make sure we have spaces after nextc = peek(parser.current_line, parser.next_nonspace + len(m.group())) if not (nextc is None or nextc == '\t' or nextc == ' '): return None # if it interrupts paragraph, make sure first line isn't blank if container.t == 'paragraph' and \ not re.search( reNonSpace, parser.current_line[parser.next_nonspace + len(m.group()):]): return None # we've got a match! advance offset and calculate padding parser.advance_next_nonspace() # to start of marker parser.advance_offset(len(m.group()), True) # to end of marker spaces_start_col = parser.column spaces_start_offset = parser.offset while True: parser.advance_offset(1, True) nextc = peek(parser.current_line, parser.offset) if parser.column - spaces_start_col < 5 and \ is_space_or_tab(nextc): pass else: break blank_item = peek(parser.current_line, parser.offset) is None spaces_after_marker = parser.column - spaces_start_col if spaces_after_marker >= 5 or \ spaces_after_marker < 1 or \ blank_item: data['padding'] = len(m.group()) + 1 parser.column = spaces_start_col parser.offset = spaces_start_offset if is_space_or_tab(peek(parser.current_line, parser.offset)): parser.advance_offset(1, True) else: data['padding'] = len(m.group()) + spaces_after_marker return data def lists_match(list_data, item_data): """ Returns True if the two list items are of the same type, with the same delimiter and bullet character. This is used in agglomerating list items into lists. """ return list_data.get('type') == item_data.get('type') and \ list_data.get('delimiter') == item_data.get('delimiter') and \ list_data.get('bullet_char') == item_data.get('bullet_char') class Block(object): accepts_lines = None @staticmethod def continue_(parser=None, container=None): return @staticmethod def finalize(parser=None, block=None): return @staticmethod def can_contain(t): return class Document(Block): accepts_lines = False @staticmethod def continue_(parser=None, container=None): return 0 @staticmethod def finalize(parser=None, block=None): return @staticmethod def can_contain(t): return t != 'item' class List(Block): accepts_lines = False @staticmethod def continue_(parser=None, container=None): return 0 @staticmethod def finalize(parser=None, block=None): item = block.first_child while item: # check for non-final list item ending with blank line: if ends_with_blank_line(item) and item.nxt: block.list_data['tight'] = False break # recurse into children of list item, to see if there are # spaces between any of them: subitem = item.first_child while subitem: if ends_with_blank_line(subitem) and \ (item.nxt or subitem.nxt): block.list_data['tight'] = False break subitem = subitem.nxt item = item.nxt @staticmethod def can_contain(t): return t == 'item' class BlockQuote(Block): accepts_lines = False @staticmethod def continue_(parser=None, container=None): ln = parser.current_line if not parser.indented and peek(ln, parser.next_nonspace) == '>': parser.advance_next_nonspace() parser.advance_offset(1, False) if is_space_or_tab(peek(ln, parser.offset)): parser.advance_offset(1, True) else: return 1 return 0 @staticmethod def finalize(parser=None, block=None): return @staticmethod def can_contain(t): return t != 'item' class Item(Block): accepts_lines = False @staticmethod def continue_(parser=None, container=None): if parser.blank: if container.first_child is None: # Blank line after empty list item return 1 else: parser.advance_next_nonspace() elif parser.indent >= (container.list_data['marker_offset'] + container.list_data['padding']): parser.advance_offset( container.list_data['marker_offset'] + container.list_data['padding'], True) else: return 1 return 0 @staticmethod def finalize(parser=None, block=None): return @staticmethod def can_contain(t): return t != 'item' class Heading(Block): accepts_lines = False @staticmethod def continue_(parser=None, container=None): # A heading can never container > 1 line, so fail to match: return 1 @staticmethod def finalize(parser=None, block=None): return @staticmethod def can_contain(t): return False class ThematicBreak(Block): accepts_lines = False @staticmethod def continue_(parser=None, container=None): # A thematic break can never container > 1 line, so fail to match: return 1 @staticmethod def finalize(parser=None, block=None): return @staticmethod def can_contain(t): return False class CodeBlock(Block): accepts_lines = True @staticmethod def continue_(parser=None, container=None): ln = parser.current_line indent = parser.indent if container.is_fenced: match = indent <= 3 and \ len(ln) >= parser.next_nonspace + 1 and \ ln[parser.next_nonspace] == container.fence_char and \ re.search(reClosingCodeFence, ln[parser.next_nonspace:]) if match and len(match.group()) >= container.fence_length: # closing fence - we're at end of line, so we can return parser.finalize(container, parser.line_number) return 2 else: # skip optional spaces of fence offset i = container.fence_offset while i > 0 and is_space_or_tab(peek(ln, parser.offset)): parser.advance_offset(1, True) i -= 1 else: # indented if indent >= CODE_INDENT: parser.advance_offset(CODE_INDENT, True) elif parser.blank: parser.advance_next_nonspace() else: return 1 return 0 @staticmethod def finalize(parser=None, block=None): if block.is_fenced: # first line becomes info string content = block.string_content newline_pos = content.index('\n') first_line = content[0:newline_pos] rest = content[newline_pos + 1:] block.info = unescape_string(first_line.strip()) block.literal = rest else: # indented block.literal = re.sub(r'(\n *)+$', '\n', block.string_content) block.string_content = None @staticmethod def can_contain(t): return False class HtmlBlock(Block): accepts_lines = True @staticmethod def continue_(parser=None, container=None): if parser.blank and (container.html_block_type == 6 or container.html_block_type == 7): return 1 else: return 0 @staticmethod def finalize(parser=None, block=None): block.literal = re.sub(r'(\n *)+$', '', block.string_content) # allow GC block.string_content = None @staticmethod def can_contain(t): return False class Paragraph(Block): accepts_lines = True @staticmethod def continue_(parser=None, container=None): return 1 if parser.blank else 0 @staticmethod def finalize(parser=None, block=None): has_reference_defs = False # try parsing the beginning as link reference definitions: while peek(block.string_content, 0) == '[': pos = parser.inline_parser.parseReference( block.string_content, parser.refmap) if not pos: break block.string_content = block.string_content[pos:] has_reference_defs = True if has_reference_defs and is_blank(block.string_content): block.unlink() @staticmethod def can_contain(t): return False class BlockStarts(object): """Block start functions. Return values: 0 = no match 1 = matched container, keep going 2 = matched leaf, no more block starts """ METHODS = [ 'block_quote', 'atx_heading', 'fenced_code_block', 'html_block', 'setext_heading', 'thematic_break', 'list_item', 'indented_code_block', ] @staticmethod def block_quote(parser, container=None): if not parser.indented and \ peek(parser.current_line, parser.next_nonspace) == '>': parser.advance_next_nonspace() parser.advance_offset(1, False) # optional following space if is_space_or_tab(peek(parser.current_line, parser.offset)): parser.advance_offset(1, True) parser.close_unmatched_blocks() parser.add_child('block_quote', parser.next_nonspace) return 1 return 0 @staticmethod def atx_heading(parser, container=None): if not parser.indented: m = re.search(reATXHeadingMarker, parser.current_line[parser.next_nonspace:]) if m: parser.advance_next_nonspace() parser.advance_offset(len(m.group()), False) parser.close_unmatched_blocks() container = parser.add_child('heading', parser.next_nonspace) # number of #s container.level = len(m.group().strip()) # remove trailing ###s: container.string_content = re.sub( r'[ \t]+#+[ \t]*$', '', re.sub( r'^[ \t]*#+[ \t]*$', '', parser.current_line[parser.offset:])) parser.advance_offset( len(parser.current_line) - parser.offset, False) return 2 return 0 @staticmethod def fenced_code_block(parser, container=None): if not parser.indented: m = re.search( reCodeFence, parser.current_line[parser.next_nonspace:]) if m: fence_length = len(m.group()) parser.close_unmatched_blocks() container = parser.add_child( 'code_block', parser.next_nonspace) container.is_fenced = True container.fence_length = fence_length container.fence_char = m.group()[0] container.fence_offset = parser.indent parser.advance_next_nonspace() parser.advance_offset(fence_length, False) return 2 return 0 @staticmethod def html_block(parser, container=None): if not parser.indented and \ peek(parser.current_line, parser.next_nonspace) == '<': s = parser.current_line[parser.next_nonspace:] for block_type in range(1, 8): if re.search(reHtmlBlockOpen[block_type], s) and \ (block_type < 7 or container.t != 'paragraph'): parser.close_unmatched_blocks() # We don't adjust parser.offset; # spaces are part of the HTML block: b = parser.add_child('html_block', parser.offset) b.html_block_type = block_type return 2 return 0 @staticmethod def setext_heading(parser, container=None): if not parser.indented and container.t == 'paragraph': m = re.search( reSetextHeadingLine, parser.current_line[parser.next_nonspace:]) if m: parser.close_unmatched_blocks() # resolve reference link definitiosn while peek(container.string_content, 0) == '[': pos = parser.inline_parser.parseReference( container.string_content, parser.refmap) if not pos: break container.string_content = container.string_content[pos:] if container.string_content: heading = Node('heading', container.sourcepos) heading.level = 1 if m.group()[0] == '=' else 2 heading.string_content = container.string_content container.insert_after(heading) container.unlink() parser.tip = heading parser.advance_offset( len(parser.current_line) - parser.offset, False) return 2 else: return 0 return 0 @staticmethod def thematic_break(parser, container=None): if not parser.indented and re.search( reThematicBreak, parser.current_line[parser.next_nonspace:]): parser.close_unmatched_blocks() parser.add_child('thematic_break', parser.next_nonspace) parser.advance_offset( len(parser.current_line) - parser.offset, False) return 2 return 0 @staticmethod def list_item(parser, container=None): if (not parser.indented or container.t == 'list'): data = parse_list_marker(parser, container) if data: parser.close_unmatched_blocks() # add the list if needed if parser.tip.t != 'list' or \ not lists_match(container.list_data, data): container = parser.add_child('list', parser.next_nonspace) container.list_data = data # add the list item container = parser.add_child('item', parser.next_nonspace) container.list_data = data return 1 return 0 @staticmethod def indented_code_block(parser, container=None): if parser.indented and \ parser.tip.t != 'paragraph' and \ not parser.blank: # indented code parser.advance_offset(CODE_INDENT, True) parser.close_unmatched_blocks() parser.add_child('code_block', parser.offset) return 2 return 0 class Parser(object): def __init__(self, options={}): self.doc = Node('document', [[1, 1], [0, 0]]) self.block_starts = BlockStarts() self.tip = self.doc self.oldtip = self.doc self.current_line = '' self.line_number = 0 self.offset = 0 self.column = 0 self.next_nonspace = 0 self.next_nonspace_column = 0 self.indent = 0 self.indented = False self.blank = False self.partially_consumed_tab = False self.all_closed = True self.last_matched_container = self.doc self.refmap = {} self.last_line_length = 0 self.inline_parser = InlineParser(options) self.options = options def add_line(self): """ Add a line to the block at the tip. We assume the tip can accept lines -- that check should be done before calling this.""" if self.partially_consumed_tab: # Skip over tab self.offset += 1 # Add space characters chars_to_tab = 4 - (self.column % 4) self.tip.string_content += (' ' * chars_to_tab) self.tip.string_content += (self.current_line[self.offset:] + '\n') def add_child(self, tag, offset): """ Add block of type tag as a child of the tip. If the tip can't accept children, close and finalize it and try its parent, and so on til we find a block that can accept children.""" while not self.blocks[self.tip.t].can_contain(tag): self.finalize(self.tip, self.line_number - 1) column_number = offset + 1 new_block = Node(tag, [[self.line_number, column_number], [0, 0]]) new_block.string_content = '' self.tip.append_child(new_block) self.tip = new_block return new_block def close_unmatched_blocks(self): """Finalize and close any unmatched blocks.""" if not self.all_closed: while self.oldtip != self.last_matched_container: parent = self.oldtip.parent self.finalize(self.oldtip, self.line_number - 1) self.oldtip = parent self.all_closed = True def find_next_nonspace(self): current_line = self.current_line i = self.offset cols = self.column try: c = current_line[i] except IndexError: c = '' while c != '': if c == ' ': i += 1 cols += 1 elif c == '\t': i += 1 cols += (4 - (cols % 4)) else: break try: c = current_line[i] except IndexError: c = '' self.blank = (c == '\n' or c == '\r' or c == '') self.next_nonspace = i self.next_nonspace_column = cols self.indent = self.next_nonspace_column - self.column self.indented = self.indent >= CODE_INDENT def advance_next_nonspace(self): self.offset = self.next_nonspace self.column = self.next_nonspace_column self.partially_consumed_tab = False def advance_offset(self, count, columns): current_line = self.current_line try: c = current_line[self.offset] except IndexError: c = None while count > 0 and c is not None: if c == '\t': chars_to_tab = 4 - (self.column % 4) if columns: self.partially_consumed_tab = chars_to_tab > count chars_to_advance = min(count, chars_to_tab) self.column += chars_to_advance self.offset += 0 if self.partially_consumed_tab else 1 count -= chars_to_advance else: self.partially_consumed_tab = False self.column += chars_to_tab self.offset += 1 count -= 1 else: self.partially_consumed_tab = False self.offset += 1 # assume ascii; block starts are ascii self.column += 1 count -= 1 try: c = current_line[self.offset] except IndexError: c = None def incorporate_line(self, ln): """Analyze a line of text and update the document appropriately. We parse markdown text by calling this on each line of input, then finalizing the document. """ all_matched = True container = self.doc self.oldtip = self.tip self.offset = 0 self.column = 0 self.blank = False self.partially_consumed_tab = False self.line_number += 1 # replace NUL characters for security if re.search(r'\u0000', ln) is not None: ln = re.sub(r'\0', '\uFFFD', ln) self.current_line = ln # For each containing block, try to parse the associated line start. # Bail out on failure: container will point to the last matching block. # Set all_matched to false if not all containers match. while True: last_child = container.last_child if not (last_child and last_child.is_open): break container = last_child self.find_next_nonspace() rv = self.blocks[container.t].continue_(self, container) if rv == 0: # we've matched, keep going pass elif rv == 1: # we've failed to match a block all_matched = False elif rv == 2: # we've hit end of line for fenced code close and can return self.last_line_length = len(ln) return else: raise ValueError( 'continue_ returned illegal value, must be 0, 1, or 2') if not all_matched: # back up to last matching block container = container.parent break self.all_closed = (container == self.oldtip) self.last_matched_container = container matched_leaf = container.t != 'paragraph' and \ self.blocks[container.t].accepts_lines starts = self.block_starts starts_len = len(starts.METHODS) # Unless last matched container is a code block, try new container # starts, adding children to the last matched container: while not matched_leaf: self.find_next_nonspace() # this is a little performance optimization: if not self.indented and \ not re.search(reMaybeSpecial, ln[self.next_nonspace:]): self.advance_next_nonspace() break i = 0 while i < starts_len: res = getattr(starts, starts.METHODS[i])(self, container) if res == 1: container = self.tip break elif res == 2: container = self.tip matched_leaf = True break else: i += 1 if i == starts_len: # nothing matched self.advance_next_nonspace() break # What remains at the offset is a text line. Add the text to the # appropriate container. if not self.all_closed and not self.blank and \ self.tip.t == 'paragraph': # lazy paragraph continuation self.add_line() else: # not a lazy continuation # finalize any blocks not matched self.close_unmatched_blocks() if self.blank and container.last_child: container.last_child.last_line_blank = True t = container.t # Block quote lines are never blank as they start with > # and we don't count blanks in fenced code for purposes of # tight/loose lists or breaking out of lists. We also # don't set last_line_blank on an empty list item, or if we # just closed a fenced block. last_line_blank = self.blank and \ not (t == 'block_quote' or (t == 'code_block' and container.is_fenced) or (t == 'item' and not container.first_child and container.sourcepos[0][0] == self.line_number)) # propagate last_line_blank up through parents: cont = container while cont: cont.last_line_blank = last_line_blank cont = cont.parent if self.blocks[t].accepts_lines: self.add_line() # if HtmlBlock, check for end condition if t == 'html_block' and \ container.html_block_type >= 1 and \ container.html_block_type <= 5 and \ re.search( reHtmlBlockClose[container.html_block_type], self.current_line[self.offset:]): self.finalize(container, self.line_number) elif self.offset < len(ln) and not self.blank: # create a paragraph container for one line container = self.add_child('paragraph', self.offset) self.advance_next_nonspace() self.add_line() self.last_line_length = len(ln) def finalize(self, block, line_number): """ Finalize a block. Close it and do any necessary postprocessing, e.g. creating string_content from strings, setting the 'tight' or 'loose' status of a list, and parsing the beginnings of paragraphs for reference definitions. Reset the tip to the parent of the closed block.""" above = block.parent block.is_open = False block.sourcepos[1] = [line_number, self.last_line_length] self.blocks[block.t].finalize(self, block) self.tip = above def process_inlines(self, block): """ Walk through a block & children recursively, parsing string content into inline content where appropriate. """ walker = block.walker() self.inline_parser.refmap = self.refmap self.inline_parser.options = self.options event = walker.nxt() while event is not None: node = event['node'] t = node.t if not event['entering'] and (t == 'paragraph' or t == 'heading'): self.inline_parser.parse(node) event = walker.nxt() def parse(self, my_input): """ The main parsing function. Returns a parsed document AST.""" self.doc = Node('document', [[1, 1], [0, 0]]) self.tip = self.doc self.refmap = {} self.line_number = 0 self.last_line_length = 0 self.offset = 0 self.column = 0 self.last_matched_container = self.doc self.current_line = '' lines = re.split(reLineEnding, my_input) length = len(lines) if len(my_input) > 0 and my_input[-1] == '\n': # ignore last blank line created by final newline length -= 1 for i in range(length): self.incorporate_line(lines[i]) while (self.tip): self.finalize(self.tip, length) self.process_inlines(self.doc) return self.doc CAMEL_RE = re.compile("(.)([A-Z](?:[a-z]+|(?<=[a-z0-9].)))") Parser.blocks = dict( (CAMEL_RE.sub(r'\1_\2', cls.__name__).lower(), cls) for cls in Block.__subclasses__()) commonmark-0.9.1/commonmark/cmark.py0000755000175000017500000000262613343503214017071 0ustar niknik00000000000000#!/usr/bin/env python from __future__ import unicode_literals import argparse import sys import commonmark def main(): parser = argparse.ArgumentParser( description="Process Markdown according to " "the CommonMark specification.") if sys.version_info < (3, 0): reload(sys) # noqa sys.setdefaultencoding('utf-8') parser.add_argument( 'infile', nargs="?", type=argparse.FileType('r'), default=sys.stdin, help="Input Markdown file to parse, defaults to STDIN") parser.add_argument( '-o', nargs="?", type=argparse.FileType('w'), default=sys.stdout, help="Output HTML/JSON file, defaults to STDOUT") parser.add_argument('-a', action="store_true", help="Print formatted AST") parser.add_argument('-aj', action="store_true", help="Output JSON AST") args = parser.parse_args() parser = commonmark.Parser() f = args.infile o = args.o lines = [] for line in f: lines.append(line) data = "".join(lines) ast = parser.parse(data) if not args.a and not args.aj: renderer = commonmark.HtmlRenderer() o.write(renderer.render(ast)) exit() if args.a: # print ast commonmark.dumpAST(ast) exit() # o.write(ast.to_JSON()) o.write(commonmark.dumpJSON(ast)) exit() if __name__ == '__main__': main() commonmark-0.9.1/commonmark/common.py0000644000175000017500000000635513462635760017302 0ustar niknik00000000000000from __future__ import absolute_import, unicode_literals import re import sys try: from urllib.parse import quote except ImportError: from urllib import quote if sys.version_info >= (3, 0): if sys.version_info >= (3, 4): import html HTMLunescape = html.unescape else: from .entitytrans import _unescape HTMLunescape = _unescape else: from commonmark import entitytrans HTMLunescape = entitytrans._unescape ENTITY = '&(?:#x[a-f0-9]{1,6}|#[0-9]{1,7}|[a-z][a-z0-9]{1,31});' TAGNAME = '[A-Za-z][A-Za-z0-9-]*' ATTRIBUTENAME = '[a-zA-Z_:][a-zA-Z0-9:._-]*' UNQUOTEDVALUE = "[^\"'=<>`\\x00-\\x20]+" SINGLEQUOTEDVALUE = "'[^']*'" DOUBLEQUOTEDVALUE = '"[^"]*"' ATTRIBUTEVALUE = "(?:" + UNQUOTEDVALUE + "|" + SINGLEQUOTEDVALUE + \ "|" + DOUBLEQUOTEDVALUE + ")" ATTRIBUTEVALUESPEC = "(?:" + "\\s*=" + "\\s*" + ATTRIBUTEVALUE + ")" ATTRIBUTE = "(?:" + "\\s+" + ATTRIBUTENAME + ATTRIBUTEVALUESPEC + "?)" OPENTAG = "<" + TAGNAME + ATTRIBUTE + "*" + "\\s*/?>" CLOSETAG = "]" HTMLCOMMENT = '|' PROCESSINGINSTRUCTION = "[<][?].*?[?][>]" DECLARATION = "]*>" CDATA = '' HTMLTAG = "(?:" + OPENTAG + "|" + CLOSETAG + "|" + HTMLCOMMENT + "|" + \ PROCESSINGINSTRUCTION + "|" + DECLARATION + "|" + CDATA + ")" reHtmlTag = re.compile('^' + HTMLTAG, re.IGNORECASE) reBackslashOrAmp = re.compile(r'[\\&]') ESCAPABLE = '[!"#$%&\'()*+,./:;<=>?@[\\\\\\]^_`{|}~-]' reEntityOrEscapedChar = re.compile( '\\\\' + ESCAPABLE + '|' + ENTITY, re.IGNORECASE) XMLSPECIAL = '[&<>"]' reXmlSpecial = re.compile(XMLSPECIAL) def unescape_char(s): if s[0] == '\\': return s[1] else: return HTMLunescape(s) def unescape_string(s): """Replace entities and backslash escapes with literal characters.""" if re.search(reBackslashOrAmp, s): return re.sub( reEntityOrEscapedChar, lambda m: unescape_char(m.group()), s) else: return s def normalize_uri(uri): try: return quote(uri.encode('utf-8'), safe=str('/@:+?=&()%#*,')) except UnicodeDecodeError: # Python 2 also throws a UnicodeDecodeError, complaining about # the width of the "safe" string. Removing this parameter # solves the issue, but yields overly aggressive quoting, but we # can correct those errors manually. s = quote(uri.encode('utf-8')) s = re.sub(r'%40', '@', s) s = re.sub(r'%3A', ':', s) s = re.sub(r'%2B', '+', s) s = re.sub(r'%3F', '?', s) s = re.sub(r'%3D', '=', s) s = re.sub(r'%26', '&', s) s = re.sub(r'%28', '(', s) s = re.sub(r'%29', ')', s) s = re.sub(r'%25', '%', s) s = re.sub(r'%23', '#', s) s = re.sub(r'%2A', '*', s) s = re.sub(r'%2C', ',', s) return s UNSAFE_MAP = { '&': '&', '<': '<', '>': '>', '"': '"', } def replace_unsafe_char(s): return UNSAFE_MAP.get(s, s) def escape_xml(s): if s is None: return '' if re.search(reXmlSpecial, s): return re.sub( reXmlSpecial, lambda m: replace_unsafe_char(m.group()), s) else: return s commonmark-0.9.1/commonmark/dump.py0000644000175000017500000000707413343503214016740 0ustar niknik00000000000000from __future__ import absolute_import, unicode_literals from builtins import str import json from commonmark.node import is_container def prepare(obj, topnode=False): """Walk the complete AST, only returning needed data. This removes circular references and allows us to output JSON. """ a = [] for subnode, entered in obj.walker(): rep = { 'type': subnode.t, } if subnode.literal: rep['literal'] = subnode.literal if subnode.string_content: rep['string_content'] = subnode.string_content if subnode.title: rep['title'] = subnode.title if subnode.info: rep['info'] = subnode.info if subnode.destination: rep['destination'] = subnode.destination if subnode.list_data: rep['list_data'] = subnode.list_data if is_container(subnode): rep['children'] = [] if entered and len(a) > 0: if a[-1]['children']: a[-1]['children'].append(rep) else: a[-1]['children'] = [rep] else: a.append(rep) return a def dumpJSON(obj): """Output AST in JSON form, this is destructive of block.""" prepared = prepare(obj) return json.dumps(prepared, indent=4, sort_keys=True) def dumpAST(obj, ind=0, topnode=False): """Print out a block/entire AST.""" indChar = ("\t" * ind) + "-> " if ind else "" print(indChar + "[" + obj.t + "]") if not obj.title == "": print("\t" + indChar + "Title: " + (obj.title or '')) if not obj.info == "": print("\t" + indChar + "Info: " + (obj.info or '')) if not obj.destination == "": print("\t" + indChar + "Destination: " + (obj.destination or '')) if obj.is_open: print("\t" + indChar + "Open: " + str(obj.is_open)) if obj.last_line_blank: print( "\t" + indChar + "Last line blank: " + str(obj.last_line_blank)) if obj.sourcepos: print("\t" + indChar + "Sourcepos: " + str(obj.sourcepos)) if not obj.string_content == "": print("\t" + indChar + "String content: " + (obj.string_content or '')) if not obj.info == "": print("\t" + indChar + "Info: " + (obj.info or '')) if not obj.literal == "": print("\t" + indChar + "Literal: " + (obj.literal or '')) if obj.list_data.get('type'): print("\t" + indChar + "List Data: ") print("\t\t" + indChar + "[type] = " + obj.list_data.get('type')) if obj.list_data.get('bullet_char'): print( "\t\t" + indChar + "[bullet_char] = " + obj.list_data['bullet_char']) if obj.list_data.get('start'): print( "\t\t" + indChar + "[start] = " + str(obj.list_data.get('start'))) if obj.list_data.get('delimiter'): print( "\t\t" + indChar + "[delimiter] = " + obj.list_data.get('delimiter')) if obj.list_data.get('padding'): print( "\t\t" + indChar + "[padding] = " + str(obj.list_data.get('padding'))) if obj.list_data.get('marker_offset'): print( "\t\t" + indChar + "[marker_offset] = " + str(obj.list_data.get('marker_offset'))) if obj.walker: print("\t" + indChar + "Children:") walker = obj.walker() nxt = walker.nxt() while nxt is not None and topnode is False: dumpAST(nxt['node'], ind + 2, topnode=True) nxt = walker.nxt() commonmark-0.9.1/commonmark/entitytrans.py0000755000175000017500000017016113343503214020360 0ustar niknik00000000000000""" Python 3.4 HTML5 entity unescaping for all! Based on https://hg.python.org/cpython/file/500d3d6f22ff/Lib/html/__init__.py """ from __future__ import unicode_literals import sys import re as _re __all__ = ['_unescape'] _html5 = { 'Aacute': '\xc1', 'aacute': '\xe1', 'Aacute;': '\xc1', 'aacute;': '\xe1', 'Abreve;': '\u0102', 'abreve;': '\u0103', 'ac;': '\u223e', 'acd;': '\u223f', 'acE;': '\u223e\u0333', 'Acirc': '\xc2', 'acirc': '\xe2', 'Acirc;': '\xc2', 'acirc;': '\xe2', 'acute': '\xb4', 'acute;': '\xb4', 'Acy;': '\u0410', 'acy;': '\u0430', 'AElig': '\xc6', 'aelig': '\xe6', 'AElig;': '\xc6', 'aelig;': '\xe6', 'af;': '\u2061', 'Afr;': '\U0001d504', 'afr;': '\U0001d51e', 'Agrave': '\xc0', 'agrave': '\xe0', 'Agrave;': '\xc0', 'agrave;': '\xe0', 'alefsym;': '\u2135', 'aleph;': '\u2135', 'Alpha;': '\u0391', 'alpha;': '\u03b1', 'Amacr;': '\u0100', 'amacr;': '\u0101', 'amalg;': '\u2a3f', 'AMP': '&', 'amp': '&', 'AMP;': '&', 'amp;': '&', 'And;': '\u2a53', 'and;': '\u2227', 'andand;': '\u2a55', 'andd;': '\u2a5c', 'andslope;': '\u2a58', 'andv;': '\u2a5a', 'ang;': '\u2220', 'ange;': '\u29a4', 'angle;': '\u2220', 'angmsd;': '\u2221', 'angmsdaa;': '\u29a8', 'angmsdab;': '\u29a9', 'angmsdac;': '\u29aa', 'angmsdad;': '\u29ab', 'angmsdae;': '\u29ac', 'angmsdaf;': '\u29ad', 'angmsdag;': '\u29ae', 'angmsdah;': '\u29af', 'angrt;': '\u221f', 'angrtvb;': '\u22be', 'angrtvbd;': '\u299d', 'angsph;': '\u2222', 'angst;': '\xc5', 'angzarr;': '\u237c', 'Aogon;': '\u0104', 'aogon;': '\u0105', 'Aopf;': '\U0001d538', 'aopf;': '\U0001d552', 'ap;': '\u2248', 'apacir;': '\u2a6f', 'apE;': '\u2a70', 'ape;': '\u224a', 'apid;': '\u224b', 'apos;': "'", 'ApplyFunction;': '\u2061', 'approx;': '\u2248', 'approxeq;': '\u224a', 'Aring': '\xc5', 'aring': '\xe5', 'Aring;': '\xc5', 'aring;': '\xe5', 'Ascr;': '\U0001d49c', 'ascr;': '\U0001d4b6', 'Assign;': '\u2254', 'ast;': '*', 'asymp;': '\u2248', 'asympeq;': '\u224d', 'Atilde': '\xc3', 'atilde': '\xe3', 'Atilde;': '\xc3', 'atilde;': '\xe3', 'Auml': '\xc4', 'auml': '\xe4', 'Auml;': '\xc4', 'auml;': '\xe4', 'awconint;': '\u2233', 'awint;': '\u2a11', 'backcong;': '\u224c', 'backepsilon;': '\u03f6', 'backprime;': '\u2035', 'backsim;': '\u223d', 'backsimeq;': '\u22cd', 'Backslash;': '\u2216', 'Barv;': '\u2ae7', 'barvee;': '\u22bd', 'Barwed;': '\u2306', 'barwed;': '\u2305', 'barwedge;': '\u2305', 'bbrk;': '\u23b5', 'bbrktbrk;': '\u23b6', 'bcong;': '\u224c', 'Bcy;': '\u0411', 'bcy;': '\u0431', 'bdquo;': '\u201e', 'becaus;': '\u2235', 'Because;': '\u2235', 'because;': '\u2235', 'bemptyv;': '\u29b0', 'bepsi;': '\u03f6', 'bernou;': '\u212c', 'Bernoullis;': '\u212c', 'Beta;': '\u0392', 'beta;': '\u03b2', 'beth;': '\u2136', 'between;': '\u226c', 'Bfr;': '\U0001d505', 'bfr;': '\U0001d51f', 'bigcap;': '\u22c2', 'bigcirc;': '\u25ef', 'bigcup;': '\u22c3', 'bigodot;': '\u2a00', 'bigoplus;': '\u2a01', 'bigotimes;': '\u2a02', 'bigsqcup;': '\u2a06', 'bigstar;': '\u2605', 'bigtriangledown;': '\u25bd', 'bigtriangleup;': '\u25b3', 'biguplus;': '\u2a04', 'bigvee;': '\u22c1', 'bigwedge;': '\u22c0', 'bkarow;': '\u290d', 'blacklozenge;': '\u29eb', 'blacksquare;': '\u25aa', 'blacktriangle;': '\u25b4', 'blacktriangledown;': '\u25be', 'blacktriangleleft;': '\u25c2', 'blacktriangleright;': '\u25b8', 'blank;': '\u2423', 'blk12;': '\u2592', 'blk14;': '\u2591', 'blk34;': '\u2593', 'block;': '\u2588', 'bne;': '=\u20e5', 'bnequiv;': '\u2261\u20e5', 'bNot;': '\u2aed', 'bnot;': '\u2310', 'Bopf;': '\U0001d539', 'bopf;': '\U0001d553', 'bot;': '\u22a5', 'bottom;': '\u22a5', 'bowtie;': '\u22c8', 'boxbox;': '\u29c9', 'boxDL;': '\u2557', 'boxDl;': '\u2556', 'boxdL;': '\u2555', 'boxdl;': '\u2510', 'boxDR;': '\u2554', 'boxDr;': '\u2553', 'boxdR;': '\u2552', 'boxdr;': '\u250c', 'boxH;': '\u2550', 'boxh;': '\u2500', 'boxHD;': '\u2566', 'boxHd;': '\u2564', 'boxhD;': '\u2565', 'boxhd;': '\u252c', 'boxHU;': '\u2569', 'boxHu;': '\u2567', 'boxhU;': '\u2568', 'boxhu;': '\u2534', 'boxminus;': '\u229f', 'boxplus;': '\u229e', 'boxtimes;': '\u22a0', 'boxUL;': '\u255d', 'boxUl;': '\u255c', 'boxuL;': '\u255b', 'boxul;': '\u2518', 'boxUR;': '\u255a', 'boxUr;': '\u2559', 'boxuR;': '\u2558', 'boxur;': '\u2514', 'boxV;': '\u2551', 'boxv;': '\u2502', 'boxVH;': '\u256c', 'boxVh;': '\u256b', 'boxvH;': '\u256a', 'boxvh;': '\u253c', 'boxVL;': '\u2563', 'boxVl;': '\u2562', 'boxvL;': '\u2561', 'boxvl;': '\u2524', 'boxVR;': '\u2560', 'boxVr;': '\u255f', 'boxvR;': '\u255e', 'boxvr;': '\u251c', 'bprime;': '\u2035', 'Breve;': '\u02d8', 'breve;': '\u02d8', 'brvbar': '\xa6', 'brvbar;': '\xa6', 'Bscr;': '\u212c', 'bscr;': '\U0001d4b7', 'bsemi;': '\u204f', 'bsim;': '\u223d', 'bsime;': '\u22cd', 'bsol;': '\\', 'bsolb;': '\u29c5', 'bsolhsub;': '\u27c8', 'bull;': '\u2022', 'bullet;': '\u2022', 'bump;': '\u224e', 'bumpE;': '\u2aae', 'bumpe;': '\u224f', 'Bumpeq;': '\u224e', 'bumpeq;': '\u224f', 'Cacute;': '\u0106', 'cacute;': '\u0107', 'Cap;': '\u22d2', 'cap;': '\u2229', 'capand;': '\u2a44', 'capbrcup;': '\u2a49', 'capcap;': '\u2a4b', 'capcup;': '\u2a47', 'capdot;': '\u2a40', 'CapitalDifferentialD;': '\u2145', 'caps;': '\u2229\ufe00', 'caret;': '\u2041', 'caron;': '\u02c7', 'Cayleys;': '\u212d', 'ccaps;': '\u2a4d', 'Ccaron;': '\u010c', 'ccaron;': '\u010d', 'Ccedil': '\xc7', 'ccedil': '\xe7', 'Ccedil;': '\xc7', 'ccedil;': '\xe7', 'Ccirc;': '\u0108', 'ccirc;': '\u0109', 'Cconint;': '\u2230', 'ccups;': '\u2a4c', 'ccupssm;': '\u2a50', 'Cdot;': '\u010a', 'cdot;': '\u010b', 'cedil': '\xb8', 'cedil;': '\xb8', 'Cedilla;': '\xb8', 'cemptyv;': '\u29b2', 'cent': '\xa2', 'cent;': '\xa2', 'CenterDot;': '\xb7', 'centerdot;': '\xb7', 'Cfr;': '\u212d', 'cfr;': '\U0001d520', 'CHcy;': '\u0427', 'chcy;': '\u0447', 'check;': '\u2713', 'checkmark;': '\u2713', 'Chi;': '\u03a7', 'chi;': '\u03c7', 'cir;': '\u25cb', 'circ;': '\u02c6', 'circeq;': '\u2257', 'circlearrowleft;': '\u21ba', 'circlearrowright;': '\u21bb', 'circledast;': '\u229b', 'circledcirc;': '\u229a', 'circleddash;': '\u229d', 'CircleDot;': '\u2299', 'circledR;': '\xae', 'circledS;': '\u24c8', 'CircleMinus;': '\u2296', 'CirclePlus;': '\u2295', 'CircleTimes;': '\u2297', 'cirE;': '\u29c3', 'cire;': '\u2257', 'cirfnint;': '\u2a10', 'cirmid;': '\u2aef', 'cirscir;': '\u29c2', 'ClockwiseContourIntegral;': '\u2232', 'CloseCurlyDoubleQuote;': '\u201d', 'CloseCurlyQuote;': '\u2019', 'clubs;': '\u2663', 'clubsuit;': '\u2663', 'Colon;': '\u2237', 'colon;': ':', 'Colone;': '\u2a74', 'colone;': '\u2254', 'coloneq;': '\u2254', 'comma;': ',', 'commat;': '@', 'comp;': '\u2201', 'compfn;': '\u2218', 'complement;': '\u2201', 'complexes;': '\u2102', 'cong;': '\u2245', 'congdot;': '\u2a6d', 'Congruent;': '\u2261', 'Conint;': '\u222f', 'conint;': '\u222e', 'ContourIntegral;': '\u222e', 'Copf;': '\u2102', 'copf;': '\U0001d554', 'coprod;': '\u2210', 'Coproduct;': '\u2210', 'COPY': '\xa9', 'copy': '\xa9', 'COPY;': '\xa9', 'copy;': '\xa9', 'copysr;': '\u2117', 'CounterClockwiseContourIntegral;': '\u2233', 'crarr;': '\u21b5', 'Cross;': '\u2a2f', 'cross;': '\u2717', 'Cscr;': '\U0001d49e', 'cscr;': '\U0001d4b8', 'csub;': '\u2acf', 'csube;': '\u2ad1', 'csup;': '\u2ad0', 'csupe;': '\u2ad2', 'ctdot;': '\u22ef', 'cudarrl;': '\u2938', 'cudarrr;': '\u2935', 'cuepr;': '\u22de', 'cuesc;': '\u22df', 'cularr;': '\u21b6', 'cularrp;': '\u293d', 'Cup;': '\u22d3', 'cup;': '\u222a', 'cupbrcap;': '\u2a48', 'CupCap;': '\u224d', 'cupcap;': '\u2a46', 'cupcup;': '\u2a4a', 'cupdot;': '\u228d', 'cupor;': '\u2a45', 'cups;': '\u222a\ufe00', 'curarr;': '\u21b7', 'curarrm;': '\u293c', 'curlyeqprec;': '\u22de', 'curlyeqsucc;': '\u22df', 'curlyvee;': '\u22ce', 'curlywedge;': '\u22cf', 'curren': '\xa4', 'curren;': '\xa4', 'curvearrowleft;': '\u21b6', 'curvearrowright;': '\u21b7', 'cuvee;': '\u22ce', 'cuwed;': '\u22cf', 'cwconint;': '\u2232', 'cwint;': '\u2231', 'cylcty;': '\u232d', 'Dagger;': '\u2021', 'dagger;': '\u2020', 'daleth;': '\u2138', 'Darr;': '\u21a1', 'dArr;': '\u21d3', 'darr;': '\u2193', 'dash;': '\u2010', 'Dashv;': '\u2ae4', 'dashv;': '\u22a3', 'dbkarow;': '\u290f', 'dblac;': '\u02dd', 'Dcaron;': '\u010e', 'dcaron;': '\u010f', 'Dcy;': '\u0414', 'dcy;': '\u0434', 'DD;': '\u2145', 'dd;': '\u2146', 'ddagger;': '\u2021', 'ddarr;': '\u21ca', 'DDotrahd;': '\u2911', 'ddotseq;': '\u2a77', 'deg': '\xb0', 'deg;': '\xb0', 'Del;': '\u2207', 'Delta;': '\u0394', 'delta;': '\u03b4', 'demptyv;': '\u29b1', 'dfisht;': '\u297f', 'Dfr;': '\U0001d507', 'dfr;': '\U0001d521', 'dHar;': '\u2965', 'dharl;': '\u21c3', 'dharr;': '\u21c2', 'DiacriticalAcute;': '\xb4', 'DiacriticalDot;': '\u02d9', 'DiacriticalDoubleAcute;': '\u02dd', 'DiacriticalGrave;': '`', 'DiacriticalTilde;': '\u02dc', 'diam;': '\u22c4', 'Diamond;': '\u22c4', 'diamond;': '\u22c4', 'diamondsuit;': '\u2666', 'diams;': '\u2666', 'die;': '\xa8', 'DifferentialD;': '\u2146', 'digamma;': '\u03dd', 'disin;': '\u22f2', 'div;': '\xf7', 'divide': '\xf7', 'divide;': '\xf7', 'divideontimes;': '\u22c7', 'divonx;': '\u22c7', 'DJcy;': '\u0402', 'djcy;': '\u0452', 'dlcorn;': '\u231e', 'dlcrop;': '\u230d', 'dollar;': '$', 'Dopf;': '\U0001d53b', 'dopf;': '\U0001d555', 'Dot;': '\xa8', 'dot;': '\u02d9', 'DotDot;': '\u20dc', 'doteq;': '\u2250', 'doteqdot;': '\u2251', 'DotEqual;': '\u2250', 'dotminus;': '\u2238', 'dotplus;': '\u2214', 'dotsquare;': '\u22a1', 'doublebarwedge;': '\u2306', 'DoubleContourIntegral;': '\u222f', 'DoubleDot;': '\xa8', 'DoubleDownArrow;': '\u21d3', 'DoubleLeftArrow;': '\u21d0', 'DoubleLeftRightArrow;': '\u21d4', 'DoubleLeftTee;': '\u2ae4', 'DoubleLongLeftArrow;': '\u27f8', 'DoubleLongLeftRightArrow;': '\u27fa', 'DoubleLongRightArrow;': '\u27f9', 'DoubleRightArrow;': '\u21d2', 'DoubleRightTee;': '\u22a8', 'DoubleUpArrow;': '\u21d1', 'DoubleUpDownArrow;': '\u21d5', 'DoubleVerticalBar;': '\u2225', 'DownArrow;': '\u2193', 'Downarrow;': '\u21d3', 'downarrow;': '\u2193', 'DownArrowBar;': '\u2913', 'DownArrowUpArrow;': '\u21f5', 'DownBreve;': '\u0311', 'downdownarrows;': '\u21ca', 'downharpoonleft;': '\u21c3', 'downharpoonright;': '\u21c2', 'DownLeftRightVector;': '\u2950', 'DownLeftTeeVector;': '\u295e', 'DownLeftVector;': '\u21bd', 'DownLeftVectorBar;': '\u2956', 'DownRightTeeVector;': '\u295f', 'DownRightVector;': '\u21c1', 'DownRightVectorBar;': '\u2957', 'DownTee;': '\u22a4', 'DownTeeArrow;': '\u21a7', 'drbkarow;': '\u2910', 'drcorn;': '\u231f', 'drcrop;': '\u230c', 'Dscr;': '\U0001d49f', 'dscr;': '\U0001d4b9', 'DScy;': '\u0405', 'dscy;': '\u0455', 'dsol;': '\u29f6', 'Dstrok;': '\u0110', 'dstrok;': '\u0111', 'dtdot;': '\u22f1', 'dtri;': '\u25bf', 'dtrif;': '\u25be', 'duarr;': '\u21f5', 'duhar;': '\u296f', 'dwangle;': '\u29a6', 'DZcy;': '\u040f', 'dzcy;': '\u045f', 'dzigrarr;': '\u27ff', 'Eacute': '\xc9', 'eacute': '\xe9', 'Eacute;': '\xc9', 'eacute;': '\xe9', 'easter;': '\u2a6e', 'Ecaron;': '\u011a', 'ecaron;': '\u011b', 'ecir;': '\u2256', 'Ecirc': '\xca', 'ecirc': '\xea', 'Ecirc;': '\xca', 'ecirc;': '\xea', 'ecolon;': '\u2255', 'Ecy;': '\u042d', 'ecy;': '\u044d', 'eDDot;': '\u2a77', 'Edot;': '\u0116', 'eDot;': '\u2251', 'edot;': '\u0117', 'ee;': '\u2147', 'efDot;': '\u2252', 'Efr;': '\U0001d508', 'efr;': '\U0001d522', 'eg;': '\u2a9a', 'Egrave': '\xc8', 'egrave': '\xe8', 'Egrave;': '\xc8', 'egrave;': '\xe8', 'egs;': '\u2a96', 'egsdot;': '\u2a98', 'el;': '\u2a99', 'Element;': '\u2208', 'elinters;': '\u23e7', 'ell;': '\u2113', 'els;': '\u2a95', 'elsdot;': '\u2a97', 'Emacr;': '\u0112', 'emacr;': '\u0113', 'empty;': '\u2205', 'emptyset;': '\u2205', 'EmptySmallSquare;': '\u25fb', 'emptyv;': '\u2205', 'EmptyVerySmallSquare;': '\u25ab', 'emsp13;': '\u2004', 'emsp14;': '\u2005', 'emsp;': '\u2003', 'ENG;': '\u014a', 'eng;': '\u014b', 'ensp;': '\u2002', 'Eogon;': '\u0118', 'eogon;': '\u0119', 'Eopf;': '\U0001d53c', 'eopf;': '\U0001d556', 'epar;': '\u22d5', 'eparsl;': '\u29e3', 'eplus;': '\u2a71', 'epsi;': '\u03b5', 'Epsilon;': '\u0395', 'epsilon;': '\u03b5', 'epsiv;': '\u03f5', 'eqcirc;': '\u2256', 'eqcolon;': '\u2255', 'eqsim;': '\u2242', 'eqslantgtr;': '\u2a96', 'eqslantless;': '\u2a95', 'Equal;': '\u2a75', 'equals;': '=', 'EqualTilde;': '\u2242', 'equest;': '\u225f', 'Equilibrium;': '\u21cc', 'equiv;': '\u2261', 'equivDD;': '\u2a78', 'eqvparsl;': '\u29e5', 'erarr;': '\u2971', 'erDot;': '\u2253', 'Escr;': '\u2130', 'escr;': '\u212f', 'esdot;': '\u2250', 'Esim;': '\u2a73', 'esim;': '\u2242', 'Eta;': '\u0397', 'eta;': '\u03b7', 'ETH': '\xd0', 'eth': '\xf0', 'ETH;': '\xd0', 'eth;': '\xf0', 'Euml': '\xcb', 'euml': '\xeb', 'Euml;': '\xcb', 'euml;': '\xeb', 'euro;': '\u20ac', 'excl;': '!', 'exist;': '\u2203', 'Exists;': '\u2203', 'expectation;': '\u2130', 'ExponentialE;': '\u2147', 'exponentiale;': '\u2147', 'fallingdotseq;': '\u2252', 'Fcy;': '\u0424', 'fcy;': '\u0444', 'female;': '\u2640', 'ffilig;': '\ufb03', 'fflig;': '\ufb00', 'ffllig;': '\ufb04', 'Ffr;': '\U0001d509', 'ffr;': '\U0001d523', 'filig;': '\ufb01', 'FilledSmallSquare;': '\u25fc', 'FilledVerySmallSquare;': '\u25aa', 'fjlig;': 'fj', 'flat;': '\u266d', 'fllig;': '\ufb02', 'fltns;': '\u25b1', 'fnof;': '\u0192', 'Fopf;': '\U0001d53d', 'fopf;': '\U0001d557', 'ForAll;': '\u2200', 'forall;': '\u2200', 'fork;': '\u22d4', 'forkv;': '\u2ad9', 'Fouriertrf;': '\u2131', 'fpartint;': '\u2a0d', 'frac12': '\xbd', 'frac12;': '\xbd', 'frac13;': '\u2153', 'frac14': '\xbc', 'frac14;': '\xbc', 'frac15;': '\u2155', 'frac16;': '\u2159', 'frac18;': '\u215b', 'frac23;': '\u2154', 'frac25;': '\u2156', 'frac34': '\xbe', 'frac34;': '\xbe', 'frac35;': '\u2157', 'frac38;': '\u215c', 'frac45;': '\u2158', 'frac56;': '\u215a', 'frac58;': '\u215d', 'frac78;': '\u215e', 'frasl;': '\u2044', 'frown;': '\u2322', 'Fscr;': '\u2131', 'fscr;': '\U0001d4bb', 'gacute;': '\u01f5', 'Gamma;': '\u0393', 'gamma;': '\u03b3', 'Gammad;': '\u03dc', 'gammad;': '\u03dd', 'gap;': '\u2a86', 'Gbreve;': '\u011e', 'gbreve;': '\u011f', 'Gcedil;': '\u0122', 'Gcirc;': '\u011c', 'gcirc;': '\u011d', 'Gcy;': '\u0413', 'gcy;': '\u0433', 'Gdot;': '\u0120', 'gdot;': '\u0121', 'gE;': '\u2267', 'ge;': '\u2265', 'gEl;': '\u2a8c', 'gel;': '\u22db', 'geq;': '\u2265', 'geqq;': '\u2267', 'geqslant;': '\u2a7e', 'ges;': '\u2a7e', 'gescc;': '\u2aa9', 'gesdot;': '\u2a80', 'gesdoto;': '\u2a82', 'gesdotol;': '\u2a84', 'gesl;': '\u22db\ufe00', 'gesles;': '\u2a94', 'Gfr;': '\U0001d50a', 'gfr;': '\U0001d524', 'Gg;': '\u22d9', 'gg;': '\u226b', 'ggg;': '\u22d9', 'gimel;': '\u2137', 'GJcy;': '\u0403', 'gjcy;': '\u0453', 'gl;': '\u2277', 'gla;': '\u2aa5', 'glE;': '\u2a92', 'glj;': '\u2aa4', 'gnap;': '\u2a8a', 'gnapprox;': '\u2a8a', 'gnE;': '\u2269', 'gne;': '\u2a88', 'gneq;': '\u2a88', 'gneqq;': '\u2269', 'gnsim;': '\u22e7', 'Gopf;': '\U0001d53e', 'gopf;': '\U0001d558', 'grave;': '`', 'GreaterEqual;': '\u2265', 'GreaterEqualLess;': '\u22db', 'GreaterFullEqual;': '\u2267', 'GreaterGreater;': '\u2aa2', 'GreaterLess;': '\u2277', 'GreaterSlantEqual;': '\u2a7e', 'GreaterTilde;': '\u2273', 'Gscr;': '\U0001d4a2', 'gscr;': '\u210a', 'gsim;': '\u2273', 'gsime;': '\u2a8e', 'gsiml;': '\u2a90', 'GT': '>', 'gt': '>', 'GT;': '>', 'Gt;': '\u226b', 'gt;': '>', 'gtcc;': '\u2aa7', 'gtcir;': '\u2a7a', 'gtdot;': '\u22d7', 'gtlPar;': '\u2995', 'gtquest;': '\u2a7c', 'gtrapprox;': '\u2a86', 'gtrarr;': '\u2978', 'gtrdot;': '\u22d7', 'gtreqless;': '\u22db', 'gtreqqless;': '\u2a8c', 'gtrless;': '\u2277', 'gtrsim;': '\u2273', 'gvertneqq;': '\u2269\ufe00', 'gvnE;': '\u2269\ufe00', 'Hacek;': '\u02c7', 'hairsp;': '\u200a', 'half;': '\xbd', 'hamilt;': '\u210b', 'HARDcy;': '\u042a', 'hardcy;': '\u044a', 'hArr;': '\u21d4', 'harr;': '\u2194', 'harrcir;': '\u2948', 'harrw;': '\u21ad', 'Hat;': '^', 'hbar;': '\u210f', 'Hcirc;': '\u0124', 'hcirc;': '\u0125', 'hearts;': '\u2665', 'heartsuit;': '\u2665', 'hellip;': '\u2026', 'hercon;': '\u22b9', 'Hfr;': '\u210c', 'hfr;': '\U0001d525', 'HilbertSpace;': '\u210b', 'hksearow;': '\u2925', 'hkswarow;': '\u2926', 'hoarr;': '\u21ff', 'homtht;': '\u223b', 'hookleftarrow;': '\u21a9', 'hookrightarrow;': '\u21aa', 'Hopf;': '\u210d', 'hopf;': '\U0001d559', 'horbar;': '\u2015', 'HorizontalLine;': '\u2500', 'Hscr;': '\u210b', 'hscr;': '\U0001d4bd', 'hslash;': '\u210f', 'Hstrok;': '\u0126', 'hstrok;': '\u0127', 'HumpDownHump;': '\u224e', 'HumpEqual;': '\u224f', 'hybull;': '\u2043', 'hyphen;': '\u2010', 'Iacute': '\xcd', 'iacute': '\xed', 'Iacute;': '\xcd', 'iacute;': '\xed', 'ic;': '\u2063', 'Icirc': '\xce', 'icirc': '\xee', 'Icirc;': '\xce', 'icirc;': '\xee', 'Icy;': '\u0418', 'icy;': '\u0438', 'Idot;': '\u0130', 'IEcy;': '\u0415', 'iecy;': '\u0435', 'iexcl': '\xa1', 'iexcl;': '\xa1', 'iff;': '\u21d4', 'Ifr;': '\u2111', 'ifr;': '\U0001d526', 'Igrave': '\xcc', 'igrave': '\xec', 'Igrave;': '\xcc', 'igrave;': '\xec', 'ii;': '\u2148', 'iiiint;': '\u2a0c', 'iiint;': '\u222d', 'iinfin;': '\u29dc', 'iiota;': '\u2129', 'IJlig;': '\u0132', 'ijlig;': '\u0133', 'Im;': '\u2111', 'Imacr;': '\u012a', 'imacr;': '\u012b', 'image;': '\u2111', 'ImaginaryI;': '\u2148', 'imagline;': '\u2110', 'imagpart;': '\u2111', 'imath;': '\u0131', 'imof;': '\u22b7', 'imped;': '\u01b5', 'Implies;': '\u21d2', 'in;': '\u2208', 'incare;': '\u2105', 'infin;': '\u221e', 'infintie;': '\u29dd', 'inodot;': '\u0131', 'Int;': '\u222c', 'int;': '\u222b', 'intcal;': '\u22ba', 'integers;': '\u2124', 'Integral;': '\u222b', 'intercal;': '\u22ba', 'Intersection;': '\u22c2', 'intlarhk;': '\u2a17', 'intprod;': '\u2a3c', 'InvisibleComma;': '\u2063', 'InvisibleTimes;': '\u2062', 'IOcy;': '\u0401', 'iocy;': '\u0451', 'Iogon;': '\u012e', 'iogon;': '\u012f', 'Iopf;': '\U0001d540', 'iopf;': '\U0001d55a', 'Iota;': '\u0399', 'iota;': '\u03b9', 'iprod;': '\u2a3c', 'iquest': '\xbf', 'iquest;': '\xbf', 'Iscr;': '\u2110', 'iscr;': '\U0001d4be', 'isin;': '\u2208', 'isindot;': '\u22f5', 'isinE;': '\u22f9', 'isins;': '\u22f4', 'isinsv;': '\u22f3', 'isinv;': '\u2208', 'it;': '\u2062', 'Itilde;': '\u0128', 'itilde;': '\u0129', 'Iukcy;': '\u0406', 'iukcy;': '\u0456', 'Iuml': '\xcf', 'iuml': '\xef', 'Iuml;': '\xcf', 'iuml;': '\xef', 'Jcirc;': '\u0134', 'jcirc;': '\u0135', 'Jcy;': '\u0419', 'jcy;': '\u0439', 'Jfr;': '\U0001d50d', 'jfr;': '\U0001d527', 'jmath;': '\u0237', 'Jopf;': '\U0001d541', 'jopf;': '\U0001d55b', 'Jscr;': '\U0001d4a5', 'jscr;': '\U0001d4bf', 'Jsercy;': '\u0408', 'jsercy;': '\u0458', 'Jukcy;': '\u0404', 'jukcy;': '\u0454', 'Kappa;': '\u039a', 'kappa;': '\u03ba', 'kappav;': '\u03f0', 'Kcedil;': '\u0136', 'kcedil;': '\u0137', 'Kcy;': '\u041a', 'kcy;': '\u043a', 'Kfr;': '\U0001d50e', 'kfr;': '\U0001d528', 'kgreen;': '\u0138', 'KHcy;': '\u0425', 'khcy;': '\u0445', 'KJcy;': '\u040c', 'kjcy;': '\u045c', 'Kopf;': '\U0001d542', 'kopf;': '\U0001d55c', 'Kscr;': '\U0001d4a6', 'kscr;': '\U0001d4c0', 'lAarr;': '\u21da', 'Lacute;': '\u0139', 'lacute;': '\u013a', 'laemptyv;': '\u29b4', 'lagran;': '\u2112', 'Lambda;': '\u039b', 'lambda;': '\u03bb', 'Lang;': '\u27ea', 'lang;': '\u27e8', 'langd;': '\u2991', 'langle;': '\u27e8', 'lap;': '\u2a85', 'Laplacetrf;': '\u2112', 'laquo': '\xab', 'laquo;': '\xab', 'Larr;': '\u219e', 'lArr;': '\u21d0', 'larr;': '\u2190', 'larrb;': '\u21e4', 'larrbfs;': '\u291f', 'larrfs;': '\u291d', 'larrhk;': '\u21a9', 'larrlp;': '\u21ab', 'larrpl;': '\u2939', 'larrsim;': '\u2973', 'larrtl;': '\u21a2', 'lat;': '\u2aab', 'lAtail;': '\u291b', 'latail;': '\u2919', 'late;': '\u2aad', 'lates;': '\u2aad\ufe00', 'lBarr;': '\u290e', 'lbarr;': '\u290c', 'lbbrk;': '\u2772', 'lbrace;': '{', 'lbrack;': '[', 'lbrke;': '\u298b', 'lbrksld;': '\u298f', 'lbrkslu;': '\u298d', 'Lcaron;': '\u013d', 'lcaron;': '\u013e', 'Lcedil;': '\u013b', 'lcedil;': '\u013c', 'lceil;': '\u2308', 'lcub;': '{', 'Lcy;': '\u041b', 'lcy;': '\u043b', 'ldca;': '\u2936', 'ldquo;': '\u201c', 'ldquor;': '\u201e', 'ldrdhar;': '\u2967', 'ldrushar;': '\u294b', 'ldsh;': '\u21b2', 'lE;': '\u2266', 'le;': '\u2264', 'LeftAngleBracket;': '\u27e8', 'LeftArrow;': '\u2190', 'Leftarrow;': '\u21d0', 'leftarrow;': '\u2190', 'LeftArrowBar;': '\u21e4', 'LeftArrowRightArrow;': '\u21c6', 'leftarrowtail;': '\u21a2', 'LeftCeiling;': '\u2308', 'LeftDoubleBracket;': '\u27e6', 'LeftDownTeeVector;': '\u2961', 'LeftDownVector;': '\u21c3', 'LeftDownVectorBar;': '\u2959', 'LeftFloor;': '\u230a', 'leftharpoondown;': '\u21bd', 'leftharpoonup;': '\u21bc', 'leftleftarrows;': '\u21c7', 'LeftRightArrow;': '\u2194', 'Leftrightarrow;': '\u21d4', 'leftrightarrow;': '\u2194', 'leftrightarrows;': '\u21c6', 'leftrightharpoons;': '\u21cb', 'leftrightsquigarrow;': '\u21ad', 'LeftRightVector;': '\u294e', 'LeftTee;': '\u22a3', 'LeftTeeArrow;': '\u21a4', 'LeftTeeVector;': '\u295a', 'leftthreetimes;': '\u22cb', 'LeftTriangle;': '\u22b2', 'LeftTriangleBar;': '\u29cf', 'LeftTriangleEqual;': '\u22b4', 'LeftUpDownVector;': '\u2951', 'LeftUpTeeVector;': '\u2960', 'LeftUpVector;': '\u21bf', 'LeftUpVectorBar;': '\u2958', 'LeftVector;': '\u21bc', 'LeftVectorBar;': '\u2952', 'lEg;': '\u2a8b', 'leg;': '\u22da', 'leq;': '\u2264', 'leqq;': '\u2266', 'leqslant;': '\u2a7d', 'les;': '\u2a7d', 'lescc;': '\u2aa8', 'lesdot;': '\u2a7f', 'lesdoto;': '\u2a81', 'lesdotor;': '\u2a83', 'lesg;': '\u22da\ufe00', 'lesges;': '\u2a93', 'lessapprox;': '\u2a85', 'lessdot;': '\u22d6', 'lesseqgtr;': '\u22da', 'lesseqqgtr;': '\u2a8b', 'LessEqualGreater;': '\u22da', 'LessFullEqual;': '\u2266', 'LessGreater;': '\u2276', 'lessgtr;': '\u2276', 'LessLess;': '\u2aa1', 'lesssim;': '\u2272', 'LessSlantEqual;': '\u2a7d', 'LessTilde;': '\u2272', 'lfisht;': '\u297c', 'lfloor;': '\u230a', 'Lfr;': '\U0001d50f', 'lfr;': '\U0001d529', 'lg;': '\u2276', 'lgE;': '\u2a91', 'lHar;': '\u2962', 'lhard;': '\u21bd', 'lharu;': '\u21bc', 'lharul;': '\u296a', 'lhblk;': '\u2584', 'LJcy;': '\u0409', 'ljcy;': '\u0459', 'Ll;': '\u22d8', 'll;': '\u226a', 'llarr;': '\u21c7', 'llcorner;': '\u231e', 'Lleftarrow;': '\u21da', 'llhard;': '\u296b', 'lltri;': '\u25fa', 'Lmidot;': '\u013f', 'lmidot;': '\u0140', 'lmoust;': '\u23b0', 'lmoustache;': '\u23b0', 'lnap;': '\u2a89', 'lnapprox;': '\u2a89', 'lnE;': '\u2268', 'lne;': '\u2a87', 'lneq;': '\u2a87', 'lneqq;': '\u2268', 'lnsim;': '\u22e6', 'loang;': '\u27ec', 'loarr;': '\u21fd', 'lobrk;': '\u27e6', 'LongLeftArrow;': '\u27f5', 'Longleftarrow;': '\u27f8', 'longleftarrow;': '\u27f5', 'LongLeftRightArrow;': '\u27f7', 'Longleftrightarrow;': '\u27fa', 'longleftrightarrow;': '\u27f7', 'longmapsto;': '\u27fc', 'LongRightArrow;': '\u27f6', 'Longrightarrow;': '\u27f9', 'longrightarrow;': '\u27f6', 'looparrowleft;': '\u21ab', 'looparrowright;': '\u21ac', 'lopar;': '\u2985', 'Lopf;': '\U0001d543', 'lopf;': '\U0001d55d', 'loplus;': '\u2a2d', 'lotimes;': '\u2a34', 'lowast;': '\u2217', 'lowbar;': '_', 'LowerLeftArrow;': '\u2199', 'LowerRightArrow;': '\u2198', 'loz;': '\u25ca', 'lozenge;': '\u25ca', 'lozf;': '\u29eb', 'lpar;': '(', 'lparlt;': '\u2993', 'lrarr;': '\u21c6', 'lrcorner;': '\u231f', 'lrhar;': '\u21cb', 'lrhard;': '\u296d', 'lrm;': '\u200e', 'lrtri;': '\u22bf', 'lsaquo;': '\u2039', 'Lscr;': '\u2112', 'lscr;': '\U0001d4c1', 'Lsh;': '\u21b0', 'lsh;': '\u21b0', 'lsim;': '\u2272', 'lsime;': '\u2a8d', 'lsimg;': '\u2a8f', 'lsqb;': '[', 'lsquo;': '\u2018', 'lsquor;': '\u201a', 'Lstrok;': '\u0141', 'lstrok;': '\u0142', 'LT': '<', 'lt': '<', 'LT;': '<', 'Lt;': '\u226a', 'lt;': '<', 'ltcc;': '\u2aa6', 'ltcir;': '\u2a79', 'ltdot;': '\u22d6', 'lthree;': '\u22cb', 'ltimes;': '\u22c9', 'ltlarr;': '\u2976', 'ltquest;': '\u2a7b', 'ltri;': '\u25c3', 'ltrie;': '\u22b4', 'ltrif;': '\u25c2', 'ltrPar;': '\u2996', 'lurdshar;': '\u294a', 'luruhar;': '\u2966', 'lvertneqq;': '\u2268\ufe00', 'lvnE;': '\u2268\ufe00', 'macr': '\xaf', 'macr;': '\xaf', 'male;': '\u2642', 'malt;': '\u2720', 'maltese;': '\u2720', 'Map;': '\u2905', 'map;': '\u21a6', 'mapsto;': '\u21a6', 'mapstodown;': '\u21a7', 'mapstoleft;': '\u21a4', 'mapstoup;': '\u21a5', 'marker;': '\u25ae', 'mcomma;': '\u2a29', 'Mcy;': '\u041c', 'mcy;': '\u043c', 'mdash;': '\u2014', 'mDDot;': '\u223a', 'measuredangle;': '\u2221', 'MediumSpace;': '\u205f', 'Mellintrf;': '\u2133', 'Mfr;': '\U0001d510', 'mfr;': '\U0001d52a', 'mho;': '\u2127', 'micro': '\xb5', 'micro;': '\xb5', 'mid;': '\u2223', 'midast;': '*', 'midcir;': '\u2af0', 'middot': '\xb7', 'middot;': '\xb7', 'minus;': '\u2212', 'minusb;': '\u229f', 'minusd;': '\u2238', 'minusdu;': '\u2a2a', 'MinusPlus;': '\u2213', 'mlcp;': '\u2adb', 'mldr;': '\u2026', 'mnplus;': '\u2213', 'models;': '\u22a7', 'Mopf;': '\U0001d544', 'mopf;': '\U0001d55e', 'mp;': '\u2213', 'Mscr;': '\u2133', 'mscr;': '\U0001d4c2', 'mstpos;': '\u223e', 'Mu;': '\u039c', 'mu;': '\u03bc', 'multimap;': '\u22b8', 'mumap;': '\u22b8', 'nabla;': '\u2207', 'Nacute;': '\u0143', 'nacute;': '\u0144', 'nang;': '\u2220\u20d2', 'nap;': '\u2249', 'napE;': '\u2a70\u0338', 'napid;': '\u224b\u0338', 'napos;': '\u0149', 'napprox;': '\u2249', 'natur;': '\u266e', 'natural;': '\u266e', 'naturals;': '\u2115', # 'nbsp': '\xa0', 'nbsp;': '\xa0', 'nbump;': '\u224e\u0338', 'nbumpe;': '\u224f\u0338', 'ncap;': '\u2a43', 'Ncaron;': '\u0147', 'ncaron;': '\u0148', 'Ncedil;': '\u0145', 'ncedil;': '\u0146', 'ncong;': '\u2247', 'ncongdot;': '\u2a6d\u0338', 'ncup;': '\u2a42', 'Ncy;': '\u041d', 'ncy;': '\u043d', 'ndash;': '\u2013', 'ne;': '\u2260', 'nearhk;': '\u2924', 'neArr;': '\u21d7', 'nearr;': '\u2197', 'nearrow;': '\u2197', 'nedot;': '\u2250\u0338', 'NegativeMediumSpace;': '\u200b', 'NegativeThickSpace;': '\u200b', 'NegativeThinSpace;': '\u200b', 'NegativeVeryThinSpace;': '\u200b', 'nequiv;': '\u2262', 'nesear;': '\u2928', 'nesim;': '\u2242\u0338', 'NestedGreaterGreater;': '\u226b', 'NestedLessLess;': '\u226a', 'NewLine;': '\n', 'nexist;': '\u2204', 'nexists;': '\u2204', 'Nfr;': '\U0001d511', 'nfr;': '\U0001d52b', 'ngE;': '\u2267\u0338', 'nge;': '\u2271', 'ngeq;': '\u2271', 'ngeqq;': '\u2267\u0338', 'ngeqslant;': '\u2a7e\u0338', 'nges;': '\u2a7e\u0338', 'nGg;': '\u22d9\u0338', 'ngsim;': '\u2275', 'nGt;': '\u226b\u20d2', 'ngt;': '\u226f', 'ngtr;': '\u226f', 'nGtv;': '\u226b\u0338', 'nhArr;': '\u21ce', 'nharr;': '\u21ae', 'nhpar;': '\u2af2', 'ni;': '\u220b', 'nis;': '\u22fc', 'nisd;': '\u22fa', 'niv;': '\u220b', 'NJcy;': '\u040a', 'njcy;': '\u045a', 'nlArr;': '\u21cd', 'nlarr;': '\u219a', 'nldr;': '\u2025', 'nlE;': '\u2266\u0338', 'nle;': '\u2270', 'nLeftarrow;': '\u21cd', 'nleftarrow;': '\u219a', 'nLeftrightarrow;': '\u21ce', 'nleftrightarrow;': '\u21ae', 'nleq;': '\u2270', 'nleqq;': '\u2266\u0338', 'nleqslant;': '\u2a7d\u0338', 'nles;': '\u2a7d\u0338', 'nless;': '\u226e', 'nLl;': '\u22d8\u0338', 'nlsim;': '\u2274', 'nLt;': '\u226a\u20d2', 'nlt;': '\u226e', 'nltri;': '\u22ea', 'nltrie;': '\u22ec', 'nLtv;': '\u226a\u0338', 'nmid;': '\u2224', 'NoBreak;': '\u2060', 'NonBreakingSpace;': '\xa0', 'Nopf;': '\u2115', 'nopf;': '\U0001d55f', 'not': '\xac', 'Not;': '\u2aec', 'not;': '\xac', 'NotCongruent;': '\u2262', 'NotCupCap;': '\u226d', 'NotDoubleVerticalBar;': '\u2226', 'NotElement;': '\u2209', 'NotEqual;': '\u2260', 'NotEqualTilde;': '\u2242\u0338', 'NotExists;': '\u2204', 'NotGreater;': '\u226f', 'NotGreaterEqual;': '\u2271', 'NotGreaterFullEqual;': '\u2267\u0338', 'NotGreaterGreater;': '\u226b\u0338', 'NotGreaterLess;': '\u2279', 'NotGreaterSlantEqual;': '\u2a7e\u0338', 'NotGreaterTilde;': '\u2275', 'NotHumpDownHump;': '\u224e\u0338', 'NotHumpEqual;': '\u224f\u0338', 'notin;': '\u2209', 'notindot;': '\u22f5\u0338', 'notinE;': '\u22f9\u0338', 'notinva;': '\u2209', 'notinvb;': '\u22f7', 'notinvc;': '\u22f6', 'NotLeftTriangle;': '\u22ea', 'NotLeftTriangleBar;': '\u29cf\u0338', 'NotLeftTriangleEqual;': '\u22ec', 'NotLess;': '\u226e', 'NotLessEqual;': '\u2270', 'NotLessGreater;': '\u2278', 'NotLessLess;': '\u226a\u0338', 'NotLessSlantEqual;': '\u2a7d\u0338', 'NotLessTilde;': '\u2274', 'NotNestedGreaterGreater;': '\u2aa2\u0338', 'NotNestedLessLess;': '\u2aa1\u0338', 'notni;': '\u220c', 'notniva;': '\u220c', 'notnivb;': '\u22fe', 'notnivc;': '\u22fd', 'NotPrecedes;': '\u2280', 'NotPrecedesEqual;': '\u2aaf\u0338', 'NotPrecedesSlantEqual;': '\u22e0', 'NotReverseElement;': '\u220c', 'NotRightTriangle;': '\u22eb', 'NotRightTriangleBar;': '\u29d0\u0338', 'NotRightTriangleEqual;': '\u22ed', 'NotSquareSubset;': '\u228f\u0338', 'NotSquareSubsetEqual;': '\u22e2', 'NotSquareSuperset;': '\u2290\u0338', 'NotSquareSupersetEqual;': '\u22e3', 'NotSubset;': '\u2282\u20d2', 'NotSubsetEqual;': '\u2288', 'NotSucceeds;': '\u2281', 'NotSucceedsEqual;': '\u2ab0\u0338', 'NotSucceedsSlantEqual;': '\u22e1', 'NotSucceedsTilde;': '\u227f\u0338', 'NotSuperset;': '\u2283\u20d2', 'NotSupersetEqual;': '\u2289', 'NotTilde;': '\u2241', 'NotTildeEqual;': '\u2244', 'NotTildeFullEqual;': '\u2247', 'NotTildeTilde;': '\u2249', 'NotVerticalBar;': '\u2224', 'npar;': '\u2226', 'nparallel;': '\u2226', 'nparsl;': '\u2afd\u20e5', 'npart;': '\u2202\u0338', 'npolint;': '\u2a14', 'npr;': '\u2280', 'nprcue;': '\u22e0', 'npre;': '\u2aaf\u0338', 'nprec;': '\u2280', 'npreceq;': '\u2aaf\u0338', 'nrArr;': '\u21cf', 'nrarr;': '\u219b', 'nrarrc;': '\u2933\u0338', 'nrarrw;': '\u219d\u0338', 'nRightarrow;': '\u21cf', 'nrightarrow;': '\u219b', 'nrtri;': '\u22eb', 'nrtrie;': '\u22ed', 'nsc;': '\u2281', 'nsccue;': '\u22e1', 'nsce;': '\u2ab0\u0338', 'Nscr;': '\U0001d4a9', 'nscr;': '\U0001d4c3', 'nshortmid;': '\u2224', 'nshortparallel;': '\u2226', 'nsim;': '\u2241', 'nsime;': '\u2244', 'nsimeq;': '\u2244', 'nsmid;': '\u2224', 'nspar;': '\u2226', 'nsqsube;': '\u22e2', 'nsqsupe;': '\u22e3', 'nsub;': '\u2284', 'nsubE;': '\u2ac5\u0338', 'nsube;': '\u2288', 'nsubset;': '\u2282\u20d2', 'nsubseteq;': '\u2288', 'nsubseteqq;': '\u2ac5\u0338', 'nsucc;': '\u2281', 'nsucceq;': '\u2ab0\u0338', 'nsup;': '\u2285', 'nsupE;': '\u2ac6\u0338', 'nsupe;': '\u2289', 'nsupset;': '\u2283\u20d2', 'nsupseteq;': '\u2289', 'nsupseteqq;': '\u2ac6\u0338', 'ntgl;': '\u2279', 'Ntilde': '\xd1', 'ntilde': '\xf1', 'Ntilde;': '\xd1', 'ntilde;': '\xf1', 'ntlg;': '\u2278', 'ntriangleleft;': '\u22ea', 'ntrianglelefteq;': '\u22ec', 'ntriangleright;': '\u22eb', 'ntrianglerighteq;': '\u22ed', 'Nu;': '\u039d', 'nu;': '\u03bd', 'num;': '#', 'numero;': '\u2116', 'numsp;': '\u2007', 'nvap;': '\u224d\u20d2', 'nVDash;': '\u22af', 'nVdash;': '\u22ae', 'nvDash;': '\u22ad', 'nvdash;': '\u22ac', 'nvge;': '\u2265\u20d2', 'nvgt;': '>\u20d2', 'nvHarr;': '\u2904', 'nvinfin;': '\u29de', 'nvlArr;': '\u2902', 'nvle;': '\u2264\u20d2', 'nvlt;': '<\u20d2', 'nvltrie;': '\u22b4\u20d2', 'nvrArr;': '\u2903', 'nvrtrie;': '\u22b5\u20d2', 'nvsim;': '\u223c\u20d2', 'nwarhk;': '\u2923', 'nwArr;': '\u21d6', 'nwarr;': '\u2196', 'nwarrow;': '\u2196', 'nwnear;': '\u2927', 'Oacute': '\xd3', 'oacute': '\xf3', 'Oacute;': '\xd3', 'oacute;': '\xf3', 'oast;': '\u229b', 'ocir;': '\u229a', 'Ocirc': '\xd4', 'ocirc': '\xf4', 'Ocirc;': '\xd4', 'ocirc;': '\xf4', 'Ocy;': '\u041e', 'ocy;': '\u043e', 'odash;': '\u229d', 'Odblac;': '\u0150', 'odblac;': '\u0151', 'odiv;': '\u2a38', 'odot;': '\u2299', 'odsold;': '\u29bc', 'OElig;': '\u0152', 'oelig;': '\u0153', 'ofcir;': '\u29bf', 'Ofr;': '\U0001d512', 'ofr;': '\U0001d52c', 'ogon;': '\u02db', 'Ograve': '\xd2', 'ograve': '\xf2', 'Ograve;': '\xd2', 'ograve;': '\xf2', 'ogt;': '\u29c1', 'ohbar;': '\u29b5', 'ohm;': '\u03a9', 'oint;': '\u222e', 'olarr;': '\u21ba', 'olcir;': '\u29be', 'olcross;': '\u29bb', 'oline;': '\u203e', 'olt;': '\u29c0', 'Omacr;': '\u014c', 'omacr;': '\u014d', 'Omega;': '\u03a9', 'omega;': '\u03c9', 'Omicron;': '\u039f', 'omicron;': '\u03bf', 'omid;': '\u29b6', 'ominus;': '\u2296', 'Oopf;': '\U0001d546', 'oopf;': '\U0001d560', 'opar;': '\u29b7', 'OpenCurlyDoubleQuote;': '\u201c', 'OpenCurlyQuote;': '\u2018', 'operp;': '\u29b9', 'oplus;': '\u2295', 'Or;': '\u2a54', 'or;': '\u2228', 'orarr;': '\u21bb', 'ord;': '\u2a5d', 'order;': '\u2134', 'orderof;': '\u2134', 'ordf': '\xaa', 'ordf;': '\xaa', 'ordm': '\xba', 'ordm;': '\xba', 'origof;': '\u22b6', 'oror;': '\u2a56', 'orslope;': '\u2a57', 'orv;': '\u2a5b', 'oS;': '\u24c8', 'Oscr;': '\U0001d4aa', 'oscr;': '\u2134', 'Oslash': '\xd8', 'oslash': '\xf8', 'Oslash;': '\xd8', 'oslash;': '\xf8', 'osol;': '\u2298', 'Otilde': '\xd5', 'otilde': '\xf5', 'Otilde;': '\xd5', 'otilde;': '\xf5', 'Otimes;': '\u2a37', 'otimes;': '\u2297', 'otimesas;': '\u2a36', 'Ouml': '\xd6', 'ouml': '\xf6', 'Ouml;': '\xd6', 'ouml;': '\xf6', 'ovbar;': '\u233d', 'OverBar;': '\u203e', 'OverBrace;': '\u23de', 'OverBracket;': '\u23b4', 'OverParenthesis;': '\u23dc', 'par;': '\u2225', 'para': '\xb6', 'para;': '\xb6', 'parallel;': '\u2225', 'parsim;': '\u2af3', 'parsl;': '\u2afd', 'part;': '\u2202', 'PartialD;': '\u2202', 'Pcy;': '\u041f', 'pcy;': '\u043f', 'percnt;': '%', 'period;': '.', 'permil;': '\u2030', 'perp;': '\u22a5', 'pertenk;': '\u2031', 'Pfr;': '\U0001d513', 'pfr;': '\U0001d52d', 'Phi;': '\u03a6', 'phi;': '\u03c6', 'phiv;': '\u03d5', 'phmmat;': '\u2133', 'phone;': '\u260e', 'Pi;': '\u03a0', 'pi;': '\u03c0', 'pitchfork;': '\u22d4', 'piv;': '\u03d6', 'planck;': '\u210f', 'planckh;': '\u210e', 'plankv;': '\u210f', 'plus;': '+', 'plusacir;': '\u2a23', 'plusb;': '\u229e', 'pluscir;': '\u2a22', 'plusdo;': '\u2214', 'plusdu;': '\u2a25', 'pluse;': '\u2a72', 'PlusMinus;': '\xb1', 'plusmn': '\xb1', 'plusmn;': '\xb1', 'plussim;': '\u2a26', 'plustwo;': '\u2a27', 'pm;': '\xb1', 'Poincareplane;': '\u210c', 'pointint;': '\u2a15', 'Popf;': '\u2119', 'popf;': '\U0001d561', 'pound': '\xa3', 'pound;': '\xa3', 'Pr;': '\u2abb', 'pr;': '\u227a', 'prap;': '\u2ab7', 'prcue;': '\u227c', 'prE;': '\u2ab3', 'pre;': '\u2aaf', 'prec;': '\u227a', 'precapprox;': '\u2ab7', 'preccurlyeq;': '\u227c', 'Precedes;': '\u227a', 'PrecedesEqual;': '\u2aaf', 'PrecedesSlantEqual;': '\u227c', 'PrecedesTilde;': '\u227e', 'preceq;': '\u2aaf', 'precnapprox;': '\u2ab9', 'precneqq;': '\u2ab5', 'precnsim;': '\u22e8', 'precsim;': '\u227e', 'Prime;': '\u2033', 'prime;': '\u2032', 'primes;': '\u2119', 'prnap;': '\u2ab9', 'prnE;': '\u2ab5', 'prnsim;': '\u22e8', 'prod;': '\u220f', 'Product;': '\u220f', 'profalar;': '\u232e', 'profline;': '\u2312', 'profsurf;': '\u2313', 'prop;': '\u221d', 'Proportion;': '\u2237', 'Proportional;': '\u221d', 'propto;': '\u221d', 'prsim;': '\u227e', 'prurel;': '\u22b0', 'Pscr;': '\U0001d4ab', 'pscr;': '\U0001d4c5', 'Psi;': '\u03a8', 'psi;': '\u03c8', 'puncsp;': '\u2008', 'Qfr;': '\U0001d514', 'qfr;': '\U0001d52e', 'qint;': '\u2a0c', 'Qopf;': '\u211a', 'qopf;': '\U0001d562', 'qprime;': '\u2057', 'Qscr;': '\U0001d4ac', 'qscr;': '\U0001d4c6', 'quaternions;': '\u210d', 'quatint;': '\u2a16', 'quest;': '?', 'questeq;': '\u225f', 'QUOT': '"', 'quot': '"', 'QUOT;': '"', 'quot;': '"', 'rAarr;': '\u21db', 'race;': '\u223d\u0331', 'Racute;': '\u0154', 'racute;': '\u0155', 'radic;': '\u221a', 'raemptyv;': '\u29b3', 'Rang;': '\u27eb', 'rang;': '\u27e9', 'rangd;': '\u2992', 'range;': '\u29a5', 'rangle;': '\u27e9', 'raquo': '\xbb', 'raquo;': '\xbb', 'Rarr;': '\u21a0', 'rArr;': '\u21d2', 'rarr;': '\u2192', 'rarrap;': '\u2975', 'rarrb;': '\u21e5', 'rarrbfs;': '\u2920', 'rarrc;': '\u2933', 'rarrfs;': '\u291e', 'rarrhk;': '\u21aa', 'rarrlp;': '\u21ac', 'rarrpl;': '\u2945', 'rarrsim;': '\u2974', 'Rarrtl;': '\u2916', 'rarrtl;': '\u21a3', 'rarrw;': '\u219d', 'rAtail;': '\u291c', 'ratail;': '\u291a', 'ratio;': '\u2236', 'rationals;': '\u211a', 'RBarr;': '\u2910', 'rBarr;': '\u290f', 'rbarr;': '\u290d', 'rbbrk;': '\u2773', 'rbrace;': '}', 'rbrack;': ']', 'rbrke;': '\u298c', 'rbrksld;': '\u298e', 'rbrkslu;': '\u2990', 'Rcaron;': '\u0158', 'rcaron;': '\u0159', 'Rcedil;': '\u0156', 'rcedil;': '\u0157', 'rceil;': '\u2309', 'rcub;': '}', 'Rcy;': '\u0420', 'rcy;': '\u0440', 'rdca;': '\u2937', 'rdldhar;': '\u2969', 'rdquo;': '\u201d', 'rdquor;': '\u201d', 'rdsh;': '\u21b3', 'Re;': '\u211c', 'real;': '\u211c', 'realine;': '\u211b', 'realpart;': '\u211c', 'reals;': '\u211d', 'rect;': '\u25ad', 'REG': '\xae', 'reg': '\xae', 'REG;': '\xae', 'reg;': '\xae', 'ReverseElement;': '\u220b', 'ReverseEquilibrium;': '\u21cb', 'ReverseUpEquilibrium;': '\u296f', 'rfisht;': '\u297d', 'rfloor;': '\u230b', 'Rfr;': '\u211c', 'rfr;': '\U0001d52f', 'rHar;': '\u2964', 'rhard;': '\u21c1', 'rharu;': '\u21c0', 'rharul;': '\u296c', 'Rho;': '\u03a1', 'rho;': '\u03c1', 'rhov;': '\u03f1', 'RightAngleBracket;': '\u27e9', 'RightArrow;': '\u2192', 'Rightarrow;': '\u21d2', 'rightarrow;': '\u2192', 'RightArrowBar;': '\u21e5', 'RightArrowLeftArrow;': '\u21c4', 'rightarrowtail;': '\u21a3', 'RightCeiling;': '\u2309', 'RightDoubleBracket;': '\u27e7', 'RightDownTeeVector;': '\u295d', 'RightDownVector;': '\u21c2', 'RightDownVectorBar;': '\u2955', 'RightFloor;': '\u230b', 'rightharpoondown;': '\u21c1', 'rightharpoonup;': '\u21c0', 'rightleftarrows;': '\u21c4', 'rightleftharpoons;': '\u21cc', 'rightrightarrows;': '\u21c9', 'rightsquigarrow;': '\u219d', 'RightTee;': '\u22a2', 'RightTeeArrow;': '\u21a6', 'RightTeeVector;': '\u295b', 'rightthreetimes;': '\u22cc', 'RightTriangle;': '\u22b3', 'RightTriangleBar;': '\u29d0', 'RightTriangleEqual;': '\u22b5', 'RightUpDownVector;': '\u294f', 'RightUpTeeVector;': '\u295c', 'RightUpVector;': '\u21be', 'RightUpVectorBar;': '\u2954', 'RightVector;': '\u21c0', 'RightVectorBar;': '\u2953', 'ring;': '\u02da', 'risingdotseq;': '\u2253', 'rlarr;': '\u21c4', 'rlhar;': '\u21cc', 'rlm;': '\u200f', 'rmoust;': '\u23b1', 'rmoustache;': '\u23b1', 'rnmid;': '\u2aee', 'roang;': '\u27ed', 'roarr;': '\u21fe', 'robrk;': '\u27e7', 'ropar;': '\u2986', 'Ropf;': '\u211d', 'ropf;': '\U0001d563', 'roplus;': '\u2a2e', 'rotimes;': '\u2a35', 'RoundImplies;': '\u2970', 'rpar;': ')', 'rpargt;': '\u2994', 'rppolint;': '\u2a12', 'rrarr;': '\u21c9', 'Rrightarrow;': '\u21db', 'rsaquo;': '\u203a', 'Rscr;': '\u211b', 'rscr;': '\U0001d4c7', 'Rsh;': '\u21b1', 'rsh;': '\u21b1', 'rsqb;': ']', 'rsquo;': '\u2019', 'rsquor;': '\u2019', 'rthree;': '\u22cc', 'rtimes;': '\u22ca', 'rtri;': '\u25b9', 'rtrie;': '\u22b5', 'rtrif;': '\u25b8', 'rtriltri;': '\u29ce', 'RuleDelayed;': '\u29f4', 'ruluhar;': '\u2968', 'rx;': '\u211e', 'Sacute;': '\u015a', 'sacute;': '\u015b', 'sbquo;': '\u201a', 'Sc;': '\u2abc', 'sc;': '\u227b', 'scap;': '\u2ab8', 'Scaron;': '\u0160', 'scaron;': '\u0161', 'sccue;': '\u227d', 'scE;': '\u2ab4', 'sce;': '\u2ab0', 'Scedil;': '\u015e', 'scedil;': '\u015f', 'Scirc;': '\u015c', 'scirc;': '\u015d', 'scnap;': '\u2aba', 'scnE;': '\u2ab6', 'scnsim;': '\u22e9', 'scpolint;': '\u2a13', 'scsim;': '\u227f', 'Scy;': '\u0421', 'scy;': '\u0441', 'sdot;': '\u22c5', 'sdotb;': '\u22a1', 'sdote;': '\u2a66', 'searhk;': '\u2925', 'seArr;': '\u21d8', 'searr;': '\u2198', 'searrow;': '\u2198', 'sect': '\xa7', 'sect;': '\xa7', 'semi;': ';', 'seswar;': '\u2929', 'setminus;': '\u2216', 'setmn;': '\u2216', 'sext;': '\u2736', 'Sfr;': '\U0001d516', 'sfr;': '\U0001d530', 'sfrown;': '\u2322', 'sharp;': '\u266f', 'SHCHcy;': '\u0429', 'shchcy;': '\u0449', 'SHcy;': '\u0428', 'shcy;': '\u0448', 'ShortDownArrow;': '\u2193', 'ShortLeftArrow;': '\u2190', 'shortmid;': '\u2223', 'shortparallel;': '\u2225', 'ShortRightArrow;': '\u2192', 'ShortUpArrow;': '\u2191', 'shy': '\xad', 'shy;': '\xad', 'Sigma;': '\u03a3', 'sigma;': '\u03c3', 'sigmaf;': '\u03c2', 'sigmav;': '\u03c2', 'sim;': '\u223c', 'simdot;': '\u2a6a', 'sime;': '\u2243', 'simeq;': '\u2243', 'simg;': '\u2a9e', 'simgE;': '\u2aa0', 'siml;': '\u2a9d', 'simlE;': '\u2a9f', 'simne;': '\u2246', 'simplus;': '\u2a24', 'simrarr;': '\u2972', 'slarr;': '\u2190', 'SmallCircle;': '\u2218', 'smallsetminus;': '\u2216', 'smashp;': '\u2a33', 'smeparsl;': '\u29e4', 'smid;': '\u2223', 'smile;': '\u2323', 'smt;': '\u2aaa', 'smte;': '\u2aac', 'smtes;': '\u2aac\ufe00', 'SOFTcy;': '\u042c', 'softcy;': '\u044c', 'sol;': '/', 'solb;': '\u29c4', 'solbar;': '\u233f', 'Sopf;': '\U0001d54a', 'sopf;': '\U0001d564', 'spades;': '\u2660', 'spadesuit;': '\u2660', 'spar;': '\u2225', 'sqcap;': '\u2293', 'sqcaps;': '\u2293\ufe00', 'sqcup;': '\u2294', 'sqcups;': '\u2294\ufe00', 'Sqrt;': '\u221a', 'sqsub;': '\u228f', 'sqsube;': '\u2291', 'sqsubset;': '\u228f', 'sqsubseteq;': '\u2291', 'sqsup;': '\u2290', 'sqsupe;': '\u2292', 'sqsupset;': '\u2290', 'sqsupseteq;': '\u2292', 'squ;': '\u25a1', 'Square;': '\u25a1', 'square;': '\u25a1', 'SquareIntersection;': '\u2293', 'SquareSubset;': '\u228f', 'SquareSubsetEqual;': '\u2291', 'SquareSuperset;': '\u2290', 'SquareSupersetEqual;': '\u2292', 'SquareUnion;': '\u2294', 'squarf;': '\u25aa', 'squf;': '\u25aa', 'srarr;': '\u2192', 'Sscr;': '\U0001d4ae', 'sscr;': '\U0001d4c8', 'ssetmn;': '\u2216', 'ssmile;': '\u2323', 'sstarf;': '\u22c6', 'Star;': '\u22c6', 'star;': '\u2606', 'starf;': '\u2605', 'straightepsilon;': '\u03f5', 'straightphi;': '\u03d5', 'strns;': '\xaf', 'Sub;': '\u22d0', 'sub;': '\u2282', 'subdot;': '\u2abd', 'subE;': '\u2ac5', 'sube;': '\u2286', 'subedot;': '\u2ac3', 'submult;': '\u2ac1', 'subnE;': '\u2acb', 'subne;': '\u228a', 'subplus;': '\u2abf', 'subrarr;': '\u2979', 'Subset;': '\u22d0', 'subset;': '\u2282', 'subseteq;': '\u2286', 'subseteqq;': '\u2ac5', 'SubsetEqual;': '\u2286', 'subsetneq;': '\u228a', 'subsetneqq;': '\u2acb', 'subsim;': '\u2ac7', 'subsub;': '\u2ad5', 'subsup;': '\u2ad3', 'succ;': '\u227b', 'succapprox;': '\u2ab8', 'succcurlyeq;': '\u227d', 'Succeeds;': '\u227b', 'SucceedsEqual;': '\u2ab0', 'SucceedsSlantEqual;': '\u227d', 'SucceedsTilde;': '\u227f', 'succeq;': '\u2ab0', 'succnapprox;': '\u2aba', 'succneqq;': '\u2ab6', 'succnsim;': '\u22e9', 'succsim;': '\u227f', 'SuchThat;': '\u220b', 'Sum;': '\u2211', 'sum;': '\u2211', 'sung;': '\u266a', 'sup1': '\xb9', 'sup1;': '\xb9', 'sup2': '\xb2', 'sup2;': '\xb2', 'sup3': '\xb3', 'sup3;': '\xb3', 'Sup;': '\u22d1', 'sup;': '\u2283', 'supdot;': '\u2abe', 'supdsub;': '\u2ad8', 'supE;': '\u2ac6', 'supe;': '\u2287', 'supedot;': '\u2ac4', 'Superset;': '\u2283', 'SupersetEqual;': '\u2287', 'suphsol;': '\u27c9', 'suphsub;': '\u2ad7', 'suplarr;': '\u297b', 'supmult;': '\u2ac2', 'supnE;': '\u2acc', 'supne;': '\u228b', 'supplus;': '\u2ac0', 'Supset;': '\u22d1', 'supset;': '\u2283', 'supseteq;': '\u2287', 'supseteqq;': '\u2ac6', 'supsetneq;': '\u228b', 'supsetneqq;': '\u2acc', 'supsim;': '\u2ac8', 'supsub;': '\u2ad4', 'supsup;': '\u2ad6', 'swarhk;': '\u2926', 'swArr;': '\u21d9', 'swarr;': '\u2199', 'swarrow;': '\u2199', 'swnwar;': '\u292a', 'szlig': '\xdf', 'szlig;': '\xdf', 'Tab;': '\t', 'target;': '\u2316', 'Tau;': '\u03a4', 'tau;': '\u03c4', 'tbrk;': '\u23b4', 'Tcaron;': '\u0164', 'tcaron;': '\u0165', 'Tcedil;': '\u0162', 'tcedil;': '\u0163', 'Tcy;': '\u0422', 'tcy;': '\u0442', 'tdot;': '\u20db', 'telrec;': '\u2315', 'Tfr;': '\U0001d517', 'tfr;': '\U0001d531', 'there4;': '\u2234', 'Therefore;': '\u2234', 'therefore;': '\u2234', 'Theta;': '\u0398', 'theta;': '\u03b8', 'thetasym;': '\u03d1', 'thetav;': '\u03d1', 'thickapprox;': '\u2248', 'thicksim;': '\u223c', 'ThickSpace;': '\u205f\u200a', 'thinsp;': '\u2009', 'ThinSpace;': '\u2009', 'thkap;': '\u2248', 'thksim;': '\u223c', 'THORN': '\xde', 'thorn': '\xfe', 'THORN;': '\xde', 'thorn;': '\xfe', 'Tilde;': '\u223c', 'tilde;': '\u02dc', 'TildeEqual;': '\u2243', 'TildeFullEqual;': '\u2245', 'TildeTilde;': '\u2248', 'times': '\xd7', 'times;': '\xd7', 'timesb;': '\u22a0', 'timesbar;': '\u2a31', 'timesd;': '\u2a30', 'tint;': '\u222d', 'toea;': '\u2928', 'top;': '\u22a4', 'topbot;': '\u2336', 'topcir;': '\u2af1', 'Topf;': '\U0001d54b', 'topf;': '\U0001d565', 'topfork;': '\u2ada', 'tosa;': '\u2929', 'tprime;': '\u2034', 'TRADE;': '\u2122', 'trade;': '\u2122', 'triangle;': '\u25b5', 'triangledown;': '\u25bf', 'triangleleft;': '\u25c3', 'trianglelefteq;': '\u22b4', 'triangleq;': '\u225c', 'triangleright;': '\u25b9', 'trianglerighteq;': '\u22b5', 'tridot;': '\u25ec', 'trie;': '\u225c', 'triminus;': '\u2a3a', 'TripleDot;': '\u20db', 'triplus;': '\u2a39', 'trisb;': '\u29cd', 'tritime;': '\u2a3b', 'trpezium;': '\u23e2', 'Tscr;': '\U0001d4af', 'tscr;': '\U0001d4c9', 'TScy;': '\u0426', 'tscy;': '\u0446', 'TSHcy;': '\u040b', 'tshcy;': '\u045b', 'Tstrok;': '\u0166', 'tstrok;': '\u0167', 'twixt;': '\u226c', 'twoheadleftarrow;': '\u219e', 'twoheadrightarrow;': '\u21a0', 'Uacute': '\xda', 'uacute': '\xfa', 'Uacute;': '\xda', 'uacute;': '\xfa', 'Uarr;': '\u219f', 'uArr;': '\u21d1', 'uarr;': '\u2191', 'Uarrocir;': '\u2949', 'Ubrcy;': '\u040e', 'ubrcy;': '\u045e', 'Ubreve;': '\u016c', 'ubreve;': '\u016d', 'Ucirc': '\xdb', 'ucirc': '\xfb', 'Ucirc;': '\xdb', 'ucirc;': '\xfb', 'Ucy;': '\u0423', 'ucy;': '\u0443', 'udarr;': '\u21c5', 'Udblac;': '\u0170', 'udblac;': '\u0171', 'udhar;': '\u296e', 'ufisht;': '\u297e', 'Ufr;': '\U0001d518', 'ufr;': '\U0001d532', 'Ugrave': '\xd9', 'ugrave': '\xf9', 'Ugrave;': '\xd9', 'ugrave;': '\xf9', 'uHar;': '\u2963', 'uharl;': '\u21bf', 'uharr;': '\u21be', 'uhblk;': '\u2580', 'ulcorn;': '\u231c', 'ulcorner;': '\u231c', 'ulcrop;': '\u230f', 'ultri;': '\u25f8', 'Umacr;': '\u016a', 'umacr;': '\u016b', 'uml': '\xa8', 'uml;': '\xa8', 'UnderBar;': '_', 'UnderBrace;': '\u23df', 'UnderBracket;': '\u23b5', 'UnderParenthesis;': '\u23dd', 'Union;': '\u22c3', 'UnionPlus;': '\u228e', 'Uogon;': '\u0172', 'uogon;': '\u0173', 'Uopf;': '\U0001d54c', 'uopf;': '\U0001d566', 'UpArrow;': '\u2191', 'Uparrow;': '\u21d1', 'uparrow;': '\u2191', 'UpArrowBar;': '\u2912', 'UpArrowDownArrow;': '\u21c5', 'UpDownArrow;': '\u2195', 'Updownarrow;': '\u21d5', 'updownarrow;': '\u2195', 'UpEquilibrium;': '\u296e', 'upharpoonleft;': '\u21bf', 'upharpoonright;': '\u21be', 'uplus;': '\u228e', 'UpperLeftArrow;': '\u2196', 'UpperRightArrow;': '\u2197', 'Upsi;': '\u03d2', 'upsi;': '\u03c5', 'upsih;': '\u03d2', 'Upsilon;': '\u03a5', 'upsilon;': '\u03c5', 'UpTee;': '\u22a5', 'UpTeeArrow;': '\u21a5', 'upuparrows;': '\u21c8', 'urcorn;': '\u231d', 'urcorner;': '\u231d', 'urcrop;': '\u230e', 'Uring;': '\u016e', 'uring;': '\u016f', 'urtri;': '\u25f9', 'Uscr;': '\U0001d4b0', 'uscr;': '\U0001d4ca', 'utdot;': '\u22f0', 'Utilde;': '\u0168', 'utilde;': '\u0169', 'utri;': '\u25b5', 'utrif;': '\u25b4', 'uuarr;': '\u21c8', 'Uuml': '\xdc', 'uuml': '\xfc', 'Uuml;': '\xdc', 'uuml;': '\xfc', 'uwangle;': '\u29a7', 'vangrt;': '\u299c', 'varepsilon;': '\u03f5', 'varkappa;': '\u03f0', 'varnothing;': '\u2205', 'varphi;': '\u03d5', 'varpi;': '\u03d6', 'varpropto;': '\u221d', 'vArr;': '\u21d5', 'varr;': '\u2195', 'varrho;': '\u03f1', 'varsigma;': '\u03c2', 'varsubsetneq;': '\u228a\ufe00', 'varsubsetneqq;': '\u2acb\ufe00', 'varsupsetneq;': '\u228b\ufe00', 'varsupsetneqq;': '\u2acc\ufe00', 'vartheta;': '\u03d1', 'vartriangleleft;': '\u22b2', 'vartriangleright;': '\u22b3', 'Vbar;': '\u2aeb', 'vBar;': '\u2ae8', 'vBarv;': '\u2ae9', 'Vcy;': '\u0412', 'vcy;': '\u0432', 'VDash;': '\u22ab', 'Vdash;': '\u22a9', 'vDash;': '\u22a8', 'vdash;': '\u22a2', 'Vdashl;': '\u2ae6', 'Vee;': '\u22c1', 'vee;': '\u2228', 'veebar;': '\u22bb', 'veeeq;': '\u225a', 'vellip;': '\u22ee', 'Verbar;': '\u2016', 'verbar;': '|', 'Vert;': '\u2016', 'vert;': '|', 'VerticalBar;': '\u2223', 'VerticalLine;': '|', 'VerticalSeparator;': '\u2758', 'VerticalTilde;': '\u2240', 'VeryThinSpace;': '\u200a', 'Vfr;': '\U0001d519', 'vfr;': '\U0001d533', 'vltri;': '\u22b2', 'vnsub;': '\u2282\u20d2', 'vnsup;': '\u2283\u20d2', 'Vopf;': '\U0001d54d', 'vopf;': '\U0001d567', 'vprop;': '\u221d', 'vrtri;': '\u22b3', 'Vscr;': '\U0001d4b1', 'vscr;': '\U0001d4cb', 'vsubnE;': '\u2acb\ufe00', 'vsubne;': '\u228a\ufe00', 'vsupnE;': '\u2acc\ufe00', 'vsupne;': '\u228b\ufe00', 'Vvdash;': '\u22aa', 'vzigzag;': '\u299a', 'Wcirc;': '\u0174', 'wcirc;': '\u0175', 'wedbar;': '\u2a5f', 'Wedge;': '\u22c0', 'wedge;': '\u2227', 'wedgeq;': '\u2259', 'weierp;': '\u2118', 'Wfr;': '\U0001d51a', 'wfr;': '\U0001d534', 'Wopf;': '\U0001d54e', 'wopf;': '\U0001d568', 'wp;': '\u2118', 'wr;': '\u2240', 'wreath;': '\u2240', 'Wscr;': '\U0001d4b2', 'wscr;': '\U0001d4cc', 'xcap;': '\u22c2', 'xcirc;': '\u25ef', 'xcup;': '\u22c3', 'xdtri;': '\u25bd', 'Xfr;': '\U0001d51b', 'xfr;': '\U0001d535', 'xhArr;': '\u27fa', 'xharr;': '\u27f7', 'Xi;': '\u039e', 'xi;': '\u03be', 'xlArr;': '\u27f8', 'xlarr;': '\u27f5', 'xmap;': '\u27fc', 'xnis;': '\u22fb', 'xodot;': '\u2a00', 'Xopf;': '\U0001d54f', 'xopf;': '\U0001d569', 'xoplus;': '\u2a01', 'xotime;': '\u2a02', 'xrArr;': '\u27f9', 'xrarr;': '\u27f6', 'Xscr;': '\U0001d4b3', 'xscr;': '\U0001d4cd', 'xsqcup;': '\u2a06', 'xuplus;': '\u2a04', 'xutri;': '\u25b3', 'xvee;': '\u22c1', 'xwedge;': '\u22c0', 'Yacute': '\xdd', 'yacute': '\xfd', 'Yacute;': '\xdd', 'yacute;': '\xfd', 'YAcy;': '\u042f', 'yacy;': '\u044f', 'Ycirc;': '\u0176', 'ycirc;': '\u0177', 'Ycy;': '\u042b', 'ycy;': '\u044b', 'yen': '\xa5', 'yen;': '\xa5', 'Yfr;': '\U0001d51c', 'yfr;': '\U0001d536', 'YIcy;': '\u0407', 'yicy;': '\u0457', 'Yopf;': '\U0001d550', 'yopf;': '\U0001d56a', 'Yscr;': '\U0001d4b4', 'yscr;': '\U0001d4ce', 'YUcy;': '\u042e', 'yucy;': '\u044e', 'yuml': '\xff', 'Yuml;': '\u0178', 'yuml;': '\xff', 'Zacute;': '\u0179', 'zacute;': '\u017a', 'Zcaron;': '\u017d', 'zcaron;': '\u017e', 'Zcy;': '\u0417', 'zcy;': '\u0437', 'Zdot;': '\u017b', 'zdot;': '\u017c', 'zeetrf;': '\u2128', 'ZeroWidthSpace;': '\u200b', 'Zeta;': '\u0396', 'zeta;': '\u03b6', 'Zfr;': '\u2128', 'zfr;': '\U0001d537', 'ZHcy;': '\u0416', 'zhcy;': '\u0436', 'zigrarr;': '\u21dd', 'Zopf;': '\u2124', 'zopf;': '\U0001d56b', 'Zscr;': '\U0001d4b5', 'zscr;': '\U0001d4cf', 'zwj;': '\u200d', 'zwnj;': '\u200c', } _invalid_charrefs = { 0x00: '\ufffd', # REPLACEMENT CHARACTER 0x0d: '\r', # CARRIAGE RETURN 0x80: '\u20ac', # EURO SIGN 0x81: '\x81', # 0x82: '\u201a', # SINGLE LOW-9 QUOTATION MARK 0x83: '\u0192', # LATIN SMALL LETTER F WITH HOOK 0x84: '\u201e', # DOUBLE LOW-9 QUOTATION MARK 0x85: '\u2026', # HORIZONTAL ELLIPSIS 0x86: '\u2020', # DAGGER 0x87: '\u2021', # DOUBLE DAGGER 0x88: '\u02c6', # MODIFIER LETTER CIRCUMFLEX ACCENT 0x89: '\u2030', # PER MILLE SIGN 0x8a: '\u0160', # LATIN CAPITAL LETTER S WITH CARON 0x8b: '\u2039', # SINGLE LEFT-POINTING ANGLE QUOTATION MARK 0x8c: '\u0152', # LATIN CAPITAL LIGATURE OE 0x8d: '\x8d', # 0x8e: '\u017d', # LATIN CAPITAL LETTER Z WITH CARON 0x8f: '\x8f', # 0x90: '\x90', # 0x91: '\u2018', # LEFT SINGLE QUOTATION MARK 0x92: '\u2019', # RIGHT SINGLE QUOTATION MARK 0x93: '\u201c', # LEFT DOUBLE QUOTATION MARK 0x94: '\u201d', # RIGHT DOUBLE QUOTATION MARK 0x95: '\u2022', # BULLET 0x96: '\u2013', # EN DASH 0x97: '\u2014', # EM DASH 0x98: '\u02dc', # SMALL TILDE 0x99: '\u2122', # TRADE MARK SIGN 0x9a: '\u0161', # LATIN SMALL LETTER S WITH CARON 0x9b: '\u203a', # SINGLE RIGHT-POINTING ANGLE QUOTATION MARK 0x9c: '\u0153', # LATIN SMALL LIGATURE OE 0x9d: '\x9d', # 0x9e: '\u017e', # LATIN SMALL LETTER Z WITH CARON 0x9f: '\u0178', # LATIN CAPITAL LETTER Y WITH DIAERESIS } _invalid_codepoints = set([ # 0x0001 to 0x0008 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, # 0x000E to 0x001F 0xe, 0xf, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, # 0x007F to 0x009F 0x7f, 0x80, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, 0x89, 0x8a, 0x8b, 0x8c, 0x8d, 0x8e, 0x8f, 0x90, 0x91, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98, 0x99, 0x9a, 0x9b, 0x9c, 0x9d, 0x9e, 0x9f, # 0xFDD0 to 0xFDEF 0xfdd0, 0xfdd1, 0xfdd2, 0xfdd3, 0xfdd4, 0xfdd5, 0xfdd6, 0xfdd7, 0xfdd8, 0xfdd9, 0xfdda, 0xfddb, 0xfddc, 0xfddd, 0xfdde, 0xfddf, 0xfde0, 0xfde1, 0xfde2, 0xfde3, 0xfde4, 0xfde5, 0xfde6, 0xfde7, 0xfde8, 0xfde9, 0xfdea, 0xfdeb, 0xfdec, 0xfded, 0xfdee, 0xfdef, # others 0xb, 0xfffe, 0xffff, 0x1fffe, 0x1ffff, 0x2fffe, 0x2ffff, 0x3fffe, 0x3ffff, 0x4fffe, 0x4ffff, 0x5fffe, 0x5ffff, 0x6fffe, 0x6ffff, 0x7fffe, 0x7ffff, 0x8fffe, 0x8ffff, 0x9fffe, 0x9ffff, 0xafffe, 0xaffff, 0xbfffe, 0xbffff, 0xcfffe, 0xcffff, 0xdfffe, 0xdffff, 0xefffe, 0xeffff, 0xffffe, 0xfffff, 0x10fffe, 0x10ffff ]) def _replace_charref(s): s = s.group(1) if s[0] == '#': # numeric charref if s[1] in 'xX': num = int(s[2:].rstrip(';'), 16) else: num = int(s[1:].rstrip(';')) if num in _invalid_charrefs: return _invalid_charrefs[num] if 0xD800 <= num <= 0xDFFF or num > 0x10FFFF: return '\uFFFD' if num in _invalid_codepoints: return '' if sys.version_info >= (3, 0): return chr(num) else: return unichr(num) # noqa else: # named charref if s in _html5: return _html5[s] # find the longest matching name (as defined by the standard) for x in range(len(s)-1, 1, -1): if s[:x] in _html5: return _html5[s[:x]] + s[x:] else: return '&' + s _charref = _re.compile(r'&(#[0-9]+;?' r'|#[xX][0-9a-fA-F]+;?' r'|[^\t\n\f <&#;]{1,32};?)') def _unescape(s): """ Convert all named and numeric character references (e.g. >, >, &x3e;) in the string s to the corresponding unicode characters. This function uses the rules defined by the HTML 5 standard for both valid and invalid character references, and the list of HTML 5 named character references defined in html.entities.html5. """ if '&' not in s: return s return _charref.sub(_replace_charref, s) commonmark-0.9.1/commonmark/inlines.py0000644000175000017500000007441113462635760017451 0ustar niknik00000000000000from __future__ import absolute_import, unicode_literals, division import re import sys from commonmark import common from commonmark.common import normalize_uri, unescape_string from commonmark.node import Node from commonmark.normalize_reference import normalize_reference if sys.version_info >= (3, 0): if sys.version_info >= (3, 4): import html HTMLunescape = html.unescape else: from .entitytrans import _unescape HTMLunescape = _unescape else: from commonmark import entitytrans HTMLunescape = entitytrans._unescape # Some regexps used in inline parser: ESCAPED_CHAR = '\\\\' + common.ESCAPABLE rePunctuation = re.compile( r'[!"#$%&\'()*+,\-./:;<=>?@\[\]\\^_`{|}~\xA1\xA7\xAB\xB6\xB7\xBB' r'\xBF\u037E\u0387\u055A-\u055F\u0589\u058A\u05BE\u05C0\u05C3' r'\u05C6\u05F3\u05F4\u0609\u060A\u060C\u060D\u061B\u061E\u061F' r'\u066A-\u066D\u06D4\u0700-\u070D\u07F7-\u07F9\u0830-\u083E' r'\u085E\u0964\u0965\u0970\u0AF0\u0DF4\u0E4F\u0E5A\u0E5B\u0F04-\u0F12' r'\u0F14\u0F3A-\u0F3D\u0F85\u0FD0-\u0FD4\u0FD9\u0FDA\u104A-\u104F\u10FB' r'\u1360-\u1368\u1400\u166D\u166E\u169B\u169C\u16EB-\u16ED\u1735\u1736' r'\u17D4-\u17D6\u17D8-\u17DA\u1800-\u180A\u1944\u1945\u1A1E\u1A1F\u1AA0-' r'\u1AA6\u1AA8-\u1AAD\u1B5A-\u1B60\u1BFC-\u1BFF\u1C3B-\u1C3F\u1C7E\u1C7F' r'\u1CC0-\u1CC7\u1CD3\u2010-\u2027\u2030-\u2043\u2045-\u2051\u2053-\u205E' r'\u207D\u207E\u208D\u208E\u2308-\u230B\u2329\u232A\u2768-\u2775\u27C5' r'\u27C6\u27E6-\u27EF\u2983-\u2998\u29D8-\u29DB\u29FC\u29FD\u2CF9-\u2CFC' r'\u2CFE\u2CFF\u2D70\u2E00-\u2E2E\u2E30-\u2E42\u3001-\u3003\u3008-\u3011' r'\u3014-\u301F\u3030\u303D\u30A0\u30FB\uA4FE\uA4FF\uA60D-\uA60F\uA673' r'\uA67E\uA6F2-\uA6F7\uA874-\uA877\uA8CE\uA8CF\uA8F8-\uA8FA\uA8FC\uA92E' r'\uA92F\uA95F\uA9C1-\uA9CD\uA9DE\uA9DF\uAA5C-\uAA5F\uAADE\uAADF\uAAF0' r'\uAAF1\uABEB\uFD3E\uFD3F\uFE10-\uFE19\uFE30-\uFE52\uFE54-\uFE61\uFE63' r'\uFE68\uFE6A\uFE6B\uFF01-\uFF03\uFF05-\uFF0A\uFF0C-\uFF0F\uFF1A\uFF1B' r'\uFF1F\uFF20\uFF3B-\uFF3D\uFF3F\uFF5B\uFF5D\uFF5F-\uFF65]|\uD800[\uDD00-' r'\uDD02\uDF9F\uDFD0]|\uD801\uDD6F|\uD802[\uDC57\uDD1F\uDD3F\uDE50-\uDE58' r'\uDE7F\uDEF0-\uDEF6\uDF39-\uDF3F\uDF99-\uDF9C]|\uD804[\uDC47-\uDC4D' r'\uDCBB\uDCBC\uDCBE-\uDCC1\uDD40-\uDD43\uDD74\uDD75\uDDC5-\uDDC9\uDDCD' r'\uDDDB\uDDDD-\uDDDF\uDE38-\uDE3D\uDEA9]|\uD805[\uDCC6\uDDC1-\uDDD7' r'\uDE41-\uDE43\uDF3C-\uDF3E]|\uD809[\uDC70-\uDC74]|\uD81A[\uDE6E\uDE6F' r'\uDEF5\uDF37-\uDF3B\uDF44]|\uD82F\uDC9F|\uD836[\uDE87-\uDE8B]' ) reLinkTitle = re.compile( '^(?:"(' + ESCAPED_CHAR + '|[^"\\x00])*"' + '|' + '\'(' + ESCAPED_CHAR + '|[^\'\\x00])*\'' + '|' + '\\((' + ESCAPED_CHAR + '|[^()\\x00])*\\))') reLinkDestinationBraces = re.compile(r'^(?:<(?:[^<>\n\\\x00]|\\.)*>)') reEscapable = re.compile('^' + common.ESCAPABLE) reEntityHere = re.compile('^' + common.ENTITY, re.IGNORECASE) reTicks = re.compile(r'`+') reTicksHere = re.compile(r'^`+') reEllipses = re.compile(r'\.\.\.') reDash = re.compile(r'--+') reEmailAutolink = re.compile( r"^<([a-zA-Z0-9.!#$%&'*+\/=?^_`{|}~-]+@[a-zA-Z0-9]" r"(?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?" r"(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*)>") reAutolink = re.compile( r'^<[A-Za-z][A-Za-z0-9.+-]{1,31}:[^<>\x00-\x20]*>', re.IGNORECASE) reSpnl = re.compile(r'^ *(?:\n *)?') reWhitespaceChar = re.compile(r'^^[ \t\n\x0b\x0c\x0d]') reWhitespace = re.compile(r'[ \t\n\x0b\x0c\x0d]+') reUnicodeWhitespaceChar = re.compile(r'^\s') reFinalSpace = re.compile(r' *$') reInitialSpace = re.compile(r'^ *') reSpaceAtEndOfLine = re.compile(r'^ *(?:\n|$)') reLinkLabel = re.compile(r'^\[(?:[^\\\[\]]|\\.){0,1000}\]') # Matches a string of non-special characters. reMain = re.compile(r'^[^\n`\[\]\\!<&*_\'"]+', re.MULTILINE) def text(s): node = Node('text', None) node.literal = s return node def smart_dashes(chars): en_count = 0 em_count = 0 if len(chars) % 3 == 0: # If divisible by 3, use all em dashes em_count = len(chars) // 3 elif len(chars) % 2 == 0: # If divisble by 2, use all en dashes en_count = len(chars) // 2 elif len(chars) % 3 == 2: # if 2 extra dashes, use en dashfor last 2; # em dashes for rest en_count = 1 em_count = (len(chars) - 2) // 3 else: # Use en dashes for last 4 hyphens; em dashes for rest en_count = 2 em_count = (len(chars) - 4) // 3 return ('\u2014' * em_count) + ('\u2013' * en_count) class InlineParser(object): """INLINE PARSER These are methods of an InlineParser class, defined below. An InlineParser keeps track of a subject (a string to be parsed) and a position in that subject. """ def __init__(self, options={}): self.subject = '' self.brackets = None self.pos = 0 self.refmap = {} self.options = options def match(self, regexString): """ If regexString matches at current position in the subject, advance position in subject and return the match; otherwise return None. """ match = re.search(regexString, self.subject[self.pos:]) if match is None: return None else: self.pos += match.end() return match.group() def peek(self): """ Returns the character at the current subject position, or None if there are no more characters.""" if self.pos < len(self.subject): return self.subject[self.pos] else: return None def spnl(self): """ Parse zero or more space characters, including at most one newline.""" self.match(reSpnl) return True # All of the parsers below try to match something at the current position # in the subject. If they succeed in matching anything, they # push an inline matched, advancing the subject. def parseBackticks(self, block): """ Attempt to parse backticks, adding either a backtick code span or a literal sequence of backticks to the 'inlines' list.""" ticks = self.match(reTicksHere) if ticks is None: return False after_open_ticks = self.pos matched = self.match(reTicks) while matched is not None: if matched == ticks: node = Node('code', None) contents = self.subject[after_open_ticks:self.pos-len(ticks)] \ .replace('\n', ' ') if contents.lstrip(' ') and contents[0] == contents[-1] == ' ': node.literal = contents[1:-1] else: node.literal = contents block.append_child(node) return True matched = self.match(reTicks) # If we got here, we didn't match a closing backtick sequence. self.pos = after_open_ticks block.append_child(text(ticks)) return True def parseBackslash(self, block): """ Parse a backslash-escaped special character, adding either the escaped character, a hard line break (if the backslash is followed by a newline), or a literal backslash to the block's children. Assumes current character is a backslash. """ subj = self.subject self.pos += 1 try: subjchar = subj[self.pos] except IndexError: subjchar = None if self.peek() == '\n': self.pos += 1 node = Node('linebreak', None) block.append_child(node) elif subjchar and re.search(reEscapable, subjchar): block.append_child(text(subjchar)) self.pos += 1 else: block.append_child(text('\\')) return True def parseAutolink(self, block): """Attempt to parse an autolink (URL or email in pointy brackets).""" m = self.match(reEmailAutolink) if m: # email dest = m[1:-1] node = Node('link', None) node.destination = normalize_uri('mailto:' + dest) node.title = '' node.append_child(text(dest)) block.append_child(node) return True else: m = self.match(reAutolink) if m: # link dest = m[1:-1] node = Node('link', None) node.destination = normalize_uri(dest) node.title = '' node.append_child(text(dest)) block.append_child(node) return True return False def parseHtmlTag(self, block): """Attempt to parse a raw HTML tag.""" m = self.match(common.reHtmlTag) if m is None: return False else: node = Node('html_inline', None) node.literal = m block.append_child(node) return True def scanDelims(self, c): """ Scan a sequence of characters == c, and return information about the number of delimiters and whether they are positioned such that they can open and/or close emphasis or strong emphasis. A utility function for strong/emph parsing. """ numdelims = 0 startpos = self.pos if c == "'" or c == '"': numdelims += 1 self.pos += 1 else: while (self.peek() == c): numdelims += 1 self.pos += 1 if numdelims == 0: return None c_before = '\n' if startpos == 0 else self.subject[startpos - 1] c_after = self.peek() if c_after is None: c_after = '\n' # Python 2 doesn't recognize '\xa0' as whitespace after_is_whitespace = re.search(reUnicodeWhitespaceChar, c_after) or \ c_after == '\xa0' after_is_punctuation = re.search(rePunctuation, c_after) before_is_whitespace = re.search( reUnicodeWhitespaceChar, c_before) or \ c_before == '\xa0' before_is_punctuation = re.search(rePunctuation, c_before) left_flanking = not after_is_whitespace and \ (not after_is_punctuation or before_is_whitespace or before_is_punctuation) right_flanking = not before_is_whitespace and \ (not before_is_punctuation or after_is_whitespace or after_is_punctuation) if c == '_': can_open = left_flanking and \ (not right_flanking or before_is_punctuation) can_close = right_flanking and \ (not left_flanking or after_is_punctuation) elif c == "'" or c == '"': can_open = left_flanking and not right_flanking can_close = right_flanking else: can_open = left_flanking can_close = right_flanking self.pos = startpos return { 'numdelims': numdelims, 'can_open': can_open, 'can_close': can_close, } def handleDelim(self, cc, block): """Handle a delimiter marker for emphasis or a quote.""" res = self.scanDelims(cc) if not res: return False numdelims = res.get('numdelims') startpos = self.pos self.pos += numdelims if cc == "'": contents = '\u2019' elif cc == '"': contents = '\u201C' else: contents = self.subject[startpos:self.pos] node = text(contents) block.append_child(node) # Add entry to stack for this opener self.delimiters = { 'cc': cc, 'numdelims': numdelims, 'origdelims': numdelims, 'node': node, 'previous': self.delimiters, 'next': None, 'can_open': res.get('can_open'), 'can_close': res.get('can_close'), } if self.delimiters['previous'] is not None: self.delimiters['previous']['next'] = self.delimiters return True def removeDelimiter(self, delim): if delim.get('previous') is not None: delim['previous']['next'] = delim.get('next') if delim.get('next') is None: # Top of stack self.delimiters = delim.get('previous') else: delim['next']['previous'] = delim.get('previous') @staticmethod def removeDelimitersBetween(bottom, top): if bottom.get('next') != top: bottom['next'] = top top['previous'] = bottom def processEmphasis(self, stack_bottom): openers_bottom = { '_': stack_bottom, '*': stack_bottom, "'": stack_bottom, '"': stack_bottom, } odd_match = False use_delims = 0 # Find first closer above stack_bottom closer = self.delimiters while closer is not None and closer.get('previous') != stack_bottom: closer = closer.get('previous') # Move forward, looking for closers, and handling each while closer is not None: if not closer.get('can_close'): closer = closer.get('next') else: # found emphasis closer. now look back for first # matching opener: opener = closer.get('previous') opener_found = False closercc = closer.get('cc') while (opener is not None and opener != stack_bottom and opener != openers_bottom[closercc]): odd_match = (closer.get('can_open') or opener.get('can_close')) and \ closer['origdelims'] % 3 != 0 and \ (opener['origdelims'] + closer['origdelims']) % 3 == 0 if opener.get('cc') == closercc and \ opener.get('can_open') and \ not odd_match: opener_found = True break opener = opener.get('previous') old_closer = closer if closercc == '*' or closercc == '_': if not opener_found: closer = closer.get('next') else: # Calculate actual number of delimiters used from # closer use_delims = 2 if ( closer['numdelims'] >= 2 and opener['numdelims'] >= 2) else 1 opener_inl = opener.get('node') closer_inl = closer.get('node') # Remove used delimiters from stack elts and inlines opener['numdelims'] -= use_delims closer['numdelims'] -= use_delims opener_inl.literal = opener_inl.literal[ :len(opener_inl.literal) - use_delims] closer_inl.literal = closer_inl.literal[ :len(closer_inl.literal) - use_delims] # Build contents for new Emph element if use_delims == 1: emph = Node('emph', None) else: emph = Node('strong', None) tmp = opener_inl.nxt while tmp and tmp != closer_inl: nxt = tmp.nxt tmp.unlink() emph.append_child(tmp) tmp = nxt opener_inl.insert_after(emph) # Remove elts between opener and closer in delimiters # stack self.removeDelimitersBetween(opener, closer) # If opener has 0 delims, remove it and the inline if opener['numdelims'] == 0: opener_inl.unlink() self.removeDelimiter(opener) if closer['numdelims'] == 0: closer_inl.unlink() tempstack = closer['next'] self.removeDelimiter(closer) closer = tempstack elif closercc == "'": closer['node'].literal = '\u2019' if opener_found: opener['node'].literal = '\u2018' closer = closer['next'] elif closercc == '"': closer['node'].literal = '\u201D' if opener_found: opener['node'].literal = '\u201C' closer = closer['next'] if not opener_found and not odd_match: # Set lower bound for future searches for openers: # We don't do this with odd_match because a ** # that doesn't match an earlier * might turn into # an opener, and the * might be matched by something # else. openers_bottom[closercc] = old_closer['previous'] if not old_closer['can_open']: # We can remove a closer that can't be an opener, # once we've seen there's no matching opener: self.removeDelimiter(old_closer) # Remove all delimiters while self.delimiters is not None and self.delimiters != stack_bottom: self.removeDelimiter(self.delimiters) def parseLinkTitle(self): """ Attempt to parse link title (sans quotes), returning the string or None if no match. """ title = self.match(reLinkTitle) if title is None: return None else: # chop off quotes from title and unescape: return unescape_string(title[1:-1]) def parseLinkDestination(self): """ Attempt to parse link destination, returning the string or None if no match. """ res = self.match(reLinkDestinationBraces) if res is None: if self.peek() == '<': return None # TODO handrolled parser; res should be None or the string savepos = self.pos openparens = 0 while True: c = self.peek() if c is None: break if c == '\\' and re.search( reEscapable, self.subject[self.pos+1:self.pos+2]): self.pos += 1 if self.peek() is not None: self.pos += 1 elif c == '(': self.pos += 1 openparens += 1 elif c == ')': if openparens < 1: break else: self.pos += 1 openparens -= 1 elif re.search(reWhitespaceChar, c): break else: self.pos += 1 if self.pos == savepos and c != ')': return None res = self.subject[savepos:self.pos] return normalize_uri(unescape_string(res)) else: # chop off surrounding <..>: return normalize_uri(unescape_string(res[1:-1])) def parseLinkLabel(self): """ Attempt to parse a link label, returning number of characters parsed. """ # Note: our regex will allow something of form [..\]; # we disallow it here rather than using lookahead in the regex: m = self.match(reLinkLabel) if m is None or len(m) > 1001: return 0 else: return len(m) def parseOpenBracket(self, block): """ Add open bracket to delimiter stack and add a text node to block's children. """ startpos = self.pos self.pos += 1 node = text('[') block.append_child(node) # Add entry to stack for this opener self.addBracket(node, startpos, False) return True def parseBang(self, block): """ If next character is [, and ! delimiter to delimiter stack and add a text node to block's children. Otherwise just add a text node. """ startpos = self.pos self.pos += 1 if self.peek() == '[': self.pos += 1 node = text('![') block.append_child(node) # Add entry to stack for this openeer self.addBracket(node, startpos + 1, True) else: block.append_child(text('!')) return True def parseCloseBracket(self, block): """ Try to match close bracket against an opening in the delimiter stack. Add either a link or image, or a plain [ character, to block's children. If there is a matching delimiter, remove it from the delimiter stack. """ title = None matched = False self.pos += 1 startpos = self.pos # get last [ or ![ opener = self.brackets if opener is None: # no matched opener, just return a literal block.append_child(text(']')) return True if not opener.get('active'): # no matched opener, just return a literal block.append_child(text(']')) # take opener off brackets stack self.removeBracket() return True # If we got here, opener is a potential opener is_image = opener.get('image') # Check to see if we have a link/image savepos = self.pos # Inline link? if self.peek() == '(': self.pos += 1 self.spnl() dest = self.parseLinkDestination() if dest is not None and self.spnl(): # make sure there's a space before the title if re.search(reWhitespaceChar, self.subject[self.pos-1]): title = self.parseLinkTitle() if self.spnl() and self.peek() == ')': self.pos += 1 matched = True else: self.pos = savepos if not matched: # Next, see if there's a link label beforelabel = self.pos n = self.parseLinkLabel() if n > 2: reflabel = self.subject[beforelabel:beforelabel + n] elif not opener.get('bracket_after'): # Empty or missing second label means to use the first # label as the reference. The reference must not # contain a bracket. If we know there's a bracket, we # don't even bother checking it. reflabel = self.subject[opener.get('index'):startpos] if n == 0: # If shortcut reference link, rewind before spaces we skipped. self.pos = savepos if reflabel: # lookup rawlabel in refmap link = self.refmap.get(normalize_reference(reflabel)) if link: dest = link['destination'] title = link['title'] matched = True if matched: node = Node('image' if is_image else 'link', None) node.destination = dest node.title = title or '' tmp = opener.get('node').nxt while tmp: nxt = tmp.nxt tmp.unlink() node.append_child(tmp) tmp = nxt block.append_child(node) self.processEmphasis(opener.get('previousDelimiter')) self.removeBracket() opener.get('node').unlink() # We remove this bracket and processEmphasis will remove # later delimiters. # Now, for a link, we also deactivate earlier link openers. # (no links in links) if not is_image: opener = self.brackets while opener is not None: if not opener.get('image'): # deactivate this opener opener['active'] = False opener = opener.get('previous') return True else: # no match # remove this opener from stack self.removeBracket() self.pos = startpos block.append_child(text(']')) return True def addBracket(self, node, index, image): if self.brackets is not None: self.brackets['bracketAfter'] = True self.brackets = { 'node': node, 'previous': self.brackets, 'previousDelimiter': self.delimiters, 'index': index, 'image': image, 'active': True, } def removeBracket(self): self.brackets = self.brackets.get('previous') def parseEntity(self, block): """Attempt to parse an entity.""" m = self.match(reEntityHere) if m: block.append_child(text(HTMLunescape(m))) return True else: return False def parseString(self, block): """ Parse a run of ordinary characters, or a single character with a special meaning in markdown, as a plain string. """ m = self.match(reMain) if m: if self.options.get('smart'): s = re.sub(reEllipses, '\u2026', m) s = re.sub(reDash, lambda x: smart_dashes(x.group()), s) block.append_child(text(s)) else: block.append_child(text(m)) return True else: return False def parseNewline(self, block): """ Parse a newline. If it was preceded by two spaces, return a hard line break; otherwise a soft line break. """ # assume we're at a \n self.pos += 1 lastc = block.last_child if lastc and lastc.t == 'text' and lastc.literal[-1] == ' ': linebreak = len(lastc.literal) >= 2 and lastc.literal[-2] == ' ' lastc.literal = re.sub(reFinalSpace, '', lastc.literal) if linebreak: node = Node('linebreak', None) else: node = Node('softbreak', None) block.append_child(node) else: block.append_child(Node('softbreak', None)) # gobble leading spaces in next line self.match(reInitialSpace) return True def parseReference(self, s, refmap): """Attempt to parse a link reference, modifying refmap.""" self.subject = s self.pos = 0 startpos = self.pos # label: match_chars = self.parseLinkLabel() if match_chars == 0 or match_chars == 2: return 0 else: rawlabel = self.subject[:match_chars] # colon: if (self.peek() == ':'): self.pos += 1 else: self.pos = startpos return 0 # link url self.spnl() dest = self.parseLinkDestination() if dest is None: self.pos = startpos return 0 beforetitle = self.pos self.spnl() title = None if self.pos != beforetitle: title = self.parseLinkTitle() if title is None: title = '' # rewind before spaces self.pos = beforetitle # make sure we're at line end: at_line_end = True if self.match(reSpaceAtEndOfLine) is None: if title == '': at_line_end = False else: # the potential title we found is not at the line end, # but it could still be a legal link reference if we # discard the title title == '' # rewind before spaces self.pos = beforetitle # and instead check if the link URL is at the line end at_line_end = self.match(reSpaceAtEndOfLine) is not None if not at_line_end: self.pos = startpos return 0 normlabel = normalize_reference(rawlabel) if normlabel == '': # label must contain non-whitespace characters self.pos = startpos return 0 if not refmap.get(normlabel): refmap[normlabel] = { 'destination': dest, 'title': title } return (self.pos - startpos) def parseInline(self, block): """ Parse the next inline element in subject, advancing subject position. On success, add the result to block's children and return True. On failure, return False. """ res = False c = self.peek() if c is None: return False if c == '\n': res = self.parseNewline(block) elif c == '\\': res = self.parseBackslash(block) elif c == '`': res = self.parseBackticks(block) elif c == '*' or c == '_': res = self.handleDelim(c, block) elif c == "'" or c == '"': res = self.options.get('smart') and self.handleDelim(c, block) elif c == '[': res = self.parseOpenBracket(block) elif c == '!': res = self.parseBang(block) elif c == ']': res = self.parseCloseBracket(block) elif c == '<': res = self.parseAutolink(block) or self.parseHtmlTag(block) elif c == '&': res = self.parseEntity(block) else: res = self.parseString(block) if not res: self.pos += 1 block.append_child(text(c)) return True def parseInlines(self, block): """ Parse string content in block into inline children, using refmap to resolve references. """ self.subject = block.string_content.strip() self.pos = 0 self.delimiters = None self.brackets = None while (self.parseInline(block)): pass # allow raw string to be garbage collected block.string_content = None self.processEmphasis(None) parse = parseInlines commonmark-0.9.1/commonmark/main.py0000755000175000017500000000236413343503214016717 0ustar niknik00000000000000# 2014 - Bibek Kafle & Roland Shoemaker # 2015-2017 - Nikolas Nyby # Port of @jgm's commonmark.js implementation of the CommonMark spec. # Basic usage: # # import commonmark # parser = commonmark.Parser() # renderer = commonmark.HtmlRenderer() # print(renderer.render(parser.parse('Hello *world*'))) from __future__ import absolute_import, unicode_literals from commonmark.blocks import Parser from commonmark.dump import dumpAST, dumpJSON from commonmark.render.html import HtmlRenderer from commonmark.render.rst import ReStructuredTextRenderer def commonmark(text, format="html"): """Render CommonMark into HTML, JSON or AST Optional keyword arguments: format: 'html' (default), 'json' or 'ast' >>> commonmark("*hello!*") '

hello

\\n' """ parser = Parser() ast = parser.parse(text) if format not in ["html", "json", "ast", "rst"]: raise ValueError("format must be 'html', 'json' or 'ast'") if format == "html": renderer = HtmlRenderer() return renderer.render(ast) if format == "json": return dumpJSON(ast) if format == "ast": return dumpAST(ast) if format == "rst": renderer = ReStructuredTextRenderer() return renderer.render(ast) commonmark-0.9.1/commonmark/node.py0000644000175000017500000001100613462635760016724 0ustar niknik00000000000000from __future__ import unicode_literals import re reContainer = re.compile( r'(document|block_quote|list|item|paragraph|' r'heading|emph|strong|link|image|' r'custom_inline|custom_block)') def is_container(node): return (re.search(reContainer, node.t) is not None) class NodeWalker(object): def __init__(self, root): self.current = root self.root = root self.entering = True def __next__(self): cur = self.current entering = self.entering if cur is None: raise StopIteration container = is_container(cur) if entering and container: if cur.first_child: self.current = cur.first_child self.entering = True else: # stay on node but exit self.entering = False elif cur == self.root: self.current = None elif cur.nxt is None: self.current = cur.parent self.entering = False else: self.current = cur.nxt self.entering = True return cur, entering next = __next__ def __iter__(self): return self def nxt(self): """ for backwards compatibility """ try: cur, entering = next(self) return { 'entering': entering, 'node': cur, } except StopIteration: return None def resume_at(self, node, entering): self.current = node self.entering = (entering is True) class Node(object): def __init__(self, node_type, sourcepos): self.t = node_type self.parent = None self.first_child = None self.last_child = None self.prv = None self.nxt = None self.sourcepos = sourcepos self.last_line_blank = False self.last_line_checked = False self.is_open = True self.string_content = '' self.literal = None self.list_data = {} self.info = None self.destination = None self.title = None self.is_fenced = False self.fence_char = None self.fence_length = 0 self.fence_offset = None self.level = None self.on_enter = None self.on_exit = None def __repr__(self): return "Node {} [{}]".format(self.t, self.literal) def pretty(self): from pprint import pprint pprint(self.__dict__) def normalize(self): prev = None for curr, _ in self.walker(): if prev is None: prev = curr continue if prev.t == 'text' and curr.t == 'text': prev.literal += curr.literal curr.unlink() else: prev = curr def is_container(self): return is_container(self) def append_child(self, child): child.unlink() child.parent = self if self.last_child: self.last_child.nxt = child child.prv = self.last_child self.last_child = child else: self.first_child = child self.last_child = child def prepend_child(self, child): child.unlink() child.parent = self if self.first_child: self.first_child.prv = child child.nxt = self.first_child self.first_child = child else: self.first_child = child self.last_child = child def unlink(self): if self.prv: self.prv.nxt = self.nxt elif self.parent: self.parent.first_child = self.nxt if self.nxt: self.nxt.prv = self.prv elif self.parent: self.parent.last_child = self.prv self.parent = None self.nxt = None self.prv = None def insert_after(self, sibling): sibling.unlink() sibling.nxt = self.nxt if sibling.nxt: sibling.nxt.prv = sibling sibling.prv = self self.nxt = sibling sibling.parent = self.parent if not sibling.nxt: sibling.parent.last_child = sibling def insert_before(self, sibling): sibling.unlink() sibling.prv = self.prv if sibling.prv: sibling.prv.nxt = sibling sibling.nxt = self self.prv = sibling sibling.parent = self.parent if not sibling.prv: sibling.parent.first_child = sibling def walker(self): return NodeWalker(self) commonmark-0.9.1/commonmark/normalize_reference.py0000644000175000017500000001436413462635760022027 0ustar niknik00000000000000"""Case-folding and whitespace normalization""" # Unicode Case Folding table has been derived from the following work: # # CaseFolding-12.0.0.txt # Date: 2019-01-22, 08:18:22 GMT # (c) 2019 Unicode(R) Inc. # Unicode and the Unicode Logo are registered trademarks # of Unicode, Inc. in the U.S. and other countries. # For terms of use, see http://www.unicode.org/terms_of_use.html # # Unicode Character Database # For documentation, see http://www.unicode.org/reports/tr44/ import re import sys from builtins import str, chr __all__ = ["normalize_reference"] if sys.version_info < (3,) and sys.maxunicode <= 0xffff: # shim for Python 2.x UCS2 build _unichr = chr def chr(cdp): if 0x10000 <= cdp < 0x110000: cdp -= 0x10000 return (_unichr(0xd800 | (cdp >> 10)) + _unichr(0xdc00 | (cdp & 0x3ff))) return _unichr(cdp) def _parse_table(tbl): xlat = {} cur_i, cur_j = -1, 0 for entry in tbl.split(';'): arr = entry.split(',') info = [int(x, 36) if x else 0 for x in arr[0].split(':')] arr = [int(x, 36) for x in arr[1:]] assert not any(x in xlat for x in arr) sfx = ''.join(map(chr, arr)) streak, stride = 0, 1 if len(info) == 2: fdt, delta = info elif len(info) == 3: fdt, streak, delta = info else: fdt, streak, delta, stride = info assert streak >= 0 and stride >= 1 cur_i += fdt + 1 cur_j -= delta assert cur_j != 0 i = cur_i last = cur_i + streak while i <= last: # uniqueness and idempotency assert i not in xlat and i + cur_j not in xlat assert i not in arr xlat[i] = chr(i + cur_j) + sfx i += stride return xlat XLAT = _parse_table( # ===== Start of Unicode Case Folding table ===== '1t:p:-w;37:-kn;a:m:kn;n:6:;6:3w,37;w:1a:-31:2;1b:5k,lj;1:4:-5k:2;6:e::' '2;f:-aa,32;:18:aa:2;19:3e;:4:-3e:2;5:7h;1:-da;:2:5t:2;3:-5p;:5p;1:1:-5' 'o;1:5o;2:-26;:-3f;:-1;:5m;1:-5o;:-2;1:-4;:2;:5s;3:-5u;:-2;1:-1;:4:5x:2' ';5:-61;:61;1:-61;2:61;1:-61;:61;1:1:-60;1:2:60:2;3:-62;:4:62:4;b:-1;:1' ';1:-1;:1;1:-1;:g:1:2;i:g::2;h:av,lo;:-aw;:2:1:2;3:2q;:-15;:12:-1l:2;13' ':3n;1:g:-3n:2;n:-8bu;:8bu;1:4k;:-8gb;2:8br;1:5g;:-7c;:-2;:8:1y:2;72:-3' '7;16:2:37:2;5:;8:-37;6:26;1:2:1;3:-r;1:1:1;1:m,lk,ld;:g:9;h:8:;c:b,lk,' 'ld;h:k;c:-7;:12;:-5;3:-a;:7;1:m:-n:2;n:1j;:-6;2:c;:4;1:-1t;1:8;:-8;2:2' ':3n;2:f:-5u;f:v:1c;27:w:v:2;15:1g::2;1h:-e;:c:e:2;e:2m::2;2o:11:-1b;2d' ':2a,136;26w:11:-5mq;12:6::6;mo:5:5m0;1on:4sm;:-1;:-9;:1:-2;1:1;:-7;:-o' ';:-vzb;7:16:tj7;18:2:;8y:44:-2bl:2;45:5yn,mp;:-b,lk;:-2,lm;:-1,lm;:p,j' 'i;:-5xb;2:5wx,37;1:2m:-5yk:2;2v:7:9;f:5:;f:7:;f:7:;f:5:;7:5fn,lv;1:2,l' 'v,lc;1:2,lv,ld;1:2,lv,n6;2:6:-5ft:2;e:7:;n:7:3c,qh;7:7:8,qh;7:7:-o,qh;' '7:7:8,qh;7:7:-1k,qh;7:7:8,qh;9:-6,qh;:5hc,qh;:6,qh;1:-3,n6;:1,n6,qh;:1' ':-5j2;1:1:1u;1:5hd,qh;1:-6;3:-5h3,qh;:5ha,qh;:a,qh;1:-7,n6;:1,n6,qh;:3' ':-5h6;3:5hb,qh;5:4,lk,lc;:1,lk,ld;2:3,n6;:1,lk,n6;:1:-5jq;1:1:2k;7:5h5' ',lk,lc;:1,lk,ld;:5,lv;1:-2,n6;:1,lk,n6;:1:-5ju;1:1:2w;1:-2x;5:33,qh;:5' 'h0,qh;:-4,qh;1:7,n6;:1,n6,qh;:1:-5gu;1:1:-2;1:5h1,qh;89:8a;3:o2;:-3d;6' ':-6ea;19:f:c;y:f;mq:p:-p;1ft:1a:-m;2n:1b;1:8ag;:-5ch;:5c1;2:4:-8a0:2;5' ':8bh;:-v;:y;:-1;1:3:-8bj:3;b:1:8cg;1:2q:-8cg:2;2y:2::2;6:nym::nym;nyn:' '16::2;1p:q::2;4h:c::2;f:1o::2;1y:2::2;3:r9h;:8:-r9h:2;c:;1:wmh;2:2:-wm' 'h:2;5:i::2;j:wn9;:b;:-4;:-a;:3;1:-1e;:o;:-l;:-xbp;:a:pr:2;d:;1:1d;:wlv' ';:-5cb;q1:27:2oo;fpr:jii,2u;:1,2x;:1,30;:1,2u,2x;:1,2u,30;:-c,38;:1,38' ';c:-z8,12u;:1,12d;:1,12j;:-9,12u;:b,12l;sp:p:-1cjn;ym:13:-8;4v:z:;1jj:' '1e:-o;2e7:v:w;gwv:v:;o8v:x:-2' # ===== End of Unicode Case Folding table ===== ) def _check_native(tbl): """ Determine if Python's own native implementation subsumes the supplied case folding table """ try: for i in tbl: stv = chr(i) if stv.casefold() == stv: return False except AttributeError: return False return True # Hoist version check out of function for performance SPACE_RE = re.compile(r'[ \t\r\n]+') if _check_native(XLAT): def normalize_reference(string): """ Normalize reference label: collapse internal whitespace to single space, remove leading/trailing whitespace, case fold. """ return SPACE_RE.sub(' ', string[1:-1].strip()).casefold() elif sys.version_info >= (3,) or sys.maxunicode > 0xffff: def normalize_reference(string): """ Normalize reference label: collapse internal whitespace to single space, remove leading/trailing whitespace, case fold. """ return SPACE_RE.sub(' ', string[1:-1].strip()).translate(XLAT) else: def _get_smp_regex(): xls = sorted(x - 0x10000 for x in XLAT if x >= 0x10000) xls.append(-1) fmt, (dsh, opn, pip, cse) = str('\\u%04x'), str('-[|]') rga, srk, erk = [str(r'[ \t\r\n]+')], 0, -2 for k in xls: new_hir = (erk ^ k) >> 10 != 0 if new_hir or erk + 1 != k: if erk >= 0 and srk != erk: if srk + 1 != erk: rga.append(dsh) rga.append(fmt % (0xdc00 + (erk & 0x3ff))) if new_hir: if erk >= 0: rga.append(cse) if k < 0: break rga.append(pip) rga.append(fmt % (0xd800 + (k >> 10))) rga.append(opn) srk = k rga.append(fmt % (0xdc00 + (srk & 0x3ff))) erk = k return re.compile(str().join(rga)) def _subst_handler(matchobj): src = matchobj.group(0) hiv = ord(src[0]) if hiv < 0xd800: return ' ' return XLAT[0x10000 + ((hiv & 0x3ff) << 10) | (ord(src[1]) & 0x3ff)] SMP_RE = _get_smp_regex() def normalize_reference(string): """ Normalize reference label: collapse internal whitespace to single space, remove leading/trailing whitespace, case fold. """ return SMP_RE.sub(_subst_handler, string[1:-1].strip()).translate(XLAT) commonmark-0.9.1/commonmark/render/0000755000175000017500000000000013545663333016705 5ustar niknik00000000000000commonmark-0.9.1/commonmark/render/__init__.py0000644000175000017500000000000013343503214020767 0ustar niknik00000000000000commonmark-0.9.1/commonmark/render/html.py0000644000175000017500000001467113462635760020235 0ustar niknik00000000000000from __future__ import unicode_literals import re from builtins import str from commonmark.common import escape_xml from commonmark.render.renderer import Renderer reUnsafeProtocol = re.compile( r'^javascript:|vbscript:|file:|data:', re.IGNORECASE) reSafeDataProtocol = re.compile( r'^data:image\/(?:png|gif|jpeg|webp)', re.IGNORECASE) def potentially_unsafe(url): return re.search(reUnsafeProtocol, url) and \ (not re.search(reSafeDataProtocol, url)) class HtmlRenderer(Renderer): def __init__(self, options={}): # by default, soft breaks are rendered as newlines in HTML options['softbreak'] = options.get('softbreak') or '\n' # set to "
" to make them hard breaks # set to " " if you want to ignore line wrapping in source self.disable_tags = 0 self.last_out = '\n' self.options = options def escape(self, text): return escape_xml(text) def tag(self, name, attrs=None, selfclosing=None): """Helper function to produce an HTML tag.""" if self.disable_tags > 0: return self.buf += '<' + name if attrs and len(attrs) > 0: for attrib in attrs: self.buf += ' ' + attrib[0] + '="' + attrib[1] + '"' if selfclosing: self.buf += ' /' self.buf += '>' self.last_out = '>' # Node methods # def text(self, node, entering=None): self.out(node.literal) def softbreak(self, node=None, entering=None): self.lit(self.options['softbreak']) def linebreak(self, node=None, entering=None): self.tag('br', [], True) self.cr() def link(self, node, entering): attrs = self.attrs(node) if entering: if not (self.options.get('safe') and potentially_unsafe(node.destination)): attrs.append(['href', self.escape(node.destination)]) if node.title: attrs.append(['title', self.escape(node.title)]) self.tag('a', attrs) else: self.tag('/a') def image(self, node, entering): if entering: if self.disable_tags == 0: if self.options.get('safe') and \ potentially_unsafe(node.destination): self.lit('') else: self.lit('<img src=') def emph(self, node, entering): self.tag('em' if entering else '/em') def strong(self, node, entering): self.tag('strong' if entering else '/strong') def paragraph(self, node, entering): grandparent = node.parent.parent attrs = self.attrs(node) if grandparent is not None and grandparent.t == 'list': if grandparent.list_data['tight']: return if entering: self.cr() self.tag('p', attrs) else: self.tag('/p') self.cr() def heading(self, node, entering): tagname = 'h' + str(node.level) attrs = self.attrs(node) if entering: self.cr() self.tag(tagname, attrs) else: self.tag('/' + tagname) self.cr() def code(self, node, entering): self.tag('code') self.out(node.literal) self.tag('/code') def code_block(self, node, entering): info_words = node.info.split() if node.info else [] attrs = self.attrs(node) if len(info_words) > 0 and len(info_words[0]) > 0: attrs.append(['class', 'language-' + self.escape(info_words[0])]) self.cr() self.tag('pre') self.tag('code', attrs) self.out(node.literal) self.tag('/code') self.tag('/pre') self.cr() def thematic_break(self, node, entering): attrs = self.attrs(node) self.cr() self.tag('hr', attrs, True) self.cr() def block_quote(self, node, entering): attrs = self.attrs(node) if entering: self.cr() self.tag('blockquote', attrs) self.cr() else: self.cr() self.tag('/blockquote') self.cr() def list(self, node, entering): tagname = 'ul' if node.list_data['type'] == 'bullet' else 'ol' attrs = self.attrs(node) if entering: start = node.list_data['start'] if start is not None and start != 1: attrs.append(['start', str(start)]) self.cr() self.tag(tagname, attrs) self.cr() else: self.cr() self.tag('/' + tagname) self.cr() def item(self, node, entering): attrs = self.attrs(node) if entering: self.tag('li', attrs) else: self.tag('/li') self.cr() def html_inline(self, node, entering): if self.options.get('safe'): self.lit('') else: self.lit(node.literal) def html_block(self, node, entering): self.cr() if self.options.get('safe'): self.lit('') else: self.lit(node.literal) self.cr() def custom_inline(self, node, entering): if entering and node.on_enter: self.lit(node.on_enter) elif (not entering) and node.on_exit: self.lit(node.on_exit) def custom_block(self, node, entering): self.cr() if entering and node.on_enter: self.lit(node.on_enter) elif (not entering) and node.on_exit: self.lit(node.on_exit) self.cr() # Helper methods # def out(self, s): self.lit(self.escape(s)) def attrs(self, node): att = [] if self.options.get('sourcepos'): pos = node.sourcepos if pos: att.append(['data-sourcepos', str(pos[0][0]) + ':' + str(pos[0][1]) + '-' + str(pos[1][0]) + ':' + str(pos[1][1])]) return att commonmark-0.9.1/commonmark/render/renderer.py0000644000175000017500000000214113343503214021046 0ustar niknik00000000000000from __future__ import unicode_literals class Renderer(object): def render(self, ast): """Walks the AST and calls member methods for each Node type. @param ast {Node} The root of the abstract syntax tree. """ walker = ast.walker() self.buf = '' self.last_out = '\n' event = walker.nxt() while event is not None: type_ = event['node'].t if hasattr(self, type_): getattr(self, type_)(event['node'], event['entering']) event = walker.nxt() return self.buf def lit(self, s): """Concatenate a literal string to the buffer. @param str {String} The string to concatenate. """ self.buf += s self.last_out = s def cr(self): if self.last_out != '\n': self.lit('\n') def out(self, s): """Concatenate a string to the buffer possibly escaping the content. Concrete renderer implementations should override this method. @param str {String} The string to concatenate. """ self.lit(s) commonmark-0.9.1/commonmark/render/rst.py0000644000175000017500000000744713343503214020066 0ustar niknik00000000000000from __future__ import unicode_literals from commonmark.render.renderer import Renderer class ReStructuredTextRenderer(Renderer): """ Render reStructuredText from Markdown Example: .. code:: python import commonmark parser = commonmark.Parser() ast = parser.parse('Hello `inline code` example') renderer = commonmark.ReStructuredTextRenderer() rst = renderer.render(ast) print(rst) # Hello ``inline code`` example """ def __init__(self, indent_char=' '): self.indent_char = indent_char self.indent_length = 0 def lit(self, s): if s == '\n': indent = '' # Avoid whitespace if we're just adding a newline elif self.last_out != '\n': indent = '' # Don't indent if we're in the middle of a line else: indent = self.indent_char * self.indent_length return super(ReStructuredTextRenderer, self).lit(indent + s) def cr(self): self.lit('\n') def indent_lines(self, literal, indent_length=4): indent = self.indent_char * indent_length new_lines = [] for line in literal.splitlines(): new_lines.append(indent + line) return '\n'.join(new_lines) # Nodes def document(self, node, entering): pass def softbreak(self, node, entering): self.cr() def linebreak(self, node, entering): self.cr() self.cr() def text(self, node, entering): self.out(node.literal) def emph(self, node, entering): self.out('*') def strong(self, node, entering): self.out('**') def paragraph(self, node, entering): if node.parent.t == 'item': pass else: self.cr() def link(self, node, entering): if entering: self.out('`') else: self.out(' <%s>`_' % node.destination) def image(self, node, entering): directive = '.. image:: ' + node.destination if entering: self.out(directive) self.cr() self.indent_length += 4 self.out(':alt: ') else: self.indent_length -= 4 def code(self, node, entering): self.out('``') self.out(node.literal) self.out('``') def code_block(self, node, entering): directive = '.. code::' language_name = None info_words = node.info.split() if node.info else [] if len(info_words) > 0 and len(info_words[0]) > 0: language_name = info_words[0] if language_name: directive += ' ' + language_name self.cr() self.out(directive) self.cr() self.cr() self.out(self.indent_lines(node.literal)) self.cr() def list(self, node, entering): if entering: self.cr() def item(self, node, entering): tagname = '*' if node.list_data['type'] == 'bullet' else '#.' if entering: self.out(tagname + ' ') else: self.cr() def block_quote(self, node, entering): if entering: self.indent_length += 4 else: self.indent_length -= 4 def heading(self, node, entering): heading_chars = [ '#', '*', '=', '-', '^', '"' ] try: heading_char = heading_chars[node.level-1] except IndexError: # Default to the last level if we're in too deep heading_char = heading_chars[-1] heading_length = len(node.first_child.literal) banner = heading_char * heading_length if entering: self.cr() else: self.cr() self.out(banner) self.cr() commonmark-0.9.1/commonmark/tests/0000755000175000017500000000000013545663333016570 5ustar niknik00000000000000commonmark-0.9.1/commonmark/tests/__init__.py0000644000175000017500000000000013343503214020652 0ustar niknik00000000000000commonmark-0.9.1/commonmark/tests/rst_tests.py0000644000175000017500000000672413343503214021170 0ustar niknik00000000000000import unittest import commonmark class TestCommonmark(unittest.TestCase): def setUp(self): self.parser = commonmark.Parser() self.renderer = commonmark.ReStructuredTextRenderer() def render_rst(self, test_str): ast = self.parser.parse(test_str) rst = self.renderer.render(ast) return rst def assertEqualRender(self, src_markdown, expected_rst): rendered_rst = self.render_rst(src_markdown) self.assertEqual(rendered_rst, expected_rst) def test_strong(self): src_markdown = 'Hello **Strong**' expected_rst = '\nHello **Strong**\n' self.assertEqualRender(src_markdown, expected_rst) def test_emphasis(self): src_markdown = 'Hello *Emphasis*' expected_rst = '\nHello *Emphasis*\n' self.assertEqualRender(src_markdown, expected_rst) def test_paragraph(self): src_markdown = 'Hello paragraph' expected_rst = '\nHello paragraph\n' self.assertEqualRender(src_markdown, expected_rst) def test_link(self): src_markdown = '[Link](http://example.com)' expected_rst = '\n`Link `_\n' self.assertEqualRender(src_markdown, expected_rst) def test_image(self): src_markdown = '![Image](http://placekitten.com/100/100)' expected_rst = """ .. image:: http://placekitten.com/100/100 :alt: Image """ self.assertEqualRender(src_markdown, expected_rst) def test_code(self): src_markdown = 'Test `inline code` with backticks' expected_rst = '\nTest ``inline code`` with backticks\n' self.assertEqualRender(src_markdown, expected_rst) def test_code_block(self): src_markdown = """ ```python # code block print '3 backticks or' print 'indent 4 spaces' ``` """ expected_rst = """ .. code:: python # code block print '3 backticks or' print 'indent 4 spaces' """ self.assertEqualRender(src_markdown, expected_rst) def test_unordered_list(self): src_markdown = """ This is a list: * List item * List item * List item """ expected_rst = """ This is a list: * List item * List item * List item """ self.assertEqualRender(src_markdown, expected_rst) def test_ordered_list(self): src_markdown = """ This is a ordered list: 1. One 2. Two 3. Three """ expected_rst = """ This is a ordered list: #. One #. Two #. Three """ self.assertEqualRender(src_markdown, expected_rst) def test_block_quote(self): src_markdown = """ Before the blockquote: > The blockquote After the blockquote """ expected_rst = """ Before the blockquote: The blockquote After the blockquote """ self.assertEqualRender(src_markdown, expected_rst) def test_heading(self): src_markdown = ''' # Heading 1 ## Heading 2 ### Heading 3 #### Heading 4 ##### Heading 5 ###### Heading 6 ''' expected_rst = ''' Heading 1 ######### Heading 2 ********* Heading 3 ========= Heading 4 --------- Heading 5 ^^^^^^^^^ Heading 6 """"""""" ''' self.assertEqualRender(src_markdown, expected_rst) def test_multiple_paragraphs(self): src_markdown = ''' Start of first paragraph that continues on a new line This is the second paragraph ''' expected_rst = ''' Start of first paragraph that continues on a new line This is the second paragraph ''' self.assertEqualRender(src_markdown, expected_rst) if __name__ == '__main__': unittest.main() commonmark-0.9.1/commonmark/tests/run_spec_tests.py0000755000175000017500000001670613545662472022222 0ustar niknik00000000000000#!/usr/bin/env python # coding: utf-8 from __future__ import division, print_function, unicode_literals import re import timeit import codecs import argparse import sys from builtins import str from commonmark.render.html import HtmlRenderer from commonmark.main import Parser, dumpAST class colors(object): HEADER = '\033[95m' OKBLUE = '\033[94m' OKGREEN = '\033[92m' WARNING = '\033[93m' FAIL = '\033[91m' ENDC = '\033[0m' def trace_calls(frame, event, arg): co = frame.f_code func_name = co.co_name if func_name == "write": return line_no = frame.f_lineno filename = co.co_filename if event == "call" and not re.match("__", func_name) and \ re.search("CommonMark.py", filename) \ and func_name != "dumpAST": print("-> " + frame.f_back.f_code.co_name + " at " + str(frame.f_back.f_lineno) + " called " + func_name + " at " + str(line_no) + " in " + filename) return trace_calls def main(): parser = argparse.ArgumentParser( description="script to run the CommonMark specification tests " + "against the CommonMark.py parser") parser.add_argument( '-t', help="Single test to run or comma separated list " + "of tests (-t 10 or -t 10,11,12,13)") parser.add_argument( '-p', action="store_true", help="Print passed test information") parser.add_argument( '-f', action="store_true", help="Print failed tests (during -np...)") parser.add_argument( '-i', action="store_true", help="Interactive Markdown input mode") parser.add_argument( '-d', action="store_true", help="Debug, trace calls") parser.add_argument( '-np', action="store_true", help="Only print section header, tick, or cross") parser.add_argument( '-s', action="store_true", help="Print percent of tests passed by category") args = parser.parse_args() if args.d: sys.settrace(trace_calls) renderer = HtmlRenderer() parser = Parser() f = codecs.open("spec.txt", encoding="utf-8") datalist = [] for line in f: datalist.append(line) data = "".join(datalist) passed = 0 failed = 0 catStats = {} examples = [] example_number = 0 current_section = "" tabChar = '\u2192' spaceChar = '\u2423' nbspChar = '\u00A0' def showSpaces(t): t = re.sub("\\t", tabChar, t) t = re.sub(" ", spaceChar, t) t = re.sub(nbspChar, spaceChar, t) return t t = re.sub("\r\n", "\n", data) tests = re.sub( re.compile("^(.|[\n])*", flags=re.M), '', t) testMatch = re.findall( re.compile( r'^`{32} example\n' r'([\s\S]*?)^\.\n([\s\S]*?)' r'^`{32}$' r'|^#{1,6} *(.*)$', re.M), tests) for match in testMatch: if not match[2] == "": current_section = match[2] else: example_number += 1 examples.append({ 'markdown': match[0], 'html': match[1], 'section': current_section, 'number': example_number}) current_section = "" startTime = timeit.default_timer() if args.i: print( colors.OKGREEN + "(To end input of Markdown block enter 'end' on " + "it's own line, to quit enter 'quit')" + colors.ENDC) while True: s = "" while True: if sys.version_info >= (3, 0): inp = input(colors.OKBLUE + 'Markdown: ' + colors.ENDC) else: inp = raw_input(colors.OKBLUE + 'Markdown: ' + colors.ENDC) # noqa if not inp == "end" and inp != "quit": s += inp + "\n" elif inp == "end": s = s[:-1] break elif inp == "quit": print(colors.HEADER+"bye!"+colors.ENDC) exit(0) ast = parser.parse(s) html = renderer.render(ast) print(colors.WARNING+"="*10+"AST====="+colors.ENDC) dumpAST(ast) print(colors.WARNING+"="*10+"HTML===="+colors.ENDC) print(html) # some tests? if args.t: tests = args.t.split(",") choice_examples = [] for t in tests: if not t == "" and len(examples) > int(t): choice_examples.append(examples[int(t)-1]) examples = choice_examples # all tests for i, example in enumerate(examples): # [0,examples[0]] if not example['section'] == "" and \ not current_section == example['section']: print('\n' + colors.HEADER + '[' + example['section'] + ']' + colors.ENDC + ' ', end='') current_section = example['section'] catStats.update({current_section: [0, 0, 0]}) catStats[current_section][2] += 1 if args.d: print(colors.HEADER+"[Parsing]"+colors.ENDC) ast = parser.parse(re.sub(tabChar, "\t", example['markdown'])) if args.d: print(colors.HEADER+"[Rendering]"+colors.ENDC) actual = renderer.render(ast) if re.sub('\t', tabChar, actual) == example['html']: passed += 1 catStats[current_section][0] += 1 if not args.f: print(colors.OKGREEN + '✓' + colors.ENDC, end='') if args.d: dumpAST(ast) if args.p or args.d and not args.np: print( colors.OKBLUE + "=== markdown ===============\n" + colors.ENDC + showSpaces(example['markdown']) + colors.OKBLUE + "\n=== expected ===============\n" + colors.ENDC + showSpaces(example['html']) + colors.OKBLUE + "\n=== got ====================\n" + colors.ENDC + showSpaces(actual)) else: failed += 1 catStats[current_section][1] += 1 if args.t: print("Test #" + str(args.t.split(",")[i]), end='') else: print("Test #" + str(i+1), end='') print(' ' + colors.FAIL + "✗" + colors.ENDC) if args.d: dumpAST(ast) if not args.np or args.f: print( colors.WARNING + "=== markdown ===============\n" + colors.ENDC + showSpaces(example['markdown']) + colors.WARNING + "\n=== expected ===============\n" + colors.ENDC + showSpaces(example['html']) + colors.WARNING + "\n=== got ====================\n" + colors.ENDC + showSpaces(actual)) print('\n' + str(passed) + ' tests passed, ' + str(failed) + ' failed') endTime = timeit.default_timer() runTime = endTime - startTime if args.s: for i in catStats.keys(): per = catStats[i][0]/catStats[i][2] print(colors.HEADER + "[" + i + "]" + colors.ENDC + "\t" + str(per*100) + "% Passed") print("runtime: " + str(runTime) + "s") if (failed > 0): sys.exit(1) if __name__ == '__main__': main() commonmark-0.9.1/commonmark/tests/unit_tests.py0000644000175000017500000001061713462635760021351 0ustar niknik00000000000000from __future__ import unicode_literals import unittest try: from hypothesis import given, example except ImportError: # Mock out hypothesis stuff for python 2.6 def given(a): def func(b): return return func example = given try: from hypothesis.strategies import text except ImportError: def text(): pass import commonmark from commonmark.blocks import Parser from commonmark.render.html import HtmlRenderer from commonmark.inlines import InlineParser from commonmark.node import NodeWalker, Node class TestCommonmark(unittest.TestCase): def test_output(self): s = commonmark.commonmark('*hello!*') self.assertEqual(s, '

hello!

\n') def test_unicode(self): s = commonmark.commonmark('
\u2020
\n') self.assertEqual(s, '
\u2020
\n', 'Unicode works in an HTML block.') commonmark.commonmark('* unicode: \u2020') commonmark.commonmark('# unicode: \u2020') commonmark.commonmark('```\n# unicode: \u2020\n```') def test_null_string_bug(self): s = commonmark.commonmark('> sometext\n>\n\n') self.assertEqual( s, '
\n
sometext\n
' '\n
\n') def test_normalize_contracts_text_nodes(self): md = '_a' ast = Parser().parse(md) def assert_text_literals(text_literals): walker = ast.walker() document, _ = walker.next() self.assertEqual(document.t, 'document') paragraph, _ = walker.next() self.assertEqual(paragraph.t, 'paragraph') for literal in text_literals: text, _ = walker.next() self.assertEqual(text.t, 'text') self.assertEqual(text.literal, literal) paragraph, _ = walker.next() self.assertEqual(paragraph.t, 'paragraph') assert_text_literals(['_', 'a']) ast.normalize() # assert text nodes are contracted assert_text_literals(['_a']) ast.normalize() # assert normalize() doesn't alter a normalized ast assert_text_literals(['_a']) def test_dumpAST_orderedlist(self): md = '1.' ast = Parser().parse(md) commonmark.dumpAST(ast) @given(text()) def test_random_text(self, s): commonmark.commonmark(s) def test_smart_dashes(self): md = 'a - b -- c --- d ---- e ----- f' EM = '\u2014' EN = '\u2013' expected_html = ( '

' + 'a - ' + 'b ' + EN + ' ' + 'c ' + EM + ' ' + 'd ' + EN + EN + ' ' + 'e ' + EM + EN + ' ' + 'f

\n') parser = commonmark.Parser(options=dict(smart=True)) ast = parser.parse(md) renderer = commonmark.HtmlRenderer() html = renderer.render(ast) self.assertEqual(html, expected_html) def test_regex_vulnerability_link_label(self): i = 200 while i <= 2000: s = commonmark.commonmark('[' + ('\\' * i) + '\n') self.assertEqual(s, '

' + '[' + ('\\' * (i // 2)) + '

\n', '[\\\\... %d deep' % (i,)) i *= 10 def test_regex_vulnerability_link_destination(self): i = 200 while i <= 2000: s = commonmark.commonmark(('[](' * i) + '\n') self.assertEqual(s, '

' + ('[](' * i) + '

\n', '[]( %d deep' % (i,)) i *= 10 class TestHtmlRenderer(unittest.TestCase): def test_init(self): HtmlRenderer() class TestInlineParser(unittest.TestCase): def test_init(self): InlineParser() class TestNode(unittest.TestCase): def test_doc_node(self): Node('document', [[1, 1], [0, 0]]) class TestNodeWalker(unittest.TestCase): def test_node_walker(self): node = Node('document', [[1, 1], [0, 0]]) NodeWalker(node) def test_node_walker_iter(self): node = Node('document', [[1, 1], [0, 0]]) for subnode, entered in node.walker(): pass class TestParser(unittest.TestCase): def setUp(self): self.parser = Parser() @given(text()) @example('') @example('* unicode: \u2020') def test_text(self, s): self.parser.parse(s) if __name__ == '__main__': unittest.main() commonmark-0.9.1/commonmark.egg-info/0000755000175000017500000000000013545663333017120 5ustar niknik00000000000000commonmark-0.9.1/commonmark.egg-info/PKG-INFO0000644000175000017500000001502313545663333020216 0ustar niknik00000000000000Metadata-Version: 2.1 Name: commonmark Version: 0.9.1 Summary: Python parser for the CommonMark Markdown spec Home-page: https://github.com/rtfd/commonmark.py Author: Bibek Kafle , Roland Shoemaker Author-email: rolandshoemaker@gmail.com Maintainer: Nikolas Nyby Maintainer-email: nikolas@gnu.org License: BSD-3-Clause Description: commonmark.py ============= commonmark.py is a pure Python port of `jgm `__'s `commonmark.js `__, a Markdown parser and renderer for the `CommonMark `__ specification, using only native modules. Once both this project and the CommonMark specification are stable we will release the first ``1.0`` version and attempt to keep up to date with changes in ``commonmark.js``. commonmark.py is tested against the CommonMark spec with Python versions 2.7, 3.4, 3.5, 3.6, and 3.7. **Current version:** 0.9.1 |Pypi Link| |Build Status| |Doc Link| Installation ------------ :: $ pip install commonmark Usage ----- :: >>> import commonmark >>> commonmark.commonmark('*hello!*') '

hello!

\n' Or, without the syntactic sugar: .. code:: python import commonmark parser = commonmark.Parser() ast = parser.parse("Hello *World*") renderer = commonmark.HtmlRenderer() html = renderer.render(ast) print(html) #

Hello World

# inspecting the abstract syntax tree json = commonmark.dumpJSON(ast) commonmark.dumpAST(ast) # pretty print generated AST structure There is also a CLI: :: $ cmark README.md -o README.html $ cmark README.md -o README.json -aj # output AST as JSON $ cmark README.md -a # pretty print generated AST structure $ cmark -h usage: cmark [-h] [-o [O]] [-a] [-aj] [infile] Process Markdown according to the CommonMark specification. positional arguments: infile Input Markdown file to parse, defaults to stdin optional arguments: -h, --help show this help message and exit -o [O] Output HTML/JSON file, defaults to stdout -a Print formatted AST -aj Output JSON AST Contributing ------------ If you would like to offer suggestions/optimizations/bugfixes through pull requests please do! Also if you find an error in the parser/renderer that isn't caught by the current test suite please open a new issue and I would also suggest you send the `commonmark.js `__ project a pull request adding your test to the existing test suite. Tests ----- To work on commonmark.py, you will need to be able to run the test suite to make sure your changes don't break anything. To run the tests, you can do something like this: :: $ pyvenv venv $ ./venv/bin/python setup.py develop test The tests script, ``commonmark/tests/run_spec_tests.py``, is pretty much a devtool. As well as running all the tests embedded in ``spec.txt`` it also allows you to run specific tests using the ``-t`` argument, provide information about passed tests with ``-p``, percentage passed by category of test with ``-s``, and enter markdown interactively with ``-i`` (In interactive mode end a block by inputting a line with just ``end``, to quit do the same but with ``quit``). ``-d`` can be used to print call tracing. :: $ ./venv/bin/python commonmark/tests/run_spec_tests.py -h usage: run_spec_tests.py [-h] [-t T] [-p] [-f] [-i] [-d] [-np] [-s] script to run the CommonMark specification tests against the commonmark.py parser. optional arguments: -h, --help show this help message and exit -t T Single test to run or comma separated list of tests (-t 10 or -t 10,11,12,13) -p Print passed test information -f Print failed tests (during -np...) -i Interactive Markdown input mode -d Debug, trace calls -np Only print section header, tick, or cross -s Print percent of tests passed by category Authors ------- - `Bibek Kafle `__ - `Roland Shoemaker `__ - `Nikolas Nyby `__ .. |Pypi Link| image:: https://img.shields.io/pypi/v/commonmark.svg :target: https://pypi.org/project/commonmark/ .. |Build Status| image:: https://travis-ci.org/rtfd/commonmark.py.svg?branch=master :target: https://travis-ci.org/rtfd/commonmark.py .. |Doc Link| image:: https://readthedocs.org/projects/commonmarkpy/badge/?version=latest :target: https://commonmarkpy.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status Keywords: markup,markdown,commonmark Platform: UNKNOWN Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 3 Classifier: Development Status :: 5 - Production/Stable Classifier: Environment :: Other Environment Classifier: Environment :: Console Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Topic :: Documentation Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content Classifier: Topic :: Software Development :: Documentation Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: Topic :: Text Processing :: Markup Classifier: Topic :: Utilities Description-Content-Type: text/x-rst Provides-Extra: test commonmark-0.9.1/commonmark.egg-info/SOURCES.txt0000644000175000017500000000135413545663333021007 0ustar niknik00000000000000.gitignore CHANGELOG.md LICENSE MANIFEST.in README.rst setup.cfg setup.py spec.txt commonmark/__init__.py commonmark/blocks.py commonmark/cmark.py commonmark/common.py commonmark/dump.py commonmark/entitytrans.py commonmark/inlines.py commonmark/main.py commonmark/node.py commonmark/normalize_reference.py commonmark.egg-info/PKG-INFO commonmark.egg-info/SOURCES.txt commonmark.egg-info/dependency_links.txt commonmark.egg-info/entry_points.txt commonmark.egg-info/requires.txt commonmark.egg-info/top_level.txt commonmark/render/__init__.py commonmark/render/html.py commonmark/render/renderer.py commonmark/render/rst.py commonmark/tests/__init__.py commonmark/tests/rst_tests.py commonmark/tests/run_spec_tests.py commonmark/tests/unit_tests.pycommonmark-0.9.1/commonmark.egg-info/dependency_links.txt0000644000175000017500000000000113545663333023166 0ustar niknik00000000000000 commonmark-0.9.1/commonmark.egg-info/entry_points.txt0000644000175000017500000000006113545663333022413 0ustar niknik00000000000000[console_scripts] cmark = commonmark.cmark:main commonmark-0.9.1/commonmark.egg-info/requires.txt0000644000175000017500000000012113545663333021512 0ustar niknik00000000000000 [:python_version < "3"] future>=0.14.0 [test] flake8==3.7.8 hypothesis==3.55.3 commonmark-0.9.1/commonmark.egg-info/top_level.txt0000644000175000017500000000001313545663333021644 0ustar niknik00000000000000commonmark commonmark-0.9.1/setup.cfg0000644000175000017500000000015513545663333015105 0ustar niknik00000000000000[metadata] description-file = README.rst [bdist_wheel] universal = 1 [egg_info] tag_build = tag_date = 0 commonmark-0.9.1/setup.py0000644000175000017500000000434213545663315015000 0ustar niknik00000000000000from setuptools import setup, find_packages, Command import io from os import path class Test(Command): user_options = [] def initialize_options(self): pass def finalize_options(self): pass def run(self): import subprocess import sys errno = subprocess.call([ sys.executable, 'commonmark/tests/run_spec_tests.py']) raise SystemExit(errno) tests_require = [ 'flake8==3.7.8', 'hypothesis==3.55.3', ] this_directory = path.abspath(path.dirname(__file__)) with io.open(path.join(this_directory, 'README.rst'), encoding='utf-8') as f: long_description = f.read() setup( name="commonmark", packages=find_packages(exclude=['tests']), version="0.9.1", license="BSD-3-Clause", description="Python parser for the CommonMark Markdown spec", long_description=long_description, long_description_content_type='text/x-rst', author="Bibek Kafle , " + "Roland Shoemaker ", author_email="rolandshoemaker@gmail.com", maintainer="Nikolas Nyby", maintainer_email="nikolas@gnu.org", url="https://github.com/rtfd/commonmark.py", keywords=["markup", "markdown", "commonmark"], entry_points={ 'console_scripts': [ 'cmark = commonmark.cmark:main', ] }, cmdclass={'test': Test}, install_requires=[ 'future>=0.14.0;python_version<"3"', ], tests_require=tests_require, extras_require={'test': tests_require}, classifiers=[ "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 3", "Development Status :: 5 - Production/Stable", "Environment :: Other Environment", "Environment :: Console", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Topic :: Documentation", "Topic :: Internet :: WWW/HTTP :: Dynamic Content", "Topic :: Software Development :: Documentation", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: Text Processing :: Markup", "Topic :: Utilities"]) commonmark-0.9.1/spec.txt0000644000175000017500000061411313462635760014765 0ustar niknik00000000000000--- title: CommonMark Spec author: John MacFarlane version: 0.29 date: '2019-04-06' license: '[CC-BY-SA 4.0](http://creativecommons.org/licenses/by-sa/4.0/)' ... # Introduction ## What is Markdown? Markdown is a plain text format for writing structured documents, based on conventions for indicating formatting in email and usenet posts. It was developed by John Gruber (with help from Aaron Swartz) and released in 2004 in the form of a [syntax description](http://daringfireball.net/projects/markdown/syntax) and a Perl script (`Markdown.pl`) for converting Markdown to HTML. In the next decade, dozens of implementations were developed in many languages. Some extended the original Markdown syntax with conventions for footnotes, tables, and other document elements. Some allowed Markdown documents to be rendered in formats other than HTML. Websites like Reddit, StackOverflow, and GitHub had millions of people using Markdown. And Markdown started to be used beyond the web, to author books, articles, slide shows, letters, and lecture notes. What distinguishes Markdown from many other lightweight markup syntaxes, which are often easier to write, is its readability. As Gruber writes: > The overriding design goal for Markdown's formatting syntax is > to make it as readable as possible. The idea is that a > Markdown-formatted document should be publishable as-is, as > plain text, without looking like it's been marked up with tags > or formatting instructions. > () The point can be illustrated by comparing a sample of [AsciiDoc](http://www.methods.co.nz/asciidoc/) with an equivalent sample of Markdown. Here is a sample of AsciiDoc from the AsciiDoc manual: ``` 1. List item one. + List item one continued with a second paragraph followed by an Indented block. + ................. $ ls *.sh $ mv *.sh ~/tmp ................. + List item continued with a third paragraph. 2. List item two continued with an open block. + -- This paragraph is part of the preceding list item. a. This list is nested and does not require explicit item continuation. + This paragraph is part of the preceding list item. b. List item b. This paragraph belongs to item two of the outer list. -- ``` And here is the equivalent in Markdown: ``` 1. List item one. List item one continued with a second paragraph followed by an Indented block. $ ls *.sh $ mv *.sh ~/tmp List item continued with a third paragraph. 2. List item two continued with an open block. This paragraph is part of the preceding list item. 1. This list is nested and does not require explicit item continuation. This paragraph is part of the preceding list item. 2. List item b. This paragraph belongs to item two of the outer list. ``` The AsciiDoc version is, arguably, easier to write. You don't need to worry about indentation. But the Markdown version is much easier to read. The nesting of list items is apparent to the eye in the source, not just in the processed document. ## Why is a spec needed? John Gruber's [canonical description of Markdown's syntax](http://daringfireball.net/projects/markdown/syntax) does not specify the syntax unambiguously. Here are some examples of questions it does not answer: 1. How much indentation is needed for a sublist? The spec says that continuation paragraphs need to be indented four spaces, but is not fully explicit about sublists. It is natural to think that they, too, must be indented four spaces, but `Markdown.pl` does not require that. This is hardly a "corner case," and divergences between implementations on this issue often lead to surprises for users in real documents. (See [this comment by John Gruber](http://article.gmane.org/gmane.text.markdown.general/1997).) 2. Is a blank line needed before a block quote or heading? Most implementations do not require the blank line. However, this can lead to unexpected results in hard-wrapped text, and also to ambiguities in parsing (note that some implementations put the heading inside the blockquote, while others do not). (John Gruber has also spoken [in favor of requiring the blank lines](http://article.gmane.org/gmane.text.markdown.general/2146).) 3. Is a blank line needed before an indented code block? (`Markdown.pl` requires it, but this is not mentioned in the documentation, and some implementations do not require it.) ``` markdown paragraph code? ``` 4. What is the exact rule for determining when list items get wrapped in `

` tags? Can a list be partially "loose" and partially "tight"? What should we do with a list like this? ``` markdown 1. one 2. two 3. three ``` Or this? ``` markdown 1. one - a - b 2. two ``` (There are some relevant comments by John Gruber [here](http://article.gmane.org/gmane.text.markdown.general/2554).) 5. Can list markers be indented? Can ordered list markers be right-aligned? ``` markdown 8. item 1 9. item 2 10. item 2a ``` 6. Is this one list with a thematic break in its second item, or two lists separated by a thematic break? ``` markdown * a * * * * * * b ``` 7. When list markers change from numbers to bullets, do we have two lists or one? (The Markdown syntax description suggests two, but the perl scripts and many other implementations produce one.) ``` markdown 1. fee 2. fie - foe - fum ``` 8. What are the precedence rules for the markers of inline structure? For example, is the following a valid link, or does the code span take precedence ? ``` markdown [a backtick (`)](/url) and [another backtick (`)](/url). ``` 9. What are the precedence rules for markers of emphasis and strong emphasis? For example, how should the following be parsed? ``` markdown *foo *bar* baz* ``` 10. What are the precedence rules between block-level and inline-level structure? For example, how should the following be parsed? ``` markdown - `a long code span can contain a hyphen like this - and it can screw things up` ``` 11. Can list items include section headings? (`Markdown.pl` does not allow this, but does allow blockquotes to include headings.) ``` markdown - # Heading ``` 12. Can list items be empty? ``` markdown * a * * b ``` 13. Can link references be defined inside block quotes or list items? ``` markdown > Blockquote [foo]. > > [foo]: /url ``` 14. If there are multiple definitions for the same reference, which takes precedence? ``` markdown [foo]: /url1 [foo]: /url2 [foo][] ``` In the absence of a spec, early implementers consulted `Markdown.pl` to resolve these ambiguities. But `Markdown.pl` was quite buggy, and gave manifestly bad results in many cases, so it was not a satisfactory replacement for a spec. Because there is no unambiguous spec, implementations have diverged considerably. As a result, users are often surprised to find that a document that renders one way on one system (say, a GitHub wiki) renders differently on another (say, converting to docbook using pandoc). To make matters worse, because nothing in Markdown counts as a "syntax error," the divergence often isn't discovered right away. ## About this document This document attempts to specify Markdown syntax unambiguously. It contains many examples with side-by-side Markdown and HTML. These are intended to double as conformance tests. An accompanying script `spec_tests.py` can be used to run the tests against any Markdown program: python test/spec_tests.py --spec spec.txt --program PROGRAM Since this document describes how Markdown is to be parsed into an abstract syntax tree, it would have made sense to use an abstract representation of the syntax tree instead of HTML. But HTML is capable of representing the structural distinctions we need to make, and the choice of HTML for the tests makes it possible to run the tests against an implementation without writing an abstract syntax tree renderer. This document is generated from a text file, `spec.txt`, written in Markdown with a small extension for the side-by-side tests. The script `tools/makespec.py` can be used to convert `spec.txt` into HTML or CommonMark (which can then be converted into other formats). In the examples, the `→` character is used to represent tabs. # Preliminaries ## Characters and lines Any sequence of [characters] is a valid CommonMark document. A [character](@) is a Unicode code point. Although some code points (for example, combining accents) do not correspond to characters in an intuitive sense, all code points count as characters for purposes of this spec. This spec does not specify an encoding; it thinks of lines as composed of [characters] rather than bytes. A conforming parser may be limited to a certain encoding. A [line](@) is a sequence of zero or more [characters] other than newline (`U+000A`) or carriage return (`U+000D`), followed by a [line ending] or by the end of file. A [line ending](@) is a newline (`U+000A`), a carriage return (`U+000D`) not followed by a newline, or a carriage return and a following newline. A line containing no characters, or a line containing only spaces (`U+0020`) or tabs (`U+0009`), is called a [blank line](@). The following definitions of character classes will be used in this spec: A [whitespace character](@) is a space (`U+0020`), tab (`U+0009`), newline (`U+000A`), line tabulation (`U+000B`), form feed (`U+000C`), or carriage return (`U+000D`). [Whitespace](@) is a sequence of one or more [whitespace characters]. A [Unicode whitespace character](@) is any code point in the Unicode `Zs` general category, or a tab (`U+0009`), carriage return (`U+000D`), newline (`U+000A`), or form feed (`U+000C`). [Unicode whitespace](@) is a sequence of one or more [Unicode whitespace characters]. A [space](@) is `U+0020`. A [non-whitespace character](@) is any character that is not a [whitespace character]. An [ASCII punctuation character](@) is `!`, `"`, `#`, `$`, `%`, `&`, `'`, `(`, `)`, `*`, `+`, `,`, `-`, `.`, `/` (U+0021–2F), `:`, `;`, `<`, `=`, `>`, `?`, `@` (U+003A–0040), `[`, `\`, `]`, `^`, `_`, `` ` `` (U+005B–0060), `{`, `|`, `}`, or `~` (U+007B–007E). A [punctuation character](@) is an [ASCII punctuation character] or anything in the general Unicode categories `Pc`, `Pd`, `Pe`, `Pf`, `Pi`, `Po`, or `Ps`. ## Tabs Tabs in lines are not expanded to [spaces]. However, in contexts where whitespace helps to define block structure, tabs behave as if they were replaced by spaces with a tab stop of 4 characters. Thus, for example, a tab can be used instead of four spaces in an indented code block. (Note, however, that internal tabs are passed through as literal tabs, not expanded to spaces.) ```````````````````````````````` example →foo→baz→→bim . 

foo→baz→→bim
```````````````````````````````` ```````````````````````````````` example →foo→baz→→bim . 
foo→baz→→bim
```````````````````````````````` ```````````````````````````````` example a→a ὐ→a . 
a→a
ὐ→a
```````````````````````````````` In the following example, a continuation paragraph of a list item is indented with a tab; this has exactly the same effect as indentation with four spaces would: ```````````````````````````````` example - foo →bar .

  • foo

    bar

```````````````````````````````` ```````````````````````````````` example - foo →→bar .

  • foo

      bar
```````````````````````````````` Normally the `>` that begins a block quote may be followed optionally by a space, which is not considered part of the content. In the following case `>` is followed by a tab, which is treated as if it were expanded into three spaces. Since one of these spaces is considered part of the delimiter, `foo` is considered to be indented six spaces inside the block quote context, so we get an indented code block starting with two spaces. ```````````````````````````````` example >→→foo . 
  foo
```````````````````````````````` ```````````````````````````````` example -→→foo . 
  •   foo
```````````````````````````````` ```````````````````````````````` example foo →bar . 
foo
bar
```````````````````````````````` ```````````````````````````````` example - foo - bar → - baz .
  • foo
    • bar
      • baz
```````````````````````````````` ```````````````````````````````` example #→Foo .

Foo

```````````````````````````````` ```````````````````````````````` example *→*→*→ .
```````````````````````````````` ## Insecure characters For security reasons, the Unicode character `U+0000` must be replaced with the REPLACEMENT CHARACTER (`U+FFFD`). # Blocks and inlines We can think of a document as a sequence of [blocks](@)---structural elements like paragraphs, block quotations, lists, headings, rules, and code blocks. Some blocks (like block quotes and list items) contain other blocks; others (like headings and paragraphs) contain [inline](@) content---text, links, emphasized text, images, code spans, and so on. ## Precedence Indicators of block structure always take precedence over indicators of inline structure. So, for example, the following is a list with two items, not a list with one item containing a code span: ```````````````````````````````` example - `one - two` .
  • `one
  • two`
```````````````````````````````` This means that parsing can proceed in two steps: first, the block structure of the document can be discerned; second, text lines inside paragraphs, headings, and other block constructs can be parsed for inline structure. The second step requires information about link reference definitions that will be available only at the end of the first step. Note that the first step requires processing lines in sequence, but the second can be parallelized, since the inline parsing of one block element does not affect the inline parsing of any other. ## Container blocks and leaf blocks We can divide blocks into two types: [container blocks](@), which can contain other blocks, and [leaf blocks](@), which cannot. # Leaf blocks This section describes the different kinds of leaf block that make up a Markdown document. ## Thematic breaks A line consisting of 0-3 spaces of indentation, followed by a sequence of three or more matching `-`, `_`, or `*` characters, each followed optionally by any number of spaces or tabs, forms a [thematic break](@). ```````````````````````````````` example *** --- ___ .
```````````````````````````````` Wrong characters: ```````````````````````````````` example +++ .

+++

```````````````````````````````` ```````````````````````````````` example === .

===

```````````````````````````````` Not enough characters: ```````````````````````````````` example -- ** __ .

-- ** __

```````````````````````````````` One to three spaces indent are allowed: ```````````````````````````````` example *** *** *** .
```````````````````````````````` Four spaces is too many: ```````````````````````````````` example *** . 
***
```````````````````````````````` ```````````````````````````````` example Foo *** .

Foo ***

```````````````````````````````` More than three characters may be used: ```````````````````````````````` example _____________________________________ .
```````````````````````````````` Spaces are allowed between the characters: ```````````````````````````````` example - - - .
```````````````````````````````` ```````````````````````````````` example ** * ** * ** * ** .
```````````````````````````````` ```````````````````````````````` example - - - - .
```````````````````````````````` Spaces are allowed at the end: ```````````````````````````````` example - - - - .
```````````````````````````````` However, no other characters may occur in the line: ```````````````````````````````` example _ _ _ _ a a------ ---a--- .

_ _ _ _ a

a------

---a---

```````````````````````````````` It is required that all of the [non-whitespace characters] be the same. So, this is not a thematic break: ```````````````````````````````` example *-* .

-

```````````````````````````````` Thematic breaks do not need blank lines before or after: ```````````````````````````````` example - foo *** - bar .
  • foo
  • bar
```````````````````````````````` Thematic breaks can interrupt a paragraph: ```````````````````````````````` example Foo *** bar .

Foo

bar

```````````````````````````````` If a line of dashes that meets the above conditions for being a thematic break could also be interpreted as the underline of a [setext heading], the interpretation as a [setext heading] takes precedence. Thus, for example, this is a setext heading, not a paragraph followed by a thematic break: ```````````````````````````````` example Foo --- bar .

Foo

bar

```````````````````````````````` When both a thematic break and a list item are possible interpretations of a line, the thematic break takes precedence: ```````````````````````````````` example * Foo * * * * Bar .
  • Foo
  • Bar
```````````````````````````````` If you want a thematic break in a list item, use a different bullet: ```````````````````````````````` example - Foo - * * * .
  • Foo
```````````````````````````````` ## ATX headings An [ATX heading](@) consists of a string of characters, parsed as inline content, between an opening sequence of 1--6 unescaped `#` characters and an optional closing sequence of any number of unescaped `#` characters. The opening sequence of `#` characters must be followed by a [space] or by the end of line. The optional closing sequence of `#`s must be preceded by a [space] and may be followed by spaces only. The opening `#` character may be indented 0-3 spaces. The raw contents of the heading are stripped of leading and trailing spaces before being parsed as inline content. The heading level is equal to the number of `#` characters in the opening sequence. Simple headings: ```````````````````````````````` example # foo ## foo ### foo #### foo ##### foo ###### foo .

foo

foo

foo

foo

foo
foo
```````````````````````````````` More than six `#` characters is not a heading: ```````````````````````````````` example ####### foo .

####### foo

```````````````````````````````` At least one space is required between the `#` characters and the heading's contents, unless the heading is empty. Note that many implementations currently do not require the space. However, the space was required by the [original ATX implementation](http://www.aaronsw.com/2002/atx/atx.py), and it helps prevent things like the following from being parsed as headings: ```````````````````````````````` example #5 bolt #hashtag .

#5 bolt

#hashtag

```````````````````````````````` This is not a heading, because the first `#` is escaped: ```````````````````````````````` example \## foo .

## foo

```````````````````````````````` Contents are parsed as inlines: ```````````````````````````````` example # foo *bar* \*baz\* .

foo bar *baz*

```````````````````````````````` Leading and trailing [whitespace] is ignored in parsing inline content: ```````````````````````````````` example # foo .

foo

```````````````````````````````` One to three spaces indentation are allowed: ```````````````````````````````` example ### foo ## foo # foo .

foo

foo

foo

```````````````````````````````` Four spaces are too much: ```````````````````````````````` example # foo . 
# foo
```````````````````````````````` ```````````````````````````````` example foo # bar .

foo # bar

```````````````````````````````` A closing sequence of `#` characters is optional: ```````````````````````````````` example ## foo ## ### bar ### .

foo

bar

```````````````````````````````` It need not be the same length as the opening sequence: ```````````````````````````````` example # foo ################################## ##### foo ## .

foo

foo
```````````````````````````````` Spaces are allowed after the closing sequence: ```````````````````````````````` example ### foo ### .

foo

```````````````````````````````` A sequence of `#` characters with anything but [spaces] following it is not a closing sequence, but counts as part of the contents of the heading: ```````````````````````````````` example ### foo ### b .

foo ### b

```````````````````````````````` The closing sequence must be preceded by a space: ```````````````````````````````` example # foo# .

foo#

```````````````````````````````` Backslash-escaped `#` characters do not count as part of the closing sequence: ```````````````````````````````` example ### foo \### ## foo #\## # foo \# .

foo ###

foo ###

foo #

```````````````````````````````` ATX headings need not be separated from surrounding content by blank lines, and they can interrupt paragraphs: ```````````````````````````````` example **** ## foo **** .

foo

```````````````````````````````` ```````````````````````````````` example Foo bar # baz Bar foo .

Foo bar

baz

Bar foo

```````````````````````````````` ATX headings can be empty: ```````````````````````````````` example ## # ### ### .

```````````````````````````````` ## Setext headings A [setext heading](@) consists of one or more lines of text, each containing at least one [non-whitespace character], with no more than 3 spaces indentation, followed by a [setext heading underline]. The lines of text must be such that, were they not followed by the setext heading underline, they would be interpreted as a paragraph: they cannot be interpretable as a [code fence], [ATX heading][ATX headings], [block quote][block quotes], [thematic break][thematic breaks], [list item][list items], or [HTML block][HTML blocks]. A [setext heading underline](@) is a sequence of `=` characters or a sequence of `-` characters, with no more than 3 spaces indentation and any number of trailing spaces. If a line containing a single `-` can be interpreted as an empty [list items], it should be interpreted this way and not as a [setext heading underline]. The heading is a level 1 heading if `=` characters are used in the [setext heading underline], and a level 2 heading if `-` characters are used. The contents of the heading are the result of parsing the preceding lines of text as CommonMark inline content. In general, a setext heading need not be preceded or followed by a blank line. However, it cannot interrupt a paragraph, so when a setext heading comes after a paragraph, a blank line is needed between them. Simple examples: ```````````````````````````````` example Foo *bar* ========= Foo *bar* --------- .

Foo bar

Foo bar

```````````````````````````````` The content of the header may span more than one line: ```````````````````````````````` example Foo *bar baz* ==== .

Foo bar baz

```````````````````````````````` The contents are the result of parsing the headings's raw content as inlines. The heading's raw content is formed by concatenating the lines and removing initial and final [whitespace]. ```````````````````````````````` example Foo *bar baz*→ ==== .

Foo bar baz

```````````````````````````````` The underlining can be any length: ```````````````````````````````` example Foo ------------------------- Foo = .

Foo

Foo

```````````````````````````````` The heading content can be indented up to three spaces, and need not line up with the underlining: ```````````````````````````````` example Foo --- Foo ----- Foo === .

Foo

Foo

Foo

```````````````````````````````` Four spaces indent is too much: ```````````````````````````````` example Foo --- Foo --- . 
Foo
---

Foo
```````````````````````````````` The setext heading underline can be indented up to three spaces, and may have trailing spaces: ```````````````````````````````` example Foo ---- .

Foo

```````````````````````````````` Four spaces is too much: ```````````````````````````````` example Foo --- .

Foo ---

```````````````````````````````` The setext heading underline cannot contain internal spaces: ```````````````````````````````` example Foo = = Foo --- - .

Foo = =

Foo

```````````````````````````````` Trailing spaces in the content line do not cause a line break: ```````````````````````````````` example Foo ----- .

Foo

```````````````````````````````` Nor does a backslash at the end: ```````````````````````````````` example Foo\ ---- .

Foo\

```````````````````````````````` Since indicators of block structure take precedence over indicators of inline structure, the following are setext headings: ```````````````````````````````` example `Foo ---- ` .

`Foo

`

<a title="a lot

of dashes"/>

```````````````````````````````` The setext heading underline cannot be a [lazy continuation line] in a list item or block quote: ```````````````````````````````` example > Foo --- .

Foo

```````````````````````````````` ```````````````````````````````` example > foo bar === .

foo bar ===

```````````````````````````````` ```````````````````````````````` example - Foo --- .
  • Foo
```````````````````````````````` A blank line is needed between a paragraph and a following setext heading, since otherwise the paragraph becomes part of the heading's content: ```````````````````````````````` example Foo Bar --- .

Foo Bar

```````````````````````````````` But in general a blank line is not required before or after setext headings: ```````````````````````````````` example --- Foo --- Bar --- Baz .

Foo

Bar

Baz

```````````````````````````````` Setext headings cannot be empty: ```````````````````````````````` example ==== .

====

```````````````````````````````` Setext heading text lines must not be interpretable as block constructs other than paragraphs. So, the line of dashes in these examples gets interpreted as a thematic break: ```````````````````````````````` example --- --- .
```````````````````````````````` ```````````````````````````````` example - foo ----- .
  • foo
```````````````````````````````` ```````````````````````````````` example foo --- . 
foo
```````````````````````````````` ```````````````````````````````` example > foo ----- .

foo

```````````````````````````````` If you want a heading with `> foo` as its literal text, you can use backslash escapes: ```````````````````````````````` example \> foo ------ .

> foo

```````````````````````````````` **Compatibility note:** Most existing Markdown implementations do not allow the text of setext headings to span multiple lines. But there is no consensus about how to interpret ``` markdown Foo bar --- baz ``` One can find four different interpretations: 1. paragraph "Foo", heading "bar", paragraph "baz" 2. paragraph "Foo bar", thematic break, paragraph "baz" 3. paragraph "Foo bar --- baz" 4. heading "Foo bar", paragraph "baz" We find interpretation 4 most natural, and interpretation 4 increases the expressive power of CommonMark, by allowing multiline headings. Authors who want interpretation 1 can put a blank line after the first paragraph: ```````````````````````````````` example Foo bar --- baz .

Foo

bar

baz

```````````````````````````````` Authors who want interpretation 2 can put blank lines around the thematic break, ```````````````````````````````` example Foo bar --- baz .

Foo bar

baz

```````````````````````````````` or use a thematic break that cannot count as a [setext heading underline], such as ```````````````````````````````` example Foo bar * * * baz .

Foo bar

baz

```````````````````````````````` Authors who want interpretation 3 can use backslash escapes: ```````````````````````````````` example Foo bar \--- baz .

Foo bar --- baz

```````````````````````````````` ## Indented code blocks An [indented code block](@) is composed of one or more [indented chunks] separated by blank lines. An [indented chunk](@) is a sequence of non-blank lines, each indented four or more spaces. The contents of the code block are the literal contents of the lines, including trailing [line endings], minus four spaces of indentation. An indented code block has no [info string]. An indented code block cannot interrupt a paragraph, so there must be a blank line between a paragraph and a following indented code block. (A blank line is not needed, however, between a code block and a following paragraph.) ```````````````````````````````` example a simple indented code block . 
a simple
  indented code block
```````````````````````````````` If there is any ambiguity between an interpretation of indentation as a code block and as indicating that material belongs to a [list item][list items], the list item interpretation takes precedence: ```````````````````````````````` example - foo bar .

  • foo

    bar

```````````````````````````````` ```````````````````````````````` example 1. foo - bar .

  1. foo

    • bar
```````````````````````````````` The contents of a code block are literal text, and do not get parsed as Markdown: ```````````````````````````````` example *hi* - one . 
<a/>
*hi*

- one
```````````````````````````````` Here we have three chunks separated by blank lines: ```````````````````````````````` example chunk1 chunk2 chunk3 . 
chunk1

chunk2



chunk3
```````````````````````````````` Any initial spaces beyond four will be included in the content, even in interior blank lines: ```````````````````````````````` example chunk1 chunk2 . 
chunk1
  
  chunk2
```````````````````````````````` An indented code block cannot interrupt a paragraph. (This allows hanging indents and the like.) ```````````````````````````````` example Foo bar .

Foo bar

```````````````````````````````` However, any non-blank line with fewer than four leading spaces ends the code block immediately. So a paragraph may occur immediately after indented code: ```````````````````````````````` example foo bar . 
foo

bar

```````````````````````````````` And indented code can occur immediately before and after other kinds of blocks: ```````````````````````````````` example # Heading foo Heading ------ foo ---- .

Heading

foo

Heading

foo
```````````````````````````````` The first line can be indented more than four spaces: ```````````````````````````````` example foo bar . 
    foo
bar
```````````````````````````````` Blank lines preceding or following an indented code block are not included in it: ```````````````````````````````` example foo . 
foo
```````````````````````````````` Trailing spaces are included in the code block's content: ```````````````````````````````` example foo . 
foo
```````````````````````````````` ## Fenced code blocks A [code fence](@) is a sequence of at least three consecutive backtick characters (`` ` ``) or tildes (`~`). (Tildes and backticks cannot be mixed.) A [fenced code block](@) begins with a code fence, indented no more than three spaces. The line with the opening code fence may optionally contain some text following the code fence; this is trimmed of leading and trailing whitespace and called the [info string](@). If the [info string] comes after a backtick fence, it may not contain any backtick characters. (The reason for this restriction is that otherwise some inline code would be incorrectly interpreted as the beginning of a fenced code block.) The content of the code block consists of all subsequent lines, until a closing [code fence] of the same type as the code block began with (backticks or tildes), and with at least as many backticks or tildes as the opening code fence. If the leading code fence is indented N spaces, then up to N spaces of indentation are removed from each line of the content (if present). (If a content line is not indented, it is preserved unchanged. If it is indented less than N spaces, all of the indentation is removed.) The closing code fence may be indented up to three spaces, and may be followed only by spaces, which are ignored. If the end of the containing block (or document) is reached and no closing code fence has been found, the code block contains all of the lines after the opening code fence until the end of the containing block (or document). (An alternative spec would require backtracking in the event that a closing code fence is not found. But this makes parsing much less efficient, and there seems to be no real down side to the behavior described here.) A fenced code block may interrupt a paragraph, and does not require a blank line either before or after. The content of a code fence is treated as literal text, not parsed as inlines. The first word of the [info string] is typically used to specify the language of the code sample, and rendered in the `class` attribute of the `code` tag. However, this spec does not mandate any particular treatment of the [info string]. Here is a simple example with backticks: ```````````````````````````````` example ``` < > ``` . 
<
 >
```````````````````````````````` With tildes: ```````````````````````````````` example ~~~ < > ~~~ . 
<
 >
```````````````````````````````` Fewer than three backticks is not enough: ```````````````````````````````` example `` foo `` .

foo

```````````````````````````````` The closing code fence must use the same character as the opening fence: ```````````````````````````````` example ``` aaa ~~~ ``` . 
aaa
~~~
```````````````````````````````` ```````````````````````````````` example ~~~ aaa ``` ~~~ . 
aaa
```
```````````````````````````````` The closing code fence must be at least as long as the opening fence: ```````````````````````````````` example ```` aaa ``` `````` . 
aaa
```
```````````````````````````````` ```````````````````````````````` example ~~~~ aaa ~~~ ~~~~ . 
aaa
~~~
```````````````````````````````` Unclosed code blocks are closed by the end of the document (or the enclosing [block quote][block quotes] or [list item][list items]): ```````````````````````````````` example ``` .
```````````````````````````````` ```````````````````````````````` example ````` ``` aaa . 

```
aaa
```````````````````````````````` ```````````````````````````````` example > ``` > aaa bbb . 
aaa

bbb

```````````````````````````````` A code block can have all empty lines as its content: ```````````````````````````````` example ``` ``` .
```````````````````````````````` A code block can be empty: ```````````````````````````````` example ``` ``` .
```````````````````````````````` Fences can be indented. If the opening fence is indented, content lines will have equivalent opening indentation removed, if present: ```````````````````````````````` example ``` aaa aaa ``` . 
aaa
aaa
```````````````````````````````` ```````````````````````````````` example ``` aaa aaa aaa ``` . 
aaa
aaa
aaa
```````````````````````````````` ```````````````````````````````` example ``` aaa aaa aaa ``` . 
aaa
 aaa
aaa
```````````````````````````````` Four spaces indentation produces an indented code block: ```````````````````````````````` example ``` aaa ``` . 
```
aaa
```
```````````````````````````````` Closing fences may be indented by 0-3 spaces, and their indentation need not match that of the opening fence: ```````````````````````````````` example ``` aaa ``` . 
aaa
```````````````````````````````` ```````````````````````````````` example ``` aaa ``` . 
aaa
```````````````````````````````` This is not a closing fence, because it is indented 4 spaces: ```````````````````````````````` example ``` aaa ``` . 
aaa
    ```
```````````````````````````````` Code fences (opening and closing) cannot contain internal spaces: ```````````````````````````````` example ``` ``` aaa .

aaa

```````````````````````````````` ```````````````````````````````` example ~~~~~~ aaa ~~~ ~~ . 
aaa
~~~ ~~
```````````````````````````````` Fenced code blocks can interrupt paragraphs, and can be followed directly by paragraphs, without a blank line between: ```````````````````````````````` example foo ``` bar ``` baz .

foo

bar

baz

```````````````````````````````` Other blocks can also occur before and after fenced code blocks without an intervening blank line: ```````````````````````````````` example foo --- ~~~ bar ~~~ # baz .

foo

bar

baz

```````````````````````````````` An [info string] can be provided after the opening code fence. Although this spec doesn't mandate any particular treatment of the info string, the first word is typically used to specify the language of the code block. In HTML output, the language is normally indicated by adding a class to the `code` element consisting of `language-` followed by the language name. ```````````````````````````````` example ```ruby def foo(x) return 3 end ``` . 
def foo(x)
  return 3
end
```````````````````````````````` ```````````````````````````````` example ~~~~ ruby startline=3 $%@#$ def foo(x) return 3 end ~~~~~~~ . 
def foo(x)
  return 3
end
```````````````````````````````` ```````````````````````````````` example ````; ```` .
```````````````````````````````` [Info strings] for backtick code blocks cannot contain backticks: ```````````````````````````````` example ``` aa ``` foo .

aa foo

```````````````````````````````` [Info strings] for tilde code blocks can contain backticks and tildes: ```````````````````````````````` example ~~~ aa ``` ~~~ foo ~~~ . 
foo
```````````````````````````````` Closing code fences cannot have [info strings]: ```````````````````````````````` example ``` ``` aaa ``` . 
``` aaa
```````````````````````````````` ## HTML blocks An [HTML block](@) is a group of lines that is treated as raw HTML (and will not be escaped in HTML output). There are seven kinds of [HTML block], which can be defined by their start and end conditions. The block begins with a line that meets a [start condition](@) (after up to three spaces optional indentation). It ends with the first subsequent line that meets a matching [end condition](@), or the last line of the document, or the last line of the [container block](#container-blocks) containing the current HTML block, if no line is encountered that meets the [end condition]. If the first line meets both the [start condition] and the [end condition], the block will contain just that line. 1. **Start condition:** line begins with the string ``, or the end of the line.\ **End condition:** line contains an end tag ``, ``, or `` (case-insensitive; it need not match the start tag). 2. **Start condition:** line begins with the string ``. 3. **Start condition:** line begins with the string ``. 4. **Start condition:** line begins with the string ``. 5. **Start condition:** line begins with the string ``. 6. **Start condition:** line begins the string `<` or ``, or the string `/>`.\ **End condition:** line is followed by a [blank line]. 7. **Start condition:** line begins with a complete [open tag] (with any [tag name] other than `script`, `style`, or `pre`) or a complete [closing tag], followed only by [whitespace] or the end of the line.\ **End condition:** line is followed by a [blank line]. HTML blocks continue until they are closed by their appropriate [end condition], or the last line of the document or other [container block](#container-blocks). This means any HTML **within an HTML block** that might otherwise be recognised as a start condition will be ignored by the parser and passed through as-is, without changing the parser's state. For instance, `
` within a HTML block started by `` will not affect
the parser state; as the HTML block was started in by start condition 6, it
will end at any blank line. This can be surprising:

```````````````````````````````` example



**Hello**,

_world_.




.



**Hello**,

world.




````````````````````````````````

In this case, the HTML block is terminated by the newline — the `**Hello**`
text remains verbatim — and regular parsing resumes, with a paragraph,
emphasised `world` and inline and block HTML following.

All types of [HTML blocks] except type 7 may interrupt
a paragraph.  Blocks of type 7 may not interrupt a paragraph.
(This restriction is intended to prevent unwanted interpretation
of long tags inside a wrapped paragraph as starting HTML blocks.)

Some simple examples follow.  Here are some basic HTML blocks
of type 6:

```````````````````````````````` example


  

    
           hi
    
  




okay.
.


  

    
           hi
    
  




okay.

````````````````````````````````


```````````````````````````````` example
 
*foo*
````````````````````````````````


Here we have two HTML blocks with a Markdown paragraph between them:

```````````````````````````````` example



*Markdown*



.



Markdown



````````````````````````````````


The tag on the first line can be partial, as long
as it is split where there would be whitespace:

```````````````````````````````` example




.




````````````````````````````````


```````````````````````````````` example




.




````````````````````````````````


An open tag need not be closed:
```````````````````````````````` example


*foo*

*bar*
.


*foo*

bar

````````````````````````````````



A partial tag need not even be completed (garbage
in, garbage out):

```````````````````````````````` example

.

````````````````````````````````


```````````````````````````````` example


foo


.


foo


````````````````````````````````


Everything until the next blank line or end of document
gets included in the HTML block.  So, in the following
example, what looks like a Markdown code block
is actually part of the HTML block, which continues until a blank
line or the end of the document is reached:

```````````````````````````````` example


``` c
int x = 33;
```
.


``` c
int x = 33;
```
````````````````````````````````


To start an [HTML block] with a tag that is *not* in the
list of block-level tags in (6), you must put the tag by
itself on the first line (and it must be complete):

```````````````````````````````` example

*bar*

.

*bar*

````````````````````````````````


In type 7 blocks, the [tag name] can be anything:

```````````````````````````````` example

*bar*

.

*bar*

````````````````````````````````


```````````````````````````````` example

*bar*

.

*bar*

````````````````````````````````


```````````````````````````````` example

*bar*
.

*bar*
````````````````````````````````


These rules are designed to allow us to work with tags that
can function as either block-level or inline-level tags.
The `` tag is a nice example.  We can surround content with
`` tags in three different ways.  In this case, we get a raw
HTML block, because the `` tag is on a line by itself:

```````````````````````````````` example

*foo*

.

*foo*

````````````````````````````````


In this case, we get a raw HTML block that just includes
the `` tag (because it ends with the following blank
line).  So the contents get interpreted as CommonMark:

```````````````````````````````` example


*foo*


.


foo


````````````````````````````````


Finally, in this case, the `` tags are interpreted
as [raw HTML] *inside* the CommonMark paragraph.  (Because
the tag is not on a line by itself, we get inline HTML
rather than an [HTML block].)

```````````````````````````````` example
*foo*
.

foo

````````````````````````````````


HTML tags designed to contain literal content
(`script`, `style`, `pre`), comments, processing instructions,
and declarations are treated somewhat differently.
Instead of ending at the first blank line, these blocks
end at the first line containing a corresponding end tag.
As a result, these blocks can contain blank lines:

A pre tag (type 1):

```````````````````````````````` example


import Text.HTML.TagSoup

main :: IO ()
main = print $ parseTags tags


okay
.


import Text.HTML.TagSoup

main :: IO ()
main = print $ parseTags tags



okay

````````````````````````````````


A script tag (type 1):

```````````````````````````````` example

okay
.


okay

````````````````````````````````


A style tag (type 1):

```````````````````````````````` example

okay
.


okay

````````````````````````````````


If there is no matching end tag, the block will end at the
end of the document (or the enclosing [block quote][block quotes]
or [list item][list items]):

```````````````````````````````` example

*foo*
.


foo

````````````````````````````````


```````````````````````````````` example
*bar*
*baz*
.
*bar*

baz

````````````````````````````````


Note that anything on the last line after the
end tag will be included in the [HTML block]:

```````````````````````````````` example
1. *bar*
.
1. *bar*
````````````````````````````````


A comment (type 2):

```````````````````````````````` example

okay
.


okay

````````````````````````````````



A processing instruction (type 3):

```````````````````````````````` example
';

?>
okay
.
';

?>

okay

````````````````````````````````


A declaration (type 4):

```````````````````````````````` example

.

````````````````````````````````


CDATA (type 5):

```````````````````````````````` example

okay
.


okay

````````````````````````````````


The opening tag can be indented 1-3 spaces, but not 4:

```````````````````````````````` example
  

    
.
  

<!-- foo -->


````````````````````````````````


```````````````````````````````` example
  


    

.
  


<div>


````````````````````````````````


An HTML block of types 1--6 can interrupt a paragraph, and need not be
preceded by a blank line.

```````````````````````````````` example
Foo


bar


.

Foo



bar


````````````````````````````````


However, a following blank line is needed, except at the end of
a document, and except for blocks of types 1--5, [above][HTML
block]:

```````````````````````````````` example


bar


*foo*
.


bar


*foo*
````````````````````````````````


HTML blocks of type 7 cannot interrupt a paragraph:

```````````````````````````````` example
Foo

baz
.

Foo

baz

````````````````````````````````


This rule differs from John Gruber's original Markdown syntax
specification, which says:

> The only restrictions are that block-level HTML elements —
> e.g. `
`, ``, `
`, `
`, etc. — must be separated from
> surrounding content by blank lines, and the start and end tags of the
> block should not be indented with tabs or spaces.

In some ways Gruber's rule is more restrictive than the one given
here:

- It requires that an HTML block be preceded by a blank line.
- It does not allow the start tag to be indented.
- It requires a matching end tag, which it also does not allow to
  be indented.

Most Markdown implementations (including some of Gruber's own) do not
respect all of these restrictions.

There is one respect, however, in which Gruber's rule is more liberal
than the one given here, since it allows blank lines to occur inside
an HTML block.  There are two reasons for disallowing them here.
First, it removes the need to parse balanced tags, which is
expensive and can require backtracking from the end of the document
if no matching end tag is found. Second, it provides a very simple
and flexible way of including Markdown content inside HTML tags:
simply separate the Markdown from the HTML using blank lines:

Compare:

```````````````````````````````` example



*Emphasized* text.



.



Emphasized text.



````````````````````````````````


```````````````````````````````` example


*Emphasized* text.


.


*Emphasized* text.


````````````````````````````````


Some Markdown implementations have adopted a convention of
interpreting content inside tags as text if the open tag has
the attribute `markdown=1`.  The rule given above seems a simpler and
more elegant way of achieving the same expressive power, which is also
much simpler to parse.

The main potential drawback is that one can no longer paste HTML
blocks into Markdown documents with 100% reliability.  However,
*in most cases* this will work fine, because the blank lines in
HTML are usually followed by HTML block tags.  For example:

```````````````````````````````` example







Hi







.





Hi





````````````````````````````````


There are problems, however, if the inner tags are indented
*and* separated by spaces, as then they will be interpreted as
an indented code block:

```````````````````````````````` example



  


    
      Hi
    

  




.

<td>
  Hi
</td>


  


  



````````````````````````````````


Fortunately, blank lines are usually not necessary and can be
deleted.  The exception is inside `
` tags, but as described
[above][HTML blocks], raw HTML blocks starting with `
`
*can* contain blank lines.

## Link reference definitions

A [link reference definition](@)
consists of a [link label], indented up to three spaces, followed
by a colon (`:`), optional [whitespace] (including up to one
[line ending]), a [link destination],
optional [whitespace] (including up to one
[line ending]), and an optional [link
title], which if it is present must be separated
from the [link destination] by [whitespace].
No further [non-whitespace characters] may occur on the line.

A [link reference definition]
does not correspond to a structural element of a document.  Instead, it
defines a label which can be used in [reference links]
and reference-style [images] elsewhere in the document.  [Link
reference definitions] can come either before or after the links that use
them.

```````````````````````````````` example
[foo]: /url "title"

[foo]
.

foo

````````````````````````````````


```````````````````````````````` example
   [foo]: 
      /url  
           'the title'  

[foo]
.

foo

````````````````````````````````


```````````````````````````````` example
[Foo*bar\]]:my_(url) 'title (with parens)'

[Foo*bar\]]
.

Foo*bar]

````````````````````````````````


```````````````````````````````` example
[Foo bar]:

'title'

[Foo bar]
.

Foo bar

````````````````````````````````


The title may extend over multiple lines:

```````````````````````````````` example
[foo]: /url '
title
line1
line2
'

[foo]
.

foo

````````````````````````````````


However, it may not contain a [blank line]:

```````````````````````````````` example
[foo]: /url 'title

with blank line'

[foo]
.

[foo]: /url 'title


with blank line'


[foo]

````````````````````````````````


The title may be omitted:

```````````````````````````````` example
[foo]:
/url

[foo]
.

foo

````````````````````````````````


The link destination may not be omitted:

```````````````````````````````` example
[foo]:

[foo]
.

[foo]:


[foo]

````````````````````````````````

 However, an empty link destination may be specified using
 angle brackets:

```````````````````````````````` example
[foo]: <>

[foo]
.

foo

````````````````````````````````

The title must be separated from the link destination by
whitespace:

```````````````````````````````` example
[foo]: (baz)

[foo]
.

[foo]: (baz)


[foo]

````````````````````````````````


Both title and destination can contain backslash escapes
and literal backslashes:

```````````````````````````````` example
[foo]: /url\bar\*baz "foo\"bar\baz"

[foo]
.

foo

````````````````````````````````


A link can come before its corresponding definition:

```````````````````````````````` example
[foo]

[foo]: url
.

foo

````````````````````````````````


If there are several matching definitions, the first one takes
precedence:

```````````````````````````````` example
[foo]

[foo]: first
[foo]: second
.

foo

````````````````````````````````


As noted in the section on [Links], matching of labels is
case-insensitive (see [matches]).

```````````````````````````````` example
[FOO]: /url

[Foo]
.

Foo

````````````````````````````````


```````````````````````````````` example
[ΑΓΩ]: /φου

[αγω]
.

αγω

````````````````````````````````


Here is a link reference definition with no corresponding link.
It contributes nothing to the document.

```````````````````````````````` example
[foo]: /url
.
````````````````````````````````


Here is another one:

```````````````````````````````` example
[
foo
]: /url
bar
.

bar

````````````````````````````````


This is not a link reference definition, because there are
[non-whitespace characters] after the title:

```````````````````````````````` example
[foo]: /url "title" ok
.

[foo]: /url "title" ok

````````````````````````````````


This is a link reference definition, but it has no title:

```````````````````````````````` example
[foo]: /url
"title" ok
.

"title" ok

````````````````````````````````


This is not a link reference definition, because it is indented
four spaces:

```````````````````````````````` example
    [foo]: /url "title"

[foo]
.

[foo]: /url "title"



[foo]

````````````````````````````````


This is not a link reference definition, because it occurs inside
a code block:

```````````````````````````````` example
```
[foo]: /url
```

[foo]
.

[foo]: /url



[foo]

````````````````````````````````


A [link reference definition] cannot interrupt a paragraph.

```````````````````````````````` example
Foo
[bar]: /baz

[bar]
.

Foo
[bar]: /baz


[bar]

````````````````````````````````


However, it can directly follow other block elements, such as headings
and thematic breaks, and it need not be followed by a blank line.

```````````````````````````````` example
# [Foo]
[foo]: /url
> bar
.

Foo




bar



````````````````````````````````

```````````````````````````````` example
[foo]: /url
bar
===
[foo]
.

bar


foo

````````````````````````````````

```````````````````````````````` example
[foo]: /url
===
[foo]
.

===
foo

````````````````````````````````


Several [link reference definitions]
can occur one after another, without intervening blank lines.

```````````````````````````````` example
[foo]: /foo-url "foo"
[bar]: /bar-url
  "bar"
[baz]: /baz-url

[foo],
[bar],
[baz]
.

foo,
bar,
baz

````````````````````````````````


[Link reference definitions] can occur
inside block containers, like lists and block quotations.  They
affect the entire document, not just the container in which they
are defined:

```````````````````````````````` example
[foo]

> [foo]: /url
.

foo





````````````````````````````````


Whether something is a [link reference definition] is
independent of whether the link reference it defines is
used in the document.  Thus, for example, the following
document contains just a link reference definition, and
no visible content:

```````````````````````````````` example
[foo]: /url
.
````````````````````````````````


## Paragraphs

A sequence of non-blank lines that cannot be interpreted as other
kinds of blocks forms a [paragraph](@).
The contents of the paragraph are the result of parsing the
paragraph's raw content as inlines.  The paragraph's raw content
is formed by concatenating the lines and removing initial and final
[whitespace].

A simple example with two paragraphs:

```````````````````````````````` example
aaa

bbb
.

aaa


bbb

````````````````````````````````


Paragraphs can contain multiple lines, but no blank lines:

```````````````````````````````` example
aaa
bbb

ccc
ddd
.

aaa
bbb


ccc
ddd

````````````````````````````````


Multiple blank lines between paragraph have no effect:

```````````````````````````````` example
aaa


bbb
.

aaa


bbb

````````````````````````````````


Leading spaces are skipped:

```````````````````````````````` example
  aaa
 bbb
.

aaa
bbb

````````````````````````````````


Lines after the first may be indented any amount, since indented
code blocks cannot interrupt paragraphs.

```````````````````````````````` example
aaa
             bbb
                                       ccc
.

aaa
bbb
ccc

````````````````````````````````


However, the first line may be indented at most three spaces,
or an indented code block will be triggered:

```````````````````````````````` example
   aaa
bbb
.

aaa
bbb

````````````````````````````````


```````````````````````````````` example
    aaa
bbb
.

aaa



bbb

````````````````````````````````


Final spaces are stripped before inline parsing, so a paragraph
that ends with two or more spaces will not end with a [hard line
break]:

```````````````````````````````` example
aaa     
bbb     
.

aaa

bbb

````````````````````````````````


## Blank lines

[Blank lines] between block-level elements are ignored,
except for the role they play in determining whether a [list]
is [tight] or [loose].

Blank lines at the beginning and end of the document are also ignored.

```````````````````````````````` example
  

aaa
  

# aaa

  
.

aaa


aaa

````````````````````````````````



# Container blocks

A [container block](#container-blocks) is a block that has other
blocks as its contents.  There are two basic kinds of container blocks:
[block quotes] and [list items].
[Lists] are meta-containers for [list items].

We define the syntax for container blocks recursively.  The general
form of the definition is:

> If X is a sequence of blocks, then the result of
> transforming X in such-and-such a way is a container of type Y
> with these blocks as its content.

So, we explain what counts as a block quote or list item by explaining
how these can be *generated* from their contents. This should suffice
to define the syntax, although it does not give a recipe for *parsing*
these constructions.  (A recipe is provided below in the section entitled
[A parsing strategy](#appendix-a-parsing-strategy).)

## Block quotes

A [block quote marker](@)
consists of 0-3 spaces of initial indent, plus (a) the character `>` together
with a following space, or (b) a single character `>` not followed by a space.

The following rules define [block quotes]:

1.  **Basic case.**  If a string of lines *Ls* constitute a sequence
    of blocks *Bs*, then the result of prepending a [block quote
    marker] to the beginning of each line in *Ls*
    is a [block quote](#block-quotes) containing *Bs*.

2.  **Laziness.**  If a string of lines *Ls* constitute a [block
    quote](#block-quotes) with contents *Bs*, then the result of deleting
    the initial [block quote marker] from one or
    more lines in which the next [non-whitespace character] after the [block
    quote marker] is [paragraph continuation
    text] is a block quote with *Bs* as its content.
    [Paragraph continuation text](@) is text
    that will be parsed as part of the content of a paragraph, but does
    not occur at the beginning of the paragraph.

3.  **Consecutiveness.**  A document cannot contain two [block
    quotes] in a row unless there is a [blank line] between them.

Nothing else counts as a [block quote](#block-quotes).

Here is a simple example:

```````````````````````````````` example
> # Foo
> bar
> baz
.



Foo


bar
baz



````````````````````````````````


The spaces after the `>` characters can be omitted:

```````````````````````````````` example
># Foo
>bar
> baz
.



Foo


bar
baz



````````````````````````````````


The `>` characters can be indented 1-3 spaces:

```````````````````````````````` example
   > # Foo
   > bar
 > baz
.



Foo


bar
baz



````````````````````````````````


Four spaces gives us a code block:

```````````````````````````````` example
    > # Foo
    > bar
    > baz
.

> # Foo
> bar
> baz


````````````````````````````````


The Laziness clause allows us to omit the `>` before
[paragraph continuation text]:

```````````````````````````````` example
> # Foo
> bar
baz
.



Foo


bar
baz



````````````````````````````````


A block quote can contain some lazy and some non-lazy
continuation lines:

```````````````````````````````` example
> bar
baz
> foo
.



bar
baz
foo



````````````````````````````````


Laziness only applies to lines that would have been continuations of
paragraphs had they been prepended with [block quote markers].
For example, the `> ` cannot be omitted in the second line of

``` markdown
> foo
> ---
```

without changing the meaning:

```````````````````````````````` example
> foo
---
.



foo





````````````````````````````````


Similarly, if we omit the `> ` in the second line of

``` markdown
> - foo
> - bar
```

then the block quote ends after the first line:

```````````````````````````````` example
> - foo
- bar
.



    

  • foo
    • 





    

  • bar
    • 


````````````````````````````````


For the same reason, we can't omit the `> ` in front of
subsequent lines of an indented or fenced code block:

```````````````````````````````` example
>     foo
    bar
.



foo





bar


````````````````````````````````


```````````````````````````````` example
> ```
foo
```
.







foo



````````````````````````````````


Note that in the following case, we have a [lazy
continuation line]:

```````````````````````````````` example
> foo
    - bar
.



foo
- bar



````````````````````````````````


To see why, note that in

```markdown
> foo
>     - bar
```

the `- bar` is indented too far to start a list, and can't
be an indented code block because indented code blocks cannot
interrupt paragraphs, so it is [paragraph continuation text].

A block quote can be empty:

```````````````````````````````` example
>
.




````````````````````````````````


```````````````````````````````` example
>
>  
> 
.




````````````````````````````````


A block quote can have initial or final blank lines:

```````````````````````````````` example
>
> foo
>  
.



foo



````````````````````````````````


A blank line always separates block quotes:

```````````````````````````````` example
> foo

> bar
.



foo






bar



````````````````````````````````


(Most current Markdown implementations, including John Gruber's
original `Markdown.pl`, will parse this example as a single block quote
with two paragraphs.  But it seems better to allow the author to decide
whether two block quotes or one are wanted.)

Consecutiveness means that if we put these block quotes together,
we get a single block quote:

```````````````````````````````` example
> foo
> bar
.



foo
bar



````````````````````````````````


To get a block quote with two paragraphs, use:

```````````````````````````````` example
> foo
>
> bar
.



foo


bar



````````````````````````````````


Block quotes can interrupt paragraphs:

```````````````````````````````` example
foo
> bar
.

foo




bar



````````````````````````````````


In general, blank lines are not needed before or after block
quotes:

```````````````````````````````` example
> aaa
***
> bbb
.



aaa








bbb



````````````````````````````````


However, because of laziness, a blank line is needed between
a block quote and a following paragraph:

```````````````````````````````` example
> bar
baz
.



bar
baz



````````````````````````````````


```````````````````````````````` example
> bar

baz
.



bar




baz

````````````````````````````````


```````````````````````````````` example
> bar
>
baz
.



bar




baz

````````````````````````````````


It is a consequence of the Laziness rule that any number
of initial `>`s may be omitted on a continuation line of a
nested block quote:

```````````````````````````````` example
> > > foo
bar
.







foo
bar







````````````````````````````````


```````````````````````````````` example
>>> foo
> bar
>>baz
.







foo
bar
baz







````````````````````````````````


When including an indented code block in a block quote,
remember that the [block quote marker] includes
both the `>` and a following space.  So *five spaces* are needed after
the `>`:

```````````````````````````````` example
>     code

>    not code
.



code







not code



````````````````````````````````



## List items

A [list marker](@) is a
[bullet list marker] or an [ordered list marker].

A [bullet list marker](@)
is a `-`, `+`, or `*` character.

An [ordered list marker](@)
is a sequence of 1--9 arabic digits (`0-9`), followed by either a
`.` character or a `)` character.  (The reason for the length
limit is that with 10 digits we start seeing integer overflows
in some browsers.)

The following rules define [list items]:

1.  **Basic case.**  If a sequence of lines *Ls* constitute a sequence of
    blocks *Bs* starting with a [non-whitespace character], and *M* is a
    list marker of width *W* followed by 1 ≤ *N* ≤ 4 spaces, then the result
    of prepending *M* and the following spaces to the first line of
    *Ls*, and indenting subsequent lines of *Ls* by *W + N* spaces, is a
    list item with *Bs* as its contents.  The type of the list item
    (bullet or ordered) is determined by the type of its list marker.
    If the list item is ordered, then it is also assigned a start
    number, based on the ordered list marker.

    Exceptions:

    1. When the first list item in a [list] interrupts
       a paragraph---that is, when it starts on a line that would
       otherwise count as [paragraph continuation text]---then (a)
       the lines *Ls* must not begin with a blank line, and (b) if
       the list item is ordered, the start number must be 1.
    2. If any line is a [thematic break][thematic breaks] then
       that line is not a list item.

For example, let *Ls* be the lines

```````````````````````````````` example
A paragraph
with two lines.

    indented code

> A block quote.
.

A paragraph
with two lines.


indented code





A block quote.



````````````````````````````````


And let *M* be the marker `1.`, and *N* = 2.  Then rule #1 says
that the following is an ordered list item with start number 1,
and the same contents as *Ls*:

```````````````````````````````` example
1.  A paragraph
    with two lines.

        indented code

    > A block quote.
.

    

  1. 

    A paragraph
with two lines.
    

    indented code

    

    

    A block quote.
    

    

    2. 


````````````````````````````````


The most important thing to notice is that the position of
the text after the list marker determines how much indentation
is needed in subsequent blocks in the list item.  If the list
marker takes up two spaces, and there are three spaces between
the list marker and the next [non-whitespace character], then blocks
must be indented five spaces in order to fall under the list
item.

Here are some examples showing how far content must be indented to be
put under the list item:

```````````````````````````````` example
- one

 two
.

    

  • one
    • 



two

````````````````````````````````


```````````````````````````````` example
- one

  two
.

    

  • 

    one
    

    two
    

    • 


````````````````````````````````


```````````````````````````````` example
 -    one

     two
.

    

  • one
    • 



 two


````````````````````````````````


```````````````````````````````` example
 -    one

      two
.

    

  • 

    one
    

    two
    

    • 


````````````````````````````````


It is tempting to think of this in terms of columns:  the continuation
blocks must be indented at least to the column of the first
[non-whitespace character] after the list marker. However, that is not quite right.
The spaces after the list marker determine how much relative indentation
is needed.  Which column this indentation reaches will depend on
how the list item is embedded in other constructions, as shown by
this example:

```````````````````````````````` example
   > > 1.  one
>>
>>     two
.





    

  1. 

    one
    

    two
    

    2. 






````````````````````````````````


Here `two` occurs in the same column as the list marker `1.`,
but is actually contained in the list item, because there is
sufficient indentation after the last containing blockquote marker.

The converse is also possible.  In the following example, the word `two`
occurs far to the right of the initial text of the list item, `one`, but
it is not considered part of the list item, because it is not indented
far enough past the blockquote marker:

```````````````````````````````` example
>>- one
>>
  >  > two
.





    

  • one
    • 



two





````````````````````````````````


Note that at least one space is needed between the list marker and
any following content, so these are not list items:

```````````````````````````````` example
-one

2.two
.

-one


2.two

````````````````````````````````


A list item may contain blocks that are separated by more than
one blank line.

```````````````````````````````` example
- foo


  bar
.

    

  • 

    foo
    

    bar
    

    • 


````````````````````````````````


A list item may contain any kind of block:

```````````````````````````````` example
1.  foo

    ```
    bar
    ```

    baz

    > bam
.

    

  1. 

    foo
    

    bar

    

    baz
    

    

    bam
    

    

    2. 


````````````````````````````````


A list item that contains an indented code block will preserve
empty lines within the code block verbatim.

```````````````````````````````` example
- Foo

      bar


      baz
.

    

  • 

    Foo
    

    bar


baz

    

    • 


````````````````````````````````

Note that ordered list start numbers must be nine digits or less:

```````````````````````````````` example
123456789. ok
.

    

  1. ok
    2. 


````````````````````````````````


```````````````````````````````` example
1234567890. not ok
.

1234567890. not ok

````````````````````````````````


A start number may begin with 0s:

```````````````````````````````` example
0. ok
.

    

  1. ok
    2. 


````````````````````````````````


```````````````````````````````` example
003. ok
.

    

  1. ok
    2. 


````````````````````````````````


A start number may not be negative:

```````````````````````````````` example
-1. not ok
.

-1. not ok

````````````````````````````````



2.  **Item starting with indented code.**  If a sequence of lines *Ls*
    constitute a sequence of blocks *Bs* starting with an indented code
    block, and *M* is a list marker of width *W* followed by
    one space, then the result of prepending *M* and the following
    space to the first line of *Ls*, and indenting subsequent lines of
    *Ls* by *W + 1* spaces, is a list item with *Bs* as its contents.
    If a line is empty, then it need not be indented.  The type of the
    list item (bullet or ordered) is determined by the type of its list
    marker.  If the list item is ordered, then it is also assigned a
    start number, based on the ordered list marker.

An indented code block will have to be indented four spaces beyond
the edge of the region where text will be included in the list item.
In the following case that is 6 spaces:

```````````````````````````````` example
- foo

      bar
.

    

  • 

    foo
    

    bar

    

    • 


````````````````````````````````


And in this case it is 11 spaces:

```````````````````````````````` example
  10.  foo

           bar
.

    

  1. 

    foo
    

    bar

    

    2. 


````````````````````````````````


If the *first* block in the list item is an indented code block,
then by rule #2, the contents must be indented *one* space after the
list marker:

```````````````````````````````` example
    indented code

paragraph

    more code
.

indented code



paragraph


more code


````````````````````````````````


```````````````````````````````` example
1.     indented code

   paragraph

       more code
.

    

  1. 

    indented code

    

    paragraph
    

    more code

    

    2. 


````````````````````````````````


Note that an additional space indent is interpreted as space
inside the code block:

```````````````````````````````` example
1.      indented code

   paragraph

       more code
.

    

  1. 

     indented code

    

    paragraph
    

    more code

    

    2. 


````````````````````````````````


Note that rules #1 and #2 only apply to two cases:  (a) cases
in which the lines to be included in a list item begin with a
[non-whitespace character], and (b) cases in which
they begin with an indented code
block.  In a case like the following, where the first block begins with
a three-space indent, the rules do not allow us to form a list item by
indenting the whole thing and prepending a list marker:

```````````````````````````````` example
   foo

bar
.

foo


bar

````````````````````````````````


```````````````````````````````` example
-    foo

  bar
.

    

  • foo
    • 



bar

````````````````````````````````


This is not a significant restriction, because when a block begins
with 1-3 spaces indent, the indentation can always be removed without
a change in interpretation, allowing rule #1 to be applied.  So, in
the above case:

```````````````````````````````` example
-  foo

   bar
.

    

  • 

    foo
    

    bar
    

    • 


````````````````````````````````


3.  **Item starting with a blank line.**  If a sequence of lines *Ls*
    starting with a single [blank line] constitute a (possibly empty)
    sequence of blocks *Bs*, not separated from each other by more than
    one blank line, and *M* is a list marker of width *W*,
    then the result of prepending *M* to the first line of *Ls*, and
    indenting subsequent lines of *Ls* by *W + 1* spaces, is a list
    item with *Bs* as its contents.
    If a line is empty, then it need not be indented.  The type of the
    list item (bullet or ordered) is determined by the type of its list
    marker.  If the list item is ordered, then it is also assigned a
    start number, based on the ordered list marker.

Here are some list items that start with a blank line but are not empty:

```````````````````````````````` example
-
  foo
-
  ```
  bar
  ```
-
      baz
.

    

  • foo
    • 

  • 

    bar

    

    • 

  • 

    baz

    

    • 


````````````````````````````````

When the list item starts with a blank line, the number of spaces
following the list marker doesn't change the required indentation:

```````````````````````````````` example
-   
  foo
.

    

  • foo
    • 


````````````````````````````````


A list item can begin with at most one blank line.
In the following example, `foo` is not part of the list
item:

```````````````````````````````` example
-

  foo
.

    

  • 



foo

````````````````````````````````


Here is an empty bullet list item:

```````````````````````````````` example
- foo
-
- bar
.

    

  • foo
    • 

  • 

  • bar
    • 


````````````````````````````````


It does not matter whether there are spaces following the [list marker]:

```````````````````````````````` example
- foo
-   
- bar
.

    

  • foo
    • 

  • 

  • bar
    • 


````````````````````````````````


Here is an empty ordered list item:

```````````````````````````````` example
1. foo
2.
3. bar
.

    

  1. foo
    2. 

  2. 

  3. bar
    4. 


````````````````````````````````


A list may start or end with an empty list item:

```````````````````````````````` example
*
.

    

  • 


````````````````````````````````

However, an empty list item cannot interrupt a paragraph:

```````````````````````````````` example
foo
*

foo
1.
.

foo
*


foo
1.

````````````````````````````````


4.  **Indentation.**  If a sequence of lines *Ls* constitutes a list item
    according to rule #1, #2, or #3, then the result of indenting each line
    of *Ls* by 1-3 spaces (the same for each line) also constitutes a
    list item with the same contents and attributes.  If a line is
    empty, then it need not be indented.

Indented one space:

```````````````````````````````` example
 1.  A paragraph
     with two lines.

         indented code

     > A block quote.
.

    

  1. 

    A paragraph
with two lines.
    

    indented code

    

    

    A block quote.
    

    

    2. 


````````````````````````````````


Indented two spaces:

```````````````````````````````` example
  1.  A paragraph
      with two lines.

          indented code

      > A block quote.
.

    

  1. 

    A paragraph
with two lines.
    

    indented code

    

    

    A block quote.
    

    

    2. 


````````````````````````````````


Indented three spaces:

```````````````````````````````` example
   1.  A paragraph
       with two lines.

           indented code

       > A block quote.
.

    

  1. 

    A paragraph
with two lines.
    

    indented code

    

    

    A block quote.
    

    

    2. 


````````````````````````````````


Four spaces indent gives a code block:

```````````````````````````````` example
    1.  A paragraph
        with two lines.

            indented code

        > A block quote.
.

1.  A paragraph
    with two lines.

        indented code

    > A block quote.


````````````````````````````````



5.  **Laziness.**  If a string of lines *Ls* constitute a [list
    item](#list-items) with contents *Bs*, then the result of deleting
    some or all of the indentation from one or more lines in which the
    next [non-whitespace character] after the indentation is
    [paragraph continuation text] is a
    list item with the same contents and attributes.  The unindented
    lines are called
    [lazy continuation line](@)s.

Here is an example with [lazy continuation lines]:

```````````````````````````````` example
  1.  A paragraph
with two lines.

          indented code

      > A block quote.
.

    

  1. 

    A paragraph
with two lines.
    

    indented code

    

    

    A block quote.
    

    

    2. 


````````````````````````````````


Indentation can be partially deleted:

```````````````````````````````` example
  1.  A paragraph
    with two lines.
.

    

  1. A paragraph
with two lines.
    2. 


````````````````````````````````


These examples show how laziness can work in nested structures:

```````````````````````````````` example
> 1. > Blockquote
continued here.
.



    

  1. 

    

    Blockquote
continued here.
    

    

    2. 




````````````````````````````````


```````````````````````````````` example
> 1. > Blockquote
> continued here.
.



    

  1. 

    

    Blockquote
continued here.
    

    

    2. 




````````````````````````````````



6.  **That's all.** Nothing that is not counted as a list item by rules
    #1--5 counts as a [list item](#list-items).

The rules for sublists follow from the general rules
[above][List items].  A sublist must be indented the same number
of spaces a paragraph would need to be in order to be included
in the list item.

So, in this case we need two spaces indent:

```````````````````````````````` example
- foo
  - bar
    - baz
      - boo
.

    

  • foo

      

    • bar

        

      • baz

          

        • boo
          • 

        

        • 

      

      • 

    

    • 


````````````````````````````````


One is not enough:

```````````````````````````````` example
- foo
 - bar
  - baz
   - boo
.

    

  • foo
    • 

  • bar
    • 

  • baz
    • 

  • boo
    • 


````````````````````````````````


Here we need four, because the list marker is wider:

```````````````````````````````` example
10) foo
    - bar
.

    

  1. foo

      

    • bar
      • 

    

    2. 


````````````````````````````````


Three is not enough:

```````````````````````````````` example
10) foo
   - bar
.

    

  1. foo
    2. 



    

  • bar
    • 


````````````````````````````````


A list may be the first block in a list item:

```````````````````````````````` example
- - foo
.

    

  • 

      

    • foo
      • 

    

    • 


````````````````````````````````


```````````````````````````````` example
1. - 2. foo
.

    

  1. 

      

    • 

        

      1. foo
        2. 

      

      • 

    

    2. 


````````````````````````````````


A list item can contain a heading:

```````````````````````````````` example
- # Foo
- Bar
  ---
  baz
.

    

  • 

    Foo
    

    • 

  • 

    Bar
    
baz
    • 


````````````````````````````````


### Motivation

John Gruber's Markdown spec says the following about list items:

1. "List markers typically start at the left margin, but may be indented
   by up to three spaces. List markers must be followed by one or more
   spaces or a tab."

2. "To make lists look nice, you can wrap items with hanging indents....
   But if you don't want to, you don't have to."

3. "List items may consist of multiple paragraphs. Each subsequent
   paragraph in a list item must be indented by either 4 spaces or one
   tab."

4. "It looks nice if you indent every line of the subsequent paragraphs,
   but here again, Markdown will allow you to be lazy."

5. "To put a blockquote within a list item, the blockquote's `>`
   delimiters need to be indented."

6. "To put a code block within a list item, the code block needs to be
   indented twice — 8 spaces or two tabs."

These rules specify that a paragraph under a list item must be indented
four spaces (presumably, from the left margin, rather than the start of
the list marker, but this is not said), and that code under a list item
must be indented eight spaces instead of the usual four.  They also say
that a block quote must be indented, but not by how much; however, the
example given has four spaces indentation.  Although nothing is said
about other kinds of block-level content, it is certainly reasonable to
infer that *all* block elements under a list item, including other
lists, must be indented four spaces.  This principle has been called the
*four-space rule*.

The four-space rule is clear and principled, and if the reference
implementation `Markdown.pl` had followed it, it probably would have
become the standard.  However, `Markdown.pl` allowed paragraphs and
sublists to start with only two spaces indentation, at least on the
outer level.  Worse, its behavior was inconsistent: a sublist of an
outer-level list needed two spaces indentation, but a sublist of this
sublist needed three spaces.  It is not surprising, then, that different
implementations of Markdown have developed very different rules for
determining what comes under a list item.  (Pandoc and python-Markdown,
for example, stuck with Gruber's syntax description and the four-space
rule, while discount, redcarpet, marked, PHP Markdown, and others
followed `Markdown.pl`'s behavior more closely.)

Unfortunately, given the divergences between implementations, there
is no way to give a spec for list items that will be guaranteed not
to break any existing documents.  However, the spec given here should
correctly handle lists formatted with either the four-space rule or
the more forgiving `Markdown.pl` behavior, provided they are laid out
in a way that is natural for a human to read.

The strategy here is to let the width and indentation of the list marker
determine the indentation necessary for blocks to fall under the list
item, rather than having a fixed and arbitrary number.  The writer can
think of the body of the list item as a unit which gets indented to the
right enough to fit the list marker (and any indentation on the list
marker).  (The laziness rule, #5, then allows continuation lines to be
unindented if needed.)

This rule is superior, we claim, to any rule requiring a fixed level of
indentation from the margin.  The four-space rule is clear but
unnatural. It is quite unintuitive that

``` markdown
- foo

  bar

  - baz
```

should be parsed as two lists with an intervening paragraph,

``` html

    

  • foo
    • 



bar


    

  • baz
    • 


```

as the four-space rule demands, rather than a single list,

``` html

    

  • 

    foo
    

    bar
    

      

    • baz
      • 

    

    • 


```

The choice of four spaces is arbitrary.  It can be learned, but it is
not likely to be guessed, and it trips up beginners regularly.

Would it help to adopt a two-space rule?  The problem is that such
a rule, together with the rule allowing 1--3 spaces indentation of the
initial list marker, allows text that is indented *less than* the
original list marker to be included in the list item. For example,
`Markdown.pl` parses

``` markdown
   - one

  two
```

as a single list item, with `two` a continuation paragraph:

``` html

    

  • 

    one
    

    two
    

    • 


```

and similarly

``` markdown
>   - one
>
>  two
```

as

``` html



    

  • 

    one
    

    two
    

    • 




```

This is extremely unintuitive.

Rather than requiring a fixed indent from the margin, we could require
a fixed indent (say, two spaces, or even one space) from the list marker (which
may itself be indented).  This proposal would remove the last anomaly
discussed.  Unlike the spec presented above, it would count the following
as a list item with a subparagraph, even though the paragraph `bar`
is not indented as far as the first paragraph `foo`:

``` markdown
 10. foo

   bar  
```

Arguably this text does read like a list item with `bar` as a subparagraph,
which may count in favor of the proposal.  However, on this proposal indented
code would have to be indented six spaces after the list marker.  And this
would break a lot of existing Markdown, which has the pattern:

``` markdown
1.  foo

        indented code
```

where the code is indented eight spaces.  The spec above, by contrast, will
parse this text as expected, since the code block's indentation is measured
from the beginning of `foo`.

The one case that needs special treatment is a list item that *starts*
with indented code.  How much indentation is required in that case, since
we don't have a "first paragraph" to measure from?  Rule #2 simply stipulates
that in such cases, we require one space indentation from the list marker
(and then the normal four spaces for the indented code).  This will match the
four-space rule in cases where the list marker plus its initial indentation
takes four spaces (a common case), but diverge in other cases.

## Lists

A [list](@) is a sequence of one or more
list items [of the same type].  The list items
may be separated by any number of blank lines.

Two list items are [of the same type](@)
if they begin with a [list marker] of the same type.
Two list markers are of the
same type if (a) they are bullet list markers using the same character
(`-`, `+`, or `*`) or (b) they are ordered list numbers with the same
delimiter (either `.` or `)`).

A list is an [ordered list](@)
if its constituent list items begin with
[ordered list markers], and a
[bullet list](@) if its constituent list
items begin with [bullet list markers].

The [start number](@)
of an [ordered list] is determined by the list number of
its initial list item.  The numbers of subsequent list items are
disregarded.

A list is [loose](@) if any of its constituent
list items are separated by blank lines, or if any of its constituent
list items directly contain two block-level elements with a blank line
between them.  Otherwise a list is [tight](@).
(The difference in HTML output is that paragraphs in a loose list are
wrapped in `
` tags, while paragraphs in a tight list are not.)

Changing the bullet or ordered list delimiter starts a new list:

```````````````````````````````` example
- foo
- bar
+ baz
.

    

  • foo
    • 

  • bar
    • 



    

  • baz
    • 


````````````````````````````````


```````````````````````````````` example
1. foo
2. bar
3) baz
.

    

  1. foo
    2. 

  2. bar
    3. 



    

  1. baz
    2. 


````````````````````````````````


In CommonMark, a list can interrupt a paragraph. That is,
no blank line is needed to separate a paragraph from a following
list:

```````````````````````````````` example
Foo
- bar
- baz
.

Foo


    

  • bar
    • 

  • baz
    • 


````````````````````````````````

`Markdown.pl` does not allow this, through fear of triggering a list
via a numeral in a hard-wrapped line:

``` markdown
The number of windows in my house is
14.  The number of doors is 6.
```

Oddly, though, `Markdown.pl` *does* allow a blockquote to
interrupt a paragraph, even though the same considerations might
apply.

In CommonMark, we do allow lists to interrupt paragraphs, for
two reasons.  First, it is natural and not uncommon for people
to start lists without blank lines:

``` markdown
I need to buy
- new shoes
- a coat
- a plane ticket
```

Second, we are attracted to a

> [principle of uniformity](@):
> if a chunk of text has a certain
> meaning, it will continue to have the same meaning when put into a
> container block (such as a list item or blockquote).

(Indeed, the spec for [list items] and [block quotes] presupposes
this principle.) This principle implies that if

``` markdown
  * I need to buy
    - new shoes
    - a coat
    - a plane ticket
```

is a list item containing a paragraph followed by a nested sublist,
as all Markdown implementations agree it is (though the paragraph
may be rendered without `
` tags, since the list is "tight"),
then

``` markdown
I need to buy
- new shoes
- a coat
- a plane ticket
```

by itself should be a paragraph followed by a nested sublist.

Since it is well established Markdown practice to allow lists to
interrupt paragraphs inside list items, the [principle of
uniformity] requires us to allow this outside list items as
well.  ([reStructuredText](http://docutils.sourceforge.net/rst.html)
takes a different approach, requiring blank lines before lists
even inside other list items.)

In order to solve of unwanted lists in paragraphs with
hard-wrapped numerals, we allow only lists starting with `1` to
interrupt paragraphs.  Thus,

```````````````````````````````` example
The number of windows in my house is
14.  The number of doors is 6.
.

The number of windows in my house is
14.  The number of doors is 6.

````````````````````````````````

We may still get an unintended result in cases like

```````````````````````````````` example
The number of windows in my house is
1.  The number of doors is 6.
.

The number of windows in my house is


    

  1. The number of doors is 6.
    2. 


````````````````````````````````

but this rule should prevent most spurious list captures.

There can be any number of blank lines between items:

```````````````````````````````` example
- foo

- bar


- baz
.

    

  • 

    foo
    

    • 

  • 

    bar
    

    • 

  • 

    baz
    

    • 


````````````````````````````````

```````````````````````````````` example
- foo
  - bar
    - baz


      bim
.

    

  • foo

      

    • bar

        

      • 

        baz
        

        bim
        

        • 

      

      • 

    

    • 


````````````````````````````````


To separate consecutive lists of the same type, or to separate a
list from an indented code block that would otherwise be parsed
as a subparagraph of the final list item, you can insert a blank HTML
comment:

```````````````````````````````` example
- foo
- bar



- baz
- bim
.

    

  • foo
    • 

  • bar
    • 




    

  • baz
    • 

  • bim
    • 


````````````````````````````````


```````````````````````````````` example
-   foo

    notcode

-   foo



    code
.

    

  • 

    foo
    

    notcode
    

    • 

  • 

    foo
    

    • 




code


````````````````````````````````


List items need not be indented to the same level.  The following
list items will be treated as items at the same list level,
since none is indented enough to belong to the previous list
item:

```````````````````````````````` example
- a
 - b
  - c
   - d
  - e
 - f
- g
.

    

  • a
    • 

  • b
    • 

  • c
    • 

  • d
    • 

  • e
    • 

  • f
    • 

  • g
    • 


````````````````````````````````


```````````````````````````````` example
1. a

  2. b

   3. c
.

    

  1. 

    a
    

    2. 

  2. 

    b
    

    3. 

  3. 

    c
    

    4. 


````````````````````````````````

Note, however, that list items may not be indented more than
three spaces.  Here `- e` is treated as a paragraph continuation
line, because it is indented more than three spaces:

```````````````````````````````` example
- a
 - b
  - c
   - d
    - e
.

    

  • a
    • 

  • b
    • 

  • c
    • 

  • d
- e
    • 


````````````````````````````````

And here, `3. c` is treated as in indented code block,
because it is indented four spaces and preceded by a
blank line.

```````````````````````````````` example
1. a

  2. b

    3. c
.

    

  1. 

    a
    

    2. 

  2. 

    b
    

    3. 



3. c


````````````````````````````````


This is a loose list, because there is a blank line between
two of the list items:

```````````````````````````````` example
- a
- b

- c
.

    

  • 

    a
    

    • 

  • 

    b
    

    • 

  • 

    c
    

    • 


````````````````````````````````


So is this, with a empty second item:

```````````````````````````````` example
* a
*

* c
.

    

  • 

    a
    

    • 

  • 

  • 

    c
    

    • 


````````````````````````````````


These are loose lists, even though there is no space between the items,
because one of the items directly contains two block-level elements
with a blank line between them:

```````````````````````````````` example
- a
- b

  c
- d
.

    

  • 

    a
    

    • 

  • 

    b
    

    c
    

    • 

  • 

    d
    

    • 


````````````````````````````````


```````````````````````````````` example
- a
- b

  [ref]: /url
- d
.

    

  • 

    a
    

    • 

  • 

    b
    

    • 

  • 

    d
    

    • 


````````````````````````````````


This is a tight list, because the blank lines are in a code block:

```````````````````````````````` example
- a
- ```
  b


  ```
- c
.

    

  • a
    • 

  • 

    b



    

    • 

  • c
    • 


````````````````````````````````


This is a tight list, because the blank line is between two
paragraphs of a sublist.  So the sublist is loose while
the outer list is tight:

```````````````````````````````` example
- a
  - b

    c
- d
.

    

  • a

      

    • 

      b
      

      c
      

      • 

    

    • 

  • d
    • 


````````````````````````````````


This is a tight list, because the blank line is inside the
block quote:

```````````````````````````````` example
* a
  > b
  >
* c
.

    

  • a

    

    b
    

    

    • 

  • c
    • 


````````````````````````````````


This list is tight, because the consecutive block elements
are not separated by blank lines:

```````````````````````````````` example
- a
  > b
  ```
  c
  ```
- d
.

    

  • a

    

    b
    

    

    c

    

    • 

  • d
    • 


````````````````````````````````


A single-paragraph list is tight:

```````````````````````````````` example
- a
.

    

  • a
    • 


````````````````````````````````


```````````````````````````````` example
- a
  - b
.

    

  • a

      

    • b
      • 

    

    • 


````````````````````````````````


This list is loose, because of the blank line between the
two block elements in the list item:

```````````````````````````````` example
1. ```
   foo
   ```

   bar
.

    

  1. 

    foo

    

    bar
    

    2. 


````````````````````````````````


Here the outer list is loose, the inner list tight:

```````````````````````````````` example
* foo
  * bar

  baz
.

    

  • 

    foo
    

      

    • bar
      • 

    

    baz
    

    • 


````````````````````````````````


```````````````````````````````` example
- a
  - b
  - c

- d
  - e
  - f
.

    

  • 

    a
    

      

    • b
      • 

    • c
      • 

    

    • 

  • 

    d
    

      

    • e
      • 

    • f
      • 

    

    • 


````````````````````````````````


# Inlines

Inlines are parsed sequentially from the beginning of the character
stream to the end (left to right, in left-to-right languages).
Thus, for example, in

```````````````````````````````` example
`hi`lo`
.

hilo`

````````````````````````````````

`hi` is parsed as code, leaving the backtick at the end as a literal
backtick.


## Backslash escapes

Any ASCII punctuation character may be backslash-escaped:

```````````````````````````````` example
\!\"\#\$\%\&\'\(\)\*\+\,\-\.\/\:\;\<\=\>\?\@\[\\\]\^\_\`\{\|\}\~
.

!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~

````````````````````````````````


Backslashes before other characters are treated as literal
backslashes:

```````````````````````````````` example
\→\A\a\ \3\φ\«
.

\→\A\a\ \3\φ\«

````````````````````````````````


Escaped characters are treated as regular characters and do
not have their usual Markdown meanings:

```````````````````````````````` example
\*not emphasized*
\
 not a tag
\[not a link](/foo)
\`not code`
1\. not a list
\* not a list
\# not a heading
\[foo]: /url "not a reference"
\ö not a character entity
.

*not emphasized*
<br/> not a tag
[not a link](/foo)
`not code`
1. not a list
* not a list
# not a heading
[foo]: /url "not a reference"
&ouml; not a character entity

````````````````````````````````


If a backslash is itself escaped, the following character is not:

```````````````````````````````` example
\\*emphasis*
.

\emphasis

````````````````````````````````


A backslash at the end of the line is a [hard line break]:

```````````````````````````````` example
foo\
bar
.

foo

bar

````````````````````````````````


Backslash escapes do not work in code blocks, code spans, autolinks, or
raw HTML:

```````````````````````````````` example
`` \[\` ``
.

\[\`

````````````````````````````````


```````````````````````````````` example
    \[\]
.

\[\]


````````````````````````````````


```````````````````````````````` example
~~~
\[\]
~~~
.

\[\]


````````````````````````````````


```````````````````````````````` example

.

http://example.com?find=\*

````````````````````````````````


```````````````````````````````` example

.

````````````````````````````````


But they work in all other contexts, including URLs and link titles,
link references, and [info strings] in [fenced code blocks]:

```````````````````````````````` example
[foo](/bar\* "ti\*tle")
.

foo

````````````````````````````````


```````````````````````````````` example
[foo]

[foo]: /bar\* "ti\*tle"
.

foo

````````````````````````````````


```````````````````````````````` example
``` foo\+bar
foo
```
.

foo


````````````````````````````````



## Entity and numeric character references

Valid HTML entity references and numeric character references
can be used in place of the corresponding Unicode character,
with the following exceptions:

- Entity and character references are not recognized in code
  blocks and code spans.

- Entity and character references cannot stand in place of
  special characters that define structural elements in
  CommonMark.  For example, although `*` can be used
  in place of a literal `*` character, `*` cannot replace
  `*` in emphasis delimiters, bullet list markers, or thematic
  breaks.

Conforming CommonMark parsers need not store information about
whether a particular character was represented in the source
using a Unicode character or an entity reference.

[Entity references](@) consist of `&` + any of the valid
HTML5 entity names + `;`. The
document 
is used as an authoritative source for the valid entity
references and their corresponding code points.

```````````````````````````````` example
  & © Æ Ď
¾ ℋ ⅆ
∲ ≧̸
.

  & © Æ Ď
¾ ℋ ⅆ
∲ ≧̸

````````````````````````````````


[Decimal numeric character
references](@)
consist of `&#` + a string of 1--7 arabic digits + `;`. A
numeric character reference is parsed as the corresponding
Unicode character. Invalid Unicode code points will be replaced by
the REPLACEMENT CHARACTER (`U+FFFD`).  For security reasons,
the code point `U+0000` will also be replaced by `U+FFFD`.

```````````````````````````````` example
# Ӓ Ϡ �
.

# Ӓ Ϡ �

````````````````````````````````


[Hexadecimal numeric character
references](@) consist of `&#` +
either `X` or `x` + a string of 1-6 hexadecimal digits + `;`.
They too are parsed as the corresponding Unicode character (this
time specified with a hexadecimal numeral instead of decimal).

```````````````````````````````` example
" ആ ಫ
.

" ആ ಫ

````````````````````````````````


Here are some nonentities:

```````````````````````````````` example
  &x; &#; &#x;
�
&#abcdef0;
&ThisIsNotDefined; &hi?;
.

&nbsp &x; &#; &#x;
&#987654321;
&#abcdef0;
&ThisIsNotDefined; &hi?;

````````````````````````````````


Although HTML5 does accept some entity references
without a trailing semicolon (such as `©`), these are not
recognized here, because it makes the grammar too ambiguous:

```````````````````````````````` example
©
.

&copy

````````````````````````````````


Strings that are not on the list of HTML5 named entities are not
recognized as entity references either:

```````````````````````````````` example
&MadeUpEntity;
.

&MadeUpEntity;

````````````````````````````````


Entity and numeric character references are recognized in any
context besides code spans or code blocks, including
URLs, [link titles], and [fenced code block][] [info strings]:

```````````````````````````````` example

.

````````````````````````````````


```````````````````````````````` example
[foo](/föö "föö")
.

foo

````````````````````````````````


```````````````````````````````` example
[foo]

[foo]: /föö "föö"
.

foo

````````````````````````````````


```````````````````````````````` example
``` föö
foo
```
.

foo


````````````````````````````````


Entity and numeric character references are treated as literal
text in code spans and code blocks:

```````````````````````````````` example
`föö`
.

f&ouml;&ouml;

````````````````````````````````


```````````````````````````````` example
    föfö
.

f&ouml;f&ouml;


````````````````````````````````


Entity and numeric character references cannot be used
in place of symbols indicating structure in CommonMark
documents.

```````````````````````````````` example
*foo*
*foo*
.

*foo*
foo

````````````````````````````````

```````````````````````````````` example
* foo

* foo
.

* foo


    

  • foo
    • 


````````````````````````````````

```````````````````````````````` example
foo

bar
.

foo

bar

````````````````````````````````

```````````````````````````````` example
	foo
.

→foo

````````````````````````````````


```````````````````````````````` example
[a](url "tit")
.

[a](url "tit")

````````````````````````````````


## Code spans

A [backtick string](@)
is a string of one or more backtick characters (`` ` ``) that is neither
preceded nor followed by a backtick.

A [code span](@) begins with a backtick string and ends with
a backtick string of equal length.  The contents of the code span are
the characters between the two backtick strings, normalized in the
following ways:

- First, [line endings] are converted to [spaces].
- If the resulting string both begins *and* ends with a [space]
  character, but does not consist entirely of [space]
  characters, a single [space] character is removed from the
  front and back.  This allows you to include code that begins
  or ends with backtick characters, which must be separated by
  whitespace from the opening or closing backtick strings.

This is a simple code span:

```````````````````````````````` example
`foo`
.

foo

````````````````````````````````


Here two backticks are used, because the code contains a backtick.
This example also illustrates stripping of a single leading and
trailing space:

```````````````````````````````` example
`` foo ` bar ``
.

foo ` bar

````````````````````````````````


This example shows the motivation for stripping leading and trailing
spaces:

```````````````````````````````` example
` `` `
.

``

````````````````````````````````

Note that only *one* space is stripped:

```````````````````````````````` example
`  ``  `
.

 `` 

````````````````````````````````

The stripping only happens if the space is on both
sides of the string:

```````````````````````````````` example
` a`
.

 a

````````````````````````````````

Only [spaces], and not [unicode whitespace] in general, are
stripped in this way:

```````````````````````````````` example
` b `
.

 b 

````````````````````````````````

No stripping occurs if the code span contains only spaces:

```````````````````````````````` example
` `
`  `
.

 
  

````````````````````````````````


[Line endings] are treated like spaces:

```````````````````````````````` example
``
foo
bar  
baz
``
.

foo bar   baz

````````````````````````````````

```````````````````````````````` example
``
foo 
``
.

foo 

````````````````````````````````


Interior spaces are not collapsed:

```````````````````````````````` example
`foo   bar 
baz`
.

foo   bar  baz

````````````````````````````````

Note that browsers will typically collapse consecutive spaces
when rendering `` elements, so it is recommended that
the following CSS be used:

    code{white-space: pre-wrap;}


Note that backslash escapes do not work in code spans. All backslashes
are treated literally:

```````````````````````````````` example
`foo\`bar`
.

foo\bar`

````````````````````````````````


Backslash escapes are never needed, because one can always choose a
string of *n* backtick characters as delimiters, where the code does
not contain any strings of exactly *n* backtick characters.

```````````````````````````````` example
``foo`bar``
.

foo`bar

````````````````````````````````

```````````````````````````````` example
` foo `` bar `
.

foo `` bar

````````````````````````````````


Code span backticks have higher precedence than any other inline
constructs except HTML tags and autolinks.  Thus, for example, this is
not parsed as emphasized text, since the second `*` is part of a code
span:

```````````````````````````````` example
*foo`*`
.

*foo*

````````````````````````````````


And this is not parsed as a link:

```````````````````````````````` example
[not a `link](/foo`)
.

[not a link](/foo)

````````````````````````````````


Code spans, HTML tags, and autolinks have the same precedence.
Thus, this is code:

```````````````````````````````` example
``
.

<a href="">`

````````````````````````````````


But this is an HTML tag:

```````````````````````````````` example
`
.

`

````````````````````````````````


And this is code:

```````````````````````````````` example
``
.

<http://foo.bar.baz>`

````````````````````````````````


But this is an autolink:

```````````````````````````````` example
`
.

http://foo.bar.`baz`

````````````````````````````````


When a backtick string is not closed by a matching backtick string,
we just have literal backticks:

```````````````````````````````` example
```foo``
.

```foo``

````````````````````````````````


```````````````````````````````` example
`foo
.

`foo

````````````````````````````````

The following case also illustrates the need for opening and
closing backtick strings to be equal in length:

```````````````````````````````` example
`foo``bar``
.

`foobar

````````````````````````````````


## Emphasis and strong emphasis

John Gruber's original [Markdown syntax
description](http://daringfireball.net/projects/markdown/syntax#em) says:

> Markdown treats asterisks (`*`) and underscores (`_`) as indicators of
> emphasis. Text wrapped with one `*` or `_` will be wrapped with an HTML
> `` tag; double `*`'s or `_`'s will be wrapped with an HTML ``
> tag.

This is enough for most users, but these rules leave much undecided,
especially when it comes to nested emphasis.  The original
`Markdown.pl` test suite makes it clear that triple `***` and
`___` delimiters can be used for strong emphasis, and most
implementations have also allowed the following patterns:

``` markdown
***strong emph***
***strong** in emph*
***emph* in strong**
**in strong *emph***
*in emph **strong***
```

The following patterns are less widely supported, but the intent
is clear and they are useful (especially in contexts like bibliography
entries):

``` markdown
*emph *with emph* in it*
**strong **with strong** in it**
```

Many implementations have also restricted intraword emphasis to
the `*` forms, to avoid unwanted emphasis in words containing
internal underscores.  (It is best practice to put these in code
spans, but users often do not.)

``` markdown
internal emphasis: foo*bar*baz
no emphasis: foo_bar_baz
```

The rules given below capture all of these patterns, while allowing
for efficient parsing strategies that do not backtrack.

First, some definitions.  A [delimiter run](@) is either
a sequence of one or more `*` characters that is not preceded or
followed by a non-backslash-escaped `*` character, or a sequence
of one or more `_` characters that is not preceded or followed by
a non-backslash-escaped `_` character.

A [left-flanking delimiter run](@) is
a [delimiter run] that is (1) not followed by [Unicode whitespace],
and either (2a) not followed by a [punctuation character], or
(2b) followed by a [punctuation character] and
preceded by [Unicode whitespace] or a [punctuation character].
For purposes of this definition, the beginning and the end of
the line count as Unicode whitespace.

A [right-flanking delimiter run](@) is
a [delimiter run] that is (1) not preceded by [Unicode whitespace],
and either (2a) not preceded by a [punctuation character], or
(2b) preceded by a [punctuation character] and
followed by [Unicode whitespace] or a [punctuation character].
For purposes of this definition, the beginning and the end of
the line count as Unicode whitespace.

Here are some examples of delimiter runs.

  - left-flanking but not right-flanking:

    ```
    ***abc
      _abc
    **"abc"
     _"abc"
    ```

  - right-flanking but not left-flanking:

    ```
     abc***
     abc_
    "abc"**
    "abc"_
    ```

  - Both left and right-flanking:

    ```
     abc***def
    "abc"_"def"
    ```

  - Neither left nor right-flanking:

    ```
    abc *** def
    a _ b
    ```

(The idea of distinguishing left-flanking and right-flanking
delimiter runs based on the character before and the character
after comes from Roopesh Chander's
[vfmd](http://www.vfmd.org/vfmd-spec/specification/#procedure-for-identifying-emphasis-tags).
vfmd uses the terminology "emphasis indicator string" instead of "delimiter
run," and its rules for distinguishing left- and right-flanking runs
are a bit more complex than the ones given here.)

The following rules define emphasis and strong emphasis:

1.  A single `*` character [can open emphasis](@)
    iff (if and only if) it is part of a [left-flanking delimiter run].

2.  A single `_` character [can open emphasis] iff
    it is part of a [left-flanking delimiter run]
    and either (a) not part of a [right-flanking delimiter run]
    or (b) part of a [right-flanking delimiter run]
    preceded by punctuation.

3.  A single `*` character [can close emphasis](@)
    iff it is part of a [right-flanking delimiter run].

4.  A single `_` character [can close emphasis] iff
    it is part of a [right-flanking delimiter run]
    and either (a) not part of a [left-flanking delimiter run]
    or (b) part of a [left-flanking delimiter run]
    followed by punctuation.

5.  A double `**` [can open strong emphasis](@)
    iff it is part of a [left-flanking delimiter run].

6.  A double `__` [can open strong emphasis] iff
    it is part of a [left-flanking delimiter run]
    and either (a) not part of a [right-flanking delimiter run]
    or (b) part of a [right-flanking delimiter run]
    preceded by punctuation.

7.  A double `**` [can close strong emphasis](@)
    iff it is part of a [right-flanking delimiter run].

8.  A double `__` [can close strong emphasis] iff
    it is part of a [right-flanking delimiter run]
    and either (a) not part of a [left-flanking delimiter run]
    or (b) part of a [left-flanking delimiter run]
    followed by punctuation.

9.  Emphasis begins with a delimiter that [can open emphasis] and ends
    with a delimiter that [can close emphasis], and that uses the same
    character (`_` or `*`) as the opening delimiter.  The
    opening and closing delimiters must belong to separate
    [delimiter runs].  If one of the delimiters can both
    open and close emphasis, then the sum of the lengths of the
    delimiter runs containing the opening and closing delimiters
    must not be a multiple of 3 unless both lengths are
    multiples of 3.

10. Strong emphasis begins with a delimiter that
    [can open strong emphasis] and ends with a delimiter that
    [can close strong emphasis], and that uses the same character
    (`_` or `*`) as the opening delimiter.  The
    opening and closing delimiters must belong to separate
    [delimiter runs].  If one of the delimiters can both open
    and close strong emphasis, then the sum of the lengths of
    the delimiter runs containing the opening and closing
    delimiters must not be a multiple of 3 unless both lengths
    are multiples of 3.

11. A literal `*` character cannot occur at the beginning or end of
    `*`-delimited emphasis or `**`-delimited strong emphasis, unless it
    is backslash-escaped.

12. A literal `_` character cannot occur at the beginning or end of
    `_`-delimited emphasis or `__`-delimited strong emphasis, unless it
    is backslash-escaped.

Where rules 1--12 above are compatible with multiple parsings,
the following principles resolve ambiguity:

13. The number of nestings should be minimized. Thus, for example,
    an interpretation `...` is always preferred to
    `...`.

14. An interpretation `...` is always
    preferred to `...`.

15. When two potential emphasis or strong emphasis spans overlap,
    so that the second begins before the first ends and ends after
    the first ends, the first takes precedence. Thus, for example,
    `*foo _bar* baz_` is parsed as `foo _bar baz_` rather
    than `*foo bar* baz`.

16. When there are two potential emphasis or strong emphasis spans
    with the same closing delimiter, the shorter one (the one that
    opens later) takes precedence. Thus, for example,
    `**foo **bar baz**` is parsed as `**foo bar baz`
    rather than `foo **bar baz`.

17. Inline code spans, links, images, and HTML tags group more tightly
    than emphasis.  So, when there is a choice between an interpretation
    that contains one of these elements and one that does not, the
    former always wins.  Thus, for example, `*[foo*](bar)` is
    parsed as `*foo*` rather than as
    `[foo](bar)`.

These rules can be illustrated through a series of examples.

Rule 1:

```````````````````````````````` example
*foo bar*
.

foo bar

````````````````````````````````


This is not emphasis, because the opening `*` is followed by
whitespace, and hence not part of a [left-flanking delimiter run]:

```````````````````````````````` example
a * foo bar*
.

a * foo bar*

````````````````````````````````


This is not emphasis, because the opening `*` is preceded
by an alphanumeric and followed by punctuation, and hence
not part of a [left-flanking delimiter run]:

```````````````````````````````` example
a*"foo"*
.

a*"foo"*

````````````````````````````````


Unicode nonbreaking spaces count as whitespace, too:

```````````````````````````````` example
* a *
.

* a *

````````````````````````````````


Intraword emphasis with `*` is permitted:

```````````````````````````````` example
foo*bar*
.

foobar

````````````````````````````````


```````````````````````````````` example
5*6*78
.

5678

````````````````````````````````


Rule 2:

```````````````````````````````` example
_foo bar_
.

foo bar

````````````````````````````````


This is not emphasis, because the opening `_` is followed by
whitespace:

```````````````````````````````` example
_ foo bar_
.

_ foo bar_

````````````````````````````````


This is not emphasis, because the opening `_` is preceded
by an alphanumeric and followed by punctuation:

```````````````````````````````` example
a_"foo"_
.

a_"foo"_

````````````````````````````````


Emphasis with `_` is not allowed inside words:

```````````````````````````````` example
foo_bar_
.

foo_bar_

````````````````````````````````


```````````````````````````````` example
5_6_78
.

5_6_78

````````````````````````````````


```````````````````````````````` example
пристаням_стремятся_
.

пристаням_стремятся_

````````````````````````````````


Here `_` does not generate emphasis, because the first delimiter run
is right-flanking and the second left-flanking:

```````````````````````````````` example
aa_"bb"_cc
.

aa_"bb"_cc

````````````````````````````````


This is emphasis, even though the opening delimiter is
both left- and right-flanking, because it is preceded by
punctuation:

```````````````````````````````` example
foo-_(bar)_
.

foo-(bar)

````````````````````````````````


Rule 3:

This is not emphasis, because the closing delimiter does
not match the opening delimiter:

```````````````````````````````` example
_foo*
.

_foo*

````````````````````````````````


This is not emphasis, because the closing `*` is preceded by
whitespace:

```````````````````````````````` example
*foo bar *
.

*foo bar *

````````````````````````````````


A newline also counts as whitespace:

```````````````````````````````` example
*foo bar
*
.

*foo bar
*

````````````````````````````````


This is not emphasis, because the second `*` is
preceded by punctuation and followed by an alphanumeric
(hence it is not part of a [right-flanking delimiter run]:

```````````````````````````````` example
*(*foo)
.

*(*foo)

````````````````````````````````


The point of this restriction is more easily appreciated
with this example:

```````````````````````````````` example
*(*foo*)*
.

(foo)

````````````````````````````````


Intraword emphasis with `*` is allowed:

```````````````````````````````` example
*foo*bar
.

foobar

````````````````````````````````



Rule 4:

This is not emphasis, because the closing `_` is preceded by
whitespace:

```````````````````````````````` example
_foo bar _
.

_foo bar _

````````````````````````````````


This is not emphasis, because the second `_` is
preceded by punctuation and followed by an alphanumeric:

```````````````````````````````` example
_(_foo)
.

_(_foo)

````````````````````````````````


This is emphasis within emphasis:

```````````````````````````````` example
_(_foo_)_
.

(foo)

````````````````````````````````


Intraword emphasis is disallowed for `_`:

```````````````````````````````` example
_foo_bar
.

_foo_bar

````````````````````````````````


```````````````````````````````` example
_пристаням_стремятся
.

_пристаням_стремятся

````````````````````````````````


```````````````````````````````` example
_foo_bar_baz_
.

foo_bar_baz

````````````````````````````````


This is emphasis, even though the closing delimiter is
both left- and right-flanking, because it is followed by
punctuation:

```````````````````````````````` example
_(bar)_.
.

(bar).

````````````````````````````````


Rule 5:

```````````````````````````````` example
**foo bar**
.

foo bar

````````````````````````````````


This is not strong emphasis, because the opening delimiter is
followed by whitespace:

```````````````````````````````` example
** foo bar**
.

** foo bar**

````````````````````````````````


This is not strong emphasis, because the opening `**` is preceded
by an alphanumeric and followed by punctuation, and hence
not part of a [left-flanking delimiter run]:

```````````````````````````````` example
a**"foo"**
.

a**"foo"**

````````````````````````````````


Intraword strong emphasis with `**` is permitted:

```````````````````````````````` example
foo**bar**
.

foobar

````````````````````````````````


Rule 6:

```````````````````````````````` example
__foo bar__
.

foo bar

````````````````````````````````


This is not strong emphasis, because the opening delimiter is
followed by whitespace:

```````````````````````````````` example
__ foo bar__
.

__ foo bar__

````````````````````````````````


A newline counts as whitespace:
```````````````````````````````` example
__
foo bar__
.

__
foo bar__

````````````````````````````````


This is not strong emphasis, because the opening `__` is preceded
by an alphanumeric and followed by punctuation:

```````````````````````````````` example
a__"foo"__
.

a__"foo"__

````````````````````````````````


Intraword strong emphasis is forbidden with `__`:

```````````````````````````````` example
foo__bar__
.

foo__bar__

````````````````````````````````


```````````````````````````````` example
5__6__78
.

5__6__78

````````````````````````````````


```````````````````````````````` example
пристаням__стремятся__
.

пристаням__стремятся__

````````````````````````````````


```````````````````````````````` example
__foo, __bar__, baz__
.

foo, bar, baz

````````````````````````````````


This is strong emphasis, even though the opening delimiter is
both left- and right-flanking, because it is preceded by
punctuation:

```````````````````````````````` example
foo-__(bar)__
.

foo-(bar)

````````````````````````````````



Rule 7:

This is not strong emphasis, because the closing delimiter is preceded
by whitespace:

```````````````````````````````` example
**foo bar **
.

**foo bar **

````````````````````````````````


(Nor can it be interpreted as an emphasized `*foo bar *`, because of
Rule 11.)

This is not strong emphasis, because the second `**` is
preceded by punctuation and followed by an alphanumeric:

```````````````````````````````` example
**(**foo)
.

**(**foo)

````````````````````````````````


The point of this restriction is more easily appreciated
with these examples:

```````````````````````````````` example
*(**foo**)*
.

(foo)

````````````````````````````````


```````````````````````````````` example
**Gomphocarpus (*Gomphocarpus physocarpus*, syn.
*Asclepias physocarpa*)**
.

Gomphocarpus (Gomphocarpus physocarpus, syn.
Asclepias physocarpa)

````````````````````````````````


```````````````````````````````` example
**foo "*bar*" foo**
.

foo "bar" foo

````````````````````````````````


Intraword emphasis:

```````````````````````````````` example
**foo**bar
.

foobar

````````````````````````````````


Rule 8:

This is not strong emphasis, because the closing delimiter is
preceded by whitespace:

```````````````````````````````` example
__foo bar __
.

__foo bar __

````````````````````````````````


This is not strong emphasis, because the second `__` is
preceded by punctuation and followed by an alphanumeric:

```````````````````````````````` example
__(__foo)
.

__(__foo)

````````````````````````````````


The point of this restriction is more easily appreciated
with this example:

```````````````````````````````` example
_(__foo__)_
.

(foo)

````````````````````````````````


Intraword strong emphasis is forbidden with `__`:

```````````````````````````````` example
__foo__bar
.

__foo__bar

````````````````````````````````


```````````````````````````````` example
__пристаням__стремятся
.

__пристаням__стремятся

````````````````````````````````


```````````````````````````````` example
__foo__bar__baz__
.

foo__bar__baz

````````````````````````````````


This is strong emphasis, even though the closing delimiter is
both left- and right-flanking, because it is followed by
punctuation:

```````````````````````````````` example
__(bar)__.
.

(bar).

````````````````````````````````


Rule 9:

Any nonempty sequence of inline elements can be the contents of an
emphasized span.

```````````````````````````````` example
*foo [bar](/url)*
.

foo bar

````````````````````````````````


```````````````````````````````` example
*foo
bar*
.

foo
bar

````````````````````````````````


In particular, emphasis and strong emphasis can be nested
inside emphasis:

```````````````````````````````` example
_foo __bar__ baz_
.

foo bar baz

````````````````````````````````


```````````````````````````````` example
_foo _bar_ baz_
.

foo bar baz

````````````````````````````````


```````````````````````````````` example
__foo_ bar_
.

foo bar

````````````````````````````````


```````````````````````````````` example
*foo *bar**
.

foo bar

````````````````````````````````


```````````````````````````````` example
*foo **bar** baz*
.

foo bar baz

````````````````````````````````

```````````````````````````````` example
*foo**bar**baz*
.

foobarbaz

````````````````````````````````

Note that in the preceding case, the interpretation

``` markdown

foobarbaz

```


is precluded by the condition that a delimiter that
can both open and close (like the `*` after `foo`)
cannot form emphasis if the sum of the lengths of
the delimiter runs containing the opening and
closing delimiters is a multiple of 3 unless
both lengths are multiples of 3.


For the same reason, we don't get two consecutive
emphasis sections in this example:

```````````````````````````````` example
*foo**bar*
.

foo**bar

````````````````````````````````


The same condition ensures that the following
cases are all strong emphasis nested inside
emphasis, even when the interior spaces are
omitted:


```````````````````````````````` example
***foo** bar*
.

foo bar

````````````````````````````````


```````````````````````````````` example
*foo **bar***
.

foo bar

````````````````````````````````


```````````````````````````````` example
*foo**bar***
.

foobar

````````````````````````````````


When the lengths of the interior closing and opening
delimiter runs are *both* multiples of 3, though,
they can match to create emphasis:

```````````````````````````````` example
foo***bar***baz
.

foobarbaz

````````````````````````````````

```````````````````````````````` example
foo******bar*********baz
.

foobar***baz

````````````````````````````````


Indefinite levels of nesting are possible:

```````````````````````````````` example
*foo **bar *baz* bim** bop*
.

foo bar baz bim bop

````````````````````````````````


```````````````````````````````` example
*foo [*bar*](/url)*
.

foo bar

````````````````````````````````


There can be no empty emphasis or strong emphasis:

```````````````````````````````` example
** is not an empty emphasis
.

** is not an empty emphasis

````````````````````````````````


```````````````````````````````` example
**** is not an empty strong emphasis
.

**** is not an empty strong emphasis

````````````````````````````````



Rule 10:

Any nonempty sequence of inline elements can be the contents of an
strongly emphasized span.

```````````````````````````````` example
**foo [bar](/url)**
.

foo bar

````````````````````````````````


```````````````````````````````` example
**foo
bar**
.

foo
bar

````````````````````````````````


In particular, emphasis and strong emphasis can be nested
inside strong emphasis:

```````````````````````````````` example
__foo _bar_ baz__
.

foo bar baz

````````````````````````````````


```````````````````````````````` example
__foo __bar__ baz__
.

foo bar baz

````````````````````````````````


```````````````````````````````` example
____foo__ bar__
.

foo bar

````````````````````````````````


```````````````````````````````` example
**foo **bar****
.

foo bar

````````````````````````````````


```````````````````````````````` example
**foo *bar* baz**
.

foo bar baz

````````````````````````````````


```````````````````````````````` example
**foo*bar*baz**
.

foobarbaz

````````````````````````````````


```````````````````````````````` example
***foo* bar**
.

foo bar

````````````````````````````````


```````````````````````````````` example
**foo *bar***
.

foo bar

````````````````````````````````


Indefinite levels of nesting are possible:

```````````````````````````````` example
**foo *bar **baz**
bim* bop**
.

foo bar baz
bim bop

````````````````````````````````


```````````````````````````````` example
**foo [*bar*](/url)**
.

foo bar

````````````````````````````````


There can be no empty emphasis or strong emphasis:

```````````````````````````````` example
__ is not an empty emphasis
.

__ is not an empty emphasis

````````````````````````````````


```````````````````````````````` example
____ is not an empty strong emphasis
.

____ is not an empty strong emphasis

````````````````````````````````



Rule 11:

```````````````````````````````` example
foo ***
.

foo ***

````````````````````````````````


```````````````````````````````` example
foo *\**
.

foo *

````````````````````````````````


```````````````````````````````` example
foo *_*
.

foo _

````````````````````````````````


```````````````````````````````` example
foo *****
.

foo *****

````````````````````````````````


```````````````````````````````` example
foo **\***
.

foo *

````````````````````````````````


```````````````````````````````` example
foo **_**
.

foo _

````````````````````````````````


Note that when delimiters do not match evenly, Rule 11 determines
that the excess literal `*` characters will appear outside of the
emphasis, rather than inside it:

```````````````````````````````` example
**foo*
.

*foo

````````````````````````````````


```````````````````````````````` example
*foo**
.

foo*

````````````````````````````````


```````````````````````````````` example
***foo**
.

*foo

````````````````````````````````


```````````````````````````````` example
****foo*
.

***foo

````````````````````````````````


```````````````````````````````` example
**foo***
.

foo*

````````````````````````````````


```````````````````````````````` example
*foo****
.

foo***

````````````````````````````````



Rule 12:

```````````````````````````````` example
foo ___
.

foo ___

````````````````````````````````


```````````````````````````````` example
foo _\__
.

foo _

````````````````````````````````


```````````````````````````````` example
foo _*_
.

foo *

````````````````````````````````


```````````````````````````````` example
foo _____
.

foo _____

````````````````````````````````


```````````````````````````````` example
foo __\___
.

foo _

````````````````````````````````


```````````````````````````````` example
foo __*__
.

foo *

````````````````````````````````


```````````````````````````````` example
__foo_
.

_foo

````````````````````````````````


Note that when delimiters do not match evenly, Rule 12 determines
that the excess literal `_` characters will appear outside of the
emphasis, rather than inside it:

```````````````````````````````` example
_foo__
.

foo_

````````````````````````````````


```````````````````````````````` example
___foo__
.

_foo

````````````````````````````````


```````````````````````````````` example
____foo_
.

___foo

````````````````````````````````


```````````````````````````````` example
__foo___
.

foo_

````````````````````````````````


```````````````````````````````` example
_foo____
.

foo___

````````````````````````````````


Rule 13 implies that if you want emphasis nested directly inside
emphasis, you must use different delimiters:

```````````````````````````````` example
**foo**
.

foo

````````````````````````````````


```````````````````````````````` example
*_foo_*
.

foo

````````````````````````````````


```````````````````````````````` example
__foo__
.

foo

````````````````````````````````


```````````````````````````````` example
_*foo*_
.

foo

````````````````````````````````


However, strong emphasis within strong emphasis is possible without
switching delimiters:

```````````````````````````````` example
****foo****
.

foo

````````````````````````````````


```````````````````````````````` example
____foo____
.

foo

````````````````````````````````



Rule 13 can be applied to arbitrarily long sequences of
delimiters:

```````````````````````````````` example
******foo******
.

foo

````````````````````````````````


Rule 14:

```````````````````````````````` example
***foo***
.

foo

````````````````````````````````


```````````````````````````````` example
_____foo_____
.

foo

````````````````````````````````


Rule 15:

```````````````````````````````` example
*foo _bar* baz_
.

foo _bar baz_

````````````````````````````````


```````````````````````````````` example
*foo __bar *baz bim__ bam*
.

foo bar *baz bim bam

````````````````````````````````


Rule 16:

```````````````````````````````` example
**foo **bar baz**
.

**foo bar baz

````````````````````````````````


```````````````````````````````` example
*foo *bar baz*
.

*foo bar baz

````````````````````````````````


Rule 17:

```````````````````````````````` example
*[bar*](/url)
.

*bar*

````````````````````````````````


```````````````````````````````` example
_foo [bar_](/url)
.

_foo bar_

````````````````````````````````


```````````````````````````````` example
*
.

*

````````````````````````````````


```````````````````````````````` example
**
.

**

````````````````````````````````


```````````````````````````````` example
__
.

__

````````````````````````````````


```````````````````````````````` example
*a `*`*
.

a *

````````````````````````````````


```````````````````````````````` example
_a `_`_
.

a _

````````````````````````````````


```````````````````````````````` example
**a
.

**ahttp://foo.bar/?q=**

````````````````````````````````


```````````````````````````````` example
__a
.

__ahttp://foo.bar/?q=__

````````````````````````````````



## Links

A link contains [link text] (the visible text), a [link destination]
(the URI that is the link destination), and optionally a [link title].
There are two basic kinds of links in Markdown.  In [inline links] the
destination and title are given immediately after the link text.  In
[reference links] the destination and title are defined elsewhere in
the document.

A [link text](@) consists of a sequence of zero or more
inline elements enclosed by square brackets (`[` and `]`).  The
following rules apply:

- Links may not contain other links, at any level of nesting. If
  multiple otherwise valid link definitions appear nested inside each
  other, the inner-most definition is used.

- Brackets are allowed in the [link text] only if (a) they
  are backslash-escaped or (b) they appear as a matched pair of brackets,
  with an open bracket `[`, a sequence of zero or more inlines, and
  a close bracket `]`.

- Backtick [code spans], [autolinks], and raw [HTML tags] bind more tightly
  than the brackets in link text.  Thus, for example,
  `` [foo`]` `` could not be a link text, since the second `]`
  is part of a code span.

- The brackets in link text bind more tightly than markers for
  [emphasis and strong emphasis]. Thus, for example, `*[foo*](url)` is a link.

A [link destination](@) consists of either

- a sequence of zero or more characters between an opening `<` and a
  closing `>` that contains no line breaks or unescaped
  `<` or `>` characters, or

- a nonempty sequence of characters that does not start with
  `<`, does not include ASCII space or control characters, and
  includes parentheses only if (a) they are backslash-escaped or
  (b) they are part of a balanced pair of unescaped parentheses.
  (Implementations may impose limits on parentheses nesting to
  avoid performance issues, but at least three levels of nesting
  should be supported.)

A [link title](@)  consists of either

- a sequence of zero or more characters between straight double-quote
  characters (`"`), including a `"` character only if it is
  backslash-escaped, or

- a sequence of zero or more characters between straight single-quote
  characters (`'`), including a `'` character only if it is
  backslash-escaped, or

- a sequence of zero or more characters between matching parentheses
  (`(...)`), including a `(` or `)` character only if it is
  backslash-escaped.

Although [link titles] may span multiple lines, they may not contain
a [blank line].

An [inline link](@) consists of a [link text] followed immediately
by a left parenthesis `(`, optional [whitespace], an optional
[link destination], an optional [link title] separated from the link
destination by [whitespace], optional [whitespace], and a right
parenthesis `)`. The link's text consists of the inlines contained
in the [link text] (excluding the enclosing square brackets).
The link's URI consists of the link destination, excluding enclosing
`<...>` if present, with backslash-escapes in effect as described
above.  The link's title consists of the link title, excluding its
enclosing delimiters, with backslash-escapes in effect as described
above.

Here is a simple inline link:

```````````````````````````````` example
[link](/uri "title")
.

link

````````````````````````````````


The title may be omitted:

```````````````````````````````` example
[link](/uri)
.

link

````````````````````````````````


Both the title and the destination may be omitted:

```````````````````````````````` example
[link]()
.

link

````````````````````````````````


```````````````````````````````` example
[link](<>)
.

link

````````````````````````````````

The destination can only contain spaces if it is
enclosed in pointy brackets:

```````````````````````````````` example
[link](/my uri)
.

[link](/my uri)

````````````````````````````````

```````````````````````````````` example
[link]()
.

link

````````````````````````````````

The destination cannot contain line breaks,
even if enclosed in pointy brackets:

```````````````````````````````` example
[link](foo
bar)
.

[link](foo
bar)

````````````````````````````````

```````````````````````````````` example
[link]()
.

[link]()

````````````````````````````````

The destination can contain `)` if it is enclosed
in pointy brackets:

```````````````````````````````` example
[a]()
.

a

````````````````````````````````

Pointy brackets that enclose links must be unescaped:

```````````````````````````````` example
[link]()
.

[link](<foo>)

````````````````````````````````

These are not links, because the opening pointy bracket
is not matched properly:

```````````````````````````````` example
[a](
[a](c)
.

[a](<b)c
[a](<b)c>
[a](c)

````````````````````````````````

Parentheses inside the link destination may be escaped:

```````````````````````````````` example
[link](\(foo\))
.

link

````````````````````````````````

Any number of parentheses are allowed without escaping, as long as they are
balanced:

```````````````````````````````` example
[link](foo(and(bar)))
.

link

````````````````````````````````

However, if you have unbalanced parentheses, you need to escape or use the
`<...>` form:

```````````````````````````````` example
[link](foo\(and\(bar\))
.

link

````````````````````````````````


```````````````````````````````` example
[link]()
.

link

````````````````````````````````


Parentheses and other symbols can also be escaped, as usual
in Markdown:

```````````````````````````````` example
[link](foo\)\:)
.

link

````````````````````````````````


A link can contain fragment identifiers and queries:

```````````````````````````````` example
[link](#fragment)

[link](http://example.com#fragment)

[link](http://example.com?foo=3#frag)
.

link


link


link

````````````````````````````````


Note that a backslash before a non-escapable character is
just a backslash:

```````````````````````````````` example
[link](foo\bar)
.

link

````````````````````````````````


URL-escaping should be left alone inside the destination, as all
URL-escaped characters are also valid URL characters. Entity and
numerical character references in the destination will be parsed
into the corresponding Unicode code points, as usual.  These may
be optionally URL-escaped when written as HTML, but this spec
does not enforce any particular policy for rendering URLs in
HTML or other formats.  Renderers may make different decisions
about how to escape or normalize URLs in the output.

```````````````````````````````` example
[link](foo%20bä)
.

link

````````````````````````````````


Note that, because titles can often be parsed as destinations,
if you try to omit the destination and keep the title, you'll
get unexpected results:

```````````````````````````````` example
[link]("title")
.

link

````````````````````````````````


Titles may be in single quotes, double quotes, or parentheses:

```````````````````````````````` example
[link](/url "title")
[link](/url 'title')
[link](/url (title))
.

link
link
link

````````````````````````````````


Backslash escapes and entity and numeric character references
may be used in titles:

```````````````````````````````` example
[link](/url "title \""")
.

link

````````````````````````````````


Titles must be separated from the link using a [whitespace].
Other [Unicode whitespace] like non-breaking space doesn't work.

```````````````````````````````` example
[link](/url "title")
.

link

````````````````````````````````


Nested balanced quotes are not allowed without escaping:

```````````````````````````````` example
[link](/url "title "and" title")
.

[link](/url "title "and" title")

````````````````````````````````


But it is easy to work around this by using a different quote type:

```````````````````````````````` example
[link](/url 'title "and" title')
.

link

````````````````````````````````


(Note:  `Markdown.pl` did allow double quotes inside a double-quoted
title, and its test suite included a test demonstrating this.
But it is hard to see a good rationale for the extra complexity this
brings, since there are already many ways---backslash escaping,
entity and numeric character references, or using a different
quote type for the enclosing title---to write titles containing
double quotes.  `Markdown.pl`'s handling of titles has a number
of other strange features.  For example, it allows single-quoted
titles in inline links, but not reference links.  And, in
reference links but not inline links, it allows a title to begin
with `"` and end with `)`.  `Markdown.pl` 1.0.1 even allows
titles with no closing quotation mark, though 1.0.2b8 does not.
It seems preferable to adopt a simple, rational rule that works
the same way in inline links and link reference definitions.)

[Whitespace] is allowed around the destination and title:

```````````````````````````````` example
[link](   /uri
  "title"  )
.

link

````````````````````````````````


But it is not allowed between the link text and the
following parenthesis:

```````````````````````````````` example
[link] (/uri)
.

[link] (/uri)

````````````````````````````````


The link text may contain balanced brackets, but not unbalanced ones,
unless they are escaped:

```````````````````````````````` example
[link [foo [bar]]](/uri)
.

link [foo [bar]]

````````````````````````````````


```````````````````````````````` example
[link] bar](/uri)
.

[link] bar](/uri)

````````````````````````````````


```````````````````````````````` example
[link [bar](/uri)
.

[link bar

````````````````````````````````


```````````````````````````````` example
[link \[bar](/uri)
.

link [bar

````````````````````````````````


The link text may contain inline content:

```````````````````````````````` example
[link *foo **bar** `#`*](/uri)
.

link foo bar #

````````````````````````````````


```````````````````````````````` example
[![moon](moon.jpg)](/uri)
.

moon

````````````````````````````````


However, links may not contain other links, at any level of nesting.

```````````````````````````````` example
[foo [bar](/uri)](/uri)
.

[foo bar](/uri)

````````````````````````````````


```````````````````````````````` example
[foo *[bar [baz](/uri)](/uri)*](/uri)
.

[foo [bar baz](/uri)](/uri)

````````````````````````````````


```````````````````````````````` example
![[[foo](uri1)](uri2)](uri3)
.

[foo](uri2)

````````````````````````````````


These cases illustrate the precedence of link text grouping over
emphasis grouping:

```````````````````````````````` example
*[foo*](/uri)
.

*foo*

````````````````````````````````


```````````````````````````````` example
[foo *bar](baz*)
.

foo *bar

````````````````````````````````


Note that brackets that *aren't* part of links do not take
precedence:

```````````````````````````````` example
*foo [bar* baz]
.

foo [bar baz]

````````````````````````````````


These cases illustrate the precedence of HTML tags, code spans,
and autolinks over link grouping:

```````````````````````````````` example
[foo 
.

[foo 

````````````````````````````````


```````````````````````````````` example
[foo`](/uri)`
.

[foo](/uri)

````````````````````````````````


```````````````````````````````` example
[foo
.

[foohttp://example.com/?search=](uri)

````````````````````````````````


There are three kinds of [reference link](@)s:
[full](#full-reference-link), [collapsed](#collapsed-reference-link),
and [shortcut](#shortcut-reference-link).

A [full reference link](@)
consists of a [link text] immediately followed by a [link label]
that [matches] a [link reference definition] elsewhere in the document.

A [link label](@)  begins with a left bracket (`[`) and ends
with the first right bracket (`]`) that is not backslash-escaped.
Between these brackets there must be at least one [non-whitespace character].
Unescaped square bracket characters are not allowed inside the
opening and closing square brackets of [link labels].  A link
label can have at most 999 characters inside the square
brackets.

One label [matches](@)
another just in case their normalized forms are equal.  To normalize a
label, strip off the opening and closing brackets,
perform the *Unicode case fold*, strip leading and trailing
[whitespace] and collapse consecutive internal
[whitespace] to a single space.  If there are multiple
matching reference link definitions, the one that comes first in the
document is used.  (It is desirable in such cases to emit a warning.)

The contents of the first link label are parsed as inlines, which are
used as the link's text.  The link's URI and title are provided by the
matching [link reference definition].

Here is a simple example:

```````````````````````````````` example
[foo][bar]

[bar]: /url "title"
.

foo

````````````````````````````````


The rules for the [link text] are the same as with
[inline links].  Thus:

The link text may contain balanced brackets, but not unbalanced ones,
unless they are escaped:

```````````````````````````````` example
[link [foo [bar]]][ref]

[ref]: /uri
.

link [foo [bar]]

````````````````````````````````


```````````````````````````````` example
[link \[bar][ref]

[ref]: /uri
.

link [bar

````````````````````````````````


The link text may contain inline content:

```````````````````````````````` example
[link *foo **bar** `#`*][ref]

[ref]: /uri
.

link foo bar #

````````````````````````````````


```````````````````````````````` example
[![moon](moon.jpg)][ref]

[ref]: /uri
.

moon

````````````````````````````````


However, links may not contain other links, at any level of nesting.

```````````````````````````````` example
[foo [bar](/uri)][ref]

[ref]: /uri
.

[foo bar]ref

````````````````````````````````


```````````````````````````````` example
[foo *bar [baz][ref]*][ref]

[ref]: /uri
.

[foo bar baz]ref

````````````````````````````````


(In the examples above, we have two [shortcut reference links]
instead of one [full reference link].)

The following cases illustrate the precedence of link text grouping over
emphasis grouping:

```````````````````````````````` example
*[foo*][ref]

[ref]: /uri
.

*foo*

````````````````````````````````


```````````````````````````````` example
[foo *bar][ref]

[ref]: /uri
.

foo *bar

````````````````````````````````


These cases illustrate the precedence of HTML tags, code spans,
and autolinks over link grouping:

```````````````````````````````` example
[foo 

[ref]: /uri
.

[foo 

````````````````````````````````


```````````````````````````````` example
[foo`][ref]`

[ref]: /uri
.

[foo][ref]

````````````````````````````````


```````````````````````````````` example
[foo

[ref]: /uri
.

[foohttp://example.com/?search=][ref]

````````````````````````````````


Matching is case-insensitive:

```````````````````````````````` example
[foo][BaR]

[bar]: /url "title"
.

foo

````````````````````````````````


Unicode case fold is used:

```````````````````````````````` example
[Толпой][Толпой] is a Russian word.

[ТОЛПОЙ]: /url
.

Толпой is a Russian word.

````````````````````````````````


Consecutive internal [whitespace] is treated as one space for
purposes of determining matching:

```````````````````````````````` example
[Foo
  bar]: /url

[Baz][Foo bar]
.

Baz

````````````````````````````````


No [whitespace] is allowed between the [link text] and the
[link label]:

```````````````````````````````` example
[foo] [bar]

[bar]: /url "title"
.

[foo] bar

````````````````````````````````


```````````````````````````````` example
[foo]
[bar]

[bar]: /url "title"
.

[foo]
bar

````````````````````````````````


This is a departure from John Gruber's original Markdown syntax
description, which explicitly allows whitespace between the link
text and the link label.  It brings reference links in line with
[inline links], which (according to both original Markdown and
this spec) cannot have whitespace after the link text.  More
importantly, it prevents inadvertent capture of consecutive
[shortcut reference links]. If whitespace is allowed between the
link text and the link label, then in the following we will have
a single reference link, not two shortcut reference links, as
intended:

``` markdown
[foo]
[bar]

[foo]: /url1
[bar]: /url2
```

(Note that [shortcut reference links] were introduced by Gruber
himself in a beta version of `Markdown.pl`, but never included
in the official syntax description.  Without shortcut reference
links, it is harmless to allow space between the link text and
link label; but once shortcut references are introduced, it is
too dangerous to allow this, as it frequently leads to
unintended results.)

When there are multiple matching [link reference definitions],
the first is used:

```````````````````````````````` example
[foo]: /url1

[foo]: /url2

[bar][foo]
.

bar

````````````````````````````````


Note that matching is performed on normalized strings, not parsed
inline content.  So the following does not match, even though the
labels define equivalent inline content:

```````````````````````````````` example
[bar][foo\!]

[foo!]: /url
.

[bar][foo!]

````````````````````````````````


[Link labels] cannot contain brackets, unless they are
backslash-escaped:

```````````````````````````````` example
[foo][ref[]

[ref[]: /uri
.

[foo][ref[]


[ref[]: /uri

````````````````````````````````


```````````````````````````````` example
[foo][ref[bar]]

[ref[bar]]: /uri
.

[foo][ref[bar]]


[ref[bar]]: /uri

````````````````````````````````


```````````````````````````````` example
[[[foo]]]

[[[foo]]]: /url
.

[[[foo]]]


[[[foo]]]: /url

````````````````````````````````


```````````````````````````````` example
[foo][ref\[]

[ref\[]: /uri
.

foo

````````````````````````````````


Note that in this example `]` is not backslash-escaped:

```````````````````````````````` example
[bar\\]: /uri

[bar\\]
.

bar\

````````````````````````````````


A [link label] must contain at least one [non-whitespace character]:

```````````````````````````````` example
[]

[]: /uri
.

[]


[]: /uri

````````````````````````````````


```````````````````````````````` example
[
 ]

[
 ]: /uri
.

[
]


[
]: /uri

````````````````````````````````


A [collapsed reference link](@)
consists of a [link label] that [matches] a
[link reference definition] elsewhere in the
document, followed by the string `[]`.
The contents of the first link label are parsed as inlines,
which are used as the link's text.  The link's URI and title are
provided by the matching reference link definition.  Thus,
`[foo][]` is equivalent to `[foo][foo]`.

```````````````````````````````` example
[foo][]

[foo]: /url "title"
.

foo

````````````````````````````````


```````````````````````````````` example
[*foo* bar][]

[*foo* bar]: /url "title"
.

foo bar

````````````````````````````````


The link labels are case-insensitive:

```````````````````````````````` example
[Foo][]

[foo]: /url "title"
.

Foo

````````````````````````````````



As with full reference links, [whitespace] is not
allowed between the two sets of brackets:

```````````````````````````````` example
[foo] 
[]

[foo]: /url "title"
.

foo
[]

````````````````````````````````


A [shortcut reference link](@)
consists of a [link label] that [matches] a
[link reference definition] elsewhere in the
document and is not followed by `[]` or a link label.
The contents of the first link label are parsed as inlines,
which are used as the link's text.  The link's URI and title
are provided by the matching link reference definition.
Thus, `[foo]` is equivalent to `[foo][]`.

```````````````````````````````` example
[foo]

[foo]: /url "title"
.

foo

````````````````````````````````


```````````````````````````````` example
[*foo* bar]

[*foo* bar]: /url "title"
.

foo bar

````````````````````````````````


```````````````````````````````` example
[[*foo* bar]]

[*foo* bar]: /url "title"
.

[foo bar]

````````````````````````````````


```````````````````````````````` example
[[bar [foo]

[foo]: /url
.

[[bar foo

````````````````````````````````


The link labels are case-insensitive:

```````````````````````````````` example
[Foo]

[foo]: /url "title"
.

Foo

````````````````````````````````


A space after the link text should be preserved:

```````````````````````````````` example
[foo] bar

[foo]: /url
.

foo bar

````````````````````````````````


If you just want bracketed text, you can backslash-escape the
opening bracket to avoid links:

```````````````````````````````` example
\[foo]

[foo]: /url "title"
.

[foo]

````````````````````````````````


Note that this is a link, because a link label ends with the first
following closing bracket:

```````````````````````````````` example
[foo*]: /url

*[foo*]
.

*foo*

````````````````````````````````


Full and compact references take precedence over shortcut
references:

```````````````````````````````` example
[foo][bar]

[foo]: /url1
[bar]: /url2
.

foo

````````````````````````````````

```````````````````````````````` example
[foo][]

[foo]: /url1
.

foo

````````````````````````````````

Inline links also take precedence:

```````````````````````````````` example
[foo]()

[foo]: /url1
.

foo

````````````````````````````````

```````````````````````````````` example
[foo](not a link)

[foo]: /url1
.

foo(not a link)

````````````````````````````````

In the following case `[bar][baz]` is parsed as a reference,
`[foo]` as normal text:

```````````````````````````````` example
[foo][bar][baz]

[baz]: /url
.

[foo]bar

````````````````````````````````


Here, though, `[foo][bar]` is parsed as a reference, since
`[bar]` is defined:

```````````````````````````````` example
[foo][bar][baz]

[baz]: /url1
[bar]: /url2
.

foobaz

````````````````````````````````


Here `[foo]` is not parsed as a shortcut reference, because it
is followed by a link label (even though `[bar]` is not defined):

```````````````````````````````` example
[foo][bar][baz]

[baz]: /url1
[foo]: /url2
.

[foo]bar

````````````````````````````````



## Images

Syntax for images is like the syntax for links, with one
difference. Instead of [link text], we have an
[image description](@).  The rules for this are the
same as for [link text], except that (a) an
image description starts with `![` rather than `[`, and
(b) an image description may contain links.
An image description has inline elements
as its contents.  When an image is rendered to HTML,
this is standardly used as the image's `alt` attribute.

```````````````````````````````` example
![foo](/url "title")
.

foo

````````````````````````````````


```````````````````````````````` example
![foo *bar*]

[foo *bar*]: train.jpg "train & tracks"
.

foo bar

````````````````````````````````


```````````````````````````````` example
![foo ![bar](/url)](/url2)
.

foo bar

````````````````````````````````


```````````````````````````````` example
![foo [bar](/url)](/url2)
.

foo bar

````````````````````````````````


Though this spec is concerned with parsing, not rendering, it is
recommended that in rendering to HTML, only the plain string content
of the [image description] be used.  Note that in
the above example, the alt attribute's value is `foo bar`, not `foo
[bar](/url)` or `foo bar`.  Only the plain string
content is rendered, without formatting.

```````````````````````````````` example
![foo *bar*][]

[foo *bar*]: train.jpg "train & tracks"
.

foo bar

````````````````````````````````


```````````````````````````````` example
![foo *bar*][foobar]

[FOOBAR]: train.jpg "train & tracks"
.

foo bar

````````````````````````````````


```````````````````````````````` example
![foo](train.jpg)
.

foo

````````````````````````````````


```````````````````````````````` example
My ![foo bar](/path/to/train.jpg  "title"   )
.

My foo bar

````````````````````````````````


```````````````````````````````` example
![foo]()
.

foo

````````````````````````````````


```````````````````````````````` example
![](/url)
.


````````````````````````````````


Reference-style:

```````````````````````````````` example
![foo][bar]

[bar]: /url
.

foo

````````````````````````````````


```````````````````````````````` example
![foo][bar]

[BAR]: /url
.

foo

````````````````````````````````


Collapsed:

```````````````````````````````` example
![foo][]

[foo]: /url "title"
.

foo

````````````````````````````````


```````````````````````````````` example
![*foo* bar][]

[*foo* bar]: /url "title"
.

foo bar

````````````````````````````````


The labels are case-insensitive:

```````````````````````````````` example
![Foo][]

[foo]: /url "title"
.

Foo

````````````````````````````````


As with reference links, [whitespace] is not allowed
between the two sets of brackets:

```````````````````````````````` example
![foo] 
[]

[foo]: /url "title"
.

foo
[]

````````````````````````````````


Shortcut:

```````````````````````````````` example
![foo]

[foo]: /url "title"
.

foo

````````````````````````````````


```````````````````````````````` example
![*foo* bar]

[*foo* bar]: /url "title"
.

foo bar

````````````````````````````````


Note that link labels cannot contain unescaped brackets:

```````````````````````````````` example
![[foo]]

[[foo]]: /url "title"
.

![[foo]]


[[foo]]: /url "title"

````````````````````````````````


The link labels are case-insensitive:

```````````````````````````````` example
![Foo]

[foo]: /url "title"
.

Foo

````````````````````````````````


If you just want a literal `!` followed by bracketed text, you can
backslash-escape the opening `[`:

```````````````````````````````` example
!\[foo]

[foo]: /url "title"
.

![foo]

````````````````````````````````


If you want a link after a literal `!`, backslash-escape the
`!`:

```````````````````````````````` example
\![foo]

[foo]: /url "title"
.

!foo

````````````````````````````````


## Autolinks

[Autolink](@)s are absolute URIs and email addresses inside
`<` and `>`. They are parsed as links, with the URL or email address
as the link label.

A [URI autolink](@) consists of `<`, followed by an
[absolute URI] followed by `>`.  It is parsed as
a link to the URI, with the URI as the link's label.

An [absolute URI](@),
for these purposes, consists of a [scheme] followed by a colon (`:`)
followed by zero or more characters other than ASCII
[whitespace] and control characters, `<`, and `>`.  If
the URI includes these characters, they must be percent-encoded
(e.g. `%20` for a space).

For purposes of this spec, a [scheme](@) is any sequence
of 2--32 characters beginning with an ASCII letter and followed
by any combination of ASCII letters, digits, or the symbols plus
("+"), period ("."), or hyphen ("-").

Here are some valid autolinks:

```````````````````````````````` example

.

http://foo.bar.baz

````````````````````````````````


```````````````````````````````` example

.

http://foo.bar.baz/test?q=hello&id=22&boolean

````````````````````````````````


```````````````````````````````` example

.

irc://foo.bar:2233/baz

````````````````````````````````


Uppercase is also fine:

```````````````````````````````` example

.

MAILTO:FOO@BAR.BAZ

````````````````````````````````


Note that many strings that count as [absolute URIs] for
purposes of this spec are not valid URIs, because their
schemes are not registered or because of other problems
with their syntax:

```````````````````````````````` example

.

a+b+c:d

````````````````````````````````


```````````````````````````````` example

.

made-up-scheme://foo,bar

````````````````````````````````


```````````````````````````````` example

.

http://../

````````````````````````````````


```````````````````````````````` example

.

localhost:5001/foo

````````````````````````````````


Spaces are not allowed in autolinks:

```````````````````````````````` example

.

<http://foo.bar/baz bim>

````````````````````````````````


Backslash-escapes do not work inside autolinks:

```````````````````````````````` example

.

http://example.com/\[\

````````````````````````````````


An [email autolink](@)
consists of `<`, followed by an [email address],
followed by `>`.  The link's label is the email address,
and the URL is `mailto:` followed by the email address.

An [email address](@),
for these purposes, is anything that matches
the [non-normative regex from the HTML5
spec](https://html.spec.whatwg.org/multipage/forms.html#e-mail-state-(type=email)):

    /^[a-zA-Z0-9.!#$%&'*+/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?
    (?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$/

Examples of email autolinks:

```````````````````````````````` example

.

foo@bar.example.com

````````````````````````````````


```````````````````````````````` example

.

foo+special@Bar.baz-bar0.com

````````````````````````````````


Backslash-escapes do not work inside email autolinks:

```````````````````````````````` example

.

<foo+@bar.example.com>

````````````````````````````````


These are not autolinks:

```````````````````````````````` example
<>
.

<>

````````````````````````````````


```````````````````````````````` example
< http://foo.bar >
.

< http://foo.bar >

````````````````````````````````


```````````````````````````````` example

.

<m:abc>

````````````````````````````````


```````````````````````````````` example

.

<foo.bar.baz>

````````````````````````````````


```````````````````````````````` example
http://example.com
.

http://example.com

````````````````````````````````


```````````````````````````````` example
foo@bar.example.com
.

foo@bar.example.com

````````````````````````````````


## Raw HTML

Text between `<` and `>` that looks like an HTML tag is parsed as a
raw HTML tag and will be rendered in HTML without escaping.
Tag and attribute names are not limited to current HTML tags,
so custom tags (and even, say, DocBook tags) may be used.

Here is the grammar for tags:

A [tag name](@) consists of an ASCII letter
followed by zero or more ASCII letters, digits, or
hyphens (`-`).

An [attribute](@) consists of [whitespace],
an [attribute name], and an optional
[attribute value specification].

An [attribute name](@)
consists of an ASCII letter, `_`, or `:`, followed by zero or more ASCII
letters, digits, `_`, `.`, `:`, or `-`.  (Note:  This is the XML
specification restricted to ASCII.  HTML5 is laxer.)

An [attribute value specification](@)
consists of optional [whitespace],
a `=` character, optional [whitespace], and an [attribute
value].

An [attribute value](@)
consists of an [unquoted attribute value],
a [single-quoted attribute value], or a [double-quoted attribute value].

An [unquoted attribute value](@)
is a nonempty string of characters not
including [whitespace], `"`, `'`, `=`, `<`, `>`, or `` ` ``.

A [single-quoted attribute value](@)
consists of `'`, zero or more
characters not including `'`, and a final `'`.

A [double-quoted attribute value](@)
consists of `"`, zero or more
characters not including `"`, and a final `"`.

An [open tag](@) consists of a `<` character, a [tag name],
zero or more [attributes], optional [whitespace], an optional `/`
character, and a `>` character.

A [closing tag](@) consists of the string ``.

An [HTML comment](@) consists of ``,
where *text* does not start with `>` or `->`, does not end with `-`,
and does not contain `--`.  (See the
[HTML5 spec](http://www.w3.org/TR/html5/syntax.html#comments).)

A [processing instruction](@)
consists of the string ``, and the string
`?>`.

A [declaration](@) consists of the
string ``, and the character `>`.

A [CDATA section](@) consists of
the string ``, and the string `]]>`.

An [HTML tag](@) consists of an [open tag], a [closing tag],
an [HTML comment], a [processing instruction], a [declaration],
or a [CDATA section].

Here are some simple open tags:

```````````````````````````````` example

.


````````````````````````````````


Empty elements:

```````````````````````````````` example

.


````````````````````````````````


[Whitespace] is allowed:

```````````````````````````````` example

.


````````````````````````````````


With attributes:

```````````````````````````````` example

.


````````````````````````````````


Custom tag names can be used:

```````````````````````````````` example
Foo 
.

Foo 

````````````````````````````````


Illegal tag names, not parsed as HTML:

```````````````````````````````` example
<33> <__>
.

<33> <__>

````````````````````````````````


Illegal attribute names:

```````````````````````````````` example

.

<a h*#ref="hi">

````````````````````````````````


Illegal attribute values:

```````````````````````````````` example

.

</a href="foo">

````````````````````````````````


Comments:

```````````````````````````````` example
foo 
.

foo 

````````````````````````````````


```````````````````````````````` example
foo 
.

foo <!-- not a comment -- two hyphens -->

````````````````````````````````


Not comments:

```````````````````````````````` example
foo  foo -->

foo 
.

foo <!--> foo -->


foo <!-- foo--->

````````````````````````````````


Processing instructions:

```````````````````````````````` example
foo 
.

foo 

````````````````````````````````


Declarations:

```````````````````````````````` example
foo 
.

foo 

````````````````````````````````


CDATA sections:

```````````````````````````````` example
foo &<]]>
.

foo &<]]>

````````````````````````````````


Entity and numeric character references are preserved in HTML
attributes:

```````````````````````````````` example
foo 
.

foo 

````````````````````````````````


Backslash escapes do not work in HTML attributes:

```````````````````````````````` example
foo 
.

foo 

````````````````````````````````


```````````````````````````````` example

.

<a href=""">

````````````````````````````````


## Hard line breaks

A line break (not in a code span or HTML tag) that is preceded
by two or more spaces and does not occur at the end of a block
is parsed as a [hard line break](@) (rendered
in HTML as a `
` tag):

```````````````````````````````` example
foo  
baz
.

foo

baz

````````````````````````````````


For a more visible alternative, a backslash before the
[line ending] may be used instead of two spaces:

```````````````````````````````` example
foo\
baz
.

foo

baz

````````````````````````````````


More than two spaces can be used:

```````````````````````````````` example
foo       
baz
.

foo

baz

````````````````````````````````


Leading spaces at the beginning of the next line are ignored:

```````````````````````````````` example
foo  
     bar
.

foo

bar

````````````````````````````````


```````````````````````````````` example
foo\
     bar
.

foo

bar

````````````````````````````````


Line breaks can occur inside emphasis, links, and other constructs
that allow inline content:

```````````````````````````````` example
*foo  
bar*
.

foo

bar

````````````````````````````````


```````````````````````````````` example
*foo\
bar*
.

foo

bar

````````````````````````````````


Line breaks do not occur inside code spans

```````````````````````````````` example
`code 
span`
.

code  span

````````````````````````````````


```````````````````````````````` example
`code\
span`
.

code\ span

````````````````````````````````


or HTML tags:

```````````````````````````````` example

.


````````````````````````````````


```````````````````````````````` example

.


````````````````````````````````


Hard line breaks are for separating inline content within a block.
Neither syntax for hard line breaks works at the end of a paragraph or
other block element:

```````````````````````````````` example
foo\
.

foo\

````````````````````````````````


```````````````````````````````` example
foo  
.

foo

````````````````````````````````


```````````````````````````````` example
### foo\
.

foo\

````````````````````````````````


```````````````````````````````` example
### foo  
.

foo

````````````````````````````````


## Soft line breaks

A regular line break (not in a code span or HTML tag) that is not
preceded by two or more spaces or a backslash is parsed as a
[softbreak](@).  (A softbreak may be rendered in HTML either as a
[line ending] or as a space. The result will be the same in
browsers. In the examples here, a [line ending] will be used.)

```````````````````````````````` example
foo
baz
.

foo
baz

````````````````````````````````


Spaces at the end of the line and beginning of the next line are
removed:

```````````````````````````````` example
foo 
 baz
.

foo
baz

````````````````````````````````


A conforming parser may render a soft line break in HTML either as a
line break or as a space.

A renderer may also provide an option to render soft line breaks
as hard line breaks.

## Textual content

Any characters not given an interpretation by the above rules will
be parsed as plain textual content.

```````````````````````````````` example
hello $.;'there
.

hello $.;'there

````````````````````````````````


```````````````````````````````` example
Foo χρῆν
.

Foo χρῆν

````````````````````````````````


Internal spaces are preserved verbatim:

```````````````````````````````` example
Multiple     spaces
.

Multiple     spaces

````````````````````````````````




# Appendix: A parsing strategy

In this appendix we describe some features of the parsing strategy
used in the CommonMark reference implementations.

## Overview

Parsing has two phases:

1. In the first phase, lines of input are consumed and the block
structure of the document---its division into paragraphs, block quotes,
list items, and so on---is constructed.  Text is assigned to these
blocks but not parsed. Link reference definitions are parsed and a
map of links is constructed.

2. In the second phase, the raw text contents of paragraphs and headings
are parsed into sequences of Markdown inline elements (strings,
code spans, links, emphasis, and so on), using the map of link
references constructed in phase 1.

At each point in processing, the document is represented as a tree of
**blocks**.  The root of the tree is a `document` block.  The `document`
may have any number of other blocks as **children**.  These children
may, in turn, have other blocks as children.  The last child of a block
is normally considered **open**, meaning that subsequent lines of input
can alter its contents.  (Blocks that are not open are **closed**.)
Here, for example, is a possible document tree, with the open blocks
marked by arrows:

``` tree
-> document
  -> block_quote
       paragraph
         "Lorem ipsum dolor\nsit amet."
    -> list (type=bullet tight=true bullet_char=-)
         list_item
           paragraph
             "Qui *quodsi iracundia*"
      -> list_item
        -> paragraph
             "aliquando id"
```

## Phase 1: block structure

Each line that is processed has an effect on this tree.  The line is
analyzed and, depending on its contents, the document may be altered
in one or more of the following ways:

1. One or more open blocks may be closed.
2. One or more new blocks may be created as children of the
   last open block.
3. Text may be added to the last (deepest) open block remaining
   on the tree.

Once a line has been incorporated into the tree in this way,
it can be discarded, so input can be read in a stream.

For each line, we follow this procedure:

1. First we iterate through the open blocks, starting with the
root document, and descending through last children down to the last
open block.  Each block imposes a condition that the line must satisfy
if the block is to remain open.  For example, a block quote requires a
`>` character.  A paragraph requires a non-blank line.
In this phase we may match all or just some of the open
blocks.  But we cannot close unmatched blocks yet, because we may have a
[lazy continuation line].

2.  Next, after consuming the continuation markers for existing
blocks, we look for new block starts (e.g. `>` for a block quote).
If we encounter a new block start, we close any blocks unmatched
in step 1 before creating the new block as a child of the last
matched block.

3.  Finally, we look at the remainder of the line (after block
markers like `>`, list markers, and indentation have been consumed).
This is text that can be incorporated into the last open
block (a paragraph, code block, heading, or raw HTML).

Setext headings are formed when we see a line of a paragraph
that is a [setext heading underline].

Reference link definitions are detected when a paragraph is closed;
the accumulated text lines are parsed to see if they begin with
one or more reference link definitions.  Any remainder becomes a
normal paragraph.

We can see how this works by considering how the tree above is
generated by four lines of Markdown:

``` markdown
> Lorem ipsum dolor
sit amet.
> - Qui *quodsi iracundia*
> - aliquando id
```

At the outset, our document model is just

``` tree
-> document
```

The first line of our text,

``` markdown
> Lorem ipsum dolor
```

causes a `block_quote` block to be created as a child of our
open `document` block, and a `paragraph` block as a child of
the `block_quote`.  Then the text is added to the last open
block, the `paragraph`:

``` tree
-> document
  -> block_quote
    -> paragraph
         "Lorem ipsum dolor"
```

The next line,

``` markdown
sit amet.
```

is a "lazy continuation" of the open `paragraph`, so it gets added
to the paragraph's text:

``` tree
-> document
  -> block_quote
    -> paragraph
         "Lorem ipsum dolor\nsit amet."
```

The third line,

``` markdown
> - Qui *quodsi iracundia*
```

causes the `paragraph` block to be closed, and a new `list` block
opened as a child of the `block_quote`.  A `list_item` is also
added as a child of the `list`, and a `paragraph` as a child of
the `list_item`.  The text is then added to the new `paragraph`:

``` tree
-> document
  -> block_quote
       paragraph
         "Lorem ipsum dolor\nsit amet."
    -> list (type=bullet tight=true bullet_char=-)
      -> list_item
        -> paragraph
             "Qui *quodsi iracundia*"
```

The fourth line,

``` markdown
> - aliquando id
```

causes the `list_item` (and its child the `paragraph`) to be closed,
and a new `list_item` opened up as child of the `list`.  A `paragraph`
is added as a child of the new `list_item`, to contain the text.
We thus obtain the final tree:

``` tree
-> document
  -> block_quote
       paragraph
         "Lorem ipsum dolor\nsit amet."
    -> list (type=bullet tight=true bullet_char=-)
         list_item
           paragraph
             "Qui *quodsi iracundia*"
      -> list_item
        -> paragraph
             "aliquando id"
```

## Phase 2: inline structure

Once all of the input has been parsed, all open blocks are closed.

We then "walk the tree," visiting every node, and parse raw
string contents of paragraphs and headings as inlines.  At this
point we have seen all the link reference definitions, so we can
resolve reference links as we go.

``` tree
document
  block_quote
    paragraph
      str "Lorem ipsum dolor"
      softbreak
      str "sit amet."
    list (type=bullet tight=true bullet_char=-)
      list_item
        paragraph
          str "Qui "
          emph
            str "quodsi iracundia"
      list_item
        paragraph
          str "aliquando id"
```

Notice how the [line ending] in the first paragraph has
been parsed as a `softbreak`, and the asterisks in the first list item
have become an `emph`.

### An algorithm for parsing nested emphasis and links

By far the trickiest part of inline parsing is handling emphasis,
strong emphasis, links, and images.  This is done using the following
algorithm.

When we're parsing inlines and we hit either

- a run of `*` or `_` characters, or
- a `[` or `![`

we insert a text node with these symbols as its literal content, and we
add a pointer to this text node to the [delimiter stack](@).

The [delimiter stack] is a doubly linked list.  Each
element contains a pointer to a text node, plus information about

- the type of delimiter (`[`, `![`, `*`, `_`)
- the number of delimiters,
- whether the delimiter is "active" (all are active to start), and
- whether the delimiter is a potential opener, a potential closer,
  or both (which depends on what sort of characters precede
  and follow the delimiters).

When we hit a `]` character, we call the *look for link or image*
procedure (see below).

When we hit the end of the input, we call the *process emphasis*
procedure (see below), with `stack_bottom` = NULL.

#### *look for link or image*

Starting at the top of the delimiter stack, we look backwards
through the stack for an opening `[` or `![` delimiter.

- If we don't find one, we return a literal text node `]`.

- If we do find one, but it's not *active*, we remove the inactive
  delimiter from the stack, and return a literal text node `]`.

- If we find one and it's active, then we parse ahead to see if
  we have an inline link/image, reference link/image, compact reference
  link/image, or shortcut reference link/image.

  + If we don't, then we remove the opening delimiter from the
    delimiter stack and return a literal text node `]`.

  + If we do, then

    * We return a link or image node whose children are the inlines
      after the text node pointed to by the opening delimiter.

    * We run *process emphasis* on these inlines, with the `[` opener
      as `stack_bottom`.

    * We remove the opening delimiter.

    * If we have a link (and not an image), we also set all
      `[` delimiters before the opening delimiter to *inactive*.  (This
      will prevent us from getting links within links.)

#### *process emphasis*

Parameter `stack_bottom` sets a lower bound to how far we
descend in the [delimiter stack].  If it is NULL, we can
go all the way to the bottom.  Otherwise, we stop before
visiting `stack_bottom`.

Let `current_position` point to the element on the [delimiter stack]
just above `stack_bottom` (or the first element if `stack_bottom`
is NULL).

We keep track of the `openers_bottom` for each delimiter
type (`*`, `_`) and each length of the closing delimiter run
(modulo 3).  Initialize this to `stack_bottom`.

Then we repeat the following until we run out of potential
closers:

- Move `current_position` forward in the delimiter stack (if needed)
  until we find the first potential closer with delimiter `*` or `_`.
  (This will be the potential closer closest
  to the beginning of the input -- the first one in parse order.)

- Now, look back in the stack (staying above `stack_bottom` and
  the `openers_bottom` for this delimiter type) for the
  first matching potential opener ("matching" means same delimiter).

- If one is found:

  + Figure out whether we have emphasis or strong emphasis:
    if both closer and opener spans have length >= 2, we have
    strong, otherwise regular.

  + Insert an emph or strong emph node accordingly, after
    the text node corresponding to the opener.

  + Remove any delimiters between the opener and closer from
    the delimiter stack.

  + Remove 1 (for regular emph) or 2 (for strong emph) delimiters
    from the opening and closing text nodes.  If they become empty
    as a result, remove them and remove the corresponding element
    of the delimiter stack.  If the closing node is removed, reset
    `current_position` to the next element in the stack.

- If none is found:

  + Set `openers_bottom` to the element before `current_position`.
    (We know that there are no openers for this kind of closer up to and
    including this point, so this puts a lower bound on future searches.)

  + If the closer at `current_position` is not a potential opener,
    remove it from the delimiter stack (since we know it can't
    be a closer either).

  + Advance `current_position` to the next element in the stack.

After we're done, we remove all delimiters above `stack_bottom` from the
delimiter stack.