pax_global_header00006660000000000000000000000064142010246220014503gustar00rootroot0000000000000052 comment=5916ebd57a52668976831b2d249e96662cb001f4 mistletoe-0.8.2/000077500000000000000000000000001420102462200135175ustar00rootroot00000000000000mistletoe-0.8.2/.gitignore000066400000000000000000000001561420102462200155110ustar00rootroot00000000000000__pycache__/ *.swp out.html venv/ .coverage htmlcov .pylintrc *.egg-info dist build/ .DS_Store *.vim .vscode/ mistletoe-0.8.2/.travis.yml000066400000000000000000000006211420102462200156270ustar00rootroot00000000000000language: python python: - "3.3" - "3.4" - "3.5" - "3.6" matrix: include: - python: 3.7 dist: xenial sudo: true before_script: - (cd test/specification && ./spec.sh) script: - make test - python3 -m test.specification after_success: - pip install coveralls - coverage run -a -m unittest - coverage run -a -m test.specification - coveralls mistletoe-0.8.2/CONTRIBUTING.md000066400000000000000000000105341420102462200157530ustar00rootroot00000000000000

Contributing

You've seen mistletoe: it branches off in all directions, bringing people together. We would love to see what you can make of mistletoe, which direction you would take it to. Or maybe you can discover some [Nargles][nargles], which, by the way, totally exists. The following instructions serve as guidelines, and you should use your best judgements when employing them. ## Getting started Refer to the [README][readme] for install instructions. Since you're going to mess with the code, it's prefered that you clone the repo directly. Check back on the dev branch regularly to avoid redoing work that others might have done. The master branch is updated only when features on the dev branch are stabilized somewhat. ## Things you can do ### Introducing new features It is suggested that you **open an issue first** before working on new features. Include your reasons, use case, and maybe plans for implementation. That way, we have a better idea of what you'll be working on, and can hopefully avoid collision. Your pull request may also get merged much faster. ### Fixing bugs Before you post an issue, try narrowing the problem down to the smallest component possible. For example, if an `InlineCode` token is not parsed correctly, include only the paragraph that introduce the error, not the entire document. You might find mistletoe's interactive mode handy when tracking down bugs. Type in your input, and you immediately see how mistletoe handles it. I created it just for this purpose. To use it, run `mistletoe` (or `python3 mistletoe`) in your shell without arguments. Markdown is a very finicky document format to parse, so if something does not work as intended, it's probably my fault and not yours. ### Writing documentations The creator might not the best person to write documentations; the users, knowing all the painpoints, have a better idea of actual use cases and possible things that can go wrong. Write docstrings or comments for functions that are missing them. mistletoe generally follows the [Google Python Style Guide][style-guide] to format comments. ## Writing code ### Commit messages * minimal cosmetic changes are fine to mix in with your commits, but try feel guilty when you do that, and if it's not too big of a hassle, break them into two commits. * give clear, instructive commit messages. Try using phrases like "added XXX feature" or "fixed XXX (#42)". * if you find yourself cramming too many things into one commit message, you should probably break them into multiple commits. * emojis are awesome. Use them like this: | Emoji | Description | | :---: | :------------------------------ | | 📚 | Update documentation. | | 🐎 | Performance improvements. | | 💡 | New features. | | 🐛 | Bug fixes. | | 🚨 | Under construction. | | ☕️ | Refactoring / cosmetic changes. | | 🌎 | Internationalization. | ### Style guide Here's the obligatory [PEP8][pep-8] link, but here's a much shorter list of things to be aware of: * mistletoe uses `CamelCase` for classnames, `snake_case` for functions and methods; * mistletoe uses *one* blank line between classes and functions, even global ones, despite PEP8's suggestion to the contrary. * mistletoe follows the eighty-character rule: if you find your line to be too lengthy, try giving variable names to expressions, and break it up that way. That said, it's okay to go over the charater limit occasionally. * mistletoe uses four spaces instead of a tab to indent. For vim users, include `set ts=4 sw=4 ai et` in your `.vimrc`. Apart from that, stay consistent with the coding style around you. But don't get boggled down by this: if you have a genius idea, I'd love to clean up for you; write down your genius idea first. ## Get in touch I tweet [@mi_before_yu][twitter]. Also yell at me over [email][email]. [nargles]: http://harrypotter.wikia.com/wiki/Nargle [readme]: README.md [wiki]: https://github.com/miyuchina/mistletoe/wiki [style-guide]: https://google.github.io/styleguide/pyguide.html [pep-8]: https://www.python.org/dev/peps/pep-0008/ [twitter]: https://twitter.com/mi_before_yu [email]: mailto:hello@afteryu.me mistletoe-0.8.2/LICENSE000066400000000000000000000020461420102462200145260ustar00rootroot00000000000000The MIT License Copyright 2017 Mi Yu Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. mistletoe-0.8.2/MANIFEST.in000066400000000000000000000000201420102462200152450ustar00rootroot00000000000000include LICENSE mistletoe-0.8.2/README.md000066400000000000000000000203631420102462200150020ustar00rootroot00000000000000

mistletoe

[![Build Status][build-badge]][travis] [![Coverage Status][cover-badge]][coveralls] [![PyPI][pypi-badge]][pypi] [![is wheel][wheel-badge]][pypi] mistletoe is a Markdown parser in pure Python, designed to be fast, spec-compliant and fully customizable. Apart from being the fastest CommonMark-compliant Markdown parser implementation in pure Python, mistletoe also supports easy definitions of custom tokens. Parsing Markdown into an abstract syntax tree also allows us to swap out renderers for different output formats, without touching any of the core components. Remember to spell mistletoe in lowercase! Features -------- * **Fast**: mistletoe is the fastest implementation of CommonMark in Python. See the [performance][performance] section for details. * **Spec-compliant**: CommonMark is [a useful, high-quality project][oilshell]. mistletoe follows the [CommonMark specification][commonmark] to resolve ambiguities during parsing. Outputs are predictable and well-defined. * **Extensible**: Strikethrough and tables are supported natively, and custom block-level and span-level tokens can easily be added. Writing a new renderer for mistletoe is a relatively trivial task. You can even write [a Lisp][scheme] in it. Output formats -------------- Renderers for the following "core" output formats exist within the mistletoe module itself: * HTML * LaTeX * AST (Abstract Syntax Tree; handy for debugging the parsing process) Renderers for the following output formats are placed in the [contrib][contrib] folder: * HTML with MathJax (_mathjax.py_) * HTML with code highlighting (using Pygments) (_pygments\_renderer.py_) * HTML with TOC (for programmatical use) (_toc\_renderer.py_) * HTML with support for GitHub wiki links (_github\_wiki.py_) * Jira Markdown (_jira\_renderer.py_) * XWiki Syntax (_xwiki20\_renderer.py_) * Scheme (_scheme.py_) Installation ------------ mistletoe is tested for Python 3.3 and above. Install mistletoe with pip: ```sh pip3 install mistletoe ``` Alternatively, clone the repo: ```sh git clone https://github.com/miyuchina/mistletoe.git cd mistletoe pip3 install -e . ``` This installs mistletoe in "editable" mode (because of the `-e` option). That means that any changes made to the source code will get visible immediately - that's because Python only makes a link to the specified directory (`.`) instead of copying the files to the standard packages folder. See the [contributing][contributing] doc for how to contribute to mistletoe. Usage ----- ### Usage from Python Here's how you can use mistletoe in a Python script: ```python import mistletoe with open('foo.md', 'r') as fin: rendered = mistletoe.markdown(fin) ``` `mistletoe.markdown()` uses mistletoe's default settings: allowing HTML mixins and rendering to HTML. The function also accepts an additional argument `renderer`. To produce LaTeX output: ```python import mistletoe from mistletoe.latex_renderer import LaTeXRenderer with open('foo.md', 'r') as fin: rendered = mistletoe.markdown(fin, LaTeXRenderer) ``` Finally, here's how you would manually specify extra tokens via a renderer. In the following example, we use `HTMLRenderer` to render the AST. The renderer itself adds `HTMLBlock` and `HTMLSpan` tokens to the parsing process. The result should be equal to the output obtained from the first example above. ```python from mistletoe import Document, HTMLRenderer with open('foo.md', 'r') as fin: with HTMLRenderer() as renderer: # or: `with HTMLRenderer(AnotherToken1, AnotherToken2) as renderer:` doc = Document(fin) # parse the lines into AST rendered = renderer.render(doc) # render the AST # internal lists of tokens to be parsed are automatically reset when exiting this `with` block ``` **Important**: As can be seen from the example above, the parsing phase is currently tightly connected with initiation and closing of a renderer. Therefore, you should never call `Document(...)` outside of a `with ... as renderer` block, unless you know what you are doing. ### Usage from command-line pip installation enables mistletoe's command-line utility. Type the following directly into your shell: ```sh mistletoe foo.md ``` This will transpile `foo.md` into HTML, and dump the output to stdout. To save the HTML, direct the output into a file: ```sh mistletoe foo.md > out.html ``` You can use a different renderer by including the full path to the renderer class after a `-r` or `--renderer` flag. For example, to transpile into LaTeX: ```sh mistletoe foo.md --renderer mistletoe.latex_renderer.LaTeXRenderer ``` Note: The renderers inside the `contrib` directory are not currently a part of the `mistletoe` module when mistletoe is installed as a regular package. So if you want to use a renderer from the `contrib` directory, you either have to add that directory to Python's [PYTHONPATH][pythonpath] or install mistletoe with the `-e` switch (see [Installation](#installation)). ### mistletoe interactive mode Running `mistletoe` without specifying a file will land you in interactive mode. Like Python's REPL, interactive mode allows you to test how your Markdown will be interpreted by mistletoe: ```html mistletoe [version 0.7.2] (interactive) Type Ctrl-D to complete input, or Ctrl-C to exit. >>> some **bold** text ... and some *italics* ...

some bold text and some italics

>>> ``` The interactive mode also accepts the `--renderer` flag: ```latex mistletoe [version 0.7.2] (interactive) Type Ctrl-D to complete input, or Ctrl-C to exit. Using renderer: LaTeXRenderer >>> some **bold** text ... and some *italics* ... \documentclass{article} \begin{document} some \textbf{bold} text and some \textit{italics} \end{document} >>> ``` Who uses mistletoe? ------------------- mistletoe is used by projects of various target audience. You can find some concrete projects in the "Used by" section on [Libraries.io][libraries-mistletoe], but this is definitely not a complete list. Also a list of [Dependents][github-dependents] is tracked by GitHub directly. ### Run mistletoe from CopyQ One notable example is running mistletoe as a Markdown converter from the advanced clipboard manager called [CopyQ][copyq]. One just needs to install the [Convert Markdown to ...][copyq-convert-md] custom script command and then run this command on any selected Markdown text. Why mistletoe? -------------- "For fun," says David Beazley. Further reading --------------- * [Performance][performance] * [Developer's Guide](dev-guide.md) Copyright & License ------------------- * mistletoe's logo uses artwork by [Freepik][icon], under [CC BY 3.0][cc-by]. * mistletoe is released under [MIT][license]. [build-badge]: https://img.shields.io/travis/miyuchina/mistletoe.svg?style=flat-square [cover-badge]: https://img.shields.io/coveralls/miyuchina/mistletoe.svg?style=flat-square [pypi-badge]: https://img.shields.io/pypi/v/mistletoe.svg?style=flat-square [wheel-badge]: https://img.shields.io/pypi/wheel/mistletoe.svg?style=flat-square [travis]: https://travis-ci.org/miyuchina/mistletoe [coveralls]: https://coveralls.io/github/miyuchina/mistletoe?branch=master [pypi]: https://pypi.python.org/pypi/mistletoe [mistune]: https://github.com/lepture/mistune [python-markdown]: https://github.com/waylan/Python-Markdown [python-markdown2]: https://github.com/trentm/python-markdown2 [commonmark-py]: https://github.com/rtfd/CommonMark-py [performance]: performance.md [oilshell]: https://www.oilshell.org/blog/2018/02/14.html [commonmark]: https://spec.commonmark.org/ [contrib]: https://github.com/miyuchina/mistletoe/tree/master/contrib [scheme]: https://github.com/miyuchina/mistletoe/blob/dev/contrib/scheme.py [contributing]: CONTRIBUTING.md [icon]: https://www.freepik.com [cc-by]: https://creativecommons.org/licenses/by/3.0/us/ [license]: LICENSE [pythonpath]: https://stackoverflow.com/questions/16107526/how-to-flexibly-change-pythonpath [libraries-mistletoe]: https://libraries.io/pypi/mistletoe [copyq]: https://hluk.github.io/CopyQ/ [copyq-convert-md]: https://github.com/hluk/copyq-commands/tree/master/Global#convert-markdown-to- [github-dependents]: https://github.com/miyuchina/mistletoe/network/dependents mistletoe-0.8.2/contrib/000077500000000000000000000000001420102462200151575ustar00rootroot00000000000000mistletoe-0.8.2/contrib/Makefile000066400000000000000000000003141420102462200166150ustar00rootroot00000000000000 INSTALL_DIR=$(HOME)/bin install: install-md2jira install-md2jira: echo "[contrib] installing: md2jira" cp md2jira.py $(INSTALL_DIR)/md2jira chmod uog+x $(INSTALL_DIR)/md2jira echo "[contrib] done" mistletoe-0.8.2/contrib/github_wiki.py000066400000000000000000000012711420102462200200370ustar00rootroot00000000000000""" GitHub Wiki support for mistletoe. """ import re from mistletoe.span_token import SpanToken from mistletoe.html_renderer import HTMLRenderer __all__ = ['GithubWiki', 'GithubWikiRenderer'] class GithubWiki(SpanToken): pattern = re.compile(r"\[\[ *(.+?) *\| *(.+?) *\]\]") def __init__(self, match): self.target = match.group(2) class GithubWikiRenderer(HTMLRenderer): def __init__(self): super().__init__(GithubWiki) def render_github_wiki(self, token): template = '{inner}' target = self.escape_url(token.target) inner = self.render_inner(token) return template.format(target=target, inner=inner) mistletoe-0.8.2/contrib/jira_renderer.py000066400000000000000000000202651420102462200203510ustar00rootroot00000000000000# Copyright 2018 Tile, Inc. All Rights Reserved. # # The MIT License # # Permission is hereby granted, free of charge, to any person obtaining a copy of # this software and associated documentation files (the "Software"), to deal in # the Software without restriction, including without limitation the rights to # use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies # of the Software, and to permit persons to whom the Software is furnished to do # so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # import html from itertools import chain from mistletoe import block_token, span_token from mistletoe.base_renderer import BaseRenderer import re import sys class JIRARenderer(BaseRenderer): """ JIRA renderer class. See mistletoe.base_renderer module for more info. """ def __init__(self, *extras): """ Args: extras (list): allows subclasses to add even more custom tokens. """ self.listTokens = [] self.lastChildOfQuotes = [] super().__init__(*chain([block_token.HTMLBlock, span_token.HTMLSpan], extras)) def render_strong(self, token): template = '*{}*' return template.format(self.render_inner(token)) def render_emphasis(self, token): template = '_{}_' return template.format(self.render_inner(token)) def render_inline_code(self, token): template = '{{{{{}}}}}' return template.format(self.render_inner(token)) def render_strikethrough(self, token): template = '-{}-' return template.format(self.render_inner(token)) def render_image(self, token): template = '!{src}!' inner = self.render_inner(token) return template.format(src=token.src) def render_link(self, token): template = '[{inner}|{target}]' target = escape_url(token.target) inner = self.render_inner(token) return template.format(target=target, inner=inner) def render_auto_link(self, token): template = '[{target}]' target = escape_url(token.target) #inner = self.render_inner(token) return template.format(target=target) def render_escape_sequence(self, token): return self.render_inner(token) def render_raw_text(self, token, escape=True): if escape: def repl(match): return '\\' + match.group(0) # The following regex tries to find special chars that are one of the following: # 1. the whole string (typically in an EscapeSequence) # 2. just after a non-whitespace # 3. just before a non-whitespace re_esc_chars = r'[{}\[\]\-*_+^~]' re_find = r'(^{esc_chars}$)|((?<=\S)({esc_chars}))|(({esc_chars})(?=\S))'.format(esc_chars=re_esc_chars) return re.sub(re_find, repl, token.content) else: return token.content @staticmethod def render_html_span(token): return token.content def render_heading(self, token): template = 'h{level}. {inner}' inner = self.render_inner(token) return template.format(level=token.level, inner=inner) + self._block_eol(token) def render_quote(self, token): self.lastChildOfQuotes.append(token.children[-1]) inner = self.render_inner(token) del (self.lastChildOfQuotes[-1]) if len(token.children) == 1 and isinstance(token.children[0], block_token.Paragraph): template = 'bq. {inner}' + self._block_eol(token)[0:-1] else: template = '{{quote}}\n{inner}{{quote}}' + self._block_eol(token) return template.format(inner=inner) def render_paragraph(self, token): return '{}'.format(self.render_inner(token)) + self._block_eol(token) def render_block_code(self, token): template = '{{code{attr}}}\n{inner}{{code}}' + self._block_eol(token) if token.language: attr = ':{}'.format(token.language) else: attr = '' inner = self.render_raw_text(token.children[0], False) return template.format(attr=attr, inner=inner) def render_list(self, token): inner = self.render_inner(token) return inner + self._block_eol(token)[0:-1] def render_list_item(self, token): template = '{prefix} {inner}' prefix = ''.join(self.listTokens) result = template.format(prefix=prefix, inner=self.render_inner(token)) return result def render_inner(self, token): if isinstance(token, block_token.List): if token.start: self.listTokens.append('#') else: self.listTokens.append('*') rendered = [self.render(child) for child in token.children] if isinstance(token, block_token.List): del (self.listTokens[-1]) return ''.join(rendered) def render_table(self, token): # This is actually gross and I wonder if there's a better way to do it. # # The primary difficulty seems to be passing down alignment options to # reach individual cells. template = '{inner}\n' if hasattr(token, 'header'): head_template = '{inner}' header = token.header head_inner = self.render_table_row(header, True) head_rendered = head_template.format(inner=head_inner) else: head_rendered = '' body_template = '{inner}' body_inner = self.render_inner(token) body_rendered = body_template.format(inner=body_inner) return template.format(inner=head_rendered+body_rendered) def render_table_row(self, token, is_header=False): if is_header: template = '{inner}||\n' else: template = '{inner}|\n' inner = ''.join([self.render_table_cell(child, is_header) for child in token.children]) return template.format(inner=inner) def render_table_cell(self, token, in_header=False): if in_header: template = '||{inner}' else: template = '|{inner}' inner = self.render_inner(token) if inner == '': inner = ' ' return template.format(inner=inner) @staticmethod def render_thematic_break(token): return '----\n' @staticmethod def render_line_break(token): # Note: In Jira, outputting just '\n' instead of '\\\n' should be usually sufficient as well. # It is not clear when it wouldn't be sufficient though, so we use the longer variant for sure. return ' ' if token.soft else '\\\\\n' @staticmethod def render_html_block(token): return token.content def render_document(self, token): self.footnotes.update(token.footnotes) return self.render_inner(token) def _block_eol(self, token): """ Jira syntax is very limited when it comes to lists: whenever we put an empty line anywhere in a list, it gets terminated and there seems to be no workaround for this. Also to have blocks like paragraphs really vertically separated, we need to put an empty line between them. This function handles these two cases. """ return '\n' if len(self.listTokens) > 0 or (len(self.lastChildOfQuotes) > 0 and token is self.lastChildOfQuotes[-1]) else '\n\n' def escape_url(raw): """ Escape urls to prevent code injection craziness. (Hopefully.) """ from urllib.parse import quote return quote(html.unescape(raw), safe='/#:()*?=%@+,&;') mistletoe-0.8.2/contrib/mathjax.py000066400000000000000000000016371420102462200171740ustar00rootroot00000000000000""" Provides MathJax support for rendering Markdown with LaTeX to html. """ from mistletoe.html_renderer import HTMLRenderer from mistletoe.latex_renderer import LaTeXRenderer class MathJaxRenderer(HTMLRenderer, LaTeXRenderer): """ MRO will first look for render functions under HTMLRenderer, then LaTeXRenderer. """ mathjax_src = '\n' def render_math(self, token): """ Ensure Math tokens are all enclosed in two dollar signs. """ if token.content.startswith('$$'): return self.render_raw_text(token) return '${}$'.format(self.render_raw_text(token)) def render_document(self, token): """ Append CDN link for MathJax to the end of . """ return super().render_document(token) + self.mathjax_src mistletoe-0.8.2/contrib/md2jira.py000077500000000000000000000066301420102462200170710ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright 2018 Tile, Inc. # # The MIT License # # Permission is hereby granted, free of charge, to any person obtaining a copy of # this software and associated documentation files (the "Software"), to deal in # the Software without restriction, including without limitation the rights to # use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies # of the Software, and to permit persons to whom the Software is furnished to do # so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # import os import sys import getopt import subprocess import shutil import mistletoe from contrib.jira_renderer import JIRARenderer usageString = '%s ' % os.path.basename(sys.argv[0]) helpString = """ Convert Markdown (CommonMark) to JIRA wiki markup -h, --help help -v, --version version -o , --output= output file, use '-' for stdout (default: stdout) If no input file is specified, stdin is used. """ """ Command-line utility to convert Markdown (CommonMark) to JIRA markup. JIRA markup spec: https://jira.atlassian.com/secure/WikiRendererHelpAction.jspa?section=all CommonMark spec: http://spec.commonmark.org/0.28/#introduction """ class CommandLineParser: def __init__(self): try: optlist, args = getopt.getopt(sys.argv[1:], 'hvo:', ['help', 'version', 'output=']) except getopt.GetoptError as err: sys.stderr.write(err.msg + '\n') sys.stderr.write(usageString + '\n') sys.exit(1) app = MarkdownToJIRA() app.run(optlist, args) class MarkdownToJIRA: def __init__(self): self.version = "1.0.2" self.options = {} self.options['output'] = '-' def run(self, optlist, args): for o, i in optlist: if o in ('-h', '--help'): sys.stderr.write(usageString + '\n') sys.stderr.write(helpString + '\n') sys.exit(1) elif o in ('-v', '--version'): sys.stdout.write('%s\n' % self.version) sys.exit(0) elif o in ('-o', '--output'): self.options['output'] = i if len(args) < 1: sys.stderr.write(usageString + '\n') sys.exit(1) with open(args[0], 'r', encoding='utf-8') if len(args) == 1 else sys.stdin as infile: rendered = mistletoe.markdown(infile, JIRARenderer) if self.options['output'] == '-': sys.stdout.write(rendered) else: with open(self.options['output'], 'w', encoding='utf-8') as outfile: outfile.write(rendered) if __name__ == '__main__': CommandLineParser() mistletoe-0.8.2/contrib/pygments_renderer.py000066400000000000000000000013001420102462200212570ustar00rootroot00000000000000from mistletoe import HTMLRenderer from pygments import highlight from pygments.styles import get_style_by_name as get_style from pygments.lexers import get_lexer_by_name as get_lexer, guess_lexer from pygments.formatters.html import HtmlFormatter class PygmentsRenderer(HTMLRenderer): formatter = HtmlFormatter() formatter.noclasses = True def __init__(self, *extras, style='default'): super().__init__(*extras) self.formatter.style = get_style(style) def render_block_code(self, token): code = token.children[0].content lexer = get_lexer(token.language) if token.language else guess_lexer(code) return highlight(code, lexer, self.formatter) mistletoe-0.8.2/contrib/scheme.py000066400000000000000000000120621420102462200167760ustar00rootroot00000000000000import re from collections import ChainMap from mistletoe import BaseRenderer, span_token, block_token from mistletoe.core_tokens import MatchObj class Program(block_token.BlockToken): def __init__(self, lines): self.children = span_token.tokenize_inner(''.join([line.strip() for line in lines])) class Expr(span_token.SpanToken): @classmethod def find(cls, string): matches = [] count = 0 start = [] for i, c in enumerate(string): if c == '(': start.append(i) elif c == ')': pos = start.pop() end_pos = i + 1 content = string[pos+1:i] matches.append(MatchObj(pos, end_pos, (pos+1, i, content))) return matches def __repr__(self): return ''.format(self.children) class Number(span_token.SpanToken): pattern = re.compile(r"(\d+)") parse_inner = False def __init__(self, match): self.number = eval(match.group(0)) def __repr__(self): return ''.format(self.number) class Variable(span_token.SpanToken): pattern = re.compile(r"([^\s()]+)") parse_inner = False def __init__(self, match): self.name = match.group(0) def __repr__(self): return ''.format(self.name) class Whitespace(span_token.SpanToken): parse_inner = False def __new__(self, _): return None class Procedure: def __init__(self, expr_token, body, env): self.params = [child.name for child in expr_token.children] self.body = body self.env = env class Scheme(BaseRenderer): def __init__(self): self.render_map = { "Program": self.render_program, "Expr": self.render_expr, "Number": self.render_number, "Variable": self.render_variable, } block_token._token_types = [] span_token._token_types = [Expr, Number, Variable, Whitespace] self.env = ChainMap({ "define": self.define, "lambda": lambda expr_token, *body: Procedure(expr_token, body, self.env), "+": lambda x, y: self.render(x) + self.render(y), "-": lambda x, y: self.render(x) - self.render(y), "*": lambda x, y: self.render(x) * self.render(y), "/": lambda x, y: self.render(x) / self.render(y), "<": lambda x, y: self.render(x) < self.render(y), ">": lambda x, y: self.render(x) > self.render(y), "<=": lambda x, y: self.render(x) <= self.render(y), ">=": lambda x, y: self.render(x) >= self.render(y), "=": lambda x, y: self.render(x) == self.render(y), "true": True, "false": False, "cons": lambda x, y: (self.render(x), self.render(y)), "car": lambda pair: self.render(pair)[0], "cdr": lambda pair: self.render(pair)[1], "and": lambda *args: all(map(self.render, args)), "or": lambda *args: any(map(self.render, args)), "not": lambda x: not self.render(x), "if": lambda cond, true, false: self.render(true) if self.render(cond) else self.render(false), "cond": self.cond, "null": None, "null?": lambda x: self.render(x) is None, "list": lambda *args: reduce(lambda x, y: (y, x), map(self.render, reversed(args)), None), "display": lambda *args: print(*map(self.render, args)), }) def render_inner(self, token): result = None for child in token.children: result = self.render(child) return result def render_expr(self, token): proc, *args = token.children proc = self.render(proc) return self.apply(proc, args) if isinstance(proc, Procedure) else proc(*args) def render_number(self, token): return token.number def render_variable(self, token): return self.env[token.name] def define(self, *args): if len(args) == 2: name_token, val_token = args self.env[name_token.name] = self.render(val_token) else: name_token, expr_token, *body = args self.env[name_token.name] = Procedure(expr_token, body, self.env) def cond(self, *exprs): for expr in exprs: test, value = expr.children if test == 'else' and 'else' not in self.env: return self.render(value) if self.render(test): return self.render(value) def apply(self, proc, args): old_env = self.env self.env = proc.env.new_child() try: for param, arg in zip(proc.params, args): self.env[param] = self.render(arg) result = None for expr in proc.body: result = self.render(expr) finally: self.env = old_env return result if __name__ == '__main__': with Scheme() as renderer: prog = ["(define x (* 2 21))", "x"] print(renderer.render(Program(prog))) mistletoe-0.8.2/contrib/toc_renderer.py000066400000000000000000000043571420102462200202150ustar00rootroot00000000000000""" Table of contents support for mistletoe. See `if __name__ == '__main__'` section for sample usage. """ import re from mistletoe.html_renderer import HTMLRenderer from mistletoe import block_token class TOCRenderer(HTMLRenderer): """ Extends HTMLRenderer class for table of contents support. Args: depth (int): the maximum level of heading to be included in TOC; omit_title (bool): whether to ignore tokens where token.level == 1; filter_conds (list): when any of these functions evaluate to true, current heading will not be included; extras (list): allows subclasses to add even more custom tokens. """ def __init__(self, depth=5, omit_title=True, filter_conds=[], *extras): super().__init__(*extras) self._headings = [] self.depth = depth self.omit_title = omit_title self.filter_conds = filter_conds @property def toc(self): """ Returns table of contents as a block_token.List instance. """ from mistletoe.block_token import List def get_indent(level): if self.omit_title: level -= 1 return ' ' * 4 * (level - 1) def build_list_item(heading): level, content = heading template = '{indent}- {content}\n' return template.format(indent=get_indent(level), content=content) lines = [build_list_item(heading) for heading in self._headings] items = block_token.tokenize(lines) return items[0] def render_heading(self, token): """ Overrides super().render_heading; stores rendered heading first, then returns it. """ rendered = super().render_heading(token) content = self.parse_rendered_heading(rendered) if not (self.omit_title and token.level == 1 or token.level > self.depth or any(cond(content) for cond in self.filter_conds)): self._headings.append((token.level, content)) return rendered @staticmethod def parse_rendered_heading(rendered): """ Helper method; converts rendered heading to plain text. """ return re.sub(r'<.+?>', '', rendered) mistletoe-0.8.2/contrib/xwiki20_renderer.py000066400000000000000000000231471420102462200207230ustar00rootroot00000000000000# The MIT License # # Permission is hereby granted, free of charge, to any person obtaining a copy of # this software and associated documentation files (the "Software"), to deal in # the Software without restriction, including without limitation the rights to # use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies # of the Software, and to permit persons to whom the Software is furnished to do # so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # import html from itertools import chain from mistletoe import block_token, span_token from mistletoe.base_renderer import BaseRenderer import sys class XWiki20Renderer(BaseRenderer): """ XWiki syntax 2.0 renderer class. See mistletoe.base_renderer module for more info. """ def __init__(self, *extras): """ Args: extras (list): allows subclasses to add even more custom tokens. """ self.listTokens = [] self.lastChildOfQuotes = [] self.firstChildOfListItems = [] localExtras = [block_token.HTMLBlock, span_token.HTMLSpan, span_token.XWikiBlockMacroStart, span_token.XWikiBlockMacroEnd] super().__init__(*chain(localExtras, extras)) def render_strong(self, token): template = '**{}**' return template.format(self.render_inner(token)) def render_emphasis(self, token): template = '//{}//' return template.format(self.render_inner(token)) def render_inline_code(self, token): # Note: XWiki also offers preformatted text syntax ('##{}##') as a shorter alternative. # We would have to escape the raw text when using it. template = '{{{{code}}}}{}{{{{/code}}}}' return template.format(self.render_raw_text(token.children[0], False)) def render_strikethrough(self, token): template = '--{}--' return template.format(self.render_inner(token)) def render_image(self, token): template = '[[image:{src}]]' inner = self.render_inner(token) return template.format(src=token.src) def render_link(self, token): template = '[[{inner}>>{target}]]' target = escape_url(token.target) inner = self.render_inner(token) return template.format(target=target, inner=inner) def render_auto_link(self, token): template = '[[{target}]]' target = escape_url(token.target) return template.format(target=target) def render_escape_sequence(self, token): return '~' + self.render_inner(token) def render_raw_text(self, token, escape=True): return (token.content.replace('~', '~~') # Note: It's probably better to leave potential XWiki macros as-is, i. e. don't escape their markers: #.replace('{{', '~{{').replace('}}', '~}}') .replace('[[', '~[[').replace(']]', '~]]') .replace('**', '~**').replace('//', '~//') .replace('##', '~##').replace('--', '~--') ) if escape else token.content def render_x_wiki_block_macro_start(self, token): return token.content + '\n' def render_x_wiki_block_macro_end(self, token): return '\n' + token.content def render_html_span(self, token): # XXX: HTMLSpan parses (contains) only individual opening and closing tags # => no easy way to wrap the whole HTML code into {{html}} like this: # # template = '{{{{html wiki="true"}}}}{}{{{{/html}}}}' # return template.format(token.content) # # => Users must do this themselves after the conversion. return token.content def render_html_block(self, token): template = '{{{{html wiki="true"}}}}\n{}\n{{{{/html}}}}' + self._block_eol(token) return template.format(token.content) def render_heading(self, token): template = '{level} {inner} {level}' inner = self.render_inner(token) return template.format(level='=' * token.level, inner=inner) + self._block_eol(token) def render_quote(self, token): self.lastChildOfQuotes.append(token.children[-1]) inner = self.render_inner(token) del (self.lastChildOfQuotes[-1]) return (''.join(map(lambda line: '>{}{}'.format('' if line.startswith('>') else ' ', line), inner.splitlines(keepends=True))) + self._block_eol(token)[0:-1]) def render_paragraph(self, token): return '{}'.format(self.render_inner(token)) + self._block_eol(token) def render_block_code(self, token): template = '{{{{code{attr}}}}}\n{inner}{{{{/code}}}}' + self._block_eol(token) if token.language: attr = ' language="{}"'.format(token.language) else: attr = '' inner = self.render_raw_text(token.children[0], False) return template.format(attr=attr, inner=inner) def render_list(self, token): inner = self.render_inner(token) return inner + self._block_eol(token)[0:-1] def render_list_item(self, token): template = '{prefix} {inner}\n' prefix = ''.join(self.listTokens) if '1' in self.listTokens: prefix += '.' self.firstChildOfListItems.append(token.children[0]) inner = self.render_inner(token) del (self.firstChildOfListItems[-1]) result = template.format(prefix=prefix, inner=inner.rstrip()) return result def render_inner(self, token): if isinstance(token, block_token.List): if token.start: self.listTokens.append('1') else: self.listTokens.append('*') rendered = [self.render(child) for child in token.children] wrap = False if isinstance(token, block_token.BlockToken) and len(token.children) > 1: # test what follows after the 1st child of this block token - wrap it to a XWiki block right after the 1st child if necessary for child in token.children[1:]: if isinstance(token, block_token.ListItem) and not isinstance(child, block_token.List): # Note: Nested list within a list item is OK, because it does its own wrapping if necessary. wrap = True break if isinstance(token, (block_token.TableCell)) and isinstance(child, block_token.BlockToken): # Note: By-design, Markdown doesn't support multiple lines in one cell, but they can be enforced by using HTML. # See e. g. https://stackoverflow.com/questions/19950648/how-to-write-lists-inside-a-markdown-table. wrap = True break if isinstance(token, block_token.List): del (self.listTokens[-1]) return (''.join(rendered) if not wrap else '{head}(((\n{tail}\n)))\n'.format(head=rendered[0].rstrip(), tail=''.join(rendered[1:]).rstrip())) def render_table(self, token): # Copied from JIRARenderer... # # This is actually gross and I wonder if there's a better way to do it. # # The primary difficulty seems to be passing down alignment options to # reach individual cells. template = '{inner}\n' if hasattr(token, 'header'): head_template = '{inner}' header = token.header head_inner = self.render_table_row(header, True) head_rendered = head_template.format(inner=head_inner) else: head_rendered = '' body_template = '{inner}' body_inner = self.render_inner(token) body_rendered = body_template.format(inner=body_inner) return template.format(inner=head_rendered+body_rendered) def render_table_row(self, token, is_header=False): if is_header: template = '{inner}\n' else: template = '{inner}\n' inner = ''.join([self.render_table_cell(child, is_header) for child in token.children]) return template.format(inner=inner) def render_table_cell(self, token, in_header=False): if in_header: template = '|={inner}' else: template = '|{inner}' inner = self.render_inner(token) return template.format(inner=inner) @staticmethod def render_thematic_break(token): return '----\n' @staticmethod def render_line_break(token): return ' ' if token.soft else '\n' def render_document(self, token): self.footnotes.update(token.footnotes) return self.render_inner(token) def _block_eol(self, token): return ('\n' if ((len(self.firstChildOfListItems) > 0 and token is self.firstChildOfListItems[-1]) or (len(self.lastChildOfQuotes) > 0 and token is self.lastChildOfQuotes[-1])) else '\n\n') def escape_url(raw): """ Escape urls to prevent code injection craziness. (Hopefully.) """ from urllib.parse import quote return quote(html.unescape(raw), safe='/#:()*?=%@+,&;') mistletoe-0.8.2/dev-guide.md000066400000000000000000000150621420102462200157160ustar00rootroot00000000000000

Developer's Guide

This document describes usage of mistletoe and its API from the developer's point of view. Understanding the AST --------------------- When a markdown document gets parsed by mistletoe, the result is represented as an "abstract syntax tree" (AST), stored in an instance of `Document`. This object contains a hierarchy of all the various tokens which were recognized during the parsing process. In order to see what exactly gets parsed, one can simply use the `ASTRenderer` on a given markdown input, for example: ```sh mistletoe text.md --renderer mistletoe.ast_renderer.ASTRenderer ``` Say that the input file contains for example: ```markdown # Heading 1 text # Heading 2 [link](https://www.example.com) ``` Then we will get this JSON output from the AST renderer: ```json { "type": "Document", "footnotes": {}, "children": [ { "type": "Heading", "level": 1, "children": [ { "type": "RawText", "content": "Heading 1" } ] }, { "type": "Paragraph", "children": [ { "type": "RawText", "content": "text" } ] }, { "type": "Heading", "level": 1, "children": [ { "type": "RawText", "content": "Heading 2" } ] }, { "type": "Paragraph", "children": [ { "type": "Link", "target": "https://www.example.com", "title": "", "children": [ { "type": "RawText", "content": "link" } ] } ] } ] } ``` When passing this tree to a renderer, it is recursively traversed and methods corresponding to individual token types get called on the renderer in order to create the output in the desired format. Creating a custom renderer -------------------------- Here's an example of how to add GitHub-style wiki links to the parsing process, and provide a renderer for this new token. ### A new token GitHub wiki links are span-level tokens, meaning that they reside inline, and don't really look like chunky paragraphs. To write a new span-level token, all we need to do is make a subclass of `SpanToken`: ```python from mistletoe.span_token import SpanToken class GithubWiki(SpanToken): pass ``` mistletoe uses regular expressions to search for span-level tokens in the parsing process. As a refresher, GitHub wiki looks something like this: `[[alternative text | target]]`. We define a class variable, `pattern`, that stores the compiled regex: ```python class GithubWiki(SpanToken): pattern = re.compile(r"\[\[ *(.+?) *\| *(.+?) *\]\]") def __init__(self, match): pass ``` The regex will be picked up by `SpanToken.find`, which is used by the tokenizer to find all tokens of its kind in the document. If regexes are too limited for your use case, consider overriding the `find` method; it should return a list of all token occurrences. Three other class variables are available for our custom token class, and their default values are shown below: ```python class SpanToken: parse_group = 1 parse_inner = True precedence = 5 ``` Note that alternative text can also contain other span-level tokens. For example, `[[*alt*|link]]` is a GitHub link with an `Emphasis` token as its child. To parse child tokens, `parse_inner` should be set to `True` (the default value in this case), and `parse_group` should correspond to the match group in which child tokens might occur (also the default value, 1, in this case). Once these two class variables are set correctly, `GithubWiki.children` attribute will automatically be set to the list of child tokens. Note that there is no need to manually set this attribute, unlike previous versions of mistletoe. Lastly, the `SpanToken` constructors take a regex match object as its argument. We can simply store off the `target` attribute from `match_obj.group(2)`. ```python from mistletoe.span_token import SpanToken class GithubWiki(SpanToken): pattern = re.compile(r"\[\[ *(.+?) *\| *(.+?) *\]\]") def __init__(self, match_obj): self.target = match_obj.group(2) ``` There you go: a new token in 5 lines of code. ### Side note about precedence Normally there is no need to override the `precedence` value of a custom token. The default value is the same as `InlineCode`, `AutoLink` and `HTMLSpan`, which means that whichever token comes first will be parsed. In our case: ```markdown `code with [[ text` | link ]] ``` ... will be parsed as: ```html code with [[ text | link ]] ``` If we set `GithubWiki.precedence = 6`, we have: ```html `code with text` ``` ### A new renderer Adding a custom token to the parsing process usually involves a lot of nasty implementation details. Fortunately, mistletoe takes care of most of them for you. Simply passing your custom token class to `super().__init__()` does the trick: ```python from mistletoe.html_renderer import HTMLRenderer class GithubWikiRenderer(HTMLRenderer): def __init__(self): super().__init__(GithubWiki) ``` We then only need to tell mistletoe how to render our new token: ```python def render_github_wiki(self, token): template = '{inner}' target = token.target inner = self.render_inner(token) return template.format(target=target, inner=inner) ``` Cleaning up, we have our new renderer class: ```python from mistletoe.html_renderer import HTMLRenderer, escape_url class GithubWikiRenderer(HTMLRenderer): def __init__(self): super().__init__(GithubWiki) def render_github_wiki(self, token): template = '{inner}' target = escape_url(token.target) inner = self.render_inner(token) return template.format(target=target, inner=inner) ``` ### Take it for a spin? It is preferred that all mistletoe's renderers be used as context managers. This is to ensure that your custom tokens are cleaned up properly, so that you can parse other Markdown documents with different token types in the same program. ```python from mistletoe import Document from contrib.github_wiki import GithubWikiRenderer with open('foo.md', 'r') as fin: with GithubWikiRenderer() as renderer: rendered = renderer.render(Document(fin)) ``` For more info, take a look at the `base_renderer` module in mistletoe. The docstrings might give you a more granular idea of customizing mistletoe to your needs. mistletoe-0.8.2/docs/000077500000000000000000000000001420102462200144475ustar00rootroot00000000000000mistletoe-0.8.2/docs/CNAME000066400000000000000000000000241420102462200152110ustar00rootroot00000000000000mistletoe.afteryu.memistletoe-0.8.2/docs/README.md000066400000000000000000000002321420102462200157230ustar00rootroot00000000000000DEPRECATED: This folder used to be the source for project pages. It is not used anymore though. This folder's content is updated by running `make docs`. mistletoe-0.8.2/docs/__init__.py000066400000000000000000000040311420102462200165560ustar00rootroot00000000000000from mistletoe import Document, HTMLRenderer, __version__ INCLUDE = {'README.md': 'index.html', 'CONTRIBUTING.md': 'contributing.html'} METADATA = """ mistletoe{} """ class DocRenderer(HTMLRenderer): def render_link(self, token): return super().render_link(self._replace_link(token)) def render_document(self, token, name="README.md"): pattern = "{}{}" self.footnotes.update(token.footnotes) for filename, new_link in getattr(self, 'files', {}).items(): for k, v in self.footnotes.items(): if v == filename: self.footnotes[k] = new_link subtitle = ' | {}'.format('version ' + __version__ if name == 'README.md' else name.split('.')[0].lower()) return pattern.format(METADATA.format(subtitle), self.render_inner(token)) def _replace_link(self, token): token.target = getattr(self, 'files', {}).get(token.target, token.target) return token def build(files=None): files = files or INCLUDE for f in files: with open(f, 'r', encoding='utf-8') as fin: rendered_file = 'docs/' + files[f] with open(rendered_file, 'w+', encoding='utf-8') as fout: with DocRenderer() as renderer: renderer.files = files print(renderer.render_document(Document(fin), f), file=fout) mistletoe-0.8.2/docs/__main__.py000066400000000000000000000001271420102462200165410ustar00rootroot00000000000000import sys from docs import build build(sys.argv[1:] if len(sys.argv) > 1 else None) mistletoe-0.8.2/docs/contributing.html000066400000000000000000000127021420102462200200460ustar00rootroot00000000000000 mistletoe | contributing

Contributing

You've seen mistletoe: it branches off in all directions, bringing people together. We would love to see what you can make of mistletoe, which direction you would take it to. Or maybe you can discover some Nargles, which, by the way, totally exists.

The following instructions serve as guidelines, and you should use your best judgements when employing them.

Getting started

Refer to the README for install instructions. Since you're going to mess with the code, it's prefered that you clone the repo directly.

Check back on the dev branch regularly to avoid redoing work that others might have done. The master branch is updated only when features on the dev branch are stabilized somewhat.

Things you can do

Introducing new features

It is suggested that you open an issue first before working on new features. Include your reasons, use case, and maybe plans for implementation. That way, we have a better idea of what you'll be working on, and can hopefully avoid collision. Your pull request may also get merged much faster.

Fixing bugs

Before you post an issue, try narrowing the problem down to the smallest component possible. For example, if an InlineCode token is not parsed correctly, include only the paragraph that introduce the error, not the entire document.

You might find mistletoe's interactive mode handy when tracking down bugs. Type in your input, and you immediately see how mistletoe handles it. I created it just for this purpose. To use it, run mistletoe (or python3 mistletoe) in your shell without arguments.

Markdown is a very finicky document format to parse, so if something does not work as intended, it's probably my fault and not yours.

Writing documentations

The creator might not the best person to write documentations; the users, knowing all the painpoints, have a better idea of actual use cases and possible things that can go wrong.

Go to the mistletoe wiki and write up your own topic. Alternatively, write docstrings or comments for functions that are missing them. mistletoe generally follows the Google Python Style Guide to format comments.

Writing code

Commit messages

  • minimal cosmetic changes are fine to mix in with your commits, but try feel guilty when you do that, and if it's not too big of a hassle, break them into two commits.

  • give clear, instructive commit messages. Try using phrases like "added XXX feature" or "fixed XXX (#42)".

  • if you find yourself cramming too many things into one commit message, you should probably break them into multiple commits.

  • emojis are awesome. Use them like this:

| Emoji | Description | | :---: | :------------------------------ | | 📚 | Update documentation. | | 🐎 | Performance improvements. | | 💡 | New features. | | 🐛 | Bug fixes. | | 🚨 | Under construction. | | ☕️ | Refactoring / cosmetic changes. | | 🌎 | Internationalization. |

Style guide

Here's the obligatory PEP8 link, but here's a much shorter list of things to be aware of:

  • mistletoe uses CamelCase for classnames, snake_case for functions and methods;
  • mistletoe uses one blank line between classes and functions, even global ones, despite PEP8's suggestion to the contrary.
  • mistletoe follows the eighty-character rule: if you find your line to be too lengthy, try giving variable names to expressions, and break it up that way. That said, it's okay to go over the charater limit occasionally.
  • mistletoe uses four spaces instead of a tab to indent. For vim users, include set ts=4 sw=4 ai et in your .vimrc.

Apart from that, stay consistent with the coding style around you. But don't get boggled down by this: if you have a genius idea, I'd love to clean up for you; write down your genius idea first.

Get in touch

I tweet @mi_before_yu. Also yell at me over email.

mistletoe-0.8.2/docs/index.html000066400000000000000000000316651420102462200164570ustar00rootroot00000000000000 mistletoe | version 0.5.2

mistletoe

Build Status Coverage Status PyPI is wheel

mistletoe is a Markdown parser in pure Python, designed to be fast, modular and fully customizable.

mistletoe is not simply a Markdown-to-HTML transpiler. It is designed, from the start, to parse Markdown into an abstract syntax tree. You can swap out renderers for different output formats, without touching any of the core components.

Remember to spell mistletoe in lowercase!

Features

  • Fast: mistletoe is as fast as the fastest implementation currently available: that is, over 4 times faster than Python-Markdown, and much faster than Python-Markdown2. See the performance section for details.

  • Modular: mistletoe is designed with modularity in mind. Its initial goal is to provide a clear and easy API to extend upon.

  • Customizable: as of now, mistletoe can render Markdown documents to LaTeX, HTML and an abstract syntax tree out of the box. Writing a new renderer for mistletoe is a relatively trivial task.

Installation

mistletoe requires Python 3.3 and above, including Python 3.7, the current development branch. It is also tested on PyPy 5.8.0. Install mistletoe with pip:

pip3 install mistletoe

Alternatively, clone the repo:

git clone https://github.com/miyuchina/mistletoe.git
cd mistletoe
pip3 install -e .

See the contributing doc for how to contribute to mistletoe.

Usage

Basic usage

Here's how you can use mistletoe in a Python script:

import mistletoe

with open('foo.md', 'r') as fin:
    rendered = mistletoe.markdown(fin)

mistletoe.markdown() uses mistletoe's default settings: allowing HTML mixins and rendering to HTML. The function also accepts an additional argument renderer. To produce LaTeX output:

import mistletoe
from mistletoe.latex_renderer import LaTeXRenderer

with open('foo.md', 'r') as fin:
    rendered = mistletoe.markdown(fin, LaTeXRenderer)

Finally, here's how you would manually specify extra tokens and a renderer for mistletoe. In the following example, we use HTMLRenderer to render the AST, which adds HTMLBlock and HTMLSpan to the normal parsing process.

from mistletoe import Document, HTMLRenderer

with open('foo.md', 'r') as fin:
    with HTMLRenderer() as renderer:
        rendered = renderer.render(Document(fin))

From the command-line

pip installation enables mistletoe's commandline utility. Type the following directly into your shell:

mistletoe foo.md

This will transpile foo.md into HTML, and dump the output to stdout. To save the HTML, direct the output into a file:

mistletoe foo.md > out.html

You can pass in custom renderers by including the full path to your renderer class after a -r or --renderer flag:

mistletoe foo.md --renderer custom_renderer.CustomRenderer

Running mistletoe without specifying a file will land you in interactive mode. Like Python's REPL, interactive mode allows you to test how your Markdown will be interpreted by mistletoe:

mistletoe [version 0.5.2] (interactive)
Type Ctrl-D to complete input, or Ctrl-C to exit.
>>> some **bold text**
... and some *italics*
... ^D
<html>
<body>
<p>some <strong>bold text</strong> and some <em>italics</em></p>
</body>
</html>
>>>

The interactive mode also accepts the --renderer flag.

Performance

mistletoe is the fastest Markdown parser implementation available in pure Python; that is, on par with mistune. Try the benchmarks yourself by running:

python3 test/benchmark.py

One of the significant bottlenecks of mistletoe compared to mistune, however, is the function overhead. Because, unlike mistune, mistletoe chooses to split functionality into modules, function lookups can take significantly longer than mistune.

To boost the performance further, it is suggested to use PyPy with mistletoe. Benchmark results show that on PyPy, mistletoe is about twice as fast as mistune:

$ pypy3 test/benchmark.py mistune mistletoe
Test document: test/samples/syntax.md
Test iterations: 1000
Running tests with mistune, mistletoe...
========================================
mistune: 13.524028996936977
mistletoe: 6.477352762129158

The above result was achieved on PyPy 5.8.0-beta0, on a 13-inch Retina MacBook Pro (Early 2015).

Developer's Guide

Here's an example to add GitHub-style wiki links to the parsing process, and provide a renderer for this new token.

A new token

GitHub wiki links are span-level tokens, meaning that they reside inline, and don't really look like chunky paragraphs. To write a new span-level token, all we need to do is make a subclass of SpanToken:

from mistletoe.span_token import SpanToken

class GithubWiki(SpanToken):
    pass

mistletoe uses regular expressions to search for span-level tokens in the parsing process. As a refresher, GitHub wiki looks something like this: [[alternative text | target]]. We define a class variable, pattern, that stores the compiled regex:

class GithubWiki(SpanToken):
    pattern = re.compile(r"\[\[ *(.+?) *\| *(.+?) *\]\]")
    def __init__(self, match_obj):
        pass

For spiritual guidance on regexes, refer to xkcd classics. For an actual representation of this author parsing Markdown with regexes, refer to this brilliant meme by Greg Hendershott.

mistletoe's span-level tokenizer will search for our pattern. When it finds a match, it will pass in the match object as argument into our constructor. We have defined our regex so that the first match group is the alternative text, and the second one is the link target.

Note that alternative text can also contain other span-level tokens. For example, [[*alt*|link]] is a GitHub link with an Emphasis token as its child. To parse child tokens, simply pass match_obj to the super constructor (which assumes children to be in match_obj.group(1)), and save off all the additional attributes we need:

from mistletoe.span_token import SpanToken

class GithubWiki(SpanToken):
    pattern = re.compile(r"\[\[ *(.+?) *\| *(.+?) *\]\]")
    def __init__(self, match_obj):
        super().__init__(match_obj)
        self.target = match_obj.group(2)

There you go: a new token in 7 lines of code.

A new renderer

Adding a custom token to the parsing process usually involves a lot of nasty implementation details. Fortunately, mistletoe takes care of most of them for you. Simply pass your custom token class to super().__init__() does the trick:

from mistletoe.html_renderer import HTMLRenderer

class GithubWikiRenderer(HTMLRenderer):
    def __init__(self):
        super().__init__(GithubWiki)

We then only need to tell mistletoe how to render our new token:

def render_github_wiki(self, token):
    template = '<a href="{target}">{inner}</a>'
    target = token.target
    inner = self.render_inner(token)
    return template.format(target=target, inner=inner)

Cleaning up, we have our new renderer class:

from mistletoe.html_renderer import HTMLRenderer, escape_url

class GithubWikiRenderer(HTMLRenderer):
    def __init__(self):
        super().__init__(GithubWiki)

    def render_github_wiki(self, token):
        template = '<a href="{target}">{inner}</a>'
        target = escape_url(token.target)
        inner = self.render_inner(token)
        return template.format(target=target, inner=inner)

Take it for a spin?

It is preferred that all mistletoe's renderers be used as context managers. This is to ensure that your custom tokens are cleaned up properly, so that you can parse other Markdown documents with different token types in the same program.

from mistletoe import Document
from contrib.github_wiki import GithubWikiRenderer

with open('foo.md', 'r') as fin:
    with GithubWikiRenderer() as renderer:
        rendered = renderer.render(Document(fin))

For more info, take a look at the base_renderer module in mistletoe. The docstrings might give you a more granular idea of customizing mistletoe to your needs.

Why mistletoe?

For me, the question becomes: why not mistune? My original motivation really has nothing to do with starting a competition. Here's a list of reasons I created mistletoe in the first place:

  • I am interested in a Markdown-to-LaTeX transpiler in Python.
  • I want to write more Python.
  • "How hard could it be?"
  • "For fun," says David Beazley.

Here's two things mistune inspired mistletoe to do:

  • Markdown parsers should be fast, and other parser implementations in Python leaves much to be desired.
  • A parser implementation for Markdown does not need to restrict itself to one flavor of Markdown.

Here's two things mistletoe does differently from mistune:

  • Per its readme, mistune will always be a single-file script. mistletoe breaks its functionality into modules.
  • mistune, as of now, can only render Markdown into HTML. It is relatively trivial to write a new renderer for mistletoe.
  • Unlike mistune, mistletoe is pushing for some extent of spec compliance with CommonMark.

The implications of these are quite profound, and there's no definite this-is-better-than-that answer. Mistune is near perfect if one wants what it provides: I have used mistune extensively in the past, and had a great experience. If you want more control, however, give mistletoe a try.

Copyright & License

  • mistletoe's logo uses artwork by Daniele De Santis, under CC BY 3.0.
  • mistletoe is released under MIT.
mistletoe-0.8.2/docs/style.css000066400000000000000000000012771420102462200163300ustar00rootroot00000000000000body { width: 60%; margin: 2em auto; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; line-height: 1.5; color: #24292e; } h1, h2 { border-bottom: 1px solid #eaecef; padding-bottom: 0.3em; } a { color: #0366d6; text-decoration: none; } code { padding: 0.2em 0.4em; margin: 0; background-color: rgba(27,31,35,0.05); border-radius: 3px; font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo, Courier, monospace; font-size: 85%; } @media screen and (max-width: 1000px) { body { width: 75%; } } @media screen and (max-width: 700px) { body { width: 90%; } } mistletoe-0.8.2/makefile000066400000000000000000000006571420102462200152270ustar00rootroot00000000000000PYTHON_EXEC=python3 .PHONY: run test coverage integration benchmark docs run: ${PYTHON_EXEC} -m mistletoe test: ${PYTHON_EXEC} -m unittest coverage: . venv/bin/activate && \ ${PYTHON_EXEC} -m coverage run -m unittest && \ coverage report && \ deactivate integration: ./test/test_ci.sh 1 benchmark: ${PYTHON_EXEC} test/benchmark.py specification: ${PYTHON_EXEC} -m test.specification docs: ${PYTHON_EXEC} -m docs mistletoe-0.8.2/mistletoe/000077500000000000000000000000001420102462200155245ustar00rootroot00000000000000mistletoe-0.8.2/mistletoe/__init__.py000066400000000000000000000013471420102462200176420ustar00rootroot00000000000000""" Make mistletoe easier to import. """ __version__ = "0.8.2" __all__ = ['html_renderer', 'ast_renderer', 'block_token', 'block_tokenizer', 'span_token', 'span_tokenizer'] from mistletoe.block_token import Document from mistletoe.base_renderer import BaseRenderer from mistletoe.html_renderer import HTMLRenderer def markdown(iterable, renderer=HTMLRenderer): """ Converts markdown input to the output supported by the given renderer. If no renderer is supplied, ``HTMLRenderer`` is used. Note that extra token types supported by the given renderer are automatically (and temporarily) added to the parsing process. """ with renderer() as renderer: return renderer.render(Document(iterable)) mistletoe-0.8.2/mistletoe/__main__.py000066400000000000000000000003331420102462200176150ustar00rootroot00000000000000""" Make mistletoe runnable as a script with default settings. """ import sys from mistletoe import cli def main(): """ Entry point. """ cli.main(sys.argv[1:]) if __name__ == "__main__": main() mistletoe-0.8.2/mistletoe/_html.py000066400000000000000000000113371420102462200172060ustar00rootroot00000000000000""" Taken from Python 3.4 standard library. Conforms to its original license. General functions for HTML manipulation. """ import re as _re from html.entities import html5 as _html5 __all__ = ['escape', 'unescape'] def escape(s, quote=True): """ Replace special characters "&", "<" and ">" to HTML-safe sequences. If the optional flag quote is true (the default), the quotation mark characters, both double quote (") and single quote (') characters are also translated. """ s = s.replace("&", "&") # Must be done first! s = s.replace("<", "<") s = s.replace(">", ">") if quote: s = s.replace('"', """) s = s.replace('\'', "'") return s # see http://www.w3.org/TR/html5/syntax.html#tokenizing-character-references _invalid_charrefs = { 0x00: '\ufffd', # REPLACEMENT CHARACTER 0x0d: '\r', # CARRIAGE RETURN 0x80: '\u20ac', # EURO SIGN 0x81: '\x81', # 0x82: '\u201a', # SINGLE LOW-9 QUOTATION MARK 0x83: '\u0192', # LATIN SMALL LETTER F WITH HOOK 0x84: '\u201e', # DOUBLE LOW-9 QUOTATION MARK 0x85: '\u2026', # HORIZONTAL ELLIPSIS 0x86: '\u2020', # DAGGER 0x87: '\u2021', # DOUBLE DAGGER 0x88: '\u02c6', # MODIFIER LETTER CIRCUMFLEX ACCENT 0x89: '\u2030', # PER MILLE SIGN 0x8a: '\u0160', # LATIN CAPITAL LETTER S WITH CARON 0x8b: '\u2039', # SINGLE LEFT-POINTING ANGLE QUOTATION MARK 0x8c: '\u0152', # LATIN CAPITAL LIGATURE OE 0x8d: '\x8d', # 0x8e: '\u017d', # LATIN CAPITAL LETTER Z WITH CARON 0x8f: '\x8f', # 0x90: '\x90', # 0x91: '\u2018', # LEFT SINGLE QUOTATION MARK 0x92: '\u2019', # RIGHT SINGLE QUOTATION MARK 0x93: '\u201c', # LEFT DOUBLE QUOTATION MARK 0x94: '\u201d', # RIGHT DOUBLE QUOTATION MARK 0x95: '\u2022', # BULLET 0x96: '\u2013', # EN DASH 0x97: '\u2014', # EM DASH 0x98: '\u02dc', # SMALL TILDE 0x99: '\u2122', # TRADE MARK SIGN 0x9a: '\u0161', # LATIN SMALL LETTER S WITH CARON 0x9b: '\u203a', # SINGLE RIGHT-POINTING ANGLE QUOTATION MARK 0x9c: '\u0153', # LATIN SMALL LIGATURE OE 0x9d: '\x9d', # 0x9e: '\u017e', # LATIN SMALL LETTER Z WITH CARON 0x9f: '\u0178', # LATIN CAPITAL LETTER Y WITH DIAERESIS } _invalid_codepoints = { # 0x0001 to 0x0008 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, # 0x000E to 0x001F 0xe, 0xf, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, # 0x007F to 0x009F 0x7f, 0x80, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, 0x89, 0x8a, 0x8b, 0x8c, 0x8d, 0x8e, 0x8f, 0x90, 0x91, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98, 0x99, 0x9a, 0x9b, 0x9c, 0x9d, 0x9e, 0x9f, # 0xFDD0 to 0xFDEF 0xfdd0, 0xfdd1, 0xfdd2, 0xfdd3, 0xfdd4, 0xfdd5, 0xfdd6, 0xfdd7, 0xfdd8, 0xfdd9, 0xfdda, 0xfddb, 0xfddc, 0xfddd, 0xfdde, 0xfddf, 0xfde0, 0xfde1, 0xfde2, 0xfde3, 0xfde4, 0xfde5, 0xfde6, 0xfde7, 0xfde8, 0xfde9, 0xfdea, 0xfdeb, 0xfdec, 0xfded, 0xfdee, 0xfdef, # others 0xb, 0xfffe, 0xffff, 0x1fffe, 0x1ffff, 0x2fffe, 0x2ffff, 0x3fffe, 0x3ffff, 0x4fffe, 0x4ffff, 0x5fffe, 0x5ffff, 0x6fffe, 0x6ffff, 0x7fffe, 0x7ffff, 0x8fffe, 0x8ffff, 0x9fffe, 0x9ffff, 0xafffe, 0xaffff, 0xbfffe, 0xbffff, 0xcfffe, 0xcffff, 0xdfffe, 0xdffff, 0xefffe, 0xeffff, 0xffffe, 0xfffff, 0x10fffe, 0x10ffff } def _replace_charref(s): s = s.group(1) if s[0] == '#': # numeric charref if s[1] in 'xX': num = int(s[2:].rstrip(';'), 16) else: num = int(s[1:].rstrip(';')) if num in _invalid_charrefs: return _invalid_charrefs[num] if 0xD800 <= num <= 0xDFFF or num > 0x10FFFF: return '\uFFFD' if num in _invalid_codepoints: return '' return chr(num) else: # named charref if s in _html5: return _html5[s] # find the longest matching name (as defined by the standard) for x in range(len(s)-1, 1, -1): if s[:x] in _html5: return _html5[s[:x]] + s[x:] else: return '&' + s _charref = _re.compile(r'&(#[0-9]+;?' r'|#[xX][0-9a-fA-F]+;?' r'|[^\t\n\f <&#;]{1,32};?)') def unescape(s): """ Convert all named and numeric character references (e.g. >, >, &x3e;) in the string s to the corresponding unicode characters. This function uses the rules defined by the HTML 5 standard for both valid and invalid character references, and the list of HTML 5 named character references defined in html.entities.html5. """ if '&' not in s: return s return _charref.sub(_replace_charref, s) mistletoe-0.8.2/mistletoe/ast_renderer.py000066400000000000000000000022251420102462200205540ustar00rootroot00000000000000""" Abstract syntax tree renderer for mistletoe. """ import json from mistletoe.base_renderer import BaseRenderer class ASTRenderer(BaseRenderer): def render(self, token): """ Returns the string representation of the AST. Overrides super().render. Delegates the logic to get_ast. """ return json.dumps(get_ast(token), indent=2) + '\n' def __getattr__(self, name): return lambda token: '' def get_ast(token): """ Recursively unrolls token attributes into dictionaries (token.children into lists). Returns: a dictionary of token's attributes. """ node = {} # Python 3.6 uses [ordered dicts] [1]. # Put in 'type' entry first to make the final tree format somewhat # similar to [MDAST] [2]. # # [1]: https://docs.python.org/3/whatsnew/3.6.html # [2]: https://github.com/syntax-tree/mdast node['type'] = token.__class__.__name__ node.update(token.__dict__) if 'header' in node: node['header'] = get_ast(node['header']) if 'children' in node: node['children'] = [get_ast(child) for child in node['children']] return node mistletoe-0.8.2/mistletoe/base_renderer.py000066400000000000000000000136211420102462200207010ustar00rootroot00000000000000""" Base class for renderers. """ import re import sys from mistletoe import block_token, span_token class BaseRenderer(object): """ Base class for renderers. All renderers should ... * ... define all render functions specified in self.render_map; * ... be a context manager (by inheriting __enter__ and __exit__); Custom renderers could ... * ... add additional tokens into the parsing process by passing custom tokens to super().__init__(); * ... add additional render functions by appending to self.render_map; Usage: Suppose SomeRenderer inherits BaseRenderer, and fin is the input file. The syntax looks something like this: >>> from mistletoe import Document >>> from some_renderer import SomeRenderer >>> with SomeRenderer() as renderer: ... rendered = renderer.render(Document(fin)) See mistletoe.html_renderer for an implementation example. Naming conventions: * The keys of self.render_map should exactly match the class name of tokens; * Render function names should be of form: "render_" + the "snake-case" form of token's class name. Attributes: render_map (dict): maps tokens to their corresponding render functions. _extras (list): a list of custom tokens to be added to the parsing process. """ _parse_name = re.compile(r"([A-Z][a-z]+|[A-Z]+(?![a-z]))") def __init__(self, *extras): self.render_map = { 'Strong': self.render_strong, 'Emphasis': self.render_emphasis, 'InlineCode': self.render_inline_code, 'RawText': self.render_raw_text, 'Strikethrough': self.render_strikethrough, 'Image': self.render_image, 'Link': self.render_link, 'AutoLink': self.render_auto_link, 'EscapeSequence': self.render_escape_sequence, 'Heading': self.render_heading, 'SetextHeading': self.render_heading, 'Quote': self.render_quote, 'Paragraph': self.render_paragraph, 'CodeFence': self.render_block_code, 'BlockCode': self.render_block_code, 'List': self.render_list, 'ListItem': self.render_list_item, 'Table': self.render_table, 'TableRow': self.render_table_row, 'TableCell': self.render_table_cell, 'ThematicBreak': self.render_thematic_break, 'LineBreak': self.render_line_break, 'Document': self.render_document, } self._extras = extras for token in extras: if issubclass(token, span_token.SpanToken): token_module = span_token else: token_module = block_token token_module.add_token(token) render_func = getattr(self, self._cls_to_func(token.__name__)) self.render_map[token.__name__] = render_func self.footnotes = {} def render(self, token): """ Grabs the class name from input token and finds its corresponding render function. Basically a janky way to do polymorphism. Arguments: token: whose __class__.__name__ is in self.render_map. """ return self.render_map[token.__class__.__name__](token) def render_inner(self, token): """ Recursively renders child tokens. Joins the rendered strings with no space in between. If newlines / spaces are needed between tokens, add them in their respective templates, or override this function in the renderer subclass, so that whitespace won't seem to appear magically for anyone reading your program. Arguments: token: a branch node who has children attribute. """ return ''.join(map(self.render, token.children)) def __enter__(self): """ Make renderer classes into context managers. """ return self def __exit__(self, exception_type, exception_val, traceback): """ Make renderer classes into context managers. Reset block_token._token_types and span_token._token_types. """ block_token.reset_tokens() span_token.reset_tokens() @classmethod def _cls_to_func(cls, cls_name): snake = '_'.join(map(str.lower, cls._parse_name.findall(cls_name))) return 'render_{}'.format(snake) @staticmethod def _tokens_from_module(module): """ Helper method; takes a module and returns a list of all token classes specified in module.__all__. Useful when custom tokens are defined in a separate module. """ return [getattr(module, name) for name in module.__all__] def render_raw_text(self, token): """ Default render method for RawText. Simply return token.content. """ return token.content def __getattr__(self, name): """ Provides a default render method for all tokens. Any token without a custom render method will simply be rendered by self.render_inner. If name does not start with 'render_', raise AttributeError as normal, for less magic during debugging. This method would only be called if the attribute requested has not been defined. Defined attributes will not be overridden. I still think this is heavy wizardry. Let me know if you would like this method removed. """ if not name.startswith('render_'): msg = '{cls} object has no attribute {name}'.format(cls=type(self).__name__, name=name) raise AttributeError(msg).with_traceback(sys.exc_info()[2]) return self.render_inner mistletoe-0.8.2/mistletoe/block_token.py000066400000000000000000000772561420102462200204110ustar00rootroot00000000000000""" Built-in block-level token classes. """ import re import sys from itertools import zip_longest import mistletoe.block_tokenizer as tokenizer from mistletoe import span_token from mistletoe.core_tokens import ( is_link_label, follows, shift_whitespace, whitespace, is_control_char, normalize_label, ) """ Tokens to be included in the parsing process, in the order specified. """ __all__ = ['BlockCode', 'Heading', 'Quote', 'CodeFence', 'ThematicBreak', 'List', 'Table', 'Footnote', 'Paragraph'] """ Stores a reference to the current document token. When parsing, footnote entries will be stored in the document by accessing this pointer. """ _root_node = None def tokenize(lines): """ A wrapper around block_tokenizer.tokenize. Pass in all block-level token constructors as arguments to block_tokenizer.tokenize. Doing so (instead of importing block_token module in block_tokenizer) avoids cyclic dependency issues, and allows for future injections of custom token classes. _token_types variable is at the bottom of this module. See also: block_tokenizer.tokenize, span_token.tokenize_inner. """ return tokenizer.tokenize(lines, _token_types) def add_token(token_cls, position=0): """ Allows external manipulation of the parsing process. This function is usually called in BaseRenderer.__enter__. Arguments: token_cls (SpanToken): token to be included in the parsing process. position (int): the position for the token class to be inserted into. """ _token_types.insert(position, token_cls) def remove_token(token_cls): """ Allows external manipulation of the parsing process. This function is usually called in BaseRenderer.__exit__. Arguments: token_cls (BlockToken): token to be removed from the parsing process. """ _token_types.remove(token_cls) def reset_tokens(): """ Resets global _token_types to all token classes in __all__. """ global _token_types _token_types = [globals()[cls_name] for cls_name in __all__] class BlockToken(object): """ Base class for block-level tokens. Recursively parse inner tokens. Naming conventions: * lines denotes a list of (possibly unparsed) input lines, and is commonly used as the argument name for constructors. * BlockToken.children is a list with all the inner tokens (thus if a token has children attribute, it is not a leaf node; if a token calls span_token.tokenize_inner, it is the boundary between span-level tokens and block-level tokens); * BlockToken.start takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize). * BlockToken.read takes the rest of the lines in the ducment as an iterator (including the start line), and consumes all the lines that should be read into this token. Default to stop at an empty line. Note that BlockToken.read does not have to return a list of lines. Because the return value of this function will be directly passed into the token constructor, we can return any relevant parsing information, sometimes even ready-made tokens, into the constructor. See block_tokenizer.tokenize. If BlockToken.read returns None, the read result is ignored, but the token class is responsible for resetting the iterator to a previous state. See block_tokenizer.FileWrapper.anchor, block_tokenizer.FileWrapper.reset. Attributes: children (list): inner tokens. """ def __init__(self, lines, tokenize_func): self.children = tokenize_func(lines) def __contains__(self, text): return any(text in child for child in self.children) @staticmethod def read(lines): line_buffer = [next(lines)] for line in lines: if line == '\n': break line_buffer.append(line) return line_buffer class Document(BlockToken): """ Document token. """ def __init__(self, lines): if isinstance(lines, str): lines = lines.splitlines(keepends=True) lines = [line if line.endswith('\n') else '{}\n'.format(line) for line in lines] self.footnotes = {} global _root_node _root_node = self span_token._root_node = self self.children = tokenize(lines) span_token._root_node = None _root_node = None class Heading(BlockToken): """ Heading token. (["### some heading ###\\n"]) Boundary between span-level and block-level tokens. Attributes: level (int): heading level. children (list): inner tokens. """ pattern = re.compile(r' {0,3}(#{1,6})(?:\n|\s+?(.*?)(?:\n|\s+?#+\s*?$))') level = 0 content = '' def __init__(self, match): self.level, content = match super().__init__(content, span_token.tokenize_inner) @classmethod def start(cls, line): match_obj = cls.pattern.match(line) if match_obj is None: return False cls.level = len(match_obj.group(1)) cls.content = (match_obj.group(2) or '').strip() if set(cls.content) == {'#'}: cls.content = '' return True @classmethod def read(cls, lines): next(lines) return cls.level, cls.content class SetextHeading(BlockToken): """ Setext headings. Not included in the parsing process, but called by Paragraph.__new__. """ def __init__(self, lines): self.level = 1 if lines.pop().lstrip().startswith('=') else 2 content = '\n'.join([line.strip() for line in lines]) super().__init__(content, span_token.tokenize_inner) @classmethod def start(cls, line): raise NotImplementedError() @classmethod def read(cls, lines): raise NotImplementedError() class Quote(BlockToken): """ Quote token. (["> # heading\\n", "> paragraph\\n"]) """ def __init__(self, parse_buffer): # span-level tokenizing happens here. self.children = tokenizer.make_tokens(parse_buffer) @staticmethod def start(line): stripped = line.lstrip(' ') if len(line) - len(stripped) > 3: return False return stripped.startswith('>') @classmethod def read(cls, lines): # first line line = cls.convert_leading_tabs(next(lines).lstrip()).split('>', 1)[1] if len(line) > 0 and line[0] == ' ': line = line[1:] line_buffer = [line] # set booleans in_code_fence = CodeFence.start(line) in_block_code = BlockCode.start(line) blank_line = line.strip() == '' # loop next_line = lines.peek() while (next_line is not None and next_line.strip() != '' and not Heading.start(next_line) and not CodeFence.start(next_line) and not ThematicBreak.start(next_line) and not List.start(next_line)): stripped = cls.convert_leading_tabs(next_line.lstrip()) prepend = 0 if stripped[0] == '>': # has leader, not lazy continuation prepend += 1 if stripped[1] == ' ': prepend += 1 stripped = stripped[prepend:] in_code_fence = CodeFence.start(stripped) in_block_code = BlockCode.start(stripped) blank_line = stripped.strip() == '' line_buffer.append(stripped) elif in_code_fence or in_block_code or blank_line: # not paragraph continuation text break else: # lazy continuation, preserve whitespace line_buffer.append(next_line) next(lines) next_line = lines.peek() # block level tokens are parsed here, so that footnotes # in quotes can be recognized before span-level tokenizing. Paragraph.parse_setext = False parse_buffer = tokenizer.tokenize_block(line_buffer, _token_types) Paragraph.parse_setext = True return parse_buffer @staticmethod def convert_leading_tabs(string): string = string.replace('>\t', ' ', 1) count = 0 for i, c in enumerate(string): if c == '\t': count += 4 elif c == ' ': count += 1 else: break if i == 0: return string return '>' + ' ' * count + string[i:] class Paragraph(BlockToken): """ Paragraph token. (["some\\n", "continuous\\n", "lines\\n"]) Boundary between span-level and block-level tokens. """ setext_pattern = re.compile(r' {0,3}(=|-)+ *$') parse_setext = True # can be disabled by Quote def __new__(cls, lines): if not isinstance(lines, list): # setext heading token, return directly return lines return super().__new__(cls) def __init__(self, lines): content = ''.join([line.lstrip() for line in lines]).strip() super().__init__(content, span_token.tokenize_inner) @staticmethod def start(line): return line.strip() != '' @classmethod def read(cls, lines): line_buffer = [next(lines)] next_line = lines.peek() while (next_line is not None and next_line.strip() != '' and not Heading.start(next_line) and not CodeFence.start(next_line) and not Quote.start(next_line)): # check if next_line starts List list_pair = ListItem.parse_marker(next_line) if (len(next_line) - len(next_line.lstrip()) < 4 and list_pair is not None): prepend, leader = list_pair # non-empty list item if next_line[:prepend].endswith(' '): # unordered list, or ordered list starting from 1 if not leader[:-1].isdigit() or leader[:-1] == '1': break # check if next_line starts HTMLBlock other than type 7 html_block = HTMLBlock.start(next_line) if html_block and html_block != 7: break # check if we see a setext underline if cls.parse_setext and cls.is_setext_heading(next_line): line_buffer.append(next(lines)) return SetextHeading(line_buffer) # check if we have a ThematicBreak (has to be after setext) if ThematicBreak.start(next_line): break # no other tokens, we're good line_buffer.append(next(lines)) next_line = lines.peek() return line_buffer @classmethod def is_setext_heading(cls, line): return cls.setext_pattern.match(line) class BlockCode(BlockToken): """ Indented code. Attributes: children (list): contains a single span_token.RawText token. language (str): always the empty string. """ def __init__(self, lines): self.language = '' self.children = (span_token.RawText(''.join(lines).strip('\n')+'\n'),) @staticmethod def start(line): return line.replace('\t', ' ', 1).startswith(' ') @classmethod def read(cls, lines): line_buffer = [] for line in lines: if line.strip() == '': line_buffer.append(line.lstrip(' ') if len(line) < 5 else line[4:]) continue if not line.replace('\t', ' ', 1).startswith(' '): lines.backstep() break line_buffer.append(cls.strip(line)) return line_buffer @staticmethod def strip(string): count = 0 for i, c in enumerate(string): if c == '\t': return string[i+1:] elif c == ' ': count += 1 else: break if count == 4: return string[i+1:] return string class CodeFence(BlockToken): """ Code fence. (["```sh\\n", "rm -rf /", ..., "```"]) Boundary between span-level and block-level tokens. Attributes: children (list): contains a single span_token.RawText token. language (str): language of code block (default to empty). """ pattern = re.compile(r'( {0,3})(`{3,}|~{3,}) *(\S*)') _open_info = None def __init__(self, match): lines, open_info = match self.language = span_token.EscapeSequence.strip(open_info[2]) self.children = (span_token.RawText(''.join(lines)),) @classmethod def start(cls, line): match_obj = cls.pattern.match(line) if not match_obj: return False prepend, leader, lang = match_obj.groups() if leader[0] in lang or leader[0] in line[match_obj.end():]: return False cls._open_info = len(prepend), leader, lang return True @classmethod def read(cls, lines): next(lines) line_buffer = [] for line in lines: stripped_line = line.lstrip(' ') diff = len(line) - len(stripped_line) if (stripped_line.startswith(cls._open_info[1]) and len(stripped_line.split(maxsplit=1)) == 1 and diff < 4): break if diff > cls._open_info[0]: stripped_line = ' ' * (diff - cls._open_info[0]) + stripped_line line_buffer.append(stripped_line) return line_buffer, cls._open_info class List(BlockToken): """ List token. Attributes: children (list): a list of ListItem tokens. loose (bool): whether the list is loose. start (NoneType or int): None if unordered, starting number if ordered. """ pattern = re.compile(r' {0,3}(?:\d{0,9}[.)]|[+\-*])(?:[ \t]*$|[ \t]+)') def __init__(self, matches): self.children = [ListItem(*match) for match in matches] self.loose = any(item.loose for item in self.children) leader = self.children[0].leader self.start = None if len(leader) != 1: self.start = int(leader[:-1]) @classmethod def start(cls, line): return cls.pattern.match(line) @classmethod def read(cls, lines): leader = None next_marker = None matches = [] while True: output, next_marker = ListItem.read(lines, next_marker) item_leader = output[2] if leader is None: leader = item_leader elif not cls.same_marker_type(leader, item_leader): lines.reset() break matches.append(output) if next_marker is None: break if matches: # Only consider the last list item loose if there's more than one element last_parse_buffer = matches[-1][0] last_parse_buffer.loose = len(last_parse_buffer) > 1 and last_parse_buffer.loose return matches @staticmethod def same_marker_type(leader, other): if len(leader) == 1: return leader == other return leader[:-1].isdigit() and other[:-1].isdigit() and leader[-1] == other[-1] class ListItem(BlockToken): """ List items. Not included in the parsing process, but called by List. """ pattern = re.compile(r'\s*(\d{0,9}[.)]|[+\-*])(\s*$|\s+)') def __init__(self, parse_buffer, prepend, leader): self.leader = leader self.prepend = prepend self.children = tokenizer.make_tokens(parse_buffer) self.loose = parse_buffer.loose @staticmethod def in_continuation(line, prepend): return line.strip() == '' or len(line) - len(line.lstrip()) >= prepend @staticmethod def other_token(line): return (Heading.start(line) or Quote.start(line) or CodeFence.start(line) or ThematicBreak.start(line)) @classmethod def parse_marker(cls, line): """ Returns a pair (prepend, leader) iff the line has a valid leader. """ match_obj = cls.pattern.match(line) if match_obj is None: return None # no valid leader leader = match_obj.group(1) content = match_obj.group(0).replace(leader+'\t', leader+' ', 1) # reassign prepend and leader prepend = len(content) if prepend == len(line.rstrip('\n')): prepend = match_obj.end(1) + 1 else: spaces = match_obj.group(2) if spaces.startswith('\t'): spaces = spaces.replace('\t', ' ', 1) spaces = spaces.replace('\t', ' ') n_spaces = len(spaces) if n_spaces > 4: prepend = match_obj.end(1) + 1 return prepend, leader @classmethod def read(cls, lines, prev_marker=None): next_marker = None lines.anchor() prepend = -1 leader = None line_buffer = [] # first line line = next(lines) prepend, leader = prev_marker if prev_marker else cls.parse_marker(line) line = line.replace(leader+'\t', leader+' ', 1).replace('\t', ' ') empty_first_line = line[prepend:].strip() == '' if not empty_first_line: line_buffer.append(line[prepend:]) next_line = lines.peek() if empty_first_line and next_line is not None and next_line.strip() == '': parse_buffer = tokenizer.tokenize_block([next(lines)], _token_types) next_line = lines.peek() if next_line is not None: marker_info = cls.parse_marker(next_line) if marker_info is not None: next_marker = marker_info return (parse_buffer, prepend, leader), next_marker # loop newline = 0 while True: # no more lines if next_line is None: # strip off newlines if newline: lines.backstep() del line_buffer[-newline:] break next_line = next_line.replace('\t', ' ') # not in continuation if not cls.in_continuation(next_line, prepend): # directly followed by another token if cls.other_token(next_line): if newline: lines.backstep() del line_buffer[-newline:] break # next_line is a new list item marker_info = cls.parse_marker(next_line) if marker_info is not None: next_marker = marker_info break # not another item, has newlines -> not continuation if newline: lines.backstep() del line_buffer[-newline:] break next(lines) line = next_line stripped = line.lstrip(' ') diff = len(line) - len(stripped) if diff > prepend: stripped = ' ' * (diff - prepend) + stripped line_buffer.append(stripped) newline = newline + 1 if next_line.strip() == '' else 0 next_line = lines.peek() # block-level tokens are parsed here, so that footnotes can be # recognized before span-level parsing. parse_buffer = tokenizer.tokenize_block(line_buffer, _token_types) return (parse_buffer, prepend, leader), next_marker class Table(BlockToken): """ Table token. Attributes: has_header (bool): whether table has header row. column_align (list): align options for each column (default to [None]). children (list): inner tokens (TableRows). """ def __init__(self, lines): if '---' in lines[1]: self.column_align = [self.parse_align(column) for column in self.split_delimiter(lines[1])] self.header = TableRow(lines[0], self.column_align) self.children = [TableRow(line, self.column_align) for line in lines[2:]] else: self.column_align = [None] self.children = [TableRow(line) for line in lines] @staticmethod def split_delimiter(delimiter): """ Helper function; returns a list of align options. Args: delimiter (str): e.g.: "| :--- | :---: | ---: |\n" Returns: a list of align options (None, 0 or 1). """ return re.findall(r':?---+:?', delimiter) @staticmethod def parse_align(column): """ Helper function; returns align option from cell content. Returns: None if align = left; 0 if align = center; 1 if align = right. """ return (0 if column[0] == ':' else 1) if column[-1] == ':' else None @staticmethod def start(line): return '|' in line @staticmethod def read(lines): lines.anchor() line_buffer = [next(lines)] while lines.peek() is not None and '|' in lines.peek(): line_buffer.append(next(lines)) if len(line_buffer) < 2 or '---' not in line_buffer[1]: lines.reset() return None return line_buffer class TableRow(BlockToken): """ Table row token. Supports escaped pipes in table cells (for primary use within code spans). Should only be called by Table.__init__(). """ # Note: Python regex requires fixed-length look-behind, # so we cannot use a more precise alternative: r"(?' and not escaped: return offset, i+1, string[offset+1:i] elif escaped: escaped = False return None else: escaped = False count = 0 for i, c in enumerate(string[offset:], start=offset): if c == '\\' and not escaped: escaped = True elif c in whitespace: break elif not escaped: if c == '(': count += 1 elif c == ')': count -= 1 elif is_control_char(c): return None elif escaped: escaped = False if count != 0: return None return offset, i, string[offset:i] @classmethod def match_link_title(cls, string, offset): new_offset = shift_whitespace(string, offset) if (new_offset == len(string) or '\n' in string[offset:new_offset] and string[new_offset] == '['): return offset, new_offset, '' if string[new_offset] == '"': closing = '"' elif string[new_offset] == "'": closing = "'" elif string[new_offset] == '(': closing = ')' elif '\n' in string[offset:new_offset]: return offset, offset, '' else: # XXX: Can this actually ever happen? return None offset = new_offset escaped = False for i, c in enumerate(string[offset+1:], start=offset+1): if c == '\\' and not escaped: escaped = True elif c == closing and not escaped: new_offset = shift_whitespace(string, i+1) if '\n' not in string[i+1:new_offset]: return None return offset, new_offset, string[offset+1:i] elif escaped: escaped = False return None @staticmethod def append_footnotes(matches, root): for key, dest, title in matches: key = normalize_label(key) dest = span_token.EscapeSequence.strip(dest.strip()) title = span_token.EscapeSequence.strip(title) if key not in root.footnotes: root.footnotes[key] = dest, title @staticmethod def backtrack(lines, string, offset): """ Called when we iterated over some lines and found nothing relevant on them. This returns those lines back to the parsing process. """ # call lstrip() to prevent returning too many lines back (hence infinite loop), like in this case: # * valid footlink definition line: `[key]: valueN\r\n` (here `offset` points to `\r` after parsing the definition) # * follow-up line containing just text: `something\n` (only this line should be re-processed and parsed as a Paragraph) lines._index -= string[offset:].lstrip().count('\n') class ThematicBreak(BlockToken): """ Thematic break token (a.k.a. horizontal rule.) """ pattern = re.compile(r' {0,3}(?:([-_*])\s*?)(?:\1\s*?){2,}$') def __init__(self, _): pass @classmethod def start(cls, line): return cls.pattern.match(line) @staticmethod def read(lines): return [next(lines)] class HTMLBlock(BlockToken): """ Block-level HTML tokens. Attributes: content (str): literal strings rendered as-is. """ _end_cond = None multiblock = re.compile(r'<(script|pre|style)[ >\n]') predefined = re.compile(r'<\/?(.+?)(?:\/?>|[ \n])') custom_tag = re.compile(r'(?:' + '|'.join((span_token._open_tag, span_token._closing_tag)) + r')\s*$') def __init__(self, lines): self.content = ''.join(lines).rstrip('\n') @classmethod def start(cls, line): stripped = line.lstrip() if len(line) - len(stripped) >= 4: return False # rule 1:
, \nokay\n",
    "html": "\n

okay

\n", "example": 138, "start_line": 2379, "end_line": 2393, "section": "HTML blocks" }, { "markdown": "\nh1 {color:red;}\n\np {color:blue;}\n\nokay\n", "html": "\nh1 {color:red;}\n\np {color:blue;}\n\n

okay

\n", "example": 139, "start_line": 2398, "end_line": 2414, "section": "HTML blocks" }, { "markdown": "\n\nfoo\n", "html": "\n\nfoo\n", "example": 140, "start_line": 2421, "end_line": 2431, "section": "HTML blocks" }, { "markdown": ">
\n> foo\n\nbar\n", "html": "
\n
\nfoo\n
\n

bar

\n", "example": 141, "start_line": 2434, "end_line": 2445, "section": "HTML blocks" }, { "markdown": "-
\n- foo\n", "html": "
    \n
  • \n
    \n
  • \n
  • foo
  • \n
\n", "example": 142, "start_line": 2448, "end_line": 2458, "section": "HTML blocks" }, { "markdown": "\n*foo*\n", "html": "\n

foo

\n", "example": 143, "start_line": 2463, "end_line": 2469, "section": "HTML blocks" }, { "markdown": "*bar*\n*baz*\n", "html": "*bar*\n

baz

\n", "example": 144, "start_line": 2472, "end_line": 2478, "section": "HTML blocks" }, { "markdown": "1. *bar*\n", "html": "1. *bar*\n", "example": 145, "start_line": 2484, "end_line": 2492, "section": "HTML blocks" }, { "markdown": "\nokay\n", "html": "\n

okay

\n", "example": 146, "start_line": 2497, "end_line": 2509, "section": "HTML blocks" }, { "markdown": "';\n\n?>\nokay\n", "html": "';\n\n?>\n

okay

\n", "example": 147, "start_line": 2515, "end_line": 2529, "section": "HTML blocks" }, { "markdown": "\n", "html": "\n", "example": 148, "start_line": 2534, "end_line": 2538, "section": "HTML blocks" }, { "markdown": "\nokay\n", "html": "\n

okay

\n", "example": 149, "start_line": 2543, "end_line": 2571, "section": "HTML blocks" }, { "markdown": " \n\n \n", "html": " \n
<!-- foo -->\n
\n", "example": 150, "start_line": 2576, "end_line": 2584, "section": "HTML blocks" }, { "markdown": "
\n\n
\n", "html": "
\n
<div>\n
\n", "example": 151, "start_line": 2587, "end_line": 2595, "section": "HTML blocks" }, { "markdown": "Foo\n
\nbar\n
\n", "html": "

Foo

\n
\nbar\n
\n", "example": 152, "start_line": 2601, "end_line": 2611, "section": "HTML blocks" }, { "markdown": "
\nbar\n
\n*foo*\n", "html": "
\nbar\n
\n*foo*\n", "example": 153, "start_line": 2617, "end_line": 2627, "section": "HTML blocks" }, { "markdown": "Foo\n\nbaz\n", "html": "

Foo\n\nbaz

\n", "example": 154, "start_line": 2632, "end_line": 2640, "section": "HTML blocks" }, { "markdown": "
\n\n*Emphasized* text.\n\n
\n", "html": "
\n

Emphasized text.

\n
\n", "example": 155, "start_line": 2673, "end_line": 2683, "section": "HTML blocks" }, { "markdown": "
\n*Emphasized* text.\n
\n", "html": "
\n*Emphasized* text.\n
\n", "example": 156, "start_line": 2686, "end_line": 2694, "section": "HTML blocks" }, { "markdown": "\n\n\n\n\n\n\n\n
\nHi\n
\n", "html": "\n\n\n\n
\nHi\n
\n", "example": 157, "start_line": 2708, "end_line": 2728, "section": "HTML blocks" }, { "markdown": "\n\n \n\n \n\n \n\n
\n Hi\n
\n", "html": "\n \n
<td>\n  Hi\n</td>\n
\n \n
\n", "example": 158, "start_line": 2735, "end_line": 2756, "section": "HTML blocks" }, { "markdown": "[foo]: /url \"title\"\n\n[foo]\n", "html": "

foo

\n", "example": 159, "start_line": 2783, "end_line": 2789, "section": "Link reference definitions" }, { "markdown": " [foo]: \n /url \n 'the title' \n\n[foo]\n", "html": "

foo

\n", "example": 160, "start_line": 2792, "end_line": 2800, "section": "Link reference definitions" }, { "markdown": "[Foo*bar\\]]:my_(url) 'title (with parens)'\n\n[Foo*bar\\]]\n", "html": "

Foo*bar]

\n", "example": 161, "start_line": 2803, "end_line": 2809, "section": "Link reference definitions" }, { "markdown": "[Foo bar]:\n\n'title'\n\n[Foo bar]\n", "html": "

Foo bar

\n", "example": 162, "start_line": 2812, "end_line": 2820, "section": "Link reference definitions" }, { "markdown": "[foo]: /url '\ntitle\nline1\nline2\n'\n\n[foo]\n", "html": "

foo

\n", "example": 163, "start_line": 2825, "end_line": 2839, "section": "Link reference definitions" }, { "markdown": "[foo]: /url 'title\n\nwith blank line'\n\n[foo]\n", "html": "

[foo]: /url 'title

\n

with blank line'

\n

[foo]

\n", "example": 164, "start_line": 2844, "end_line": 2854, "section": "Link reference definitions" }, { "markdown": "[foo]:\n/url\n\n[foo]\n", "html": "

foo

\n", "example": 165, "start_line": 2859, "end_line": 2866, "section": "Link reference definitions" }, { "markdown": "[foo]:\n\n[foo]\n", "html": "

[foo]:

\n

[foo]

\n", "example": 166, "start_line": 2871, "end_line": 2878, "section": "Link reference definitions" }, { "markdown": "[foo]: /url\\bar\\*baz \"foo\\\"bar\\baz\"\n\n[foo]\n", "html": "

foo

\n", "example": 167, "start_line": 2884, "end_line": 2890, "section": "Link reference definitions" }, { "markdown": "[foo]\n\n[foo]: url\n", "html": "

foo

\n", "example": 168, "start_line": 2895, "end_line": 2901, "section": "Link reference definitions" }, { "markdown": "[foo]\n\n[foo]: first\n[foo]: second\n", "html": "

foo

\n", "example": 169, "start_line": 2907, "end_line": 2914, "section": "Link reference definitions" }, { "markdown": "[FOO]: /url\n\n[Foo]\n", "html": "

Foo

\n", "example": 170, "start_line": 2920, "end_line": 2926, "section": "Link reference definitions" }, { "markdown": "[ΑΓΩ]: /φου\n\n[αγω]\n", "html": "

αγω

\n", "example": 171, "start_line": 2929, "end_line": 2935, "section": "Link reference definitions" }, { "markdown": "[foo]: /url\n", "html": "", "example": 172, "start_line": 2941, "end_line": 2944, "section": "Link reference definitions" }, { "markdown": "[\nfoo\n]: /url\nbar\n", "html": "

bar

\n", "example": 173, "start_line": 2949, "end_line": 2956, "section": "Link reference definitions" }, { "markdown": "[foo]: /url \"title\" ok\n", "html": "

[foo]: /url "title" ok

\n", "example": 174, "start_line": 2962, "end_line": 2966, "section": "Link reference definitions" }, { "markdown": "[foo]: /url\n\"title\" ok\n", "html": "

"title" ok

\n", "example": 175, "start_line": 2971, "end_line": 2976, "section": "Link reference definitions" }, { "markdown": " [foo]: /url \"title\"\n\n[foo]\n", "html": "
[foo]: /url "title"\n
\n

[foo]

\n", "example": 176, "start_line": 2982, "end_line": 2990, "section": "Link reference definitions" }, { "markdown": "```\n[foo]: /url\n```\n\n[foo]\n", "html": "
[foo]: /url\n
\n

[foo]

\n", "example": 177, "start_line": 2996, "end_line": 3006, "section": "Link reference definitions" }, { "markdown": "Foo\n[bar]: /baz\n\n[bar]\n", "html": "

Foo\n[bar]: /baz

\n

[bar]

\n", "example": 178, "start_line": 3011, "end_line": 3020, "section": "Link reference definitions" }, { "markdown": "# [Foo]\n[foo]: /url\n> bar\n", "html": "

Foo

\n
\n

bar

\n
\n", "example": 179, "start_line": 3026, "end_line": 3035, "section": "Link reference definitions" }, { "markdown": "[foo]: /foo-url \"foo\"\n[bar]: /bar-url\n \"bar\"\n[baz]: /baz-url\n\n[foo],\n[bar],\n[baz]\n", "html": "

foo,\nbar,\nbaz

\n", "example": 180, "start_line": 3041, "end_line": 3054, "section": "Link reference definitions" }, { "markdown": "[foo]\n\n> [foo]: /url\n", "html": "

foo

\n
\n
\n", "example": 181, "start_line": 3062, "end_line": 3070, "section": "Link reference definitions" }, { "markdown": "aaa\n\nbbb\n", "html": "

aaa

\n

bbb

\n", "example": 182, "start_line": 3085, "end_line": 3092, "section": "Paragraphs" }, { "markdown": "aaa\nbbb\n\nccc\nddd\n", "html": "

aaa\nbbb

\n

ccc\nddd

\n", "example": 183, "start_line": 3097, "end_line": 3108, "section": "Paragraphs" }, { "markdown": "aaa\n\n\nbbb\n", "html": "

aaa

\n

bbb

\n", "example": 184, "start_line": 3113, "end_line": 3121, "section": "Paragraphs" }, { "markdown": " aaa\n bbb\n", "html": "

aaa\nbbb

\n", "example": 185, "start_line": 3126, "end_line": 3132, "section": "Paragraphs" }, { "markdown": "aaa\n bbb\n ccc\n", "html": "

aaa\nbbb\nccc

\n", "example": 186, "start_line": 3138, "end_line": 3146, "section": "Paragraphs" }, { "markdown": " aaa\nbbb\n", "html": "

aaa\nbbb

\n", "example": 187, "start_line": 3152, "end_line": 3158, "section": "Paragraphs" }, { "markdown": " aaa\nbbb\n", "html": "
aaa\n
\n

bbb

\n", "example": 188, "start_line": 3161, "end_line": 3168, "section": "Paragraphs" }, { "markdown": "aaa \nbbb \n", "html": "

aaa
\nbbb

\n", "example": 189, "start_line": 3175, "end_line": 3181, "section": "Paragraphs" }, { "markdown": " \n\naaa\n \n\n# aaa\n\n \n", "html": "

aaa

\n

aaa

\n", "example": 190, "start_line": 3192, "end_line": 3204, "section": "Blank lines" }, { "markdown": "> # Foo\n> bar\n> baz\n", "html": "
\n

Foo

\n

bar\nbaz

\n
\n", "example": 191, "start_line": 3258, "end_line": 3268, "section": "Block quotes" }, { "markdown": "># Foo\n>bar\n> baz\n", "html": "
\n

Foo

\n

bar\nbaz

\n
\n", "example": 192, "start_line": 3273, "end_line": 3283, "section": "Block quotes" }, { "markdown": " > # Foo\n > bar\n > baz\n", "html": "
\n

Foo

\n

bar\nbaz

\n
\n", "example": 193, "start_line": 3288, "end_line": 3298, "section": "Block quotes" }, { "markdown": " > # Foo\n > bar\n > baz\n", "html": "
> # Foo\n> bar\n> baz\n
\n", "example": 194, "start_line": 3303, "end_line": 3312, "section": "Block quotes" }, { "markdown": "> # Foo\n> bar\nbaz\n", "html": "
\n

Foo

\n

bar\nbaz

\n
\n", "example": 195, "start_line": 3318, "end_line": 3328, "section": "Block quotes" }, { "markdown": "> bar\nbaz\n> foo\n", "html": "
\n

bar\nbaz\nfoo

\n
\n", "example": 196, "start_line": 3334, "end_line": 3344, "section": "Block quotes" }, { "markdown": "> foo\n---\n", "html": "
\n

foo

\n
\n
\n", "example": 197, "start_line": 3358, "end_line": 3366, "section": "Block quotes" }, { "markdown": "> - foo\n- bar\n", "html": "
\n
    \n
  • foo
  • \n
\n
\n
    \n
  • bar
  • \n
\n", "example": 198, "start_line": 3378, "end_line": 3390, "section": "Block quotes" }, { "markdown": "> foo\n bar\n", "html": "
\n
foo\n
\n
\n
bar\n
\n", "example": 199, "start_line": 3396, "end_line": 3406, "section": "Block quotes" }, { "markdown": "> ```\nfoo\n```\n", "html": "
\n
\n
\n

foo

\n
\n", "example": 200, "start_line": 3409, "end_line": 3419, "section": "Block quotes" }, { "markdown": "> foo\n - bar\n", "html": "
\n

foo\n- bar

\n
\n", "example": 201, "start_line": 3425, "end_line": 3433, "section": "Block quotes" }, { "markdown": ">\n", "html": "
\n
\n", "example": 202, "start_line": 3449, "end_line": 3454, "section": "Block quotes" }, { "markdown": ">\n> \n> \n", "html": "
\n
\n", "example": 203, "start_line": 3457, "end_line": 3464, "section": "Block quotes" }, { "markdown": ">\n> foo\n> \n", "html": "
\n

foo

\n
\n", "example": 204, "start_line": 3469, "end_line": 3477, "section": "Block quotes" }, { "markdown": "> foo\n\n> bar\n", "html": "
\n

foo

\n
\n
\n

bar

\n
\n", "example": 205, "start_line": 3482, "end_line": 3493, "section": "Block quotes" }, { "markdown": "> foo\n> bar\n", "html": "
\n

foo\nbar

\n
\n", "example": 206, "start_line": 3504, "end_line": 3512, "section": "Block quotes" }, { "markdown": "> foo\n>\n> bar\n", "html": "
\n

foo

\n

bar

\n
\n", "example": 207, "start_line": 3517, "end_line": 3526, "section": "Block quotes" }, { "markdown": "foo\n> bar\n", "html": "

foo

\n
\n

bar

\n
\n", "example": 208, "start_line": 3531, "end_line": 3539, "section": "Block quotes" }, { "markdown": "> aaa\n***\n> bbb\n", "html": "
\n

aaa

\n
\n
\n
\n

bbb

\n
\n", "example": 209, "start_line": 3545, "end_line": 3557, "section": "Block quotes" }, { "markdown": "> bar\nbaz\n", "html": "
\n

bar\nbaz

\n
\n", "example": 210, "start_line": 3563, "end_line": 3571, "section": "Block quotes" }, { "markdown": "> bar\n\nbaz\n", "html": "
\n

bar

\n
\n

baz

\n", "example": 211, "start_line": 3574, "end_line": 3583, "section": "Block quotes" }, { "markdown": "> bar\n>\nbaz\n", "html": "
\n

bar

\n
\n

baz

\n", "example": 212, "start_line": 3586, "end_line": 3595, "section": "Block quotes" }, { "markdown": "> > > foo\nbar\n", "html": "
\n
\n
\n

foo\nbar

\n
\n
\n
\n", "example": 213, "start_line": 3602, "end_line": 3614, "section": "Block quotes" }, { "markdown": ">>> foo\n> bar\n>>baz\n", "html": "
\n
\n
\n

foo\nbar\nbaz

\n
\n
\n
\n", "example": 214, "start_line": 3617, "end_line": 3631, "section": "Block quotes" }, { "markdown": "> code\n\n> not code\n", "html": "
\n
code\n
\n
\n
\n

not code

\n
\n", "example": 215, "start_line": 3639, "end_line": 3651, "section": "Block quotes" }, { "markdown": "A paragraph\nwith two lines.\n\n indented code\n\n> A block quote.\n", "html": "

A paragraph\nwith two lines.

\n
indented code\n
\n
\n

A block quote.

\n
\n", "example": 216, "start_line": 3694, "end_line": 3709, "section": "List items" }, { "markdown": "1. A paragraph\n with two lines.\n\n indented code\n\n > A block quote.\n", "html": "
    \n
  1. \n

    A paragraph\nwith two lines.

    \n
    indented code\n
    \n
    \n

    A block quote.

    \n
    \n
  2. \n
\n", "example": 217, "start_line": 3716, "end_line": 3735, "section": "List items" }, { "markdown": "- one\n\n two\n", "html": "
    \n
  • one
  • \n
\n

two

\n", "example": 218, "start_line": 3749, "end_line": 3758, "section": "List items" }, { "markdown": "- one\n\n two\n", "html": "
    \n
  • \n

    one

    \n

    two

    \n
  • \n
\n", "example": 219, "start_line": 3761, "end_line": 3772, "section": "List items" }, { "markdown": " - one\n\n two\n", "html": "
    \n
  • one
  • \n
\n
 two\n
\n", "example": 220, "start_line": 3775, "end_line": 3785, "section": "List items" }, { "markdown": " - one\n\n two\n", "html": "
    \n
  • \n

    one

    \n

    two

    \n
  • \n
\n", "example": 221, "start_line": 3788, "end_line": 3799, "section": "List items" }, { "markdown": " > > 1. one\n>>\n>> two\n", "html": "
\n
\n
    \n
  1. \n

    one

    \n

    two

    \n
  2. \n
\n
\n
\n", "example": 222, "start_line": 3810, "end_line": 3825, "section": "List items" }, { "markdown": ">>- one\n>>\n > > two\n", "html": "
\n
\n
    \n
  • one
  • \n
\n

two

\n
\n
\n", "example": 223, "start_line": 3837, "end_line": 3850, "section": "List items" }, { "markdown": "-one\n\n2.two\n", "html": "

-one

\n

2.two

\n", "example": 224, "start_line": 3856, "end_line": 3863, "section": "List items" }, { "markdown": "- foo\n\n\n bar\n", "html": "
    \n
  • \n

    foo

    \n

    bar

    \n
  • \n
\n", "example": 225, "start_line": 3869, "end_line": 3881, "section": "List items" }, { "markdown": "1. foo\n\n ```\n bar\n ```\n\n baz\n\n > bam\n", "html": "
    \n
  1. \n

    foo

    \n
    bar\n
    \n

    baz

    \n
    \n

    bam

    \n
    \n
  2. \n
\n", "example": 226, "start_line": 3886, "end_line": 3908, "section": "List items" }, { "markdown": "- Foo\n\n bar\n\n\n baz\n", "html": "
    \n
  • \n

    Foo

    \n
    bar\n\n\nbaz\n
    \n
  • \n
\n", "example": 227, "start_line": 3914, "end_line": 3932, "section": "List items" }, { "markdown": "123456789. ok\n", "html": "
    \n
  1. ok
  2. \n
\n", "example": 228, "start_line": 3936, "end_line": 3942, "section": "List items" }, { "markdown": "1234567890. not ok\n", "html": "

1234567890. not ok

\n", "example": 229, "start_line": 3945, "end_line": 3949, "section": "List items" }, { "markdown": "0. ok\n", "html": "
    \n
  1. ok
  2. \n
\n", "example": 230, "start_line": 3954, "end_line": 3960, "section": "List items" }, { "markdown": "003. ok\n", "html": "
    \n
  1. ok
  2. \n
\n", "example": 231, "start_line": 3963, "end_line": 3969, "section": "List items" }, { "markdown": "-1. not ok\n", "html": "

-1. not ok

\n", "example": 232, "start_line": 3974, "end_line": 3978, "section": "List items" }, { "markdown": "- foo\n\n bar\n", "html": "
    \n
  • \n

    foo

    \n
    bar\n
    \n
  • \n
\n", "example": 233, "start_line": 3998, "end_line": 4010, "section": "List items" }, { "markdown": " 10. foo\n\n bar\n", "html": "
    \n
  1. \n

    foo

    \n
    bar\n
    \n
  2. \n
\n", "example": 234, "start_line": 4015, "end_line": 4027, "section": "List items" }, { "markdown": " indented code\n\nparagraph\n\n more code\n", "html": "
indented code\n
\n

paragraph

\n
more code\n
\n", "example": 235, "start_line": 4034, "end_line": 4046, "section": "List items" }, { "markdown": "1. indented code\n\n paragraph\n\n more code\n", "html": "
    \n
  1. \n
    indented code\n
    \n

    paragraph

    \n
    more code\n
    \n
  2. \n
\n", "example": 236, "start_line": 4049, "end_line": 4065, "section": "List items" }, { "markdown": "1. indented code\n\n paragraph\n\n more code\n", "html": "
    \n
  1. \n
     indented code\n
    \n

    paragraph

    \n
    more code\n
    \n
  2. \n
\n", "example": 237, "start_line": 4071, "end_line": 4087, "section": "List items" }, { "markdown": " foo\n\nbar\n", "html": "

foo

\n

bar

\n", "example": 238, "start_line": 4098, "end_line": 4105, "section": "List items" }, { "markdown": "- foo\n\n bar\n", "html": "
    \n
  • foo
  • \n
\n

bar

\n", "example": 239, "start_line": 4108, "end_line": 4117, "section": "List items" }, { "markdown": "- foo\n\n bar\n", "html": "
    \n
  • \n

    foo

    \n

    bar

    \n
  • \n
\n", "example": 240, "start_line": 4125, "end_line": 4136, "section": "List items" }, { "markdown": "-\n foo\n-\n ```\n bar\n ```\n-\n baz\n", "html": "
    \n
  • foo
  • \n
  • \n
    bar\n
    \n
  • \n
  • \n
    baz\n
    \n
  • \n
\n", "example": 241, "start_line": 4153, "end_line": 4174, "section": "List items" }, { "markdown": "- \n foo\n", "html": "
    \n
  • foo
  • \n
\n", "example": 242, "start_line": 4179, "end_line": 4186, "section": "List items" }, { "markdown": "-\n\n foo\n", "html": "
    \n
  • \n
\n

foo

\n", "example": 243, "start_line": 4193, "end_line": 4202, "section": "List items" }, { "markdown": "- foo\n-\n- bar\n", "html": "
    \n
  • foo
  • \n
  • \n
  • bar
  • \n
\n", "example": 244, "start_line": 4207, "end_line": 4217, "section": "List items" }, { "markdown": "- foo\n- \n- bar\n", "html": "
    \n
  • foo
  • \n
  • \n
  • bar
  • \n
\n", "example": 245, "start_line": 4222, "end_line": 4232, "section": "List items" }, { "markdown": "1. foo\n2.\n3. bar\n", "html": "
    \n
  1. foo
  2. \n
  3. \n
  4. bar
  5. \n
\n", "example": 246, "start_line": 4237, "end_line": 4247, "section": "List items" }, { "markdown": "*\n", "html": "
    \n
  • \n
\n", "example": 247, "start_line": 4252, "end_line": 4258, "section": "List items" }, { "markdown": "foo\n*\n\nfoo\n1.\n", "html": "

foo\n*

\n

foo\n1.

\n", "example": 248, "start_line": 4262, "end_line": 4273, "section": "List items" }, { "markdown": " 1. A paragraph\n with two lines.\n\n indented code\n\n > A block quote.\n", "html": "
    \n
  1. \n

    A paragraph\nwith two lines.

    \n
    indented code\n
    \n
    \n

    A block quote.

    \n
    \n
  2. \n
\n", "example": 249, "start_line": 4284, "end_line": 4303, "section": "List items" }, { "markdown": " 1. A paragraph\n with two lines.\n\n indented code\n\n > A block quote.\n", "html": "
    \n
  1. \n

    A paragraph\nwith two lines.

    \n
    indented code\n
    \n
    \n

    A block quote.

    \n
    \n
  2. \n
\n", "example": 250, "start_line": 4308, "end_line": 4327, "section": "List items" }, { "markdown": " 1. A paragraph\n with two lines.\n\n indented code\n\n > A block quote.\n", "html": "
    \n
  1. \n

    A paragraph\nwith two lines.

    \n
    indented code\n
    \n
    \n

    A block quote.

    \n
    \n
  2. \n
\n", "example": 251, "start_line": 4332, "end_line": 4351, "section": "List items" }, { "markdown": " 1. A paragraph\n with two lines.\n\n indented code\n\n > A block quote.\n", "html": "
1.  A paragraph\n    with two lines.\n\n        indented code\n\n    > A block quote.\n
\n", "example": 252, "start_line": 4356, "end_line": 4371, "section": "List items" }, { "markdown": " 1. A paragraph\nwith two lines.\n\n indented code\n\n > A block quote.\n", "html": "
    \n
  1. \n

    A paragraph\nwith two lines.

    \n
    indented code\n
    \n
    \n

    A block quote.

    \n
    \n
  2. \n
\n", "example": 253, "start_line": 4386, "end_line": 4405, "section": "List items" }, { "markdown": " 1. A paragraph\n with two lines.\n", "html": "
    \n
  1. A paragraph\nwith two lines.
  2. \n
\n", "example": 254, "start_line": 4410, "end_line": 4418, "section": "List items" }, { "markdown": "> 1. > Blockquote\ncontinued here.\n", "html": "
\n
    \n
  1. \n
    \n

    Blockquote\ncontinued here.

    \n
    \n
  2. \n
\n
\n", "example": 255, "start_line": 4423, "end_line": 4437, "section": "List items" }, { "markdown": "> 1. > Blockquote\n> continued here.\n", "html": "
\n
    \n
  1. \n
    \n

    Blockquote\ncontinued here.

    \n
    \n
  2. \n
\n
\n", "example": 256, "start_line": 4440, "end_line": 4454, "section": "List items" }, { "markdown": "- foo\n - bar\n - baz\n - boo\n", "html": "
    \n
  • foo\n
      \n
    • bar\n
        \n
      • baz\n
          \n
        • boo
        • \n
        \n
      • \n
      \n
    • \n
    \n
  • \n
\n", "example": 257, "start_line": 4467, "end_line": 4488, "section": "List items" }, { "markdown": "- foo\n - bar\n - baz\n - boo\n", "html": "
    \n
  • foo
  • \n
  • bar
  • \n
  • baz
  • \n
  • boo
  • \n
\n", "example": 258, "start_line": 4493, "end_line": 4505, "section": "List items" }, { "markdown": "10) foo\n - bar\n", "html": "
    \n
  1. foo\n
      \n
    • bar
    • \n
    \n
  2. \n
\n", "example": 259, "start_line": 4510, "end_line": 4521, "section": "List items" }, { "markdown": "10) foo\n - bar\n", "html": "
    \n
  1. foo
  2. \n
\n
    \n
  • bar
  • \n
\n", "example": 260, "start_line": 4526, "end_line": 4536, "section": "List items" }, { "markdown": "- - foo\n", "html": "
    \n
  • \n
      \n
    • foo
    • \n
    \n
  • \n
\n", "example": 261, "start_line": 4541, "end_line": 4551, "section": "List items" }, { "markdown": "1. - 2. foo\n", "html": "
    \n
  1. \n
      \n
    • \n
        \n
      1. foo
      2. \n
      \n
    • \n
    \n
  2. \n
\n", "example": 262, "start_line": 4554, "end_line": 4568, "section": "List items" }, { "markdown": "- # Foo\n- Bar\n ---\n baz\n", "html": "
    \n
  • \n

    Foo

    \n
  • \n
  • \n

    Bar

    \nbaz
  • \n
\n", "example": 263, "start_line": 4573, "end_line": 4587, "section": "List items" }, { "markdown": "- foo\n- bar\n+ baz\n", "html": "
    \n
  • foo
  • \n
  • bar
  • \n
\n
    \n
  • baz
  • \n
\n", "example": 264, "start_line": 4809, "end_line": 4821, "section": "Lists" }, { "markdown": "1. foo\n2. bar\n3) baz\n", "html": "
    \n
  1. foo
  2. \n
  3. bar
  4. \n
\n
    \n
  1. baz
  2. \n
\n", "example": 265, "start_line": 4824, "end_line": 4836, "section": "Lists" }, { "markdown": "Foo\n- bar\n- baz\n", "html": "

Foo

\n
    \n
  • bar
  • \n
  • baz
  • \n
\n", "example": 266, "start_line": 4843, "end_line": 4853, "section": "Lists" }, { "markdown": "The number of windows in my house is\n14. The number of doors is 6.\n", "html": "

The number of windows in my house is\n14. The number of doors is 6.

\n", "example": 267, "start_line": 4920, "end_line": 4926, "section": "Lists" }, { "markdown": "The number of windows in my house is\n1. The number of doors is 6.\n", "html": "

The number of windows in my house is

\n
    \n
  1. The number of doors is 6.
  2. \n
\n", "example": 268, "start_line": 4930, "end_line": 4938, "section": "Lists" }, { "markdown": "- foo\n\n- bar\n\n\n- baz\n", "html": "
    \n
  • \n

    foo

    \n
  • \n
  • \n

    bar

    \n
  • \n
  • \n

    baz

    \n
  • \n
\n", "example": 269, "start_line": 4944, "end_line": 4963, "section": "Lists" }, { "markdown": "- foo\n - bar\n - baz\n\n\n bim\n", "html": "
    \n
  • foo\n
      \n
    • bar\n
        \n
      • \n

        baz

        \n

        bim

        \n
      • \n
      \n
    • \n
    \n
  • \n
\n", "example": 270, "start_line": 4965, "end_line": 4987, "section": "Lists" }, { "markdown": "- foo\n- bar\n\n\n\n- baz\n- bim\n", "html": "
    \n
  • foo
  • \n
  • bar
  • \n
\n\n
    \n
  • baz
  • \n
  • bim
  • \n
\n", "example": 271, "start_line": 4995, "end_line": 5013, "section": "Lists" }, { "markdown": "- foo\n\n notcode\n\n- foo\n\n\n\n code\n", "html": "
    \n
  • \n

    foo

    \n

    notcode

    \n
  • \n
  • \n

    foo

    \n
  • \n
\n\n
code\n
\n", "example": 272, "start_line": 5016, "end_line": 5039, "section": "Lists" }, { "markdown": "- a\n - b\n - c\n - d\n - e\n - f\n - g\n - h\n- i\n", "html": "
    \n
  • a
  • \n
  • b
  • \n
  • c
  • \n
  • d
  • \n
  • e
  • \n
  • f
  • \n
  • g
  • \n
  • h
  • \n
  • i
  • \n
\n", "example": 273, "start_line": 5047, "end_line": 5069, "section": "Lists" }, { "markdown": "1. a\n\n 2. b\n\n 3. c\n", "html": "
    \n
  1. \n

    a

    \n
  2. \n
  3. \n

    b

    \n
  4. \n
  5. \n

    c

    \n
  6. \n
\n", "example": 274, "start_line": 5072, "end_line": 5090, "section": "Lists" }, { "markdown": "- a\n- b\n\n- c\n", "html": "
    \n
  • \n

    a

    \n
  • \n
  • \n

    b

    \n
  • \n
  • \n

    c

    \n
  • \n
\n", "example": 275, "start_line": 5096, "end_line": 5113, "section": "Lists" }, { "markdown": "* a\n*\n\n* c\n", "html": "
    \n
  • \n

    a

    \n
  • \n
  • \n
  • \n

    c

    \n
  • \n
\n", "example": 276, "start_line": 5118, "end_line": 5133, "section": "Lists" }, { "markdown": "- a\n- b\n\n c\n- d\n", "html": "
    \n
  • \n

    a

    \n
  • \n
  • \n

    b

    \n

    c

    \n
  • \n
  • \n

    d

    \n
  • \n
\n", "example": 277, "start_line": 5140, "end_line": 5159, "section": "Lists" }, { "markdown": "- a\n- b\n\n [ref]: /url\n- d\n", "html": "
    \n
  • \n

    a

    \n
  • \n
  • \n

    b

    \n
  • \n
  • \n

    d

    \n
  • \n
\n", "example": 278, "start_line": 5162, "end_line": 5180, "section": "Lists" }, { "markdown": "- a\n- ```\n b\n\n\n ```\n- c\n", "html": "
    \n
  • a
  • \n
  • \n
    b\n\n\n
    \n
  • \n
  • c
  • \n
\n", "example": 279, "start_line": 5185, "end_line": 5204, "section": "Lists" }, { "markdown": "- a\n - b\n\n c\n- d\n", "html": "
    \n
  • a\n
      \n
    • \n

      b

      \n

      c

      \n
    • \n
    \n
  • \n
  • d
  • \n
\n", "example": 280, "start_line": 5211, "end_line": 5229, "section": "Lists" }, { "markdown": "* a\n > b\n >\n* c\n", "html": "
    \n
  • a\n
    \n

    b

    \n
    \n
  • \n
  • c
  • \n
\n", "example": 281, "start_line": 5235, "end_line": 5249, "section": "Lists" }, { "markdown": "- a\n > b\n ```\n c\n ```\n- d\n", "html": "
    \n
  • a\n
    \n

    b

    \n
    \n
    c\n
    \n
  • \n
  • d
  • \n
\n", "example": 282, "start_line": 5255, "end_line": 5273, "section": "Lists" }, { "markdown": "- a\n", "html": "
    \n
  • a
  • \n
\n", "example": 283, "start_line": 5278, "end_line": 5284, "section": "Lists" }, { "markdown": "- a\n - b\n", "html": "
    \n
  • a\n
      \n
    • b
    • \n
    \n
  • \n
\n", "example": 284, "start_line": 5287, "end_line": 5298, "section": "Lists" }, { "markdown": "1. ```\n foo\n ```\n\n bar\n", "html": "
    \n
  1. \n
    foo\n
    \n

    bar

    \n
  2. \n
\n", "example": 285, "start_line": 5304, "end_line": 5318, "section": "Lists" }, { "markdown": "* foo\n * bar\n\n baz\n", "html": "
    \n
  • \n

    foo

    \n
      \n
    • bar
    • \n
    \n

    baz

    \n
  • \n
\n", "example": 286, "start_line": 5323, "end_line": 5338, "section": "Lists" }, { "markdown": "- a\n - b\n - c\n\n- d\n - e\n - f\n", "html": "
    \n
  • \n

    a

    \n
      \n
    • b
    • \n
    • c
    • \n
    \n
  • \n
  • \n

    d

    \n
      \n
    • e
    • \n
    • f
    • \n
    \n
  • \n
\n", "example": 287, "start_line": 5341, "end_line": 5366, "section": "Lists" }, { "markdown": "`hi`lo`\n", "html": "

hilo`

\n", "example": 288, "start_line": 5375, "end_line": 5379, "section": "Inlines" }, { "markdown": "\\!\\\"\\#\\$\\%\\&\\'\\(\\)\\*\\+\\,\\-\\.\\/\\:\\;\\<\\=\\>\\?\\@\\[\\\\\\]\\^\\_\\`\\{\\|\\}\\~\n", "html": "

!"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~

\n", "example": 289, "start_line": 5389, "end_line": 5393, "section": "Backslash escapes" }, { "markdown": "\\\t\\A\\a\\ \\3\\φ\\«\n", "html": "

\\\t\\A\\a\\ \\3\\φ\\«

\n", "example": 290, "start_line": 5399, "end_line": 5403, "section": "Backslash escapes" }, { "markdown": "\\*not emphasized*\n\\
not a tag\n\\[not a link](/foo)\n\\`not code`\n1\\. not a list\n\\* not a list\n\\# not a heading\n\\[foo]: /url \"not a reference\"\n", "html": "

*not emphasized*\n<br/> not a tag\n[not a link](/foo)\n`not code`\n1. not a list\n* not a list\n# not a heading\n[foo]: /url "not a reference"

\n", "example": 291, "start_line": 5409, "end_line": 5427, "section": "Backslash escapes" }, { "markdown": "\\\\*emphasis*\n", "html": "

\\emphasis

\n", "example": 292, "start_line": 5432, "end_line": 5436, "section": "Backslash escapes" }, { "markdown": "foo\\\nbar\n", "html": "

foo
\nbar

\n", "example": 293, "start_line": 5441, "end_line": 5447, "section": "Backslash escapes" }, { "markdown": "`` \\[\\` ``\n", "html": "

\\[\\`

\n", "example": 294, "start_line": 5453, "end_line": 5457, "section": "Backslash escapes" }, { "markdown": " \\[\\]\n", "html": "
\\[\\]\n
\n", "example": 295, "start_line": 5460, "end_line": 5465, "section": "Backslash escapes" }, { "markdown": "~~~\n\\[\\]\n~~~\n", "html": "
\\[\\]\n
\n", "example": 296, "start_line": 5468, "end_line": 5475, "section": "Backslash escapes" }, { "markdown": "\n", "html": "

http://example.com?find=\\*

\n", "example": 297, "start_line": 5478, "end_line": 5482, "section": "Backslash escapes" }, { "markdown": "\n", "html": "\n", "example": 298, "start_line": 5485, "end_line": 5489, "section": "Backslash escapes" }, { "markdown": "[foo](/bar\\* \"ti\\*tle\")\n", "html": "

foo

\n", "example": 299, "start_line": 5495, "end_line": 5499, "section": "Backslash escapes" }, { "markdown": "[foo]\n\n[foo]: /bar\\* \"ti\\*tle\"\n", "html": "

foo

\n", "example": 300, "start_line": 5502, "end_line": 5508, "section": "Backslash escapes" }, { "markdown": "``` foo\\+bar\nfoo\n```\n", "html": "
foo\n
\n", "example": 301, "start_line": 5511, "end_line": 5518, "section": "Backslash escapes" }, { "markdown": "  & © Æ Ď\n¾ ℋ ⅆ\n∲ ≧̸\n", "html": "

  & © Æ Ď\n¾ ℋ ⅆ\n∲ ≧̸

\n", "example": 302, "start_line": 5538, "end_line": 5546, "section": "Entity and numeric character references" }, { "markdown": "# Ӓ Ϡ � �\n", "html": "

# Ӓ Ϡ � �

\n", "example": 303, "start_line": 5557, "end_line": 5561, "section": "Entity and numeric character references" }, { "markdown": "" ആ ಫ\n", "html": "

" ആ ಫ

\n", "example": 304, "start_line": 5570, "end_line": 5574, "section": "Entity and numeric character references" }, { "markdown": "  &x; &#; &#x;\n&ThisIsNotDefined; &hi?;\n", "html": "

&nbsp &x; &#; &#x;\n&ThisIsNotDefined; &hi?;

\n", "example": 305, "start_line": 5579, "end_line": 5585, "section": "Entity and numeric character references" }, { "markdown": "©\n", "html": "

&copy

\n", "example": 306, "start_line": 5592, "end_line": 5596, "section": "Entity and numeric character references" }, { "markdown": "&MadeUpEntity;\n", "html": "

&MadeUpEntity;

\n", "example": 307, "start_line": 5602, "end_line": 5606, "section": "Entity and numeric character references" }, { "markdown": "\n", "html": "\n", "example": 308, "start_line": 5613, "end_line": 5617, "section": "Entity and numeric character references" }, { "markdown": "[foo](/föö \"föö\")\n", "html": "

foo

\n", "example": 309, "start_line": 5620, "end_line": 5624, "section": "Entity and numeric character references" }, { "markdown": "[foo]\n\n[foo]: /föö \"föö\"\n", "html": "

foo

\n", "example": 310, "start_line": 5627, "end_line": 5633, "section": "Entity and numeric character references" }, { "markdown": "``` föö\nfoo\n```\n", "html": "
foo\n
\n", "example": 311, "start_line": 5636, "end_line": 5643, "section": "Entity and numeric character references" }, { "markdown": "`föö`\n", "html": "

f&ouml;&ouml;

\n", "example": 312, "start_line": 5649, "end_line": 5653, "section": "Entity and numeric character references" }, { "markdown": " föfö\n", "html": "
f&ouml;f&ouml;\n
\n", "example": 313, "start_line": 5656, "end_line": 5661, "section": "Entity and numeric character references" }, { "markdown": "`foo`\n", "html": "

foo

\n", "example": 314, "start_line": 5678, "end_line": 5682, "section": "Code spans" }, { "markdown": "`` foo ` bar ``\n", "html": "

foo ` bar

\n", "example": 315, "start_line": 5688, "end_line": 5692, "section": "Code spans" }, { "markdown": "` `` `\n", "html": "

``

\n", "example": 316, "start_line": 5698, "end_line": 5702, "section": "Code spans" }, { "markdown": "``\nfoo\n``\n", "html": "

foo

\n", "example": 317, "start_line": 5707, "end_line": 5713, "section": "Code spans" }, { "markdown": "`foo bar\n baz`\n", "html": "

foo bar baz

\n", "example": 318, "start_line": 5719, "end_line": 5724, "section": "Code spans" }, { "markdown": "`a  b`\n", "html": "

a  b

\n", "example": 319, "start_line": 5730, "end_line": 5734, "section": "Code spans" }, { "markdown": "`foo `` bar`\n", "html": "

foo `` bar

\n", "example": 320, "start_line": 5750, "end_line": 5754, "section": "Code spans" }, { "markdown": "`foo\\`bar`\n", "html": "

foo\\bar`

\n", "example": 321, "start_line": 5760, "end_line": 5764, "section": "Code spans" }, { "markdown": "*foo`*`\n", "html": "

*foo*

\n", "example": 322, "start_line": 5776, "end_line": 5780, "section": "Code spans" }, { "markdown": "[not a `link](/foo`)\n", "html": "

[not a link](/foo)

\n", "example": 323, "start_line": 5785, "end_line": 5789, "section": "Code spans" }, { "markdown": "``\n", "html": "

<a href="">`

\n", "example": 324, "start_line": 5795, "end_line": 5799, "section": "Code spans" }, { "markdown": "
`\n", "html": "

`

\n", "example": 325, "start_line": 5804, "end_line": 5808, "section": "Code spans" }, { "markdown": "``\n", "html": "

<http://foo.bar.baz>`

\n", "example": 326, "start_line": 5813, "end_line": 5817, "section": "Code spans" }, { "markdown": "`\n", "html": "

http://foo.bar.`baz`

\n", "example": 327, "start_line": 5822, "end_line": 5826, "section": "Code spans" }, { "markdown": "```foo``\n", "html": "

```foo``

\n", "example": 328, "start_line": 5832, "end_line": 5836, "section": "Code spans" }, { "markdown": "`foo\n", "html": "

`foo

\n", "example": 329, "start_line": 5839, "end_line": 5843, "section": "Code spans" }, { "markdown": "`foo``bar``\n", "html": "

`foobar

\n", "example": 330, "start_line": 5848, "end_line": 5852, "section": "Code spans" }, { "markdown": "*foo bar*\n", "html": "

foo bar

\n", "example": 331, "start_line": 6061, "end_line": 6065, "section": "Emphasis and strong emphasis" }, { "markdown": "a * foo bar*\n", "html": "

a * foo bar*

\n", "example": 332, "start_line": 6071, "end_line": 6075, "section": "Emphasis and strong emphasis" }, { "markdown": "a*\"foo\"*\n", "html": "

a*"foo"*

\n", "example": 333, "start_line": 6082, "end_line": 6086, "section": "Emphasis and strong emphasis" }, { "markdown": "* a *\n", "html": "

* a *

\n", "example": 334, "start_line": 6091, "end_line": 6095, "section": "Emphasis and strong emphasis" }, { "markdown": "foo*bar*\n", "html": "

foobar

\n", "example": 335, "start_line": 6100, "end_line": 6104, "section": "Emphasis and strong emphasis" }, { "markdown": "5*6*78\n", "html": "

5678

\n", "example": 336, "start_line": 6107, "end_line": 6111, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo bar_\n", "html": "

foo bar

\n", "example": 337, "start_line": 6116, "end_line": 6120, "section": "Emphasis and strong emphasis" }, { "markdown": "_ foo bar_\n", "html": "

_ foo bar_

\n", "example": 338, "start_line": 6126, "end_line": 6130, "section": "Emphasis and strong emphasis" }, { "markdown": "a_\"foo\"_\n", "html": "

a_"foo"_

\n", "example": 339, "start_line": 6136, "end_line": 6140, "section": "Emphasis and strong emphasis" }, { "markdown": "foo_bar_\n", "html": "

foo_bar_

\n", "example": 340, "start_line": 6145, "end_line": 6149, "section": "Emphasis and strong emphasis" }, { "markdown": "5_6_78\n", "html": "

5_6_78

\n", "example": 341, "start_line": 6152, "end_line": 6156, "section": "Emphasis and strong emphasis" }, { "markdown": "пристаням_стремятся_\n", "html": "

пристаням_стремятся_

\n", "example": 342, "start_line": 6159, "end_line": 6163, "section": "Emphasis and strong emphasis" }, { "markdown": "aa_\"bb\"_cc\n", "html": "

aa_"bb"_cc

\n", "example": 343, "start_line": 6169, "end_line": 6173, "section": "Emphasis and strong emphasis" }, { "markdown": "foo-_(bar)_\n", "html": "

foo-(bar)

\n", "example": 344, "start_line": 6180, "end_line": 6184, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo*\n", "html": "

_foo*

\n", "example": 345, "start_line": 6192, "end_line": 6196, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo bar *\n", "html": "

*foo bar *

\n", "example": 346, "start_line": 6202, "end_line": 6206, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo bar\n*\n", "html": "

*foo bar\n*

\n", "example": 347, "start_line": 6211, "end_line": 6217, "section": "Emphasis and strong emphasis" }, { "markdown": "*(*foo)\n", "html": "

*(*foo)

\n", "example": 348, "start_line": 6224, "end_line": 6228, "section": "Emphasis and strong emphasis" }, { "markdown": "*(*foo*)*\n", "html": "

(foo)

\n", "example": 349, "start_line": 6234, "end_line": 6238, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo*bar\n", "html": "

foobar

\n", "example": 350, "start_line": 6243, "end_line": 6247, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo bar _\n", "html": "

_foo bar _

\n", "example": 351, "start_line": 6256, "end_line": 6260, "section": "Emphasis and strong emphasis" }, { "markdown": "_(_foo)\n", "html": "

_(_foo)

\n", "example": 352, "start_line": 6266, "end_line": 6270, "section": "Emphasis and strong emphasis" }, { "markdown": "_(_foo_)_\n", "html": "

(foo)

\n", "example": 353, "start_line": 6275, "end_line": 6279, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo_bar\n", "html": "

_foo_bar

\n", "example": 354, "start_line": 6284, "end_line": 6288, "section": "Emphasis and strong emphasis" }, { "markdown": "_пристаням_стремятся\n", "html": "

_пристаням_стремятся

\n", "example": 355, "start_line": 6291, "end_line": 6295, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo_bar_baz_\n", "html": "

foo_bar_baz

\n", "example": 356, "start_line": 6298, "end_line": 6302, "section": "Emphasis and strong emphasis" }, { "markdown": "_(bar)_.\n", "html": "

(bar).

\n", "example": 357, "start_line": 6309, "end_line": 6313, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo bar**\n", "html": "

foo bar

\n", "example": 358, "start_line": 6318, "end_line": 6322, "section": "Emphasis and strong emphasis" }, { "markdown": "** foo bar**\n", "html": "

** foo bar**

\n", "example": 359, "start_line": 6328, "end_line": 6332, "section": "Emphasis and strong emphasis" }, { "markdown": "a**\"foo\"**\n", "html": "

a**"foo"**

\n", "example": 360, "start_line": 6339, "end_line": 6343, "section": "Emphasis and strong emphasis" }, { "markdown": "foo**bar**\n", "html": "

foobar

\n", "example": 361, "start_line": 6348, "end_line": 6352, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo bar__\n", "html": "

foo bar

\n", "example": 362, "start_line": 6357, "end_line": 6361, "section": "Emphasis and strong emphasis" }, { "markdown": "__ foo bar__\n", "html": "

__ foo bar__

\n", "example": 363, "start_line": 6367, "end_line": 6371, "section": "Emphasis and strong emphasis" }, { "markdown": "__\nfoo bar__\n", "html": "

__\nfoo bar__

\n", "example": 364, "start_line": 6375, "end_line": 6381, "section": "Emphasis and strong emphasis" }, { "markdown": "a__\"foo\"__\n", "html": "

a__"foo"__

\n", "example": 365, "start_line": 6387, "end_line": 6391, "section": "Emphasis and strong emphasis" }, { "markdown": "foo__bar__\n", "html": "

foo__bar__

\n", "example": 366, "start_line": 6396, "end_line": 6400, "section": "Emphasis and strong emphasis" }, { "markdown": "5__6__78\n", "html": "

5__6__78

\n", "example": 367, "start_line": 6403, "end_line": 6407, "section": "Emphasis and strong emphasis" }, { "markdown": "пристаням__стремятся__\n", "html": "

пристаням__стремятся__

\n", "example": 368, "start_line": 6410, "end_line": 6414, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo, __bar__, baz__\n", "html": "

foo, bar, baz

\n", "example": 369, "start_line": 6417, "end_line": 6421, "section": "Emphasis and strong emphasis" }, { "markdown": "foo-__(bar)__\n", "html": "

foo-(bar)

\n", "example": 370, "start_line": 6428, "end_line": 6432, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo bar **\n", "html": "

**foo bar **

\n", "example": 371, "start_line": 6441, "end_line": 6445, "section": "Emphasis and strong emphasis" }, { "markdown": "**(**foo)\n", "html": "

**(**foo)

\n", "example": 372, "start_line": 6454, "end_line": 6458, "section": "Emphasis and strong emphasis" }, { "markdown": "*(**foo**)*\n", "html": "

(foo)

\n", "example": 373, "start_line": 6464, "end_line": 6468, "section": "Emphasis and strong emphasis" }, { "markdown": "**Gomphocarpus (*Gomphocarpus physocarpus*, syn.\n*Asclepias physocarpa*)**\n", "html": "

Gomphocarpus (Gomphocarpus physocarpus, syn.\nAsclepias physocarpa)

\n", "example": 374, "start_line": 6471, "end_line": 6477, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo \"*bar*\" foo**\n", "html": "

foo "bar" foo

\n", "example": 375, "start_line": 6480, "end_line": 6484, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo**bar\n", "html": "

foobar

\n", "example": 376, "start_line": 6489, "end_line": 6493, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo bar __\n", "html": "

__foo bar __

\n", "example": 377, "start_line": 6501, "end_line": 6505, "section": "Emphasis and strong emphasis" }, { "markdown": "__(__foo)\n", "html": "

__(__foo)

\n", "example": 378, "start_line": 6511, "end_line": 6515, "section": "Emphasis and strong emphasis" }, { "markdown": "_(__foo__)_\n", "html": "

(foo)

\n", "example": 379, "start_line": 6521, "end_line": 6525, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo__bar\n", "html": "

__foo__bar

\n", "example": 380, "start_line": 6530, "end_line": 6534, "section": "Emphasis and strong emphasis" }, { "markdown": "__пристаням__стремятся\n", "html": "

__пристаням__стремятся

\n", "example": 381, "start_line": 6537, "end_line": 6541, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo__bar__baz__\n", "html": "

foo__bar__baz

\n", "example": 382, "start_line": 6544, "end_line": 6548, "section": "Emphasis and strong emphasis" }, { "markdown": "__(bar)__.\n", "html": "

(bar).

\n", "example": 383, "start_line": 6555, "end_line": 6559, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo [bar](/url)*\n", "html": "

foo bar

\n", "example": 384, "start_line": 6567, "end_line": 6571, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo\nbar*\n", "html": "

foo\nbar

\n", "example": 385, "start_line": 6574, "end_line": 6580, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo __bar__ baz_\n", "html": "

foo bar baz

\n", "example": 386, "start_line": 6586, "end_line": 6590, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo _bar_ baz_\n", "html": "

foo bar baz

\n", "example": 387, "start_line": 6593, "end_line": 6597, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo_ bar_\n", "html": "

foo bar

\n", "example": 388, "start_line": 6600, "end_line": 6604, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo *bar**\n", "html": "

foo bar

\n", "example": 389, "start_line": 6607, "end_line": 6611, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo **bar** baz*\n", "html": "

foo bar baz

\n", "example": 390, "start_line": 6614, "end_line": 6618, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo**bar**baz*\n", "html": "

foobarbaz

\n", "example": 391, "start_line": 6620, "end_line": 6624, "section": "Emphasis and strong emphasis" }, { "markdown": "***foo** bar*\n", "html": "

foo bar

\n", "example": 392, "start_line": 6645, "end_line": 6649, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo **bar***\n", "html": "

foo bar

\n", "example": 393, "start_line": 6652, "end_line": 6656, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo**bar***\n", "html": "

foobar

\n", "example": 394, "start_line": 6659, "end_line": 6663, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo **bar *baz* bim** bop*\n", "html": "

foo bar baz bim bop

\n", "example": 395, "start_line": 6668, "end_line": 6672, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo [*bar*](/url)*\n", "html": "

foo bar

\n", "example": 396, "start_line": 6675, "end_line": 6679, "section": "Emphasis and strong emphasis" }, { "markdown": "** is not an empty emphasis\n", "html": "

** is not an empty emphasis

\n", "example": 397, "start_line": 6684, "end_line": 6688, "section": "Emphasis and strong emphasis" }, { "markdown": "**** is not an empty strong emphasis\n", "html": "

**** is not an empty strong emphasis

\n", "example": 398, "start_line": 6691, "end_line": 6695, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo [bar](/url)**\n", "html": "

foo bar

\n", "example": 399, "start_line": 6704, "end_line": 6708, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo\nbar**\n", "html": "

foo\nbar

\n", "example": 400, "start_line": 6711, "end_line": 6717, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo _bar_ baz__\n", "html": "

foo bar baz

\n", "example": 401, "start_line": 6723, "end_line": 6727, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo __bar__ baz__\n", "html": "

foo bar baz

\n", "example": 402, "start_line": 6730, "end_line": 6734, "section": "Emphasis and strong emphasis" }, { "markdown": "____foo__ bar__\n", "html": "

foo bar

\n", "example": 403, "start_line": 6737, "end_line": 6741, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo **bar****\n", "html": "

foo bar

\n", "example": 404, "start_line": 6744, "end_line": 6748, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo *bar* baz**\n", "html": "

foo bar baz

\n", "example": 405, "start_line": 6751, "end_line": 6755, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo*bar*baz**\n", "html": "

foobarbaz

\n", "example": 406, "start_line": 6758, "end_line": 6762, "section": "Emphasis and strong emphasis" }, { "markdown": "***foo* bar**\n", "html": "

foo bar

\n", "example": 407, "start_line": 6765, "end_line": 6769, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo *bar***\n", "html": "

foo bar

\n", "example": 408, "start_line": 6772, "end_line": 6776, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo *bar **baz**\nbim* bop**\n", "html": "

foo bar baz\nbim bop

\n", "example": 409, "start_line": 6781, "end_line": 6787, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo [*bar*](/url)**\n", "html": "

foo bar

\n", "example": 410, "start_line": 6790, "end_line": 6794, "section": "Emphasis and strong emphasis" }, { "markdown": "__ is not an empty emphasis\n", "html": "

__ is not an empty emphasis

\n", "example": 411, "start_line": 6799, "end_line": 6803, "section": "Emphasis and strong emphasis" }, { "markdown": "____ is not an empty strong emphasis\n", "html": "

____ is not an empty strong emphasis

\n", "example": 412, "start_line": 6806, "end_line": 6810, "section": "Emphasis and strong emphasis" }, { "markdown": "foo ***\n", "html": "

foo ***

\n", "example": 413, "start_line": 6816, "end_line": 6820, "section": "Emphasis and strong emphasis" }, { "markdown": "foo *\\**\n", "html": "

foo *

\n", "example": 414, "start_line": 6823, "end_line": 6827, "section": "Emphasis and strong emphasis" }, { "markdown": "foo *_*\n", "html": "

foo _

\n", "example": 415, "start_line": 6830, "end_line": 6834, "section": "Emphasis and strong emphasis" }, { "markdown": "foo *****\n", "html": "

foo *****

\n", "example": 416, "start_line": 6837, "end_line": 6841, "section": "Emphasis and strong emphasis" }, { "markdown": "foo **\\***\n", "html": "

foo *

\n", "example": 417, "start_line": 6844, "end_line": 6848, "section": "Emphasis and strong emphasis" }, { "markdown": "foo **_**\n", "html": "

foo _

\n", "example": 418, "start_line": 6851, "end_line": 6855, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo*\n", "html": "

*foo

\n", "example": 419, "start_line": 6862, "end_line": 6866, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo**\n", "html": "

foo*

\n", "example": 420, "start_line": 6869, "end_line": 6873, "section": "Emphasis and strong emphasis" }, { "markdown": "***foo**\n", "html": "

*foo

\n", "example": 421, "start_line": 6876, "end_line": 6880, "section": "Emphasis and strong emphasis" }, { "markdown": "****foo*\n", "html": "

***foo

\n", "example": 422, "start_line": 6883, "end_line": 6887, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo***\n", "html": "

foo*

\n", "example": 423, "start_line": 6890, "end_line": 6894, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo****\n", "html": "

foo***

\n", "example": 424, "start_line": 6897, "end_line": 6901, "section": "Emphasis and strong emphasis" }, { "markdown": "foo ___\n", "html": "

foo ___

\n", "example": 425, "start_line": 6907, "end_line": 6911, "section": "Emphasis and strong emphasis" }, { "markdown": "foo _\\__\n", "html": "

foo _

\n", "example": 426, "start_line": 6914, "end_line": 6918, "section": "Emphasis and strong emphasis" }, { "markdown": "foo _*_\n", "html": "

foo *

\n", "example": 427, "start_line": 6921, "end_line": 6925, "section": "Emphasis and strong emphasis" }, { "markdown": "foo _____\n", "html": "

foo _____

\n", "example": 428, "start_line": 6928, "end_line": 6932, "section": "Emphasis and strong emphasis" }, { "markdown": "foo __\\___\n", "html": "

foo _

\n", "example": 429, "start_line": 6935, "end_line": 6939, "section": "Emphasis and strong emphasis" }, { "markdown": "foo __*__\n", "html": "

foo *

\n", "example": 430, "start_line": 6942, "end_line": 6946, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo_\n", "html": "

_foo

\n", "example": 431, "start_line": 6949, "end_line": 6953, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo__\n", "html": "

foo_

\n", "example": 432, "start_line": 6960, "end_line": 6964, "section": "Emphasis and strong emphasis" }, { "markdown": "___foo__\n", "html": "

_foo

\n", "example": 433, "start_line": 6967, "end_line": 6971, "section": "Emphasis and strong emphasis" }, { "markdown": "____foo_\n", "html": "

___foo

\n", "example": 434, "start_line": 6974, "end_line": 6978, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo___\n", "html": "

foo_

\n", "example": 435, "start_line": 6981, "end_line": 6985, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo____\n", "html": "

foo___

\n", "example": 436, "start_line": 6988, "end_line": 6992, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo**\n", "html": "

foo

\n", "example": 437, "start_line": 6998, "end_line": 7002, "section": "Emphasis and strong emphasis" }, { "markdown": "*_foo_*\n", "html": "

foo

\n", "example": 438, "start_line": 7005, "end_line": 7009, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo__\n", "html": "

foo

\n", "example": 439, "start_line": 7012, "end_line": 7016, "section": "Emphasis and strong emphasis" }, { "markdown": "_*foo*_\n", "html": "

foo

\n", "example": 440, "start_line": 7019, "end_line": 7023, "section": "Emphasis and strong emphasis" }, { "markdown": "****foo****\n", "html": "

foo

\n", "example": 441, "start_line": 7029, "end_line": 7033, "section": "Emphasis and strong emphasis" }, { "markdown": "____foo____\n", "html": "

foo

\n", "example": 442, "start_line": 7036, "end_line": 7040, "section": "Emphasis and strong emphasis" }, { "markdown": "******foo******\n", "html": "

foo

\n", "example": 443, "start_line": 7047, "end_line": 7051, "section": "Emphasis and strong emphasis" }, { "markdown": "***foo***\n", "html": "

foo

\n", "example": 444, "start_line": 7056, "end_line": 7060, "section": "Emphasis and strong emphasis" }, { "markdown": "_____foo_____\n", "html": "

foo

\n", "example": 445, "start_line": 7063, "end_line": 7067, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo _bar* baz_\n", "html": "

foo _bar baz_

\n", "example": 446, "start_line": 7072, "end_line": 7076, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo __bar *baz bim__ bam*\n", "html": "

foo bar *baz bim bam

\n", "example": 447, "start_line": 7079, "end_line": 7083, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo **bar baz**\n", "html": "

**foo bar baz

\n", "example": 448, "start_line": 7088, "end_line": 7092, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo *bar baz*\n", "html": "

*foo bar baz

\n", "example": 449, "start_line": 7095, "end_line": 7099, "section": "Emphasis and strong emphasis" }, { "markdown": "*[bar*](/url)\n", "html": "

*bar*

\n", "example": 450, "start_line": 7104, "end_line": 7108, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo [bar_](/url)\n", "html": "

_foo bar_

\n", "example": 451, "start_line": 7111, "end_line": 7115, "section": "Emphasis and strong emphasis" }, { "markdown": "*\n", "html": "

*

\n", "example": 452, "start_line": 7118, "end_line": 7122, "section": "Emphasis and strong emphasis" }, { "markdown": "**\n", "html": "

**

\n", "example": 453, "start_line": 7125, "end_line": 7129, "section": "Emphasis and strong emphasis" }, { "markdown": "__\n", "html": "

__

\n", "example": 454, "start_line": 7132, "end_line": 7136, "section": "Emphasis and strong emphasis" }, { "markdown": "*a `*`*\n", "html": "

a *

\n", "example": 455, "start_line": 7139, "end_line": 7143, "section": "Emphasis and strong emphasis" }, { "markdown": "_a `_`_\n", "html": "

a _

\n", "example": 456, "start_line": 7146, "end_line": 7150, "section": "Emphasis and strong emphasis" }, { "markdown": "**a\n", "html": "

**ahttp://foo.bar/?q=**

\n", "example": 457, "start_line": 7153, "end_line": 7157, "section": "Emphasis and strong emphasis" }, { "markdown": "__a\n", "html": "

__ahttp://foo.bar/?q=__

\n", "example": 458, "start_line": 7160, "end_line": 7164, "section": "Emphasis and strong emphasis" }, { "markdown": "[link](/uri \"title\")\n", "html": "

link

\n", "example": 459, "start_line": 7241, "end_line": 7245, "section": "Links" }, { "markdown": "[link](/uri)\n", "html": "

link

\n", "example": 460, "start_line": 7250, "end_line": 7254, "section": "Links" }, { "markdown": "[link]()\n", "html": "

link

\n", "example": 461, "start_line": 7259, "end_line": 7263, "section": "Links" }, { "markdown": "[link](<>)\n", "html": "

link

\n", "example": 462, "start_line": 7266, "end_line": 7270, "section": "Links" }, { "markdown": "[link](/my uri)\n", "html": "

[link](/my uri)

\n", "example": 463, "start_line": 7276, "end_line": 7280, "section": "Links" }, { "markdown": "[link]()\n", "html": "

[link](</my uri>)

\n", "example": 464, "start_line": 7283, "end_line": 7287, "section": "Links" }, { "markdown": "[link](foo\nbar)\n", "html": "

[link](foo\nbar)

\n", "example": 465, "start_line": 7290, "end_line": 7296, "section": "Links" }, { "markdown": "[link]()\n", "html": "

[link]()

\n", "example": 466, "start_line": 7299, "end_line": 7305, "section": "Links" }, { "markdown": "[link](\\(foo\\))\n", "html": "

link

\n", "example": 467, "start_line": 7309, "end_line": 7313, "section": "Links" }, { "markdown": "[link](foo(and(bar)))\n", "html": "

link

\n", "example": 468, "start_line": 7318, "end_line": 7322, "section": "Links" }, { "markdown": "[link](foo\\(and\\(bar\\))\n", "html": "

link

\n", "example": 469, "start_line": 7327, "end_line": 7331, "section": "Links" }, { "markdown": "[link]()\n", "html": "

link

\n", "example": 470, "start_line": 7334, "end_line": 7338, "section": "Links" }, { "markdown": "[link](foo\\)\\:)\n", "html": "

link

\n", "example": 471, "start_line": 7344, "end_line": 7348, "section": "Links" }, { "markdown": "[link](#fragment)\n\n[link](http://example.com#fragment)\n\n[link](http://example.com?foo=3#frag)\n", "html": "

link

\n

link

\n

link

\n", "example": 472, "start_line": 7353, "end_line": 7363, "section": "Links" }, { "markdown": "[link](foo\\bar)\n", "html": "

link

\n", "example": 473, "start_line": 7369, "end_line": 7373, "section": "Links" }, { "markdown": "[link](foo%20bä)\n", "html": "

link

\n", "example": 474, "start_line": 7385, "end_line": 7389, "section": "Links" }, { "markdown": "[link](\"title\")\n", "html": "

link

\n", "example": 475, "start_line": 7396, "end_line": 7400, "section": "Links" }, { "markdown": "[link](/url \"title\")\n[link](/url 'title')\n[link](/url (title))\n", "html": "

link\nlink\nlink

\n", "example": 476, "start_line": 7405, "end_line": 7413, "section": "Links" }, { "markdown": "[link](/url \"title \\\""\")\n", "html": "

link

\n", "example": 477, "start_line": 7419, "end_line": 7423, "section": "Links" }, { "markdown": "[link](/url \"title\")\n", "html": "

link

\n", "example": 478, "start_line": 7429, "end_line": 7433, "section": "Links" }, { "markdown": "[link](/url \"title \"and\" title\")\n", "html": "

[link](/url "title "and" title")

\n", "example": 479, "start_line": 7438, "end_line": 7442, "section": "Links" }, { "markdown": "[link](/url 'title \"and\" title')\n", "html": "

link

\n", "example": 480, "start_line": 7447, "end_line": 7451, "section": "Links" }, { "markdown": "[link]( /uri\n \"title\" )\n", "html": "

link

\n", "example": 481, "start_line": 7471, "end_line": 7476, "section": "Links" }, { "markdown": "[link] (/uri)\n", "html": "

[link] (/uri)

\n", "example": 482, "start_line": 7482, "end_line": 7486, "section": "Links" }, { "markdown": "[link [foo [bar]]](/uri)\n", "html": "

link [foo [bar]]

\n", "example": 483, "start_line": 7492, "end_line": 7496, "section": "Links" }, { "markdown": "[link] bar](/uri)\n", "html": "

[link] bar](/uri)

\n", "example": 484, "start_line": 7499, "end_line": 7503, "section": "Links" }, { "markdown": "[link [bar](/uri)\n", "html": "

[link bar

\n", "example": 485, "start_line": 7506, "end_line": 7510, "section": "Links" }, { "markdown": "[link \\[bar](/uri)\n", "html": "

link [bar

\n", "example": 486, "start_line": 7513, "end_line": 7517, "section": "Links" }, { "markdown": "[link *foo **bar** `#`*](/uri)\n", "html": "

link foo bar #

\n", "example": 487, "start_line": 7522, "end_line": 7526, "section": "Links" }, { "markdown": "[![moon](moon.jpg)](/uri)\n", "html": "

\"moon\"

\n", "example": 488, "start_line": 7529, "end_line": 7533, "section": "Links" }, { "markdown": "[foo [bar](/uri)](/uri)\n", "html": "

[foo bar](/uri)

\n", "example": 489, "start_line": 7538, "end_line": 7542, "section": "Links" }, { "markdown": "[foo *[bar [baz](/uri)](/uri)*](/uri)\n", "html": "

[foo [bar baz](/uri)](/uri)

\n", "example": 490, "start_line": 7545, "end_line": 7549, "section": "Links" }, { "markdown": "![[[foo](uri1)](uri2)](uri3)\n", "html": "

\"[foo](uri2)\"

\n", "example": 491, "start_line": 7552, "end_line": 7556, "section": "Links" }, { "markdown": "*[foo*](/uri)\n", "html": "

*foo*

\n", "example": 492, "start_line": 7562, "end_line": 7566, "section": "Links" }, { "markdown": "[foo *bar](baz*)\n", "html": "

foo *bar

\n", "example": 493, "start_line": 7569, "end_line": 7573, "section": "Links" }, { "markdown": "*foo [bar* baz]\n", "html": "

foo [bar baz]

\n", "example": 494, "start_line": 7579, "end_line": 7583, "section": "Links" }, { "markdown": "[foo \n", "html": "

[foo

\n", "example": 495, "start_line": 7589, "end_line": 7593, "section": "Links" }, { "markdown": "[foo`](/uri)`\n", "html": "

[foo](/uri)

\n", "example": 496, "start_line": 7596, "end_line": 7600, "section": "Links" }, { "markdown": "[foo\n", "html": "

[foohttp://example.com/?search=](uri)

\n", "example": 497, "start_line": 7603, "end_line": 7607, "section": "Links" }, { "markdown": "[foo][bar]\n\n[bar]: /url \"title\"\n", "html": "

foo

\n", "example": 498, "start_line": 7641, "end_line": 7647, "section": "Links" }, { "markdown": "[link [foo [bar]]][ref]\n\n[ref]: /uri\n", "html": "

link [foo [bar]]

\n", "example": 499, "start_line": 7656, "end_line": 7662, "section": "Links" }, { "markdown": "[link \\[bar][ref]\n\n[ref]: /uri\n", "html": "

link [bar

\n", "example": 500, "start_line": 7665, "end_line": 7671, "section": "Links" }, { "markdown": "[link *foo **bar** `#`*][ref]\n\n[ref]: /uri\n", "html": "

link foo bar #

\n", "example": 501, "start_line": 7676, "end_line": 7682, "section": "Links" }, { "markdown": "[![moon](moon.jpg)][ref]\n\n[ref]: /uri\n", "html": "

\"moon\"

\n", "example": 502, "start_line": 7685, "end_line": 7691, "section": "Links" }, { "markdown": "[foo [bar](/uri)][ref]\n\n[ref]: /uri\n", "html": "

[foo bar]ref

\n", "example": 503, "start_line": 7696, "end_line": 7702, "section": "Links" }, { "markdown": "[foo *bar [baz][ref]*][ref]\n\n[ref]: /uri\n", "html": "

[foo bar baz]ref

\n", "example": 504, "start_line": 7705, "end_line": 7711, "section": "Links" }, { "markdown": "*[foo*][ref]\n\n[ref]: /uri\n", "html": "

*foo*

\n", "example": 505, "start_line": 7720, "end_line": 7726, "section": "Links" }, { "markdown": "[foo *bar][ref]\n\n[ref]: /uri\n", "html": "

foo *bar

\n", "example": 506, "start_line": 7729, "end_line": 7735, "section": "Links" }, { "markdown": "[foo \n\n[ref]: /uri\n", "html": "

[foo

\n", "example": 507, "start_line": 7741, "end_line": 7747, "section": "Links" }, { "markdown": "[foo`][ref]`\n\n[ref]: /uri\n", "html": "

[foo][ref]

\n", "example": 508, "start_line": 7750, "end_line": 7756, "section": "Links" }, { "markdown": "[foo\n\n[ref]: /uri\n", "html": "

[foohttp://example.com/?search=][ref]

\n", "example": 509, "start_line": 7759, "end_line": 7765, "section": "Links" }, { "markdown": "[foo][BaR]\n\n[bar]: /url \"title\"\n", "html": "

foo

\n", "example": 510, "start_line": 7770, "end_line": 7776, "section": "Links" }, { "markdown": "[Толпой][Толпой] is a Russian word.\n\n[ТОЛПОЙ]: /url\n", "html": "

Толпой is a Russian word.

\n", "example": 511, "start_line": 7781, "end_line": 7787, "section": "Links" }, { "markdown": "[Foo\n bar]: /url\n\n[Baz][Foo bar]\n", "html": "

Baz

\n", "example": 512, "start_line": 7793, "end_line": 7800, "section": "Links" }, { "markdown": "[foo] [bar]\n\n[bar]: /url \"title\"\n", "html": "

[foo] bar

\n", "example": 513, "start_line": 7806, "end_line": 7812, "section": "Links" }, { "markdown": "[foo]\n[bar]\n\n[bar]: /url \"title\"\n", "html": "

[foo]\nbar

\n", "example": 514, "start_line": 7815, "end_line": 7823, "section": "Links" }, { "markdown": "[foo]: /url1\n\n[foo]: /url2\n\n[bar][foo]\n", "html": "

bar

\n", "example": 515, "start_line": 7856, "end_line": 7864, "section": "Links" }, { "markdown": "[bar][foo\\!]\n\n[foo!]: /url\n", "html": "

[bar][foo!]

\n", "example": 516, "start_line": 7871, "end_line": 7877, "section": "Links" }, { "markdown": "[foo][ref[]\n\n[ref[]: /uri\n", "html": "

[foo][ref[]

\n

[ref[]: /uri

\n", "example": 517, "start_line": 7883, "end_line": 7890, "section": "Links" }, { "markdown": "[foo][ref[bar]]\n\n[ref[bar]]: /uri\n", "html": "

[foo][ref[bar]]

\n

[ref[bar]]: /uri

\n", "example": 518, "start_line": 7893, "end_line": 7900, "section": "Links" }, { "markdown": "[[[foo]]]\n\n[[[foo]]]: /url\n", "html": "

[[[foo]]]

\n

[[[foo]]]: /url

\n", "example": 519, "start_line": 7903, "end_line": 7910, "section": "Links" }, { "markdown": "[foo][ref\\[]\n\n[ref\\[]: /uri\n", "html": "

foo

\n", "example": 520, "start_line": 7913, "end_line": 7919, "section": "Links" }, { "markdown": "[bar\\\\]: /uri\n\n[bar\\\\]\n", "html": "

bar\\

\n", "example": 521, "start_line": 7924, "end_line": 7930, "section": "Links" }, { "markdown": "[]\n\n[]: /uri\n", "html": "

[]

\n

[]: /uri

\n", "example": 522, "start_line": 7935, "end_line": 7942, "section": "Links" }, { "markdown": "[\n ]\n\n[\n ]: /uri\n", "html": "

[\n]

\n

[\n]: /uri

\n", "example": 523, "start_line": 7945, "end_line": 7956, "section": "Links" }, { "markdown": "[foo][]\n\n[foo]: /url \"title\"\n", "html": "

foo

\n", "example": 524, "start_line": 7968, "end_line": 7974, "section": "Links" }, { "markdown": "[*foo* bar][]\n\n[*foo* bar]: /url \"title\"\n", "html": "

foo bar

\n", "example": 525, "start_line": 7977, "end_line": 7983, "section": "Links" }, { "markdown": "[Foo][]\n\n[foo]: /url \"title\"\n", "html": "

Foo

\n", "example": 526, "start_line": 7988, "end_line": 7994, "section": "Links" }, { "markdown": "[foo] \n[]\n\n[foo]: /url \"title\"\n", "html": "

foo\n[]

\n", "example": 527, "start_line": 8001, "end_line": 8009, "section": "Links" }, { "markdown": "[foo]\n\n[foo]: /url \"title\"\n", "html": "

foo

\n", "example": 528, "start_line": 8021, "end_line": 8027, "section": "Links" }, { "markdown": "[*foo* bar]\n\n[*foo* bar]: /url \"title\"\n", "html": "

foo bar

\n", "example": 529, "start_line": 8030, "end_line": 8036, "section": "Links" }, { "markdown": "[[*foo* bar]]\n\n[*foo* bar]: /url \"title\"\n", "html": "

[foo bar]

\n", "example": 530, "start_line": 8039, "end_line": 8045, "section": "Links" }, { "markdown": "[[bar [foo]\n\n[foo]: /url\n", "html": "

[[bar foo

\n", "example": 531, "start_line": 8048, "end_line": 8054, "section": "Links" }, { "markdown": "[Foo]\n\n[foo]: /url \"title\"\n", "html": "

Foo

\n", "example": 532, "start_line": 8059, "end_line": 8065, "section": "Links" }, { "markdown": "[foo] bar\n\n[foo]: /url\n", "html": "

foo bar

\n", "example": 533, "start_line": 8070, "end_line": 8076, "section": "Links" }, { "markdown": "\\[foo]\n\n[foo]: /url \"title\"\n", "html": "

[foo]

\n", "example": 534, "start_line": 8082, "end_line": 8088, "section": "Links" }, { "markdown": "[foo*]: /url\n\n*[foo*]\n", "html": "

*foo*

\n", "example": 535, "start_line": 8094, "end_line": 8100, "section": "Links" }, { "markdown": "[foo][bar]\n\n[foo]: /url1\n[bar]: /url2\n", "html": "

foo

\n", "example": 536, "start_line": 8106, "end_line": 8113, "section": "Links" }, { "markdown": "[foo][]\n\n[foo]: /url1\n", "html": "

foo

\n", "example": 537, "start_line": 8115, "end_line": 8121, "section": "Links" }, { "markdown": "[foo]()\n\n[foo]: /url1\n", "html": "

foo

\n", "example": 538, "start_line": 8125, "end_line": 8131, "section": "Links" }, { "markdown": "[foo](not a link)\n\n[foo]: /url1\n", "html": "

foo(not a link)

\n", "example": 539, "start_line": 8133, "end_line": 8139, "section": "Links" }, { "markdown": "[foo][bar][baz]\n\n[baz]: /url\n", "html": "

[foo]bar

\n", "example": 540, "start_line": 8144, "end_line": 8150, "section": "Links" }, { "markdown": "[foo][bar][baz]\n\n[baz]: /url1\n[bar]: /url2\n", "html": "

foobaz

\n", "example": 541, "start_line": 8156, "end_line": 8163, "section": "Links" }, { "markdown": "[foo][bar][baz]\n\n[baz]: /url1\n[foo]: /url2\n", "html": "

[foo]bar

\n", "example": 542, "start_line": 8169, "end_line": 8176, "section": "Links" }, { "markdown": "![foo](/url \"title\")\n", "html": "

\"foo\"

\n", "example": 543, "start_line": 8192, "end_line": 8196, "section": "Images" }, { "markdown": "![foo *bar*]\n\n[foo *bar*]: train.jpg \"train & tracks\"\n", "html": "

\"foo

\n", "example": 544, "start_line": 8199, "end_line": 8205, "section": "Images" }, { "markdown": "![foo ![bar](/url)](/url2)\n", "html": "

\"foo

\n", "example": 545, "start_line": 8208, "end_line": 8212, "section": "Images" }, { "markdown": "![foo [bar](/url)](/url2)\n", "html": "

\"foo

\n", "example": 546, "start_line": 8215, "end_line": 8219, "section": "Images" }, { "markdown": "![foo *bar*][]\n\n[foo *bar*]: train.jpg \"train & tracks\"\n", "html": "

\"foo

\n", "example": 547, "start_line": 8229, "end_line": 8235, "section": "Images" }, { "markdown": "![foo *bar*][foobar]\n\n[FOOBAR]: train.jpg \"train & tracks\"\n", "html": "

\"foo

\n", "example": 548, "start_line": 8238, "end_line": 8244, "section": "Images" }, { "markdown": "![foo](train.jpg)\n", "html": "

\"foo\"

\n", "example": 549, "start_line": 8247, "end_line": 8251, "section": "Images" }, { "markdown": "My ![foo bar](/path/to/train.jpg \"title\" )\n", "html": "

My \"foo

\n", "example": 550, "start_line": 8254, "end_line": 8258, "section": "Images" }, { "markdown": "![foo]()\n", "html": "

\"foo\"

\n", "example": 551, "start_line": 8261, "end_line": 8265, "section": "Images" }, { "markdown": "![](/url)\n", "html": "

\"\"

\n", "example": 552, "start_line": 8268, "end_line": 8272, "section": "Images" }, { "markdown": "![foo][bar]\n\n[bar]: /url\n", "html": "

\"foo\"

\n", "example": 553, "start_line": 8277, "end_line": 8283, "section": "Images" }, { "markdown": "![foo][bar]\n\n[BAR]: /url\n", "html": "

\"foo\"

\n", "example": 554, "start_line": 8286, "end_line": 8292, "section": "Images" }, { "markdown": "![foo][]\n\n[foo]: /url \"title\"\n", "html": "

\"foo\"

\n", "example": 555, "start_line": 8297, "end_line": 8303, "section": "Images" }, { "markdown": "![*foo* bar][]\n\n[*foo* bar]: /url \"title\"\n", "html": "

\"foo

\n", "example": 556, "start_line": 8306, "end_line": 8312, "section": "Images" }, { "markdown": "![Foo][]\n\n[foo]: /url \"title\"\n", "html": "

\"Foo\"

\n", "example": 557, "start_line": 8317, "end_line": 8323, "section": "Images" }, { "markdown": "![foo] \n[]\n\n[foo]: /url \"title\"\n", "html": "

\"foo\"\n[]

\n", "example": 558, "start_line": 8329, "end_line": 8337, "section": "Images" }, { "markdown": "![foo]\n\n[foo]: /url \"title\"\n", "html": "

\"foo\"

\n", "example": 559, "start_line": 8342, "end_line": 8348, "section": "Images" }, { "markdown": "![*foo* bar]\n\n[*foo* bar]: /url \"title\"\n", "html": "

\"foo

\n", "example": 560, "start_line": 8351, "end_line": 8357, "section": "Images" }, { "markdown": "![[foo]]\n\n[[foo]]: /url \"title\"\n", "html": "

![[foo]]

\n

[[foo]]: /url "title"

\n", "example": 561, "start_line": 8362, "end_line": 8369, "section": "Images" }, { "markdown": "![Foo]\n\n[foo]: /url \"title\"\n", "html": "

\"Foo\"

\n", "example": 562, "start_line": 8374, "end_line": 8380, "section": "Images" }, { "markdown": "!\\[foo]\n\n[foo]: /url \"title\"\n", "html": "

![foo]

\n", "example": 563, "start_line": 8386, "end_line": 8392, "section": "Images" }, { "markdown": "\\![foo]\n\n[foo]: /url \"title\"\n", "html": "

!foo

\n", "example": 564, "start_line": 8398, "end_line": 8404, "section": "Images" }, { "markdown": "\n", "html": "

http://foo.bar.baz

\n", "example": 565, "start_line": 8431, "end_line": 8435, "section": "Autolinks" }, { "markdown": "\n", "html": "

http://foo.bar.baz/test?q=hello&id=22&boolean

\n", "example": 566, "start_line": 8438, "end_line": 8442, "section": "Autolinks" }, { "markdown": "\n", "html": "

irc://foo.bar:2233/baz

\n", "example": 567, "start_line": 8445, "end_line": 8449, "section": "Autolinks" }, { "markdown": "\n", "html": "

MAILTO:FOO@BAR.BAZ

\n", "example": 568, "start_line": 8454, "end_line": 8458, "section": "Autolinks" }, { "markdown": "\n", "html": "

a+b+c:d

\n", "example": 569, "start_line": 8466, "end_line": 8470, "section": "Autolinks" }, { "markdown": "\n", "html": "

made-up-scheme://foo,bar

\n", "example": 570, "start_line": 8473, "end_line": 8477, "section": "Autolinks" }, { "markdown": "\n", "html": "

http://../

\n", "example": 571, "start_line": 8480, "end_line": 8484, "section": "Autolinks" }, { "markdown": "\n", "html": "

localhost:5001/foo

\n", "example": 572, "start_line": 8487, "end_line": 8491, "section": "Autolinks" }, { "markdown": "\n", "html": "

<http://foo.bar/baz bim>

\n", "example": 573, "start_line": 8496, "end_line": 8500, "section": "Autolinks" }, { "markdown": "\n", "html": "

http://example.com/\\[\\

\n", "example": 574, "start_line": 8505, "end_line": 8509, "section": "Autolinks" }, { "markdown": "\n", "html": "

foo@bar.example.com

\n", "example": 575, "start_line": 8527, "end_line": 8531, "section": "Autolinks" }, { "markdown": "\n", "html": "

foo+special@Bar.baz-bar0.com

\n", "example": 576, "start_line": 8534, "end_line": 8538, "section": "Autolinks" }, { "markdown": "\n", "html": "

<foo+@bar.example.com>

\n", "example": 577, "start_line": 8543, "end_line": 8547, "section": "Autolinks" }, { "markdown": "<>\n", "html": "

<>

\n", "example": 578, "start_line": 8552, "end_line": 8556, "section": "Autolinks" }, { "markdown": "< http://foo.bar >\n", "html": "

< http://foo.bar >

\n", "example": 579, "start_line": 8559, "end_line": 8563, "section": "Autolinks" }, { "markdown": "\n", "html": "

<m:abc>

\n", "example": 580, "start_line": 8566, "end_line": 8570, "section": "Autolinks" }, { "markdown": "\n", "html": "

<foo.bar.baz>

\n", "example": 581, "start_line": 8573, "end_line": 8577, "section": "Autolinks" }, { "markdown": "http://example.com\n", "html": "

http://example.com

\n", "example": 582, "start_line": 8580, "end_line": 8584, "section": "Autolinks" }, { "markdown": "foo@bar.example.com\n", "html": "

foo@bar.example.com

\n", "example": 583, "start_line": 8587, "end_line": 8591, "section": "Autolinks" }, { "markdown": "\n", "html": "

\n", "example": 584, "start_line": 8669, "end_line": 8673, "section": "Raw HTML" }, { "markdown": "\n", "html": "

\n", "example": 585, "start_line": 8678, "end_line": 8682, "section": "Raw HTML" }, { "markdown": "\n", "html": "

\n", "example": 586, "start_line": 8687, "end_line": 8693, "section": "Raw HTML" }, { "markdown": "\n", "html": "

\n", "example": 587, "start_line": 8698, "end_line": 8704, "section": "Raw HTML" }, { "markdown": "Foo \n", "html": "

Foo

\n", "example": 588, "start_line": 8709, "end_line": 8713, "section": "Raw HTML" }, { "markdown": "<33> <__>\n", "html": "

<33> <__>

\n", "example": 589, "start_line": 8718, "end_line": 8722, "section": "Raw HTML" }, { "markdown": "
\n", "html": "

<a h*#ref="hi">

\n", "example": 590, "start_line": 8727, "end_line": 8731, "section": "Raw HTML" }, { "markdown": "
\n", "html": "

<a href="hi'> <a href=hi'>

\n", "example": 591, "start_line": 8736, "end_line": 8740, "section": "Raw HTML" }, { "markdown": "< a><\nfoo>\n", "html": "

< a><\nfoo><bar/ >

\n", "example": 592, "start_line": 8745, "end_line": 8751, "section": "Raw HTML" }, { "markdown": "
\n", "html": "

<a href='bar'title=title>

\n", "example": 593, "start_line": 8756, "end_line": 8760, "section": "Raw HTML" }, { "markdown": "
\n", "html": "

\n", "example": 594, "start_line": 8765, "end_line": 8769, "section": "Raw HTML" }, { "markdown": "\n", "html": "

</a href="foo">

\n", "example": 595, "start_line": 8774, "end_line": 8778, "section": "Raw HTML" }, { "markdown": "foo \n", "html": "

foo

\n", "example": 596, "start_line": 8783, "end_line": 8789, "section": "Raw HTML" }, { "markdown": "foo \n", "html": "

foo <!-- not a comment -- two hyphens -->

\n", "example": 597, "start_line": 8792, "end_line": 8796, "section": "Raw HTML" }, { "markdown": "foo foo -->\n\nfoo \n", "html": "

foo <!--> foo -->

\n

foo <!-- foo--->

\n", "example": 598, "start_line": 8801, "end_line": 8808, "section": "Raw HTML" }, { "markdown": "foo \n", "html": "

foo

\n", "example": 599, "start_line": 8813, "end_line": 8817, "section": "Raw HTML" }, { "markdown": "foo \n", "html": "

foo

\n", "example": 600, "start_line": 8822, "end_line": 8826, "section": "Raw HTML" }, { "markdown": "foo &<]]>\n", "html": "

foo &<]]>

\n", "example": 601, "start_line": 8831, "end_line": 8835, "section": "Raw HTML" }, { "markdown": "foo \n", "html": "

foo

\n", "example": 602, "start_line": 8841, "end_line": 8845, "section": "Raw HTML" }, { "markdown": "foo \n", "html": "

foo

\n", "example": 603, "start_line": 8850, "end_line": 8854, "section": "Raw HTML" }, { "markdown": "\n", "html": "

<a href=""">

\n", "example": 604, "start_line": 8857, "end_line": 8861, "section": "Raw HTML" }, { "markdown": "foo \nbaz\n", "html": "

foo
\nbaz

\n", "example": 605, "start_line": 8871, "end_line": 8877, "section": "Hard line breaks" }, { "markdown": "foo\\\nbaz\n", "html": "

foo
\nbaz

\n", "example": 606, "start_line": 8883, "end_line": 8889, "section": "Hard line breaks" }, { "markdown": "foo \nbaz\n", "html": "

foo
\nbaz

\n", "example": 607, "start_line": 8894, "end_line": 8900, "section": "Hard line breaks" }, { "markdown": "foo \n bar\n", "html": "

foo
\nbar

\n", "example": 608, "start_line": 8905, "end_line": 8911, "section": "Hard line breaks" }, { "markdown": "foo\\\n bar\n", "html": "

foo
\nbar

\n", "example": 609, "start_line": 8914, "end_line": 8920, "section": "Hard line breaks" }, { "markdown": "*foo \nbar*\n", "html": "

foo
\nbar

\n", "example": 610, "start_line": 8926, "end_line": 8932, "section": "Hard line breaks" }, { "markdown": "*foo\\\nbar*\n", "html": "

foo
\nbar

\n", "example": 611, "start_line": 8935, "end_line": 8941, "section": "Hard line breaks" }, { "markdown": "`code \nspan`\n", "html": "

code span

\n", "example": 612, "start_line": 8946, "end_line": 8951, "section": "Hard line breaks" }, { "markdown": "`code\\\nspan`\n", "html": "

code\\ span

\n", "example": 613, "start_line": 8954, "end_line": 8959, "section": "Hard line breaks" }, { "markdown": "
\n", "html": "

\n", "example": 614, "start_line": 8964, "end_line": 8970, "section": "Hard line breaks" }, { "markdown": "\n", "html": "

\n", "example": 615, "start_line": 8973, "end_line": 8979, "section": "Hard line breaks" }, { "markdown": "foo\\\n", "html": "

foo\\

\n", "example": 616, "start_line": 8986, "end_line": 8990, "section": "Hard line breaks" }, { "markdown": "foo \n", "html": "

foo

\n", "example": 617, "start_line": 8993, "end_line": 8997, "section": "Hard line breaks" }, { "markdown": "### foo\\\n", "html": "

foo\\

\n", "example": 618, "start_line": 9000, "end_line": 9004, "section": "Hard line breaks" }, { "markdown": "### foo \n", "html": "

foo

\n", "example": 619, "start_line": 9007, "end_line": 9011, "section": "Hard line breaks" }, { "markdown": "foo\nbaz\n", "html": "

foo\nbaz

\n", "example": 620, "start_line": 9022, "end_line": 9028, "section": "Soft line breaks" }, { "markdown": "foo \n baz\n", "html": "

foo\nbaz

\n", "example": 621, "start_line": 9034, "end_line": 9040, "section": "Soft line breaks" }, { "markdown": "hello $.;'there\n", "html": "

hello $.;'there

\n", "example": 622, "start_line": 9054, "end_line": 9058, "section": "Textual content" }, { "markdown": "Foo χρῆν\n", "html": "

Foo χρῆν

\n", "example": 623, "start_line": 9061, "end_line": 9065, "section": "Textual content" }, { "markdown": "Multiple spaces\n", "html": "

Multiple spaces

\n", "example": 624, "start_line": 9070, "end_line": 9074, "section": "Textual content" } ]mistletoe-0.8.2/test/specification/commonmark.py000066400000000000000000000113501420102462200220330ustar00rootroot00000000000000import re import sys import json from mistletoe import markdown from traceback import print_tb from argparse import ArgumentParser def run_tests(test_entries, start=None, end=None, quiet=False, verbose=False, known=False): if known: print('ignoring tests:', ', '.join(map(str, KNOWN)) + '\n') start = start or 0 end = end or sys.maxsize results = [run_test(test_entry, quiet) for test_entry in test_entries if test_entry['example'] >= start and test_entry['example'] <= end and (not known or test_entry['example'] not in KNOWN)] if verbose: print_failure_in_sections(results) fails = len(list(filter(lambda x: not x[0], results))) if fails: print('failed:', fails) print(' total:', len(results)) else: print('All tests passing.') return not fails def run_test(test_entry, quiet=False): test_case = test_entry['markdown'].splitlines(keepends=True) try: output = markdown(test_case) success = test_entry['html'] == output if not success and not quiet: print_test_entry(test_entry, output) return success, test_entry['section'] except Exception as exception: if not quiet: print_exception(exception, test_entry) return False, test_entry['section'] def load_tests(specfile): with open(specfile, 'r', encoding='utf-8') as fin: return json.load(fin) def locate_section(section, tests): start = None end = None for test in tests: if re.search(section, test['section'], re.IGNORECASE): if start is None: start = test['example'] elif start is not None and end is None: end = test['example'] - 1 return start, end if start: return start, tests[-1]['example'] - 1 raise RuntimeError("Section '{}' not found, aborting.".format(section)) def print_exception(exception, test_entry): print_test_entry(test_entry, '-- exception --', fout=sys.stderr) print(exception.__class__.__name__ + ':', exception, file=sys.stderr) print('Traceback: ', file=sys.stderr) print_tb(exception.__traceback__) def print_test_entry(test_entry, output, fout=sys.stdout): print('example: ', repr(test_entry['example']), file=fout) print('markdown:', repr(test_entry['markdown']), file=fout) print('html: ', repr(test_entry['html']), file=fout) print('output: ', repr(output), file=fout) print(file=fout) def print_failure_in_sections(results): section = results[0][1] failed = 0 total = 0 for result in results: if section != result[1]: if failed: section_str = "Failed in section '{}':".format(section) result_str = "{:>3} / {:>3}".format(failed, total) print('{:70} {}'.format(section_str, result_str)) section = result[1] failed = 0 total = 0 if not result[0]: failed += 1 total += 1 if failed: section_str = "Failed in section '{}':".format(section) result_str = "{:>3} / {:>3}".format(failed, total) print('{:70} {}'.format(section_str, result_str)) print() def main(): parser = ArgumentParser(description="Custom script for running Commonmark tests.") parser.add_argument('start', type=int, nargs='?', default=None, help="Run tests starting from this position.") parser.add_argument('end', type=int, nargs='?', default=None, help="Run tests until this position.") parser.add_argument('-v', '--verbose', dest='verbose', action='store_true', help="Output failure count in every section.") parser.add_argument('-q', '--quiet', dest='quiet', action='store_true', help="Suppress failed test entry output.") parser.add_argument('-s', '--section', dest='section', default=None, help="Only run tests in specified section.") parser.add_argument('-f', '--file', dest='tests', type=load_tests, default='test/specification/commonmark.json', help="Specify alternative specfile to run.") parser.add_argument('-n', '--ignore-known', dest='known', action='store_true', help="Ignore tests entries that are known to fail.") args = parser.parse_args() start = args.start end = args.end verbose = args.verbose quiet = args.quiet tests = args.tests known = args.known if args.section is not None: start, end = locate_section(args.section, tests) if not run_tests(tests, start, end, quiet, verbose, known): sys.exit(1) if __name__ == '__main__': main() mistletoe-0.8.2/test/specification/spec.sh000077500000000000000000000007151420102462200206120ustar00rootroot00000000000000#!/usr/bin/env bash set -e REPO="https://github.com/commonmark/CommonMark.git" VERSION="0.28" function main { echo "Cloning from repo: $REPO..." git clone --quiet $REPO echo "Using version $VERSION..." cd "CommonMark" git checkout --quiet $VERSION echo "Dumping tests file..." python3 "test/spec_tests.py" --dump-tests > "../commonmark.json" echo "Cleaning up..." cd .. rm -rf CommonMark echo "Done." } main mistletoe-0.8.2/test/test_ast_renderer.py000066400000000000000000000036231420102462200205700ustar00rootroot00000000000000import unittest from mistletoe import Document, ast_renderer class TestASTRenderer(unittest.TestCase): def test(self): self.maxDiff = None d = Document(['# heading 1\n', '\n', 'hello\n', 'world\n']) output = ast_renderer.get_ast(d) target = {'type': 'Document', 'footnotes': {}, 'children': [{ 'type': 'Heading', 'level': 1, 'children': [{ 'type': 'RawText', 'content': 'heading 1' }] }, { 'type': 'Paragraph', 'children': [{ 'type': 'RawText', 'content': 'hello' }, { 'type': 'LineBreak', 'soft': True, 'content': '' }, { 'type': 'RawText', 'content': 'world' }] }]} self.assertEqual(output, target) def test_footnotes(self): self.maxDiff = None d = Document(['[bar][baz]\n', '\n', '[baz]: spam\n']) target = {'type': 'Document', 'footnotes': {'baz': ('spam', '')}, 'children': [{ 'type': 'Paragraph', 'children': [{ 'type': 'Link', 'target': 'spam', 'title': '', 'children': [{ 'type': 'RawText', 'content': 'bar' }] }] }]} output = ast_renderer.get_ast(d) self.assertEqual(output, target) mistletoe-0.8.2/test/test_block_token.py000066400000000000000000000406261420102462200204110ustar00rootroot00000000000000import unittest from unittest.mock import patch, call from mistletoe import block_token, span_token from mistletoe.block_tokenizer import FileWrapper class TestToken(unittest.TestCase): def setUp(self): self.addCleanup(lambda: span_token._token_types.__setitem__(-1, span_token.RawText)) patcher = patch('mistletoe.span_token.RawText') self.mock = patcher.start() span_token._token_types[-1] = self.mock self.addCleanup(patcher.stop) def _test_match(self, token_cls, lines, arg, **kwargs): token = next(iter(block_token.tokenize(lines))) self.assertIsInstance(token, token_cls) self._test_token(token, arg, **kwargs) def _test_token(self, token, arg, **kwargs): for attr, value in kwargs.items(): self.assertEqual(getattr(token, attr), value) self.mock.assert_any_call(arg) class TestATXHeading(TestToken): def test_match(self): lines = ['### heading 3\n'] arg = 'heading 3' self._test_match(block_token.Heading, lines, arg, level=3) def test_children_with_enclosing_hashes(self): lines = ['# heading 3 ##### \n'] arg = 'heading 3' self._test_match(block_token.Heading, lines, arg, level=1) def test_not_heading(self): lines = ['####### paragraph\n'] arg = '####### paragraph' self._test_match(block_token.Paragraph, lines, arg) def test_heading_in_paragraph(self): lines = ['foo\n', '# heading\n', 'bar\n'] token1, token2, token3 = block_token.tokenize(lines) self.assertIsInstance(token1, block_token.Paragraph) self.assertIsInstance(token2, block_token.Heading) self.assertIsInstance(token3, block_token.Paragraph) class TestSetextHeading(TestToken): def test_match(self): lines = ['some heading\n', '---\n'] arg = 'some heading' self._test_match(block_token.SetextHeading, lines, arg, level=2) def test_next(self): lines = ['some\n', 'heading\n', '---\n', '\n', 'foobar\n'] tokens = iter(block_token.tokenize(lines)) self.assertIsInstance(next(tokens), block_token.SetextHeading) self.assertIsInstance(next(tokens), block_token.Paragraph) self.mock.assert_has_calls([call('some'), call('heading'), call('foobar')]) with self.assertRaises(StopIteration) as e: token = next(tokens) class TestQuote(unittest.TestCase): def test_match(self): with patch('mistletoe.block_token.Paragraph') as mock: token = next(iter(block_token.tokenize(['> line 1\n', '> line 2\n']))) self.assertIsInstance(token, block_token.Quote) def test_lazy_continuation(self): with patch('mistletoe.block_token.Paragraph') as mock: token = next(iter(block_token.tokenize(['> line 1\n', 'line 2\n']))) self.assertIsInstance(token, block_token.Quote) class TestCodeFence(TestToken): def test_match_fenced_code(self): lines = ['```sh\n', 'rm dir\n', 'mkdir test\n', '```\n'] arg = 'rm dir\nmkdir test\n' self._test_match(block_token.CodeFence, lines, arg, language='sh') def test_match_fenced_code_with_tilde(self): lines = ['~~~sh\n', 'rm dir\n', 'mkdir test\n', '~~~\n'] arg = 'rm dir\nmkdir test\n' self._test_match(block_token.CodeFence, lines, arg, language='sh') def test_not_match_fenced_code_when_only_inline_code(self): lines = ['`~` is called tilde'] token = next(iter(block_token.tokenize(lines))) self.assertIsInstance(token, block_token.Paragraph) token1 = token.children[0] self.assertIsInstance(token1, span_token.InlineCode) self.mock.assert_has_calls([call('~'), call(' is called tilde')]) def test_mixed_code_fence(self): lines = ['~~~markdown\n', '```sh\n', 'some code\n', '```\n', '~~~\n'] arg = '```sh\nsome code\n```\n' self._test_match(block_token.CodeFence, lines, arg, language='markdown') def test_fence_code_lazy_continuation(self): lines = ['```sh\n', 'rm dir\n', '\n', 'mkdir test\n', '```\n'] arg = 'rm dir\n\nmkdir test\n' self._test_match(block_token.CodeFence, lines, arg, language='sh') def test_no_wrapping_newlines_code_fence(self): lines = ['```\n', 'hey', '```\n', 'paragraph\n'] arg = 'hey' self._test_match(block_token.CodeFence, lines, arg, language='') def test_unclosed_code_fence(self): lines = ['```\n', 'hey'] arg = 'hey' self._test_match(block_token.CodeFence, lines, arg, language='') class TestBlockCode(TestToken): def test_parse_indented_code(self): lines = [' rm dir\n', ' mkdir test\n'] arg = 'rm dir\nmkdir test\n' self._test_match(block_token.BlockCode, lines, arg, language='') class TestParagraph(TestToken): def test_parse(self): lines = ['some\n', 'continuous\n', 'lines\n'] arg = 'some' self._test_match(block_token.Paragraph, lines, arg) def test_read(self): lines = ['this\n', '```\n', 'is some\n', '```\n', 'code\n'] try: token1, token2, token3 = block_token.tokenize(lines) except ValueError as e: raise AssertionError("Token number mismatch.") from e self.assertIsInstance(token1, block_token.Paragraph) self.assertIsInstance(token2, block_token.CodeFence) self.assertIsInstance(token3, block_token.Paragraph) class TestListItem(unittest.TestCase): def test_parse_marker(self): lines = ['- foo\n', '* bar\n', ' + baz\n', '1. item 1\n', '2) item 2\n', '123456789. item x\n'] for line in lines: self.assertTrue(block_token.ListItem.parse_marker(line)) bad_lines = ['> foo\n', '1item 1\n', '2| item 2\n', '1234567890. item x\n'] for line in bad_lines: self.assertFalse(block_token.ListItem.parse_marker(line)) def test_tokenize(self): lines = [' - foo\n', ' bar\n', '\n', ' baz\n'] token1, token2 = next(iter(block_token.tokenize(lines))).children[0].children self.assertIsInstance(token1, block_token.Paragraph) self.assertTrue('foo' in token1) self.assertIsInstance(token2, block_token.BlockCode) def test_sublist(self): lines = ['- foo\n', ' - bar\n'] token1, token2 = block_token.tokenize(lines)[0].children[0].children self.assertIsInstance(token1, block_token.Paragraph) self.assertIsInstance(token2, block_token.List) def test_deep_list(self): lines = ['- foo\n', ' - bar\n', ' - baz\n'] f = FileWrapper(lines) ptoken, ltoken = block_token.tokenize(lines)[0].children[0].children self.assertIsInstance(ptoken, block_token.Paragraph) self.assertIsInstance(ltoken, block_token.List) self.assertTrue('foo' in ptoken) ptoken, ltoken = ltoken.children[0].children self.assertIsInstance(ptoken, block_token.Paragraph) self.assertTrue('bar' in ptoken) self.assertIsInstance(ltoken, block_token.List) self.assertTrue('baz' in ltoken) def test_loose_list(self): lines = ['- foo\n', ' ~~~\n', ' bar\n', ' \n', ' baz\n' ' ~~~\n'] f = FileWrapper(lines) list_item = block_token.tokenize(lines)[0].children[0] self.assertEqual(list_item.loose, False) def test_tight_list(self): lines = ['- foo\n', '\n', '# bar\n'] f = FileWrapper(lines) list_item = block_token.tokenize(lines)[0].children[0] self.assertEqual(list_item.loose, False) class TestList(unittest.TestCase): def test_different_markers(self): lines = ['- foo\n', '* bar\n', '1. baz\n', '2) spam\n'] l1, l2, l3, l4 = block_token.tokenize(lines) self.assertIsInstance(l1, block_token.List) self.assertTrue('foo' in l1) self.assertIsInstance(l2, block_token.List) self.assertTrue('bar' in l2) self.assertIsInstance(l3, block_token.List) self.assertTrue('baz' in l3) self.assertIsInstance(l4, block_token.List) self.assertTrue('spam' in l4) def test_sublist(self): lines = ['- foo\n', ' + bar\n'] token, = block_token.tokenize(lines) self.assertIsInstance(token, block_token.List) class TestTable(unittest.TestCase): def test_parse_align(self): test_func = block_token.Table.parse_align self.assertEqual(test_func(':------'), None) self.assertEqual(test_func(':-----:'), 0) self.assertEqual(test_func('------:'), 1) def test_parse_delimiter(self): test_func = block_token.Table.split_delimiter self.assertEqual(list(test_func('| :--- | :---: | ---:|\n')), [':---', ':---:', '---:']) def test_match(self): lines = ['| header 1 | header 2 | header 3 |\n', '| --- | --- | --- |\n', '| cell 1 | cell 2 | cell 3 |\n', '| more 1 | more 2 | more 3 |\n'] with patch('mistletoe.block_token.TableRow') as mock: token = next(iter(block_token.tokenize(lines))) self.assertIsInstance(token, block_token.Table) self.assertTrue(hasattr(token, 'header')) self.assertEqual(token.column_align, [None, None, None]) token.children calls = [call(line, [None, None, None]) for line in lines[:1]+lines[2:]] mock.assert_has_calls(calls) def test_easy_table(self): lines = ['header 1 | header 2\n', ' ---: | :---\n', ' cell 1 | cell 2\n'] with patch('mistletoe.block_token.TableRow') as mock: token, = block_token.tokenize(lines) self.assertIsInstance(token, block_token.Table) self.assertTrue(hasattr(token, 'header')) self.assertEqual(token.column_align, [1, None]) token.children calls = [call(line, [1, None]) for line in lines[:1] + lines[2:]] mock.assert_has_calls(calls) def test_not_easy_table(self): lines = ['not header 1 | not header 2\n', 'foo | bar\n'] token, = block_token.tokenize(lines) self.assertIsInstance(token, block_token.Paragraph) class TestTableRow(unittest.TestCase): def test_match(self): with patch('mistletoe.block_token.TableCell') as mock: line = '| cell 1 | cell 2 |\n' token = block_token.TableRow(line) self.assertEqual(token.row_align, [None]) mock.assert_has_calls([call('cell 1', None), call('cell 2', None)]) def test_easy_table_row(self): with patch('mistletoe.block_token.TableCell') as mock: line = 'cell 1 | cell 2\n' token = block_token.TableRow(line) self.assertEqual(token.row_align, [None]) mock.assert_has_calls([call('cell 1', None), call('cell 2', None)]) def test_short_row(self): with patch('mistletoe.block_token.TableCell') as mock: line = '| cell 1 |\n' token = block_token.TableRow(line, [None, None]) self.assertEqual(token.row_align, [None, None]) mock.assert_has_calls([call('cell 1', None), call('', None)]) def test_escaped_pipe_in_cell(self): with patch('mistletoe.block_token.TableCell') as mock: line = '| pipe: `\\|` | cell 2\n' token = block_token.TableRow(line, [None, None]) self.assertEqual(token.row_align, [None, None]) mock.assert_has_calls([call('pipe: `|`', None), call('cell 2', None)]) @unittest.skip('Even GitHub fails in here, workaround: always put a space before `|`') def test_not_really_escaped_pipe_in_cell(self): with patch('mistletoe.block_token.TableCell') as mock: line = '|ending with a \\\\|cell 2\n' token = block_token.TableRow(line, [None, None]) self.assertEqual(token.row_align, [None, None]) mock.assert_has_calls([call('ending with a \\\\', None), call('cell 2', None)]) class TestTableCell(TestToken): def test_match(self): token = block_token.TableCell('cell 2') self._test_token(token, 'cell 2', align=None) class TestFootnote(unittest.TestCase): def test_parse_simple(self): lines = ['[key 1]: value1\n', '[key 2]: value2\n'] token = block_token.Document(lines) self.assertEqual(token.footnotes, {"key 1": ("value1", ""), "key 2": ("value2", "")}) def test_parse_with_title(self): lines = ['[key 1]: value1 "title1"\n', '[key 2]: value2\n', '"title2"\n'] token = block_token.Document(lines) self.assertEqual(token.footnotes, {"key 1": ("value1", "title1"), "key 2": ("value2", "title2")}) # this tests an edge case, it shouldn't occur in normal documents def test_parse_with_para_right_after(self): lines = ['[key 1]: value1\n', # 'something1\n', # if uncommented, # this and the next line should be treated as a paragraph # - this line gets skipped instead now '[key 2]: value2\n', 'something2\n', '\n', '[key 3]: value3\r\n', # '\r', or any other whitespace 'something3\n'] token = block_token.Document(lines) self.assertEqual(token.footnotes, {"key 1": ("value1", ""), "key 2": ("value2", ""), "key 3": ("value3", "")}) self.assertEqual(len(token.children), 2) self.assertIsInstance(token.children[0], block_token.Paragraph) self.assertEqual(token.children[0].children[0].content, "something2") self.assertEqual(token.children[1].children[0].content, "something3") def test_parse_opening_bracket_as_paragraph(self): # ... and no error is raised lines = ['[\n'] token = block_token.Document(lines) self.assertEqual(len(token.footnotes), 0) self.assertEqual(len(token.children), 1) self.assertIsInstance(token.children[0], block_token.Paragraph) self.assertEqual(token.children[0].children[0].content, '[') def test_parse_opening_brackets_as_paragraph(self): # ... and no lines are skipped lines = ['[\n', '[ \n', ']\n'] token = block_token.Document(lines) self.assertEqual(len(token.footnotes), 0) self.assertEqual(len(token.children), 1) para = token.children[0] self.assertIsInstance(para, block_token.Paragraph) self.assertEqual(len(para.children), 5, 'expected: RawText, LineBreak, RawText, LineBreak, RawText') self.assertEqual(para.children[0].content, '[') class TestDocument(unittest.TestCase): def test_store_footnote(self): lines = ['[key 1]: value1\n', '[key 2]: value2\n'] document = block_token.Document(lines) self.assertEqual(document.footnotes['key 1'], ('value1', '')) self.assertEqual(document.footnotes['key 2'], ('value2', '')) def test_auto_splitlines(self): lines = "some\ncontinual\nlines\n" document = block_token.Document(lines) self.assertIsInstance(document.children[0], block_token.Paragraph) self.assertEqual(len(document.children), 1) class TestThematicBreak(unittest.TestCase): def test_match(self): def test_case(line): token = next(iter(block_token.tokenize([line]))) self.assertIsInstance(token, block_token.ThematicBreak) cases = ['---\n', '* * *\n', '_ _ _\n'] for case in cases: test_case(case) class TestContains(unittest.TestCase): def test_contains(self): lines = ['# heading\n', '\n', 'paragraph\n', 'with\n', '`code`\n'] token = block_token.Document(lines) self.assertTrue('heading' in token) self.assertTrue('code' in token) self.assertFalse('foo' in token) mistletoe-0.8.2/test/test_ci.sh000077500000000000000000000015451420102462200164740ustar00rootroot00000000000000#!/usr/bin/env bash set -e function main { if [[ "$1" == "" ]]; then echo "[Error] Specify how far you want to go back." exit 1 fi CURR_BRANCH="$(get_current_branch)" git checkout --quiet HEAD~$1 render_to_file "out2.html" OLD_SHA=$(get_sha "out2.html") git checkout --quiet "$CURR_BRANCH" render_to_file "out.html" NEW_SHA=$(get_sha "out2.html") if [[ "$OLD_SHA" == "$NEW_SHA" ]]; then cleanup else get_diff fi } function get_current_branch { git rev-parse --abbrev-ref HEAD } function render_to_file { python3 -m mistletoe "test/samples/syntax.md" > "$1" } function get_sha { md5 -q "$1" } function cleanup { echo "All good." rm out2.html } function get_diff { echo "Diff exits; prompting for review..." diff out.html out2.html | view - } main $1 mistletoe-0.8.2/test/test_cli.py000066400000000000000000000127151420102462200166640ustar00rootroot00000000000000from unittest import TestCase from unittest.mock import call, patch, sentinel, mock_open, Mock from mistletoe import cli class TestCLI(TestCase): @patch('mistletoe.cli.parse', return_value=Mock(filenames=[], renderer=sentinel.Renderer)) @patch('mistletoe.cli.interactive') def test_main_to_interactive(self, mock_interactive, mock_parse): cli.main(None) mock_interactive.assert_called_with(sentinel.Renderer) @patch('mistletoe.cli.parse', return_value=Mock(filenames=['foo.md'], renderer=sentinel.Renderer)) @patch('mistletoe.cli.convert') def test_main_to_convert(self, mock_convert, mock_parse): cli.main(None) mock_convert.assert_called_with(['foo.md'], sentinel.Renderer) @patch('importlib.import_module', return_value=Mock(Renderer=sentinel.RendererCls)) def test_parse_renderer(self, mock_import_module): namespace = cli.parse(['-r', 'foo.Renderer']) mock_import_module.assert_called_with('foo') self.assertEqual(namespace.renderer, sentinel.RendererCls) def test_parse_filenames(self): filenames = ['foo.md', 'bar.md'] namespace = cli.parse(filenames) self.assertEqual(namespace.filenames, filenames) @patch('mistletoe.cli.convert_file') def test_convert(self, mock_convert_file): filenames = ['foo', 'bar'] cli.convert(filenames, sentinel.RendererCls) calls = [call(filename, sentinel.RendererCls) for filename in filenames] mock_convert_file.assert_has_calls(calls) @patch('mistletoe.markdown', return_value='rendered text') @patch('sys.stdout.buffer.write') @patch('builtins.open', new_callable=mock_open) def test_convert_file_success(self, mock_open_, mock_write, mock_markdown): filename = 'foo' cli.convert_file(filename, sentinel.RendererCls) mock_open_.assert_called_with(filename, 'r', encoding='utf-8') mock_write.assert_called_with('rendered text'.encode()) @patch('builtins.open', side_effect=OSError) @patch('sys.exit') def test_convert_file_fail(self, mock_exit, mock_open_): filename = 'foo' cli.convert_file(filename, sentinel.RendererCls) mock_open_.assert_called_with(filename, 'r', encoding='utf-8') mock_exit.assert_called_with('Cannot open file "foo".') @patch('mistletoe.cli._import_readline') @patch('mistletoe.cli._print_heading') @patch('mistletoe.markdown', return_value='rendered text') @patch('builtins.print') def test_interactive(self, mock_print, mock_markdown, mock_print_heading, mock_import_readline): def MockInputFactory(return_values): _counter = -1 def mock_input(prompt=''): nonlocal _counter _counter += 1 if _counter < len(return_values): return return_values[_counter] elif _counter == len(return_values): raise EOFError else: raise KeyboardInterrupt return mock_input return_values = ['foo', 'bar', 'baz'] with patch('builtins.input', MockInputFactory(return_values)): cli.interactive(sentinel.RendererCls) mock_import_readline.assert_called_with() mock_print_heading.assert_called_with(sentinel.RendererCls) mock_markdown.assert_called_with(['foo\n', 'bar\n', 'baz\n'], sentinel.RendererCls) calls = [call('\nrendered text', end=''), call('\nExiting.')] mock_print.assert_has_calls(calls) @patch('importlib.import_module', return_value=Mock(Renderer=sentinel.RendererCls)) def test_import_success(self, mock_import_module): self.assertEqual(sentinel.RendererCls, cli._import('foo.Renderer')) @patch('sys.exit') def test_import_incomplete_path(self, mock_exit): cli._import('foo') error_msg = '[error] please supply full path to your custom renderer.' mock_exit.assert_called_with(error_msg) @patch('importlib.import_module', side_effect=ImportError) @patch('sys.exit') def test_import_module_error(self, mock_exit, mock_import_module): cli._import('foo.Renderer') mock_exit.assert_called_with('[error] cannot import module "foo".') @patch('importlib.import_module', return_value=Mock(spec=[])) @patch('sys.exit') def test_import_class_error(self, mock_exit, mock_import_module): cli._import('foo.Renderer') error_msg = '[error] cannot find renderer "Renderer" from module "foo".' mock_exit.assert_called_with(error_msg) @patch('builtins.__import__') @patch('builtins.print') def test_import_readline_success(self, mock_print, mock_import): cli._import_readline() mock_print.assert_not_called() @patch('builtins.__import__', side_effect=ImportError) @patch('builtins.print') def test_import_readline_fail(self, mock_print, mock_import): cli._import_readline() mock_print.assert_called_with('[warning] readline library not available.') @patch('builtins.print') def test_print_heading(self, mock_print): cli._print_heading(Mock(__name__='Renderer')) version = cli.mistletoe.__version__ msgs = ['mistletoe [version {}] (interactive)'.format(version), 'Type Ctrl-D to complete input, or Ctrl-C to exit.', 'Using renderer: Renderer'] calls = [call(msg) for msg in msgs] mock_print.assert_has_calls(calls) mistletoe-0.8.2/test/test_contrib/000077500000000000000000000000001420102462200171755ustar00rootroot00000000000000mistletoe-0.8.2/test/test_contrib/__init__.py000066400000000000000000000000001420102462200212740ustar00rootroot00000000000000mistletoe-0.8.2/test/test_contrib/test_github_wiki.py000066400000000000000000000022401420102462200231110ustar00rootroot00000000000000from unittest import TestCase, mock from mistletoe import span_token, Document from mistletoe.span_token import tokenize_inner, _token_types from contrib.github_wiki import GithubWiki, GithubWikiRenderer class TestGithubWiki(TestCase): def setUp(self): span_token._root_node = Document([]) self.renderer = GithubWikiRenderer() self.renderer.__enter__() self.addCleanup(self.renderer.__exit__, None, None, None) def test_parse(self): MockRawText = mock.Mock(autospec='mistletoe.span_token.RawText') RawText = _token_types.pop() _token_types.append(MockRawText) try: tokens = tokenize_inner('text with [[wiki | target]]') token = tokens[1] self.assertIsInstance(token, GithubWiki) self.assertEqual(token.target, 'target') MockRawText.assert_has_calls([mock.call('text with '), mock.call('wiki')]) finally: _token_types[-1] = RawText def test_render(self): token = next(iter(tokenize_inner('[[wiki|target]]'))) output = '
wiki' self.assertEqual(self.renderer.render(token), output) mistletoe-0.8.2/test/test_contrib/test_jira_renderer.py000066400000000000000000000147621420102462200234330ustar00rootroot00000000000000# Copyright 2018 Tile, Inc. All Rights Reserved. # # The MIT License # # Permission is hereby granted, free of charge, to any person obtaining a copy of # this software and associated documentation files (the "Software"), to deal in # the Software without restriction, including without limitation the rights to # use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies # of the Software, and to permit persons to whom the Software is furnished to do # so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. from unittest import TestCase, mock from test.base_test import BaseRendererTest from mistletoe.block_token import Document from mistletoe.span_token import tokenize_inner from mistletoe import Document from contrib.jira_renderer import JIRARenderer import random import string filesBasedTest = BaseRendererTest.filesBasedTest class TestJIRARenderer(BaseRendererTest): def setUp(self): super().setUp() self.renderer = JIRARenderer() self.renderer.__enter__() self.addCleanup(self.renderer.__exit__, None, None, None) self.sampleOutputExtension = 'jira' def genRandomString(self, n, hasWhitespace=False): source = string.ascii_letters + string.digits if hasWhitespace: source = source + ' \t' result = ''.join(random.SystemRandom().choice(source) for _ in range(n)) return result def textFormatTest(self, inputTemplate, outputTemplate): input = self.genRandomString(80, False) token = next(iter(tokenize_inner(inputTemplate.format(input)))) expected = outputTemplate.format(input) actual = self.renderer.render(token) self.assertEqual(expected, actual) def test_escape_simple(self): self.textFormatTest('---fancy text---', '\\-\\-\\-fancy text\\-\\-\\-') def test_escape_single_chars(self): self.textFormatTest('**fancy \\*@\\* text**', '*fancy \\*@\\* text*') def test_escape_none_when_whitespaces(self): self.textFormatTest('obj = {{ a: (b * c) + d }}', 'obj = {{ a: (b * c) + d }}') def test_escape_in_inline_code(self): # Note: Jira puts inline code into "{{...}}" as seen in this test. self.textFormatTest('**code: `a = b + c;// [1]`**', '*code: {{{{a = b + c;// \\[1\\]}}}}*') def test_escape_link(self): # Note: There seems to be no way of how to escape plain text URL in Jira. self.textFormatTest('http://www.example.com', 'http://www.example.com') def test_render_strong(self): self.textFormatTest('**a{}**', '*a{}*') def test_render_emphasis(self): self.textFormatTest('*a{}*', '_a{}_') def test_render_inline_code(self): self.textFormatTest('`a{}b`', '{{{{a{}b}}}}') def test_render_strikethrough(self): self.textFormatTest('~~{}~~', '-{}-') def test_render_image(self): token = next(iter(tokenize_inner('![image](foo.jpg)'))) expected = '!foo.jpg!' actual = self.renderer.render(token) self.assertEqual(expected, actual) def test_render_footnote_image(self): # token = next(tokenize_inner('![image]\n\n[image]: foo.jpg')) # expected = '!foo.jpg!' # actual = self.renderer.render(token) # self.assertEqual(expected, actual) pass def test_render_link(self): url = 'http://{0}.{1}.{2}'.format(self.genRandomString(5), self.genRandomString(5), self.genRandomString(3)) body = self.genRandomString(80, True) token = next(iter(tokenize_inner('[{body}]({url})'.format(url=url, body=body)))) expected = '[{body}|{url}]'.format(url=url, body=body) actual = self.renderer.render(token) self.assertEqual(expected, actual) def test_render_footnote_link(self): pass def test_render_auto_link(self): url = 'http://{0}.{1}.{2}'.format(self.genRandomString(5), self.genRandomString(5), self.genRandomString(3)) token = next(iter(tokenize_inner('<{url}>'.format(url=url)))) expected = '[{url}]'.format(url=url) actual = self.renderer.render(token) self.assertEqual(expected, actual) def test_render_escape_sequence(self): pass def test_render_html_span(self): pass def test_render_heading(self): pass def test_render_quote(self): pass def test_render_paragraph(self): pass def test_render_block_code(self): markdown = """\ ```java public static void main(String[] args) { // a = 1 * 2; } ``` """ expected = """\ {code:java} public static void main(String[] args) { // a = 1 * 2; } {code} """ self.markdownResultTest(markdown, expected) def test_render_list(self): pass def test_render_list_item(self): pass def test_render_inner(self): pass def test_render_table(self): pass def test_render_table_row(self): pass def test_render_table_cell(self): pass def test_render_thematic_break(self): pass def test_render_html_block(self): pass def test_render_document(self): pass def test_table_header(self): markdown = """\ | header row | |--------------| | first cell | """ expected = """\ ||header row|| |first cell| """ self.markdownResultTest(markdown, expected) def test_table_empty_cell(self): """ Empty cells need to have a space in them, see . """ markdown = """\ | A | B | C | |-----------| | 1 | | 3 | """ expected = """\ ||A||B||C|| |1| |3| """ self.markdownResultTest(markdown, expected) @filesBasedTest def test_render__basic_blocks(self): pass @filesBasedTest def test_render__lists(self): pass @filesBasedTest def test_render__quotes(self): pass mistletoe-0.8.2/test/test_contrib/test_mathjax.py000066400000000000000000000017461420102462200222520ustar00rootroot00000000000000import unittest from mistletoe import Document from contrib.mathjax import MathJaxRenderer class TestMathJaxRenderer(unittest.TestCase): mathjax_src = '\n' def test_render_html(self): with MathJaxRenderer() as renderer: token = Document(['# heading 1\n', 'paragraph\n']) output = renderer.render(token) target = '

heading 1

\n

paragraph

\n' target += self.mathjax_src self.assertEqual(output, target) def test_render_math(self): with MathJaxRenderer() as renderer: raw = ['# heading 1\n', '$$paragraph$$\n', 'with $ math $\n'] token = Document(raw) output = renderer.render(token) target = '

heading 1

\n

$$paragraph$$\nwith $$ math $$

\n' target += self.mathjax_src self.assertEqual(output, target) mistletoe-0.8.2/test/test_contrib/test_toc_renderer.py000066400000000000000000000047261420102462200232720ustar00rootroot00000000000000from unittest import TestCase from mistletoe import block_token from mistletoe.block_token import Document, Heading from contrib.toc_renderer import TOCRenderer class TestTOCRenderer(TestCase): def test_parse_rendered_heading(self): rendered_heading = '

some text

' content = TOCRenderer.parse_rendered_heading(rendered_heading) self.assertEqual(content, 'some text') def test_render_heading(self): renderer = TOCRenderer() Heading.start('### some *text*\n') token = Heading(Heading.read(iter(['foo']))) rendered_heading = renderer.render_heading(token) self.assertEqual(renderer._headings[0], (3, 'some text')) def test_depth(self): renderer = TOCRenderer(depth=3) token = Document(['# title\n', '## heading\n', '#### heading\n']) renderer.render(token) self.assertEqual(renderer._headings, [(2, 'heading')]) def test_omit_title(self): renderer = TOCRenderer(omit_title=True) token = Document(['# title\n', '\n', '## heading\n']) renderer.render(token) self.assertEqual(renderer._headings, [(2, 'heading')]) def test_filter_conditions(self): import re filter_conds = [lambda x: re.match(r'heading', x), lambda x: re.match(r'foo', x)] renderer = TOCRenderer(filter_conds=filter_conds) token = Document(['# title\n', '\n', '## heading\n', '\n', '#### not heading\n']) renderer.render(token) self.assertEqual(renderer._headings, [(4, 'not heading')]) def test_get_toc(self): headings = [(1, 'heading 1'), (2, 'subheading 1'), (2, 'subheading 2'), (3, 'subsubheading 1'), (2, 'subheading 3'), (1, 'heading 2')] renderer = TOCRenderer(omit_title=False) renderer._headings = headings toc = renderer.toc self.assertIsInstance(toc, block_token.List) # for now, we check at least the most nested heading (hierarchy: `List -> ListItem -> {Paragraph -> RawText.content | List -> ...}`): heading_item = toc.children[0].children[1].children[1].children[1].children[0] self.assertIsInstance(heading_item, block_token.ListItem) self.assertEqual(heading_item.children[0].children[0].content, 'subsubheading 1') mistletoe-0.8.2/test/test_contrib/test_xwiki20_renderer.py000066400000000000000000000135351420102462200240000ustar00rootroot00000000000000# The MIT License # # Permission is hereby granted, free of charge, to any person obtaining a copy of # this software and associated documentation files (the "Software"), to deal in # the Software without restriction, including without limitation the rights to # use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies # of the Software, and to permit persons to whom the Software is furnished to do # so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. from unittest import TestCase, mock from test.base_test import BaseRendererTest from mistletoe.block_token import Document from mistletoe.span_token import tokenize_inner from mistletoe import Document from contrib.xwiki20_renderer import XWiki20Renderer import random import string filesBasedTest = BaseRendererTest.filesBasedTest class TestXWiki20Renderer(BaseRendererTest): def setUp(self): super().setUp() self.renderer = XWiki20Renderer() self.renderer.__enter__() self.addCleanup(self.renderer.__exit__, None, None, None) self.sampleOutputExtension = 'xwiki20' def genRandomString(self, n, hasWhitespace=False): source = string.ascii_letters + string.digits if hasWhitespace: source = source + ' \t' result = ''.join(random.SystemRandom().choice(source) for _ in range(n)) return result def textFormatTest(self, inputTemplate, outputTemplate): input = self.genRandomString(80, False) token = next(iter(tokenize_inner(inputTemplate.format(input)))) expected = outputTemplate.format(input) actual = self.renderer.render(token) self.assertEqual(expected, actual) def test_escaping(self): self.textFormatTest('**code: `a = 1;// comment`, plain text URL: http://example.com**', '**code: {{{{code}}}}a = 1;// comment{{{{/code}}}}, plain text URL: http:~//example.com**') def test_render_strong(self): self.textFormatTest('**a{}**', '**a{}**') def test_render_emphasis(self): self.textFormatTest('*a{}*', '//a{}//') def test_render_inline_code(self): self.textFormatTest('`a{}b`', '{{{{code}}}}a{}b{{{{/code}}}}') def test_render_strikethrough(self): self.textFormatTest('~~{}~~', '--{}--') def test_render_image(self): token = next(iter(tokenize_inner('![image](foo.jpg)'))) expected = '[[image:foo.jpg]]' actual = self.renderer.render(token) self.assertEqual(expected, actual) def test_render_link(self): url = 'http://{0}.{1}.{2}'.format(self.genRandomString(5), self.genRandomString(5), self.genRandomString(3)) body = self.genRandomString(80, True) token = next(iter(tokenize_inner('[{body}]({url})'.format(url=url, body=body)))) expected = '[[{body}>>{url}]]'.format(url=url, body=body) actual = self.renderer.render(token) self.assertEqual(expected, actual) def test_render_auto_link(self): url = 'http://{0}.{1}.{2}'.format(self.genRandomString(5), self.genRandomString(5), self.genRandomString(3)) token = next(iter(tokenize_inner('<{url}>'.format(url=url)))) expected = '[[{url}]]'.format(url=url) actual = self.renderer.render(token) self.assertEqual(expected, actual) def test_render_html_span(self): markdown = 'text styles: italic, bold' # See fixme at the `render_html_span` method... # expected = 'text styles: {{html wiki="true"}}italic{{/html}}, {{html wiki="true"}}bold{{/html}}\n\n' expected = 'text styles: italic, bold\n\n' self.markdownResultTest(markdown, expected) def test_render_html_block(self): markdown = 'paragraph\n\n
some cool code
' expected = 'paragraph\n\n{{html wiki="true"}}\n
some cool code
\n{{/html}}\n\n' self.markdownResultTest(markdown, expected) def test_render_xwiki_macros_simple(self): markdown = """\ {{warning}} Use this feature with *caution*. See {{Wikipedia article="SomeArticle"/}}. {{test}}Another inline macro{{/test}}. {{/warning}} """ # Note: There is a trailing ' ' at the end of the second line. It will be a bit complicated to get rid of it. expected = """\ {{warning}} Use this feature with //caution//. See {{Wikipedia article="SomeArticle"/}}. {{test}}Another inline macro{{/test}}. \n\ {{/warning}} """ self.markdownResultTest(markdown, expected) def test_render_xwiki_macros_in_list(self): markdown = """\ * list item {{warning}} Use this feature with *caution*. See {{Wikipedia article="SomeArticle"/}}. {{test}}Another inline macro{{/test}}. {{/warning}} """ # Note: There is a trailing ' ' at the end of the second line. It will be a bit complicated to get rid of it. expected = """\ * list item((( {{warning}} Use this feature with //caution//. See {{Wikipedia article="SomeArticle"/}}. {{test}}Another inline macro{{/test}}. \n\ {{/warning}} ))) """ self.markdownResultTest(markdown, expected) @filesBasedTest def test_render__basic_blocks(self): pass @filesBasedTest def test_render__lists(self): pass @filesBasedTest def test_render__quotes(self): pass mistletoe-0.8.2/test/test_core_tokens.py000066400000000000000000000062271420102462200204310ustar00rootroot00000000000000from unittest import TestCase from unittest.mock import patch from mistletoe.core_tokens import (MatchObj, Delimiter, follows, shift_whitespace, is_control_char, deactivate_delimiters, preceded_by, succeeded_by) class TestCoreTokens(TestCase): def test_match_obj(self): match = MatchObj(0, 2, (0, 1, 'a'), (1, 2, 'b')) self.assertEqual(match.start(), 0) self.assertEqual(match.start(1), 0) self.assertEqual(match.start(2), 1) self.assertEqual(match.end(), 2) self.assertEqual(match.end(1), 1) self.assertEqual(match.end(2), 2) self.assertEqual(match.group(), 'ab') self.assertEqual(match.group(1), 'a') self.assertEqual(match.group(2), 'b') def test_delimiter(self): delimiter = Delimiter(4, 6, 'abcd**') self.assertEqual(delimiter.type, '**') self.assertEqual(delimiter.number, 2) self.assertEqual(delimiter.active, True) self.assertEqual(delimiter.start, 4) self.assertEqual(delimiter.end, 6) def test_delimiter_remove_left(self): delimiter = Delimiter(4, 6, 'abcd**') self.assertTrue(delimiter.remove(1, left=True)) self.assertEqual(delimiter.number, 1) self.assertEqual(delimiter.start, 5) self.assertEqual(delimiter.end, 6) def test_delimiter_remove_right(self): delimiter = Delimiter(4, 6, 'abcd**') self.assertTrue(delimiter.remove(1, left=False)) self.assertEqual(delimiter.number, 1) self.assertEqual(delimiter.start, 4) self.assertEqual(delimiter.end, 5) def test_delimiter_remove_empty(self): delimiter = Delimiter(4, 6, 'abcd**') self.assertFalse(delimiter.remove(2)) def test_follows(self): string = '(foobar)' self.assertTrue(follows(string, 6, ')')) self.assertFalse(follows(string, 6, '(')) self.assertFalse(follows(string, 7, ')')) def test_shift_whitespace(self): string = ' \n\t\rfoo' self.assertEqual(shift_whitespace(string, 0), 4) self.assertEqual(shift_whitespace('', 0), 0) def test_is_control_char(self): char = chr(0) self.assertTrue(is_control_char(char)) self.assertFalse(is_control_char('a')) def test_deactivate_delimiters(self): s = 'abc' delimiters = [Delimiter(0, 1, s), Delimiter(1, 2, s), Delimiter(2, 3, s)] deactivate_delimiters(delimiters, 2, 'b') self.assertTrue(delimiters[0].active) self.assertFalse(delimiters[1].active) self.assertTrue(delimiters[2].active) def test_preceded_by(self): whitespace = ' \t\n\r' self.assertTrue(preceded_by(1, ' abc', whitespace)) self.assertTrue(preceded_by(0, 'aabc', whitespace)) self.assertFalse(preceded_by(1, 'aabc', whitespace)) self.assertFalse(preceded_by(0, 'aabc', 'abc')) def test_succeeded_by(self): whitespace = ' \t\n\r' self.assertTrue(succeeded_by(3, 'abc ', whitespace)) self.assertTrue(succeeded_by(4, 'abcc', whitespace)) self.assertFalse(succeeded_by(3, 'abcc', whitespace)) self.assertFalse(succeeded_by(4, 'abcc', 'abc')) mistletoe-0.8.2/test/test_html_renderer.py000066400000000000000000000133351420102462200207460ustar00rootroot00000000000000from unittest import TestCase, mock from mistletoe.html_renderer import HTMLRenderer class TestRenderer(TestCase): def setUp(self): self.renderer = HTMLRenderer() self.renderer.render_inner = mock.Mock(return_value='inner') self.renderer.__enter__() self.addCleanup(self.renderer.__exit__, None, None, None) def _test_token(self, token_name, output, children=True, without_attrs=None, **kwargs): render_func = self.renderer.render_map[token_name] children = mock.MagicMock(spec=list) if children else None mock_token = mock.Mock(children=children, **kwargs) without_attrs = without_attrs or [] for attr in without_attrs: delattr(mock_token, attr) self.assertEqual(render_func(mock_token), output) class TestHTMLRenderer(TestRenderer): def test_strong(self): self._test_token('Strong', 'inner') def test_emphasis(self): self._test_token('Emphasis', 'inner') def test_inline_code(self): from mistletoe.span_token import tokenize_inner rendered = self.renderer.render(tokenize_inner('`foo`')[0]) self.assertEqual(rendered, 'foo') def test_strikethrough(self): self._test_token('Strikethrough', 'inner') def test_image(self): output = '' self._test_token('Image', output, src='src', title='title') def test_link(self): output = 'inner' self._test_token('Link', output, target='target', title='title') def test_autolink(self): output = 'inner' self._test_token('AutoLink', output, target='link', mailto=False) def test_escape_sequence(self): self._test_token('EscapeSequence', 'inner') def test_raw_text(self): self._test_token('RawText', 'john & jane', children=False, content='john & jane') def test_html_span(self): self._test_token('HTMLSpan', 'text', children=False, content='text') def test_heading(self): output = '

inner

' self._test_token('Heading', output, level=3) def test_quote(self): output = '
\n
' self._test_token('Quote', output) def test_paragraph(self): self._test_token('Paragraph', '

inner

') def test_block_code(self): from mistletoe.block_token import tokenize rendered = self.renderer.render(tokenize(['```sh\n', 'foo\n', '```\n'])[0]) output = '
foo\n
' self.assertEqual(rendered, output) def test_block_code_no_language(self): from mistletoe.block_token import tokenize rendered = self.renderer.render(tokenize(['```\n', 'foo\n', '```\n'])[0]) output = '
foo\n
' self.assertEqual(rendered, output) def test_list(self): output = '
    \n\n
' self._test_token('List', output, start=None) def test_list_item(self): output = '
  • ' self._test_token('ListItem', output) def test_table_with_header(self): func_path = 'mistletoe.html_renderer.HTMLRenderer.render_table_row' with mock.patch(func_path, autospec=True) as mock_func: mock_func.return_value = 'row' output = ('\n' '\nrow\n' '\ninner\n' '
    ') self._test_token('Table', output) def test_table_without_header(self): func_path = 'mistletoe.html_renderer.HTMLRenderer.render_table_row' with mock.patch(func_path, autospec=True) as mock_func: mock_func.return_value = 'row' output = '\n\ninner\n
    ' self._test_token('Table', output, without_attrs=['header',]) def test_table_row(self): self._test_token('TableRow', '\n\n') def test_table_cell(self): output = 'inner\n' self._test_token('TableCell', output, align=None) def test_table_cell0(self): output = 'inner\n' self._test_token('TableCell', output, align=0) def test_table_cell1(self): output = 'inner\n' self._test_token('TableCell', output, align=1) def test_thematic_break(self): self._test_token('ThematicBreak', '
    ', children=False) def test_html_block(self): content = output = '

    hello

    \n

    this is\na paragraph

    \n' self._test_token('HTMLBlock', output, children=False, content=content) def test_line_break(self): self._test_token('LineBreak', '
    \n', children=False, soft=False) def test_document(self): self._test_token('Document', '', footnotes={}) class TestHTMLRendererFootnotes(TestCase): def setUp(self): self.renderer = HTMLRenderer() self.renderer.__enter__() self.addCleanup(self.renderer.__exit__, None, None, None) def test_footnote_image(self): from mistletoe import Document token = Document(['![alt][foo]\n', '\n', '[foo]: bar "title"\n']) output = '

    alt

    \n' self.assertEqual(self.renderer.render(token), output) def test_footnote_link(self): from mistletoe import Document token = Document(['[name][foo]\n', '\n', '[foo]: target\n']) output = '

    name

    \n' self.assertEqual(self.renderer.render(token), output) mistletoe-0.8.2/test/test_latex_renderer.py000066400000000000000000000122151420102462200211130ustar00rootroot00000000000000from unittest import TestCase, mock import mistletoe.latex_token as latex_token from mistletoe.latex_renderer import LaTeXRenderer class TestLaTeXRenderer(TestCase): def setUp(self): self.renderer = LaTeXRenderer() self.renderer.render_inner = mock.Mock(return_value='inner') self.renderer.__enter__() self.addCleanup(self.renderer.__exit__, None, None, None) def _test_token(self, token_name, output, children=True, without_attrs=None, **kwargs): render_func = self.renderer.render_map[token_name] children = mock.MagicMock(spec=list) if children else None mock_token = mock.Mock(children=children, **kwargs) without_attrs = without_attrs or [] for attr in without_attrs: delattr(mock_token, attr) self.assertEqual(render_func(mock_token), output) def test_strong(self): self._test_token('Strong', '\\textbf{inner}') def test_emphasis(self): self._test_token('Emphasis', '\\textit{inner}') def test_inline_code(self): func_path = 'mistletoe.latex_renderer.LaTeXRenderer.render_raw_text' with mock.patch(func_path, return_value='inner'): self._test_token('InlineCode', '\\verb|inner|') def test_strikethrough(self): self._test_token('Strikethrough', '\\sout{inner}') def test_image(self): output = '\n\\includegraphics{src}\n' self._test_token('Image', output, src='src') def test_link(self): output = '\\href{target}{inner}' self._test_token('Link', output, target='target') def test_autolink(self): output = '\\url{target}' self._test_token('AutoLink', output, target='target') def test_math(self): output = '$ 1 + 2 = 3 $' self._test_token('Math', output, children=False, content='$ 1 + 2 = 3 $') def test_raw_text(self): output = '\\$\\&\\#\\{\\}' self._test_token('RawText', output, children=False, content='$&#{}') def test_heading(self): output = '\n\\section{inner}\n' self._test_token('Heading', output, level=1) def test_quote(self): output = '\\begin{displayquote}\ninner\\end{displayquote}\n' self._test_token('Quote', output) def test_paragraph(self): output = '\ninner\n' self._test_token('Paragraph', output) def test_block_code(self): func_path = 'mistletoe.latex_renderer.LaTeXRenderer.render_raw_text' with mock.patch(func_path, return_value='inner'): output = '\n\\begin{lstlisting}[language=sh]\ninner\\end{lstlisting}\n' self._test_token('BlockCode', output, language='sh') def test_list(self): output = '\\begin{itemize}\ninner\\end{itemize}\n' self._test_token('List', output, start=None) def test_list_item(self): self._test_token('ListItem', '\\item inner\n') def test_table_with_header(self): func_path = 'mistletoe.latex_renderer.LaTeXRenderer.render_table_row' with mock.patch(func_path, autospec=True, return_value='row\n'): output = '\\begin{tabular}{l c r}\nrow\n\\hline\ninner\\end{tabular}\n' self._test_token('Table', output, column_align=[None, 0, 1]) def test_table_without_header(self): output = ('\\begin{tabular}\ninner\\end{tabular}\n') self._test_token('Table', output, without_attrs=['header'], column_align=[None]) def test_table_row(self): self._test_token('TableRow', ' \\\\\n') def test_table_cell(self): self._test_token('TableCell', 'inner') def test_thematic_break(self): self._test_token('ThematicBreak', '\\hrulefill\n') def test_line_break(self): self._test_token('LineBreak', '\\newline\n', soft=False) def test_document(self): output = ('\\documentclass{article}\n' '\\begin{document}\n' 'inner' '\\end{document}\n') self._test_token('Document', output, footnotes={}) class TestLaTeXFootnotes(TestCase): def setUp(self): self.renderer = LaTeXRenderer() self.renderer.__enter__() self.addCleanup(self.renderer.__exit__, None, None, None) def test_footnote_image(self): from mistletoe import Document raw = ['![alt][foo]\n', '\n', '[foo]: bar "title"\n'] target = ('\\documentclass{article}\n' '\\usepackage{graphicx}\n' '\\begin{document}\n' '\n' '\n\\includegraphics{bar}\n' '\n' '\\end{document}\n') self.assertEqual(self.renderer.render(Document(raw)), target) def test_footnote_link(self): from mistletoe import Document raw = ['[name][key]\n', '\n', '[key]: target\n'] target = ('\\documentclass{article}\n' '\\usepackage{hyperref}\n' '\\begin{document}\n' '\n' '\\href{target}{name}' '\n' '\\end{document}\n') self.assertEqual(self.renderer.render(Document(raw)), target) mistletoe-0.8.2/test/test_latex_token.py000066400000000000000000000010401420102462200204170ustar00rootroot00000000000000import unittest from mistletoe.span_token import tokenize_inner from mistletoe.latex_token import Math from mistletoe.latex_renderer import LaTeXRenderer class TestLaTeXToken(unittest.TestCase): def setUp(self): self.renderer = LaTeXRenderer() self.renderer.__enter__() self.addCleanup(self.renderer.__exit__, None, None, None) def test_span(self): token = next(iter(tokenize_inner('$ 1 + 2 = 3 $'))) self.assertIsInstance(token, Math) self.assertEqual(token.content, '$ 1 + 2 = 3 $') mistletoe-0.8.2/test/test_span_token.py000066400000000000000000000115741420102462200202600ustar00rootroot00000000000000import unittest from unittest.mock import patch from mistletoe import span_token from functools import wraps class TestBranchToken(unittest.TestCase): def setUp(self): self.addCleanup(lambda: span_token._token_types.__setitem__(-1, span_token.RawText)) patcher = patch('mistletoe.span_token.RawText') self.mock = patcher.start() span_token._token_types[-1] = self.mock self.addCleanup(patcher.stop) def _test_parse(self, token_cls, raw, arg, **kwargs): token = next(iter(span_token.tokenize_inner(raw))) self.assertIsInstance(token, token_cls) self._test_token(token, arg, **kwargs) return token def _test_token(self, token, arg, children=True, **kwargs): for attr, value in kwargs.items(): self.assertEqual(getattr(token, attr), value) if children: self.mock.assert_any_call(arg) class TestStrong(TestBranchToken): def test_parse(self): self._test_parse(span_token.Strong, '**some text**', 'some text') self._test_parse(span_token.Strong, '__some text__', 'some text') class TestEmphasis(TestBranchToken): def test_parse(self): self._test_parse(span_token.Emphasis, '*some text*', 'some text') self._test_parse(span_token.Emphasis, '_some text_', 'some text') class TestInlineCode(TestBranchToken): def _test_parse_enclosed(self, encl_type, encl_delimiter): token = self._test_parse(encl_type, '{delim}`some text`{delim}'.format(delim=encl_delimiter), 'some text') self.assertEqual(len(token.children), 1) self.assertIsInstance(token.children[0], span_token.InlineCode) def test_parse(self): self._test_parse(span_token.InlineCode, '`some text`', 'some text') def test_parse_in_bold(self): self._test_parse_enclosed(span_token.Strong, '**') self._test_parse_enclosed(span_token.Strong, '__') def test_parse_in_emphasis(self): self._test_parse_enclosed(span_token.Emphasis, '*') self._test_parse_enclosed(span_token.Emphasis, '_') def test_parse_in_strikethrough(self): self._test_parse_enclosed(span_token.Strikethrough, '~~') class TestStrikethrough(TestBranchToken): def test_parse(self): self._test_parse(span_token.Strikethrough, '~~some text~~', 'some text') def test_parse_multiple(self): tokens = iter(span_token.tokenize_inner('~~one~~ ~~two~~')) self._test_token(next(tokens), 'one') self._test_token(next(tokens), 'two') class TestLink(TestBranchToken): def test_parse(self): self._test_parse(span_token.Link, '[name 1](target1)', 'name 1', target='target1', title='') def test_parse_multi_links(self): tokens = iter(span_token.tokenize_inner('[n1](t1) & [n2](t2)')) self._test_token(next(tokens), 'n1', target='t1') self._test_token(next(tokens), ' & ', children=False) self._test_token(next(tokens), 'n2', target='t2') def test_parse_children(self): token = next(iter(span_token.tokenize_inner('[![alt](src)](target)'))) child = next(iter(token.children)) self._test_token(child, 'alt', src='src') class TestAutoLink(TestBranchToken): def test_parse(self): self._test_parse(span_token.AutoLink, '', 'ftp://foo.com', target='ftp://foo.com') class TestImage(TestBranchToken): def test_parse(self): self._test_parse(span_token.Image, '![alt](link)', 'alt', src='link') self._test_parse(span_token.Image, '![alt](link "title")', 'alt', src='link', title='title') def test_no_alternative_text(self): self._test_parse(span_token.Image, '![](link)', '', children=False, src='link') class TestEscapeSequence(TestBranchToken): def test_parse(self): self._test_parse(span_token.EscapeSequence, '\*', '*') def test_parse_in_text(self): tokens = iter(span_token.tokenize_inner('some \*text*')) self._test_token(next(tokens), 'some ', children=False) self._test_token(next(tokens), '*') self._test_token(next(tokens), 'text*', children=False) class TestRawText(unittest.TestCase): def test_attribute(self): token = span_token.RawText('some text') self.assertEqual(token.content, 'some text') def test_no_children(self): token = span_token.RawText('some text') with self.assertRaises(AttributeError): token.children class TestLineBreak(unittest.TestCase): def test_parse(self): token, = span_token.tokenize_inner(' \n') self.assertIsInstance(token, span_token.LineBreak) class TestContains(unittest.TestCase): def test_contains(self): token = next(iter(span_token.tokenize_inner('**with some *emphasis* text**'))) self.assertTrue('text' in token) self.assertTrue('emphasis' in token) self.assertFalse('foo' in token) mistletoe-0.8.2/test/test_traverse.py000066400000000000000000000021011420102462200177340ustar00rootroot00000000000000from textwrap import dedent import unittest from mistletoe import Document from mistletoe.utils import traverse class TestTraverse(unittest.TestCase): def test_a(self): doc = Document( dedent( """\ a **b** c [*d*](link) """ ) ) tree = [ ( t.node.__class__.__name__, t.parent.__class__.__name__ if t.parent else None, t.depth ) for t in traverse(doc, include_source=True) ] self.assertEqual( tree, [ ('Document', None, 0), ('Paragraph', 'Document', 1), ('Paragraph', 'Document', 1), ('RawText', 'Paragraph', 2), ('Strong', 'Paragraph', 2), ('RawText', 'Paragraph', 2), ('Link', 'Paragraph', 2), ('RawText', 'Strong', 3), ('Emphasis', 'Link', 3), ('RawText', 'Emphasis', 4), ] )