pax_global_header00006660000000000000000000000064146164733610014525gustar00rootroot0000000000000052 comment=332967db4c98054500f76e1de5d299d19b218a6e parsexp-0.17.0/000077500000000000000000000000001461647336100132745ustar00rootroot00000000000000parsexp-0.17.0/.gitignore000066400000000000000000000000411461647336100152570ustar00rootroot00000000000000_build *.install *.merlin _opam parsexp-0.17.0/.ocamlformat000066400000000000000000000000231461647336100155740ustar00rootroot00000000000000profile=janestreet parsexp-0.17.0/CHANGES.md000066400000000000000000000010361461647336100146660ustar00rootroot00000000000000## Release v0.17.0 - Added `Parsexp.Conv_many_and_locations` - In eager parser, add an option `reraise_notrace` to better support call sites that use exceptions as control-flow. ## Release v0.16.0 - Expose type equalities for `State.t` and `Stack.t` types in off-the-shelf parser modules. This lets you use one of those modules, but without giving up the flexibility of interaction with the parser state. ## Old pre-v0.15 changelogs (very likely stale and incomplete) # v0.11 Drop dependency on Base. # v0.10 Initial release parsexp-0.17.0/CONTRIBUTING.md000066400000000000000000000044101461647336100155240ustar00rootroot00000000000000This repository contains open source software that is developed and maintained by [Jane Street][js]. Contributions to this project are welcome and should be submitted via GitHub pull requests. Signing contributions --------------------- We require that you sign your contributions. Your signature certifies that you wrote the patch or otherwise have the right to pass it on as an open-source patch. The rules are pretty simple: if you can certify the below (from [developercertificate.org][dco]): ``` Developer Certificate of Origin Version 1.1 Copyright (C) 2004, 2006 The Linux Foundation and its contributors. 1 Letterman Drive Suite D4700 San Francisco, CA, 94129 Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Developer's Certificate of Origin 1.1 By making a contribution to this project, I certify that: (a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or (b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or (c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it. (d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. ``` Then you just add a line to every git commit message: ``` Signed-off-by: Joe Smith ``` Use your real name (sorry, no pseudonyms or anonymous contributions.) If you set your `user.name` and `user.email` git configs, you can sign your commit automatically with git commit -s. [dco]: http://developercertificate.org/ [js]: https://opensource.janestreet.com/ parsexp-0.17.0/LICENSE.md000066400000000000000000000021461461647336100147030ustar00rootroot00000000000000The MIT License Copyright (c) 2016--2024 Jane Street Group, LLC Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. parsexp-0.17.0/Makefile000066400000000000000000000004031461647336100147310ustar00rootroot00000000000000INSTALL_ARGS := $(if $(PREFIX),--prefix $(PREFIX),) default: dune build install: dune install $(INSTALL_ARGS) uninstall: dune uninstall $(INSTALL_ARGS) reinstall: uninstall install clean: dune clean .PHONY: default install uninstall reinstall clean parsexp-0.17.0/README.org000066400000000000000000000121721461647336100147450ustar00rootroot00000000000000* Parsexp - S-expression parser =Parsexp= contains functionality for parsing s-expressions, which are defined in the [[https://github.com/janestreet/base][Base]] library as follows: #+begin_src ocaml module Sexp : sig type t = Atom of string | List of t list end #+end_src =Parsexp= is platform independent and as such doesn't provide IO operations. The companion library [[https://github.com/janestreet/parsexp_io][Parsexp_io]] provides facilities for loading s-expressions from files. ** Syntax of s-expression *** Lexical conventions of s-expression Whitespace, which consists of space, newline, horizontal tab, and form feed, is ignored unless within an OCaml-string, where it is treated according to OCaml-conventions. The left parenthesis opens a new list, the right one closes it. Lists can be empty. The double quote denotes the beginning and end of a string using similar lexing conventions to the ones of OCaml (see the [[http://caml.inria.fr/pub/docs/manual-ocaml/][OCaml-manual]] for details). Differences are: - octal escape sequences (=\o123=) are not supported by parsexp - backslash that's not a part of any escape sequence is kept as it is instead of resulting in parse error - a backslash followed by a space does not form an escape sequence, so it's interpreted as is, while it is interpreted as just a space by OCaml All characters other than double quotes, left- and right parentheses, whitespace, carriage return, and comment-introducing characters or sequences (see next paragraph) are considered part of a contiguous string. *** Comments There are three kinds of comments: - _line comments_ are introduced with =;=, and end at the newline. - _sexp comments_ are introduced with =#;=, and end at the end of the following s-expression - _block comments_ are introduced with =#|= and end with =|#=. These can be nested, and double-quotes within them must be balanced and be lexically correct OCaml strings. *** Grammar of s-expressions s-expressions are either strings (= atoms) or lists. The lists can recursively contain further s-expressions or be empty, and must be balanced, /i.e./ parentheses must match. *** Examples #+begin_src scheme this_is_an_atom_123'&^%! ; this is a comment "another atom in an OCaml-string \"string in a string\" \123" ; empty list follows below () ; a more complex example ( ( list in a list ; comment within a list (list in a list in a list) 42 is the answer to all questions #; (this S-expression (has been commented out) ) #| Block comments #| can be "nested" |# |# ) ) #+end_src ** API =Parsexp= offers a few different parsers. Each of them comes in 3 variants: 1. =single=, for parsing a source expected to contain a single s-expression. Zero or more than 1 s-expressions is considered an error and reported as such. 2. =many=, for parsing a source expected to contain multiple s-expressions, returned as a list 3. =eager=, for reporting s-expressions to the user as soon as they are found in the input source Parsers all implement the same API. For each parsers, =Parsexp= offers a low-level API, where one feeds characters one-by-one to the parser, as well as a convenience functions to parse a whole string. *** Low-level API With the low-level API, ones must create a parser state and feed it characters one by one. This allows =Parsexp= to be used with any kind of input as well as to be =Lwt= or =Async= friendly. Essentially, if you want to parse from a different input source, you have to write the fetch-character-feed-to-parser loop yourself. *** Parsers =Parsexp= offers the following parsers: - _normal parsers_ returning values of type =Sexp.t= - _with positions_ returning s-expressions as well as a set of positions (see next section) - _just positions_ returning only a set of positions - _CST_ return a concrete syntax tree, including all comment and locations In addition, it provides convenience functions for parsing strings and converting the result with a =Sexp.t -> 'a= function, reporting errors with accurate locations. Each parsing/conversion functions comes in two variants: one raising exception in case of error and one returning a =Result.t= value. In general you should use the latter since parsing is an operation that can always fail. *** Positions sets and error reporting To deal with error locations when converting S-expressions to OCaml values, =Parsexp= introduces the notion of compact positions sets. A positions set represents positions in the input source of all s-expressions. It has a small memory footprint and is relatively cheap to construct. Using a positions set and the corresponding s-expression, one can reconstruct the location of any sub s-expressions in the input source. Depending on the input source, one can either: - parse a first time without recording positions and parse a second time only producing positions in case of error - parse only once producing both the s-expressions and the positions sets The first method has the advantage that it is faster where there are no errors, however it is not suitable for sources that can't guarantee repeatable reads such as files. parsexp-0.17.0/bench/000077500000000000000000000000001461647336100143535ustar00rootroot00000000000000parsexp-0.17.0/bench/dune000066400000000000000000000001671461647336100152350ustar00rootroot00000000000000(library (name parsexp_bench) (libraries core_kernel.caml_unix core parsexp sexplib) (preprocess (pps ppx_jane))) parsexp-0.17.0/bench/parsexp_vs_sexplib.ml000066400000000000000000000036751461647336100206400ustar00rootroot00000000000000open Core open Poly module Unix = Caml_unix let data_multi = let data_fn = "data.sexp" in if Stdlib.Sys.file_exists data_fn then sprintf "(%s)" (In_channel.read_all data_fn) else ( (* Collect all the jbuilds in $ROOT/lib *) let ic = Unix.open_process_in "find ../../../lib -name jbuild -exec cat {} \\;" in let s = In_channel.input_all ic in assert (Unix.close_process_in ic = WEXITED 0); sprintf "%s" s) ;; let data = sprintf "(%s)" data_multi (* To make sure the input is valid before starting the bench *) let (_ : Sexp.t) = Sexplib.Sexp.of_string data let _, positions = Parsexp.Single_and_positions.parse_string_exn data let () = let pos_mem_kb = Parsexp.Positions.memory_footprint_in_bytes positions / 1024 in printf "input size: %d KB\nposition set size: %d KB (%d KW)\n%!" (String.length data / 1024) pos_mem_kb (pos_mem_kb / match Word_size.word_size with | W32 -> 4 | W64 -> 8) ;; (* Obj.tag is a C call so the compiler can't consider the value as dead-code *) let don't_optimize_out x = ignore (Obj.tag (Obj.repr x) : int) let%bench "sexplib" = don't_optimize_out (Sexplib.Sexp.of_string data : Sexp.t) let%bench "sexplib.annotated" = don't_optimize_out (Sexplib.Sexp.Annotated.of_string data : Sexp.Annotated.t) ;; let%bench "parsexp" = don't_optimize_out (Parsexp.Single.parse_string_exn data : Sexp.t) let%bench "parsexp+positions" = don't_optimize_out (Parsexp.Single_and_positions.parse_string_exn data : _ * _) ;; let eager_parse_sexps str ~f = let module P = Parsexp.Eager in let results = Queue.create () in let got_sexp _state sexp = Core.Queue.enqueue results (f sexp) in let state = P.State.create got_sexp ~no_sexp_is_error:false in let stack = P.feed_string state str P.Stack.empty in P.feed_eoi state stack; Queue.to_list results ;; let%bench "parsexp eager" = don't_optimize_out (eager_parse_sexps data_multi ~f:(fun (_ : Sexp.t) -> ()) : unit list) ;; parsexp-0.17.0/bench/parsexp_vs_sexplib.mli000066400000000000000000000000551461647336100207760ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/dune-project000066400000000000000000000000211461647336100156070ustar00rootroot00000000000000(lang dune 3.11) parsexp-0.17.0/fuzz/000077500000000000000000000000001461647336100142725ustar00rootroot00000000000000parsexp-0.17.0/fuzz/dune000066400000000000000000000002361461647336100151510ustar00rootroot00000000000000(executables (modes byte exe) (names parser_equivalence) (libraries aflPersistent base parsexp parsexp_test sexplib stdio) (preprocess (pps ppx_jane))) parsexp-0.17.0/fuzz/parser_equivalence.ml000066400000000000000000000231271461647336100205060ustar00rootroot00000000000000(* About this file: It checks the equivalence of the various parsexp parsers, as well some sexplib parsers. This check is manual, it is not run in hydra in any way. This requires instrumentation, so I build my own compiler like so: # ./configure -prefix $PWD/installed -afl-instrument && make world world.opt install Without a custom compiler, the stdlib will not be instrumented. The way I run it: {v function fuzz-run { ( mkdir -p input echo a > input/testcase rm -rf output/min-* shopt -s nullglob export AFL_SKIP_CPUFREQ= for prop in {0..14}; do export PROP=$prop timeout 2m afl-fuzz -i input -o output ./parser_equivalence.exe mkdir -p output/min-$prop for f in output/crashes/id\:*; do afl-tmin -i $f -o output/min-$prop/$(basename $f) -- ./parser_equivalence.exe done done ) } function fuzz-result { ( shopt -s nullglob for v in output/min-*; do prop=$(echo $(basename $v) | cut -d - -f 2-) for f in $(md5sum output/min-$prop/* | sort -u | awk '{ tbl[$1] = $2 } END { for (k in tbl) { print tbl[k] } }' | sort | grep -v '^-$'); do echo --- prop:$prop PROP=$prop NICE_EXN= ./parser_equivalence.exe < $f done done ) < /dev/null |& less -R } v} This is needed because the cli of afl-fuzz is wonky. Run fuzz-run to actually run tests. Run fuzz-result either during the tests or after to see the failures so far. *) open Base module P = Parsexp let compare_exn _ _ = 0 type 'a or_exn = ('a, exn) Result.t [@@deriving compare, sexp_of] let parsexp_single str = Result.try_with (fun () -> P.Single.parse_string_exn str) let parsexp_many str = Result.try_with (fun () -> P.Many.parse_string_exn str) let parsexp_eager str ~no_sexp_is_error = let r = ref [] in let state = P.Eager.State.create ~no_sexp_is_error (fun _ v -> r := v :: !r) in Result.try_with (fun () -> let stack = P.Eager.Stack.empty in let stack = P.Eager.feed_string state str stack in P.Eager.feed_eoi state stack; List.rev !r) ;; let parsexp_single_and_positions str = Result.try_with (fun () -> P.Single_and_positions.parse_string_exn str) ;; let parsexp_many_and_positions str = Result.try_with (fun () -> P.Many_and_positions.parse_string_exn str) ;; let parsexp_eager_and_positions str = let r = ref [] in let state = P.Eager_and_positions.State.create ~no_sexp_is_error:false (fun _ v -> r := v :: !r) in Result.try_with (fun () -> let stack = P.Eager_and_positions.Stack.empty in let stack = P.Eager_and_positions.feed_string state str stack in P.Eager_and_positions.feed_eoi state stack; List.rev !r) ;; let parsexp_single_just_positions str = Result.try_with (fun () -> P.Single_just_positions.parse_string_exn str) ;; let parsexp_many_just_positions str = Result.try_with (fun () -> P.Many_just_positions.parse_string_exn str) ;; let parsexp_eager_just_positions str = let r = ref [] in let state = P.Eager_just_positions.State.create ~no_sexp_is_error:false (fun _ v -> r := v :: !r) in Result.try_with (fun () -> let stack = P.Eager_just_positions.Stack.empty in let stack = P.Eager_just_positions.feed_string state str stack in P.Eager_just_positions.feed_eoi state stack; List.rev !r) ;; let parsexp_many_cst str = Result.try_with (fun () -> P.Many_cst.parse_string_exn str) let parsexp_eager_cst str = let r = ref [] in let state = P.Eager_cst.State.create ~no_sexp_is_error:false (fun _ v -> r := v :: !r) in Result.try_with (fun () -> let stack = P.Eager_cst.Stack.empty in let stack = P.Eager_cst.feed_string state str stack in P.Eager_cst.feed_eoi state stack; List.rev !r) ;; let sexplib_single str = Result.try_with (fun () -> Sexplib.Sexp.of_string str) let sexplib_many str = Result.try_with (fun () -> Parsexp_test.Import.sexplib_sexps_of_string str) ;; let single_of_many = function | Ok [ x ] -> Ok x | Ok _ -> Error (Failure "not single") | Error _ as e -> e ;; let fst_or_error = function | Ok p -> Ok (fst p) | Error _ as e -> e ;; let snd_or_error = function | Ok p -> Ok (snd p) | Error _ as e -> e ;; let main ~nice_exn ~property = let buf = Buffer.create 100000 in AflPersistent.run (fun () -> let str = ignore (Stdio.In_channel.input_buffer Stdio.In_channel.stdin buf ~len:100000); let str = Buffer.contents buf in Buffer.reset buf; str in let equal_or_raise (type a) sexp_of_a compare_a name1 name2 v1 v2 = if [%compare: a or_exn] v1 v2 <> 0 then if nice_exn then raise_s [%sexp (str : string) , ((name1 : string), (v1 : a or_exn)) , ((name2 : string), (v2 : a or_exn))] else Stdlib.raise_notrace Stdlib.Exit in match property with | 0 -> equal_or_raise [%sexp_of: Sexp.t] [%compare: Sexp.t] "parsexp_single" "parsexp_many" (parsexp_single str) (single_of_many (parsexp_many str)) | 1 -> equal_or_raise [%sexp_of: Sexp.t list] [%compare: Sexp.t list] "parsexp_many" "parsexp_eager_no_sexp_is_error:false" (parsexp_many str) (parsexp_eager str ~no_sexp_is_error:false) | 2 -> equal_or_raise [%sexp_of: Sexp.t list] [%compare: Sexp.t list] "parsexp_many" "parsexp_eager_no_sexp_is_error:true" (match parsexp_many str with | Ok [] -> Error (Failure "empty") | x -> x) (parsexp_eager str ~no_sexp_is_error:true) | 3 -> equal_or_raise [%sexp_of: Sexp.t] [%compare: Sexp.t] "parsexp_single" "parsexp_single_and_positions" (parsexp_single str) (fst_or_error (parsexp_single_and_positions str)) | 4 -> equal_or_raise [%sexp_of: Sexp.t list] [%compare: Sexp.t list] "parsexp_many" "parsexp_many_and_positions" (parsexp_many str) (fst_or_error (parsexp_many_and_positions str)) | 5 -> equal_or_raise [%sexp_of: Sexp.t list] [%compare: Sexp.t list] "parsexp_many" "parsexp_many_and_positions" (parsexp_many str) (match parsexp_eager_and_positions str with | Ok l -> Ok (List.map l ~f:fst) | Error _ as e -> e) | 6 -> equal_or_raise [%sexp_of: Sexp.t * P.Positions.t] [%compare: Sexp.t * P.Positions.t] "parsexp_single_and_positions" "parsexp_many_and_positions" (parsexp_single_and_positions str) (match parsexp_many_and_positions str with | Ok ([ v ], positions) -> Ok (v, positions) | Error _ as e -> e | Ok (_, _) -> Error (Failure "not many")) | 7 -> equal_or_raise [%sexp_of: P.Positions.t] [%compare: P.Positions.t] "parsexp_single_just_positions" "parsexp_single_and_positions" (parsexp_single_just_positions str) (snd_or_error (parsexp_single_and_positions str)) | 8 -> equal_or_raise [%sexp_of: P.Positions.t] [%compare: P.Positions.t] "parsexp_many_just_positions" "parsexp_many_and_positions" (parsexp_many_just_positions str) (snd_or_error (parsexp_many_and_positions str)) | 9 -> equal_or_raise [%sexp_of: P.Positions.t list] [%compare: P.Positions.t list] "parsexp_eager_just_positions" "parsexp_eager_and_positions" (parsexp_eager_just_positions str) (match parsexp_eager_and_positions str with | Ok l -> Ok (List.map l ~f:snd) | Error _ as e -> e) | 10 -> equal_or_raise [%sexp_of: P.Positions.pos array] [%compare: P.Positions.pos array] "parsexp_many_just_positions" "parsexp_eager_just_positions" (match parsexp_many_just_positions str with | Ok a -> Ok (P.Positions.to_array a) | Error _ as e -> e) (match parsexp_eager_and_positions str with | Ok l -> Ok (Array.concat_map (Array.of_list l) ~f:(fun a -> P.Positions.to_array (snd a))) | Error _ as e -> e) | 11 -> equal_or_raise [%sexp_of: Sexp.t list] [%compare: Sexp.t list] "parsexp_many" "parsexp_many_cst" (parsexp_many str) (match parsexp_many_cst str with | Error _ as e -> e | Ok l -> Ok (P.Cst.Forget.t_or_comments l)) | 12 -> equal_or_raise [%sexp_of: P.Cst.t_or_comment list] [%compare: P.Cst.t_or_comment list] "parsexp_many_cst" "parsexp_eager_cst" (parsexp_many_cst str) (parsexp_eager_cst str) | 13 -> equal_or_raise [%sexp_of: Sexp.t] [%compare: Sexp.t] "parsexp_single" "sexplib_single" (parsexp_single str) (sexplib_single str) | 14 -> equal_or_raise [%sexp_of: Sexp.t list] [%compare: Sexp.t list] "parsexp_many" "sexplib_many" (parsexp_many str) (sexplib_many str) | _ -> raise_s [%sexp "property out of range", (property : int)]) ;; let nice_exn = match Stdlib.Sys.getenv "NICE_EXN" with | exception (Not_found_s _ | Stdlib.Not_found) -> false | (_ : string) -> true ;; let property = match Stdlib.Sys.getenv "PROP" with | exception (Not_found_s _ | Stdlib.Not_found) -> assert false | s -> Int.of_string s ;; let () = main ~nice_exn ~property parsexp-0.17.0/fuzz/parser_equivalence.mli000066400000000000000000000000551461647336100206520ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/parsexp.opam000066400000000000000000000027401461647336100156370ustar00rootroot00000000000000opam-version: "2.0" version: "v0.17.0" maintainer: "Jane Street developers" authors: ["Jane Street Group, LLC"] homepage: "https://github.com/janestreet/parsexp" bug-reports: "https://github.com/janestreet/parsexp/issues" dev-repo: "git+https://github.com/janestreet/parsexp.git" doc: "https://ocaml.janestreet.com/ocaml-core/latest/doc/parsexp/index.html" license: "MIT" build: [ ["dune" "build" "-p" name "-j" jobs] ] depends: [ "ocaml" {>= "5.1.0"} "sexplib0" {>= "v0.17" & < "v0.18"} "dune" {>= "3.11.0"} ] available: arch != "arm32" & arch != "x86_32" synopsis: "S-expression parsing library" description: " This library provides generic parsers for parsing S-expressions from strings or other medium. The library is focused on performances but still provide full generic parsers that can be used with strings, bigstrings, lexing buffers, character streams or any other sources effortlessly. It provides three different class of parsers: - the normal parsers, producing [Sexp.t] or [Sexp.t list] values - the parsers with positions, building compact position sequences so that one can recover original positions in order to report properly located errors at little cost - the Concrete Syntax Tree parsers, produce values of type [Parsexp.Cst.t] which record the concrete layout of the s-expression syntax, including comments This library is portable and doesn't provide IO functions. To read s-expressions from files or other external sources, you should use parsexp_io. " parsexp-0.17.0/prefix/000077500000000000000000000000001461647336100145715ustar00rootroot00000000000000parsexp-0.17.0/prefix/src/000077500000000000000000000000001461647336100153605ustar00rootroot00000000000000parsexp-0.17.0/prefix/src/atom_prefix.ml000066400000000000000000000046661461647336100202430ustar00rootroot00000000000000open! Ppx_compare_lib.Builtin open! Import include Atom_prefix_intf type t = { signified : Signified.t ; signifier_begin_offset : int ; signifier_end_offset : int } [@@deriving compare, sexp_of] let offset_of_start_of_current_quoted_atom position_builder = match Positions.Builder.contents position_builder |> Positions.to_list |> List.rev with | [] -> failwith "BUG: No positions saved even though we have begun a quoted atom." | { offset; line = _; col = _ } :: _ -> offset ;; let incomplete prefix_of_prefix : Signified.t = Incomplete { prefix_of_prefix } let complete prefix : Signified.t = Complete { prefix } let create_unquoted (state : (_, _) Automaton.t) make_signified = let prefix = Buffer.contents state.atom_buffer in { signified = make_signified prefix ; signifier_begin_offset = state.offset - String.length prefix ; signifier_end_offset = state.offset } ;; let create_quoted (state : (_, _) Automaton.t) make_signified = let prefix = Buffer.contents state.atom_buffer in { signified = make_signified prefix ; signifier_begin_offset = offset_of_start_of_current_quoted_atom state.user_state ; signifier_end_offset = state.offset } ;; let create (state : (_, _) Automaton.t) : t Create_result.t = match Automaton.context state with | Sexp_comment -> Awkward_position | Sexp -> (match Parsexp_symbolic_automaton.Automaton.State.of_int state.automaton_state with | Whitespace -> Whitespace | Error | After_cr | Line_comment | After_hash | Block_comment _ -> Awkward_position | Unquoted_string _ -> Some (create_unquoted state complete) | Quoted_string (Normal | Ignoring_blanks) -> Some (create_quoted state complete) | Quoted_string ( After_backslash | After_backslash_cr | After_backslash_digit | After_backslash_2digits | After_backslash_x | After_backslash_x_hex ) -> Some (create_quoted state incomplete)) ;; let create_opt state = match create state with | Some t -> Some t | Awkward_position | Whitespace -> None ;; let get_signified t = t.signified let get_signifier_length t = t.signifier_end_offset - t.signifier_begin_offset let get_signifier t ~parser_input = String.sub parser_input t.signifier_begin_offset (get_signifier_length t) ;; module Unstable = struct type nonrec t = t = { signified : Signified.Unstable.t ; signifier_begin_offset : int ; signifier_end_offset : int } [@@deriving sexp] end parsexp-0.17.0/prefix/src/atom_prefix.mli000066400000000000000000000000451461647336100203770ustar00rootroot00000000000000include Atom_prefix_intf.Atom_prefix parsexp-0.17.0/prefix/src/atom_prefix_intf.ml000066400000000000000000000051501461647336100212500ustar00rootroot00000000000000(** Preview incomplete atoms when the parser is in one This interface borrows the jargon of semiotics to distinguish between 1. the signifier: the string we write when serializing a sexp 2. the signified: the atom we mean to exist in the parsed OCaml value *) open! Ppx_compare_lib.Builtin open! Import module Signified = struct type t = | Complete of { prefix : string } (** [Complete] means [prefix] is all the signified from the start of the atom to the end of the input seen so far. *) | Incomplete of { prefix_of_prefix : string } (** [Incomplete] means the parser is uncertain how to handle some suffix of the input it has seen so far. E.g. midway through a backslash-escape sequence *) [@@deriving compare, sexp_of] module Unstable = struct type nonrec t = t = | Complete of { prefix : string } | Incomplete of { prefix_of_prefix : string } [@@deriving sexp] end end module Create_result = struct type 't t = | Awkward_position | Whitespace | Some of 't end module type Atom_prefix = sig module Signified = Signified type t [@@deriving compare, sexp_of] (** [create state] inspects the parser state and returns [Some t] if the parser is known to be in a (non-comment) atom. [create] returns [Awkward_position] in the middle of comments (including sexp comments) or when the parser is uncertain whether or not it is in an atom. For example, [#] not in double-quotes can be either an atom or a block comment. The parser must consume the next character to determine which. [Whitespace] state is reported separately because the caller may choose to interpret it as an atom prefix of length 0: we know that it would be safe to start an atom here. *) val create : (Positions.Builder.t, _) Automaton.t -> t Create_result.t val create_opt : (Positions.Builder.t, _) Automaton.t -> t option val get_signified : t -> Signified.t (** [get_signifier t ~parser_input] returns the substring of [parser_input] this [t] corresponds to, starting at the beginning of the partial atom, ending at the position the parser was at when [create] was called. [parser_input] should contain the entire document being parsed (or at least the prefix that was fed to the parser so far). [get_signifier] does not necessarily return a valid sexp: consider when the parser is partway through a quoted string. *) val get_signifier : t -> parser_input:string -> string val get_signifier_length : t -> int module Unstable : sig type nonrec t = t [@@deriving sexp] end end parsexp-0.17.0/prefix/src/dependencies.dot000066400000000000000000000002601461647336100205140ustar00rootroot00000000000000digraph { sexp_prefix -> parsexp_prefix; atom_prefix -> sexp_prefix_intf; sexp_prefix_intf -> sexp_prefix; import -> atom_prefix_intf; atom_prefix_intf -> atom_prefix; } parsexp-0.17.0/prefix/src/dune000066400000000000000000000003651461647336100162420ustar00rootroot00000000000000(library (name parsexp_prefix) (libraries parsexp parsexp_symbolic_automaton ppx_sexp_conv.runtime-lib sexplib0) (preprocess (pps ppx_js_style -check-doc-comments ppx_sexp_conv ppx_compare ppx_enumerate ppx_hash ppx_variants_conv))) parsexp-0.17.0/prefix/src/import.ml000066400000000000000000000002161461647336100172230ustar00rootroot00000000000000include Sexplib0.Sexp_conv module Automaton = Parsexp.Private.Automaton module List = ListLabels module Positions = Parsexp.Private.Positions parsexp-0.17.0/prefix/src/parsexp_prefix.ml000066400000000000000000000004661461647336100207570ustar00rootroot00000000000000(** Parses prefixes of valid sexps, for implementing tab completion. This library is separate from [Parsexp] because it allows itself heavier dependencies. It is not released publicly, although we may reconsider after it matures. *) module Atom_prefix = Atom_prefix module Sexp_prefix = Sexp_prefix parsexp-0.17.0/prefix/src/sexp_prefix.ml000066400000000000000000000026731461647336100202560ustar00rootroot00000000000000open! Import open! Ppx_sexp_conv_lib include Sexp_prefix_intf type t = Sexp.t list * in_sexp and in_sexp = | Hole of Atom_prefix.t option | In_list of t [@@deriving sexp_of] let create (state : (_, _) Automaton.t) stack = let rec of_stack (stack : Automaton.Stack.t) acc = match stack with | Empty -> acc | Open stack -> of_stack stack ([], In_list acc) | Sexp (sexp, stack) -> let sexps, partial = acc in of_stack stack (sexp :: sexps, partial) in match state.mode with | Eager _ | Single -> failwith "The automaton must be in [Many] mode." | Many -> (match Atom_prefix.create state with | Awkward_position -> None | Whitespace -> Some (of_stack stack ([], Hole None)) | Some atom -> Some (of_stack stack ([], Hole (Some atom)))) ;; let of_substring s ~pos ~len = let state, stack = Automaton.of_substring Many Sexp_with_positions s ~pos ~len in create state stack ;; let get_a_signifier (t : t) ~parser_input = let buf = Buffer.create 128 in let rec add_in_sexp = function | Hole None -> () | Hole (Some atom_prefix) -> Buffer.add_string buf (Atom_prefix.get_signifier atom_prefix ~parser_input) | In_list t -> Buffer.add_string buf "("; add_t t and add_t (sexps, in_sexp) = List.iter sexps ~f:(fun sexp -> Buffer.add_string buf (Sexp.to_string sexp); Buffer.add_char buf ' '); add_in_sexp in_sexp in add_t t; Buffer.contents buf ;; parsexp-0.17.0/prefix/src/sexp_prefix.mli000066400000000000000000000000451461647336100204160ustar00rootroot00000000000000include Sexp_prefix_intf.Sexp_prefix parsexp-0.17.0/prefix/src/sexp_prefix_intf.ml000066400000000000000000000031421461647336100212660ustar00rootroot00000000000000(** Prefixes of valid sexps, terminating in a hole where additional input is expected to complete the sexp. *) open Import open Ppx_sexp_conv_lib module type Sexp_prefix = sig (** The outermost constructor represents the outermost sexp. (This is the opposite of [Automaton.Stack.t], where the outermost constructor is the innermost sexp.) [Hole] also encodes the information about the incomplete atom, which is otherwise stored in [State.t]. *) type t = Sexp.t list * in_sexp and in_sexp = | Hole of Atom_prefix.t option | In_list of t [@@deriving sexp_of] (** [create state] returns [Some t] if the parser is known to be in a sexp. "In a sexp" does not include comments, even comments nested inside sexps. [create] returns [None] when the parser is uncertain whether or not it is in sexp. For example, [#] not in double-quotes can be either an atom or a block comment. The parser must consume the next character to determine which. The automaton must be in [Many] mode. *) val create : (Positions.Builder.t, Automaton.Stack.t) Automaton.t -> Automaton.Stack.t -> t option val of_substring : string -> pos:int -> len:int -> t option (** [get_a_signifier t ~parser_input] returns a string which, when prepended to the unconsumed parser input, will parse to the same sexp(s) as the original parser input. Unlike [Atom_prefix.get_signifier], [get_a_signifier] does not promise to return the original signifier of this prefix in the parser input. *) val get_a_signifier : t -> parser_input:string -> string end parsexp-0.17.0/prefix/test/000077500000000000000000000000001461647336100155505ustar00rootroot00000000000000parsexp-0.17.0/prefix/test/coverage.ml000066400000000000000000000052571461647336100177060ustar00rootroot00000000000000open! Core open! Import include Coverage_intf module type Seen = sig type t include Intable.S with type t := t (** [of_int_exn t < cardinality] for all [t]. *) val cardinality : int end module Seen : sig type 'a t val create : (module Seen with type t = 'a) -> 'a t val add : 'a t -> 'a -> unit val get_unseen : 'a t -> 'a list end = struct type 'a t = { of_int_exn : int -> 'a ; seen : bool array ; to_int_exn : 'a -> int } let create (type a) (module M : Seen with type t = a) = { of_int_exn = M.of_int_exn ; seen = Array.create false ~len:M.cardinality ; to_int_exn = M.to_int_exn } ;; let add t a = t.seen.(t.to_int_exn a) <- true let get_unseen t = Array.filter_mapi t.seen ~f:(fun i was_seen -> match was_seen with | true -> None | false -> Some (t.of_int_exn i)) |> Array.to_list ;; end let with_prefix_coverage full_string ~f = let module Prefixes_seen = struct type t = string let cardinality = String.length full_string + 1 (* "" is a prefix too *) let of_int_exn len = String.subo full_string ~len let to_int_exn = String.length end in let seen = Seen.create (module Prefixes_seen) in let saw_prefix ~prefix = match String.is_prefix full_string ~prefix with | false -> raise_s [%message "Not a prefix." prefix (full_string : string)] | true -> Seen.add seen prefix in let get_unseen () = List.filter (Seen.get_unseen seen) ~f:(function | "" -> (* For unquoted atoms, we never emit the empty prefix because we only know the atom has begun when we see the first character. *) false | unseen -> (match unseen.[String.length unseen - 1] with (* Some prefixes we don't emit because we don't know how to decode them until we see the next character. *) | '#' | '|' -> false (* block comments *) | _ -> true)) in f ({ f = saw_prefix } : Saw_prefix.t); match get_unseen () with | [] -> () | _ :: _ as unseen_prefixes -> raise_s [%message (unseen_prefixes : string list) (full_string : string)] ;; let with_state_coverage ~f = let module State = Parsexp_symbolic_automaton.Automaton.State in let module States_seen = struct type t = int let cardinality = List.length [%all: State.t] let of_int_exn = Fn.id let to_int_exn = Fn.id end in let seen = Seen.create (module States_seen) in f ({ f = (fun state -> Seen.add seen state.automaton_state) } : Saw_state.t); match Seen.get_unseen seen with | [] -> () | _ :: _ as unseen_states -> let unseen_states = List.map unseen_states ~f:State.of_int in raise_s [%message (unseen_states : State.t list)] ;; parsexp-0.17.0/prefix/test/coverage.mli000066400000000000000000000000371461647336100200460ustar00rootroot00000000000000include Coverage_intf.Coverage parsexp-0.17.0/prefix/test/coverage_intf.ml000066400000000000000000000006341461647336100207200ustar00rootroot00000000000000open! Core open! Import module Saw_prefix = struct type t = { f : prefix:string -> unit } end module Saw_state = struct type t = { f : 'u 's. ('u, 's) Automaton.t -> unit } end module type Coverage = sig module Saw_prefix = Saw_prefix module Saw_state = Saw_state val with_prefix_coverage : string -> f:(Saw_prefix.t -> unit) -> unit val with_state_coverage : f:(Saw_state.t -> unit) -> unit end parsexp-0.17.0/prefix/test/dune000066400000000000000000000003451461647336100164300ustar00rootroot00000000000000(library (name parsexp_prefix_test) (libraries core_kernel.composition_infix core expect_test_helpers_core parsexp parsexp_prefix parsexp_symbolic_automaton sexp_string_quickcheck tilde_f) (preprocess (pps ppx_jane))) parsexp-0.17.0/prefix/test/import.ml000066400000000000000000000012671461647336100174220ustar00rootroot00000000000000open! Core include Composition_infix include Expect_test_helpers_core include Parsexp_prefix module Atom_string = Sexp_string_quickcheck.Atom_string module Automaton = Parsexp.Private.Automaton module Positions = Parsexp.Private.Positions module Sexp_string = Sexp_string_quickcheck.Sexp_string let state_after_every_prefix mode s = let len = String.length s in let state, stack = Automaton.of_substring mode Sexp_with_positions s ~pos:0 ~len in ignore (Automaton.feed_eoi state stack : Automaton.Stack.t); List.init (len + 1) ~f:(fun len -> let state, stack = Automaton.of_substring mode Sexp_with_positions s ~pos:0 ~len in len, state, stack) @ [ len + 1, state, stack ] ;; parsexp-0.17.0/prefix/test/test_atom_prefix.ml000066400000000000000000000075511461647336100214660ustar00rootroot00000000000000open! Core open! Import (** High-level testing strategy: We generate parser inputs that parse to single sexp atoms. For each parser input, we check every prefix. For every prefix s.t. [Atom_prefix.create] returns [Some atom_prefix], we check that 1. The parser input parses to the same atom if we replace the prefix with [Atom_prefix.get_signifier atom_prefix] 2. [Atom_prefix.get_signifier] does not include any comments. Across all the prefix, we check that [Atom_prefix.get_signified] has returned every prefix of the atom. Across all parser inputs, we check that we have covered every [state.automaton_state]. *) let get_signified signifier = match Parsexp.Single.parse_string_exn signifier with | List _ -> assert false | Atom signified -> signified ;; let eager_cst_consume_string s on_parsed_value = let state = Parsexp.Eager_cst.State.create on_parsed_value in let stack = Parsexp.Eager_cst.Stack.empty in let stack = Parsexp.Eager_cst.feed_string state s stack in ignore (stack : Parsexp.Eager_cst.Stack.t) ;; let parse_a_comment_eagerly signifier = Option.try_with_join (fun () -> with_return_option (fun { return } -> eager_cst_consume_string signifier (fun _ -> function | Sexp _ -> () | Comment comment -> return comment))) ;; let assert_signifier_has_no_comments of_atom ~len ~parser_input = let prefix_signifier = Atom_prefix.get_signifier of_atom ~parser_input in Option.iter (parse_a_comment_eagerly prefix_signifier) ~f:(fun comment -> raise_s [%message "Signifier included a comment." (comment : Parsexp.Cst.comment) (prefix_signifier : string) (len : int) (parser_input : string)]) ;; let round_trip_prefix of_atom ~len ~parser_input ~verbose = let prefix_signifier = Atom_prefix.get_signifier of_atom ~parser_input in let suffix = String.drop_prefix parser_input len in if verbose then print_s [%message (prefix_signifier : string) (of_atom : Atom_prefix.t)]; prefix_signifier ^ suffix ;; let show_state state len = let of_atom = Atom_prefix.create_opt state in let positions = Positions.Builder.contents state.user_state |> Positions.to_list in let state = Parsexp_symbolic_automaton.Automaton.State.of_int state.automaton_state in [%sexp { len : int ; of_atom : Atom_prefix.t option ; state : Parsexp_symbolic_automaton.Automaton.State.t ; positions : Positions.pos list }] ;; let assert_prefix_can_round_trip of_atom ~len ~parser_input ~verbose = let observed = get_signified (round_trip_prefix of_atom ~len ~parser_input ~verbose) in let expected = get_signified parser_input in require_equal [%here] (module String) observed expected ;; let test_every_prefix (saw_state : Coverage.Saw_state.t) ~parser_input ~verbose = let%bind.Tilde_f saw_prefix = Coverage.with_prefix_coverage (get_signified parser_input) in let%bind.Tilde_f len, state, _ = Tilde_f.of_local (List.iter (state_after_every_prefix Single parser_input)) in saw_state.f state; if verbose then print_s (show_state state len); let%bind.Tilde_f of_atom = Tilde_f.of_local (Option.iter (Atom_prefix.create_opt state)) in match Atom_prefix.get_signified of_atom with | Incomplete { prefix_of_prefix = prefix } | Complete { prefix } -> saw_prefix.f ~prefix; assert_signifier_has_no_comments of_atom ~len ~parser_input; assert_prefix_can_round_trip of_atom ~len ~parser_input ~verbose; Tilde_f.return () ;; let%expect_test _ = Tilde_f.run (let%bind.Tilde_f () = Tilde_f.of_unlabeled (fun f -> require_does_not_raise [%here] (fun () -> f ())) in let%bind.Tilde_f state_coverage = Coverage.with_state_coverage in let%bind.Tilde_f parser_input = Base_quickcheck.Test.run_exn (module Atom_string) in test_every_prefix state_coverage ~parser_input ~verbose:false); [%expect {| |}] ;; parsexp-0.17.0/prefix/test/test_atom_prefix.mli000066400000000000000000000000551461647336100216270ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/prefix/test/test_sexp_prefix_examples.ml000066400000000000000000000163011461647336100233740ustar00rootroot00000000000000open! Core open! Import let mark_prefix s ~len = let before = String.sub s ~pos:0 ~len in let after = String.sub s ~pos:len ~len:(String.length s - len) in [%string "%{before}│%{after}"] ;; let show mode s ~len = let state, stack = Automaton.of_substring mode Sexp_with_positions s ~pos:0 ~len in let sexp_prefix = Sexp_prefix.create state stack in print_s [%sexp (sexp_prefix : Sexp_prefix.t option)] ;; let test cr mode s = for len = 0 to String.length s do print_endline (mark_prefix s ~len); require_does_not_raise [%here] ~cr ~hide_positions:true (fun () -> show mode s ~len); print_endline "" done ;; let%expect_test "[Sexp_prefix.create] with [Many_and_positions]" = let state, stack = Parsexp.Many_and_positions.(State.create (), Stack.empty) in let stack = Parsexp.Many_and_positions.feed_string state "abc" stack in let sexp_prefix = Sexp_prefix.create state stack in print_s [%sexp (sexp_prefix : Sexp_prefix.t option)]; [%expect {| (( () (Hole (( (signified (Complete (prefix abc))) (signifier_begin_offset 0) (signifier_end_offset 3)))))) |}] ;; let%expect_test "[Many]" = test CR Many {|a #;() ("b\r\n\x61 c") #| ; |# d|}; [%expect {| │a #;() ("b\r\n\x61 c") #| ; |# d ((() (Hole ()))) a│ #;() ("b\r\n\x61 c") #| ; |# d (( () (Hole (( (signified (Complete (prefix a))) (signifier_begin_offset 0) (signifier_end_offset 1)))))) a │#;() ("b\r\n\x61 c") #| ; |# d (((a) (Hole ()))) a #│;() ("b\r\n\x61 c") #| ; |# d () a #;│() ("b\r\n\x61 c") #| ; |# d () a #;(│) ("b\r\n\x61 c") #| ; |# d () a #;()│ ("b\r\n\x61 c") #| ; |# d (((a) (Hole ()))) a #;() │("b\r\n\x61 c") #| ; |# d (((a) (Hole ()))) a #;() (│"b\r\n\x61 c") #| ; |# d (((a) (In_list (() (Hole ()))))) a #;() ("│b\r\n\x61 c") #| ; |# d (( (a) (In_list ( () (Hole (( (signified (Complete (prefix ""))) (signifier_begin_offset 8) (signifier_end_offset 9)))))))) a #;() ("b│\r\n\x61 c") #| ; |# d (( (a) (In_list ( () (Hole (( (signified (Complete (prefix b))) (signifier_begin_offset 8) (signifier_end_offset 10)))))))) a #;() ("b\│r\n\x61 c") #| ; |# d (( (a) (In_list ( () (Hole (( (signified (Incomplete (prefix_of_prefix b))) (signifier_begin_offset 8) (signifier_end_offset 11)))))))) a #;() ("b\r│\n\x61 c") #| ; |# d (( (a) (In_list ( () (Hole (( (signified (Complete (prefix "b\r"))) (signifier_begin_offset 8) (signifier_end_offset 12)))))))) a #;() ("b\r\│n\x61 c") #| ; |# d (( (a) (In_list ( () (Hole (( (signified (Incomplete (prefix_of_prefix "b\r"))) (signifier_begin_offset 8) (signifier_end_offset 13)))))))) a #;() ("b\r\n│\x61 c") #| ; |# d (( (a) (In_list ( () (Hole (( (signified (Complete (prefix "b\r\n"))) (signifier_begin_offset 8) (signifier_end_offset 14)))))))) a #;() ("b\r\n\│x61 c") #| ; |# d (( (a) (In_list ( () (Hole (( (signified (Incomplete (prefix_of_prefix "b\r\n"))) (signifier_begin_offset 8) (signifier_end_offset 15)))))))) a #;() ("b\r\n\x│61 c") #| ; |# d (( (a) (In_list ( () (Hole (( (signified (Incomplete (prefix_of_prefix "b\r\n"))) (signifier_begin_offset 8) (signifier_end_offset 16)))))))) a #;() ("b\r\n\x6│1 c") #| ; |# d (( (a) (In_list ( () (Hole (( (signified (Incomplete (prefix_of_prefix "b\r\n"))) (signifier_begin_offset 8) (signifier_end_offset 17)))))))) a #;() ("b\r\n\x61│ c") #| ; |# d (( (a) (In_list ( () (Hole (( (signified (Complete (prefix "b\r\na"))) (signifier_begin_offset 8) (signifier_end_offset 18)))))))) a #;() ("b\r\n\x61 │c") #| ; |# d (( (a) (In_list ( () (Hole (( (signified (Complete (prefix "b\r\na "))) (signifier_begin_offset 8) (signifier_end_offset 19)))))))) a #;() ("b\r\n\x61 c│") #| ; |# d (( (a) (In_list ( () (Hole (( (signified (Complete (prefix "b\r\na c"))) (signifier_begin_offset 8) (signifier_end_offset 20)))))))) a #;() ("b\r\n\x61 c"│) #| ; |# d (((a) (In_list (("b\r\na c") (Hole ()))))) a #;() ("b\r\n\x61 c")│ #| ; |# d (((a ("b\r\na c")) (Hole ()))) a #;() ("b\r\n\x61 c") │#| ; |# d (((a ("b\r\na c")) (Hole ()))) a #;() ("b\r\n\x61 c") #│| ; |# d () a #;() ("b\r\n\x61 c") #|│ ; |# d () a #;() ("b\r\n\x61 c") #| │; |# d () a #;() ("b\r\n\x61 c") #| ;│ |# d () a #;() ("b\r\n\x61 c") #| ; │|# d () a #;() ("b\r\n\x61 c") #| ; |│# d () a #;() ("b\r\n\x61 c") #| ; |#│ d (((a ("b\r\na c")) (Hole ()))) a #;() ("b\r\n\x61 c") #| ; |# │d (((a ("b\r\na c")) (Hole ()))) a #;() ("b\r\n\x61 c") #| ; |# d│ (( (a ("b\r\na c")) (Hole (( (signified (Complete (prefix d))) (signifier_begin_offset 31) (signifier_end_offset 32)))))) |}] ;; let%expect_test "deep nesting" = test CR Many "((((a))))"; [%expect {| │((((a)))) ((() (Hole ()))) (│(((a)))) ((() (In_list (() (Hole ()))))) ((│((a)))) ((() (In_list (() (In_list (() (Hole ()))))))) (((│(a)))) ((() (In_list (() (In_list (() (In_list (() (Hole ()))))))))) ((((│a)))) ((() (In_list (() (In_list (() (In_list (() (In_list (() (Hole ()))))))))))) ((((a│)))) (( () (In_list ( () (In_list ( () (In_list ( () (In_list ( () (Hole (( (signified (Complete (prefix a))) (signifier_begin_offset 4) (signifier_end_offset 5)))))))))))))) ((((a)│))) ((() (In_list (() (In_list (() (In_list (((a)) (Hole ()))))))))) ((((a))│)) ((() (In_list (() (In_list ((((a))) (Hole ()))))))) ((((a)))│) ((() (In_list (((((a)))) (Hole ()))))) ((((a))))│ (((((((a))))) (Hole ()))) |}] ;; let%expect_test "only [Many] mode" = test Comment Single ""; [%expect {| │ (* require-failed: lib/parsexp/prefix/test/test_sexp_prefix_examples.ml:LINE:COL. *) ("unexpectedly raised" (Failure "The automaton must be in [Many] mode.")) |}]; test Comment (Eager { got_sexp = (fun _ stack -> stack) ; reraise_notrace = false ; no_sexp_is_error = false }) ""; [%expect {| │ (* require-failed: lib/parsexp/prefix/test/test_sexp_prefix_examples.ml:LINE:COL. *) ("unexpectedly raised" (Failure "The automaton must be in [Many] mode.")) |}]; ignore () ;; parsexp-0.17.0/prefix/test/test_sexp_prefix_examples.mli000066400000000000000000000000551461647336100235440ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/prefix/test/test_sexp_prefix_quickcheck.ml000066400000000000000000000025041461647336100236700ustar00rootroot00000000000000open! Core open! Import let round_trip_prefix prefix ~len ~parser_input ~verbose = let prefix_signifier = Sexp_prefix.get_a_signifier prefix ~parser_input in let suffix = String.drop_prefix parser_input len in if verbose then print_s [%message (prefix : Sexp_prefix.t) (prefix_signifier : string) (suffix : string)]; prefix_signifier ^ suffix ;; let test_prefix state stack ~len ~parser_input ~verbose = Option.iter (Sexp_prefix.create state stack) ~f:(fun prefix -> let observed = round_trip_prefix prefix ~len ~parser_input ~verbose in if not ([%compare.equal: Sexp_string.Compare_sexps.t] parser_input observed) then raise_s [%message (len : int) (observed : string) (parser_input : string)]) ;; let test_input (saw_state : Coverage.Saw_state.t) parser_input = List.iter (state_after_every_prefix Many parser_input) ~f:(fun (len, state, stack) -> saw_state.f state; test_prefix state stack ~len ~parser_input ~verbose:false) ;; (* Check that a value is unchanged after we round-trip some prefix of its string representation through [Prefix]. *) let%expect_test (_ [@tags "64-bits-only"]) = require_does_not_raise [%here] (fun () -> Coverage.with_state_coverage ~f:(fun saw_state -> Base_quickcheck.Test.run_exn (module Sexp_string) ~f:(test_input saw_state))); [%expect {| |}] ;; parsexp-0.17.0/prefix/test/test_sexp_prefix_quickcheck.mli000066400000000000000000000000551461647336100240400ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/src/000077500000000000000000000000001461647336100140635ustar00rootroot00000000000000parsexp-0.17.0/src/automaton.ml000066400000000000000000000044771461647336100164400ustar00rootroot00000000000000open! Import module Stack = Automaton_stack include Automaton_state let feed (type u s) (state : (u, s) Automaton_state.t) char (stack : s) : s = let idx = (automaton_state state lsl 8) lor Char.code char in Automaton_tables.transitions.(idx).f state char stack [@@inline always] ;; let feed_eoi (type u s) (state : (u, s) Automaton_state.t) (stack : s) : s = let stack = Automaton_tables.transitions_eoi.(automaton_state state).f state stack in set_error_state state; stack ;; let old_parser_cont_state state : Old_parser_cont_state.t = match context state with | Sexp_comment -> Parsing_sexp_comment | Sexp -> (match ( Automaton_tables.old_parser_approx_cont_states.(automaton_state state) , has_unclosed_paren state ) with | Parsing_toplevel_whitespace, true -> Parsing_list | s, _ -> s) ;; let rec feed_substring_unsafe str state stack i stop = if i < stop then ( let c = String.unsafe_get str i in let stack = feed state c stack in feed_substring_unsafe str state stack (i + 1) stop) else stack ;; let rec feed_subbytes_unsafe str state stack i stop = if i < stop then ( let c = Bytes.unsafe_get str i in let stack = feed state c stack in feed_subbytes_unsafe str state stack (i + 1) stop) else stack ;; let feed_substring state str ~pos ~len stack = let str_len = String.length str in if pos < 0 || len < 0 || pos > str_len - len then invalid_arg "Parsexp.feed_substring"; feed_substring_unsafe str state stack pos (pos + len) ;; let feed_subbytes state str ~pos ~len stack = let str_len = Bytes.length str in if pos < 0 || len < 0 || pos > str_len - len then invalid_arg "Parsexp.feed_subbytes"; feed_subbytes_unsafe str state stack pos (pos + len) ;; let feed_string state str stack = feed_substring_unsafe str state stack 0 (String.length str) ;; let feed_bytes state str stack = feed_subbytes_unsafe str state stack 0 (Bytes.length str) let empty_stack : type u s. (u, s) Kind.t -> s = function | Sexp -> Stack.empty | Sexp_with_positions -> Stack.empty | Positions -> Stack.Just_positions.empty | Cst -> Stack.For_cst.empty ;; let of_substring (type u s) (mode : (u, s) Mode.t) (kind : (u, s) Kind.t) s ~pos ~len = let state = create mode kind in let stack = feed_substring state s ~pos ~len (empty_stack kind) in state, stack ;; parsexp-0.17.0/src/automaton.mli000066400000000000000000000000411461647336100165700ustar00rootroot00000000000000include Automaton_intf.Automaton parsexp-0.17.0/src/automaton_action.ml000066400000000000000000000362721461647336100177730ustar00rootroot00000000000000open! Import open Automaton_state open State let raise_error : type a b. (a, b) Automaton_state.t -> _ = fun state ~at_eof reason -> set_error_state state; Parse_error.Private.raise reason { line = state.line_number ; col = state.offset - state.bol_offset ; offset = state.offset } ~at_eof ~atom_buffer:state.atom_buffer ;; type nonrec context = Automaton_state.Context.t = | Sexp_comment | Sexp let context = Automaton_state.context type ('u, 's) t = ('u, 's) Automaton_state.t -> char -> 's -> 's module Poly = struct type nonrec t = { f : 'u 's. ('u, 's) t } [@@unboxed] end module Epsilon = struct type ('u, 's) t = ('u, 's) Automaton_state.t -> 's -> 's module Poly = struct type nonrec t = { f : 'u 's. ('u, 's) t } [@@unboxed] end end let current_pos ?(delta = 0) state : Positions.pos = let offset = state.offset + delta in { line = state.line_number; col = offset - state.bol_offset; offset } ;; let set_automaton_state state x = state.automaton_state <- x let advance state = state.offset <- state.offset + 1 let advance_eol : type u s. (u, s) Automaton_state.t -> unit = fun state -> let newline_offset = state.offset in state.offset <- newline_offset + 1; state.bol_offset <- state.offset; state.line_number <- state.line_number + 1; match state.kind with | Positions -> Positions.Builder.add_newline state.user_state ~offset:newline_offset | Sexp_with_positions -> Positions.Builder.add_newline state.user_state ~offset:newline_offset | _ -> () ;; let block_comment_depth state = state.block_comment_depth let add_token_char : type u s. (u, s) t = fun state char stack -> match state.kind with | Cst -> Buffer.add_char state.user_state.token_buffer char; stack | _ -> stack ;; let add_atom_char state c stack = Buffer.add_char state.atom_buffer c; stack ;; let add_quoted_atom_char state c stack = Buffer.add_char state.atom_buffer c; add_token_char state c stack ;; let is_ignoring state = match context state with | Sexp -> false | Sexp_comment -> true ;; let is_not_ignoring state = not (is_ignoring state) let check_new_sexp_allowed state = let is_single = match state.mode with | Single -> true | _ -> false in if is_single && state.full_sexps > 0 && is_not_ignoring state then raise_error state ~at_eof:false Too_many_sexps ;; let add_pos state ~delta = Positions.Builder.add state.user_state ~offset:(state.offset + delta) ;; let add_first_char : type u s. (u, s) t = fun state char stack -> check_new_sexp_allowed state; Buffer.add_char state.atom_buffer char; (* For non-quoted atoms, we save both positions at the end. We can always determine the start position from the end position and the atom length for non-quoted atoms. Doing it this way allows us to detect single characater atoms for which we need to save the position twice. *) stack ;; let eps_add_first_char_hash : type u s. (u, s) Epsilon.t = fun state stack -> check_new_sexp_allowed state; Buffer.add_char state.atom_buffer '#'; stack ;; let start_quoted_string : type u s. (u, s) t = fun state _char stack -> check_new_sexp_allowed state; match state.kind with | Positions -> if is_not_ignoring state then add_pos state ~delta:0; stack | Sexp_with_positions -> if is_not_ignoring state then add_pos state ~delta:0; stack | Cst -> state.user_state.token_start_pos <- current_pos state; Buffer.add_char state.user_state.token_buffer '"'; stack | Sexp -> stack ;; let add_escaped state c stack = let c' = match c with | 'n' -> '\n' | 'r' -> '\r' | 'b' -> '\b' | 't' -> '\t' | '\\' | '\'' | '"' -> c | _ -> Buffer.add_char state.atom_buffer '\\'; c in Buffer.add_char state.atom_buffer c'; add_token_char state c stack ;; let eps_add_escaped_cr state stack = Buffer.add_char state.atom_buffer '\r'; stack ;; let dec_val c = Char.code c - Char.code '0' let hex_val c = match c with | '0' .. '9' -> Char.code c - Char.code '0' | 'a' .. 'f' -> Char.code c - Char.code 'a' + 10 | _ -> Char.code c - Char.code 'A' + 10 ;; let add_dec_escape_char state c stack = state.escaped_value <- (state.escaped_value * 10) + dec_val c; add_token_char state c stack ;; let add_last_dec_escape_char state c stack = let value = (state.escaped_value * 10) + dec_val c in state.escaped_value <- 0; if value > 255 then raise_error state ~at_eof:false Escape_sequence_out_of_range; Buffer.add_char state.atom_buffer (Char.chr value); add_token_char state c stack ;; let comment_add_last_dec_escape_char state c stack = let value = (state.escaped_value * 10) + dec_val c in state.escaped_value <- 0; if value > 255 then raise_error state ~at_eof:false Escape_sequence_out_of_range; add_token_char state c stack ;; let add_hex_escape_char state c stack = state.escaped_value <- (state.escaped_value lsl 4) lor hex_val c; add_token_char state c stack ;; let add_last_hex_escape_char state c stack = let value = (state.escaped_value lsl 4) lor hex_val c in state.escaped_value <- 0; Buffer.add_char state.atom_buffer (Char.chr value); add_token_char state c stack ;; let opening : type u s. (u, s) Automaton_state.t -> char -> s -> s = fun state _char stack -> check_new_sexp_allowed state; state.depth <- state.depth + 1; match state.kind with | Positions -> if is_not_ignoring state then add_pos state ~delta:0; stack | Sexp -> if is_not_ignoring state then Open stack else stack | Sexp_with_positions -> if is_not_ignoring state then ( add_pos state ~delta:0; Open stack) else stack | Cst -> Open (current_pos state, stack) ;; let do_reset_positions state = Positions.Builder.reset state.user_state { line = state.line_number ; col = state.offset - state.bol_offset ; offset = state.offset } ;; let reset_positions : type u s. (u, s) Automaton_state.t -> unit = fun state -> match state.kind with | Positions -> do_reset_positions state | Sexp_with_positions -> do_reset_positions state | Sexp -> () | Cst -> () ;; let toplevel_sexp_or_comment_added state stack ~delta = match state.mode with | Single | Many -> stack | Eager { got_sexp = f; reraise_notrace; _ } -> (* Modify the offset so that [f] get a state pointing to the end of the current s-expression *) let saved_offset = state.offset in state.offset <- state.offset + delta; let saved_full_sexps = state.full_sexps in (match f state stack with | exception e -> set_error_state state; if reraise_notrace then raise_notrace e else raise e | stack -> (* This assert is not a full protection against the user mutating the state but it should catch most cases. *) assert (state.offset = saved_offset + delta && state.full_sexps = saved_full_sexps); state.offset <- saved_offset; reset_positions state; stack) ;; let is_top_level state = is_not_ignoring state && state.depth = 0 let comment_added_assuming_cst state stack ~delta = if is_top_level state then toplevel_sexp_or_comment_added state stack ~delta else stack ;; let maybe_pop_ignoring_stack state = match state.ignoring_stack with | inner_comment_depth :: _tl when inner_comment_depth > state.depth -> raise_error state ~at_eof:false Sexp_comment_without_sexp | inner_comment_depth :: tl when inner_comment_depth = state.depth -> state.ignoring_stack <- tl; true | _ -> false ;; let sexp_added : type u s. (u, s) Automaton_state.t -> s -> delta:int -> s = fun state stack ~delta -> let is_comment = maybe_pop_ignoring_stack state in if is_top_level state then ( if not is_comment then state.full_sexps <- state.full_sexps + 1; if (not is_comment) || match state.kind with | Cst -> true | _ -> false then toplevel_sexp_or_comment_added state stack ~delta else stack) else stack ;; let rec make_list acc : Automaton_stack.t -> Automaton_stack.t = function | Empty -> assert false | Open stack -> (* [opaque_identity] is needed to prevent the compiler from sharing multiple copies of [List []] after inlining. Such sharing is problematic because it prevents us from computing the correct sexp locations, since we base that on physical identity. *) Sexp (List (Sys.opaque_identity acc), stack) | Sexp (sexp, stack) -> make_list (sexp :: acc) stack ;; let add_comment_to_stack_cst comment (stack : Automaton_stack.For_cst.t) : Automaton_stack.For_cst.t = match stack with | In_sexp_comment r -> In_sexp_comment { r with rev_comments = comment :: r.rev_comments } | _ -> T_or_comment (Comment comment, stack) ;; let add_sexp_to_stack_cst sexp : Automaton_stack.For_cst.t -> Automaton_stack.For_cst.t = function | In_sexp_comment { hash_semi_pos; rev_comments; stack } -> let comment : Cst.comment = Sexp_comment { hash_semi_pos; comments = List.rev rev_comments; sexp } in add_comment_to_stack_cst comment stack | stack -> T_or_comment (Sexp sexp, stack) ;; let rec make_list_cst end_pos acc : Automaton_stack.For_cst.t -> Automaton_stack.For_cst.t = function | T_or_comment (t, stack) -> make_list_cst end_pos (t :: acc) stack | Open (start_pos, stack) -> let sexp : Cst.t = List { loc = { start_pos; end_pos }; elements = acc } in add_sexp_to_stack_cst sexp stack | Empty | In_sexp_comment _ -> assert false ;; let closing : type u s. (u, s) Automaton_state.t -> char -> s -> s = fun state _char stack -> if state.depth > 0 then ( let stack : s = match state.kind with | Positions -> (* Note we store end positions as inclusive in [Positions.t], so we use [delta:0], while in the [Cst] case we save directly the final ranges, so we use [delta:1]. *) if is_not_ignoring state then add_pos state ~delta:0; stack | Sexp -> if is_not_ignoring state then make_list [] stack else stack | Sexp_with_positions -> if is_not_ignoring state then ( add_pos state ~delta:0; make_list [] stack) else stack | Cst -> make_list_cst (current_pos state ~delta:1) [] stack in state.depth <- state.depth - 1; sexp_added state stack ~delta:1) else raise_error state ~at_eof:false Closed_paren_without_opened ;; let make_loc ?(delta = 0) (state : (Automaton_state.For_cst.t, _) Automaton_state.t) : Positions.range = { start_pos = state.user_state.token_start_pos; end_pos = current_pos state ~delta } ;; (* This is always called on the position exactly following the last character of a non-quoted atom *) let add_non_quoted_atom_pos state ~atom = let len = String.length atom in if len = 1 then Positions.Builder.add_twice state.user_state ~offset:(state.offset - 1) else ( add_pos state ~delta:(-len); add_pos state ~delta:(-1)) ;; let eps_push_atom : type u s. (u, s) Epsilon.t = fun state stack -> let str = Buffer.contents state.atom_buffer in Buffer.clear state.atom_buffer; let stack : s = match state.kind with | Positions -> if is_not_ignoring state then add_non_quoted_atom_pos state ~atom:str; stack | Sexp -> if is_not_ignoring state then Sexp (Atom str, stack) else stack | Sexp_with_positions -> if is_not_ignoring state then ( add_non_quoted_atom_pos state ~atom:str; Sexp (Atom str, stack)) else stack | Cst -> let loc : Positions.range = { start_pos = current_pos state ~delta:(-String.length str) ; end_pos = current_pos state ~delta:0 } in let sexp : Cst.t = Atom { loc; atom = str; unescaped = Some str } in add_sexp_to_stack_cst sexp stack in sexp_added state stack ~delta:0 ;; let push_quoted_atom : type u s. (u, s) t = fun state _char stack -> let str = Buffer.contents state.atom_buffer in Buffer.clear state.atom_buffer; let stack : s = match state.kind with | Positions -> if is_not_ignoring state then add_pos state ~delta:0; stack | Sexp -> if is_not_ignoring state then Sexp (Atom str, stack) else stack | Sexp_with_positions -> if is_not_ignoring state then ( add_pos state ~delta:0; Sexp (Atom str, stack)) else stack | Cst -> let buf = state.user_state.token_buffer in Buffer.add_char buf '"'; let s = Buffer.contents buf in Buffer.clear buf; let sexp : Cst.t = Atom { loc = make_loc state ~delta:1; atom = str; unescaped = Some s } in add_sexp_to_stack_cst sexp stack in sexp_added state stack ~delta:1 ;; let start_sexp_comment : type u s. (u, s) t = fun state _char stack -> state.ignoring_stack <- state.depth :: state.ignoring_stack; match state.kind with | Cst -> In_sexp_comment { hash_semi_pos = current_pos state ~delta:(-1); rev_comments = []; stack } | _ -> stack ;; let start_block_comment : type u s. (u, s) Automaton_state.t -> char -> s -> s = fun state char stack -> state.block_comment_depth <- state.block_comment_depth + 1; match state.kind with | Positions -> stack | Sexp -> stack | Sexp_with_positions -> stack | Cst -> if state.block_comment_depth = 1 then ( state.user_state.token_start_pos <- current_pos state ~delta:(-1); Buffer.add_char state.user_state.token_buffer '#'); Buffer.add_char state.user_state.token_buffer char; stack ;; let end_block_comment : type u s. (u, s) Automaton_state.t -> char -> s -> s = fun state char stack -> state.block_comment_depth <- state.block_comment_depth - 1; match state.kind with | Positions -> stack | Sexp -> stack | Sexp_with_positions -> stack | Cst -> let buf = state.user_state.token_buffer in Buffer.add_char buf char; if state.block_comment_depth = 0 then ( let s = Buffer.contents buf in Buffer.clear buf; let comment : Cst.comment = Plain_comment { loc = make_loc state ~delta:1; comment = s } in let stack = add_comment_to_stack_cst comment stack in comment_added_assuming_cst state stack ~delta:1) else stack ;; let start_line_comment : type u s. (u, s) t = fun state char stack -> match state.kind with | Cst -> state.user_state.token_start_pos <- current_pos state; Buffer.add_char state.user_state.token_buffer char; stack | _ -> stack ;; let end_line_comment : type u s. (u, s) Epsilon.t = fun state stack -> match state.kind with | Positions -> stack | Sexp -> stack | Sexp_with_positions -> stack | Cst -> let buf = state.user_state.token_buffer in let s = Buffer.contents buf in Buffer.clear buf; let comment : Cst.comment = Plain_comment { loc = make_loc state; comment = s } in let stack = add_comment_to_stack_cst comment stack in comment_added_assuming_cst state stack ~delta:0 ;; let eps_eoi_check : type u s. (u, s) Epsilon.t = fun state stack -> if state.depth > 0 then raise_error state ~at_eof:true Unclosed_paren; if is_ignoring state then raise_error state ~at_eof:true Sexp_comment_without_sexp; if state.full_sexps = 0 then ( match state.mode with | Many | Eager { no_sexp_is_error = false; _ } -> () | Single | Eager { no_sexp_is_error = true; _ } -> raise_error state ~at_eof:true No_sexp_found_in_input); stack ;; parsexp-0.17.0/src/automaton_action.mli000066400000000000000000000053621461647336100201400ustar00rootroot00000000000000(** Internal bits used by the generated automaton, not part of the public API *) open! Import val raise_error : _ Automaton_state.t -> at_eof:bool -> Parse_error.Private.Reason.t -> _ type context = Automaton_state.Context.t = | Sexp_comment | Sexp val context : _ Automaton_state.t -> context val set_automaton_state : ('u, 's) Automaton_state.t -> int -> unit (** Advance the position counters. [advance_eol] is for when we read a newline character. *) val advance : ('u, 's) Automaton_state.t -> unit val advance_eol : ('u, 's) Automaton_state.t -> unit (** Number of opened #| *) val block_comment_depth : ('u, 's) Automaton_state.t -> int type ('u, 's) t = ('u, 's) Automaton_state.t -> char -> 's -> 's module Poly : sig type nonrec t = { f : 'u 's. ('u, 's) t } [@@unboxed] end module Epsilon : sig type ('u, 's) t = ('u, 's) Automaton_state.t -> 's -> 's module Poly : sig type nonrec t = { f : 'u 's. ('u, 's) t } [@@unboxed] end end (** Add a character to the atom buffer. [add_quoted_atom_char] does the same for quoted atoms *) val add_atom_char : _ t val add_quoted_atom_char : _ t (** Add a character that just follows a '\\' and the '\\' itself if necessary. *) val add_escaped : _ t (** [escaped_value <- escaped_value * 10 + (char - '0')] These functions make the assumption that [char] is between '0' and '9'. [add_dec_escape_char] also assumes the result doesn't overflow. The automaton definition must make sure this is the case. [add_last_dec_escape_char] also adds the resulting character to the atom buffer. *) val add_dec_escape_char : _ t val add_last_dec_escape_char : _ t (** Same but for quoted strings inside comments. Useful because it can fail. *) val comment_add_last_dec_escape_char : _ t (** Same as [add_dec_escape_char] but for hexadicemal escape sequences *) val add_hex_escape_char : _ t val add_last_hex_escape_char : _ t (** Ignore one more full sexp to come *) val start_sexp_comment : _ t (** Add the first char of an unquoted atom. *) val add_first_char : _ t val start_quoted_string : _ t (** Takes note of a control character in quoted atoms or the uninterpreted characters of comments, for which there is no corresponding [add_*] call (a backslash and the x in "\xff" or any character in a line comment). This does not get called for the opening ([start_quoted_string]) or closing ([push_quoted_atom]) quotes themselves. *) val add_token_char : _ t val opening : _ t val closing : _ t val push_quoted_atom : _ t val start_block_comment : _ t val end_block_comment : _ t val start_line_comment : _ t val end_line_comment : _ Epsilon.t val eps_push_atom : _ Epsilon.t val eps_add_first_char_hash : _ Epsilon.t val eps_eoi_check : _ Epsilon.t val eps_add_escaped_cr : _ Epsilon.t parsexp-0.17.0/src/automaton_intf.ml000066400000000000000000000014271461647336100174500ustar00rootroot00000000000000(** Convenience functions for feeding the automaton. *) open! Import module type Automaton = sig module Stack = Automaton_stack include module type of struct include Automaton_state end val feed_bytes : (_, 'stack) t -> bytes -> 'stack -> 'stack val feed_string : (_, 'stack) t -> string -> 'stack -> 'stack val feed_subbytes : (_, 'stack) t -> bytes -> pos:int -> len:int -> 'stack -> 'stack val feed_substring : (_, 'stack) t -> string -> pos:int -> len:int -> 'stack -> 'stack val feed : ('a, 'b) t -> char -> 'b -> 'b val feed_eoi : ('a, 'b) t -> 'b -> 'b val old_parser_cont_state : ('a, 'b) t -> Old_parser_cont_state.t val of_substring : ('u, 's) Mode.t -> ('u, 's) Kind.t -> string -> pos:int -> len:int -> ('u, 's) t * 's end parsexp-0.17.0/src/automaton_stack.ml000066400000000000000000000020211461647336100176040ustar00rootroot00000000000000open! Import include Automaton_stack_intf module For_cst = struct type t = | Empty | T_or_comment of Cst.t_or_comment * t | Open of Positions.pos * t | In_sexp_comment of { hash_semi_pos : Positions.pos ; rev_comments : Cst.comment list ; stack : t } let empty = Empty let get_many = let rec loop acc = function | Empty -> acc | T_or_comment (t, stack) -> loop (t :: acc) stack | Open _ | In_sexp_comment _ -> failwith "Automaton_stack.For_cst.get_many" in fun stack -> loop [] stack ;; end module Just_positions = struct type t = unit let empty = () end type t = | Empty | Open of t | Sexp of Sexp.t * t let empty = Empty let get_single = function | Sexp (sexp, Empty) -> sexp | _ -> failwith "Automaton_stack.get_single" ;; let get_many = let rec loop acc = function | Empty -> acc | Open _ -> failwith "Automaton_stack.get_many" | Sexp (sexp, stack) -> loop (sexp :: acc) stack in fun stack -> loop [] stack ;; parsexp-0.17.0/src/automaton_stack.mli000066400000000000000000000000551461647336100177620ustar00rootroot00000000000000include Automaton_stack_intf.Automaton_stack parsexp-0.17.0/src/automaton_stack_intf.ml000066400000000000000000000026661461647336100206430ustar00rootroot00000000000000open! Import (** Automaton stack represents a prefix of a sexp (or of a sexp sequence) while it's being parsed. We have three different types of the stack depending on how much detail about the sexp structure is being recorded (just the sexp, sexp with comments and positions (CST), or just positions. *) module type Automaton_stack = sig module For_cst : sig type t = | Empty (** at top-level *) | T_or_comment of Cst.t_or_comment * t (** after the given sexp or comment *) | Open of Positions.pos * t (** after the opening paren *) | In_sexp_comment of { hash_semi_pos : Positions.pos ; rev_comments : Cst.comment list ; stack : t } (** [In_sexp_comment] only indicates if the next s-expression is to be commented out, but if we are nested below parens below an sexp comment, the stack would look like [Open (.., In_sexp_comment ..)]. *) val empty : t (** Raises if [t] contains a partial sexp. *) val get_many : t -> Cst.t_or_comment list end (** The recorded positions are stored elsewhere *) module Just_positions : sig type t = unit val empty : t end type t = | Empty | Open of t | Sexp of Sexp.t * t val empty : t (** Raises if [t] is not exactly one complete sexp. *) val get_single : t -> Sexp.t (** Raises if [t] contains a partial sexp. *) val get_many : t -> Sexp.t list end parsexp-0.17.0/src/automaton_state.ml000066400000000000000000000045261461647336100176330ustar00rootroot00000000000000open! Import include Automaton_state_intf type ('u, 's) t = ('u, 's) State.t open State let initial_user_state : type u s. (u, s) Kind.t -> Positions.pos -> u = fun kind initial_pos -> match kind with | Positions -> Positions.Builder.create ~initial_pos () | Sexp -> () | Sexp_with_positions -> Positions.Builder.create ~initial_pos () | Cst -> (* [token_start_pos] is set to a dummy location here. It is properly set when we start to capture a token from the input *) { token_buffer = Buffer.create 128; token_start_pos = Positions.beginning_of_file } ;; (*$ Parsexp_cinaps_helpers.Gen_automaton_state.print_constants () *) let initial_state = 0 let error_state = 1 (*$*) let create ?(initial_pos = Positions.beginning_of_file) mode kind = { kind ; depth = 0 ; automaton_state = initial_state ; block_comment_depth = 0 ; ignoring_stack = [] ; escaped_value = 0 ; atom_buffer = Buffer.create 128 ; user_state = initial_user_state kind initial_pos ; mode ; full_sexps = 0 ; offset = initial_pos.offset ; line_number = initial_pos.line ; bol_offset = initial_pos.offset - initial_pos.col } ;; let mode t = t.mode let positions t = Positions.Builder.contents t.user_state let atom_buffer t = t.atom_buffer let offset state = state.offset let line state = state.line_number let column state = state.offset - state.bol_offset let position t = { Positions.col = column t; line = line t; offset = offset t } let reset_user_state : type u s. (u, s) t -> unit = fun t -> match t.kind with | Positions -> Positions.Builder.reset t.user_state (position t) | Sexp -> () | Sexp_with_positions -> Positions.Builder.reset t.user_state (position t) | Cst -> Buffer.clear t.user_state.token_buffer ;; let reset ?(pos = Positions.beginning_of_file) t = t.depth <- 0; t.automaton_state <- initial_state; t.block_comment_depth <- 0; t.ignoring_stack <- []; t.escaped_value <- 0; t.full_sexps <- 0; t.offset <- pos.offset; t.line_number <- pos.line; t.bol_offset <- pos.offset - pos.col; reset_user_state t; Buffer.clear t.atom_buffer ;; let context state : Context.t = match state.ignoring_stack with | _ :: _ -> Sexp_comment | [] -> Sexp ;; let has_unclosed_paren state = state.depth > 0 let set_error_state state = state.automaton_state <- error_state let automaton_state state = state.automaton_state parsexp-0.17.0/src/automaton_state.mli000066400000000000000000000000551461647336100177750ustar00rootroot00000000000000include Automaton_state_intf.Automaton_state parsexp-0.17.0/src/automaton_state_intf.ml000066400000000000000000000050251461647336100206460ustar00rootroot00000000000000open! Import module Context = struct type t = | Sexp_comment | Sexp end module For_cst = struct type t = { token_buffer : Buffer.t (** Starting positions of the current token **) ; mutable token_start_pos : Positions.pos } [@@deriving sexp_of] end module Kind = struct type ('u, 's) t = | Positions : (Positions.Builder.t, unit) t | Sexp : (unit, Automaton_stack.t) t | Sexp_with_positions : (Positions.Builder.t, Automaton_stack.t) t | Cst : (For_cst.t, Automaton_stack.For_cst.t) t end module rec State : sig type ('u, 's) t = { mutable automaton_state : int ; kind : ('u, 's) Kind.t ; mutable depth : int (** Number of opened #| when parsing a block comment *) ; mutable block_comment_depth : int (** Stack of ignoring depths; the current depth is pushed each time a #; comment is entered. *) ; mutable ignoring_stack : int list (** When parsing an escape sequence of the form "\\NNN" or "\\XX", this accumulates the computed number *) ; mutable escaped_value : int (** Buffer for accumulating atoms *) ; atom_buffer : Buffer.t ; user_state : 'u ; mode : ('u, 's) Mode.t ; mutable full_sexps : int ; mutable offset : int (** global offset **) ; mutable line_number : int ; mutable bol_offset : int (** offset of beginning of line **) } end = State and Mode : sig type ('u, 's) t = | Single | Many | Eager of { got_sexp : ('u, 's) State.t -> 's -> 's ; reraise_notrace : bool ; mutable no_sexp_is_error : bool } end = Mode module type Automaton_state = sig module Context = Context module For_cst = For_cst module Kind = Kind module Mode = Mode module State = State type ('u, 's) t = ('u, 's) State.t val create : ?initial_pos:Positions.pos -> ('u, 's) Mode.t -> ('u, 's) Kind.t -> ('u, 's) t val reset : ?pos:Positions.pos -> _ t -> unit val positions : (Positions.Builder.t, _) t -> Positions.t val mode : ('u, 's) t -> ('u, 's) Mode.t (** Number of characters fed to the parser *) val offset : _ t -> int (** Position in the text *) val line : _ t -> int val column : _ t -> int (** Whether there are some unclosed parentheses *) val has_unclosed_paren : ('u, 's) t -> bool val set_error_state : _ t -> unit (**/**) (*_ Only for converting errors to the old parser errors *) val atom_buffer : _ t -> Buffer.t (*_ For coverate tests *) val automaton_state : ('u, 's) t -> int val context : _ t -> Context.t end parsexp-0.17.0/src/automaton_tables.ml000066400000000000000000002707771461647336100200020ustar00rootroot00000000000000open Automaton_action let raise = Automaton_action.raise_error type u' type s' (*$ Parsexp_cinaps_helpers.Gen_automaton_tables.print_code () *) let tr_00_f state char stack = let stack = add_first_char state char stack in set_automaton_state state 3; advance state; stack ;; let tr_00 : Automaton_action.Poly.t = { f = tr_00_f } let tr_01_f state _char stack = set_automaton_state state 0; advance state; stack ;; let tr_01 : Automaton_action.Poly.t = { f = tr_01_f } let tr_02_f state _char stack = set_automaton_state state 0; advance_eol state; stack ;; let tr_02 : Automaton_action.Poly.t = { f = tr_02_f } let tr_03_f state _char stack = set_automaton_state state 2; advance state; stack ;; let tr_03 : Automaton_action.Poly.t = { f = tr_03_f } let tr_04_f state char stack = let stack = start_quoted_string state char stack in set_automaton_state state 8; advance state; stack ;; let tr_04 : Automaton_action.Poly.t = { f = tr_04_f } let tr_05_f state _char stack = set_automaton_state state 7; advance state; stack ;; let tr_05 : Automaton_action.Poly.t = { f = tr_05_f } let tr_06_f state char stack = let stack = opening state char stack in set_automaton_state state 0; advance state; stack ;; let tr_06 : Automaton_action.Poly.t = { f = tr_06_f } let tr_07_f state char stack = let stack = closing state char stack in set_automaton_state state 0; advance state; stack ;; let tr_07 : Automaton_action.Poly.t = { f = tr_07_f } let tr_08_f state char stack = let stack = start_line_comment state char stack in set_automaton_state state 6; advance state; stack ;; let tr_08 : Automaton_action.Poly.t = { f = tr_08_f } let tr_09_f state char stack = let stack = add_first_char state char stack in set_automaton_state state 5; advance state; stack ;; let tr_09 : Automaton_action.Poly.t = { f = tr_09_f } let tr_10_f _state _char _stack = raise _state ~at_eof:false Automaton_in_error_state let tr_10 : Automaton_action.Poly.t = { f = tr_10_f } let tr_11_f _state _char _stack = raise _state ~at_eof:false Unexpected_character_after_cr let tr_11 : Automaton_action.Poly.t = { f = tr_11_f } let tr_12_f state char stack = let stack = add_atom_char state char stack in set_automaton_state state 3; advance state; stack ;; let tr_12 : Automaton_action.Poly.t = { f = tr_12_f } let tr_13_f state _char stack = let stack = eps_push_atom state stack in set_automaton_state state 0; advance state; stack ;; let tr_13 : Automaton_action.Poly.t = { f = tr_13_f } let tr_14_f state _char stack = let stack = eps_push_atom state stack in set_automaton_state state 0; advance_eol state; stack ;; let tr_14 : Automaton_action.Poly.t = { f = tr_14_f } let tr_15_f state _char stack = let stack = eps_push_atom state stack in set_automaton_state state 2; advance state; stack ;; let tr_15 : Automaton_action.Poly.t = { f = tr_15_f } let tr_16_f state char stack = let stack = eps_push_atom state stack in let stack = start_quoted_string state char stack in set_automaton_state state 8; advance state; stack ;; let tr_16 : Automaton_action.Poly.t = { f = tr_16_f } let tr_17_f state char stack = let stack = add_atom_char state char stack in set_automaton_state state 4; advance state; stack ;; let tr_17 : Automaton_action.Poly.t = { f = tr_17_f } let tr_18_f state char stack = let stack = eps_push_atom state stack in let stack = opening state char stack in set_automaton_state state 0; advance state; stack ;; let tr_18 : Automaton_action.Poly.t = { f = tr_18_f } let tr_19_f state char stack = let stack = eps_push_atom state stack in let stack = closing state char stack in set_automaton_state state 0; advance state; stack ;; let tr_19 : Automaton_action.Poly.t = { f = tr_19_f } let tr_20_f state char stack = let stack = eps_push_atom state stack in let stack = start_line_comment state char stack in set_automaton_state state 6; advance state; stack ;; let tr_20 : Automaton_action.Poly.t = { f = tr_20_f } let tr_21_f state char stack = let stack = add_atom_char state char stack in set_automaton_state state 5; advance state; stack ;; let tr_21 : Automaton_action.Poly.t = { f = tr_21_f } let tr_22_f _state _char _stack = raise _state ~at_eof:false Comment_token_in_unquoted_atom ;; let tr_22 : Automaton_action.Poly.t = { f = tr_22_f } let tr_23_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 6; advance state; stack ;; let tr_23 : Automaton_action.Poly.t = { f = tr_23_f } let tr_24_f state _char stack = let stack = end_line_comment state stack in set_automaton_state state 0; advance_eol state; stack ;; let tr_24 : Automaton_action.Poly.t = { f = tr_24_f } let tr_25_f state _char stack = let stack = end_line_comment state stack in set_automaton_state state 2; advance state; stack ;; let tr_25 : Automaton_action.Poly.t = { f = tr_25_f } let tr_26_f state char stack = let stack = eps_add_first_char_hash state stack in let stack = add_atom_char state char stack in set_automaton_state state 3; advance state; stack ;; let tr_26 : Automaton_action.Poly.t = { f = tr_26_f } let tr_27_f state _char stack = let stack = eps_add_first_char_hash state stack in let stack = eps_push_atom state stack in set_automaton_state state 0; advance state; stack ;; let tr_27 : Automaton_action.Poly.t = { f = tr_27_f } let tr_28_f state _char stack = let stack = eps_add_first_char_hash state stack in let stack = eps_push_atom state stack in set_automaton_state state 0; advance_eol state; stack ;; let tr_28 : Automaton_action.Poly.t = { f = tr_28_f } let tr_29_f state _char stack = let stack = eps_add_first_char_hash state stack in let stack = eps_push_atom state stack in set_automaton_state state 2; advance state; stack ;; let tr_29 : Automaton_action.Poly.t = { f = tr_29_f } let tr_30_f state char stack = let stack = eps_add_first_char_hash state stack in let stack = eps_push_atom state stack in let stack = start_quoted_string state char stack in set_automaton_state state 8; advance state; stack ;; let tr_30 : Automaton_action.Poly.t = { f = tr_30_f } let tr_31_f state char stack = let stack = eps_add_first_char_hash state stack in let stack = add_atom_char state char stack in set_automaton_state state 4; advance state; stack ;; let tr_31 : Automaton_action.Poly.t = { f = tr_31_f } let tr_32_f state char stack = let stack = eps_add_first_char_hash state stack in let stack = eps_push_atom state stack in let stack = opening state char stack in set_automaton_state state 0; advance state; stack ;; let tr_32 : Automaton_action.Poly.t = { f = tr_32_f } let tr_33_f state char stack = let stack = eps_add_first_char_hash state stack in let stack = eps_push_atom state stack in let stack = closing state char stack in set_automaton_state state 0; advance state; stack ;; let tr_33 : Automaton_action.Poly.t = { f = tr_33_f } let tr_34_f state char stack = let stack = start_sexp_comment state char stack in set_automaton_state state 0; advance state; stack ;; let tr_34 : Automaton_action.Poly.t = { f = tr_34_f } let tr_35_f state char stack = let stack = start_block_comment state char stack in set_automaton_state state 16; advance state; stack ;; let tr_35 : Automaton_action.Poly.t = { f = tr_35_f } let tr_36_f state char stack = let stack = add_quoted_atom_char state char stack in set_automaton_state state 8; advance state; stack ;; let tr_36 : Automaton_action.Poly.t = { f = tr_36_f } let tr_37_f state char stack = let stack = add_quoted_atom_char state char stack in set_automaton_state state 8; advance_eol state; stack ;; let tr_37 : Automaton_action.Poly.t = { f = tr_37_f } let tr_38_f state char stack = let stack = push_quoted_atom state char stack in set_automaton_state state 0; advance state; stack ;; let tr_38 : Automaton_action.Poly.t = { f = tr_38_f } let tr_39_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 9; advance state; stack ;; let tr_39 : Automaton_action.Poly.t = { f = tr_39_f } let tr_40_f state char stack = let stack = add_escaped state char stack in set_automaton_state state 8; advance state; stack ;; let tr_40 : Automaton_action.Poly.t = { f = tr_40_f } let tr_41_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 15; advance_eol state; stack ;; let tr_41 : Automaton_action.Poly.t = { f = tr_41_f } let tr_42_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 10; advance state; stack ;; let tr_42 : Automaton_action.Poly.t = { f = tr_42_f } let tr_43_f state char stack = let stack = add_dec_escape_char state char stack in set_automaton_state state 11; advance state; stack ;; let tr_43 : Automaton_action.Poly.t = { f = tr_43_f } let tr_44_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 13; advance state; stack ;; let tr_44 : Automaton_action.Poly.t = { f = tr_44_f } let tr_45_f state char stack = let stack = eps_add_escaped_cr state stack in let stack = add_quoted_atom_char state char stack in set_automaton_state state 8; advance state; stack ;; let tr_45 : Automaton_action.Poly.t = { f = tr_45_f } let tr_46_f state char stack = let stack = eps_add_escaped_cr state stack in let stack = push_quoted_atom state char stack in set_automaton_state state 0; advance state; stack ;; let tr_46 : Automaton_action.Poly.t = { f = tr_46_f } let tr_47_f state char stack = let stack = eps_add_escaped_cr state stack in let stack = add_token_char state char stack in set_automaton_state state 9; advance state; stack ;; let tr_47 : Automaton_action.Poly.t = { f = tr_47_f } let tr_48_f _state _char _stack = raise _state ~at_eof:false Unexpected_char_parsing_dec_escape ;; let tr_48 : Automaton_action.Poly.t = { f = tr_48_f } let tr_49_f state char stack = let stack = add_dec_escape_char state char stack in set_automaton_state state 12; advance state; stack ;; let tr_49 : Automaton_action.Poly.t = { f = tr_49_f } let tr_50_f state char stack = let stack = add_last_dec_escape_char state char stack in set_automaton_state state 8; advance state; stack ;; let tr_50 : Automaton_action.Poly.t = { f = tr_50_f } let tr_51_f _state _char _stack = raise _state ~at_eof:false Unexpected_char_parsing_hex_escape ;; let tr_51 : Automaton_action.Poly.t = { f = tr_51_f } let tr_52_f state char stack = let stack = add_hex_escape_char state char stack in set_automaton_state state 14; advance state; stack ;; let tr_52 : Automaton_action.Poly.t = { f = tr_52_f } let tr_53_f state char stack = let stack = add_last_hex_escape_char state char stack in set_automaton_state state 8; advance state; stack ;; let tr_53 : Automaton_action.Poly.t = { f = tr_53_f } let tr_54_f state char stack = let stack = add_quoted_atom_char state char stack in set_automaton_state state 8; advance state; stack ;; let tr_54 : Automaton_action.Poly.t = { f = tr_54_f } let tr_55_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 15; advance state; stack ;; let tr_55 : Automaton_action.Poly.t = { f = tr_55_f } let tr_56_f state char stack = let stack = add_quoted_atom_char state char stack in set_automaton_state state 8; advance_eol state; stack ;; let tr_56 : Automaton_action.Poly.t = { f = tr_56_f } let tr_57_f state char stack = let stack = push_quoted_atom state char stack in set_automaton_state state 0; advance state; stack ;; let tr_57 : Automaton_action.Poly.t = { f = tr_57_f } let tr_58_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 9; advance state; stack ;; let tr_58 : Automaton_action.Poly.t = { f = tr_58_f } let tr_59_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 16; advance state; stack ;; let tr_59 : Automaton_action.Poly.t = { f = tr_59_f } let tr_60_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 16; advance_eol state; stack ;; let tr_60 : Automaton_action.Poly.t = { f = tr_60_f } let tr_61_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 19; advance state; stack ;; let tr_61 : Automaton_action.Poly.t = { f = tr_61_f } let tr_62_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 18; advance state; stack ;; let tr_62 : Automaton_action.Poly.t = { f = tr_62_f } let tr_63_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 17; advance state; stack ;; let tr_63 : Automaton_action.Poly.t = { f = tr_63_f } let tr_64_f state char stack = let stack = end_block_comment state char stack in set_automaton_state state (if block_comment_depth state <> 0 then 16 else 0); advance state; stack ;; let tr_64 : Automaton_action.Poly.t = { f = tr_64_f } let tr_65_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 19; advance_eol state; stack ;; let tr_65 : Automaton_action.Poly.t = { f = tr_65_f } let tr_66_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 20; advance state; stack ;; let tr_66 : Automaton_action.Poly.t = { f = tr_66_f } let tr_67_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 26; advance_eol state; stack ;; let tr_67 : Automaton_action.Poly.t = { f = tr_67_f } let tr_68_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 21; advance state; stack ;; let tr_68 : Automaton_action.Poly.t = { f = tr_68_f } let tr_69_f state char stack = let stack = add_dec_escape_char state char stack in set_automaton_state state 22; advance state; stack ;; let tr_69 : Automaton_action.Poly.t = { f = tr_69_f } let tr_70_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 24; advance state; stack ;; let tr_70 : Automaton_action.Poly.t = { f = tr_70_f } let tr_71_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 19; advance state; stack ;; let tr_71 : Automaton_action.Poly.t = { f = tr_71_f } let tr_72_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 16; advance state; stack ;; let tr_72 : Automaton_action.Poly.t = { f = tr_72_f } let tr_73_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 20; advance state; stack ;; let tr_73 : Automaton_action.Poly.t = { f = tr_73_f } let tr_74_f state char stack = let stack = add_dec_escape_char state char stack in set_automaton_state state 23; advance state; stack ;; let tr_74 : Automaton_action.Poly.t = { f = tr_74_f } let tr_75_f state char stack = let stack = comment_add_last_dec_escape_char state char stack in set_automaton_state state 19; advance state; stack ;; let tr_75 : Automaton_action.Poly.t = { f = tr_75_f } let tr_76_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 25; advance state; stack ;; let tr_76 : Automaton_action.Poly.t = { f = tr_76_f } let tr_77_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 26; advance state; stack ;; let tr_77 : Automaton_action.Poly.t = { f = tr_77_f } let tr_78_f state char stack = let stack = add_token_char state char stack in set_automaton_state state 19; advance_eol state; stack ;; let tr_78 : Automaton_action.Poly.t = { f = tr_78_f } let tr_eoi_00_f state stack = eps_eoi_check state stack let tr_eoi_00 : Automaton_action.Epsilon.Poly.t = { f = tr_eoi_00_f } let tr_eoi_01_f state _stack = raise state ~at_eof:true Automaton_in_error_state let tr_eoi_01 : Automaton_action.Epsilon.Poly.t = { f = tr_eoi_01_f } let tr_eoi_02_f state _stack = raise state ~at_eof:true Unexpected_character_after_cr let tr_eoi_02 : Automaton_action.Epsilon.Poly.t = { f = tr_eoi_02_f } let tr_eoi_03_f state stack = let stack = eps_push_atom state stack in eps_eoi_check state stack ;; let tr_eoi_03 : Automaton_action.Epsilon.Poly.t = { f = tr_eoi_03_f } let tr_eoi_04_f state stack = let stack = end_line_comment state stack in eps_eoi_check state stack ;; let tr_eoi_04 : Automaton_action.Epsilon.Poly.t = { f = tr_eoi_04_f } let tr_eoi_05_f state stack = let stack = eps_add_first_char_hash state stack in let stack = eps_push_atom state stack in eps_eoi_check state stack ;; let tr_eoi_05 : Automaton_action.Epsilon.Poly.t = { f = tr_eoi_05_f } let tr_eoi_06_f state _stack = raise state ~at_eof:true Unterminated_quoted_string let tr_eoi_06 : Automaton_action.Epsilon.Poly.t = { f = tr_eoi_06_f } let tr_eoi_07_f state _stack = raise state ~at_eof:true Unterminated_block_comment let tr_eoi_07 : Automaton_action.Epsilon.Poly.t = { f = tr_eoi_07_f } let transitions = [| tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_01 ; tr_02 ; tr_00 ; tr_01 ; tr_03 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_01 ; tr_00 ; tr_04 ; tr_05 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_06 ; tr_07 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_08 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_09 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_00 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_10 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_02 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_11 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_13 ; tr_14 ; tr_12 ; tr_13 ; tr_15 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_13 ; tr_12 ; tr_16 ; tr_17 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_18 ; tr_19 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_20 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_21 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_13 ; tr_14 ; tr_12 ; tr_13 ; tr_15 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_13 ; tr_12 ; tr_16 ; tr_17 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_18 ; tr_19 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_20 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_22 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_13 ; tr_14 ; tr_12 ; tr_13 ; tr_15 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_13 ; tr_12 ; tr_16 ; tr_22 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_18 ; tr_19 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_20 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_21 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_12 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_24 ; tr_23 ; tr_23 ; tr_25 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_23 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_27 ; tr_28 ; tr_26 ; tr_27 ; tr_29 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_27 ; tr_26 ; tr_30 ; tr_31 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_32 ; tr_33 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_34 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_35 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_26 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_37 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_38 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_39 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_36 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_41 ; tr_40 ; tr_40 ; tr_42 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_43 ; tr_43 ; tr_43 ; tr_43 ; tr_43 ; tr_43 ; tr_43 ; tr_43 ; tr_43 ; tr_43 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_44 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_40 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_41 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_46 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_47 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_45 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_49 ; tr_49 ; tr_49 ; tr_49 ; tr_49 ; tr_49 ; tr_49 ; tr_49 ; tr_49 ; tr_49 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_50 ; tr_50 ; tr_50 ; tr_50 ; tr_50 ; tr_50 ; tr_50 ; tr_50 ; tr_50 ; tr_50 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_52 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_53 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_55 ; tr_56 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_55 ; tr_54 ; tr_57 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_58 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_54 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_60 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_61 ; tr_62 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_63 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_60 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_61 ; tr_64 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_63 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_60 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_61 ; tr_62 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_35 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_59 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_65 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_59 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_66 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_67 ; tr_61 ; tr_61 ; tr_68 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_69 ; tr_69 ; tr_69 ; tr_69 ; tr_69 ; tr_69 ; tr_69 ; tr_69 ; tr_69 ; tr_69 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_70 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_67 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_72 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_73 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_74 ; tr_74 ; tr_74 ; tr_74 ; tr_74 ; tr_74 ; tr_74 ; tr_74 ; tr_74 ; tr_74 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_75 ; tr_75 ; tr_75 ; tr_75 ; tr_75 ; tr_75 ; tr_75 ; tr_75 ; tr_75 ; tr_75 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_48 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_76 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_61 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_51 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_77 ; tr_78 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_77 ; tr_71 ; tr_72 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_73 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 ; tr_71 |] ;; let transitions_eoi = [| tr_eoi_00 ; tr_eoi_01 ; tr_eoi_02 ; tr_eoi_03 ; tr_eoi_03 ; tr_eoi_03 ; tr_eoi_04 ; tr_eoi_05 ; tr_eoi_06 ; tr_eoi_06 ; tr_eoi_06 ; tr_eoi_06 ; tr_eoi_06 ; tr_eoi_06 ; tr_eoi_06 ; tr_eoi_06 ; tr_eoi_07 ; tr_eoi_07 ; tr_eoi_07 ; tr_eoi_07 ; tr_eoi_07 ; tr_eoi_07 ; tr_eoi_07 ; tr_eoi_07 ; tr_eoi_07 ; tr_eoi_07 ; tr_eoi_07 |] ;; let old_parser_approx_cont_states : Old_parser_cont_state.t array = [| Parsing_toplevel_whitespace ; Parsing_toplevel_whitespace ; Parsing_nested_whitespace ; Parsing_atom ; Parsing_atom ; Parsing_atom ; Parsing_toplevel_whitespace ; Parsing_atom ; Parsing_atom ; Parsing_atom ; Parsing_atom ; Parsing_atom ; Parsing_atom ; Parsing_atom ; Parsing_atom ; Parsing_atom ; Parsing_block_comment ; Parsing_block_comment ; Parsing_block_comment ; Parsing_block_comment ; Parsing_block_comment ; Parsing_block_comment ; Parsing_block_comment ; Parsing_block_comment ; Parsing_block_comment ; Parsing_block_comment ; Parsing_block_comment |] ;; (*$*) parsexp-0.17.0/src/automaton_tables.mli000066400000000000000000000003301461647336100201230ustar00rootroot00000000000000(** Parser automaton *) type u' type s' val transitions : Automaton_action.Poly.t array val transitions_eoi : Automaton_action.Epsilon.Poly.t array val old_parser_approx_cont_states : Old_parser_cont_state.t array parsexp-0.17.0/src/cinaps/000077500000000000000000000000001461647336100153405ustar00rootroot00000000000000parsexp-0.17.0/src/cinaps/dune000066400000000000000000000001731461647336100162170ustar00rootroot00000000000000(library (name parsexp_cinaps_helpers) (libraries base parsexp_symbolic_automaton stdio) (preprocess (pps ppx_jane))) parsexp-0.17.0/src/cinaps/gen_automaton_state.ml000066400000000000000000000005021461647336100217270ustar00rootroot00000000000000open! Base let print_constants () = let initial_int = Parsexp_symbolic_automaton.Automaton.State.(to_int initial) in let error_int = Parsexp_symbolic_automaton.Automaton.State.to_int Error in Stdio.print_endline [%string {| let initial_state = %{initial_int#Int} let error_state = %{error_int#Int} |}] ;; parsexp-0.17.0/src/cinaps/gen_automaton_state.mli000066400000000000000000000000431461647336100221000ustar00rootroot00000000000000val print_constants : unit -> unit parsexp-0.17.0/src/cinaps/gen_automaton_tables.ml000066400000000000000000000102371461647336100220670ustar00rootroot00000000000000(* Parsing of S-expression. The parsing is written as an automaton for which we provide different implementations of actions. *) open! Base module Automaton = Parsexp_symbolic_automaton.Automaton module Parse_error_reason = Parsexp_symbolic_automaton.Parse_error_reason module Table = Parsexp_symbolic_automaton.Table (* Sharing of transitions *) module Sharing = struct let create_assign_id () = let cache = Hashtbl.Poly.create () in ( cache , fun x -> if not (Hashtbl.mem cache x) then Hashtbl.add_exn cache ~key:x ~data:(Hashtbl.length cache) ) ;; let share (table : Table.t) = let transitions, assign_transition_id = create_assign_id () in let transitions_eoi, assign_transition_eoi_id = create_assign_id () in Array.iter table.transitions ~f:assign_transition_id; Array.iter table.transitions_eoi ~f:assign_transition_eoi_id; transitions, transitions_eoi ;; end let pr fmt = Printf.ksprintf Stdio.print_endline fmt let ordered_ids tbl = Hashtbl.fold tbl ~init:[] ~f:(fun ~key:x ~data:id acc -> (id, x) :: acc) |> List.sort ~compare:(fun (id1, _) (id2, _) -> compare id1 id2) ;; let print_named_transition (id, tr) = (match (tr : Table.Transition.t Table.Or_parse_error_reason.t) with | Error error -> pr "let tr_%02d_f _state _char _stack =" id; pr " raise _state ~at_eof:false %s" (Parse_error_reason.to_string error) | Ok { action = eps_actions, action; goto; advance } -> let eps_actions = List.filter_map ~f:Automaton.epsilon_action_to_runtime_function eps_actions in let action = Automaton.action_to_runtime_function action in pr "let tr_%02d_f state %schar stack =" id (if Option.is_none action && not ([%compare.equal: Table.Goto_state.t] goto End_block_comment) then "_" else ""); List.iter eps_actions ~f:(pr " let stack = %s state stack in"); (match action with | None -> () | Some s -> pr " let stack = %s state char stack in" s); (match goto with | State n -> pr " set_automaton_state state %d;" n | End_block_comment -> pr " let stack = end_block_comment state char stack in"; pr " set_automaton_state state (if block_comment_depth state <> 0 then %d else \ %d);" (Automaton.State.to_int (Block_comment Normal)) (Automaton.State.to_int Whitespace)); pr " %s state;" (match advance with | Advance -> "advance" | Advance_eol -> "advance_eol"); pr " stack"); pr "let tr_%02d : Automaton_action.Poly.t = { f = tr_%02d_f }" id id ;; let print_named_transition_eoi (id, tr) = (match (tr : Automaton.Epsilon_action.t list Table.Or_parse_error_reason.t) with | Error error -> pr "let tr_eoi_%02d_f state _stack =" id; pr " raise state ~at_eof:true %s" (Parse_error_reason.to_string error) | Ok eps_actions -> pr "let tr_eoi_%02d_f state stack =" id; let eps_actions = List.filter_map eps_actions ~f:Automaton.epsilon_action_to_runtime_function in List.iter eps_actions ~f:(pr " let stack = %s state stack in"); pr " eps_eoi_check state stack"); pr "let tr_eoi_%02d : Automaton_action.Epsilon.Poly.t = { f = tr_eoi_%02d_f }" id id ;; let print_table suffix tbl ids = Array.map tbl ~f:(fun tr -> Printf.sprintf "tr%s_%02d" suffix (Hashtbl.find_exn ids tr)) |> Array.to_list |> String.concat ~sep:";" |> Stdio.printf "let transitions%s = [| %s |]" suffix ;; let print_old_parser_approx_cont_states () = List.map Automaton.State.all ~f:Automaton.State.old_parser_approx_cont_state |> String.concat ~sep:";" |> Stdio.printf "let old_parser_approx_cont_states : Old_parser_cont_state.t array = [| %s |]" ;; let print_code () = let table = Parsexp_symbolic_automaton.table in let named_transitions, named_transitions_eoi = Sharing.share table in List.iter (ordered_ids named_transitions) ~f:print_named_transition; List.iter (ordered_ids named_transitions_eoi) ~f:print_named_transition_eoi; print_table "" table.transitions named_transitions; print_table "_eoi" table.transitions_eoi named_transitions_eoi; print_old_parser_approx_cont_states () ;; parsexp-0.17.0/src/cinaps/gen_automaton_tables.mli000066400000000000000000000000521461647336100222320ustar00rootroot00000000000000open! Base val print_code : unit -> unit parsexp-0.17.0/src/cinaps/gen_parse_error.ml000066400000000000000000000003461461647336100210510ustar00rootroot00000000000000open! Base let print_constructors () = List.iter [%all: Parsexp_symbolic_automaton.Parse_error_reason.t] ~f:(fun reason -> Stdio.print_endline [%string "| %{reason#Parsexp_symbolic_automaton.Parse_error_reason}"]) ;; parsexp-0.17.0/src/cinaps/gen_parse_error.mli000066400000000000000000000000461461647336100212170ustar00rootroot00000000000000val print_constructors : unit -> unit parsexp-0.17.0/src/cinaps/parsexp_cinaps_helpers.ml000066400000000000000000000003351461647336100224340ustar00rootroot00000000000000(* The naming convention is [Gen_] + the module where we use those helpers. *) module Gen_automaton_state = Gen_automaton_state module Gen_automaton_tables = Gen_automaton_tables module Gen_parse_error = Gen_parse_error parsexp-0.17.0/src/conv.ml000066400000000000000000000027431461647336100153700ustar00rootroot00000000000000open! Import include Conv_intf module Make (Mode : Mode) (Sexp_parser : Parser.S with type parsed_value = Mode.parsed_sexp) (Positions_parser : Parser.S with type parsed_value = Positions.t) = struct let reraise positions parsed_value ~sub user_exn = let location = Mode.find positions parsed_value ~sub in Of_sexp_error.raise ~user_exn ~sub_sexp:sub ~location ;; let parse_string_exn str f = let parsed_value = Sexp_parser.parse_string_exn str in match Mode.apply_f parsed_value ~f with | x -> x | exception Sexp.Of_sexp_error (exn, sub) -> let positions = Positions_parser.parse_string_exn str in reraise positions parsed_value exn ~sub ;; let parse_string str f : (_, Conv_error.t) result = match parse_string_exn str f with | x -> Ok x | exception Parse_error.Parse_error e -> Error (Parse_error e) | exception Of_sexp_error.Of_sexp_error e -> Error (Of_sexp_error e) ;; let conv_exn (parsed_value, positions) f = match Mode.apply_f parsed_value ~f with | x -> x | exception Sexp.Of_sexp_error (exn, sub) -> reraise positions parsed_value exn ~sub ;; let conv x f = match conv_exn x f with | x -> Ok x | exception Of_sexp_error.Of_sexp_error e -> Error e ;; let conv_combine result f : (_, Conv_error.t) result = match result with | Error e -> Error (Parse_error e) | Ok x -> (match conv x f with | Ok _ as r -> r | Error e -> Error (Of_sexp_error e)) ;; end parsexp-0.17.0/src/conv.mli000066400000000000000000000000461461647336100155330ustar00rootroot00000000000000include Conv_intf.Conv (** @inline *) parsexp-0.17.0/src/conv_error.ml000066400000000000000000000012551461647336100165760ustar00rootroot00000000000000open! Import type t = | Parse_error of Parse_error.t | Of_sexp_error of Of_sexp_error.t [@@deriving_inline sexp_of] let sexp_of_t = (function | Parse_error arg0__001_ -> let res0__002_ = Parse_error.sexp_of_t arg0__001_ in Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "Parse_error"; res0__002_ ] | Of_sexp_error arg0__003_ -> let res0__004_ = Of_sexp_error.sexp_of_t arg0__003_ in Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "Of_sexp_error"; res0__004_ ] : t -> Sexplib0.Sexp.t) ;; [@@@end] let report ppf ~filename t = match t with | Parse_error e -> Parse_error.report ppf ~filename e | Of_sexp_error e -> Of_sexp_error.report ppf ~filename e ;; parsexp-0.17.0/src/conv_error.mli000066400000000000000000000005251461647336100167460ustar00rootroot00000000000000open! Import type t = | Parse_error of Parse_error.t | Of_sexp_error of Of_sexp_error.t [@@deriving_inline sexp_of] include sig [@@@ocaml.warning "-32"] val sexp_of_t : t -> Sexplib0.Sexp.t end [@@ocaml.doc "@inline"] [@@@end] (** Similar to [Parse_error.report] *) val report : Format.formatter -> filename:string -> t -> unit parsexp-0.17.0/src/conv_intf.ml000066400000000000000000000031771461647336100164120ustar00rootroot00000000000000open! Import module type S = sig type 'a res type chunk_to_conv type parsed_sexp val parse_string : string -> (chunk_to_conv -> 'a) -> ('a res, Conv_error.t) result val parse_string_exn : string -> (chunk_to_conv -> 'a) -> 'a res val conv : parsed_sexp * Positions.t -> (chunk_to_conv -> 'a) -> ('a res, Of_sexp_error.t) result val conv_exn : parsed_sexp * Positions.t -> (chunk_to_conv -> 'a) -> 'a res (** Convenience function for merging parsing and conversion errors. For instance if you have a [load] function as follow: {[ val load : string -> (Sexp.t list * Positions.t, Parse_error.t) result ]} then you can create a [load_conv] function as follow: {[ let load_conv : string -> (Sexp.t -> 'a) -> ('a list, Conv_error.t) result = fun filename f -> conv_combine (load filename) f ]} *) val conv_combine : (parsed_sexp * Positions.t, Parse_error.t) result -> (chunk_to_conv -> 'a) -> ('a res, Conv_error.t) result end module type Mode = sig type parsed_sexp type 'a res type chunk_to_conv val apply_f : parsed_sexp -> f:(chunk_to_conv -> 'r) -> 'r res val find : Positions.t -> parsed_sexp -> sub:Sexp.t -> Positions.range option end module type Conv = sig module type Mode = Mode module type S = S module Make (Mode : Mode) (Sexp_parser : Parser.S with type parsed_value = Mode.parsed_sexp) (Positions_parser : Parser.S with type parsed_value = Positions.t) : S with type parsed_sexp := Mode.parsed_sexp with type chunk_to_conv := Mode.chunk_to_conv with type 'a res := 'a Mode.res end parsexp-0.17.0/src/cst.ml000066400000000000000000000117311461647336100152110ustar00rootroot00000000000000open! Import type t = | Atom of { loc : Positions.range ; atom : string ; unescaped : string option } | List of { loc : Positions.range ; elements : t_or_comment list } and t_or_comment = | Sexp of t | Comment of comment and comment = | Plain_comment of { loc : Positions.range ; comment : string } | Sexp_comment of { hash_semi_pos : Positions.pos ; comments : comment list ; sexp : t } [@@deriving_inline sexp_of] let rec sexp_of_t = (function | Atom { loc = loc__002_; atom = atom__004_; unescaped = unescaped__006_ } -> let bnds__001_ = ([] : _ Stdlib.List.t) in let bnds__001_ = let arg__007_ = sexp_of_option sexp_of_string unescaped__006_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "unescaped"; arg__007_ ] :: bnds__001_ : _ Stdlib.List.t) in let bnds__001_ = let arg__005_ = sexp_of_string atom__004_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "atom"; arg__005_ ] :: bnds__001_ : _ Stdlib.List.t) in let bnds__001_ = let arg__003_ = Positions.sexp_of_range loc__002_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "loc"; arg__003_ ] :: bnds__001_ : _ Stdlib.List.t) in Sexplib0.Sexp.List (Sexplib0.Sexp.Atom "Atom" :: bnds__001_) | List { loc = loc__009_; elements = elements__011_ } -> let bnds__008_ = ([] : _ Stdlib.List.t) in let bnds__008_ = let arg__012_ = sexp_of_list sexp_of_t_or_comment elements__011_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "elements"; arg__012_ ] :: bnds__008_ : _ Stdlib.List.t) in let bnds__008_ = let arg__010_ = Positions.sexp_of_range loc__009_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "loc"; arg__010_ ] :: bnds__008_ : _ Stdlib.List.t) in Sexplib0.Sexp.List (Sexplib0.Sexp.Atom "List" :: bnds__008_) : t -> Sexplib0.Sexp.t) and sexp_of_t_or_comment = (function | Sexp arg0__013_ -> let res0__014_ = sexp_of_t arg0__013_ in Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "Sexp"; res0__014_ ] | Comment arg0__015_ -> let res0__016_ = sexp_of_comment arg0__015_ in Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "Comment"; res0__016_ ] : t_or_comment -> Sexplib0.Sexp.t) and sexp_of_comment = (function | Plain_comment { loc = loc__018_; comment = comment__020_ } -> let bnds__017_ = ([] : _ Stdlib.List.t) in let bnds__017_ = let arg__021_ = sexp_of_string comment__020_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "comment"; arg__021_ ] :: bnds__017_ : _ Stdlib.List.t) in let bnds__017_ = let arg__019_ = Positions.sexp_of_range loc__018_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "loc"; arg__019_ ] :: bnds__017_ : _ Stdlib.List.t) in Sexplib0.Sexp.List (Sexplib0.Sexp.Atom "Plain_comment" :: bnds__017_) | Sexp_comment { hash_semi_pos = hash_semi_pos__023_ ; comments = comments__025_ ; sexp = sexp__027_ } -> let bnds__022_ = ([] : _ Stdlib.List.t) in let bnds__022_ = let arg__028_ = sexp_of_t sexp__027_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "sexp"; arg__028_ ] :: bnds__022_ : _ Stdlib.List.t) in let bnds__022_ = let arg__026_ = sexp_of_list sexp_of_comment comments__025_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "comments"; arg__026_ ] :: bnds__022_ : _ Stdlib.List.t) in let bnds__022_ = let arg__024_ = Positions.sexp_of_pos hash_semi_pos__023_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "hash_semi_pos"; arg__024_ ] :: bnds__022_ : _ Stdlib.List.t) in Sexplib0.Sexp.List (Sexplib0.Sexp.Atom "Sexp_comment" :: bnds__022_) : comment -> Sexplib0.Sexp.t) ;; [@@@end] let compare = Stdlib.compare let compare_t_or_comment = Stdlib.compare let compare_comment = Stdlib.compare module Forget = struct (* In cps to prevent non-tail recursion. The polymorphism in the signature ensures that each function returns only through the continuation. *) module Cps : sig val forget_t : t -> (Sexp.t -> 'r) -> 'r val forget_toc : t_or_comment -> (Sexp.t option -> 'r) -> 'r val forget_tocs : t_or_comment list -> (Sexp.t list -> 'r) -> 'r end = struct let rec forget_t t k = match t with | Atom { atom; _ } -> k (Sexp.Atom atom) | List { elements; _ } -> forget_tocs elements (fun xs -> k (Sexp.List xs)) and forget_tocs tocs k = match tocs with | [] -> k [] | toc :: tocs -> forget_toc toc (function | None -> forget_tocs tocs k | Some x -> forget_tocs tocs (fun xs -> k (x :: xs))) and forget_toc toc k = match toc with | Comment _ -> k None | Sexp t -> forget_t t (fun x -> k (Some x)) ;; end let t x = Cps.forget_t x (fun y -> y) let t_or_comment x = Cps.forget_toc x (fun y -> y) let t_or_comments x = Cps.forget_tocs x (fun y -> y) end parsexp-0.17.0/src/cst.mli000066400000000000000000000030361461647336100153610ustar00rootroot00000000000000(** Concrete syntax tree of s-expressions *) (** This module exposes a type that describe the full contents of a source file containing s-expressions. One can use this type to do low-level rewriting of s-expression files. *) open! Import type t = | Atom of { loc : Positions.range ; atom : string (** Source syntax of atom. The parser only fills this for atoms that are quoted in the source, but it makes sense for unquoted atoms too (to ensure they get printed unquoted). *) ; unescaped : string option } | List of { loc : Positions.range ; elements : t_or_comment list } and t_or_comment = | Sexp of t | Comment of comment and comment = | Plain_comment of { loc : Positions.range ; comment : string } (** Line or block comment *) | Sexp_comment of { hash_semi_pos : Positions.pos ; comments : comment list ; sexp : t } [@@deriving_inline sexp_of] include sig [@@@ocaml.warning "-32"] val sexp_of_t : t -> Sexplib0.Sexp.t val sexp_of_t_or_comment : t_or_comment -> Sexplib0.Sexp.t val sexp_of_comment : comment -> Sexplib0.Sexp.t end [@@ocaml.doc "@inline"] [@@@end] val compare : t -> t -> int val compare_t_or_comment : t_or_comment -> t_or_comment -> int val compare_comment : comment -> comment -> int module Forget : sig val t : t -> Ppx_sexp_conv_lib.Sexp.t val t_or_comment : t_or_comment -> Ppx_sexp_conv_lib.Sexp.t option val t_or_comments : t_or_comment list -> Ppx_sexp_conv_lib.Sexp.t list end parsexp-0.17.0/src/dune000066400000000000000000000002751461647336100147450ustar00rootroot00000000000000(library (name parsexp) (public_name parsexp) (libraries sexplib0) (preprocess no_preprocessing) (lint (pps ppx_js_style -check-doc-comments ppx_sexp_conv -deriving-keep-w32=impl))) parsexp-0.17.0/src/import.ml000066400000000000000000000003471461647336100157330ustar00rootroot00000000000000module Ppx_sexp_conv_lib = struct module Conv_error = Sexplib0.Sexp_conv_error module Conv = Sexplib0.Sexp_conv module Sexp = Sexplib0.Sexp end module Sexp = Sexplib0.Sexp include Sexplib0.Sexp_conv module List = ListLabels parsexp-0.17.0/src/of_sexp_error.ml000066400000000000000000000037541461647336100173020ustar00rootroot00000000000000open! Import type t = { user_exn : exn ; sub_sexp : Sexp.t ; location : Positions.range option } [@@deriving_inline sexp_of] let sexp_of_t = (fun { user_exn = user_exn__002_; sub_sexp = sub_sexp__004_; location = location__006_ } -> let bnds__001_ = ([] : _ Stdlib.List.t) in let bnds__001_ = let arg__007_ = sexp_of_option Positions.sexp_of_range location__006_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "location"; arg__007_ ] :: bnds__001_ : _ Stdlib.List.t) in let bnds__001_ = let arg__005_ = Sexp.sexp_of_t sub_sexp__004_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "sub_sexp"; arg__005_ ] :: bnds__001_ : _ Stdlib.List.t) in let bnds__001_ = let arg__003_ = sexp_of_exn user_exn__002_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "user_exn"; arg__003_ ] :: bnds__001_ : _ Stdlib.List.t) in Sexplib0.Sexp.List bnds__001_ : t -> Sexplib0.Sexp.t) ;; [@@@end] let user_exn t = t.user_exn let sub_sexp t = t.sub_sexp let location t = t.location let report ppf ~filename t = let line, start, stop = match t.location with | None -> 1, 0, 0 | Some { start_pos; end_pos } -> start_pos.line, start_pos.col, start_pos.col + end_pos.offset - start_pos.offset in Format.fprintf ppf "File \"%s\", line %d, characters %d-%d:\n\ Error: s-expression conversion error;\n\ exception %s\n" filename line start stop (Sexplib0.Sexp_conv.printexc_prefer_sexp t.user_exn) ;; exception Of_sexp_error of t [@@deriving_inline sexp_of] let () = Sexplib0.Sexp_conv.Exn_converter.add [%extension_constructor Of_sexp_error] (function | Of_sexp_error arg0__008_ -> let res0__009_ = sexp_of_t arg0__008_ in Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "of_sexp_error.ml.Of_sexp_error"; res0__009_ ] | _ -> assert false) ;; [@@@end] let raise ~user_exn ~sub_sexp ~location = raise (Of_sexp_error { user_exn; sub_sexp; location }) ;; parsexp-0.17.0/src/of_sexp_error.mli000066400000000000000000000012661461647336100174470ustar00rootroot00000000000000open! Import type t [@@deriving_inline sexp_of] include sig [@@@ocaml.warning "-32"] val sexp_of_t : t -> Sexplib0.Sexp.t end [@@ocaml.doc "@inline"] [@@@end] (** Exception raised by the user function *) val user_exn : t -> exn (** S-expression that failed to be converted *) val sub_sexp : t -> Sexp.t (** Position of [sub_sexp t] in the original source, if found *) val location : t -> Positions.range option (** Similar to [Parse_error.report] *) val report : Format.formatter -> filename:string -> t -> unit (** Exception raised in case of a conversion error *) exception Of_sexp_error of t val raise : user_exn:exn -> sub_sexp:Sexp.t -> location:Positions.range option -> 'a parsexp-0.17.0/src/old_parser_cont_state.ml000066400000000000000000000014551461647336100207770ustar00rootroot00000000000000open! Import type t = | Parsing_toplevel_whitespace | Parsing_nested_whitespace | Parsing_atom | Parsing_list | Parsing_sexp_comment | Parsing_block_comment [@@deriving_inline sexp_of] let sexp_of_t = (function | Parsing_toplevel_whitespace -> Sexplib0.Sexp.Atom "Parsing_toplevel_whitespace" | Parsing_nested_whitespace -> Sexplib0.Sexp.Atom "Parsing_nested_whitespace" | Parsing_atom -> Sexplib0.Sexp.Atom "Parsing_atom" | Parsing_list -> Sexplib0.Sexp.Atom "Parsing_list" | Parsing_sexp_comment -> Sexplib0.Sexp.Atom "Parsing_sexp_comment" | Parsing_block_comment -> Sexplib0.Sexp.Atom "Parsing_block_comment" : t -> Sexplib0.Sexp.t) ;; [@@@end] let to_string t = match sexp_of_t t with | Atom s -> s | List _ -> failwith "BUG: [sexp_of_t] returned a [List _]" ;; parsexp-0.17.0/src/old_parser_cont_state.mli000066400000000000000000000005311461647336100211420ustar00rootroot00000000000000open! Import type t = | Parsing_toplevel_whitespace | Parsing_nested_whitespace | Parsing_atom | Parsing_list | Parsing_sexp_comment | Parsing_block_comment [@@deriving_inline sexp_of] include sig [@@@ocaml.warning "-32"] val sexp_of_t : t -> Sexplib0.Sexp.t end [@@ocaml.doc "@inline"] [@@@end] val to_string : t -> string parsexp-0.17.0/src/parse_error.ml000066400000000000000000000062021461647336100167400ustar00rootroot00000000000000open! Import include Parse_error_intf type t = { position : Positions.pos ; message : string ; old_parser_exn : [ `Parse_error | `Failure ] } let sexp_of_t { position; message; old_parser_exn = _ } : Sexp.t = List [ List [ Atom "position"; Positions.sexp_of_pos position ] ; List [ Atom "message"; sexp_of_string message ] ] ;; let position t = t.position let message t = t.message let old_parser_exn t = t.old_parser_exn let report ppf ~filename t = let pos = position t in let msg = message t in Format.fprintf ppf "File \"%s\", line %d, character %d:\nError: s-expression parsing error;\n%s\n" filename pos.line pos.col msg ;; exception Parse_error of t [@@deriving_inline sexp] let () = Sexplib0.Sexp_conv.Exn_converter.add [%extension_constructor Parse_error] (function | Parse_error arg0__001_ -> let res0__002_ = sexp_of_t arg0__001_ in Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "parse_error.ml.Parse_error"; res0__002_ ] | _ -> assert false) ;; [@@@end] let raise (reason : Reason.t) position ~at_eof ~atom_buffer = let message = (* These messages where choosen such that we can build the various Sexplib parsing functions on top of Parsexp and keep the same exceptions. At the time of writing this, a simple layer on top of parsexp to implement the sexplib API is passing all the sexplib tests. Note that parsexp matches the semantic of Sexp.parse which is slightly different from the ocamllex/ocamlyacc based parser of Sexplib. The latter one is less tested and assumed to be less used. *) match reason with | Unexpected_char_parsing_hex_escape -> "unterminated hexadecimal escape sequence" | Unexpected_char_parsing_dec_escape -> "unterminated decimal escape sequence" | Unterminated_quoted_string -> "unterminated quoted string" | Unterminated_block_comment -> "unterminated block comment" | Escape_sequence_out_of_range -> "escape sequence in quoted string out of range" | Unclosed_paren -> "unclosed parentheses at end of input" | Too_many_sexps -> "s-expression followed by data" | Closed_paren_without_opened -> "unexpected character: ')'" | Comment_token_in_unquoted_atom -> if String.equal (Buffer.contents atom_buffer) "|" then "illegal end of comment" else "comment tokens in unquoted atom" | Sexp_comment_without_sexp -> "unterminated sexp comment" | Unexpected_character_after_cr -> if at_eof then "unexpected end of input after carriage return" else "unexpected character after carriage return" | No_sexp_found_in_input -> "no s-expression found in input" | Automaton_in_error_state -> failwith "Parsexp.Parser_automaton: parser is dead" in let old_parser_exn = match reason, at_eof with | Too_many_sexps, _ | _, true -> `Failure | Comment_token_in_unquoted_atom, _ when String.equal (Buffer.contents atom_buffer) "|" -> `Failure | _ -> `Parse_error in raise (Parse_error { position; message; old_parser_exn }) ;; module Private = struct module Reason = Reason let old_parser_exn = old_parser_exn let raise = raise end parsexp-0.17.0/src/parse_error.mli000066400000000000000000000000451461647336100171100ustar00rootroot00000000000000include Parse_error_intf.Parse_error parsexp-0.17.0/src/parse_error_intf.ml000066400000000000000000000027701461647336100177660ustar00rootroot00000000000000open! Import module Reason = struct (*_ Some of these come from [Parsexp_symbolic_automaton.Parse_error_reason]. *) type t = (*$ Parsexp_cinaps_helpers.Gen_parse_error.print_constructors () *) | Automaton_in_error_state | Comment_token_in_unquoted_atom | Unexpected_char_parsing_dec_escape | Unexpected_char_parsing_hex_escape | Unexpected_character_after_cr | Unterminated_block_comment | Unterminated_quoted_string (*$*) | Closed_paren_without_opened | Escape_sequence_out_of_range | No_sexp_found_in_input | Sexp_comment_without_sexp | Too_many_sexps | Unclosed_paren end module type Parse_error = sig type t [@@deriving_inline sexp_of] include sig [@@@ocaml.warning "-32"] val sexp_of_t : t -> Sexplib0.Sexp.t end [@@ocaml.doc "@inline"] [@@@end] val position : t -> Positions.pos val message : t -> string (** Report an error in a style similar to OCaml, for instance: File "blah", line 42, character 10: Error: s-expression parsing error; unterminated quoted string. *) val report : Format.formatter -> filename:string -> t -> unit exception Parse_error of t (**/**) module Private : sig module Reason = Reason (** To match the old behavior, the old parser sometimes raised [Failure] and sometimes raised [Parse_error] *) val old_parser_exn : t -> [ `Parse_error | `Failure ] val raise : Reason.t -> Positions.pos -> at_eof:bool -> atom_buffer:Buffer.t -> 'a end end parsexp-0.17.0/src/parser.ml000066400000000000000000000131501461647336100157110ustar00rootroot00000000000000open! Import include Parser_intf module A = Automaton let kind_to_stack : type stack. ('state, stack) Automaton_state.Kind.t -> (module Stack with type t = stack) = fun (type state) (kind : (state, stack) Automaton_state.Kind.t) -> match kind with | Sexp -> (module Automaton_stack : Stack with type t = stack) | Positions -> (module Automaton_stack.Just_positions : Stack with type t = stack) | Sexp_with_positions -> (module Automaton_stack : Stack with type t = stack) | Cst -> (module Automaton_stack.For_cst : Stack with type t = stack) ;; let make (type stack state parsed_value) kind mode make_value : (module S with type parsed_value = parsed_value and type State.t = (state, stack) Automaton_state.t and type Stack.t = stack) = (module struct type nonrec parsed_value = parsed_value module Stack = (val kind_to_stack kind : Stack with type t = stack) module State = struct type t = (state, Stack.t) Automaton_state.t let create ?pos () = A.create ?initial_pos:pos mode kind let reset = A.reset let offset = A.offset let line = A.line let column = A.column let position t : Positions.pos = { offset = offset t; line = line t; col = column t } ;; let stop state = A.set_error_state state end let feed = A.feed let feed_eoi state stack = make_value state (A.feed_eoi state stack) let feed_substring = Automaton.feed_substring let feed_string = Automaton.feed_string let feed_subbytes = Automaton.feed_subbytes let feed_bytes = Automaton.feed_bytes let parse_string_exn str = let state = State.create () in feed_eoi state (feed_string state str Stack.empty) ;; let parse_string str = match parse_string_exn str with | x -> Ok x | exception Parse_error.Parse_error e -> Error e ;; end) ;; let make_eager (type stack state parsed_value) kind make_value : (module S_eager with type parsed_value = parsed_value and type State.t = (state, stack) Automaton_state.t and type Stack.t = stack) = (module struct type nonrec parsed_value = parsed_value module Stack = (val kind_to_stack kind : Stack with type t = stack) module State = struct module Read_only = struct type t = (state, Stack.t) Automaton_state.t let offset = A.offset let line = A.line let column = A.column let position t : Positions.pos = { offset = offset t; line = line t; col = column t } ;; end include Read_only let create ?pos ?(reraise_notrace = false) ?(no_sexp_is_error = false) f = let got_sexp state stack = let parsed_value = make_value state stack in f state parsed_value; Stack.empty in A.create ?initial_pos:pos (Eager { got_sexp; reraise_notrace; no_sexp_is_error }) kind ;; let reset = A.reset let stop t = A.set_error_state t let old_parser_cont_state t = Automaton.old_parser_cont_state t end let feed = A.feed let feed_eoi state stack = ignore (A.feed_eoi state stack : Stack.t) let feed_substring = Automaton.feed_substring let feed_string = Automaton.feed_string let feed_subbytes = Automaton.feed_subbytes let feed_bytes = Automaton.feed_bytes module Lexbuf_consumer = struct type t = State.t exception Got_sexp of parsed_value * Positions.pos let got_sexp state parsed_value = raise_notrace (Got_sexp (parsed_value, State.position state)) ;; let create () = State.create got_sexp let pos_of_lexbuf lexbuf = let p = lexbuf.Lexing.lex_curr_p in { Positions.line = p.pos_lnum; col = p.pos_cnum - p.pos_bol; offset = p.pos_cnum } ;; let update_lexbuf (lexbuf : Lexing.lexbuf) (pos : Positions.pos) = let p = pos.offset - lexbuf.lex_abs_pos in lexbuf.lex_curr_pos <- p; lexbuf.lex_start_pos <- p; lexbuf.lex_curr_p <- { lexbuf.lex_curr_p with pos_lnum = pos.line ; pos_cnum = pos.offset ; pos_bol = pos.offset - pos.col } ;; let rec feed_lexbuf t (lexbuf : Lexing.lexbuf) stack = let stack = feed_subbytes t lexbuf.lex_buffer stack ~pos:lexbuf.lex_curr_pos ~len:(lexbuf.lex_buffer_len - lexbuf.lex_curr_pos) in lexbuf.lex_curr_pos <- lexbuf.lex_buffer_len; lexbuf.lex_start_pos <- lexbuf.lex_buffer_len; if not lexbuf.lex_eof_reached then ( lexbuf.refill_buff lexbuf; feed_lexbuf t lexbuf stack) else feed_eoi t stack ;; let parse_gen t (lexbuf : Lexing.lexbuf) = A.reset t ~pos:(pos_of_lexbuf lexbuf); match feed_lexbuf t lexbuf Stack.empty with | () -> update_lexbuf lexbuf (State.position t); None | exception Got_sexp (parsed_value, pos) -> update_lexbuf lexbuf pos; Some parsed_value | exception exn -> update_lexbuf lexbuf (State.position t); raise exn ;; let set_no_sexp_is_error t x = match A.mode t with | Eager e -> e.no_sexp_is_error <- x | _ -> assert false ;; let parse t lexbuf = set_no_sexp_is_error t true; match parse_gen t lexbuf with | Some x -> x | None -> failwith "Parsexp.parse_gen: None" ;; let parse_opt t lexbuf = set_no_sexp_is_error t false; parse_gen t lexbuf ;; end end) ;; parsexp-0.17.0/src/parser.mli000066400000000000000000000000521461647336100160570ustar00rootroot00000000000000include Parser_intf.Parser (** @inline *) parsexp-0.17.0/src/parser_intf.ml000066400000000000000000000142661461647336100167420ustar00rootroot00000000000000open! Import module type State = sig (** State of the parser *) type t (** Create a new parser state. [pos] is the initial position, it defaults to [{line=1;col=0;offset=0}]. *) val create : ?pos:Positions.pos -> unit -> t (** Reset the given parsing state. The following always succeed: {[ reset t ?pos; assert (t = create ?pos ()) ]} *) val reset : ?pos:Positions.pos -> t -> unit (** Number of characters fed to the parser *) val offset : t -> int (** Position in the text *) val line : t -> int val column : t -> int val position : t -> Positions.pos (** Prevent the state from receiving any more characters. Trying to feed more characters will result in an exception, unless the state is reset. *) val stop : t -> unit end module type Stack = sig (** Parser stack. The stack is not in [state] for optimization purposes. *) type t val empty : t end module type S = sig (** Values produced by the parser *) type parsed_value module State : State module Stack : Stack (** Feed one character to the parser. In case of error, it raises [Parse_error] *) val feed : State.t -> char -> Stack.t -> Stack.t (** Instruct the parser that the end of input was reached. In case of error, it raises [Parse_error] *) val feed_eoi : State.t -> Stack.t -> parsed_value (** {3 Convenience functions} *) val feed_string : State.t -> string -> Stack.t -> Stack.t val feed_substring : State.t -> string -> pos:int -> len:int -> Stack.t -> Stack.t val feed_bytes : State.t -> bytes -> Stack.t -> Stack.t val feed_subbytes : State.t -> bytes -> pos:int -> len:int -> Stack.t -> Stack.t (** {3 High-level functions} *) val parse_string : string -> (parsed_value, Parse_error.t) result val parse_string_exn : string -> parsed_value end module type S_eager = sig (** Same as [Parser] but gives back a s-expression as soon as they are found in the input. For instance you can use this function to parse a stream and stop at the first s-expression: {[ exception Got_sexp of Sexp.t let fetch_sexp stream = let module P = Parsexp.Sexp_parsing.Eager in let rec hot_loop state stream stack = match Stream.peek stream with | None -> P.feed_eoi state stack | Some char -> let stack = P.feed state char stack in Stream.junk stream; hot_loop state stream stack in let got_sexp state sexp = raise_notrace (Got_sexp sexp) in let count = Stream.count stream in let state = P.State.create ~f:got_sexp ~no_sexp_is_error:true in match hot_loop state stream P.Stack.empty with | () -> assert false | exception (Got_sexp sexp) -> (* This test is true if the s-expression includes the last character passed to the parser *) if P.State.offset state > Stream.count stream - count then Stream.junk stream; sexp ]} *) (** Values produces by the parser *) type parsed_value module State : sig include State module Read_only : sig (** Read-only handle to a parser state *) type t val offset : t -> int val line : t -> int val column : t -> int val position : t -> Positions.pos end (** [create ~f] create a new eager parser state. [f] will be called on each s-expression found. If [f] raises, then the parser is made unusable ([stop t] is invoked). [no_sexp_is_error] controls the behavior of the parse when the end of input is reached and no s-expression has been found. When [no_sexp_is_error] is [false] (the default) [feed_eoi] just returns [()], when it is [false] [feed_eoi] raises. In any case, if the end of input is reached while parsing an incomplete s-expression such as [(abc], error is raised. [reraise_notrace] controls whether or not the exceptions raised by [f] are reraised without a backtrace. ([false]: with, [true]: without) Generally you should pass [true] if you're using [raise_nontrace] in [f], especially if using exceptions for control flow. [f] must not save the read-only parser state it receives to access it after returning. It is unspecified what values it will read if it does so. *) val create : ?pos:Positions.pos -> ?reraise_notrace:bool (** default: false *) -> ?no_sexp_is_error:bool (** default: false *) -> (Read_only.t -> parsed_value -> unit) -> t (**/**) val old_parser_cont_state : t -> Old_parser_cont_state.t (**/**) end module Stack : Stack val feed : State.t -> char -> Stack.t -> Stack.t val feed_eoi : State.t -> Stack.t -> unit val feed_string : State.t -> string -> Stack.t -> Stack.t val feed_substring : State.t -> string -> pos:int -> len:int -> Stack.t -> Stack.t val feed_bytes : State.t -> bytes -> Stack.t -> Stack.t val feed_subbytes : State.t -> bytes -> pos:int -> len:int -> Stack.t -> Stack.t module Lexbuf_consumer : sig type t val create : unit -> t (** Consume exactly one s-expression from the given lexing buffer *) val parse : t -> Lexing.lexbuf -> parsed_value (** Consume exactly one s-expression from the given lexing buffer. Returns [None] if the end of input is reached before seeing any s-expression. *) val parse_opt : t -> Lexing.lexbuf -> parsed_value option end end module type Parser = sig module type S = S module type S_eager = S_eager module type Stack = Stack val make : ('state, 'stack) Automaton_state.Kind.t -> ('state, 'stack) Automaton_state.Mode.t -> (('state, 'stack) Automaton_state.t -> 'stack -> 'a) -> (module S with type parsed_value = 'a and type State.t = ('state, 'stack) Automaton_state.t and type Stack.t = 'stack) val make_eager : ('state, 'stack) Automaton_state.Kind.t -> (('state, 'stack) Automaton_state.t -> 'stack -> 'a) -> (module S_eager with type parsed_value = 'a and type State.t = ('state, 'stack) Automaton_state.t and type Stack.t = 'stack) end parsexp-0.17.0/src/parsexp.ml000066400000000000000000000066721461647336100161120ustar00rootroot00000000000000open! Import module type Conv = Conv.S module type Parser = Parser.S module type Eager_parser = Parser.S_eager module Conv_error = Conv_error module Of_sexp_error = Of_sexp_error module Old_parser_cont_state = Old_parser_cont_state module Parse_error = Parse_error module Positions = Positions module Cst = Cst module A = Automaton exception Parse_error = Parse_error.Parse_error exception Of_sexp_error = Of_sexp_error.Of_sexp_error let const c _ = c module Single = (val Parser.make Sexp Single (const Automaton_stack.get_single)) module Many = (val Parser.make Sexp Many (const Automaton_stack.get_many)) module Eager = (val Parser.make_eager Sexp (const Automaton_stack.get_single)) let and_get_positions get_sexp state stack = get_sexp stack, A.positions state let and_positions mode get_sexp = Parser.make Sexp_with_positions mode (and_get_positions get_sexp) ;; module Single_and_positions = (val and_positions Single Automaton_stack.get_single) module Many_and_positions = (val and_positions Many Automaton_stack.get_many) module Eager_and_positions = (val Parser.make_eager Sexp_with_positions (Automaton_stack.get_single |> and_get_positions)) let just_get_positions state () = A.positions state let just_positions mode = Parser.make Positions mode just_get_positions module Single_just_positions = (val just_positions Single) module Many_just_positions = (val just_positions Many) module Eager_just_positions = (val Parser.make_eager Positions just_get_positions) let cst mode f = Parser.make Cst mode (const f) module Many_cst = (val cst Many Automaton_stack.For_cst.get_many) module Eager_cst = (val Parser.make_eager Cst (fun _ stack -> match Automaton_stack.For_cst.get_many stack with | [ sexp ] -> sexp | _ -> assert false)) type 'a id = 'a type sexp_list = Sexp.t list module Conv_single = Conv.Make (struct type 'a res = 'a type parsed_sexp = Sexp.t type chunk_to_conv = Sexp.t let apply_f x ~f = f x let find = Positions.find_sub_sexp_phys end) (Single) (Single_just_positions) module Conv_many = Conv.Make (struct type 'a res = 'a list type parsed_sexp = Sexp.t list type chunk_to_conv = Sexp.t let apply_f x ~f = List.rev (List.rev_map x ~f) let find = Positions.find_sub_sexp_in_list_phys end) (Many) (Many_just_positions) module Conv_many_and_locations = Conv.Make (struct type 'a res = 'a list type parsed_sexp = Sexp.t list * Positions.t type chunk_to_conv = Sexp.t * Positions.range let find positions (s, _) ~sub = Positions.find_sub_sexp_in_list_phys positions s ~sub ;; let apply_f (sexps, positions) ~f = let iter = Positions.Iterator.create positions in List.rev (List.rev_map sexps ~f:(fun sexp -> let location = Positions.Iterator.advance_sexp_exn iter sexp in f (sexp, location))) ;; end) (Many_and_positions) (Many_just_positions) module Conv_many_at_once = Conv.Make (struct type 'a res = 'a type parsed_sexp = Sexp.t list type chunk_to_conv = Sexp.t list let apply_f x ~f = f x let find = Positions.find_sub_sexp_in_list_phys end) (Many) (Many_just_positions) module Private = struct module Automaton = Automaton module Automaton_stack = Automaton_stack module Automaton_state = Automaton_state module Positions = Positions end parsexp-0.17.0/src/parsexp.mli000066400000000000000000000000541461647336100162470ustar00rootroot00000000000000include Parsexp_intf.Parsexp (** @inline *) parsexp-0.17.0/src/parsexp_intf.ml000066400000000000000000000100001461647336100171060ustar00rootroot00000000000000(** Parsing of s-expression *) open! Import module type Parsexp = sig module Conv_error = Conv_error module Of_sexp_error = Of_sexp_error module Old_parser_cont_state = Old_parser_cont_state module Parse_error = Parse_error module Positions = Positions module Cst = Cst module type Conv = Conv.S module type Parser = Parser.S module type Eager_parser = Parser.S_eager (** Exception raised in case of a conversion error *) exception Of_sexp_error of Of_sexp_error.t (** Exception raised in case of a parsing error *) exception Parse_error of Parse_error.t module Single : Parser with type parsed_value = Sexp.t and type State.t = (unit, Automaton_stack.t) Automaton_state.t and type Stack.t = Automaton_stack.t module Many : Parser with type parsed_value = Sexp.t list and type State.t = (unit, Automaton_stack.t) Automaton_state.t and type Stack.t = Automaton_stack.t module Eager : Eager_parser with type parsed_value = Sexp.t and type State.t = (unit, Automaton_stack.t) Automaton_state.t and type Stack.t = Automaton_stack.t module Single_and_positions : Parser with type parsed_value = Sexp.t * Positions.t and type State.t = (Positions.Builder.t, Automaton_stack.t) Automaton_state.t and type Stack.t = Automaton_stack.t module Many_and_positions : Parser with type parsed_value = Sexp.t list * Positions.t and type State.t = (Positions.Builder.t, Automaton_stack.t) Automaton_state.t and type Stack.t = Automaton_stack.t module Eager_and_positions : Eager_parser with type parsed_value = Sexp.t * Positions.t and type State.t = (Positions.Builder.t, Automaton_stack.t) Automaton_state.t and type Stack.t = Automaton_stack.t module Single_just_positions : Parser with type parsed_value = Positions.t and type State.t = (Positions.Builder.t, unit) Automaton_state.t and type Stack.t = unit module Many_just_positions : Parser with type parsed_value = Positions.t and type State.t = (Positions.Builder.t, unit) Automaton_state.t and type Stack.t = unit module Eager_just_positions : Eager_parser with type parsed_value = Positions.t and type State.t = (Positions.Builder.t, unit) Automaton_state.t and type Stack.t = unit module Many_cst : Parser with type parsed_value = Cst.t_or_comment list and type State.t = (Automaton_state.For_cst.t, Automaton_stack.For_cst.t) Automaton_state.t and type Stack.t = Automaton_stack.For_cst.t module Eager_cst : Eager_parser with type parsed_value = Cst.t_or_comment and type State.t = (Automaton_state.For_cst.t, Automaton_stack.For_cst.t) Automaton_state.t and type Stack.t = Automaton_stack.For_cst.t (*_ These type synonyms are introduced because ocaml <4.06 do not support destructive substitutions with `type 'a t1 = t2` or `type t1 = 'a t2`. *) type 'a id = 'a type sexp_list = Sexp.t list module Conv_single : Conv with type 'a res := 'a id and type parsed_sexp := Sexp.t and type chunk_to_conv := Sexp.t module Conv_many : Conv with type 'a res := 'a list and type parsed_sexp := sexp_list and type chunk_to_conv := Sexp.t module Conv_many_and_locations : Conv with type 'a res := 'a list and type parsed_sexp := sexp_list * Positions.t and type chunk_to_conv := Sexp.t * Positions.range module Conv_many_at_once : Conv with type 'a res := 'a id and type parsed_sexp := sexp_list and type chunk_to_conv := sexp_list (*_ For tests *) (*_ See the Jane Street Style Guide for an explanation of [Private] submodules: https://opensource.janestreet.com/standards/#private-submodules *) module Private : sig module Automaton = Automaton module Automaton_stack = Automaton_stack module Automaton_state = Automaton_state module Positions = Positions end end parsexp-0.17.0/src/positions.ml000066400000000000000000000415541461647336100164550ustar00rootroot00000000000000(* This module builds a buffer of "instructions", in order to represent a compact sequence of delimiting positions and newlines. The parser stores the positions of each: - newline - beginning of atom - end of atom - left parenthesis - right parenthesis Instructions are encoded as a sequence bits. The next instruction is determined by looking at the next few bits: - bit 0 represents a saved position followed by an offset increment - bits 10 represent an offset increment - bits 110 are followed by 5 bits of payload. The 5-bit payloads of any subsequent 110- instructions are squashed to form a number (least significant 5-bit chunk first). This number + 5 represents an offset increment - bits 1110 marks the beginning of a new line (with offset incremented) - bits 1111 represent a position saved twice followed by an offset increment For instance let's consider the following sexp: {[ {| (abc "foo bar" ) |} ]} the sequence of instructions to record in order to reconstruct the position of any sub-sexp is: - 0 save position and advance 1: first '(' - 0 save position and advance 1: start of "abc" - 10 advance 1 - 0 save position and advance 1: end of "abc" - 1110 newline - 1100_0001 advance 6 - 0 save position and advance 1: start of "foo\n bar" - 10 advance 1 - 10 advance 1 - 10 advance 1 - 1110 newline - 1100_0000 advance 5 - 0 save position and advance 1: end of "foo\n bar" - 1110 newline - 0 save position and advance 1: last ')' (we save the position after the closing parenthesis) The total sequence is 42 bits, so we need 6 bytes to store it The sequence of bits is encoded as a sequence of 16-bit values, where the earlier bits are most significant. Note that the parser stores the end positions as inclusive. This way only single character atoms require a double positions. If we were storing end positions as exclusive, we would need double positions for [)(] and [a(], which are likely to be frequent in s-expressions printed with the non [_hum] printer. We expect single character atoms to be less frequent so it makes sense to penalize them instead. *) open! Import type pos = { line : int ; col : int ; offset : int } [@@deriving_inline sexp_of] let sexp_of_pos = (fun { line = line__002_; col = col__004_; offset = offset__006_ } -> let bnds__001_ = ([] : _ Stdlib.List.t) in let bnds__001_ = let arg__007_ = sexp_of_int offset__006_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "offset"; arg__007_ ] :: bnds__001_ : _ Stdlib.List.t) in let bnds__001_ = let arg__005_ = sexp_of_int col__004_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "col"; arg__005_ ] :: bnds__001_ : _ Stdlib.List.t) in let bnds__001_ = let arg__003_ = sexp_of_int line__002_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "line"; arg__003_ ] :: bnds__001_ : _ Stdlib.List.t) in Sexplib0.Sexp.List bnds__001_ : pos -> Sexplib0.Sexp.t) ;; [@@@end] let compare_pos = Stdlib.compare let beginning_of_file = { line = 1; col = 0; offset = 0 } let shift_pos pos ~cols = { pos with col = pos.col + cols; offset = pos.offset + cols } type range = { start_pos : pos ; end_pos : pos } [@@deriving_inline sexp_of] let sexp_of_range = (fun { start_pos = start_pos__009_; end_pos = end_pos__011_ } -> let bnds__008_ = ([] : _ Stdlib.List.t) in let bnds__008_ = let arg__012_ = sexp_of_pos end_pos__011_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "end_pos"; arg__012_ ] :: bnds__008_ : _ Stdlib.List.t) in let bnds__008_ = let arg__010_ = sexp_of_pos start_pos__009_ in (Sexplib0.Sexp.List [ Sexplib0.Sexp.Atom "start_pos"; arg__010_ ] :: bnds__008_ : _ Stdlib.List.t) in Sexplib0.Sexp.List bnds__008_ : range -> Sexplib0.Sexp.t) ;; [@@@end] let compare_range = Stdlib.compare let make_range_incl ~start_pos ~last_pos = { start_pos; end_pos = shift_pos last_pos ~cols:1 } ;; module Chunk : sig (** Represents an array of [length/2] signed 16-bit values *) type t (** Length in bytes. *) val length : int val alloc : unit -> t (** [get16 ~pos] and [set16 ~pos] manipulate the [pos/2]th stored value. [pos] must be even. [set16 x] only uses the 16 least significant bits of [x]. *) val get16 : t -> pos:int -> int val set16 : t -> pos:int -> int -> unit end = struct type t = bytes (* OCaml strings always waste two bytes at the end, so we take a power of two minus two to be sure we don't waste space. *) let length = 62 let alloc () = Bytes.create length external get16 : bytes -> pos:int -> int = "%caml_bytes_get16" external set16 : bytes -> pos:int -> int -> unit = "%caml_bytes_set16" (* If we want to make a [Positions.t] serializable: {[ external bswap16 : int -> int = "%bswap16";; let get16 = if Caml.Sys.arch_big_endian then fun buf ~pos -> get16 buf ~pos |> bswap16 else get16 let set16 = if Caml.Sys.arch_big_endian then fun buf ~pos x -> set16 buf ~pos (bswap16 x) else set16 ]} *) end type t_ = { chunks : Chunk.t list ; (* [num_bytes * 8 + extra_bits] is the number of bits stored in [chunks]. The last [extra_bits] bits will be stored as the *least* significant bits of the appropriate pair of bytes of the last chunk. *) num_bytes : int ; extra_bits : int ; initial_pos : pos } type t = t_ Lazy.t let memory_footprint_in_bytes (lazy t) = let num_fields = 4 in let header_words = 1 in let word_bytes = match Sys.word_size with | 32 -> 4 | 64 -> 8 | _ -> assert false in let chunk_words = let div_ceil a b = (a + b - 1) / b in let n = div_ceil (Chunk.length + 1 (* NUL terminating bytes *) + 1 (* number of wasted bytes to fill a word *)) word_bytes in n + header_words in let pos_fields = 3 in let pos_words = header_words + pos_fields in let list_cons_words = header_words + 2 in (header_words + num_fields + pos_words + (List.length t.chunks * (chunk_words + list_cons_words))) * word_bytes ;; module Builder = struct type t = { mutable chunk : Chunk.t ; mutable chunk_pos : int ; mutable filled_chunks : Chunk.t list (* Filled chunks in reverse order *) ; mutable offset : int (* Offset of the last saved position or newline plus one, or [initial_pos] *) ; mutable int_buf : int (* the [num_bits] least significant bits of [int_buf] are the bits not yet pushed to [chunk]. *) ; mutable num_bits : int (* number of bits stored in [int_buf] *) ; mutable initial_pos : pos } let invariant t = assert (t.chunk_pos >= 0 && t.chunk_pos <= Chunk.length); assert (t.offset >= t.initial_pos.offset); assert (t.num_bits <= 15) ;; let check_invariant = false let invariant t = if check_invariant then invariant t let create ?(initial_pos = beginning_of_file) () = { chunk = Chunk.alloc () ; chunk_pos = 0 ; filled_chunks = [] ; offset = initial_pos.offset ; int_buf = 0 ; num_bits = 0 ; initial_pos } ;; let reset t (pos : pos) = (* We need a new chunk as [contents] keeps the current chunk in the closure of the lazy value. *) t.chunk <- Chunk.alloc (); t.chunk_pos <- 0; t.filled_chunks <- []; t.offset <- pos.offset; t.int_buf <- 0; t.num_bits <- 0; t.initial_pos <- pos ;; let[@inline never] alloc_new_chunk t = t.filled_chunks <- t.chunk :: t.filled_chunks; t.chunk <- Chunk.alloc (); t.chunk_pos <- 0 ;; let add_uint16 t n = if t.chunk_pos = Chunk.length then alloc_new_chunk t; Chunk.set16 t.chunk ~pos:t.chunk_pos n ;; let add_bits t n ~num_bits = let int_buf = (t.int_buf lsl num_bits) lor n in let num_bits = t.num_bits + num_bits in t.int_buf <- int_buf; if num_bits < 16 then t.num_bits <- num_bits else ( let num_bits = num_bits - 16 in t.num_bits <- num_bits; add_uint16 t (int_buf lsr num_bits); t.chunk_pos <- t.chunk_pos + 2 (* no need to clear the bits of int_buf we just wrote, as further set16 will ignore these extra bits. *)) ;; let contents t = (* Flush the current [t.int_buf] *) add_uint16 t t.int_buf; let rev_chunks = t.chunk :: t.filled_chunks in let chunk_pos = t.chunk_pos in let extra_bits = t.num_bits in let initial_pos = t.initial_pos in lazy { chunks = List.rev rev_chunks ; num_bytes = ((List.length rev_chunks - 1) * Chunk.length) + chunk_pos ; extra_bits ; initial_pos } ;; let long_shift t n = let n = ref (n - 5) in while !n > 0 do add_bits t (0b1100_0000 lor (!n land 0b0001_1111)) ~num_bits:8; n := !n lsr 5 done ;; (* precondition: n >= 5 *) let[@inline never] add_gen_slow t n ~instr ~instr_bits = long_shift t n; add_bits t instr ~num_bits:instr_bits ;; let shift4 = 0b10_10_10_10 let[@inline always] add_gen t ~offset ~instr ~instr_bits = invariant t; let n = offset - t.offset in t.offset <- offset + 1; match n with | 0 | 1 | 2 | 3 | 4 -> let num_bits = (n lsl 1) + instr_bits in add_bits t ((shift4 lsl instr_bits) lor instr land ((1 lsl num_bits) - 1)) ~num_bits | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 -> add_bits t (((0b1100_0000 lor (n - 5)) lsl instr_bits) lor instr) ~num_bits:(8 + instr_bits) | _ -> if n < 0 then invalid_arg "Parsexp.Positions.add_gen"; add_gen_slow t n ~instr ~instr_bits ;; let add t ~offset = add_gen t ~offset ~instr:0b0 ~instr_bits:1 let add_twice t ~offset = add_gen t ~offset ~instr:0b1111 ~instr_bits:4 let add_newline t ~offset = add_gen t ~offset ~instr:0b1110 ~instr_bits:4 end type positions = t let rec sub_sexp_count (sexp : Sexp.t) = match sexp with | Atom _ -> 1 | List l -> List.fold_left l ~init:1 ~f:(fun acc x -> acc + sub_sexp_count x) ;; let sexp_positions_count sexp = sub_sexp_count sexp * 2 module Iterator : sig type t val create : positions -> t exception No_more (* [advance t ~skip] ignores [skip] saved positions and returns the next saved position. Raises [No_more] when reaching the end of the position set. *) val advance_exn : t -> skip:int -> pos val advance_sexp_exn : t -> Sexp.t -> range end = struct type t = { mutable chunk : Chunk.t ; mutable chunks : Chunk.t list ; (* [num_bytes * 8 + extra_bits] is the number of bits available from [instr_pos] in [chunk :: chunks]. *) mutable num_bytes : int ; extra_bits : int ; mutable instr_pos : int (* position in [chunk] *) ; mutable offset : int ; mutable line : int ; mutable bol : int ; mutable int_buf : int ; mutable num_bits : int (* Number of bits not yet consumed in [int_buf] *) ; mutable pending : pos option } let create ((lazy p) : positions) = match p.chunks with | [] -> assert false | chunk :: chunks -> { chunk ; chunks ; num_bytes = p.num_bytes ; extra_bits = p.extra_bits ; instr_pos = 0 ; offset = p.initial_pos.offset ; line = p.initial_pos.line ; bol = p.initial_pos.offset - p.initial_pos.col ; int_buf = 0 ; num_bits = 0 ; pending = None } ;; exception No_more let no_more () = raise_notrace No_more let[@inline never] fetch_chunk t = match t.chunks with | [] -> assert false | chunk :: chunks -> t.instr_pos <- 0; t.num_bytes <- t.num_bytes - Chunk.length; t.chunk <- chunk; t.chunks <- chunks ;; let fetch t = if t.instr_pos > t.num_bytes then no_more (); if t.instr_pos = Chunk.length then fetch_chunk t; let v = Chunk.get16 t.chunk ~pos:t.instr_pos in let added_bits = if t.instr_pos = t.num_bytes then t.extra_bits else 16 in t.int_buf <- (t.int_buf lsl added_bits) lor (v land ((1 lsl added_bits) - 1)); t.num_bits <- t.num_bits + added_bits; t.instr_pos <- t.instr_pos + 2 ;; let next_instruction_bits t ~num_bits = if t.num_bits < num_bits then ( fetch t; if t.num_bits < num_bits then no_more ()); let n = (t.int_buf lsr (t.num_bits - num_bits)) land ((1 lsl num_bits) - 1) in t.num_bits <- t.num_bits - num_bits; n ;; (* [offset_shift] and [offset_shift_num_bits] encode the offset number specified by the immediately preceding [110] instructions. *) let rec advance t ~skip ~offset_shift ~offset_shift_num_bits = match next_instruction_bits t ~num_bits:1 with | 0 -> (* bit seq 0 -> new item *) let offset = t.offset + offset_shift in t.offset <- offset + 1; if skip = 0 then { line = t.line; col = offset - t.bol; offset } else advance t ~skip:(skip - 1) ~offset_shift:0 ~offset_shift_num_bits:0 | _ -> (match next_instruction_bits t ~num_bits:1 with | 0 -> (* bit seq 10 -> shift *) t.offset <- t.offset + offset_shift + 1; advance t ~skip ~offset_shift:0 ~offset_shift_num_bits:0 | _ -> (match next_instruction_bits t ~num_bits:1 with | 0 -> (* bit seq 110 -> long shift *) let n = next_instruction_bits t ~num_bits:5 in let offset_shift = if offset_shift_num_bits = 0 then 5 else offset_shift in advance t ~skip ~offset_shift:(offset_shift + (n lsl offset_shift_num_bits)) ~offset_shift_num_bits:(offset_shift_num_bits + 5) | _ -> (match next_instruction_bits t ~num_bits:1 with | 0 -> (* bit seq 1110 -> newline *) t.offset <- t.offset + offset_shift + 1; t.bol <- t.offset; t.line <- t.line + 1; advance t ~skip ~offset_shift:0 ~offset_shift_num_bits:0 | _ -> (* bit seq 1111 -> 2 new items *) let offset = t.offset + offset_shift in t.offset <- offset + 1; if skip <= 1 then ( let pos = { line = t.line; col = offset - t.bol; offset } in if skip = 0 then t.pending <- Some pos; pos) else advance t ~skip:(skip - 2) ~offset_shift:0 ~offset_shift_num_bits:0))) ;; let advance_exn t ~skip = match t.pending with | Some pos -> t.pending <- None; if skip = 0 then pos else advance t ~skip:(skip - 1) ~offset_shift:0 ~offset_shift_num_bits:0 | None -> advance t ~skip ~offset_shift:0 ~offset_shift_num_bits:0 ;; let advance_sexp_exn t sexp = let positions_count = sexp_positions_count sexp in let start_pos = advance_exn t ~skip:0 in let last_pos = advance_exn t ~skip:(positions_count - 2) in make_range_incl ~start_pos ~last_pos ;; end let find t a b = if a < 0 || b <= a then invalid_arg "Parsexp.Positions.find"; let iter = Iterator.create t in try let start_pos = Iterator.advance_exn iter ~skip:a in let last_pos = Iterator.advance_exn iter ~skip:(b - a - 1) in make_range_incl ~start_pos ~last_pos with | Iterator.No_more -> failwith "Parsexp.Position.find" ;; module Sexp_search = struct exception Found of int let rec loop ~sub index (sexp : Sexp.t) = if sexp == sub then raise_notrace (Found index) else ( match sexp with | Atom _ -> index + 2 | List l -> let index = loop_list ~sub (index + 1) l in index + 1) and loop_list ~sub index (sexps : Sexp.t list) = List.fold_left sexps ~init:index ~f:(loop ~sub) ;; let finalize t ~sub a = let b = a + (sub_sexp_count sub * 2) - 1 in Some (find t a b) ;; let find_sub_sexp_phys t sexp ~sub = match loop ~sub 0 sexp with | (_ : int) -> None | exception Found n -> finalize t ~sub n ;; let find_sub_sexp_in_list_phys t sexps ~sub = match loop_list ~sub 0 sexps with | (_ : int) -> None | exception Found n -> finalize t ~sub n ;; end let find_sub_sexp_phys = Sexp_search.find_sub_sexp_phys let find_sub_sexp_in_list_phys = Sexp_search.find_sub_sexp_in_list_phys let to_list t = let iter = Iterator.create t in let rec loop acc = match Iterator.advance_exn iter ~skip:0 with | exception Iterator.No_more -> List.rev acc | pos -> loop (pos :: acc) in loop [] ;; let to_array t = to_list t |> Array.of_list let compare t1 t2 = Stdlib.compare (to_array t1) (to_array t2) let sexp_of_t t = sexp_of_array sexp_of_pos (to_array t) parsexp-0.17.0/src/positions.mli000066400000000000000000000106211461647336100166150ustar00rootroot00000000000000(** Compact set of positions *) open! Import (** A [t] value represent a sequence of positions. The focus is on small memory footprint. Given a s-expression and a sequence of positions, one can reconstruct the location of every sub s-expression. This is used to report location informations without having to annotate every node in the s-expression during parsing. The s-expression parser saves the positions of each opening and closing parentheses as well as the positions of the first and last character of each atom. Note that a [t] can hold the same given positions no more than twice. The parser stores the same position twice for non-quoted single character atoms. *) type t [@@deriving_inline sexp_of] include sig [@@@ocaml.warning "-32"] val sexp_of_t : t -> Sexplib0.Sexp.t end [@@ocaml.doc "@inline"] [@@@end] val compare : t -> t -> int (** Represent a position in the input *) type pos = { line : int (** Line number. The first line has number [1]. *) ; col : int (** Column number. The first column has number [0]. *) ; offset : int (** Number of bytes from the beginning of the input. The first byte has offset [0]. *) } [@@deriving_inline sexp_of] include sig [@@@ocaml.warning "-32"] val sexp_of_pos : pos -> Sexplib0.Sexp.t end [@@ocaml.doc "@inline"] [@@@end] val compare_pos : pos -> pos -> int val beginning_of_file : pos val shift_pos : pos -> cols:int -> pos (** Range of positions, as reported in error messages. We follow the lexing conventions of OCaml, i.e. [start_pos] points to the first character and [end_pos] points to the position just after the last character. This allow for instance to represent empty ranges with [start_pos = end_pos]. *) type range = { start_pos : pos ; end_pos : pos } [@@deriving_inline sexp_of] include sig [@@@ocaml.warning "-32"] val sexp_of_range : range -> Sexplib0.Sexp.t end [@@ocaml.doc "@inline"] [@@@end] val compare_range : range -> range -> int (** Make a range from two positions where both positions are inclusive, i.e. [start_pos] points to the first character and [end_pos] points to the last one. The character at [last_pos] is assumed to not be a newline character. *) val make_range_incl : start_pos:pos -> last_pos:pos -> range module Builder : sig type positions type t val create : ?initial_pos:pos -> unit -> t (** [add], [add_twice] and [add_newline] must be called with strictly increasing [offset] values. *) (** int is absolute offset of the position *) val add : t -> offset:int -> unit val add_twice : t -> offset:int -> unit (** int is absolute offset of the newline character *) val add_newline : t -> offset:int -> unit val contents : t -> positions val reset : t -> pos -> unit end with type positions := t (** Build the list of all positions in [t]. *) val to_list : t -> pos list (** Build the array of all positions in [t]. *) val to_array : t -> pos array (** [find t start stop] returns the range of positions starting at position with index [start] in [t] and ending at position with index [stop]. [find t i j] is the same as: {[ let a = to_array t in make_range_incl ~start_pos:a.(i) ~last_pos:a.(j) ]} but more efficient. *) val find : t -> int -> int -> range (** [find_sub_sexp_phys t sexp ~sub] looks for [sub] in [sexp] and return its location, assuming [t] is the sequence of positions associated with [sexp]. Comparison is done using physical equality. *) val find_sub_sexp_phys : t -> Sexp.t -> sub:Sexp.t -> range option val find_sub_sexp_in_list_phys : t -> Sexp.t list -> sub:Sexp.t -> range option (** Returns how much memory is used by [t] *) val memory_footprint_in_bytes : t -> int (** API for iterating over positions in an efficient way *) module Iterator : sig type positions = t type t val create : positions -> t (** Exception raised when the iterator has reached the end of the sequence. *) exception No_more (** [advance t ~skip] skips the next [skip] positions in the sequence, advance to the next position and return it. Raises [No_more] when reaching the end of the position set. *) val advance_exn : t -> skip:int -> pos (** Advance over a whole s-expression worth of positions. Returns the position range corresponding to that s-expression. *) val advance_sexp_exn : t -> Sexp.t -> range end with type positions := t parsexp-0.17.0/symbolic_automaton/000077500000000000000000000000001461647336100172045ustar00rootroot00000000000000parsexp-0.17.0/symbolic_automaton/src/000077500000000000000000000000001461647336100177735ustar00rootroot00000000000000parsexp-0.17.0/symbolic_automaton/src/automaton.ml000066400000000000000000000141551461647336100223420ustar00rootroot00000000000000open! Base include Automaton_intf module State = State module Quoted_string_transition = struct type t = | T of Action.t * State.Quoted_string.t | E of Epsilon_action.t * State.Quoted_string.t | Error of Parse_error_reason.t | End_of_quoted_string end type context = | In_block_comment | In_atom let quoted_string_transition : context -> State.Quoted_string.t * char -> Quoted_string_transition.t = fun context x -> (* Distinguishing atom and block comments is to optimize block comments. But we musn't optimize the exception on things like \321. *) let if_atom then_ else_ : Action.t = match context with | In_atom -> then_ | In_block_comment -> else_ in let if_atom_eps then_ else_ : Epsilon_action.t = match context with | In_atom -> then_ | In_block_comment -> else_ in match x with | Normal, '"' -> End_of_quoted_string | Normal, '\\' -> T (Add_token_char, After_backslash) | Normal, _ -> T (if_atom Add_quoted_atom_char Add_token_char, Normal) | After_backslash, '\n' -> T (Add_token_char, Ignoring_blanks) | After_backslash, '\r' -> T (Add_token_char, After_backslash_cr) | After_backslash, 'x' -> T (Add_token_char, After_backslash_x) | After_backslash, '0' .. '9' -> T (Add_dec_escape_char, After_backslash_digit) | After_backslash, _ -> T (if_atom Add_escaped Add_token_char, Normal) | After_backslash_cr, '\n' -> T (Add_token_char, Ignoring_blanks) | After_backslash_cr, _ -> E (if_atom_eps Add_escaped_cr Nop, Normal) | After_backslash_x, ('0' .. '9' | 'a' .. 'f' | 'A' .. 'F') -> T (if_atom Add_hex_escape_char Add_token_char, After_backslash_x_hex) | After_backslash_x, _ -> Error Unexpected_char_parsing_hex_escape | After_backslash_x_hex, ('0' .. '9' | 'a' .. 'f' | 'A' .. 'F') -> T (if_atom Add_last_hex_escape_char Add_token_char, Normal) | After_backslash_x_hex, _ -> Error Unexpected_char_parsing_hex_escape | After_backslash_digit, '0' .. '9' -> T (Add_dec_escape_char, After_backslash_2digits) | After_backslash_digit, _ -> Error Unexpected_char_parsing_dec_escape | After_backslash_2digits, '0' .. '9' -> T (if_atom Add_last_dec_escape_char Comment_add_last_dec_escape_char, Normal) | After_backslash_2digits, _ -> Error Unexpected_char_parsing_dec_escape | Ignoring_blanks, (' ' | '\t') -> T (Add_token_char, Ignoring_blanks) | Ignoring_blanks, _ -> E (Nop, Normal) ;; module Block_comment_transition = struct type t = | T of Action.t * State.Block_comment.t | E of Epsilon_action.t * State.Block_comment.t | Error of Parse_error_reason.t | End_comment end let block_comment_transition : State.Block_comment.t * char -> Block_comment_transition.t = function | Quoted_string state, c -> (match quoted_string_transition In_block_comment (state, c) with | End_of_quoted_string -> T (Add_token_char, Normal) | T (action, state) -> T (action, Quoted_string state) | E (action, state) -> E (action, Quoted_string state) | Error error -> Error error) | After_hash, '|' -> T (Start_block_comment, Normal) | After_pipe, '#' -> End_comment | _, '"' -> T (Add_token_char, Quoted_string Normal) | _, '|' -> T (Add_token_char, After_pipe) | _, '#' -> T (Add_token_char, After_hash) | _, _ -> T (Add_token_char, Normal) ;; let transition : State.t * char -> Transition.t = function | Whitespace, '(' -> T (Opening, Whitespace) | Whitespace, ')' -> T (Closing, Whitespace) | Whitespace, '\r' -> T (Nop, After_cr) | Whitespace, (' ' | '\t' | '\012' | '\n') -> T (Nop, Whitespace) | Whitespace, ';' -> T (Start_line_comment, Line_comment) | Whitespace, '"' -> T (Start_quoted_string, Quoted_string Normal) | Whitespace, '#' -> T (Nop, After_hash) | Whitespace, '|' -> T (Add_first_char, Unquoted_string After_pipe) | Whitespace, _ -> T (Add_first_char, Unquoted_string Normal) | After_cr, '\n' -> T (Nop, Whitespace) | After_cr, _ -> Error Unexpected_character_after_cr | Unquoted_string _, (';' | '(' | ')' | '"' | ' ' | '\t' | '\012' | '\r' | '\n') -> E (Push_atom, Whitespace) | Unquoted_string After_hash, '|' | Unquoted_string After_pipe, '#' -> Error Comment_token_in_unquoted_atom | Unquoted_string _, '#' -> T (Add_atom_char, Unquoted_string After_hash) | Unquoted_string _, '|' -> T (Add_atom_char, Unquoted_string After_pipe) | Unquoted_string _, _ -> T (Add_atom_char, Unquoted_string Normal) | Line_comment, ('\r' | '\n') -> E (End_line_comment, Whitespace) | Line_comment, _ -> T (Add_token_char, Line_comment) | After_hash, ';' -> T (Start_sexp_comment, Whitespace) | After_hash, '|' -> T (Start_block_comment, Block_comment Normal) | After_hash, _ -> E (Add_first_char_hash, Unquoted_string Normal) | Quoted_string state, c -> (match quoted_string_transition In_atom (state, c) with | End_of_quoted_string -> T (Push_quoted_atom, Whitespace) | T (action, state) -> T (action, Quoted_string state) | E (action, state) -> E (action, Quoted_string state) | Error error -> Error error) | Block_comment state, c -> (match block_comment_transition (state, c) with | T (action, state) -> T (action, Block_comment state) | E (action, state) -> E (action, Block_comment state) | End_comment -> End_block_comment | Error error -> Error error) | Error, _ -> Error Automaton_in_error_state ;; let transition_eoi : State.t -> Final_transition.t = function | Whitespace -> Eoi_check | After_cr -> Error Unexpected_character_after_cr | Unquoted_string _ -> E (Push_atom, Whitespace) | Line_comment -> E (End_line_comment, Whitespace) | After_hash -> E (Add_first_char_hash, Unquoted_string Normal) | Quoted_string _ -> Error Unterminated_quoted_string | Block_comment _ -> Error Unterminated_block_comment | Error -> Error Automaton_in_error_state ;; let action_to_runtime_function : Action.t -> string option = function | Nop -> None | t -> Some (String.uncapitalize (Action.Variants.to_name t)) ;; let epsilon_action_to_runtime_function : Epsilon_action.t -> string option = function | Nop -> None | End_line_comment -> Some "end_line_comment" | t -> Some ("eps_" ^ String.uncapitalize (Epsilon_action.Variants.to_name t)) ;; parsexp-0.17.0/symbolic_automaton/src/automaton.mli000066400000000000000000000000411461647336100225000ustar00rootroot00000000000000include Automaton_intf.Automaton parsexp-0.17.0/symbolic_automaton/src/automaton_intf.ml000066400000000000000000000037661461647336100233700ustar00rootroot00000000000000open! Base (** Action associated to transitions. Actions correspond to the similarly named functions in [Parsexp.Automaton_actions]. **) module Action = struct type t = | Nop | Opening | Closing | Add_atom_char | Add_quoted_atom_char | Add_first_char | Add_escaped | Add_hex_escape_char | Add_dec_escape_char | Add_last_hex_escape_char | Add_last_dec_escape_char | Add_token_char | Comment_add_last_dec_escape_char | Push_quoted_atom | Start_quoted_string | Start_block_comment | Start_sexp_comment | Start_line_comment [@@deriving compare, sexp_of, hash, variants] end (** Action associated to epsilon transitions, i.e. transitions that do not consume a character. Actions correspond to the similarly named functions in [Parsexp.Automaton_actions]. Having epsilon actions makes the definition of the automaton much simpler. **) module Epsilon_action = struct type t = | Nop | Push_atom | Add_first_char_hash | Add_escaped_cr | End_line_comment [@@deriving compare, sexp_of, hash, variants] end module Transition = struct type t = | T of Action.t * State.t | E of Epsilon_action.t * State.t | Error of Parse_error_reason.t | End_block_comment (* can't be a normal transition, as the new state isn't known statically *) [@@deriving compare] end module Final_transition = struct type t = | Eoi_check | E of Epsilon_action.t * State.t | Error of Parse_error_reason.t end module type S = sig val transition : State.t * char -> Transition.t val transition_eoi : State.t -> Final_transition.t end module type Automaton = sig module Action = Action module Epsilon_action = Epsilon_action module Transition = Transition module Final_transition = Final_transition module State = State module type S = S include S val action_to_runtime_function : Action.t -> string option val epsilon_action_to_runtime_function : Epsilon_action.t -> string option end parsexp-0.17.0/symbolic_automaton/src/dune000066400000000000000000000001361461647336100206510ustar00rootroot00000000000000(library (name parsexp_symbolic_automaton) (libraries base) (preprocess (pps ppx_jane))) parsexp-0.17.0/symbolic_automaton/src/parse_error_reason.ml000066400000000000000000000001571461647336100242220ustar00rootroot00000000000000open! Base include Parse_error_reason_intf include Parse_error_reason_intf.T let to_string = Variants.to_name parsexp-0.17.0/symbolic_automaton/src/parse_error_reason.mli000066400000000000000000000000631461647336100243670ustar00rootroot00000000000000include Parse_error_reason_intf.Parse_error_reason parsexp-0.17.0/symbolic_automaton/src/parse_error_reason_intf.ml000066400000000000000000000010341461647336100252350ustar00rootroot00000000000000open! Base (** Subset of the [Parsexp.Parse_error.Reason.t] type **) module T = struct type t = | Automaton_in_error_state | Comment_token_in_unquoted_atom | Unexpected_char_parsing_dec_escape | Unexpected_char_parsing_hex_escape | Unexpected_character_after_cr | Unterminated_block_comment | Unterminated_quoted_string [@@deriving compare, enumerate, hash, sexp_of, variants] end module type Parse_error_reason = sig include module type of struct include T end val to_string : t -> string end parsexp-0.17.0/symbolic_automaton/src/parsexp_symbolic_automaton.ml000066400000000000000000000020771461647336100260050ustar00rootroot00000000000000(** Abstract version of the parsing automaton. It is used in two places: - to define the automaton and generate parser code. - for tests At parser runtime, we instead use an integer for states and a table of functions for transitions *) (** It is possible that a modern and good parser generator can simplify the parser definition further. We didn't use an existing parser generator like ocamlyacc or menhir because we wanted to have a very precise control over performance and a good API for incremental parsing. At the time [Parsexp] was written, sexp parsers written using parser generators ran more slowly, allocated more, and were difficult to integrate with monadic IOs. [Sexplib] had both a lex/yacc generated parser and a handwritten one for performance. [Parsexp] had to match the performance of the handwritten parser (which we did, with less allocation). *) open! Base module Automaton = Automaton module Parse_error_reason = Parse_error_reason module Table = Table let table = Table.compile (module Automaton) parsexp-0.17.0/symbolic_automaton/src/state.ml000066400000000000000000000021421461647336100214440ustar00rootroot00000000000000open! Base include State_intf type t = | Whitespace | Error | After_cr | Unquoted_string of Unquoted_string.t | Line_comment | After_hash | Quoted_string of Quoted_string.t | Block_comment of Block_comment.t [@@deriving enumerate, compare, sexp_of] include Comparator.Make (struct type nonrec t = t [@@deriving compare, sexp_of] end) let to_int t = let rec loop i t l = match l with | [] -> assert false | x :: l -> if [%compare.equal: t] t x then i else loop (i + 1) t l in loop 0 t all ;; let of_int i = List.nth_exn all i let count = List.length all let initial = Whitespace (* This is assumed in parser_automaton_internal.ml *) let old_parser_approx_cont_state = function | Whitespace -> "Parsing_toplevel_whitespace" | After_cr -> "Parsing_nested_whitespace" | Unquoted_string _ | Quoted_string _ -> "Parsing_atom" | After_hash -> "Parsing_atom" | Block_comment _ -> "Parsing_block_comment" | Line_comment -> "Parsing_toplevel_whitespace" (* This cannot happen with the old parser so the result is a dummy value *) | Error -> "Parsing_toplevel_whitespace" ;; parsexp-0.17.0/symbolic_automaton/src/state.mli000066400000000000000000000000311461647336100216100ustar00rootroot00000000000000include State_intf.State parsexp-0.17.0/symbolic_automaton/src/state_intf.ml000066400000000000000000000022641461647336100224710ustar00rootroot00000000000000open! Base module Quoted_string = struct type t = | Normal | After_backslash | After_backslash_cr | After_backslash_digit | After_backslash_2digits | After_backslash_x | After_backslash_x_hex | Ignoring_blanks [@@deriving enumerate, compare, sexp_of] end module Block_comment = struct type t = | Normal | After_pipe | After_hash | Quoted_string of Quoted_string.t [@@deriving enumerate, compare, sexp_of] end module Unquoted_string = struct type t = | Normal | After_hash | After_pipe [@@deriving enumerate, compare, sexp_of] end module type State = sig module Block_comment = Block_comment module Quoted_string = Quoted_string module Unquoted_string = Unquoted_string type t = | Whitespace | Error | After_cr (** After '\r' *) | Unquoted_string of Unquoted_string.t | Line_comment | After_hash | Quoted_string of Quoted_string.t | Block_comment of Block_comment.t [@@deriving enumerate, compare, sexp_of] include Comparator.S with type t := t val count : int val initial : t val old_parser_approx_cont_state : t -> string val of_int : int -> t val to_int : t -> int end parsexp-0.17.0/symbolic_automaton/src/table.ml000066400000000000000000000032441461647336100214170ustar00rootroot00000000000000open! Base include Table_intf type t = { transitions : Transition.t Or_parse_error_reason.t array ; transitions_eoi : Automaton.Epsilon_action.t list Or_parse_error_reason.t array } [@@deriving sexp_of] let advance : char -> Advance.t = function | '\n' -> Advance_eol | _ -> Advance ;; let compile (module A : Automaton.S) : t = let rec squash acc state c : Transition.t Or_parse_error_reason.t = match A.transition (state, c) with | T (action, state) -> Ok { action = List.rev acc, action ; goto = State (State.to_int state) ; advance = advance c } | E (action, state) -> squash (action :: acc) state c | Error error -> Error error | End_block_comment -> Ok { action = List.rev acc, Nop; goto = End_block_comment; advance = advance c } in let rec squash_eoi acc state : Automaton.Epsilon_action.t list Or_parse_error_reason.t = match A.transition_eoi state with | Eoi_check -> Ok (List.rev acc) | E (eps_action, state) -> squash_eoi (eps_action :: acc) state | Error error -> Error error in let transitions = let dummy_transition : Transition.t = { action = [], Nop; goto = State 0; advance = Advance } in Array.create ~len:(State.count * 256) (Ok dummy_transition : _ Or_parse_error_reason.t) in let transitions_eoi = Array.create (Ok [] : _ Or_parse_error_reason.t) ~len:State.count in for s = 0 to State.count - 1 do let state = State.of_int s in for c = 0 to 255 do transitions.((s * 256) + c) <- squash [] state (Char.of_int_exn c) done; transitions_eoi.(s) <- squash_eoi [] state done; { transitions; transitions_eoi } ;; parsexp-0.17.0/symbolic_automaton/src/table.mli000066400000000000000000000000311461647336100215570ustar00rootroot00000000000000include Table_intf.Table parsexp-0.17.0/symbolic_automaton/src/table_intf.ml000066400000000000000000000022201461647336100224300ustar00rootroot00000000000000(** Compile [Automaton] to transition table. *) open! Base module Action = struct type t = Automaton.Epsilon_action.t list * Automaton.Action.t [@@deriving compare, sexp_of, hash] end module Goto_state = struct type t = | State of int | End_block_comment [@@deriving compare, sexp_of, hash] end module Advance = struct type t = | Advance | Advance_eol [@@deriving compare, sexp_of, hash] end module Transition = struct type t = { action : Action.t ; goto : Goto_state.t ; advance : Advance.t } [@@deriving compare, sexp_of, hash] end module Or_parse_error_reason = struct type 'a t = | Ok of 'a | Error of Parse_error_reason.t [@@deriving compare, sexp_of, hash] end module type Table = sig module Action = Action module Goto_state = Goto_state module Advance = Advance module Transition = Transition module Or_parse_error_reason = Or_parse_error_reason type t = { transitions : Transition.t Or_parse_error_reason.t array ; transitions_eoi : Automaton.Epsilon_action.t list Or_parse_error_reason.t array } [@@deriving sexp_of] val compile : (module Automaton.S) -> t end parsexp-0.17.0/test/000077500000000000000000000000001461647336100142535ustar00rootroot00000000000000parsexp-0.17.0/test/broken_phys_equal.ml000066400000000000000000000021161461647336100203170ustar00rootroot00000000000000open! Core type t = { name : string ; numbers : int list ; opt : string option } [@@deriving sexp, sexp_grammar] let%expect_test "Returns the incorrect sexp" = let fill char length = String.init length ~f:(const char) in let string = sexp_of_t { name = ""; numbers = []; opt = None } |> Sexp.to_string_mach in let sexps, positions = Parsexp.Many_and_positions.parse_string_exn string in let sub = match sexps with | [ List [ _; _; List [ Atom "opt"; opt ] ] ] -> opt | _ -> assert false in let range = Parsexp.Positions.find_sub_sexp_in_list_phys positions sexps ~sub |> Option.value_exn in let underline = String.concat [ fill ' ' range.start_pos.col; fill '^' (range.end_pos.col - range.start_pos.col) ] in print_endline ("| " ^ string); print_endline ("| " ^ underline); [%expect {| | ((name"")(numbers())(opt())) | ^^ |}]; print_s [%sexp (range : Parsexp.Positions.range)]; [%expect {| ((start_pos ((line 1) (col 24) (offset 24))) (end_pos ((line 1) (col 26) (offset 26)))) |}] ;; parsexp-0.17.0/test/broken_phys_equal.mli000066400000000000000000000000551461647336100204700ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/test/dune000066400000000000000000000002471461647336100151340ustar00rootroot00000000000000(library (name parsexp_test) (libraries base stdio parsexp parsexp_symbolic_automaton expect_test_helpers_core) (preprocess (pps ppx_jane))) (ocamllex lexer) parsexp-0.17.0/test/import.ml000066400000000000000000000024351461647336100161230ustar00rootroot00000000000000include Base include Stdio include Expect_test_helpers_core include Parsexp module Automaton_stack = Private.Automaton_stack let dummy_sexplib_error () = (* a hack to raise an arbitrary Parse_error *) let _ = Sexplib.Sexp.of_string ")" in assert false ;; let sexplib_sexps_of_string str : Sexp.t list = (* [ws_buf] must contain a single space character *) let feed_end_of_input ~this_parse ~ws_buf = (* When parsing atoms, the incremental parser cannot tell whether it is at the end until it hits whitespace. We therefore feed it one space to determine whether it is finished. *) match this_parse ~pos:0 ~len:1 ws_buf with | Sexplib.Sexp.Done (sexp, _) -> Ok sexp | Cont (cont_state, _) -> Error cont_state in let of_string_bigstring this_parse ws_buf str = let rec loop parse_pos = match this_parse str ~parse_pos with | Sexplib.Sexp.Done (sexp, parse_pos) -> sexp :: loop (Some parse_pos) | Cont (_, this_parse) -> (match feed_end_of_input ~this_parse ~ws_buf with | Ok sexp -> [ sexp ] | Error Sexplib.Sexp.Cont_state.Parsing_toplevel_whitespace -> [] | Error _ -> dummy_sexplib_error ()) in loop None in of_string_bigstring (fun x ~parse_pos -> Sexplib.Sexp.parse ?parse_pos x) " " str ;; parsexp-0.17.0/test/lexer.mll000066400000000000000000000006001461647336100160740ustar00rootroot00000000000000rule assoc sexp_parser = parse | eof { [] } | ' '* '\n' { assoc sexp_parser lexbuf } | '#' [^'\n']* '\n' { assoc sexp_parser lexbuf } | (['a'-'z']+ as key) ' '* '=' ' '* { let value = Parsexp.Eager.Lexbuf_consumer.parse sexp_parser lexbuf in eol lexbuf; (key, value) :: assoc sexp_parser lexbuf } and eol = parse | ' '* '\n' { () } parsexp-0.17.0/test/test_coverage.ml000066400000000000000000000456321461647336100174510ustar00rootroot00000000000000open! Import open Parsexp_symbolic_automaton.Automaton (* For the coverage tests, we write as input a list of string and expected s-expression. Each s-expression is parsed with both parsexp and sexplib and compared to the expected s-expression. We assert that both parsers agree with the expected result. While parsing, we record visited transition. At the end of all tests, we check that the full table is visited. These tests don't check that comments and sexps with layout are parsed correctly, they only test the main parser. *) (* Compute the classes of character that give the same transition whatever the state. *) module Char_class = struct open Parsexp_symbolic_automaton.Table module T = struct type t = Transition.t Or_parse_error_reason.t list [@@deriving compare, sexp_of, hash] end let compute (table : t) = let transitions_to_class = Hashtbl.create (module T) in let classes = Array.create 0 ~len:256 in for i = 0 to 255 do let transitions = List.init State.count ~f:(fun state -> table.transitions.((state * 256) + i)) in let cl = Hashtbl.find_or_add transitions_to_class transitions ~default:(fun () -> Hashtbl.length transitions_to_class) in classes.(i) <- cl done; Hashtbl.length transitions_to_class, classes ;; let count, class_table = compute Parsexp_symbolic_automaton.table let () = assert (count <= 256) let of_char ch = class_table.(Char.to_int ch) let%expect_test "display char classes" = let classes = Array.to_list class_table in for cl = 0 to count - 1 do let groups = List.filter_mapi classes ~f:(fun i cl' -> Option.some_if (cl = cl') i) |> List.group ~break:(fun a b -> a + 1 <> b) |> List.map ~f:(function | [] -> assert false | [ x ] -> Printf.sprintf "%C" (Char.of_int_exn x) | x :: l -> Printf.sprintf "%C..%C" (Char.of_int_exn x) (Option.value (List.last l) ~default:x |> Char.of_int_exn)) in let open Stdlib.Format in printf "@[<10>class %2d:%t@]@." cl (fun ppf -> List.iter groups ~f:(fprintf ppf "@ %s")) done; [%expect {| class 0: '\000'..'\b' '\011' '\014'..'\031' '!' '$'..'\'' '*'..'/' ':' '<'..'@' 'G'..'[' ']'..'`' 'g'..'w' 'y'..'{' '}'..'\255' class 1: '\t' ' ' class 2: '\n' class 3: '\012' class 4: '\r' class 5: '"' class 6: '#' class 7: '(' class 8: ')' class 9: '0'..'9' class 10: ';' class 11: 'A'..'F' 'a'..'f' class 12: '\\' class 13: 'x' class 14: '|' |}] ;; end module A = Private.Automaton type 'a result = | Sexp of 'a | Error [@@deriving compare, sexp_of] let transition_count = State.count * Char_class.count let transition_seen = Array.create false ~len:transition_count let transition_eoi_seen = Array.create false ~len:State.count let feed state char stack = transition_seen.((A.automaton_state state * Char_class.count) + Char_class.of_char char) <- true; A.feed state char stack ;; let feed_eoi state stack = transition_eoi_seen.(A.automaton_state state) <- true; A.feed_eoi state stack ;; let parse_string_gen mode s = let state = A.create mode Sexp in let len = String.length s in let rec loop pos stack = if pos = len then feed_eoi state stack else ( let stack = feed state s.[pos] stack in loop (pos + 1) stack) in loop 0 Automaton_stack.empty ;; let parse_string s = Automaton_stack.get_single (parse_string_gen Single s) let parse_string_many s = Automaton_stack.get_many (parse_string_gen Many s) let cases_where_sexplib_disagree_with_itself = ref [] let add_to_list cell x = cell := x :: !cell let test_one_case_dealing_with_a_single_sexp (str, expected) = let parsexp = match parse_string str with | sexp -> Ok sexp | exception Parse_error.Parse_error error -> Error error in let matches_expected = match parsexp, expected with | Ok a, Sexp b -> Sexp.equal a b | Error _, Error -> true | _ -> false in if not matches_expected then Stdlib.Format.printf "round-trip failure:@.input: %S@.expected: %a@.got: %a@.@." str Sexp.pp_hum [%sexp (expected : Sexp.t result)] Sexp.pp_hum [%sexp (parsexp : (Sexp.t, Parse_error.t) Result.t)]; let sexplib = match Sexplib.Sexp.of_string str with | sexp -> Ok sexp | exception exn -> Error exn in let matches_sexplib = match parsexp, sexplib with | Ok a, Ok b -> Sexp.equal a b | Error _, Error _ -> true | _ -> false in if not matches_sexplib then Stdlib.Format.printf "parsexp and sexplib disagree:@.input: %S@.parsexp: %a@.sexplib: %a@.@." str Sexp.pp_hum [%sexp (parsexp : (Sexp.t, Parse_error.t) Result.t)] Sexp.pp_hum [%sexp (sexplib : (Sexp.t, exn) Result.t)]; let sexplib_lexer = match Sexplib.Sexp.scan_sexps (Lexing.from_string str) with | [ sexp ] -> Ok sexp | exception exn -> Error exn | [] | _ :: _ :: _ -> dummy_sexplib_error () in let matches = match sexplib, sexplib_lexer with | Ok a, Ok b -> Sexp.equal a b | Error _, Error _ -> true | _ -> false in if not matches then add_to_list cases_where_sexplib_disagree_with_itself (str, sexplib, sexplib_lexer) ;; let test_one_case_dealing_with_many_sexps (str, expected) = let parsexp = match parse_string_many str with | sexps -> Ok sexps | exception Parse_error.Parse_error error -> Error error in let matches_expected = match parsexp, expected with | Ok a, Sexp b -> [%compare.equal: Sexp.t list] a b | Error _, Error -> true | _ -> false in if not matches_expected then Stdlib.Format.printf "round-trip failure:@.input: %S@.expected: %a@.got: %a@.@." str Sexp.pp_hum [%sexp (expected : Sexp.t list result)] Sexp.pp_hum [%sexp (parsexp : (Sexp.t list, Parse_error.t) Result.t)]; let sexplib = match sexplib_sexps_of_string str with | sexps -> Ok sexps | exception (Sexplib.Sexp.Parse_error _ as exn) -> Error exn in let matches_expected = match sexplib, expected with | Ok a, Sexp b -> [%compare.equal: Sexp.t list] a b | Error _, Error -> true | _ -> false in if not matches_expected then Stdlib.Format.printf "sexplib does not match expected:@.input: %S@.expected: %a@.sexplib: %a@.@." str Sexp.pp_hum [%sexp (expected : Sexp.t list result)] Sexp.pp_hum [%sexp (sexplib : (Sexp.t list, exn) Result.t)] ;; let witness_for_class cl = let rec loop i = if i = 256 then raise Stdlib.Not_found; if Char_class.of_char (Char.of_int_exn i) = cl then Char.of_int_exn i else loop (i + 1) in let ch = loop 0 in match ch with | '\000' -> if Char_class.of_char 'z' = cl then 'z' else ch | _ -> ch ;; let some_cases : (string * Sexp.t result) list = [ "foo", Sexp (Atom "foo") ; " foo", Sexp (Atom "foo") ; " zoo", Sexp (Atom "zoo") ; " 123", Sexp (Atom "123") ; "(foo bar)", Sexp (List [ Atom "foo"; Atom "bar" ]) ; "\\", Sexp (Atom "\\") ; "|", Sexp (Atom "|") ; "#", Sexp (Atom "#") ; "; foo\nx", Sexp (Atom "x") ; "; foo\r\nx", Sexp (Atom "x") ; "(# )", Sexp (List [ Atom "#" ]) ; "##", Sexp (Atom "##") ; "#|x|#z", Sexp (Atom "z") ; "(#\n)", Sexp (List [ Atom "#" ]) ; "(#\r\n)", Sexp (List [ Atom "#" ]) ; "#x", Sexp (Atom "#x") ; "(#())", Sexp (List [ Atom "#"; List [] ]) ; "(#)", Sexp (List [ Atom "#" ]) ; "(#\"x\")", Sexp (List [ Atom "#"; Atom "x" ]) ; "#; x z", Sexp (Atom "z") ; "foo#", Sexp (Atom "foo#") ; "(foo# )", Sexp (List [ Atom "foo#" ]) ; "(foo#\n)", Sexp (List [ Atom "foo#" ]) ; "(foo#\r\n)", Sexp (List [ Atom "foo#" ]) ; "foo#x", Sexp (Atom "foo#x") ; "foo##", Sexp (Atom "foo##") ; "(foo#\"bar\")", Sexp (List [ Atom "foo#"; Atom "bar" ]) ; "(foo#())", Sexp (List [ Atom "foo#"; List [] ]) ; "(foo#)", Sexp (List [ Atom "foo#" ]) ; "(foo#;\n)", Sexp (List [ Atom "foo#" ]) ; "foo#|", Error ; "foo|#", Error ; "foo|", Sexp (Atom "foo|") ; "foo|x", Sexp (Atom "foo|x") ; "(foo| )", Sexp (List [ Atom "foo|" ]) ; "(foo|\nz)", Sexp (List [ Atom "foo|"; Atom "z" ]) ; "(foo|\r\nz)", Sexp (List [ Atom "foo|"; Atom "z" ]) ; "(foo|\"bar\")", Sexp (List [ Atom "foo|"; Atom "bar" ]) ; "(foo|())", Sexp (List [ Atom "foo|"; List [] ]) ; "(foo|)", Sexp (List [ Atom "foo|" ]) ; "(foo|;\n)", Sexp (List [ Atom "foo|" ]) ; "foo||", Sexp (Atom "foo||") ; "(", Error ; ")", Error ; "\r", Error ; "\"", Error ; "x(", Error ; "x)", Error ; "x\r", Error ; "x\"", Error ; "\rx", Error ; "#|\n|#z", Sexp (Atom "z") ; "#|\"foo\"|#z", Sexp (Atom "z") ; "#| #| x |# |#z", Sexp (Atom "z") ; "#| \"x |#\" |#z", Sexp (Atom "z") ; "#| #x |#z", Sexp (Atom "z") ; "#| #\n |#z", Sexp (Atom "z") ; "#| ## |#z", Sexp (Atom "z") ; "#| #\"x\" |#z", Sexp (Atom "z") ; "#| |x |#z", Sexp (Atom "z") ; "#| |\n |#z", Sexp (Atom "z") ; "#| |\"x\" |#z", Sexp (Atom "z") ; "#| || |#z", Sexp (Atom "z") ; "#| \"\n\" |#z", Sexp (Atom "z") ; "#| \"\\n\" |#z", Sexp (Atom "z") ; "#| \"\\z\" |#z", Sexp (Atom "z") ; "#| \"\\\n\" |#z", Sexp (Atom "z") ; "#| \"\\\r\n\" |#z", Sexp (Atom "z") ; "#| \"\\123\" |#z", Sexp (Atom "z") ; "#| \"\\x42\" |#z", Sexp (Atom "z") ; "#| \"\\\rx\" |#z", Sexp (Atom "z") ; "#| \"\\\r\" |#z", Sexp (Atom "z") ; "#| \"\\\r\\n\" |#z", Sexp (Atom "z") ; "#| \"\\1z\" |#z", Error ; "#| \"\\12z\" |#z", Error ; "#| \"\\xz\" |#z", Error ; "#| \"\\x1z\" |#z", Error ; "#| \"\\\n x\" |#z", Sexp (Atom "z") ; "#| \"\\\n \t\" |#z", Sexp (Atom "z") ; "#| \"\\\n \n\" |#z", Sexp (Atom "z") ; "#| \"\\\n \\z\" |#z", Sexp (Atom "z") ; "\"", Error ; "\"\\", Error ; "\"\\\r", Error ; "\"\\1", Error ; "\"\\12", Error ; "\"\\x", Error ; "\"\\x1", Error ; "\"\\\n ", Error ; "#|", Error ; "#| #", Error ; "#| |", Error ; "#| \"", Error ; "#| \"\\", Error ; "#| \"\\\r", Error ; "#| \"\\1", Error ; "#| \"\\12", Error ; "#| \"\\x", Error ; "#| \"\\x1", Error ; "#| \"\\\n ", Error ; "\012z", Sexp (Atom "z") ; "\rz", Error ; "\r\t", Error ; "\r\012", Error ; "\r\r", Error ; "\r\"\"", Error ; "\r#", Error ; "\r()", Error ; "(\r)", Error ; "\r123", Error ; "\r;", Error ; "\rA", Error ; "\r\\", Error ; "\r|", Error ; "(abc\012)", Sexp (List [ Atom "abc" ]) ; "abcA", Sexp (Atom "abcA") ; "abc\\", Sexp (Atom "abc\\") ; "ax", Sexp (Atom "ax") ; "a#z", Sexp (Atom "a#z") ; "(a#\012)", Sexp (List [ Atom "a#" ]) ; "a#0", Sexp (Atom "a#0") ; "a#A", Sexp (Atom "a#A") ; "a#\\", Sexp (Atom "a#\\") ; "a|z", Sexp (Atom "a|z") ; "(a|\012)", Sexp (List [ Atom "a|" ]) ; "a|0", Sexp (Atom "a|0") ; "a|A", Sexp (Atom "a|A") ; "a|\\", Sexp (Atom "a|\\") ; ";\012\nz", Sexp (Atom "z") ; ";\"\nz", Sexp (Atom "z") ; ";#\nz", Sexp (Atom "z") ; ";(\nz", Sexp (Atom "z") ; ";)\nz", Sexp (Atom "z") ; ";0\nz", Sexp (Atom "z") ; ";;\nz", Sexp (Atom "z") ; ";\\\nz", Sexp (Atom "z") ; ";x\nz", Sexp (Atom "z") ; ";|\nz", Sexp (Atom "z") ; "#z", Sexp (Atom "#z") ; "(#\012)", Sexp (List [ Atom "#" ]) ; "#0", Sexp (Atom "#0") ; "#A", Sexp (Atom "#A") ; "#\\", Sexp (Atom "#\\") ; "#|z|#z", Sexp (Atom "z") ; "#|x|#z", Sexp (Atom "z") ; "#|\t|#z", Sexp (Atom "z") ; "#|\012|#z", Sexp (Atom "z") ; "#|(|#z", Sexp (Atom "z") ; "#|)|#z", Sexp (Atom "z") ; "#|;|#z", Sexp (Atom "z") ; "#|A|#z", Sexp (Atom "z") ; "#|||#z", Sexp (Atom "z") ; "#||#z", Sexp (Atom "z") ; "#|\r|#z", Sexp (Atom "z") ; "#|0|#z", Sexp (Atom "z") ; "#|\\|#z", Sexp (Atom "z") ; "#| |z|#z", Sexp (Atom "z") ; "#| |x|#z", Sexp (Atom "z") ; "#| |\t|#z", Sexp (Atom "z") ; "#| |\012|#z", Sexp (Atom "z") ; "#| |(|#z", Sexp (Atom "z") ; "#| |)|#z", Sexp (Atom "z") ; "#| |;|#z", Sexp (Atom "z") ; "#| |A|#z", Sexp (Atom "z") ; "#| |||#z", Sexp (Atom "z") ; "#| ||#z", Sexp (Atom "z") ; "#| |\r|#z", Sexp (Atom "z") ; "#| |0|#z", Sexp (Atom "z") ; "#| |\\|#z", Sexp (Atom "z") ; "#| #z|#z", Sexp (Atom "z") ; "#| #x|#z", Sexp (Atom "z") ; "#| #\t|#z", Sexp (Atom "z") ; "#| #\012|#z", Sexp (Atom "z") ; "#| #(|#z", Sexp (Atom "z") ; "#| #)|#z", Sexp (Atom "z") ; "#| #;|#z", Sexp (Atom "z") ; "#| #A|#z", Sexp (Atom "z") ; "#| #\r|#z", Sexp (Atom "z") ; "#| #0|#z", Sexp (Atom "z") ; "#| #\\|#z", Sexp (Atom "z") ] ;; let quoted_string_cases : (string * Sexp.t result) list = [ "\"foo\\\\bar\n1\r\n2\\\n3\\\r\n4\\\n 5\"", Sexp (Atom "foo\\bar\n1\r\n2345") ; "\"\\\r\"", Sexp (Atom "\r") ; "\"\\\n \"", Sexp (Atom "") ; "\"\\\n \n\"", Sexp (Atom "\n") ; "\"\\\n \\123\"", Sexp (Atom "\123") ; {|"\123"|}, Sexp (Atom "\123") ; {|"\x42"|}, Sexp (Atom "\x42") ; "\"\\\rx\"", Sexp (Atom "\rx") ; "\"\\\r\"", Sexp (Atom "\r") ; "\"\\\r\\123\"", Sexp (Atom "\r\123") ; "\"\\1z\"", Error ; "\"\\12z\"", Error ; "\"\\xz\"", Error ; "\"\\x1z\"", Error ; "\"\t\"", Sexp (Atom "\t") ; "\"\012\"", Sexp (Atom "\012") ; "\"#\"", Sexp (Atom "#") ; "\"(\"", Sexp (Atom "(") ; "\")\"", Sexp (Atom ")") ; "\";\"", Sexp (Atom ";") ; "\"|\"", Sexp (Atom "|") ; "\"\\z\"", Sexp (Atom "\\z") ; "\"\\ \"", Sexp (Atom "\\ ") ; "\"\\\t\"", Sexp (Atom "\\\t") ; "\"\\\012\"", Sexp (Atom "\\\012") ; "\"\\#\"", Sexp (Atom "\\#") ; "\"\\(\"", Sexp (Atom "\\(") ; "\"\\)\"", Sexp (Atom "\\)") ; "\"\\;\"", Sexp (Atom "\\;") ; "\"\\A\"", Sexp (Atom "\\A") ; "\"\\|\"", Sexp (Atom "\\|") ; "\"\\\"", Error ; "\"\\\rz\"", Sexp (Atom "\rz") ; "\"\\\r\t\"", Sexp (Atom "\r\t") ; "\"\\\r\012\"", Sexp (Atom "\r\012") ; "\"\\\r#\"", Sexp (Atom "\r#") ; "\"\\\r(\"", Sexp (Atom "\r(") ; "\"\\\r)\"", Sexp (Atom "\r)") ; "\"\\\r;\"", Sexp (Atom "\r;") ; "\"\\\rA\"", Sexp (Atom "\rA") ; "\"\\\r|\"", Sexp (Atom "\r|") ; "\"\\\r\"", Sexp (Atom "\r") ; "\"\\\r\r\"", Sexp (Atom "\r\r") ; "\"\\\r0\"", Sexp (Atom "\r0") ; "\"\\1z\"", Error ; "\"\\1\t\"", Error ; "\"\\1\012\"", Error ; "\"\\1#\"", Error ; "\"\\1(\"", Error ; "\"\\1)\"", Error ; "\"\\1;\"", Error ; "\"\\1A\"", Error ; "\"\\1|\"", Error ; "\"\\1\"", Error ; "\"\\1\r\"", Error ; "\"\\1\n\"", Error ; "\"\\1\\\"", Error ; "\"\\1x\"", Error ; "\"\\12z\"", Error ; "\"\\12\t\"", Error ; "\"\\12\012\"", Error ; "\"\\12#\"", Error ; "\"\\12(\"", Error ; "\"\\12)\"", Error ; "\"\\12;\"", Error ; "\"\\12A\"", Error ; "\"\\12|\"", Error ; "\"\\12\"", Error ; "\"\\12\r\"", Error ; "\"\\12\n\"", Error ; "\"\\12\\\"", Error ; "\"\\12x\"", Error ; "\"\\xz\"", Error ; "\"\\x\t\"", Error ; "\"\\x\012\"", Error ; "\"\\x#\"", Error ; "\"\\x(\"", Error ; "\"\\x)\"", Error ; "\"\\x;\"", Error ; "\"\\xA\"", Error ; "\"\\x|\"", Error ; "\"\\x\"", Error ; "\"\\x\r\"", Error ; "\"\\x\n\"", Error ; "\"\\x\\\"", Error ; "\"\\xx\"", Error ; "\"\\x1z\"", Error ; "\"\\x1\t\"", Error ; "\"\\x1\012\"", Error ; "\"\\x1#\"", Error ; "\"\\x1(\"", Error ; "\"\\x1)\"", Error ; "\"\\x1;\"", Error ; "\"\\x1A\"", Sexp (Atom "\x1a") ; "\"\\x1|\"", Error ; "\"\\x1\"", Error ; "\"\\x1\r\"", Error ; "\"\\x1\n\"", Error ; "\"\\x1\\\"", Error ; "\"\\x1x\"", Error ; "\"\\\nz\"", Sexp (Atom "z") ; "\"\\\nx\"", Sexp (Atom "x") ; "\"\\\n\t\"", Sexp (Atom "") ; "\"\\\n\012\"", Sexp (Atom "\012") ; "\"\\\n#\"", Sexp (Atom "#") ; "\"\\\n(\"", Sexp (Atom "(") ; "\"\\\n)\"", Sexp (Atom ")") ; "\"\\\n;\"", Sexp (Atom ";") ; "\"\\\nA\"", Sexp (Atom "A") ; "\"\\\n|\"", Sexp (Atom "|") ; "\"\\\n\"", Sexp (Atom "") ; "\"\\\n\r\"", Sexp (Atom "\r") ; "\"\\\n0\"", Sexp (Atom "0") ] ;; let trailing_whitespace_cases : (string * Sexp.t result) list = [ "x ", Sexp (Atom "x"); "x\n", Sexp (Atom "x"); "x;", Sexp (Atom "x") ] ;; let cases = some_cases @ quoted_string_cases @ trailing_whitespace_cases @ List.map quoted_string_cases ~f:(fun (s, result) -> ( "#| " ^ s ^ " |#blah" , match result with | Sexp _ -> Sexp (Sexp.Atom "blah") | Error -> Error )) ;; let zero_sexp_cases : (string * Sexp.t list result) list = [ " ", Sexp []; "\n", Sexp []; ";", Sexp [] ] ;; let%expect_test "coverage" = List.iter cases ~f:(fun (input, result) -> let result_many = match result with | Sexp x -> Sexp [ x ] | Error -> Error in test_one_case_dealing_with_a_single_sexp (input, result); test_one_case_dealing_with_many_sexps (input, result_many)); List.iter zero_sexp_cases ~f:test_one_case_dealing_with_many_sexps; (* Transition after an error *) let after_error () = let state = A.create Many Sexp in match feed state ')' Automaton_stack.empty with | (_ : Automaton_stack.t) -> assert false | exception _ -> state in List.iter Char.all ~f:(fun c -> let state = after_error () in match feed state c Automaton_stack.empty with | (_ : Automaton_stack.t) -> assert false | exception _ -> ()); (let state = after_error () in match feed_eoi state Automaton_stack.empty with | (_ : Automaton_stack.t) -> assert false | exception _ -> ()); for state = 0 to State.count - 1 do for cl = 0 to Char_class.count - 1 do if not transition_seen.((state * Char_class.count) + cl) then printf !"state %{sexp:State.t}, feed %C\n" (State.of_int state) (witness_for_class cl) done done; for s = 0 to State.count - 1 do if not transition_eoi_seen.(s) then printf !"state %{sexp:State.t}, feed EOI\n" (State.of_int s) done; (* This output represent the list of transition not visited. It must be empty. *) [%expect {| |}]; printf "%d of %d" (List.length !cases_where_sexplib_disagree_with_itself) (List.length cases); [%expect "131 of 409"]; List.iter !cases_where_sexplib_disagree_with_itself ~f:(fun (input, sexplib, sexplib_lexer) -> let suppress = (* there are many examples where continuation-based parser fails and lexer succeeds so we filter these out: lexer interprets broken hex and decimal escape sequences as if they are not escape sequences at all Also lexer interprets "\\\r" differently: it keeps the slash, but we drop the slash *) String.is_substring ~substring:{|"\x|} input || String.is_substring ~substring:{|"\1|} input || String.is_substring ~substring:({|"\|} ^ "\r") input in if not suppress then Stdlib.Format.printf "the various sexplib parsers disagree between themselves on this \ case:@.input: %S@.sexplib: %a@.sexplib_lexer: %a@.@." input Sexp.pp_hum [%sexp (sexplib : (Sexp.t, exn) Result.t)] Sexp.pp_hum [%sexp (sexplib_lexer : (Sexp.t, exn) Result.t)]); [%expect {| the various sexplib parsers disagree between themselves on this case: input: "\"\\ \"" sexplib: (Ok "\\ ") sexplib_lexer: (Ok " ") |}] ;; parsexp-0.17.0/test/test_coverage.mli000066400000000000000000000000551461647336100176100ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/test/test_cst.ml000066400000000000000000000016571461647336100164460ustar00rootroot00000000000000open! Import let parse_and_print input = let x = Many_cst.parse_string_exn input in print_s [%sexp (x : Cst.t_or_comment list)] ;; let%expect_test "escape sequence in block comment" = parse_and_print {|#| "\255" |#|}; [%expect {| (( Comment ( Plain_comment (loc ( (start_pos ( (line 1) (col 0) (offset 0))) (end_pos ( (line 1) (col 12) (offset 12))))) (comment "#| \"\\255\" |#")))) |}] ;; let%expect_test "quoted atom" = parse_and_print {| "foo bar" |}; [%expect {| (( Sexp ( Atom (loc ( (start_pos ( (line 1) (col 1) (offset 1))) (end_pos ( (line 1) (col 10) (offset 10))))) (atom "foo bar") (unescaped ("\"foo bar\""))))) |}] ;; parsexp-0.17.0/test/test_cst.mli000066400000000000000000000000551461647336100166060ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/test/test_parsexp.ml000066400000000000000000000305341461647336100173330ustar00rootroot00000000000000open! Import let test s = Single.parse_string s |> [%sexp_of: (Sexp.t, Parse_error.t) Result.t] |> print_s ;; let chars_of_string str = Queue.of_array (String.to_array str) let%expect_test "unterminated sexp" = test "(abc"; [%expect {| (Error ( (position ( (line 1) (col 4) (offset 4))) (message "unclosed parentheses at end of input"))) |}]; test " "; [%expect {| (Error ( (position ( (line 1) (col 2) (offset 2))) (message "no s-expression found in input"))) |}] ;; let%expect_test "parsexp bug: it accepts invalid syntax" = test "(#;)"; [%expect {| (Error ( (position ( (line 1) (col 3) (offset 3))) (message "unterminated sexp comment"))) |}]; test "#;(#;) a"; [%expect {| (Error ( (position ( (line 1) (col 5) (offset 5))) (message "unterminated sexp comment"))) |}]; test "#;"; [%expect {| (Error ( (position ( (line 1) (col 2) (offset 2))) (message "unterminated sexp comment"))) |}]; test "#;(#;q) (a)"; [%expect {| (Ok (a)) |}]; test "#;(#;(q)) (a)"; [%expect {| (Ok (a)) |}]; test "#;(#;(q)) a"; [%expect {| (Ok a) |}]; test "#;(#;((q))) a"; [%expect {| (Ok a) |}]; test "#;#;(#;x)y a"; [%expect {| (Ok a) |}] ;; let%expect_test "regression test (used to raise s-expression followed by data)" = test "a #;0"; [%expect {| (Ok a) |}] ;; let parse_eager_cst ~no_sexp_is_error str = let r = ref [] in let state = Eager_cst.State.create ~no_sexp_is_error (fun _ v -> r := v :: !r) in Result.try_with (fun () -> let stack = Eager_cst.Stack.empty in let stack = Eager_cst.feed_string state str stack in Eager_cst.feed_eoi state stack; List.rev !r) ;; let parse_eager_cst_no_sexp_is_error = parse_eager_cst ~no_sexp_is_error:true let parse_eager_cst = parse_eager_cst ~no_sexp_is_error:false let%expect_test "regression test (we didn't run the callback on comments, resulting in \ missing comments or assertion failures)" = let test s = parse_eager_cst s |> [%sexp_of: (Cst.t_or_comment list, exn) Result.t] |> print_s in test ";"; [%expect {| (Ok (( Comment ( Plain_comment (loc ( (start_pos ( (line 1) (col 0) (offset 0))) (end_pos ( (line 1) (col 1) (offset 1))))) (comment ";"))))) |}]; test "#||#"; [%expect {| (Ok (( Comment ( Plain_comment (loc ( (start_pos ( (line 1) (col 0) (offset 0))) (end_pos ( (line 1) (col 4) (offset 4))))) (comment "#||#"))))) |}]; test "#;#;a b"; [%expect {| (Ok (( Comment ( Sexp_comment (hash_semi_pos ( (line 1) (col 0) (offset 0))) (comments (( Sexp_comment (hash_semi_pos ( (line 1) (col 2) (offset 2))) (comments ()) (sexp ( Atom (loc ( (start_pos ( (line 1) (col 4) (offset 4))) (end_pos ( (line 1) (col 5) (offset 5))))) (atom a) (unescaped (a))))))) (sexp ( Atom (loc ( (start_pos ( (line 1) (col 6) (offset 6))) (end_pos ( (line 1) (col 7) (offset 7))))) (atom b) (unescaped (b)))))))) |}]; test "a;\nb"; [%expect {| (Ok ( (Sexp ( Atom (loc ( (start_pos ( (line 1) (col 0) (offset 0))) (end_pos ( (line 1) (col 1) (offset 1))))) (atom a) (unescaped (a)))) (Comment ( Plain_comment (loc ( (start_pos ( (line 1) (col 1) (offset 1))) (end_pos ( (line 1) (col 2) (offset 2))))) (comment ";"))) (Sexp ( Atom (loc ( (start_pos ( (line 2) (col 0) (offset 3))) (end_pos ( (line 2) (col 1) (offset 4))))) (atom b) (unescaped (b)))))) |}] ;; let%expect_test "regression test (we counted comments as sexps for the purpose of \ ~no_sexp_is_error" = let test s = parse_eager_cst_no_sexp_is_error s |> [%sexp_of: (Cst.t_or_comment list, exn) Result.t] |> print_s in test ";"; [%expect {| (Error ( parse_error.ml.Parse_error ( (position ( (line 1) (col 1) (offset 1))) (message "no s-expression found in input")))) |}]; test "#||#"; [%expect {| (Error ( parse_error.ml.Parse_error ( (position ( (line 1) (col 4) (offset 4))) (message "no s-expression found in input")))) |}]; test "#;#;a b"; [%expect {| (Error ( parse_error.ml.Parse_error ( (position ( (line 1) (col 7) (offset 7))) (message "no s-expression found in input")))) |}] ;; module P = Eager let rec hot_loop state stream stack = match Queue.peek stream with | None -> P.feed_eoi state stack | Some char -> let stack = P.feed state char stack in Queue.dequeue_and_ignore_exn stream; hot_loop state stream stack ;; exception Got_sexp of Sexp.t let fetch_sexp (stream : char Queue.t) = let got_sexp _state sexp = Exn.raise_without_backtrace (Got_sexp sexp) in let count = Queue.length stream in let state = P.State.create ~reraise_notrace:true got_sexp in match hot_loop state stream P.Stack.empty with | () -> None | exception Got_sexp sexp -> (* This test is true if the s-expression includes the last character passed to the parser *) if P.State.offset state > count - Queue.length stream then Queue.dequeue_and_ignore_exn stream; Some sexp ;; let iter_sexps (stream : char Queue.t) ~f = let got_sexp _state sexp = f sexp in let state = P.State.create got_sexp in hot_loop state stream P.Stack.empty ;; let input = {| (Hello World) (a b c) "Hello world" a(b)c"d" (1 (2 3)) |} let%expect_test "eager parser raise" = let stream = chars_of_string input in let rec loop stream = match fetch_sexp stream with | None -> assert (Queue.is_empty stream) | Some sexp -> Stdlib.Format.printf "got: %a@." Sexp.pp_hum sexp; loop stream in loop stream; [%expect {| got: (Hello World) got: (a b c) got: "Hello world" got: a got: (b) got: c got: d got: (1 (2 3)) |}] ;; let all_short_strings () = let all_chars = (* should have an example from every character class (see [Char_class] in test_coverage.ml) *) [ '\000' ; '\001' ; '\t' ; ' ' ; '\n' ; '\012' ; '\r' ; '"' ; '#' ; '(' ; ')' ; '8' ; ';' ; 'A' ; 'b' ; '\\' ; 'x' ; '|' ] in List.bind [ 0; 1; 2; 3; 4 ] ~f:(fun len -> List.all (List.init len ~f:(fun _ -> all_chars))) |> List.map ~f:String.of_char_list ;; (* A thorough test of [Eager] semantics expressed in terms of [Many]. - feed chars one by one, - note when sexps are detected, - assert that the position reported is how many chars we fed or 1 fewer - assert that the resulting chunks if we split at those positions can be parsed to the same sexps as the sexps being reported - assert that the whole thing parses iff it parses with [Many] *) let%expect_test "eager parser semantics" = let exception Got_sexp of { offset : int ; sexp : Sexp.t } in let got_sexp state sexp = Exn.raise_without_backtrace (Got_sexp { sexp; offset = P.State.Read_only.offset state }) in let inputs = all_short_strings () in printf "testing %d inputs\n" (List.length inputs); List.iter inputs ~f:(fun input -> let rec go start state stack i = if Int.( = ) i (String.length input) then ( match P.feed_eoi state stack with | exception Got_sexp { offset; sexp } -> let ate = P.State.offset state in assert (Int.( = ) offset ate); let fed = i - start in assert (List.mem [ fed ] ~equal:Int.( = ) ate); (match Single.parse_string (String.sub ~pos:start ~len:ate input) with | Error _ -> assert false | Ok sexp2 -> assert (Sexp.( = ) sexp sexp2); Ok ()) | exception exn -> Error (i, exn) | () -> (match Many.parse_string (String.sub ~pos:start ~len:(i - start) input) with | Ok [] -> Ok () | Ok (_ :: _) -> failwith "lost a sexp" | Error error -> raise_s [%message "succeeded with Eager, but failed with Many" ~pos:(start : int) ~len:(i - start : int) (error : Parse_error.t)])) else ( match P.feed state input.[i] stack with | exception Got_sexp { sexp; offset } -> let ate = P.State.offset state in assert (Int.( = ) offset ate); let fed = i + 1 - start in assert (List.mem [ fed; fed - 1 ] ~equal:Int.( = ) ate); let end_ = start + ate in (match Single.parse_string (String.sub ~pos:start ~len:ate input) with | Error _ -> raise_s [%message "Eager gave us a sexp that won't parse" (start : int) (i : int)] | Ok sexp2 -> assert (Sexp.( = ) sexp sexp2); let state = P.State.create got_sexp in go end_ state P.Stack.empty end_) | exception exn -> Error (i, exn) | stack -> go start state stack (i + 1)) in let state = P.State.create got_sexp in match go 0 state P.Stack.empty 0 with | Ok () -> () | Error exn -> (match Many.parse_string input with | Ok result_from_many -> print_s [%message "failed to parse with Eager, but succeeded with Many" (input : string) (exn : int * Exn.t) (result_from_many : Sexp.t list)] | Error _bad -> ()) | exception bad_thing -> print_s [%message "Bad" (input : string) (bad_thing : Exn.t)]); [%expect {| testing 111151 inputs |}] ;; let%expect_test "eager parser continue" = let stream = chars_of_string input in iter_sexps stream ~f:(Stdlib.Format.printf "got: %a@." Sexp.pp_hum); [%expect {| got: (Hello World) got: (a b c) got: "Hello world" got: a got: (b) got: c got: d got: (1 (2 3)) |}] ;; let%expect_test "eager parser incorrect mutation" = let stream = chars_of_string input in let state = ref (P.State.create (fun _ _ -> assert false)) in let got_sexp _state _sexp = P.State.reset !state in state := P.State.create got_sexp; show_raise ~hide_positions:true (fun () -> hot_loop !state stream P.Stack.empty); [%expect {| (raised "Assert_failure automaton_action.ml:LINE:COL") |}] ;; let%expect_test "eager parser feed after raise without reset" = let stream = chars_of_string input in let got_sexp _state _sexp = raise Stdlib.Exit in let state = P.State.create got_sexp in (try hot_loop state stream P.Stack.empty with | Stdlib.Exit -> ()); show_raise (fun () -> hot_loop state stream P.Stack.empty); [%expect {| (raised (Failure "Parsexp.Parser_automaton: parser is dead")) |}] ;; let%expect_test ("test for memory leak of most recently parsed sexp" [@tags "no-js"]) = (* read a sexp option *) let stream = chars_of_string input in let open Core in let option1 = fetch_sexp stream in print_s [%sexp (option1 : Sexp.t option)]; [%expect {| ((Hello World)) |}]; let weak = Weak.create 1 in Weak.set weak 0 option1; (* sexp option no longer live after a major collection *) Gc.full_major (); print_s [%sexp (Weak.check weak 0 : bool)]; [%expect {| false |}] ;; parsexp-0.17.0/test/test_parsexp.mli000066400000000000000000000000551461647336100174770ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/test/test_parsexp_and_positions.ml000066400000000000000000000054331461647336100222640ustar00rootroot00000000000000open! Import let test input = Single_just_positions.parse_string input |> Result.map ~f:Positions.to_list |> [%sexp_of: (Positions.pos list, Parse_error.t) Result.t] |> print_s ;; let test_many input = Many_just_positions.parse_string input |> Result.map ~f:Positions.to_list |> [%sexp_of: (Positions.pos list, Parse_error.t) Result.t] |> print_s ;; let%expect_test "single" = test "(1 2 3)"; [%expect {| (Ok ( ((line 1) (col 0) (offset 0)) ((line 1) (col 1) (offset 1)) ((line 1) (col 1) (offset 1)) ((line 1) (col 3) (offset 3)) ((line 1) (col 3) (offset 3)) ((line 1) (col 5) (offset 5)) ((line 1) (col 5) (offset 5)) ((line 1) (col 6) (offset 6)))) |}]; test "; plop\natom"; [%expect {| (Ok ( ((line 2) (col 0) (offset 7)) ((line 2) (col 3) (offset 10)))) |}]; test "(1 (2 3)(4 5))"; [%expect {| (Ok ( ((line 1) (col 0) (offset 0)) ((line 1) (col 1) (offset 1)) ((line 1) (col 1) (offset 1)) ((line 1) (col 3) (offset 3)) ((line 1) (col 4) (offset 4)) ((line 1) (col 4) (offset 4)) ((line 1) (col 6) (offset 6)) ((line 1) (col 6) (offset 6)) ((line 1) (col 7) (offset 7)) ((line 1) (col 8) (offset 8)) ((line 1) (col 9) (offset 9)) ((line 1) (col 9) (offset 9)) ((line 1) (col 11) (offset 11)) ((line 1) (col 11) (offset 11)) ((line 1) (col 12) (offset 12)) ((line 1) (col 13) (offset 13)))) |}] ;; let%expect_test "many" = test_many "(1 2) (3)"; [%expect {| (Ok ( ((line 1) (col 0) (offset 0)) ((line 1) (col 1) (offset 1)) ((line 1) (col 1) (offset 1)) ((line 1) (col 3) (offset 3)) ((line 1) (col 3) (offset 3)) ((line 1) (col 4) (offset 4)) ((line 1) (col 6) (offset 6)) ((line 1) (col 7) (offset 7)) ((line 1) (col 7) (offset 7)) ((line 1) (col 8) (offset 8)))) |}]; test_many "(1 2 3)\nhello"; [%expect {| (Ok ( ((line 1) (col 0) (offset 0)) ((line 1) (col 1) (offset 1)) ((line 1) (col 1) (offset 1)) ((line 1) (col 3) (offset 3)) ((line 1) (col 3) (offset 3)) ((line 1) (col 5) (offset 5)) ((line 1) (col 5) (offset 5)) ((line 1) (col 6) (offset 6)) ((line 2) (col 0) (offset 8)) ((line 2) (col 4) (offset 12)))) |}]; test_many "(1 2)(3)"; [%expect {| (Ok ( ((line 1) (col 0) (offset 0)) ((line 1) (col 1) (offset 1)) ((line 1) (col 1) (offset 1)) ((line 1) (col 3) (offset 3)) ((line 1) (col 3) (offset 3)) ((line 1) (col 4) (offset 4)) ((line 1) (col 5) (offset 5)) ((line 1) (col 6) (offset 6)) ((line 1) (col 6) (offset 6)) ((line 1) (col 7) (offset 7)))) |}] ;; parsexp-0.17.0/test/test_parsexp_and_positions.mli000066400000000000000000000000551461647336100224300ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/test/test_parsexp_conv.ml000066400000000000000000000056721461647336100203650ustar00rootroot00000000000000open! Import type 'a conv_result = ('a, Conv_error.t) Result.t [@@deriving sexp_of] let test a_of_sexp input = match Conv_single.parse_string input a_of_sexp with | Ok _ -> () | Error error -> Conv_error.report Stdlib.Format.std_formatter error ~filename:"" ;; let test_many a_of_sexp input = match Conv_many.parse_string input a_of_sexp with | Ok _ -> () | Error error -> Conv_error.report Stdlib.Format.std_formatter error ~filename:"" ;; let%expect_test "parsing errors" = (* For the following tests, you can check that the reported character positions match the positions reported by emacs when you move the cursor over the faulty s-expressions. *) test [%of_sexp: int list] {| (1 2 3|}; [%expect {| File "", line 2, character 6: Error: s-expression parsing error; unclosed parentheses at end of input |}]; test [%of_sexp: int list] {| (1 2 "abc)|}; [%expect {| File "", line 2, character 10: Error: s-expression parsing error; unterminated quoted string |}]; test_many [%of_sexp: int list] {| (1 2 3) "a|}; [%expect {| File "", line 3, character 2: Error: s-expression parsing error; unterminated quoted string |}] ;; let%expect_test "conversion errors" = (* For the following tests, you can check that the reported character positions match the positions reported by emacs when you move the cursor over the faulty s-expressions. *) test [%of_sexp: int list] {| (1 2 3 abc 4 5 6) |}; [%expect {| File "", line 2, characters 7-10: Error: s-expression conversion error; exception (Failure "int_of_sexp: (Failure int_of_string)") |}]; test [%of_sexp: int list] {| (1 2 (1 2 3)) |}; [%expect {| File "", line 2, characters 5-12: Error: s-expression conversion error; exception (Failure "int_of_sexp: atom needed") |}]; test [%of_sexp: int list] {| (1 2 (1 2 3)) |}; [%expect {| File "", line 2, characters 5-12: Error: s-expression conversion error; exception (Failure "int_of_sexp: atom needed") |}]; test_many [%of_sexp: int list] {| (1 2 3) (a) |}; [%expect {| File "", line 3, characters 1-2: Error: s-expression conversion error; exception (Failure "int_of_sexp: (Failure int_of_string)") |}] ;; let%expect_test "Conv_many_and_locations" = let input = {|() (abc) (1 (2 (3))) 123 (+ x y) |} in let _ = Conv_many_and_locations.parse_string_exn input (fun (sexp, { start_pos; end_pos }) -> let pos = start_pos.offset in let len = end_pos.offset - start_pos.offset in let sub = String.sub input ~pos ~len in let sexp_from_loc = Parsexp.Single.parse_string_exn sub in Expect_test_helpers_core.require_equal [%here] (module Sexp) sexp sexp_from_loc; print_s [%sexp (sexp : Sexp.t)]) in [%expect {| () (abc) (1 (2 (3))) 123 (+ x y) |}] ;; parsexp-0.17.0/test/test_parsexp_conv.mli000066400000000000000000000000551461647336100205240ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/test/test_parsexp_lexing.ml000066400000000000000000000015321461647336100206750ustar00rootroot00000000000000open! Import let assoc s = Lexer.assoc (Eager.Lexbuf_consumer.create ()) (Lexing.from_string s) |> [%sexp_of: (string * Sexp.t) list] |> print_s ;; let sexps s = let lexbuf = Lexing.from_string s in let p = Eager.Lexbuf_consumer.create () in let rec loop () = match Eager.Lexbuf_consumer.parse_opt p lexbuf with | None -> [] | Some sexp -> sexp :: loop () in loop () |> List.iter ~f:print_s ;; let%expect_test "simple test" = assoc {| # Hello x = (a b c) y = 42 z = "blah" a = 123 |}; [%expect {| ((x (a b c)) (y 42) (z blah) (a 123)) |}] ;; let%expect_test "empty lexing" = sexps ""; [%expect {| |}] ;; let%expect_test "the lexer doesn't consume more than it should" = sexps {| abc"123" |}; [%expect {| abc 123 |}]; sexps {| abc() |}; [%expect {| abc () |}] ;; parsexp-0.17.0/test/test_parsexp_lexing.mli000066400000000000000000000000551461647336100210450ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/test/test_positions.ml000066400000000000000000000162211461647336100176750ustar00rootroot00000000000000open! Import let cases = (* Cases of inputs, [.] represent a saved position, [:] represent a position saved twice. *) [ "" ; "." ; "1." ; "12." ; "123." ; "1234." ; "12345." ; "123456." ; ".\n" ; "\n." ; "\n\n\n" ; ". . . xxxxx\nxxxx . xxx . xxx\n" ; "....." ; ":" ; "..:.." ; Printf.sprintf "%*s." 40 "" ; Printf.sprintf "%*s." 300 "" ; String.concat (List.init (62 * 4) ~f:(fun _i -> "..")) ] ;; (* Extract the positions of the '.' in the input *) let build_positions s = let builder = Positions.Builder.create () in for i = 0 to String.length s - 1 do match s.[i] with | '\n' -> Positions.Builder.add_newline builder ~offset:i | '.' -> Positions.Builder.add builder ~offset:i | ':' -> Positions.Builder.add_twice builder ~offset:i | _ -> () done; Positions.Builder.contents builder ;; (* Same but with a trivial implementation (i.e. without using [Positions]). *) let build_positions_simple s = let rec loop i (pos : Positions.pos) = if i = String.length s then [] else ( let offset = pos.offset + 1 in match s.[i] with | '\n' -> loop (i + 1) { line = pos.line + 1; col = 0; offset } | '.' -> pos :: loop (i + 1) { pos with offset; col = pos.col + 1 } | ':' -> pos :: pos :: loop (i + 1) { pos with offset; col = pos.col + 1 } | _ -> loop (i + 1) { pos with offset; col = pos.col + 1 }) in loop 0 Positions.beginning_of_file ;; let%expect_test "build_positions_simple" = let f s = print_s [%sexp (build_positions_simple s : Positions.pos list)] in f ""; [%expect {| () |}]; f "."; [%expect {| (( (line 1) (col 0) (offset 0))) |}]; f ". ."; [%expect {| (((line 1) (col 0) (offset 0)) ((line 1) (col 2) (offset 2))) |}]; f ". xxx \n xxx ."; [%expect {| (((line 1) (col 0) (offset 0)) ((line 2) (col 5) (offset 12))) |}] ;; let check_all_subsexps_map_to_their_position s sexps positions = let check_subsexp subsexp = match Positions.find_sub_sexp_in_list_phys positions sexps ~sub:subsexp with | None -> failwith "not found" | Some range -> assert ( Sexp.( = ) (Single.parse_string_exn (String.sub s ~pos:range.start_pos.offset ~len:(range.end_pos.offset - range.start_pos.offset))) subsexp) in let rec iter sexps = List.iter sexps ~f:(fun sexp -> check_subsexp sexp; match sexp with | Atom _ -> () | List l -> iter l) in iter sexps ;; let%expect_test "build_positions_ignore_commented_expr" = let f s = let sexps, positions = Many_and_positions.parse_string_exn s in print_s [%sexp (Positions.to_list positions : Positions.pos list)]; check_all_subsexps_map_to_their_position s sexps positions in f "a #;((b)) c"; [%expect {| (((line 1) (col 0) (offset 0)) ((line 1) (col 0) (offset 0)) ((line 1) (col 10) (offset 10)) ((line 1) (col 10) (offset 10))) |}] ;; let%expect_test "all" = List.iter cases ~f:(fun input -> let expected = build_positions_simple input in let got = build_positions input |> Positions.to_list in require [%here] ([%compare.equal: Positions.pos list] got expected) ~if_false_then_print_s: (lazy [%sexp { input : string; expected : Positions.pos list; got : Positions.pos list }])) ;; let%expect_test "find" = List.iter cases ~f:(fun input -> let positions = build_positions_simple input |> Array.of_list in let from_parsexp = build_positions input in let count = Array.length positions in for i = 0 to count - 1 do for j = i + 1 to count - 1 do let expected = Positions.make_range_incl ~start_pos:positions.(i) ~last_pos:positions.(j) in let got = Result.try_with (fun () -> Positions.find from_parsexp i j) in require [%here] (match got with | Ok got -> [%compare.equal: Positions.range] got expected | Error _ -> false) ~if_false_then_print_s: (lazy [%sexp { input : string ; i : int ; j : int ; expected : (Positions.range, exn) Result.t = Ok expected ; got : (Positions.range, exn) Result.t }]) done done); [%expect {| |}] ;; let cases_for_find_sub_sexp = [ "( ( ( abc ) (+ 1 2) ) )" ] module Annotated = struct type t = | Atom of Positions.range * Sexp.t | List of Positions.range * Sexp.t * t list let of_sexp_and_positions = let rec loop (sexp : Sexp.t) (positions : Positions.pos list) = match sexp, positions with | Atom _, start_pos :: last_pos :: rest -> Atom (Positions.make_range_incl ~start_pos ~last_pos, sexp), rest | List l, start_pos :: rest -> let annots_rev, rest = List.fold_left l ~init:([], rest) ~f:(fun (acc, positions) sexp -> let annot, rest = loop sexp positions in annot :: acc, rest) in (match rest with | [] -> assert false | last_pos :: rest -> ( List (Positions.make_range_incl ~start_pos ~last_pos, sexp, List.rev annots_rev) , rest )) | _ -> assert false in fun sexp positions -> let t, rest = loop sexp positions in assert (List.is_empty rest); t ;; let rec iter t ~f = match t with | Atom (range, sexp) -> f range sexp | List (range, sexp, children) -> f range sexp; List.iter children ~f:(iter ~f) ;; end let%expect_test "find_sub_sexp_phys" = List.iter cases_for_find_sub_sexp ~f:(fun input -> let sexp, positions = Single_and_positions.parse_string_exn input in let annot = Annotated.of_sexp_and_positions sexp (Positions.to_list positions) in Annotated.iter annot ~f:(fun expected sub -> let got = Positions.find_sub_sexp_phys positions sexp ~sub in require [%here] (Option.value_map got ~default:false ~f:([%compare.equal: Positions.range] expected)) ~if_false_then_print_s: (lazy [%sexp { input : string ; sexp : Sexp.t ; sub : Sexp.t ; expected : Positions.range option = Some expected ; got : Positions.range option }]))); [%expect {| |}] ;; let%expect_test "advance_sexp_exn" = let input = {|() (abc) (1 (2 (3))) 123 (+ x y) |} in let sexps, positions = Many_and_positions.parse_string_exn input in let iterator = Positions.Iterator.create positions in List.iter sexps ~f:(fun sexp -> let ({ start_pos; end_pos } : Positions.range) = Positions.Iterator.advance_sexp_exn iterator sexp in let pos = start_pos.offset in let len = end_pos.offset - start_pos.offset in let sub = String.sub input ~pos ~len in let sexp_from_loc = Parsexp.Single.parse_string_exn sub in Expect_test_helpers_core.require_equal [%here] (module Sexp) sexp sexp_from_loc; print_s [%sexp (sexp : Sexp.t)]); [%expect {| () (abc) (1 (2 (3))) 123 (+ x y) |}] ;; parsexp-0.17.0/test/test_positions.mli000066400000000000000000000000551461647336100200440ustar00rootroot00000000000000(*_ This signature is deliberately empty. *) parsexp-0.17.0/test/test_sexplib_vs_parsexp.ml000066400000000000000000000056701461647336100215740ustar00rootroot00000000000000open! Import (* Compare parsexp to the default cpp based sexplib parser *) let test s = let sexplib = Sexplib.Sexp.of_string (String.strip s) in let parsexp = Single.parse_string_exn s in if Sexp.equal sexplib parsexp then ( Stdlib.print_string "same\n"; print_s [%sexp (parsexp : Sexp.t)]) else ( Stdlib.print_string "FAILURE\n"; print_s [%sexp ~~(sexplib : Sexp.t)]; print_s [%sexp ~~(parsexp : Sexp.t)]); let sexplib_lexer = Sexplib.Sexp.scan_sexp (Lexing.from_string s) in if not (Sexp.equal sexplib sexplib_lexer) then ( Stdlib.print_string "\nNote: the various sexplib parsers disagree between themselves\n"; print_s [%sexp ~~(sexplib_lexer : Sexp.t)]) ;; let%expect_test _ = test {| "a\q" |}; [%expect {| same "a\\q" |}] ;; let%expect_test _ = test {| "a\ b" |}; [%expect {| same "a\\ b" Note: the various sexplib parsers disagree between themselves (sexplib_lexer "a b") |}] ;; let test_cont_state input = let sexplib : Sexp.t = try match Sexplib.Sexp.parse input with | Done (sexp, _) -> [%sexp Done (sexp : Sexp.t)] | Cont (st, _) -> [%sexp Cont (Sexplib.Sexp.Cont_state.to_string st : string)] with | _ -> [%sexp Raised] in let parsexp = let state = Private.Automaton.create Single Sexp in ignore (String.fold input ~init:Private.Automaton.Stack.empty ~f:(fun stack ch -> Private.Automaton.feed state ch stack) : Private.Automaton.Stack.t); let old_state = Private.Automaton.old_parser_cont_state state in [%sexp Cont (old_state : Old_parser_cont_state.t)] in if Sexp.equal sexplib parsexp then ( Stdlib.print_string "same\n"; print_s [%sexp (parsexp : Sexp.t)]) else ( Stdlib.print_string "FAILURE\n"; print_s [%sexp ~~(sexplib : Sexp.t)]; print_s [%sexp ~~(parsexp : Sexp.t)]) ;; let%expect_test _ = test_cont_state " "; [%expect {| same (Cont Parsing_toplevel_whitespace) |}]; test_cont_state "\r"; [%expect {| same (Cont Parsing_nested_whitespace) |}]; test_cont_state "\"toto"; [%expect {| same (Cont Parsing_atom) |}]; test_cont_state "#"; [%expect {| same (Cont Parsing_atom) |}]; test_cont_state "#| toto"; [%expect {| same (Cont Parsing_block_comment) |}]; test_cont_state "#| \"bla"; [%expect {| same (Cont Parsing_block_comment) |}]; test_cont_state "#; toto"; [%expect {| same (Cont Parsing_sexp_comment) |}]; test_cont_state "; toto"; [%expect {| same (Cont Parsing_toplevel_whitespace) |}]; test_cont_state "("; [%expect {| same (Cont Parsing_list) |}]; test_cont_state "#; ("; [%expect {| same (Cont Parsing_sexp_comment) |}]; test_cont_state "#; #|"; [%expect {| same (Cont Parsing_sexp_comment) |}]; test_cont_state "#;\r"; [%expect {| same (Cont Parsing_sexp_comment) |}] ;; parsexp-0.17.0/test/test_sexplib_vs_parsexp.mli000066400000000000000000000000551461647336100217350ustar00rootroot00000000000000(*_ This signature is deliberately empty. *)