csv-1.3.2/0000755000175000017500000000000012236526204010774 5ustar trchtrchcsv-1.3.2/INSTALL.txt0000644000175000017500000000166112236526203012646 0ustar trchtrch(* OASIS_START *) (* DO NOT EDIT (digest: aeed0b6f112359d79bc311b45f9c68e2) *) This is the INSTALL file for the csv distribution. This package uses OASIS to generate its build system. See section OASIS for full information. Dependencies ============ In order to compile this package, you will need: * ocaml for all, doc API * findlib Installing ========== 1. Uncompress the source archive and go to the root of the package 2. Run 'ocaml setup.ml -configure' 3. Run 'ocaml setup.ml -build' 4. Run 'ocaml setup.ml -install' Uninstalling ============ 1. Go to the root of the package 2. Run 'ocaml setup.ml -uninstall' OASIS ===== OASIS is a program that generates a setup.ml file using a simple '_oasis' configuration file. The generated setup only depends on the standard OCaml installation: no additional library is required. (* OASIS_STOP *) Testing ======= If you would like to run some tests, just issue 'ocaml setup.ml -test'. csv-1.3.2/README.txt0000644000175000017500000000112312236526204012467 0ustar trchtrch(* OASIS_START *) (* DO NOT EDIT (digest: 1f7364ec362701bd28a62729f5e9882e) *) This is the README file for the csv distribution. A pure OCaml library to read and write CSV files. This is a pure OCaml library to read and write CSV files, including all extensions used by Excel — e.g. quotes, newlines, 8 bit characters in fields, \"0 etc. The library comes with a handy command line tool called csvtool for handling CSV files from shell scripts. See the files INSTALL.txt for building and installation instructions. Home page: https://forge.ocamlcore.org/projects/csv/ (* OASIS_STOP *) csv-1.3.2/README.md0000644000175000017500000000131512236526203012252 0ustar trchtrchOCaml CSV ========= The [comma-separated values](http://en.wikipedia.org/wiki/Comma-separated_values) format — or CSV for short — is a simple tabular format supported by all major spreadsheets. This library implements functions to read and write files in this format (including Excel extensions) as well as some convenience functions to manipulate such data. Compile ------- This package uses [oasis](https://github.com/ocaml/oasis) to generate its configure, build and install scripts. You do not need oasis to install and use the library. Simply run ocaml setup.ml -configure ocaml setup.ml -build Install ------- ocaml setup.ml -install Uninstall --------- ocaml setup.ml -uninstall csv-1.3.2/src/0000755000175000017500000000000012236526204011563 5ustar trchtrchcsv-1.3.2/src/csv.mli0000644000175000017500000003324012236526203013062 0ustar trchtrch(* File: csv.mli Copyright (C) 2006 Richard Jones email: rjones@redhat.com Christophe Troestler email: Christophe.Troestler@umons.ac.be WWW: http://math.umons.ac.be/anum/software/ This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License version 2.1 or later as published by the Free Software Foundation, with the special exception on linking described in the file LICENSE. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the file LICENSE for more details. *) (** Read and write the CSV (comma separated values) format. @author Richard Jones @author Christophe Troestler *) type t = string list list (** Representation of CSV data in memory. This is a list of rows, each row being a list of columns. *) (** {2 Input/output objects} *) (** The most basic input object for best interoperability. *) class type in_obj_channel = object method input : string -> int -> int -> int (** [input buf ofs len] reads up to [len] octets from the channel and puts them in the substring [buf.[ofs .. ofs+len-1]]. Returns the number of octets actually read (and stored). When the channel is non-blocking, and there are currently no bytes to read, the number 0 will be returned. @raise End_of_file when there are no more bytes to read. *) method close_in : unit -> unit (** Closes the channel for input. *) end (** The most basic output object for best interoperability. *) class type out_obj_channel = object method output : string -> int -> int -> int (** [output s ofs len] writes up to [len] bytes of the substring [s.[ofs .. ofs+len-1]]. Return the number of bytes actually written. When the channel is non-blocking, and there are currently no bytes to write, the number 0 must be returned. *) method close_out : unit -> unit (** Flushes the buffer, if any, and closes the channel for output. *) end (** {2 Input} *) exception Failure of int * int * string (** [Failure(nrecord, nfield, msg)] is raised to indicate a parsing error for the field number [nfield] on the record number [nrecord], the description [msg] says what is wrong. The first record and the first field of a record are numbered [1] (to correspond to the usual spreadsheet numbering but differing from [List.nth] of the OCaml representation). *) type in_channel (** Stateful handle to input CSV files. *) val of_in_obj : ?separator:char -> ?excel_tricks:bool -> in_obj_channel -> in_channel (** [of_in_obj ?separator ?excel_tricks in_chan] creates a new "channel" to access the data in CSV form available from the channel [in_chan]. @param separator What character the separator is. The default is [',']. You should be aware however that, in the countries where comma is used as a decimal separator, Excel will use [';'] as the separator. @param excel_tricks enables Excel tricks, namely the fact that '"' followed by '0' in a quoted string means ASCII NULL and the fact that a field of the form ="..." only returns the string inside the quotes. Default: [true]. *) val of_channel : ?separator:char -> ?excel_tricks:bool -> Pervasives.in_channel -> in_channel (** Same as {!Csv.of_in_obj} except that the data is read from a standard channel. *) val of_string : ?separator:char -> ?excel_tricks:bool -> string -> in_channel (** Same as {!Csv.of_in_obj} except that the data is read from a string. *) val load : ?separator:char -> ?excel_tricks:bool-> string -> t (** [load fname] loads the CSV file [fname]. If [filename] is ["-"] then load from [stdin]. @param separator What character the separator is. The default is [',']. You should be aware however that, in the countries where comma is used as a decimal separator, Excel will use [';'] as the separator. @param excel_tricks enables Excel tricks, namely the fact that '"' followed by '0' in a quoted string means ASCII NULL and the fact that a field of the form ="..." only returns the string inside the quotes. Default: [true]. *) val load_in : ?separator:char -> ?excel_tricks:bool -> Pervasives.in_channel -> t (** [load_in ch] loads a CSV file from the input channel [ch]. See {!Csv.load} for the meaning of [separator] and [excel_tricks]. *) val to_in_obj : in_channel -> in_obj_channel (** For efficiency reasons, the [in_channel] buffers the data from the original channel. If you want to examine the data by other means than the methods below (say after a failure), you need to use this function in order not to "loose" data in the buffer. *) val close_in : in_channel -> unit (** [close_in ic] closes the channel [ic]. The underlying channel is closed as well. *) val next : in_channel -> string list (** [next ic] returns the next record in the CSV file. @raise End_of_file if no more record can be read. @raise Csv.Failure if the CSV format is not respected. The partial record read is available with [#current_record]. *) val fold_left : ('a -> string list -> 'a) -> 'a -> in_channel -> 'a (** [fold_left f a ic] computes (f ... (f (f a r0) r1) ... rN) where r1,...,rN are the records in the CSV file. If [f] raises an exception, the record available at that moment is accessible through {!Csv.current_record}. *) val fold_right : (string list -> 'a -> 'a) -> in_channel -> 'a -> 'a (** [fold_right f ic a] computes (f r1 ... (f rN-1 (f rN a)) ...) where r1,...,rN-1, rN are the records in the CSV file. All records are read before applying [f] so this method is not convenient if your file is large. *) val iter : f:(string list -> unit) -> in_channel -> unit (** [iter f ic] iterates [f] on all remaining records. If [f] raises an exception, the record available at that moment is accessible through {!Csv.current_record}. *) val input_all : in_channel -> t (** [input_all ic] return a list of the CSV records till the end of the file. *) val current_record : in_channel -> string list (** The current record under examination. This is useful in order to gather the parsed data in case of [Failure]. *) val load_rows : ?separator:char -> ?excel_tricks:bool -> (string list -> unit) -> Pervasives.in_channel -> unit (** @deprecated use {!Csv.iter} on a {!Csv.in_channel} created with {!Csv.of_channel}. *) (************************************************************************) (** {2 Output} *) type out_channel val to_out_obj : ?separator:char -> ?excel_tricks:bool -> out_obj_channel -> out_channel (** [to_out_obj ?separator ?excel_tricks out_chan] creates a new "channel" to output the data in CSV form. @param separator What character the separator is. The default is [',']. @param excel_tricks enables Excel tricks, namely the fact that '\000' is represented as '"' followed by '0' and the fact that a field with leading or trailing spaces or a leading '0' will be encoded as ="..." (to avoid Excel "helping" you). Default: [false]. *) val to_channel : ?separator:char -> ?excel_tricks:bool -> Pervasives.out_channel -> out_channel (** Same as {!Csv.to_out_obj} but output to a standard channel. *) val output_record : out_channel -> string list -> unit (** [output_record oc r] write the record [r] is CSV form to the channel [oc]. *) val output_all : out_channel -> t -> unit (** [output_all oc csv] outputs all records in [csv] to the channel [oc]. *) val save_out : ?separator:char -> ?excel_tricks:bool -> Pervasives.out_channel -> t -> unit (** @deprecated Save string list list to a channel. *) val save : ?separator:char -> ?excel_tricks:bool -> string -> t -> unit (** [save fname csv] Save the [csv] data to the file [fname]. *) val print : ?separator:char -> ?excel_tricks:bool -> t -> unit (** Print the CSV data. *) val print_readable : t -> unit (** Print the CSV data to [stdout] in a human-readable format. Not much is guaranteed about how the CSV is printed, except that it will be easier to follow than a "raw" output done with {!Csv.print}. This is a one-way operation. There is no easy way to parse the output of this command back into CSV data. *) val save_out_readable : Pervasives.out_channel -> t -> unit (** As for {!Csv.print_readable}, allowing the output to be sent to a channel. *) (************************************************************************) (** {2 Functions acting on CSV data loaded in memory} *) val lines : t -> int (** Return the number of lines in a CSV data. *) val columns : t -> int (** Work out the (maximum) number of columns in a CSV file. Note that each line may be a different length, so this finds the one with the most columns. *) val trim : ?top:bool -> ?left:bool -> ?right:bool -> ?bottom:bool -> t -> t (** This takes a CSV file and trims empty cells. * * All four of the option arguments ([~top], [~left], [~right], [~bottom]) * default to [true]. * * The exact behaviour is: * * [~right]: If true, remove any empty cells at the right hand end of * any row. The number of columns in the resulting CSV structure will * not necessarily be the same for each row. * * [~top]: If true, remove any empty rows (no cells, or containing just empty * cells) from the top of the CSV structure. * * [~bottom]: If true, remove any empty rows from the bottom of the * CSV structure. * * [~left]: If true, remove any empty columns from the left of the * CSV structure. Note that [~left] and [~right] are quite different: * [~left] considers the whole CSV structure, whereas [~right] considers * each row in isolation. *) val square : t -> t (** Make the CSV data "square" (actually rectangular). This pads out each row with empty cells so that all rows are the same length as the longest row. After this operation, every row will have length {!Csv.columns}. *) val is_square : t -> bool (** Return true iff the CSV is "square" (actually rectangular). This means that each row has the same number of cells. *) val set_columns : int -> t -> t (** [set_columns cols csv] makes the CSV data square by forcing the width to the given number of [cols]. Any short rows are padded with blank cells. Any long rows are truncated. *) val set_rows : int -> t -> t (** [set_rows rows csv] makes the CSV data have exactly [rows] rows by adding empty rows or truncating rows as necessary. Note that [set_rows] does not make the CSV square. If you want it to be square, call either {!Csv.square} or {!Csv.set_columns} after. *) val set_size : int -> int -> t -> t (** [set_size rows cols csv] makes the CSV data square by forcing the size to [rows * cols], adding blank cells or truncating as necessary. It is the same as calling [set_columns cols (set_rows rows csv)] *) val sub : int -> int -> int -> int -> t -> t (** [sub r c rows cols csv] returns a subset of [csv]. The subset is defined as having top left corner at row [r], column [c] (counting from [0]) and being [rows] deep and [cols] wide. The returned CSV will be "square". *) val compare : t -> t -> int (** Compare two CSV files for equality, ignoring blank cells at the end of a row, and empty rows appended to one or the other. This is "semantic" equality - roughly speaking, the two CSV files would look the same if opened in a spreadsheet program. *) val concat : t list -> t (** Concatenate CSV files so that they appear side by side, arranged left to right across the page. Each CSV file (except the final one) is first squared. (To concatenate CSV files so that they appear from top to bottom, just use [List.concat]). *) val transpose : t -> t (** Permutes the lines and columns of the CSV data. Nonexistent cells become empty cells after transpose if they must be created. *) val to_array : t -> string array array val of_array : string array array -> t (** Convenience functions to convert to and from a matrix representation. [to_array] will produce a ragged matrix (not all rows will have the same length) unless you call {!Csv.square} first. *) val associate : string list -> t -> (string * string) list list (** [associate header data] takes a block of data and converts each * row in turn into an assoc list which maps column header to data cell. * * Typically a spreadsheet will have the format: * {v * header1 header2 header3 * data11 data12 data13 * data21 data22 data23 * ... * v} * * This function arranges the data into a more usable form which is * robust against changes in column ordering. The output of the * function is: * {v * [ ["header1", "data11"; "header2", "data12"; "header3", "data13"]; * ["header1", "data21"; "header2", "data22"; "header3", "data23"]; * etc. ] * v} * * Each row is turned into an assoc list (see [List.assoc]). * * If a row is too short, it is padded with empty cells ([""]). If * a row is too long, it is truncated. * * You would typically call this function as: * * {v * let header, data = match csv with h :: d -> h, d | [] -> assert false;; * let data = Csv.associate header data;; * v} * * The header strings are shared, so the actual space in memory consumed * by the spreadsheet is not much larger. *) csv-1.3.2/src/csv.ml0000644000175000017500000006660512236526203012724 0ustar trchtrch(* File: csv.ml Copyright (C) 2005-2009 Richard Jones email: rjones@redhat.com Christophe Troestler email: Christophe.Troestler@umons.ac.be WWW: http://math.umons.ac.be/anum/software/ This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License version 2.1 or later as published by the Free Software Foundation, with the special exception on linking described in the file LICENSE. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the file LICENSE for more details. *) (* MOTIVATION. There are already several solutions to parse CSV files in OCaml. They did not suit my needs however because: - The files I need to deal with have a header which does not always reflect the data structure (say the first row are the names of neurones but there are two columns per name). In other words I want to be able to deal with heterogeneous files. - I do not want to read the the whole file at once (I may but I just want to be able to choose). Higher order functions like fold are fine provided the read stops at the line an exception is raised (so it can be reread again). - For similarly encoded line, being able to specify once a decoder and then use a type safe version would be nice. - Speed is not neglected (we would like to be able to parse a ~2.5Mb file under 0.1 sec on my machine (2GHz Core Duo)). We follow the CVS format documentation available at http://www.creativyst.com/Doc/Articles/CSV/CSV01.htm *) type t = string list list (* Specialize to int for speed *) let max i j = if (i:int) < j then j else i (* Enhance the List module with tail rec functions. *) module List = struct include List (* Implementation of [map] in JSC Core. *) let map_slow l ~f = List.rev (List.rev_map f l) let rec count_map ~f l ctr = match l with | [] -> [] | [x1] -> let f1 = f x1 in [f1] | [x1; x2] -> let f1 = f x1 in let f2 = f x2 in [f1; f2] | [x1; x2; x3] -> let f1 = f x1 in let f2 = f x2 in let f3 = f x3 in [f1; f2; f3] | [x1; x2; x3; x4] -> let f1 = f x1 in let f2 = f x2 in let f3 = f x3 in let f4 = f x4 in [f1; f2; f3; f4] | x1 :: x2 :: x3 :: x4 :: x5 :: tl -> let f1 = f x1 in let f2 = f x2 in let f3 = f x3 in let f4 = f x4 in let f5 = f x5 in f1 :: f2 :: f3 :: f4 :: f5 :: (if ctr > 1000 then map_slow ~f tl else count_map ~f tl (ctr + 1)) let map f l = count_map ~f l 0 (* Implementation of [append] in JSC core. *) let slow_append l1 l2 = List.rev_append (List.rev l1) l2 let rec count_append l1 l2 count = match l1 with | [] -> l2 | [x1] -> x1 :: l2 | [x1; x2] -> x1 :: x2 :: l2 | [x1; x2; x3] -> x1 :: x2 :: x3 :: l2 | [x1; x2; x3; x4] -> x1 :: x2 :: x3 :: x4 :: l2 | x1 :: x2 :: x3 :: x4 :: x5 :: tl -> x1 :: x2 :: x3 :: x4 :: x5 :: (if count > 1000 then slow_append tl l2 else count_append tl l2 (count + 1)) let append l1 l2 = count_append l1 l2 0 (* Tail recursive [combine]. *) let rec rev_combine acc l1 l2 = match l1, l2 with | ([], []) -> acc | (a1::l1, a2::l2) -> rev_combine ((a1, a2) :: acc) l1 l2 | (_, _) -> invalid_arg "List.combine" let slow_combine l1 l2 = List.rev (rev_combine [] l1 l2) let rec count_combine l1 l2 count = match l1, l2 with | ([], []) -> [] | ([x1], [y1]) -> [x1, y1] | ([x1; x2], [y1; y2]) -> [x1, y1; x2, y2] | ([x1; x2; x3], [y1; y2; y3]) -> [x1, y1; x2, y2; x3, y3] | ([x1; x2; x3; x4], [y1; y2; y3; y4]) -> [x1, y1; x2, y2; x3, y3; x4, y4] | (x1 :: x2 :: x3 :: x4 :: tl1), (y1 :: y2 :: y3 :: y4 :: tl2) -> (x1, y1) :: (x2, y2) :: (x3, y3) :: (x4, y4) :: (if count > 1000 then slow_combine tl1 tl2 else count_combine tl1 tl2 (count + 1)) | (_, _) -> invalid_arg "List.combine" let combine l1 l2 = count_combine l1 l2 0 end class type in_obj_channel = object method input : string -> int -> int -> int method close_in : unit -> unit end class type out_obj_channel = object method output : string -> int -> int -> int method close_out : unit -> unit end (* Specialize [min] to integers for performance reasons (> 150% faster). *) let min x y = if (x:int) <= y then x else y (* * Input *) exception Failure of int * int * string let buffer_len = 0x1FFF (* We buffer the input as this allows the be efficient while using very basic input channels. The drawback is that if we want to use another tool, there will be data hold in the buffer. That is why we allow to convert a CSV handle to an object sharing the same buffer. Because if this, we actually decided to implement the CSV handle as an object that is coercible to a input-object. FIXME: This is not made for non-blocking channels. Can we fix it? *) type in_channel = { in_chan : in_obj_channel; in_buf : string; (* The data in the in_buf is at indexes i s.t. in0 <= i < in1. Invariant: 0 <= in0 ; in1 <= buffer_len in1 < 0 indicates a closed channel. *) mutable in0 : int; mutable in1 : int; mutable end_of_file : bool; (* If we encounter an End_of_file exception, we set this flag to avoid reading again because we do not know how the channel will react past an end of file. That allows us to assume that reading past an end of file will keep raising End_of_file. *) current_field : Buffer.t; (* buffer reused to scan fields *) mutable record : string list; (* The current record *) mutable record_n : int; (* For error messages *) separator : char; excel_tricks : bool; } let of_in_obj ?(separator=',') ?(excel_tricks=true) in_chan = { in_chan = in_chan; in_buf = String.create buffer_len; in0 = 0; in1 = 0; end_of_file = false; current_field = Buffer.create 0xFF; record = []; record_n = 0; (* => first record numbered 1 *) separator = separator; excel_tricks = excel_tricks; } let of_channel ?separator ?excel_tricks fh = of_in_obj ?separator ?excel_tricks (object val fh = fh method input s ofs len = try let r = Pervasives.input fh s ofs len in if r = 0 then raise End_of_file; r with Sys_blocked_io -> 0 method close_in() = Pervasives.close_in fh end) let of_string ?separator ?excel_tricks str = of_in_obj ?separator ?excel_tricks (object val mutable position = 0 method input buf ofs len = if position >= String.length str then raise End_of_file else ( let actual = min len (String.length str - position) in String.blit str position buf ofs actual ; position <- position + actual ; actual ) method close_in() = () end) (* [fill_in_buf chan] refills in_buf if needed (when empty). After this [in0 < in1] or [in0 = in1 = 0], the latter indicating that there is currently no bytes to read (for a non-blocking channel). @raise End_of_file if there are no more bytes to read. *) let fill_in_buf ic = if ic.end_of_file then raise End_of_file; if ic.in0 >= ic.in1 then begin try ic.in0 <- 0; ic.in1 <- ic.in_chan#input ic.in_buf 0 buffer_len; with End_of_file -> ic.end_of_file <- true; raise End_of_file end let close_in ic = if ic.in1 >= 0 then begin ic.in0 <- 0; ic.in1 <- -1; ic.in_chan#close_in(); (* may raise an exception *) end let to_in_obj ic = object val ic = ic method input buf ofs len = if ofs < 0 || len < 0 || ofs + len > String.length buf then invalid_arg "Csv.to_in_obj#input"; if ic.in1 < 0 then raise(Sys_error "Bad file descriptor"); fill_in_buf ic; let r = min len (ic.in1 - ic.in0) in String.blit ic.in_buf ic.in0 buf ofs r; ic.in0 <- ic.in0 + r; r method close_in() = close_in ic end (* * CSV input format parsing *) let is_space c = c = ' ' || c = '\t' (* See documentation *) let is_real_space c = c = ' ' (* when separator = '\t' *) (* Given a buffer, returns its content stripped of *final* whitespace. *) let strip_contents buf = let n = ref(Buffer.length buf - 1) in while !n >= 0 && is_space(Buffer.nth buf !n) do decr n done; Buffer.sub buf 0 (!n + 1) (* Return the substring after stripping its final space. It is assumed the substring parameters are valid. *) let strip_substring buf ofs len = let n = ref(ofs + len - 1) in while !n >= ofs && is_space(String.unsafe_get buf !n) do decr n done; String.sub buf ofs (!n - ofs + 1) (* Skip the possible '\n' following a '\r'. Reaching End_of_file is not considered an error -- just do nothing. *) let skip_CR ic = try fill_in_buf ic; if String.unsafe_get ic.in_buf ic.in0 = '\n' then ic.in0 <- ic.in0 + 1 with End_of_file -> () (* Unquoted field. Read till a delimiter, a newline, or the end of the file. Skip the next delimiter or newline. @return [true] if more fields follow, [false] if the record is complete. *) let rec seek_unquoted_separator ic i = if i >= ic.in1 then ( (* End not found, need to look at the next chunk *) Buffer.add_substring ic.current_field ic.in_buf ic.in0 (i - ic.in0); ic.in0 <- i; fill_in_buf ic; (* or raise End_of_file *) seek_unquoted_separator ic 0 ) else let c = String.unsafe_get ic.in_buf i in if c = ic.separator || c = '\n' || c = '\r' then ( if Buffer.length ic.current_field = 0 then (* Avoid copying the string to the buffer if unnecessary *) ic.record <- strip_substring ic.in_buf ic.in0 (i - ic.in0) :: ic.record else ( Buffer.add_substring ic.current_field ic.in_buf ic.in0 (i - ic.in0); ic.record <- strip_contents ic.current_field :: ic.record; ); ic.in0 <- i + 1; if c = '\r' then (skip_CR ic; false) else (c = ic.separator) ) else seek_unquoted_separator ic (i+1) let add_unquoted_field ic = try seek_unquoted_separator ic ic.in0 with End_of_file -> ic.record <- strip_contents ic.current_field :: ic.record; false (* Quoted field closed. Read past a separator or a newline and decode the field or raise [End_of_file]. @return [true] if more fields follow, [false] if the record is complete. *) let rec seek_quoted_separator ic field_no = fill_in_buf ic; (* or raise End_of_file *) let c = String.unsafe_get ic.in_buf ic.in0 in ic.in0 <- ic.in0 + 1; if c = ic.separator || c = '\n' || c = '\r' then ( ic.record <- Buffer.contents ic.current_field :: ic.record; if c = '\r' then (skip_CR ic; false) else (c = ic.separator) ) else if is_space c then seek_quoted_separator ic field_no (* skip space *) else raise(Failure(ic.record_n, field_no, "Non-space char after closing the quoted field")) let rec examine_quoted_field ic field_no after_quote i = if i >= ic.in1 then ( (* End of field not found, need to look at the next chunk *) Buffer.add_substring ic.current_field ic.in_buf ic.in0 (i - ic.in0); ic.in0 <- i; fill_in_buf ic; (* or raise End_of_file *) examine_quoted_field ic field_no after_quote 0 ) else let c = String.unsafe_get ic.in_buf i in if !after_quote then ( if c = '\"' then ( after_quote := false; (* [c] is kept so a quote will be included in the field *) examine_quoted_field ic field_no after_quote (i+1) ) else if c = ic.separator || is_space c || c = '\n' || c = '\r' then ( seek_quoted_separator ic field_no (* field already saved; in0=i; after_quote=true *) ) else if ic.excel_tricks && c = '0' then ( (* Supposedly, '"' '0' means ASCII NULL *) after_quote := false; Buffer.add_char ic.current_field '\000'; ic.in0 <- i + 1; (* skip the '0' *) examine_quoted_field ic field_no after_quote (i+1) ) else raise(Failure(ic.record_n, field_no, "Bad '\"' in quoted field")) ) else if c = '\"' then ( after_quote := true; (* Save the field so far, without the quote *) Buffer.add_substring ic.current_field ic.in_buf ic.in0 (i - ic.in0); ic.in0 <- i + 1; (* skip the quote *) examine_quoted_field ic field_no after_quote ic.in0 ) else examine_quoted_field ic field_no after_quote (i+1) let add_quoted_field ic field_no = let after_quote = ref false in (* preserved through exn *) try examine_quoted_field ic field_no after_quote ic.in0 with End_of_file -> (* Add the field even if not closed well *) ic.record <- Buffer.contents ic.current_field :: ic.record; if !after_quote then false else raise(Failure(ic.record_n, field_no, "Quoted field closed by end of file")) let skip_spaces ic = let is_space = if ic.separator = '\t' then is_real_space else is_space in (* Skip spaces: after this [in0] is a non-space char. *) while ic.in0 < ic.in1 && is_space(String.unsafe_get ic.in_buf ic.in0) do ic.in0 <- ic.in0 + 1 done; while ic.in0 >= ic.in1 do fill_in_buf ic; while ic.in0 < ic.in1 && is_space(String.unsafe_get ic.in_buf ic.in0) do ic.in0 <- ic.in0 + 1 done; done (* We suppose to be at the beginning of a field. Add the next field to [record]. @return [true] if more fields follow, [false] if the record is complete. Return Failure (if there is a format error), End_of_line (if the row is complete) or End_of_file (if there is not more data to read). *) let add_next_field ic field_no = Buffer.clear ic.current_field; try skip_spaces ic; (* Now, in0 < in1 or End_of_file was raised *) let c = String.unsafe_get ic.in_buf ic.in0 in if c = '\"' then ( ic.in0 <- ic.in0 + 1; add_quoted_field ic field_no ) else if ic.excel_tricks && c = '=' then begin ic.in0 <- ic.in0 + 1; (* mark '=' as read *) try fill_in_buf ic; if String.unsafe_get ic.in_buf ic.in0 = '\"' then ( (* Excel trick ="..." to prevent spaces around the field to be removed. *) ic.in0 <- ic.in0 + 1; (* skip '"' *) add_quoted_field ic field_no ) else ( Buffer.add_char ic.current_field '='; add_unquoted_field ic ) with End_of_file -> ic.record <- "=" :: ic.record; false end else add_unquoted_field ic with End_of_file -> (* If it is the first field, coming from [next()], the field is made of spaces. If after the first, we are sure we read a delimiter before (but maybe the field is empty). Thus add en empty field. *) ic.record <- "" :: ic.record; false let next ic = if ic.in1 < 0 then raise(Sys_error "Bad file descriptor"); fill_in_buf ic; (* or End_of_file which means no more records *) ic.record <- []; ic.record_n <- ic.record_n + 1; (* the current line being read *) let more_fields = ref true and field_no = ref 1 in (* the current field being read *) while !more_fields do more_fields := add_next_field ic !field_no; incr field_no; done; ic.record <- List.rev ic.record; ic.record let current_record ic = ic.record let fold_left f a0 ic = let a = ref a0 in try while true do a := f !a (next ic) done; assert false with End_of_file -> !a let iter ~f ic = try while true do f (next ic) done; with End_of_file -> () let input_all ic = List.rev(fold_left (fun l r -> r :: l) [] ic) let fold_right f ic a0 = (* We to collect all records before applying [f] -- last row first. *) let lr = fold_left (fun l r -> r :: l) [] ic in List.fold_left (fun a r -> f r a) a0 lr let load ?separator ?excel_tricks fname = let fh = if fname = "-" then stdin else open_in fname in let csv = of_channel ?separator ?excel_tricks fh in let t = input_all csv in close_in csv; t let load_in ?separator ?excel_tricks ch = input_all (of_channel ?separator ?excel_tricks ch) (* @deprecated *) let load_rows ?separator ?excel_tricks f ch = iter f (of_channel ?separator ?excel_tricks ch) (* * Output *) (* FIXME: Rework this part *) type out_channel = { out_chan : out_obj_channel; out_separator : char; out_separator_string : string; out_excel_tricks : bool; } let to_out_obj ?(separator=',') ?(excel_tricks=false) out_chan = { out_chan = out_chan; out_separator = separator; out_separator_string = String.make 1 separator; out_excel_tricks = excel_tricks; } let to_channel ?separator ?excel_tricks fh = to_out_obj ?separator ?excel_tricks (object val fh = fh method output s ofs len = output fh s ofs len; len method close_out () = close_out fh end) let rec really_output oc s ofs len = let w = oc.out_chan#output s ofs len in if w < len then really_output oc s (ofs+w) (len-w) (* Determine whether the string s must be quoted and how many chars it must be extended to contain the escaped values. Return -1 if there is no need to quote. It is assumed that the string length [len] is > 0. *) let must_quote separator excel_tricks s len = let quote = ref(is_space(String.unsafe_get s 0) || is_space(String.unsafe_get s (len - 1))) in let n = ref 0 in for i = 0 to len - 1 do let c = String.unsafe_get s i in if c = separator || c = '\n' || c = '\r' then quote := true else if c = '"' || (excel_tricks && c = '\000') then ( quote := true; incr n) done; if !quote then !n else -1 let need_excel_trick s len = let c = String.unsafe_get s 0 in is_space c || c = '0' || is_space(String.unsafe_get s (len - 1)) (* Do some work to avoid quoting a field unless it is absolutely required. *) let write_escaped oc field = if String.length field > 0 then begin let len = String.length field in let use_excel_trick = oc.out_excel_tricks && need_excel_trick field len and n = must_quote oc.out_separator oc.out_excel_tricks field len in if n < 0 && not use_excel_trick then really_output oc field 0 len else ( let field = if n = 0 then field else (* There are some quotes to escape *) let s = String.create (len + n) in let j = ref 0 in for i = 0 to len - 1 do let c = String.unsafe_get field i in if c = '"' then ( String.unsafe_set s !j '"'; incr j; String.unsafe_set s !j '"'; incr j ) else if oc.out_excel_tricks && c = '\000' then ( String.unsafe_set s !j '"'; incr j; String.unsafe_set s !j '0'; incr j ) else (String.unsafe_set s !j c; incr j) done; s in if use_excel_trick then really_output oc "=\"" 0 2 else really_output oc "\"" 0 1; really_output oc field 0 (String.length field); really_output oc "\"" 0 1 ) end let output_record oc = function | [] -> really_output oc "\n" 0 1 | [f] -> write_escaped oc f; really_output oc "\n" 0 1 | f :: tl -> write_escaped oc f; List.iter (fun f -> really_output oc oc.out_separator_string 0 1; write_escaped oc f; ) tl; really_output oc "\n" 0 1 let output_all oc t = List.iter (fun r -> output_record oc r) t let print ?separator ?excel_tricks t = let csv = to_channel ?separator ?excel_tricks stdout in output_all csv t; flush stdout let save_out ?separator ?excel_tricks ch t = let csv = to_channel ?separator ?excel_tricks ch in output_all csv t let save ?separator ?excel_tricks fname t = let ch = open_out fname in let csv = to_channel ?separator ?excel_tricks ch in output_all csv t; close_out ch (* * Acting on CSV data in memory *) let lines = List.length let columns csv = let m = ref 0 in List.iter (fun row -> m := max !m (List.length row)) csv; !m let rec dropwhile f = function | [] -> [] | x :: xs when f x -> dropwhile f xs | xs -> xs let rec empty_row = function | [] -> true | "" :: xs -> empty_row xs | _ :: _ -> false let trim ?(top=true) ?(left=true) ?(right=true) ?(bottom=true) csv = let csv = if top then dropwhile empty_row csv else csv in let csv = if right then List.map (fun row -> let row = List.rev row in let row = dropwhile ((=) "") row in let row = List.rev row in row) csv else csv in let csv = if bottom then ( let csv = List.rev csv in let csv = dropwhile empty_row csv in let csv = List.rev csv in csv ) else csv in let empty_left_cell = function [] -> true | x :: _ when x = "" -> true | _ -> false in let empty_left_col = List.fold_left (fun a row -> a && empty_left_cell row) true in let remove_left_col = List.map (function [] -> [] | _ :: xs -> xs) in let rec loop csv = if empty_left_col csv then remove_left_col csv else csv in let csv = if left then loop csv else csv in csv let square csv = let columns = columns csv in List.map ( fun row -> let n = List.length row in let row = List.rev row in let rec loop acc = function | 0 -> acc | i -> "" :: loop acc (i-1) in let row = loop row (columns - n) in List.rev row ) csv let is_square csv = let columns = columns csv in List.for_all (fun row -> List.length row = columns) csv let rec set_columns cols = function | [] -> [] | r :: rs -> let rec loop i cells = if i < cols then ( match cells with | [] -> "" :: loop (succ i) [] | c :: cs -> c :: loop (succ i) cs ) else [] in loop 0 r :: set_columns cols rs let rec set_rows rows csv = if rows > 0 then ( match csv with | [] -> [] :: set_rows (pred rows) [] | r :: rs -> r :: set_rows (pred rows) rs ) else [] let set_size rows cols csv = set_columns cols (set_rows rows csv) (* from extlib: *) let rec drop n = function | _ :: l when n > 0 -> drop (n-1) l | l -> l let sub r c rows cols csv = let csv = drop r csv in let csv = List.map (drop c) csv in let csv = set_rows rows csv in let csv = set_columns cols csv in csv (* Compare two rows for semantic equality - ignoring any blank cells * at the end of each row. *) let rec compare_row (row1 : string list) row2 = match row1, row2 with | [], [] -> 0 | x :: xs, y :: ys -> let c = compare x y in if c <> 0 then c else compare_row xs ys | "" :: xs , [] -> compare_row xs [] | _ :: _, [] -> 1 | [], "" :: ys -> compare_row [] ys | [], _ :: _ -> -1 (* Semantic equality for CSV files. *) let rec compare (csv1 : t) csv2 = match csv1, csv2 with | [], [] -> 0 | x :: xs, y :: ys -> let c = compare_row x y in if c <> 0 then c else compare xs ys | x :: xs, [] -> let c = compare_row x [] in if c <> 0 then c else compare xs [] | [], y :: ys -> let c = compare_row [] y in if c <> 0 then c else compare [] ys (* Concatenate - arrange left to right. *) let rec concat = function | [] -> [] | [csv] -> csv | left_csv :: csvs -> (* Concatenate the remaining CSV files. *) let right_csv = concat csvs in (* Set the height of the left and right CSVs to the same. *) let nr_rows = max (lines left_csv) (lines right_csv) in let left_csv = set_rows nr_rows left_csv in let right_csv = set_rows nr_rows right_csv in (* Square off the left CSV. *) let left_csv = square left_csv in (* Prepend the right CSV rows with the left CSV rows. *) List.map ( fun (left_row, right_row) -> List.append left_row right_row ) (List.combine left_csv right_csv) let transpose = (* Suppose the CSV data is presented with the last row first. Then new rows may be constructed in a tail rec way. We use mutable rows in order to preserve tail recursiveness. *) (* Return the new 1st row; whether all rows are empty. *) let rec row_of_1st_col tr_row empty = function | [] -> (tr_row, empty) (* No more rows *) | r :: rows -> match !r with | [] -> (* Last row empty *) let tr_row = if tr_row = [] then tr_row else "" :: tr_row in row_of_1st_col tr_row empty rows | a :: tl -> r := tl; let tr_row = if a = "" && tr_row = [] then [] else a :: tr_row in row_of_1st_col tr_row false rows in let rec tr tr_csv csv = let row, empty = row_of_1st_col [] true csv in (* remove [csv] 1st col *) if empty then List.rev tr_csv else tr (row :: tr_csv) csv in fun csv -> tr [] (List.rev_map ref csv) let to_array csv = Array.of_list (List.map Array.of_list csv) let of_array csv = List.map Array.to_list (Array.to_list csv) let associate header data = let nr_cols = List.length header in let rec trunc = function | 0, _ -> [] | n, [] -> "" :: trunc (n-1, []) | n, (x :: xs) -> x :: trunc (n-1, xs) in List.map ( fun row -> let row = trunc (nr_cols, row) in List.combine header row ) data let save_out_readable chan csv = (* Escape all the strings in the CSV file first. *) (* XXX Why are we doing this? I commented it out anyway. let csv = List.map (List.map String.escaped) csv in *) (* Find the width of each column. *) let widths = (* Don't consider rows with only a single element - typically * long titles. *) let csv = List.filter (function [_] -> false | _ -> true) csv in (* Square the CSV file - makes the next step simpler to implement. *) let csv = square csv in match csv with | [] -> [] | row1 :: rest -> let lengths_row1 = List.map String.length row1 in let lengths_rest = List.map (List.map String.length) rest in let max2rows r1 r2 = let rp = try List.combine r1 r2 with Invalid_argument _ -> failwith (Printf.sprintf "Csv.save_out_readable: internal \ error: length r1 = %d, length r2 = %d" (List.length r1) (List.length r2)) in List.map (fun ((a : int), (b : int)) -> max a b) rp in List.fold_left max2rows lengths_row1 lengths_rest in (* Print out each cell at the correct width. *) let rec repeat f = function | 0 -> () | i -> f (); repeat f (i-1) in List.iter ( function | [cell] -> (* Single column. *) output_string chan cell; output_char chan '\n' | row -> (* Other. *) (* Pair up each cell with its max width. *) let row = let rec loop = function | ([], _) -> [] | (_, []) -> failwith "Csv.save_out_readable: internal error" | (cell :: cells, width :: widths) -> (cell, width) :: loop (cells, widths) in loop (row, widths) in List.iter ( fun (cell, width) -> output_string chan cell; let n = String.length cell in repeat (fun () -> output_char chan ' ') (width - n + 1) ) row; output_char chan '\n' ) csv let print_readable = save_out_readable stdout csv-1.3.2/src/csv.mllib0000644000175000017500000000013012236526204013371 0ustar trchtrch# OASIS_START # DO NOT EDIT (digest: b20daea8bfacec3891fcf5b7be4adca0) Csv # OASIS_STOP csv-1.3.2/src/Makefile0000644000175000017500000000165712236526203013233 0ustar trchtrchROOT=.. include $(ROOT)/Makefile.conf # Native code versions must be installed if and only if make opt was # called previously (i.e. the files exist): INSTALL_FILES = csv.mli $(wildcard *.cmi *.cma *.cmx *.a *.cmxa) .PHONY: default all opt byte native default: opt all: byte opt: native byte: csv.cma native: csv.cmxa csv.cma: csv.cmo csv.cmxa: csv.cmx .PHONY: doc doc: test -f $(DOC_DIR) || mkdir -p $(DOC_DIR) $(OCAMLDOC) -html -d $(DOC_DIR) $(OCAMLDOC_FLAGS) $(wildcard *.mli) .PHONY: install uninstall # TODO: copy HTML doc install: META $(INSTALL_FILES) ocamlfind install $(PACKAGE) $^ uninstall: ocamlfind remove $(PACKAGE) META: META.in @ sed -e "s/@PACKAGE@/$(PACKAGE)/" $< \ | sed -e "s/@VERSION@/$(VERSION)/" > $@ @ echo "Updated \"$@\"." # Make.bat -- easy compilation on win32 Make.bat: $(MAKE) clean # Filter out all "make" messages $(MAKE) all | grep --invert-match "make" > $@ include $(ROOT)/Makefile.ocaml csv-1.3.2/src/META0000644000175000017500000000047512236526204012242 0ustar trchtrch# OASIS_START # DO NOT EDIT (digest: 2e5599ee740dd369f8fbe4dce85222f8) version = "1.3.2" description = "A pure OCaml library to read and write CSV files." archive(byte) = "csv.cma" archive(byte, plugin) = "csv.cma" archive(native) = "csv.cmxa" archive(native, plugin) = "csv.cmxs" exists_if = "csv.cma" # OASIS_STOP csv-1.3.2/myocamlbuild.ml0000644000175000017500000003314012236526204014010 0ustar trchtrch(* OASIS_START *) (* DO NOT EDIT (digest: 36fec46b39d49f881fbb5b0db76af130) *) module OASISGettext = struct (* # 21 "src/oasis/OASISGettext.ml" *) let ns_ str = str let s_ str = str let f_ (str : ('a, 'b, 'c, 'd) format4) = str let fn_ fmt1 fmt2 n = if n = 1 then fmt1^^"" else fmt2^^"" let init = [] end module OASISExpr = struct (* # 21 "src/oasis/OASISExpr.ml" *) open OASISGettext type test = string type flag = string type t = | EBool of bool | ENot of t | EAnd of t * t | EOr of t * t | EFlag of flag | ETest of test * string type 'a choices = (t * 'a) list let eval var_get t = let rec eval' = function | EBool b -> b | ENot e -> not (eval' e) | EAnd (e1, e2) -> (eval' e1) && (eval' e2) | EOr (e1, e2) -> (eval' e1) || (eval' e2) | EFlag nm -> let v = var_get nm in assert(v = "true" || v = "false"); (v = "true") | ETest (nm, vl) -> let v = var_get nm in (v = vl) in eval' t let choose ?printer ?name var_get lst = let rec choose_aux = function | (cond, vl) :: tl -> if eval var_get cond then vl else choose_aux tl | [] -> let str_lst = if lst = [] then s_ "" else String.concat (s_ ", ") (List.map (fun (cond, vl) -> match printer with | Some p -> p vl | None -> s_ "") lst) in match name with | Some nm -> failwith (Printf.sprintf (f_ "No result for the choice list '%s': %s") nm str_lst) | None -> failwith (Printf.sprintf (f_ "No result for a choice list: %s") str_lst) in choose_aux (List.rev lst) end # 117 "myocamlbuild.ml" module BaseEnvLight = struct (* # 21 "src/base/BaseEnvLight.ml" *) module MapString = Map.Make(String) type t = string MapString.t let default_filename = Filename.concat (Sys.getcwd ()) "setup.data" let load ?(allow_empty=false) ?(filename=default_filename) () = if Sys.file_exists filename then begin let chn = open_in_bin filename in let st = Stream.of_channel chn in let line = ref 1 in let st_line = Stream.from (fun _ -> try match Stream.next st with | '\n' -> incr line; Some '\n' | c -> Some c with Stream.Failure -> None) in let lexer = Genlex.make_lexer ["="] st_line in let rec read_file mp = match Stream.npeek 3 lexer with | [Genlex.Ident nm; Genlex.Kwd "="; Genlex.String value] -> Stream.junk lexer; Stream.junk lexer; Stream.junk lexer; read_file (MapString.add nm value mp) | [] -> mp | _ -> failwith (Printf.sprintf "Malformed data file '%s' line %d" filename !line) in let mp = read_file MapString.empty in close_in chn; mp end else if allow_empty then begin MapString.empty end else begin failwith (Printf.sprintf "Unable to load environment, the file '%s' doesn't exist." filename) end let var_get name env = let rec var_expand str = let buff = Buffer.create ((String.length str) * 2) in Buffer.add_substitute buff (fun var -> try var_expand (MapString.find var env) with Not_found -> failwith (Printf.sprintf "No variable %s defined when trying to expand %S." var str)) str; Buffer.contents buff in var_expand (MapString.find name env) let var_choose lst env = OASISExpr.choose (fun nm -> var_get nm env) lst end # 215 "myocamlbuild.ml" module MyOCamlbuildFindlib = struct (* # 21 "src/plugins/ocamlbuild/MyOCamlbuildFindlib.ml" *) (** OCamlbuild extension, copied from * http://brion.inria.fr/gallium/index.php/Using_ocamlfind_with_ocamlbuild * by N. Pouillard and others * * Updated on 2009/02/28 * * Modified by Sylvain Le Gall *) open Ocamlbuild_plugin (* these functions are not really officially exported *) let run_and_read = Ocamlbuild_pack.My_unix.run_and_read let blank_sep_strings = Ocamlbuild_pack.Lexers.blank_sep_strings let split s ch = let x = ref [] in let rec go s = let pos = String.index s ch in x := (String.before s pos)::!x; go (String.after s (pos + 1)) in try go s with Not_found -> !x let split_nl s = split s '\n' let before_space s = try String.before s (String.index s ' ') with Not_found -> s (* this lists all supported packages *) let find_packages () = List.map before_space (split_nl & run_and_read "ocamlfind list") (* this is supposed to list available syntaxes, but I don't know how to do it. *) let find_syntaxes () = ["camlp4o"; "camlp4r"] (* ocamlfind command *) let ocamlfind x = S[A"ocamlfind"; x] let dispatch = function | Before_options -> (* by using Before_options one let command line options have an higher priority *) (* on the contrary using After_options will guarantee to have the higher priority *) (* override default commands by ocamlfind ones *) Options.ocamlc := ocamlfind & A"ocamlc"; Options.ocamlopt := ocamlfind & A"ocamlopt"; Options.ocamldep := ocamlfind & A"ocamldep"; Options.ocamldoc := ocamlfind & A"ocamldoc"; Options.ocamlmktop := ocamlfind & A"ocamlmktop" | After_rules -> (* When one link an OCaml library/binary/package, one should use -linkpkg *) flag ["ocaml"; "link"; "program"] & A"-linkpkg"; (* For each ocamlfind package one inject the -package option when * compiling, computing dependencies, generating documentation and * linking. *) List.iter begin fun pkg -> flag ["ocaml"; "compile"; "pkg_"^pkg] & S[A"-package"; A pkg]; flag ["ocaml"; "ocamldep"; "pkg_"^pkg] & S[A"-package"; A pkg]; flag ["ocaml"; "doc"; "pkg_"^pkg] & S[A"-package"; A pkg]; flag ["ocaml"; "link"; "pkg_"^pkg] & S[A"-package"; A pkg]; flag ["ocaml"; "infer_interface"; "pkg_"^pkg] & S[A"-package"; A pkg]; end (find_packages ()); (* Like -package but for extensions syntax. Morover -syntax is useless * when linking. *) List.iter begin fun syntax -> flag ["ocaml"; "compile"; "syntax_"^syntax] & S[A"-syntax"; A syntax]; flag ["ocaml"; "ocamldep"; "syntax_"^syntax] & S[A"-syntax"; A syntax]; flag ["ocaml"; "doc"; "syntax_"^syntax] & S[A"-syntax"; A syntax]; flag ["ocaml"; "infer_interface"; "syntax_"^syntax] & S[A"-syntax"; A syntax]; end (find_syntaxes ()); (* The default "thread" tag is not compatible with ocamlfind. * Indeed, the default rules add the "threads.cma" or "threads.cmxa" * options when using this tag. When using the "-linkpkg" option with * ocamlfind, this module will then be added twice on the command line. * * To solve this, one approach is to add the "-thread" option when using * the "threads" package using the previous plugin. *) flag ["ocaml"; "pkg_threads"; "compile"] (S[A "-thread"]); flag ["ocaml"; "pkg_threads"; "doc"] (S[A "-I"; A "+threads"]); flag ["ocaml"; "pkg_threads"; "link"] (S[A "-thread"]); flag ["ocaml"; "pkg_threads"; "infer_interface"] (S[A "-thread"]) | _ -> () end module MyOCamlbuildBase = struct (* # 21 "src/plugins/ocamlbuild/MyOCamlbuildBase.ml" *) (** Base functions for writing myocamlbuild.ml @author Sylvain Le Gall *) open Ocamlbuild_plugin module OC = Ocamlbuild_pack.Ocaml_compiler type dir = string type file = string type name = string type tag = string (* # 56 "src/plugins/ocamlbuild/MyOCamlbuildBase.ml" *) type t = { lib_ocaml: (name * dir list) list; lib_c: (name * dir * file list) list; flags: (tag list * (spec OASISExpr.choices)) list; (* Replace the 'dir: include' from _tags by a precise interdepends in * directory. *) includes: (dir * dir list) list; } let env_filename = Pathname.basename BaseEnvLight.default_filename let dispatch_combine lst = fun e -> List.iter (fun dispatch -> dispatch e) lst let tag_libstubs nm = "use_lib"^nm^"_stubs" let nm_libstubs nm = nm^"_stubs" let dispatch t e = let env = BaseEnvLight.load ~filename:env_filename ~allow_empty:true () in match e with | Before_options -> let no_trailing_dot s = if String.length s >= 1 && s.[0] = '.' then String.sub s 1 ((String.length s) - 1) else s in List.iter (fun (opt, var) -> try opt := no_trailing_dot (BaseEnvLight.var_get var env) with Not_found -> Printf.eprintf "W: Cannot get variable %s" var) [ Options.ext_obj, "ext_obj"; Options.ext_lib, "ext_lib"; Options.ext_dll, "ext_dll"; ] | After_rules -> (* Declare OCaml libraries *) List.iter (function | nm, [] -> ocaml_lib nm | nm, dir :: tl -> ocaml_lib ~dir:dir (dir^"/"^nm); List.iter (fun dir -> List.iter (fun str -> flag ["ocaml"; "use_"^nm; str] (S[A"-I"; P dir])) ["compile"; "infer_interface"; "doc"]) tl) t.lib_ocaml; (* Declare directories dependencies, replace "include" in _tags. *) List.iter (fun (dir, include_dirs) -> Pathname.define_context dir include_dirs) t.includes; (* Declare C libraries *) List.iter (fun (lib, dir, headers) -> (* Handle C part of library *) flag ["link"; "library"; "ocaml"; "byte"; tag_libstubs lib] (S[A"-dllib"; A("-l"^(nm_libstubs lib)); A"-cclib"; A("-l"^(nm_libstubs lib))]); flag ["link"; "library"; "ocaml"; "native"; tag_libstubs lib] (S[A"-cclib"; A("-l"^(nm_libstubs lib))]); flag ["link"; "program"; "ocaml"; "byte"; tag_libstubs lib] (S[A"-dllib"; A("dll"^(nm_libstubs lib))]); (* When ocaml link something that use the C library, then one need that file to be up to date. *) dep ["link"; "ocaml"; "program"; tag_libstubs lib] [dir/"lib"^(nm_libstubs lib)^"."^(!Options.ext_lib)]; dep ["compile"; "ocaml"; "program"; tag_libstubs lib] [dir/"lib"^(nm_libstubs lib)^"."^(!Options.ext_lib)]; (* TODO: be more specific about what depends on headers *) (* Depends on .h files *) dep ["compile"; "c"] headers; (* Setup search path for lib *) flag ["link"; "ocaml"; "use_"^lib] (S[A"-I"; P(dir)]); ) t.lib_c; (* Add flags *) List.iter (fun (tags, cond_specs) -> let spec = BaseEnvLight.var_choose cond_specs env in flag tags & spec) t.flags | _ -> () let dispatch_default t = dispatch_combine [ dispatch t; MyOCamlbuildFindlib.dispatch; ] end # 476 "myocamlbuild.ml" open Ocamlbuild_plugin;; let package_default = { MyOCamlbuildBase.lib_ocaml = [("csv", ["src"])]; lib_c = []; flags = []; includes = [("tests", ["src"]); ("examples", ["src"])]; } ;; let dispatch_default = MyOCamlbuildBase.dispatch_default package_default;; # 490 "myocamlbuild.ml" (* OASIS_STOP *) Ocamlbuild_plugin.dispatch dispatch_default;; csv-1.3.2/LICENSE.txt0000644000175000017500000006534512236526203012633 0ustar trchtrchThis Library is distributed under the terms of the GNU Lesser General Public License version 2.1 (included below). As a special exception to the GNU Lesser General Public License, you may link, statically or dynamically, a "work that uses the Library" with a publicly distributed version of the Library to produce an executable file containing portions of the Library, and distribute that executable file under terms of your choice, without any of the additional requirements listed in clause 6 of the GNU Lesser General Public License. By "a publicly distributed version of the Library", we mean either the unmodified Library as distributed, or a modified version of the Library that is distributed under the conditions defined in clause 3 of the GNU Library General Public License. This exception does not however invalidate any other reasons why the executable file might be covered by the GNU Lesser General Public License. GNU LESSER GENERAL PUBLIC LICENSE Version 2.1, February 1999 Copyright (C) 1991, 1999 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. [This is the first released version of the Lesser GPL. It also counts as the successor of the GNU Library Public License, version 2, hence the version number 2.1.] Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public Licenses are intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This license, the Lesser General Public License, applies to some specially designated software packages--typically libraries--of the Free Software Foundation and other authors who decide to use it. You can use it too, but we suggest you first think carefully about whether this license or the ordinary General Public License is the better strategy to use in any particular case, based on the explanations below. When we speak of free software, we are referring to freedom of use, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish); that you receive source code or can get it if you want it; that you can change the software and use pieces of it in new free programs; and that you are informed that you can do these things. To protect your rights, we need to make restrictions that forbid distributors to deny you these rights or to ask you to surrender these rights. These restrictions translate to certain responsibilities for you if you distribute copies of the library or if you modify it. For example, if you distribute copies of the library, whether gratis or for a fee, you must give the recipients all the rights that we gave you. You must make sure that they, too, receive or can get the source code. If you link other code with the library, you must provide complete object files to the recipients, so that they can relink them with the library after making changes to the library and recompiling it. And you must show them these terms so they know their rights. We protect your rights with a two-step method: (1) we copyright the library, and (2) we offer you this license, which gives you legal permission to copy, distribute and/or modify the library. To protect each distributor, we want to make it very clear that there is no warranty for the free library. Also, if the library is modified by someone else and passed on, the recipients should know that what they have is not the original version, so that the original author's reputation will not be affected by problems that might be introduced by others. Finally, software patents pose a constant threat to the existence of any free program. We wish to make sure that a company cannot effectively restrict the users of a free program by obtaining a restrictive license from a patent holder. Therefore, we insist that any patent license obtained for a version of the library must be consistent with the full freedom of use specified in this license. Most GNU software, including some libraries, is covered by the ordinary GNU General Public License. This license, the GNU Lesser General Public License, applies to certain designated libraries, and is quite different from the ordinary General Public License. We use this license for certain libraries in order to permit linking those libraries into non-free programs. When a program is linked with a library, whether statically or using a shared library, the combination of the two is legally speaking a combined work, a derivative of the original library. The ordinary General Public License therefore permits such linking only if the entire combination fits its criteria of freedom. The Lesser General Public License permits more lax criteria for linking other code with the library. We call this license the "Lesser" General Public License because it does Less to protect the user's freedom than the ordinary General Public License. It also provides other free software developers Less of an advantage over competing non-free programs. These disadvantages are the reason we use the ordinary General Public License for many libraries. However, the Lesser license provides advantages in certain special circumstances. For example, on rare occasions, there may be a special need to encourage the widest possible use of a certain library, so that it becomes a de-facto standard. To achieve this, non-free programs must be allowed to use the library. A more frequent case is that a free library does the same job as widely used non-free libraries. In this case, there is little to gain by limiting the free library to free software only, so we use the Lesser General Public License. In other cases, permission to use a particular library in non-free programs enables a greater number of people to use a large body of free software. For example, permission to use the GNU C Library in non-free programs enables many more people to use the whole GNU operating system, as well as its variant, the GNU/Linux operating system. Although the Lesser General Public License is Less protective of the users' freedom, it does ensure that the user of a program that is linked with the Library has the freedom and the wherewithal to run that program using a modified version of the Library. The precise terms and conditions for copying, distribution and modification follow. Pay close attention to the difference between a "work based on the library" and a "work that uses the library". The former contains code derived from the library, whereas the latter must be combined with the library in order to run. GNU LESSER GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License Agreement applies to any software library or other program which contains a notice placed by the copyright holder or other authorized party saying it may be distributed under the terms of this Lesser General Public License (also called "this License"). Each licensee is addressed as "you". A "library" means a collection of software functions and/or data prepared so as to be conveniently linked with application programs (which use some of those functions and data) to form executables. The "Library", below, refers to any such software library or work which has been distributed under these terms. A "work based on the Library" means either the Library or any derivative work under copyright law: that is to say, a work containing the Library or a portion of it, either verbatim or with modifications and/or translated straightforwardly into another language. (Hereinafter, translation is included without limitation in the term "modification".) "Source code" for a work means the preferred form of the work for making modifications to it. For a library, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the library. Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running a program using the Library is not restricted, and output from such a program is covered only if its contents constitute a work based on the Library (independent of the use of the Library in a tool for writing it). Whether that is true depends on what the Library does and what the program that uses the Library does. 1. You may copy and distribute verbatim copies of the Library's complete source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and distribute a copy of this License along with the Library. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Library or any portion of it, thus forming a work based on the Library, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) The modified work must itself be a software library. b) You must cause the files modified to carry prominent notices stating that you changed the files and the date of any change. c) You must cause the whole of the work to be licensed at no charge to all third parties under the terms of this License. d) If a facility in the modified Library refers to a function or a table of data to be supplied by an application program that uses the facility, other than as an argument passed when the facility is invoked, then you must make a good faith effort to ensure that, in the event an application does not supply such function or table, the facility still operates, and performs whatever part of its purpose remains meaningful. (For example, a function in a library to compute square roots has a purpose that is entirely well-defined independent of the application. Therefore, Subsection 2d requires that any application-supplied function or table used by this function must be optional: if the application does not supply it, the square root function must still compute square roots.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Library, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Library, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Library. In addition, mere aggregation of another work not based on the Library with the Library (or with a work based on the Library) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may opt to apply the terms of the ordinary GNU General Public License instead of this License to a given copy of the Library. To do this, you must alter all the notices that refer to this License, so that they refer to the ordinary GNU General Public License, version 2, instead of to this License. (If a newer version than version 2 of the ordinary GNU General Public License has appeared, then you can specify that version instead if you wish.) Do not make any other change in these notices. Once this change is made in a given copy, it is irreversible for that copy, so the ordinary GNU General Public License applies to all subsequent copies and derivative works made from that copy. This option is useful when you wish to copy part of the code of the Library into a program that is not a library. 4. You may copy and distribute the Library (or a portion or derivative of it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange. If distribution of object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place satisfies the requirement to distribute the source code, even though third parties are not compelled to copy the source along with the object code. 5. A program that contains no derivative of any portion of the Library, but is designed to work with the Library by being compiled or linked with it, is called a "work that uses the Library". Such a work, in isolation, is not a derivative work of the Library, and therefore falls outside the scope of this License. However, linking a "work that uses the Library" with the Library creates an executable that is a derivative of the Library (because it contains portions of the Library), rather than a "work that uses the library". The executable is therefore covered by this License. Section 6 states terms for distribution of such executables. When a "work that uses the Library" uses material from a header file that is part of the Library, the object code for the work may be a derivative work of the Library even though the source code is not. Whether this is true is especially significant if the work can be linked without the Library, or if the work is itself a library. The threshold for this to be true is not precisely defined by law. If such an object file uses only numerical parameters, data structure layouts and accessors, and small macros and small inline functions (ten lines or less in length), then the use of the object file is unrestricted, regardless of whether it is legally a derivative work. (Executables containing this object code plus portions of the Library will still fall under Section 6.) Otherwise, if the work is a derivative of the Library, you may distribute the object code for the work under the terms of Section 6. Any executables containing that work also fall under Section 6, whether or not they are linked directly with the Library itself. 6. As an exception to the Sections above, you may also combine or link a "work that uses the Library" with the Library to produce a work containing portions of the Library, and distribute that work under terms of your choice, provided that the terms permit modification of the work for the customer's own use and reverse engineering for debugging such modifications. You must give prominent notice with each copy of the work that the Library is used in it and that the Library and its use are covered by this License. You must supply a copy of this License. If the work during execution displays copyright notices, you must include the copyright notice for the Library among them, as well as a reference directing the user to the copy of this License. Also, you must do one of these things: a) Accompany the work with the complete corresponding machine-readable source code for the Library including whatever changes were used in the work (which must be distributed under Sections 1 and 2 above); and, if the work is an executable linked with the Library, with the complete machine-readable "work that uses the Library", as object code and/or source code, so that the user can modify the Library and then relink to produce a modified executable containing the modified Library. (It is understood that the user who changes the contents of definitions files in the Library will not necessarily be able to recompile the application to use the modified definitions.) b) Use a suitable shared library mechanism for linking with the Library. A suitable mechanism is one that (1) uses at run time a copy of the library already present on the user's computer system, rather than copying library functions into the executable, and (2) will operate properly with a modified version of the library, if the user installs one, as long as the modified version is interface-compatible with the version that the work was made with. c) Accompany the work with a written offer, valid for at least three years, to give the same user the materials specified in Subsection 6a, above, for a charge no more than the cost of performing this distribution. d) If distribution of the work is made by offering access to copy from a designated place, offer equivalent access to copy the above specified materials from the same place. e) Verify that the user has already received a copy of these materials or that you have already sent this user a copy. For an executable, the required form of the "work that uses the Library" must include any data and utility programs needed for reproducing the executable from it. However, as a special exception, the materials to be distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. It may happen that this requirement contradicts the license restrictions of other proprietary libraries that do not normally accompany the operating system. Such a contradiction means you cannot use both them and the Library together in an executable that you distribute. 7. You may place library facilities that are a work based on the Library side-by-side in a single library together with other library facilities not covered by this License, and distribute such a combined library, provided that the separate distribution of the work based on the Library and of the other library facilities is otherwise permitted, and provided that you do these two things: a) Accompany the combined library with a copy of the same work based on the Library, uncombined with any other library facilities. This must be distributed under the terms of the Sections above. b) Give prominent notice with the combined library of the fact that part of it is a work based on the Library, and explaining where to find the accompanying uncombined form of the same work. 8. You may not copy, modify, sublicense, link with, or distribute the Library except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, link with, or distribute the Library is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 9. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Library or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Library (or any work based on the Library), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Library or works based on it. 10. Each time you redistribute the Library (or any work based on the Library), the recipient automatically receives a license from the original licensor to copy, distribute, link with or modify the Library subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties with this License. 11. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Library at all. For example, if a patent license would not permit royalty-free redistribution of the Library by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Library. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply, and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 12. If the distribution and/or use of the Library is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Library under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 13. The Free Software Foundation may publish revised and/or new versions of the Lesser General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Library specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Library does not specify a license version number, you may choose any version ever published by the Free Software Foundation. 14. If you wish to incorporate parts of the Library into other free programs whose distribution conditions are incompatible with these, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Libraries If you develop a new library, and you want it to be of the greatest possible use to the public, we recommend making it free software that everyone can redistribute and change. You can do so by permitting redistribution under these terms (or, alternatively, under the terms of the ordinary General Public License). To apply these terms, attach the following notices to the library. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Also add information on how to contact you by electronic and paper mail. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the library, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the library `Frob' (a library for tweaking knobs) written by James Random Hacker. , 1 April 1990 Ty Coon, President of Vice That's all there is to it! csv-1.3.2/API.odocl0000644000175000017500000000013412236526204012425 0ustar trchtrch# OASIS_START # DO NOT EDIT (digest: f6e881d172d1787cd0a44c015176b6db) src/Csv # OASIS_STOP csv-1.3.2/tests/0000755000175000017500000000000012236526203012135 5ustar trchtrchcsv-1.3.2/tests/testcsv9.csv0000644000175000017500000000002212236526203014430 0ustar trchtrchA1,A2,A3 B1,B2 C1 csv-1.3.2/tests/testcsv4.csv0000644000175000017500000000000012236526203014417 0ustar trchtrchcsv-1.3.2/tests/testcsv8.csv0000644000175000017500000000003612236526203014434 0ustar trchtrch"Foo" "Bar" "Baz" "Boof" a c csv-1.3.2/tests/testcsv7.csv0000644000175000017500000000013212236526203014430 0ustar trchtrch Initial , and , final, , spaces , do not matter " Quoted spaces ", are, " important " csv-1.3.2/tests/test.ml0000644000175000017500000000435212236526203013452 0ustar trchtrchopen Printf let do_testcsv ?separator filename expected = try let csv = Csv.load ?separator filename in if csv <> expected then ( printf "input file: %s\n" filename; printf "Csv library produced:\n"; Csv.print csv; printf "Expected:\n"; Csv.print expected; failwith "failed" ) with Csv.Failure(nrow, nfield, err) -> printf "The file %S line %i, field %i, does not conform to the CSV \ specifications: %s\n" filename nrow nfield err; failwith "failed" let () = do_testcsv "testcsv1.csv" [ [ "This is a test\nwith commas,,,,,\n\nand carriage returns." ] ] let () = do_testcsv "testcsv2.csv" [ [ "Normal field"; "Quoted field"; "Quoted field with \"\" quotes" ] ] let () = do_testcsv "testcsv3.csv" [ [ "" ]; [ ""; "" ]; [ ""; ""; "" ]; [ ""; ""; ""; "" ]; [ ""; ""; ""; ""; "" ] ] let () = do_testcsv "testcsv4.csv" [] let () = do_testcsv "testcsv5.csv" [ [ "This is a test\nwith commas,,,,,\n\nand carriage returns."; "a second field"; "a third field" ]; [ "a fourth field on a new line" ] ] let () = do_testcsv "testcsv6.csv" [ [ "This is a test\nwith commas,,,,,\n\nand carriage returns\nand \000"; "a second field"; "a third field" ]; [ "a fourth field on a new line" ] ] let () = do_testcsv "testcsv7.csv" [ [ "Initial"; "and"; "final"; ""; "spaces"; "do not matter" ]; [ " Quoted spaces "; "are"; " important " ] ] let () = do_testcsv ~separator:'\t' "testcsv8.csv" [["Foo"; "Bar"]; ["Baz"; "Boof"]; ["a"; ""; "c"]] let () = let csv1 = [ [ "a"; "b"; "c"; ""; "" ]; [ "f"; "g"; "h"; "i"; "" ]; [ "" ]; [ ] ] in let csv2 = Csv.trim ~top:false ~left:false ~right:true ~bottom:true csv1 in assert(compare csv1 csv2 <> 0); assert(Csv.compare csv1 csv2 = 0) let () = let csv1 = [ [ "a"; "b"; "c"; ""; "" ]; [ "f"; "g"; "h"; "i"; "" ]; [ "" ]; [ ] ] in let csv2 = [ [ "a"; "b"; "c"; "d"; "" ]; [ "f"; "g"; "h"; "i"; "" ]; [ "" ]; [ ] ] in assert (Csv.compare csv1 csv2 < 0) let () = print_endline "All tests succeeded." csv-1.3.2/tests/testcsv5.csv0000644000175000017500000000016212236526203014431 0ustar trchtrch"This is a test with commas,,,,, and carriage returns.",a second field,a third field a fourth field on a new linecsv-1.3.2/tests/testcsv2.csv0000644000175000017500000000007412236526203014430 0ustar trchtrchNormal field,"Quoted field","Quoted field with """" quotes" csv-1.3.2/tests/testcsv6.csv0000644000175000017500000000017012236526203014431 0ustar trchtrch"This is a test with commas,,,,, and carriage returns and "0",a second field,a third field a fourth field on a new linecsv-1.3.2/tests/testcsv1.csv0000644000175000017500000000007012236526203014423 0ustar trchtrch"This is a test with commas,,,,, and carriage returns."csv-1.3.2/tests/testcsv3.csv0000644000175000017500000000001612236526203014425 0ustar trchtrch , ,, ,,, ,,,,csv-1.3.2/AUTHORS.txt0000644000175000017500000000034612236526204012665 0ustar trchtrch(* OASIS_START *) (* DO NOT EDIT (digest: c0436713be9bd4c24dc3ff6d25d92eea) *) Authors of csv Richard Jones Christophe Troestler Current maintainers of csv Christophe Troestler (* OASIS_STOP *) csv-1.3.2/Makefile0000644000175000017500000000351012236526203012432 0ustar trchtrchPKGNAME = $(shell oasis query name) PKGVERSION = $(shell oasis query version) PKG_TARBALL = $(PKGNAME)-$(PKGVERSION).tar.gz OCAMLFORGE_FILE_NO = 1148 DISTFILES = LICENSE.txt INSTALL.txt README.md _oasis \ _tags Makefile setup.ml \ $(filter-out %~, $(wildcard src/*) $(wildcard examples/*) $(wildcard tests/*)) WEB = shell.forge.ocamlcore.org:/home/groups/csv/htdocs .PHONY: all byte native configure doc test install uninstall reinstall \ upload-doc all byte native: configure ocaml setup.ml -build configure: setup.ml ocaml $< -configure --enable-tests setup.ml: _oasis oasis setup -setup-update dynamic test doc install uninstall reinstall: all ocaml setup.ml -$@ upload-doc: doc scp -C -p -r _build/API.docdir $(WEB) csvtool: all ./csvtool.native pastecol 1-3 2,1,2 \ tests/testcsv9.csv tests/testcsv9.csv csv.godiva: csv.godiva.in @ sed -e "s/@PACKAGE@/$(PKGNAME)/" $< \ | sed -e "s/@VERSION@/$(PKGVERSION)/" \ | sed -e "s/@TARBALL@/$(PKG_TARBALL)/" \ | sed -e "s/@DOWNLOAD@/$(OCAMLFORGE_FILE_NO)/" > $@ @ echo "Updated \"$@\"." # Assume the environment variable $GODI_LOCALBASE is set .PHONY: godi tar dist web godi: csv.godiva godiva $< # "Force" a tag to be defined for each released tarball dist tar: setup.ml @ if [ -z "$(PKGNAME)" ]; then echo "PKGNAME not defined"; exit 1; fi @ if [ -z "$(PKGVERSION)" ]; then \ echo "PKGVERSION not defined"; exit 1; fi mkdir $(PKGNAME)-$(PKGVERSION) cp -r --parents $(DISTFILES) $(PKGNAME)-$(PKGVERSION)/ # Generate a setup.ml independent of oasis cd $(PKGNAME)-$(PKGVERSION) && oasis setup tar -zcvf $(PKG_TARBALL) $(PKGNAME)-$(PKGVERSION) $(RM) -rf $(PKGNAME)-$(PKGVERSION) web: doc $(MAKE) -C doc $@ .PHONY: clean distclean clean:: ocaml setup.ml -clean $(RM) $(PKG_TARBALL) distclean: ocaml setup.ml -distclean $(RM) $(wildcard *.ba[0-9] *.bak *~ *.odocl) csv-1.3.2/examples/0000755000175000017500000000000012236526203012611 5ustar trchtrchcsv-1.3.2/examples/csvtool.ml0000644000175000017500000006406312236526203014645 0ustar trchtrch(* Handy tool for managing CSV files. @author Richard Jones *) open Printf (*------------------------------ start of code from extlib *) exception Invalid_string let find str sub = let sublen = String.length sub in if sublen = 0 then 0 else let found = ref 0 in let len = String.length str in try for i = 0 to len - sublen do let j = ref 0 in while String.unsafe_get str (i + !j) = String.unsafe_get sub !j do incr j; if !j = sublen then begin found := i; raise Exit; end; done; done; raise Invalid_string with Exit -> !found let split str sep = let p = find str sep in let len = String.length sep in let slen = String.length str in String.sub str 0 p, String.sub str (p + len) (slen - p - len) let nsplit str sep = if str = "" then [] else ( let rec nsplit str sep = try let s1 , s2 = split str sep in s1 :: nsplit s2 sep with Invalid_string -> [str] in nsplit str sep ) type 'a mut_list = { hd: 'a; mutable tl: 'a list } external inj : 'a mut_list -> 'a list = "%identity" let dummy_node () = { hd = Obj.magic (); tl = [] } let rec drop n = function | _ :: l when n > 0 -> drop (n-1) l | l -> l let take n l = let rec loop n dst = function | h :: t when n > 0 -> let r = { hd = h; tl = [] } in dst.tl <- inj r; loop (n-1) r t | _ -> () in let dummy = dummy_node() in loop n dummy l; dummy.tl (*------------------------------ end of extlib code *) (* Parse column specs. *) type colspec = range list and range = | Col of int (* 0 *) | Range of int * int (* 2-5 *) | ToEnd of int (* 7- *) let parse_colspec ~count_zero colspec = let cols = nsplit colspec "," in let cols = List.map ( fun col -> try (try let first, second = split col "-" in if second <> "" then Range (int_of_string first, int_of_string second) else ToEnd (int_of_string first) with Invalid_string -> Col (int_of_string col) ) with Failure "int_of_string" -> failwith (colspec ^ ":" ^ col ^ ": invalid column-spec") ) cols in (* Adjust so columns always count from zero. *) if not count_zero then List.map ( function | Col c -> Col (c-1) | Range (s, e) -> Range (s-1, e-1) | ToEnd e -> ToEnd (e-1) ) cols else cols let rec width_of_colspec = function | [] -> 0 | Col c :: rest -> 1 + width_of_colspec rest | Range (s, e) :: rest -> (e-s+1) + width_of_colspec rest | ToEnd _ :: _ -> failwith "width_of_colspec: cannot calculate width of an open column spec (one which contains 'N-')" (* For closed column specs, this preserves the correct width in the * result. *) let cols_of_colspec colspec row = let rec loop = function | [] -> [] | Col c :: rest -> (try List.nth row c with Failure "nth" -> "") :: loop rest | Range (s, e) :: rest -> let width = e-s+1 in let range = take width (drop s row) in let range = List.hd (Csv.set_columns width [range]) in List.append range (loop rest) | ToEnd e :: rest -> List.append (drop e row) (loop rest) in loop colspec (* The actual commands. *) let cmd_cols ~input_sep ~output_sep ~chan colspec files = List.iter ( fun filename -> let csv = Csv.load ~separator:input_sep filename in let csv = List.map (cols_of_colspec colspec) csv in Csv.output_all (Csv.to_channel ~separator:output_sep chan) csv ) files let cmd_namedcols ~input_sep ~output_sep ~chan names files = List.iter ( fun filename -> let csv = Csv.load ~separator:input_sep filename in let header, data = match csv with | [] -> failwith "no rows in this CSV file" | h :: t -> h, t in (* Do the headers requested exist in the CSV file? If not, * throw an error. *) List.iter ( fun name -> if not (List.mem name header) then failwith ("namedcol: requested header not in CSV file: " ^ name) ) names; let data = Csv.associate header data in let data = List.map ( fun row -> List.map (fun name -> List.assoc name row) names ) data in let data = names :: data in Csv.output_all (Csv.to_channel ~separator:output_sep chan) data ) files let cmd_width ~input_sep ~chan files = let width = List.fold_left ( fun width filename -> let csv = Csv.load ~separator:input_sep filename in let width = max width (Csv.columns csv) in width ) 0 files in fprintf chan "%d\n" width let cmd_height ~input_sep ~chan files = let height = List.fold_left ( fun height filename -> let csv = Csv.load ~separator:input_sep filename in let height = height + Csv.lines csv in height ) 0 files in fprintf chan "%d\n" height let cmd_readable ~input_sep ~chan files = let csv = List.concat (List.map (Csv.load ~separator:input_sep) files) in Csv.save_out_readable chan csv let cmd_cat ~input_sep ~output_sep ~chan files = (* Avoid loading the whole file into memory. *) let chan = Csv.to_channel ~separator:output_sep chan in let f row = Csv.output_record chan row in List.iter ( fun filename -> let in_chan, close = match filename with | "-" -> stdin, false | filename -> open_in filename, true in Csv.iter f (Csv.of_channel ~separator:input_sep in_chan); if close then close_in in_chan ) files let cmd_paste ~input_sep ~output_sep ~chan files = (* Return the 1st row, concatenation of all 1st rows; whether all CSV files are empty; and the CSV files without their 1st row. *) let rec add_columns = function | [] -> ([], true, []) (* empty CSV file list *) | [] :: csvs -> (* exhausted the first CSV file *) let row, empty, csvs = add_columns csvs in (row, empty, [] :: csvs) | (r :: csv0) :: csvs -> let row, empty, csvs = add_columns csvs in (r @ row, false, csv0 :: csvs) in let rec paste_rows csvs final_csv = let row, empty, csvs = add_columns csvs in if empty then List.rev final_csv else paste_rows csvs (row :: final_csv) in let csvs = List.map (Csv.load ~separator:input_sep) files in let csv = paste_rows csvs [] in Csv.output_all (Csv.to_channel ~separator:output_sep chan) csv (* Given [colspec1] and [colspec2], return an associative list that indicates the correspondence between the i th column specified by [colspec1] and the corresponding one in [colspec2]. *) let rec colspec_map colspec1 colspec2 = match colspec1 with | [] -> [] | Col i :: tl1 -> (match colspec2 with | Col k :: tl2 -> (i,k) :: colspec_map tl1 tl2 | Range(k,l) :: tl2 -> let colspec2 = if k < l then Range(k+1, l) :: tl2 else if k = l then tl2 else (* k > l *) Range(k-1, l) :: tl2 in (i,k) :: colspec_map tl1 colspec2 | ToEnd k :: _ -> (i, k) :: colspec_map tl1 [ToEnd(k+1)] | [] -> failwith "pastecol: the second range does not contain \ enough columns") | Range(i,j) :: tl1 -> let colspec1 = if i < j then Range(i+1, j) :: tl1 else if i = j then tl1 else (* i > j *) Range(i-1, j) :: tl1 in (match colspec2 with | Col k :: tl2 -> (i,k) :: colspec_map colspec1 tl2 | Range(k,l) :: tl2 -> let colspec2 = if k < l then Range(k+1, l) :: tl2 else if k = l then tl2 else (* k > l *) Range(k-1, l) :: tl2 in (i,k) :: colspec_map colspec1 colspec2 | ToEnd k :: _ -> (i,k) :: colspec_map colspec1 [ToEnd(k+1)] | [] -> failwith "pastecol: the second range does not contain \ enough columns") | ToEnd i :: _ -> let m = sprintf "pastecol: the first range cannot contain an open \ range like %i-" i in failwith m (* When several bindings are defined for an initial column, use the last one. ASSUME that the associative map is sorted w.r.t. the first data. *) let rec reduce_colspec_map = function | (i,_) :: (((j,_) :: _) as tl) when (i: int) = j -> reduce_colspec_map tl (* maybe (j,_) is also supplanted *) | m :: tl -> m :: reduce_colspec_map tl | [] -> [] let cmd_pastecol ~input_sep ~output_sep ~chan colspec1 colspec2 file1 file2 = let csv1 = Csv.load ~separator:input_sep file1 in let csv2 = Csv.load ~separator:input_sep file2 in let m = colspec_map colspec1 colspec2 in let m = List.stable_sort (fun (i,_) (j,_) -> compare (i:int) j) m in let m = reduce_colspec_map m in let rec update m curr_col row1 row2 = match m with | [] -> row1 (* substitutions exhausted *) | (i, j) :: m_tl -> let c, row1 = match row1 with | [] -> "", [] (* row exhausted but some remaining substitutions must be performed. Create new columns. *) | c :: row1_tl -> c, row1_tl in if curr_col = i then let c' = try List.nth row2 j with _ -> "" in c' :: update m_tl (curr_col + 1) row1 row2 else (* curr_col < i *) c :: update m (curr_col + 1) row1 row2 in let csv = List.map2 (update m 0) csv1 csv2 in Csv.output_all (Csv.to_channel ~separator:output_sep chan) csv let cmd_set_columns ~input_sep ~output_sep ~chan cols files = (* Avoid loading the whole file into memory. *) let f row = let csv = [row] in let csv = Csv.set_columns cols csv in Csv.output_all (Csv.to_channel ~separator:output_sep chan) csv in List.iter ( fun filename -> let in_chan, close = match filename with | "-" -> stdin, false | filename -> open_in filename, true in Csv.iter f (Csv.of_channel ~separator:input_sep in_chan); if close then close_in in_chan ) files let cmd_set_rows ~input_sep ~output_sep ~chan rows files = let csv = List.concat (List.map (Csv.load ~separator:input_sep) files) in let csv = Csv.set_rows rows csv in Csv.output_all (Csv.to_channel ~separator:output_sep chan) csv let cmd_head ~input_sep ~output_sep ~chan rows files = (* Avoid loading the whole file into memory, or even loading * later files. *) let nr_rows = ref rows in let chan = Csv.to_channel ~separator:output_sep chan in let f row = if !nr_rows > 0 then ( decr nr_rows; Csv.output_record chan row ) in List.iter ( fun filename -> if !nr_rows > 0 then ( let in_chan, close = match filename with | "-" -> stdin, false | filename -> open_in filename, true in Csv.iter f (Csv.of_channel ~separator:input_sep in_chan); if close then close_in in_chan ) ) files let cmd_drop ~input_sep ~output_sep ~chan rows files = (* Avoid loading the whole file into memory. *) let nr_rows = ref rows in let chan = Csv.to_channel ~separator:output_sep chan in let f row = if !nr_rows = 0 then Csv.output_record chan row else decr nr_rows in List.iter ( fun filename -> let in_chan, close = match filename with | "-" -> stdin, false | filename -> open_in filename, true in Csv.iter f (Csv.of_channel ~separator:input_sep in_chan); if close then close_in in_chan ) files let cmd_square ~input_sep ~output_sep ~chan files = let csv = List.concat (List.map (Csv.load ~separator:input_sep) files) in let csv = Csv.square csv in Csv.output_all (Csv.to_channel ~separator:output_sep chan) csv let cmd_sub ~input_sep ~output_sep ~chan r c rows cols files = let csv = List.concat (List.map (Csv.load ~separator:input_sep) files) in let csv = Csv.sub r c rows cols csv in Csv.output_all (Csv.to_channel ~separator:output_sep chan) csv let cmd_replace ~input_sep ~output_sep ~chan colspec update files = let csv = List.concat (List.map (Csv.load ~separator:input_sep) files) in (* Load the update CSV file in. *) let update = Csv.load ~separator:input_sep update in (* Compare two rows for equality by considering only the columns * in colspec. *) let equal row1 row2 = let row1 = cols_of_colspec colspec row1 in let row2 = cols_of_colspec colspec row2 in 0 = Csv.compare [row1] [row2] in (* Look for rows in the original to be replaced by rows from the * update file. This is an ugly O(n^2) hack (XXX). *) let csv = List.filter ( fun row -> not (List.exists (equal row) update) ) csv in let csv = csv @ update in Csv.output_all (Csv.to_channel ~separator:output_sep chan) csv let cmd_transpose ~input_sep ~output_sep ~chan files = List.iter (fun file -> let tr = Csv.transpose (Csv.load ~separator:input_sep file) in Csv.output_all (Csv.to_channel ~separator:output_sep chan) tr ) files let cmd_call ~input_sep ~output_sep ~chan command files = (* Avoid loading the whole file into memory. *) let f row = let cmd = command ^ " " ^ String.concat " " (List.map Filename.quote row) in let code = Sys.command cmd in if code <> 0 then ( eprintf "%s: terminated with exit code %d\n" command code; exit code ) in List.iter ( fun filename -> let in_chan, close = match filename with | "-" -> stdin, false | filename -> open_in filename, true in Csv.iter f (Csv.of_channel ~separator:input_sep in_chan); if close then close_in in_chan ) files let rec uniq = function | [] -> [] | [x] -> [x] | x :: y :: xs when Pervasives.compare x y = 0 -> uniq (x :: xs) | x :: y :: xs -> x :: uniq (y :: xs) let cmd_join ~input_sep ~output_sep ~chan colspec1 colspec2 files = (* Load in the files separately. *) let csvs = List.map (Csv.load ~separator:input_sep) files in (* For each CSV file, construct a hash table from row class (key) to * the (possibly empty) output columns (values). * Also construct a hash which has the unique list of row classes. *) let keys = Hashtbl.create 1023 in let hashes = List.map ( fun csv -> let hash = Hashtbl.create 1023 in List.iter ( fun row -> let key = cols_of_colspec colspec1 row in let value = cols_of_colspec colspec2 row in if not (Hashtbl.mem keys key) then Hashtbl.add keys key true; Hashtbl.add hash key value ) csv; hash ) csvs in (* Get the keys. *) let keys = Hashtbl.fold (fun key _ xs -> key :: xs) keys [] in let value_width = width_of_colspec colspec2 in let empty_value = List.hd (Csv.set_columns value_width [[""]]) in let multiple_values = List.hd (Csv.set_columns value_width [["!MULTIPLE VALUES"]]) in (* Generate output CSV. *) let keys = List.sort Pervasives.compare keys in let keys = List.map (fun key -> key, []) keys in let csv = List.fold_left ( fun keys hash -> List.map ( fun (key, values) -> let value = try Hashtbl.find_all hash key with Not_found -> [] in let value = match value with | [] -> empty_value | [value] -> value | _::_ -> multiple_values in key, (value :: values) ) keys ) keys hashes in let csv = List.map ( fun (key, values) -> key @ List.flatten (List.rev values) ) csv in Csv.output_all (Csv.to_channel ~separator:output_sep chan) csv let rec cmd_trim ~input_sep ~output_sep ~chan (top, left, right, bottom) files = let csv = List.concat (List.map (Csv.load ~separator:input_sep) files) in let csv = Csv.trim ~top ~left ~right ~bottom csv in Csv.output_all (Csv.to_channel ~separator:output_sep chan) csv and trim_flags flags = let set c = try ignore (String.index flags c); true with Not_found -> false in let top = set 't' in let left = set 'l' in let right = set 'r' in let bottom = set 'b' in (top, left, right, bottom) (* Process the arguments. *) let usage = "csvtool - Copyright (C) 2005-2006 Richard W.M. Jones, Merjis Ltd. csvtool is a tool for performing manipulations on CSV files from shell scripts. Summary: csvtool [-options] command [command-args] input.csv [input2.csv [...]] Commands: col Return one or more columns from the CSV file. For , see below. Example: csvtool col 1-3,6 input.csv > output.csv namedcol Assuming the first row of the CSV file is a list of column headings, this returned the column(s) with the named headings. is a comma-separated list of names. Example: csvtool namedcol Account,Cost input.csv > output.csv width Print the maximum width of the CSV file (number of columns in the widest row). height Print the number of rows in the CSV file. For most CSV files this is equivalent to 'wc -l', but note that some CSV files can contain a row which breaks over two (or more) lines. setcolumns cols Set the number of columns to cols (this also makes the CSV file square). Any short rows are padding with blank cells. Any long rows are truncated. setrows rows 'setrows n' sets the number of rows to 'n'. If there are fewer than 'n' rows in the CSV files, then empty blank lines are added. head rows take rows 'head n' and 'take n' (which are synonyms) take the first 'n' rows. If there are fewer than 'n' rows, padding is not added. drop rows Drop the first 'rows' rows and return the rest (if any). Example: To remove the headings from a CSV file with headings: csvtool drop 1 input.csv > output.csv To extract rows 11 through 20 from a file: csvtool drop 10 input.csv | csvtool take 10 - > output.csv cat This concatenates the input files together and writes them to the output. You can use this to change the separator character. Example: csvtool -t TAB -u COMMA cat input.tsv > output.csv paste Concatenate the columns of the files together and write them to the output. Example: csvtool paste input1.csv input2.csv > output.csv pastecol input.csv update.csv Replace the content of the columns referenced by in the file input.csv with the one of the corresponding column specified by in update.csv. Example: csvtool pastecol 2-3 1- input.csv update.csv.csv > output.csv join Join (collate) multiple CSV files together. controls which columns are compared. controls which columns are copied into the new file. Example: csvtool join 1 2 coll1.csv coll2.csv > output.csv In the above example, if coll1.csv contains: Computers,$40 Software,$100 and coll2.csv contains: Computers,$50 then the output will be: Computers,$40,$50 Software,$100, square Make the CSV square, so all rows have the same length. Example: csvtool square input.csv > input-square.csv trim [tlrb]+ Trim empty cells at the top/left/right/bottom of the CSV file. Example: csvtool trim t input.csv # trims empty rows at the top only csvtool trim tb input.csv # trims empty rows at the top & bottom csvtool trim lr input.csv # trims empty columns at left & right csvtool trim tlrb input.csv # trims empty rows/columns all around sub r c rows cols Take a square subset of the CSV, top left at row r, column c, which is rows deep and cols wide. 'r' and 'c' count from 1, or from 0 if -z option is given. replace update.csv original.csv Replace rows in original.csv with rows from update.csv. The columns in only are used to compare rows in input.csv and update.csv to see if they are candidates for replacement. Example: csvtool replace 3 updates.csv original.csv > new.csv mv new.csv original.csv transpose input.csv Transpose the lines and columns of the CSV file. call command This calls the external command (or shell function) 'command' followed by a parameter for each column in the CSV file. The external command is called once for each row in the CSV file. If any command returns a non-zero exit code then the whole program terminates. Tip: Use the shell command 'export -f funcname' to export a shell function for use as a command. Within the function, use the positional parameters $1, $2, ... to refer to the columns. Example (with a shell function): function test { echo Column 1: $1 echo Column 2: $2 } export -f test csvtool call test my.csv In the above example, if my.csv contains: how,now brown,cow then the output is: Column 1: how Column 2: now Column 1: brown Column 2: cow readable Print the input CSV in a readable format. Column specs: A is a comma-separated list of column numbers or column ranges. Examples: 1 Column 1 (the first, leftmost column) 2,5,7 Columns 2, 5 and 7 1-3,5 Columns 1, 2, 3 and 5 1,5- Columns 1, 5 and up. Columns are numbered starting from 1 unless the -z option is given. Input files: csvtool takes a list of input file(s) from the command line. If an input filename is '-' then take input from stdin. Output file: Normally the output is written to stdout. Use the -o option to override this. Separators: The default separator character is , (comma). To change this on input or output see the -t and -u options respectively. Use -t TAB or -u TAB (literally T-A-B!) to specify tab-separated files. Options:" let () = let input_sep = ref ',' in let set_input_sep = function | "TAB" -> input_sep := '\t' | "COMMA" -> input_sep := ',' | "SPACE" -> input_sep := ' ' | s -> input_sep := s.[0] in let output_sep = ref ',' in let set_output_sep = function | "TAB" -> output_sep := '\t' | "COMMA" -> output_sep := ',' | "SPACE" -> output_sep := ' ' | s -> output_sep := s.[0] in let count_zero = ref false in let output_file = ref "" in let rest = ref [] in let set_rest str = rest := str :: !rest in let argspec = [ "-t", Arg.String set_input_sep, "Input separator char. Use -t TAB for tab separated input."; "-u", Arg.String set_output_sep, "Output separator char. Use -u TAB for tab separated output."; "-o", Arg.Set_string output_file, "Write output to file (instead of stdout)"; "-z", Arg.Set count_zero, "Number columns from 0 instead of 1"; "-", Arg.Unit (fun () -> set_rest "-"), "" (* Hack to allow '-' for input from stdin. *) ] in Arg.parse argspec set_rest usage; let input_sep = !input_sep in let output_sep = !output_sep in let count_zero = !count_zero in let output_file = !output_file in let rest = List.rev !rest in (* Set up the output file. *) let chan = if output_file <> "" then open_out output_file else stdout in (match rest with | ("col"|"cols") :: colspec :: files -> let colspec = parse_colspec ~count_zero colspec in cmd_cols ~input_sep ~output_sep ~chan colspec files | ("namedcol"|"namedcols") :: names :: files -> let names = nsplit names "," in cmd_namedcols ~input_sep ~output_sep ~chan names files | ("width"|"columns") :: files -> cmd_width ~input_sep ~chan files | ("height"|"rows") :: files -> cmd_height ~input_sep ~chan files | "readable" :: files -> cmd_readable ~input_sep ~chan files | ("cat"|"concat") :: files -> cmd_cat ~input_sep ~output_sep ~chan files | "paste" :: files -> cmd_paste ~input_sep ~output_sep ~chan files | "pastecol" :: colspec1 :: colspec2 :: file1 :: file2 :: _ -> let colspec1 = parse_colspec ~count_zero colspec1 in let colspec2 = parse_colspec ~count_zero colspec2 in cmd_pastecol ~input_sep ~output_sep ~chan colspec1 colspec2 file1 file2 | ("join"|"collate") :: colspec1 :: colspec2 :: ((_::_::_) as files) -> let colspec1 = parse_colspec ~count_zero colspec1 in let colspec2 = parse_colspec ~count_zero colspec2 in cmd_join ~input_sep ~output_sep ~chan colspec1 colspec2 files | "square" :: files -> cmd_square ~input_sep ~output_sep ~chan files | "sub" :: r :: c :: rows :: cols :: files -> let r = int_of_string r in let r = if not count_zero then r-1 else r in let c = int_of_string c in let c = if not count_zero then c-1 else c in let rows = int_of_string rows in let cols = int_of_string cols in cmd_sub ~input_sep ~output_sep ~chan r c rows cols files | "replace" :: colspec :: update :: files -> let colspec = parse_colspec ~count_zero colspec in cmd_replace ~input_sep ~output_sep ~chan colspec update files | ("setcolumns"|"set_columns"|"set-columns"| "setcols"|"set_cols"|"set-cols") :: cols :: files -> let cols = int_of_string cols in cmd_set_columns ~input_sep ~output_sep ~chan cols files | ("setrows"|"set_rows"|"set-rows") :: rows :: files -> let rows = int_of_string rows in cmd_set_rows ~input_sep ~output_sep ~chan rows files | ("head"|"take") :: rows :: files -> let rows = int_of_string rows in cmd_head ~input_sep ~output_sep ~chan rows files | "drop" :: rows :: files -> let rows = int_of_string rows in cmd_drop ~input_sep ~output_sep ~chan rows files | "transpose" :: files -> cmd_transpose ~input_sep ~output_sep ~chan files | "call" :: command :: files -> cmd_call ~input_sep ~output_sep ~chan command files | "trim" :: flags :: files -> let flags = trim_flags flags in cmd_trim ~input_sep ~output_sep ~chan flags files | _ -> prerr_endline (Sys.executable_name ^ " --help for usage"); exit 2 ); if output_file <> "" then close_out chan csv-1.3.2/examples/example.ml0000644000175000017500000000264612236526203014606 0ustar trchtrch(* See also 'test.ml' for examples, and 'csv.mli' for documentation. *) open Printf let embedded_csv = "\ \"Banner clickins\" \"Clickin\",\"Number\",\"Percentage\", \"brand.adwords\",\"4,878\",\"14.4\" \"vacation.advert2.adwords\",\"4,454\",\"13.1\" \"affiliates.generic.tc1\",\"1,608\",\"4.7\" \"brand.overture\",\"1,576\",\"4.6\" \"vacation.cheap.adwords\",\"1,515\",\"4.5\" \"affiliates.generic.vacation.biggestchoice\",\"1,072\",\"3.2\" \"breaks.no-destination.adwords\",\"1,015\",\"3.0\" \"fly.no-destination.flightshome.adwords\",\"833\",\"2.5\" \"exchange.adwords\",\"728\",\"2.1\" \"holidays.cyprus.cheap\",\"574\",\"1.7\" \"travel.adwords\",\"416\",\"1.2\" \"affiliates.vacation.generic.onlinediscount.200\",\"406\",\"1.2\" \"promo.home.topX.ACE.189\",\"373\",\"1.1\" \"homepage.hp_tx1b_20050126\",\"369\",\"1.1\" \"travel.agents.adwords\",\"358\",\"1.1\" \"promo.home.topX.SSH.366\",\"310\",\"0.9\"" let csvs = List.map (fun name -> name, Csv.load name) [ "examples/example1.csv"; "examples/example2.csv" ] let () = let ecsv = Csv.input_all(Csv.of_string embedded_csv) in printf "---Embedded CSV---------------------------------\n" ; Csv.print_readable ecsv; List.iter ( fun (name, csv) -> printf "---%s----------------------------------------\n" name; Csv.print_readable csv ) csvs; printf "Compare (Embedded CSV) example1.csv = %i\n" (Csv.compare ecsv (snd(List.hd csvs))) csv-1.3.2/examples/example2.csv0000644000175000017500000000035312236526203015044 0ustar trchtrch"Visitors per search engine" "Search engine","Number", "Google","15,437" "MSN","2,372" "AOLSearch","1,885" "Yahoo","1,555" "Ask-Jeeves","998" "Ntlworld","303" "Myway","268" "Myway.com","169" "Freeserve","73" "BBC","41" "Altavista","26"csv-1.3.2/examples/example1.csv0000644000175000017500000000127212236526203015044 0ustar trchtrch"Banner clickins" "Clickin","Number","Percentage", "brand.adwords","4,878","14.4" "vacation.advert2.adwords","4,454","13.1" "affiliates.generic.tc1","1,608","4.7" "brand.overture","1,576","4.6" "vacation.cheap.adwords","1,515","4.5" "affiliates.generic.vacation.biggestchoice","1,072","3.2" "breaks.no-destination.adwords","1,015","3.0" "fly.no-destination.flightshome.adwords","833","2.5" "exchange.adwords","728","2.1" "holidays.cyprus.cheap","574","1.7" "travel.adwords","416","1.2" "affiliates.vacation.generic.onlinediscount.200","406","1.2" "promo.home.topX.ACE.189","373","1.1" "homepage.hp_tx1b_20050126","369","1.1" "travel.agents.adwords","358","1.1" "promo.home.topX.SSH.366","310","0.9"csv-1.3.2/setup.ml0000644000175000017500000047407512236526204012507 0ustar trchtrch let () = try Topdirs.dir_directory (Sys.getenv "OCAML_TOPLEVEL_PATH") with Not_found -> ();; (* OASIS_START *) (* DO NOT EDIT (digest: 6f4cb2c8f6c74abd9ee546e9f8eb86c4) *) (* Regenerated by OASIS v0.3.0 Visit http://oasis.forge.ocamlcore.org for more information and documentation about functions used in this file. *) module OASISGettext = struct (* # 21 "src/oasis/OASISGettext.ml" *) let ns_ str = str let s_ str = str let f_ (str : ('a, 'b, 'c, 'd) format4) = str let fn_ fmt1 fmt2 n = if n = 1 then fmt1^^"" else fmt2^^"" let init = [] end module OASISContext = struct (* # 21 "src/oasis/OASISContext.ml" *) open OASISGettext type level = [ `Debug | `Info | `Warning | `Error] type t = { quiet: bool; info: bool; debug: bool; ignore_plugins: bool; ignore_unknown_fields: bool; printf: level -> string -> unit; } let printf lvl str = let beg = match lvl with | `Error -> s_ "E: " | `Warning -> s_ "W: " | `Info -> s_ "I: " | `Debug -> s_ "D: " in prerr_endline (beg^str) let default = ref { quiet = false; info = false; debug = false; ignore_plugins = false; ignore_unknown_fields = false; printf = printf; } let quiet = {!default with quiet = true} let args () = ["-quiet", Arg.Unit (fun () -> default := {!default with quiet = true}), (s_ " Run quietly"); "-info", Arg.Unit (fun () -> default := {!default with info = true}), (s_ " Display information message"); "-debug", Arg.Unit (fun () -> default := {!default with debug = true}), (s_ " Output debug message")] end module OASISString = struct (* # 1 "src/oasis/OASISString.ml" *) (** Various string utilities. Mostly inspired by extlib and batteries ExtString and BatString libraries. @author Sylvain Le Gall *) let nsplitf str f = if str = "" then [] else let buf = Buffer.create 13 in let lst = ref [] in let push () = lst := Buffer.contents buf :: !lst; Buffer.clear buf in let str_len = String.length str in for i = 0 to str_len - 1 do if f str.[i] then push () else Buffer.add_char buf str.[i] done; push (); List.rev !lst (** [nsplit c s] Split the string [s] at char [c]. It doesn't include the separator. *) let nsplit str c = nsplitf str ((=) c) let find ~what ?(offset=0) str = let what_idx = ref 0 in let str_idx = ref offset in while !str_idx < String.length str && !what_idx < String.length what do if str.[!str_idx] = what.[!what_idx] then incr what_idx else what_idx := 0; incr str_idx done; if !what_idx <> String.length what then raise Not_found else !str_idx - !what_idx let sub_start str len = let str_len = String.length str in if len >= str_len then "" else String.sub str len (str_len - len) let sub_end ?(offset=0) str len = let str_len = String.length str in if len >= str_len then "" else String.sub str 0 (str_len - len) let starts_with ~what ?(offset=0) str = let what_idx = ref 0 in let str_idx = ref offset in let ok = ref true in while !ok && !str_idx < String.length str && !what_idx < String.length what do if str.[!str_idx] = what.[!what_idx] then incr what_idx else ok := false; incr str_idx done; if !what_idx = String.length what then true else false let strip_starts_with ~what str = if starts_with ~what str then sub_start str (String.length what) else raise Not_found let ends_with ~what ?(offset=0) str = let what_idx = ref ((String.length what) - 1) in let str_idx = ref ((String.length str) - 1) in let ok = ref true in while !ok && offset <= !str_idx && 0 <= !what_idx do if str.[!str_idx] = what.[!what_idx] then decr what_idx else ok := false; decr str_idx done; if !what_idx = -1 then true else false let strip_ends_with ~what str = if ends_with ~what str then sub_end str (String.length what) else raise Not_found let replace_chars f s = let buf = String.make (String.length s) 'X' in for i = 0 to String.length s - 1 do buf.[i] <- f s.[i] done; buf end module OASISUtils = struct (* # 21 "src/oasis/OASISUtils.ml" *) open OASISGettext module MapString = Map.Make(String) let map_string_of_assoc assoc = List.fold_left (fun acc (k, v) -> MapString.add k v acc) MapString.empty assoc module SetString = Set.Make(String) let set_string_add_list st lst = List.fold_left (fun acc e -> SetString.add e acc) st lst let set_string_of_list = set_string_add_list SetString.empty let compare_csl s1 s2 = String.compare (String.lowercase s1) (String.lowercase s2) module HashStringCsl = Hashtbl.Make (struct type t = string let equal s1 s2 = (String.lowercase s1) = (String.lowercase s2) let hash s = Hashtbl.hash (String.lowercase s) end) let varname_of_string ?(hyphen='_') s = if String.length s = 0 then begin invalid_arg "varname_of_string" end else begin let buf = OASISString.replace_chars (fun c -> if ('a' <= c && c <= 'z') || ('A' <= c && c <= 'Z') || ('0' <= c && c <= '9') then c else hyphen) s; in let buf = (* Start with a _ if digit *) if '0' <= s.[0] && s.[0] <= '9' then "_"^buf else buf in String.lowercase buf end let varname_concat ?(hyphen='_') p s = let what = String.make 1 hyphen in let p = try OASISString.strip_ends_with ~what p with Not_found -> p in let s = try OASISString.strip_starts_with ~what s with Not_found -> s in p^what^s let is_varname str = str = varname_of_string str let failwithf fmt = Printf.ksprintf failwith fmt end module PropList = struct (* # 21 "src/oasis/PropList.ml" *) open OASISGettext type name = string exception Not_set of name * string option exception No_printer of name exception Unknown_field of name * name let () = Printexc.register_printer (function | Not_set (nm, Some rsn) -> Some (Printf.sprintf (f_ "Field '%s' is not set: %s") nm rsn) | Not_set (nm, None) -> Some (Printf.sprintf (f_ "Field '%s' is not set") nm) | No_printer nm -> Some (Printf.sprintf (f_ "No default printer for value %s") nm) | Unknown_field (nm, schm) -> Some (Printf.sprintf (f_ "Field %s is not defined in schema %s") nm schm) | _ -> None) module Data = struct type t = (name, unit -> unit) Hashtbl.t let create () = Hashtbl.create 13 let clear t = Hashtbl.clear t (* # 71 "src/oasis/PropList.ml" *) end module Schema = struct type ('ctxt, 'extra) value = { get: Data.t -> string; set: Data.t -> ?context:'ctxt -> string -> unit; help: (unit -> string) option; extra: 'extra; } type ('ctxt, 'extra) t = { name: name; fields: (name, ('ctxt, 'extra) value) Hashtbl.t; order: name Queue.t; name_norm: string -> string; } let create ?(case_insensitive=false) nm = { name = nm; fields = Hashtbl.create 13; order = Queue.create (); name_norm = (if case_insensitive then String.lowercase else fun s -> s); } let add t nm set get extra help = let key = t.name_norm nm in if Hashtbl.mem t.fields key then failwith (Printf.sprintf (f_ "Field '%s' is already defined in schema '%s'") nm t.name); Hashtbl.add t.fields key { set = set; get = get; help = help; extra = extra; }; Queue.add nm t.order let mem t nm = Hashtbl.mem t.fields nm let find t nm = try Hashtbl.find t.fields (t.name_norm nm) with Not_found -> raise (Unknown_field (nm, t.name)) let get t data nm = (find t nm).get data let set t data nm ?context x = (find t nm).set data ?context x let fold f acc t = Queue.fold (fun acc k -> let v = find t k in f acc k v.extra v.help) acc t.order let iter f t = fold (fun () -> f) () t let name t = t.name end module Field = struct type ('ctxt, 'value, 'extra) t = { set: Data.t -> ?context:'ctxt -> 'value -> unit; get: Data.t -> 'value; sets: Data.t -> ?context:'ctxt -> string -> unit; gets: Data.t -> string; help: (unit -> string) option; extra: 'extra; } let new_id = let last_id = ref 0 in fun () -> incr last_id; !last_id let create ?schema ?name ?parse ?print ?default ?update ?help extra = (* Default value container *) let v = ref None in (* If name is not given, create unique one *) let nm = match name with | Some s -> s | None -> Printf.sprintf "_anon_%d" (new_id ()) in (* Last chance to get a value: the default *) let default () = match default with | Some d -> d | None -> raise (Not_set (nm, Some (s_ "no default value"))) in (* Get data *) let get data = (* Get value *) try (Hashtbl.find data nm) (); match !v with | Some x -> x | None -> default () with Not_found -> default () in (* Set data *) let set data ?context x = let x = match update with | Some f -> begin try f ?context (get data) x with Not_set _ -> x end | None -> x in Hashtbl.replace data nm (fun () -> v := Some x) in (* Parse string value, if possible *) let parse = match parse with | Some f -> f | None -> fun ?context s -> failwith (Printf.sprintf (f_ "Cannot parse field '%s' when setting value %S") nm s) in (* Set data, from string *) let sets data ?context s = set ?context data (parse ?context s) in (* Output value as string, if possible *) let print = match print with | Some f -> f | None -> fun _ -> raise (No_printer nm) in (* Get data, as a string *) let gets data = print (get data) in begin match schema with | Some t -> Schema.add t nm sets gets extra help | None -> () end; { set = set; get = get; sets = sets; gets = gets; help = help; extra = extra; } let fset data t ?context x = t.set data ?context x let fget data t = t.get data let fsets data t ?context s = t.sets data ?context s let fgets data t = t.gets data end module FieldRO = struct let create ?schema ?name ?parse ?print ?default ?update ?help extra = let fld = Field.create ?schema ?name ?parse ?print ?default ?update ?help extra in fun data -> Field.fget data fld end end module OASISMessage = struct (* # 21 "src/oasis/OASISMessage.ml" *) open OASISGettext open OASISContext let generic_message ~ctxt lvl fmt = let cond = if ctxt.quiet then false else match lvl with | `Debug -> ctxt.debug | `Info -> ctxt.info | _ -> true in Printf.ksprintf (fun str -> if cond then begin ctxt.printf lvl str end) fmt let debug ~ctxt fmt = generic_message ~ctxt `Debug fmt let info ~ctxt fmt = generic_message ~ctxt `Info fmt let warning ~ctxt fmt = generic_message ~ctxt `Warning fmt let error ~ctxt fmt = generic_message ~ctxt `Error fmt end module OASISVersion = struct (* # 21 "src/oasis/OASISVersion.ml" *) open OASISGettext type s = string type t = string type comparator = | VGreater of t | VGreaterEqual of t | VEqual of t | VLesser of t | VLesserEqual of t | VOr of comparator * comparator | VAnd of comparator * comparator (* Range of allowed characters *) let is_digit c = '0' <= c && c <= '9' let is_alpha c = ('a' <= c && c <= 'z') || ('A' <= c && c <= 'Z') let is_special = function | '.' | '+' | '-' | '~' -> true | _ -> false let rec version_compare v1 v2 = if v1 <> "" || v2 <> "" then begin (* Compare ascii string, using special meaning for version * related char *) let val_ascii c = if c = '~' then -1 else if is_digit c then 0 else if c = '\000' then 0 else if is_alpha c then Char.code c else (Char.code c) + 256 in let len1 = String.length v1 in let len2 = String.length v2 in let p = ref 0 in (** Compare ascii part *) let compare_vascii () = let cmp = ref 0 in while !cmp = 0 && !p < len1 && !p < len2 && not (is_digit v1.[!p] && is_digit v2.[!p]) do cmp := (val_ascii v1.[!p]) - (val_ascii v2.[!p]); incr p done; if !cmp = 0 && !p < len1 && !p = len2 then val_ascii v1.[!p] else if !cmp = 0 && !p = len1 && !p < len2 then - (val_ascii v2.[!p]) else !cmp in (** Compare digit part *) let compare_digit () = let extract_int v p = let start_p = !p in while !p < String.length v && is_digit v.[!p] do incr p done; let substr = String.sub v !p ((String.length v) - !p) in let res = match String.sub v start_p (!p - start_p) with | "" -> 0 | s -> int_of_string s in res, substr in let i1, tl1 = extract_int v1 (ref !p) in let i2, tl2 = extract_int v2 (ref !p) in i1 - i2, tl1, tl2 in match compare_vascii () with | 0 -> begin match compare_digit () with | 0, tl1, tl2 -> if tl1 <> "" && is_digit tl1.[0] then 1 else if tl2 <> "" && is_digit tl2.[0] then -1 else version_compare tl1 tl2 | n, _, _ -> n end | n -> n end else begin 0 end let version_of_string str = str let string_of_version t = t let chop t = try let pos = String.rindex t '.' in String.sub t 0 pos with Not_found -> t let rec comparator_apply v op = match op with | VGreater cv -> (version_compare v cv) > 0 | VGreaterEqual cv -> (version_compare v cv) >= 0 | VLesser cv -> (version_compare v cv) < 0 | VLesserEqual cv -> (version_compare v cv) <= 0 | VEqual cv -> (version_compare v cv) = 0 | VOr (op1, op2) -> (comparator_apply v op1) || (comparator_apply v op2) | VAnd (op1, op2) -> (comparator_apply v op1) && (comparator_apply v op2) let rec string_of_comparator = function | VGreater v -> "> "^(string_of_version v) | VEqual v -> "= "^(string_of_version v) | VLesser v -> "< "^(string_of_version v) | VGreaterEqual v -> ">= "^(string_of_version v) | VLesserEqual v -> "<= "^(string_of_version v) | VOr (c1, c2) -> (string_of_comparator c1)^" || "^(string_of_comparator c2) | VAnd (c1, c2) -> (string_of_comparator c1)^" && "^(string_of_comparator c2) let rec varname_of_comparator = let concat p v = OASISUtils.varname_concat p (OASISUtils.varname_of_string (string_of_version v)) in function | VGreater v -> concat "gt" v | VLesser v -> concat "lt" v | VEqual v -> concat "eq" v | VGreaterEqual v -> concat "ge" v | VLesserEqual v -> concat "le" v | VOr (c1, c2) -> (varname_of_comparator c1)^"_or_"^(varname_of_comparator c2) | VAnd (c1, c2) -> (varname_of_comparator c1)^"_and_"^(varname_of_comparator c2) let version_0_3_or_after t = comparator_apply t (VGreaterEqual (string_of_version "0.3")) end module OASISLicense = struct (* # 21 "src/oasis/OASISLicense.ml" *) (** License for _oasis fields @author Sylvain Le Gall *) type license = string type license_exception = string type license_version = | Version of OASISVersion.t | VersionOrLater of OASISVersion.t | NoVersion type license_dep_5_unit = { license: license; excption: license_exception option; version: license_version; } type license_dep_5 = | DEP5Unit of license_dep_5_unit | DEP5Or of license_dep_5 list | DEP5And of license_dep_5 list type t = | DEP5License of license_dep_5 | OtherLicense of string (* URL *) end module OASISExpr = struct (* # 21 "src/oasis/OASISExpr.ml" *) open OASISGettext type test = string type flag = string type t = | EBool of bool | ENot of t | EAnd of t * t | EOr of t * t | EFlag of flag | ETest of test * string type 'a choices = (t * 'a) list let eval var_get t = let rec eval' = function | EBool b -> b | ENot e -> not (eval' e) | EAnd (e1, e2) -> (eval' e1) && (eval' e2) | EOr (e1, e2) -> (eval' e1) || (eval' e2) | EFlag nm -> let v = var_get nm in assert(v = "true" || v = "false"); (v = "true") | ETest (nm, vl) -> let v = var_get nm in (v = vl) in eval' t let choose ?printer ?name var_get lst = let rec choose_aux = function | (cond, vl) :: tl -> if eval var_get cond then vl else choose_aux tl | [] -> let str_lst = if lst = [] then s_ "" else String.concat (s_ ", ") (List.map (fun (cond, vl) -> match printer with | Some p -> p vl | None -> s_ "") lst) in match name with | Some nm -> failwith (Printf.sprintf (f_ "No result for the choice list '%s': %s") nm str_lst) | None -> failwith (Printf.sprintf (f_ "No result for a choice list: %s") str_lst) in choose_aux (List.rev lst) end module OASISTypes = struct (* # 21 "src/oasis/OASISTypes.ml" *) type name = string type package_name = string type url = string type unix_dirname = string type unix_filename = string type host_dirname = string type host_filename = string type prog = string type arg = string type args = string list type command_line = (prog * arg list) type findlib_name = string type findlib_full = string type compiled_object = | Byte | Native | Best type dependency = | FindlibPackage of findlib_full * OASISVersion.comparator option | InternalLibrary of name type tool = | ExternalTool of name | InternalExecutable of name type vcs = | Darcs | Git | Svn | Cvs | Hg | Bzr | Arch | Monotone | OtherVCS of url type plugin_kind = [ `Configure | `Build | `Doc | `Test | `Install | `Extra ] type plugin_data_purpose = [ `Configure | `Build | `Install | `Clean | `Distclean | `Install | `Uninstall | `Test | `Doc | `Extra | `Other of string ] type 'a plugin = 'a * name * OASISVersion.t option type all_plugin = plugin_kind plugin type plugin_data = (all_plugin * plugin_data_purpose * (unit -> unit)) list (* # 102 "src/oasis/OASISTypes.ml" *) type 'a conditional = 'a OASISExpr.choices type custom = { pre_command: (command_line option) conditional; post_command: (command_line option) conditional; } type common_section = { cs_name: name; cs_data: PropList.Data.t; cs_plugin_data: plugin_data; } type build_section = { bs_build: bool conditional; bs_install: bool conditional; bs_path: unix_dirname; bs_compiled_object: compiled_object; bs_build_depends: dependency list; bs_build_tools: tool list; bs_c_sources: unix_filename list; bs_data_files: (unix_filename * unix_filename option) list; bs_ccopt: args conditional; bs_cclib: args conditional; bs_dlllib: args conditional; bs_dllpath: args conditional; bs_byteopt: args conditional; bs_nativeopt: args conditional; } type library = { lib_modules: string list; lib_pack: bool; lib_internal_modules: string list; lib_findlib_parent: findlib_name option; lib_findlib_name: findlib_name option; lib_findlib_containers: findlib_name list; } type executable = { exec_custom: bool; exec_main_is: unix_filename; } type flag = { flag_description: string option; flag_default: bool conditional; } type source_repository = { src_repo_type: vcs; src_repo_location: url; src_repo_browser: url option; src_repo_module: string option; src_repo_branch: string option; src_repo_tag: string option; src_repo_subdir: unix_filename option; } type test = { test_type: [`Test] plugin; test_command: command_line conditional; test_custom: custom; test_working_directory: unix_filename option; test_run: bool conditional; test_tools: tool list; } type doc_format = | HTML of unix_filename | DocText | PDF | PostScript | Info of unix_filename | DVI | OtherDoc type doc = { doc_type: [`Doc] plugin; doc_custom: custom; doc_build: bool conditional; doc_install: bool conditional; doc_install_dir: unix_filename; doc_title: string; doc_authors: string list; doc_abstract: string option; doc_format: doc_format; doc_data_files: (unix_filename * unix_filename option) list; doc_build_tools: tool list; } type section = | Library of common_section * build_section * library | Executable of common_section * build_section * executable | Flag of common_section * flag | SrcRepo of common_section * source_repository | Test of common_section * test | Doc of common_section * doc type section_kind = [ `Library | `Executable | `Flag | `SrcRepo | `Test | `Doc ] type package = { oasis_version: OASISVersion.t; ocaml_version: OASISVersion.comparator option; findlib_version: OASISVersion.comparator option; name: package_name; version: OASISVersion.t; license: OASISLicense.t; license_file: unix_filename option; copyrights: string list; maintainers: string list; authors: string list; homepage: url option; synopsis: string; description: string option; categories: url list; conf_type: [`Configure] plugin; conf_custom: custom; build_type: [`Build] plugin; build_custom: custom; install_type: [`Install] plugin; install_custom: custom; uninstall_custom: custom; clean_custom: custom; distclean_custom: custom; files_ab: unix_filename list; sections: section list; plugins: [`Extra] plugin list; schema_data: PropList.Data.t; plugin_data: plugin_data; } end module OASISUnixPath = struct (* # 21 "src/oasis/OASISUnixPath.ml" *) type unix_filename = string type unix_dirname = string type host_filename = string type host_dirname = string let current_dir_name = "." let parent_dir_name = ".." let is_current_dir fn = fn = current_dir_name || fn = "" let concat f1 f2 = if is_current_dir f1 then f2 else let f1' = try OASISString.strip_ends_with ~what:"/" f1 with Not_found -> f1 in f1'^"/"^f2 let make = function | hd :: tl -> List.fold_left (fun f p -> concat f p) hd tl | [] -> invalid_arg "OASISUnixPath.make" let dirname f = try String.sub f 0 (String.rindex f '/') with Not_found -> current_dir_name let basename f = try let pos_start = (String.rindex f '/') + 1 in String.sub f pos_start ((String.length f) - pos_start) with Not_found -> f let chop_extension f = try let last_dot = String.rindex f '.' in let sub = String.sub f 0 last_dot in try let last_slash = String.rindex f '/' in if last_slash < last_dot then sub else f with Not_found -> sub with Not_found -> f let capitalize_file f = let dir = dirname f in let base = basename f in concat dir (String.capitalize base) let uncapitalize_file f = let dir = dirname f in let base = basename f in concat dir (String.uncapitalize base) end module OASISHostPath = struct (* # 21 "src/oasis/OASISHostPath.ml" *) open Filename module Unix = OASISUnixPath let make = function | [] -> invalid_arg "OASISHostPath.make" | hd :: tl -> List.fold_left Filename.concat hd tl let of_unix ufn = if Sys.os_type = "Unix" then ufn else make (List.map (fun p -> if p = Unix.current_dir_name then current_dir_name else if p = Unix.parent_dir_name then parent_dir_name else p) (OASISString.nsplit ufn '/')) end module OASISSection = struct (* # 21 "src/oasis/OASISSection.ml" *) open OASISTypes let section_kind_common = function | Library (cs, _, _) -> `Library, cs | Executable (cs, _, _) -> `Executable, cs | Flag (cs, _) -> `Flag, cs | SrcRepo (cs, _) -> `SrcRepo, cs | Test (cs, _) -> `Test, cs | Doc (cs, _) -> `Doc, cs let section_common sct = snd (section_kind_common sct) let section_common_set cs = function | Library (_, bs, lib) -> Library (cs, bs, lib) | Executable (_, bs, exec) -> Executable (cs, bs, exec) | Flag (_, flg) -> Flag (cs, flg) | SrcRepo (_, src_repo) -> SrcRepo (cs, src_repo) | Test (_, tst) -> Test (cs, tst) | Doc (_, doc) -> Doc (cs, doc) (** Key used to identify section *) let section_id sct = let k, cs = section_kind_common sct in k, cs.cs_name let string_of_section sct = let k, nm = section_id sct in (match k with | `Library -> "library" | `Executable -> "executable" | `Flag -> "flag" | `SrcRepo -> "src repository" | `Test -> "test" | `Doc -> "doc") ^" "^nm let section_find id scts = List.find (fun sct -> id = section_id sct) scts module CSection = struct type t = section let id = section_id let compare t1 t2 = compare (id t1) (id t2) let equal t1 t2 = (id t1) = (id t2) let hash t = Hashtbl.hash (id t) end module MapSection = Map.Make(CSection) module SetSection = Set.Make(CSection) end module OASISBuildSection = struct (* # 21 "src/oasis/OASISBuildSection.ml" *) end module OASISExecutable = struct (* # 21 "src/oasis/OASISExecutable.ml" *) open OASISTypes let unix_exec_is (cs, bs, exec) is_native ext_dll suffix_program = let dir = OASISUnixPath.concat bs.bs_path (OASISUnixPath.dirname exec.exec_main_is) in let is_native_exec = match bs.bs_compiled_object with | Native -> true | Best -> is_native () | Byte -> false in OASISUnixPath.concat dir (cs.cs_name^(suffix_program ())), if not is_native_exec && not exec.exec_custom && bs.bs_c_sources <> [] then Some (dir^"/dll"^cs.cs_name^"_stubs"^(ext_dll ())) else None end module OASISLibrary = struct (* # 21 "src/oasis/OASISLibrary.ml" *) open OASISTypes open OASISUtils open OASISGettext open OASISSection type library_name = name type findlib_part_name = name type 'a map_of_findlib_part_name = 'a OASISUtils.MapString.t exception InternalLibraryNotFound of library_name exception FindlibPackageNotFound of findlib_name type group_t = | Container of findlib_name * group_t list | Package of (findlib_name * common_section * build_section * library * group_t list) (* Look for a module file, considering capitalization or not. *) let find_module source_file_exists (cs, bs, lib) modul = let possible_base_fn = List.map (OASISUnixPath.concat bs.bs_path) [modul; OASISUnixPath.uncapitalize_file modul; OASISUnixPath.capitalize_file modul] in (* TODO: we should be able to be able to determine the source for every * files. Hence we should introduce a Module(source: fn) for the fields * Modules and InternalModules *) List.fold_left (fun acc base_fn -> match acc with | `No_sources _ -> begin let file_found = List.fold_left (fun acc ext -> if source_file_exists (base_fn^ext) then (base_fn^ext) :: acc else acc) [] [".ml"; ".mli"; ".mll"; ".mly"] in match file_found with | [] -> acc | lst -> `Sources (base_fn, lst) end | `Sources _ -> acc) (`No_sources possible_base_fn) possible_base_fn let source_unix_files ~ctxt (cs, bs, lib) source_file_exists = List.fold_left (fun acc modul -> match find_module source_file_exists (cs, bs, lib) modul with | `Sources (base_fn, lst) -> (base_fn, lst) :: acc | `No_sources _ -> OASISMessage.warning ~ctxt (f_ "Cannot find source file matching \ module '%s' in library %s") modul cs.cs_name; acc) [] (lib.lib_modules @ lib.lib_internal_modules) let generated_unix_files ~ctxt ~is_native ~has_native_dynlink ~ext_lib ~ext_dll ~source_file_exists (cs, bs, lib) = let find_modules lst ext = let find_module modul = match find_module source_file_exists (cs, bs, lib) modul with | `Sources (base_fn, _) -> [base_fn] | `No_sources lst -> OASISMessage.warning ~ctxt (f_ "Cannot find source file matching \ module '%s' in library %s") modul cs.cs_name; lst in List.map (fun nm -> List.map (fun base_fn -> base_fn ^"."^ext) (find_module nm)) lst in (* The headers that should be compiled along *) let headers = if lib.lib_pack then [] else find_modules lib.lib_modules "cmi" in (* The .cmx that be compiled along *) let cmxs = let should_be_built = (not lib.lib_pack) && (* Do not install .cmx packed submodules *) match bs.bs_compiled_object with | Native -> true | Best -> is_native | Byte -> false in if should_be_built then find_modules (lib.lib_modules @ lib.lib_internal_modules) "cmx" else [] in let acc_nopath = [] in (* Compute what libraries should be built *) let acc_nopath = (* Add the packed header file if required *) let add_pack_header acc = if lib.lib_pack then [cs.cs_name^".cmi"] :: acc else acc in let byte acc = add_pack_header ([cs.cs_name^".cma"] :: acc) in let native acc = let acc = add_pack_header (if has_native_dynlink then [cs.cs_name^".cmxs"] :: acc else acc) in [cs.cs_name^".cmxa"] :: [cs.cs_name^ext_lib] :: acc in match bs.bs_compiled_object with | Native -> byte (native acc_nopath) | Best when is_native -> byte (native acc_nopath) | Byte | Best -> byte acc_nopath in (* Add C library to be built *) let acc_nopath = if bs.bs_c_sources <> [] then begin ["lib"^cs.cs_name^"_stubs"^ext_lib] :: ["dll"^cs.cs_name^"_stubs"^ext_dll] :: acc_nopath end else acc_nopath in (* All the files generated *) List.rev_append (List.rev_map (List.rev_map (OASISUnixPath.concat bs.bs_path)) acc_nopath) (headers @ cmxs) type data = common_section * build_section * library type tree = | Node of (data option) * (tree MapString.t) | Leaf of data let findlib_mapping pkg = (* Map from library name to either full findlib name or parts + parent. *) let fndlb_parts_of_lib_name = let fndlb_parts cs lib = let name = match lib.lib_findlib_name with | Some nm -> nm | None -> cs.cs_name in let name = String.concat "." (lib.lib_findlib_containers @ [name]) in name in List.fold_left (fun mp -> function | Library (cs, _, lib) -> begin let lib_name = cs.cs_name in let fndlb_parts = fndlb_parts cs lib in if MapString.mem lib_name mp then failwithf (f_ "The library name '%s' is used more than once.") lib_name; match lib.lib_findlib_parent with | Some lib_name_parent -> MapString.add lib_name (`Unsolved (lib_name_parent, fndlb_parts)) mp | None -> MapString.add lib_name (`Solved fndlb_parts) mp end | Executable _ | Test _ | Flag _ | SrcRepo _ | Doc _ -> mp) MapString.empty pkg.sections in (* Solve the above graph to be only library name to full findlib name. *) let fndlb_name_of_lib_name = let rec solve visited mp lib_name lib_name_child = if SetString.mem lib_name visited then failwithf (f_ "Library '%s' is involved in a cycle \ with regard to findlib naming.") lib_name; let visited = SetString.add lib_name visited in try match MapString.find lib_name mp with | `Solved fndlb_nm -> fndlb_nm, mp | `Unsolved (lib_nm_parent, post_fndlb_nm) -> let pre_fndlb_nm, mp = solve visited mp lib_nm_parent lib_name in let fndlb_nm = pre_fndlb_nm^"."^post_fndlb_nm in fndlb_nm, MapString.add lib_name (`Solved fndlb_nm) mp with Not_found -> failwithf (f_ "Library '%s', which is defined as the findlib parent of \ library '%s', doesn't exist.") lib_name lib_name_child in let mp = MapString.fold (fun lib_name status mp -> match status with | `Solved _ -> (* Solved initialy, no need to go further *) mp | `Unsolved _ -> let _, mp = solve SetString.empty mp lib_name "" in mp) fndlb_parts_of_lib_name fndlb_parts_of_lib_name in MapString.map (function | `Solved fndlb_nm -> fndlb_nm | `Unsolved _ -> assert false) mp in (* Convert an internal library name to a findlib name. *) let findlib_name_of_library_name lib_nm = try MapString.find lib_nm fndlb_name_of_lib_name with Not_found -> raise (InternalLibraryNotFound lib_nm) in (* Add a library to the tree. *) let add sct mp = let fndlb_fullname = let cs, _, _ = sct in let lib_name = cs.cs_name in findlib_name_of_library_name lib_name in let rec add_children nm_lst (children : tree MapString.t) = match nm_lst with | (hd :: tl) -> begin let node = try add_node tl (MapString.find hd children) with Not_found -> (* New node *) new_node tl in MapString.add hd node children end | [] -> (* Should not have a nameless library. *) assert false and add_node tl node = if tl = [] then begin match node with | Node (None, children) -> Node (Some sct, children) | Leaf (cs', _, _) | Node (Some (cs', _, _), _) -> (* TODO: allow to merge Package, i.e. * archive(byte) = "foo.cma foo_init.cmo" *) let cs, _, _ = sct in failwithf (f_ "Library '%s' and '%s' have the same findlib name '%s'") cs.cs_name cs'.cs_name fndlb_fullname end else begin match node with | Leaf data -> Node (Some data, add_children tl MapString.empty) | Node (data_opt, children) -> Node (data_opt, add_children tl children) end and new_node = function | [] -> Leaf sct | hd :: tl -> Node (None, MapString.add hd (new_node tl) MapString.empty) in add_children (OASISString.nsplit fndlb_fullname '.') mp in let rec group_of_tree mp = MapString.fold (fun nm node acc -> let cur = match node with | Node (Some (cs, bs, lib), children) -> Package (nm, cs, bs, lib, group_of_tree children) | Node (None, children) -> Container (nm, group_of_tree children) | Leaf (cs, bs, lib) -> Package (nm, cs, bs, lib, []) in cur :: acc) mp [] in let group_mp = List.fold_left (fun mp -> function | Library (cs, bs, lib) -> add (cs, bs, lib) mp | _ -> mp) MapString.empty pkg.sections in let groups = group_of_tree group_mp in let library_name_of_findlib_name = Lazy.lazy_from_fun (fun () -> (* Revert findlib_name_of_library_name. *) MapString.fold (fun k v mp -> MapString.add v k mp) fndlb_name_of_lib_name MapString.empty) in let library_name_of_findlib_name fndlb_nm = try MapString.find fndlb_nm (Lazy.force library_name_of_findlib_name) with Not_found -> raise (FindlibPackageNotFound fndlb_nm) in groups, findlib_name_of_library_name, library_name_of_findlib_name let findlib_of_group = function | Container (fndlb_nm, _) | Package (fndlb_nm, _, _, _, _) -> fndlb_nm let root_of_group grp = let rec root_lib_aux = (* We do a DFS in the group. *) function | Container (_, children) -> List.fold_left (fun res grp -> if res = None then root_lib_aux grp else res) None children | Package (_, cs, bs, lib, _) -> Some (cs, bs, lib) in match root_lib_aux grp with | Some res -> res | None -> failwithf (f_ "Unable to determine root library of findlib library '%s'") (findlib_of_group grp) end module OASISFlag = struct (* # 21 "src/oasis/OASISFlag.ml" *) end module OASISPackage = struct (* # 21 "src/oasis/OASISPackage.ml" *) end module OASISSourceRepository = struct (* # 21 "src/oasis/OASISSourceRepository.ml" *) end module OASISTest = struct (* # 21 "src/oasis/OASISTest.ml" *) end module OASISDocument = struct (* # 21 "src/oasis/OASISDocument.ml" *) end module OASISExec = struct (* # 21 "src/oasis/OASISExec.ml" *) open OASISGettext open OASISUtils open OASISMessage (* TODO: I don't like this quote, it is there because $(rm) foo expands to * 'rm -f' foo... *) let run ~ctxt ?f_exit_code ?(quote=true) cmd args = let cmd = if quote then if Sys.os_type = "Win32" then if String.contains cmd ' ' then (* Double the 1st double quote... win32... sigh *) "\""^(Filename.quote cmd) else cmd else Filename.quote cmd else cmd in let cmdline = String.concat " " (cmd :: args) in info ~ctxt (f_ "Running command '%s'") cmdline; match f_exit_code, Sys.command cmdline with | None, 0 -> () | None, i -> failwithf (f_ "Command '%s' terminated with error code %d") cmdline i | Some f, i -> f i let run_read_output ~ctxt ?f_exit_code cmd args = let fn = Filename.temp_file "oasis-" ".txt" in try begin let () = run ~ctxt ?f_exit_code cmd (args @ [">"; Filename.quote fn]) in let chn = open_in fn in let routput = ref [] in begin try while true do routput := (input_line chn) :: !routput done with End_of_file -> () end; close_in chn; Sys.remove fn; List.rev !routput end with e -> (try Sys.remove fn with _ -> ()); raise e let run_read_one_line ~ctxt ?f_exit_code cmd args = match run_read_output ~ctxt ?f_exit_code cmd args with | [fst] -> fst | lst -> failwithf (f_ "Command return unexpected output %S") (String.concat "\n" lst) end module OASISFileUtil = struct (* # 21 "src/oasis/OASISFileUtil.ml" *) open OASISGettext let file_exists_case fn = let dirname = Filename.dirname fn in let basename = Filename.basename fn in if Sys.file_exists dirname then if basename = Filename.current_dir_name then true else List.mem basename (Array.to_list (Sys.readdir dirname)) else false let find_file ?(case_sensitive=true) paths exts = (* Cardinal product of two list *) let ( * ) lst1 lst2 = List.flatten (List.map (fun a -> List.map (fun b -> a,b) lst2) lst1) in let rec combined_paths lst = match lst with | p1 :: p2 :: tl -> let acc = (List.map (fun (a,b) -> Filename.concat a b) (p1 * p2)) in combined_paths (acc :: tl) | [e] -> e | [] -> [] in let alternatives = List.map (fun (p,e) -> if String.length e > 0 && e.[0] <> '.' then p ^ "." ^ e else p ^ e) ((combined_paths paths) * exts) in List.find (if case_sensitive then file_exists_case else Sys.file_exists) alternatives let which ~ctxt prg = let path_sep = match Sys.os_type with | "Win32" -> ';' | _ -> ':' in let path_lst = OASISString.nsplit (Sys.getenv "PATH") path_sep in let exec_ext = match Sys.os_type with | "Win32" -> "" :: (OASISString.nsplit (Sys.getenv "PATHEXT") path_sep) | _ -> [""] in find_file ~case_sensitive:false [path_lst; [prg]] exec_ext (**/**) let rec fix_dir dn = (* Windows hack because Sys.file_exists "src\\" = false when * Sys.file_exists "src" = true *) let ln = String.length dn in if Sys.os_type = "Win32" && ln > 0 && dn.[ln - 1] = '\\' then fix_dir (String.sub dn 0 (ln - 1)) else dn let q = Filename.quote (**/**) let cp ~ctxt ?(recurse=false) src tgt = if recurse then match Sys.os_type with | "Win32" -> OASISExec.run ~ctxt "xcopy" [q src; q tgt; "/E"] | _ -> OASISExec.run ~ctxt "cp" ["-r"; q src; q tgt] else OASISExec.run ~ctxt (match Sys.os_type with | "Win32" -> "copy" | _ -> "cp") [q src; q tgt] let mkdir ~ctxt tgt = OASISExec.run ~ctxt (match Sys.os_type with | "Win32" -> "md" | _ -> "mkdir") [q tgt] let rec mkdir_parent ~ctxt f tgt = let tgt = fix_dir tgt in if Sys.file_exists tgt then begin if not (Sys.is_directory tgt) then OASISUtils.failwithf (f_ "Cannot create directory '%s', a file of the same name already \ exists") tgt end else begin mkdir_parent ~ctxt f (Filename.dirname tgt); if not (Sys.file_exists tgt) then begin f tgt; mkdir ~ctxt tgt end end let rmdir ~ctxt tgt = if Sys.readdir tgt = [||] then begin match Sys.os_type with | "Win32" -> OASISExec.run ~ctxt "rd" [q tgt] | _ -> OASISExec.run ~ctxt "rm" ["-r"; q tgt] end let glob ~ctxt fn = let basename = Filename.basename fn in if String.length basename >= 2 && basename.[0] = '*' && basename.[1] = '.' then begin let ext_len = (String.length basename) - 2 in let ext = String.sub basename 2 ext_len in let dirname = Filename.dirname fn in Array.fold_left (fun acc fn -> try let fn_ext = String.sub fn ((String.length fn) - ext_len) ext_len in if fn_ext = ext then (Filename.concat dirname fn) :: acc else acc with Invalid_argument _ -> acc) [] (Sys.readdir dirname) end else begin if file_exists_case fn then [fn] else [] end end # 2142 "setup.ml" module BaseEnvLight = struct (* # 21 "src/base/BaseEnvLight.ml" *) module MapString = Map.Make(String) type t = string MapString.t let default_filename = Filename.concat (Sys.getcwd ()) "setup.data" let load ?(allow_empty=false) ?(filename=default_filename) () = if Sys.file_exists filename then begin let chn = open_in_bin filename in let st = Stream.of_channel chn in let line = ref 1 in let st_line = Stream.from (fun _ -> try match Stream.next st with | '\n' -> incr line; Some '\n' | c -> Some c with Stream.Failure -> None) in let lexer = Genlex.make_lexer ["="] st_line in let rec read_file mp = match Stream.npeek 3 lexer with | [Genlex.Ident nm; Genlex.Kwd "="; Genlex.String value] -> Stream.junk lexer; Stream.junk lexer; Stream.junk lexer; read_file (MapString.add nm value mp) | [] -> mp | _ -> failwith (Printf.sprintf "Malformed data file '%s' line %d" filename !line) in let mp = read_file MapString.empty in close_in chn; mp end else if allow_empty then begin MapString.empty end else begin failwith (Printf.sprintf "Unable to load environment, the file '%s' doesn't exist." filename) end let var_get name env = let rec var_expand str = let buff = Buffer.create ((String.length str) * 2) in Buffer.add_substitute buff (fun var -> try var_expand (MapString.find var env) with Not_found -> failwith (Printf.sprintf "No variable %s defined when trying to expand %S." var str)) str; Buffer.contents buff in var_expand (MapString.find name env) let var_choose lst env = OASISExpr.choose (fun nm -> var_get nm env) lst end # 2240 "setup.ml" module BaseContext = struct (* # 21 "src/base/BaseContext.ml" *) open OASISContext let args = args let default = default end module BaseMessage = struct (* # 21 "src/base/BaseMessage.ml" *) (** Message to user, overrid for Base @author Sylvain Le Gall *) open OASISMessage open BaseContext let debug fmt = debug ~ctxt:!default fmt let info fmt = info ~ctxt:!default fmt let warning fmt = warning ~ctxt:!default fmt let error fmt = error ~ctxt:!default fmt end module BaseEnv = struct (* # 21 "src/base/BaseEnv.ml" *) open OASISGettext open OASISUtils open PropList module MapString = BaseEnvLight.MapString type origin_t = | ODefault | OGetEnv | OFileLoad | OCommandLine type cli_handle_t = | CLINone | CLIAuto | CLIWith | CLIEnable | CLIUser of (Arg.key * Arg.spec * Arg.doc) list type definition_t = { hide: bool; dump: bool; cli: cli_handle_t; arg_help: string option; group: string option; } let schema = Schema.create "environment" (* Environment data *) let env = Data.create () (* Environment data from file *) let env_from_file = ref MapString.empty (* Lexer for var *) let var_lxr = Genlex.make_lexer [] let rec var_expand str = let buff = Buffer.create ((String.length str) * 2) in Buffer.add_substitute buff (fun var -> try (* TODO: this is a quick hack to allow calling Test.Command * without defining executable name really. I.e. if there is * an exec Executable toto, then $(toto) should be replace * by its real name. It is however useful to have this function * for other variable that depend on the host and should be * written better than that. *) let st = var_lxr (Stream.of_string var) in match Stream.npeek 3 st with | [Genlex.Ident "utoh"; Genlex.Ident nm] -> OASISHostPath.of_unix (var_get nm) | [Genlex.Ident "utoh"; Genlex.String s] -> OASISHostPath.of_unix s | [Genlex.Ident "ocaml_escaped"; Genlex.Ident nm] -> String.escaped (var_get nm) | [Genlex.Ident "ocaml_escaped"; Genlex.String s] -> String.escaped s | [Genlex.Ident nm] -> var_get nm | _ -> failwithf (f_ "Unknown expression '%s' in variable expansion of %s.") var str with | Unknown_field (_, _) -> failwithf (f_ "No variable %s defined when trying to expand %S.") var str | Stream.Error e -> failwithf (f_ "Syntax error when parsing '%s' when trying to \ expand %S: %s") var str e) str; Buffer.contents buff and var_get name = let vl = try Schema.get schema env name with Unknown_field _ as e -> begin try MapString.find name !env_from_file with Not_found -> raise e end in var_expand vl let var_choose ?printer ?name lst = OASISExpr.choose ?printer ?name var_get lst let var_protect vl = let buff = Buffer.create (String.length vl) in String.iter (function | '$' -> Buffer.add_string buff "\\$" | c -> Buffer.add_char buff c) vl; Buffer.contents buff let var_define ?(hide=false) ?(dump=true) ?short_desc ?(cli=CLINone) ?arg_help ?group name (* TODO: type constraint on the fact that name must be a valid OCaml id *) dflt = let default = [ OFileLoad, (fun () -> MapString.find name !env_from_file); ODefault, dflt; OGetEnv, (fun () -> Sys.getenv name); ] in let extra = { hide = hide; dump = dump; cli = cli; arg_help = arg_help; group = group; } in (* Try to find a value that can be defined *) let var_get_low lst = let errors, res = List.fold_left (fun (errors, res) (o, v) -> if res = None then begin try errors, Some (v ()) with | Not_found -> errors, res | Failure rsn -> (rsn :: errors), res | e -> (Printexc.to_string e) :: errors, res end else errors, res) ([], None) (List.sort (fun (o1, _) (o2, _) -> Pervasives.compare o2 o1) lst) in match res, errors with | Some v, _ -> v | None, [] -> raise (Not_set (name, None)) | None, lst -> raise (Not_set (name, Some (String.concat (s_ ", ") lst))) in let help = match short_desc with | Some fs -> Some fs | None -> None in let var_get_lst = FieldRO.create ~schema ~name ~parse:(fun ?(context=ODefault) s -> [context, fun () -> s]) ~print:var_get_low ~default ~update:(fun ?context x old_x -> x @ old_x) ?help extra in fun () -> var_expand (var_get_low (var_get_lst env)) let var_redefine ?hide ?dump ?short_desc ?cli ?arg_help ?group name dflt = if Schema.mem schema name then begin (* TODO: look suspsicious, we want to memorize dflt not dflt () *) Schema.set schema env ~context:ODefault name (dflt ()); fun () -> var_get name end else begin var_define ?hide ?dump ?short_desc ?cli ?arg_help ?group name dflt end let var_ignore (e : unit -> string) = () let print_hidden = var_define ~hide:true ~dump:false ~cli:CLIAuto ~arg_help:"Print even non-printable variable. (debug)" "print_hidden" (fun () -> "false") let var_all () = List.rev (Schema.fold (fun acc nm def _ -> if not def.hide || bool_of_string (print_hidden ()) then nm :: acc else acc) [] schema) let default_filename = BaseEnvLight.default_filename let load ?allow_empty ?filename () = env_from_file := BaseEnvLight.load ?allow_empty ?filename () let unload () = env_from_file := MapString.empty; Data.clear env let dump ?(filename=default_filename) () = let chn = open_out_bin filename in let output nm value = Printf.fprintf chn "%s=%S\n" nm value in let mp_todo = (* Dump data from schema *) Schema.fold (fun mp_todo nm def _ -> if def.dump then begin try let value = Schema.get schema env nm in output nm value with Not_set _ -> () end; MapString.remove nm mp_todo) !env_from_file schema in (* Dump data defined outside of schema *) MapString.iter output mp_todo; (* End of the dump *) close_out chn let print () = let printable_vars = Schema.fold (fun acc nm def short_descr_opt -> if not def.hide || bool_of_string (print_hidden ()) then begin try let value = Schema.get schema env nm in let txt = match short_descr_opt with | Some s -> s () | None -> nm in (txt, value) :: acc with Not_set _ -> acc end else acc) [] schema in let max_length = List.fold_left max 0 (List.rev_map String.length (List.rev_map fst printable_vars)) in let dot_pad str = String.make ((max_length - (String.length str)) + 3) '.' in Printf.printf "\nConfiguration: \n"; List.iter (fun (name,value) -> Printf.printf "%s: %s %s\n" name (dot_pad name) value) (List.rev printable_vars); Printf.printf "\n%!" let args () = let arg_concat = OASISUtils.varname_concat ~hyphen:'-' in [ "--override", Arg.Tuple ( let rvr = ref "" in let rvl = ref "" in [ Arg.Set_string rvr; Arg.Set_string rvl; Arg.Unit (fun () -> Schema.set schema env ~context:OCommandLine !rvr !rvl) ] ), "var+val Override any configuration variable."; ] @ List.flatten (Schema.fold (fun acc name def short_descr_opt -> let var_set s = Schema.set schema env ~context:OCommandLine name s in let arg_name = OASISUtils.varname_of_string ~hyphen:'-' name in let hlp = match short_descr_opt with | Some txt -> txt () | None -> "" in let arg_hlp = match def.arg_help with | Some s -> s | None -> "str" in let default_value = try Printf.sprintf (f_ " [%s]") (Schema.get schema env name) with Not_set _ -> "" in let args = match def.cli with | CLINone -> [] | CLIAuto -> [ arg_concat "--" arg_name, Arg.String var_set, Printf.sprintf (f_ "%s %s%s") arg_hlp hlp default_value ] | CLIWith -> [ arg_concat "--with-" arg_name, Arg.String var_set, Printf.sprintf (f_ "%s %s%s") arg_hlp hlp default_value ] | CLIEnable -> let dflt = if default_value = " [true]" then s_ " [default: enabled]" else s_ " [default: disabled]" in [ arg_concat "--enable-" arg_name, Arg.Unit (fun () -> var_set "true"), Printf.sprintf (f_ " %s%s") hlp dflt; arg_concat "--disable-" arg_name, Arg.Unit (fun () -> var_set "false"), Printf.sprintf (f_ " %s%s") hlp dflt ] | CLIUser lst -> lst in args :: acc) [] schema) end module BaseArgExt = struct (* # 21 "src/base/BaseArgExt.ml" *) open OASISUtils open OASISGettext let parse argv args = (* Simulate command line for Arg *) let current = ref 0 in try Arg.parse_argv ~current:current (Array.concat [[|"none"|]; argv]) (Arg.align args) (failwithf (f_ "Don't know what to do with arguments: '%s'")) (s_ "configure options:") with | Arg.Help txt -> print_endline txt; exit 0 | Arg.Bad txt -> prerr_endline txt; exit 1 end module BaseCheck = struct (* # 21 "src/base/BaseCheck.ml" *) open BaseEnv open BaseMessage open OASISUtils open OASISGettext let prog_best prg prg_lst = var_redefine prg (fun () -> let alternate = List.fold_left (fun res e -> match res with | Some _ -> res | None -> try Some (OASISFileUtil.which ~ctxt:!BaseContext.default e) with Not_found -> None) None prg_lst in match alternate with | Some prg -> prg | None -> raise Not_found) let prog prg = prog_best prg [prg] let prog_opt prg = prog_best prg [prg^".opt"; prg] let ocamlfind = prog "ocamlfind" let version var_prefix cmp fversion () = (* Really compare version provided *) let var = var_prefix^"_version_"^(OASISVersion.varname_of_comparator cmp) in var_redefine ~hide:true var (fun () -> let version_str = match fversion () with | "[Distributed with OCaml]" -> begin try (var_get "ocaml_version") with Not_found -> warning (f_ "Variable ocaml_version not defined, fallback \ to default"); Sys.ocaml_version end | res -> res in let version = OASISVersion.version_of_string version_str in if OASISVersion.comparator_apply version cmp then version_str else failwithf (f_ "Cannot satisfy version constraint on %s: %s (version: %s)") var_prefix (OASISVersion.string_of_comparator cmp) version_str) () let package_version pkg = OASISExec.run_read_one_line ~ctxt:!BaseContext.default (ocamlfind ()) ["query"; "-format"; "%v"; pkg] let package ?version_comparator pkg () = let var = OASISUtils.varname_concat "pkg_" (OASISUtils.varname_of_string pkg) in let findlib_dir pkg = let dir = OASISExec.run_read_one_line ~ctxt:!BaseContext.default (ocamlfind ()) ["query"; "-format"; "%d"; pkg] in if Sys.file_exists dir && Sys.is_directory dir then dir else failwithf (f_ "When looking for findlib package %s, \ directory %s return doesn't exist") pkg dir in let vl = var_redefine var (fun () -> findlib_dir pkg) () in ( match version_comparator with | Some ver_cmp -> ignore (version var ver_cmp (fun _ -> package_version pkg) ()) | None -> () ); vl end module BaseOCamlcConfig = struct (* # 21 "src/base/BaseOCamlcConfig.ml" *) open BaseEnv open OASISUtils open OASISGettext module SMap = Map.Make(String) let ocamlc = BaseCheck.prog_opt "ocamlc" let ocamlc_config_map = (* Map name to value for ocamlc -config output (name ^": "^value) *) let rec split_field mp lst = match lst with | line :: tl -> let mp = try let pos_semicolon = String.index line ':' in if pos_semicolon > 1 then ( let name = String.sub line 0 pos_semicolon in let linelen = String.length line in let value = if linelen > pos_semicolon + 2 then String.sub line (pos_semicolon + 2) (linelen - pos_semicolon - 2) else "" in SMap.add name value mp ) else ( mp ) with Not_found -> ( mp ) in split_field mp tl | [] -> mp in let cache = lazy (var_protect (Marshal.to_string (split_field SMap.empty (OASISExec.run_read_output ~ctxt:!BaseContext.default (ocamlc ()) ["-config"])) [])) in var_redefine "ocamlc_config_map" ~hide:true ~dump:false (fun () -> (* TODO: update if ocamlc change !!! *) Lazy.force cache) let var_define nm = (* Extract data from ocamlc -config *) let avlbl_config_get () = Marshal.from_string (ocamlc_config_map ()) 0 in let chop_version_suffix s = try String.sub s 0 (String.index s '+') with _ -> s in let nm_config, value_config = match nm with | "ocaml_version" -> "version", chop_version_suffix | _ -> nm, (fun x -> x) in var_redefine nm (fun () -> try let map = avlbl_config_get () in let value = SMap.find nm_config map in value_config value with Not_found -> failwithf (f_ "Cannot find field '%s' in '%s -config' output") nm (ocamlc ())) end module BaseStandardVar = struct (* # 21 "src/base/BaseStandardVar.ml" *) open OASISGettext open OASISTypes open OASISExpr open BaseCheck open BaseEnv let ocamlfind = BaseCheck.ocamlfind let ocamlc = BaseOCamlcConfig.ocamlc let ocamlopt = prog_opt "ocamlopt" let ocamlbuild = prog "ocamlbuild" (**/**) let rpkg = ref None let pkg_get () = match !rpkg with | Some pkg -> pkg | None -> failwith (s_ "OASIS Package is not set") let var_cond = ref [] let var_define_cond ~since_version f dflt = let holder = ref (fun () -> dflt) in let since_version = OASISVersion.VGreaterEqual (OASISVersion.version_of_string since_version) in var_cond := (fun ver -> if OASISVersion.comparator_apply ver since_version then holder := f ()) :: !var_cond; fun () -> !holder () (**/**) let pkg_name = var_define ~short_desc:(fun () -> s_ "Package name") "pkg_name" (fun () -> (pkg_get ()).name) let pkg_version = var_define ~short_desc:(fun () -> s_ "Package version") "pkg_version" (fun () -> (OASISVersion.string_of_version (pkg_get ()).version)) let c = BaseOCamlcConfig.var_define let os_type = c "os_type" let system = c "system" let architecture = c "architecture" let ccomp_type = c "ccomp_type" let ocaml_version = c "ocaml_version" (* TODO: Check standard variable presence at runtime *) let standard_library_default = c "standard_library_default" let standard_library = c "standard_library" let standard_runtime = c "standard_runtime" let bytecomp_c_compiler = c "bytecomp_c_compiler" let native_c_compiler = c "native_c_compiler" let model = c "model" let ext_obj = c "ext_obj" let ext_asm = c "ext_asm" let ext_lib = c "ext_lib" let ext_dll = c "ext_dll" let default_executable_name = c "default_executable_name" let systhread_supported = c "systhread_supported" let flexlink = BaseCheck.prog "flexlink" let flexdll_version = var_define ~short_desc:(fun () -> "FlexDLL version (Win32)") "flexdll_version" (fun () -> let lst = OASISExec.run_read_output ~ctxt:!BaseContext.default (flexlink ()) ["-help"] in match lst with | line :: _ -> Scanf.sscanf line "FlexDLL version %s" (fun ver -> ver) | [] -> raise Not_found) (**/**) let p name hlp dflt = var_define ~short_desc:hlp ~cli:CLIAuto ~arg_help:"dir" name dflt let (/) a b = if os_type () = Sys.os_type then Filename.concat a b else if os_type () = "Unix" then OASISUnixPath.concat a b else OASISUtils.failwithf (f_ "Cannot handle os_type %s filename concat") (os_type ()) (**/**) let prefix = p "prefix" (fun () -> s_ "Install architecture-independent files dir") (fun () -> match os_type () with | "Win32" -> let program_files = Sys.getenv "PROGRAMFILES" in program_files/(pkg_name ()) | _ -> "/usr/local") let exec_prefix = p "exec_prefix" (fun () -> s_ "Install architecture-dependent files in dir") (fun () -> "$prefix") let bindir = p "bindir" (fun () -> s_ "User executables") (fun () -> "$exec_prefix"/"bin") let sbindir = p "sbindir" (fun () -> s_ "System admin executables") (fun () -> "$exec_prefix"/"sbin") let libexecdir = p "libexecdir" (fun () -> s_ "Program executables") (fun () -> "$exec_prefix"/"libexec") let sysconfdir = p "sysconfdir" (fun () -> s_ "Read-only single-machine data") (fun () -> "$prefix"/"etc") let sharedstatedir = p "sharedstatedir" (fun () -> s_ "Modifiable architecture-independent data") (fun () -> "$prefix"/"com") let localstatedir = p "localstatedir" (fun () -> s_ "Modifiable single-machine data") (fun () -> "$prefix"/"var") let libdir = p "libdir" (fun () -> s_ "Object code libraries") (fun () -> "$exec_prefix"/"lib") let datarootdir = p "datarootdir" (fun () -> s_ "Read-only arch-independent data root") (fun () -> "$prefix"/"share") let datadir = p "datadir" (fun () -> s_ "Read-only architecture-independent data") (fun () -> "$datarootdir") let infodir = p "infodir" (fun () -> s_ "Info documentation") (fun () -> "$datarootdir"/"info") let localedir = p "localedir" (fun () -> s_ "Locale-dependent data") (fun () -> "$datarootdir"/"locale") let mandir = p "mandir" (fun () -> s_ "Man documentation") (fun () -> "$datarootdir"/"man") let docdir = p "docdir" (fun () -> s_ "Documentation root") (fun () -> "$datarootdir"/"doc"/"$pkg_name") let htmldir = p "htmldir" (fun () -> s_ "HTML documentation") (fun () -> "$docdir") let dvidir = p "dvidir" (fun () -> s_ "DVI documentation") (fun () -> "$docdir") let pdfdir = p "pdfdir" (fun () -> s_ "PDF documentation") (fun () -> "$docdir") let psdir = p "psdir" (fun () -> s_ "PS documentation") (fun () -> "$docdir") let destdir = p "destdir" (fun () -> s_ "Prepend a path when installing package") (fun () -> raise (PropList.Not_set ("destdir", Some (s_ "undefined by construct")))) let findlib_version = var_define "findlib_version" (fun () -> BaseCheck.package_version "findlib") let is_native = var_define "is_native" (fun () -> try let _s : string = ocamlopt () in "true" with PropList.Not_set _ -> let _s : string = ocamlc () in "false") let ext_program = var_define "suffix_program" (fun () -> match os_type () with | "Win32" -> ".exe" | _ -> "") let rm = var_define ~short_desc:(fun () -> s_ "Remove a file.") "rm" (fun () -> match os_type () with | "Win32" -> "del" | _ -> "rm -f") let rmdir = var_define ~short_desc:(fun () -> s_ "Remove a directory.") "rmdir" (fun () -> match os_type () with | "Win32" -> "rd" | _ -> "rm -rf") let debug = var_define ~short_desc:(fun () -> s_ "Turn ocaml debug flag on") ~cli:CLIEnable "debug" (fun () -> "true") let profile = var_define ~short_desc:(fun () -> s_ "Turn ocaml profile flag on") ~cli:CLIEnable "profile" (fun () -> "false") let tests = var_define_cond ~since_version:"0.3" (fun () -> var_define ~short_desc:(fun () -> s_ "Compile tests executable and library and run them") ~cli:CLIEnable "tests" (fun () -> "false")) "true" let docs = var_define_cond ~since_version:"0.3" (fun () -> var_define ~short_desc:(fun () -> s_ "Create documentations") ~cli:CLIEnable "docs" (fun () -> "true")) "true" let native_dynlink = var_define ~short_desc:(fun () -> s_ "Compiler support generation of .cmxs.") ~cli:CLINone "native_dynlink" (fun () -> let res = let ocaml_lt_312 () = OASISVersion.comparator_apply (OASISVersion.version_of_string (ocaml_version ())) (OASISVersion.VLesser (OASISVersion.version_of_string "3.12.0")) in let flexdll_lt_030 () = OASISVersion.comparator_apply (OASISVersion.version_of_string (flexdll_version ())) (OASISVersion.VLesser (OASISVersion.version_of_string "0.30")) in let has_native_dynlink = let ocamlfind = ocamlfind () in try let fn = OASISExec.run_read_one_line ~ctxt:!BaseContext.default ocamlfind ["query"; "-predicates"; "native"; "dynlink"; "-format"; "%d/%a"] in Sys.file_exists fn with _ -> false in if not has_native_dynlink then false else if ocaml_lt_312 () then false else if (os_type () = "Win32" || os_type () = "Cygwin") && flexdll_lt_030 () then begin BaseMessage.warning (f_ ".cmxs generation disabled because FlexDLL needs to be \ at least 0.30. Please upgrade FlexDLL from %s to 0.30.") (flexdll_version ()); false end else true in string_of_bool res) let init pkg = rpkg := Some pkg; List.iter (fun f -> f pkg.oasis_version) !var_cond end module BaseFileAB = struct (* # 21 "src/base/BaseFileAB.ml" *) open BaseEnv open OASISGettext open BaseMessage let to_filename fn = let fn = OASISHostPath.of_unix fn in if not (Filename.check_suffix fn ".ab") then warning (f_ "File '%s' doesn't have '.ab' extension") fn; Filename.chop_extension fn let replace fn_lst = let buff = Buffer.create 13 in List.iter (fun fn -> let fn = OASISHostPath.of_unix fn in let chn_in = open_in fn in let chn_out = open_out (to_filename fn) in ( try while true do Buffer.add_string buff (var_expand (input_line chn_in)); Buffer.add_char buff '\n' done with End_of_file -> () ); Buffer.output_buffer chn_out buff; Buffer.clear buff; close_in chn_in; close_out chn_out) fn_lst end module BaseLog = struct (* # 21 "src/base/BaseLog.ml" *) open OASISUtils let default_filename = Filename.concat (Filename.dirname BaseEnv.default_filename) "setup.log" module SetTupleString = Set.Make (struct type t = string * string let compare (s11, s12) (s21, s22) = match String.compare s11 s21 with | 0 -> String.compare s12 s22 | n -> n end) let load () = if Sys.file_exists default_filename then begin let chn = open_in default_filename in let scbuf = Scanf.Scanning.from_file default_filename in let rec read_aux (st, lst) = if not (Scanf.Scanning.end_of_input scbuf) then begin let acc = try Scanf.bscanf scbuf "%S %S\n" (fun e d -> let t = e, d in if SetTupleString.mem t st then st, lst else SetTupleString.add t st, t :: lst) with Scanf.Scan_failure _ -> failwith (Scanf.bscanf scbuf "%l" (fun line -> Printf.sprintf "Malformed log file '%s' at line %d" default_filename line)) in read_aux acc end else begin close_in chn; List.rev lst end in read_aux (SetTupleString.empty, []) end else begin [] end let register event data = let chn_out = open_out_gen [Open_append; Open_creat; Open_text] 0o644 default_filename in Printf.fprintf chn_out "%S %S\n" event data; close_out chn_out let unregister event data = if Sys.file_exists default_filename then begin let lst = load () in let chn_out = open_out default_filename in let write_something = ref false in List.iter (fun (e, d) -> if e <> event || d <> data then begin write_something := true; Printf.fprintf chn_out "%S %S\n" e d end) lst; close_out chn_out; if not !write_something then Sys.remove default_filename end let filter events = let st_events = List.fold_left (fun st e -> SetString.add e st) SetString.empty events in List.filter (fun (e, _) -> SetString.mem e st_events) (load ()) let exists event data = List.exists (fun v -> (event, data) = v) (load ()) end module BaseBuilt = struct (* # 21 "src/base/BaseBuilt.ml" *) open OASISTypes open OASISGettext open BaseStandardVar open BaseMessage type t = | BExec (* Executable *) | BExecLib (* Library coming with executable *) | BLib (* Library *) | BDoc (* Document *) let to_log_event_file t nm = "built_"^ (match t with | BExec -> "exec" | BExecLib -> "exec_lib" | BLib -> "lib" | BDoc -> "doc")^ "_"^nm let to_log_event_done t nm = "is_"^(to_log_event_file t nm) let register t nm lst = BaseLog.register (to_log_event_done t nm) "true"; List.iter (fun alt -> let registered = List.fold_left (fun registered fn -> if OASISFileUtil.file_exists_case fn then begin BaseLog.register (to_log_event_file t nm) (if Filename.is_relative fn then Filename.concat (Sys.getcwd ()) fn else fn); true end else registered) false alt in if not registered then warning (f_ "Cannot find an existing alternative files among: %s") (String.concat (s_ ", ") alt)) lst let unregister t nm = List.iter (fun (e, d) -> BaseLog.unregister e d) (BaseLog.filter [to_log_event_file t nm; to_log_event_done t nm]) let fold t nm f acc = List.fold_left (fun acc (_, fn) -> if OASISFileUtil.file_exists_case fn then begin f acc fn end else begin warning (f_ "File '%s' has been marked as built \ for %s but doesn't exist") fn (Printf.sprintf (match t with | BExec | BExecLib -> (f_ "executable %s") | BLib -> (f_ "library %s") | BDoc -> (f_ "documentation %s")) nm); acc end) acc (BaseLog.filter [to_log_event_file t nm]) let is_built t nm = List.fold_left (fun is_built (_, d) -> (try bool_of_string d with _ -> false)) false (BaseLog.filter [to_log_event_done t nm]) let of_executable ffn (cs, bs, exec) = let unix_exec_is, unix_dll_opt = OASISExecutable.unix_exec_is (cs, bs, exec) (fun () -> bool_of_string (is_native ())) ext_dll ext_program in let evs = (BExec, cs.cs_name, [[ffn unix_exec_is]]) :: (match unix_dll_opt with | Some fn -> [BExecLib, cs.cs_name, [[ffn fn]]] | None -> []) in evs, unix_exec_is, unix_dll_opt let of_library ffn (cs, bs, lib) = let unix_lst = OASISLibrary.generated_unix_files ~ctxt:!BaseContext.default ~source_file_exists:(fun fn -> OASISFileUtil.file_exists_case (OASISHostPath.of_unix fn)) ~is_native:(bool_of_string (is_native ())) ~has_native_dynlink:(bool_of_string (native_dynlink ())) ~ext_lib:(ext_lib ()) ~ext_dll:(ext_dll ()) (cs, bs, lib) in let evs = [BLib, cs.cs_name, List.map (List.map ffn) unix_lst] in evs, unix_lst end module BaseCustom = struct (* # 21 "src/base/BaseCustom.ml" *) open BaseEnv open BaseMessage open OASISTypes open OASISGettext let run cmd args extra_args = OASISExec.run ~ctxt:!BaseContext.default ~quote:false (var_expand cmd) (List.map var_expand (args @ (Array.to_list extra_args))) let hook ?(failsafe=false) cstm f e = let optional_command lst = let printer = function | Some (cmd, args) -> String.concat " " (cmd :: args) | None -> s_ "No command" in match var_choose ~name:(s_ "Pre/Post Command") ~printer lst with | Some (cmd, args) -> begin try run cmd args [||] with e when failsafe -> warning (f_ "Command '%s' fail with error: %s") (String.concat " " (cmd :: args)) (match e with | Failure msg -> msg | e -> Printexc.to_string e) end | None -> () in let res = optional_command cstm.pre_command; f e in optional_command cstm.post_command; res end module BaseDynVar = struct (* # 21 "src/base/BaseDynVar.ml" *) open OASISTypes open OASISGettext open BaseEnv open BaseBuilt let init pkg = (* TODO: disambiguate exec vs other variable by adding exec_VARNAME. *) (* TODO: provide compile option for library libary_byte_args_VARNAME... *) List.iter (function | Executable (cs, bs, exec) -> if var_choose bs.bs_build then var_ignore (var_redefine (* We don't save this variable *) ~dump:false ~short_desc:(fun () -> Printf.sprintf (f_ "Filename of executable '%s'") cs.cs_name) (OASISUtils.varname_of_string cs.cs_name) (fun () -> let fn_opt = fold BExec cs.cs_name (fun _ fn -> Some fn) None in match fn_opt with | Some fn -> fn | None -> raise (PropList.Not_set (cs.cs_name, Some (Printf.sprintf (f_ "Executable '%s' not yet built.") cs.cs_name))))) | Library _ | Flag _ | Test _ | SrcRepo _ | Doc _ -> ()) pkg.sections end module BaseTest = struct (* # 21 "src/base/BaseTest.ml" *) open BaseEnv open BaseMessage open OASISTypes open OASISExpr open OASISGettext let test lst pkg extra_args = let one_test (failure, n) (test_plugin, cs, test) = if var_choose ~name:(Printf.sprintf (f_ "test %s run") cs.cs_name) ~printer:string_of_bool test.test_run then begin let () = info (f_ "Running test '%s'") cs.cs_name in let back_cwd = match test.test_working_directory with | Some dir -> let cwd = Sys.getcwd () in let chdir d = info (f_ "Changing directory to '%s'") d; Sys.chdir d in chdir dir; fun () -> chdir cwd | None -> fun () -> () in try let failure_percent = BaseCustom.hook test.test_custom (test_plugin pkg (cs, test)) extra_args in back_cwd (); (failure_percent +. failure, n + 1) with e -> begin back_cwd (); raise e end end else begin info (f_ "Skipping test '%s'") cs.cs_name; (failure, n) end in let (failed, n) = List.fold_left one_test (0.0, 0) lst in let failure_percent = if n = 0 then 0.0 else failed /. (float_of_int n) in let msg = Printf.sprintf (f_ "Tests had a %.2f%% failure rate") (100. *. failure_percent) in if failure_percent > 0.0 then failwith msg else info "%s" msg; (* Possible explanation why the tests where not run. *) if OASISVersion.version_0_3_or_after pkg.oasis_version && not (bool_of_string (BaseStandardVar.tests ())) && lst <> [] then BaseMessage.warning "Tests are turned off, consider enabling with \ 'ocaml setup.ml -configure --enable-tests'" end module BaseDoc = struct (* # 21 "src/base/BaseDoc.ml" *) open BaseEnv open BaseMessage open OASISTypes open OASISGettext let doc lst pkg extra_args = let one_doc (doc_plugin, cs, doc) = if var_choose ~name:(Printf.sprintf (f_ "documentation %s build") cs.cs_name) ~printer:string_of_bool doc.doc_build then begin info (f_ "Building documentation '%s'") cs.cs_name; BaseCustom.hook doc.doc_custom (doc_plugin pkg (cs, doc)) extra_args end in List.iter one_doc lst; if OASISVersion.version_0_3_or_after pkg.oasis_version && not (bool_of_string (BaseStandardVar.docs ())) && lst <> [] then BaseMessage.warning "Docs are turned off, consider enabling with \ 'ocaml setup.ml -configure --enable-docs'" end module BaseSetup = struct (* # 21 "src/base/BaseSetup.ml" *) open BaseEnv open BaseMessage open OASISTypes open OASISSection open OASISGettext open OASISUtils type std_args_fun = package -> string array -> unit type ('a, 'b) section_args_fun = name * (package -> (common_section * 'a) -> string array -> 'b) type t = { configure: std_args_fun; build: std_args_fun; doc: ((doc, unit) section_args_fun) list; test: ((test, float) section_args_fun) list; install: std_args_fun; uninstall: std_args_fun; clean: std_args_fun list; clean_doc: (doc, unit) section_args_fun list; clean_test: (test, unit) section_args_fun list; distclean: std_args_fun list; distclean_doc: (doc, unit) section_args_fun list; distclean_test: (test, unit) section_args_fun list; package: package; oasis_fn: string option; oasis_version: string; oasis_digest: Digest.t option; oasis_exec: string option; oasis_setup_args: string list; setup_update: bool; } (* Associate a plugin function with data from package *) let join_plugin_sections filter_map lst = List.rev (List.fold_left (fun acc sct -> match filter_map sct with | Some e -> e :: acc | None -> acc) [] lst) (* Search for plugin data associated with a section name *) let lookup_plugin_section plugin action nm lst = try List.assoc nm lst with Not_found -> failwithf (f_ "Cannot find plugin %s matching section %s for %s action") plugin nm action let configure t args = (* Run configure *) BaseCustom.hook t.package.conf_custom (fun () -> (* Reload if preconf has changed it *) begin try unload (); load (); with _ -> () end; (* Run plugin's configure *) t.configure t.package args; (* Dump to allow postconf to change it *) dump ()) (); (* Reload environment *) unload (); load (); (* Save environment *) print (); (* Replace data in file *) BaseFileAB.replace t.package.files_ab let build t args = BaseCustom.hook t.package.build_custom (t.build t.package) args let doc t args = BaseDoc.doc (join_plugin_sections (function | Doc (cs, e) -> Some (lookup_plugin_section "documentation" (s_ "build") cs.cs_name t.doc, cs, e) | _ -> None) t.package.sections) t.package args let test t args = BaseTest.test (join_plugin_sections (function | Test (cs, e) -> Some (lookup_plugin_section "test" (s_ "run") cs.cs_name t.test, cs, e) | _ -> None) t.package.sections) t.package args let all t args = let rno_doc = ref false in let rno_test = ref false in Arg.parse_argv ~current:(ref 0) (Array.of_list ((Sys.executable_name^" all") :: (Array.to_list args))) [ "-no-doc", Arg.Set rno_doc, s_ "Don't run doc target"; "-no-test", Arg.Set rno_test, s_ "Don't run test target"; ] (failwithf (f_ "Don't know what to do with '%s'")) ""; info "Running configure step"; configure t [||]; info "Running build step"; build t [||]; (* Load setup.log dynamic variables *) BaseDynVar.init t.package; if not !rno_doc then begin info "Running doc step"; doc t [||]; end else begin info "Skipping doc step" end; if not !rno_test then begin info "Running test step"; test t [||] end else begin info "Skipping test step" end let install t args = BaseCustom.hook t.package.install_custom (t.install t.package) args let uninstall t args = BaseCustom.hook t.package.uninstall_custom (t.uninstall t.package) args let reinstall t args = uninstall t args; install t args let clean, distclean = let failsafe f a = try f a with e -> warning (f_ "Action fail with error: %s") (match e with | Failure msg -> msg | e -> Printexc.to_string e) in let generic_clean t cstm mains docs tests args = BaseCustom.hook ~failsafe:true cstm (fun () -> (* Clean section *) List.iter (function | Test (cs, test) -> let f = try List.assoc cs.cs_name tests with Not_found -> fun _ _ _ -> () in failsafe (f t.package (cs, test)) args | Doc (cs, doc) -> let f = try List.assoc cs.cs_name docs with Not_found -> fun _ _ _ -> () in failsafe (f t.package (cs, doc)) args | Library _ | Executable _ | Flag _ | SrcRepo _ -> ()) t.package.sections; (* Clean whole package *) List.iter (fun f -> failsafe (f t.package) args) mains) () in let clean t args = generic_clean t t.package.clean_custom t.clean t.clean_doc t.clean_test args in let distclean t args = (* Call clean *) clean t args; (* Call distclean code *) generic_clean t t.package.distclean_custom t.distclean t.distclean_doc t.distclean_test args; (* Remove generated file *) List.iter (fun fn -> if Sys.file_exists fn then begin info (f_ "Remove '%s'") fn; Sys.remove fn end) (BaseEnv.default_filename :: BaseLog.default_filename :: (List.rev_map BaseFileAB.to_filename t.package.files_ab)) in clean, distclean let version t _ = print_endline t.oasis_version let update_setup_ml, no_update_setup_ml_cli = let b = ref true in b, ("-no-update-setup-ml", Arg.Clear b, s_ " Don't try to update setup.ml, even if _oasis has changed.") let update_setup_ml t = let oasis_fn = match t.oasis_fn with | Some fn -> fn | None -> "_oasis" in let oasis_exec = match t.oasis_exec with | Some fn -> fn | None -> "oasis" in let ocaml = Sys.executable_name in let setup_ml, args = match Array.to_list Sys.argv with | setup_ml :: args -> setup_ml, args | [] -> failwith (s_ "Expecting non-empty command line arguments.") in let ocaml, setup_ml = if Sys.executable_name = Sys.argv.(0) then (* We are not running in standard mode, probably the script * is precompiled. *) "ocaml", "setup.ml" else ocaml, setup_ml in let no_update_setup_ml_cli, _, _ = no_update_setup_ml_cli in let do_update () = let oasis_exec_version = OASISExec.run_read_one_line ~ctxt:!BaseContext.default ~f_exit_code: (function | 0 -> () | 1 -> failwithf (f_ "Executable '%s' is probably an old version \ of oasis (< 0.3.0), please update to version \ v%s.") oasis_exec t.oasis_version | 127 -> failwithf (f_ "Cannot find executable '%s', please install \ oasis v%s.") oasis_exec t.oasis_version | n -> failwithf (f_ "Command '%s version' exited with code %d.") oasis_exec n) oasis_exec ["version"] in if OASISVersion.comparator_apply (OASISVersion.version_of_string oasis_exec_version) (OASISVersion.VGreaterEqual (OASISVersion.version_of_string t.oasis_version)) then begin (* We have a version >= for the executable oasis, proceed with * update. *) (* TODO: delegate this check to 'oasis setup'. *) if Sys.os_type = "Win32" then failwithf (f_ "It is not possible to update the running script \ setup.ml on Windows. Please update setup.ml by \ running '%s'.") (String.concat " " (oasis_exec :: "setup" :: t.oasis_setup_args)) else begin OASISExec.run ~ctxt:!BaseContext.default ~f_exit_code: (function | 0 -> () | n -> failwithf (f_ "Unable to update setup.ml using '%s', \ please fix the problem and retry.") oasis_exec) oasis_exec ("setup" :: t.oasis_setup_args); OASISExec.run ~ctxt:!BaseContext.default ocaml (setup_ml :: args) end end else failwithf (f_ "The version of '%s' (v%s) doesn't match the version of \ oasis used to generate the %s file. Please install at \ least oasis v%s.") oasis_exec oasis_exec_version setup_ml t.oasis_version in if !update_setup_ml then begin try match t.oasis_digest with | Some dgst -> if Sys.file_exists oasis_fn && dgst <> Digest.file "_oasis" then begin do_update (); true end else false | None -> false with e -> error (f_ "Error when updating setup.ml. If you want to avoid this error, \ you can bypass the update of %s by running '%s %s %s %s'") setup_ml ocaml setup_ml no_update_setup_ml_cli (String.concat " " args); raise e end else false let setup t = let catch_exn = ref true in try let act_ref = ref (fun _ -> failwithf (f_ "No action defined, run '%s %s -help'") Sys.executable_name Sys.argv.(0)) in let extra_args_ref = ref [] in let allow_empty_env_ref = ref false in let arg_handle ?(allow_empty_env=false) act = Arg.Tuple [ Arg.Rest (fun str -> extra_args_ref := str :: !extra_args_ref); Arg.Unit (fun () -> allow_empty_env_ref := allow_empty_env; act_ref := act); ] in Arg.parse (Arg.align ([ "-configure", arg_handle ~allow_empty_env:true configure, s_ "[options*] Configure the whole build process."; "-build", arg_handle build, s_ "[options*] Build executables and libraries."; "-doc", arg_handle doc, s_ "[options*] Build documents."; "-test", arg_handle test, s_ "[options*] Run tests."; "-all", arg_handle ~allow_empty_env:true all, s_ "[options*] Run configure, build, doc and test targets."; "-install", arg_handle install, s_ "[options*] Install libraries, data, executables \ and documents."; "-uninstall", arg_handle uninstall, s_ "[options*] Uninstall libraries, data, executables \ and documents."; "-reinstall", arg_handle reinstall, s_ "[options*] Uninstall and install libraries, data, \ executables and documents."; "-clean", arg_handle ~allow_empty_env:true clean, s_ "[options*] Clean files generated by a build."; "-distclean", arg_handle ~allow_empty_env:true distclean, s_ "[options*] Clean files generated by a build and configure."; "-version", arg_handle ~allow_empty_env:true version, s_ " Display version of OASIS used to generate this setup.ml."; "-no-catch-exn", Arg.Clear catch_exn, s_ " Don't catch exception, useful for debugging."; ] @ (if t.setup_update then [no_update_setup_ml_cli] else []) @ (BaseContext.args ()))) (failwithf (f_ "Don't know what to do with '%s'")) (s_ "Setup and run build process current package\n"); (* Build initial environment *) load ~allow_empty:!allow_empty_env_ref (); (** Initialize flags *) List.iter (function | Flag (cs, {flag_description = hlp; flag_default = choices}) -> begin let apply ?short_desc () = var_ignore (var_define ~cli:CLIEnable ?short_desc (OASISUtils.varname_of_string cs.cs_name) (fun () -> string_of_bool (var_choose ~name:(Printf.sprintf (f_ "default value of flag %s") cs.cs_name) ~printer:string_of_bool choices))) in match hlp with | Some hlp -> apply ~short_desc:(fun () -> hlp) () | None -> apply () end | _ -> ()) t.package.sections; BaseStandardVar.init t.package; BaseDynVar.init t.package; if t.setup_update && update_setup_ml t then () else !act_ref t (Array.of_list (List.rev !extra_args_ref)) with e when !catch_exn -> error "%s" (Printexc.to_string e); exit 1 end # 4480 "setup.ml" module InternalConfigurePlugin = struct (* # 21 "src/plugins/internal/InternalConfigurePlugin.ml" *) (** Configure using internal scheme @author Sylvain Le Gall *) open BaseEnv open OASISTypes open OASISUtils open OASISGettext open BaseMessage (** Configure build using provided series of check to be done * and then output corresponding file. *) let configure pkg argv = let var_ignore_eval var = let _s : string = var () in () in let errors = ref SetString.empty in let buff = Buffer.create 13 in let add_errors fmt = Printf.kbprintf (fun b -> errors := SetString.add (Buffer.contents b) !errors; Buffer.clear b) buff fmt in let warn_exception e = warning "%s" (Printexc.to_string e) in (* Check tools *) let check_tools lst = List.iter (function | ExternalTool tool -> begin try var_ignore_eval (BaseCheck.prog tool) with e -> warn_exception e; add_errors (f_ "Cannot find external tool '%s'") tool end | InternalExecutable nm1 -> (* Check that matching tool is built *) List.iter (function | Executable ({cs_name = nm2}, {bs_build = build}, _) when nm1 = nm2 -> if not (var_choose build) then add_errors (f_ "Cannot find buildable internal executable \ '%s' when checking build depends") nm1 | _ -> ()) pkg.sections) lst in let build_checks sct bs = if var_choose bs.bs_build then begin if bs.bs_compiled_object = Native then begin try var_ignore_eval BaseStandardVar.ocamlopt with e -> warn_exception e; add_errors (f_ "Section %s requires native compilation") (OASISSection.string_of_section sct) end; (* Check tools *) check_tools bs.bs_build_tools; (* Check depends *) List.iter (function | FindlibPackage (findlib_pkg, version_comparator) -> begin try var_ignore_eval (BaseCheck.package ?version_comparator findlib_pkg) with e -> warn_exception e; match version_comparator with | None -> add_errors (f_ "Cannot find findlib package %s") findlib_pkg | Some ver_cmp -> add_errors (f_ "Cannot find findlib package %s (%s)") findlib_pkg (OASISVersion.string_of_comparator ver_cmp) end | InternalLibrary nm1 -> (* Check that matching library is built *) List.iter (function | Library ({cs_name = nm2}, {bs_build = build}, _) when nm1 = nm2 -> if not (var_choose build) then add_errors (f_ "Cannot find buildable internal library \ '%s' when checking build depends") nm1 | _ -> ()) pkg.sections) bs.bs_build_depends end in (* Parse command line *) BaseArgExt.parse argv (BaseEnv.args ()); (* OCaml version *) begin match pkg.ocaml_version with | Some ver_cmp -> begin try var_ignore_eval (BaseCheck.version "ocaml" ver_cmp BaseStandardVar.ocaml_version) with e -> warn_exception e; add_errors (f_ "OCaml version %s doesn't match version constraint %s") (BaseStandardVar.ocaml_version ()) (OASISVersion.string_of_comparator ver_cmp) end | None -> () end; (* Findlib version *) begin match pkg.findlib_version with | Some ver_cmp -> begin try var_ignore_eval (BaseCheck.version "findlib" ver_cmp BaseStandardVar.findlib_version) with e -> warn_exception e; add_errors (f_ "Findlib version %s doesn't match version constraint %s") (BaseStandardVar.findlib_version ()) (OASISVersion.string_of_comparator ver_cmp) end | None -> () end; (* FlexDLL *) if BaseStandardVar.os_type () = "Win32" || BaseStandardVar.os_type () = "Cygwin" then begin try var_ignore_eval BaseStandardVar.flexlink with e -> warn_exception e; add_errors (f_ "Cannot find 'flexlink'") end; (* Check build depends *) List.iter (function | Executable (_, bs, _) | Library (_, bs, _) as sct -> build_checks sct bs | Doc (_, doc) -> if var_choose doc.doc_build then check_tools doc.doc_build_tools | Test (_, test) -> if var_choose test.test_run then check_tools test.test_tools | _ -> ()) pkg.sections; (* Check if we need native dynlink (presence of libraries that compile to * native) *) begin let has_cmxa = List.exists (function | Library (_, bs, _) -> var_choose bs.bs_build && (bs.bs_compiled_object = Native || (bs.bs_compiled_object = Best && bool_of_string (BaseStandardVar.is_native ()))) | _ -> false) pkg.sections in if has_cmxa then var_ignore_eval BaseStandardVar.native_dynlink end; (* Check errors *) if SetString.empty != !errors then begin List.iter (fun e -> error "%s" e) (SetString.elements !errors); failwithf (fn_ "%d configuration error" "%d configuration errors" (SetString.cardinal !errors)) (SetString.cardinal !errors) end end module InternalInstallPlugin = struct (* # 21 "src/plugins/internal/InternalInstallPlugin.ml" *) (** Install using internal scheme @author Sylvain Le Gall *) open BaseEnv open BaseStandardVar open BaseMessage open OASISTypes open OASISLibrary open OASISGettext open OASISUtils let exec_hook = ref (fun (cs, bs, exec) -> cs, bs, exec) let lib_hook = ref (fun (cs, bs, lib) -> cs, bs, lib, []) let doc_hook = ref (fun (cs, doc) -> cs, doc) let install_file_ev = "install-file" let install_dir_ev = "install-dir" let install_findlib_ev = "install-findlib" let win32_max_command_line_length = 8000 let split_install_command ocamlfind findlib_name meta files = if Sys.os_type = "Win32" then (* Arguments for the first command: *) let first_args = ["install"; findlib_name; meta] in (* Arguments for remaining commands: *) let other_args = ["install"; findlib_name; "-add"] in (* Extract as much files as possible from [files], [len] is the current command line length: *) let rec get_files len acc files = match files with | [] -> (List.rev acc, []) | file :: rest -> let len = len + 1 + String.length file in if len > win32_max_command_line_length then (List.rev acc, files) else get_files len (file :: acc) rest in (* Split the command into several commands. *) let rec split args files = match files with | [] -> [] | _ -> (* Length of "ocamlfind install [META|-add]" *) let len = List.fold_left (fun len arg -> len + 1 (* for the space *) + String.length arg) (String.length ocamlfind) args in match get_files len [] files with | ([], _) -> failwith (s_ "Command line too long.") | (firsts, others) -> let cmd = args @ firsts in (* Use -add for remaining commands: *) let () = let findlib_ge_132 = OASISVersion.comparator_apply (OASISVersion.version_of_string (BaseStandardVar.findlib_version ())) (OASISVersion.VGreaterEqual (OASISVersion.version_of_string "1.3.2")) in if not findlib_ge_132 then failwithf (f_ "Installing the library %s require to use the flag \ '-add' of ocamlfind because the command line is too \ long. This flag is only available for findlib 1.3.2. \ Please upgrade findlib from %s to 1.3.2") findlib_name (BaseStandardVar.findlib_version ()) in let cmds = split other_args others in cmd :: cmds in (* The first command does not use -add: *) split first_args files else ["install" :: findlib_name :: meta :: files] let install pkg argv = let in_destdir = try let destdir = destdir () in (* Practically speaking destdir is prepended * at the beginning of the target filename *) fun fn -> destdir^fn with PropList.Not_set _ -> fun fn -> fn in let install_file ?tgt_fn src_file envdir = let tgt_dir = in_destdir (envdir ()) in let tgt_file = Filename.concat tgt_dir (match tgt_fn with | Some fn -> fn | None -> Filename.basename src_file) in (* Create target directory if needed *) OASISFileUtil.mkdir_parent ~ctxt:!BaseContext.default (fun dn -> info (f_ "Creating directory '%s'") dn; BaseLog.register install_dir_ev dn) tgt_dir; (* Really install files *) info (f_ "Copying file '%s' to '%s'") src_file tgt_file; OASISFileUtil.cp ~ctxt:!BaseContext.default src_file tgt_file; BaseLog.register install_file_ev tgt_file in (* Install data into defined directory *) let install_data srcdir lst tgtdir = let tgtdir = OASISHostPath.of_unix (var_expand tgtdir) in List.iter (fun (src, tgt_opt) -> let real_srcs = OASISFileUtil.glob ~ctxt:!BaseContext.default (Filename.concat srcdir src) in if real_srcs = [] then failwithf (f_ "Wildcard '%s' doesn't match any files") src; List.iter (fun fn -> install_file fn (fun () -> match tgt_opt with | Some s -> OASISHostPath.of_unix (var_expand s) | None -> tgtdir)) real_srcs) lst in (** Install all libraries *) let install_libs pkg = let files_of_library (f_data, acc) data_lib = let cs, bs, lib, lib_extra = !lib_hook data_lib in if var_choose bs.bs_install && BaseBuilt.is_built BaseBuilt.BLib cs.cs_name then begin let acc = (* Start with acc + lib_extra *) List.rev_append lib_extra acc in let acc = (* Add uncompiled header from the source tree *) let path = OASISHostPath.of_unix bs.bs_path in List.fold_left (fun acc modul -> try List.find OASISFileUtil.file_exists_case (List.map (Filename.concat path) [modul^".mli"; modul^".ml"; String.uncapitalize modul^".mli"; String.capitalize modul^".mli"; String.uncapitalize modul^".ml"; String.capitalize modul^".ml"]) :: acc with Not_found -> begin warning (f_ "Cannot find source header for module %s \ in library %s") modul cs.cs_name; acc end) acc lib.lib_modules in let acc = (* Get generated files *) BaseBuilt.fold BaseBuilt.BLib cs.cs_name (fun acc fn -> fn :: acc) acc in let f_data () = (* Install data associated with the library *) install_data bs.bs_path bs.bs_data_files (Filename.concat (datarootdir ()) pkg.name); f_data () in (f_data, acc) end else begin (f_data, acc) end in (* Install one group of library *) let install_group_lib grp = (* Iterate through all group nodes *) let rec install_group_lib_aux data_and_files grp = let data_and_files, children = match grp with | Container (_, children) -> data_and_files, children | Package (_, cs, bs, lib, children) -> files_of_library data_and_files (cs, bs, lib), children in List.fold_left install_group_lib_aux data_and_files children in (* Findlib name of the root library *) let findlib_name = findlib_of_group grp in (* Determine root library *) let root_lib = root_of_group grp in (* All files to install for this library *) let f_data, files = install_group_lib_aux (ignore, []) grp in (* Really install, if there is something to install *) if files = [] then begin warning (f_ "Nothing to install for findlib library '%s'") findlib_name end else begin let meta = (* Search META file *) let (_, bs, _) = root_lib in let res = Filename.concat bs.bs_path "META" in if not (OASISFileUtil.file_exists_case res) then failwithf (f_ "Cannot find file '%s' for findlib library %s") res findlib_name; res in let files = (* Make filename shorter to avoid hitting command max line length * too early, esp. on Windows. *) let remove_prefix p n = let plen = String.length p in let nlen = String.length n in if plen <= nlen && String.sub n 0 plen = p then begin let fn_sep = if Sys.os_type = "Win32" then '\\' else '/' in let cutpoint = plen + (if plen < nlen && n.[plen] = fn_sep then 1 else 0) in String.sub n cutpoint (nlen - cutpoint) end else n in List.map (remove_prefix (Sys.getcwd ())) files in info (f_ "Installing findlib library '%s'") findlib_name; let ocamlfind = ocamlfind () in let commands = split_install_command ocamlfind findlib_name meta files in List.iter (OASISExec.run ~ctxt:!BaseContext.default ocamlfind) commands; BaseLog.register install_findlib_ev findlib_name end; (* Install data files *) f_data (); in let group_libs, _, _ = findlib_mapping pkg in (* We install libraries in groups *) List.iter install_group_lib group_libs in let install_execs pkg = let install_exec data_exec = let (cs, bs, exec) = !exec_hook data_exec in if var_choose bs.bs_install && BaseBuilt.is_built BaseBuilt.BExec cs.cs_name then begin let exec_libdir () = Filename.concat (libdir ()) pkg.name in BaseBuilt.fold BaseBuilt.BExec cs.cs_name (fun () fn -> install_file ~tgt_fn:(cs.cs_name ^ ext_program ()) fn bindir) (); BaseBuilt.fold BaseBuilt.BExecLib cs.cs_name (fun () fn -> install_file fn exec_libdir) (); install_data bs.bs_path bs.bs_data_files (Filename.concat (datarootdir ()) pkg.name) end in List.iter (function | Executable (cs, bs, exec)-> install_exec (cs, bs, exec) | _ -> ()) pkg.sections in let install_docs pkg = let install_doc data = let (cs, doc) = !doc_hook data in if var_choose doc.doc_install && BaseBuilt.is_built BaseBuilt.BDoc cs.cs_name then begin let tgt_dir = OASISHostPath.of_unix (var_expand doc.doc_install_dir) in BaseBuilt.fold BaseBuilt.BDoc cs.cs_name (fun () fn -> install_file fn (fun () -> tgt_dir)) (); install_data Filename.current_dir_name doc.doc_data_files doc.doc_install_dir end in List.iter (function | Doc (cs, doc) -> install_doc (cs, doc) | _ -> ()) pkg.sections in install_libs pkg; install_execs pkg; install_docs pkg (* Uninstall already installed data *) let uninstall _ argv = List.iter (fun (ev, data) -> if ev = install_file_ev then begin if OASISFileUtil.file_exists_case data then begin info (f_ "Removing file '%s'") data; Sys.remove data end else begin warning (f_ "File '%s' doesn't exist anymore") data end end else if ev = install_dir_ev then begin if Sys.file_exists data && Sys.is_directory data then begin if Sys.readdir data = [||] then begin info (f_ "Removing directory '%s'") data; OASISFileUtil.rmdir ~ctxt:!BaseContext.default data end else begin warning (f_ "Directory '%s' is not empty (%s)") data (String.concat ", " (Array.to_list (Sys.readdir data))) end end else begin warning (f_ "Directory '%s' doesn't exist anymore") data end end else if ev = install_findlib_ev then begin info (f_ "Removing findlib library '%s'") data; OASISExec.run ~ctxt:!BaseContext.default (ocamlfind ()) ["remove"; data] end else failwithf (f_ "Unknown log event '%s'") ev; BaseLog.unregister ev data) (* We process event in reverse order *) (List.rev (BaseLog.filter [install_file_ev; install_dir_ev; install_findlib_ev;])) end # 5233 "setup.ml" module OCamlbuildCommon = struct (* # 21 "src/plugins/ocamlbuild/OCamlbuildCommon.ml" *) (** Functions common to OCamlbuild build and doc plugin *) open OASISGettext open BaseEnv open BaseStandardVar let ocamlbuild_clean_ev = "ocamlbuild-clean" let ocamlbuildflags = var_define ~short_desc:(fun () -> "OCamlbuild additional flags") "ocamlbuildflags" (fun () -> "") (** Fix special arguments depending on environment *) let fix_args args extra_argv = List.flatten [ if (os_type ()) = "Win32" then [ "-classic-display"; "-no-log"; "-no-links"; "-install-lib-dir"; (Filename.concat (standard_library ()) "ocamlbuild") ] else []; if not (bool_of_string (is_native ())) || (os_type ()) = "Win32" then [ "-byte-plugin" ] else []; args; if bool_of_string (debug ()) then ["-tag"; "debug"] else []; if bool_of_string (profile ()) then ["-tag"; "profile"] else []; OASISString.nsplit (ocamlbuildflags ()) ' '; Array.to_list extra_argv; ] (** Run 'ocamlbuild -clean' if not already done *) let run_clean extra_argv = let extra_cli = String.concat " " (Array.to_list extra_argv) in (* Run if never called with these args *) if not (BaseLog.exists ocamlbuild_clean_ev extra_cli) then begin OASISExec.run ~ctxt:!BaseContext.default (ocamlbuild ()) (fix_args ["-clean"] extra_argv); BaseLog.register ocamlbuild_clean_ev extra_cli; at_exit (fun () -> try BaseLog.unregister ocamlbuild_clean_ev extra_cli with _ -> ()) end (** Run ocamlbuild, unregister all clean events *) let run_ocamlbuild args extra_argv = (* TODO: enforce that target in args must be UNIX encoded i.e. toto/index.html *) OASISExec.run ~ctxt:!BaseContext.default (ocamlbuild ()) (fix_args args extra_argv); (* Remove any clean event, we must run it again *) List.iter (fun (e, d) -> BaseLog.unregister e d) (BaseLog.filter [ocamlbuild_clean_ev]) (** Determine real build directory *) let build_dir extra_argv = let rec search_args dir = function | "-build-dir" :: dir :: tl -> search_args dir tl | _ :: tl -> search_args dir tl | [] -> dir in search_args "_build" (fix_args [] extra_argv) end module OCamlbuildPlugin = struct (* # 21 "src/plugins/ocamlbuild/OCamlbuildPlugin.ml" *) (** Build using ocamlbuild @author Sylvain Le Gall *) open OASISTypes open OASISGettext open OASISUtils open BaseEnv open OCamlbuildCommon open BaseStandardVar open BaseMessage let cond_targets_hook = ref (fun lst -> lst) let build pkg argv = (* Return the filename in build directory *) let in_build_dir fn = Filename.concat (build_dir argv) fn in (* Return the unix filename in host build directory *) let in_build_dir_of_unix fn = in_build_dir (OASISHostPath.of_unix fn) in let cond_targets = List.fold_left (fun acc -> function | Library (cs, bs, lib) when var_choose bs.bs_build -> begin let evs, unix_files = BaseBuilt.of_library in_build_dir_of_unix (cs, bs, lib) in let ends_with nd fn = let nd_len = String.length nd in (String.length fn >= nd_len) && (String.sub fn (String.length fn - nd_len) nd_len) = nd in let tgts = List.flatten (List.filter (fun l -> l <> []) (List.map (List.filter (fun fn -> ends_with ".cma" fn || ends_with ".cmxs" fn || ends_with ".cmxa" fn || ends_with (ext_lib ()) fn || ends_with (ext_dll ()) fn)) unix_files)) in match tgts with | _ :: _ -> (evs, tgts) :: acc | [] -> failwithf (f_ "No possible ocamlbuild targets for library %s") cs.cs_name end | Executable (cs, bs, exec) when var_choose bs.bs_build -> begin let evs, unix_exec_is, unix_dll_opt = BaseBuilt.of_executable in_build_dir_of_unix (cs, bs, exec) in let target ext = let unix_tgt = (OASISUnixPath.concat bs.bs_path (OASISUnixPath.chop_extension exec.exec_main_is))^ext in let evs = (* Fix evs, we want to use the unix_tgt, without copying *) List.map (function | BaseBuilt.BExec, nm, lst when nm = cs.cs_name -> BaseBuilt.BExec, nm, [[in_build_dir_of_unix unix_tgt]] | ev -> ev) evs in evs, [unix_tgt] in (* Add executable *) let acc = match bs.bs_compiled_object with | Native -> (target ".native") :: acc | Best when bool_of_string (is_native ()) -> (target ".native") :: acc | Byte | Best -> (target ".byte") :: acc in acc end | Library _ | Executable _ | Test _ | SrcRepo _ | Flag _ | Doc _ -> acc) [] (* Keep the pkg.sections ordered *) (List.rev pkg.sections); in (* Check and register built files *) let check_and_register (bt, bnm, lst) = List.iter (fun fns -> if not (List.exists OASISFileUtil.file_exists_case fns) then failwithf (f_ "No one of expected built files %s exists") (String.concat (s_ ", ") (List.map (Printf.sprintf "'%s'") fns))) lst; (BaseBuilt.register bt bnm lst) in let cond_targets = (* Run the hook *) !cond_targets_hook cond_targets in (* Run a list of target... *) run_ocamlbuild (List.flatten (List.map snd cond_targets)) argv; (* ... and register events *) List.iter check_and_register (List.flatten (List.map fst cond_targets)) let clean pkg extra_args = run_clean extra_args; List.iter (function | Library (cs, _, _) -> BaseBuilt.unregister BaseBuilt.BLib cs.cs_name | Executable (cs, _, _) -> BaseBuilt.unregister BaseBuilt.BExec cs.cs_name; BaseBuilt.unregister BaseBuilt.BExecLib cs.cs_name | _ -> ()) pkg.sections end module OCamlbuildDocPlugin = struct (* # 21 "src/plugins/ocamlbuild/OCamlbuildDocPlugin.ml" *) (* Create documentation using ocamlbuild .odocl files @author Sylvain Le Gall *) open OASISTypes open OASISGettext open OASISMessage open OCamlbuildCommon open BaseStandardVar let doc_build path pkg (cs, doc) argv = let index_html = OASISUnixPath.make [ path; cs.cs_name^".docdir"; "index.html"; ] in let tgt_dir = OASISHostPath.make [ build_dir argv; OASISHostPath.of_unix path; cs.cs_name^".docdir"; ] in run_ocamlbuild [index_html] argv; List.iter (fun glb -> BaseBuilt.register BaseBuilt.BDoc cs.cs_name [OASISFileUtil.glob ~ctxt:!BaseContext.default (Filename.concat tgt_dir glb)]) ["*.html"; "*.css"] let doc_clean t pkg (cs, doc) argv = run_clean argv; BaseBuilt.unregister BaseBuilt.BDoc cs.cs_name end # 5558 "setup.ml" module CustomPlugin = struct (* # 21 "src/plugins/custom/CustomPlugin.ml" *) (** Generate custom configure/build/doc/test/install system @author *) open BaseEnv open OASISGettext open OASISTypes type t = { cmd_main: command_line conditional; cmd_clean: (command_line option) conditional; cmd_distclean: (command_line option) conditional; } let run = BaseCustom.run let main t _ extra_args = let cmd, args = var_choose ~name:(s_ "main command") t.cmd_main in run cmd args extra_args let clean t pkg extra_args = match var_choose t.cmd_clean with | Some (cmd, args) -> run cmd args extra_args | _ -> () let distclean t pkg extra_args = match var_choose t.cmd_distclean with | Some (cmd, args) -> run cmd args extra_args | _ -> () module Build = struct let main t pkg extra_args = main t pkg extra_args; List.iter (fun sct -> let evs = match sct with | Library (cs, bs, lib) when var_choose bs.bs_build -> begin let evs, _ = BaseBuilt.of_library OASISHostPath.of_unix (cs, bs, lib) in evs end | Executable (cs, bs, exec) when var_choose bs.bs_build -> begin let evs, _, _ = BaseBuilt.of_executable OASISHostPath.of_unix (cs, bs, exec) in evs end | _ -> [] in List.iter (fun (bt, bnm, lst) -> BaseBuilt.register bt bnm lst) evs) pkg.sections let clean t pkg extra_args = clean t pkg extra_args; (* TODO: this seems to be pretty generic (at least wrt to ocamlbuild * considering moving this to BaseSetup? *) List.iter (function | Library (cs, _, _) -> BaseBuilt.unregister BaseBuilt.BLib cs.cs_name | Executable (cs, _, _) -> BaseBuilt.unregister BaseBuilt.BExec cs.cs_name; BaseBuilt.unregister BaseBuilt.BExecLib cs.cs_name | _ -> ()) pkg.sections let distclean t pkg extra_args = distclean t pkg extra_args end module Test = struct let main t pkg (cs, test) extra_args = try main t pkg extra_args; 0.0 with Failure s -> BaseMessage.warning (f_ "Test '%s' fails: %s") cs.cs_name s; 1.0 let clean t pkg (cs, test) extra_args = clean t pkg extra_args let distclean t pkg (cs, test) extra_args = distclean t pkg extra_args end module Doc = struct let main t pkg (cs, _) extra_args = main t pkg extra_args; BaseBuilt.register BaseBuilt.BDoc cs.cs_name [] let clean t pkg (cs, _) extra_args = clean t pkg extra_args; BaseBuilt.unregister BaseBuilt.BDoc cs.cs_name let distclean t pkg (cs, _) extra_args = distclean t pkg extra_args end end # 5694 "setup.ml" open OASISTypes;; let setup_t = { BaseSetup.configure = InternalConfigurePlugin.configure; build = OCamlbuildPlugin.build; test = [ ("conformity", CustomPlugin.Test.main { CustomPlugin.cmd_main = [(OASISExpr.EBool true, ("$test", []))]; cmd_clean = [(OASISExpr.EBool true, None)]; cmd_distclean = [(OASISExpr.EBool true, None)]; }) ]; doc = [("API", OCamlbuildDocPlugin.doc_build ".")]; install = InternalInstallPlugin.install; uninstall = InternalInstallPlugin.uninstall; clean = [OCamlbuildPlugin.clean]; clean_test = [ ("conformity", CustomPlugin.Test.clean { CustomPlugin.cmd_main = [(OASISExpr.EBool true, ("$test", []))]; cmd_clean = [(OASISExpr.EBool true, None)]; cmd_distclean = [(OASISExpr.EBool true, None)]; }) ]; clean_doc = [("API", OCamlbuildDocPlugin.doc_clean ".")]; distclean = []; distclean_test = [ ("conformity", CustomPlugin.Test.distclean { CustomPlugin.cmd_main = [(OASISExpr.EBool true, ("$test", []))]; cmd_clean = [(OASISExpr.EBool true, None)]; cmd_distclean = [(OASISExpr.EBool true, None)]; }) ]; distclean_doc = []; package = { oasis_version = "0.3"; ocaml_version = None; findlib_version = None; name = "csv"; version = "1.3.2"; license = OASISLicense.DEP5License (OASISLicense.DEP5Unit { OASISLicense.license = "LGPL"; excption = Some "OCaml linking"; version = OASISLicense.Version "2.1"; }); license_file = None; copyrights = []; maintainers = ["Christophe Troestler "]; authors = ["Richard Jones"; "Christophe Troestler"]; homepage = Some "https://forge.ocamlcore.org/projects/csv/"; synopsis = "A pure OCaml library to read and write CSV files."; description = Some "This is a pure OCaml library to read and write CSV files,\nincluding all extensions used by Excel \226\128\148 e.g. quotes, newlines,\n8 bit characters in fields, \\\"0 etc. The library comes with a handy\ncommand line tool called csvtool for handling CSV files from shell scripts."; categories = ["http://database"; "http://science"]; conf_type = (`Configure, "internal", Some "0.3"); conf_custom = { pre_command = [(OASISExpr.EBool true, None)]; post_command = [(OASISExpr.EBool true, None)]; }; build_type = (`Build, "ocamlbuild", Some "0.3"); build_custom = { pre_command = [(OASISExpr.EBool true, None)]; post_command = [(OASISExpr.EBool true, None)]; }; install_type = (`Install, "internal", Some "0.3"); install_custom = { pre_command = [(OASISExpr.EBool true, None)]; post_command = [(OASISExpr.EBool true, None)]; }; uninstall_custom = { pre_command = [(OASISExpr.EBool true, None)]; post_command = [(OASISExpr.EBool true, None)]; }; clean_custom = { pre_command = [(OASISExpr.EBool true, None)]; post_command = [(OASISExpr.EBool true, None)]; }; distclean_custom = { pre_command = [(OASISExpr.EBool true, None)]; post_command = [(OASISExpr.EBool true, None)]; }; files_ab = []; sections = [ Library ({ cs_name = "csv"; cs_data = PropList.Data.create (); cs_plugin_data = []; }, { bs_build = [(OASISExpr.EBool true, true)]; bs_install = [(OASISExpr.EBool true, true)]; bs_path = "src"; bs_compiled_object = Best; bs_build_depends = []; bs_build_tools = [ExternalTool "ocamlbuild"]; bs_c_sources = []; bs_data_files = []; bs_ccopt = [(OASISExpr.EBool true, [])]; bs_cclib = [(OASISExpr.EBool true, [])]; bs_dlllib = [(OASISExpr.EBool true, [])]; bs_dllpath = [(OASISExpr.EBool true, [])]; bs_byteopt = [(OASISExpr.EBool true, [])]; bs_nativeopt = [(OASISExpr.EBool true, [])]; }, { lib_modules = ["Csv"]; lib_pack = false; lib_internal_modules = []; lib_findlib_parent = None; lib_findlib_name = None; lib_findlib_containers = []; }); Doc ({ cs_name = "API"; cs_data = PropList.Data.create (); cs_plugin_data = []; }, { doc_type = (`Doc, "OCamlbuild", Some "0.3"); doc_custom = { pre_command = [(OASISExpr.EBool true, None)]; post_command = [(OASISExpr.EBool true, None)]; }; doc_build = [ (OASISExpr.ENot (OASISExpr.EFlag "docs"), false); (OASISExpr.EFlag "docs", true) ]; doc_install = [(OASISExpr.EBool true, true)]; doc_install_dir = "$docdir/api"; doc_title = "API reference for Csv"; doc_authors = []; doc_abstract = None; doc_format = OtherDoc; doc_data_files = []; doc_build_tools = [ExternalTool "ocamldoc"; ExternalTool "ocamlbuild"]; }); Executable ({ cs_name = "csvtool"; cs_data = PropList.Data.create (); cs_plugin_data = []; }, { bs_build = [(OASISExpr.EBool true, true)]; bs_install = [(OASISExpr.EBool true, true)]; bs_path = "examples/"; bs_compiled_object = Best; bs_build_depends = [InternalLibrary "csv"]; bs_build_tools = [ExternalTool "ocamlbuild"]; bs_c_sources = []; bs_data_files = []; bs_ccopt = [(OASISExpr.EBool true, [])]; bs_cclib = [(OASISExpr.EBool true, [])]; bs_dlllib = [(OASISExpr.EBool true, [])]; bs_dllpath = [(OASISExpr.EBool true, [])]; bs_byteopt = [(OASISExpr.EBool true, [])]; bs_nativeopt = [(OASISExpr.EBool true, [])]; }, {exec_custom = false; exec_main_is = "csvtool.ml"; }); Executable ({ cs_name = "example"; cs_data = PropList.Data.create (); cs_plugin_data = []; }, { bs_build = [(OASISExpr.EBool true, true)]; bs_install = [(OASISExpr.EBool true, false)]; bs_path = "examples/"; bs_compiled_object = Best; bs_build_depends = [InternalLibrary "csv"]; bs_build_tools = [ExternalTool "ocamlbuild"]; bs_c_sources = []; bs_data_files = []; bs_ccopt = [(OASISExpr.EBool true, [])]; bs_cclib = [(OASISExpr.EBool true, [])]; bs_dlllib = [(OASISExpr.EBool true, [])]; bs_dllpath = [(OASISExpr.EBool true, [])]; bs_byteopt = [(OASISExpr.EBool true, [])]; bs_nativeopt = [(OASISExpr.EBool true, [])]; }, {exec_custom = false; exec_main_is = "example.ml"; }); Executable ({ cs_name = "test"; cs_data = PropList.Data.create (); cs_plugin_data = []; }, { bs_build = [(OASISExpr.EBool true, true)]; bs_install = [(OASISExpr.EBool true, false)]; bs_path = "tests/"; bs_compiled_object = Best; bs_build_depends = [InternalLibrary "csv"]; bs_build_tools = [ExternalTool "ocamlbuild"]; bs_c_sources = []; bs_data_files = []; bs_ccopt = [(OASISExpr.EBool true, [])]; bs_cclib = [(OASISExpr.EBool true, [])]; bs_dlllib = [(OASISExpr.EBool true, [])]; bs_dllpath = [(OASISExpr.EBool true, [])]; bs_byteopt = [(OASISExpr.EBool true, [])]; bs_nativeopt = [(OASISExpr.EBool true, [])]; }, {exec_custom = false; exec_main_is = "test.ml"; }); Test ({ cs_name = "conformity"; cs_data = PropList.Data.create (); cs_plugin_data = []; }, { test_type = (`Test, "custom", Some "0.3"); test_command = [(OASISExpr.EBool true, ("$test", []))]; test_custom = { pre_command = [(OASISExpr.EBool true, None)]; post_command = [(OASISExpr.EBool true, None)]; }; test_working_directory = Some "tests/"; test_run = [ (OASISExpr.ENot (OASISExpr.EFlag "tests"), false); (OASISExpr.EFlag "tests", true) ]; test_tools = []; }); SrcRepo ({ cs_name = "trunk"; cs_data = PropList.Data.create (); cs_plugin_data = []; }, { src_repo_type = Svn; src_repo_location = "svn://svn.forge.ocamlcore.org/svn/csv/trunk"; src_repo_browser = Some "https://forge.ocamlcore.org/scm/browser.php?group_id=113"; src_repo_module = None; src_repo_branch = None; src_repo_tag = None; src_repo_subdir = None; }) ]; plugins = [(`Extra, "META", Some "0.3"); (`Extra, "StdFiles", Some "0.3")]; schema_data = PropList.Data.create (); plugin_data = []; }; oasis_fn = Some "_oasis"; oasis_version = "0.3.0"; oasis_digest = Some "\212\178\238\1387\180*\022\005\228\158\155\n>O\169"; oasis_exec = None; oasis_setup_args = []; setup_update = false; };; let setup () = BaseSetup.setup setup_t;; # 5987 "setup.ml" (* OASIS_STOP *) let () = setup ();; csv-1.3.2/_tags0000644000175000017500000000136212236526204012016 0ustar trchtrch# OASIS_START # DO NOT EDIT (digest: 990159b2445be5f3ae423215ba83a477) # Ignore VCS directories, you can use the same kind of rule outside # OASIS_START/STOP if you want to exclude directories that contains # useless stuff for the build process <**/.svn>: -traverse <**/.svn>: not_hygienic ".bzr": -traverse ".bzr": not_hygienic ".hg": -traverse ".hg": not_hygienic ".git": -traverse ".git": not_hygienic "_darcs": -traverse "_darcs": not_hygienic # Library csv "src/csv.cmxs": use_csv # Executable csvtool : use_csv # Executable example : use_csv : use_csv # Executable test : use_csv : use_csv # OASIS_STOP <**/*.ml>: annot csv-1.3.2/_oasis0000644000175000017500000000341412236526203012175 0ustar trchtrch# -*-conf-*- OASISFormat: 0.3 Name: csv Version: 1.3.2 Synopsis: A pure OCaml library to read and write CSV files. Description: This is a pure OCaml library to read and write CSV files, including all extensions used by Excel — e.g. quotes, newlines, 8 bit characters in fields, \"0 etc. The library comes with a handy command line tool called csvtool for handling CSV files from shell scripts. Authors: Richard Jones, Christophe Troestler Maintainers: Christophe Troestler Categories: http://database, http://science License: LGPL-2.1 with OCaml linking exception Homepage: https://forge.ocamlcore.org/projects/csv/ Plugins: META (0.3), StdFiles (0.3) Library csv Path: src BuildTools: ocamlbuild Modules: Csv Document API Title: API reference for Csv Type: OCamlbuild (0.3) InstallDir: $docdir/api BuildTools: ocamldoc, ocamlbuild XOCamlbuildPath: . XOCamlbuildLibraries: csv # Examples Executable csvtool Path: examples/ MainIs: csvtool.ml BuildDepends: csv CompiledObject: best BuildTools: ocamlbuild Install: true Executable example Path: examples/ MainIs: example.ml BuildDepends: csv CompiledObject: best BuildTools: ocamlbuild Install: false # Testing Executable test Path: tests/ MainIs: test.ml BuildDepends: csv CompiledObject: best BuildTools: ocamlbuild Install: false Test conformity WorkingDirectory: tests/ Command: $test Run: true SourceRepository trunk Type: svn Location: svn://svn.forge.ocamlcore.org/svn/csv/trunk Browser: https://forge.ocamlcore.org/scm/browser.php?group_id=113