Abstract FLEXI-STREAMS implements "virtual" bivalent streams that can be layered atop real binary or bivalent streams and that can be used to read and write character data in various single- or multi-octet encodings which can be changed on the fly. It also supplies in-memory binary streams which are similar to string streams. The library needs a Common Lisp implementation that supports Gray streams and relies on David Lichteblau's trivial-gray-streams to offer portability between different Lisps. The code comes with a BSD-style license so you can basically do with it whatever you want. Download shortcut: http://weitz.de/files/flexi-streams.tar.gz.
The library needs a Common Lisp implementation that supports Gray streams and relies on David Lichteblau's trivial-gray-streams to offer portability between different Lisps.
The code comes with a BSD-style license so you can basically do with it whatever you want.
Download shortcut: http://weitz.de/files/flexi-streams.tar.gz.
make-external-format
external-format-name
external-format-eol-style
external-format-little-endian
external-format-id
external-format-equal
*default-eol-style*
*default-little-endian*
external-format-condition
external-format-condition-external-format
external-format-error
external-format-encoding-error
*substitution-char*
accept-overlong-sequence
flexi-stream
flexi-input-stream
flexi-output-stream
flexi-io-stream
make-flexi-stream
flexi-stream-external-format
flexi-stream-element-type
flexi-stream-column
flexi-stream-position
flexi-stream-bound
flexi-stream-stream
unread-byte
peek-byte
octet
flexi-stream-error
flexi-stream-out-of-sync-error
flexi-stream-element-type-error
flexi-stream-element-type-error-element-type
in-memory-stream
in-memory-input-stream
in-memory-output-stream
list-stream
vector-stream
make-in-memory-input-stream
make-in-memory-output-stream
get-output-stream-sequence
output-stream-sequence-length
with-input-from-sequence
with-output-to-sequence
in-memory-stream-error
in-memory-stream-closed-error
in-memory-stream-position-spec-error
in-memory-stream-position-spec-error-position-spec
string-to-octets
octets-to-string
octet-length
char-length
(defun foo (pathspec) "With standard LispWorks streams." (with-open-file (out pathspec :direction :output :if-exists :supersede :external-format '(:utf-8 :eol-style :crlf)) (write-line "ÄÖÜ1" out)) (with-open-file (out pathspec :direction :output :if-exists :append :external-format '(:latin-1 :eol-style :lf)) (write-line "ÄÖÜ2" out)) (with-open-file (out pathspec :direction :output :if-exists :append :element-type 'octet) (write-byte #xeb out) (write-sequence #(#xa3 #xa4 #xa5) out)) (with-open-file (out pathspec :direction :output :if-exists :append :external-format '(:unicode :little-endian nil :eol-style :crlf)) (write-line "ÄÖÜ3" out))) (defun bar (pathspec) "With a flexi stream." (with-open-file (out pathspec :direction :output :if-exists :supersede :external-format '(:latin-1 :eol-style :lf)) (setq out (make-flexi-stream out :external-format :utf-8)) (write-line "ÄÖÜ1" out) (setf (flexi-stream-external-format out) '(:latin-1 :eol-style :lf)) (write-line "ÄÖÜ2" out) (write-byte #xeb out) (write-sequence #(#xa3 #xa4 #xa5) out) (setf (flexi-stream-external-format out) :ucs-2be) (write-line "ÄÖÜ3" out)))
And applying this function
(defun baz (pathspec) (let (result) (with-open-file (in pathspec :element-type 'octet) (setq in (make-flexi-stream in :external-format :utf-8)) (push (read-line in) result) (push (read-byte in) result) (setf (flexi-stream-external-format in) '(:latin-1 :eol-style :lf)) (push (read-line in) result) (setf (flexi-stream-external-format in) :greek) (push (read-char in) result) (setf (flexi-stream-external-format in) :latin0) (let ((string (make-string 3 :element-type 'character))) (read-sequence string in) (push string result)) (let ((octets (make-array 2 :element-type 'octet))) (read-sequence octets in) (push octets result)) (setf (flexi-stream-external-format in) :ucs-2be) (push (read-line in) result)) (nreverse result)))
("ÄÖÜ1" 196 "ÖÜ2" #\λ "£€¥" #(0 196) "ÖÜ3")
For more examples see the source code of CL-RFC2047, Drakma, Chunga, or CL-WBXML.
(CHAR-CODE #\Newline)
(CHAR-CODE #\Linefeed)
FLEXI-STREAMS together with this documentation can be downloaded from http://weitz.de/files/flexi-streams.tar.gz. The current version is 1.0.7.
Before you install FLEXI-STREAMS you first need to install the trivial-gray-streams library unless you already have it.
FLEXI-STREAMS comes with a system definition for ASDF so you can install the library with
(asdf:oos 'asdf:load-op :flexi-streams)
You can run a test suite which tests some (but not all) aspects of the library with
(asdf:oos 'asdf:test-op :flexi-streams)
Luís Oliveira maintains a darcs repository of FLEXI-STREAMS at http://common-lisp.net/~loliveira/ediware/.
A Mercurial repository of older versions is available at http://arcanes.fr.eu.org/~pierre/2007/02/weitz/ thanks to Pierre Thierry.
If you want to send patches, please read this first.
EXTERNAL-FORMAT
MAKE-EXTERNAL-FORMAT
An external format consists of a basic encoding (like ISO 8859-1 or UTF-8), a definition how line endings are denoted - by a carriage return character (ASCII 13), by a line feed character (ASCII 10), or by both of these characters in a row -, and optionally (for encodings that use units larger than 8 bits) information about the endianess of the encoding.
The following encodings are currently supported by FLEXI-STREAMS:
:UTF-8
:UTF-16
:UTF-32
:ISO-8859-15
:KOI8-R
:CODE-PAGE
:ID
A couple of alternative names are allowed that are listed below:
:UTF8
:UTF16
:UCS-2
:UCS2
:UNICODE
:UTF32
:UCS-4
:UCS4
:ISO-8859-1
:LATIN-1
:LATIN1
:ISO-8859-2
:LATIN-2
:LATIN2
:ISO-8859-3
:LATIN-3
:LATIN3
:ISO-8859-4
:LATIN-4
:LATIN4
:ISO-8859-5
:CYRILLIC
:ISO-8859-6
:ARABIC
:ISO-8859-7
:GREEK
:ISO-8859-8
:HEBREW
:ISO-8859-9
:LATIN-5
:LATIN5
:ISO-8859-10
:LATIN-6
:LATIN6
:ISO-8859-11
:THAI
:ISO-8859-13
:LATIN-7
:LATIN7
:ISO-8859-14
:LATIN-8
:LATIN8
:LATIN-9
:LATIN9
:LATIN-0
:LATIN0
:ISO-8859-16
:LATIN-10
:LATIN10
:CODEPAGE
WIN32:CODE-PAGE(only on LWW)
:KOI8R
:US-ASCII
:ASCII
(Note that we treat UCS-2 exactly like UTF-16 although there are subtle differences. Also note that even though we support encodings like UTF-32 some Lisps only supports characters contained within the Basic Multilingual Plane (like LispWorks) or even less (like CMUCL), so if other characters are read from a flexi stream, READ-CHAR will try to be helpful and return the corresponding Unicode code point - an integer - instead. This might lead to an error if you're using functions like READ-LINE, though.)
READ-CHAR
READ-LINE
Whenever a FLEXI-STREAMS function accepts an external format as one of its arguments, you can provide either an EXTERNAL-FORMAT object or a shortcut which can be a list or a symbol. The list shortcuts have a syntax similar to the one used by LispWorks - the cars are the names of and encoding and the cdrs of these lists correspond to the keyword arguments to MAKE-EXTERNAL-FORMAT, so for example
(:latin-1 :eol-style :crlf)
(make-external-format :latin-1 :eol-style :crlf)
:thai
(make-external-format :thai)
:UCS-2LE
(:UCS-2 :LITTLE-ENDIAN T)
:UCS-2BE
(:UCS-2 :LITTLE-ENDIAN NIL)
:UCS-4LE
(:UCS-4 :LITTLE-ENDIAN T)
:UCS-4BE
(:UCS-4 :LITTLE-ENDIAN NIL)
:UTF-16LE
(:UTF-16 :LITTLE-ENDIAN T)
:UTF-16BE
(:UTF-16 :LITTLE-ENDIAN NIL)
:UTF-32LE
(:UTF-32 :LITTLE-ENDIAN T)
:UTF-32BE
(:UTF-32 :LITTLE-ENDIAN NIL)
:IBM437
(:CODE-PAGE :ID 437)
:IBM850
(:CODE-PAGE :ID 850)
:IBM852
(:CODE-PAGE :ID 852)
:IBM855
(:CODE-PAGE :ID 855)
:IBM857
(:CODE-PAGE :ID 857)
:IBM860
(:CODE-PAGE :ID 860)
:IBM861
(:CODE-PAGE :ID 861)
:IBM862
(:CODE-PAGE :ID 862)
:IBM863
(:CODE-PAGE :ID 863)
:IBM864
(:CODE-PAGE :ID 864)
:IBM865
(:CODE-PAGE :ID 865)
:IBM866
(:CODE-PAGE :ID 866)
:IBM869
(:CODE-PAGE :ID 869)
:WINDOWS-1250
(:CODE-PAGE :ID 1250)
:WINDOWS-1251
(:CODE-PAGE :ID 1251)
:WINDOWS-1252
(:CODE-PAGE :ID 1252)
:WINDOWS-1253
(:CODE-PAGE :ID 1253)
:WINDOWS-1254
(:CODE-PAGE :ID 1254)
:WINDOWS-1255
(:CODE-PAGE :ID 1255)
:WINDOWS-1256
(:CODE-PAGE :ID 1256)
:WINDOWS-1257
(:CODE-PAGE :ID 1257)
:WINDOWS-1258
(:CODE-PAGE :ID 1258)
Note that if you provide a shortcut, it will be converted to an EXTERNAL-FORMAT object first. So, if you're concerned about efficiency, create these objects once and re-use them.
[Function] make-external-format name &key eol-style little-endian id => external-format
Creates and returns an EXTERNAL-FORMAT object. name is a symbol, eol-style is one of the keywords :CR, :LF, or :CRLF, and little-endian is a generalized boolean. The default value for eol-style is the value of *DEFAULT-EOL-STYLE* except for Windows code pages where it is :CRLF. The default value for little-endian is the value of *DEFAULT-LITTLE-ENDIAN* - this value is ignored unless name denotes one of UTF-16 or UTF-32. id must be an integer denoting a Windows code page known by FLEXI-STREAMS if name is :CODE-PAGE or WIN32:CODE-PAGE, otherwise the value is ignored. See the section about external formats for more info. Examples (run on Windows): CL-USER 1 > (make-external-format :latin-1) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :CRLF) 2067DA84> CL-USER 2 > (make-external-format :latin-1 :eol-style :lf) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) 2068B4D4> CL-USER 3 > (make-external-format :ibm437) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:CODE-PAGE :ID 437 :EOL-STYLE :CRLF) 2069B33C> CL-USER 4 > (make-external-format :ucs-2) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-16 :EOL-STYLE :CRLF :LITTLE-ENDIAN T) 206B4F4C> CL-USER 5 > (make-external-format :ucs-2be) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-16 :EOL-STYLE :CRLF :LITTLE-ENDIAN NIL) 2067DBE4> CL-USER 6 > (make-external-format :ucs-2be :eol-style :cr) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-16 :EOL-STYLE :CR :LITTLE-ENDIAN NIL) 206B54AC>
name
eol-style
:CR
:LF
:CRLF
little-endian
*DEFAULT-EOL-STYLE*
*DEFAULT-LITTLE-ENDIAN*
id
WIN32:CODE-PAGE
Examples (run on Windows):
CL-USER 1 > (make-external-format :latin-1) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :CRLF) 2067DA84> CL-USER 2 > (make-external-format :latin-1 :eol-style :lf) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1 :EOL-STYLE :LF) 2068B4D4> CL-USER 3 > (make-external-format :ibm437) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:CODE-PAGE :ID 437 :EOL-STYLE :CRLF) 2069B33C> CL-USER 4 > (make-external-format :ucs-2) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-16 :EOL-STYLE :CRLF :LITTLE-ENDIAN T) 206B4F4C> CL-USER 5 > (make-external-format :ucs-2be) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-16 :EOL-STYLE :CRLF :LITTLE-ENDIAN NIL) 2067DBE4> CL-USER 6 > (make-external-format :ucs-2be :eol-style :cr) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-16 :EOL-STYLE :CR :LITTLE-ENDIAN NIL) 206B54AC>
[Readers] external-format-name external-format => name external-format-eol-style external-format => eol-style external-format-little-endian external-format => little-endian external-format-id external-format => id
These methods can be used to query an EXTERNAL-FORMAT object for its attributes.
[Functions] external-format-equal external-format-1 external-format-2 => generalized-boolean
Checks whether the two external formats external-format-1 and external-format-2 are equivalent with respect to their effects on flexi streams. Examples (run on Windows): CL-USER 1 > (make-external-format :ucs-4le) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-32 :EOL-STYLE :CRLF :LITTLE-ENDIAN T) 2067FB74> CL-USER 2 > (external-format-equal * (make-external-format :utf32 :little-endian t)) T CL-USER 3 > (make-external-format :code-page :id 437) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:CODE-PAGE :ID 437 :EOL-STYLE :CRLF) 2069428C> CL-USER 4 > (external-format-equal * (make-external-format :ibm437)) T
external-format-1
external-format-2
CL-USER 1 > (make-external-format :ucs-4le) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-32 :EOL-STYLE :CRLF :LITTLE-ENDIAN T) 2067FB74> CL-USER 2 > (external-format-equal * (make-external-format :utf32 :little-endian t)) T CL-USER 3 > (make-external-format :code-page :id 437) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:CODE-PAGE :ID 437 :EOL-STYLE :CRLF) 2069428C> CL-USER 4 > (external-format-equal * (make-external-format :ibm437)) T
[Special variable] *default-eol-style*
The default value for the eol-style keyword argument of MAKE-EXTERNAL-FORMAT. Its initial value is :CRLF on Windows and :LF on other operating systems.
[Special variable] *default-little-endian*
The default value for the little-endian keyword argument of MAKE-EXTERNAL-FORMAT. Its initial value corresponds to the endianess of the platform FLEXI-STREAMS is used on as revealed by the :LITTLE-ENDIAN feature.
:LITTLE-ENDIAN
[Condition] external-format-condition
All conditions related to external formats are of this type. There's a slot for the external format which can be accessed with EXTERNAL-FORMAT-CONDITION-EXTERNAL-FORMAT.
EXTERNAL-FORMAT-CONDITION-EXTERNAL-FORMAT
[Reader] external-format-condition-external-format condition => external-format
If condition is of type EXTERNAL-FORMAT-CONDITION, this function will return the associated external format. Note that there are situation which happen during the creation of external formats where this method returns NIL.
condition
EXTERNAL-FORMAT-CONDITION
NIL
[Condition] external-format-error
All errors related to external formats are of this type. This is a subtype of EXTERNAL-FORMAT-CONDITION.
[Condition] external-format-encoding-error
All errors related to encoding problems with external formats are of this type. (This includes situation where an end of file is encountered in the middle of a multi-octet character.) When this condition is signalled during reading, USE-VALUE restart is provided. See also *SUBSTITUTION-CHAR* and the example for it. EXTERNAL-FORMAT-ENCODING-ERROR is a subtype of EXTERNAL-FORMAT-ERROR.
USE-VALUE
*SUBSTITUTION-CHAR*
EXTERNAL-FORMAT-ENCODING-ERROR
EXTERNAL-FORMAT-ERROR
[Special variable] *substitution-char*
If this value is not NIL, it should be a character which is used (as if by a USE-VALUE restart) whenever during reading an error of type EXTERNAL-FORMAT-ENCODING-ERROR would have been signalled otherwise. CL-USER 1 > (defun foo () ;; not a valid UTF-8 sequence (with-input-from-sequence (in '(#xe4 #xf6 #xfc)) (setq in (make-flexi-stream in :external-format :utf8)) (read-line in))) FOO CL-USER 2 > (foo) Error: Unexpected value #xF6 in UTF-8 sequence. 1 (continue) Specify a character to be used instead. 2 (abort) Return to level 0. 3 Return to top loop level 0. Type :b for backtrace, :c <option number> to proceed, or :? for other options CL-USER 3 : 1 > :c Type a character: x Error: End of file while in UTF-8 sequence. 1 (continue) Specify a character to be used instead. 2 (abort) Return to level 0. 3 Return to top loop level 0. Type :b for backtrace, :c <option number> to proceed, or :? for other options CL-USER 4 : 1 > :c Type a character: y "xy" T CL-USER 5 > (handler-bind ((external-format-encoding-error (lambda (condition) (use-value #\-)))) (foo)) "--" T CL-USER 6 > (let ((*substitution-char* #\?)) (foo)) "??" T
CL-USER 1 > (defun foo () ;; not a valid UTF-8 sequence (with-input-from-sequence (in '(#xe4 #xf6 #xfc)) (setq in (make-flexi-stream in :external-format :utf8)) (read-line in))) FOO CL-USER 2 > (foo) Error: Unexpected value #xF6 in UTF-8 sequence. 1 (continue) Specify a character to be used instead. 2 (abort) Return to level 0. 3 Return to top loop level 0. Type :b for backtrace, :c <option number> to proceed, or :? for other options CL-USER 3 : 1 > :c Type a character: x Error: End of file while in UTF-8 sequence. 1 (continue) Specify a character to be used instead. 2 (abort) Return to level 0. 3 Return to top loop level 0. Type :b for backtrace, :c <option number> to proceed, or :? for other options CL-USER 4 : 1 > :c Type a character: y "xy" T CL-USER 5 > (handler-bind ((external-format-encoding-error (lambda (condition) (use-value #\-)))) (foo)) "--" T CL-USER 6 > (let ((*substitution-char* #\?)) (foo)) "??" T
[Restart] accept-overlong-sequence
This is a restart which is established whenever a UTF-8 "overlong" sequence is encountered. If you invoke this restart, the corresponding code point will be accepted although it was encoded in an illegal way.
MAKE-FLEXI-STREAM
READ-BYTE
READ-SEQUENCE
OCTET
WRITE-BYTE
WRITE-SEQUENCE
A flexi stream behaves like an ordinary Lisp stream. It is an input stream if the underlying binary stream is an input stream, and it is an output stream when the underlying binary stream is an output stream. You can write characters as well as octets to an output flexi stream and similarly you can read characters and octets from an input flexi stream.
A flexi stream always has an external format associated with it which is deployed whenever you read characters from the stream or write characters to it. You can change the external format while you use the stream.
Once you're using a flexi stream you should not read from or write to the underlying stream directly anymore.
If you close a flexi stream, the underlying stream will also be closed. However, it also suffices to close the underlying stream directly should you not want to use the flexi stream anymore. So, the following usage (where IN is implicitly closed at the end) is OK:
IN
(with-open-file (in "/foo/bar/baz.txt") (let ((flexi (make-flexi-stream in :external-format :hebrew))) (read-line flexi)))
Output flexi streams will try to keep track of the column they're in but you can also set the column directly. This value will be incremented by one for each character written to the stream and it will be set to 0 if you send a #\Newline character. The column will be set to NIL if an OCTET is sent to the stream. Once the column is NIL it'll stay like that unless it is explicitly set to another value.
0
#\Newline
Input flexi streams keep track of their position within the stream. This value is incremented by one for each OCTET read from the stream, and it is incremented by the number of octets actually read for each character read from the stream. So, if the encoding is UTF-8, reading the character #\ä (a-umlaut) will advance the position by two. If the encoding is UTF-32 and the end-of-line style is :CRLF, reading a #\Newline will advance the position by eight.
#\ä
You can also set the bound of an input flexi stream. Initially it is NIL, but when it's an integer and the stream's position has gone beyond this bound, the stream will behave as if no more input is available.
Caveat: You can only unread a character from a flexi stream if you haven't changed the external format after you read it.
Caveat: The underlying stream should either be a binary stream (i.e. have an element type that is a subtype of integer) or it should explicitly use an external format with :LF as its end-of-line style. Otherwise it might perform unwanted conversion of line endings on its own. (LispWorks does this even if you write binary data to the stream using WRITE-SEQUENCE.)
[Standard class] flexi-stream
Every flexi stream returned by MAKE-FLEXI-STREAM is of this type which is a subtype of STREAM.
STREAM
[Standard class] flexi-input-stream
A flexi stream is of this type if its underlying stream is an input stream. This is a subtype of FLEXI-STREAM.
FLEXI-STREAM
[Standard class] flexi-output-stream
A flexi stream is of this type if its underlying stream is an output stream. This is a subtype of FLEXI-STREAM.
[Standard class] flexi-io-stream
A flexi stream is of this type if it is both a FLEXI-INPUT-STREAM as well as a FLEXI-OUTPUT-STREAM.
FLEXI-INPUT-STREAM
FLEXI-OUTPUT-STREAM
[Function] make-flexi-stream stream &key external-format element-type column position bound => flexi-stream
Creates and returns a flexi stream, i.e. an object of type FLEXI-STREAM. stream is the underlying Lisp stream. external-format is the initial external format to be used by the stream, the default is the value of evaluating (MAKE-EXTERNAL-FORMAT :LATIN1). element-type is the initial element type of the flexi stream the default of which is LW:SIMPLE-CHAR for LispWorks and CHARACTER otherwise. column is the initial column of the stream and should only be provided for output streams, the default is 0. position is the initial octet position of the stream and must only be provided for input streams, the default is 0. bound should be NIL (the default) or an integer and must only be provided for input streams. If the octet position of the stream has gone beyond this bound, the stream will behave as if no more input is available. See the section about flexi streams for more information.
stream
external-format
(MAKE-EXTERNAL-FORMAT :LATIN1)
element-type
LW:SIMPLE-CHAR
CHARACTER
column
position
bound
[Accessors] flexi-stream-external-format flexi-stream => external-format (setf (flexi-stream-external-format flexi-stream) external-format) flexi-stream-element-type flexi-stream => element-type (setf (flexi-stream-element-type flexi-stream) element-type) flexi-stream-column flexi-output-stream => column (setf (flexi-stream-column flexi-output-stream) column) flexi-stream-position flexi-input-stream => position (setf (flexi-stream-position flexi-input-stream) position) flexi-stream-bound flexi-input-stream => bound (setf (flexi-stream-bound flexi-input-stream) bound)
These methods can be used to get and set the corresponding attributes of a flexi stream. (SETF FLEXI-STREAM-EXTERNAL-FORMAT) accepts keyword symbols (names of external formats), lists (which should be valid lists of parameters to MAKE-EXTERNAL-FORMAT), or EXTERNAL-FORMAT objects: CL-USER 1 > (setf (flexi-stream-external-format *my-stream*) :ucs-4le) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-32 :EOL-STYLE :CRLF :LITTLE-ENDIAN T) 206920DC> CL-USER 2 > (setf (flexi-stream-external-format *my-stream*) '(:ucs-2be :eol-style :br)) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-16 :EOL-STYLE :BR :LITTLE-ENDIAN NIL) 20696934> CL-USER 3 > (setf (flexi-stream-external-format *my-stream*) (make-external-format :ibm437)) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:CODE-PAGE :ID 437 :EOL-STYLE :CRLF) 2068716C>
(SETF FLEXI-STREAM-EXTERNAL-FORMAT) accepts keyword symbols (names of external formats), lists (which should be valid lists of parameters to MAKE-EXTERNAL-FORMAT), or EXTERNAL-FORMAT objects:
(SETF FLEXI-STREAM-EXTERNAL-FORMAT)
CL-USER 1 > (setf (flexi-stream-external-format *my-stream*) :ucs-4le) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-32 :EOL-STYLE :CRLF :LITTLE-ENDIAN T) 206920DC> CL-USER 2 > (setf (flexi-stream-external-format *my-stream*) '(:ucs-2be :eol-style :br)) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-16 :EOL-STYLE :BR :LITTLE-ENDIAN NIL) 20696934> CL-USER 3 > (setf (flexi-stream-external-format *my-stream*) (make-external-format :ibm437)) #<FLEXI-STREAMS::EXTERNAL-FORMAT (:CODE-PAGE :ID 437 :EOL-STYLE :CRLF) 2068716C>
[Reader] flexi-stream-stream flexi-stream => stream
This method returns the underlying stream of a flexi stream.
[Generic function] unread-byte byte stream => nil
Similar to UNREAD-CHAR in that it "unreads" the last octet from stream which must be a flexi stream. Note that you can only call UNREAD-BYTE after a corresponding READ-BYTE, not after READ-CHAR.
UNREAD-CHAR
UNREAD-BYTE
[Generic function] peek-byte stream &optional peek-type eof-error-p eof-value => byte
PEEK-BYTE is like PEEK-CHAR, i.e. it returns an octet from stream (which must be a flexi stream) without actually removing it. If peek-type is NIL, the next octet is returned, if peek-type is T, the next octet which is not 0 is returned, if peek-type is an octet, the next octet which equals peek-type is returned. eof-error-p and eof-value are interpreted as usual. Note that the parameters aren't in the same order as with PEEK-CHAR because it doesn't make much sense to make stream an optional argument.
PEEK-BYTE
PEEK-CHAR
peek-type
T
eof-error-p
eof-value
Note that the parameters aren't in the same order as with PEEK-CHAR because it doesn't make much sense to make stream an optional argument.
[Type] octet
Just a shortcut for (UNSIGNED-BYTE 8).
(UNSIGNED-BYTE 8)
[Condition] flexi-stream-error
All errors related to flexi streams are of this type. This is a subtype of STREAM-ERROR.
STREAM-ERROR
[Condition] flexi-stream-out-of-sync-error
This can happen if you're trying to write to an IO stream which had prior to that "looked ahead" while reading and now can't "rewind" to the octet where you should be.
[Condition] flexi-stream-element-type-error
All errors related to problems with the element type of flexi streams are of this type. This is a subtype of FLEXI-STREAM-ERROR and has an additional slot for the element type which can be accessed with FLEXI-STREAM-ELEMENT-TYPE-ERROR-ELEMENT-TYPE.
FLEXI-STREAM-ERROR
FLEXI-STREAM-ELEMENT-TYPE-ERROR-ELEMENT-TYPE
[Reader] flexi-stream-element-type-error-element-type condition => element-type
If condition is of type FLEXI-STREAM-ELEMENT-TYPE-ERROR, this function will return the offending element type.
FLEXI-STREAM-ELEMENT-TYPE-ERROR
[Standard class] in-memory-stream
Every in-memory stream returned by MAKE-IN-MEMORY-INPUT-STREAM or MAKE-IN-MEMORY-OUTPUT-STREAM is of this type which is a subtype of STREAM.
MAKE-IN-MEMORY-INPUT-STREAM
MAKE-IN-MEMORY-OUTPUT-STREAM
[Standard class] in-memory-input-stream
Every in-memory stream returned by MAKE-IN-MEMORY-INPUT-STREAM is of this type which is a subtype of IN-MEMORY-STREAM.
IN-MEMORY-STREAM
[Standard class] in-memory-output-stream
Every in-memory stream returned by MAKE-IN-MEMORY-OUTPUT-STREAM is of this type which is a subtype of IN-MEMORY-STREAM.
[Standard class] list-stream
Every in-memory input stream is of this type if it reads from a list.
[Standard class] vector-stream
Every in-memory stream is of this type if it reads from or writes to a vector.
[Generic function] make-in-memory-input-stream sequence &key start end transformer => in-memory-input-stream
Returns a binary input stream (of type IN-MEMORY-INPUT-STREAM) which will supply, in order, the octets in the subsequence of sequence bounded by start (the default is 0) and end (the default is the length of sequence). sequence must either be a list or a vector of octets. Each octet returned will be transformed in turn by the optional transformer function.
IN-MEMORY-INPUT-STREAM
sequence
start
end
transformer
[Function] make-in-memory-output-stream &key element-type transformer => in-memory-output-stream
Returns a binary output stream (of type IN-MEMORY-OUTPUT-STREAM) which accepts objects of type element-type (a subtype of OCTET) and makes available a sequence (see GET-OUTPUT-STREAM-SEQUENCE) that contains the octets that were actually output. The octets stored will each be transformed by the optional transformer function.
IN-MEMORY-OUTPUT-STREAM
GET-OUTPUT-STREAM-SEQUENCE
[Generic function] get-output-stream-sequence stream &key as-list => sequence
Returns a vector containing, in order, all the octets that have been output to the in-memory output stream stream. This operation clears any octets on stream, so the vector contains only those octets which have been output since the last call to GET-OUTPUT-STREAM-SEQUENCE or since the creation of the stream, whichever occurred most recently. If as-list is true the return value is coerced to a list.
as-list
[Generic function] output-stream-sequence-length stream => length
Returns the current length of the underlying vector of the in-memory output stream stream, i.e. this is the length of the sequence that GET-OUTPUT-STREAM-SEQUENCE would return if called at this very moment.
[Macro] with-input-from-sequence (var sequence &key start end transformer) statement* => result*
Creates an in-memory input stream from the sequence sequence using the parameters start and end (see MAKE-IN-MEMORY-INPUT-STREAM), binds var to this stream and then executes the statement* forms. A function transformer may optionally be specified to transform the returned octets. The stream is automatically closed on exit from WITH-OUTPUT-TO-SEQUENCE, no matter whether the exit is normal or abnormal. The return value of this macro is the return value of the last statement of statement*.
var
statement*
WITH-OUTPUT-TO-SEQUENCE
[Macro] with-output-to-sequence (var &key as-list element-type transformer) statement* => sequence
Creates an in-memory output stream, binds var to this stream and then executes the statement* forms. The stream stores data of type element-type (a subtype of OCTET) which is (optionally) transformed by the function transformer prior to storage. The stream is automatically closed on exit from WITH-OUTPUT-TO-SEQUENCE, no matter whether the exit is normal or abnormal. The return value of this macro is a vector (or a list if as-list is true) containing the octets that were sent to the stream within the body of the macro.
[Condition] in-memory-stream-error
All errors related to in-memory streams are of this type. This is a subtype of STREAM-ERROR.
[Condition] in-memory-stream-closed-error
An error of this type is signalled if one tries to read from or write to an in-memory stream which had already been closed. This is a subtype of IN-MEMORY-STREAM-ERROR.
IN-MEMORY-STREAM-ERROR
[Condition] in-memory-stream-position-spec-error
Errors of this type are signalled if an erroneous position spec is used in conjunction with FILE-POSITION. This is a subtype of IN-MEMORY-STREAM-ERROR and has an additional slot for the position spec which can be accessed with IN-MEMORY-STREAM-POSITION-SPEC-ERROR-POSITION-SPEC.
FILE-POSITION
IN-MEMORY-STREAM-POSITION-SPEC-ERROR-POSITION-SPEC
[Reader] in-memory-stream-position-spec-error-position-spec condition => position-spec
If condition is of type IN-MEMORY-STREAM-POSITION-SPEC-ERROR, this function will return the offending position spec.
IN-MEMORY-STREAM-POSITION-SPEC-ERROR
[Function] string-to-octets string &key external-format start end => vector
Converts the Lisp string string from start to end to an array of octets corresponding to the external format designated by external-format. The defaults for start and end are 0 and the length of the string. The default for external-format is :LATIN1. In spite of the name, string can be any sequence of characters, but the function is optimized for strings.
string
In spite of the name, string can be any sequence of characters, but the function is optimized for strings.
[Function] octets-to-string sequence &key external-format start end => string
Converts the Lisp sequence sequence of octets from start to end to a string using the external format designated by external-format. The defaults for start and end are 0 and the length of the sequence. The default for external-format is :LATIN1. This function is optimized for the case of sequence being a vector. Don't use lists if you are in hurry.
This function is optimized for the case of sequence being a vector. Don't use lists if you are in hurry.
[Function] octet-length string &key external-format start end => length
Returns the length of the subsequence of string from start to end in octets if encoded using the external format designated by external-format. The defaults for start and end are 0 and the length of string. The default for external-format is :LATIN1. In spite of the name, string can be any sequence of characters, but the function is optimized for strings.
[Function] char-length sequence &key external-format start end => length
Kind of the inverse of OCTET-LENGTH. Returns the length of the subsequence (of octets) of sequence from start to end in characters if decoded using the external format designated by external-format. The defaults for start and end are 0 and the length of the sequence. The default for external-format is :LATIN1. Note that this function doesn't check for the validity of the data in sequence. This function is optimized for the case of sequence being a vector. Don't use lists if you are in hurry.
OCTET-LENGTH
For LispWorks and CLISP, FILE-POSITION for flexi streams will work as if the function had been applied to the underlying stream, and for in-memory streams it will try to do something sensible if the underlying data structure is a vector (i.e. not a list). Patches for other Common Lisp implementations should be sent to the trivial-gray-streams maintainers.
$Header: /usr/local/cvsrep/flexi-streams/doc/index.html,v 1.126 2008/08/26 10:59:24 edi Exp $
BACK TO MY HOMEPAGE flexi-streams-1.0.7/encode.lisp 0000775 0000000 0000000 00000033527 11175756704 0016462 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/encode.lisp,v 1.26 2008/05/26 10:55:08 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) (defgeneric char-to-octets (format char writer) (declare #.*standard-optimize-settings*) (:documentation "Converts the character CHAR to a sequence of octets using the external format FORMAT. The conversion is performed by calling the unary function \(which must be a functional object) WRITER repeatedly each octet. The return value of this function is unspecified.")) (defgeneric write-sequence* (format stream sequence start end) (declare #.*standard-optimize-settings*) (:documentation "A generic function which dispatches on the external format and does the real work for STREAM-WRITE-SEQUENCE.")) (defgeneric string-to-octets* (format string start end) (declare #.*standard-optimize-settings*) (:documentation "A generic function which dispatches on the external format and does the real work for STRING-TO-OCTETS.")) (defmethod string-to-octets* :around (format (list list) start end) (declare #.*standard-optimize-settings*) (string-to-octets* format (coerce list 'string*) start end)) (defmacro define-sequence-writers ((format-class) &body body) "Non-hygienic utility macro which defines methods for WRITE-SEQUENCE* and STRING-TO-OCTETS* for the class FORMAT-CLASS. For BODY see the docstring of DEFINE-CHAR-ENCODERS." (let ((body `((locally (declare #.*fixnum-optimize-settings*) ,@body)))) `(progn (defmethod string-to-octets* ((format ,format-class) string start end) (declare #.*standard-optimize-settings*) (declare (fixnum start end) (string string)) (let ((octets (make-array (compute-number-of-octets format string start end) :element-type 'octet)) (j 0)) (declare (fixnum j)) (loop for i of-type fixnum from start below end do (macrolet ((octet-writer (form) `(progn (setf (aref (the (array octet *) octets) j) ,form) (incf j)))) (symbol-macrolet ((char-getter (char string i))) (progn ,@body)))) octets)) (defmethod write-sequence* ((format ,format-class) stream sequence start end) (declare #.*standard-optimize-settings*) (declare (fixnum start end)) (with-accessors ((column flexi-stream-column)) stream (let* ((octet-seen-p nil) (buffer-pos 0) ;; estimate should be good enough... (factor (encoding-factor format)) ;; we don't want arbitrarily large buffer, do we? (buffer-size (min +buffer-size+ (ceiling (* factor (- end start))))) (buffer (make-octet-buffer buffer-size))) (declare (fixnum buffer-pos buffer-size) (boolean octet-seen-p) (type (array octet *) buffer)) (macrolet ((octet-writer (form) `(write-octet ,form))) (labels ((flush-buffer () "Sends all octets in BUFFER to the underlying stream." (write-sequence buffer stream :end buffer-pos) (setq buffer-pos 0)) (write-octet (octet) "Adds one octet to the buffer and flushes it if necessary." (declare (type octet octet)) (when (>= buffer-pos buffer-size) (flush-buffer)) (setf (aref buffer buffer-pos) octet) (incf buffer-pos)) (write-object (object) "Dispatches to WRITE-OCTET or WRITE-CHARACTER depending on the type of OBJECT." (etypecase object (octet (setq octet-seen-p t) (write-octet object)) (character (symbol-macrolet ((char-getter object)) ,@body))))) (macrolet ((iterate (&body output-forms) "An unhygienic macro to implement the actual iteration through SEQUENCE. OUTPUT-FORM is the form to retrieve one sequence element and put its octet representation into the buffer." `(loop for index of-type fixnum from start below end do (progn ,@output-forms) finally (when (plusp buffer-pos) (flush-buffer))))) (etypecase sequence (string (iterate (symbol-macrolet ((char-getter (char sequence index))) ,@body))) (array (iterate (symbol-macrolet ((char-getter (aref sequence index))) ,@body))) (list (iterate (write-object (nth index sequence)))))) ;; update the column slot, setting it to NIL if we sent ;; octets (setq column (cond (octet-seen-p nil) (t (let ((last-newline-pos (position #\Newline sequence :test #'char= :start start :end end :from-end t))) (cond (last-newline-pos (- end last-newline-pos 1)) (column (+ column (- end start)))))))))))))))) (defmacro define-char-encoders ((lf-format-class cr-format-class crlf-format-class) &body body) "Non-hygienic utility macro which defines several encoding-related methods for the classes LF-FORMAT-CLASS, CR-FORMAT-CLASS, and CRLF-FORMAT-CLASS where it is assumed that CR-FORMAT-CLASS is the same encoding as LF-FORMAT-CLASS but with CR instead of LF line endings and similar for CRLF-FORMAT-CLASS, i.e. LF-FORMAT-CLASS is the base class. BODY is a code template for the code to convert one character to octets. BODY must contain a symbol CHAR-GETTER representing the form which is used to obtain the character and a forms like \(OCTET-WRITE ) to write the octet . The CHAR-GETTER form might be called more than once." `(progn (defmethod char-to-octets ((format ,lf-format-class) char writer) (declare #.*fixnum-optimize-settings*) (declare (character char) (function writer)) (symbol-macrolet ((char-getter char)) (macrolet ((octet-writer (form) `(funcall writer ,form))) ,@body))) (define-sequence-writers (,lf-format-class) ,@body) (define-sequence-writers (,cr-format-class) ;; modify the body so that the getter replaces a #\Newline ;; with a #\Return ,@(sublis `((char-getter . ,(with-unique-names (char) `(let ((,char char-getter)) (declare (character ,char)) (if (char= ,char #\Newline) #\Return ,char))))) body)) (define-sequence-writers (,crlf-format-class) ;; modify the body so that we potentially write octets for ;; two characters (#\Return and #\Linefeed) - the original ;; body is wrapped with the WRITE-CHAR local function ,(with-unique-names (char write-char) `(flet ((,write-char (,char) ,@(sublis `((char-getter . ,char)) body))) (let ((,char char-getter)) (declare (character ,char)) (cond ((char= ,char #\Newline) (,write-char #\Return) (,write-char #\Linefeed)) (t (,write-char ,char))))))))) (define-char-encoders (flexi-latin-1-format flexi-cr-latin-1-format flexi-crlf-latin-1-format) (let ((octet (char-code char-getter))) (when (> octet 255) (signal-encoding-error format "~S (code ~A) is not a LATIN-1 character." char-getter octet)) (octet-writer octet))) (define-char-encoders (flexi-ascii-format flexi-cr-ascii-format flexi-crlf-ascii-format) (let ((octet (char-code char-getter))) (when (> octet 127) (signal-encoding-error format "~S (code ~A) is not an ASCII character." char-getter octet)) (octet-writer octet))) (define-char-encoders (flexi-8-bit-format flexi-cr-8-bit-format flexi-crlf-8-bit-format) (with-accessors ((encoding-hash external-format-encoding-hash)) format (let ((octet (gethash (char-code char-getter) encoding-hash))) (unless octet (signal-encoding-error format "~S (code ~A) is not in this encoding." char-getter octet)) (octet-writer octet)))) (define-char-encoders (flexi-utf-8-format flexi-cr-utf-8-format flexi-crlf-utf-8-format) ;; the old version using LDB was more elegant, but some Lisps had ;; trouble optimizing it (let ((char-code (char-code char-getter))) (tagbody (cond ((< char-code #x80) (octet-writer char-code) (go zero)) ((< char-code #x800) (octet-writer (logior* #b11000000 (ash* char-code -6))) (go one)) ((< char-code #x10000) (octet-writer (logior* #b11100000 (ash* char-code -12))) (go two)) (t (octet-writer (logior* #b11110000 (ash* char-code -18))))) (octet-writer (logior* #b10000000 (logand* #b00111111 (ash* char-code -12)))) two (octet-writer (logior* #b10000000 (logand* #b00111111 (ash* char-code -6)))) one (octet-writer (logior* #b10000000 (logand* #b00111111 char-code))) zero))) (define-char-encoders (flexi-utf-16-le-format flexi-cr-utf-16-le-format flexi-crlf-utf-16-le-format) (flet ((write-word (word) (octet-writer (logand* #x00ff word)) (octet-writer (ash* (logand* #xff00 word) -8)))) (declare (inline write-word)) (let ((char-code (char-code char-getter))) (declare (type char-code-integer char-code)) (cond ((< char-code #x10000) (write-word char-code)) (t (decf char-code #x10000) (write-word (logior* #xd800 (ash* char-code -10))) (write-word (logior* #xdc00 (logand* #x03ff char-code)))))))) (define-char-encoders (flexi-utf-16-be-format flexi-cr-utf-16-be-format flexi-crlf-utf-16-be-format) (flet ((write-word (word) (octet-writer (ash* (logand* #xff00 word) -8)) (octet-writer (logand* #x00ff word)))) (declare (inline write-word)) (let ((char-code (char-code char-getter))) (declare (type char-code-integer char-code)) (cond ((< char-code #x10000) (write-word char-code)) (t (decf char-code #x10000) (write-word (logior* #xd800 (ash* char-code -10))) (write-word (logior* #xdc00 (logand* #x03ff char-code)))))))) (define-char-encoders (flexi-utf-32-le-format flexi-cr-utf-32-le-format flexi-crlf-utf-32-le-format) (let ((char-code (char-code char-getter))) (octet-writer (logand* #x00ff char-code)) (octet-writer (logand* #x00ff (ash* char-code -8))) (octet-writer (logand* #x00ff (ash* char-code -16))) (octet-writer (logand* #x00ff (ash* char-code -24))))) (define-char-encoders (flexi-utf-32-be-format flexi-cr-utf-32-be-format flexi-crlf-utf-32-be-format) (let ((char-code (char-code char-getter))) (octet-writer (logand* #x00ff (ash* char-code -24))) (octet-writer (logand* #x00ff (ash* char-code -16))) (octet-writer (logand* #x00ff (ash* char-code -8))) (octet-writer (logand* #x00ff char-code)))) (defmethod char-to-octets ((format flexi-cr-mixin) char writer) (declare #.*fixnum-optimize-settings*) (declare (character char)) (if (char= char #\Newline) (call-next-method format #\Return writer) (call-next-method))) (defmethod char-to-octets ((format flexi-crlf-mixin) char writer) (declare #.*fixnum-optimize-settings*) (declare (character char)) (cond ((char= char #\Newline) (call-next-method format #\Return writer) (call-next-method format #\Linefeed writer)) (t (call-next-method)))) flexi-streams-1.0.7/external-format.lisp 0000664 0000000 0000000 00000041416 11175756704 0020326 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/external-format.lisp,v 1.24 2008/05/26 10:55:08 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) (defclass external-format () ((name :initarg :name :reader external-format-name :documentation "The name of the external format - a keyword.") (id :initarg :id :initform nil :reader external-format-id :documentation "If the external format denotes a Windows code page this ID specifies which one to use. Otherwise the value is ignored \(and usually NIL).") (little-endian :initarg :little-endian :initform *default-little-endian* :reader external-format-little-endian :documentation "Whether multi-octet values are read and written with the least significant octet first. For 8-bit encodings like :ISO-8859-1 this value is ignored.") (eol-style :initarg :eol-style :reader external-format-eol-style :documentation "The character\(s) to or from which a #\Newline will be translated - one of the keywords :CR, :LF, or :CRLF.")) (:documentation "EXTERNAL-FORMAT objects are used to denote encodings for flexi streams or for the string functions defined in strings.lisp.")) (defmethod make-load-form ((thing external-format) &optional environment) "Defines a way to reconstruct external formats. Needed for OpenMCL." (make-load-form-saving-slots thing :environment environment)) (defclass flexi-cr-mixin () () (:documentation "A mixin for external-formats where the end-of-line designator is #\Return.")) (defclass flexi-crlf-mixin () () (:documentation "A mixin for external-formats where the end-of-line designator is the sequence #\Return #\Linefeed.")) (defclass flexi-8-bit-format (external-format) ((encoding-hash :accessor external-format-encoding-hash) (decoding-table :accessor external-format-decoding-table)) (:documentation "The class for all flexi streams which use an 8-bit encoding and thus need additional slots for the encoding/decoding tables.")) (defclass flexi-cr-8-bit-format (flexi-cr-mixin flexi-8-bit-format) () (:documentation "Special class for external formats which use an 8-bit encoding /and/ have #\Return as the line-end character.")) (defclass flexi-crlf-8-bit-format (flexi-crlf-mixin flexi-8-bit-format) () (:documentation "Special class for external formats which use an 8-bit encoding /and/ have the sequence #\Return #\Linefeed as the line-end character.")) (defclass flexi-ascii-format (flexi-8-bit-format) () (:documentation "Special class for external formats which use the US-ASCII encoding.")) (defclass flexi-cr-ascii-format (flexi-cr-mixin flexi-ascii-format) () (:documentation "Special class for external formats which use the US-ASCII encoding /and/ have #\Return as the line-end character.")) (defclass flexi-crlf-ascii-format (flexi-crlf-mixin flexi-ascii-format) () (:documentation "Special class for external formats which use the US-ASCII encoding /and/ have the sequence #\Return #\Linefeed as the line-end character.")) (defclass flexi-latin-1-format (flexi-8-bit-format) () (:documentation "Special class for external formats which use the ISO-8859-1 encoding.")) (defclass flexi-cr-latin-1-format (flexi-cr-mixin flexi-latin-1-format) () (:documentation "Special class for external formats which use the ISO-8859-1 encoding /and/ have #\Return as the line-end character.")) (defclass flexi-crlf-latin-1-format (flexi-crlf-mixin flexi-latin-1-format) () (:documentation "Special class for external formats which use the ISO-8859-1 encoding /and/ have the sequence #\Return #\Linefeed as the line-end character.")) (defclass flexi-utf-32-format (external-format) () (:documentation "Abstract class for external formats which use the UTF-32 encoding.")) (defclass flexi-utf-32-le-format (flexi-utf-32-format) () (:documentation "Special class for external formats which use the UTF-32 encoding with little-endian byte ordering.")) (defclass flexi-cr-utf-32-le-format (flexi-cr-mixin flexi-utf-32-le-format) () (:documentation "Special class for external formats which use the UTF-32 encoding with little-endian byte ordering /and/ have #\Return as the line-end character.")) (defclass flexi-crlf-utf-32-le-format (flexi-crlf-mixin flexi-utf-32-le-format) () (:documentation "Special class for external formats which use the UTF-32 encoding with little-endian byte ordering /and/ have the sequence #\Return #\Linefeed as the line-end character.")) (defclass flexi-utf-32-be-format (flexi-utf-32-format) () (:documentation "Special class for external formats which use the UTF-32 encoding with big-endian byte ordering.")) (defclass flexi-cr-utf-32-be-format (flexi-cr-mixin flexi-utf-32-be-format) () (:documentation "Special class for external formats which use the UTF-32 encoding with big-endian byte ordering /and/ have #\Return as the line-end character.")) (defclass flexi-crlf-utf-32-be-format (flexi-crlf-mixin flexi-utf-32-be-format) () (:documentation "Special class for external formats which use the the UTF-32 encoding with big-endian byte ordering /and/ have the sequence #\Return #\Linefeed as the line-end character.")) (defclass flexi-utf-16-format (external-format) () (:documentation "Abstract class for external formats which use the UTF-16 encoding.")) (defclass flexi-utf-16-le-format (flexi-utf-16-format) () (:documentation "Special class for external formats which use the UTF-16 encoding with little-endian byte ordering.")) (defclass flexi-cr-utf-16-le-format (flexi-cr-mixin flexi-utf-16-le-format) () (:documentation "Special class for external formats which use the UTF-16 encoding with little-endian byte ordering /and/ have #\Return as the line-end character.")) (defclass flexi-crlf-utf-16-le-format (flexi-crlf-mixin flexi-utf-16-le-format) () (:documentation "Special class for external formats which use the UTF-16 encoding with little-endian byte ordering /and/ have the sequence #\Return #\Linefeed as the line-end character.")) (defclass flexi-utf-16-be-format (flexi-utf-16-format) () (:documentation "Special class for external formats which use the UTF-16 encoding with big-endian byte ordering.")) (defclass flexi-cr-utf-16-be-format (flexi-cr-mixin flexi-utf-16-be-format) () (:documentation "Special class for external formats which use the UTF-16 encoding with big-endian byte ordering /and/ have #\Return as the line-end character.")) (defclass flexi-crlf-utf-16-be-format (flexi-crlf-mixin flexi-utf-16-be-format) () (:documentation "Special class for external formats which use the UTF-16 encoding with big-endian byte ordering /and/ have the sequence #\Return #\Linefeed as the line-end character.")) (defclass flexi-utf-8-format (external-format) () (:documentation "Special class for external formats which use the UTF-8 encoding.")) (defclass flexi-cr-utf-8-format (flexi-cr-mixin flexi-utf-8-format) () (:documentation "Special class for external formats which use the UTF-8 encoding /and/ have #\Return as the line-end character.")) (defclass flexi-crlf-utf-8-format (flexi-crlf-mixin flexi-utf-8-format) () (:documentation "Special class for external formats which use the UTF-8 encoding /and/ have the sequence #\Return #\Linefeed as the line-end character.")) (defmethod initialize-instance :after ((external-format flexi-8-bit-format) &rest initargs) "Sets the fixed encoding/decoding tables for this particular external format." (declare #.*standard-optimize-settings*) (declare (ignore initargs)) (with-accessors ((encoding-hash external-format-encoding-hash) (decoding-table external-format-decoding-table) (name external-format-name) (id external-format-id)) external-format (multiple-value-setq (encoding-hash decoding-table) (cond ((ascii-name-p name) (values +ascii-hash+ +ascii-table+)) ((koi8-r-name-p name) (values +koi8-r-hash+ +koi8-r-table+)) ((iso-8859-name-p name) (values (cdr (assoc name +iso-8859-hashes+ :test #'eq)) (cdr (assoc name +iso-8859-tables+ :test #'eq)))) ((code-page-name-p name) (values (cdr (assoc id +code-page-hashes+)) (cdr (assoc id +code-page-tables+)))))))) (defun external-format-class-name (real-name &key eol-style little-endian id) "Given the initargs for a general external format returns the name \(a symbol) of the most specific subclass matching these arguments." (declare #.*standard-optimize-settings*) (declare (ignore id)) (cond ((ascii-name-p real-name) (ecase eol-style (:lf 'flexi-ascii-format) (:cr 'flexi-cr-ascii-format) (:crlf 'flexi-crlf-ascii-format))) ((eq real-name :iso-8859-1) (ecase eol-style (:lf 'flexi-latin-1-format) (:cr 'flexi-cr-latin-1-format) (:crlf 'flexi-crlf-latin-1-format))) ((or (koi8-r-name-p real-name) (iso-8859-name-p real-name) (code-page-name-p real-name)) (ecase eol-style (:lf 'flexi-8-bit-format) (:cr 'flexi-cr-8-bit-format) (:crlf 'flexi-crlf-8-bit-format))) (t (ecase real-name (:utf-8 (ecase eol-style (:lf 'flexi-utf-8-format) (:cr 'flexi-cr-utf-8-format) (:crlf 'flexi-crlf-utf-8-format))) (:utf-16 (ecase eol-style (:lf (if little-endian 'flexi-utf-16-le-format 'flexi-utf-16-be-format)) (:cr (if little-endian 'flexi-cr-utf-16-le-format 'flexi-cr-utf-16-be-format)) (:crlf (if little-endian 'flexi-crlf-utf-16-le-format 'flexi-crlf-utf-16-be-format)))) (:utf-32 (ecase eol-style (:lf (if little-endian 'flexi-utf-32-le-format 'flexi-utf-32-be-format)) (:cr (if little-endian 'flexi-cr-utf-32-le-format 'flexi-cr-utf-32-be-format)) (:crlf (if little-endian 'flexi-crlf-utf-32-le-format 'flexi-crlf-utf-32-be-format)))))))) (defun make-external-format% (name &key (little-endian *default-little-endian*) id eol-style) "Used internally by MAKE-EXTERNAL-FORMAT to default some of the keywords arguments and to determine the right subclass of EXTERNAL-FORMAT." (declare #.*standard-optimize-settings*) (let* ((real-name (normalize-external-format-name name)) (initargs (cond ((or (iso-8859-name-p real-name) (koi8-r-name-p real-name) (ascii-name-p real-name)) (list :eol-style (or eol-style *default-eol-style*))) ((code-page-name-p real-name) (list :id (or (known-code-page-id-p id) (error 'external-format-error :format-control "Unknown code page ID ~S" :format-arguments (list id))) ;; default EOL style for Windows code pages is :CRLF :eol-style (or eol-style :crlf))) (t (list :eol-style (or eol-style *default-eol-style*) :little-endian little-endian))))) (apply #'make-instance (apply #'external-format-class-name real-name initargs) :name real-name initargs))) (defun make-external-format (name &rest args &key (little-endian *default-little-endian*) id eol-style) "Creates and returns an external format object as specified. NAME is a keyword like :LATIN1 or :UTF-8, LITTLE-ENDIAN specifies the `endianess' of the external format and is ignored for 8-bit encodings, EOL-STYLE is one of the keywords :CR, :LF, or :CRLF which denote the end-of-line character \(sequence), ID is the ID of a Windows code page \(and ignored for other encodings)." (declare #.*standard-optimize-settings*) ;; the keyword arguments are only there for arglist display in the IDE (declare (ignore id little-endian)) (let ((shortcut-args (cdr (assoc name +shortcut-map+)))) (cond (shortcut-args (apply #'make-external-format% (append shortcut-args `(:eol-style ,eol-style)))) (t (apply #'make-external-format% name args))))) (defun maybe-convert-external-format (external-format) "Given an external format designator \(a keyword, a list, or an EXTERNAL-FORMAT object) returns the corresponding EXTERNAL-FORMAT object." (declare #.*standard-optimize-settings*) (typecase external-format (symbol (make-external-format external-format)) (list (apply #'make-external-format external-format)) (otherwise external-format))) (defun external-format-equal (ef1 ef2) "Checks whether two EXTERNAL-FORMAT objects denote the same encoding." (declare #.*standard-optimize-settings*) (let* ((name1 (external-format-name ef1)) (code-page-name-p (code-page-name-p name1))) ;; they must habe the same canonical name (and (eq name1 (external-format-name ef2)) ;; if both are code pages the IDs must be the same (or (not code-page-name-p) (eql (external-format-id ef1) (external-format-id ef2))) ;; for non-8-bit encodings the endianess must be the same (or code-page-name-p (ascii-name-p name1) (koi8-r-name-p name1) (iso-8859-name-p name1) (eq name1 :utf-8) (eq (not (external-format-little-endian ef1)) (not (external-format-little-endian ef2)))) ;; the EOL style must also be the same (eq (external-format-eol-style ef1) (external-format-eol-style ef2))))) (defun normalize-external-format (external-format) "Returns a list which is a `normalized' representation of the external format EXTERNAL-FORMAT. Used internally by PRINT-OBJECT, for example. Basically, the result is an argument list that can be fed back to MAKE-EXTERNAL-FORMAT to create an equivalent object." (declare #.*standard-optimize-settings*) (let ((name (external-format-name external-format)) (eol-style (external-format-eol-style external-format))) (cond ((or (ascii-name-p name) (koi8-r-name-p name) (iso-8859-name-p name) (eq name :utf-8)) (list name :eol-style eol-style)) ((code-page-name-p name) (list name :id (external-format-id external-format) :eol-style eol-style)) (t (list name :eol-style eol-style :little-endian (external-format-little-endian external-format)))))) (defmethod print-object ((object external-format) stream) "How an EXTERNAL-FORMAT object is rendered. Uses NORMALIZE-EXTERNAL-FORMAT." (print-unreadable-object (object stream :type t :identity t) (prin1 (normalize-external-format object) stream))) flexi-streams-1.0.7/flexi-streams.asd 0000664 0000000 0000000 00000005476 11175756704 0017607 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: CL-USER; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/flexi-streams.asd,v 1.79 2008/08/26 10:59:22 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :cl-user) (defpackage :flexi-streams-system (:use :asdf :cl)) (in-package :flexi-streams-system) (defsystem :flexi-streams :version "1.0.7" :serial t :components ((:file "packages") (:file "mapping") (:file "ascii") (:file "koi8-r") (:file "iso-8859") (:file "code-pages") (:file "specials") (:file "util") (:file "conditions") (:file "external-format") (:file "length") (:file "encode") (:file "decode") (:file "in-memory") (:file "stream") #+:lispworks (:file "lw-char-stream") (:file "output") (:file "input") (:file "io") (:file "strings")) :depends-on (:trivial-gray-streams)) (defsystem :flexi-streams-test :components ((:module "test" :serial t :components ((:file "packages") (:file "test")))) :depends-on (:flexi-streams)) (defmethod perform ((o test-op) (c (eql (find-system 'flexi-streams)))) (operate 'load-op 'flexi-streams-test) (funcall (intern (symbol-name :run-all-tests) (find-package :flexi-streams-test)))) flexi-streams-1.0.7/in-memory.lisp 0000664 0000000 0000000 00000041646 11175756704 0017137 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/in-memory.lisp,v 1.31 2008/05/19 07:57:07 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) (defclass in-memory-stream (trivial-gray-stream-mixin) ((transformer :initarg :transformer :accessor in-memory-stream-transformer :documentation "A function used to transform the written/read octet to the value stored/retrieved in/from the underlying vector.") #+:cmu (open-p :initform t :accessor in-memory-stream-open-p :documentation "For CMUCL we have to keep track of this manually.")) (:documentation "An IN-MEMORY-STREAM is a binary stream that reads octets from or writes octets to a sequence in RAM.")) (defclass in-memory-input-stream (in-memory-stream fundamental-binary-input-stream) () (:documentation "An IN-MEMORY-INPUT-STREAM is a binary stream that reads octets from a sequence in RAM.")) #+:cmu (defmethod output-stream-p ((stream in-memory-input-stream)) "Explicitly states whether this is an output stream." (declare (optimize speed)) nil) (defclass in-memory-output-stream (in-memory-stream fundamental-binary-output-stream) () (:documentation "An IN-MEMORY-OUTPUT-STREAM is a binary stream that writes octets to a sequence in RAM.")) #+:cmu (defmethod input-stream-p ((stream in-memory-output-stream)) "Explicitly states whether this is an input stream." (declare (optimize speed)) nil) (defclass list-stream () ((list :initarg :list :accessor list-stream-list :documentation "The underlying list of the stream.")) (:documentation "A LIST-STREAM is a mixin for IN-MEMORY streams where the underlying sequence is a list.")) (defclass vector-stream () ((vector :initarg :vector :accessor vector-stream-vector :documentation "The underlying vector of the stream which \(for output) must always be adjustable and have a fill pointer.")) (:documentation "A VECTOR-STREAM is a mixin for IN-MEMORY streams where the underlying sequence is a vector.")) (defclass list-input-stream (list-stream in-memory-input-stream) () (:documentation "A binary input stream that gets its data from an associated list of octets.")) (defclass vector-input-stream (vector-stream in-memory-input-stream) ((index :initarg :index :accessor vector-stream-index :type (integer 0 #.array-dimension-limit) :documentation "An index into the underlying vector denoting the current position.") (end :initarg :end :accessor vector-stream-end :type (integer 0 #.array-dimension-limit) :documentation "An index into the underlying vector denoting the end of the available data.")) (:documentation "A binary input stream that gets its data from an associated vector of octets.")) (defclass vector-output-stream (vector-stream in-memory-output-stream) () (:documentation "A binary output stream that writes its data to an associated vector.")) #+:cmu (defmethod open-stream-p ((stream in-memory-stream)) "Returns a true value if STREAM is open. See ANSI standard." (declare #.*standard-optimize-settings*) (in-memory-stream-open-p stream)) #+:cmu (defmethod close ((stream in-memory-stream) &key abort) "Closes the stream STREAM. See ANSI standard." (declare #.*standard-optimize-settings*) (declare (ignore abort)) (prog1 (in-memory-stream-open-p stream) (setf (in-memory-stream-open-p stream) nil))) (defmethod check-if-open ((stream in-memory-stream)) "Checks if STREAM is open and signals an error otherwise." (declare #.*standard-optimize-settings*) (unless (open-stream-p stream) (error 'in-memory-stream-closed-error :stream stream))) (defmethod stream-element-type ((stream in-memory-stream)) "The element type is always OCTET by definition." (declare #.*standard-optimize-settings*) 'octet) (defmethod transform-octet ((stream in-memory-stream) octet) "Applies the transformer of STREAM to octet and returns the result." (declare #.*standard-optimize-settings*) (funcall (or (in-memory-stream-transformer stream) #'identity) octet)) (defmethod stream-read-byte ((stream list-input-stream)) "Reads one byte by simply popping it off of the top of the list." (declare #.*standard-optimize-settings*) (check-if-open stream) (with-accessors ((list list-stream-list)) stream (transform-octet stream (or (pop list) (return-from stream-read-byte :eof))))) (defmethod stream-listen ((stream list-input-stream)) "Checks whether list is not empty." (declare #.*standard-optimize-settings*) (check-if-open stream) (with-accessors ((list list-stream-list)) stream list)) (defmethod stream-read-sequence ((stream list-input-stream) sequence start end &key) "Repeatedly pops elements from the list until it's empty." (declare #.*standard-optimize-settings*) (declare (fixnum start end)) (with-accessors ((list list-stream-list)) stream (loop for index of-type fixnum from start below end while list do (setf (elt sequence index) (pop list)) finally (return index)))) (defmethod stream-read-byte ((stream vector-input-stream)) "Reads one byte and increments INDEX pointer unless we're beyond END pointer." (declare #.*standard-optimize-settings*) (check-if-open stream) (with-accessors ((index vector-stream-index) (end vector-stream-end) (vector vector-stream-vector)) stream (let ((current-index index)) (declare (fixnum current-index)) (cond ((< current-index (the fixnum end)) (incf (the fixnum index)) (transform-octet stream (aref vector current-index))) (t :eof))))) (defmethod stream-listen ((stream vector-input-stream)) "Checking whether INDEX is beyond END." (declare #.*standard-optimize-settings*) (check-if-open stream) (with-accessors ((index vector-stream-index) (end vector-stream-end)) stream (< (the fixnum index) (the fixnum end)))) (defmethod stream-read-sequence ((stream vector-input-stream) sequence start end &key) "Traverses both sequences in parallel until the end of one of them is reached." (declare #.*standard-optimize-settings*) (declare (fixnum start end)) (loop with vector-end of-type fixnum = (vector-stream-end stream) with vector = (vector-stream-vector stream) for index of-type fixnum from start below end for vector-index of-type fixnum = (vector-stream-index stream) while (< vector-index vector-end) do (setf (elt sequence index) (aref vector vector-index)) (incf (the fixnum (vector-stream-index stream))) finally (return index))) (defmethod stream-write-byte ((stream vector-output-stream) byte) "Writes a byte \(octet) by extending the underlying vector." (declare #.*standard-optimize-settings*) (check-if-open stream) (with-accessors ((vector vector-stream-vector)) stream (vector-push-extend (transform-octet stream byte) vector))) (defmethod stream-write-sequence ((stream vector-output-stream) sequence start end &key) "Just calls VECTOR-PUSH-EXTEND repeatedly." (declare #.*standard-optimize-settings*) (declare (fixnum start end)) (with-accessors ((vector vector-stream-vector)) stream (loop for index of-type fixnum from start below end do (vector-push-extend (elt sequence index) vector)) sequence)) (defmethod stream-file-position ((stream vector-input-stream)) "Simply returns the index into the underlying vector." (declare #.*standard-optimize-settings*) (with-accessors ((index vector-stream-index)) stream index)) (defmethod (setf stream-file-position) (position-spec (stream vector-input-stream)) "Sets the index into the underlying vector if POSITION-SPEC is acceptable." (declare #.*standard-optimize-settings*) (with-accessors ((index vector-stream-index) (end vector-stream-end)) stream (setq index (case position-spec (:start 0) (:end end) (otherwise (unless (integerp position-spec) (error 'in-memory-stream-position-spec-error :format-control "Unknown file position designator: ~S." :format-arguments (list position-spec) :stream stream :position-spec position-spec)) (unless (<= 0 position-spec end) (error 'in-memory-stream-position-spec-error :format-control "File position designator ~S is out of bounds." :format-arguments (list position-spec) :stream stream :position-spec position-spec)) position-spec))) position-spec)) (defmethod stream-file-position ((stream vector-output-stream)) "Simply returns the fill pointer of the underlying vector." (declare #.*standard-optimize-settings*) (with-accessors ((vector vector-stream-vector)) stream (fill-pointer vector))) (defmethod (setf stream-file-position) (position-spec (stream vector-output-stream)) "Sets the fill pointer underlying vector if POSITION-SPEC is acceptable. Adjusts the vector if necessary." (declare #.*standard-optimize-settings*) (with-accessors ((vector vector-stream-vector)) stream (let* ((total-size (array-total-size vector)) (new-fill-pointer (case position-spec (:start 0) (:end (warn "File position designator :END doesn't really make sense for an output stream.") total-size) (otherwise (unless (integerp position-spec) (error 'in-memory-stream-position-spec-error :format-control "Unknown file position designator: ~S." :format-arguments (list position-spec) :stream stream :position-spec position-spec)) (unless (<= 0 position-spec array-total-size-limit) (error 'in-memory-stream-position-spec-error :format-control "File position designator ~S is out of bounds." :format-arguments (list position-spec) :stream stream :position-spec position-spec)) position-spec)))) (declare (fixnum total-size new-fill-pointer)) (when (> new-fill-pointer total-size) (adjust-array vector new-fill-pointer)) (setf (fill-pointer vector) new-fill-pointer) position-spec))) (defmethod make-in-memory-input-stream ((vector vector) &key (start 0) (end (length vector)) transformer) "Returns a binary input stream which will supply, in order, the octets in the subsequence of VECTOR bounded by START and END. Each octet returned will be transformed in turn by the optional TRANSFORMER function." (declare #.*standard-optimize-settings*) (make-instance 'vector-input-stream :vector vector :index start :end end :transformer transformer)) (defmethod make-in-memory-input-stream ((list list) &key (start 0) (end (length list)) transformer) "Returns a binary input stream which will supply, in order, the octets in the subsequence of LIST bounded by START and END. Each octet returned will be transformed in turn by the optional TRANSFORMER function." (declare #.*standard-optimize-settings*) (make-instance 'list-input-stream :list (subseq list start end) :transformer transformer)) (defun make-output-vector (&key (element-type 'octet)) "Creates and returns an array which can be used as the underlying vector for a VECTOR-OUTPUT-STREAM." (declare #.*standard-optimize-settings*) (make-array 0 :adjustable t :fill-pointer 0 :element-type element-type)) (defun make-in-memory-output-stream (&key (element-type 'octet) transformer) "Returns a binary output stream which accepts objects of type ELEMENT-TYPE \(a subtype of OCTET) and makes available a sequence that contains the octes that were actually output. The octets stored will each be transformed by the optional TRANSFORMER function." (declare #.*standard-optimize-settings*) (make-instance 'vector-output-stream :vector (make-output-vector :element-type element-type) :transformer transformer)) (defmethod get-output-stream-sequence ((stream in-memory-output-stream) &key as-list) "Returns a vector containing, in order, all the octets that have been output to the IN-MEMORY stream STREAM. This operation clears any octets on STREAM, so the vector contains only those octets which have been output since the last call to GET-OUTPUT-STREAM-SEQUENCE or since the creation of the stream, whichever occurred most recently. If AS-LIST is true the return value is coerced to a list." (declare #.*standard-optimize-settings*) (with-accessors ((vector vector-stream-vector)) stream (prog1 (if as-list (coerce vector 'list) vector) (setq vector (make-output-vector))))) (defmethod output-stream-sequence-length ((stream in-memory-output-stream)) "Returns the current length of the underlying vector of the IN-MEMORY output stream STREAM." (declare (optimize speed)) (with-accessors ((vector vector-stream-vector)) stream (length (the (simple-array * (*)) vector)))) (defmacro with-input-from-sequence ((var sequence &key start end transformer) &body body) "Creates an IN-MEMORY input stream from SEQUENCE using the parameters START and END, binds VAR to this stream and then executes the code in BODY. A function TRANSFORMER may optionally be specified to transform the returned octets. The stream is automatically closed on exit from WITH-INPUT-FROM-SEQUENCE, no matter whether the exit is normal or abnormal. The return value of this macro is the return value of BODY." (with-rebinding (sequence) `(let (,var) (unwind-protect (progn (setq ,var (make-in-memory-input-stream ,sequence :start (or ,start 0) :end (or ,end (length ,sequence)) :transformer ,transformer)) ,@body) (when ,var (close ,var)))))) (defmacro with-output-to-sequence ((var &key as-list (element-type ''octet) transformer) &body body) "Creates an IN-MEMORY output stream, binds VAR to this stream and then executes the code in BODY. The stream stores data of type ELEMENT-TYPE \(a subtype of OCTET) which is \(optionally) transformed by the function TRANSFORMER prior to storage. The stream is automatically closed on exit from WITH-OUTPUT-TO-SEQUENCE, no matter whether the exit is normal or abnormal. The return value of this macro is a vector \(or a list if AS-LIST is true) containing the octets that were sent to the stream within BODY." `(let (,var) (unwind-protect (progn (setq ,var (make-in-memory-output-stream :element-type ,element-type :transformer ,transformer)) ,@body (get-output-stream-sequence ,var :as-list ,as-list)) (when ,var (close ,var))))) flexi-streams-1.0.7/input.lisp 0000664 0000000 0000000 00000031754 11175756704 0016361 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/input.lisp,v 1.78 2008/05/25 19:25:44 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) #-:lispworks (defmethod read-byte* ((flexi-input-stream flexi-input-stream)) "Reads one byte \(octet) from the underlying stream of FLEXI-OUTPUT-STREAM \(or from the internal stack if it's not empty)." (declare #.*standard-optimize-settings*) ;; we're using S instead of STREAM here because of an ;; issue with SBCL: ;; (with-accessors ((position flexi-stream-position) (bound flexi-stream-bound) (octet-stack flexi-stream-octet-stack) (s flexi-stream-stream)) flexi-input-stream (declare (integer position) (type (or null integer) bound)) (when (and bound (>= position bound)) (return-from read-byte* nil)) (incf position) (or (pop octet-stack) (read-byte s nil nil) (progn (decf position) nil)))) #+:lispworks (defmethod read-byte* ((flexi-input-stream flexi-input-stream)) "Reads one byte \(octet) from the underlying \(binary) stream of FLEXI-OUTPUT-STREAM \(or from the internal stack if it's not empty)." (declare #.*standard-optimize-settings*) (with-accessors ((position flexi-stream-position) (bound flexi-stream-bound) (octet-stack flexi-stream-octet-stack) (stream flexi-stream-stream)) flexi-input-stream (declare (integer position) (type (or null integer) bound)) (when (and bound (>= position bound)) (return-from read-byte* nil)) (incf position) (or (pop octet-stack) (read-byte stream nil nil) (progn (decf position) nil)))) #+:lispworks (defmethod read-byte* ((flexi-input-stream flexi-char-input-stream)) "Reads one byte \(octet) from the underlying stream of FLEXI-OUTPUT-STREAM \(or from the internal stack if it's not empty). Only used for LispWorks bivalent streams which aren't binary." (declare #.*standard-optimize-settings*) (with-accessors ((position flexi-stream-position) (bound flexi-stream-bound) (octet-stack flexi-stream-octet-stack) (stream flexi-stream-stream)) flexi-input-stream (declare (integer position) (type (or null integer) bound)) (when (and bound (>= position bound)) (return-from read-byte* nil)) (incf position) (or (pop octet-stack) ;; we use READ-SEQUENCE because READ-BYTE doesn't work with all ;; bivalent streams in LispWorks (let* ((buffer (make-array 1 :element-type 'octet)) (new-position (read-sequence buffer stream))) (cond ((zerop new-position) (decf position) nil) (t (aref buffer 0))))))) (defmethod stream-clear-input ((flexi-input-stream flexi-input-stream)) "Calls the corresponding method for the underlying input stream and also clears the value of the OCTET-STACK slot." (declare #.*standard-optimize-settings*) ;; note that we don't reset the POSITION slot (with-accessors ((octet-stack flexi-stream-octet-stack) (stream flexi-stream-stream)) flexi-input-stream (setq octet-stack nil) (clear-input stream))) (defmethod stream-listen ((flexi-input-stream flexi-input-stream)) "Calls the corresponding method for the underlying input stream but first checks if \(old) input is available in the OCTET-STACK slot." (declare #.*standard-optimize-settings*) (with-accessors ((position flexi-stream-position) (bound flexi-stream-bound) (octet-stack flexi-stream-octet-stack) (stream flexi-stream-stream)) flexi-input-stream (declare (integer position) (type (or null integer) bound)) (when (and bound (>= position bound)) (return-from stream-listen nil)) (or octet-stack (listen stream)))) (defmethod stream-read-byte ((stream flexi-input-stream)) "Reads one byte \(octet) from the underlying stream." (declare #.*standard-optimize-settings*) ;; set LAST-CHAR-CODE slot to NIL because we can't UNREAD-CHAR after ;; this operation (with-accessors ((last-char-code flexi-stream-last-char-code) (last-octet flexi-stream-last-octet)) stream (setq last-char-code nil) (let ((octet (read-byte* stream))) (setq last-octet octet) (or octet :eof)))) (defun unread-char% (char flexi-input-stream) "Used internally to put a character CHAR which was already read back on the stream. Uses the OCTET-STACK slot and decrements the POSITION slot accordingly." (declare #.*standard-optimize-settings*) (with-accessors ((position flexi-stream-position) (octet-stack flexi-stream-octet-stack) (external-format flexi-stream-external-format)) flexi-input-stream (let ((counter 0) octets-reversed) (declare (fixnum counter)) (flet ((writer (octet) (incf counter) (push octet octets-reversed))) (declare (dynamic-extent (function writer))) (char-to-octets external-format char #'writer) (decf position counter) (setq octet-stack (nreconc octets-reversed octet-stack)))))) (defmethod stream-read-char ((stream flexi-input-stream)) (declare #.*standard-optimize-settings*) ;; note that we do nothing for the :LF EOL style because we assume ;; that #\Newline is the same as #\Linefeed in all Lisps which will ;; use this library (with-accessors ((external-format flexi-stream-external-format) (last-octet flexi-stream-last-octet) (last-char-code flexi-stream-last-char-code)) stream ;; set LAST-OCTET slot to NIL because we can't UNREAD-BYTE after ;; this operation (setq last-octet nil) (flet ((reader () (read-byte* stream)) (unreader (char) (unread-char% char stream))) (declare (dynamic-extent (function reader) (function unreader))) (let* ((*current-unreader* #'unreader) (char-code (or (octets-to-char-code external-format #'reader) (return-from stream-read-char :eof)))) ;; remember this character and its char code for UNREAD-CHAR (setq last-char-code char-code) (or (code-char char-code) char-code))))) (defmethod stream-read-char-no-hang ((stream flexi-input-stream)) "Reads one character if the underlying stream has at least one octet available." (declare #.*standard-optimize-settings*) ;; note that this may block for non-8-bit encodings - I think ;; there's no easy way to handle this correctly (and (stream-listen stream) (stream-read-char stream))) (defmethod stream-read-sequence ((flexi-input-stream flexi-input-stream) sequence start end &key) "An optimized version which uses a buffer underneath. The function can deliver characters as well as octets and it decides what to do based on the element type of the sequence \(which takes precedence) and the element type of the stream. What you'll really get might also depend on your Lisp. Some of the implementations are more picky than others - see for example FLEXI-STREAMS-TEST::SEQUENCE-TEST." (declare #.*standard-optimize-settings*) (declare (fixnum start end)) (with-accessors ((octet-stack flexi-stream-octet-stack) (external-format flexi-stream-external-format) (last-octet flexi-stream-last-octet) (last-char-code flexi-stream-last-char-code) (element-type flexi-stream-element-type) (stream flexi-stream-stream)) flexi-input-stream (when (>= start end) (return-from stream-read-sequence start)) (when (or (subtypep (etypecase sequence (vector (array-element-type sequence)) (list t)) 'integer) (and (not (stringp sequence)) (type-equal element-type 'octet))) ;; if binary data is requested, just read from the underlying ;; stream directly and skip the rest (but flush octet stack ;; first) (let ((index start)) (declare (fixnum index)) (when octet-stack (replace sequence octet-stack :start1 start :end1 end) (let ((octets-flushed (min (length octet-stack) (- end start)))) (incf index octets-flushed) (setq octet-stack (nthcdr octets-flushed octet-stack)))) (setq index (read-sequence sequence stream :start index :end end)) (when (> index start) (setq last-char-code nil last-octet (elt sequence (1- index)))) (return-from stream-read-sequence index))) ;; otherwise hand over to the external format to do the work (read-sequence* external-format flexi-input-stream sequence start end))) (defmethod stream-unread-char ((stream flexi-input-stream) char) "Implements UNREAD-CHAR for streams of type FLEXI-INPUT-STREAM. Makes sure CHAR will only be unread if it was the last character read and if it was read with the same encoding that's currently being used by the stream." (declare #.*standard-optimize-settings*) (with-accessors ((last-char-code flexi-stream-last-char-code)) stream (unless last-char-code (error 'flexi-stream-error :format-control "No character to unread from this stream \(or external format has changed or last reading operation was binary).")) (unless (= (char-code char) last-char-code) (error 'flexi-stream-error :format-control "Last character read (~S) was different from ~S." :format-arguments (list (code-char last-char-code) char))) (unread-char% char stream) (setq last-char-code nil) nil)) (defmethod unread-byte (byte (flexi-input-stream flexi-input-stream)) "Similar to UNREAD-CHAR in that it `unreads' the last octet from STREAM. Note that you can only call UNREAD-BYTE after a corresponding READ-BYTE." (declare #.*standard-optimize-settings*) (with-accessors ((last-octet flexi-stream-last-octet) (octet-stack flexi-stream-octet-stack) (position flexi-stream-position)) flexi-input-stream (unless last-octet (error 'flexi-stream-error :format-control "No byte to unread from this stream \(or last reading operation read a character).")) (unless (= byte last-octet) (error 'flexi-stream-error :format-control "Last byte read was different from #x~X." :format-arguments (list byte))) (setq last-octet nil) (decf (the integer position)) (push byte octet-stack) nil)) (defmethod peek-byte ((flexi-input-stream flexi-input-stream) &optional peek-type (eof-error-p t) eof-value) "PEEK-BYTE is like PEEK-CHAR, i.e. it returns an octet from FLEXI-INPUT-STREAM without actually removing it. If PEEK-TYPE is NIL the next octet is returned, if PEEK-TYPE is T, the next octet which is not 0 is returned, if PEEK-TYPE is an octet, the next octet which equals PEEK-TYPE is returned. EOF-ERROR-P and EOF-VALUE are interpreted as usual." (declare #.*standard-optimize-settings*) (loop for octet = (read-byte flexi-input-stream eof-error-p eof-value) until (cond ((null peek-type)) ((eql octet eof-value)) ((eq peek-type t) (plusp octet)) (t (= octet peek-type))) finally (unless (eql octet eof-value) (unread-byte octet flexi-input-stream)) (return octet))) flexi-streams-1.0.7/io.lisp 0000775 0000000 0000000 00000011004 11175756704 0015616 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/io.lisp,v 1.2 2008/05/20 23:44:45 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) (defmethod reset-input-state ((flexi-io-stream flexi-io-stream)) "This method is used to clear any state associated with previous input before output is attempted on the stream. It can fail if the octet stack is not empty and the stream can't be `rewound'." (declare #.*standard-optimize-settings*) (with-accessors ((last-char-code flexi-stream-last-char-code) (last-octet flexi-stream-last-octet) (octet-stack flexi-stream-octet-stack) (stream flexi-stream-stream)) flexi-io-stream (when octet-stack (unless (maybe-rewind stream (length octet-stack)) (error 'flexi-stream-out-of-sync-error :stream flexi-io-stream)) (setq octet-stack nil)) (setq last-octet nil last-char-code nil))) (defmethod stream-write-byte :before ((stream flexi-io-stream) byte) (declare #.*standard-optimize-settings*) (declare (ignore byte)) (reset-input-state stream)) (defmethod stream-write-char :before ((stream flexi-io-stream) char) (declare #.*standard-optimize-settings*) (declare (ignore char)) (reset-input-state stream)) (defmethod stream-write-sequence :before ((stream flexi-io-stream) sequence start end &key) (declare #.*standard-optimize-settings*) (declare (ignore sequence start end)) (reset-input-state stream)) (defmethod stream-clear-output :before ((stream flexi-io-stream)) (declare #.*standard-optimize-settings*) (reset-input-state stream)) (defmethod reset-output-state ((flexi-io-stream flexi-io-stream)) "This method is used to clear any state associated with previous output before the stream is used for input." (declare #.*standard-optimize-settings*) (with-accessors ((column flexi-stream-column)) flexi-io-stream (setq column nil))) (defmethod stream-read-byte :before ((stream flexi-io-stream)) (declare #.*standard-optimize-settings*) (reset-output-state stream)) (defmethod stream-read-char :before ((stream flexi-io-stream)) (declare #.*standard-optimize-settings*) (reset-output-state stream)) (defmethod stream-read-sequence :before ((stream flexi-io-stream) sequence start end &key) (declare #.*standard-optimize-settings*) (declare (ignore sequence start end)) (reset-output-state stream)) (defmethod stream-unread-char :before ((stream flexi-io-stream) char) (declare #.*standard-optimize-settings*) (declare (ignore char)) (reset-output-state stream)) (defmethod unread-byte :before (byte (stream flexi-io-stream)) (declare #.*standard-optimize-settings*) (declare (ignore byte)) (reset-output-state stream)) (defmethod stream-clear-input :before ((stream flexi-io-stream)) (declare #.*standard-optimize-settings*) (reset-output-state stream)) (defmethod write-byte* :after (byte (stream flexi-io-stream)) "Keep POSITION slot up to date even when performing output." (declare #.*standard-optimize-settings*) (declare (ignore byte)) (with-accessors ((position flexi-stream-position)) stream (incf position))) flexi-streams-1.0.7/iso-8859.lisp 0000664 0000000 0000000 00000040753 11175756704 0016426 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/iso-8859.lisp,v 1.7 2008/05/18 21:32:15 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) ;;; the following code was auto-generated from files which can be ;;; found at (defconstant +iso-8859-tables+ `((:iso-8859-1 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255))) (:iso-8859-2 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 260 728 321 164 317 346 167 168 352 350 356 377 173 381 379 176 261 731 322 180 318 347 711 184 353 351 357 378 733 382 380 340 193 194 258 196 313 262 199 268 201 280 203 282 205 206 270 272 323 327 211 212 336 214 215 344 366 218 368 220 221 354 223 341 225 226 259 228 314 263 231 269 233 281 235 283 237 238 271 273 324 328 243 244 337 246 247 345 367 250 369 252 253 355 729))) (:iso-8859-3 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 294 728 163 164 65533 292 167 168 304 350 286 308 173 65533 379 176 295 178 179 180 181 293 183 184 305 351 287 309 189 65533 380 192 193 194 65533 196 266 264 199 200 201 202 203 204 205 206 207 65533 209 210 211 212 288 214 215 284 217 218 219 220 364 348 223 224 225 226 65533 228 267 265 231 232 233 234 235 236 237 238 239 65533 241 242 243 244 289 246 247 285 249 250 251 252 365 349 729))) (:iso-8859-4 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 260 312 342 164 296 315 167 168 352 274 290 358 173 381 175 176 261 731 343 180 297 316 711 184 353 275 291 359 330 382 331 256 193 194 195 196 197 198 302 268 201 280 203 278 205 206 298 272 325 332 310 212 213 214 215 216 370 218 219 220 360 362 223 257 225 226 227 228 229 230 303 269 233 281 235 279 237 238 299 273 326 333 311 244 245 246 247 248 371 250 251 252 361 363 729))) (:iso-8859-5 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 173 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 8470 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 167 1118 1119))) (:iso-8859-6 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 65533 65533 65533 164 65533 65533 65533 65533 65533 65533 65533 1548 173 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 1563 65533 65533 65533 1567 65533 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 65533 65533 65533 65533 65533 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533))) (:iso-8859-7 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 8216 8217 163 8364 8367 166 167 168 169 890 171 172 173 65533 8213 176 177 178 179 900 901 902 183 904 905 906 187 908 189 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 65533 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 65533))) (:iso-8859-8 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 65533 162 163 164 165 166 167 168 169 215 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 247 187 188 189 190 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 65533 8215 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 65533 65533 8206 8207 65533))) (:iso-8859-9 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 286 209 210 211 212 213 214 215 216 217 218 219 220 304 350 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 287 241 242 243 244 245 246 247 248 249 250 251 252 305 351 255))) (:iso-8859-10 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 260 274 290 298 296 310 167 315 272 352 358 381 173 362 330 176 261 275 291 299 297 311 183 316 273 353 359 382 8213 363 331 256 193 194 195 196 197 198 302 268 201 280 203 278 205 206 207 208 325 332 211 212 213 214 360 216 370 218 219 220 221 222 223 257 225 226 227 228 229 230 303 269 233 281 235 279 237 238 239 240 326 333 243 244 245 246 361 248 371 250 251 252 253 254 312))) (:iso-8859-11 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 3585 3586 3587 3588 3589 3590 3591 3592 3593 3594 3595 3596 3597 3598 3599 3600 3601 3602 3603 3604 3605 3606 3607 3608 3609 3610 3611 3612 3613 3614 3615 3616 3617 3618 3619 3620 3621 3622 3623 3624 3625 3626 3627 3628 3629 3630 3631 3632 3633 3634 3635 3636 3637 3638 3639 3640 3641 3642 65533 65533 65533 65533 3647 3648 3649 3650 3651 3652 3653 3654 3655 3656 3657 3658 3659 3660 3661 3662 3663 3664 3665 3666 3667 3668 3669 3670 3671 3672 3673 3674 3675 65533 65533 65533 65533))) (:iso-8859-13 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 8221 162 163 164 8222 166 167 216 169 342 171 172 173 174 198 176 177 178 179 8220 181 182 183 248 185 343 187 188 189 190 230 260 302 256 262 196 197 280 274 268 201 377 278 290 310 298 315 352 323 325 211 332 213 214 215 370 321 346 362 220 379 381 223 261 303 257 263 228 229 281 275 269 233 378 279 291 311 299 316 353 324 326 243 333 245 246 247 371 322 347 363 252 380 382 8217))) (:iso-8859-14 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 7682 7683 163 266 267 7690 167 7808 169 7810 7691 7922 173 174 376 7710 7711 288 289 7744 7745 182 7766 7809 7767 7811 7776 7923 7812 7813 7777 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 372 209 210 211 212 213 214 7786 216 217 218 219 220 221 374 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 373 241 242 243 244 245 246 7787 248 249 250 251 252 253 375 255))) (:iso-8859-15 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 8364 165 352 167 353 169 170 171 172 173 174 175 176 177 178 179 381 181 182 183 382 185 186 187 338 339 376 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255))) (:iso-8859-16 . ,(make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 260 261 321 8364 8222 352 167 353 169 536 171 377 173 378 379 176 177 268 322 381 8221 182 183 382 269 537 187 338 339 376 380 192 193 194 258 196 262 198 199 200 201 202 203 204 205 206 207 272 323 210 211 212 336 214 346 368 217 218 219 220 280 538 223 224 225 226 259 228 263 230 231 232 233 234 235 236 237 238 239 273 324 242 243 244 337 246 347 369 249 250 251 252 281 539 255)))) "A list of the ISO-8859 encodings where each element is a cons with the car being a keyword denoting the encoding and the cdr being a vector enumerating the corresponding character codes.") flexi-streams-1.0.7/koi8-r.lisp 0000664 0000000 0000000 00000005431 11175756704 0016324 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/koi8-r.lisp,v 1.2 2008/05/18 21:32:15 edi Exp $ ;;; Copyright (c) 2006, Igor Plekhov. All rights reserved. ;;; Copyright (c) 2006-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) ;; http://unicode.org/Public/MAPPINGS/VENDORS/MISC/KOI8-R.TXT (defconstant +koi8-r-table+ (make-decoding-table '(0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 9472 9474 9484 9488 9492 9496 9500 9508 9516 9524 9532 9600 9604 9608 9612 9616 9617 9618 9619 8992 9632 8729 8730 8776 8804 8805 160 8993 176 178 183 247 9552 9553 9554 1105 9555 9556 9557 9558 9559 9560 9561 9562 9563 9564 9565 9566 9567 9568 9569 1025 9570 9571 9572 9573 9574 9575 9576 9577 9578 9579 9580 169 1102 1072 1073 1094 1076 1077 1092 1075 1093 1080 1081 1082 1083 1084 1085 1086 1087 1103 1088 1089 1090 1091 1078 1074 1100 1099 1079 1096 1101 1097 1095 1098 1070 1040 1041 1062 1044 1045 1060 1043 1061 1048 1049 1050 1051 1052 1053 1054 1055 1071 1056 1057 1058 1059 1046 1042 1068 1067 1047 1064 1069 1065 1063 1066)) "An array enumerating the character codes for the KOI8-R encoding.") flexi-streams-1.0.7/length.lisp 0000775 0000000 0000000 00000042134 11175756704 0016500 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/length.lisp,v 1.6 2008/05/29 10:25:14 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) (defgeneric encoding-factor (format) (:documentation "Given an external format FORMAT, returns a factor which denotes the octets to characters ratio to expect when encoding/decoding. If the returned value is an integer, the factor is assumed to be exact. If it is a \(double) float, the factor is supposed to be based on heuristics and usually not exact. This factor is used in string.lisp.") (declare #.*standard-optimize-settings*)) (defmethod encoding-factor ((format flexi-8-bit-format)) (declare #.*standard-optimize-settings*) ;; 8-bit encodings map octets to characters in an exact one-to-one ;; fashion 1) (defmethod encoding-factor ((format flexi-utf-8-format)) (declare #.*standard-optimize-settings*) ;; UTF-8 characters can be anything from one to six octets, but we ;; assume that the "overhead" is only about 5 percent - this ;; estimate is obviously very much dependant on the content 1.05d0) (defmethod encoding-factor ((format flexi-utf-16-format)) (declare #.*standard-optimize-settings*) ;; usually one character maps to two octets, but characters with ;; code points above #x10000 map to four octets - we assume that we ;; usually don't see these characters but of course have to return a ;; float 2.0d0) (defmethod encoding-factor ((format flexi-utf-32-format)) (declare #.*standard-optimize-settings*) ;; UTF-32 always matches every character to four octets 4) (defmethod encoding-factor ((format flexi-crlf-mixin)) (declare #.*standard-optimize-settings*) ;; if the sequence #\Return #\Linefeed is the line-end marker, this ;; obviously makes encodings potentially longer and definitely makes ;; the estimate unexact (* 1.02d0 (call-next-method))) (defgeneric check-end (format start end i) (declare #.*fixnum-optimize-settings*) (:documentation "Helper function used below to determine if we tried to read past the end of the sequence.") (:method (format start end i) (declare #.*fixnum-optimize-settings*) (declare (ignore start)) (declare (fixnum end i)) (when (> i end) (signal-encoding-error format "This sequence can't be decoded ~ using ~A as it is too short. ~A octet~:P missing at then end." (external-format-name format) (- i end)))) (:method ((format flexi-utf-16-format) start end i) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end i)) (declare (ignore i)) ;; don't warn twice (when (evenp (- end start)) (call-next-method)))) (defgeneric compute-number-of-chars (format sequence start end) (declare #.*standard-optimize-settings*) (:documentation "Computes the exact number of characters required to decode the sequence of octets in SEQUENCE from START to END using the external format FORMAT.")) (defmethod compute-number-of-chars :around (format (list list) start end) (declare #.*standard-optimize-settings*) (call-next-method format (coerce list 'vector) start end)) (defmethod compute-number-of-chars ((format flexi-8-bit-format) sequence start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end)) (declare (ignore sequence)) (- end start)) (defmethod compute-number-of-chars ((format flexi-crlf-mixin) sequence start end) ;; this method only applies to the 8-bit formats as all other ;; formats with CRLF line endings have their own specialized methods ;; below (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (vector sequence)) (let ((i start) (length (- end start))) (declare (fixnum i length)) (loop (when (>= i end) (return)) (let ((position (search #.(vector +cr+ +lf+) sequence :start2 i :end2 end :test #'=))) (unless position (return)) (setq i (1+ position)) (decf length))) length)) (defmethod compute-number-of-chars ((format flexi-utf-8-format) sequence start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (vector sequence)) (let ((sum 0) (i start)) (declare (fixnum i sum)) (loop (when (>= i end) (return)) (let* ((octet (aref sequence i)) ;; note that there are no validity checks here (length (cond ((not (logbitp 7 octet)) 1) ((= #b11000000 (logand* octet #b11100000)) 2) ((= #b11100000 (logand* octet #b11110000)) 3) (t 4)))) (declare (fixnum length) (type octet octet)) (incf sum) (incf i length))) (check-end format start end i) sum)) (defmethod compute-number-of-chars ((format flexi-crlf-utf-8-format) sequence start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (vector sequence)) (let ((sum 0) (i start) (last-octet 0)) (declare (fixnum i sum) (type octet last-octet)) (loop (when (>= i end) (return)) (let* ((octet (aref sequence i)) ;; note that there are no validity checks here (length (cond ((not (logbitp 7 octet)) 1) ((= #b11000000 (logand* octet #b11100000)) 2) ((= #b11100000 (logand* octet #b11110000)) 3) (t 4)))) (declare (fixnum length) (type octet octet)) (unless (and (= octet +lf+) (= last-octet +cr+)) (incf sum)) (incf i length) (setq last-octet octet))) (check-end format start end i) sum)) (defmethod compute-number-of-chars :before ((format flexi-utf-16-format) sequence start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (vector sequence)) (declare (ignore sequence)) (when (oddp (- end start)) (signal-encoding-error format "~A octet~:P cannot be decoded ~ using UTF-16 as ~:*~A is not even." (- end start)))) (defmethod compute-number-of-chars ((format flexi-utf-16-le-format) sequence start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end)) (let ((sum 0) (i start)) (declare (fixnum i sum)) (decf end 2) (loop (when (> i end) (return)) (let* ((high-octet (aref sequence (1+ i))) (length (cond ((<= #xd8 high-octet #xdf) 4) (t 2)))) (declare (fixnum length) (type octet high-octet)) (incf sum) (incf i length))) (check-end format start (+ end 2) i) sum)) (defmethod compute-number-of-chars ((format flexi-utf-16-be-format) sequence start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (vector sequence)) (let ((sum 0) (i start)) (declare (fixnum i sum)) (decf end 2) (loop (when (> i end) (return)) (let* ((high-octet (aref sequence i)) (length (cond ((<= #xd8 high-octet #xdf) 4) (t 2)))) (declare (fixnum length) (type octet high-octet)) (incf sum) (incf i length))) (check-end format start (+ end 2) i) sum)) (defmethod compute-number-of-chars ((format flexi-crlf-utf-16-le-format) sequence start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (vector sequence)) (let ((sum 0) (i start) (last-octet 0)) (declare (fixnum i sum) (type octet last-octet)) (decf end 2) (loop (when (> i end) (return)) (let* ((high-octet (aref sequence (1+ i))) (length (cond ((<= #xd8 high-octet #xdf) 4) (t 2)))) (declare (fixnum length) (type octet high-octet)) (unless (and (zerop high-octet) (= (the octet (aref sequence i)) +lf+) (= last-octet +cr+)) (incf sum)) (setq last-octet (if (zerop high-octet) (aref sequence i) 0)) (incf i length))) (check-end format start (+ end 2) i) sum)) (defmethod compute-number-of-chars ((format flexi-crlf-utf-16-be-format) sequence start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (vector sequence)) (let ((sum 0) (i start) (last-octet 0)) (declare (fixnum i sum) (type octet last-octet)) (decf end 2) (loop (when (> i end) (return)) (let* ((high-octet (aref sequence i)) (length (cond ((<= #xd8 high-octet #xdf) 4) (t 2)))) (declare (fixnum length) (type octet high-octet)) (unless (and (zerop high-octet) (= (the octet (aref sequence (1+ i))) +lf+) (= last-octet +cr+)) (incf sum)) (setq last-octet (if (zerop high-octet) (aref sequence (1+ i)) 0)) (incf i length))) (check-end format start (+ end 2) i) sum)) (defmethod compute-number-of-chars :before ((format flexi-utf-32-format) sequence start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end)) (declare (ignore sequence)) (let ((length (- end start))) (when (plusp (mod length 4)) (signal-encoding-error format "~A octet~:P cannot be decoded ~ using UTF-32 as ~:*~A is not a multiple-value of four." length)))) (defmethod compute-number-of-chars ((format flexi-utf-32-format) sequence start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end)) (declare (ignore sequence)) (ceiling (- end start) 4)) (defmethod compute-number-of-chars ((format flexi-crlf-utf-32-le-format) sequence start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (vector sequence)) (let ((i start) (length (ceiling (- end start) 4))) (decf end 8) (loop (when (> i end) (return)) (cond ((loop for j of-type fixnum from i for octet across #.(vector +cr+ 0 0 0 +lf+ 0 0 0) always (= octet (aref sequence j))) (decf length) (incf i 8)) (t (incf i 4)))) length)) (defmethod compute-number-of-chars ((format flexi-crlf-utf-32-be-format) sequence start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (vector sequence)) (let ((i start) (length (ceiling (- end start) 4))) (decf end 8) (loop (when (> i end) (return)) (cond ((loop for j of-type fixnum from i for octet across #.(vector 0 0 0 +cr+ 0 0 0 +lf+) always (= octet (aref sequence j))) (decf length) (incf i 8)) (t (incf i 4)))) length)) (defgeneric compute-number-of-octets (format sequence start end) (declare #.*standard-optimize-settings*) (:documentation "Computes the exact number of octets required to encode the sequence of characters in SEQUENCE from START to END using the external format FORMAT.")) (defmethod compute-number-of-octets :around (format (list list) start end) (declare #.*standard-optimize-settings*) (call-next-method format (coerce list 'string*) start end)) (defmethod compute-number-of-octets ((format flexi-8-bit-format) string start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end)) (declare (ignore string)) (- end start)) (defmethod compute-number-of-octets ((format flexi-utf-8-format) string start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (string string)) (let ((sum 0) (i start)) (declare (fixnum i sum)) (loop (when (>= i end) (return)) (let* ((char-code (char-code (char string i))) (char-length (cond ((< char-code #x80) 1) ((< char-code #x800) 2) ((< char-code #x10000) 3) (t 4)))) (declare (fixnum char-length) (type char-code-integer char-code)) (incf sum char-length) (incf i))) sum)) (defmethod compute-number-of-octets ((format flexi-crlf-utf-8-format) string start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (string string)) (let ((sum 0) (i start)) (declare (fixnum i sum)) (loop (when (>= i end) (return)) (let* ((char-code (char-code (char string i))) (char-length (cond ((= char-code #.(char-code #\Newline)) 2) ((< char-code #x80) 1) ((< char-code #x800) 2) ((< char-code #x10000) 3) (t 4)))) (declare (fixnum char-length) (type char-code-integer char-code)) (incf sum char-length) (incf i))) sum)) (defmethod compute-number-of-octets ((format flexi-utf-16-format) string start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (string string)) (let ((sum 0) (i start)) (declare (fixnum i sum)) (loop (when (>= i end) (return)) (let* ((char-code (char-code (char string i))) (char-length (cond ((< char-code #x10000) 2) (t 4)))) (declare (fixnum char-length) (type char-code-integer char-code)) (incf sum char-length) (incf i))) sum)) (defmethod compute-number-of-octets ((format flexi-crlf-utf-16-le-format) string start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (string string)) (let ((sum 0) (i start)) (declare (fixnum i sum)) (loop (when (>= i end) (return)) (let* ((char-code (char-code (char string i))) (char-length (cond ((= char-code #.(char-code #\Newline)) 4) ((< char-code #x10000) 2) (t 4)))) (declare (fixnum char-length) (type char-code-integer char-code)) (incf sum char-length) (incf i))) sum)) (defmethod compute-number-of-octets ((format flexi-crlf-utf-16-be-format) string start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (string string)) (let ((sum 0) (i start)) (declare (fixnum i sum)) (loop (when (>= i end) (return)) (let* ((char-code (char-code (char string i))) (char-length (cond ((= char-code #.(char-code #\Newline)) 4) ((< char-code #x10000) 2) (t 4)))) (declare (fixnum char-length) (type char-code-integer char-code)) (incf sum char-length) (incf i))) sum)) (defmethod compute-number-of-octets ((format flexi-utf-32-format) string start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end)) (declare (ignore string)) (* 4 (- end start))) (defmethod compute-number-of-octets ((format flexi-crlf-mixin) string start end) (declare #.*fixnum-optimize-settings*) (declare (fixnum start end) (string string)) (+ (call-next-method) (* (case (external-format-name format) (:utf-32 4) (otherwise 1)) (count #\Newline string :start start :end end :test #'char=)))) (defgeneric character-length (format char) (declare #.*fixnum-optimize-settings*) (:documentation "Returns the number of octets needed to encode the single character CHAR.") (:method (format char) (compute-number-of-octets format (string char) 0 1))) (defmethod character-length :around ((format flexi-crlf-mixin) (char (eql #\Newline))) (declare #.*fixnum-optimize-settings*) (+ (call-next-method format +cr+) (call-next-method format +lf+))) (defmethod character-length ((format flexi-8-bit-format) char) (declare #.*fixnum-optimize-settings*) (declare (ignore char)) 1) (defmethod character-length ((format flexi-utf-32-format) char) (declare #.*fixnum-optimize-settings*) (declare (ignore char)) 4) flexi-streams-1.0.7/lw-char-stream.lisp 0000775 0000000 0000000 00000007104 11175756704 0020043 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/lw-char-stream.lisp,v 1.1 2008/05/23 14:43:09 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) (defclass flexi-char-output-stream (flexi-output-stream) () (:documentation "This class is for output streams where the underlying stream is bivalent but not binary. It exists solely for the purpose of optimizing output to binary streams on LispWorks. See WRITE-BYTE*.")) (defclass flexi-char-input-stream (flexi-input-stream) () (:documentation "This class is for input streams where the underlying stream is bivalent but not binary. It exists solely for the purpose of optimizing input to binary streams on LispWorks. See READ-BYTE*.")) (defclass flexi-char-io-stream (flexi-char-input-stream flexi-char-output-stream flexi-io-stream) () (:documentation "This class is for bidirectional streams where the underlying stream is bivalent but not binary. It exists solely for the purpose of optimizing input and output from/to binary streams on LispWorks. See READ-BYTE* and WRITE-BYTE*.")) (defmethod initialize-instance :after ((flexi-stream flexi-output-stream) &rest initargs) "Might change the class of FLEXI-STREAM for optimization purposes. Only needed for LispWorks." (declare #.*standard-optimize-settings*) (declare (ignore initargs)) (with-accessors ((stream flexi-stream-stream)) flexi-stream (unless (subtypep (stream-element-type stream) 'octet) (change-class flexi-stream (typecase flexi-stream (flexi-io-stream 'flexi-char-io-stream) (otherwise 'flexi-char-output-stream)))))) (defmethod initialize-instance :after ((flexi-stream flexi-input-stream) &rest initargs) "Might change the class of FLEXI-STREAM for optimization purposes. Only needed for LispWorks." (declare #.*standard-optimize-settings*) (declare (ignore initargs)) (with-accessors ((stream flexi-stream-stream)) flexi-stream (unless (subtypep (stream-element-type stream) 'octet) (change-class flexi-stream (typecase flexi-stream (flexi-io-stream 'flexi-char-io-stream) (otherwise 'flexi-char-input-stream)))))) flexi-streams-1.0.7/mapping.lisp 0000775 0000000 0000000 00000006111 11175756704 0016645 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/mapping.lisp,v 1.3 2008/05/25 19:07:53 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) (deftype octet () "A shortcut for \(UNSIGNED-BYTE 8)." '(unsigned-byte 8)) (deftype char* () "Convenience shortcut to paper over the difference between LispWorks and the other Lisps." #+:lispworks 'lw:simple-char #-:lispworks 'character) (deftype string* () "Convenience shortcut to paper over the difference between LispWorks and the other Lisps." #+:lispworks 'lw:text-string #-:lispworks 'string) (deftype char-code-integer () "The subtype of integers which can be returned by the function CHAR-CODE." '(integer 0 #.(1- char-code-limit))) (deftype code-point () "The subtype of integers that's just big enough to hold all Unicode codepoints. See for example ." '(mod #x110000)) (defmacro defconstant (name value &optional doc) "Make sure VALUE is evaluated only once \(to appease SBCL)." `(cl:defconstant ,name (if (boundp ',name) (symbol-value ',name) ,value) ,@(when doc (list doc)))) (defun invert-table (table) "`Inverts' an array which maps octets to character codes to a hash table which maps character codes to octets." (let ((hash (make-hash-table))) (loop for octet from 0 for char-code across table unless (= char-code 65533) do (setf (gethash char-code hash) octet)) hash)) (defun make-decoding-table (list) "Creates and returns an array which contains the elements in the list LIST and has an element type that's suitable for character codes." (make-array (length list) :element-type 'char-code-integer :initial-contents list)) flexi-streams-1.0.7/output.lisp 0000664 0000000 0000000 00000015572 11175756704 0016562 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/output.lisp,v 1.65 2008/05/24 23:15:25 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) (defgeneric write-byte* (byte stream) (declare #.*standard-optimize-settings*) (:documentation "Writes one byte \(octet) to the underlying stream STREAM.")) #-:lispworks (defmethod write-byte* (byte (flexi-output-stream flexi-output-stream)) (declare #.*standard-optimize-settings*) (with-accessors ((stream flexi-stream-stream)) flexi-output-stream (write-byte byte stream))) #+:lispworks (defmethod write-byte* (byte (flexi-output-stream flexi-output-stream)) (declare #.*standard-optimize-settings*) (with-accessors ((stream flexi-stream-stream)) flexi-output-stream (write-byte byte stream))) #+:lispworks (defmethod write-byte* (byte (flexi-output-stream flexi-char-output-stream)) "This method is only used for LispWorks bivalent streams which aren't binary." (declare #.*standard-optimize-settings*) ;; we use WRITE-SEQUENCE because WRITE-BYTE doesn't work with all ;; bivalent streams in LispWorks (4.4.6) (with-accessors ((stream flexi-stream-stream)) flexi-output-stream (write-sequence (make-array 1 :element-type 'octet :initial-element byte) stream) byte)) (defmethod stream-write-char ((stream flexi-output-stream) char) (declare #.*standard-optimize-settings*) (with-accessors ((external-format flexi-stream-external-format)) stream (flet ((writer (octet) (write-byte* octet stream))) (declare (dynamic-extent (function writer))) (char-to-octets external-format char #'writer)))) (defmethod stream-write-char :after ((stream flexi-output-stream) char) (declare #.*standard-optimize-settings*) ;; update the column unless we're in the middle of the line and ;; the current value is NIL (with-accessors ((column flexi-stream-column)) stream (cond ((char= char #\Newline) (setq column 0)) (column (incf (the integer column)))))) (defmethod stream-clear-output ((flexi-output-stream flexi-output-stream)) "Simply calls the corresponding method for the underlying output stream." (declare #.*standard-optimize-settings*) (with-accessors ((stream flexi-stream-stream)) flexi-output-stream (clear-output stream))) (defmethod stream-finish-output ((flexi-output-stream flexi-output-stream)) "Simply calls the corresponding method for the underlying output stream." (declare #.*standard-optimize-settings*) (with-accessors ((stream flexi-stream-stream)) flexi-output-stream (finish-output stream))) (defmethod stream-force-output ((flexi-output-stream flexi-output-stream)) "Simply calls the corresponding method for the underlying output stream." (declare #.*standard-optimize-settings*) (with-accessors ((stream flexi-stream-stream)) flexi-output-stream (force-output stream))) (defmethod stream-line-column ((flexi-output-stream flexi-output-stream)) "Returns the column stored in the COLUMN slot of the FLEXI-OUTPUT-STREAM object STREAM." (declare #.*standard-optimize-settings*) (with-accessors ((column flexi-stream-column)) flexi-output-stream column)) (defmethod stream-write-byte ((flexi-output-stream flexi-output-stream) byte) "Writes a byte \(octet) to the underlying stream." (declare #.*standard-optimize-settings*) (with-accessors ((column flexi-stream-column)) flexi-output-stream ;; set column to NIL because we don't know how to handle binary ;; output mixed with character output (setq column nil) (write-byte* byte flexi-output-stream))) #+:allegro (defmethod stream-terpri ((stream flexi-output-stream)) "Writes a #\Newline character to the underlying stream." (declare #.*standard-optimize-settings*) ;; needed for AllegroCL - grrr... (stream-write-char stream #\Newline)) (defmethod stream-write-sequence ((flexi-output-stream flexi-output-stream) sequence start end &key) "An optimized version which uses a buffer underneath. The function can accepts characters as well as octets and it decides what to do based on the element type of the sequence \(if possible) or on the individual elements, i.e. you can mix characters and octets in SEQUENCE if you want. Whether that really works might also depend on your Lisp, some of the implementations are more picky than others." (declare #.*standard-optimize-settings*) (declare (fixnum start end)) (with-accessors ((column flexi-stream-column) (external-format flexi-stream-external-format) (stream flexi-stream-stream)) flexi-output-stream (when (>= start end) (return-from stream-write-sequence sequence)) (when (and (vectorp sequence) (subtypep (array-element-type sequence) 'integer)) ;; if this is pure binary output, just send all the stuff to the ;; underlying stream directly and skip the rest (setq column nil) (return-from stream-write-sequence (write-sequence sequence stream :start start :end end))) ;; otherwise hand over to the external format to do the work (write-sequence* external-format flexi-output-stream sequence start end)) sequence) (defmethod stream-write-string ((stream flexi-output-stream) string &optional (start 0) (end (length string))) "Simply hands over to the optimized method for STREAM-WRITE-SEQUENCE." (declare #.*standard-optimize-settings*) (stream-write-sequence stream string start (or end (length string)))) flexi-streams-1.0.7/packages.lisp 0000664 0000000 0000000 00000007061 11175756704 0016772 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: CL-USER; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/packages.lisp,v 1.39 2008/05/30 07:50:31 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :cl-user) (unless (find-symbol (symbol-name :stream-file-position) :trivial-gray-streams) (error "You need a newer version of TRIVIAL-GRAY-STREAMS.")) (defpackage :flexi-streams (:use :cl :trivial-gray-streams) (:nicknames :flex) (:shadow #+:lispworks :with-accessors :defconstant) (:export :*default-eol-style* :*default-little-endian* :*substitution-char* :accept-overlong-sequence :char-length :external-format-condition :external-format-condition-external-format :external-format-eol-style :external-format-error :external-format-encoding-error :external-format-equal :external-format-id :external-format-little-endian :external-format-name :flexi-input-stream :flexi-output-stream :flexi-io-stream :flexi-stream :flexi-stream-bound :flexi-stream-column :flexi-stream-external-format :flexi-stream-element-type :flexi-stream-element-type-error :flexi-stream-element-type-error-element-type :flexi-stream-error :flexi-stream-out-of-sync-error :flexi-stream-position :flexi-stream-stream :get-output-stream-sequence :in-memory-stream :in-memory-stream-closed-error :in-memory-stream-error :in-memory-stream-position-spec-error :in-memory-stream-position-spec-error-position-spec :in-memory-input-stream :in-memory-output-stream :list-stream :make-external-format :make-in-memory-input-stream :make-in-memory-output-stream :make-flexi-stream :octet :octet-length :octets-to-string :output-stream-sequence-length :peek-byte :string-to-octets :unread-byte :vector-stream :with-input-from-sequence :with-output-to-sequence)) flexi-streams-1.0.7/specials.lisp 0000664 0000000 0000000 00000016036 11175756704 0017021 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/specials.lisp,v 1.33 2008/05/25 01:40:54 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) (defvar *standard-optimize-settings* '(optimize speed (safety 0) (space 0) (debug 1) (compilation-speed 0)) "The standard optimize settings used by most declaration expressions.") (defvar *fixnum-optimize-settings* '(optimize speed (safety 0) (space 0) (debug 1) (compilation-speed 0) #+:lispworks (hcl:fixnum-safety 0)) "Like *STANDARD-OPTIMIZE-SETTINGS*, but \(on LispWorks) with all arithmetic being fixnum arithmetic.") (defconstant +lf+ (char-code #\Linefeed)) (defconstant +cr+ (char-code #\Return)) (defvar *current-unreader* nil "A unary function which might be called to `unread' a character \(i.e. the sequence of octets it represents). Used by the function OCTETS-TO-CHAR-CODE and must always be bound to a suitable functional object when this function is called.") (defvar +name-map+ '((:utf8 . :utf-8) (:utf16 . :utf-16) (:ucs2 . :utf-16) (:ucs-2 . :utf-16) (:unicode . :utf-16) (:utf32 . :utf-32) (:ucs4 . :utf-32) (:ucs-4 . :utf-32) (:ascii . :us-ascii) (:koi8r . :koi8-r) (:latin-1 . :iso-8859-1) (:latin1 . :iso-8859-1) (:latin-2 . :iso-8859-2) (:latin2 . :iso-8859-2) (:latin-3 . :iso-8859-3) (:latin3 . :iso-8859-3) (:latin-4 . :iso-8859-4) (:latin4 . :iso-8859-4) (:cyrillic . :iso-8859-5) (:arabic . :iso-8859-6) (:greek . :iso-8859-7) (:hebrew . :iso-8859-8) (:latin-5 . :iso-8859-9) (:latin5 . :iso-8859-9) (:latin-6 . :iso-8859-10) (:latin6 . :iso-8859-10) (:thai . :iso-8859-11) (:latin-7 . :iso-8859-13) (:latin7 . :iso-8859-13) (:latin-8 . :iso-8859-14) (:latin8 . :iso-8859-14) (:latin-9 . :iso-8859-15) (:latin9 . :iso-8859-15) (:latin-0 . :iso-8859-15) (:latin0 . :iso-8859-15) (:latin-10 . :iso-8859-16) (:latin10 . :iso-8859-16) (:codepage . :code-page) #+(and :lispworks :win32) (win32:code-page . :code-page)) "An alist which mapes alternative names for external formats to their canonical counterparts.") (defvar +shortcut-map+ '((:ucs-2le . (:ucs-2 :little-endian t)) (:ucs-2be . (:ucs-2 :little-endian nil)) (:ucs-4le . (:ucs-4 :little-endian t)) (:ucs-4be . (:ucs-4 :little-endian nil)) (:utf-16le . (:utf-16 :little-endian t)) (:utf-16be . (:utf-16 :little-endian nil)) (:utf-32le . (:utf-32 :little-endian t)) (:utf-32be . (:utf-32 :little-endian nil)) (:ibm437 . (:code-page :id 437)) (:ibm850 . (:code-page :id 850)) (:ibm852 . (:code-page :id 852)) (:ibm855 . (:code-page :id 855)) (:ibm857 . (:code-page :id 857)) (:ibm860 . (:code-page :id 860)) (:ibm861 . (:code-page :id 861)) (:ibm862 . (:code-page :id 862)) (:ibm863 . (:code-page :id 863)) (:ibm864 . (:code-page :id 864)) (:ibm865 . (:code-page :id 865)) (:ibm866 . (:code-page :id 866)) (:ibm869 . (:code-page :id 869)) (:windows-1250 . (:code-page :id 1250)) (:windows-1251 . (:code-page :id 1251)) (:windows-1252 . (:code-page :id 1252)) (:windows-1253 . (:code-page :id 1253)) (:windows-1254 . (:code-page :id 1254)) (:windows-1255 . (:code-page :id 1255)) (:windows-1256 . (:code-page :id 1256)) (:windows-1257 . (:code-page :id 1257)) (:windows-1258 . (:code-page :id 1258))) "An alist which maps shortcuts for external formats to their long forms.") (defvar *default-eol-style* #+:win32 :crlf #-:win32 :lf "The end-of-line style used by external formats if none is explicitly given. Depends on the OS the code is compiled on.") (defvar *default-little-endian* #+:little-endian t #-:little-endian nil "Whether external formats are little-endian by default \(i.e. unless explicitly specified). Depends on the platform the code is compiled on.") (defvar *substitution-char* nil "If this value is not NIL, it should be a character which is used \(as if by a USE-VALUE restart) whenever during reading an error of type FLEXI-STREAM-ENCODING-ERROR would have been signalled otherwise.") (defconstant +iso-8859-hashes+ (loop for (name . table) in +iso-8859-tables+ collect (cons name (invert-table table))) "An alist which maps names for ISO-8859 encodings to hash tables which map character codes to the corresponding octets.") (defconstant +code-page-hashes+ (loop for (id . table) in +code-page-tables+ collect (cons id (invert-table table))) "An alist which maps IDs of Windows code pages to hash tables which map character codes to the corresponding octets.") (defconstant +ascii-hash+ (invert-table +ascii-table+) "A hash table which maps US-ASCII character codes to the corresponding octets.") (defconstant +koi8-r-hash+ (invert-table +koi8-r-table+) "A hash table which maps KOI8-R character codes to the corresponding octets.") (defconstant +buffer-size+ 8192 "Default size for buffers used for internal purposes.") (pushnew :flexi-streams *features*) ;; stuff for Nikodemus Siivola's HYPERDOC ;; see ;; and ;; also used by LW-ADD-ONS (defvar *hyperdoc-base-uri* "http://weitz.de/flexi-streams/") (let ((exported-symbols-alist (loop for symbol being the external-symbols of :flexi-streams collect (cons symbol (concatenate 'string "#" (string-downcase symbol)))))) (defun hyperdoc-lookup (symbol type) (declare (ignore type)) (cdr (assoc symbol exported-symbols-alist :test #'eq)))) flexi-streams-1.0.7/stream.lisp 0000664 0000000 0000000 00000024544 11175756704 0016514 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/stream.lisp,v 1.61 2008/05/19 22:32:56 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) (defclass flexi-stream (trivial-gray-stream-mixin) ((stream :initarg :stream :reader flexi-stream-stream :documentation "The actual stream that's used for input and/or output. It must be capable of reading/writing octets with READ-SEQUENCE and/or WRITE-SEQUENCE.") (external-format :initform (make-external-format :iso-8859-1) :initarg :flexi-stream-external-format :accessor flexi-stream-external-format :documentation "The encoding currently used by this stream. Can be changed on the fly.") (element-type :initform 'char* :initarg :element-type :accessor flexi-stream-element-type :documentation "The element type of this stream.")) (:documentation "A FLEXI-STREAM object is a stream that's `layered' atop an existing binary/bivalent stream in order to allow for multi-octet external formats. FLEXI-STREAM itself is a mixin and should not be instantiated.")) (defmethod initialize-instance :after ((flexi-stream flexi-stream) &rest initargs) "Makes sure the EXTERNAL-FORMAT and ELEMENT-TYPE slots contain reasonable values." (declare #.*standard-optimize-settings*) (declare (ignore initargs)) (with-accessors ((external-format flexi-stream-external-format) (element-type flexi-stream-element-type)) flexi-stream (unless (or (subtypep element-type 'character) (subtypep element-type 'octet)) (error 'flexi-stream-element-type-error :element-type element-type :stream flexi-stream)) (setq external-format (maybe-convert-external-format external-format)))) (defmethod (setf flexi-stream-external-format) :around (new-value (flexi-stream flexi-stream)) "Converts the new value to an EXTERNAL-FORMAT object if necessary." (declare #.*standard-optimize-settings*) (call-next-method (maybe-convert-external-format new-value) flexi-stream)) (defmethod (setf flexi-stream-element-type) :before (new-value (flexi-stream flexi-stream)) "Checks whether the new value makes sense before it is set." (declare #.*standard-optimize-settings*) (unless (or (subtypep new-value 'character) (type-equal new-value 'octet)) (error 'flexi-stream-element-type-error :element-type new-value :stream flexi-stream))) (defmethod stream-element-type ((stream flexi-stream)) "Returns the element type that was provided by the creator of the stream." (declare #.*standard-optimize-settings*) (with-accessors ((element-type flexi-stream-element-type)) stream element-type)) (defmethod close ((stream flexi-stream) &key abort) "Closes the flexi stream by closing the underlying `real' stream." (declare #.*standard-optimize-settings*) (with-accessors ((stream flexi-stream-stream)) stream (cond ((open-stream-p stream) (close stream :abort abort)) (t nil)))) (defmethod open-stream-p ((stream flexi-stream)) "A flexi stream is open if its underlying stream is open." (declare #.*standard-optimize-settings*) (with-accessors ((stream flexi-stream-stream)) stream (open-stream-p stream))) (defmethod stream-file-position ((stream flexi-stream)) "Dispatch to method for underlying stream." (declare #.*standard-optimize-settings*) (with-accessors ((stream flexi-stream-stream)) stream (stream-file-position stream))) (defmethod (setf stream-file-position) (position-spec (stream flexi-stream)) "Dispatch to method for underlying stream." (declare #.*standard-optimize-settings*) (with-accessors ((stream flexi-stream-stream)) stream (setf (stream-file-position stream) position-spec))) (defclass flexi-output-stream (flexi-stream fundamental-binary-output-stream fundamental-character-output-stream) ((column :initform 0 :accessor flexi-stream-column :documentation "The current output column. A non-negative integer or NIL.")) (:documentation "A FLEXI-OUTPUT-STREAM is a FLEXI-STREAM that can actually be instatiated and used for output. Don't use MAKE-INSTANCE to create a new FLEXI-OUTPUT-STREAM but use MAKE-FLEXI-STREAM instead.")) #+:cmu (defmethod input-stream-p ((stream flexi-output-stream)) "Explicitly states whether this is an input stream." (declare #.*standard-optimize-settings*) nil) (defclass flexi-input-stream (flexi-stream fundamental-binary-input-stream fundamental-character-input-stream) ((last-char-code :initform nil :accessor flexi-stream-last-char-code :documentation "This slot either holds NIL or the last character \(code) read successfully. This is mainly used for UNREAD-CHAR sanity checks.") (last-octet :initform nil :accessor flexi-stream-last-octet :documentation "This slot either holds NIL or the last octet read successfully from the stream using a `binary' operation such as READ-BYTE. This is mainly used for UNREAD-BYTE sanity checks.") (octet-stack :initform nil :accessor flexi-stream-octet-stack :documentation "A small buffer which holds octets that were already read from the underlying stream but not yet used to produce characters. This is mainly used if we have to look ahead for a CR/LF line ending.") (position :initform 0 :initarg :position :type integer :accessor flexi-stream-position :documentation "The position within the stream where each octet read counts as one.") (bound :initform nil :initarg :bound :type (or null integer) :accessor flexi-stream-bound :documentation "When this is not NIL, it must be an integer and the stream will behave as if no more data is available as soon as POSITION is greater or equal than this value.")) (:documentation "A FLEXI-INPUT-STREAM is a FLEXI-STREAM that can actually be instatiated and used for input. Don't use MAKE-INSTANCE to create a new FLEXI-INPUT-STREAM but use MAKE-FLEXI-STREAM instead.")) #+:cmu (defmethod output-stream-p ((stream flexi-input-stream)) "Explicitly states whether this is an output stream." (declare #.*standard-optimize-settings*) nil) (defclass flexi-io-stream (flexi-input-stream flexi-output-stream) () (:documentation "A FLEXI-IO-STREAM is a FLEXI-STREAM that can actually be instatiated and used for input and output. Don't use MAKE-INSTANCE to create a new FLEXI-IO-STREAM but use MAKE-FLEXI-STREAM instead.")) #+:cmu (defmethod input-stream-p ((stream flexi-io-stream)) "Explicitly states whether this is an input stream." (declare #.*standard-optimize-settings*) t) #+:cmu (defmethod output-stream-p ((stream flexi-io-stream)) "Explicitly states whether this is an output stream." (declare #.*standard-optimize-settings*) t) (defun make-flexi-stream (stream &rest args &key (external-format (make-external-format :iso-8859-1)) element-type column position bound) "Creates and returns a new flexi stream. STREAM must be an open binary or `bivalent' stream, i.e. it must be capable of reading/writing octets with READ-SEQUENCE and/or WRITE-SEQUENCE. The resulting flexi stream is an input stream if and only if STREAM is an input stream. Likewise, it's an output stream if and only if STREAM is an output stream. The default for ELEMENT-TYPE is LW:SIMPLE-CHAR on LispWorks and CHARACTER on other Lisps. EXTERNAL-FORMAT must be an EXTERNAL-FORMAT object or a symbol or a list denoting such an object. COLUMN is the initial column of the stream which is either a non-negative integer or NIL. The COLUMN argument must only be used for output streams. POSITION \(only used for input streams) should be an integer and it denotes the position the stream is in - it will be increased by one for each octet read. BOUND \(only used for input streams) should be NIL or an integer. If BOUND is not NIL and POSITION has gone beyond BOUND, then the stream will behave as if no more input is available." (declare #.*standard-optimize-settings*) ;; these arguments are ignored - they are only there to provide a ;; meaningful parameter list for IDEs (declare (ignore element-type column position bound)) (unless (and (streamp stream) (open-stream-p stream)) (error "~S should have been an open stream." stream)) (apply #'make-instance ;; actual type depends on STREAM (cond ((and (input-stream-p stream) (output-stream-p stream)) 'flexi-io-stream) ((input-stream-p stream) 'flexi-input-stream) ((output-stream-p stream) 'flexi-output-stream)) :stream stream :flexi-stream-external-format external-format (sans args :external-format))) flexi-streams-1.0.7/strings.lisp 0000664 0000000 0000000 00000007762 11175756704 0016715 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/strings.lisp,v 1.34 2008/05/26 10:55:08 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) (defun string-to-octets (string &key (external-format :latin1) (start 0) (end (length string))) "Converts the Lisp string STRING from START to END to an array of octets corresponding to the external format designated by EXTERNAL-FORMAT. In spite of the name, STRING can be any sequence of characters, but the function is optimized for strings." (declare #.*standard-optimize-settings*) (setq external-format (maybe-convert-external-format external-format)) ;; the external format knows how to do it... (string-to-octets* external-format string start end)) (defun octets-to-string (sequence &key (external-format :latin1) (start 0) (end (length sequence))) "Converts the Lisp sequence SEQUENCE of octets from START to END to a string using the external format designated by EXTERNAL-FORMAT. This function is optimized for the case of SEQUENCE being a vector. Don't use lists if you're in a hurry." (declare #.*standard-optimize-settings*) (declare (fixnum start end)) (setq external-format (maybe-convert-external-format external-format)) ;; the external format knows how to do it... (octets-to-string* external-format sequence start end)) (defun octet-length (string &key (external-format :latin1) (start 0) (end (length string))) "Returns the length of the substring of STRING from START to END in octets if encoded using the external format EXTERNAL-FORMAT. In spite of the name, STRING can be any sequence of characters, but the function is optimized for strings." (declare #.*standard-optimize-settings*) (declare (fixnum start end)) (setq external-format (maybe-convert-external-format external-format)) (compute-number-of-octets external-format string start end)) (defun char-length (sequence &key (external-format :latin1) (start 0) (end (length sequence))) "Kind of the inverse of OCTET-LENGTH. Returns the length of the subsequence \(of octets) of SEQUENCE from START to END in characters if decoded using the external format EXTERNAL-FORMAT. Note that this function doesn't check for the validity of the data in SEQUENCE. This function is optimized for the case of SEQUENCE being a vector. Don't use lists if you're in a hurry." (declare #.*standard-optimize-settings*) (declare (fixnum start end)) (setq external-format (maybe-convert-external-format external-format)) (compute-number-of-chars external-format sequence start end)) flexi-streams-1.0.7/test/ 0000775 0000000 0000000 00000000000 11175756704 0015276 5 ustar 00root root 0000000 0000000 flexi-streams-1.0.7/test/README 0000775 0000000 0000000 00000000414 11175756704 0016160 0 ustar 00root root 0000000 0000000 The reference files in this directory were created/converted using a mixture of GNU recode and the native internationalization facilities of LispWorks and AllegroCL, i.e. we're not testing FLEXI-STREAMS against files created by itself (which would be kind of useless). flexi-streams-1.0.7/test/hebrew_latin8_cr.txt 0000664 0000000 0000000 00000004465 11175756704 0021267 0 ustar 00root root 0000000 0000000 : 1 - 2 : - :- 3 - - 4 : 5 : - - 6 : - 7 :- - - 8 : - 9 :- 10 :- 11 - - :- 12 - - :- : - - 13 14 : - 15 :- - - 16 - : 17 :- 18 :- : - - 19 20 : - - - - 21 :- - 22 : - : - - 23 24 :- - - - 25 - :- 26 :- - - - 27 : 28 - :- - -- 29 - -- - - : - - - 30 -- - - :- - -- 31 : - flexi-streams-1.0.7/test/hebrew_latin8_crlf.txt 0000664 0000000 0000000 00000004571 11175756704 0021607 0 ustar 00root root 0000000 0000000 : 1 - 2 : - :- 3 - - 4 : 5 : - - 6 : - 7 :- - - 8 : - 9 :- 10 :- 11 - - :- 12 - - :- : - - 13 14 : - 15 :- - - 16 - : 17 :- 18 :- : - - 19 20 : - - - - 21 :- - 22 : - : - - 23 24 :- - - - 25 - :- 26 :- - - - 27 : 28 - :- - -- 29 - -- - - : - - - 30 -- - - :- - -- 31 : - flexi-streams-1.0.7/test/hebrew_latin8_lf.txt 0000664 0000000 0000000 00000004465 11175756704 0021264 0 ustar 00root root 0000000 0000000 : 1 - 2 : - :- 3 - - 4 : 5 : - - 6 : - 7 :- - - 8 : - 9 :- 10 :- 11 - - :- 12 - - :- : - - 13 14 : - 15 :- - - 16 - : 17 :- 18 :- : - - 19 20 : - - - - 21 :- - 22 : - : - - 23 24 :- - - - 25 - :- 26 :- - - - 27 : 28 - :- - -- 29 - -- - - : - - - 30 -- - - :- - -- 31 : - flexi-streams-1.0.7/test/hebrew_utf8_cr.txt 0000664 0000000 0000000 00000007751 11175756704 0020757 0 ustar 00root root 0000000 0000000 :ץראה תאו םימשה תא םיהלא ארב תישארב א 1 םוהת ינפ-לע ךשחו והבו והת התיה ץראהו ב 2 :םימה ינפ-לע תפחרמ םיהלא חורו :רוא-יהיו רוא יהי םיהלא רמאיו ג 3 םיהלא לדביו בוט-יכ רואה-תא םיהלא אריו ד 4 :ךשחה ןיבו רואה ןיב הליל ארק ךשחלו םוי רואל םיהלא ארקיו ה 5 :דחא םוי רקב-יהיו ברע-יהיו יהיו םימה ךותב עיקר יהי םיהלא רמאיו ו 6 :םימל םימ ןיב לידבמ רשא םימה ןיב לדביו עיקרה-תא םיהלא שעיו ז 7 עיקרל לעמ רשא םימה ןיבו עיקרל תחתמ :ןכ-יהיו רקב-יהיו ברע-יהיו םימש עיקרל םיהלא ארקיו ח 8 :ינש םוי םוקמ-לא םימשה תחתמ םימה ווקי םיהלא רמאיו ט 9 :ןכ-יהיו השביה הארתו דחא ארק םימה הוקמלו ץרא השביל םיהלא ארקיו י 10 :בוט-יכ םיהלא אריו םימי ערז עירזמ בשע אשד ץראה אשדת םיהלא רמאיו אי 11 ץראה-לע וב-וערז רשא ונימל ירפ השע ירפ ץע :ןכ-יהיו ץעו והנימל ערז עירזמ בשע אשד ץראה אצותו בי 12 םיהלא אריו והנימל וב-וערז רשא ירפ-השע :בוט-יכ :ישילש םוי רקב-יהיו ברע-יהיו גי 13 לידבהל םימשה עיקרב תראמ יהי םיהלא רמאיו די 14 םידעומלו תתאל ויהו הלילה ןיבו םויה ןיב :םינשו םימילו ץראה-לע ריאהל םימשה עיקרב תרואמל ויהו וט 15 :ןכ-יהיו רואמה-תא םילדגה תראמה ינש-תא םיהלא שעיו זט 16 תלשממל ןטקה רואמה-תאו םויה תלשממל לדגה :םיבכוכה תאו הלילה ריאהל םימשה עיקרב םיהלא םתא ןתיו זי 17 :ץראה-לע ןיבו רואה ןיב לידבהלו הלילבו םויב לשמלו חי 18 :בוט-יכ םיהלא אריו ךשחה :יעיבר םוי רקב-יהיו ברע-יהיו טי 19 ףועו היח שפנ ץרש םימה וצרשי םיהלא רמאיו כ 20 :םימשה עיקר ינפ-לע ץראה-לע ףפועי שפנ-לכ תאו םילדגה םנינתה-תא םיהלא ארביו אכ 21 תאו םהנימל םימה וצרש רשא תשמרה היחה :בוט-יכ םיהלא אריו והנימל ףנכ ףוע-לכ ואלמו וברו ורפ רמאל םיהלא םתא ךרביו בכ 22 :ץראב ברי ףועהו םימיב םימה-תא :ישימח םוי רקב-יהיו ברע-יהיו גכ 23 המהב הנימל היח שפנ ץראה אצות םיהלא רמאיו דכ 24 :ןכ-יהיו הנימל ץרא-ותיחו שמרו המהבה-תאו הנימל ץראה תיח-תא םיהלא שעיו הכ 25 םיהלא אריו והנימל המדאה שמר-לכ תאו הנימל :בוט-יכ ונתומדכ ונמלצב םדא השענ םיהלא רמאיו וכ 26 המהבבו םימשה ףועבו םיה תגדב ודריו :ץראה-לע שמרה שמרה-לכבו ץראה-לכבו ארב םיהלא םלצב ומלצב םדאה-תא םיהלא ארביו זכ 27 :םתא ארב הבקנו רכז ותא וברו ורפ םיהלא םהל רמאיו םיהלא םתא ךרביו חכ 28 ףועבו םיה תגדב ודרו השבכו ץראה-תא ואלמו :ץראה-לע תשמרה היח-לכבו םימשה ערז בשע-לכ-תא םכל יתתנ הנה םיהלא רמאיו טכ 29 וב-רשא ץעה-לכ-תאו ץראה-לכ ינפ-לע רשא ערז :הלכאל היהי םכל ערז ערז ץע-ירפ שמור לכלו םימשה ףוע-לכלו ץראה תיח-לכלו ל 30 בשע קרי-לכ-תא היח שפנ וב-רשא ץראה-לע :ןכ-יהיו הלכאל דאמ בוט-הנהו השע רשא-לכ-תא םיהלא אריו אל 31 :יששה םוי רקב-יהיו בר flexi-streams-1.0.7/test/hebrew_utf8_crlf.txt 0000664 0000000 0000000 00000010055 11175756704 0021270 0 ustar 00root root 0000000 0000000 :ץראה תאו םימשה תא םיהלא ארב תישארב א 1 םוהת ינפ-לע ךשחו והבו והת התיה ץראהו ב 2 :םימה ינפ-לע תפחרמ םיהלא חורו :רוא-יהיו רוא יהי םיהלא רמאיו ג 3 םיהלא לדביו בוט-יכ רואה-תא םיהלא אריו ד 4 :ךשחה ןיבו רואה ןיב הליל ארק ךשחלו םוי רואל םיהלא ארקיו ה 5 :דחא םוי רקב-יהיו ברע-יהיו יהיו םימה ךותב עיקר יהי םיהלא רמאיו ו 6 :םימל םימ ןיב לידבמ רשא םימה ןיב לדביו עיקרה-תא םיהלא שעיו ז 7 עיקרל לעמ רשא םימה ןיבו עיקרל תחתמ :ןכ-יהיו רקב-יהיו ברע-יהיו םימש עיקרל םיהלא ארקיו ח 8 :ינש םוי םוקמ-לא םימשה תחתמ םימה ווקי םיהלא רמאיו ט 9 :ןכ-יהיו השביה הארתו דחא ארק םימה הוקמלו ץרא השביל םיהלא ארקיו י 10 :בוט-יכ םיהלא אריו םימי ערז עירזמ בשע אשד ץראה אשדת םיהלא רמאיו אי 11 ץראה-לע וב-וערז רשא ונימל ירפ השע ירפ ץע :ןכ-יהיו ץעו והנימל ערז עירזמ בשע אשד ץראה אצותו בי 12 םיהלא אריו והנימל וב-וערז רשא ירפ-השע :בוט-יכ :ישילש םוי רקב-יהיו ברע-יהיו גי 13 לידבהל םימשה עיקרב תראמ יהי םיהלא רמאיו די 14 םידעומלו תתאל ויהו הלילה ןיבו םויה ןיב :םינשו םימילו ץראה-לע ריאהל םימשה עיקרב תרואמל ויהו וט 15 :ןכ-יהיו רואמה-תא םילדגה תראמה ינש-תא םיהלא שעיו זט 16 תלשממל ןטקה רואמה-תאו םויה תלשממל לדגה :םיבכוכה תאו הלילה ריאהל םימשה עיקרב םיהלא םתא ןתיו זי 17 :ץראה-לע ןיבו רואה ןיב לידבהלו הלילבו םויב לשמלו חי 18 :בוט-יכ םיהלא אריו ךשחה :יעיבר םוי רקב-יהיו ברע-יהיו טי 19 ףועו היח שפנ ץרש םימה וצרשי םיהלא רמאיו כ 20 :םימשה עיקר ינפ-לע ץראה-לע ףפועי שפנ-לכ תאו םילדגה םנינתה-תא םיהלא ארביו אכ 21 תאו םהנימל םימה וצרש רשא תשמרה היחה :בוט-יכ םיהלא אריו והנימל ףנכ ףוע-לכ ואלמו וברו ורפ רמאל םיהלא םתא ךרביו בכ 22 :ץראב ברי ףועהו םימיב םימה-תא :ישימח םוי רקב-יהיו ברע-יהיו גכ 23 המהב הנימל היח שפנ ץראה אצות םיהלא רמאיו דכ 24 :ןכ-יהיו הנימל ץרא-ותיחו שמרו המהבה-תאו הנימל ץראה תיח-תא םיהלא שעיו הכ 25 םיהלא אריו והנימל המדאה שמר-לכ תאו הנימל :בוט-יכ ונתומדכ ונמלצב םדא השענ םיהלא רמאיו וכ 26 המהבבו םימשה ףועבו םיה תגדב ודריו :ץראה-לע שמרה שמרה-לכבו ץראה-לכבו ארב םיהלא םלצב ומלצב םדאה-תא םיהלא ארביו זכ 27 :םתא ארב הבקנו רכז ותא וברו ורפ םיהלא םהל רמאיו םיהלא םתא ךרביו חכ 28 ףועבו םיה תגדב ודרו השבכו ץראה-תא ואלמו :ץראה-לע תשמרה היח-לכבו םימשה ערז בשע-לכ-תא םכל יתתנ הנה םיהלא רמאיו טכ 29 וב-רשא ץעה-לכ-תאו ץראה-לכ ינפ-לע רשא ערז :הלכאל היהי םכל ערז ערז ץע-ירפ שמור לכלו םימשה ףוע-לכלו ץראה תיח-לכלו ל 30 בשע קרי-לכ-תא היח שפנ וב-רשא ץראה-לע :ןכ-יהיו הלכאל דאמ בוט-הנהו השע רשא-לכ-תא םיהלא אריו אל 31 :יששה םוי רקב-יהיו בר flexi-streams-1.0.7/test/hebrew_utf8_lf.txt 0000664 0000000 0000000 00000007751 11175756704 0020754 0 ustar 00root root 0000000 0000000 :ץראה תאו םימשה תא םיהלא ארב תישארב א 1 םוהת ינפ-לע ךשחו והבו והת התיה ץראהו ב 2 :םימה ינפ-לע תפחרמ םיהלא חורו :רוא-יהיו רוא יהי םיהלא רמאיו ג 3 םיהלא לדביו בוט-יכ רואה-תא םיהלא אריו ד 4 :ךשחה ןיבו רואה ןיב הליל ארק ךשחלו םוי רואל םיהלא ארקיו ה 5 :דחא םוי רקב-יהיו ברע-יהיו יהיו םימה ךותב עיקר יהי םיהלא רמאיו ו 6 :םימל םימ ןיב לידבמ רשא םימה ןיב לדביו עיקרה-תא םיהלא שעיו ז 7 עיקרל לעמ רשא םימה ןיבו עיקרל תחתמ :ןכ-יהיו רקב-יהיו ברע-יהיו םימש עיקרל םיהלא ארקיו ח 8 :ינש םוי םוקמ-לא םימשה תחתמ םימה ווקי םיהלא רמאיו ט 9 :ןכ-יהיו השביה הארתו דחא ארק םימה הוקמלו ץרא השביל םיהלא ארקיו י 10 :בוט-יכ םיהלא אריו םימי ערז עירזמ בשע אשד ץראה אשדת םיהלא רמאיו אי 11 ץראה-לע וב-וערז רשא ונימל ירפ השע ירפ ץע :ןכ-יהיו ץעו והנימל ערז עירזמ בשע אשד ץראה אצותו בי 12 םיהלא אריו והנימל וב-וערז רשא ירפ-השע :בוט-יכ :ישילש םוי רקב-יהיו ברע-יהיו גי 13 לידבהל םימשה עיקרב תראמ יהי םיהלא רמאיו די 14 םידעומלו תתאל ויהו הלילה ןיבו םויה ןיב :םינשו םימילו ץראה-לע ריאהל םימשה עיקרב תרואמל ויהו וט 15 :ןכ-יהיו רואמה-תא םילדגה תראמה ינש-תא םיהלא שעיו זט 16 תלשממל ןטקה רואמה-תאו םויה תלשממל לדגה :םיבכוכה תאו הלילה ריאהל םימשה עיקרב םיהלא םתא ןתיו זי 17 :ץראה-לע ןיבו רואה ןיב לידבהלו הלילבו םויב לשמלו חי 18 :בוט-יכ םיהלא אריו ךשחה :יעיבר םוי רקב-יהיו ברע-יהיו טי 19 ףועו היח שפנ ץרש םימה וצרשי םיהלא רמאיו כ 20 :םימשה עיקר ינפ-לע ץראה-לע ףפועי שפנ-לכ תאו םילדגה םנינתה-תא םיהלא ארביו אכ 21 תאו םהנימל םימה וצרש רשא תשמרה היחה :בוט-יכ םיהלא אריו והנימל ףנכ ףוע-לכ ואלמו וברו ורפ רמאל םיהלא םתא ךרביו בכ 22 :ץראב ברי ףועהו םימיב םימה-תא :ישימח םוי רקב-יהיו ברע-יהיו גכ 23 המהב הנימל היח שפנ ץראה אצות םיהלא רמאיו דכ 24 :ןכ-יהיו הנימל ץרא-ותיחו שמרו המהבה-תאו הנימל ץראה תיח-תא םיהלא שעיו הכ 25 םיהלא אריו והנימל המדאה שמר-לכ תאו הנימל :בוט-יכ ונתומדכ ונמלצב םדא השענ םיהלא רמאיו וכ 26 המהבבו םימשה ףועבו םיה תגדב ודריו :ץראה-לע שמרה שמרה-לכבו ץראה-לכבו ארב םיהלא םלצב ומלצב םדאה-תא םיהלא ארביו זכ 27 :םתא ארב הבקנו רכז ותא וברו ורפ םיהלא םהל רמאיו םיהלא םתא ךרביו חכ 28 ףועבו םיה תגדב ודרו השבכו ץראה-תא ואלמו :ץראה-לע תשמרה היח-לכבו םימשה ערז בשע-לכ-תא םכל יתתנ הנה םיהלא רמאיו טכ 29 וב-רשא ץעה-לכ-תאו ץראה-לכ ינפ-לע רשא ערז :הלכאל היהי םכל ערז ערז ץע-ירפ שמור לכלו םימשה ףוע-לכלו ץראה תיח-לכלו ל 30 בשע קרי-לכ-תא היח שפנ וב-רשא ץראה-לע :ןכ-יהיו הלכאל דאמ בוט-הנהו השע רשא-לכ-תא םיהלא אריו אל 31 :יששה םוי רקב-יהיו בר flexi-streams-1.0.7/test/kafka_cp1252_cr.txt 0000664 0000000 0000000 00000012741 11175756704 0020601 0 ustar 00root root 0000000 0000000 Als Gregor Samsa eines Morgens aus unruhigen Trumen erwachte, fand er sich in seinem Bett zu einem ungeheueren Ungeziefer verwandelt. Er lag auf seinem panzerartig harten Rcken und sah, wenn er den Kopf ein wenig hob, seinen gewlbten, braunen, von bogenfrmigen Versteifungen geteilten Bauch, auf dessen Hhe sich die Bettdecke, zum gnzlichen Niedergleiten bereit, kaum noch erhalten konnte. Seine vielen, im Vergleich zu seinem sonstigen Umfang klglich dnnen Beine flimmerten ihm hilflos vor den Augen. Was ist mit mir geschehen?, dachte er. Es war kein Traum. Sein Zimmer, ein richtiges, nur etwas zu kleines Menschenzimmer, lag ruhig zwischen den vier wohlbekannten Wnden. ber dem Tisch, auf dem eine auseinandergepackte Musterkollektion von Tuchwaren ausgebreitet war - Samsa war Reisender - hing das Bild, das er vor kurzem aus einer illustrierten Zeitschrift ausgeschnitten und in einem hbschen, vergoldeten Rahmen untergebracht hatte. Es stellte eine Dame dar, die mit einem Pelzhut und einer Pelzboa versehen, aufrecht dasa und einen schweren Pelzmuff, in dem ihr ganzer Unterarm verschwunden war, dem Beschauer entgegenhob. Gregors Blick richtete sich dann zum Fenster, und das trbe Wetter - man hrte Regentropfen auf das Fensterblech aufschlagen - machte ihn ganz melancholisch. Wie wre es, wenn ich noch ein wenig weiterschliefe und alle Narrheiten verge, dachte er, aber das war gnzlich undurchfhrbar, denn er war gewhnt, auf der rechten Seite zu schlafen, konnte sich aber in seinem gegenwrtigen Zustand nicht in diese Lage bringen. Mit welcher Kraft er sich auch auf die rechte Seite warf, immer wieder schaukelte er in die Rckenlage zurck. Er versuchte es wohl hundertmal, schlo die Augen, um die zappelnden Beine nicht sehen zu mssen, und lie erst ab, als er in der Seite einen noch nie gefhlten, leichten, dumpfen Schmerz zu fhlen begann. Ach Gott, dachte er, was fr einen anstrengenden Beruf habe ich gewhlt! Tag aus, Tag ein auf der Reise. Die geschftlichen Aufregungen sind viel grer, als im eigentlichen Geschft zu Hause, und auerdem ist mir noch diese Plage des Reisens auferlegt, die Sorgen um die Zuganschlsse, das unregelmige, schlechte Essen, ein immer wechselnder, nie andauernder, nie herzlich werdender menschlicher Verkehr. Der Teufel soll das alles holen! Er fhlte ein leichtes Jucken oben auf dem Bauch; schob sich auf dem Rcken langsam nher zum Bettpfosten, um den Kopf besser heben zu knnen; fand die juckende Stelle, die mit lauter kleinen weien Pnktchen besetzt war, die er nicht zu beurteilen verstand; und wollte mit einem Bein die Stelle betasten, zog es aber gleich zurck, denn bei der Berhrung umwehten ihn Klteschauer. Er glitt wieder in seine frhere Lage zurck. Dies frhzeitige Aufstehen, dachte er, macht einen ganz bldsinnig. Der Mensch mu seinen Schlaf haben. Andere Reisende leben wie Haremsfrauen. Wenn ich zum Beispiel im Laufe des Vormittags ins Gasthaus zurckgehe, um die erlangten Auftrge zu berschreiben, sitzen diese Herren erst beim Frhstck. Das sollte ich bei meinem Chef versuchen; ich wrde auf der Stelle hinausfliegen. Wer wei brigens, ob das nicht sehr gut fr mich wre. Wenn ich mich nicht wegen meiner Eltern zurckhielte, ich htte lngst gekndigt, ich wre vor den Chef hin getreten und htte ihm meine Meinung von Grund des Herzens aus gesagt. Vom Pult htte er fallen mssen! Es ist auch eine sonderbare Art, sich auf das Pult zu setzen und von der Hhe herab mit dem Angestellten zu reden, der berdies wegen der Schwerhrigkeit des Chefs ganz nahe herantreten mu. Nun, die Hoffnung ist noch nicht gnzlich aufgegeben; habe ich einmal das Geld beisammen, um die Schuld der Eltern an ihn abzuzahlen - es drfte noch fnf bis sechs Jahre dauern - , mache ich die Sache unbedingt. Dann wird der groe Schnitt gemacht. Vorlufig allerdings mu ich aufstehen, denn mein Zug fhrt um fnf. Und er sah zur Weckuhr hinber, die auf dem Kasten tickte. Himmlischer Vater!, dachte er. Es war halb sieben Uhr, und die Zeiger gingen ruhig vorwrts, es war sogar halb vorber, es nherte sich schon dreiviertel. Sollte der Wecker nicht gelutet haben? Man sah vom Bett aus, da er auf vier Uhr richtig eingestellt war; gewi hatte er auch gelutet. Ja, aber war es mglich, dieses mbelerschtternde Luten ruhig zu verschlafen? Nun, ruhig hatte er ja nicht geschlafen, aber wahrscheinlich desto fester. Was aber sollte er jetzt tun? Der nchste Zug ging um sieben Uhr; um den einzuholen, htte er sich unsinnig beeilen mssen, und die Kollektion war noch nicht eingepackt, und er selbst fhlte sich durchaus nicht besonders frisch und beweglich. Und selbst wenn er den Zug einholte, ein Donnerwetter des Chefs war nicht zu vermeiden, denn der Geschftsdiener hatte beim Fnfuhrzug gewartet und die Meldung von seiner Versumnis lngst erstattet. Es war eine Kreatur des Chefs, ohne Rckgrat und Verstand. Wie nun, wenn er sich krank meldete? Das wre aber uerst peinlich und verdchtig, denn Gregor war whrend seines fnfjhrigen Dienstes noch nicht einmal krank gewesen. Gewi wrde der Chef mit dem Krankenkassenarzt kommen, wrde den Eltern wegen des faulen Sohnes Vorwrfe machen und alle Einwnde durch den Hinweis auf den Krankenkassenarzt abschneiden, fr den es ja berhaupt nur ganz gesunde, aber arbeitsscheue Menschen gibt. Und htte er brigens in diesem Falle so ganz unrecht? Gregor fhlte sich tatschlich, abgesehen von einer nach dem langen Schlaf wirklich berflssigen Schlfrigkeit, ganz wohl und hatte sogar einen besonders krftigen Hunger. flexi-streams-1.0.7/test/kafka_cp1252_crlf.txt 0000664 0000000 0000000 00000012754 11175756704 0021127 0 ustar 00root root 0000000 0000000 Als Gregor Samsa eines Morgens aus unruhigen Trumen erwachte, fand er sich in seinem Bett zu einem ungeheueren Ungeziefer verwandelt. Er lag auf seinem panzerartig harten Rcken und sah, wenn er den Kopf ein wenig hob, seinen gewlbten, braunen, von bogenfrmigen Versteifungen geteilten Bauch, auf dessen Hhe sich die Bettdecke, zum gnzlichen Niedergleiten bereit, kaum noch erhalten konnte. Seine vielen, im Vergleich zu seinem sonstigen Umfang klglich dnnen Beine flimmerten ihm hilflos vor den Augen. Was ist mit mir geschehen?, dachte er. Es war kein Traum. Sein Zimmer, ein richtiges, nur etwas zu kleines Menschenzimmer, lag ruhig zwischen den vier wohlbekannten Wnden. ber dem Tisch, auf dem eine auseinandergepackte Musterkollektion von Tuchwaren ausgebreitet war - Samsa war Reisender - hing das Bild, das er vor kurzem aus einer illustrierten Zeitschrift ausgeschnitten und in einem hbschen, vergoldeten Rahmen untergebracht hatte. Es stellte eine Dame dar, die mit einem Pelzhut und einer Pelzboa versehen, aufrecht dasa und einen schweren Pelzmuff, in dem ihr ganzer Unterarm verschwunden war, dem Beschauer entgegenhob. Gregors Blick richtete sich dann zum Fenster, und das trbe Wetter - man hrte Regentropfen auf das Fensterblech aufschlagen - machte ihn ganz melancholisch. Wie wre es, wenn ich noch ein wenig weiterschliefe und alle Narrheiten verge, dachte er, aber das war gnzlich undurchfhrbar, denn er war gewhnt, auf der rechten Seite zu schlafen, konnte sich aber in seinem gegenwrtigen Zustand nicht in diese Lage bringen. Mit welcher Kraft er sich auch auf die rechte Seite warf, immer wieder schaukelte er in die Rckenlage zurck. Er versuchte es wohl hundertmal, schlo die Augen, um die zappelnden Beine nicht sehen zu mssen, und lie erst ab, als er in der Seite einen noch nie gefhlten, leichten, dumpfen Schmerz zu fhlen begann. Ach Gott, dachte er, was fr einen anstrengenden Beruf habe ich gewhlt! Tag aus, Tag ein auf der Reise. Die geschftlichen Aufregungen sind viel grer, als im eigentlichen Geschft zu Hause, und auerdem ist mir noch diese Plage des Reisens auferlegt, die Sorgen um die Zuganschlsse, das unregelmige, schlechte Essen, ein immer wechselnder, nie andauernder, nie herzlich werdender menschlicher Verkehr. Der Teufel soll das alles holen! Er fhlte ein leichtes Jucken oben auf dem Bauch; schob sich auf dem Rcken langsam nher zum Bettpfosten, um den Kopf besser heben zu knnen; fand die juckende Stelle, die mit lauter kleinen weien Pnktchen besetzt war, die er nicht zu beurteilen verstand; und wollte mit einem Bein die Stelle betasten, zog es aber gleich zurck, denn bei der Berhrung umwehten ihn Klteschauer. Er glitt wieder in seine frhere Lage zurck. Dies frhzeitige Aufstehen, dachte er, macht einen ganz bldsinnig. Der Mensch mu seinen Schlaf haben. Andere Reisende leben wie Haremsfrauen. Wenn ich zum Beispiel im Laufe des Vormittags ins Gasthaus zurckgehe, um die erlangten Auftrge zu berschreiben, sitzen diese Herren erst beim Frhstck. Das sollte ich bei meinem Chef versuchen; ich wrde auf der Stelle hinausfliegen. Wer wei brigens, ob das nicht sehr gut fr mich wre. Wenn ich mich nicht wegen meiner Eltern zurckhielte, ich htte lngst gekndigt, ich wre vor den Chef hin getreten und htte ihm meine Meinung von Grund des Herzens aus gesagt. Vom Pult htte er fallen mssen! Es ist auch eine sonderbare Art, sich auf das Pult zu setzen und von der Hhe herab mit dem Angestellten zu reden, der berdies wegen der Schwerhrigkeit des Chefs ganz nahe herantreten mu. Nun, die Hoffnung ist noch nicht gnzlich aufgegeben; habe ich einmal das Geld beisammen, um die Schuld der Eltern an ihn abzuzahlen - es drfte noch fnf bis sechs Jahre dauern - , mache ich die Sache unbedingt. Dann wird der groe Schnitt gemacht. Vorlufig allerdings mu ich aufstehen, denn mein Zug fhrt um fnf. Und er sah zur Weckuhr hinber, die auf dem Kasten tickte. Himmlischer Vater!, dachte er. Es war halb sieben Uhr, und die Zeiger gingen ruhig vorwrts, es war sogar halb vorber, es nherte sich schon dreiviertel. Sollte der Wecker nicht gelutet haben? Man sah vom Bett aus, da er auf vier Uhr richtig eingestellt war; gewi hatte er auch gelutet. Ja, aber war es mglich, dieses mbelerschtternde Luten ruhig zu verschlafen? Nun, ruhig hatte er ja nicht geschlafen, aber wahrscheinlich desto fester. Was aber sollte er jetzt tun? Der nchste Zug ging um sieben Uhr; um den einzuholen, htte er sich unsinnig beeilen mssen, und die Kollektion war noch nicht eingepackt, und er selbst fhlte sich durchaus nicht besonders frisch und beweglich. Und selbst wenn er den Zug einholte, ein Donnerwetter des Chefs war nicht zu vermeiden, denn der Geschftsdiener hatte beim Fnfuhrzug gewartet und die Meldung von seiner Versumnis lngst erstattet. Es war eine Kreatur des Chefs, ohne Rckgrat und Verstand. Wie nun, wenn er sich krank meldete? Das wre aber uerst peinlich und verdchtig, denn Gregor war whrend seines fnfjhrigen Dienstes noch nicht einmal krank gewesen. Gewi wrde der Chef mit dem Krankenkassenarzt kommen, wrde den Eltern wegen des faulen Sohnes Vorwrfe machen und alle Einwnde durch den Hinweis auf den Krankenkassenarzt abschneiden, fr den es ja berhaupt nur ganz gesunde, aber arbeitsscheue Menschen gibt. Und htte er brigens in diesem Falle so ganz unrecht? Gregor fhlte sich tatschlich, abgesehen von einer nach dem langen Schlaf wirklich berflssigen Schlfrigkeit, ganz wohl und hatte sogar einen besonders krftigen Hunger. flexi-streams-1.0.7/test/kafka_cp1252_lf.txt 0000664 0000000 0000000 00000012741 11175756704 0020576 0 ustar 00root root 0000000 0000000 Als Gregor Samsa eines Morgens aus unruhigen Trumen erwachte, fand er sich in seinem Bett zu einem ungeheueren Ungeziefer verwandelt. Er lag auf seinem panzerartig harten Rcken und sah, wenn er den Kopf ein wenig hob, seinen gewlbten, braunen, von bogenfrmigen Versteifungen geteilten Bauch, auf dessen Hhe sich die Bettdecke, zum gnzlichen Niedergleiten bereit, kaum noch erhalten konnte. Seine vielen, im Vergleich zu seinem sonstigen Umfang klglich dnnen Beine flimmerten ihm hilflos vor den Augen. Was ist mit mir geschehen?, dachte er. Es war kein Traum. Sein Zimmer, ein richtiges, nur etwas zu kleines Menschenzimmer, lag ruhig zwischen den vier wohlbekannten Wnden. ber dem Tisch, auf dem eine auseinandergepackte Musterkollektion von Tuchwaren ausgebreitet war - Samsa war Reisender - hing das Bild, das er vor kurzem aus einer illustrierten Zeitschrift ausgeschnitten und in einem hbschen, vergoldeten Rahmen untergebracht hatte. Es stellte eine Dame dar, die mit einem Pelzhut und einer Pelzboa versehen, aufrecht dasa und einen schweren Pelzmuff, in dem ihr ganzer Unterarm verschwunden war, dem Beschauer entgegenhob. Gregors Blick richtete sich dann zum Fenster, und das trbe Wetter - man hrte Regentropfen auf das Fensterblech aufschlagen - machte ihn ganz melancholisch. Wie wre es, wenn ich noch ein wenig weiterschliefe und alle Narrheiten verge, dachte er, aber das war gnzlich undurchfhrbar, denn er war gewhnt, auf der rechten Seite zu schlafen, konnte sich aber in seinem gegenwrtigen Zustand nicht in diese Lage bringen. Mit welcher Kraft er sich auch auf die rechte Seite warf, immer wieder schaukelte er in die Rckenlage zurck. Er versuchte es wohl hundertmal, schlo die Augen, um die zappelnden Beine nicht sehen zu mssen, und lie erst ab, als er in der Seite einen noch nie gefhlten, leichten, dumpfen Schmerz zu fhlen begann. Ach Gott, dachte er, was fr einen anstrengenden Beruf habe ich gewhlt! Tag aus, Tag ein auf der Reise. Die geschftlichen Aufregungen sind viel grer, als im eigentlichen Geschft zu Hause, und auerdem ist mir noch diese Plage des Reisens auferlegt, die Sorgen um die Zuganschlsse, das unregelmige, schlechte Essen, ein immer wechselnder, nie andauernder, nie herzlich werdender menschlicher Verkehr. Der Teufel soll das alles holen! Er fhlte ein leichtes Jucken oben auf dem Bauch; schob sich auf dem Rcken langsam nher zum Bettpfosten, um den Kopf besser heben zu knnen; fand die juckende Stelle, die mit lauter kleinen weien Pnktchen besetzt war, die er nicht zu beurteilen verstand; und wollte mit einem Bein die Stelle betasten, zog es aber gleich zurck, denn bei der Berhrung umwehten ihn Klteschauer. Er glitt wieder in seine frhere Lage zurck. Dies frhzeitige Aufstehen, dachte er, macht einen ganz bldsinnig. Der Mensch mu seinen Schlaf haben. Andere Reisende leben wie Haremsfrauen. Wenn ich zum Beispiel im Laufe des Vormittags ins Gasthaus zurckgehe, um die erlangten Auftrge zu berschreiben, sitzen diese Herren erst beim Frhstck. Das sollte ich bei meinem Chef versuchen; ich wrde auf der Stelle hinausfliegen. Wer wei brigens, ob das nicht sehr gut fr mich wre. Wenn ich mich nicht wegen meiner Eltern zurckhielte, ich htte lngst gekndigt, ich wre vor den Chef hin getreten und htte ihm meine Meinung von Grund des Herzens aus gesagt. Vom Pult htte er fallen mssen! Es ist auch eine sonderbare Art, sich auf das Pult zu setzen und von der Hhe herab mit dem Angestellten zu reden, der berdies wegen der Schwerhrigkeit des Chefs ganz nahe herantreten mu. Nun, die Hoffnung ist noch nicht gnzlich aufgegeben; habe ich einmal das Geld beisammen, um die Schuld der Eltern an ihn abzuzahlen - es drfte noch fnf bis sechs Jahre dauern - , mache ich die Sache unbedingt. Dann wird der groe Schnitt gemacht. Vorlufig allerdings mu ich aufstehen, denn mein Zug fhrt um fnf. Und er sah zur Weckuhr hinber, die auf dem Kasten tickte. Himmlischer Vater!, dachte er. Es war halb sieben Uhr, und die Zeiger gingen ruhig vorwrts, es war sogar halb vorber, es nherte sich schon dreiviertel. Sollte der Wecker nicht gelutet haben? Man sah vom Bett aus, da er auf vier Uhr richtig eingestellt war; gewi hatte er auch gelutet. Ja, aber war es mglich, dieses mbelerschtternde Luten ruhig zu verschlafen? Nun, ruhig hatte er ja nicht geschlafen, aber wahrscheinlich desto fester. Was aber sollte er jetzt tun? Der nchste Zug ging um sieben Uhr; um den einzuholen, htte er sich unsinnig beeilen mssen, und die Kollektion war noch nicht eingepackt, und er selbst fhlte sich durchaus nicht besonders frisch und beweglich. Und selbst wenn er den Zug einholte, ein Donnerwetter des Chefs war nicht zu vermeiden, denn der Geschftsdiener hatte beim Fnfuhrzug gewartet und die Meldung von seiner Versumnis lngst erstattet. Es war eine Kreatur des Chefs, ohne Rckgrat und Verstand. Wie nun, wenn er sich krank meldete? Das wre aber uerst peinlich und verdchtig, denn Gregor war whrend seines fnfjhrigen Dienstes noch nicht einmal krank gewesen. Gewi wrde der Chef mit dem Krankenkassenarzt kommen, wrde den Eltern wegen des faulen Sohnes Vorwrfe machen und alle Einwnde durch den Hinweis auf den Krankenkassenarzt abschneiden, fr den es ja berhaupt nur ganz gesunde, aber arbeitsscheue Menschen gibt. Und htte er brigens in diesem Falle so ganz unrecht? Gregor fhlte sich tatschlich, abgesehen von einer nach dem langen Schlaf wirklich berflssigen Schlfrigkeit, ganz wohl und hatte sogar einen besonders krftigen Hunger. flexi-streams-1.0.7/test/kafka_latin1_cr.txt 0000664 0000000 0000000 00000012741 11175756704 0021055 0 ustar 00root root 0000000 0000000 Als Gregor Samsa eines Morgens aus unruhigen Trumen erwachte, fand er sich in seinem Bett zu einem ungeheueren Ungeziefer verwandelt. Er lag auf seinem panzerartig harten Rcken und sah, wenn er den Kopf ein wenig hob, seinen gewlbten, braunen, von bogenfrmigen Versteifungen geteilten Bauch, auf dessen Hhe sich die Bettdecke, zum gnzlichen Niedergleiten bereit, kaum noch erhalten konnte. Seine vielen, im Vergleich zu seinem sonstigen Umfang klglich dnnen Beine flimmerten ihm hilflos vor den Augen. Was ist mit mir geschehen?, dachte er. Es war kein Traum. Sein Zimmer, ein richtiges, nur etwas zu kleines Menschenzimmer, lag ruhig zwischen den vier wohlbekannten Wnden. ber dem Tisch, auf dem eine auseinandergepackte Musterkollektion von Tuchwaren ausgebreitet war - Samsa war Reisender - hing das Bild, das er vor kurzem aus einer illustrierten Zeitschrift ausgeschnitten und in einem hbschen, vergoldeten Rahmen untergebracht hatte. Es stellte eine Dame dar, die mit einem Pelzhut und einer Pelzboa versehen, aufrecht dasa und einen schweren Pelzmuff, in dem ihr ganzer Unterarm verschwunden war, dem Beschauer entgegenhob. Gregors Blick richtete sich dann zum Fenster, und das trbe Wetter - man hrte Regentropfen auf das Fensterblech aufschlagen - machte ihn ganz melancholisch. Wie wre es, wenn ich noch ein wenig weiterschliefe und alle Narrheiten verge, dachte er, aber das war gnzlich undurchfhrbar, denn er war gewhnt, auf der rechten Seite zu schlafen, konnte sich aber in seinem gegenwrtigen Zustand nicht in diese Lage bringen. Mit welcher Kraft er sich auch auf die rechte Seite warf, immer wieder schaukelte er in die Rckenlage zurck. Er versuchte es wohl hundertmal, schlo die Augen, um die zappelnden Beine nicht sehen zu mssen, und lie erst ab, als er in der Seite einen noch nie gefhlten, leichten, dumpfen Schmerz zu fhlen begann. Ach Gott, dachte er, was fr einen anstrengenden Beruf habe ich gewhlt! Tag aus, Tag ein auf der Reise. Die geschftlichen Aufregungen sind viel grer, als im eigentlichen Geschft zu Hause, und auerdem ist mir noch diese Plage des Reisens auferlegt, die Sorgen um die Zuganschlsse, das unregelmige, schlechte Essen, ein immer wechselnder, nie andauernder, nie herzlich werdender menschlicher Verkehr. Der Teufel soll das alles holen! Er fhlte ein leichtes Jucken oben auf dem Bauch; schob sich auf dem Rcken langsam nher zum Bettpfosten, um den Kopf besser heben zu knnen; fand die juckende Stelle, die mit lauter kleinen weien Pnktchen besetzt war, die er nicht zu beurteilen verstand; und wollte mit einem Bein die Stelle betasten, zog es aber gleich zurck, denn bei der Berhrung umwehten ihn Klteschauer. Er glitt wieder in seine frhere Lage zurck. Dies frhzeitige Aufstehen, dachte er, macht einen ganz bldsinnig. Der Mensch mu seinen Schlaf haben. Andere Reisende leben wie Haremsfrauen. Wenn ich zum Beispiel im Laufe des Vormittags ins Gasthaus zurckgehe, um die erlangten Auftrge zu berschreiben, sitzen diese Herren erst beim Frhstck. Das sollte ich bei meinem Chef versuchen; ich wrde auf der Stelle hinausfliegen. Wer wei brigens, ob das nicht sehr gut fr mich wre. Wenn ich mich nicht wegen meiner Eltern zurckhielte, ich htte lngst gekndigt, ich wre vor den Chef hin getreten und htte ihm meine Meinung von Grund des Herzens aus gesagt. Vom Pult htte er fallen mssen! Es ist auch eine sonderbare Art, sich auf das Pult zu setzen und von der Hhe herab mit dem Angestellten zu reden, der berdies wegen der Schwerhrigkeit des Chefs ganz nahe herantreten mu. Nun, die Hoffnung ist noch nicht gnzlich aufgegeben; habe ich einmal das Geld beisammen, um die Schuld der Eltern an ihn abzuzahlen - es drfte noch fnf bis sechs Jahre dauern - , mache ich die Sache unbedingt. Dann wird der groe Schnitt gemacht. Vorlufig allerdings mu ich aufstehen, denn mein Zug fhrt um fnf. Und er sah zur Weckuhr hinber, die auf dem Kasten tickte. Himmlischer Vater!, dachte er. Es war halb sieben Uhr, und die Zeiger gingen ruhig vorwrts, es war sogar halb vorber, es nherte sich schon dreiviertel. Sollte der Wecker nicht gelutet haben? Man sah vom Bett aus, da er auf vier Uhr richtig eingestellt war; gewi hatte er auch gelutet. Ja, aber war es mglich, dieses mbelerschtternde Luten ruhig zu verschlafen? Nun, ruhig hatte er ja nicht geschlafen, aber wahrscheinlich desto fester. Was aber sollte er jetzt tun? Der nchste Zug ging um sieben Uhr; um den einzuholen, htte er sich unsinnig beeilen mssen, und die Kollektion war noch nicht eingepackt, und er selbst fhlte sich durchaus nicht besonders frisch und beweglich. Und selbst wenn er den Zug einholte, ein Donnerwetter des Chefs war nicht zu vermeiden, denn der Geschftsdiener hatte beim Fnfuhrzug gewartet und die Meldung von seiner Versumnis lngst erstattet. Es war eine Kreatur des Chefs, ohne Rckgrat und Verstand. Wie nun, wenn er sich krank meldete? Das wre aber uerst peinlich und verdchtig, denn Gregor war whrend seines fnfjhrigen Dienstes noch nicht einmal krank gewesen. Gewi wrde der Chef mit dem Krankenkassenarzt kommen, wrde den Eltern wegen des faulen Sohnes Vorwrfe machen und alle Einwnde durch den Hinweis auf den Krankenkassenarzt abschneiden, fr den es ja berhaupt nur ganz gesunde, aber arbeitsscheue Menschen gibt. Und htte er brigens in diesem Falle so ganz unrecht? Gregor fhlte sich tatschlich, abgesehen von einer nach dem langen Schlaf wirklich berflssigen Schlfrigkeit, ganz wohl und hatte sogar einen besonders krftigen Hunger. flexi-streams-1.0.7/test/kafka_latin1_crlf.txt 0000664 0000000 0000000 00000012754 11175756704 0021403 0 ustar 00root root 0000000 0000000 Als Gregor Samsa eines Morgens aus unruhigen Trumen erwachte, fand er sich in seinem Bett zu einem ungeheueren Ungeziefer verwandelt. Er lag auf seinem panzerartig harten Rcken und sah, wenn er den Kopf ein wenig hob, seinen gewlbten, braunen, von bogenfrmigen Versteifungen geteilten Bauch, auf dessen Hhe sich die Bettdecke, zum gnzlichen Niedergleiten bereit, kaum noch erhalten konnte. Seine vielen, im Vergleich zu seinem sonstigen Umfang klglich dnnen Beine flimmerten ihm hilflos vor den Augen. Was ist mit mir geschehen?, dachte er. Es war kein Traum. Sein Zimmer, ein richtiges, nur etwas zu kleines Menschenzimmer, lag ruhig zwischen den vier wohlbekannten Wnden. ber dem Tisch, auf dem eine auseinandergepackte Musterkollektion von Tuchwaren ausgebreitet war - Samsa war Reisender - hing das Bild, das er vor kurzem aus einer illustrierten Zeitschrift ausgeschnitten und in einem hbschen, vergoldeten Rahmen untergebracht hatte. Es stellte eine Dame dar, die mit einem Pelzhut und einer Pelzboa versehen, aufrecht dasa und einen schweren Pelzmuff, in dem ihr ganzer Unterarm verschwunden war, dem Beschauer entgegenhob. Gregors Blick richtete sich dann zum Fenster, und das trbe Wetter - man hrte Regentropfen auf das Fensterblech aufschlagen - machte ihn ganz melancholisch. Wie wre es, wenn ich noch ein wenig weiterschliefe und alle Narrheiten verge, dachte er, aber das war gnzlich undurchfhrbar, denn er war gewhnt, auf der rechten Seite zu schlafen, konnte sich aber in seinem gegenwrtigen Zustand nicht in diese Lage bringen. Mit welcher Kraft er sich auch auf die rechte Seite warf, immer wieder schaukelte er in die Rckenlage zurck. Er versuchte es wohl hundertmal, schlo die Augen, um die zappelnden Beine nicht sehen zu mssen, und lie erst ab, als er in der Seite einen noch nie gefhlten, leichten, dumpfen Schmerz zu fhlen begann. Ach Gott, dachte er, was fr einen anstrengenden Beruf habe ich gewhlt! Tag aus, Tag ein auf der Reise. Die geschftlichen Aufregungen sind viel grer, als im eigentlichen Geschft zu Hause, und auerdem ist mir noch diese Plage des Reisens auferlegt, die Sorgen um die Zuganschlsse, das unregelmige, schlechte Essen, ein immer wechselnder, nie andauernder, nie herzlich werdender menschlicher Verkehr. Der Teufel soll das alles holen! Er fhlte ein leichtes Jucken oben auf dem Bauch; schob sich auf dem Rcken langsam nher zum Bettpfosten, um den Kopf besser heben zu knnen; fand die juckende Stelle, die mit lauter kleinen weien Pnktchen besetzt war, die er nicht zu beurteilen verstand; und wollte mit einem Bein die Stelle betasten, zog es aber gleich zurck, denn bei der Berhrung umwehten ihn Klteschauer. Er glitt wieder in seine frhere Lage zurck. Dies frhzeitige Aufstehen, dachte er, macht einen ganz bldsinnig. Der Mensch mu seinen Schlaf haben. Andere Reisende leben wie Haremsfrauen. Wenn ich zum Beispiel im Laufe des Vormittags ins Gasthaus zurckgehe, um die erlangten Auftrge zu berschreiben, sitzen diese Herren erst beim Frhstck. Das sollte ich bei meinem Chef versuchen; ich wrde auf der Stelle hinausfliegen. Wer wei brigens, ob das nicht sehr gut fr mich wre. Wenn ich mich nicht wegen meiner Eltern zurckhielte, ich htte lngst gekndigt, ich wre vor den Chef hin getreten und htte ihm meine Meinung von Grund des Herzens aus gesagt. Vom Pult htte er fallen mssen! Es ist auch eine sonderbare Art, sich auf das Pult zu setzen und von der Hhe herab mit dem Angestellten zu reden, der berdies wegen der Schwerhrigkeit des Chefs ganz nahe herantreten mu. Nun, die Hoffnung ist noch nicht gnzlich aufgegeben; habe ich einmal das Geld beisammen, um die Schuld der Eltern an ihn abzuzahlen - es drfte noch fnf bis sechs Jahre dauern - , mache ich die Sache unbedingt. Dann wird der groe Schnitt gemacht. Vorlufig allerdings mu ich aufstehen, denn mein Zug fhrt um fnf. Und er sah zur Weckuhr hinber, die auf dem Kasten tickte. Himmlischer Vater!, dachte er. Es war halb sieben Uhr, und die Zeiger gingen ruhig vorwrts, es war sogar halb vorber, es nherte sich schon dreiviertel. Sollte der Wecker nicht gelutet haben? Man sah vom Bett aus, da er auf vier Uhr richtig eingestellt war; gewi hatte er auch gelutet. Ja, aber war es mglich, dieses mbelerschtternde Luten ruhig zu verschlafen? Nun, ruhig hatte er ja nicht geschlafen, aber wahrscheinlich desto fester. Was aber sollte er jetzt tun? Der nchste Zug ging um sieben Uhr; um den einzuholen, htte er sich unsinnig beeilen mssen, und die Kollektion war noch nicht eingepackt, und er selbst fhlte sich durchaus nicht besonders frisch und beweglich. Und selbst wenn er den Zug einholte, ein Donnerwetter des Chefs war nicht zu vermeiden, denn der Geschftsdiener hatte beim Fnfuhrzug gewartet und die Meldung von seiner Versumnis lngst erstattet. Es war eine Kreatur des Chefs, ohne Rckgrat und Verstand. Wie nun, wenn er sich krank meldete? Das wre aber uerst peinlich und verdchtig, denn Gregor war whrend seines fnfjhrigen Dienstes noch nicht einmal krank gewesen. Gewi wrde der Chef mit dem Krankenkassenarzt kommen, wrde den Eltern wegen des faulen Sohnes Vorwrfe machen und alle Einwnde durch den Hinweis auf den Krankenkassenarzt abschneiden, fr den es ja berhaupt nur ganz gesunde, aber arbeitsscheue Menschen gibt. Und htte er brigens in diesem Falle so ganz unrecht? Gregor fhlte sich tatschlich, abgesehen von einer nach dem langen Schlaf wirklich berflssigen Schlfrigkeit, ganz wohl und hatte sogar einen besonders krftigen Hunger. flexi-streams-1.0.7/test/kafka_latin1_lf.txt 0000664 0000000 0000000 00000012741 11175756704 0021052 0 ustar 00root root 0000000 0000000 Als Gregor Samsa eines Morgens aus unruhigen Trumen erwachte, fand er sich in seinem Bett zu einem ungeheueren Ungeziefer verwandelt. Er lag auf seinem panzerartig harten Rcken und sah, wenn er den Kopf ein wenig hob, seinen gewlbten, braunen, von bogenfrmigen Versteifungen geteilten Bauch, auf dessen Hhe sich die Bettdecke, zum gnzlichen Niedergleiten bereit, kaum noch erhalten konnte. Seine vielen, im Vergleich zu seinem sonstigen Umfang klglich dnnen Beine flimmerten ihm hilflos vor den Augen. Was ist mit mir geschehen?, dachte er. Es war kein Traum. Sein Zimmer, ein richtiges, nur etwas zu kleines Menschenzimmer, lag ruhig zwischen den vier wohlbekannten Wnden. ber dem Tisch, auf dem eine auseinandergepackte Musterkollektion von Tuchwaren ausgebreitet war - Samsa war Reisender - hing das Bild, das er vor kurzem aus einer illustrierten Zeitschrift ausgeschnitten und in einem hbschen, vergoldeten Rahmen untergebracht hatte. Es stellte eine Dame dar, die mit einem Pelzhut und einer Pelzboa versehen, aufrecht dasa und einen schweren Pelzmuff, in dem ihr ganzer Unterarm verschwunden war, dem Beschauer entgegenhob. Gregors Blick richtete sich dann zum Fenster, und das trbe Wetter - man hrte Regentropfen auf das Fensterblech aufschlagen - machte ihn ganz melancholisch. Wie wre es, wenn ich noch ein wenig weiterschliefe und alle Narrheiten verge, dachte er, aber das war gnzlich undurchfhrbar, denn er war gewhnt, auf der rechten Seite zu schlafen, konnte sich aber in seinem gegenwrtigen Zustand nicht in diese Lage bringen. Mit welcher Kraft er sich auch auf die rechte Seite warf, immer wieder schaukelte er in die Rckenlage zurck. Er versuchte es wohl hundertmal, schlo die Augen, um die zappelnden Beine nicht sehen zu mssen, und lie erst ab, als er in der Seite einen noch nie gefhlten, leichten, dumpfen Schmerz zu fhlen begann. Ach Gott, dachte er, was fr einen anstrengenden Beruf habe ich gewhlt! Tag aus, Tag ein auf der Reise. Die geschftlichen Aufregungen sind viel grer, als im eigentlichen Geschft zu Hause, und auerdem ist mir noch diese Plage des Reisens auferlegt, die Sorgen um die Zuganschlsse, das unregelmige, schlechte Essen, ein immer wechselnder, nie andauernder, nie herzlich werdender menschlicher Verkehr. Der Teufel soll das alles holen! Er fhlte ein leichtes Jucken oben auf dem Bauch; schob sich auf dem Rcken langsam nher zum Bettpfosten, um den Kopf besser heben zu knnen; fand die juckende Stelle, die mit lauter kleinen weien Pnktchen besetzt war, die er nicht zu beurteilen verstand; und wollte mit einem Bein die Stelle betasten, zog es aber gleich zurck, denn bei der Berhrung umwehten ihn Klteschauer. Er glitt wieder in seine frhere Lage zurck. Dies frhzeitige Aufstehen, dachte er, macht einen ganz bldsinnig. Der Mensch mu seinen Schlaf haben. Andere Reisende leben wie Haremsfrauen. Wenn ich zum Beispiel im Laufe des Vormittags ins Gasthaus zurckgehe, um die erlangten Auftrge zu berschreiben, sitzen diese Herren erst beim Frhstck. Das sollte ich bei meinem Chef versuchen; ich wrde auf der Stelle hinausfliegen. Wer wei brigens, ob das nicht sehr gut fr mich wre. Wenn ich mich nicht wegen meiner Eltern zurckhielte, ich htte lngst gekndigt, ich wre vor den Chef hin getreten und htte ihm meine Meinung von Grund des Herzens aus gesagt. Vom Pult htte er fallen mssen! Es ist auch eine sonderbare Art, sich auf das Pult zu setzen und von der Hhe herab mit dem Angestellten zu reden, der berdies wegen der Schwerhrigkeit des Chefs ganz nahe herantreten mu. Nun, die Hoffnung ist noch nicht gnzlich aufgegeben; habe ich einmal das Geld beisammen, um die Schuld der Eltern an ihn abzuzahlen - es drfte noch fnf bis sechs Jahre dauern - , mache ich die Sache unbedingt. Dann wird der groe Schnitt gemacht. Vorlufig allerdings mu ich aufstehen, denn mein Zug fhrt um fnf. Und er sah zur Weckuhr hinber, die auf dem Kasten tickte. Himmlischer Vater!, dachte er. Es war halb sieben Uhr, und die Zeiger gingen ruhig vorwrts, es war sogar halb vorber, es nherte sich schon dreiviertel. Sollte der Wecker nicht gelutet haben? Man sah vom Bett aus, da er auf vier Uhr richtig eingestellt war; gewi hatte er auch gelutet. Ja, aber war es mglich, dieses mbelerschtternde Luten ruhig zu verschlafen? Nun, ruhig hatte er ja nicht geschlafen, aber wahrscheinlich desto fester. Was aber sollte er jetzt tun? Der nchste Zug ging um sieben Uhr; um den einzuholen, htte er sich unsinnig beeilen mssen, und die Kollektion war noch nicht eingepackt, und er selbst fhlte sich durchaus nicht besonders frisch und beweglich. Und selbst wenn er den Zug einholte, ein Donnerwetter des Chefs war nicht zu vermeiden, denn der Geschftsdiener hatte beim Fnfuhrzug gewartet und die Meldung von seiner Versumnis lngst erstattet. Es war eine Kreatur des Chefs, ohne Rckgrat und Verstand. Wie nun, wenn er sich krank meldete? Das wre aber uerst peinlich und verdchtig, denn Gregor war whrend seines fnfjhrigen Dienstes noch nicht einmal krank gewesen. Gewi wrde der Chef mit dem Krankenkassenarzt kommen, wrde den Eltern wegen des faulen Sohnes Vorwrfe machen und alle Einwnde durch den Hinweis auf den Krankenkassenarzt abschneiden, fr den es ja berhaupt nur ganz gesunde, aber arbeitsscheue Menschen gibt. Und htte er brigens in diesem Falle so ganz unrecht? Gregor fhlte sich tatschlich, abgesehen von einer nach dem langen Schlaf wirklich berflssigen Schlfrigkeit, ganz wohl und hatte sogar einen besonders krftigen Hunger. flexi-streams-1.0.7/test/kafka_utf8_cr.txt 0000664 0000000 0000000 00000013154 11175756704 0020552 0 ustar 00root root 0000000 0000000 Als Gregor Samsa eines Morgens aus unruhigen Träumen erwachte, fand er sich in seinem Bett zu einem ungeheueren Ungeziefer verwandelt. Er lag auf seinem panzerartig harten Rücken und sah, wenn er den Kopf ein wenig hob, seinen gewölbten, braunen, von bogenförmigen Versteifungen geteilten Bauch, auf dessen Höhe sich die Bettdecke, zum gänzlichen Niedergleiten bereit, kaum noch erhalten konnte. Seine vielen, im Vergleich zu seinem sonstigen Umfang kläglich dünnen Beine flimmerten ihm hilflos vor den Augen. »Was ist mit mir geschehen?«, dachte er. Es war kein Traum. Sein Zimmer, ein richtiges, nur etwas zu kleines Menschenzimmer, lag ruhig zwischen den vier wohlbekannten Wänden. Über dem Tisch, auf dem eine auseinandergepackte Musterkollektion von Tuchwaren ausgebreitet war - Samsa war Reisender - hing das Bild, das er vor kurzem aus einer illustrierten Zeitschrift ausgeschnitten und in einem hübschen, vergoldeten Rahmen untergebracht hatte. Es stellte eine Dame dar, die mit einem Pelzhut und einer Pelzboa versehen, aufrecht dasaß und einen schweren Pelzmuff, in dem ihr ganzer Unterarm verschwunden war, dem Beschauer entgegenhob. Gregors Blick richtete sich dann zum Fenster, und das trübe Wetter - man hörte Regentropfen auf das Fensterblech aufschlagen - machte ihn ganz melancholisch. »Wie wäre es, wenn ich noch ein wenig weiterschliefe und alle Narrheiten vergäße«, dachte er, aber das war gänzlich undurchführbar, denn er war gewöhnt, auf der rechten Seite zu schlafen, konnte sich aber in seinem gegenwärtigen Zustand nicht in diese Lage bringen. Mit welcher Kraft er sich auch auf die rechte Seite warf, immer wieder schaukelte er in die Rückenlage zurück. Er versuchte es wohl hundertmal, schloß die Augen, um die zappelnden Beine nicht sehen zu müssen, und ließ erst ab, als er in der Seite einen noch nie gefühlten, leichten, dumpfen Schmerz zu fühlen begann. »Ach Gott«, dachte er, »was für einen anstrengenden Beruf habe ich gewählt! Tag aus, Tag ein auf der Reise. Die geschäftlichen Aufregungen sind viel größer, als im eigentlichen Geschäft zu Hause, und außerdem ist mir noch diese Plage des Reisens auferlegt, die Sorgen um die Zuganschlüsse, das unregelmäßige, schlechte Essen, ein immer wechselnder, nie andauernder, nie herzlich werdender menschlicher Verkehr. Der Teufel soll das alles holen!« Er fühlte ein leichtes Jucken oben auf dem Bauch; schob sich auf dem Rücken langsam näher zum Bettpfosten, um den Kopf besser heben zu können; fand die juckende Stelle, die mit lauter kleinen weißen Pünktchen besetzt war, die er nicht zu beurteilen verstand; und wollte mit einem Bein die Stelle betasten, zog es aber gleich zurück, denn bei der Berührung umwehten ihn Kälteschauer. Er glitt wieder in seine frühere Lage zurück. »Dies frühzeitige Aufstehen«, dachte er, »macht einen ganz blödsinnig. Der Mensch muß seinen Schlaf haben. Andere Reisende leben wie Haremsfrauen. Wenn ich zum Beispiel im Laufe des Vormittags ins Gasthaus zurückgehe, um die erlangten Aufträge zu überschreiben, sitzen diese Herren erst beim Frühstück. Das sollte ich bei meinem Chef versuchen; ich würde auf der Stelle hinausfliegen. Wer weiß übrigens, ob das nicht sehr gut für mich wäre. Wenn ich mich nicht wegen meiner Eltern zurückhielte, ich hätte längst gekündigt, ich wäre vor den Chef hin getreten und hätte ihm meine Meinung von Grund des Herzens aus gesagt. Vom Pult hätte er fallen müssen! Es ist auch eine sonderbare Art, sich auf das Pult zu setzen und von der Höhe herab mit dem Angestellten zu reden, der überdies wegen der Schwerhörigkeit des Chefs ganz nahe herantreten muß. Nun, die Hoffnung ist noch nicht gänzlich aufgegeben; habe ich einmal das Geld beisammen, um die Schuld der Eltern an ihn abzuzahlen - es dürfte noch fünf bis sechs Jahre dauern - , mache ich die Sache unbedingt. Dann wird der große Schnitt gemacht. Vorläufig allerdings muß ich aufstehen, denn mein Zug fährt um fünf.« Und er sah zur Weckuhr hinüber, die auf dem Kasten tickte. »Himmlischer Vater!«, dachte er. Es war halb sieben Uhr, und die Zeiger gingen ruhig vorwärts, es war sogar halb vorüber, es näherte sich schon dreiviertel. Sollte der Wecker nicht geläutet haben? Man sah vom Bett aus, daß er auf vier Uhr richtig eingestellt war; gewiß hatte er auch geläutet. Ja, aber war es möglich, dieses möbelerschütternde Läuten ruhig zu verschlafen? Nun, ruhig hatte er ja nicht geschlafen, aber wahrscheinlich desto fester. Was aber sollte er jetzt tun? Der nächste Zug ging um sieben Uhr; um den einzuholen, hätte er sich unsinnig beeilen müssen, und die Kollektion war noch nicht eingepackt, und er selbst fühlte sich durchaus nicht besonders frisch und beweglich. Und selbst wenn er den Zug einholte, ein Donnerwetter des Chefs war nicht zu vermeiden, denn der Geschäftsdiener hatte beim Fünfuhrzug gewartet und die Meldung von seiner Versäumnis längst erstattet. Es war eine Kreatur des Chefs, ohne Rückgrat und Verstand. Wie nun, wenn er sich krank meldete? Das wäre aber äußerst peinlich und verdächtig, denn Gregor war während seines fünfjährigen Dienstes noch nicht einmal krank gewesen. Gewiß würde der Chef mit dem Krankenkassenarzt kommen, würde den Eltern wegen des faulen Sohnes Vorwürfe machen und alle Einwände durch den Hinweis auf den Krankenkassenarzt abschneiden, für den es ja überhaupt nur ganz gesunde, aber arbeitsscheue Menschen gibt. Und hätte er übrigens in diesem Falle so ganz unrecht? Gregor fühlte sich tatsächlich, abgesehen von einer nach dem langen Schlaf wirklich überflüssigen Schläfrigkeit, ganz wohl und hatte sogar einen besonders kräftigen Hunger. flexi-streams-1.0.7/test/kafka_utf8_crlf.txt 0000664 0000000 0000000 00000013167 11175756704 0021100 0 ustar 00root root 0000000 0000000 Als Gregor Samsa eines Morgens aus unruhigen Träumen erwachte, fand er sich in seinem Bett zu einem ungeheueren Ungeziefer verwandelt. Er lag auf seinem panzerartig harten Rücken und sah, wenn er den Kopf ein wenig hob, seinen gewölbten, braunen, von bogenförmigen Versteifungen geteilten Bauch, auf dessen Höhe sich die Bettdecke, zum gänzlichen Niedergleiten bereit, kaum noch erhalten konnte. Seine vielen, im Vergleich zu seinem sonstigen Umfang kläglich dünnen Beine flimmerten ihm hilflos vor den Augen. »Was ist mit mir geschehen?«, dachte er. Es war kein Traum. Sein Zimmer, ein richtiges, nur etwas zu kleines Menschenzimmer, lag ruhig zwischen den vier wohlbekannten Wänden. Über dem Tisch, auf dem eine auseinandergepackte Musterkollektion von Tuchwaren ausgebreitet war - Samsa war Reisender - hing das Bild, das er vor kurzem aus einer illustrierten Zeitschrift ausgeschnitten und in einem hübschen, vergoldeten Rahmen untergebracht hatte. Es stellte eine Dame dar, die mit einem Pelzhut und einer Pelzboa versehen, aufrecht dasaß und einen schweren Pelzmuff, in dem ihr ganzer Unterarm verschwunden war, dem Beschauer entgegenhob. Gregors Blick richtete sich dann zum Fenster, und das trübe Wetter - man hörte Regentropfen auf das Fensterblech aufschlagen - machte ihn ganz melancholisch. »Wie wäre es, wenn ich noch ein wenig weiterschliefe und alle Narrheiten vergäße«, dachte er, aber das war gänzlich undurchführbar, denn er war gewöhnt, auf der rechten Seite zu schlafen, konnte sich aber in seinem gegenwärtigen Zustand nicht in diese Lage bringen. Mit welcher Kraft er sich auch auf die rechte Seite warf, immer wieder schaukelte er in die Rückenlage zurück. Er versuchte es wohl hundertmal, schloß die Augen, um die zappelnden Beine nicht sehen zu müssen, und ließ erst ab, als er in der Seite einen noch nie gefühlten, leichten, dumpfen Schmerz zu fühlen begann. »Ach Gott«, dachte er, »was für einen anstrengenden Beruf habe ich gewählt! Tag aus, Tag ein auf der Reise. Die geschäftlichen Aufregungen sind viel größer, als im eigentlichen Geschäft zu Hause, und außerdem ist mir noch diese Plage des Reisens auferlegt, die Sorgen um die Zuganschlüsse, das unregelmäßige, schlechte Essen, ein immer wechselnder, nie andauernder, nie herzlich werdender menschlicher Verkehr. Der Teufel soll das alles holen!« Er fühlte ein leichtes Jucken oben auf dem Bauch; schob sich auf dem Rücken langsam näher zum Bettpfosten, um den Kopf besser heben zu können; fand die juckende Stelle, die mit lauter kleinen weißen Pünktchen besetzt war, die er nicht zu beurteilen verstand; und wollte mit einem Bein die Stelle betasten, zog es aber gleich zurück, denn bei der Berührung umwehten ihn Kälteschauer. Er glitt wieder in seine frühere Lage zurück. »Dies frühzeitige Aufstehen«, dachte er, »macht einen ganz blödsinnig. Der Mensch muß seinen Schlaf haben. Andere Reisende leben wie Haremsfrauen. Wenn ich zum Beispiel im Laufe des Vormittags ins Gasthaus zurückgehe, um die erlangten Aufträge zu überschreiben, sitzen diese Herren erst beim Frühstück. Das sollte ich bei meinem Chef versuchen; ich würde auf der Stelle hinausfliegen. Wer weiß übrigens, ob das nicht sehr gut für mich wäre. Wenn ich mich nicht wegen meiner Eltern zurückhielte, ich hätte längst gekündigt, ich wäre vor den Chef hin getreten und hätte ihm meine Meinung von Grund des Herzens aus gesagt. Vom Pult hätte er fallen müssen! Es ist auch eine sonderbare Art, sich auf das Pult zu setzen und von der Höhe herab mit dem Angestellten zu reden, der überdies wegen der Schwerhörigkeit des Chefs ganz nahe herantreten muß. Nun, die Hoffnung ist noch nicht gänzlich aufgegeben; habe ich einmal das Geld beisammen, um die Schuld der Eltern an ihn abzuzahlen - es dürfte noch fünf bis sechs Jahre dauern - , mache ich die Sache unbedingt. Dann wird der große Schnitt gemacht. Vorläufig allerdings muß ich aufstehen, denn mein Zug fährt um fünf.« Und er sah zur Weckuhr hinüber, die auf dem Kasten tickte. »Himmlischer Vater!«, dachte er. Es war halb sieben Uhr, und die Zeiger gingen ruhig vorwärts, es war sogar halb vorüber, es näherte sich schon dreiviertel. Sollte der Wecker nicht geläutet haben? Man sah vom Bett aus, daß er auf vier Uhr richtig eingestellt war; gewiß hatte er auch geläutet. Ja, aber war es möglich, dieses möbelerschütternde Läuten ruhig zu verschlafen? Nun, ruhig hatte er ja nicht geschlafen, aber wahrscheinlich desto fester. Was aber sollte er jetzt tun? Der nächste Zug ging um sieben Uhr; um den einzuholen, hätte er sich unsinnig beeilen müssen, und die Kollektion war noch nicht eingepackt, und er selbst fühlte sich durchaus nicht besonders frisch und beweglich. Und selbst wenn er den Zug einholte, ein Donnerwetter des Chefs war nicht zu vermeiden, denn der Geschäftsdiener hatte beim Fünfuhrzug gewartet und die Meldung von seiner Versäumnis längst erstattet. Es war eine Kreatur des Chefs, ohne Rückgrat und Verstand. Wie nun, wenn er sich krank meldete? Das wäre aber äußerst peinlich und verdächtig, denn Gregor war während seines fünfjährigen Dienstes noch nicht einmal krank gewesen. Gewiß würde der Chef mit dem Krankenkassenarzt kommen, würde den Eltern wegen des faulen Sohnes Vorwürfe machen und alle Einwände durch den Hinweis auf den Krankenkassenarzt abschneiden, für den es ja überhaupt nur ganz gesunde, aber arbeitsscheue Menschen gibt. Und hätte er übrigens in diesem Falle so ganz unrecht? Gregor fühlte sich tatsächlich, abgesehen von einer nach dem langen Schlaf wirklich überflüssigen Schläfrigkeit, ganz wohl und hatte sogar einen besonders kräftigen Hunger. flexi-streams-1.0.7/test/kafka_utf8_lf.txt 0000664 0000000 0000000 00000013154 11175756704 0020547 0 ustar 00root root 0000000 0000000 Als Gregor Samsa eines Morgens aus unruhigen Träumen erwachte, fand er sich in seinem Bett zu einem ungeheueren Ungeziefer verwandelt. Er lag auf seinem panzerartig harten Rücken und sah, wenn er den Kopf ein wenig hob, seinen gewölbten, braunen, von bogenförmigen Versteifungen geteilten Bauch, auf dessen Höhe sich die Bettdecke, zum gänzlichen Niedergleiten bereit, kaum noch erhalten konnte. Seine vielen, im Vergleich zu seinem sonstigen Umfang kläglich dünnen Beine flimmerten ihm hilflos vor den Augen. »Was ist mit mir geschehen?«, dachte er. Es war kein Traum. Sein Zimmer, ein richtiges, nur etwas zu kleines Menschenzimmer, lag ruhig zwischen den vier wohlbekannten Wänden. Über dem Tisch, auf dem eine auseinandergepackte Musterkollektion von Tuchwaren ausgebreitet war - Samsa war Reisender - hing das Bild, das er vor kurzem aus einer illustrierten Zeitschrift ausgeschnitten und in einem hübschen, vergoldeten Rahmen untergebracht hatte. Es stellte eine Dame dar, die mit einem Pelzhut und einer Pelzboa versehen, aufrecht dasaß und einen schweren Pelzmuff, in dem ihr ganzer Unterarm verschwunden war, dem Beschauer entgegenhob. Gregors Blick richtete sich dann zum Fenster, und das trübe Wetter - man hörte Regentropfen auf das Fensterblech aufschlagen - machte ihn ganz melancholisch. »Wie wäre es, wenn ich noch ein wenig weiterschliefe und alle Narrheiten vergäße«, dachte er, aber das war gänzlich undurchführbar, denn er war gewöhnt, auf der rechten Seite zu schlafen, konnte sich aber in seinem gegenwärtigen Zustand nicht in diese Lage bringen. Mit welcher Kraft er sich auch auf die rechte Seite warf, immer wieder schaukelte er in die Rückenlage zurück. Er versuchte es wohl hundertmal, schloß die Augen, um die zappelnden Beine nicht sehen zu müssen, und ließ erst ab, als er in der Seite einen noch nie gefühlten, leichten, dumpfen Schmerz zu fühlen begann. »Ach Gott«, dachte er, »was für einen anstrengenden Beruf habe ich gewählt! Tag aus, Tag ein auf der Reise. Die geschäftlichen Aufregungen sind viel größer, als im eigentlichen Geschäft zu Hause, und außerdem ist mir noch diese Plage des Reisens auferlegt, die Sorgen um die Zuganschlüsse, das unregelmäßige, schlechte Essen, ein immer wechselnder, nie andauernder, nie herzlich werdender menschlicher Verkehr. Der Teufel soll das alles holen!« Er fühlte ein leichtes Jucken oben auf dem Bauch; schob sich auf dem Rücken langsam näher zum Bettpfosten, um den Kopf besser heben zu können; fand die juckende Stelle, die mit lauter kleinen weißen Pünktchen besetzt war, die er nicht zu beurteilen verstand; und wollte mit einem Bein die Stelle betasten, zog es aber gleich zurück, denn bei der Berührung umwehten ihn Kälteschauer. Er glitt wieder in seine frühere Lage zurück. »Dies frühzeitige Aufstehen«, dachte er, »macht einen ganz blödsinnig. Der Mensch muß seinen Schlaf haben. Andere Reisende leben wie Haremsfrauen. Wenn ich zum Beispiel im Laufe des Vormittags ins Gasthaus zurückgehe, um die erlangten Aufträge zu überschreiben, sitzen diese Herren erst beim Frühstück. Das sollte ich bei meinem Chef versuchen; ich würde auf der Stelle hinausfliegen. Wer weiß übrigens, ob das nicht sehr gut für mich wäre. Wenn ich mich nicht wegen meiner Eltern zurückhielte, ich hätte längst gekündigt, ich wäre vor den Chef hin getreten und hätte ihm meine Meinung von Grund des Herzens aus gesagt. Vom Pult hätte er fallen müssen! Es ist auch eine sonderbare Art, sich auf das Pult zu setzen und von der Höhe herab mit dem Angestellten zu reden, der überdies wegen der Schwerhörigkeit des Chefs ganz nahe herantreten muß. Nun, die Hoffnung ist noch nicht gänzlich aufgegeben; habe ich einmal das Geld beisammen, um die Schuld der Eltern an ihn abzuzahlen - es dürfte noch fünf bis sechs Jahre dauern - , mache ich die Sache unbedingt. Dann wird der große Schnitt gemacht. Vorläufig allerdings muß ich aufstehen, denn mein Zug fährt um fünf.« Und er sah zur Weckuhr hinüber, die auf dem Kasten tickte. »Himmlischer Vater!«, dachte er. Es war halb sieben Uhr, und die Zeiger gingen ruhig vorwärts, es war sogar halb vorüber, es näherte sich schon dreiviertel. Sollte der Wecker nicht geläutet haben? Man sah vom Bett aus, daß er auf vier Uhr richtig eingestellt war; gewiß hatte er auch geläutet. Ja, aber war es möglich, dieses möbelerschütternde Läuten ruhig zu verschlafen? Nun, ruhig hatte er ja nicht geschlafen, aber wahrscheinlich desto fester. Was aber sollte er jetzt tun? Der nächste Zug ging um sieben Uhr; um den einzuholen, hätte er sich unsinnig beeilen müssen, und die Kollektion war noch nicht eingepackt, und er selbst fühlte sich durchaus nicht besonders frisch und beweglich. Und selbst wenn er den Zug einholte, ein Donnerwetter des Chefs war nicht zu vermeiden, denn der Geschäftsdiener hatte beim Fünfuhrzug gewartet und die Meldung von seiner Versäumnis längst erstattet. Es war eine Kreatur des Chefs, ohne Rückgrat und Verstand. Wie nun, wenn er sich krank meldete? Das wäre aber äußerst peinlich und verdächtig, denn Gregor war während seines fünfjährigen Dienstes noch nicht einmal krank gewesen. Gewiß würde der Chef mit dem Krankenkassenarzt kommen, würde den Eltern wegen des faulen Sohnes Vorwürfe machen und alle Einwände durch den Hinweis auf den Krankenkassenarzt abschneiden, für den es ja überhaupt nur ganz gesunde, aber arbeitsscheue Menschen gibt. Und hätte er übrigens in diesem Falle so ganz unrecht? Gregor fühlte sich tatsächlich, abgesehen von einer nach dem langen Schlaf wirklich überflüssigen Schläfrigkeit, ganz wohl und hatte sogar einen besonders kräftigen Hunger. flexi-streams-1.0.7/test/packages.lisp 0000664 0000000 0000000 00000003413 11175756704 0017746 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: CL-USER; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/test/packages.lisp,v 1.8 2008/08/01 10:12:43 edi Exp $ ;;; Copyright (c) 2006-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :cl-user) (defpackage :flexi-streams-test (:use :cl :flexi-streams) (:import-from :flexi-streams :with-unique-names :with-rebinding :char* :normalize-external-format) (:export :run-all-tests)) flexi-streams-1.0.7/test/russian_koi8r_cr.txt 0000664 0000000 0000000 00000000627 11175756704 0021330 0 ustar 00root root 0000000 0000000 Unicode, 10-12 1997 . Unicode, , Unicode , , . flexi-streams-1.0.7/test/russian_koi8r_crlf.txt 0000664 0000000 0000000 00000000635 11175756704 0021651 0 ustar 00root root 0000000 0000000 Unicode, 10-12 1997 . Unicode, , Unicode , , . flexi-streams-1.0.7/test/russian_koi8r_lf.txt 0000664 0000000 0000000 00000000627 11175756704 0021325 0 ustar 00root root 0000000 0000000 Unicode, 10-12 1997 . Unicode, , Unicode , , . flexi-streams-1.0.7/test/russian_utf8_cr.txt 0000664 0000000 0000000 00000001327 11175756704 0021160 0 ustar 00root root 0000000 0000000 Зарегистрируйтесь сейчас на Десятую Международную Конференцию по Unicode, которая состоится 10-12 марта 1997 года в Майнце в Германии. Конференция соберет широкий круг экспертов по вопросам глобального Интернета и Unicode, локализации и интернационализации, воплощению и применению Unicode в различных операционных системах и программных приложениях, шрифтах, верстке и многоязычных компьютерных системах. flexi-streams-1.0.7/test/russian_utf8_crlf.txt 0000664 0000000 0000000 00000001335 11175756704 0021501 0 ustar 00root root 0000000 0000000 Зарегистрируйтесь сейчас на Десятую Международную Конференцию по Unicode, которая состоится 10-12 марта 1997 года в Майнце в Германии. Конференция соберет широкий круг экспертов по вопросам глобального Интернета и Unicode, локализации и интернационализации, воплощению и применению Unicode в различных операционных системах и программных приложениях, шрифтах, верстке и многоязычных компьютерных системах. flexi-streams-1.0.7/test/russian_utf8_lf.txt 0000664 0000000 0000000 00000001327 11175756704 0021155 0 ustar 00root root 0000000 0000000 Зарегистрируйтесь сейчас на Десятую Международную Конференцию по Unicode, которая состоится 10-12 марта 1997 года в Майнце в Германии. Конференция соберет широкий круг экспертов по вопросам глобального Интернета и Unicode, локализации и интернационализации, воплощению и применению Unicode в различных операционных системах и программных приложениях, шрифтах, верстке и многоязычных компьютерных системах. flexi-streams-1.0.7/test/test.lisp 0000664 0000000 0000000 00000110121 11175756704 0017142 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS-TEST; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/test/test.lisp,v 1.40 2008/07/25 09:56:12 edi Exp $ ;;; Copyright (c) 2006-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams-test) (defmacro with-test-suite ((test-description &key show-progress-p) &body body) "Defines a test suite. Three utilities are available inside of the body of the macro: The function FAIL, and the macros CHECK and WITH-EXPECTED-ERROR. FAIL, the lowest level utility, marks the test defined by WITH-TEST-SUITE as failed. CHECK checks whether its argument is true, otherwise it calls FAIL. If during evaluation of the specified expression any condition is signalled, this is also considered a failure. WITH-EXPECTED-ERROR executes its body and considers the test a success if the specified error was signalled, otherwise it calls FAIL. WITH-TEST-SUITE prints a simple progress report if SHOW-PROGRESS-P is true." (with-unique-names (successp testcount) (with-rebinding (show-progress-p) `(let ((,successp t) (,testcount 1)) (when (and ,show-progress-p (not (numberp ,show-progress-p))) (setq ,show-progress-p 1)) (flet ((fail (format-str &rest format-args) (apply #'format t format-str format-args) (setq ,successp nil)) (maybe-show-progress () (when (and ,show-progress-p (zerop (mod ,testcount ,show-progress-p))) (format t ".") (when (zerop (mod ,testcount (* 10 ,show-progress-p))) (terpri)) (force-output)) (incf ,testcount))) (macrolet ((check (expression) `(progn (maybe-show-progress) (handler-case (unless ,expression (fail "~&Test ~S failed.~%" ',expression)) (error (c) (fail "~&Test ~S failed signalling error of type ~A: ~A.~%" ',expression (type-of c) c))))) (with-expected-error ((condition-type) &body body) `(progn (maybe-show-progress) (handler-case (progn ,@body) (,condition-type () t) (:no-error (&rest args) (declare (ignore args)) (fail "~&Expected condition ~S not signalled.~%" ',condition-type)))))) (format t "~&Test suite: ~S~%" ,test-description) ,@body)) ,successp)))) ;; LW can't indent this correctly because it's in a MACROLET #+:lispworks (editor:setup-indent "with-expected-error" 1 2 4) (defconstant +buffer-size+ 8192 "Size of buffers for COPY-STREAM* below.") (defvar *copy-function* nil "Which function to use when copying from one stream to the other - see for example COPY-FILE below.") (defvar *this-file* (load-time-value (or #.*compile-file-pathname* *load-pathname*)) "The pathname of the file \(`test.lisp') where this variable was defined.") #+:lispworks (defun get-env-variable-as-directory (name) (lw:when-let (string (lw:environment-variable name)) (when (plusp (length string)) (cond ((find (char string (1- (length string))) "\\/" :test #'char=) string) (t (lw:string-append string "/")))))) (defvar *tmp-dir* (load-time-value (merge-pathnames "odd-streams-test/" #+:allegro (system:temporary-directory) #+:lispworks (pathname (or (get-env-variable-as-directory "TEMP") (get-env-variable-as-directory "TMP") #+:win32 "C:/" #-:win32 "/tmp/")) #-(or :allegro :lispworks) #p"/tmp/")) "The pathname of a temporary directory used for testing.") (defvar *test-files* '(("kafka" (:utf8 :latin1 :cp1252)) ("tilton" (:utf8 :ascii)) ("hebrew" (:utf8 :latin8)) ("russian" (:utf8 :koi8r)) ("unicode_demo" (:utf8 :ucs2 :ucs4))) "A list of test files where each entry consists of the name prefix and a list of encodings.") (defun create-file-variants (file-name symbol) "For a name suffix FILE-NAME and a symbol SYMBOL denoting an encoding returns a list of pairs where the car is a full file name and the cdr is the corresponding external format. This list contains all possible variants w.r.t. to line-end conversion and endianness." (let ((args (ecase symbol (:ascii '(:ascii)) (:latin1 '(:latin-1)) (:latin8 '(:hebrew)) (:cp1252 '(:code-page :id 1252)) (:koi8r '(:koi8-r)) (:utf8 '(:utf-8)) (:ucs2 '(:utf-16)) (:ucs4 '(:utf-32)))) (endianp (member symbol '(:ucs2 :ucs4)))) (loop for little-endian in (if endianp '(t nil) '(t)) for endian-suffix in (if endianp '("_le" "_be") '("")) nconc (loop for eol-style in '(:lf :cr :crlf) collect (cons (format nil "~A_~(~A~)_~(~A~)~A.txt" file-name symbol eol-style endian-suffix) (apply #'make-external-format (append args `(:eol-style ,eol-style :little-endian ,little-endian)))))))) (defun create-test-combinations (file-name symbols &optional simplep) "For a name suffix FILE-NAME and a list of symbols SYMBOLS denoting different encodings of the corresponding file returns a list of lists which can be used as arglists by COMPARE-FILES. If SIMPLEP is true, a list which can be used for the string and sequence tests below is returned." (let ((file-variants (loop for symbol in symbols nconc (create-file-variants file-name symbol)))) (loop for (name-in . external-format-in) in file-variants when simplep collect (list name-in external-format-in) else nconc (loop for (name-out . external-format-out) in file-variants collect (list name-in external-format-in name-out external-format-out))))) (defun file-equal (file1 file2) "Returns a true value iff FILE1 and FILE2 have the same contents \(viewed as binary files)." (with-open-file (stream1 file1 :element-type 'octet) (with-open-file (stream2 file2 :element-type 'octet) (and (= (file-length stream1) (file-length stream2)) (loop for byte1 = (read-byte stream1 nil nil) for byte2 = (read-byte stream2 nil nil) while (and byte1 byte2) always (= byte1 byte2)))))) (defun copy-stream (stream-in external-format-in stream-out external-format-out) "Copies the contents of the binary stream STREAM-IN to the binary stream STREAM-OUT using flexi streams - STREAM-IN is read with the external format EXTERNAL-FORMAT-IN and STREAM-OUT is written with EXTERNAL-FORMAT-OUT." (let ((in (make-flexi-stream stream-in :external-format external-format-in)) (out (make-flexi-stream stream-out :external-format external-format-out))) (loop for line = (read-line in nil nil) while line do (write-line line out)))) (defun copy-stream* (stream-in external-format-in stream-out external-format-out) "Like COPY-STREAM, but uses READ-SEQUENCE and WRITE-SEQUENCE instead of READ-LINE and WRITE-LINE." (let ((in (make-flexi-stream stream-in :external-format external-format-in)) (out (make-flexi-stream stream-out :external-format external-format-out)) (buffer (make-array +buffer-size+ :element-type 'char*))) (loop (let ((position (read-sequence buffer in))) (when (zerop position) (return)) (write-sequence buffer out :end position))))) (defun copy-file (path-in external-format-in path-out external-format-out direction-out direction-in) "Copies the contents of the file denoted by the pathname PATH-IN to the file denoted by the pathname PATH-OUT using flexi streams - STREAM-IN is read with the external format EXTERNAL-FORMAT-IN and STREAM-OUT is written with EXTERNAL-FORMAT-OUT. The input file is opened with the :DIRECTION keyword argument DIRECTION-IN, the output file is opened with the :DIRECTION keyword argument DIRECTION-OUT." (with-open-file (in path-in :element-type 'octet :direction direction-in :if-does-not-exist :error :if-exists :overwrite) (with-open-file (out path-out :element-type 'octet :direction direction-out :if-does-not-exist :create :if-exists :supersede) (funcall *copy-function* in external-format-in out external-format-out)))) #+:lispworks (defun copy-file-lw (path-in external-format-in path-out external-format-out direction-out direction-in) "Same as COPY-FILE, but uses character streams instead of binary streams. Only used to test LispWorks-specific behaviour." (with-open-file (in path-in :external-format '(:latin-1 :eol-style :lf) :element-type 'base-char :direction direction-in :if-does-not-exist :error :if-exists :overwrite) (with-open-file (out path-out :external-format '(:latin-1 :eol-style :lf) :element-type 'base-char :direction direction-out :direction :output :if-does-not-exist :create :if-exists :supersede) (funcall *copy-function* in external-format-in out external-format-out)))) (defun compare-files (&key verbose) "Each test in this suite copies the contents of one file \(in the `test' directory) to another file \(in a temporary directory) using flexi streams with different external formats. The resulting file is compared with an existing file in the `test' directory to check if the outcome is as expected. Uses various variants of the :DIRECTION keyword when opening the files. Returns a true value iff all tests succeeded. Prints information about each individual comparison if VERBOSE is true." (with-test-suite ("Reading/writing files" :show-progress-p (not verbose)) (flet ((one-comparison (path-in external-format-in path-out external-format-out verbose) (when verbose (format t "~&File ~S, using copy function ~S" (file-namestring path-in) *copy-function*) (format t "~& and external formats ~S --> ~S" (normalize-external-format external-format-in) (normalize-external-format external-format-out))) (let ((full-path-in (merge-pathnames path-in *this-file*)) (full-path-out (ensure-directories-exist (merge-pathnames path-out *tmp-dir*))) (full-path-orig (merge-pathnames path-out *this-file*))) (dolist (direction-out '(:output :io)) (dolist (direction-in '(:input :io)) (when verbose (format t "~&...directions ~S --> ~S" direction-in direction-out)) (copy-file full-path-in external-format-in full-path-out external-format-out direction-out direction-in) (check (file-equal full-path-out full-path-orig)) #+:lispworks (progn (when verbose (format t "~&...directions ~S --> ~S \(LispWorks)" direction-in direction-out)) (copy-file-lw full-path-in external-format-in full-path-out external-format-out direction-out direction-in) (check (file-equal full-path-out full-path-orig)))))))) (loop with compare-files-args-list = (loop for (file-name symbols) in *test-files* nconc (create-test-combinations file-name symbols)) for *copy-function* in '(copy-stream copy-stream*) do (loop for (path-in external-format-in path-out external-format-out) in compare-files-args-list do (one-comparison path-in external-format-in path-out external-format-out verbose)))))) (defun file-as-octet-vector (pathspec) "Returns the contents of the file denoted by PATHSPEC as a vector of octets." (with-open-file (in pathspec :element-type 'octet) (let ((vector (make-array (file-length in) :element-type 'octet))) (read-sequence vector in) vector))) (defun file-as-string (pathspec external-format) "Reads the contents of the file denoted by PATHSPEC using the external format EXTERNAL-FORMAT and returns the result as a string." (with-open-file (in pathspec :element-type 'octet) (let* ((number-of-octets (file-length in)) (in (make-flexi-stream in :external-format external-format)) (string (make-array number-of-octets :element-type #+:lispworks 'lw:simple-char #-:lispworks 'character :fill-pointer t))) (setf (fill-pointer string) (read-sequence string in)) string))) (defun old-string-to-octets (string &key (external-format (make-external-format :latin1)) (start 0) end) "The old version of STRING-TO-OCTETS. We can use it to test in-memory streams." (declare (optimize speed)) (with-output-to-sequence (out) (let ((flexi (make-flexi-stream out :external-format external-format))) (write-string string flexi :start start :end end)))) (defun old-octets-to-string (vector &key (external-format (make-external-format :latin1)) (start 0) (end (length vector))) "The old version of OCTETS-TO-STRING. We can use it to test in-memory streams." (declare (optimize speed)) (with-input-from-sequence (in vector :start start :end end) (let ((flexi (make-flexi-stream in :external-format external-format)) (result (make-array (- end start) :element-type #+:lispworks 'lw:simple-char #-:lispworks 'character :fill-pointer t))) (setf (fill-pointer result) (read-sequence result flexi)) result))) (defun string-tests (&key verbose) "Tests whether conversion from strings to octets and vice versa works as expected. Also tests with the old versions of the conversion functions in order to test in-memory streams." (with-test-suite ("String tests" :show-progress-p (and (not verbose) 10)) (flet ((one-string-test (pathspec external-format verbose) (when verbose (format t "~&With external format ~S:" (normalize-external-format external-format))) (let* ((full-path (merge-pathnames pathspec *this-file*)) (octets-vector (file-as-octet-vector full-path)) (octets-list (coerce octets-vector 'list)) (string (file-as-string full-path external-format))) (when verbose (format t "~&...testing OCTETS-TO-STRING")) (check (string= (octets-to-string octets-vector :external-format external-format) string)) (check (string= (octets-to-string octets-list :external-format external-format) string)) (when verbose (format t "~&...testing STRING-TO-OCTETS")) (check (equalp (string-to-octets string :external-format external-format) octets-vector)) (when verbose (format t "~&...testing in-memory streams")) (check (string= (old-octets-to-string octets-vector :external-format external-format) string)) (check (string= (old-octets-to-string octets-list :external-format external-format) string)) (check (equalp (old-string-to-octets string :external-format external-format) octets-vector))))) (loop with simple-test-args-list = (loop for (file-name symbols) in *test-files* nconc (create-test-combinations file-name symbols t)) for (pathspec external-format) in simple-test-args-list do (one-string-test pathspec external-format verbose))))) (defun sequence-equal (seq1 seq2) "Whether the two sequences have the same elements." (and (= (length seq1) (length seq2)) (loop for i below (length seq1) always (eql (elt seq1 i) (elt seq2 i))))) (defun sequence-tests (&key verbose) "Several tests to confirm that READ-SEQUENCE and WRITE-SEQUENCE behave as expected." (with-test-suite ("Sequence tests" :show-progress-p (and (not verbose) 10)) (flet ((one-sequence-test (pathspec external-format verbose) (when verbose (format t "~&With external format ~S:" (normalize-external-format external-format))) (let* ((full-path (merge-pathnames pathspec *this-file*)) (file-string (file-as-string full-path external-format)) (string-length (length file-string)) (octets (file-as-octet-vector full-path)) (octet-length (length octets))) (when (external-format-equal external-format (make-external-format :utf8)) (when verbose (format t "~&...reading octets")) #-:openmcl ;; FLEXI-STREAMS puts integers into the list, but OpenMCL ;; thinks they are characters... (with-open-file (in full-path :element-type 'octet) (let* ((in (make-flexi-stream in :external-format external-format)) (list (make-list octet-length))) (setf (flexi-stream-element-type in) 'octet) #-:clisp (read-sequence list in) #+:clisp (ext:read-byte-sequence list in) (check (sequence-equal list octets)))) (with-open-file (in full-path :element-type 'octet) (let* ((in (make-flexi-stream in :external-format external-format)) (third (floor octet-length 3)) (half (floor octet-length 2)) (vector (make-array half :element-type 'octet))) (check (sequence-equal (loop repeat third collect (read-byte in)) (subseq octets 0 third))) (read-sequence vector in) (check (sequence-equal vector (subseq octets third (+ third half))))))) (when verbose (format t "~&...reading characters")) (with-open-file (in full-path :element-type 'octet) (let* ((in (make-flexi-stream in :external-format external-format)) (string (make-string (- string-length 10) :element-type 'char*))) (setf (flexi-stream-element-type in) 'octet) (check (sequence-equal (loop repeat 10 collect (read-char in)) (subseq file-string 0 10))) (read-sequence string in) (check (sequence-equal string (subseq file-string 10))))) (with-open-file (in full-path :element-type 'octet) (let* ((in (make-flexi-stream in :external-format external-format)) (list (make-list (- string-length 100)))) (check (sequence-equal (loop repeat 50 collect (read-char in)) (subseq file-string 0 50))) #-:clisp (read-sequence list in) #+:clisp (ext:read-char-sequence list in) (check (sequence-equal list (subseq file-string 50 (- string-length 50)))) (check (sequence-equal (loop repeat 50 collect (read-char in)) (subseq file-string (- string-length 50)))))) (with-open-file (in full-path :element-type 'octet) (let* ((in (make-flexi-stream in :external-format external-format)) (array (make-array (- string-length 50)))) (check (sequence-equal (loop repeat 25 collect (read-char in)) (subseq file-string 0 25))) #-:clisp (read-sequence array in) #+:clisp (ext:read-char-sequence array in) (check (sequence-equal array (subseq file-string 25 (- string-length 25)))) (check (sequence-equal (loop repeat 25 collect (read-char in)) (subseq file-string (- string-length 25)))))) (let ((path-out (ensure-directories-exist (merge-pathnames pathspec *tmp-dir*)))) (when verbose (format t "~&...writing sequences")) (with-open-file (out path-out :direction :output :if-exists :supersede :element-type 'octet) (let ((out (make-flexi-stream out :external-format external-format))) (write-sequence octets out))) (check (file-equal full-path path-out)) (with-open-file (out path-out :direction :output :if-exists :supersede :element-type 'octet) (let ((out (make-flexi-stream out :external-format external-format))) (write-sequence file-string out))) (check (file-equal full-path path-out)) (with-open-file (out path-out :direction :output :if-exists :supersede :element-type 'octet) (let ((out (make-flexi-stream out :external-format external-format))) (write-sequence file-string out :end 100) (write-sequence octets out :start (length (string-to-octets file-string :external-format external-format :end 100))))) (check (file-equal full-path path-out)))))) (loop with simple-test-args-list = (loop for (file-name symbols) in *test-files* nconc (create-test-combinations file-name symbols t)) for (pathspec external-format) in simple-test-args-list do (one-sequence-test pathspec external-format verbose))))) (defmacro using-values ((&rest values) &body body) "Executes BODY and feeds an element from VALUES to the USE-VALUE restart each time a EXTERNAL-FORMAT-ENCODING-ERROR is signalled. Signals an error when there are more or less EXTERNAL-FORMAT-ENCODING-ERRORs than there are elements in VALUES." (with-unique-names (value-stack condition-counter) `(let ((,value-stack ',values) (,condition-counter 0)) (handler-bind ((external-format-encoding-error #'(lambda (c) (declare (ignore c)) (unless ,value-stack (error "Too many encoding errors signalled, expected only ~A." ,(length values))) (incf ,condition-counter) (use-value (pop ,value-stack))))) (prog1 (progn ,@body) (when ,value-stack (error "~A encoding errors signalled, but ~A were expected." ,condition-counter ,(length values)))))))) (defun accept-overlong (octets code-point) "Converts the `overlong' UTF-8 sequence OCTETS to using OCTETS-TO-STRINGS, accepts the expected error with the corresponding restart and checks that the result is CODE-POINT." (handler-bind ((external-format-encoding-error (lambda (c) (declare (ignore c)) (invoke-restart 'accept-overlong-sequence)))) (string= (octets-to-string octets :external-format :utf-8) (string (code-char code-point))))) (defun read-flexi-line (sequence external-format) "Creates and returns a string from the octet sequence SEQUENCE using the external format EXTERNAL-FORMAT." (with-input-from-sequence (in sequence) (setq in (make-flexi-stream in :external-format external-format)) (read-line in))) (defun read-flexi-line* (sequence external-format) "Like READ-FLEXI-LINE but uses OCTETS-TO-STRING internally." (octets-to-string sequence :external-format external-format)) (defun error-handling-tests (&key verbose) "Tests several possible errors and how they are handled." (with-test-suite ("Testing error handling" :show-progress-p (not verbose)) (macrolet ((want-encoding-error (input format) `(with-expected-error (external-format-encoding-error) (read-flexi-line* ,input ,format)))) (when verbose (format t "~&\"Overlong\" UTF-8 sequences")) (want-encoding-error #(#b11000000 #b10000000) :utf-8) (want-encoding-error #(#b11000001 #b10000000) :utf-8) (want-encoding-error #(#b11100000 #b10011111 #b10000000) :utf-8) (want-encoding-error #(#b11110000 #b10001111 #b10000000 #b10000000) :utf-8) (check (accept-overlong #(#b11000000 #b10000000) #b00000000)) (check (accept-overlong #(#b11000001 #b10000000) #b01000000)) (check (accept-overlong #(#b11100000 #b10011111 #b10000000) #b011111000000)) (check (accept-overlong #(#b11110000 #b10001111 #b10000000 #b10000000) #b1111000000000000)) (when verbose (format t "~&Invalid lead octets in UTF-8")) (want-encoding-error #(#b11111000) :utf-8) (want-encoding-error #(#b11111001) :utf-8) (want-encoding-error #(#b11111100) :utf-8) (want-encoding-error #(#b11111101) :utf-8) (want-encoding-error #(#b11111110) :utf-8) (want-encoding-error #(#b11111111) :utf-8) (when verbose (format t "~&Illegal code points")) (want-encoding-error #(#x00 #x00 #x11 #x00) :utf-32le) (want-encoding-error #(#x00 #xd8) :utf-16le) (want-encoding-error #(#xff #xdf) :utf-16le)) (macrolet ((want-encoding-error (input format) `(with-expected-error (external-format-encoding-error) (read-flexi-line* ,input ,format)))) (when verbose (format t "~&UTF-8 sequences which are too short")) (want-encoding-error #(#xe4 #xf6 #xfc) :utf8) (want-encoding-error #(#xc0) :utf8) (want-encoding-error #(#xe0 #xff) :utf8) (want-encoding-error #(#xf0 #xff #xff) :utf8) (when verbose (format t "~&UTF-16 sequences with an odd number of octets")) (want-encoding-error #(#x01) :utf-16le) (want-encoding-error #(#x01 #x01 #x01) :utf-16le) (want-encoding-error #(#x01) :utf-16be) (want-encoding-error #(#x01 #x01 #x01) :utf-16be) (when verbose (format t "~&Missing words in UTF-16")) (want-encoding-error #(#x01 #xd8) :utf-16le) (want-encoding-error #(#xd8 #x01) :utf-16be) (when verbose (format t "~&Missing octets in UTF-32")) (want-encoding-error #(#x01) :utf-32le) (want-encoding-error #(#x01 #x01) :utf-32le) (want-encoding-error #(#x01 #x01 #x01) :utf-32le) (want-encoding-error #(#x01 #x01 #x01 #x01 #x01) :utf-32le) (want-encoding-error #(#x01) :utf-32be) (want-encoding-error #(#x01 #x01) :utf-32be) (want-encoding-error #(#x01 #x01 #x01) :utf-32be) (want-encoding-error #(#x01 #x01 #x01 #x01 #x01) :utf-32be)) (when verbose (format t "~&Handling of EOF in the middle of CRLF")) (check (string= #.(string #\Return) (read-flexi-line `(,(char-code #\Return)) '(:ascii :eol-style :crlf)))) (let ((*substitution-char* #\?)) (when verbose (format t "~&Fixed substitution character #\?") (format t "~&:ASCII doesn't have characters with char codes > 127")) (check (string= "a??" (read-flexi-line `(,(char-code #\a) 128 200) :ascii))) (check (string= "a??" (read-flexi-line* `#(,(char-code #\a) 128 200) :ascii))) (when verbose (format t "~&:WINDOWS-1253 doesn't have a characters with codes 170 and 210")) (check (string= "a??" (read-flexi-line `(,(char-code #\a) 170 210) :windows-1253))) (check (string= "a??" (read-flexi-line* `#(,(char-code #\a) 170 210) :windows-1253))) (when verbose (format t "~&Not a valid UTF-8 sequence")) (check (string= "??" (read-flexi-line '(#xe4 #xf6 #xfc) :utf8)))) (let ((*substitution-char* nil)) (when verbose (format t "~&Variable substitution using USE-VALUE restart") (format t "~&:ASCII doesn't have characters with char codes > 127")) (check (string= "abc" (using-values (#\b #\c) (read-flexi-line `(,(char-code #\a) 128 200) :ascii)))) (check (string= "abc" (using-values (#\b #\c) (read-flexi-line* `#(,(char-code #\a) 128 200) :ascii)))) (when verbose (format t "~&:WINDOWS-1253 doesn't have a characters with codes 170 and 210")) (check (string= "axy" (using-values (#\x #\y) (read-flexi-line `(,(char-code #\a) 170 210) :windows-1253)))) (check (string= "axy" (using-values (#\x #\y) (read-flexi-line* `#(,(char-code #\a) 170 210) :windows-1253)))) (when verbose (format t "~&Not a valid UTF-8 sequence")) (check (string= "QW" (using-values (#\Q #\W) (read-flexi-line '(#xe4 #xf6 #xfc) :utf8)))) (when verbose (format t "~&UTF-8 can't start neither with #b11111110 nor with #b11111111")) (check (string= "QW" (using-values (#\Q #\W) (read-flexi-line '(#b11111110 #b11111111) :utf8)))) (when verbose (format t "~&Only one octet in UTF-16 sequence")) (check (string= "E" (using-values (#\E) (read-flexi-line '(#x01) :utf-16le)))) (when verbose (format t "~&Two octets in UTF-16, but value of resulting word suggests that another word follows")) (check (string= "R" (using-values (#\R) (read-flexi-line '(#x01 #xd8) :utf-16le)))) (when verbose (format t "~&The second word must fit into the [#xdc00; #xdfff] interval, but it is #xdbff")) (check (string= "T" (using-values (#\T) (read-flexi-line '(#x01 #xd8 #xff #xdb) :utf-16le)))) (check (string= "T" (using-values (#\T) (read-flexi-line* #(#x01 #xd8 #xff #xdb) :utf-16le)))) (when verbose (format t "~&The same as for little endian above, but using inverse order of bytes in words")) (check (string= "E" (using-values (#\E) (read-flexi-line '(#x01) :utf-16be)))) (check (string= "R" (using-values (#\R) (read-flexi-line '(#xd8 #x01) :utf-16be)))) (check (string= "T" (using-values (#\T) (read-flexi-line '(#xd8 #x01 #xdb #xff) :utf-16be)))) (check (string= "T" (using-values (#\T) (read-flexi-line* #(#xd8 #x01 #xdb #xff) :utf-16be)))) (when verbose (format t "~&EOF in the middle of a 4-octet sequence in UTF-32")) (check (string= "Y" (using-values (#\Y) (read-flexi-line '(#x01) :utf-32le)))) (check (string= "Y" (using-values (#\Y) (read-flexi-line '(#x01 #x01) :utf-32le)))) (check (string= "Y" (using-values (#\Y) (read-flexi-line '(#x01 #x01 #x01) :utf-32le)))) (check (string= "aY" (using-values (#\Y) (read-flexi-line `(,(char-code #\a) #x00 #x00 #x00 #x01) :utf-32le)))) (check (string= "Y" (using-values (#\Y) (read-flexi-line '(#x01) :utf-32be)))) (check (string= "Y" (using-values (#\Y) (read-flexi-line '(#x01 #x01) :utf-32be)))) (check (string= "Y" (using-values (#\Y) (read-flexi-line '(#x01 #x01 #x01) :utf-32be)))) (check (string= "aY" (using-values (#\Y) (read-flexi-line `(#x00 #x00 #x00 ,(char-code #\a) #x01) :utf-32be))))))) (defun unread-char-tests (&key verbose) "Tests whether UNREAD-CHAR behaves as expected." (with-test-suite ("UNREAD-CHAR behaviour." :show-progress-p (and (not verbose) 100)) (flet ((test-one-file (file-name external-format) (when verbose (format t "~& ...and external format ~A" (normalize-external-format external-format))) (with-open-file (in (merge-pathnames file-name *this-file*) :element-type 'flex:octet) (let ((in (make-flexi-stream in :external-format external-format))) (loop repeat 300 for char = (read-char in) do (unread-char char in) (check (char= (read-char in) char))))))) (loop for (file-name symbols) in *test-files* when verbose do (format t "~&With file ~S" file-name) do (loop for symbol in symbols do (loop for (file-name . external-format) in (create-file-variants file-name symbol) do (test-one-file file-name external-format))))))) (defun run-all-tests (&key verbose) "Runs all tests for FLEXI-STREAMS and returns a true value iff all tests succeeded. VERBOSE is interpreted by the individual test suites above." (let ((successp t)) (macrolet ((run-test-suite (&body body) `(unless (progn ,@body) (setq successp nil)))) (run-test-suite (compare-files :verbose verbose)) (run-test-suite (string-tests :verbose verbose)) (run-test-suite (sequence-tests :verbose verbose)) (run-test-suite (error-handling-tests :verbose verbose)) (run-test-suite (unread-char-tests :verbose verbose)) (format t "~2&~:[Some tests failed~;All tests passed~]." successp) successp))) flexi-streams-1.0.7/test/tilton_ascii_cr.txt 0000664 0000000 0000000 00000007132 11175756704 0021207 0 ustar 00root root 0000000 0000000 Programmers who lock onto a design decision and cling to it in the face of contradictory new information -- well, that's almost everyone in my experience, so I better not say what I think of them or people will start saying bad things about me on c.l.l. -- Ken Tilton % This reminds me of the NYC cabby who accepted a fare to Chicago. When they got there and could not find the friend who was supposed to pay the fare he just laughed and said he should have known. -- Ken Tilton % >> Actually, I believe that Aikido, Jazz and Lisp are different appearances >> of the same thing. Yes, the Tao. /Everything/ is a different appearance of the tao. -- Ken Tilton "Ken, I went to the library and read up on Buddhism, and believe me, you are no Buddhist." -- Kenny's mom % That absolutely terrifies the herd-following, lockstep-marching, mainstream-saluting cowards that obediently dash out or online to scoop up books on The Latest Thing. They learn and use atrocities like Java, C++, XML, and even Python for the security it gives them and then sit there slaving away miserably, tediously, joylously paying off mortgages and supporting ungrateful teenagers who despise them, only to look out the double-sealed thermo-pane windows of their central-heated, sound-proofed, dead-bolted, suffocating little nests into the howling gale thinking "what do they know that I do not know?" when they see us under a lean-to hunched over our laptops to shield them from the rain laughing our asses off as we write great code between bong hits.... what was the question? -- Ken Tilton % Shut up! (That last phrase has four or more syllables if pronounced as intended.) -- Ken Tilton % Nonsense. You'll be using it for the GUI, not protein-folding. -- Ken Tilton (responding to a comment that LTK was slow because it was based on TK) % Continuations certainly are clever, but if we learned anything from the rejection of the cover art for "Smell the Glove", it is that "there is a fine line between stupid... and clever". -- Ken Tilton % Ah, there's no place like academia for dispassionate, intellectually honest discussion of new ideas on their merits. Thank god for tenure giving your bold antagonist the protection they needed to shout down your iconoclastic..... hang on... -- Ken Tilton % Whoever objected must be in my killfile, ... -- Ken Tilton % From memory (but I think I have it right): "But Jesus said, Suffer captured variables, and forbid them not, to come unto thine macro bodies: for of such is are DSLs made." -- Ken Tilton Can I get an Amen? % Awareness of defect is the first step to recovery. -- Ken Tilton % You made a bad analogy (there are no good ones, but you found a new low) ... -- Ken Tilton % Yes, it is true that Kent Pitman was raised by a closet full of Lisp Machines, but the exception only proves the rule. -- Ken Tilton (in a postscript after positing that computer languages are not learned in infancy) % I suggest you try bartender's school to support yourself, start programming for fun again. -- Ken Tilton (responding to a comment that 98% of anything to do with computers was not interesting code) % You could add four lanes to my carpal tunnel and I still could not write all the code I am dying to write. -- Ken Tilton % Neutrality? I want to bury other languages, not have a gateway to them. -- Ken Tilton % Ken: "Cute puppy. Did you get it for companionship or to pick up chicks?" Simon: "Hunh? My puppy /always/ gives me companionship." -- Ken Tilton (on how he was understood by a native english speaker) % flexi-streams-1.0.7/test/tilton_ascii_crlf.txt 0000664 0000000 0000000 00000007272 11175756704 0021536 0 ustar 00root root 0000000 0000000 Programmers who lock onto a design decision and cling to it in the face of contradictory new information -- well, that's almost everyone in my experience, so I better not say what I think of them or people will start saying bad things about me on c.l.l. -- Ken Tilton % This reminds me of the NYC cabby who accepted a fare to Chicago. When they got there and could not find the friend who was supposed to pay the fare he just laughed and said he should have known. -- Ken Tilton % >> Actually, I believe that Aikido, Jazz and Lisp are different appearances >> of the same thing. Yes, the Tao. /Everything/ is a different appearance of the tao. -- Ken Tilton "Ken, I went to the library and read up on Buddhism, and believe me, you are no Buddhist." -- Kenny's mom % That absolutely terrifies the herd-following, lockstep-marching, mainstream-saluting cowards that obediently dash out or online to scoop up books on The Latest Thing. They learn and use atrocities like Java, C++, XML, and even Python for the security it gives them and then sit there slaving away miserably, tediously, joylously paying off mortgages and supporting ungrateful teenagers who despise them, only to look out the double-sealed thermo-pane windows of their central-heated, sound-proofed, dead-bolted, suffocating little nests into the howling gale thinking "what do they know that I do not know?" when they see us under a lean-to hunched over our laptops to shield them from the rain laughing our asses off as we write great code between bong hits.... what was the question? -- Ken Tilton % Shut up! (That last phrase has four or more syllables if pronounced as intended.) -- Ken Tilton % Nonsense. You'll be using it for the GUI, not protein-folding. -- Ken Tilton (responding to a comment that LTK was slow because it was based on TK) % Continuations certainly are clever, but if we learned anything from the rejection of the cover art for "Smell the Glove", it is that "there is a fine line between stupid... and clever". -- Ken Tilton % Ah, there's no place like academia for dispassionate, intellectually honest discussion of new ideas on their merits. Thank god for tenure giving your bold antagonist the protection they needed to shout down your iconoclastic..... hang on... -- Ken Tilton % Whoever objected must be in my killfile, ... -- Ken Tilton % From memory (but I think I have it right): "But Jesus said, Suffer captured variables, and forbid them not, to come unto thine macro bodies: for of such is are DSLs made." -- Ken Tilton Can I get an Amen? % Awareness of defect is the first step to recovery. -- Ken Tilton % You made a bad analogy (there are no good ones, but you found a new low) ... -- Ken Tilton % Yes, it is true that Kent Pitman was raised by a closet full of Lisp Machines, but the exception only proves the rule. -- Ken Tilton (in a postscript after positing that computer languages are not learned in infancy) % I suggest you try bartender's school to support yourself, start programming for fun again. -- Ken Tilton (responding to a comment that 98% of anything to do with computers was not interesting code) % You could add four lanes to my carpal tunnel and I still could not write all the code I am dying to write. -- Ken Tilton % Neutrality? I want to bury other languages, not have a gateway to them. -- Ken Tilton % Ken: "Cute puppy. Did you get it for companionship or to pick up chicks?" Simon: "Hunh? My puppy /always/ gives me companionship." -- Ken Tilton (on how he was understood by a native english speaker) % flexi-streams-1.0.7/test/tilton_ascii_lf.txt 0000664 0000000 0000000 00000007132 11175756704 0021204 0 ustar 00root root 0000000 0000000 Programmers who lock onto a design decision and cling to it in the face of contradictory new information -- well, that's almost everyone in my experience, so I better not say what I think of them or people will start saying bad things about me on c.l.l. -- Ken Tilton % This reminds me of the NYC cabby who accepted a fare to Chicago. When they got there and could not find the friend who was supposed to pay the fare he just laughed and said he should have known. -- Ken Tilton % >> Actually, I believe that Aikido, Jazz and Lisp are different appearances >> of the same thing. Yes, the Tao. /Everything/ is a different appearance of the tao. -- Ken Tilton "Ken, I went to the library and read up on Buddhism, and believe me, you are no Buddhist." -- Kenny's mom % That absolutely terrifies the herd-following, lockstep-marching, mainstream-saluting cowards that obediently dash out or online to scoop up books on The Latest Thing. They learn and use atrocities like Java, C++, XML, and even Python for the security it gives them and then sit there slaving away miserably, tediously, joylously paying off mortgages and supporting ungrateful teenagers who despise them, only to look out the double-sealed thermo-pane windows of their central-heated, sound-proofed, dead-bolted, suffocating little nests into the howling gale thinking "what do they know that I do not know?" when they see us under a lean-to hunched over our laptops to shield them from the rain laughing our asses off as we write great code between bong hits.... what was the question? -- Ken Tilton % Shut up! (That last phrase has four or more syllables if pronounced as intended.) -- Ken Tilton % Nonsense. You'll be using it for the GUI, not protein-folding. -- Ken Tilton (responding to a comment that LTK was slow because it was based on TK) % Continuations certainly are clever, but if we learned anything from the rejection of the cover art for "Smell the Glove", it is that "there is a fine line between stupid... and clever". -- Ken Tilton % Ah, there's no place like academia for dispassionate, intellectually honest discussion of new ideas on their merits. Thank god for tenure giving your bold antagonist the protection they needed to shout down your iconoclastic..... hang on... -- Ken Tilton % Whoever objected must be in my killfile, ... -- Ken Tilton % From memory (but I think I have it right): "But Jesus said, Suffer captured variables, and forbid them not, to come unto thine macro bodies: for of such is are DSLs made." -- Ken Tilton Can I get an Amen? % Awareness of defect is the first step to recovery. -- Ken Tilton % You made a bad analogy (there are no good ones, but you found a new low) ... -- Ken Tilton % Yes, it is true that Kent Pitman was raised by a closet full of Lisp Machines, but the exception only proves the rule. -- Ken Tilton (in a postscript after positing that computer languages are not learned in infancy) % I suggest you try bartender's school to support yourself, start programming for fun again. -- Ken Tilton (responding to a comment that 98% of anything to do with computers was not interesting code) % You could add four lanes to my carpal tunnel and I still could not write all the code I am dying to write. -- Ken Tilton % Neutrality? I want to bury other languages, not have a gateway to them. -- Ken Tilton % Ken: "Cute puppy. Did you get it for companionship or to pick up chicks?" Simon: "Hunh? My puppy /always/ gives me companionship." -- Ken Tilton (on how he was understood by a native english speaker) % flexi-streams-1.0.7/test/tilton_utf8_cr.txt 0000664 0000000 0000000 00000007132 11175756704 0021005 0 ustar 00root root 0000000 0000000 Programmers who lock onto a design decision and cling to it in the face of contradictory new information -- well, that's almost everyone in my experience, so I better not say what I think of them or people will start saying bad things about me on c.l.l. -- Ken Tilton % This reminds me of the NYC cabby who accepted a fare to Chicago. When they got there and could not find the friend who was supposed to pay the fare he just laughed and said he should have known. -- Ken Tilton % >> Actually, I believe that Aikido, Jazz and Lisp are different appearances >> of the same thing. Yes, the Tao. /Everything/ is a different appearance of the tao. -- Ken Tilton "Ken, I went to the library and read up on Buddhism, and believe me, you are no Buddhist." -- Kenny's mom % That absolutely terrifies the herd-following, lockstep-marching, mainstream-saluting cowards that obediently dash out or online to scoop up books on The Latest Thing. They learn and use atrocities like Java, C++, XML, and even Python for the security it gives them and then sit there slaving away miserably, tediously, joylously paying off mortgages and supporting ungrateful teenagers who despise them, only to look out the double-sealed thermo-pane windows of their central-heated, sound-proofed, dead-bolted, suffocating little nests into the howling gale thinking "what do they know that I do not know?" when they see us under a lean-to hunched over our laptops to shield them from the rain laughing our asses off as we write great code between bong hits.... what was the question? -- Ken Tilton % Shut up! (That last phrase has four or more syllables if pronounced as intended.) -- Ken Tilton % Nonsense. You'll be using it for the GUI, not protein-folding. -- Ken Tilton (responding to a comment that LTK was slow because it was based on TK) % Continuations certainly are clever, but if we learned anything from the rejection of the cover art for "Smell the Glove", it is that "there is a fine line between stupid... and clever". -- Ken Tilton % Ah, there's no place like academia for dispassionate, intellectually honest discussion of new ideas on their merits. Thank god for tenure giving your bold antagonist the protection they needed to shout down your iconoclastic..... hang on... -- Ken Tilton % Whoever objected must be in my killfile, ... -- Ken Tilton % From memory (but I think I have it right): "But Jesus said, Suffer captured variables, and forbid them not, to come unto thine macro bodies: for of such is are DSLs made." -- Ken Tilton Can I get an Amen? % Awareness of defect is the first step to recovery. -- Ken Tilton % You made a bad analogy (there are no good ones, but you found a new low) ... -- Ken Tilton % Yes, it is true that Kent Pitman was raised by a closet full of Lisp Machines, but the exception only proves the rule. -- Ken Tilton (in a postscript after positing that computer languages are not learned in infancy) % I suggest you try bartender's school to support yourself, start programming for fun again. -- Ken Tilton (responding to a comment that 98% of anything to do with computers was not interesting code) % You could add four lanes to my carpal tunnel and I still could not write all the code I am dying to write. -- Ken Tilton % Neutrality? I want to bury other languages, not have a gateway to them. -- Ken Tilton % Ken: "Cute puppy. Did you get it for companionship or to pick up chicks?" Simon: "Hunh? My puppy /always/ gives me companionship." -- Ken Tilton (on how he was understood by a native english speaker) % flexi-streams-1.0.7/test/tilton_utf8_crlf.txt 0000664 0000000 0000000 00000007272 11175756704 0021334 0 ustar 00root root 0000000 0000000 Programmers who lock onto a design decision and cling to it in the face of contradictory new information -- well, that's almost everyone in my experience, so I better not say what I think of them or people will start saying bad things about me on c.l.l. -- Ken Tilton % This reminds me of the NYC cabby who accepted a fare to Chicago. When they got there and could not find the friend who was supposed to pay the fare he just laughed and said he should have known. -- Ken Tilton % >> Actually, I believe that Aikido, Jazz and Lisp are different appearances >> of the same thing. Yes, the Tao. /Everything/ is a different appearance of the tao. -- Ken Tilton "Ken, I went to the library and read up on Buddhism, and believe me, you are no Buddhist." -- Kenny's mom % That absolutely terrifies the herd-following, lockstep-marching, mainstream-saluting cowards that obediently dash out or online to scoop up books on The Latest Thing. They learn and use atrocities like Java, C++, XML, and even Python for the security it gives them and then sit there slaving away miserably, tediously, joylously paying off mortgages and supporting ungrateful teenagers who despise them, only to look out the double-sealed thermo-pane windows of their central-heated, sound-proofed, dead-bolted, suffocating little nests into the howling gale thinking "what do they know that I do not know?" when they see us under a lean-to hunched over our laptops to shield them from the rain laughing our asses off as we write great code between bong hits.... what was the question? -- Ken Tilton % Shut up! (That last phrase has four or more syllables if pronounced as intended.) -- Ken Tilton % Nonsense. You'll be using it for the GUI, not protein-folding. -- Ken Tilton (responding to a comment that LTK was slow because it was based on TK) % Continuations certainly are clever, but if we learned anything from the rejection of the cover art for "Smell the Glove", it is that "there is a fine line between stupid... and clever". -- Ken Tilton % Ah, there's no place like academia for dispassionate, intellectually honest discussion of new ideas on their merits. Thank god for tenure giving your bold antagonist the protection they needed to shout down your iconoclastic..... hang on... -- Ken Tilton % Whoever objected must be in my killfile, ... -- Ken Tilton % From memory (but I think I have it right): "But Jesus said, Suffer captured variables, and forbid them not, to come unto thine macro bodies: for of such is are DSLs made." -- Ken Tilton Can I get an Amen? % Awareness of defect is the first step to recovery. -- Ken Tilton % You made a bad analogy (there are no good ones, but you found a new low) ... -- Ken Tilton % Yes, it is true that Kent Pitman was raised by a closet full of Lisp Machines, but the exception only proves the rule. -- Ken Tilton (in a postscript after positing that computer languages are not learned in infancy) % I suggest you try bartender's school to support yourself, start programming for fun again. -- Ken Tilton (responding to a comment that 98% of anything to do with computers was not interesting code) % You could add four lanes to my carpal tunnel and I still could not write all the code I am dying to write. -- Ken Tilton % Neutrality? I want to bury other languages, not have a gateway to them. -- Ken Tilton % Ken: "Cute puppy. Did you get it for companionship or to pick up chicks?" Simon: "Hunh? My puppy /always/ gives me companionship." -- Ken Tilton (on how he was understood by a native english speaker) % flexi-streams-1.0.7/test/tilton_utf8_lf.txt 0000664 0000000 0000000 00000007132 11175756704 0021002 0 ustar 00root root 0000000 0000000 Programmers who lock onto a design decision and cling to it in the face of contradictory new information -- well, that's almost everyone in my experience, so I better not say what I think of them or people will start saying bad things about me on c.l.l. -- Ken Tilton % This reminds me of the NYC cabby who accepted a fare to Chicago. When they got there and could not find the friend who was supposed to pay the fare he just laughed and said he should have known. -- Ken Tilton % >> Actually, I believe that Aikido, Jazz and Lisp are different appearances >> of the same thing. Yes, the Tao. /Everything/ is a different appearance of the tao. -- Ken Tilton "Ken, I went to the library and read up on Buddhism, and believe me, you are no Buddhist." -- Kenny's mom % That absolutely terrifies the herd-following, lockstep-marching, mainstream-saluting cowards that obediently dash out or online to scoop up books on The Latest Thing. They learn and use atrocities like Java, C++, XML, and even Python for the security it gives them and then sit there slaving away miserably, tediously, joylously paying off mortgages and supporting ungrateful teenagers who despise them, only to look out the double-sealed thermo-pane windows of their central-heated, sound-proofed, dead-bolted, suffocating little nests into the howling gale thinking "what do they know that I do not know?" when they see us under a lean-to hunched over our laptops to shield them from the rain laughing our asses off as we write great code between bong hits.... what was the question? -- Ken Tilton % Shut up! (That last phrase has four or more syllables if pronounced as intended.) -- Ken Tilton % Nonsense. You'll be using it for the GUI, not protein-folding. -- Ken Tilton (responding to a comment that LTK was slow because it was based on TK) % Continuations certainly are clever, but if we learned anything from the rejection of the cover art for "Smell the Glove", it is that "there is a fine line between stupid... and clever". -- Ken Tilton % Ah, there's no place like academia for dispassionate, intellectually honest discussion of new ideas on their merits. Thank god for tenure giving your bold antagonist the protection they needed to shout down your iconoclastic..... hang on... -- Ken Tilton % Whoever objected must be in my killfile, ... -- Ken Tilton % From memory (but I think I have it right): "But Jesus said, Suffer captured variables, and forbid them not, to come unto thine macro bodies: for of such is are DSLs made." -- Ken Tilton Can I get an Amen? % Awareness of defect is the first step to recovery. -- Ken Tilton % You made a bad analogy (there are no good ones, but you found a new low) ... -- Ken Tilton % Yes, it is true that Kent Pitman was raised by a closet full of Lisp Machines, but the exception only proves the rule. -- Ken Tilton (in a postscript after positing that computer languages are not learned in infancy) % I suggest you try bartender's school to support yourself, start programming for fun again. -- Ken Tilton (responding to a comment that 98% of anything to do with computers was not interesting code) % You could add four lanes to my carpal tunnel and I still could not write all the code I am dying to write. -- Ken Tilton % Neutrality? I want to bury other languages, not have a gateway to them. -- Ken Tilton % Ken: "Cute puppy. Did you get it for companionship or to pick up chicks?" Simon: "Hunh? My puppy /always/ gives me companionship." -- Ken Tilton (on how he was understood by a native english speaker) % flexi-streams-1.0.7/test/unicode_demo_ucs2_cr_be.txt 0000664 0000000 0000000 00000035612 11175756704 0022566 0 ustar 00root root 0000000 0000000 U T F - 8 e n c o d e d s a m p l e p l a i n - t e x t f i l e > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > M a r k u s K u h n [ m a k s k u n ] < h t t p : / / w w w . c l . c a m . a c . u k / ~ m g k 2 5 / > 2 0 0 2 - 0 7 - 2 5 T h e A S C I I c o m p a t i b l e U T F - 8 e n c o d i n g u s e d i n t h i s p l a i n - t e x t f i l e i s d e f i n e d i n U n i c o d e , I S O 1 0 6 4 6 - 1 , a n d R F C 2 2 7 9 . U s i n g U n i c o d e / U T F - 8 , y o u c a n w r i t e i n e m a i l s a n d s o u r c e c o d e t h i n g s s u c h a s M a t h e m a t i c s a n d s c i e n c e s : ". E" d a = Q , n ! " , " f ( i ) = " g ( i ) , ###% % % % % % %### ###% a + b ### " x"! : # x# = "# " x# , "' = ( "( ) , ###%% % % % % ### #### c ### ! " ! " !$ " ! " ! " ! , ### ### ### " ### " < a "` b "a c "d d "j " ! (' A' ! ' B' ) , ### # ### ### # a q - b q### 2 H + O ! 2 H O , R = 4 . 7 k , # 2 0 0 m m ### i = 1 ### L i n g u i s t i c s a n d d i c t i o n a r i e s : i 1 n tY n Y nY l fY n[ t1 k Y s o s i e1 n Y [ p s i lT n ] , Y e n [ j[ n ] , Y o g a [ j o gQ ] A P L : ( ( V#s V ) =#s#t V ) / V! , V #7!#s!#t""" >#N#U# N i c e r t y p o g r a p h y i n p l a i n t e x t f i l e s : %T%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%W %Q %Q %Q " s i n g l e a n d d o u b l e q u o t e s %Q %Q %Q %Q " C u r l y a p o s t r o p h e s : W e v e b e e n h e r e %Q %Q %Q %Q " L a t i n - 1 a p o s t r o p h e a n d a c c e n t s : ' ` %Q %Q %Q %Q " d e u t s c h e A n f h r u n g s z e i c h e n %Q %Q %Q %Q " , ! , 0 , " , 3 4 , , " 5 / + 5 , !" , & %Q %Q %Q %Q " A S C I I s a f e t y t e s t : 1 l I | , 0 O D , 8 B %Q %Q %m% % % % % % % % % %n %Q %Q " t h e e u r o s y m b o l : % 1 4 . 9 5 % %Q %Q %p% % % % % % % % % %o %Q %Z%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%] C o m b i n i n g c h a r a c t e r s : S T A R G T E S G - 1 , a = v = r , a " b G r e e k ( i n P o l y t o n i c ) : T h e G r e e k a n t h e m : r w x t y t u , r w x t D z r w q t . p y s u p 1 q v p s , f , q ! F r o m a s p e e c h o f D e m o s t h e n e s i n t h e 4 t h c e n t u r y B C : P v P p w w } , f , E 0 p q s v E x z y S { z r p y v u w A s , p r q 0 u , e E t y P v y s s . P s V 1 p s " t Q y , v ' { , P v t V q Q q . | s , E s y v p Q v w u , v q 6 q , P q s y s s 1 x ! 6 t } , E z q } . p p w Q q , y v v w u w v C y s v r t t @ Q s , q ! v A y . s , x G e o r g i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : U n i c o d e - , 1 0 - 1 2 , . , . U n i c o d e - , , U n i c o d e - , , , . R u s s i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : 0@538AB@8@C9B5AL A59G0A =0 5AOBCN 564C=0@>4=CN >=D5@5=F8N ?> U n i c o d e , :>B>@0O A>AB>8BAO 1 0 - 1 2 <0@B0 1 9 9 7 3>40 2 09=F5 2 5@<0=88 . >=D5@5=F8O A>15@5B H8@>:89 :@C3 M:A?5@B>2 ?> 2>?@>A0< 3;>10;L=>3> =B5@=5B0 8 U n i c o d e , ;>:0;870F88 8 8=B5@=0F8>=0;870F88 , 2>?;>I5=8N 8 ?@8<5=5=8N U n i c o d e 2 @07;8G=KE >?5@0F8>==KE A8AB5<0E 8 ?@>3@0<<=KE ?@8;>65=8OE , H@8DB0E , 25@AB:5 8 <=>3>O7KG=KE :><?LNB5@=KE A8AB5<0E . T h a i ( U C S L e v e l 2 ) : E x c e r p t f r o m a p o e t r y o n T h e R o m a n c e o f T h e T h r e e K i n g d o m s ( a C h i n e s e c l a s s i c ' S a n G u a ' ) : [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - - - - ] O A H4.1H@*7H-!B#!A**1@' #0@(-9J9I6IC+!H *4*-)1#4"LH-+I2A%1D *--LD #IBH@%2@21 2 #17-15@G5H 6H I2@!7-6'4#4@G1+2 B.4K@#5"1 1H'+1'@!7-!2 +!2"0H2! 1H'1'*31 @+!7-1D*D%H@*7-2@+2 #1+!2H2@I2!2@%"-2*1 H2"-I--8I"8A"C+IA1 C I*2'1I@G ' 7H 'C %1%4 8"8"5%1H-@+8 H2-2@ (#4+2 I2#I-D+I I-##2H2 1##%1" $E+2C#I3 99I##%1L / ( T h e a b o v e i s a t w o - c o l u m n t e x t . I f c o m b i n i n g c h a r a c t e r s a r e h a n d l e d c o r r e c t l y , t h e l i n e s o f t h e s e c o n d c o l u m n s h o u l d b e a l i g n e d w i t h t h e | c h a r a c t e r a b o v e . ) E t h i o p i a n : P r o v e r b s i n t h e A m h a r i c l a n g u a g e : 0 s(5 % 05b e ct `F b % dq A % b ` Ed c # #u `b M s `Ed s= b % `` psb 2p( ( b @5 `@5e A ` ) b - be- `3 5-b 0 dq (dq p- b - Hp .. 3 - b (du ce bu 5E cu Eb %+ Msu Ksub c * e + b 5 ) + ) -b p bpI p 6 cIb - b (-5 u 0b - `M+= - b R u n e s : ( O l d E n g l i s h , w h i c h t r a n s c r i b e d i n t o L a t i n r e a d s ' H e c w a e t h t h a t h e b u d e t h a e m l a n d e n o r t h w e a r d u m w i t h t h a W e s t s a e . ' a n d m e a n s ' H e s a i d t h a t h e l i v e d i n t h e n o r t h e r n l a n d n e a r t h e W e s t e r n S e a . ' ) B r a i l l e : (L(('( (<(( (M( (((9(0( (c(( (M( (((9 (:(( ((((( ( ( (((( (:( (9(2 (y(;( ( ( ( ( ((3(( (1(( (('(; (((3( (9(( (2 (y( (((( ( (; (( (( ( ((%(( (( (:(( (( (( (+ ((9 (9( ( ((;((9( (( ( (9( ( ((;(( (9( (%( ((;( (((;( (( ( (9( (!( (( ( (3(( (;(2 (N( ((((( (( (( (+ ( ( (2 (A( ( (N( ((((((0( ( (( ( (:(( (((( (%((( (0(a(( ((( ( (( (( (9(9(( (( (!((( ( ( ((%( (( ( ((( ( ( ((2 (U(( (M( (((9 (:(( (( (((( (( ( ((((($( (( ((2 (M((( (J ((( (0( ( ((( ( ( (((9 (9(( (J (( (*( (( ( (9 (*( (( (*((+((( (1(( (9(;( ( ( (( ( ( ( (%(( ((9 (((( (((3( ( ((((($( (( ((2 (J ( ( (#( ((('( (((2 (( (((+( ( (9(((( ( ( ( (((( ( ( ( (( ( (($( (( ( (( (9( (((((( (( (( ( (( ( ((( ( (( ((;(9 ( (9( ( (((((2 (C(%( (9( (:( (((( (( (3( (( ( (( ((( ( ( ( (9( (( ( ( ((( (( ( ( (9 (%( (((((*(+ ((( (( ()((( ( (( (( ( (%(( ( ( ( (( (9( (J(3( ( ((9(0( ((( ( ( (((2 (y(3 (:( (( (9(;(( ((( ((;( ( ( ( ( ( ( (((((( ( (( (((( ( ( ((((9( (9(( (M( (((9 (:(( (( (((( (( ( ((((($( (( ((2 ( T h e f i r s t c o u p l e o f p a r a g r a p h s o f " A C h r i s t m a s C a r o l " b y D i c k e n s ) C o m p a c t f o n t s e l e c t i o n e x a m p l e t e x t : A B C D E F G H I J K L M N O P Q R S T U V W X Y Z / 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z " & 0!"S`x~ 01234 " ""! "'"*"a" !!!!! %%<%T%X%%&:&@ $@ P#N1 G r e e t i n g s i n v a r i o u s l a n g u a g e s : H e l l o w o r l d , s y , 00000 B o x d r a w i n g a l i g n m e n t t e s t s : % % %T%P%P%f%P%P%W % % % %,% % % %m% % %,% % %n %m% % %,% % %n %%%%3%%% %%%% %w %{ %%/% % %0% % %q%r%q%r%s%s%s %Q% % %h% %%Q %%T%P%g%P%W% %%R%P%j%P%U% %%S% %A% %V% %% % %B% %% %%C%D% %v%<%t%z%K%x% %<%( % %K%% % %r%q%r%q%s%s%s %Q%%r %q%%Q %%Q %Q% %% % %% %%Q % %Q% %% % %% % %E%F% %u %y %%7% %%8% % %q%r%q%r%s%s%s %`%a %s %^%c % %b %_%$ % %<% %<% %<%$ % %k% %B% %k%$ %#%?%~%<%|%?%+ %%%% % %%% %N %%%% % % %r%q%r%q%s%s%s %Q%%q %r%%Q %%Q %Q% %% % %% %%Q % %Q% %% %} %% %%%%%%%% % % %N %O % % % %Q%% %e% %%Q %%Z%P%d%P%]% %%X%P%j%P%[% %%Y% %@% %\% %%% %B% %% %%%%%%%% % % %N %O % % % %Z%P%P%i%P%P%] %% % %4% % % %p% % %4% % %o %p% % %4% % %o %%%%;%%% %%%%%% %%L%L% %N %%M%M% % %%%%%%%% %%%%%% flexi-streams-1.0.7/test/unicode_demo_ucs2_cr_le.txt 0000664 0000000 0000000 00000035612 11175756704 0022600 0 ustar 00root root 0000000 0000000 U T F - 8 e n c o d e d s a m p l e p l a i n - t e x t f i l e > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > M a r k u s K u h n [ m a k s k u n ] < h t t p : / / w w w . c l . c a m . a c . u k / ~ m g k 2 5 / > 2 0 0 2 - 0 7 - 2 5 T h e A S C I I c o m p a t i b l e U T F - 8 e n c o d i n g u s e d i n t h i s p l a i n - t e x t f i l e i s d e f i n e d i n U n i c o d e , I S O 1 0 6 4 6 - 1 , a n d R F C 2 2 7 9 . U s i n g U n i c o d e / U T F - 8 , y o u c a n w r i t e i n e m a i l s a n d s o u r c e c o d e t h i n g s s u c h a s M a t h e m a t i c s a n d s c i e n c e s : ." E "d a = Q , n ! ", " f ( i ) = " g ( i ) , ### % % % % % %%### ###%a + b ### "x " !: #x # = " #"x #, '" = ( (" ) , ###% % % % % % ### #### c ### ! " ! " $! " ! " ! " !, ### ### ### " ### " < a `" b a" c d" d j" " ! ( 'A ' ! 'B ') , ### # ### ### #a q - b q ### 2 H + O ! 2 H O , R = 4 . 7 k , # 2 0 0 m m ###i = 1 ### L i n g u i s t i c s a n d d i c t i o n a r i e s : i 1n t Yn Yn Yl f Yn [t 1k Ys o s i e 1n Y [ p s i l Tn ] , Y e n [ j [n ] , Y o g a [ j o g Q] A P L : ( ( V s#V ) = s#t#V ) / V !, V 7#!s#!t#"""> N#U## N i c e r t y p o g r a p h y i n p l a i n t e x t f i l e s : T%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%W% Q% Q% Q% " s i n g l e a n d d o u b l e q u o t e s Q% Q% Q% Q% " C u r l y a p o s t r o p h e s : W e v e b e e n h e r e Q% Q% Q% Q% " L a t i n - 1 a p o s t r o p h e a n d a c c e n t s : ' ` Q% Q% Q% Q% " d e u t s c h e A n f h r u n g s z e i c h e n Q% Q% Q% Q% " , ! , 0 , " , 3 4 , , "5 / + 5 , "!, & Q% Q% Q% Q% " A S C I I s a f e t y t e s t : 1 l I | , 0 O D , 8 B Q% Q% m% % % % % % % % % %n% Q% Q% " t h e e u r o s y m b o l : % 1 4 . 9 5 % Q% Q% p% % % % % % % % % %o% Q% Z%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%]% C o m b i n i n g c h a r a c t e r s : S T A R G T E S G - 1 , a = v = r , a " b G r e e k ( i n P o l y t o n i c ) : T h e G r e e k a n t h e m : r w x t y t u , r w x t D z r w q t . p y s u p 1 q v p s , f , q ! F r o m a s p e e c h o f D e m o s t h e n e s i n t h e 4 t h c e n t u r y B C : P v P p w w } , f , E 0 p q s v E x z y S { z r p y v u w A s , p r q 0 u , e E t y P v y s s . P s V 1 p s " t Q y , v ' { , P v t V q Q q . | s , E s y v p Q v w u , v q 6 q , P q s y s s 1 x ! 6 t } , E z q } . p p w Q q , y v v w u w v C y s v r t t @ Q s , q ! v A y . s , x G e o r g i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : U n i c o d e - , 1 0 - 1 2 , . , . U n i c o d e - , , U n i c o d e - , , , . R u s s i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : 0@538AB@8@C9B5AL A59G0A =0 5AOBCN 564C=0@>4=CN >=D5@5=F8N ?> U n i c o d e , :>B>@0O A>AB>8BAO 1 0 - 1 2 <0@B0 1 9 9 7 3>40 2 09=F5 2 5@<0=88. >=D5@5=F8O A>15@5B H8@>:89 :@C3 M:A?5@B>2 ?> 2>?@>A0< 3;>10;L=>3> =B5@=5B0 8 U n i c o d e , ;>:0;870F88 8 8=B5@=0F8>=0;870F88, 2>?;>I5=8N 8 ?@8<5=5=8N U n i c o d e 2 @07;8G=KE >?5@0F8>==KE A8AB5<0E 8 ?@>3@0<<=KE ?@8;>65=8OE, H@8DB0E, 25@AB:5 8 <=>3>O7KG=KE :><?LNB5@=KE A8AB5<0E. T h a i ( U C S L e v e l 2 ) : E x c e r p t f r o m a p o e t r y o n T h e R o m a n c e o f T h e T h r e e K i n g d o m s ( a C h i n e s e c l a s s i c ' S a n G u a ' ) : [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - - - - ] O A H4.1H@*7H-!B#!A**1@' #0@(-9J9I6IC+!H *4*-)1#4"LH-+I2A%1D *--LD #IBH@%2@21 2 #17-15@G5H 6H I2@!7-6'4#4@G1+2 B.4K@#5"1 1H'+1'@!7-!2 +!2"0H2! 1H'1'*31 @+!7-1D*D%H@*7-2@+2 #1+!2H2@I2!2@%"-2*1 H2"-I--8I"8A"C+IA1 C I*2'1I@G ' 7H 'C %1%4 8"8"5%1H-@+8 H2-2@ (#4+2 I2#I-D+I I-##2H2 1##%1" $E+2C#I3 99I##%1L / ( T h e a b o v e i s a t w o - c o l u m n t e x t . I f c o m b i n i n g c h a r a c t e r s a r e h a n d l e d c o r r e c t l y , t h e l i n e s o f t h e s e c o n d c o l u m n s h o u l d b e a l i g n e d w i t h t h e | c h a r a c t e r a b o v e . ) E t h i o p i a n : P r o v e r b s i n t h e A m h a r i c l a n g u a g e : 0 s(5 % 05b e ct `F b % dq A % b ` Ed c # #u `b M s `Ed s= b % `` psb 2p( ( b @5 `@5e A ` ) b - be- `3 5-b 0 dq (dq p- b - Hp .. 3 - b (du ce bu 5E cu Eb %+ Msu Ksub c * e + b 5 ) + ) -b p bpI p 6 cIb - b (-5 u 0b - `M+= - b R u n e s : ( O l d E n g l i s h , w h i c h t r a n s c r i b e d i n t o L a t i n r e a d s ' H e c w a e t h t h a t h e b u d e t h a e m l a n d e n o r t h w e a r d u m w i t h t h a W e s t s a e . ' a n d m e a n s ' H e s a i d t h a t h e l i v e d i n t h e n o r t h e r n l a n d n e a r t h e W e s t e r n S e a . ' ) B r a i l l e : L(('(( <((( M( (((9(0(( c(( ( M( (((9( :((( ((((( (( (((( :( (9(2( y(;(( (( (( (3(( ( 1(( (('(;( ((3( ( 9(( (2( y(( ((( ( (;( ( ( ( (( (%(( ((( :((( ( (( (+( (9( 9(( ((;((9( (( (( 9(( ((;((( 9(( %( ((;( (((;(( ( (( 9(( !( (( ( (3(( (;(2( N( (((((( ( (( (+( ( (2( A( (( N( ((((((0(( (( (( :((( (((( %((( ( 0(a(( (((( ((( ( (9(9((( (( !(((( (( (%( ( ( (( (( (( ((2( U((( M( (((9( :((( (( (((( (( ( (((($( (( ((2( M(((( J( (( (0( ( ((( ( (( ((9( 9(( ( J( ( (*(( ( ( (9( *( ( ( (*((+(((( 1(( ( 9(;(( (( ( ( ( ( (%(( ((9( (((( ((3( ( ( (((($( (( ((2( J( ( (#( ( (('(( ((2( ( (((+(( (9(((( (( (( ((( (( ( (( ( (($( (( (( (( 9(( ((((( ( ( (( (( ( ( ((( ( (( ((;(9( ( 9(( (((((2( C(%( ( 9(( :( (((( ( ( ( 3(( ( ( (( (((( (( ( 9(( ( ( ( (((( ( (( (9( %( (((((*(+( (( ((( )(((( (( ( ( ( (%((( ( (( (( 9(( J(3( ( ((9(0(( (( (( (((2( y(3( :( ((( 9(;(( (((( (;( ( ( ( (( (( ((((( (( ( (((( ( ( ((((9(( 9(( ( M( (((9( :((( (( (((( (( ( (((($( (( ((2( ( T h e f i r s t c o u p l e o f p a r a g r a p h s o f " A C h r i s t m a s C a r o l " b y D i c k e n s ) C o m p a c t f o n t s e l e c t i o n e x a m p l e t e x t : A B C D E F G H I J K L M N O P Q R S T U V W X Y Z / 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z " & 0 "!S`x~ 01234 """ !'"*"a" " !!!!! %<%T%X%%%:&@& @$ PN#1 G r e e t i n g s i n v a r i o u s l a n g u a g e s : H e l l o w o r l d , s y , 00000 B o x d r a w i n g a l i g n m e n t t e s t s : % % T%P%P%f%P%P%W% % % %,% % %% m% % %,% % %n% m% % %,% % %n% %%%3%%%% %%%% w% {% %/%% %0%% % q%r%q%r%s%s%s% Q% % %h% %%Q% %T%P%g%P%W%% %R%P%j%P%U%% %S% %A% %V%% % % %B% %%% %C%D%% v%<%t%z%K%x% %<%(% %K%%% % r%q%r%q%s%s%s% Q%%r% q%%Q% %Q% Q%% %% % %% %Q% % Q%% %% % %% %E%F%% u% y% %7%% %8%% % q%r%q%r%s%s%s% `%a% s% ^%c% %b% _%$% %<% %<% %<%$% %k% %B% %k%$% #%?%~%<%|%?%+% %%%% %%%% N% %%%% % % r%q%r%q%s%s%s% Q%%q% r%%Q% %Q% Q%% %% % %% %Q% % Q%% %% }% %% %%%%%%%% % % N% O% % % % Q%% %e% %%Q% %Z%P%d%P%]%% %X%P%j%P%[%% %Y% %@% %\%% %% %B% %%% %%%%%%%% % % N% O% % % % Z%P%P%i%P%P%]% % % %4% % %% p% % %4% % %o% p% % %4% % %o% %%%;%%%% %%%%%% %L%L%% N% %M%M%% % %%%%%%%% %%%%%% flexi-streams-1.0.7/test/unicode_demo_ucs2_crlf_be.txt 0000664 0000000 0000000 00000036462 11175756704 0023114 0 ustar 00root root 0000000 0000000 U T F - 8 e n c o d e d s a m p l e p l a i n - t e x t f i l e > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > M a r k u s K u h n [ m a k s k u n ] < h t t p : / / w w w . c l . c a m . a c . u k / ~ m g k 2 5 / > 2 0 0 2 - 0 7 - 2 5 T h e A S C I I c o m p a t i b l e U T F - 8 e n c o d i n g u s e d i n t h i s p l a i n - t e x t f i l e i s d e f i n e d i n U n i c o d e , I S O 1 0 6 4 6 - 1 , a n d R F C 2 2 7 9 . U s i n g U n i c o d e / U T F - 8 , y o u c a n w r i t e i n e m a i l s a n d s o u r c e c o d e t h i n g s s u c h a s M a t h e m a t i c s a n d s c i e n c e s : ". E" d a = Q , n ! " , " f ( i ) = " g ( i ) , ###% % % % % % %### ###% a + b ### " x"! : # x# = "# " x# , "' = ( "( ) , ###%% % % % % ### #### c ### ! " ! " !$ " ! " ! " ! , ### ### ### " ### " < a "` b "a c "d d "j " ! (' A' ! ' B' ) , ### # ### ### # a q - b q### 2 H + O ! 2 H O , R = 4 . 7 k , # 2 0 0 m m ### i = 1 ### L i n g u i s t i c s a n d d i c t i o n a r i e s : i 1 n tY n Y nY l fY n[ t1 k Y s o s i e1 n Y [ p s i lT n ] , Y e n [ j[ n ] , Y o g a [ j o gQ ] A P L : ( ( V#s V ) =#s#t V ) / V! , V #7!#s!#t""" >#N#U# N i c e r t y p o g r a p h y i n p l a i n t e x t f i l e s : %T%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%W %Q %Q %Q " s i n g l e a n d d o u b l e q u o t e s %Q %Q %Q %Q " C u r l y a p o s t r o p h e s : W e v e b e e n h e r e %Q %Q %Q %Q " L a t i n - 1 a p o s t r o p h e a n d a c c e n t s : ' ` %Q %Q %Q %Q " d e u t s c h e A n f h r u n g s z e i c h e n %Q %Q %Q %Q " , ! , 0 , " , 3 4 , , " 5 / + 5 , !" , & %Q %Q %Q %Q " A S C I I s a f e t y t e s t : 1 l I | , 0 O D , 8 B %Q %Q %m% % % % % % % % % %n %Q %Q " t h e e u r o s y m b o l : % 1 4 . 9 5 % %Q %Q %p% % % % % % % % % %o %Q %Z%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%] C o m b i n i n g c h a r a c t e r s : S T A R G T E S G - 1 , a = v = r , a " b G r e e k ( i n P o l y t o n i c ) : T h e G r e e k a n t h e m : r w x t y t u , r w x t D z r w q t . p y s u p 1 q v p s , f , q ! F r o m a s p e e c h o f D e m o s t h e n e s i n t h e 4 t h c e n t u r y B C : P v P p w w } , f , E 0 p q s v E x z y S { z r p y v u w A s , p r q 0 u , e E t y P v y s s . P s V 1 p s " t Q y , v ' { , P v t V q Q q . | s , E s y v p Q v w u , v q 6 q , P q s y s s 1 x ! 6 t } , E z q } . p p w Q q , y v v w u w v C y s v r t t @ Q s , q ! v A y . s , x G e o r g i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : U n i c o d e - , 1 0 - 1 2 , . , . U n i c o d e - , , U n i c o d e - , , , . R u s s i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : 0@538AB@8@C9B5AL A59G0A =0 5AOBCN 564C=0@>4=CN >=D5@5=F8N ?> U n i c o d e , :>B>@0O A>AB>8BAO 1 0 - 1 2 <0@B0 1 9 9 7 3>40 2 09=F5 2 5@<0=88 . >=D5@5=F8O A>15@5B H8@>:89 :@C3 M:A?5@B>2 ?> 2>?@>A0< 3;>10;L=>3> =B5@=5B0 8 U n i c o d e , ;>:0;870F88 8 8=B5@=0F8>=0;870F88 , 2>?;>I5=8N 8 ?@8<5=5=8N U n i c o d e 2 @07;8G=KE >?5@0F8>==KE A8AB5<0E 8 ?@>3@0<<=KE ?@8;>65=8OE , H@8DB0E , 25@AB:5 8 <=>3>O7KG=KE :><?LNB5@=KE A8AB5<0E . T h a i ( U C S L e v e l 2 ) : E x c e r p t f r o m a p o e t r y o n T h e R o m a n c e o f T h e T h r e e K i n g d o m s ( a C h i n e s e c l a s s i c ' S a n G u a ' ) : [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - - - - ] O A H4.1H@*7H-!B#!A**1@' #0@(-9J9I6IC+!H *4*-)1#4"LH-+I2A%1D *--LD #IBH@%2@21 2 #17-15@G5H 6H I2@!7-6'4#4@G1+2 B.4K@#5"1 1H'+1'@!7-!2 +!2"0H2! 1H'1'*31 @+!7-1D*D%H@*7-2@+2 #1+!2H2@I2!2@%"-2*1 H2"-I--8I"8A"C+IA1 C I*2'1I@G ' 7H 'C %1%4 8"8"5%1H-@+8 H2-2@ (#4+2 I2#I-D+I I-##2H2 1##%1" $E+2C#I3 99I##%1L / ( T h e a b o v e i s a t w o - c o l u m n t e x t . I f c o m b i n i n g c h a r a c t e r s a r e h a n d l e d c o r r e c t l y , t h e l i n e s o f t h e s e c o n d c o l u m n s h o u l d b e a l i g n e d w i t h t h e | c h a r a c t e r a b o v e . ) E t h i o p i a n : P r o v e r b s i n t h e A m h a r i c l a n g u a g e : 0 s(5 % 05b e ct `F b % dq A % b ` Ed c # #u `b M s `Ed s= b % `` psb 2p( ( b @5 `@5e A ` ) b - be- `3 5-b 0 dq (dq p- b - Hp .. 3 - b (du ce bu 5E cu Eb %+ Msu Ksub c * e + b 5 ) + ) -b p bpI p 6 cIb - b (-5 u 0b - `M+= - b R u n e s : ( O l d E n g l i s h , w h i c h t r a n s c r i b e d i n t o L a t i n r e a d s ' H e c w a e t h t h a t h e b u d e t h a e m l a n d e n o r t h w e a r d u m w i t h t h a W e s t s a e . ' a n d m e a n s ' H e s a i d t h a t h e l i v e d i n t h e n o r t h e r n l a n d n e a r t h e W e s t e r n S e a . ' ) B r a i l l e : (L(('( (<(( (M( (((9(0( (c(( (M( (((9 (:(( ((((( ( ( (((( (:( (9(2 (y(;( ( ( ( ( ((3(( (1(( (('(; (((3( (9(( (2 (y( (((( ( (; (( (( ( ((%(( (( (:(( (( (( (+ ((9 (9( ( ((;((9( (( ( (9( ( ((;(( (9( (%( ((;( (((;( (( ( (9( (!( (( ( (3(( (;(2 (N( ((((( (( (( (+ ( ( (2 (A( ( (N( ((((((0( ( (( ( (:(( (((( (%((( (0(a(( ((( ( (( (( (9(9(( (( (!((( ( ( ((%( (( ( ((( ( ( ((2 (U(( (M( (((9 (:(( (( (((( (( ( ((((($( (( ((2 (M((( (J ((( (0( ( ((( ( ( (((9 (9(( (J (( (*( (( ( (9 (*( (( (*((+((( (1(( (9(;( ( ( (( ( ( ( (%(( ((9 (((( (((3( ( ((((($( (( ((2 (J ( ( (#( ((('( (((2 (( (((+( ( (9(((( ( ( ( (((( ( ( ( (( ( (($( (( ( (( (9( (((((( (( (( ( (( ( ((( ( (( ((;(9 ( (9( ( (((((2 (C(%( (9( (:( (((( (( (3( (( ( (( ((( ( ( ( (9( (( ( ( ((( (( ( ( (9 (%( (((((*(+ ((( (( ()((( ( (( (( ( (%(( ( ( ( (( (9( (J(3( ( ((9(0( ((( ( ( (((2 (y(3 (:( (( (9(;(( ((( ((;( ( ( ( ( ( ( (((((( ( (( (((( ( ( ((((9( (9(( (M( (((9 (:(( (( (((( (( ( ((((($( (( ((2 ( T h e f i r s t c o u p l e o f p a r a g r a p h s o f " A C h r i s t m a s C a r o l " b y D i c k e n s ) C o m p a c t f o n t s e l e c t i o n e x a m p l e t e x t : A B C D E F G H I J K L M N O P Q R S T U V W X Y Z / 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z " & 0!"S`x~ 01234 " ""! "'"*"a" !!!!! %%<%T%X%%&:&@ $@ P#N1 G r e e t i n g s i n v a r i o u s l a n g u a g e s : H e l l o w o r l d , s y , 00000 B o x d r a w i n g a l i g n m e n t t e s t s : % % %T%P%P%f%P%P%W % % % %,% % % %m% % %,% % %n %m% % %,% % %n %%%%3%%% %%%% %w %{ %%/% % %0% % %q%r%q%r%s%s%s %Q% % %h% %%Q %%T%P%g%P%W% %%R%P%j%P%U% %%S% %A% %V% %% % %B% %% %%C%D% %v%<%t%z%K%x% %<%( % %K%% % %r%q%r%q%s%s%s %Q%%r %q%%Q %%Q %Q% %% % %% %%Q % %Q% %% % %% % %E%F% %u %y %%7% %%8% % %q%r%q%r%s%s%s %`%a %s %^%c % %b %_%$ % %<% %<% %<%$ % %k% %B% %k%$ %#%?%~%<%|%?%+ %%%% % %%% %N %%%% % % %r%q%r%q%s%s%s %Q%%q %r%%Q %%Q %Q% %% % %% %%Q % %Q% %% %} %% %%%%%%%% % % %N %O % % % %Q%% %e% %%Q %%Z%P%d%P%]% %%X%P%j%P%[% %%Y% %@% %\% %%% %B% %% %%%%%%%% % % %N %O % % % %Z%P%P%i%P%P%] %% % %4% % % %p% % %4% % %o %p% % %4% % %o %%%%;%%% %%%%%% %%L%L% %N %%M%M% % %%%%%%%% %%%%%% flexi-streams-1.0.7/test/unicode_demo_ucs2_crlf_le.txt 0000664 0000000 0000000 00000036462 11175756704 0023126 0 ustar 00root root 0000000 0000000 U T F - 8 e n c o d e d s a m p l e p l a i n - t e x t f i l e > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > M a r k u s K u h n [ m a k s k u n ] < h t t p : / / w w w . c l . c a m . a c . u k / ~ m g k 2 5 / > 2 0 0 2 - 0 7 - 2 5 T h e A S C I I c o m p a t i b l e U T F - 8 e n c o d i n g u s e d i n t h i s p l a i n - t e x t f i l e i s d e f i n e d i n U n i c o d e , I S O 1 0 6 4 6 - 1 , a n d R F C 2 2 7 9 . U s i n g U n i c o d e / U T F - 8 , y o u c a n w r i t e i n e m a i l s a n d s o u r c e c o d e t h i n g s s u c h a s M a t h e m a t i c s a n d s c i e n c e s : ." E "d a = Q , n ! ", " f ( i ) = " g ( i ) , ### % % % % % %%### ###%a + b ### "x " !: #x # = " #"x #, '" = ( (" ) , ###% % % % % % ### #### c ### ! " ! " $! " ! " ! " !, ### ### ### " ### " < a `" b a" c d" d j" " ! ( 'A ' ! 'B ') , ### # ### ### #a q - b q ### 2 H + O ! 2 H O , R = 4 . 7 k , # 2 0 0 m m ###i = 1 ### L i n g u i s t i c s a n d d i c t i o n a r i e s : i 1n t Yn Yn Yl f Yn [t 1k Ys o s i e 1n Y [ p s i l Tn ] , Y e n [ j [n ] , Y o g a [ j o g Q] A P L : ( ( V s#V ) = s#t#V ) / V !, V 7#!s#!t#"""> N#U## N i c e r t y p o g r a p h y i n p l a i n t e x t f i l e s : T%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%W% Q% Q% Q% " s i n g l e a n d d o u b l e q u o t e s Q% Q% Q% Q% " C u r l y a p o s t r o p h e s : W e v e b e e n h e r e Q% Q% Q% Q% " L a t i n - 1 a p o s t r o p h e a n d a c c e n t s : ' ` Q% Q% Q% Q% " d e u t s c h e A n f h r u n g s z e i c h e n Q% Q% Q% Q% " , ! , 0 , " , 3 4 , , "5 / + 5 , "!, & Q% Q% Q% Q% " A S C I I s a f e t y t e s t : 1 l I | , 0 O D , 8 B Q% Q% m% % % % % % % % % %n% Q% Q% " t h e e u r o s y m b o l : % 1 4 . 9 5 % Q% Q% p% % % % % % % % % %o% Q% Z%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%]% C o m b i n i n g c h a r a c t e r s : S T A R G T E S G - 1 , a = v = r , a " b G r e e k ( i n P o l y t o n i c ) : T h e G r e e k a n t h e m : r w x t y t u , r w x t D z r w q t . p y s u p 1 q v p s , f , q ! F r o m a s p e e c h o f D e m o s t h e n e s i n t h e 4 t h c e n t u r y B C : P v P p w w } , f , E 0 p q s v E x z y S { z r p y v u w A s , p r q 0 u , e E t y P v y s s . P s V 1 p s " t Q y , v ' { , P v t V q Q q . | s , E s y v p Q v w u , v q 6 q , P q s y s s 1 x ! 6 t } , E z q } . p p w Q q , y v v w u w v C y s v r t t @ Q s , q ! v A y . s , x G e o r g i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : U n i c o d e - , 1 0 - 1 2 , . , . U n i c o d e - , , U n i c o d e - , , , . R u s s i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : 0@538AB@8@C9B5AL A59G0A =0 5AOBCN 564C=0@>4=CN >=D5@5=F8N ?> U n i c o d e , :>B>@0O A>AB>8BAO 1 0 - 1 2 <0@B0 1 9 9 7 3>40 2 09=F5 2 5@<0=88. >=D5@5=F8O A>15@5B H8@>:89 :@C3 M:A?5@B>2 ?> 2>?@>A0< 3;>10;L=>3> =B5@=5B0 8 U n i c o d e , ;>:0;870F88 8 8=B5@=0F8>=0;870F88, 2>?;>I5=8N 8 ?@8<5=5=8N U n i c o d e 2 @07;8G=KE >?5@0F8>==KE A8AB5<0E 8 ?@>3@0<<=KE ?@8;>65=8OE, H@8DB0E, 25@AB:5 8 <=>3>O7KG=KE :><?LNB5@=KE A8AB5<0E. T h a i ( U C S L e v e l 2 ) : E x c e r p t f r o m a p o e t r y o n T h e R o m a n c e o f T h e T h r e e K i n g d o m s ( a C h i n e s e c l a s s i c ' S a n G u a ' ) : [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - - - - ] O A H4.1H@*7H-!B#!A**1@' #0@(-9J9I6IC+!H *4*-)1#4"LH-+I2A%1D *--LD #IBH@%2@21 2 #17-15@G5H 6H I2@!7-6'4#4@G1+2 B.4K@#5"1 1H'+1'@!7-!2 +!2"0H2! 1H'1'*31 @+!7-1D*D%H@*7-2@+2 #1+!2H2@I2!2@%"-2*1 H2"-I--8I"8A"C+IA1 C I*2'1I@G ' 7H 'C %1%4 8"8"5%1H-@+8 H2-2@ (#4+2 I2#I-D+I I-##2H2 1##%1" $E+2C#I3 99I##%1L / ( T h e a b o v e i s a t w o - c o l u m n t e x t . I f c o m b i n i n g c h a r a c t e r s a r e h a n d l e d c o r r e c t l y , t h e l i n e s o f t h e s e c o n d c o l u m n s h o u l d b e a l i g n e d w i t h t h e | c h a r a c t e r a b o v e . ) E t h i o p i a n : P r o v e r b s i n t h e A m h a r i c l a n g u a g e : 0 s(5 % 05b e ct `F b % dq A % b ` Ed c # #u `b M s `Ed s= b % `` psb 2p( ( b @5 `@5e A ` ) b - be- `3 5-b 0 dq (dq p- b - Hp .. 3 - b (du ce bu 5E cu Eb %+ Msu Ksub c * e + b 5 ) + ) -b p bpI p 6 cIb - b (-5 u 0b - `M+= - b R u n e s : ( O l d E n g l i s h , w h i c h t r a n s c r i b e d i n t o L a t i n r e a d s ' H e c w a e t h t h a t h e b u d e t h a e m l a n d e n o r t h w e a r d u m w i t h t h a W e s t s a e . ' a n d m e a n s ' H e s a i d t h a t h e l i v e d i n t h e n o r t h e r n l a n d n e a r t h e W e s t e r n S e a . ' ) B r a i l l e : L(('(( <((( M( (((9(0(( c(( ( M( (((9( :((( ((((( (( (((( :( (9(2( y(;(( (( (( (3(( ( 1(( (('(;( ((3( ( 9(( (2( y(( ((( ( (;( ( ( ( (( (%(( ((( :((( ( (( (+( (9( 9(( ((;((9( (( (( 9(( ((;((( 9(( %( ((;( (((;(( ( (( 9(( !( (( ( (3(( (;(2( N( (((((( ( (( (+( ( (2( A( (( N( ((((((0(( (( (( :((( (((( %((( ( 0(a(( (((( ((( ( (9(9((( (( !(((( (( (%( ( ( (( (( (( ((2( U((( M( (((9( :((( (( (((( (( ( (((($( (( ((2( M(((( J( (( (0( ( ((( ( (( ((9( 9(( ( J( ( (*(( ( ( (9( *( ( ( (*((+(((( 1(( ( 9(;(( (( ( ( ( ( (%(( ((9( (((( ((3( ( ( (((($( (( ((2( J( ( (#( ( (('(( ((2( ( (((+(( (9(((( (( (( ((( (( ( (( ( (($( (( (( (( 9(( ((((( ( ( (( (( ( ( ((( ( (( ((;(9( ( 9(( (((((2( C(%( ( 9(( :( (((( ( ( ( 3(( ( ( (( (((( (( ( 9(( ( ( ( (((( ( (( (9( %( (((((*(+( (( ((( )(((( (( ( ( ( (%((( ( (( (( 9(( J(3( ( ((9(0(( (( (( (((2( y(3( :( ((( 9(;(( (((( (;( ( ( ( (( (( ((((( (( ( (((( ( ( ((((9(( 9(( ( M( (((9( :((( (( (((( (( ( (((($( (( ((2( ( T h e f i r s t c o u p l e o f p a r a g r a p h s o f " A C h r i s t m a s C a r o l " b y D i c k e n s ) C o m p a c t f o n t s e l e c t i o n e x a m p l e t e x t : A B C D E F G H I J K L M N O P Q R S T U V W X Y Z / 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z " & 0 "!S`x~ 01234 """ !'"*"a" " !!!!! %<%T%X%%%:&@& @$ PN#1 G r e e t i n g s i n v a r i o u s l a n g u a g e s : H e l l o w o r l d , s y , 00000 B o x d r a w i n g a l i g n m e n t t e s t s : % % T%P%P%f%P%P%W% % % %,% % %% m% % %,% % %n% m% % %,% % %n% %%%3%%%% %%%% w% {% %/%% %0%% % q%r%q%r%s%s%s% Q% % %h% %%Q% %T%P%g%P%W%% %R%P%j%P%U%% %S% %A% %V%% % % %B% %%% %C%D%% v%<%t%z%K%x% %<%(% %K%%% % r%q%r%q%s%s%s% Q%%r% q%%Q% %Q% Q%% %% % %% %Q% % Q%% %% % %% %E%F%% u% y% %7%% %8%% % q%r%q%r%s%s%s% `%a% s% ^%c% %b% _%$% %<% %<% %<%$% %k% %B% %k%$% #%?%~%<%|%?%+% %%%% %%%% N% %%%% % % r%q%r%q%s%s%s% Q%%q% r%%Q% %Q% Q%% %% % %% %Q% % Q%% %% }% %% %%%%%%%% % % N% O% % % % Q%% %e% %%Q% %Z%P%d%P%]%% %X%P%j%P%[%% %Y% %@% %\%% %% %B% %%% %%%%%%%% % % N% O% % % % Z%P%P%i%P%P%]% % % %4% % %% p% % %4% % %o% p% % %4% % %o% %%%;%%%% %%%%%% %L%L%% N% %M%M%% % %%%%%%%% %%%%%% flexi-streams-1.0.7/test/unicode_demo_ucs2_lf_be.txt 0000664 0000000 0000000 00000035612 11175756704 0022563 0 ustar 00root root 0000000 0000000 U T F - 8 e n c o d e d s a m p l e p l a i n - t e x t f i l e > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > M a r k u s K u h n [ m a k s k u n ] < h t t p : / / w w w . c l . c a m . a c . u k / ~ m g k 2 5 / > 2 0 0 2 - 0 7 - 2 5 T h e A S C I I c o m p a t i b l e U T F - 8 e n c o d i n g u s e d i n t h i s p l a i n - t e x t f i l e i s d e f i n e d i n U n i c o d e , I S O 1 0 6 4 6 - 1 , a n d R F C 2 2 7 9 . U s i n g U n i c o d e / U T F - 8 , y o u c a n w r i t e i n e m a i l s a n d s o u r c e c o d e t h i n g s s u c h a s M a t h e m a t i c s a n d s c i e n c e s : ". E" d a = Q , n ! " , " f ( i ) = " g ( i ) , ###% % % % % % %### ###% a + b ### " x"! : # x# = "# " x# , "' = ( "( ) , ###%% % % % % ### #### c ### ! " ! " !$ " ! " ! " ! , ### ### ### " ### " < a "` b "a c "d d "j " ! (' A' ! ' B' ) , ### # ### ### # a q - b q### 2 H + O ! 2 H O , R = 4 . 7 k , # 2 0 0 m m ### i = 1 ### L i n g u i s t i c s a n d d i c t i o n a r i e s : i 1 n tY n Y nY l fY n[ t1 k Y s o s i e1 n Y [ p s i lT n ] , Y e n [ j[ n ] , Y o g a [ j o gQ ] A P L : ( ( V#s V ) =#s#t V ) / V! , V #7!#s!#t""" >#N#U# N i c e r t y p o g r a p h y i n p l a i n t e x t f i l e s : %T%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%W %Q %Q %Q " s i n g l e a n d d o u b l e q u o t e s %Q %Q %Q %Q " C u r l y a p o s t r o p h e s : W e v e b e e n h e r e %Q %Q %Q %Q " L a t i n - 1 a p o s t r o p h e a n d a c c e n t s : ' ` %Q %Q %Q %Q " d e u t s c h e A n f h r u n g s z e i c h e n %Q %Q %Q %Q " , ! , 0 , " , 3 4 , , " 5 / + 5 , !" , & %Q %Q %Q %Q " A S C I I s a f e t y t e s t : 1 l I | , 0 O D , 8 B %Q %Q %m% % % % % % % % % %n %Q %Q " t h e e u r o s y m b o l : % 1 4 . 9 5 % %Q %Q %p% % % % % % % % % %o %Q %Z%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%] C o m b i n i n g c h a r a c t e r s : S T A R G T E S G - 1 , a = v = r , a " b G r e e k ( i n P o l y t o n i c ) : T h e G r e e k a n t h e m : r w x t y t u , r w x t D z r w q t . p y s u p 1 q v p s , f , q ! F r o m a s p e e c h o f D e m o s t h e n e s i n t h e 4 t h c e n t u r y B C : P v P p w w } , f , E 0 p q s v E x z y S { z r p y v u w A s , p r q 0 u , e E t y P v y s s . P s V 1 p s " t Q y , v ' { , P v t V q Q q . | s , E s y v p Q v w u , v q 6 q , P q s y s s 1 x ! 6 t } , E z q } . p p w Q q , y v v w u w v C y s v r t t @ Q s , q ! v A y . s , x G e o r g i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : U n i c o d e - , 1 0 - 1 2 , . , . U n i c o d e - , , U n i c o d e - , , , . R u s s i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : 0@538AB@8@C9B5AL A59G0A =0 5AOBCN 564C=0@>4=CN >=D5@5=F8N ?> U n i c o d e , :>B>@0O A>AB>8BAO 1 0 - 1 2 <0@B0 1 9 9 7 3>40 2 09=F5 2 5@<0=88 . >=D5@5=F8O A>15@5B H8@>:89 :@C3 M:A?5@B>2 ?> 2>?@>A0< 3;>10;L=>3> =B5@=5B0 8 U n i c o d e , ;>:0;870F88 8 8=B5@=0F8>=0;870F88 , 2>?;>I5=8N 8 ?@8<5=5=8N U n i c o d e 2 @07;8G=KE >?5@0F8>==KE A8AB5<0E 8 ?@>3@0<<=KE ?@8;>65=8OE , H@8DB0E , 25@AB:5 8 <=>3>O7KG=KE :><?LNB5@=KE A8AB5<0E . T h a i ( U C S L e v e l 2 ) : E x c e r p t f r o m a p o e t r y o n T h e R o m a n c e o f T h e T h r e e K i n g d o m s ( a C h i n e s e c l a s s i c ' S a n G u a ' ) : [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - - - - ] O A H4.1H@*7H-!B#!A**1@' #0@(-9J9I6IC+!H *4*-)1#4"LH-+I2A%1D *--LD #IBH@%2@21 2 #17-15@G5H 6H I2@!7-6'4#4@G1+2 B.4K@#5"1 1H'+1'@!7-!2 +!2"0H2! 1H'1'*31 @+!7-1D*D%H@*7-2@+2 #1+!2H2@I2!2@%"-2*1 H2"-I--8I"8A"C+IA1 C I*2'1I@G ' 7H 'C %1%4 8"8"5%1H-@+8 H2-2@ (#4+2 I2#I-D+I I-##2H2 1##%1" $E+2C#I3 99I##%1L / ( T h e a b o v e i s a t w o - c o l u m n t e x t . I f c o m b i n i n g c h a r a c t e r s a r e h a n d l e d c o r r e c t l y , t h e l i n e s o f t h e s e c o n d c o l u m n s h o u l d b e a l i g n e d w i t h t h e | c h a r a c t e r a b o v e . ) E t h i o p i a n : P r o v e r b s i n t h e A m h a r i c l a n g u a g e : 0 s(5 % 05b e ct `F b % dq A % b ` Ed c # #u `b M s `Ed s= b % `` psb 2p( ( b @5 `@5e A ` ) b - be- `3 5-b 0 dq (dq p- b - Hp .. 3 - b (du ce bu 5E cu Eb %+ Msu Ksub c * e + b 5 ) + ) -b p bpI p 6 cIb - b (-5 u 0b - `M+= - b R u n e s : ( O l d E n g l i s h , w h i c h t r a n s c r i b e d i n t o L a t i n r e a d s ' H e c w a e t h t h a t h e b u d e t h a e m l a n d e n o r t h w e a r d u m w i t h t h a W e s t s a e . ' a n d m e a n s ' H e s a i d t h a t h e l i v e d i n t h e n o r t h e r n l a n d n e a r t h e W e s t e r n S e a . ' ) B r a i l l e : (L(('( (<(( (M( (((9(0( (c(( (M( (((9 (:(( ((((( ( ( (((( (:( (9(2 (y(;( ( ( ( ( ((3(( (1(( (('(; (((3( (9(( (2 (y( (((( ( (; (( (( ( ((%(( (( (:(( (( (( (+ ((9 (9( ( ((;((9( (( ( (9( ( ((;(( (9( (%( ((;( (((;( (( ( (9( (!( (( ( (3(( (;(2 (N( ((((( (( (( (+ ( ( (2 (A( ( (N( ((((((0( ( (( ( (:(( (((( (%((( (0(a(( ((( ( (( (( (9(9(( (( (!((( ( ( ((%( (( ( ((( ( ( ((2 (U(( (M( (((9 (:(( (( (((( (( ( ((((($( (( ((2 (M((( (J ((( (0( ( ((( ( ( (((9 (9(( (J (( (*( (( ( (9 (*( (( (*((+((( (1(( (9(;( ( ( (( ( ( ( (%(( ((9 (((( (((3( ( ((((($( (( ((2 (J ( ( (#( ((('( (((2 (( (((+( ( (9(((( ( ( ( (((( ( ( ( (( ( (($( (( ( (( (9( (((((( (( (( ( (( ( ((( ( (( ((;(9 ( (9( ( (((((2 (C(%( (9( (:( (((( (( (3( (( ( (( ((( ( ( ( (9( (( ( ( ((( (( ( ( (9 (%( (((((*(+ ((( (( ()((( ( (( (( ( (%(( ( ( ( (( (9( (J(3( ( ((9(0( ((( ( ( (((2 (y(3 (:( (( (9(;(( ((( ((;( ( ( ( ( ( ( (((((( ( (( (((( ( ( ((((9( (9(( (M( (((9 (:(( (( (((( (( ( ((((($( (( ((2 ( T h e f i r s t c o u p l e o f p a r a g r a p h s o f " A C h r i s t m a s C a r o l " b y D i c k e n s ) C o m p a c t f o n t s e l e c t i o n e x a m p l e t e x t : A B C D E F G H I J K L M N O P Q R S T U V W X Y Z / 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z " & 0!"S`x~ 01234 " ""! "'"*"a" !!!!! %%<%T%X%%&:&@ $@ P#N1 G r e e t i n g s i n v a r i o u s l a n g u a g e s : H e l l o w o r l d , s y , 00000 B o x d r a w i n g a l i g n m e n t t e s t s : % % %T%P%P%f%P%P%W % % % %,% % % %m% % %,% % %n %m% % %,% % %n %%%%3%%% %%%% %w %{ %%/% % %0% % %q%r%q%r%s%s%s %Q% % %h% %%Q %%T%P%g%P%W% %%R%P%j%P%U% %%S% %A% %V% %% % %B% %% %%C%D% %v%<%t%z%K%x% %<%( % %K%% % %r%q%r%q%s%s%s %Q%%r %q%%Q %%Q %Q% %% % %% %%Q % %Q% %% % %% % %E%F% %u %y %%7% %%8% % %q%r%q%r%s%s%s %`%a %s %^%c % %b %_%$ % %<% %<% %<%$ % %k% %B% %k%$ %#%?%~%<%|%?%+ %%%% % %%% %N %%%% % % %r%q%r%q%s%s%s %Q%%q %r%%Q %%Q %Q% %% % %% %%Q % %Q% %% %} %% %%%%%%%% % % %N %O % % % %Q%% %e% %%Q %%Z%P%d%P%]% %%X%P%j%P%[% %%Y% %@% %\% %%% %B% %% %%%%%%%% % % %N %O % % % %Z%P%P%i%P%P%] %% % %4% % % %p% % %4% % %o %p% % %4% % %o %%%%;%%% %%%%%% %%L%L% %N %%M%M% % %%%%%%%% %%%%%% flexi-streams-1.0.7/test/unicode_demo_ucs2_lf_le.txt 0000664 0000000 0000000 00000035612 11175756704 0022575 0 ustar 00root root 0000000 0000000 U T F - 8 e n c o d e d s a m p l e p l a i n - t e x t f i l e > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > M a r k u s K u h n [ m a k s k u n ] < h t t p : / / w w w . c l . c a m . a c . u k / ~ m g k 2 5 / > 2 0 0 2 - 0 7 - 2 5 T h e A S C I I c o m p a t i b l e U T F - 8 e n c o d i n g u s e d i n t h i s p l a i n - t e x t f i l e i s d e f i n e d i n U n i c o d e , I S O 1 0 6 4 6 - 1 , a n d R F C 2 2 7 9 . U s i n g U n i c o d e / U T F - 8 , y o u c a n w r i t e i n e m a i l s a n d s o u r c e c o d e t h i n g s s u c h a s M a t h e m a t i c s a n d s c i e n c e s : ." E "d a = Q , n ! ", " f ( i ) = " g ( i ) , ### % % % % % %%### ###%a + b ### "x " !: #x # = " #"x #, '" = ( (" ) , ###% % % % % % ### #### c ### ! " ! " $! " ! " ! " !, ### ### ### " ### " < a `" b a" c d" d j" " ! ( 'A ' ! 'B ') , ### # ### ### #a q - b q ### 2 H + O ! 2 H O , R = 4 . 7 k , # 2 0 0 m m ###i = 1 ### L i n g u i s t i c s a n d d i c t i o n a r i e s : i 1n t Yn Yn Yl f Yn [t 1k Ys o s i e 1n Y [ p s i l Tn ] , Y e n [ j [n ] , Y o g a [ j o g Q] A P L : ( ( V s#V ) = s#t#V ) / V !, V 7#!s#!t#"""> N#U## N i c e r t y p o g r a p h y i n p l a i n t e x t f i l e s : T%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%W% Q% Q% Q% " s i n g l e a n d d o u b l e q u o t e s Q% Q% Q% Q% " C u r l y a p o s t r o p h e s : W e v e b e e n h e r e Q% Q% Q% Q% " L a t i n - 1 a p o s t r o p h e a n d a c c e n t s : ' ` Q% Q% Q% Q% " d e u t s c h e A n f h r u n g s z e i c h e n Q% Q% Q% Q% " , ! , 0 , " , 3 4 , , "5 / + 5 , "!, & Q% Q% Q% Q% " A S C I I s a f e t y t e s t : 1 l I | , 0 O D , 8 B Q% Q% m% % % % % % % % % %n% Q% Q% " t h e e u r o s y m b o l : % 1 4 . 9 5 % Q% Q% p% % % % % % % % % %o% Q% Z%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%P%]% C o m b i n i n g c h a r a c t e r s : S T A R G T E S G - 1 , a = v = r , a " b G r e e k ( i n P o l y t o n i c ) : T h e G r e e k a n t h e m : r w x t y t u , r w x t D z r w q t . p y s u p 1 q v p s , f , q ! F r o m a s p e e c h o f D e m o s t h e n e s i n t h e 4 t h c e n t u r y B C : P v P p w w } , f , E 0 p q s v E x z y S { z r p y v u w A s , p r q 0 u , e E t y P v y s s . P s V 1 p s " t Q y , v ' { , P v t V q Q q . | s , E s y v p Q v w u , v q 6 q , P q s y s s 1 x ! 6 t } , E z q } . p p w Q q , y v v w u w v C y s v r t t @ Q s , q ! v A y . s , x G e o r g i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : U n i c o d e - , 1 0 - 1 2 , . , . U n i c o d e - , , U n i c o d e - , , , . R u s s i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : 0@538AB@8@C9B5AL A59G0A =0 5AOBCN 564C=0@>4=CN >=D5@5=F8N ?> U n i c o d e , :>B>@0O A>AB>8BAO 1 0 - 1 2 <0@B0 1 9 9 7 3>40 2 09=F5 2 5@<0=88. >=D5@5=F8O A>15@5B H8@>:89 :@C3 M:A?5@B>2 ?> 2>?@>A0< 3;>10;L=>3> =B5@=5B0 8 U n i c o d e , ;>:0;870F88 8 8=B5@=0F8>=0;870F88, 2>?;>I5=8N 8 ?@8<5=5=8N U n i c o d e 2 @07;8G=KE >?5@0F8>==KE A8AB5<0E 8 ?@>3@0<<=KE ?@8;>65=8OE, H@8DB0E, 25@AB:5 8 <=>3>O7KG=KE :><?LNB5@=KE A8AB5<0E. T h a i ( U C S L e v e l 2 ) : E x c e r p t f r o m a p o e t r y o n T h e R o m a n c e o f T h e T h r e e K i n g d o m s ( a C h i n e s e c l a s s i c ' S a n G u a ' ) : [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - - - - ] O A H4.1H@*7H-!B#!A**1@' #0@(-9J9I6IC+!H *4*-)1#4"LH-+I2A%1D *--LD #IBH@%2@21 2 #17-15@G5H 6H I2@!7-6'4#4@G1+2 B.4K@#5"1 1H'+1'@!7-!2 +!2"0H2! 1H'1'*31 @+!7-1D*D%H@*7-2@+2 #1+!2H2@I2!2@%"-2*1 H2"-I--8I"8A"C+IA1 C I*2'1I@G ' 7H 'C %1%4 8"8"5%1H-@+8 H2-2@ (#4+2 I2#I-D+I I-##2H2 1##%1" $E+2C#I3 99I##%1L / ( T h e a b o v e i s a t w o - c o l u m n t e x t . I f c o m b i n i n g c h a r a c t e r s a r e h a n d l e d c o r r e c t l y , t h e l i n e s o f t h e s e c o n d c o l u m n s h o u l d b e a l i g n e d w i t h t h e | c h a r a c t e r a b o v e . ) E t h i o p i a n : P r o v e r b s i n t h e A m h a r i c l a n g u a g e : 0 s(5 % 05b e ct `F b % dq A % b ` Ed c # #u `b M s `Ed s= b % `` psb 2p( ( b @5 `@5e A ` ) b - be- `3 5-b 0 dq (dq p- b - Hp .. 3 - b (du ce bu 5E cu Eb %+ Msu Ksub c * e + b 5 ) + ) -b p bpI p 6 cIb - b (-5 u 0b - `M+= - b R u n e s : ( O l d E n g l i s h , w h i c h t r a n s c r i b e d i n t o L a t i n r e a d s ' H e c w a e t h t h a t h e b u d e t h a e m l a n d e n o r t h w e a r d u m w i t h t h a W e s t s a e . ' a n d m e a n s ' H e s a i d t h a t h e l i v e d i n t h e n o r t h e r n l a n d n e a r t h e W e s t e r n S e a . ' ) B r a i l l e : L(('(( <((( M( (((9(0(( c(( ( M( (((9( :((( ((((( (( (((( :( (9(2( y(;(( (( (( (3(( ( 1(( (('(;( ((3( ( 9(( (2( y(( ((( ( (;( ( ( ( (( (%(( ((( :((( ( (( (+( (9( 9(( ((;((9( (( (( 9(( ((;((( 9(( %( ((;( (((;(( ( (( 9(( !( (( ( (3(( (;(2( N( (((((( ( (( (+( ( (2( A( (( N( ((((((0(( (( (( :((( (((( %((( ( 0(a(( (((( ((( ( (9(9((( (( !(((( (( (%( ( ( (( (( (( ((2( U((( M( (((9( :((( (( (((( (( ( (((($( (( ((2( M(((( J( (( (0( ( ((( ( (( ((9( 9(( ( J( ( (*(( ( ( (9( *( ( ( (*((+(((( 1(( ( 9(;(( (( ( ( ( ( (%(( ((9( (((( ((3( ( ( (((($( (( ((2( J( ( (#( ( (('(( ((2( ( (((+(( (9(((( (( (( ((( (( ( (( ( (($( (( (( (( 9(( ((((( ( ( (( (( ( ( ((( ( (( ((;(9( ( 9(( (((((2( C(%( ( 9(( :( (((( ( ( ( 3(( ( ( (( (((( (( ( 9(( ( ( ( (((( ( (( (9( %( (((((*(+( (( ((( )(((( (( ( ( ( (%((( ( (( (( 9(( J(3( ( ((9(0(( (( (( (((2( y(3( :( ((( 9(;(( (((( (;( ( ( ( (( (( ((((( (( ( (((( ( ( ((((9(( 9(( ( M( (((9( :((( (( (((( (( ( (((($( (( ((2( ( T h e f i r s t c o u p l e o f p a r a g r a p h s o f " A C h r i s t m a s C a r o l " b y D i c k e n s ) C o m p a c t f o n t s e l e c t i o n e x a m p l e t e x t : A B C D E F G H I J K L M N O P Q R S T U V W X Y Z / 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z " & 0 "!S`x~ 01234 """ !'"*"a" " !!!!! %<%T%X%%%:&@& @$ PN#1 G r e e t i n g s i n v a r i o u s l a n g u a g e s : H e l l o w o r l d , s y , 00000 B o x d r a w i n g a l i g n m e n t t e s t s : % % T%P%P%f%P%P%W% % % %,% % %% m% % %,% % %n% m% % %,% % %n% %%%3%%%% %%%% w% {% %/%% %0%% % q%r%q%r%s%s%s% Q% % %h% %%Q% %T%P%g%P%W%% %R%P%j%P%U%% %S% %A% %V%% % % %B% %%% %C%D%% v%<%t%z%K%x% %<%(% %K%%% % r%q%r%q%s%s%s% Q%%r% q%%Q% %Q% Q%% %% % %% %Q% % Q%% %% % %% %E%F%% u% y% %7%% %8%% % q%r%q%r%s%s%s% `%a% s% ^%c% %b% _%$% %<% %<% %<%$% %k% %B% %k%$% #%?%~%<%|%?%+% %%%% %%%% N% %%%% % % r%q%r%q%s%s%s% Q%%q% r%%Q% %Q% Q%% %% % %% %Q% % Q%% %% }% %% %%%%%%%% % % N% O% % % % Q%% %e% %%Q% %Z%P%d%P%]%% %X%P%j%P%[%% %Y% %@% %\%% %% %B% %%% %%%%%%%% % % N% O% % % % Z%P%P%i%P%P%]% % % %4% % %% p% % %4% % %o% p% % %4% % %o% %%%;%%%% %%%%%% %L%L%% N% %M%M%% % %%%%%%%% %%%%%% flexi-streams-1.0.7/test/unicode_demo_ucs4_cr_be.txt 0000664 0000000 0000000 00000073424 11175756704 0022573 0 ustar 00root root 0000000 0000000 U T F - 8 e n c o d e d s a m p l e p l a i n - t e x t f i l e > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > M a r k u s K u h n [ m a k s k u n ] < h t t p : / / w w w . c l . c a m . a c . u k / ~ m g k 2 5 / > 2 0 0 2 - 0 7 - 2 5 T h e A S C I I c o m p a t i b l e U T F - 8 e n c o d i n g u s e d i n t h i s p l a i n - t e x t f i l e i s d e f i n e d i n U n i c o d e , I S O 1 0 6 4 6 - 1 , a n d R F C 2 2 7 9 . U s i n g U n i c o d e / U T F - 8 , y o u c a n w r i t e i n e m a i l s a n d s o u r c e c o d e t h i n g s s u c h a s M a t h e m a t i c s a n d s c i e n c e s : ". E " d a = Q , n ! " , " f ( i ) = " g ( i ) , # # # % % % % % % % # # # # # # % a + b # # # " x " ! : # x # = " # " x # , "' = ( "( ) , # # # % % % % % % # # # # # # # c # # # ! " ! " !$ " ! " ! " ! , # # # # # # # # # " # # # " < a "` b "a c "d d "j " ! ( ' A ' ! ' B ' ) , # # # # # # # # # # # a q - b q # # # 2 H + O ! 2 H O , R = 4 . 7 k , # 2 0 0 m m # # # i = 1 # # # L i n g u i s t i c s a n d d i c t i o n a r i e s : i 1 n t Y n Y n Y l f Y n [ t 1 k Y s o s i e 1 n Y [ p s i l T n ] , Y e n [ j [ n ] , Y o g a [ j o g Q ] A P L : ( ( V #s V ) = #s #t V ) / V ! , V #7 ! #s ! #t " " " > #N #U # N i c e r t y p o g r a p h y i n p l a i n t e x t f i l e s : %T %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %W %Q %Q %Q " s i n g l e a n d d o u b l e q u o t e s %Q %Q %Q %Q " C u r l y a p o s t r o p h e s : W e v e b e e n h e r e %Q %Q %Q %Q " L a t i n - 1 a p o s t r o p h e a n d a c c e n t s : ' ` %Q %Q %Q %Q " d e u t s c h e A n f h r u n g s z e i c h e n %Q %Q %Q %Q " , ! , 0 , " , 3 4 , , " 5 / + 5 , !" , & %Q %Q %Q %Q " A S C I I s a f e t y t e s t : 1 l I | , 0 O D , 8 B %Q %Q %m % % % % % % % % % %n %Q %Q " t h e e u r o s y m b o l : % 1 4 . 9 5 % %Q %Q %p % % % % % % % % % %o %Q %Z %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %] C o m b i n i n g c h a r a c t e r s : S T A R G T E S G - 1 , a = v = r , a " b G r e e k ( i n P o l y t o n i c ) : T h e G r e e k a n t h e m : r w x t y t u , r w x t D z r w q t . p y s u p 1 q v p s , f , q ! F r o m a s p e e c h o f D e m o s t h e n e s i n t h e 4 t h c e n t u r y B C : P v P p w w } , f , E 0 p q s v E x z y S { z r p y v u w A s , p r q 0 u , e E t y P v y s s . P s V 1 p s " t Q y , v ' { , P v t V q Q q . | s , E s y v p Q v w u , v q 6 q , P q s y s s 1 x ! 6 t } , E z q } . p p w Q q , y v v w u w v C y s v r t t @ Q s , q ! v A y . s , x G e o r g i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : U n i c o d e - , 1 0 - 1 2 , . , . U n i c o d e - , , U n i c o d e - , , , . R u s s i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : 0 @ 5 3 8 A B @ 8 @ C 9 B 5 A L A 5 9 G 0 A = 0 5 A O B C N 5 6 4 C = 0 @ > 4 = C N > = D 5 @ 5 = F 8 N ? > U n i c o d e , : > B > @ 0 O A > A B > 8 B A O 1 0 - 1 2 < 0 @ B 0 1 9 9 7 3 > 4 0 2 0 9 = F 5 2 5 @ < 0 = 8 8 . > = D 5 @ 5 = F 8 O A > 1 5 @ 5 B H 8 @ > : 8 9 : @ C 3 M : A ? 5 @ B > 2 ? > 2 > ? @ > A 0 < 3 ; > 1 0 ; L = > 3 > = B 5 @ = 5 B 0 8 U n i c o d e , ; > : 0 ; 8 7 0 F 8 8 8 8 = B 5 @ = 0 F 8 > = 0 ; 8 7 0 F 8 8 , 2 > ? ; > I 5 = 8 N 8 ? @ 8 < 5 = 5 = 8 N U n i c o d e 2 @ 0 7 ; 8 G = K E > ? 5 @ 0 F 8 > = = K E A 8 A B 5 < 0 E 8 ? @ > 3 @ 0 < < = K E ? @ 8 ; > 6 5 = 8 O E , H @ 8 D B 0 E , 2 5 @ A B : 5 8 < = > 3 > O 7 K G = K E : > < ? L N B 5 @ = K E A 8 A B 5 < 0 E . T h a i ( U C S L e v e l 2 ) : E x c e r p t f r o m a p o e t r y o n T h e R o m a n c e o f T h e T h r e e K i n g d o m s ( a C h i n e s e c l a s s i c ' S a n G u a ' ) : [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - - - - ] O A H 4 . 1 H @ * 7 H - ! B # ! A * * 1 @ ' # 0 @ ( - 9 J 9 I 6 I C + ! H * 4 * - ) 1 # 4 " L H - + I 2 A % 1 D * - - L D # I B H @ % 2 @ 2 1 2 # 1 7 - 1 5 @ G 5 H 6 H I 2 @ ! 7 - 6 ' 4 # 4 @ G 1 + 2 B . 4 K @ # 5 " 1 1 H ' + 1 ' @ ! 7 - ! 2 + ! 2 " 0 H 2 ! 1 H ' 1 ' * 3 1 @ + ! 7 - 1 D * D % H @ * 7 - 2 @ + 2 # 1 + ! 2 H 2 @ I 2 ! 2 @ % " - 2 * 1 H 2 " - I - - 8 I " 8 A " C + I A 1 C I * 2 ' 1 I @ G ' 7 H ' C % 1 % 4 8 " 8 " 5 % 1 H - @ + 8 H 2 - 2 @ ( # 4 + 2 I 2 # I - D + I I - # # 2 H 2 1 # # % 1 " $ E + 2 C # I 3 9 9 I # # % 1 L / ( T h e a b o v e i s a t w o - c o l u m n t e x t . I f c o m b i n i n g c h a r a c t e r s a r e h a n d l e d c o r r e c t l y , t h e l i n e s o f t h e s e c o n d c o l u m n s h o u l d b e a l i g n e d w i t h t h e | c h a r a c t e r a b o v e . ) E t h i o p i a n : P r o v e r b s i n t h e A m h a r i c l a n g u a g e : 0 s ( 5 % 0 5 b e c t ` F b % d q A % b ` E d c # # u ` b M s ` E d s = b % ` ` p s b 2 p ( ( b @ 5 ` @ 5 e A ` ) b - b e - ` 3 5 - b 0 d q ( d q p - b - H p . . 3 - b ( d u c e b u 5 E c u E b % + M s u K s u b c * e + b 5 ) + ) - b p b p I p 6 c I b - b ( - 5 u 0 b - ` M + = - b R u n e s : ( O l d E n g l i s h , w h i c h t r a n s c r i b e d i n t o L a t i n r e a d s ' H e c w a e t h t h a t h e b u d e t h a e m l a n d e n o r t h w e a r d u m w i t h t h a W e s t s a e . ' a n d m e a n s ' H e s a i d t h a t h e l i v e d i n t h e n o r t h e r n l a n d n e a r t h e W e s t e r n S e a . ' ) B r a i l l e : (L ( (' ( (< ( ( (M ( ( ( (9 (0 ( (c ( ( (M ( ( ( (9 (: ( ( ( ( ( ( ( ( ( ( ( ( ( (: ( (9 (2 (y (; ( ( ( ( ( ( (3 ( ( (1 ( ( ( (' (; ( ( (3 ( (9 ( ( (2 (y ( ( ( ( ( ( (; ( ( ( ( ( ( (% ( ( ( ( (: ( ( ( ( ( ( (+ ( (9 (9 ( ( ( (; ( (9 ( ( ( ( (9 ( ( ( (; ( ( (9 ( (% ( ( (; ( ( ( (; ( ( ( ( (9 ( (! ( ( ( ( (3 ( ( (; (2 (N ( ( ( ( ( ( ( ( ( ( (+ ( ( (2 (A ( ( (N ( ( ( ( ( ( (0 ( ( ( ( ( (: ( ( ( ( ( ( (% ( ( ( (0 (a ( ( ( ( ( ( ( ( ( ( (9 (9 ( ( ( ( (! ( ( ( ( ( ( (% ( ( ( ( ( ( ( ( ( ( (2 (U ( ( (M ( ( ( (9 (: ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ($ ( ( ( ( (2 (M ( ( ( (J ( ( ( (0 ( ( ( ( ( ( ( ( ( (9 (9 ( ( (J ( ( (* ( ( ( ( (9 (* ( ( ( (* ( (+ ( ( ( (1 ( ( (9 (; ( ( ( ( ( ( ( ( (% ( ( ( (9 ( ( ( ( ( ( (3 ( ( ( ( ( ( ($ ( ( ( ( (2 (J ( ( (# ( ( ( (' ( ( ( (2 ( ( ( ( (+ ( ( (9 ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ($ ( ( ( ( ( ( (9 ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (; (9 ( (9 ( ( ( ( ( ( (2 (C (% ( (9 ( (: ( ( ( ( ( ( ( (3 ( ( ( ( ( ( ( ( ( ( ( ( (9 ( ( ( ( ( ( ( ( ( ( ( ( (9 (% ( ( ( ( ( (* (+ ( ( ( ( ( () ( ( ( ( ( ( ( ( ( (% ( ( ( ( ( ( ( (9 ( (J (3 ( ( ( (9 (0 ( ( ( ( ( ( ( ( (2 (y (3 (: ( ( ( (9 (; ( ( ( ( ( ( (; ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (9 ( (9 ( ( (M ( ( ( (9 (: ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ($ ( ( ( ( (2 ( T h e f i r s t c o u p l e o f p a r a g r a p h s o f " A C h r i s t m a s C a r o l " b y D i c k e n s ) C o m p a c t f o n t s e l e c t i o n e x a m p l e t e x t : A B C D E F G H I J K L M N O P Q R S T U V W X Y Z / 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z " & 0 !" S ` x ~ 0 1 2 3 4 " " " ! "' "* "a " ! ! ! ! ! % %< %T %X % % &: &@ $@ P #N 1 G r e e t i n g s i n v a r i o u s l a n g u a g e s : H e l l o w o r l d , s y , 0 0 0 0 0 B o x d r a w i n g a l i g n m e n t t e s t s : % % %T %P %P %f %P %P %W % % % %, % % % %m % % %, % % %n %m % % %, % % %n % % % %3 % % % % % % % %w %{ % %/ % % %0 % % %q %r %q %r %s %s %s %Q % % %h % % %Q % %T %P %g %P %W % % %R %P %j %P %U % % %S % %A % %V % % % % %B % % % % %C %D % %v %< %t %z %K %x % %< %( % %K %% % %r %q %r %q %s %s %s %Q % %r %q % %Q % %Q %Q % % % % % % % %Q % %Q % % % % % % % %E %F % %u %y % %7 % % %8 % % %q %r %q %r %s %s %s %` %a %s %^ %c % %b %_ %$ % %< % %< % %< %$ % %k % %B % %k %$ %# %? %~ %< %| %? %+ % % % % % % % % %N % % % % % % %r %q %r %q %s %s %s %Q % %q %r % %Q % %Q %Q % % % % % % % %Q % %Q % % % %} % % % % % % % % % % % % %N %O % % % %Q % % %e % % %Q % %Z %P %d %P %] % % %X %P %j %P %[ % % %Y % %@ % %\ % % % % %B % % % % % % % % % % % % % %N %O % % % %Z %P %P %i %P %P %] % % % %4 % % % %p % % %4 % % %o %p % % %4 % % %o % % % %; % % % % % % % % % % %L %L % %N % %M %M % % % % % % % % % % % % % % % % flexi-streams-1.0.7/test/unicode_demo_ucs4_cr_le.txt 0000664 0000000 0000000 00000073424 11175756704 0022605 0 ustar 00root root 0000000 0000000 U T F - 8 e n c o d e d s a m p l e p l a i n - t e x t f i l e > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > M a r k u s K u h n [ m a k s k u n ] < h t t p : / / w w w . c l . c a m . a c . u k / ~ m g k 2 5 / > 2 0 0 2 - 0 7 - 2 5 T h e A S C I I c o m p a t i b l e U T F - 8 e n c o d i n g u s e d i n t h i s p l a i n - t e x t f i l e i s d e f i n e d i n U n i c o d e , I S O 1 0 6 4 6 - 1 , a n d R F C 2 2 7 9 . U s i n g U n i c o d e / U T F - 8 , y o u c a n w r i t e i n e m a i l s a n d s o u r c e c o d e t h i n g s s u c h a s M a t h e m a t i c s a n d s c i e n c e s : ." E " d a = Q , n ! " , " f ( i ) = " g ( i ) , # # # % % % % % % % # # # # # # % a + b # # # " x " ! : # x # = " # " x # , '" = ( (" ) , # # # % % % % % % # # # # # # # c # # # ! " ! " $! " ! " ! " ! , # # # # # # # # # " # # # " < a `" b a" c d" d j" " ! ( ' A ' ! ' B ' ) , # # # # # # # # # # # a q - b q # # # 2 H + O ! 2 H O , R = 4 . 7 k , # 2 0 0 m m # # # i = 1 # # # L i n g u i s t i c s a n d d i c t i o n a r i e s : i 1 n t Y n Y n Y l f Y n [ t 1 k Y s o s i e 1 n Y [ p s i l T n ] , Y e n [ j [ n ] , Y o g a [ j o g Q ] A P L : ( ( V s# V ) = s# t# V ) / V ! , V 7# ! s# ! t# " " " > N# U# # N i c e r t y p o g r a p h y i n p l a i n t e x t f i l e s : T% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% W% Q% Q% Q% " s i n g l e a n d d o u b l e q u o t e s Q% Q% Q% Q% " C u r l y a p o s t r o p h e s : W e v e b e e n h e r e Q% Q% Q% Q% " L a t i n - 1 a p o s t r o p h e a n d a c c e n t s : ' ` Q% Q% Q% Q% " d e u t s c h e A n f h r u n g s z e i c h e n Q% Q% Q% Q% " , ! , 0 , " , 3 4 , , " 5 / + 5 , "! , & Q% Q% Q% Q% " A S C I I s a f e t y t e s t : 1 l I | , 0 O D , 8 B Q% Q% m% % % % % % % % % % n% Q% Q% " t h e e u r o s y m b o l : % 1 4 . 9 5 % Q% Q% p% % % % % % % % % % o% Q% Z% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% ]% C o m b i n i n g c h a r a c t e r s : S T A R G T E S G - 1 , a = v = r , a " b G r e e k ( i n P o l y t o n i c ) : T h e G r e e k a n t h e m : r w x t y t u , r w x t D z r w q t . p y s u p 1 q v p s , f , q ! F r o m a s p e e c h o f D e m o s t h e n e s i n t h e 4 t h c e n t u r y B C : P v P p w w } , f , E 0 p q s v E x z y S { z r p y v u w A s , p r q 0 u , e E t y P v y s s . P s V 1 p s " t Q y , v ' { , P v t V q Q q . | s , E s y v p Q v w u , v q 6 q , P q s y s s 1 x ! 6 t } , E z q } . p p w Q q , y v v w u w v C y s v r t t @ Q s , q ! v A y . s , x G e o r g i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : U n i c o d e - , 1 0 - 1 2 , . , . U n i c o d e - , , U n i c o d e - , , , . R u s s i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : 0 @ 5 3 8 A B @ 8 @ C 9 B 5 A L A 5 9 G 0 A = 0 5 A O B C N 5 6 4 C = 0 @ > 4 = C N > = D 5 @ 5 = F 8 N ? > U n i c o d e , : > B > @ 0 O A > A B > 8 B A O 1 0 - 1 2 < 0 @ B 0 1 9 9 7 3 > 4 0 2 0 9 = F 5 2 5 @ < 0 = 8 8 . > = D 5 @ 5 = F 8 O A > 1 5 @ 5 B H 8 @ > : 8 9 : @ C 3 M : A ? 5 @ B > 2 ? > 2 > ? @ > A 0 < 3 ; > 1 0 ; L = > 3 > = B 5 @ = 5 B 0 8 U n i c o d e , ; > : 0 ; 8 7 0 F 8 8 8 8 = B 5 @ = 0 F 8 > = 0 ; 8 7 0 F 8 8 , 2 > ? ; > I 5 = 8 N 8 ? @ 8 < 5 = 5 = 8 N U n i c o d e 2 @ 0 7 ; 8 G = K E > ? 5 @ 0 F 8 > = = K E A 8 A B 5 < 0 E 8 ? @ > 3 @ 0 < < = K E ? @ 8 ; > 6 5 = 8 O E , H @ 8 D B 0 E , 2 5 @ A B : 5 8 < = > 3 > O 7 K G = K E : > < ? L N B 5 @ = K E A 8 A B 5 < 0 E . T h a i ( U C S L e v e l 2 ) : E x c e r p t f r o m a p o e t r y o n T h e R o m a n c e o f T h e T h r e e K i n g d o m s ( a C h i n e s e c l a s s i c ' S a n G u a ' ) : [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - - - - ] O A H 4 . 1 H @ * 7 H - ! B # ! A * * 1 @ ' # 0 @ ( - 9 J 9 I 6 I C + ! H * 4 * - ) 1 # 4 " L H - + I 2 A % 1 D * - - L D # I B H @ % 2 @ 2 1 2 # 1 7 - 1 5 @ G 5 H 6 H I 2 @ ! 7 - 6 ' 4 # 4 @ G 1 + 2 B . 4 K @ # 5 " 1 1 H ' + 1 ' @ ! 7 - ! 2 + ! 2 " 0 H 2 ! 1 H ' 1 ' * 3 1 @ + ! 7 - 1 D * D % H @ * 7 - 2 @ + 2 # 1 + ! 2 H 2 @ I 2 ! 2 @ % " - 2 * 1 H 2 " - I - - 8 I " 8 A " C + I A 1 C I * 2 ' 1 I @ G ' 7 H ' C % 1 % 4 8 " 8 " 5 % 1 H - @ + 8 H 2 - 2 @ ( # 4 + 2 I 2 # I - D + I I - # # 2 H 2 1 # # % 1 " $ E + 2 C # I 3 9 9 I # # % 1 L / ( T h e a b o v e i s a t w o - c o l u m n t e x t . I f c o m b i n i n g c h a r a c t e r s a r e h a n d l e d c o r r e c t l y , t h e l i n e s o f t h e s e c o n d c o l u m n s h o u l d b e a l i g n e d w i t h t h e | c h a r a c t e r a b o v e . ) E t h i o p i a n : P r o v e r b s i n t h e A m h a r i c l a n g u a g e : 0 s ( 5 % 0 5 b e c t ` F b % d q A % b ` E d c # # u ` b M s ` E d s = b % ` ` p s b 2 p ( ( b @ 5 ` @ 5 e A ` ) b - b e - ` 3 5 - b 0 d q ( d q p - b - H p . . 3 - b ( d u c e b u 5 E c u E b % + M s u K s u b c * e + b 5 ) + ) - b p b p I p 6 c I b - b ( - 5 u 0 b - ` M + = - b R u n e s : ( O l d E n g l i s h , w h i c h t r a n s c r i b e d i n t o L a t i n r e a d s ' H e c w a e t h t h a t h e b u d e t h a e m l a n d e n o r t h w e a r d u m w i t h t h a W e s t s a e . ' a n d m e a n s ' H e s a i d t h a t h e l i v e d i n t h e n o r t h e r n l a n d n e a r t h e W e s t e r n S e a . ' ) B r a i l l e : L( ( '( ( <( ( ( M( ( ( ( 9( 0( ( c( ( ( M( ( ( ( 9( :( ( ( ( ( ( ( ( ( ( ( ( ( ( :( ( 9( 2( y( ;( ( ( ( ( ( ( 3( ( ( 1( ( ( ( '( ;( ( ( 3( ( 9( ( ( 2( y( ( ( ( ( ( ( ;( ( ( ( ( ( ( %( ( ( ( ( :( ( ( ( ( ( ( +( ( 9( 9( ( ( ( ;( ( 9( ( ( ( ( 9( ( ( ( ;( ( ( 9( ( %( ( ( ;( ( ( ( ;( ( ( ( ( 9( ( !( ( ( ( ( 3( ( ( ;( 2( N( ( ( ( ( ( ( ( ( ( ( +( ( ( 2( A( ( ( N( ( ( ( ( ( ( 0( ( ( ( ( ( :( ( ( ( ( ( ( %( ( ( ( 0( a( ( ( ( ( ( ( ( ( ( ( 9( 9( ( ( ( ( !( ( ( ( ( ( ( %( ( ( ( ( ( ( ( ( ( ( 2( U( ( ( M( ( ( ( 9( :( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( $( ( ( ( ( 2( M( ( ( ( J( ( ( ( 0( ( ( ( ( ( ( ( ( ( 9( 9( ( ( J( ( ( *( ( ( ( ( 9( *( ( ( ( *( ( +( ( ( ( 1( ( ( 9( ;( ( ( ( ( ( ( ( ( %( ( ( ( 9( ( ( ( ( ( ( 3( ( ( ( ( ( ( $( ( ( ( ( 2( J( ( ( #( ( ( ( '( ( ( ( 2( ( ( ( ( +( ( ( 9( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( $( ( ( ( ( ( ( 9( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ;( 9( ( 9( ( ( ( ( ( ( 2( C( %( ( 9( ( :( ( ( ( ( ( ( ( 3( ( ( ( ( ( ( ( ( ( ( ( ( 9( ( ( ( ( ( ( ( ( ( ( ( ( 9( %( ( ( ( ( ( *( +( ( ( ( ( ( )( ( ( ( ( ( ( ( ( ( %( ( ( ( ( ( ( ( 9( ( J( 3( ( ( ( 9( 0( ( ( ( ( ( ( ( ( 2( y( 3( :( ( ( ( 9( ;( ( ( ( ( ( ( ;( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( 9( ( 9( ( ( M( ( ( ( 9( :( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( $( ( ( ( ( 2( ( T h e f i r s t c o u p l e o f p a r a g r a p h s o f " A C h r i s t m a s C a r o l " b y D i c k e n s ) C o m p a c t f o n t s e l e c t i o n e x a m p l e t e x t : A B C D E F G H I J K L M N O P Q R S T U V W X Y Z / 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z " & 0 "! S ` x ~ 0 1 2 3 4 " " " ! '" *" a" " ! ! ! ! ! % <% T% X% % % :& @& @$ P N# 1 G r e e t i n g s i n v a r i o u s l a n g u a g e s : H e l l o w o r l d , s y , 0 0 0 0 0 B o x d r a w i n g a l i g n m e n t t e s t s : % % T% P% P% f% P% P% W% % % % ,% % % % m% % % ,% % % n% m% % % ,% % % n% % % % 3% % % % % % % % w% {% % /% % % 0% % % q% r% q% r% s% s% s% Q% % % h% % % Q% % T% P% g% P% W% % % R% P% j% P% U% % % S% % A% % V% % % % % B% % % % % C% D% % v% <% t% z% K% x% % <% (% % K% %% % r% q% r% q% s% s% s% Q% % r% q% % Q% % Q% Q% % % % % % % % Q% % Q% % % % % % % % E% F% % u% y% % 7% % % 8% % % q% r% q% r% s% s% s% `% a% s% ^% c% % b% _% $% % <% % <% % <% $% % k% % B% % k% $% #% ?% ~% <% |% ?% +% % % % % % % % % N% % % % % % % r% q% r% q% s% s% s% Q% % q% r% % Q% % Q% Q% % % % % % % % Q% % Q% % % % }% % % % % % % % % % % % % N% O% % % % Q% % % e% % % Q% % Z% P% d% P% ]% % % X% P% j% P% [% % % Y% % @% % \% % % % % B% % % % % % % % % % % % % % N% O% % % % Z% P% P% i% P% P% ]% % % % 4% % % % p% % % 4% % % o% p% % % 4% % % o% % % % ;% % % % % % % % % % % L% L% % N% % M% M% % % % % % % % % % % % % % % % % flexi-streams-1.0.7/test/unicode_demo_ucs4_crlf_be.txt 0000664 0000000 0000000 00000075144 11175756704 0023116 0 ustar 00root root 0000000 0000000 U T F - 8 e n c o d e d s a m p l e p l a i n - t e x t f i l e > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > M a r k u s K u h n [ m a k s k u n ] < h t t p : / / w w w . c l . c a m . a c . u k / ~ m g k 2 5 / > 2 0 0 2 - 0 7 - 2 5 T h e A S C I I c o m p a t i b l e U T F - 8 e n c o d i n g u s e d i n t h i s p l a i n - t e x t f i l e i s d e f i n e d i n U n i c o d e , I S O 1 0 6 4 6 - 1 , a n d R F C 2 2 7 9 . U s i n g U n i c o d e / U T F - 8 , y o u c a n w r i t e i n e m a i l s a n d s o u r c e c o d e t h i n g s s u c h a s M a t h e m a t i c s a n d s c i e n c e s : ". E " d a = Q , n ! " , " f ( i ) = " g ( i ) , # # # % % % % % % % # # # # # # % a + b # # # " x " ! : # x # = " # " x # , "' = ( "( ) , # # # % % % % % % # # # # # # # c # # # ! " ! " !$ " ! " ! " ! , # # # # # # # # # " # # # " < a "` b "a c "d d "j " ! ( ' A ' ! ' B ' ) , # # # # # # # # # # # a q - b q # # # 2 H + O ! 2 H O , R = 4 . 7 k , # 2 0 0 m m # # # i = 1 # # # L i n g u i s t i c s a n d d i c t i o n a r i e s : i 1 n t Y n Y n Y l f Y n [ t 1 k Y s o s i e 1 n Y [ p s i l T n ] , Y e n [ j [ n ] , Y o g a [ j o g Q ] A P L : ( ( V #s V ) = #s #t V ) / V ! , V #7 ! #s ! #t " " " > #N #U # N i c e r t y p o g r a p h y i n p l a i n t e x t f i l e s : %T %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %W %Q %Q %Q " s i n g l e a n d d o u b l e q u o t e s %Q %Q %Q %Q " C u r l y a p o s t r o p h e s : W e v e b e e n h e r e %Q %Q %Q %Q " L a t i n - 1 a p o s t r o p h e a n d a c c e n t s : ' ` %Q %Q %Q %Q " d e u t s c h e A n f h r u n g s z e i c h e n %Q %Q %Q %Q " , ! , 0 , " , 3 4 , , " 5 / + 5 , !" , & %Q %Q %Q %Q " A S C I I s a f e t y t e s t : 1 l I | , 0 O D , 8 B %Q %Q %m % % % % % % % % % %n %Q %Q " t h e e u r o s y m b o l : % 1 4 . 9 5 % %Q %Q %p % % % % % % % % % %o %Q %Z %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %] C o m b i n i n g c h a r a c t e r s : S T A R G T E S G - 1 , a = v = r , a " b G r e e k ( i n P o l y t o n i c ) : T h e G r e e k a n t h e m : r w x t y t u , r w x t D z r w q t . p y s u p 1 q v p s , f , q ! F r o m a s p e e c h o f D e m o s t h e n e s i n t h e 4 t h c e n t u r y B C : P v P p w w } , f , E 0 p q s v E x z y S { z r p y v u w A s , p r q 0 u , e E t y P v y s s . P s V 1 p s " t Q y , v ' { , P v t V q Q q . | s , E s y v p Q v w u , v q 6 q , P q s y s s 1 x ! 6 t } , E z q } . p p w Q q , y v v w u w v C y s v r t t @ Q s , q ! v A y . s , x G e o r g i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : U n i c o d e - , 1 0 - 1 2 , . , . U n i c o d e - , , U n i c o d e - , , , . R u s s i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : 0 @ 5 3 8 A B @ 8 @ C 9 B 5 A L A 5 9 G 0 A = 0 5 A O B C N 5 6 4 C = 0 @ > 4 = C N > = D 5 @ 5 = F 8 N ? > U n i c o d e , : > B > @ 0 O A > A B > 8 B A O 1 0 - 1 2 < 0 @ B 0 1 9 9 7 3 > 4 0 2 0 9 = F 5 2 5 @ < 0 = 8 8 . > = D 5 @ 5 = F 8 O A > 1 5 @ 5 B H 8 @ > : 8 9 : @ C 3 M : A ? 5 @ B > 2 ? > 2 > ? @ > A 0 < 3 ; > 1 0 ; L = > 3 > = B 5 @ = 5 B 0 8 U n i c o d e , ; > : 0 ; 8 7 0 F 8 8 8 8 = B 5 @ = 0 F 8 > = 0 ; 8 7 0 F 8 8 , 2 > ? ; > I 5 = 8 N 8 ? @ 8 < 5 = 5 = 8 N U n i c o d e 2 @ 0 7 ; 8 G = K E > ? 5 @ 0 F 8 > = = K E A 8 A B 5 < 0 E 8 ? @ > 3 @ 0 < < = K E ? @ 8 ; > 6 5 = 8 O E , H @ 8 D B 0 E , 2 5 @ A B : 5 8 < = > 3 > O 7 K G = K E : > < ? L N B 5 @ = K E A 8 A B 5 < 0 E . T h a i ( U C S L e v e l 2 ) : E x c e r p t f r o m a p o e t r y o n T h e R o m a n c e o f T h e T h r e e K i n g d o m s ( a C h i n e s e c l a s s i c ' S a n G u a ' ) : [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - - - - ] O A H 4 . 1 H @ * 7 H - ! B # ! A * * 1 @ ' # 0 @ ( - 9 J 9 I 6 I C + ! H * 4 * - ) 1 # 4 " L H - + I 2 A % 1 D * - - L D # I B H @ % 2 @ 2 1 2 # 1 7 - 1 5 @ G 5 H 6 H I 2 @ ! 7 - 6 ' 4 # 4 @ G 1 + 2 B . 4 K @ # 5 " 1 1 H ' + 1 ' @ ! 7 - ! 2 + ! 2 " 0 H 2 ! 1 H ' 1 ' * 3 1 @ + ! 7 - 1 D * D % H @ * 7 - 2 @ + 2 # 1 + ! 2 H 2 @ I 2 ! 2 @ % " - 2 * 1 H 2 " - I - - 8 I " 8 A " C + I A 1 C I * 2 ' 1 I @ G ' 7 H ' C % 1 % 4 8 " 8 " 5 % 1 H - @ + 8 H 2 - 2 @ ( # 4 + 2 I 2 # I - D + I I - # # 2 H 2 1 # # % 1 " $ E + 2 C # I 3 9 9 I # # % 1 L / ( T h e a b o v e i s a t w o - c o l u m n t e x t . I f c o m b i n i n g c h a r a c t e r s a r e h a n d l e d c o r r e c t l y , t h e l i n e s o f t h e s e c o n d c o l u m n s h o u l d b e a l i g n e d w i t h t h e | c h a r a c t e r a b o v e . ) E t h i o p i a n : P r o v e r b s i n t h e A m h a r i c l a n g u a g e : 0 s ( 5 % 0 5 b e c t ` F b % d q A % b ` E d c # # u ` b M s ` E d s = b % ` ` p s b 2 p ( ( b @ 5 ` @ 5 e A ` ) b - b e - ` 3 5 - b 0 d q ( d q p - b - H p . . 3 - b ( d u c e b u 5 E c u E b % + M s u K s u b c * e + b 5 ) + ) - b p b p I p 6 c I b - b ( - 5 u 0 b - ` M + = - b R u n e s : ( O l d E n g l i s h , w h i c h t r a n s c r i b e d i n t o L a t i n r e a d s ' H e c w a e t h t h a t h e b u d e t h a e m l a n d e n o r t h w e a r d u m w i t h t h a W e s t s a e . ' a n d m e a n s ' H e s a i d t h a t h e l i v e d i n t h e n o r t h e r n l a n d n e a r t h e W e s t e r n S e a . ' ) B r a i l l e : (L ( (' ( (< ( ( (M ( ( ( (9 (0 ( (c ( ( (M ( ( ( (9 (: ( ( ( ( ( ( ( ( ( ( ( ( ( (: ( (9 (2 (y (; ( ( ( ( ( ( (3 ( ( (1 ( ( ( (' (; ( ( (3 ( (9 ( ( (2 (y ( ( ( ( ( ( (; ( ( ( ( ( ( (% ( ( ( ( (: ( ( ( ( ( ( (+ ( (9 (9 ( ( ( (; ( (9 ( ( ( ( (9 ( ( ( (; ( ( (9 ( (% ( ( (; ( ( ( (; ( ( ( ( (9 ( (! ( ( ( ( (3 ( ( (; (2 (N ( ( ( ( ( ( ( ( ( ( (+ ( ( (2 (A ( ( (N ( ( ( ( ( ( (0 ( ( ( ( ( (: ( ( ( ( ( ( (% ( ( ( (0 (a ( ( ( ( ( ( ( ( ( ( (9 (9 ( ( ( ( (! ( ( ( ( ( ( (% ( ( ( ( ( ( ( ( ( ( (2 (U ( ( (M ( ( ( (9 (: ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ($ ( ( ( ( (2 (M ( ( ( (J ( ( ( (0 ( ( ( ( ( ( ( ( ( (9 (9 ( ( (J ( ( (* ( ( ( ( (9 (* ( ( ( (* ( (+ ( ( ( (1 ( ( (9 (; ( ( ( ( ( ( ( ( (% ( ( ( (9 ( ( ( ( ( ( (3 ( ( ( ( ( ( ($ ( ( ( ( (2 (J ( ( (# ( ( ( (' ( ( ( (2 ( ( ( ( (+ ( ( (9 ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ($ ( ( ( ( ( ( (9 ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (; (9 ( (9 ( ( ( ( ( ( (2 (C (% ( (9 ( (: ( ( ( ( ( ( ( (3 ( ( ( ( ( ( ( ( ( ( ( ( (9 ( ( ( ( ( ( ( ( ( ( ( ( (9 (% ( ( ( ( ( (* (+ ( ( ( ( ( () ( ( ( ( ( ( ( ( ( (% ( ( ( ( ( ( ( (9 ( (J (3 ( ( ( (9 (0 ( ( ( ( ( ( ( ( (2 (y (3 (: ( ( ( (9 (; ( ( ( ( ( ( (; ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (9 ( (9 ( ( (M ( ( ( (9 (: ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ($ ( ( ( ( (2 ( T h e f i r s t c o u p l e o f p a r a g r a p h s o f " A C h r i s t m a s C a r o l " b y D i c k e n s ) C o m p a c t f o n t s e l e c t i o n e x a m p l e t e x t : A B C D E F G H I J K L M N O P Q R S T U V W X Y Z / 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z " & 0 !" S ` x ~ 0 1 2 3 4 " " " ! "' "* "a " ! ! ! ! ! % %< %T %X % % &: &@ $@ P #N 1 G r e e t i n g s i n v a r i o u s l a n g u a g e s : H e l l o w o r l d , s y , 0 0 0 0 0 B o x d r a w i n g a l i g n m e n t t e s t s : % % %T %P %P %f %P %P %W % % % %, % % % %m % % %, % % %n %m % % %, % % %n % % % %3 % % % % % % % %w %{ % %/ % % %0 % % %q %r %q %r %s %s %s %Q % % %h % % %Q % %T %P %g %P %W % % %R %P %j %P %U % % %S % %A % %V % % % % %B % % % % %C %D % %v %< %t %z %K %x % %< %( % %K %% % %r %q %r %q %s %s %s %Q % %r %q % %Q % %Q %Q % % % % % % % %Q % %Q % % % % % % % %E %F % %u %y % %7 % % %8 % % %q %r %q %r %s %s %s %` %a %s %^ %c % %b %_ %$ % %< % %< % %< %$ % %k % %B % %k %$ %# %? %~ %< %| %? %+ % % % % % % % % %N % % % % % % %r %q %r %q %s %s %s %Q % %q %r % %Q % %Q %Q % % % % % % % %Q % %Q % % % %} % % % % % % % % % % % % %N %O % % % %Q % % %e % % %Q % %Z %P %d %P %] % % %X %P %j %P %[ % % %Y % %@ % %\ % % % % %B % % % % % % % % % % % % % %N %O % % % %Z %P %P %i %P %P %] % % % %4 % % % %p % % %4 % % %o %p % % %4 % % %o % % % %; % % % % % % % % % % %L %L % %N % %M %M % % % % % % % % % % % % % % % % flexi-streams-1.0.7/test/unicode_demo_ucs4_crlf_le.txt 0000664 0000000 0000000 00000075144 11175756704 0023130 0 ustar 00root root 0000000 0000000 U T F - 8 e n c o d e d s a m p l e p l a i n - t e x t f i l e > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > M a r k u s K u h n [ m a k s k u n ] < h t t p : / / w w w . c l . c a m . a c . u k / ~ m g k 2 5 / > 2 0 0 2 - 0 7 - 2 5 T h e A S C I I c o m p a t i b l e U T F - 8 e n c o d i n g u s e d i n t h i s p l a i n - t e x t f i l e i s d e f i n e d i n U n i c o d e , I S O 1 0 6 4 6 - 1 , a n d R F C 2 2 7 9 . U s i n g U n i c o d e / U T F - 8 , y o u c a n w r i t e i n e m a i l s a n d s o u r c e c o d e t h i n g s s u c h a s M a t h e m a t i c s a n d s c i e n c e s : ." E " d a = Q , n ! " , " f ( i ) = " g ( i ) , # # # % % % % % % % # # # # # # % a + b # # # " x " ! : # x # = " # " x # , '" = ( (" ) , # # # % % % % % % # # # # # # # c # # # ! " ! " $! " ! " ! " ! , # # # # # # # # # " # # # " < a `" b a" c d" d j" " ! ( ' A ' ! ' B ' ) , # # # # # # # # # # # a q - b q # # # 2 H + O ! 2 H O , R = 4 . 7 k , # 2 0 0 m m # # # i = 1 # # # L i n g u i s t i c s a n d d i c t i o n a r i e s : i 1 n t Y n Y n Y l f Y n [ t 1 k Y s o s i e 1 n Y [ p s i l T n ] , Y e n [ j [ n ] , Y o g a [ j o g Q ] A P L : ( ( V s# V ) = s# t# V ) / V ! , V 7# ! s# ! t# " " " > N# U# # N i c e r t y p o g r a p h y i n p l a i n t e x t f i l e s : T% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% W% Q% Q% Q% " s i n g l e a n d d o u b l e q u o t e s Q% Q% Q% Q% " C u r l y a p o s t r o p h e s : W e v e b e e n h e r e Q% Q% Q% Q% " L a t i n - 1 a p o s t r o p h e a n d a c c e n t s : ' ` Q% Q% Q% Q% " d e u t s c h e A n f h r u n g s z e i c h e n Q% Q% Q% Q% " , ! , 0 , " , 3 4 , , " 5 / + 5 , "! , & Q% Q% Q% Q% " A S C I I s a f e t y t e s t : 1 l I | , 0 O D , 8 B Q% Q% m% % % % % % % % % % n% Q% Q% " t h e e u r o s y m b o l : % 1 4 . 9 5 % Q% Q% p% % % % % % % % % % o% Q% Z% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% ]% C o m b i n i n g c h a r a c t e r s : S T A R G T E S G - 1 , a = v = r , a " b G r e e k ( i n P o l y t o n i c ) : T h e G r e e k a n t h e m : r w x t y t u , r w x t D z r w q t . p y s u p 1 q v p s , f , q ! F r o m a s p e e c h o f D e m o s t h e n e s i n t h e 4 t h c e n t u r y B C : P v P p w w } , f , E 0 p q s v E x z y S { z r p y v u w A s , p r q 0 u , e E t y P v y s s . P s V 1 p s " t Q y , v ' { , P v t V q Q q . | s , E s y v p Q v w u , v q 6 q , P q s y s s 1 x ! 6 t } , E z q } . p p w Q q , y v v w u w v C y s v r t t @ Q s , q ! v A y . s , x G e o r g i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : U n i c o d e - , 1 0 - 1 2 , . , . U n i c o d e - , , U n i c o d e - , , , . R u s s i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : 0 @ 5 3 8 A B @ 8 @ C 9 B 5 A L A 5 9 G 0 A = 0 5 A O B C N 5 6 4 C = 0 @ > 4 = C N > = D 5 @ 5 = F 8 N ? > U n i c o d e , : > B > @ 0 O A > A B > 8 B A O 1 0 - 1 2 < 0 @ B 0 1 9 9 7 3 > 4 0 2 0 9 = F 5 2 5 @ < 0 = 8 8 . > = D 5 @ 5 = F 8 O A > 1 5 @ 5 B H 8 @ > : 8 9 : @ C 3 M : A ? 5 @ B > 2 ? > 2 > ? @ > A 0 < 3 ; > 1 0 ; L = > 3 > = B 5 @ = 5 B 0 8 U n i c o d e , ; > : 0 ; 8 7 0 F 8 8 8 8 = B 5 @ = 0 F 8 > = 0 ; 8 7 0 F 8 8 , 2 > ? ; > I 5 = 8 N 8 ? @ 8 < 5 = 5 = 8 N U n i c o d e 2 @ 0 7 ; 8 G = K E > ? 5 @ 0 F 8 > = = K E A 8 A B 5 < 0 E 8 ? @ > 3 @ 0 < < = K E ? @ 8 ; > 6 5 = 8 O E , H @ 8 D B 0 E , 2 5 @ A B : 5 8 < = > 3 > O 7 K G = K E : > < ? L N B 5 @ = K E A 8 A B 5 < 0 E . T h a i ( U C S L e v e l 2 ) : E x c e r p t f r o m a p o e t r y o n T h e R o m a n c e o f T h e T h r e e K i n g d o m s ( a C h i n e s e c l a s s i c ' S a n G u a ' ) : [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - - - - ] O A H 4 . 1 H @ * 7 H - ! B # ! A * * 1 @ ' # 0 @ ( - 9 J 9 I 6 I C + ! H * 4 * - ) 1 # 4 " L H - + I 2 A % 1 D * - - L D # I B H @ % 2 @ 2 1 2 # 1 7 - 1 5 @ G 5 H 6 H I 2 @ ! 7 - 6 ' 4 # 4 @ G 1 + 2 B . 4 K @ # 5 " 1 1 H ' + 1 ' @ ! 7 - ! 2 + ! 2 " 0 H 2 ! 1 H ' 1 ' * 3 1 @ + ! 7 - 1 D * D % H @ * 7 - 2 @ + 2 # 1 + ! 2 H 2 @ I 2 ! 2 @ % " - 2 * 1 H 2 " - I - - 8 I " 8 A " C + I A 1 C I * 2 ' 1 I @ G ' 7 H ' C % 1 % 4 8 " 8 " 5 % 1 H - @ + 8 H 2 - 2 @ ( # 4 + 2 I 2 # I - D + I I - # # 2 H 2 1 # # % 1 " $ E + 2 C # I 3 9 9 I # # % 1 L / ( T h e a b o v e i s a t w o - c o l u m n t e x t . I f c o m b i n i n g c h a r a c t e r s a r e h a n d l e d c o r r e c t l y , t h e l i n e s o f t h e s e c o n d c o l u m n s h o u l d b e a l i g n e d w i t h t h e | c h a r a c t e r a b o v e . ) E t h i o p i a n : P r o v e r b s i n t h e A m h a r i c l a n g u a g e : 0 s ( 5 % 0 5 b e c t ` F b % d q A % b ` E d c # # u ` b M s ` E d s = b % ` ` p s b 2 p ( ( b @ 5 ` @ 5 e A ` ) b - b e - ` 3 5 - b 0 d q ( d q p - b - H p . . 3 - b ( d u c e b u 5 E c u E b % + M s u K s u b c * e + b 5 ) + ) - b p b p I p 6 c I b - b ( - 5 u 0 b - ` M + = - b R u n e s : ( O l d E n g l i s h , w h i c h t r a n s c r i b e d i n t o L a t i n r e a d s ' H e c w a e t h t h a t h e b u d e t h a e m l a n d e n o r t h w e a r d u m w i t h t h a W e s t s a e . ' a n d m e a n s ' H e s a i d t h a t h e l i v e d i n t h e n o r t h e r n l a n d n e a r t h e W e s t e r n S e a . ' ) B r a i l l e : L( ( '( ( <( ( ( M( ( ( ( 9( 0( ( c( ( ( M( ( ( ( 9( :( ( ( ( ( ( ( ( ( ( ( ( ( ( :( ( 9( 2( y( ;( ( ( ( ( ( ( 3( ( ( 1( ( ( ( '( ;( ( ( 3( ( 9( ( ( 2( y( ( ( ( ( ( ( ;( ( ( ( ( ( ( %( ( ( ( ( :( ( ( ( ( ( ( +( ( 9( 9( ( ( ( ;( ( 9( ( ( ( ( 9( ( ( ( ;( ( ( 9( ( %( ( ( ;( ( ( ( ;( ( ( ( ( 9( ( !( ( ( ( ( 3( ( ( ;( 2( N( ( ( ( ( ( ( ( ( ( ( +( ( ( 2( A( ( ( N( ( ( ( ( ( ( 0( ( ( ( ( ( :( ( ( ( ( ( ( %( ( ( ( 0( a( ( ( ( ( ( ( ( ( ( ( 9( 9( ( ( ( ( !( ( ( ( ( ( ( %( ( ( ( ( ( ( ( ( ( ( 2( U( ( ( M( ( ( ( 9( :( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( $( ( ( ( ( 2( M( ( ( ( J( ( ( ( 0( ( ( ( ( ( ( ( ( ( 9( 9( ( ( J( ( ( *( ( ( ( ( 9( *( ( ( ( *( ( +( ( ( ( 1( ( ( 9( ;( ( ( ( ( ( ( ( ( %( ( ( ( 9( ( ( ( ( ( ( 3( ( ( ( ( ( ( $( ( ( ( ( 2( J( ( ( #( ( ( ( '( ( ( ( 2( ( ( ( ( +( ( ( 9( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( $( ( ( ( ( ( ( 9( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ;( 9( ( 9( ( ( ( ( ( ( 2( C( %( ( 9( ( :( ( ( ( ( ( ( ( 3( ( ( ( ( ( ( ( ( ( ( ( ( 9( ( ( ( ( ( ( ( ( ( ( ( ( 9( %( ( ( ( ( ( *( +( ( ( ( ( ( )( ( ( ( ( ( ( ( ( ( %( ( ( ( ( ( ( ( 9( ( J( 3( ( ( ( 9( 0( ( ( ( ( ( ( ( ( 2( y( 3( :( ( ( ( 9( ;( ( ( ( ( ( ( ;( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( 9( ( 9( ( ( M( ( ( ( 9( :( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( $( ( ( ( ( 2( ( T h e f i r s t c o u p l e o f p a r a g r a p h s o f " A C h r i s t m a s C a r o l " b y D i c k e n s ) C o m p a c t f o n t s e l e c t i o n e x a m p l e t e x t : A B C D E F G H I J K L M N O P Q R S T U V W X Y Z / 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z " & 0 "! S ` x ~ 0 1 2 3 4 " " " ! '" *" a" " ! ! ! ! ! % <% T% X% % % :& @& @$ P N# 1 G r e e t i n g s i n v a r i o u s l a n g u a g e s : H e l l o w o r l d , s y , 0 0 0 0 0 B o x d r a w i n g a l i g n m e n t t e s t s : % % T% P% P% f% P% P% W% % % % ,% % % % m% % % ,% % % n% m% % % ,% % % n% % % % 3% % % % % % % % w% {% % /% % % 0% % % q% r% q% r% s% s% s% Q% % % h% % % Q% % T% P% g% P% W% % % R% P% j% P% U% % % S% % A% % V% % % % % B% % % % % C% D% % v% <% t% z% K% x% % <% (% % K% %% % r% q% r% q% s% s% s% Q% % r% q% % Q% % Q% Q% % % % % % % % Q% % Q% % % % % % % % E% F% % u% y% % 7% % % 8% % % q% r% q% r% s% s% s% `% a% s% ^% c% % b% _% $% % <% % <% % <% $% % k% % B% % k% $% #% ?% ~% <% |% ?% +% % % % % % % % % N% % % % % % % r% q% r% q% s% s% s% Q% % q% r% % Q% % Q% Q% % % % % % % % Q% % Q% % % % }% % % % % % % % % % % % % N% O% % % % Q% % % e% % % Q% % Z% P% d% P% ]% % % X% P% j% P% [% % % Y% % @% % \% % % % % B% % % % % % % % % % % % % % N% O% % % % Z% P% P% i% P% P% ]% % % % 4% % % % p% % % 4% % % o% p% % % 4% % % o% % % % ;% % % % % % % % % % % L% L% % N% % M% M% % % % % % % % % % % % % % % % % flexi-streams-1.0.7/test/unicode_demo_ucs4_lf_be.txt 0000664 0000000 0000000 00000073424 11175756704 0022570 0 ustar 00root root 0000000 0000000 U T F - 8 e n c o d e d s a m p l e p l a i n - t e x t f i l e > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > M a r k u s K u h n [ m a k s k u n ] < h t t p : / / w w w . c l . c a m . a c . u k / ~ m g k 2 5 / > 2 0 0 2 - 0 7 - 2 5 T h e A S C I I c o m p a t i b l e U T F - 8 e n c o d i n g u s e d i n t h i s p l a i n - t e x t f i l e i s d e f i n e d i n U n i c o d e , I S O 1 0 6 4 6 - 1 , a n d R F C 2 2 7 9 . U s i n g U n i c o d e / U T F - 8 , y o u c a n w r i t e i n e m a i l s a n d s o u r c e c o d e t h i n g s s u c h a s M a t h e m a t i c s a n d s c i e n c e s : ". E " d a = Q , n ! " , " f ( i ) = " g ( i ) , # # # % % % % % % % # # # # # # % a + b # # # " x " ! : # x # = " # " x # , "' = ( "( ) , # # # % % % % % % # # # # # # # c # # # ! " ! " !$ " ! " ! " ! , # # # # # # # # # " # # # " < a "` b "a c "d d "j " ! ( ' A ' ! ' B ' ) , # # # # # # # # # # # a q - b q # # # 2 H + O ! 2 H O , R = 4 . 7 k , # 2 0 0 m m # # # i = 1 # # # L i n g u i s t i c s a n d d i c t i o n a r i e s : i 1 n t Y n Y n Y l f Y n [ t 1 k Y s o s i e 1 n Y [ p s i l T n ] , Y e n [ j [ n ] , Y o g a [ j o g Q ] A P L : ( ( V #s V ) = #s #t V ) / V ! , V #7 ! #s ! #t " " " > #N #U # N i c e r t y p o g r a p h y i n p l a i n t e x t f i l e s : %T %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %W %Q %Q %Q " s i n g l e a n d d o u b l e q u o t e s %Q %Q %Q %Q " C u r l y a p o s t r o p h e s : W e v e b e e n h e r e %Q %Q %Q %Q " L a t i n - 1 a p o s t r o p h e a n d a c c e n t s : ' ` %Q %Q %Q %Q " d e u t s c h e A n f h r u n g s z e i c h e n %Q %Q %Q %Q " , ! , 0 , " , 3 4 , , " 5 / + 5 , !" , & %Q %Q %Q %Q " A S C I I s a f e t y t e s t : 1 l I | , 0 O D , 8 B %Q %Q %m % % % % % % % % % %n %Q %Q " t h e e u r o s y m b o l : % 1 4 . 9 5 % %Q %Q %p % % % % % % % % % %o %Q %Z %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %P %] C o m b i n i n g c h a r a c t e r s : S T A R G T E S G - 1 , a = v = r , a " b G r e e k ( i n P o l y t o n i c ) : T h e G r e e k a n t h e m : r w x t y t u , r w x t D z r w q t . p y s u p 1 q v p s , f , q ! F r o m a s p e e c h o f D e m o s t h e n e s i n t h e 4 t h c e n t u r y B C : P v P p w w } , f , E 0 p q s v E x z y S { z r p y v u w A s , p r q 0 u , e E t y P v y s s . P s V 1 p s " t Q y , v ' { , P v t V q Q q . | s , E s y v p Q v w u , v q 6 q , P q s y s s 1 x ! 6 t } , E z q } . p p w Q q , y v v w u w v C y s v r t t @ Q s , q ! v A y . s , x G e o r g i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : U n i c o d e - , 1 0 - 1 2 , . , . U n i c o d e - , , U n i c o d e - , , , . R u s s i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : 0 @ 5 3 8 A B @ 8 @ C 9 B 5 A L A 5 9 G 0 A = 0 5 A O B C N 5 6 4 C = 0 @ > 4 = C N > = D 5 @ 5 = F 8 N ? > U n i c o d e , : > B > @ 0 O A > A B > 8 B A O 1 0 - 1 2 < 0 @ B 0 1 9 9 7 3 > 4 0 2 0 9 = F 5 2 5 @ < 0 = 8 8 . > = D 5 @ 5 = F 8 O A > 1 5 @ 5 B H 8 @ > : 8 9 : @ C 3 M : A ? 5 @ B > 2 ? > 2 > ? @ > A 0 < 3 ; > 1 0 ; L = > 3 > = B 5 @ = 5 B 0 8 U n i c o d e , ; > : 0 ; 8 7 0 F 8 8 8 8 = B 5 @ = 0 F 8 > = 0 ; 8 7 0 F 8 8 , 2 > ? ; > I 5 = 8 N 8 ? @ 8 < 5 = 5 = 8 N U n i c o d e 2 @ 0 7 ; 8 G = K E > ? 5 @ 0 F 8 > = = K E A 8 A B 5 < 0 E 8 ? @ > 3 @ 0 < < = K E ? @ 8 ; > 6 5 = 8 O E , H @ 8 D B 0 E , 2 5 @ A B : 5 8 < = > 3 > O 7 K G = K E : > < ? L N B 5 @ = K E A 8 A B 5 < 0 E . T h a i ( U C S L e v e l 2 ) : E x c e r p t f r o m a p o e t r y o n T h e R o m a n c e o f T h e T h r e e K i n g d o m s ( a C h i n e s e c l a s s i c ' S a n G u a ' ) : [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - - - - ] O A H 4 . 1 H @ * 7 H - ! B # ! A * * 1 @ ' # 0 @ ( - 9 J 9 I 6 I C + ! H * 4 * - ) 1 # 4 " L H - + I 2 A % 1 D * - - L D # I B H @ % 2 @ 2 1 2 # 1 7 - 1 5 @ G 5 H 6 H I 2 @ ! 7 - 6 ' 4 # 4 @ G 1 + 2 B . 4 K @ # 5 " 1 1 H ' + 1 ' @ ! 7 - ! 2 + ! 2 " 0 H 2 ! 1 H ' 1 ' * 3 1 @ + ! 7 - 1 D * D % H @ * 7 - 2 @ + 2 # 1 + ! 2 H 2 @ I 2 ! 2 @ % " - 2 * 1 H 2 " - I - - 8 I " 8 A " C + I A 1 C I * 2 ' 1 I @ G ' 7 H ' C % 1 % 4 8 " 8 " 5 % 1 H - @ + 8 H 2 - 2 @ ( # 4 + 2 I 2 # I - D + I I - # # 2 H 2 1 # # % 1 " $ E + 2 C # I 3 9 9 I # # % 1 L / ( T h e a b o v e i s a t w o - c o l u m n t e x t . I f c o m b i n i n g c h a r a c t e r s a r e h a n d l e d c o r r e c t l y , t h e l i n e s o f t h e s e c o n d c o l u m n s h o u l d b e a l i g n e d w i t h t h e | c h a r a c t e r a b o v e . ) E t h i o p i a n : P r o v e r b s i n t h e A m h a r i c l a n g u a g e : 0 s ( 5 % 0 5 b e c t ` F b % d q A % b ` E d c # # u ` b M s ` E d s = b % ` ` p s b 2 p ( ( b @ 5 ` @ 5 e A ` ) b - b e - ` 3 5 - b 0 d q ( d q p - b - H p . . 3 - b ( d u c e b u 5 E c u E b % + M s u K s u b c * e + b 5 ) + ) - b p b p I p 6 c I b - b ( - 5 u 0 b - ` M + = - b R u n e s : ( O l d E n g l i s h , w h i c h t r a n s c r i b e d i n t o L a t i n r e a d s ' H e c w a e t h t h a t h e b u d e t h a e m l a n d e n o r t h w e a r d u m w i t h t h a W e s t s a e . ' a n d m e a n s ' H e s a i d t h a t h e l i v e d i n t h e n o r t h e r n l a n d n e a r t h e W e s t e r n S e a . ' ) B r a i l l e : (L ( (' ( (< ( ( (M ( ( ( (9 (0 ( (c ( ( (M ( ( ( (9 (: ( ( ( ( ( ( ( ( ( ( ( ( ( (: ( (9 (2 (y (; ( ( ( ( ( ( (3 ( ( (1 ( ( ( (' (; ( ( (3 ( (9 ( ( (2 (y ( ( ( ( ( ( (; ( ( ( ( ( ( (% ( ( ( ( (: ( ( ( ( ( ( (+ ( (9 (9 ( ( ( (; ( (9 ( ( ( ( (9 ( ( ( (; ( ( (9 ( (% ( ( (; ( ( ( (; ( ( ( ( (9 ( (! ( ( ( ( (3 ( ( (; (2 (N ( ( ( ( ( ( ( ( ( ( (+ ( ( (2 (A ( ( (N ( ( ( ( ( ( (0 ( ( ( ( ( (: ( ( ( ( ( ( (% ( ( ( (0 (a ( ( ( ( ( ( ( ( ( ( (9 (9 ( ( ( ( (! ( ( ( ( ( ( (% ( ( ( ( ( ( ( ( ( ( (2 (U ( ( (M ( ( ( (9 (: ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ($ ( ( ( ( (2 (M ( ( ( (J ( ( ( (0 ( ( ( ( ( ( ( ( ( (9 (9 ( ( (J ( ( (* ( ( ( ( (9 (* ( ( ( (* ( (+ ( ( ( (1 ( ( (9 (; ( ( ( ( ( ( ( ( (% ( ( ( (9 ( ( ( ( ( ( (3 ( ( ( ( ( ( ($ ( ( ( ( (2 (J ( ( (# ( ( ( (' ( ( ( (2 ( ( ( ( (+ ( ( (9 ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ($ ( ( ( ( ( ( (9 ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (; (9 ( (9 ( ( ( ( ( ( (2 (C (% ( (9 ( (: ( ( ( ( ( ( ( (3 ( ( ( ( ( ( ( ( ( ( ( ( (9 ( ( ( ( ( ( ( ( ( ( ( ( (9 (% ( ( ( ( ( (* (+ ( ( ( ( ( () ( ( ( ( ( ( ( ( ( (% ( ( ( ( ( ( ( (9 ( (J (3 ( ( ( (9 (0 ( ( ( ( ( ( ( ( (2 (y (3 (: ( ( ( (9 (; ( ( ( ( ( ( (; ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (9 ( (9 ( ( (M ( ( ( (9 (: ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ($ ( ( ( ( (2 ( T h e f i r s t c o u p l e o f p a r a g r a p h s o f " A C h r i s t m a s C a r o l " b y D i c k e n s ) C o m p a c t f o n t s e l e c t i o n e x a m p l e t e x t : A B C D E F G H I J K L M N O P Q R S T U V W X Y Z / 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z " & 0 !" S ` x ~ 0 1 2 3 4 " " " ! "' "* "a " ! ! ! ! ! % %< %T %X % % &: &@ $@ P #N 1 G r e e t i n g s i n v a r i o u s l a n g u a g e s : H e l l o w o r l d , s y , 0 0 0 0 0 B o x d r a w i n g a l i g n m e n t t e s t s : % % %T %P %P %f %P %P %W % % % %, % % % %m % % %, % % %n %m % % %, % % %n % % % %3 % % % % % % % %w %{ % %/ % % %0 % % %q %r %q %r %s %s %s %Q % % %h % % %Q % %T %P %g %P %W % % %R %P %j %P %U % % %S % %A % %V % % % % %B % % % % %C %D % %v %< %t %z %K %x % %< %( % %K %% % %r %q %r %q %s %s %s %Q % %r %q % %Q % %Q %Q % % % % % % % %Q % %Q % % % % % % % %E %F % %u %y % %7 % % %8 % % %q %r %q %r %s %s %s %` %a %s %^ %c % %b %_ %$ % %< % %< % %< %$ % %k % %B % %k %$ %# %? %~ %< %| %? %+ % % % % % % % % %N % % % % % % %r %q %r %q %s %s %s %Q % %q %r % %Q % %Q %Q % % % % % % % %Q % %Q % % % %} % % % % % % % % % % % % %N %O % % % %Q % % %e % % %Q % %Z %P %d %P %] % % %X %P %j %P %[ % % %Y % %@ % %\ % % % % %B % % % % % % % % % % % % % %N %O % % % %Z %P %P %i %P %P %] % % % %4 % % % %p % % %4 % % %o %p % % %4 % % %o % % % %; % % % % % % % % % % %L %L % %N % %M %M % % % % % % % % % % % % % % % % flexi-streams-1.0.7/test/unicode_demo_ucs4_lf_le.txt 0000664 0000000 0000000 00000073424 11175756704 0022602 0 ustar 00root root 0000000 0000000 U T F - 8 e n c o d e d s a m p l e p l a i n - t e x t f i l e > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > M a r k u s K u h n [ m a k s k u n ] < h t t p : / / w w w . c l . c a m . a c . u k / ~ m g k 2 5 / > 2 0 0 2 - 0 7 - 2 5 T h e A S C I I c o m p a t i b l e U T F - 8 e n c o d i n g u s e d i n t h i s p l a i n - t e x t f i l e i s d e f i n e d i n U n i c o d e , I S O 1 0 6 4 6 - 1 , a n d R F C 2 2 7 9 . U s i n g U n i c o d e / U T F - 8 , y o u c a n w r i t e i n e m a i l s a n d s o u r c e c o d e t h i n g s s u c h a s M a t h e m a t i c s a n d s c i e n c e s : ." E " d a = Q , n ! " , " f ( i ) = " g ( i ) , # # # % % % % % % % # # # # # # % a + b # # # " x " ! : # x # = " # " x # , '" = ( (" ) , # # # % % % % % % # # # # # # # c # # # ! " ! " $! " ! " ! " ! , # # # # # # # # # " # # # " < a `" b a" c d" d j" " ! ( ' A ' ! ' B ' ) , # # # # # # # # # # # a q - b q # # # 2 H + O ! 2 H O , R = 4 . 7 k , # 2 0 0 m m # # # i = 1 # # # L i n g u i s t i c s a n d d i c t i o n a r i e s : i 1 n t Y n Y n Y l f Y n [ t 1 k Y s o s i e 1 n Y [ p s i l T n ] , Y e n [ j [ n ] , Y o g a [ j o g Q ] A P L : ( ( V s# V ) = s# t# V ) / V ! , V 7# ! s# ! t# " " " > N# U# # N i c e r t y p o g r a p h y i n p l a i n t e x t f i l e s : T% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% W% Q% Q% Q% " s i n g l e a n d d o u b l e q u o t e s Q% Q% Q% Q% " C u r l y a p o s t r o p h e s : W e v e b e e n h e r e Q% Q% Q% Q% " L a t i n - 1 a p o s t r o p h e a n d a c c e n t s : ' ` Q% Q% Q% Q% " d e u t s c h e A n f h r u n g s z e i c h e n Q% Q% Q% Q% " , ! , 0 , " , 3 4 , , " 5 / + 5 , "! , & Q% Q% Q% Q% " A S C I I s a f e t y t e s t : 1 l I | , 0 O D , 8 B Q% Q% m% % % % % % % % % % n% Q% Q% " t h e e u r o s y m b o l : % 1 4 . 9 5 % Q% Q% p% % % % % % % % % % o% Q% Z% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% P% ]% C o m b i n i n g c h a r a c t e r s : S T A R G T E S G - 1 , a = v = r , a " b G r e e k ( i n P o l y t o n i c ) : T h e G r e e k a n t h e m : r w x t y t u , r w x t D z r w q t . p y s u p 1 q v p s , f , q ! F r o m a s p e e c h o f D e m o s t h e n e s i n t h e 4 t h c e n t u r y B C : P v P p w w } , f , E 0 p q s v E x z y S { z r p y v u w A s , p r q 0 u , e E t y P v y s s . P s V 1 p s " t Q y , v ' { , P v t V q Q q . | s , E s y v p Q v w u , v q 6 q , P q s y s s 1 x ! 6 t } , E z q } . p p w Q q , y v v w u w v C y s v r t t @ Q s , q ! v A y . s , x G e o r g i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : U n i c o d e - , 1 0 - 1 2 , . , . U n i c o d e - , , U n i c o d e - , , , . R u s s i a n : F r o m a U n i c o d e c o n f e r e n c e i n v i t a t i o n : 0 @ 5 3 8 A B @ 8 @ C 9 B 5 A L A 5 9 G 0 A = 0 5 A O B C N 5 6 4 C = 0 @ > 4 = C N > = D 5 @ 5 = F 8 N ? > U n i c o d e , : > B > @ 0 O A > A B > 8 B A O 1 0 - 1 2 < 0 @ B 0 1 9 9 7 3 > 4 0 2 0 9 = F 5 2 5 @ < 0 = 8 8 . > = D 5 @ 5 = F 8 O A > 1 5 @ 5 B H 8 @ > : 8 9 : @ C 3 M : A ? 5 @ B > 2 ? > 2 > ? @ > A 0 < 3 ; > 1 0 ; L = > 3 > = B 5 @ = 5 B 0 8 U n i c o d e , ; > : 0 ; 8 7 0 F 8 8 8 8 = B 5 @ = 0 F 8 > = 0 ; 8 7 0 F 8 8 , 2 > ? ; > I 5 = 8 N 8 ? @ 8 < 5 = 5 = 8 N U n i c o d e 2 @ 0 7 ; 8 G = K E > ? 5 @ 0 F 8 > = = K E A 8 A B 5 < 0 E 8 ? @ > 3 @ 0 < < = K E ? @ 8 ; > 6 5 = 8 O E , H @ 8 D B 0 E , 2 5 @ A B : 5 8 < = > 3 > O 7 K G = K E : > < ? L N B 5 @ = K E A 8 A B 5 < 0 E . T h a i ( U C S L e v e l 2 ) : E x c e r p t f r o m a p o e t r y o n T h e R o m a n c e o f T h e T h r e e K i n g d o m s ( a C h i n e s e c l a s s i c ' S a n G u a ' ) : [ - - - - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - - - - ] O A H 4 . 1 H @ * 7 H - ! B # ! A * * 1 @ ' # 0 @ ( - 9 J 9 I 6 I C + ! H * 4 * - ) 1 # 4 " L H - + I 2 A % 1 D * - - L D # I B H @ % 2 @ 2 1 2 # 1 7 - 1 5 @ G 5 H 6 H I 2 @ ! 7 - 6 ' 4 # 4 @ G 1 + 2 B . 4 K @ # 5 " 1 1 H ' + 1 ' @ ! 7 - ! 2 + ! 2 " 0 H 2 ! 1 H ' 1 ' * 3 1 @ + ! 7 - 1 D * D % H @ * 7 - 2 @ + 2 # 1 + ! 2 H 2 @ I 2 ! 2 @ % " - 2 * 1 H 2 " - I - - 8 I " 8 A " C + I A 1 C I * 2 ' 1 I @ G ' 7 H ' C % 1 % 4 8 " 8 " 5 % 1 H - @ + 8 H 2 - 2 @ ( # 4 + 2 I 2 # I - D + I I - # # 2 H 2 1 # # % 1 " $ E + 2 C # I 3 9 9 I # # % 1 L / ( T h e a b o v e i s a t w o - c o l u m n t e x t . I f c o m b i n i n g c h a r a c t e r s a r e h a n d l e d c o r r e c t l y , t h e l i n e s o f t h e s e c o n d c o l u m n s h o u l d b e a l i g n e d w i t h t h e | c h a r a c t e r a b o v e . ) E t h i o p i a n : P r o v e r b s i n t h e A m h a r i c l a n g u a g e : 0 s ( 5 % 0 5 b e c t ` F b % d q A % b ` E d c # # u ` b M s ` E d s = b % ` ` p s b 2 p ( ( b @ 5 ` @ 5 e A ` ) b - b e - ` 3 5 - b 0 d q ( d q p - b - H p . . 3 - b ( d u c e b u 5 E c u E b % + M s u K s u b c * e + b 5 ) + ) - b p b p I p 6 c I b - b ( - 5 u 0 b - ` M + = - b R u n e s : ( O l d E n g l i s h , w h i c h t r a n s c r i b e d i n t o L a t i n r e a d s ' H e c w a e t h t h a t h e b u d e t h a e m l a n d e n o r t h w e a r d u m w i t h t h a W e s t s a e . ' a n d m e a n s ' H e s a i d t h a t h e l i v e d i n t h e n o r t h e r n l a n d n e a r t h e W e s t e r n S e a . ' ) B r a i l l e : L( ( '( ( <( ( ( M( ( ( ( 9( 0( ( c( ( ( M( ( ( ( 9( :( ( ( ( ( ( ( ( ( ( ( ( ( ( :( ( 9( 2( y( ;( ( ( ( ( ( ( 3( ( ( 1( ( ( ( '( ;( ( ( 3( ( 9( ( ( 2( y( ( ( ( ( ( ( ;( ( ( ( ( ( ( %( ( ( ( ( :( ( ( ( ( ( ( +( ( 9( 9( ( ( ( ;( ( 9( ( ( ( ( 9( ( ( ( ;( ( ( 9( ( %( ( ( ;( ( ( ( ;( ( ( ( ( 9( ( !( ( ( ( ( 3( ( ( ;( 2( N( ( ( ( ( ( ( ( ( ( ( +( ( ( 2( A( ( ( N( ( ( ( ( ( ( 0( ( ( ( ( ( :( ( ( ( ( ( ( %( ( ( ( 0( a( ( ( ( ( ( ( ( ( ( ( 9( 9( ( ( ( ( !( ( ( ( ( ( ( %( ( ( ( ( ( ( ( ( ( ( 2( U( ( ( M( ( ( ( 9( :( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( $( ( ( ( ( 2( M( ( ( ( J( ( ( ( 0( ( ( ( ( ( ( ( ( ( 9( 9( ( ( J( ( ( *( ( ( ( ( 9( *( ( ( ( *( ( +( ( ( ( 1( ( ( 9( ;( ( ( ( ( ( ( ( ( %( ( ( ( 9( ( ( ( ( ( ( 3( ( ( ( ( ( ( $( ( ( ( ( 2( J( ( ( #( ( ( ( '( ( ( ( 2( ( ( ( ( +( ( ( 9( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( $( ( ( ( ( ( ( 9( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ;( 9( ( 9( ( ( ( ( ( ( 2( C( %( ( 9( ( :( ( ( ( ( ( ( ( 3( ( ( ( ( ( ( ( ( ( ( ( ( 9( ( ( ( ( ( ( ( ( ( ( ( ( 9( %( ( ( ( ( ( *( +( ( ( ( ( ( )( ( ( ( ( ( ( ( ( ( %( ( ( ( ( ( ( ( 9( ( J( 3( ( ( ( 9( 0( ( ( ( ( ( ( ( ( 2( y( 3( :( ( ( ( 9( ;( ( ( ( ( ( ( ;( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( 9( ( 9( ( ( M( ( ( ( 9( :( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( $( ( ( ( ( 2( ( T h e f i r s t c o u p l e o f p a r a g r a p h s o f " A C h r i s t m a s C a r o l " b y D i c k e n s ) C o m p a c t f o n t s e l e c t i o n e x a m p l e t e x t : A B C D E F G H I J K L M N O P Q R S T U V W X Y Z / 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z " & 0 "! S ` x ~ 0 1 2 3 4 " " " ! '" *" a" " ! ! ! ! ! % <% T% X% % % :& @& @$ P N# 1 G r e e t i n g s i n v a r i o u s l a n g u a g e s : H e l l o w o r l d , s y , 0 0 0 0 0 B o x d r a w i n g a l i g n m e n t t e s t s : % % T% P% P% f% P% P% W% % % % ,% % % % m% % % ,% % % n% m% % % ,% % % n% % % % 3% % % % % % % % w% {% % /% % % 0% % % q% r% q% r% s% s% s% Q% % % h% % % Q% % T% P% g% P% W% % % R% P% j% P% U% % % S% % A% % V% % % % % B% % % % % C% D% % v% <% t% z% K% x% % <% (% % K% %% % r% q% r% q% s% s% s% Q% % r% q% % Q% % Q% Q% % % % % % % % Q% % Q% % % % % % % % E% F% % u% y% % 7% % % 8% % % q% r% q% r% s% s% s% `% a% s% ^% c% % b% _% $% % <% % <% % <% $% % k% % B% % k% $% #% ?% ~% <% |% ?% +% % % % % % % % % N% % % % % % % r% q% r% q% s% s% s% Q% % q% r% % Q% % Q% Q% % % % % % % % Q% % Q% % % % }% % % % % % % % % % % % % N% O% % % % Q% % % e% % % Q% % Z% P% d% P% ]% % % X% P% j% P% [% % % Y% % @% % \% % % % % B% % % % % % % % % % % % % % N% O% % % % Z% P% P% i% P% P% ]% % % % 4% % % % p% % % 4% % % o% p% % % 4% % % o% % % % ;% % % % % % % % % % % L% L% % N% % M% M% % % % % % % % % % % % % % % % % flexi-streams-1.0.7/test/unicode_demo_utf8_cr.txt 0000664 0000000 0000000 00000033344 11175756704 0022132 0 ustar 00root root 0000000 0000000 UTF-8 encoded sample plain-text file ‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾ Markus Kuhn [ˈmaʳkʊs kuːn] — 2002-07-25 The ASCII compatible UTF-8 encoding used in this plain-text file is defined in Unicode, ISO 10646-1, and RFC 2279. Using Unicode/UTF-8, you can write in emails and source code things such as Mathematics and sciences: ∮ E⋅da = Q, n → ∞, ∑ f(i) = ∏ g(i), ⎧⎡⎛┌─────┐⎞⎤⎫ ⎪⎢⎜│a²+b³ ⎟⎥⎪ ∀x∈ℝ: ⌈x⌉ = −⌊−x⌋, α ∧ ¬β = ¬(¬α ∨ β), ⎪⎢⎜│───── ⎟⎥⎪ ⎪⎢⎜⎷ c₈ ⎟⎥⎪ ℕ ⊆ ℕ₀ ⊂ ℤ ⊂ ℚ ⊂ ℝ ⊂ ℂ, ⎨⎢⎜ ⎟⎥⎬ ⎪⎢⎜ ∞ ⎟⎥⎪ ⊥ < a ≠ b ≡ c ≤ d ≪ ⊤ ⇒ (⟦A⟧ ⇔ ⟪B⟫), ⎪⎢⎜ ⎲ ⎟⎥⎪ ⎪⎢⎜ ⎳aⁱ-bⁱ⎟⎥⎪ 2H₂ + O₂ ⇌ 2H₂O, R = 4.7 kΩ, ⌀ 200 mm ⎩⎣⎝i=1 ⎠⎦⎭ Linguistics and dictionaries: ði ıntəˈnæʃənəl fəˈnɛtık əsoʊsiˈeıʃn Y [ˈʏpsilɔn], Yen [jɛn], Yoga [ˈjoːgɑ] APL: ((V⍳V)=⍳⍴V)/V←,V ⌷←⍳→⍴∆∇⊃‾⍎⍕⌈ Nicer typography in plain text files: ╔══════════════════════════════════════════╗ ║ ║ ║ • ‘single’ and “double” quotes ║ ║ ║ ║ • Curly apostrophes: “We’ve been here” ║ ║ ║ ║ • Latin-1 apostrophe and accents: '´` ║ ║ ║ ║ • ‚deutsche‘ „Anführungszeichen“ ║ ║ ║ ║ • †, ‡, ‰, •, 3–4, —, −5/+5, ™, … ║ ║ ║ ║ • ASCII safety test: 1lI|, 0OD, 8B ║ ║ ╭─────────╮ ║ ║ • the euro symbol: │ 14.95 € │ ║ ║ ╰─────────╯ ║ ╚══════════════════════════════════════════╝ Combining characters: STARGΛ̊TE SG-1, a = v̇ = r̈, a⃑ ⊥ b⃑ Greek (in Polytonic): The Greek anthem: Σὲ γνωρίζω ἀπὸ τὴν κόψη τοῦ σπαθιοῦ τὴν τρομερή, σὲ γνωρίζω ἀπὸ τὴν ὄψη ποὺ μὲ βία μετράει τὴ γῆ. ᾿Απ᾿ τὰ κόκκαλα βγαλμένη τῶν ῾Ελλήνων τὰ ἱερά καὶ σὰν πρῶτα ἀνδρειωμένη χαῖρε, ὦ χαῖρε, ᾿Ελευθεριά! From a speech of Demosthenes in the 4th century BC: Οὐχὶ ταὐτὰ παρίσταταί μοι γιγνώσκειν, ὦ ἄνδρες ᾿Αθηναῖοι, ὅταν τ᾿ εἰς τὰ πράγματα ἀποβλέψω καὶ ὅταν πρὸς τοὺς λόγους οὓς ἀκούω· τοὺς μὲν γὰρ λόγους περὶ τοῦ τιμωρήσασθαι Φίλιππον ὁρῶ γιγνομένους, τὰ δὲ πράγματ᾿ εἰς τοῦτο προήκοντα, ὥσθ᾿ ὅπως μὴ πεισόμεθ᾿ αὐτοὶ πρότερον κακῶς σκέψασθαι δέον. οὐδέν οὖν ἄλλο μοι δοκοῦσιν οἱ τὰ τοιαῦτα λέγοντες ἢ τὴν ὑπόθεσιν, περὶ ἧς βουλεύεσθαι, οὐχὶ τὴν οὖσαν παριστάντες ὑμῖν ἁμαρτάνειν. ἐγὼ δέ, ὅτι μέν ποτ᾿ ἐξῆν τῇ πόλει καὶ τὰ αὑτῆς ἔχειν ἀσφαλῶς καὶ Φίλιππον τιμωρήσασθαι, καὶ μάλ᾿ ἀκριβῶς οἶδα· ἐπ᾿ ἐμοῦ γάρ, οὐ πάλαι γέγονεν ταῦτ᾿ ἀμφότερα· νῦν μέντοι πέπεισμαι τοῦθ᾿ ἱκανὸν προλαβεῖν ἡμῖν εἶναι τὴν πρώτην, ὅπως τοὺς συμμάχους σώσομεν. ἐὰν γὰρ τοῦτο βεβαίως ὑπάρξῃ, τότε καὶ περὶ τοῦ τίνα τιμωρήσεταί τις καὶ ὃν τρόπον ἐξέσται σκοπεῖν· πρὶν δὲ τὴν ἀρχὴν ὀρθῶς ὑποθέσθαι, μάταιον ἡγοῦμαι περὶ τῆς τελευτῆς ὁντινοῦν ποιεῖσθαι λόγον. Δημοσθένους, Γ´ ᾿Ολυνθιακὸς Georgian: From a Unicode conference invitation: გთხოვთ ახლავე გაიაროთ რეგისტრაცია Unicode-ის მეათე საერთაშორისო კონფერენციაზე დასასწრებად, რომელიც გაიმართება 10-12 მარტს, ქ. მაინცში, გერმანიაში. კონფერენცია შეჰკრებს ერთად მსოფლიოს ექსპერტებს ისეთ დარგებში როგორიცაა ინტერნეტი და Unicode-ი, ინტერნაციონალიზაცია და ლოკალიზაცია, Unicode-ის გამოყენება ოპერაციულ სისტემებსა, და გამოყენებით პროგრამებში, შრიფტებში, ტექსტების დამუშავებასა და მრავალენოვან კომპიუტერულ სისტემებში. Russian: From a Unicode conference invitation: Зарегистрируйтесь сейчас на Десятую Международную Конференцию по Unicode, которая состоится 10-12 марта 1997 года в Майнце в Германии. Конференция соберет широкий круг экспертов по вопросам глобального Интернета и Unicode, локализации и интернационализации, воплощению и применению Unicode в различных операционных системах и программных приложениях, шрифтах, верстке и многоязычных компьютерных системах. Thai (UCS Level 2): Excerpt from a poetry on The Romance of The Three Kingdoms (a Chinese classic 'San Gua'): [----------------------------|------------------------] ๏ แผ่นดินฮั่นเสื่อมโทรมแสนสังเวช พระปกเกศกองบู๊กู้ขึ้นใหม่ สิบสองกษัตริย์ก่อนหน้าแลถัดไป สององค์ไซร้โง่เขลาเบาปัญญา ทรงนับถือขันทีเป็นที่พึ่ง บ้านเมืองจึงวิปริตเป็นนักหนา โฮจิ๋นเรียกทัพทั่วหัวเมืองมา หมายจะฆ่ามดชั่วตัวสำคัญ เหมือนขับไสไล่เสือจากเคหา รับหมาป่าเข้ามาเลยอาสัญ ฝ่ายอ้องอุ้นยุแยกให้แตกกัน ใช้สาวนั้นเป็นชนวนชื่นชวนใจ พลันลิฉุยกุยกีกลับก่อเหตุ ช่างอาเพศจริงหนาฟ้าร้องไห้ ต้องรบราฆ่าฟันจนบรรลัย ฤๅหาใครค้ำชูกู้บรรลังก์ ฯ (The above is a two-column text. If combining characters are handled correctly, the lines of the second column should be aligned with the | character above.) Ethiopian: Proverbs in the Amharic language: ሰማይ አይታረስ ንጉሥ አይከሰስ። ብላ ካለኝ እንደአባቴ በቆመጠኝ። ጌጥ ያለቤቱ ቁምጥና ነው። ደሀ በሕልሙ ቅቤ ባይጠጣ ንጣት በገደለው። የአፍ ወለምታ በቅቤ አይታሽም። አይጥ በበላ ዳዋ ተመታ። ሲተረጉሙ ይደረግሙ። ቀስ በቀስ፥ ዕንቁላል በእግሩ ይሄዳል። ድር ቢያብር አንበሳ ያስር። ሰው እንደቤቱ እንጅ እንደ ጉረቤቱ አይተዳደርም። እግዜር የከፈተውን ጉሮሮ ሳይዘጋው አይድርም። የጎረቤት ሌባ፥ ቢያዩት ይስቅ ባያዩት ያጠልቅ። ሥራ ከመፍታት ልጄን ላፋታት። ዓባይ ማደሪያ የለው፥ ግንድ ይዞ ይዞራል። የእስላም አገሩ መካ የአሞራ አገሩ ዋርካ። ተንጋሎ ቢተፉ ተመልሶ ባፉ። ወዳጅህ ማር ቢሆን ጨርስህ አትላሰው። እግርህን በፍራሽህ ልክ ዘርጋ። Runes: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ (Old English, which transcribed into Latin reads 'He cwaeth that he bude thaem lande northweardum with tha Westsae.' and means 'He said that he lived in the northern land near the Western Sea.') Braille: ⡌⠁⠧⠑ ⠼⠁⠒ ⡍⠜⠇⠑⠹⠰⠎ ⡣⠕⠌ ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠙⠑⠁⠙⠒ ⠞⠕ ⠃⠑⠛⠔ ⠺⠊⠹⠲ ⡹⠻⠑ ⠊⠎ ⠝⠕ ⠙⠳⠃⠞ ⠱⠁⠞⠑⠧⠻ ⠁⠃⠳⠞ ⠹⠁⠞⠲ ⡹⠑ ⠗⠑⠛⠊⠌⠻ ⠕⠋ ⠙⠊⠎ ⠃⠥⠗⠊⠁⠇ ⠺⠁⠎ ⠎⠊⠛⠝⠫ ⠃⠹ ⠹⠑ ⠊⠇⠻⠛⠹⠍⠁⠝⠂ ⠹⠑ ⠊⠇⠻⠅⠂ ⠹⠑ ⠥⠝⠙⠻⠞⠁⠅⠻⠂ ⠁⠝⠙ ⠹⠑ ⠡⠊⠑⠋ ⠍⠳⠗⠝⠻⠲ ⡎⠊⠗⠕⠕⠛⠑ ⠎⠊⠛⠝⠫ ⠊⠞⠲ ⡁⠝⠙ ⡎⠊⠗⠕⠕⠛⠑⠰⠎ ⠝⠁⠍⠑ ⠺⠁⠎ ⠛⠕⠕⠙ ⠥⠏⠕⠝ ⠰⡡⠁⠝⠛⠑⠂ ⠋⠕⠗ ⠁⠝⠹⠹⠔⠛ ⠙⠑ ⠡⠕⠎⠑ ⠞⠕ ⠏⠥⠞ ⠙⠊⠎ ⠙⠁⠝⠙ ⠞⠕⠲ ⡕⠇⠙ ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠁⠎ ⠙⠑⠁⠙ ⠁⠎ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲ ⡍⠔⠙⠖ ⡊ ⠙⠕⠝⠰⠞ ⠍⠑⠁⠝ ⠞⠕ ⠎⠁⠹ ⠹⠁⠞ ⡊ ⠅⠝⠪⠂ ⠕⠋ ⠍⠹ ⠪⠝ ⠅⠝⠪⠇⠫⠛⠑⠂ ⠱⠁⠞ ⠹⠻⠑ ⠊⠎ ⠏⠜⠞⠊⠊⠥⠇⠜⠇⠹ ⠙⠑⠁⠙ ⠁⠃⠳⠞ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲ ⡊ ⠍⠊⠣⠞ ⠙⠁⠧⠑ ⠃⠑⠲ ⠔⠊⠇⠔⠫⠂ ⠍⠹⠎⠑⠇⠋⠂ ⠞⠕ ⠗⠑⠛⠜⠙ ⠁ ⠊⠕⠋⠋⠔⠤⠝⠁⠊⠇ ⠁⠎ ⠹⠑ ⠙⠑⠁⠙⠑⠌ ⠏⠊⠑⠊⠑ ⠕⠋ ⠊⠗⠕⠝⠍⠕⠝⠛⠻⠹ ⠔ ⠹⠑ ⠞⠗⠁⠙⠑⠲ ⡃⠥⠞ ⠹⠑ ⠺⠊⠎⠙⠕⠍ ⠕⠋ ⠳⠗ ⠁⠝⠊⠑⠌⠕⠗⠎ ⠊⠎ ⠔ ⠹⠑ ⠎⠊⠍⠊⠇⠑⠆ ⠁⠝⠙ ⠍⠹ ⠥⠝⠙⠁⠇⠇⠪⠫ ⠙⠁⠝⠙⠎ ⠩⠁⠇⠇ ⠝⠕⠞ ⠙⠊⠌⠥⠗⠃ ⠊⠞⠂ ⠕⠗ ⠹⠑ ⡊⠳⠝⠞⠗⠹⠰⠎ ⠙⠕⠝⠑ ⠋⠕⠗⠲ ⡹⠳ ⠺⠊⠇⠇ ⠹⠻⠑⠋⠕⠗⠑ ⠏⠻⠍⠊⠞ ⠍⠑ ⠞⠕ ⠗⠑⠏⠑⠁⠞⠂ ⠑⠍⠏⠙⠁⠞⠊⠊⠁⠇⠇⠹⠂ ⠹⠁⠞ ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠁⠎ ⠙⠑⠁⠙ ⠁⠎ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲ (The first couple of paragraphs of "A Christmas Carol" by Dickens) Compact font selection example text: ABCDEFGHIJKLMNOPQRSTUVWXYZ /0123456789 abcdefghijklmnopqrstuvwxyz £©µÀÆÖÞßéöÿ –—‘“”„†•…‰™œŠŸž€ ΑΒΓΔΩαβγδω АБВГДабвгд ∀∂∈ℝ∧∪≡∞ ↑↗↨↻⇣ ┐┼╔╘░►☺♀ fi�⑀₂ἠḂӥẄɐː⍎אԱა Greetings in various languages: Hello world, Καλημέρα κόσμε, コンニチハ Box drawing alignment tests: █ ▉ ╔══╦══╗ ┌──┬──┐ ╭──┬──╮ ╭──┬──╮ ┏━━┳━━┓ ┎┒┏┑ ╷ ╻ ┏┯┓ ┌┰┐ ▊ ╱╲╱╲╳╳╳ ║┌─╨─┐║ │╔═╧═╗│ │╒═╪═╕│ │╓─╁─╖│ ┃┌─╂─┐┃ ┗╃╄┙ ╶┼╴╺╋╸┠┼┨ ┝╋┥ ▋ ╲╱╲╱╳╳╳ ║│╲ ╱│║ │║ ║│ ││ │ ││ │║ ┃ ║│ ┃│ ╿ │┃ ┍╅╆┓ ╵ ╹ ┗┷┛ └┸┘ ▌ ╱╲╱╲╳╳╳ ╠╡ ╳ ╞╣ ├╢ ╟┤ ├┼─┼─┼┤ ├╫─╂─╫┤ ┣┿╾┼╼┿┫ ┕┛┖┚ ┌┄┄┐ ╎ ┏┅┅┓ ┋ ▍ ╲╱╲╱╳╳╳ ║│╱ ╲│║ │║ ║│ ││ │ ││ │║ ┃ ║│ ┃│ ╽ │┃ ░░▒▒▓▓██ ┊ ┆ ╎ ╏ ┇ ┋ ▎ ║└─╥─┘║ │╚═╤═╝│ │╘═╪═╛│ │╙─╀─╜│ ┃└─╂─┘┃ ░░▒▒▓▓██ ┊ ┆ ╎ ╏ ┇ ┋ ▏ ╚══╩══╝ └──┴──┘ ╰──┴──╯ ╰──┴──╯ ┗━━┻━━┛ ▗▄▖▛▀▜ └╌╌┘ ╎ ┗╍╍┛ ┋ ▁▂▃▄▅▆▇█ ▝▀▘▙▄▟ flexi-streams-1.0.7/test/unicode_demo_utf8_crlf.txt 0000664 0000000 0000000 00000033670 11175756704 0022456 0 ustar 00root root 0000000 0000000 UTF-8 encoded sample plain-text file ‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾ Markus Kuhn [ˈmaʳkʊs kuːn] — 2002-07-25 The ASCII compatible UTF-8 encoding used in this plain-text file is defined in Unicode, ISO 10646-1, and RFC 2279. Using Unicode/UTF-8, you can write in emails and source code things such as Mathematics and sciences: ∮ E⋅da = Q, n → ∞, ∑ f(i) = ∏ g(i), ⎧⎡⎛┌─────┐⎞⎤⎫ ⎪⎢⎜│a²+b³ ⎟⎥⎪ ∀x∈ℝ: ⌈x⌉ = −⌊−x⌋, α ∧ ¬β = ¬(¬α ∨ β), ⎪⎢⎜│───── ⎟⎥⎪ ⎪⎢⎜⎷ c₈ ⎟⎥⎪ ℕ ⊆ ℕ₀ ⊂ ℤ ⊂ ℚ ⊂ ℝ ⊂ ℂ, ⎨⎢⎜ ⎟⎥⎬ ⎪⎢⎜ ∞ ⎟⎥⎪ ⊥ < a ≠ b ≡ c ≤ d ≪ ⊤ ⇒ (⟦A⟧ ⇔ ⟪B⟫), ⎪⎢⎜ ⎲ ⎟⎥⎪ ⎪⎢⎜ ⎳aⁱ-bⁱ⎟⎥⎪ 2H₂ + O₂ ⇌ 2H₂O, R = 4.7 kΩ, ⌀ 200 mm ⎩⎣⎝i=1 ⎠⎦⎭ Linguistics and dictionaries: ði ıntəˈnæʃənəl fəˈnɛtık əsoʊsiˈeıʃn Y [ˈʏpsilɔn], Yen [jɛn], Yoga [ˈjoːgɑ] APL: ((V⍳V)=⍳⍴V)/V←,V ⌷←⍳→⍴∆∇⊃‾⍎⍕⌈ Nicer typography in plain text files: ╔══════════════════════════════════════════╗ ║ ║ ║ • ‘single’ and “double” quotes ║ ║ ║ ║ • Curly apostrophes: “We’ve been here” ║ ║ ║ ║ • Latin-1 apostrophe and accents: '´` ║ ║ ║ ║ • ‚deutsche‘ „Anführungszeichen“ ║ ║ ║ ║ • †, ‡, ‰, •, 3–4, —, −5/+5, ™, … ║ ║ ║ ║ • ASCII safety test: 1lI|, 0OD, 8B ║ ║ ╭─────────╮ ║ ║ • the euro symbol: │ 14.95 € │ ║ ║ ╰─────────╯ ║ ╚══════════════════════════════════════════╝ Combining characters: STARGΛ̊TE SG-1, a = v̇ = r̈, a⃑ ⊥ b⃑ Greek (in Polytonic): The Greek anthem: Σὲ γνωρίζω ἀπὸ τὴν κόψη τοῦ σπαθιοῦ τὴν τρομερή, σὲ γνωρίζω ἀπὸ τὴν ὄψη ποὺ μὲ βία μετράει τὴ γῆ. ᾿Απ᾿ τὰ κόκκαλα βγαλμένη τῶν ῾Ελλήνων τὰ ἱερά καὶ σὰν πρῶτα ἀνδρειωμένη χαῖρε, ὦ χαῖρε, ᾿Ελευθεριά! From a speech of Demosthenes in the 4th century BC: Οὐχὶ ταὐτὰ παρίσταταί μοι γιγνώσκειν, ὦ ἄνδρες ᾿Αθηναῖοι, ὅταν τ᾿ εἰς τὰ πράγματα ἀποβλέψω καὶ ὅταν πρὸς τοὺς λόγους οὓς ἀκούω· τοὺς μὲν γὰρ λόγους περὶ τοῦ τιμωρήσασθαι Φίλιππον ὁρῶ γιγνομένους, τὰ δὲ πράγματ᾿ εἰς τοῦτο προήκοντα, ὥσθ᾿ ὅπως μὴ πεισόμεθ᾿ αὐτοὶ πρότερον κακῶς σκέψασθαι δέον. οὐδέν οὖν ἄλλο μοι δοκοῦσιν οἱ τὰ τοιαῦτα λέγοντες ἢ τὴν ὑπόθεσιν, περὶ ἧς βουλεύεσθαι, οὐχὶ τὴν οὖσαν παριστάντες ὑμῖν ἁμαρτάνειν. ἐγὼ δέ, ὅτι μέν ποτ᾿ ἐξῆν τῇ πόλει καὶ τὰ αὑτῆς ἔχειν ἀσφαλῶς καὶ Φίλιππον τιμωρήσασθαι, καὶ μάλ᾿ ἀκριβῶς οἶδα· ἐπ᾿ ἐμοῦ γάρ, οὐ πάλαι γέγονεν ταῦτ᾿ ἀμφότερα· νῦν μέντοι πέπεισμαι τοῦθ᾿ ἱκανὸν προλαβεῖν ἡμῖν εἶναι τὴν πρώτην, ὅπως τοὺς συμμάχους σώσομεν. ἐὰν γὰρ τοῦτο βεβαίως ὑπάρξῃ, τότε καὶ περὶ τοῦ τίνα τιμωρήσεταί τις καὶ ὃν τρόπον ἐξέσται σκοπεῖν· πρὶν δὲ τὴν ἀρχὴν ὀρθῶς ὑποθέσθαι, μάταιον ἡγοῦμαι περὶ τῆς τελευτῆς ὁντινοῦν ποιεῖσθαι λόγον. Δημοσθένους, Γ´ ᾿Ολυνθιακὸς Georgian: From a Unicode conference invitation: გთხოვთ ახლავე გაიაროთ რეგისტრაცია Unicode-ის მეათე საერთაშორისო კონფერენციაზე დასასწრებად, რომელიც გაიმართება 10-12 მარტს, ქ. მაინცში, გერმანიაში. კონფერენცია შეჰკრებს ერთად მსოფლიოს ექსპერტებს ისეთ დარგებში როგორიცაა ინტერნეტი და Unicode-ი, ინტერნაციონალიზაცია და ლოკალიზაცია, Unicode-ის გამოყენება ოპერაციულ სისტემებსა, და გამოყენებით პროგრამებში, შრიფტებში, ტექსტების დამუშავებასა და მრავალენოვან კომპიუტერულ სისტემებში. Russian: From a Unicode conference invitation: Зарегистрируйтесь сейчас на Десятую Международную Конференцию по Unicode, которая состоится 10-12 марта 1997 года в Майнце в Германии. Конференция соберет широкий круг экспертов по вопросам глобального Интернета и Unicode, локализации и интернационализации, воплощению и применению Unicode в различных операционных системах и программных приложениях, шрифтах, верстке и многоязычных компьютерных системах. Thai (UCS Level 2): Excerpt from a poetry on The Romance of The Three Kingdoms (a Chinese classic 'San Gua'): [----------------------------|------------------------] ๏ แผ่นดินฮั่นเสื่อมโทรมแสนสังเวช พระปกเกศกองบู๊กู้ขึ้นใหม่ สิบสองกษัตริย์ก่อนหน้าแลถัดไป สององค์ไซร้โง่เขลาเบาปัญญา ทรงนับถือขันทีเป็นที่พึ่ง บ้านเมืองจึงวิปริตเป็นนักหนา โฮจิ๋นเรียกทัพทั่วหัวเมืองมา หมายจะฆ่ามดชั่วตัวสำคัญ เหมือนขับไสไล่เสือจากเคหา รับหมาป่าเข้ามาเลยอาสัญ ฝ่ายอ้องอุ้นยุแยกให้แตกกัน ใช้สาวนั้นเป็นชนวนชื่นชวนใจ พลันลิฉุยกุยกีกลับก่อเหตุ ช่างอาเพศจริงหนาฟ้าร้องไห้ ต้องรบราฆ่าฟันจนบรรลัย ฤๅหาใครค้ำชูกู้บรรลังก์ ฯ (The above is a two-column text. If combining characters are handled correctly, the lines of the second column should be aligned with the | character above.) Ethiopian: Proverbs in the Amharic language: ሰማይ አይታረስ ንጉሥ አይከሰስ። ብላ ካለኝ እንደአባቴ በቆመጠኝ። ጌጥ ያለቤቱ ቁምጥና ነው። ደሀ በሕልሙ ቅቤ ባይጠጣ ንጣት በገደለው። የአፍ ወለምታ በቅቤ አይታሽም። አይጥ በበላ ዳዋ ተመታ። ሲተረጉሙ ይደረግሙ። ቀስ በቀስ፥ ዕንቁላል በእግሩ ይሄዳል። ድር ቢያብር አንበሳ ያስር። ሰው እንደቤቱ እንጅ እንደ ጉረቤቱ አይተዳደርም። እግዜር የከፈተውን ጉሮሮ ሳይዘጋው አይድርም። የጎረቤት ሌባ፥ ቢያዩት ይስቅ ባያዩት ያጠልቅ። ሥራ ከመፍታት ልጄን ላፋታት። ዓባይ ማደሪያ የለው፥ ግንድ ይዞ ይዞራል። የእስላም አገሩ መካ የአሞራ አገሩ ዋርካ። ተንጋሎ ቢተፉ ተመልሶ ባፉ። ወዳጅህ ማር ቢሆን ጨርስህ አትላሰው። እግርህን በፍራሽህ ልክ ዘርጋ። Runes: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ (Old English, which transcribed into Latin reads 'He cwaeth that he bude thaem lande northweardum with tha Westsae.' and means 'He said that he lived in the northern land near the Western Sea.') Braille: ⡌⠁⠧⠑ ⠼⠁⠒ ⡍⠜⠇⠑⠹⠰⠎ ⡣⠕⠌ ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠙⠑⠁⠙⠒ ⠞⠕ ⠃⠑⠛⠔ ⠺⠊⠹⠲ ⡹⠻⠑ ⠊⠎ ⠝⠕ ⠙⠳⠃⠞ ⠱⠁⠞⠑⠧⠻ ⠁⠃⠳⠞ ⠹⠁⠞⠲ ⡹⠑ ⠗⠑⠛⠊⠌⠻ ⠕⠋ ⠙⠊⠎ ⠃⠥⠗⠊⠁⠇ ⠺⠁⠎ ⠎⠊⠛⠝⠫ ⠃⠹ ⠹⠑ ⠊⠇⠻⠛⠹⠍⠁⠝⠂ ⠹⠑ ⠊⠇⠻⠅⠂ ⠹⠑ ⠥⠝⠙⠻⠞⠁⠅⠻⠂ ⠁⠝⠙ ⠹⠑ ⠡⠊⠑⠋ ⠍⠳⠗⠝⠻⠲ ⡎⠊⠗⠕⠕⠛⠑ ⠎⠊⠛⠝⠫ ⠊⠞⠲ ⡁⠝⠙ ⡎⠊⠗⠕⠕⠛⠑⠰⠎ ⠝⠁⠍⠑ ⠺⠁⠎ ⠛⠕⠕⠙ ⠥⠏⠕⠝ ⠰⡡⠁⠝⠛⠑⠂ ⠋⠕⠗ ⠁⠝⠹⠹⠔⠛ ⠙⠑ ⠡⠕⠎⠑ ⠞⠕ ⠏⠥⠞ ⠙⠊⠎ ⠙⠁⠝⠙ ⠞⠕⠲ ⡕⠇⠙ ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠁⠎ ⠙⠑⠁⠙ ⠁⠎ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲ ⡍⠔⠙⠖ ⡊ ⠙⠕⠝⠰⠞ ⠍⠑⠁⠝ ⠞⠕ ⠎⠁⠹ ⠹⠁⠞ ⡊ ⠅⠝⠪⠂ ⠕⠋ ⠍⠹ ⠪⠝ ⠅⠝⠪⠇⠫⠛⠑⠂ ⠱⠁⠞ ⠹⠻⠑ ⠊⠎ ⠏⠜⠞⠊⠊⠥⠇⠜⠇⠹ ⠙⠑⠁⠙ ⠁⠃⠳⠞ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲ ⡊ ⠍⠊⠣⠞ ⠙⠁⠧⠑ ⠃⠑⠲ ⠔⠊⠇⠔⠫⠂ ⠍⠹⠎⠑⠇⠋⠂ ⠞⠕ ⠗⠑⠛⠜⠙ ⠁ ⠊⠕⠋⠋⠔⠤⠝⠁⠊⠇ ⠁⠎ ⠹⠑ ⠙⠑⠁⠙⠑⠌ ⠏⠊⠑⠊⠑ ⠕⠋ ⠊⠗⠕⠝⠍⠕⠝⠛⠻⠹ ⠔ ⠹⠑ ⠞⠗⠁⠙⠑⠲ ⡃⠥⠞ ⠹⠑ ⠺⠊⠎⠙⠕⠍ ⠕⠋ ⠳⠗ ⠁⠝⠊⠑⠌⠕⠗⠎ ⠊⠎ ⠔ ⠹⠑ ⠎⠊⠍⠊⠇⠑⠆ ⠁⠝⠙ ⠍⠹ ⠥⠝⠙⠁⠇⠇⠪⠫ ⠙⠁⠝⠙⠎ ⠩⠁⠇⠇ ⠝⠕⠞ ⠙⠊⠌⠥⠗⠃ ⠊⠞⠂ ⠕⠗ ⠹⠑ ⡊⠳⠝⠞⠗⠹⠰⠎ ⠙⠕⠝⠑ ⠋⠕⠗⠲ ⡹⠳ ⠺⠊⠇⠇ ⠹⠻⠑⠋⠕⠗⠑ ⠏⠻⠍⠊⠞ ⠍⠑ ⠞⠕ ⠗⠑⠏⠑⠁⠞⠂ ⠑⠍⠏⠙⠁⠞⠊⠊⠁⠇⠇⠹⠂ ⠹⠁⠞ ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠁⠎ ⠙⠑⠁⠙ ⠁⠎ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲ (The first couple of paragraphs of "A Christmas Carol" by Dickens) Compact font selection example text: ABCDEFGHIJKLMNOPQRSTUVWXYZ /0123456789 abcdefghijklmnopqrstuvwxyz £©µÀÆÖÞßéöÿ –—‘“”„†•…‰™œŠŸž€ ΑΒΓΔΩαβγδω АБВГДабвгд ∀∂∈ℝ∧∪≡∞ ↑↗↨↻⇣ ┐┼╔╘░►☺♀ fi�⑀₂ἠḂӥẄɐː⍎אԱა Greetings in various languages: Hello world, Καλημέρα κόσμε, コンニチハ Box drawing alignment tests: █ ▉ ╔══╦══╗ ┌──┬──┐ ╭──┬──╮ ╭──┬──╮ ┏━━┳━━┓ ┎┒┏┑ ╷ ╻ ┏┯┓ ┌┰┐ ▊ ╱╲╱╲╳╳╳ ║┌─╨─┐║ │╔═╧═╗│ │╒═╪═╕│ │╓─╁─╖│ ┃┌─╂─┐┃ ┗╃╄┙ ╶┼╴╺╋╸┠┼┨ ┝╋┥ ▋ ╲╱╲╱╳╳╳ ║│╲ ╱│║ │║ ║│ ││ │ ││ │║ ┃ ║│ ┃│ ╿ │┃ ┍╅╆┓ ╵ ╹ ┗┷┛ └┸┘ ▌ ╱╲╱╲╳╳╳ ╠╡ ╳ ╞╣ ├╢ ╟┤ ├┼─┼─┼┤ ├╫─╂─╫┤ ┣┿╾┼╼┿┫ ┕┛┖┚ ┌┄┄┐ ╎ ┏┅┅┓ ┋ ▍ ╲╱╲╱╳╳╳ ║│╱ ╲│║ │║ ║│ ││ │ ││ │║ ┃ ║│ ┃│ ╽ │┃ ░░▒▒▓▓██ ┊ ┆ ╎ ╏ ┇ ┋ ▎ ║└─╥─┘║ │╚═╤═╝│ │╘═╪═╛│ │╙─╀─╜│ ┃└─╂─┘┃ ░░▒▒▓▓██ ┊ ┆ ╎ ╏ ┇ ┋ ▏ ╚══╩══╝ └──┴──┘ ╰──┴──╯ ╰──┴──╯ ┗━━┻━━┛ ▗▄▖▛▀▜ └╌╌┘ ╎ ┗╍╍┛ ┋ ▁▂▃▄▅▆▇█ ▝▀▘▙▄▟ flexi-streams-1.0.7/test/unicode_demo_utf8_lf.txt 0000664 0000000 0000000 00000033344 11175756704 0022127 0 ustar 00root root 0000000 0000000 UTF-8 encoded sample plain-text file ‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾ Markus Kuhn [ˈmaʳkʊs kuːn] — 2002-07-25 The ASCII compatible UTF-8 encoding used in this plain-text file is defined in Unicode, ISO 10646-1, and RFC 2279. Using Unicode/UTF-8, you can write in emails and source code things such as Mathematics and sciences: ∮ E⋅da = Q, n → ∞, ∑ f(i) = ∏ g(i), ⎧⎡⎛┌─────┐⎞⎤⎫ ⎪⎢⎜│a²+b³ ⎟⎥⎪ ∀x∈ℝ: ⌈x⌉ = −⌊−x⌋, α ∧ ¬β = ¬(¬α ∨ β), ⎪⎢⎜│───── ⎟⎥⎪ ⎪⎢⎜⎷ c₈ ⎟⎥⎪ ℕ ⊆ ℕ₀ ⊂ ℤ ⊂ ℚ ⊂ ℝ ⊂ ℂ, ⎨⎢⎜ ⎟⎥⎬ ⎪⎢⎜ ∞ ⎟⎥⎪ ⊥ < a ≠ b ≡ c ≤ d ≪ ⊤ ⇒ (⟦A⟧ ⇔ ⟪B⟫), ⎪⎢⎜ ⎲ ⎟⎥⎪ ⎪⎢⎜ ⎳aⁱ-bⁱ⎟⎥⎪ 2H₂ + O₂ ⇌ 2H₂O, R = 4.7 kΩ, ⌀ 200 mm ⎩⎣⎝i=1 ⎠⎦⎭ Linguistics and dictionaries: ði ıntəˈnæʃənəl fəˈnɛtık əsoʊsiˈeıʃn Y [ˈʏpsilɔn], Yen [jɛn], Yoga [ˈjoːgɑ] APL: ((V⍳V)=⍳⍴V)/V←,V ⌷←⍳→⍴∆∇⊃‾⍎⍕⌈ Nicer typography in plain text files: ╔══════════════════════════════════════════╗ ║ ║ ║ • ‘single’ and “double” quotes ║ ║ ║ ║ • Curly apostrophes: “We’ve been here” ║ ║ ║ ║ • Latin-1 apostrophe and accents: '´` ║ ║ ║ ║ • ‚deutsche‘ „Anführungszeichen“ ║ ║ ║ ║ • †, ‡, ‰, •, 3–4, —, −5/+5, ™, … ║ ║ ║ ║ • ASCII safety test: 1lI|, 0OD, 8B ║ ║ ╭─────────╮ ║ ║ • the euro symbol: │ 14.95 € │ ║ ║ ╰─────────╯ ║ ╚══════════════════════════════════════════╝ Combining characters: STARGΛ̊TE SG-1, a = v̇ = r̈, a⃑ ⊥ b⃑ Greek (in Polytonic): The Greek anthem: Σὲ γνωρίζω ἀπὸ τὴν κόψη τοῦ σπαθιοῦ τὴν τρομερή, σὲ γνωρίζω ἀπὸ τὴν ὄψη ποὺ μὲ βία μετράει τὴ γῆ. ᾿Απ᾿ τὰ κόκκαλα βγαλμένη τῶν ῾Ελλήνων τὰ ἱερά καὶ σὰν πρῶτα ἀνδρειωμένη χαῖρε, ὦ χαῖρε, ᾿Ελευθεριά! From a speech of Demosthenes in the 4th century BC: Οὐχὶ ταὐτὰ παρίσταταί μοι γιγνώσκειν, ὦ ἄνδρες ᾿Αθηναῖοι, ὅταν τ᾿ εἰς τὰ πράγματα ἀποβλέψω καὶ ὅταν πρὸς τοὺς λόγους οὓς ἀκούω· τοὺς μὲν γὰρ λόγους περὶ τοῦ τιμωρήσασθαι Φίλιππον ὁρῶ γιγνομένους, τὰ δὲ πράγματ᾿ εἰς τοῦτο προήκοντα, ὥσθ᾿ ὅπως μὴ πεισόμεθ᾿ αὐτοὶ πρότερον κακῶς σκέψασθαι δέον. οὐδέν οὖν ἄλλο μοι δοκοῦσιν οἱ τὰ τοιαῦτα λέγοντες ἢ τὴν ὑπόθεσιν, περὶ ἧς βουλεύεσθαι, οὐχὶ τὴν οὖσαν παριστάντες ὑμῖν ἁμαρτάνειν. ἐγὼ δέ, ὅτι μέν ποτ᾿ ἐξῆν τῇ πόλει καὶ τὰ αὑτῆς ἔχειν ἀσφαλῶς καὶ Φίλιππον τιμωρήσασθαι, καὶ μάλ᾿ ἀκριβῶς οἶδα· ἐπ᾿ ἐμοῦ γάρ, οὐ πάλαι γέγονεν ταῦτ᾿ ἀμφότερα· νῦν μέντοι πέπεισμαι τοῦθ᾿ ἱκανὸν προλαβεῖν ἡμῖν εἶναι τὴν πρώτην, ὅπως τοὺς συμμάχους σώσομεν. ἐὰν γὰρ τοῦτο βεβαίως ὑπάρξῃ, τότε καὶ περὶ τοῦ τίνα τιμωρήσεταί τις καὶ ὃν τρόπον ἐξέσται σκοπεῖν· πρὶν δὲ τὴν ἀρχὴν ὀρθῶς ὑποθέσθαι, μάταιον ἡγοῦμαι περὶ τῆς τελευτῆς ὁντινοῦν ποιεῖσθαι λόγον. Δημοσθένους, Γ´ ᾿Ολυνθιακὸς Georgian: From a Unicode conference invitation: გთხოვთ ახლავე გაიაროთ რეგისტრაცია Unicode-ის მეათე საერთაშორისო კონფერენციაზე დასასწრებად, რომელიც გაიმართება 10-12 მარტს, ქ. მაინცში, გერმანიაში. კონფერენცია შეჰკრებს ერთად მსოფლიოს ექსპერტებს ისეთ დარგებში როგორიცაა ინტერნეტი და Unicode-ი, ინტერნაციონალიზაცია და ლოკალიზაცია, Unicode-ის გამოყენება ოპერაციულ სისტემებსა, და გამოყენებით პროგრამებში, შრიფტებში, ტექსტების დამუშავებასა და მრავალენოვან კომპიუტერულ სისტემებში. Russian: From a Unicode conference invitation: Зарегистрируйтесь сейчас на Десятую Международную Конференцию по Unicode, которая состоится 10-12 марта 1997 года в Майнце в Германии. Конференция соберет широкий круг экспертов по вопросам глобального Интернета и Unicode, локализации и интернационализации, воплощению и применению Unicode в различных операционных системах и программных приложениях, шрифтах, верстке и многоязычных компьютерных системах. Thai (UCS Level 2): Excerpt from a poetry on The Romance of The Three Kingdoms (a Chinese classic 'San Gua'): [----------------------------|------------------------] ๏ แผ่นดินฮั่นเสื่อมโทรมแสนสังเวช พระปกเกศกองบู๊กู้ขึ้นใหม่ สิบสองกษัตริย์ก่อนหน้าแลถัดไป สององค์ไซร้โง่เขลาเบาปัญญา ทรงนับถือขันทีเป็นที่พึ่ง บ้านเมืองจึงวิปริตเป็นนักหนา โฮจิ๋นเรียกทัพทั่วหัวเมืองมา หมายจะฆ่ามดชั่วตัวสำคัญ เหมือนขับไสไล่เสือจากเคหา รับหมาป่าเข้ามาเลยอาสัญ ฝ่ายอ้องอุ้นยุแยกให้แตกกัน ใช้สาวนั้นเป็นชนวนชื่นชวนใจ พลันลิฉุยกุยกีกลับก่อเหตุ ช่างอาเพศจริงหนาฟ้าร้องไห้ ต้องรบราฆ่าฟันจนบรรลัย ฤๅหาใครค้ำชูกู้บรรลังก์ ฯ (The above is a two-column text. If combining characters are handled correctly, the lines of the second column should be aligned with the | character above.) Ethiopian: Proverbs in the Amharic language: ሰማይ አይታረስ ንጉሥ አይከሰስ። ብላ ካለኝ እንደአባቴ በቆመጠኝ። ጌጥ ያለቤቱ ቁምጥና ነው። ደሀ በሕልሙ ቅቤ ባይጠጣ ንጣት በገደለው። የአፍ ወለምታ በቅቤ አይታሽም። አይጥ በበላ ዳዋ ተመታ። ሲተረጉሙ ይደረግሙ። ቀስ በቀስ፥ ዕንቁላል በእግሩ ይሄዳል። ድር ቢያብር አንበሳ ያስር። ሰው እንደቤቱ እንጅ እንደ ጉረቤቱ አይተዳደርም። እግዜር የከፈተውን ጉሮሮ ሳይዘጋው አይድርም። የጎረቤት ሌባ፥ ቢያዩት ይስቅ ባያዩት ያጠልቅ። ሥራ ከመፍታት ልጄን ላፋታት። ዓባይ ማደሪያ የለው፥ ግንድ ይዞ ይዞራል። የእስላም አገሩ መካ የአሞራ አገሩ ዋርካ። ተንጋሎ ቢተፉ ተመልሶ ባፉ። ወዳጅህ ማር ቢሆን ጨርስህ አትላሰው። እግርህን በፍራሽህ ልክ ዘርጋ። Runes: ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ (Old English, which transcribed into Latin reads 'He cwaeth that he bude thaem lande northweardum with tha Westsae.' and means 'He said that he lived in the northern land near the Western Sea.') Braille: ⡌⠁⠧⠑ ⠼⠁⠒ ⡍⠜⠇⠑⠹⠰⠎ ⡣⠕⠌ ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠙⠑⠁⠙⠒ ⠞⠕ ⠃⠑⠛⠔ ⠺⠊⠹⠲ ⡹⠻⠑ ⠊⠎ ⠝⠕ ⠙⠳⠃⠞ ⠱⠁⠞⠑⠧⠻ ⠁⠃⠳⠞ ⠹⠁⠞⠲ ⡹⠑ ⠗⠑⠛⠊⠌⠻ ⠕⠋ ⠙⠊⠎ ⠃⠥⠗⠊⠁⠇ ⠺⠁⠎ ⠎⠊⠛⠝⠫ ⠃⠹ ⠹⠑ ⠊⠇⠻⠛⠹⠍⠁⠝⠂ ⠹⠑ ⠊⠇⠻⠅⠂ ⠹⠑ ⠥⠝⠙⠻⠞⠁⠅⠻⠂ ⠁⠝⠙ ⠹⠑ ⠡⠊⠑⠋ ⠍⠳⠗⠝⠻⠲ ⡎⠊⠗⠕⠕⠛⠑ ⠎⠊⠛⠝⠫ ⠊⠞⠲ ⡁⠝⠙ ⡎⠊⠗⠕⠕⠛⠑⠰⠎ ⠝⠁⠍⠑ ⠺⠁⠎ ⠛⠕⠕⠙ ⠥⠏⠕⠝ ⠰⡡⠁⠝⠛⠑⠂ ⠋⠕⠗ ⠁⠝⠹⠹⠔⠛ ⠙⠑ ⠡⠕⠎⠑ ⠞⠕ ⠏⠥⠞ ⠙⠊⠎ ⠙⠁⠝⠙ ⠞⠕⠲ ⡕⠇⠙ ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠁⠎ ⠙⠑⠁⠙ ⠁⠎ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲ ⡍⠔⠙⠖ ⡊ ⠙⠕⠝⠰⠞ ⠍⠑⠁⠝ ⠞⠕ ⠎⠁⠹ ⠹⠁⠞ ⡊ ⠅⠝⠪⠂ ⠕⠋ ⠍⠹ ⠪⠝ ⠅⠝⠪⠇⠫⠛⠑⠂ ⠱⠁⠞ ⠹⠻⠑ ⠊⠎ ⠏⠜⠞⠊⠊⠥⠇⠜⠇⠹ ⠙⠑⠁⠙ ⠁⠃⠳⠞ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲ ⡊ ⠍⠊⠣⠞ ⠙⠁⠧⠑ ⠃⠑⠲ ⠔⠊⠇⠔⠫⠂ ⠍⠹⠎⠑⠇⠋⠂ ⠞⠕ ⠗⠑⠛⠜⠙ ⠁ ⠊⠕⠋⠋⠔⠤⠝⠁⠊⠇ ⠁⠎ ⠹⠑ ⠙⠑⠁⠙⠑⠌ ⠏⠊⠑⠊⠑ ⠕⠋ ⠊⠗⠕⠝⠍⠕⠝⠛⠻⠹ ⠔ ⠹⠑ ⠞⠗⠁⠙⠑⠲ ⡃⠥⠞ ⠹⠑ ⠺⠊⠎⠙⠕⠍ ⠕⠋ ⠳⠗ ⠁⠝⠊⠑⠌⠕⠗⠎ ⠊⠎ ⠔ ⠹⠑ ⠎⠊⠍⠊⠇⠑⠆ ⠁⠝⠙ ⠍⠹ ⠥⠝⠙⠁⠇⠇⠪⠫ ⠙⠁⠝⠙⠎ ⠩⠁⠇⠇ ⠝⠕⠞ ⠙⠊⠌⠥⠗⠃ ⠊⠞⠂ ⠕⠗ ⠹⠑ ⡊⠳⠝⠞⠗⠹⠰⠎ ⠙⠕⠝⠑ ⠋⠕⠗⠲ ⡹⠳ ⠺⠊⠇⠇ ⠹⠻⠑⠋⠕⠗⠑ ⠏⠻⠍⠊⠞ ⠍⠑ ⠞⠕ ⠗⠑⠏⠑⠁⠞⠂ ⠑⠍⠏⠙⠁⠞⠊⠊⠁⠇⠇⠹⠂ ⠹⠁⠞ ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠁⠎ ⠙⠑⠁⠙ ⠁⠎ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲ (The first couple of paragraphs of "A Christmas Carol" by Dickens) Compact font selection example text: ABCDEFGHIJKLMNOPQRSTUVWXYZ /0123456789 abcdefghijklmnopqrstuvwxyz £©µÀÆÖÞßéöÿ –—‘“”„†•…‰™œŠŸž€ ΑΒΓΔΩαβγδω АБВГДабвгд ∀∂∈ℝ∧∪≡∞ ↑↗↨↻⇣ ┐┼╔╘░►☺♀ fi�⑀₂ἠḂӥẄɐː⍎אԱა Greetings in various languages: Hello world, Καλημέρα κόσμε, コンニチハ Box drawing alignment tests: █ ▉ ╔══╦══╗ ┌──┬──┐ ╭──┬──╮ ╭──┬──╮ ┏━━┳━━┓ ┎┒┏┑ ╷ ╻ ┏┯┓ ┌┰┐ ▊ ╱╲╱╲╳╳╳ ║┌─╨─┐║ │╔═╧═╗│ │╒═╪═╕│ │╓─╁─╖│ ┃┌─╂─┐┃ ┗╃╄┙ ╶┼╴╺╋╸┠┼┨ ┝╋┥ ▋ ╲╱╲╱╳╳╳ ║│╲ ╱│║ │║ ║│ ││ │ ││ │║ ┃ ║│ ┃│ ╿ │┃ ┍╅╆┓ ╵ ╹ ┗┷┛ └┸┘ ▌ ╱╲╱╲╳╳╳ ╠╡ ╳ ╞╣ ├╢ ╟┤ ├┼─┼─┼┤ ├╫─╂─╫┤ ┣┿╾┼╼┿┫ ┕┛┖┚ ┌┄┄┐ ╎ ┏┅┅┓ ┋ ▍ ╲╱╲╱╳╳╳ ║│╱ ╲│║ │║ ║│ ││ │ ││ │║ ┃ ║│ ┃│ ╽ │┃ ░░▒▒▓▓██ ┊ ┆ ╎ ╏ ┇ ┋ ▎ ║└─╥─┘║ │╚═╤═╝│ │╘═╪═╛│ │╙─╀─╜│ ┃└─╂─┘┃ ░░▒▒▓▓██ ┊ ┆ ╎ ╏ ┇ ┋ ▏ ╚══╩══╝ └──┴──┘ ╰──┴──╯ ╰──┴──╯ ┗━━┻━━┛ ▗▄▖▛▀▜ └╌╌┘ ╎ ┗╍╍┛ ┋ ▁▂▃▄▅▆▇█ ▝▀▘▙▄▟ flexi-streams-1.0.7/util.lisp 0000664 0000000 0000000 00000017657 11175756704 0016205 0 ustar 00root root 0000000 0000000 ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: FLEXI-STREAMS; Base: 10 -*- ;;; $Header: /usr/local/cvsrep/flexi-streams/util.lisp,v 1.24 2008/05/25 21:26:12 edi Exp $ ;;; Copyright (c) 2005-2008, Dr. Edmund Weitz. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package :flexi-streams) #+:lispworks (eval-when (:compile-toplevel :load-toplevel :execute) (import '(lw:with-unique-names lw:when-let))) #-:lispworks (defmacro when-let ((var form) &body body) "Evaluates FORM and binds VAR to the result, then executes BODY if VAR has a true value." `(let ((,var ,form)) (when ,var ,@body))) #-:lispworks (defmacro with-unique-names ((&rest bindings) &body body) "Syntax: WITH-UNIQUE-NAMES ( { var | (var x) }* ) declaration* form* Executes a series of forms with each VAR bound to a fresh, uninterned symbol. The uninterned symbol is as if returned by a call to GENSYM with the string denoted by X - or, if X is not supplied, the string denoted by VAR - as argument. The variable bindings created are lexical unless special declarations are specified. The scopes of the name bindings and declarations do not include the Xs. The forms are evaluated in order, and the values of all but the last are discarded \(that is, the body is an implicit PROGN)." ;; reference implementation posted to comp.lang.lisp as ;; by Vebjorn Ljosa - see also ;; `(let ,(mapcar #'(lambda (binding) (check-type binding (or cons symbol)) (if (consp binding) (destructuring-bind (var x) binding (check-type var symbol) `(,var (gensym ,(etypecase x (symbol (symbol-name x)) (character (string x)) (string x))))) `(,binding (gensym ,(symbol-name binding))))) bindings) ,@body)) #+:lispworks (eval-when (:compile-toplevel :load-toplevel :execute) (setf (macro-function 'with-rebinding) (macro-function 'lw:rebinding))) #-:lispworks (defmacro with-rebinding (bindings &body body) "WITH-REBINDING ( { var | (var prefix) }* ) form* Evaluates a series of forms in the lexical environment that is formed by adding the binding of each VAR to a fresh, uninterned symbol, and the binding of that fresh, uninterned symbol to VAR's original value, i.e., its value in the current lexical environment. The uninterned symbol is created as if by a call to GENSYM with the string denoted by PREFIX - or, if PREFIX is not supplied, the string denoted by VAR - as argument. The forms are evaluated in order, and the values of all but the last are discarded \(that is, the body is an implicit PROGN)." ;; reference implementation posted to comp.lang.lisp as ;; by Vebjorn Ljosa - see also ;; (loop for binding in bindings for var = (if (consp binding) (car binding) binding) for name = (gensym) collect `(,name ,var) into renames collect ``(,,var ,,name) into temps finally (return `(let ,renames (with-unique-names ,bindings `(let (,,@temps) ,,@body)))))) (defun normalize-external-format-name (name) "Converts NAME \(a symbol) to a `canonical' name for an external format, e.g. :LATIN1 will be converted to :ISO-8859-1. Also checks if there is an external format with that name and signals an error otherwise." (let ((real-name (or (cdr (assoc name +name-map+ :test #'eq)) name))) (unless (find real-name +name-map+ :test #'eq :key #'cdr) (error 'external-format-error :format-control "~S is not known to be a name for an external format." :format-arguments (list name))) real-name)) (defun ascii-name-p (name) "Checks whether NAME is the keyword :ASCII." (eq name :us-ascii)) (defun koi8-r-name-p (name) "Checks whether NAME is the keyword :KOI8-R." (eq name :koi8-r)) (defun code-page-name-p (name) "Checks whether NAME is the keyword :CODE-PAGE." (eq name :code-page)) (defun iso-8859-name-p (name) "Checks whether NAME \(a keyword) names one of the known ISO-8859 encodings." (find name +iso-8859-tables+ :key #'car)) (defun known-code-page-id-p (id) "Checks whether ID \(a number) denotes one of the known Windows code pages." (and (find id +code-page-tables+ :key #'car) id)) #+:lispworks (defun sans (plist &rest keys) "Returns PLIST with keyword arguments from KEYS removed." (sys::remove-properties plist keys)) #-:lispworks (defun sans (plist &rest keys) "Returns PLIST with keyword arguments from KEYS removed." ;; stolen from Usenet posting <3247672165664225@naggum.no> by Erik ;; Naggum (let ((sans ())) (loop (let ((tail (nth-value 2 (get-properties plist keys)))) ;; this is how it ends (unless tail (return (nreconc sans plist))) ;; copy all the unmatched keys (loop until (eq plist tail) do (push (pop plist) sans) (push (pop plist) sans)) ;; skip the matched key (setq plist (cddr plist)))))) #+:lispworks (defmacro with-accessors (slot-entries instance &body body) "For LispWorks, we prefer SLOT-VALUE over accessors for better performance." ;; note that we assume that the variables have the same names as the ;; slots `(with-slots ,(mapcar #'car slot-entries) ,instance ,@body)) (defun make-octet-buffer (&optional (size +buffer-size+)) "Creates and returns a fresh buffer \(a specialized array) of size +BUFFER-SIZE+ to hold octets." (declare #.*standard-optimize-settings*) (make-array size :element-type 'octet)) (defun type-equal (type1 type2) "Whether TYPE1 and TYPE2 denote the same type." (declare #.*standard-optimize-settings*) (and (subtypep type1 type2) (subtypep type2 type1))) (defun maybe-rewind (stream octets) "Tries to `rewind' the \(binary) stream STREAM by OCTETS octets. Returns a true value if it succeeds." (when-let (position (file-position stream)) (file-position stream (- position octets)))) (defmacro logand* (x y) "Solely for optimization purposes. Some Lisps need it, some don't." `(the fixnum (logand ,x ,y))) (defmacro logior* (x y) "Solely for optimization purposes. Some Lisps need it, some don't." `(the fixnum (logior ,x ,y))) (defmacro ash* (integer count) "Solely for optimization purposes. Some Lisps need it, some don't." `(the fixnum (ash ,integer ,count)))