pax_global_header00006660000000000000000000000064125331016320014506gustar00rootroot0000000000000052 comment=15b0a2752a05a6bdf16ccdf8a81d63a56993dd4a pycangjie-1.3/000077500000000000000000000000001253310163200133225ustar00rootroot00000000000000pycangjie-1.3/.gitignore000066400000000000000000000007141253310163200153140ustar00rootroot00000000000000# Compiled Object files *.slo *.lo *.o *.pyc *.pyo # Compiled Dynamic libraries *.so # Compiled Static libraries *.lai *.la *.a # Generated source code *.c # Autotools stuff Makefile Makefile.in aclocal.m4 autom4te.cache config.guess config.h config.h.in config.log config.status config.sub configure depcomp install-sh libtool ltmain.sh m4 missing py-compile stamp-h1 test-driver test-suite.log tests/*.log tests/*.trs tests/run_tests .deps .libs AUTHORS pycangjie-1.3/.travis.yml000066400000000000000000000006251253310163200154360ustar00rootroot00000000000000language: c before_install: - sudo apt-get update # First install libcangjie - sudo apt-get install libsqlite3-dev - git clone git://github.com/Cangjians/libcangjie - cd libcangjie && git checkout 1.x && ./autogen.sh --prefix=/usr && make && sudo make install && cd .. - git clean -dfx - sudo apt-get install python3-dev cython script: ./autogen.sh && make && make check && make distcheck pycangjie-1.3/COPYING000066400000000000000000000167431253310163200143700ustar00rootroot00000000000000 GNU LESSER GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. This version of the GNU Lesser General Public License incorporates the terms and conditions of version 3 of the GNU General Public License, supplemented by the additional permissions listed below. 0. Additional Definitions. As used herein, "this License" refers to version 3 of the GNU Lesser General Public License, and the "GNU GPL" refers to version 3 of the GNU General Public License. "The Library" refers to a covered work governed by this License, other than an Application or a Combined Work as defined below. An "Application" is any work that makes use of an interface provided by the Library, but which is not otherwise based on the Library. Defining a subclass of a class defined by the Library is deemed a mode of using an interface provided by the Library. A "Combined Work" is a work produced by combining or linking an Application with the Library. The particular version of the Library with which the Combined Work was made is also called the "Linked Version". The "Minimal Corresponding Source" for a Combined Work means the Corresponding Source for the Combined Work, excluding any source code for portions of the Combined Work that, considered in isolation, are based on the Application, and not on the Linked Version. The "Corresponding Application Code" for a Combined Work means the object code and/or source code for the Application, including any data and utility programs needed for reproducing the Combined Work from the Application, but excluding the System Libraries of the Combined Work. 1. Exception to Section 3 of the GNU GPL. You may convey a covered work under sections 3 and 4 of this License without being bound by section 3 of the GNU GPL. 2. Conveying Modified Versions. If you modify a copy of the Library, and, in your modifications, a facility refers to a function or data to be supplied by an Application that uses the facility (other than as an argument passed when the facility is invoked), then you may convey a copy of the modified version: a) under this License, provided that you make a good faith effort to ensure that, in the event an Application does not supply the function or data, the facility still operates, and performs whatever part of its purpose remains meaningful, or b) under the GNU GPL, with none of the additional permissions of this License applicable to that copy. 3. Object Code Incorporating Material from Library Header Files. The object code form of an Application may incorporate material from a header file that is part of the Library. You may convey such object code under terms of your choice, provided that, if the incorporated material is not limited to numerical parameters, data structure layouts and accessors, or small macros, inline functions and templates (ten or fewer lines in length), you do both of the following: a) Give prominent notice with each copy of the object code that the Library is used in it and that the Library and its use are covered by this License. b) Accompany the object code with a copy of the GNU GPL and this license document. 4. Combined Works. You may convey a Combined Work under terms of your choice that, taken together, effectively do not restrict modification of the portions of the Library contained in the Combined Work and reverse engineering for debugging such modifications, if you also do each of the following: a) Give prominent notice with each copy of the Combined Work that the Library is used in it and that the Library and its use are covered by this License. b) Accompany the Combined Work with a copy of the GNU GPL and this license document. c) For a Combined Work that displays copyright notices during execution, include the copyright notice for the Library among these notices, as well as a reference directing the user to the copies of the GNU GPL and this license document. d) Do one of the following: 0) Convey the Minimal Corresponding Source under the terms of this License, and the Corresponding Application Code in a form suitable for, and under terms that permit, the user to recombine or relink the Application with a modified version of the Linked Version to produce a modified Combined Work, in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source. 1) Use a suitable shared library mechanism for linking with the Library. A suitable mechanism is one that (a) uses at run time a copy of the Library already present on the user's computer system, and (b) will operate properly with a modified version of the Library that is interface-compatible with the Linked Version. e) Provide Installation Information, but only if you would otherwise be required to provide such information under section 6 of the GNU GPL, and only to the extent that such information is necessary to install and execute a modified version of the Combined Work produced by recombining or relinking the Application with a modified version of the Linked Version. (If you use option 4d0, the Installation Information must accompany the Minimal Corresponding Source and Corresponding Application Code. If you use option 4d1, you must provide the Installation Information in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source.) 5. Combined Libraries. You may place library facilities that are a work based on the Library side by side in a single library together with other library facilities that are not Applications and are not covered by this License, and convey such a combined library under terms of your choice, if you do both of the following: a) Accompany the combined library with a copy of the same work based on the Library, uncombined with any other library facilities, conveyed under the terms of this License. b) Give prominent notice with the combined library that part of it is a work based on the Library, and explaining where to find the accompanying uncombined form of the same work. 6. Revised Versions of the GNU Lesser General Public License. The Free Software Foundation may publish revised and/or new versions of the GNU Lesser General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Library as you received it specifies that a certain numbered version of the GNU Lesser General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that published version or of any later version published by the Free Software Foundation. If the Library as you received it does not specify a version number of the GNU Lesser General Public License, you may choose any version of the GNU Lesser General Public License ever published by the Free Software Foundation. If the Library as you received it specifies that a proxy can decide whether future versions of the GNU Lesser General Public License shall apply, that proxy's public statement of acceptance of any version is permanent authorization for you to choose that version for the Library. pycangjie-1.3/INSTALL.md000066400000000000000000000034201253310163200147510ustar00rootroot00000000000000## Arch Linux pycangjie has not been included in official repository yet, but only in AUR. If it works for you and you like it, please don't hesitate to vote it on [upvote it](https://aur.archlinux.org/packages/pycangjie/). So you can install it with `yaourt`: ``` $ yaourt pycangjie ``` ## Debian Unstable / Sid pycangjie is included in the default Debian repositories, so you can just install it with `apt-get`, as the root user: ``` # apt-get install python3-pycangjie ``` ## Fedora pycangjie is included in the default Fedora repositories, so you can just install it with `yum`: ``` $ sudo yum install python3-cangjie ``` ## Ubuntu 14.04 pycangjie is included in the default Ubuntu repositories, **starting with the 14.04 release**, so you can install it with `apt-get`: ``` $ sudo apt-get install python3-pycangjie ``` ## NixOS pycangjie is included in the default NixOS channel, **starting with the 14.02 release**, so you can install it with `nix-env`: ``` $ nix-env --install pycangjie ``` ## Build from the sources ### Dependencies To build these bindings, you will need the following: * Python >= 3.2 * Cython >= 0.17 * a C compiler and library (we recommend GCC and the GNU C library) * the libcangjie library and development headers ### Install from a release tarball Download a release tarball from [the download section](http://cangjians.github.io/downloads/pycangjie/) of our website. From the root folder of the unpacked tarball, do the usual Autotools dance: ``` $ ./configure $ make $ sudo make install ``` ### Install from Git First, you need to clone the development repository: ``` $ git clone git://github.com/Cangjians/pycangjie ``` Then, from the root folder of the Git clone, do the usual Autotools dance: ``` $ ./autogen.sh $ make $ sudo make install ``` pycangjie-1.3/Makefile.am000066400000000000000000000057211253310163200153630ustar00rootroot00000000000000# Point to our macro directory and pick up user flags from the environment ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS} # -- src/ ---------------------------- cangjie_PYTHON = \ src/cangjie/__init__.py \ $(NULL) cangjiedir = $(pkgpyexecdir) pkgpyexec_LTLIBRARIES = \ _core.la \ errors.la \ filters.la \ versions.la \ $(NULL) nodist__core_la_SOURCES = \ src/cangjie/_core.c \ $(NULL) _core_la_CFLAGS = $(LIBCANGJIE_CFLAGS) $(PYTHON_INCLUDE) _core_la_LDFLAGS = -avoid-version -module _core_la_LIBADD = $(LIBCANGJIE_LIBS) nodist_errors_la_SOURCES = \ src/cangjie/errors.c \ $(NULL) errors_la_CFLAGS = $(LIBCANGJIE_CFLAGS) $(PYTHON_INCLUDE) errors_la_LDFLAGS = -avoid-version -module errors_la_LIBADD = $(LIBCANGJIE_LIBS) nodist_filters_la_SOURCES = \ src/cangjie/filters.c \ $(NULL) filters_la_CFLAGS = $(LIBCANGJIE_CFLAGS) $(PYTHON_INCLUDE) filters_la_LDFLAGS = -avoid-version -module filters_la_LIBADD = $(LIBCANGJIE_LIBS) nodist_versions_la_SOURCES = \ src/cangjie/versions.c \ $(NULL) versions_la_CFLAGS = $(LIBCANGJIE_CFLAGS) $(PYTHON_INCLUDE) versions_la_LDFLAGS = -avoid-version -module versions_la_LIBADD = $(LIBCANGJIE_LIBS) pkgpyexecdir = $(pyexecdir)/cangjie src/cangjie/_core.c: src/cangjie/_core.pyx src/cangjie/_core.pxd $(MKDIR_P) src/cangjie $(CYTHON) -3 --verbose -o $@ -I $(srcdir)/src/cangjie $(srcdir)/src/cangjie/_core.pyx src/cangjie/errors.c: src/cangjie/errors.pyx src/cangjie/_core.pxd $(MKDIR_P) src/cangjie $(CYTHON) -3 --verbose -o $@ -I $(srcdir)/src/cangjie $(srcdir)/src/cangjie/errors.pyx src/cangjie/filters.c: src/cangjie/filters.pyx src/cangjie/_core.pxd $(MKDIR_P) src/cangjie $(CYTHON) -3 --verbose -o $@ -I $(srcdir)/src/cangjie $(srcdir)/src/cangjie/filters.pyx src/cangjie/versions.c: src/cangjie/versions.pyx src/cangjie/_core.pxd $(MKDIR_P) src/cangjie $(CYTHON) -3 --verbose -o $@ -I $(srcdir)/src/cangjie $(srcdir)/src/cangjie/versions.pyx # -- Testing ------------------------- TESTS = tests/run_tests tests/run_tests: tests/run_tests.in $(MKDIR_P) tests sed -e 's&@PYTHON_BIN@&$(PYTHON)&g' \ -e 's&@SRCDIR@&$(srcdir)&g' \ -e 's&@BUILDDIR@&$(builddir)&g' $< > $@ chmod +x $@ # -- Common -------------------------- AUTHORS: @if test -d "$(srcdir)/.git"; then \ echo Creating $@ && \ ( cd "$(top_srcdir)" && \ echo -e '# Generated by Makefile. Do not edit.\n#'; \ echo -e '# pycangjie was written by these people:\n'; \ git log --no-merges --pretty=format:"%an <%ae>" \ | sort | uniq ) > $@.tmp && mv -f $@.tmp $@ \ || ( rm -f $@.tmp ; echo Failed to generate $@ >&2 ); \ fi CLEANFILES = \ src/cangjie/*.c \ tests/run_tests \ $(NULL) clean-local: -rm -rf tests/__pycache__ EXTRA_DIST = \ autogen.sh \ README.md \ $(wildcard docs/*.md) \ src/cangjie/_core.pyx \ src/cangjie/_core.pxd \ src/cangjie/errors.pyx \ src/cangjie/filters.pyx \ src/cangjie/versions.pyx \ tests/run_tests.in \ tests/__init__.py \ $(wildcard $(srcdir)/tests/test_*.py) \ $(NULL) .PHONY: AUTHORS pycangjie-1.3/README.md000066400000000000000000000020021253310163200145730ustar00rootroot00000000000000This is a Python wrapper to libcangjie, the library implementing the Cangjie input method. Below is a trivial example of how to use it: ```python import cangjie cj = cangjie.Cangjie(cangjie.versions.CANGJIE3, cangjie.filters.BIG5 | cangjie.filters.HKSCS) for c in cj.get_characters("a*y"): print(c) ``` For more details, refer to the documentation, either [online](http://cangjians.github.io/projects/pycangjie/documentation) or the one shipped with this software. Development happens [on github](https://github.com/Cangjians/pycangjie), and stable release tarballs can be found in [the download section](http://cangjians.github.io/downloads/pycangjie/) of our website. ## Legalities pycangjie is offered under the terms of the [GNU Lesser General Public License, either version 3 or any later version](http://www.gnu.org/licenses/lgpl.html). We won't ask you to sign a copyright assignment or any other kind of silly and tedious legal document, so just send us patches and/or pull requests! pycangjie-1.3/autogen.sh000077500000000000000000000002061253310163200153210ustar00rootroot00000000000000#!/bin/sh set -e set -x libtoolize --automake --copy aclocal -I m4 autoheader automake --add-missing --copy autoconf ./configure $* pycangjie-1.3/configure.ac000066400000000000000000000030131253310163200156050ustar00rootroot00000000000000AC_PREREQ(2.63) AC_INIT([pycangjie], [1.3], [https://github.com/Cangjians/pycangjie/issues], [cangjie], [https://github.com/Cangjians/pycangjie]) AC_CONFIG_HEADERS([config.h]) AC_CONFIG_MACRO_DIR([m4]) AM_INIT_AUTOMAKE([1.11 no-dist-gzip dist-xz tar-ustar foreign]) AC_PROG_MKDIR_P m4_ifdef([AM_SILENT_RULES],[AM_SILENT_RULES([yes])]) # Initialize libtool LT_INIT([disable-static]) # Check for libcangjie PKG_CHECK_MODULES(LIBCANGJIE, [cangjie >= 1.0]) # Check for Python 3 AM_PATH_PYTHON([3.2.3]) AC_PATH_PROGS([PYTHON_CONFIG], [python$PYTHON_VERSION-config python-config], [no], [`dirname $PYTHON`]) AS_IF([test "$PYTHON_CONFIG" = no], [AC_MSG_ERROR([cannot find python-config for $PYTHON.])]) AC_MSG_CHECKING([python include flags]) PYTHON_INCLUDE=`$PYTHON_CONFIG --includes` AC_MSG_RESULT([$PYTHON_INCLUDE]) AC_SUBST(PYTHON_INCLUDE) # Check for Cython presence AC_PATH_PROGS([CYTHON], [cython3 cython], [no]) AS_IF([test "$CYTHON" = no], [AC_MSG_ERROR([cannot find cython.])]) # Check for Cython version CYTHON_MIN_VERSION=0.14 AC_MSG_CHECKING(for cython version) CYTHON_VERSION=`$CYTHON --version 2>&1 | cut -d" " -f3` AC_MSG_RESULT($CYTHON_VERSION) $PYTHON -c "from distutils.version import StrictVersion as ver; import sys; sys.exit(0 if ver(\"$CYTHON_VERSION\") >= ver(\"$CYTHON_MIN_VERSION\") else 1)" AS_IF([test $? = 1], [AC_MSG_ERROR([Please use cython >= $CYTHON_MIN_VERSION])]) AC_SUBST(CYTHON) # OUTPUT files AC_CONFIG_FILES([ Makefile ]) AC_OUTPUT pycangjie-1.3/docs/000077500000000000000000000000001253310163200142525ustar00rootroot00000000000000pycangjie-1.3/docs/cangjie.md000066400000000000000000000066111253310163200162000ustar00rootroot00000000000000--- layout: default title: pycangjie documentation - The Cangjie class name: projects project: pycangjie sub: doc --- ## The Cangjie class Now this is really the interesting part: it is the actual entry point to the library. For everything worth doing with pycangjie, you will want to get a hold on a `Cangjie` object. It has 2 attributes: * `version` is an `int` representing the version (see versions.md) of the Cangjie input method you are interested in. * `filter_flags` is an `int` corresponding to how you want to filter the output (see filters.md) of the `get_characters()` query method. #### First step: get a Cangjie object Given that everything comes out of it, you **must** create a `Cangjie` object before you do anything else, for example: ```python import cangjie cj = cangjie.Cangjie(cangjie.versions.CANGJIE3, cangjie.filters.BIG5 | cangjie.filters.HKSCS) ``` The first parameter is one of the supported versions (see versions.md), and the third is a bitwise OR of the available output filters (see filters.md). These will have an impact on the `get_characters()` query method. Note that in the example above, given the output filters that have been used, the only characters which would be returned are the ones which are part of the Big 5 encoding and the Hong Kong Supplemental Character Set. Output filters thus control the set of characters which will be returned, not the ones which are filtered out. #### Querying for characters There are three query methods. First, and most importantly, `get_characters()` returns a list of characters corresponding to a given Cangjie code. This is really what is intended by a Cangjie input method: the user types an input code, and expects one or more characters in return. Here is an example of usage: ```python # cj is a Cangjie object chars = cj.get_characters("abcd") ``` After execution of the above, `chars` will be a `list` of one or more `CangjieChar` instances (see cangjiechar.md) corresponding to the "abcd" code in the Cangjie version passed to the constructor of the `Cangjie` class. Wildcards ("*") are supported inside a code. For example, the "d*d" code could be used. It is often desired to show the user the radical corresponding to the characters they input, rather than (or along with) the latin letters they actually type. You can do that with the `get_radical()` method, used as follows: ```python # cj is a Cangjie object radical = cj.get_radical("d"); ``` After execution, `radical` contains the radical corresponding to the "d" code. Finally, users (especially Quick users) often want a convenient way to input punctuation and symbols without knowing their Cangjie code. As such, we provide the `get_characters_by_shortcode()` method, which is used in exactly the same way as the `get_characters()` one: ```python # cj is a Cangjie object chars = cj.get_characters_by_shortcode(","); ``` After execution, `chars` would contain a `CangjieChar` instance for each fullwidth / Chinese comma. #### Validating user input In your application, you might want to check whether the user input are valid keys of Cangjie codes (to pass them to the `get_characters()` method), or not (for example, to pass other characters to the `get_characters_by_shortcode()` method). You can do that with the `is_input_key()` method: ```python # cj is a Cangjie object cj.is_input_key("a"); # This is True cj.is_input_key("@"); # This is False ``` pycangjie-1.3/docs/cangjiechar.md000066400000000000000000000022771253310163200170420ustar00rootroot00000000000000--- layout: default title: pycangjie documentation - The CangjieChar class name: projects project: pycangjie sub: doc --- ## The CangjieChar class This is the basic currency unit in pycangjie. `CangjieChar`'s are what the methods return when you want to find what character(s) corresponds to a certain Cangjie code. The `CangjieChar` simply has 3 attributes: * `chchar` is an `str` containing the character that the user actually wants * `code` is an `str` containing the Cangjie code for that character * `frequency is an `int` representing a useful measure to order the characters when presenting them to the user. The higher it is, the more frequent that character is. For example, in Cangjie, the character (`chchar`) "木" has the `code` "d". Also, a character with a `frequency` of 1000 should be presented before a character with a frequency of 500. `CangjieChar`'s are really something that your application will consume, not something you will create and manage. Nevertheless, if you do want to create one, the `CangjieChar` constructor takes 3 arguments, which are the 3 attributes above, in the order we described them: ```python import cangjie c = cangjie.CangjieChar("木", "d", 123) ``` pycangjie-1.3/docs/errors.md000066400000000000000000000014351253310163200161130ustar00rootroot00000000000000--- layout: default title: pycangjie documentation - Exceptions name: projects project: pycangjie sub: doc --- ## Exceptions pycangjie defines a set of exceptions mapping to the error codes returned by the libcangjie functions. Functions which are supposed to get you a list of characters will raise `CangjieNoCharsError` when no characters correspond to your query. If the libcangjie database could not be opened for some reason, then `CangjieDBOpenError` is raised. If an error happens when querying the database, the method will raise `CangjieDBError`. When memory could not be allocated, the method will raise `CangjieMemoryError`. Finally, methods will raise `CangjieInvalidInputError` when they are passed an invalid input. All exceptions are defined in the `cangjie.errors` module. pycangjie-1.3/docs/filters.md000066400000000000000000000024031253310163200162430ustar00rootroot00000000000000--- layout: default title: pycangjie documentation - Output filtering name: projects project: pycangjie sub: doc --- ## Output filtering Often when using pycangjie you will get lists of characters (for example, which characters correspond to the code "d*d" ?). It is possible to filter these lists so you only get the characters you are interested in. The following filters are defined in the `cangjie.filters` module: * Use `BIG5` if you are interested in the characters which are part of the Big 5 charset. * Use `HKSCS` if you want the characters which are part of the Hong Kong Supplemental Character Set. * Use `PUNCTUATION` to get the punctuation characters. * Use `CHINESE` if you want all Chinese characters. * Use `ZHUYIN` in order to get the characters part of Taiwan's Zhuyin/Bopomofo alphabet. * Use `KANJI` for the Japanese Kanji characters. * Use `KATAKANA` and you'll get the Japanese Katakana characters. * Use `HIRAGANA` if the Japanese Hiragana matter to you. * Finally, use `SYMBOLS` to get access to lots of miscellaneous symbols. Note that these are bit flags. If you want more than one, just OR them together. For example, if you want the Big 5 and HKSCS characters, use: ```python import cangjie cangjie.filters.BIG5 | cangjie.filters.HKSCS ``` pycangjie-1.3/docs/index.md000066400000000000000000000012061253310163200157020ustar00rootroot00000000000000--- layout: default title: pycangjie documentation name: projects project: pycangjie sub: doc --- pycangjie is a very simple library: * just import the `cangjie` module * everything happens by instanciating a `Cangjie` object, and then calling its various methods. These pages will document proper use of pycangjie. Although the interesting part is the last of the list, we recommend you read them in order: 1. the possible exceptions (see errors.md) 2. the supported Cangjie versions (see versions.md) 3. the available output filters (see filters.md) 4. the `CangjieChar` class (see cangjiechar.md) 6. the `Cangjie` class (see cangjie.md) pycangjie-1.3/docs/versions.md000066400000000000000000000005521253310163200164460ustar00rootroot00000000000000--- layout: default title: pycangjie documentation - Cangjie versions name: projects project: pycangjie sub: doc --- ## The Cangjie versions libcangjie supports **2 different versions of the Cangjie input method**: versions 3 and 5. Obviously, so does pycangjie. These are defined in the `cangjie.versions` module, as `CANGJIE3` and `CANGJIE5` respectively. pycangjie-1.3/src/000077500000000000000000000000001253310163200141115ustar00rootroot00000000000000pycangjie-1.3/src/cangjie/000077500000000000000000000000001253310163200155115ustar00rootroot00000000000000pycangjie-1.3/src/cangjie/__init__.py000066400000000000000000000017511253310163200176260ustar00rootroot00000000000000# Copyright (c) 2012-2013 - The pycangjie authors # # This file is part of pycangjie, the Python bindings to libcangjie. # # pycangjie is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # pycangjie is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with pycangjie. If not, see . # The stuff in core should be exposed directly here, but it seems to be # impossible to have a __init__.so file from ._core import * # Native extentions don't get exposed automatically from . import filters from . import versions pycangjie-1.3/src/cangjie/_core.pxd000066400000000000000000000053571253310163200173270ustar00rootroot00000000000000# Copyright (c) 2012-2013 - The pycangjie authors # # This file is part of pycangjie, the Python bindings to libcangjie. # # pycangjie is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # pycangjie is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with pycangjie. If not, see . from libc.stdint cimport uint32_t cdef extern from "cangjie.h": cdef enum CangjieVersion: CANGJIE_VERSION_3 CANGJIE_VERSION_5 cdef enum CangjieFilter: CANGJIE_FILTER_BIG5 CANGJIE_FILTER_HKSCS CANGJIE_FILTER_PUNCTUATION CANGJIE_FILTER_CHINESE CANGJIE_FILTER_ZHUYIN CANGJIE_FILTER_KANJI CANGJIE_FILTER_KATAKANA CANGJIE_FILTER_HIRAGANA CANGJIE_FILTER_SYMBOLS cdef int CANGJIE_OK cdef int CANGJIE_NOCHARS cdef int CANGJIE_DBOPEN cdef int CANGJIE_DBERROR cdef int CANGJIE_NOMEM cdef int CANGJIE_INVALID cdef struct CCangjieChar "CangjieChar": char chchar[5] char code[6] uint32_t frequency int cangjie_char_new(CCangjieChar **c, char *chchar, char *code, uint32_t frequency) int cangjie_char_free(CCangjieChar *c) cdef struct CCangjieCharList "CangjieCharList": CCangjieChar *c CCangjieCharList *prev CCangjieCharList *next int cangjie_char_list_free(CCangjieCharList *l) cdef struct CCangjie "Cangjie": uint32_t version uint32_t filter_flags int cangjie_new(CCangjie **cj, CangjieVersion version, CangjieFilter filter_flags) int cangjie_get_characters(CCangjie *cj, char *code, CCangjieCharList **l) int cangjie_get_characters_by_shortcode(CCangjie *cj, char *code, CCangjieCharList **l) int cangjie_get_radical(CCangjie *cj, char key, char **radical) int cangjie_is_input_key(CCangjie *cj, char key) int cangjie_free(CCangjie *cj) pycangjie-1.3/src/cangjie/_core.pyx000066400000000000000000000134711253310163200173500ustar00rootroot00000000000000# Copyright (c) 2012-2013 - The pycangjie authors # # This file is part of pycangjie, the Python bindings to libcangjie. # # pycangjie is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # pycangjie is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with pycangjie. If not, see . cimport _core from .errors import CangjieError, CangjieInvalidInputError, handle_error_code cdef class CangjieChar: cdef _core.CCangjieChar *cobj def __cinit__(self, char *chchar, char *code, uint32_t frequency): ret = _core.cangjie_char_new(&self.cobj, chchar, code, frequency) handle_error_code(ret, msg="An unknown error happened while " "initializing a CangjieChar object (error " "code %d)" % ret) @property def chchar(self): return self.cobj.chchar.decode("utf-8") @property def code(self): return self.cobj.code.decode("utf-8") @property def frequency(self): return self.cobj.frequency def __dealloc__(self): if self.cobj is not NULL: _core.cangjie_char_free(self.cobj) def __str__(self): return ("" % (self.chchar, self.code, self.frequency)) def __richcmp__(self, other, op): equality = (self.chchar == other.chchar and self.code == other.code and self.frequency == other.frequency) if op == 2: return equality if op == 3: return not equality raise NotImplementedError("Only (in)equality is implemented") cdef class CangjieCharList: cdef _core.CCangjieCharList *cobj def __cinit__(self): self.cobj = NULL def __iter__(self): cdef _core.CCangjieCharList *iter_ = self.cobj if self.cobj == NULL: raise StopIteration() while True: c = iter_.c yield CangjieChar(c.chchar, c.code, c.frequency) if iter_.next == NULL: break iter_ = iter_.next def __dealloc__(self): if self.cobj is not NULL: _core.cangjie_char_list_free(self.cobj) cdef class Cangjie: cdef _core.CCangjie *cobj def __cinit__(self, _core.CangjieVersion version, _core.CangjieFilter filter_flags): """Constructor for the Cangjie class The `version` parameter must be one of the available constants in the `cangjie.versions` module. The `filter_flags` parameter must be a bitwise-AND of one or more of the available constants in the `cangjie.filters` module. """ ret = _core.cangjie_new(&self.cobj, version, filter_flags) handle_error_code(ret, msg="An unknown error happened while " "initializing a Cangjie object (error code" " %d)" % ret) @property def filter_flags(self): return self.cobj.filter_flags @property def version(self): return self.cobj.version def get_characters(self, str code): l = CangjieCharList() b_code = code.encode("utf-8") ret = _core.cangjie_get_characters(self.cobj, b_code, &l.cobj) handle_error_code(ret, msg="An unknown error happened while trying to" " get the characters for code '%s' (error " "code %d)" % (code, ret)) return list(l) def get_characters_by_shortcode(self, str code): l = CangjieCharList() b_code = code.encode("utf-8") ret = _core.cangjie_get_characters_by_shortcode(self.cobj, b_code, &l.cobj) handle_error_code(ret, msg="An unknown error happened while trying to" " get the characters for short code '%s' " "(error code %d)" % (code, ret)) return list(l) def get_radical(self, str key): b_key = key.encode("utf-8") if len(b_key) > 1: raise CangjieInvalidInputError() # A char is in fact an integer in C b_key = ord(b_key) cdef char *radical ret = _core.cangjie_get_radical(self.cobj, b_key, &radical) handle_error_code(ret, msg="An unknown error happened while trying to" " get the radical for code '%s' (error " "code %d)" % (key, ret)) return (radical).decode("utf-8") def is_input_key(self, str key): b_key = key.encode("utf-8") if len(b_key) > 1: raise CangjieInvalidInputError() # A char is in fact an integer in C b_key = ord(b_key) ret = _core.cangjie_is_input_key(self.cobj, b_key) if ret not in (_core.CANGJIE_OK, _core.CANGJIE_INVALID): raise CangjieError("An unknown error happened while checking " "whether '%s' is a valid input code (error " "code %d)" % ret) return ret == _core.CANGJIE_OK def __dealloc__(self): if self.cobj is not NULL: _core.cangjie_free(self.cobj) pycangjie-1.3/src/cangjie/errors.pyx000066400000000000000000000035171253310163200175750ustar00rootroot00000000000000# Copyright (c) 2013 - The pycangjie authors # # This file is part of pycangjie, the Python bindings to libcangjie. # # pycangjie is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # pycangjie is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with pycangjie. If not, see . cimport _core class CangjieError(Exception): def __init__(self, *args): if args: super(CangjieError, self).__init__(*args) else: super(CangjieError, self).__init__(self.message) class CangjieNoCharsError(CangjieError): message = "No characters found" class CangjieDBOpenError(CangjieError): message = "Error opening the database" class CangjieDBError(CangjieError): message = "Error querying the database" class CangjieMemoryError(CangjieError): message = "Not enough memory" class CangjieInvalidInputError(CangjieError): message = "Invalid input" def handle_error_code(int code, msg=None): if code == _core.CANGJIE_NOCHARS: raise CangjieNoCharsError() if code == _core.CANGJIE_DBOPEN: raise CangjieDBOpenError() if code == _core.CANGJIE_DBERROR: raise CangjieDBError() if code == _core.CANGJIE_NOMEM: raise CangjieMemoryError() if code == _core.CANGJIE_INVALID: raise CangjieInvalidInputError() if code != _core.CANGJIE_OK: raise CangjieError(msg) pycangjie-1.3/src/cangjie/filters.pyx000066400000000000000000000022441253310163200177250ustar00rootroot00000000000000# Copyright (c) 2012-2013 - The pycangjie authors # # This file is part of pycangjie, the Python bindings to libcangjie. # # pycangjie is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # pycangjie is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with pycangjie. If not, see . cimport _core # Make the filter flags available in the Python module BIG5 = _core.CANGJIE_FILTER_BIG5 HKSCS = _core.CANGJIE_FILTER_HKSCS PUNCTUATION = _core.CANGJIE_FILTER_PUNCTUATION CHINESE = _core.CANGJIE_FILTER_CHINESE ZHUYIN = _core.CANGJIE_FILTER_ZHUYIN KANJI = _core.CANGJIE_FILTER_KANJI KATAKANA = _core.CANGJIE_FILTER_KATAKANA HIRAGANA = _core.CANGJIE_FILTER_HIRAGANA SYMBOLS = _core.CANGJIE_FILTER_SYMBOLS pycangjie-1.3/src/cangjie/versions.pyx000066400000000000000000000016241253310163200201260ustar00rootroot00000000000000# Copyright (c) 2012-2013 - The pycangjie authors # # This file is part of pycangjie, the Python bindings to libcangjie. # # pycangjie is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # pycangjie is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with pycangjie. If not, see . cimport _core # Make the version constants available in the Python module CANGJIE3 = _core.CANGJIE_VERSION_3 CANGJIE5 = _core.CANGJIE_VERSION_5 pycangjie-1.3/tests/000077500000000000000000000000001253310163200144645ustar00rootroot00000000000000pycangjie-1.3/tests/__init__.py000066400000000000000000000141111253310163200165730ustar00rootroot00000000000000# Copyright (c) 2013 - The pycangjie authors # # This file is part of pycangjie, the Python bindings to libcangjie. # # pycangjie is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # pycangjie is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with pycangjie. If not, see . import itertools import operator import string import subprocess import unittest import cangjie class MetaTest(type): """Metaclass for our test cases The goal is to provide every TestCase class with methods like test_a(), test_b(), etc..., in other words, one method per potential Cangjie input code. Well, not quite, because that would be 12356630 methods (the number of strings composed of 1 to 5 lowercase ascii letters), and even though my laptop has 8Go of RAM, the test process gets killed by the OOM killer. :) So we cheat, and use libcangjie's wildcard support, so that we only generate 26 + 26^2 = 702 methods. """ def __init__(cls, name, bases, dct): super(MetaTest, cls).__init__(name, bases, dct) def gen_codes(): """Generate the 702 possible input codes""" # First, the 1-character codes for c in string.ascii_lowercase: yield c # Next, the 2-characters-with-wildcard codes for t in itertools.product(string.ascii_lowercase, repeat=2): yield '*'.join(t) def tester(code): def func(cls): return cls.run_test(code) return func # Generate the test_* methods for code in gen_codes(): setattr(cls, "test_%s" % code.replace("*", ""), tester(code)) class BaseTestCase(unittest.TestCase): """Base test class, grouping the common stuff for all our unit tests""" def __init__(self, name): super().__init__(name) self.cli_cmd = ["/usr/bin/libcangjie_cli"] + self.cli_options self.language = (cangjie.filters.BIG5 | cangjie.filters.HKSCS | cangjie.filters.PUNCTUATION | cangjie.filters.CHINESE | cangjie.filters.ZHUYIN | cangjie.filters.KANJI | cangjie.filters.KATAKANA | cangjie.filters.HIRAGANA | cangjie.filters.SYMBOLS) def setUp(self): self.cj = cangjie.Cangjie(self.version, self.language) def tearDown(self): del self.cj def run_command(self, cmd): """Run a command, deal with errors, and return its stdout""" proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) out, err = proc.communicate() try: cangjie.errors.handle_error_code(proc.returncode, msg="Unknown error while running" " libcangjie_cli (%d)" % proc.returncode) except cangjie.errors.CangjieNoCharsError: return "" try: return out.decode("utf-8") except UnicodeDecodeError: # Python's 'utf-8' codec trips over b"\xed\xa1\x9d\xed\xbc\xb2", # but according to [1] and [2], it is a valid sequence of 2 chars: # U+D85D \xed\xa1\x9d # U+DF32 \xed\xbc\xb2 # [1] http://www.utf8-chartable.de/unicode-utf8-table.pl?start=55389&utf8=string-literal # [2] http://www.utf8-chartable.de/unicode-utf8-table.pl?start=57138&utf8=string-literal # TODO: Investigate this further, and eventually open a bug report out2 = [] for line in out.split("\n".encode("utf-8")): try: out2.append(line.decode("utf-8")) except UnicodeDecodeError: pass return "\n".join(out2) def run_test(self, input_code): """Run the actual test This compares the output of the libcangjie_cli tool with the output from pycangjie. The idea is that if pycangjie produces the same results as a C++ tool compiled against libcangjie, then pycangjie properly wraps libcangjie. We do not try to verify that pycangjie produces valid results here, validity is to be checked in libcangjie. Note that this whole test is based on scraping the output of libcangjie_cli, which is quite fragile. """ # Get a list of CangjieChar from libcangjie_cli as a reference tmp_expected = self.run_command(self.cli_cmd+[input_code]).split("\n") tmp_expected = map(lambda x: x.strip(" \n"), tmp_expected) tmp_expected = filter(lambda x: len(x) > 0, tmp_expected) expected = [] for item in tmp_expected: chchar, code, frequency = item.split(", ") chchar = chchar.split(" ")[-1] code = code.split(" ")[-1].strip("'") frequency = int(frequency.split(" ")[-1]) expected.append(cangjie._core.CangjieChar(chchar.encode("utf-8"), code.encode("utf-8"), frequency)) expected = sorted(expected, key=operator.attrgetter('chchar', 'code')) try: # And compare with what pycangjie produces results = sorted(self.cj.get_characters(input_code), key=operator.attrgetter('chchar', 'code')) self.assertEqual(results, expected) except cangjie.errors.CangjieNoCharsError: self.assertEqual(len(expected), 0) pycangjie-1.3/tests/run_tests.in000066400000000000000000000033371253310163200170500ustar00rootroot00000000000000#!@PYTHON_BIN@ # Copyright (c) 2013 - The pycangjie authors # # This file is part of pycangjie, the Python bindings to libcangjie. # # pycangjie is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # pycangjie is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with pycangjie. If not, see . import glob import os import shutil import sys import unittest # -- Make sure we test the development tree ---------------------------------- dst = os.path.join("@BUILDDIR@", "built_cangjie") if os.path.exists(dst): shutil.rmtree(dst) os.mkdir(dst) for f in glob.glob(os.path.join("@SRCDIR@", "src", "cangjie", "*.py")): shutil.copy2(f, dst) for f in glob.glob(os.path.join("@BUILDDIR@", ".libs", "*.so")): shutil.copy2(f, dst) sys.path.insert(0, "@BUILDDIR@") import built_cangjie sys.modules["cangjie"] = built_cangjie # -- Load and run our unit tests --------------------------------------------- sys.path.insert(1, "@SRCDIR@") loader = unittest.TestLoader() suite = loader.discover("@SRCDIR@/tests") runner = unittest.TextTestRunner(verbosity=2) result = runner.run(suite) # -- Clean up behind ourselves ----------------------------------------------- if os.path.exists(dst): shutil.rmtree(dst) if result.failures or result.errors: sys.exit(1) pycangjie-1.3/tests/test_cangjie.py000066400000000000000000000025101253310163200174730ustar00rootroot00000000000000# Copyright (c) 2013 - The pycangjie authors # # This file is part of pycangjie, the Python bindings to libcangjie. # # pycangjie is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # pycangjie is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with pycangjie. If not, see . import unittest import cangjie from tests import BaseTestCase, MetaTest class VersionThreeTestCase(BaseTestCase, metaclass=MetaTest): cli_options = ["--filter=big5,hkscs,punctuation,chinese,zhuyin,kanji,katakana,hiragana,symbols", "--mode=code", "--cj-version=3"] version = cangjie.versions.CANGJIE3 class VersionFiveTestCase(BaseTestCase, metaclass=MetaTest): cli_options = ["--filter=big5,hkscs,punctuation,chinese,zhuyin,kanji,katakana,hiragana,symbols", "--mode=code", "--cj-version=5"] version = cangjie.versions.CANGJIE5