pax_global_header 0000666 0000000 0000000 00000000064 13067513116 0014515 g ustar 00root root 0000000 0000000 52 comment=d9ef6840a30c784fef1c7ed2c8f8c5302d5204e3
AppRecommender-0.7.5/ 0000775 0000000 0000000 00000000000 13067513116 0014427 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/.gitignore 0000664 0000000 0000000 00000000503 13067513116 0016415 0 ustar 00root root 0000000 0000000 *.pyc
TODO
doc/doxygen-1.7.3
doc/html/
popcon-data.tgz
popcon-data/
psql-out
udd-import.log
udd.sql.gz
*.log
.vagrant/
*.swp
apprecommender.egg-info/
apprecommender/tests/test_data/.sample_axi/
apprecommender/tests/test_data/.sample_axi/
apprecommender/tests/.apt_run/
apprecommender/web/submissions/
.travis.yml.container
AppRecommender-0.7.5/.gitlab-ci.yml 0000664 0000000 0000000 00000000410 13067513116 0017056 0 ustar 00root root 0000000 0000000 before_script:
- ./install_dependencies.sh
- sudo pip2 install flake8
- sudo /usr/bin/python2 -m apprecommender.main.cli --init
- sudo /usr/bin/python2 -m apprecommender.main.cli --train
test:
script:
- flake8 .
- /usr/bin/python2 setup.py test
AppRecommender-0.7.5/99app-recommender 0000664 0000000 0000000 00000000254 13067513116 0017613 0 ustar 00root root 0000000 0000000 DPkg::Post-Invoke {"[ ! -x /usr/bin/apprec-apt ] || apprec-apt --post-invoke";};
DPkg::Pre-Install-Pkgs {"[ ! -x /usr/bin/apprec-apt ] || apprec-apt --pre-install-pkgs";};
AppRecommender-0.7.5/GPLv3.txt 0000664 0000000 0000000 00000104514 13067513116 0016070 0 ustar 00root root 0000000 0000000 GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc.
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
Copyright (C)
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
Copyright (C)
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
.
AppRecommender-0.7.5/README 0000664 0000000 0000000 00000003175 13067513116 0015315 0 ustar 00root root 0000000 0000000 AppRecommender - Application recommender for GNU/Linux systems
---------------------------------------------------------------
Install dependencies
---------------------
# apt-get install python python-xapian python-apt python-cluster python-simplejson python-numpy apt-xapian-index python-xdg debtags python-pip python-sklearn python-matplotlib python-fuzzywuzzy -y
sudo update-apt-xapian-index
pip install setuptools
Run AppRecommender in Terminal
------------------------------
$ cd ./bin
$ ./apprec.py -s cb
Run "$ ./apprec.py -h" to view the recommender strategies
Prepare AppRecommender data
---------------------------
Run the following commands:
$ ./install_dependencies.sh
$ cd ./bin
$ ./apprec.py --init
Run Machine Learning Training
----------------------------
$ cd ./bin
$ ./apprec.py --train
License
-------
Copyright 2011 Tassia Cames Araujo
Copyright 2015-2016 Lucas Albuquerque Medeiros de Moura
Copyright 2015-2016 Luciano Prestes Cavalcanti
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
AppRecommender-0.7.5/README.md 0000664 0000000 0000000 00000002417 13067513116 0015712 0 ustar 00root root 0000000 0000000 AppRecommender [](https://gitlab.com/AppRecommender/AppRecommender/builds)
===============================================================
Application recommender for GNU/Linux systems
---------------------------------------------------------------
Install dependencies
---------------------
$ apt-get install python python-xapian python-apt python-cluster python-webpy python-simplejson python-numpy apt-xapian-index python-xdg debtags python-pip python-sklearn python-matplotlib python-stemmer -y
$ sudo update-apt-xapian-index
$ pip install setuptools
Run AppRecommender web UI
--------------------------
$ cd ./src/web
$ ./server.py
Open a browser and access http://localhost:8080
More info at https://github.com/tassia/AppRecommender/wiki
Run AppRecommender in Terminal
------------------------------
$ cd ./bin
$ ./apprec.py -s cb
Run "$ ./apprec.py -h" to view the recommender strategies
Prepare AppRecommender data
---------------------------
Run the following commands:
$ ./install_dependencies.sh
$ cd ./bin
$ ./apprec.py --init
Run Machine Learning Training
----------------------------
$ cd ./bin
$ ./apprec.py --train
AppRecommender-0.7.5/Vagrantfile 0000664 0000000 0000000 00000000571 13067513116 0016617 0 ustar 00root root 0000000 0000000 # -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
config.vm.box = "debian/jessie64"
config.vm.box_check_update = false
config.vm.network :forwarded_port, host: 8080, guest: 8080
config.vm.provision :shell, path: "vagrant/bootstrap.sh", privileged: false
config.vm.provider "virtualbox" do |vm|
vm.memory = 1024
vm.cpus = 2
end
end
AppRecommender-0.7.5/app_recommender.cfg 0000664 0000000 0000000 00000002354 13067513116 0020254 0 ustar 00root root 0000000 0000000 # Config file for AppRecommender
[general]
# logging level
debug = 0
verbose = 0
[data_sources]
# base_dir = /var/lib/apprecommender/
base_dir = ~/.app-recommender/
user_data_dir = user_data/
output = apprec.log
# filters for valid packages
filters_dir = filters
pkgs_filter = desktopapps
# package information indexes
axi = /var/lib/apt-xapian-index/index
axi_programs = axi_programs
axi_desktopapps = axi_desktopapps
# old, reindex, cluster, recluster
#index_mode = old
# popcon indexes
# check if there are popcon indexes available
popcon = 0
popcon_programs = popcon_programs
popcon_desktopapps = popcon_desktopapps
popcon_index = popcon_desktopapps
popcon_dir = popcon-entries
# number of popcon submission for indexing
max_popcon = 100000000
# popcon clustering
clusters_dir = clusters_dir_full
k_medoids = 100
# Connection to DDE
dde_url = http://46.4.235.200:8000/q/udd/packages/all/%s?t=json
self.dde_server = 46.4.235.200
self.dde_port = 8000
[recommender]
# search weighting scheme ('trad' or 'bm25')
weight = bm25
# bm25 parameters
bm25_k1 = 1.2
bm25_k2 = 0
bm25_k3 = 7
bm25_b = 0.75
bm25_nl = 0.5
# recommendation strategy
strategy = cb
# user content profile size
profile_size = 50
# neighborhood size
k_neighbors = 50
popcon_profiling = full
AppRecommender-0.7.5/apprecommender/ 0000775 0000000 0000000 00000000000 13067513116 0017430 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/apprecommender/Makefile 0000664 0000000 0000000 00000000077 13067513116 0021074 0 ustar 00root root 0000000 0000000 doc:
./generate_doc.sh
clean :
@find . -name \*.pyc -delete
AppRecommender-0.7.5/apprecommender/__init__.py 0000664 0000000 0000000 00000000166 13067513116 0021544 0 ustar 00root root 0000000 0000000 # \mainpage Application recommender for GNU/Linux systems
# Main repository: http://github.com/tassia/AppRecommender
AppRecommender-0.7.5/apprecommender/apt_cache.py 0000664 0000000 0000000 00000001344 13067513116 0021713 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import apt
import xapian
class AptCache:
DEFAULT_AXI_PATH = "/var/lib/apt-xapian-index/index"
def __init__(self):
self.axi = xapian.Database(AptCache.DEFAULT_AXI_PATH)
self.cache = apt.Cache()
def __getitem__(self, pkg_name):
return self.get(pkg_name)
def __contains__(self, pkg_name):
return self.xapian_has_pkg(pkg_name) and pkg_name in self.cache
def get(self, pkg_name):
if self.xapian_has_pkg(pkg_name):
return self.cache[pkg_name]
raise KeyError("The cache has no package named '{}'".format(pkg_name))
def xapian_has_pkg(self, pkg_name):
term = 'XP' + pkg_name
return self.axi.get_termfreq(term) > 0L
AppRecommender-0.7.5/apprecommender/config.py 0000664 0000000 0000000 00000022605 13067513116 0021254 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
config - python module for configuration options.
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import sys
import os
import logging
import logging.handlers
from ConfigParser import ConfigParser, MissingSectionHeaderError
from apprecommender.singleton import Singleton
class Config(Singleton):
"""
Class to handle configuration options.
"""
def __init__(self):
"""
Set default configuration options.
"""
try:
self.config_parser = ConfigParser()
self.config_parser.read(
['/etc/apprecommender/recommender.conf',
os.path.expanduser('~/.app_recommender.rc'),
os.path.expanduser('app_recommender.cfg')])
self.home_dir = os.path.expanduser('~/.apprecommender')
except (MissingSectionHeaderError), err:
logging.error("Error in config file syntax: %s", str(err))
os.abort()
if not hasattr(self, 'initialized'):
# data_source options
self.base_dir = os.environ.get(
'APPREC_DATA', '/var/lib/apprecommender')
self.base_dir = os.path.expanduser(self.base_dir)
self.output = os.path.join(self.home_dir, 'apprec.log')
self.user_data_dir = os.path.join(self.base_dir, "user_data/")
# general options
self.debug = 0
self.verbose = 0
# filters for valid packages
self.filters_dir = os.path.join(self.base_dir, "filters")
self.pkgs_filter = os.path.join(self.filters_dir, "desktopapps")
# package information packages
self.axi = "/var/lib/apt-xapian-index/index"
self.axi_programs = os.path.join(self.base_dir, "axi_programs")
self.axi_desktopapps = os.path.join(self.base_dir,
"axi_desktopapps")
# popcon indexes
self.index_mode = "old"
# check if there are popcon indexes available
self.popcon = 0
self.popcon_programs = os.path.join(self.base_dir,
"popcon_programs")
self.popcon_desktopapps = os.path.join(self.base_dir,
"popcon_desktopapps")
self.popcon_index = self.popcon_desktopapps
self.popcon_dir = os.path.join(self.base_dir, "popcon-entries")
self.max_popcon = 1000
# popcon clustering
self.clusters_dir = os.path.join(self.base_dir, "clusters-dir")
self.k_medoids = 100
# self.dde_url = "http://dde.debian.net/dde/" \
# "q/udd/packs/all/%s?t=json"
self.dde_url = "http://46.4.235.200:8000/" \
"q/udd/packages/prio-debian-sid/%s?t=json"
self.dde_server = "46.4.235.200"
self.dde_port = 8000
# recomender options
self.strategy = "cb"
self.weight = "bm25"
self.bm25_k1 = 1.2
self.bm25_k2 = 0
self.bm25_k3 = 7
self.bm25_b = 0.75
self.bm25_nl = 0.5
# user content profile size
self.profile_size = 10
self.num_recommendations = 8
self.because = False
# neighborhood size
self.k_neighbors = 50
# popcon profiling method: full, voted
self.popcon_profiling = "full"
self.load_config_file()
self.initialized = 1
logging.info("Basic config")
def read_option(self, section, option):
"""
Read option from configuration file if it is defined there or return
default value.
"""
var = "self.%s" % option
if self.config_parser.has_option(section, option):
return self.config_parser.get(section, option)
else:
return eval(var)
def load_config_file(self):
"""
Load options from configuration file and command line arguments.
"""
self.debug = int(self.read_option('general', 'debug'))
self.debug = int(self.read_option('general', 'verbose'))
self.base_dir = os.environ.get(
'APPREC_DATA', '/var/lib/apprecommender')
self.base_dir = os.path.expanduser(self.base_dir)
self.output = os.path.join(
self.home_dir, self.read_option('data_sources',
'output'))
self.user_data_dir = os.path.join(
self.base_dir, self.read_option('data_sources',
'user_data_dir'))
self.filters_dir = os.path.join(
self.base_dir, self.read_option('data_sources',
'filters_dir'))
self.pkgs_filter = os.path.join(
self.filters_dir, self.read_option('data_sources',
'pkgs_filter'))
self.axi = self.read_option('data_sources', 'axi')
self.axi_programs = os.path.join(
self.base_dir, self.read_option('data_sources',
'axi_programs'))
self.axi_desktopapps = os.path.join(
self.base_dir, self.read_option('data_sources',
'axi_desktopapps'))
# self.index_mode = self.read_option('data_sources', 'index_mode')
self.popcon = int(self.read_option('data_sources', 'popcon'))
self.popcon_programs = os.path.join(
self.base_dir, self.read_option('data_sources',
'popcon_programs'))
self.popcon_desktopapps = os.path.join(
self.base_dir, self.read_option('data_sources',
'popcon_desktopapps'))
self.popcon_index = os.path.join(
self.base_dir, self.read_option('data_sources',
'popcon_index'))
self.popcon_dir = os.path.join(
self.base_dir, self.read_option('data_sources',
'popcon_dir'))
self.max_popcon = int(self.read_option('data_sources', 'max_popcon'))
self.clusters_dir = os.path.join(
self.base_dir, self.read_option('data_sources',
'clusters_dir'))
self.k_medoids = int(self.read_option('data_sources', 'k_medoids'))
self.dde_url = self.read_option('data_sources', 'dde_url')
self.dde_server = self.read_option('data_sources', 'dde_server')
self.dde_port = self.read_option('data_sources', 'dde_port')
self.weight = self.read_option('recommender', 'weight')
self.bm25_k1 = float(self.read_option('recommender', 'bm25_k1'))
self.bm25_k2 = float(self.read_option('recommender', 'bm25_k2'))
self.bm25_k3 = float(self.read_option('recommender', 'bm25_k3'))
self.bm25_b = float(self.read_option('recommender', 'bm25_b'))
self.bm25_nl = float(self.read_option('recommender', 'bm25_nl'))
self.strategy = self.read_option('recommender', 'strategy')
self.profile_size = int(
self.read_option('recommender', 'profile_size'))
self.k_neighbors = int(
self.read_option('recommender', 'k_neighbors'))
self.popcon_profiling = self.read_option(
'recommender', 'popcon_profiling')
def set_logger(self):
"""
Configure application logger and log level.
"""
self.logger = logging.getLogger('') # root logger is used by default
self.logger.setLevel(logging.WARNING)
if self.debug == 1:
log_level = logging.DEBUG
elif self.verbose == 1:
log_level = logging.INFO
else:
log_level = logging.WARNING
console_handler = logging.StreamHandler(sys.stdout)
console_handler.setFormatter(logging.Formatter(
'%(levelname)s: %(message)s'))
console_handler.setLevel(log_level)
self.logger.addHandler(console_handler)
if not os.path.exists(self.home_dir):
os.makedirs(self.home_dir)
file_handler = logging.handlers.RotatingFileHandler(self.output,
maxBytes=50000000,
backupCount=5)
log_format = '%(asctime)s %(levelname)-8s %(message)s'
file_handler.setFormatter(logging.Formatter(
log_format, datefmt='%Y-%m-%d %H:%M:%S'))
file_handler.setLevel(log_level)
self.logger.addHandler(file_handler)
logging.info("Set up logger")
AppRecommender-0.7.5/apprecommender/data.py 0000664 0000000 0000000 00000024671 13067513116 0020725 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
data - python module for data sources classes and methods.
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import xapian
import logging
import random
import operator
import math
import commands
from apprecommender.utils import print_progress_bar
from apprecommender.data_classification import time_weight
from apprecommender.error import Error
def axi_get_pkgs(axi):
pkgs_names = []
for docid in range(1, axi.get_lastdocid() + 1):
try:
doc = axi.get_document(docid)
except:
pass
docterms_XP = [t.term for t in doc.termlist()
if t.term.startswith("XP")]
pkgs_names.append(docterms_XP[0].lstrip('XP'))
return pkgs_names
def axi_search_pkgs(axi, pkgs_list):
terms = ["XP" + item for item in pkgs_list]
query = xapian.Query(xapian.Query.OP_OR, terms)
enquire = xapian.Enquire(axi)
enquire.set_query(query)
mset = enquire.get_mset(0, axi.get_doccount())
return mset
def axi_search_pkg_tags(axi, pkg):
enquire = xapian.Enquire(axi)
enquire.set_query(xapian.Query("XP" + pkg))
matches = enquire.get_mset(0, 1)
if not matches:
logging.debug("Package %s not found in items repository" % pkg)
return False
for m in matches:
tags = [term.term for term in axi.get_document(m.docid).termlist() if
term.term.startswith("XT")]
if not tags:
return "notags"
else:
return tags
def print_index(index):
output = "\n---\n" + xapian.Database.__repr__(index) + "\n---\n"
for term in index.allterms():
output += term.term + "\n"
output += str([index.get_document(posting.docid).get_data()
for posting in index.postlist(term.term)])
output += "\n---"
return output
def get_user_installed_pkgs():
dpkg_output = commands.getoutput('/usr/bin/dpkg --get-selections')
packages = [pkg.split('\t')[0] for pkg in dpkg_output.splitlines()
if 'deinstall' not in pkg.split('\t')[-1]]
packages = set(packages)
return packages
def get_all_terms(index, docs, content_filter, normalized_weights):
# Store all terms in one single document
terms_packages = {}
terms_doc = xapian.Document()
for d in docs:
package = d.document.get_data()
for term in index.get_document(d.docid).termlist():
if content_filter(term.term):
if normalized_weights:
terms_doc.add_term(term.term,
int(math.ceil(
normalized_weights[d.docid])))
else:
terms_doc.add_term(term.term)
if term.term.startswith('XP'):
continue
elif term.term in terms_packages:
terms_packages[term.term].append(package)
else:
terms_packages[term.term] = [package]
return (terms_doc, terms_packages)
def get_tfidf_terms_weights(terms_doc, index, terms_package, time_context=0):
# Compute sublinear tfidf for each term
weights = {}
for term in terms_doc.termlist():
try:
# Even if it shouldn't raise error...
# math.log: ValueError: math domain error
tf = 1 + math.log(term.wdf)
idf = math.log(index.get_doccount() /
float(index.get_termfreq(term.term)))
tfidf = tf * idf
weights[term.term] = tfidf
if time_context:
weight = time_weight(term.term,
terms_package[term.term])
weights[term.term] *= weight
except:
pass
# print_best_weight_terms(terms_package)
return weights
def tfidf_weighting(index, docs, content_filter, normalized_weights=0,
time_context=0):
"""
Return a dictionary of terms and weights of all terms of a set of
documents, based on the frequency of terms in the selected set (docids).
"""
terms_doc, terms_packages = get_all_terms(index, docs, content_filter,
normalized_weights)
weights = get_tfidf_terms_weights(terms_doc, index, terms_packages,
time_context)
sorted_weights = list(reversed(sorted(weights.items(),
key=operator.itemgetter(1))))
return sorted_weights
def tfidf_plus(index, docs, content_filter, time_context=0):
"""
Return a dictionary of terms and weights of all terms of a set of
documents, based on the frequency of terms in the selected set (docids).
"""
normalized_weigths = {}
population = [d.weight for d in docs]
mean = sum(population) / len(population)
variance = sum([(p - mean) * (p - mean)
for p in population]) / len(population)
standard_deviation = math.sqrt(variance)
for d in docs:
if standard_deviation > 1:
# values between [0-1] would cause the opposite effect
normalized_weigths[d.docid] = d.weight / standard_deviation
else:
normalized_weigths[d.docid] = d.weight
return tfidf_weighting(index, docs, content_filter, normalized_weigths,
time_context)
def split_pkg_data(user_pkg, partition_size):
round_partition = {}
for i in range(partition_size):
if len(user_pkg) > 0:
random_key = random.choice(user_pkg.keys())
else:
logging.critical("Empty user_pkg.")
raise Error
round_partition[random_key] = user_pkg.pop(random_key)
return round_partition
class FilteredXapianIndex(xapian.WritableDatabase):
"""
Filtered Xapian Index
"""
def __init__(self, terms, index_path, path):
xapian.WritableDatabase.__init__(self, path,
xapian.DB_CREATE_OR_OVERWRITE)
index = xapian.Database(index_path)
for docid in range(1, index.get_lastdocid() + 1):
try:
doc = index.get_document(docid)
docterms = [term.term for term in doc.termlist()]
tagged = False
for t in terms:
if t in docterms:
tagged = True
if tagged:
self.add_document(doc)
logging.info("Added doc %d." % docid)
else:
logging.info("Discarded doc %d." % docid)
except:
logging.info("Doc %d not found in axi." % docid)
logging.info("Filter: %s" % terms)
logging.info("Index size: %d" % index.get_doccount())
logging.info("Filtered Index size: %d (lastdocid: %d)." %
(self.get_doccount(), self.get_lastdocid()))
def __str__(self):
return print_index(self)
class SampleAptXapianIndex(xapian.WritableDatabase):
"""
Sample data source for packages information, generated from a list of
packages.
"""
def __init__(self, pkgs_list, axi, path):
xapian.WritableDatabase.__init__(self, path,
xapian.DB_CREATE_OR_OVERWRITE)
self.sample = axi_search_pkgs(axi, pkgs_list)
len_sample = len(self.sample)
for index, package in enumerate(self.sample):
self.doc_id = self.add_document(axi.get_document(package.docid))
print_progress_bar(index + 1, len_sample)
def __str__(self):
return print_index(self)
class PopconSubmission():
def __init__(self, path, user_id=0, binary=1):
self.packages = dict()
self.path = path
self.binary = binary
self.load()
if user_id:
self.user_id = user_id
def __str__(self):
output = "\nPopularity-contest submission ID " + self.user_id
for pkg, weight in self.packages.items():
output += "\n " + pkg + ": " + str(weight)
return output
def get_filtered(self, filter_list):
filtered = {}
for pkg in self.packages.keys():
if pkg in filter_list:
filtered[pkg] = self.packages[pkg]
return filtered
def load(self, binary=1):
"""
Parse a popcon submission, generating the names of the valid packages
in the vote.
"""
with open(self.path) as submission:
for line in submission:
if line.startswith("POPULARITY"):
self.user_id = line.split()[2].lstrip("ID:")
self.arch = line.split()[3].lstrip("ARCH:")
elif not line.startswith("END-POPULARITY"):
data = line.rstrip('\n').split()
if len(data) > 2:
pkg = data[2]
if len(data) > 3:
exec_file = data[3]
# Binary weight
if self.binary:
self.packages[pkg] = 1
# Weights inherited from Enrico's anapop
# No executable files to track
elif exec_file == '':
self.packages[pkg] = 1
# Recently used packages
elif len(data) == 4:
self.packages[pkg] = 10
# Unused packages
elif data[4] == '':
self.packages[pkg] = 3
# Recently installed packages
elif data[4] == '':
self.packages[pkg] = 8
AppRecommender-0.7.5/apprecommender/data_classification.py 0000664 0000000 0000000 00000007051 13067513116 0023771 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import commands
import calendar
import logging
import math
import operator
import time
pkgs_times = {}
pkgs_time_weight = {}
best_weight_terms = {}
user_tfidf_weights = {}
def pkg_name_with_error(pkg):
return len(pkg.split()) > 1
def get_time_from_package(pkg, pkg_bin=True):
if pkg_name_with_error(pkg):
pkgs_times[pkg] = [None, None]
if pkg in pkgs_times:
modify, access = pkgs_times[pkg]
else:
modify = get_time('Y', pkg, pkg_bin)
access = get_time('X', pkg, pkg_bin)
pkgs_times[pkg] = [modify, access]
return pkgs_times[pkg]
def get_alternative_pkg(pkg):
dpkg_command = "dpkg -L {0}| grep /usr/bin/"
dpkg_command += " || dpkg -L {0}| grep /usr/sbin/"
pkg_bin = commands.getoutput(dpkg_command.format(pkg))
possible_pkgs = {}
for pkg_path in pkg_bin.splitlines():
possible_pkgs[pkg_path] = get_time('X', pkg_path)
if bool(possible_pkgs):
return sorted(possible_pkgs.items(), key=operator.itemgetter(1))[0][0]
return None
def get_time(option, pkg, pkg_bin=True):
stat_base = "stat -c '%{option}'"
stat_base += " `which {package}`" if pkg_bin else " {package}"
stat_error = 'stat:'
stat_time = stat_base.format(option=option, package=pkg)
pkg_time = commands.getoutput(stat_time)
if stat_error not in pkg_time:
return pkg_time
return None
def linear_percent_function(modify, access, time_now):
modify, access = int(modify), int(access)
time_access = access - modify
time_actual = time_now - modify
percent = float(time_access) / float(time_actual)
return percent
def get_pkg_time_weight(pkg):
modify, access = get_time_from_package(pkg)
if not modify and not access:
modify, access = get_time_from_package(get_alternative_pkg(pkg))
if not modify and not access:
return 0
time_now = calendar.timegm(time.gmtime())
return linear_percent_function(modify, access, time_now)
def calculate_time_curve(pkg_time_weight):
if not pkg_time_weight:
return 0
const_a = 10
lambda_value = 1
return const_a * (1 / math.exp((1 - pkg_time_weight) * lambda_value))
def time_weight(term, term_list):
weight = []
weight_len = 5
weight_delta = 0.2
for pkg in term_list:
if pkg in pkgs_time_weight:
weight.append(pkgs_time_weight[pkg])
else:
pkg_time_weight = get_pkg_time_weight(pkg)
pkgs_time_weight[pkg] = pkg_time_weight
weight.append(calculate_time_curve(pkg_time_weight))
weight = list(reversed(sorted(weight)))
if len(weight) < weight_len:
for i in range(len(weight), weight_len):
weight.append(weight[-1] - weight_delta)
time_weight = float(sum(weight[0:weight_len])) / float(weight_len)
best_weight_terms[term] = time_weight
return time_weight
def print_best_weight_terms(terms_package):
index = 0
total = 0
logging.info("BEST TERMS")
for term in sorted(best_weight_terms, key=best_weight_terms.get,
reverse=True):
if index < 10:
logging.info("\n")
logging.info(term, best_weight_terms[term])
logging.info('-')
for pkg in terms_package[term]:
logging.info("[{0}: {1} {2}]\n".format(pkg,
get_pkg_time_weight(pkg),
get_alternative_pkg(pkg)))
total += 1
if total > 5:
break
total = 0
index += 1
AppRecommender-0.7.5/apprecommender/decider.py 0000664 0000000 0000000 00000020047 13067513116 0021404 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import commands
import re
import xapian
from apprecommender.apt_cache import AptCache
from sklearn.feature_extraction.stop_words import ENGLISH_STOP_WORDS
class PkgInitDecider():
"""
Class used to decide if a package can be considered to recommended to an
user or not.
"""
INVALID_PREFIXES = {'ruby', 'python', 'python3', 'golang', 'gir',
'texlive'}
INVALID_SUFFIXES = {'examples', 'dbg', 'data', 'dev', 'utils', 'common',
'fonts', 'png', 'core', 'default'}
def __init__(self):
self.cache = AptCache()
self.user_role_programs = self.get_user_role_programs()
def is_in_apt_cache(self, pkg):
return pkg in self.cache
def get_package_dependencies(self, pkg):
return [dep[0].name for dep in pkg.dependencies]
def get_user_installed_packages(self):
manual_installed = commands.getoutput('apt-mark showmanual')
return manual_installed.splitlines()
def get_user_role_programs(self):
user_pkgs = self.get_user_installed_packages()
user_programs = set()
for pkg in user_pkgs:
if pkg in self.cache:
pkg_candidate = self.cache[pkg].candidate
pkg_tags = pkg_candidate.record.get('Tag', None)
if not pkg_tags:
continue
if 'role::program' in pkg_tags:
user_programs.add(pkg)
return user_programs
def is_section_doc(self, pkg_section):
return pkg_section == 'doc'
def is_valid_dependency(self, pkg_tags, pkg_section):
tags_dep = 'role::program' in pkg_tags or 'devel::editor' in pkg_tags
section_dep = pkg_section == 'interpreters'
return tags_dep or section_dep
def is_program_dependencies_installed(self, pkg):
pkg_dependencies = self.get_package_dependencies(pkg)
dep_programs = set()
for dep in pkg_dependencies:
if dep in self.cache:
pkg = self.cache[dep].candidate
if pkg is not None:
pkg_tags = pkg.record.get('Tag', None)
pkg_section = pkg.section
if pkg_tags is None:
continue
is_valid_dependency = self.is_valid_dependency(
pkg_tags, pkg_section)
if is_valid_dependency:
dep_programs.add(dep)
return len(dep_programs - self.user_role_programs) == 0
def is_pkg_a_prefix_or_suffix(self, pkg):
splited_pkg = pkg.split('-')
if len(splited_pkg) == 1:
return False
pkg_prefix = splited_pkg[0]
for prefix in PkgInitDecider.INVALID_PREFIXES:
if pkg_prefix.startswith(prefix):
return True
for suffix in PkgInitDecider.INVALID_SUFFIXES:
if (splited_pkg[-1].endswith(suffix) or
splited_pkg[-2].endswith(suffix)):
return True
return False
def __call__(self, pkg):
if not self.is_in_apt_cache(pkg):
return False
pkg_candidate = self.cache[pkg].candidate
if not pkg_candidate:
return False
if not self.is_program_dependencies_installed(pkg_candidate):
return False
if self.is_pkg_a_prefix_or_suffix(pkg):
return False
if self.is_section_doc(pkg_candidate.section):
return False
return True
class PkgMatchDecider(xapian.MatchDecider):
"""
Extend xapian.MatchDecider to not consider installed packages.
"""
def __init__(self, pkgs_list):
"""
Set initial parameters.
"""
xapian.MatchDecider.__init__(self)
self.pkgs_list = pkgs_list
def __call__(self, xapian_document):
"""
True if the package is not already installed and is not a lib or
a xapian_document.
"""
pkg = xapian_document.get_data()
is_new = pkg not in self.pkgs_list
is_new = is_new and ':' not in pkg
if "kde" in pkg:
return is_new and "kde" in self.pkgs_list
if "gnome" in pkg:
return is_new and "gnome" in self.pkgs_list
if re.match(r'^lib.*', pkg) or re.match(r'.*doc$', pkg):
return False
return is_new
class PkgExpandDecider(xapian.ExpandDecider):
"""
Extend xapian.ExpandDecider to consider packages only.
"""
def __init__(self, pkgs_list):
"""
Set initial parameters.
"""
xapian.ExpandDecider.__init__(self)
self.pkgs_list = pkgs_list
def __call__(self, term):
"""
True if the term is a package.
"""
pkg = term.lstrip("XP")
is_new_pkg = pkg not in self.pkgs_list and term.startswith("XP")
if "kde" in pkg:
return is_new_pkg and "kde" in self.pkgs_list
if "gnome" in pkg:
return is_new_pkg and "gnome" in self.pkgs_list
return is_new_pkg
class PkgReverseDependeciesDecider(xapian.MatchDecider):
"""
Extend xapian.MatchDecider to consider only packages on valid list
"""
def __init__(self, reverse_dependencies, user_installed_pkgs):
"""
Set initial parameters.
"""
xapian.MatchDecider.__init__(self)
self.reverse_dependencies = reverse_dependencies
self.pkg_init_decider = PkgInitDecider()
self.pkg_match_decider = PkgMatchDecider(user_installed_pkgs)
self.cache = AptCache()
def __call__(self, xapian_document):
"""
True if the package is on pkg_list
"""
pkg = xapian_document.get_data()
if pkg not in self.reverse_dependencies:
return False
if pkg not in self.cache:
return False
if self.cache[pkg].section == 'doc':
return False
decider = self.pkg_init_decider
pkg_candidate = self.cache[pkg].candidate
if not decider.is_program_dependencies_installed(pkg_candidate):
return False
return self.pkg_match_decider(xapian_document)
class TagExpandDecider(xapian.ExpandDecider):
"""
Extend xapian.ExpandDecider to consider tags only.
"""
def __call__(self, term):
"""
True if the term is a package tag.
"""
return term.startswith("XT")
class FilterTag(xapian.ExpandDecider):
"""
Extend xapian.ExpandDecider to consider only tag terms.
"""
def __init__(self, valid_tags):
"""
Set initial parameters.
"""
xapian.ExpandDecider.__init__(self)
self.valid_tags = valid_tags
def __call__(self, term):
"""
Return true if the term is a tag, else false.
"""
if self.valid_tags:
is_valid = term.lstrip("XT") in self.valid_tags
else:
is_valid = 1
return term.startswith("XT") and is_valid
class FilterDescription(xapian.ExpandDecider):
"""
Extend xapian.ExpandDecider to consider only package description terms.
"""
def __init__(self):
xapian.ExpandDecider.__init__(self)
self.stop_words = ENGLISH_STOP_WORDS
def __call__(self, term):
"""
Return true if the term or its stemmed version is part of a package
description.
"""
if term not in self.stop_words:
return term.islower() and re.search('[a-z]', term)
return False
class FilterTag_or_Description(xapian.ExpandDecider):
"""
Extend xapian.ExpandDecider to consider only package description terms.
"""
def __init__(self, valid_tags):
"""
Set initial parameters.
"""
xapian.ExpandDecider.__init__(self)
self.valid_tags = valid_tags
def __call__(self, term):
"""
Return true if the term or its stemmed version is part of a package
description.
"""
is_tag = FilterTag(self.valid_tags)(term)
is_description = FilterDescription()(term)
return is_tag or is_description
AppRecommender-0.7.5/apprecommender/error.py 0000664 0000000 0000000 00000001666 13067513116 0021144 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
error.py - python module for error definition.
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
class Error(Exception):
"""
Base class for exceptions.
"""
pass
AppRecommender-0.7.5/apprecommender/evaluation.py 0000664 0000000 0000000 00000040462 13067513116 0022157 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
evaluation - python module for classes and methods related to recommenders
evaluation.
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import math
import logging
from abc import ABCMeta, abstractmethod
from collections import defaultdict
from apprecommender.error import Error
from apprecommender.singleton import Singleton
from apprecommender.recommender import RecommendationResult
from apprecommender.user import User
from apprecommender.data import split_pkg_data
class Metric(Singleton):
"""
Base class for metrics. Strategy design pattern.
"""
def get_errors(self, evaluation):
"""
Compute prediction errors.
"""
keys = evaluation.predicted_item_scores.keys()
keys.extend(evaluation.real_item_scores.keys())
errors = []
for k in keys:
if k not in evaluation.real_item_scores:
evaluation.real_item_scores[k] = 0.0
if k not in evaluation.predicted_item_scores:
evaluation.predicted_item_scores[k] = 0.0
errors.append(float(evaluation.predicted_item_scores[k] -
evaluation.real_item_scores[k]))
return errors
class SimpleAccuracy(Metric):
"""
Classification accuracy metric which consider classes sizes.
"""
def __init__(self):
"""
Set metric description.
"""
self.desc = " S_Accuracy "
def run(self, evaluation):
"""
Compute metric.
"""
simple_accurary = float((evaluation.repository_size -
evaluation.false_positive_len) -
evaluation.false_negative_len)
return simple_accurary / evaluation.repository_size
class Accuracy(Metric):
"""
Classification accuracy metric which consider classes sizes.
"""
def __init__(self):
"""
Set metric description.
"""
self.desc = " Accuracy "
def run(self, evaluation):
"""
Compute metric.
"""
error_1 = (float(evaluation.false_positive_len) /
(evaluation.repository_size -
evaluation.real_relevant_len))
error_2 = (float(evaluation.false_negative_len) /
evaluation.real_relevant_len)
accuracy = 1 - (float(error_1 + error_2) / 2)
return accuracy
class Precision(Metric):
"""
Classification accuracy metric defined as the percentage of relevant itens
among the predicted ones.
"""
def __init__(self):
"""
Set metric description.
"""
self.desc = " Precision "
def run(self, evaluation):
"""
Compute metric.
"""
precision = float(evaluation.true_positive_len)
if not precision:
return 0.0
return precision / float(evaluation.predicted_relevant_len)
class Recall(Metric):
"""
Classification ccuracy metric defined as the percentage of relevant itens
which were predicted as so.
"""
def __init__(self):
"""
Set metric description.
"""
self.desc = " Recall "
def run(self, evaluation):
"""
Compute metric.
"""
recall = float(evaluation.true_positive_len)
if not recall:
return 0.0
return recall / evaluation.real_relevant_len
class FPR(Metric):
"""
False positive rate (used for ploting ROC curve).
"""
def __init__(self):
"""
Set metric description.
"""
self.desc = " FPR "
def run(self, evaluation):
"""
Compute metric.
"""
if not evaluation.false_positive_len:
return 0.0
return (float(evaluation.false_positive_len) /
evaluation.real_negative_len)
class MCC(Metric):
"""
Matthews correlation coefficient.
"""
def __init__(self):
"""
Set metric description.
"""
self.desc = " MCC "
def run(self, evaluation):
"""
Compute metric.
"""
VP = evaluation.true_positive_len
FP = evaluation.false_positive_len
FN = evaluation.false_negative_len
VN = evaluation.true_negative_len
if ((VP + FP) == 0 or (VP + FN) == 0 or
(VN + FP) == 0 or (VN + FN) == 0):
return 0
MCC = (((VP * VN) - (FP * FN)) /
math.sqrt((VP + FP) * (VP + FN) * (VN + FP) * (VN + FN)))
return MCC
class F_score(Metric):
"""
Classification accuracy metric which correlates precision and
recall into an unique measure.
"""
def __init__(self, k):
"""
Set metric description.
"""
self.desc = " F(%.1f) " % k
self.k = k
def run(self, evaluation):
"""
Compute metric.
"""
p = Precision().run(evaluation)
r = Recall().run(evaluation)
if ((self.k * self.k * p) + r) > 0:
return float(((1 + (self.k * self.k)) * ((p * r) /
((self.k * self.k * p) + r))))
else:
return 0
class MAE(Metric):
"""
Prediction accuracy metric defined as the mean absolute error.
"""
def __init__(self):
"""
Set metric description.
"""
self.desc = " MAE "
def run(self, evaluation):
"""
Compute metric.
"""
errors = self.get_errors(evaluation)
return sum(errors) / len(errors)
class MSE(Metric):
"""
Prediction accuracy metric defined as the mean square error.
"""
def __init__(self):
"""
Set metric description.
"""
self.desc = " MSE "
def run(self, evaluation):
"""
Compute metric.
"""
errors = self.get_errors(evaluation)
square_errors = [pow(x, 2) for x in errors]
return sum(square_errors) / len(square_errors)
class RMSE(MSE):
"""
Prediction accuracy metric defined as the root mean square error.
"""
def __init__(self):
"""
Set metric description.
"""
self.desc = " RMSE "
def run(self, evaluation):
"""
Compute metric.
"""
return math.sqrt(MSE.run(evaluation))
class Coverage(Metric):
"""
Evaluation metric defined as the percentage of itens covered by the
recommender (have been recommended at least once).
"""
def __init__(self):
"""
Set initial parameters.
"""
self.desc = " Coverage "
def run(self, evaluations_set):
"""
Compute metric.
"""
covered = set()
for evaluation in evaluations_set:
covered.update(set(evaluation.predicted_relevant))
return float(len(covered)) / evaluation.repository_size
class Evaluation:
"""
Class designed to perform prediction evaluation, given data and metric.
"""
def __init__(self, predicted, real, repository_size):
"""
Set initial parameters.
"""
self.repository_size = repository_size
self.predicted_item_scores = predicted.item_score
self.predicted_relevant = predicted.get_prediction()
self.predicted_relevant_len = len(self.predicted_relevant)
self.real_item_scores = real.item_score
self.real_relevant = real.get_prediction()
self.real_relevant_len = len(self.real_relevant)
self.true_positive = [v[0] for v in self.predicted_relevant if v[0] in
[w[0] for w in self.real_relevant]]
self.false_positive = [v[0] for v in self.predicted_relevant
if not v[0] in [w[0]
for w in self.real_relevant]]
self.false_negative = [v[0] for v in self.real_relevant if not v[0] in
[w[0] for w in self.predicted_relevant]]
self.true_positive_len = len(self.true_positive)
self.false_positive_len = len(self.false_positive)
self.false_negative_len = len(self.false_negative)
self.real_negative_len = self.repository_size - self.real_relevant_len
self.true_negative_len = (self.real_negative_len -
len(self.false_positive))
logging.debug("TP: %d" % len(self.true_positive))
logging.debug("FP: %d" % len(self.false_positive))
logging.debug("FN: %d" % len(self.false_negative))
logging.debug("TN: %d" % self.true_negative_len)
logging.debug("Repo_size: %d" % self.repository_size)
logging.debug("Relevant: %d" % len(self.real_relevant))
logging.debug("Irrelevant: %d" % self.real_negative_len)
def run(self, metric):
"""
Perform the evaluation with the given metric.
"""
return metric.run(self)
class CrossValidation:
__metaclass__ = ABCMeta
"""
Class designed to perform cross-validation process.
:param partition_proportion: A value that should dictates the number of
data that will be on the traning data, the
rest of data will be used as the validation
set.
:param rounds: The number of rounds that cross validation
will be used on.
:param rec: The recommendation strategy used.
:param metrics_list: Array of the metrics that will be used to
evaluate the algorithm.
:param result_proportion: The percentage of recommendations that should
be used based on the total number of packages
available.
"""
def __init__(self, partition_proportion, rounds, rec,
metrics_list, result_proportion):
"""
Set initial parameters.
"""
if partition_proportion < 1 and partition_proportion > 0:
self.partition_proportion = partition_proportion
else:
logging.critical("Partition proportion must be a value"
"in the interval [0,1].")
raise Error
self.rounds = rounds
self.recommender = rec
self.metrics_list = metrics_list
self.cross_results = defaultdict(list)
self.result_proportion = result_proportion
@abstractmethod
def get_evaluation(self, predicted_result, real_result):
raise NotImplementedError("Method not implemented.")
@abstractmethod
def get_model(self):
raise NotImplementedError("Method not implemented.")
def get_result_size(self):
result_size = (self.recommender.items_repository.get_doccount() *
self.result_proportion)
result_size = int(result_size)
logging.debug("size %d" % result_size)
return result_size
@abstractmethod
def get_real_results(self, round_partition):
raise NotImplementedError("Method not implemented.")
def get_partition_size(self, cross_item_score):
return int(len(cross_item_score) * self.partition_proportion)
@abstractmethod
def get_predicted_results(self, round_user, round_partition, result_size):
raise NotImplementedError("Method not implemented.")
@abstractmethod
def get_user_score(self, user):
raise NotImplementedError("Method not implemented.")
def reset_cross_item_score(self, cross_item_score, round_partition):
while len(round_partition) > 0:
item, score = round_partition.popitem()
cross_item_score[item] = score
return cross_item_score
def run_metrics(self, predicted_result, real_result):
logging.debug("Predicted result: %s", predicted_result)
evaluation = self.get_evaluation(predicted_result, real_result)
for metric in self.metrics_list:
result = evaluation.run(metric)
self.cross_results[metric.desc].append(result)
def run(self, user):
"""
Perform cross-validation.
"""
'''
A dictionary containing all the usefull user packages.
Its key is a package name, and its value is the input vector
associated with that package.
'''
cross_item_score = self.get_user_score(user)
# The amount of data that will be used to train the algorithm
partition_size = self.get_partition_size(cross_item_score)
# main iteration
for r in range(self.rounds):
round_partition = split_pkg_data(cross_item_score, partition_size)
logging.debug("Round partition: %s", str(round_partition))
logging.debug("Cross item-score: %s", str(cross_item_score))
# The algorithm model created with the selected training data.
round_model = self.get_model(round_partition)
result_size = self.get_result_size()
if not result_size:
logging.critical("Recommendation size is zero.")
raise Error
predicted_result = self.get_predicted_results(
round_model, cross_item_score, result_size)
if not predicted_result.size:
logging.critical("No recommendation produced"
" Abort cross-validation.")
raise Error
# partition is considered the expected result
real_result = self.get_real_results(cross_item_score)
self.run_metrics(predicted_result, real_result)
# moving back items from round_partition to cross_item_score
cross_item_score = self.reset_cross_item_score(cross_item_score,
round_partition)
class CrossValidationRecommender(CrossValidation):
def get_evaluation(self, predicted_result, real_result):
num_docs = self.recommender.items_repository.get_doccount()
return Evaluation(predicted_result, real_result, num_docs)
def get_model(self, cross_item_score):
return User(cross_item_score)
def get_pkg_score(self, user):
cross_item_score = {}
for pkg in user.pkg_profile:
cross_item_score[pkg] = user.item_score[pkg]
return cross_item_score
def get_real_results(self, round_partition):
return RecommendationResult(round_partition)
def get_predicted_results(self, round_user, round_partition, result_size):
return self.recommender.get_recommendation(round_user, result_size)
def get_user_score(user):
cross_item_score = {}
for pkg in user.pkg_profile:
cross_item_score[pkg] = user.item_score[pkg]
return cross_item_score
def __str__(self):
"""
String representation of the object.
"""
str = "\n"
metrics_desc = ""
for metric in self.metrics_list:
metrics_desc += "%s|" % (metric.desc)
str += "| Round |%s\n" % metrics_desc
for r in range(self.rounds):
metrics_result = ""
for metric in self.metrics_list:
metrics_result += (" %2.1f%% |" %
(self.cross_results[metric.desc][r] * 100))
str += "| %d |%s\n" % (r, metrics_result)
metrics_mean = ""
for metric in self.metrics_list:
mean = float(sum(self.cross_results[metric.desc]) /
len(self.cross_results[metric.desc]))
metrics_mean += " %2.1f%% |" % (mean * 100)
str += "| Mean |%s\n" % (metrics_mean)
return str
AppRecommender-0.7.5/apprecommender/experiments/ 0000775 0000000 0000000 00000000000 13067513116 0021773 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/apprecommender/experiments/README 0000664 0000000 0000000 00000000363 13067513116 0022655 0 ustar 00root root 0000000 0000000 AppRecommender experiments and tests
---------------------------------------
Install dependencies:
# apt-get install \
python-unittest2 python-gnuplot gnuplot
# cd ./src; git clone https://github.com/rueckstiess/expsuite (deprecated tests)
AppRecommender-0.7.5/apprecommender/experiments/deprecated/ 0000775 0000000 0000000 00000000000 13067513116 0024073 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/apprecommender/experiments/deprecated/README 0000664 0000000 0000000 00000000111 13067513116 0024744 0 ustar 00root root 0000000 0000000 Experiments handled by expsuite:
https://github.com/rueckstiess/expsuite
AppRecommender-0.7.5/apprecommender/experiments/deprecated/clustering-suite.py 0000775 0000000 0000000 00000004021 13067513116 0027753 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
recommender suite - recommender experiments suite
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import sys
import os
import logging
sys.path.insert(0, '../')
from apprecommender.config import Config
from apprecommender.data import PopconXapianIndex
if __name__ == '__main__':
cfg = Config()
cfg.index_mode = "recluster"
logging.info("Starting clustering experiments")
logging.info("Medoids: %d\t Max popcon:%d" %
(cfg.k_medoids, cfg.max_popcon))
cfg.popcon_dir = os.path.expanduser(
"~/org/popcon.debian.org/popcon-mail/popcon-entries/")
cfg.popcon_index = cfg.popcon_index + ("_%dmedoids%dmax" %
(cfg.k_medoids, cfg.max_popcon))
cfg.clusters_dir = cfg.clusters_dir + ("_%dmedoids%dmax" %
(cfg.k_medoids, cfg.max_popcon))
pxi = PopconXapianIndex(cfg)
logging.info("Overall dispersion: %f\n" % pxi.cluster_dispersion)
# Write clustering log
output = open(("results/clustering/%dmedoids%dmax" %
(cfg.k_medoids, cfg.max_popcon)), 'w')
output.write("# k_medoids\tmax_popcon\tdispersion\n")
output.write("%d %f\n" %
(cfg.k_medoids, cfg.max_popcon, pxi.cluster_dispersion))
output.close()
AppRecommender-0.7.5/apprecommender/experiments/deprecated/experiments.cfg 0000664 0000000 0000000 00000001222 13067513116 0027114 0 ustar 00root root 0000000 0000000 [DEFAULT]
repetitions = 1
iterations = 10
path = 'results'
experiment = 'grid'
weight = ['bm25', 'trad']
;profile_size = range(10,100,10)
;sample = [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
sample = [0.6, 0.7, 0.8, 0.9]
[content]
strategy = ['cb','cbt','cbd']
[clustering]
experiment = 'single'
;iterations = 4
;medoids = range(2,6)
iterations = 6
medoids = [100,500,1000,5000,10000,50000]
;disabled for this experiment
weight = 0
profile_size = 0
sample = 0
[colaborative]
users_repository=["data/popcon","data/popcon-100","data/popcon-500","data/popcon-1000","data/popcon-5000","data/popcon-10000","data/popcon-50000"]
neighbors = range(10,1010,50)
AppRecommender-0.7.5/apprecommender/experiments/deprecated/k-suite.py 0000775 0000000 0000000 00000017373 13067513116 0026044 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
k-suite - experiment different neighborhood sizes
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import os
import sys
import random
import Gnuplot
sys.path.insert(0, '../')
from apprecommender.config import Config
from apprecommender.recommender import Recommender, RecommendationResult
from apprecommender.user import User, PopconSystem
from apprecommender.evaluation import (Evaluation, Precision,
Recall, FPR, F_score, MCC)
def plot_roc(k, roc_points, log_file):
g = Gnuplot.Gnuplot()
g('set style data points')
g.xlabel('False Positive Rate')
g.ylabel('True Positive Rate')
g('set xrange [0:1.0]')
g('set yrange [0:1.0]')
g.title("Setup: %s-k%d" % (log_file.split("/")[-1], k))
g.plot(Gnuplot.Data([[0, 0], [1, 1]], with_="lines lt 7"),
Gnuplot.Data(roc_points))
g.hardcopy(log_file + ("-k%.3d.png" % k), terminal="png")
g.hardcopy(log_file + ("-k%.3d.ps" % k),
terminal="postscript", enhanced=1, color=1)
def plot_summary(precision, f05, mcc, log_file):
g = Gnuplot.Gnuplot()
g('set style data lines')
g.xlabel('Neighborhood (k)')
g.title("Setup: %s-size20" % (log_file.split("/")[-1]))
g.plot(
Gnuplot.Data([[k, sum(precision[k]) / len(precision[k])]
for k in precision.keys()], title="P"),
Gnuplot.Data([[k, sum(f05[k]) / len(f05[k])]
for k in f05.keys()], title="F05"),
Gnuplot.Data([[k, sum(mcc[k]) / len(mcc[k])] for k in mcc.keys()],
title="MCC"))
g.hardcopy(log_file + (".png"), terminal="png")
g.hardcopy(log_file + (".ps"), terminal="postscript", enhanced=1, color=1)
class ExperimentResults:
def __init__(self, repo_size):
self.repository_size = repo_size
self.precision = []
self.recall = []
self.fpr = []
self.f05 = []
self.mcc = []
def add_result(self, ranking, sample):
predicted = RecommendationResult(dict.fromkeys(ranking, 1))
real = RecommendationResult(sample)
evaluation = Evaluation(predicted, real, self.repository_size)
self.precision.append(evaluation.run(Precision()))
self.recall.append(evaluation.run(Recall()))
self.fpr.append(evaluation.run(FPR()))
self.f05.append(evaluation.run(F_score(0.5)))
self.mcc.append(evaluation.run(MCC()))
def get_roc_point(self):
tpr = self.recall
fpr = self.fpr
if not tpr or not fpr:
return [0, 0]
return [sum(fpr) / len(fpr), sum(tpr) / len(tpr)]
def get_precision_summary(self):
if not self.precision:
return 0
return sum(self.precision) / len(self.precision)
def get_f05_summary(self):
if not self.f05:
return 0
return sum(self.f05) / len(self.f05)
def get_mcc_summary(self):
if not self.mcc:
return 0
return sum(self.mcc) / len(self.mcc)
if __name__ == '__main__':
if len(sys.argv) < 3:
print "Usage: k-suite strategy_str sample_file"
exit(1)
threshold = 20
iterations = 30
neighbors = [3, 5, 10, 50, 100, 150, 200, 300, 400, 500]
cfg = Config()
cfg.strategy = sys.argv[1]
sample_file = sys.argv[2]
population_sample = []
with open(sample_file, 'r') as f:
for line in f.readlines():
user_id = line.strip('\n')
population_sample.append(
os.path.join(cfg.popcon_dir, user_id[:2], user_id))
# setup dictionaries and files
roc_summary = {}
recommended = {}
precision_summary = {}
f05_summary = {}
mcc_summary = {}
sample_dir = ("results/k-suite/%s" % sample_file.split('/')[-1])
if not os.path.exists(sample_dir):
os.makedirs(sample_dir)
log_file = os.path.join(sample_dir, cfg.strategy)
with open(log_file, 'w') as f:
f.write("# %s\n\n" % sample_file.split('/')[-1])
f.write("# strategy %s recommendation_size %d iterations %d\n\n" %
(cfg.strategy, threshold, iterations))
f.write("# k coverage \tprecision \tf05 \tmcc\n\n")
for k in neighbors:
roc_summary[k] = []
recommended[k] = set()
precision_summary[k] = []
f05_summary[k] = []
mcc_summary[k] = []
with open(log_file + "-k%.3d" % k, 'w') as f:
f.write("# %s\n\n" % sample_file.split('/')[-1])
f.write("# strategy-k %s-k%.3d\n\n" % (cfg.strategy, k))
f.write("# roc_point \tprecision \tf05 \tmcc\n\n")
# main loop per user
for submission_file in population_sample:
user = PopconSystem(submission_file)
user.filter_pkg_profile(cfg.pkgs_filter)
user.maximal_pkg_profile()
for k in neighbors:
cfg.k_neighbors = k
rec = Recommender(cfg)
repo_size = rec.items_repository.get_doccount()
results = ExperimentResults(repo_size)
# n iterations for same recommender and user
for n in range(iterations):
# Fill sample profile
profile_len = len(user.pkg_profile)
item_score = {}
for pkg in user.pkg_profile:
item_score[pkg] = user.item_score[pkg]
sample = {}
sample_size = int(profile_len * 0.9)
for i in range(sample_size):
key = random.choice(item_score.keys())
sample[key] = item_score.pop(key)
iteration_user = User(item_score)
recommendation = rec.get_recommendation(
iteration_user, threshold)
if hasattr(recommendation, "ranking"):
results.add_result(recommendation.ranking, sample)
recommended[k] = recommended[
k].union(recommendation.ranking)
# save summary
roc_point = results.get_roc_point()
roc_summary[k].append(roc_point)
precision = results.get_precision_summary()
precision_summary[k].append(precision)
f05 = results.get_f05_summary()
f05_summary[k].append(f05)
mcc = results.get_mcc_summary()
mcc_summary[k].append(mcc)
with open(log_file + "-k%.3d" % k, 'a') as f:
f.write("[%.2f,%.2f] \t%.4f \t%.4f \t%.4f\n" %
(roc_point[0], roc_point[1], precision, f05, mcc))
# back to main flow
with open(log_file, 'a') as f:
plot_summary(precision_summary, f05_summary, mcc_summary, log_file)
for k in neighbors:
coverage = len(recommended[size]) / float(repo_size) # noqa ??
f.write("%3d \t%.2f \t%.4f \t%.4f \t%.4f\n" %
(k, coverage, float(sum(precision_summary[
k])) / len(precision_summary[k]),
float(sum(f05_summary[k])) / len(f05_summary[k]),
float(sum(mcc_summary[k])) / len(mcc_summary[k])))
plot_roc(k, roc_summary[k], log_file)
AppRecommender-0.7.5/apprecommender/experiments/deprecated/runner.py 0000775 0000000 0000000 00000016655 13067513116 0025776 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
recommender suite - recommender experiments suite
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import expsuite
import sys
import logging
import random
import Gnuplot
sys.path.insert(0, '../')
from apprecommender.config import Config
from apprecommender.data import PopconXapianIndex
from apprecommender.recommender import Recommender, RecommendationResult
from apprecommender.evaluation import (Evaluation, Accuracy,
Precision, Recall, F1)
from apprecommender.user import LocalSystem, User
class ClusteringSuite(expsuite.PyExperimentSuite):
def reset(self, params, rep):
self.cfg = Config()
self.cfg.popcon_index = "../tests/test_data/.sample_pxi"
self.cfg.popcon_dir = "../tests/test_data/popcon_dir"
self.cfg.clusters_dir = "../tests/test_data/clusters_dir"
if params['name'] == "clustering":
logging.info("Starting 'clustering' experiments suite...")
self.cfg.index_mode = "recluster"
def iterate(self, params, rep, n):
if params['name'] == "clustering":
logging.info("Running iteration %d" % params['medoids'][n])
self.cfg.k_medoids = params['medoids'][n]
pxi = PopconXapianIndex(self.cfg)
result = {'k_medoids': params['medoids'][n],
'dispersion': pxi.cluster_dispersion}
else:
result = {}
return result
class ContentBasedSuite(expsuite.PyExperimentSuite):
def reset(self, params, rep):
if params['name'].startswith("content"):
cfg = Config()
# if the index was not built yet
# app_axi = AppAptXapianIndex(cfg.axi,"results/arnaldo/AppAxi")
cfg.axi = "data/AppAxi"
cfg.index_mode = "old"
cfg.weight = params['weight']
self.rec = Recommender(cfg)
self.rec.set_strategy(params['strategy'])
self.repo_size = self.rec.items_repository.get_doccount()
self.user = LocalSystem()
self.user.app_pkg_profile(self.rec.items_repository)
self.sample_size = int(
len(self.user.pkg_profile) * params['sample'])
# iteration should be set to 10 in config file
# self.profile_size = range(10,101,10)
def iterate(self, params, rep, n):
if params['name'].startswith("content"):
item_score = dict.fromkeys(self.user.pkg_profile, 1)
# Prepare partition
sample = {}
for i in range(self.sample_size):
key = random.choice(item_score.keys())
sample[key] = item_score.pop(key)
# Get full recommendation
user = User(item_score)
recommendation = self.rec.get_recommendation(user, self.repo_size)
# Write recall log
recall_file = "results/content/recall/%s-%s-%.2f-%d" % \
(params['strategy'], params[
'weight'], params['sample'], n)
output = open(recall_file, 'w')
output.write("# weight=%s\n" % params['weight'])
output.write("# strategy=%s\n" % params['strategy'])
output.write("# sample=%f\n" % params['sample'])
output.write("\n%d %d %d\n" %
(self.repo_size, len(item_score), self.sample_size))
notfound = []
ranks = []
for pkg in sample.keys():
if pkg in recommendation.ranking:
ranks.append(recommendation.ranking.index(pkg))
else:
notfound.append(pkg)
for r in sorted(ranks):
output.write(str(r) + "\n")
if notfound:
output.write("Out of recommendation:\n")
for pkg in notfound:
output.write(pkg + "\n")
output.close()
# Plot metrics summary
accuracy = []
precision = []
recall = []
f1 = []
g = Gnuplot.Gnuplot()
g('set style data lines')
g.xlabel('Recommendation size')
for size in range(1, len(recommendation.ranking) + 1, 100):
predicted = RecommendationResult(
dict.fromkeys(recommendation.ranking[:size], 1))
real = RecommendationResult(sample)
evaluation = Evaluation(predicted, real, self.repo_size)
accuracy.append([size, evaluation.run(Accuracy())])
precision.append([size, evaluation.run(Precision())])
recall.append([size, evaluation.run(Recall())])
f1.append([size, evaluation.run(F1())])
g.plot(Gnuplot.Data(accuracy, title="Accuracy"),
Gnuplot.Data(precision, title="Precision"),
Gnuplot.Data(recall, title="Recall"),
Gnuplot.Data(f1, title="F1"))
g.hardcopy(recall_file + "-plot.ps", enhanced=1, color=1)
# Iteration log
result = {'iteration': n,
'weight': params['weight'],
'strategy': params['strategy'],
'accuracy': accuracy[20],
'precision': precision[20],
'recall:': recall[20],
'f1': f1[20]}
return result
# class CollaborativeSuite(expsuite.PyExperimentSuite):
# def reset(self, params, rep):
# if params['name'].startswith("collaborative"):
#
# def iterate(self, params, rep, n):
# if params['name'].startswith("collaborative"):
# for root, dirs, files in os.walk(self.source_dir):
# for popcon_file in files:
# submission = PopconSubmission(
# os.path.join(root,popcon_file))
# user = User(submission.packages)
# user.maximal_pkg_profile()
# rec.get_recommendation(user)
# precision = 0
# result = {'weight': params['weight'],
# 'strategy': params['strategy'],
# 'profile_size': self.profile_size[n],
# 'accuracy': accuracy,
# 'precision': precision,
# 'recall:': recall,
# 'f1': }
# else:
# result = {}
# return result
if __name__ == '__main__':
if "clustering" in sys.argv or len(sys.argv) < 3:
ClusteringSuite().start()
if "content" in sys.argv or len(sys.argv) < 3:
ContentBasedSuite().start()
# if "collaborative" in sys.argv or len(sys.argv)<3:
# CollaborativeSuite().start()
AppRecommender-0.7.5/apprecommender/experiments/deprecated/strategies-suite.py 0000775 0000000 0000000 00000026716 13067513116 0027765 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
recommender suite - recommender experiments suite
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import sys
import random
import Gnuplot
sys.path.insert(0, '../')
from apprecommender.config import Config
from apprecommender.recommender import Recommender, RecommendationResult
from apprecommender.user import User, PopconSystem
from apprecommender.evaluation import (Evaluation, Accuracy,
Precision, Recall, F_score)
# iterations = 3
# sample_proportions = [0.9]
# weighting = [('bm25',1.2)]
# collaborative = ['knn']
# content_based = []
# hybrid = ['knnco']
# profile_size = [50,100]
# popcon_size = ["1000"]
# neighbors = [50]
iterations = 10
sample_proportions = [0.5, 0.6, 0.7, 0.8, 0.9]
weighting = [('bm25', 1.2), ('bm25', 1.6), ('bm25', 2.0), ('trad', 0)]
content_based = ['cb', 'cbt', 'cbd', 'cbh',
'cb_eset', 'cbt_eset', 'cbd_eset', 'cbh_eset']
collaborative = ['knn_eset', 'knn', 'knn_plus']
hybrid = ['knnco', 'knnco_eset']
profile_size = range(20, 100, 20)
# popcon_size = [1000,10000,50000,'full']
neighbors = range(10, 510, 50)
def write_recall_log(label, n, sample, recommendation, profile_size, repo_size,
log_file):
# Write recall log
output = open(("%s-%d" % (log_file, n)), 'w')
output.write("# %s-n\n" % label["description"])
output.write("# %s-%d\n" % (label["values"], n))
output.write("\n%d %d %d\n" %
(repo_size, profile_size, len(sample)))
if hasattr(recommendation, "ranking"):
notfound = []
ranks = []
for pkg in sample.keys():
if pkg in recommendation.ranking:
ranks.append(recommendation.ranking.index(pkg))
else:
notfound.append(pkg)
for r in sorted(ranks):
output.write(str(r) + "\n")
if notfound:
output.write("Out of recommendation:\n")
for pkg in notfound:
output.write(pkg + "\n")
output.close()
def plot_summary(precision, recall, f1, f05, accuracy, log_file):
# Plot metrics summary
g = Gnuplot.Gnuplot()
g('set style data lines')
g.xlabel('Recommendation size')
g.title("Setup: %s" % log_file.split("/")[-1])
g.plot(Gnuplot.Data(accuracy, title="Accuracy"),
Gnuplot.Data(precision, title="Precision"),
Gnuplot.Data(recall, title="Recall"),
Gnuplot.Data(f1, title="F_1"),
Gnuplot.Data(f05, title="F_0.5"))
g.hardcopy(log_file + ".png", terminal="png")
g.hardcopy(log_file + ".ps", terminal="postscript", enhanced=1, color=1)
g('set logscale x')
g('replot')
g.hardcopy(log_file + "-logscale.png", terminal="png")
g.hardcopy(log_file + "-logscale.ps",
terminal="postscript", enhanced=1, color=1)
def get_label(cfg, sample_proportion):
label = {}
if cfg.strategy in content_based:
label["description"] = "strategy-filter-profile-k1_bm25-sample"
label["values"] = ("%s-profile%d-%s-kbm%.1f-sample%.1f" %
(cfg.strategy, cfg.profile_size,
cfg.pkgs_filter.split("/")[-1],
cfg.bm25_k1, sample_proportion))
elif cfg.strategy in collaborative:
label["description"] = "strategy-knn-filter-k1_bm25-sample"
label["values"] = ("%s-k%d-%s-kbm%.1f-sample%.1f" %
(cfg.strategy, cfg.k_neighbors,
cfg.pkgs_filter.split("/")[-1],
cfg.bm25_k1, sample_proportion))
elif cfg.strategy in hybrid:
label["description"] = "strategy-knn-filter-profile-k1_bm25-sample"
label["values"] = ("%s-k%d-profile%d-%s-kbm%.1f-sample%.1f" %
(cfg.strategy, cfg.k_neighbors, cfg.profile_size,
cfg.pkgs_filter.split("/")[-1],
cfg.bm25_k1, sample_proportion))
else:
print "Unknown strategy"
return label
class ExperimentResults:
def __init__(self, repo_size):
self.repository_size = repo_size
self.accuracy = {}
self.precision = {}
self.recall = {}
self.f1 = {}
self.f05 = {}
points = [1] + range(10, 200, 10) + range(
200, self.repository_size, 100)
for size in points:
self.accuracy[size] = []
self.precision[size] = []
self.recall[size] = []
self.f1[size] = []
self.f05[size] = []
def add_result(self, ranking, sample):
for size in self.accuracy.keys():
predicted = RecommendationResult(dict.fromkeys(ranking[:size], 1))
real = RecommendationResult(sample)
evaluation = Evaluation(predicted, real, self.repository_size)
self.accuracy[size].append(evaluation.run(Accuracy()))
self.precision[size].append(evaluation.run(Precision()))
self.recall[size].append(evaluation.run(Recall()))
self.f1[size].append(evaluation.run(F_score(1)))
self.f05[size].append(evaluation.run(F_score(0.5)))
def get_precision_summary(self):
summary = [[size, sum(values) / len(values)]
for size, values in self.precision.items()]
return sorted(summary)
def get_recall_summary(self):
summary = [[size, sum(values) / len(values)]
for size, values in self.recall.items()]
return sorted(summary)
def get_f1_summary(self):
summary = [[size, sum(values) / len(values)]
for size, values in self.f1.items()]
return sorted(summary)
def get_f05_summary(self):
summary = [[size, sum(values) / len(values)]
for size, values in self.f05.items()]
return sorted(summary)
def get_accuracy_summary(self):
summary = [[size, sum(values) / len(values)]
for size, values in self.accuracy.items()]
return sorted(summary)
def best_precision(self):
size = max(self.precision, key=lambda x: max(self.precision[x]))
return (size, max(self.precision[size]))
def best_f1(self):
size = max(self.f1, key=lambda x: max(self.f1[x]))
return (size, max(self.f1[size]))
def best_f05(self):
size = max(self.f05, key=lambda x: max(self.f05[x]))
return (size, max(self.f05[size]))
def run_strategy(cfg, user):
for weight in weighting:
cfg.weight = weight[0]
cfg.bm25_k1 = weight[1]
rec = Recommender(cfg)
repo_size = rec.items_repository.get_doccount()
for proportion in sample_proportions:
results = ExperimentResults(repo_size)
label = get_label(cfg, proportion)
log_file = "results/strategies/" + label["values"]
for n in range(iterations):
# Fill sample profile
profile_size = len(user.pkg_profile)
item_score = {}
for pkg in user.pkg_profile:
item_score[pkg] = user.item_score[pkg]
sample = {}
sample_size = int(profile_size * proportion)
for i in range(sample_size):
key = random.choice(item_score.keys())
sample[key] = item_score.pop(key)
iteration_user = User(item_score)
recommendation = rec.get_recommendation(
iteration_user, repo_size)
write_recall_log(
label, n, sample, recommendation, profile_size, repo_size,
log_file)
if hasattr(recommendation, "ranking"):
results.add_result(recommendation.ranking, sample)
with open(log_file, 'w') as f:
precision_10 = sum(results.precision[10]) / len(
results.precision[10])
f1_10 = sum(results.f1[10]) / len(results.f1[10])
f05_10 = sum(results.f05[10]) / len(results.f05[10])
f.write("# %s\n# %s\n\ncoverage %d\n\n" %
(label["description"], label["values"],
recommendation.size))
f.write("# best results (recommendation size; metric)\n")
f.write(
"precision (%d; %.2f)\nf1 (%d; %.2f)\nf05 (%d; %.2f)\n\n" %
(results.best_precision()[0], results.best_precision()[1],
results.best_f1()[0], results.best_f1()[1],
results.best_f05()[0], results.best_f05()[1]))
f.write("# recommendation size 10\nprecision (10; %.2f)\nf1 (10; %.2f)\nf05 (10; %.2f)" % # noqa
(precision_10, f1_10, f05_10))
precision = results.get_precision_summary()
recall = results.get_recall_summary()
f1 = results.get_f1_summary()
f05 = results.get_f05_summary()
accuracy = results.get_accuracy_summary()
plot_summary(precision, recall, f1, f05, accuracy, log_file)
def run_content(user, cfg):
for strategy in content_based:
cfg.strategy = strategy
for size in profile_size:
cfg.profile_size = size
run_strategy(cfg, user)
def run_collaborative(user, cfg):
for strategy in collaborative:
cfg.strategy = strategy
for k in neighbors:
cfg.k_neighbors = k
# for size in popcon_size:
# if size:
# cfg.popcon_desktopapps = popcon_desktopapps+"_"+size
# cfg.popcon_programs = popcon_programs+"_"+size
run_strategy(cfg, user)
def run_hybrid(user, cfg):
for strategy in hybrid:
cfg.strategy = strategy
for k in neighbors:
cfg.k_neighbors = k
# for size in popcon_size:
# if size:
# cfg.popcon_desktopapps = popcon_desktopapps+"_"+size
# cfg.popcon_programs = popcon_programs+"_"+size
for size in profile_size:
cfg.profile_size = size
run_strategy(cfg, user)
if __name__ == '__main__':
# user = LocalSystem()
# user =
# RandomPopcon(cfg.popcon_dir,os.path.join(cfg.filters_dir,"desktopapps"))
cfg = Config()
user = PopconSystem(
"/root/.app-recommender/popcon-entries/8b/8b44fcdbcf676e711a153d5db09979d7") # noqa
# user =
# PopconSystem("/root/.app-recommender/popcon-entries/4a/4a67a295ec14826db2aa1d90be2f1623")
user.filter_pkg_profile(cfg.pkgs_filter)
user.maximal_pkg_profile()
if "content" in sys.argv or len(sys.argv) < 2:
run_content(user, cfg)
if "collaborative" in sys.argv or len(sys.argv) < 2:
run_collaborative(user, cfg)
if "hybrid" in sys.argv or len(sys.argv) < 2:
run_hybrid(user, cfg)
AppRecommender-0.7.5/apprecommender/experiments/extract-sample-db.py 0000775 0000000 0000000 00000003671 13067513116 0025673 0 ustar 00root root 0000000 0000000 #! /usr/bin/env python
"""
sample-popcon - extract a sample from popcon population
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import xapian
import sys
if __name__ == '__main__':
try:
sample_file = sys.argv[1]
popcon = xapian.WritableDatabase(sys.argv[2], xapian.DB_OPEN)
except:
print "Usage: extract-sample-db sample_file popcon_index"
exit(1)
enquire = xapian.Enquire(popcon)
print sample_file.split("/")
new_popcon = xapian.WritableDatabase(
sys.argv[2] + "-" + sample_file.split("/")[-1],
xapian.DB_CREATE_OR_OVERWRITE)
print ("Popcon repository size: %d" % popcon.get_doccount())
for submission in open(sample_file):
print "ID" + submission.strip()
query = xapian.Query("ID" + submission.strip())
enquire.set_query(query)
mset = enquire.get_mset(0, 20)
for m in mset:
print "Adding doc %s" % m.docid
new_popcon.add_document(popcon.get_document(m.docid))
print "Removing doc %s" % m.docid
popcon.delete_document(m.docid)
print ("Popcon repository size: %d" % popcon.get_doccount())
print ("Popcon repository size: %d" % new_popcon.get_doccount())
AppRecommender-0.7.5/apprecommender/experiments/hybrid.py 0000775 0000000 0000000 00000022352 13067513116 0023635 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
hybrid-suite
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import Gnuplot
import numpy
import os
import random
import sys
from apprecommender.config import Config
from apprecommender.recommender import Recommender, RecommendationResult
from apprecommender.evaluation import Evaluation, Precision, F_score
from apprecommender.user import User, PopconSystem
# hybrid_strategies = ['knnco','knnco_eset']
if __name__ == '__main__':
if len(sys.argv) < 2:
print "Usage: hybrid strategy sample_file"
exit(1)
iterations = 20
profile_size = [10, 40, 70, 100, 170, 240]
neighbor_size = [3, 10, 50, 70, 100, 150, 200]
# iterations = 1
# profile_size = [10,20,30]
# neighbor_size = [10,20,30]
cfg = Config()
population_sample = []
strategy = sys.argv[1]
sample_file = sys.argv[2]
sample_str = sample_file.split('/')[-1]
with open(sample_file, 'r') as f:
for line in f.readlines():
user_id = line.strip('\n')
population_sample.append(
os.path.join(cfg.popcon_dir, user_id[:2], user_id))
sample_dir = ("results/hybrid/%s/%s" % (sample_str, strategy))
if not os.path.exists(sample_dir):
os.makedirs(sample_dir)
cfg.strategy = strategy
p_10_summary = {}
f05_100_summary = {}
c_10 = {}
c_100 = {}
log_file = os.path.join(sample_dir, sample_str + "-" + cfg.strategy)
graph_10 = {}
graph_100 = {}
graph_10_jpg = {}
graph_100_jpg = {}
comment_10 = {}
comment_100 = {}
for k in neighbor_size:
graph_10[k] = log_file + ("-neighborhood%.3d-010.png" % k)
graph_100[k] = log_file + ("-neighborhood%.3d-100.png" % k)
graph_10_jpg[k] = graph_10[k].strip(".png") + ".jpg"
graph_100_jpg[k] = graph_100[k].strip(".png") + ".jpg"
comment_10[k] = graph_10_jpg[k] + ".comment"
comment_100[k] = graph_100_jpg[k] + ".comment"
with open(comment_10[k], 'w') as f:
f.write("# %s\n" % sample_str)
f.write("# strategy %s\n# threshold 10\n# iterations %d\n\n" %
(cfg.strategy, iterations))
f.write("# neighborhood\tprofile\tmean_p_10\tdev_p_10\tc_10\n\n")
with open(comment_100[k], 'w') as f:
f.write("# %s\n" % sample_str)
f.write("# strategy %s\n# threshold 100\n# iterations %d\n\n" %
(cfg.strategy, iterations))
f.write(
'# neighborhood\tprofile\tmean_f05_100\tdev_f05_100\tc_100\n\n') # noqa
c_10[k] = {}
c_100[k] = {}
p_10_summary[k] = {}
f05_100_summary[k] = {}
for size in profile_size:
c_10[k][size] = set()
c_100[k][size] = set()
p_10_summary[k][size] = []
f05_100_summary[k][size] = []
with open(log_file + "-neighborhood%.3d-profile%.3d" % (k, size),
'w') as f:
f.write("# %s\n" % sample_str)
f.write("# strategy %s-neighborhood%.3d-profile%.3d\n\n" %
(cfg.strategy, k, size))
f.write("# p_10\t\tf05_100\n\n")
# main loop per user
for submission_file in population_sample:
user = PopconSystem(submission_file)
user.filter_pkg_profile(cfg.pkgs_filter)
user.maximal_pkg_profile()
for k in neighbor_size:
cfg.k_neighbors = k
for size in profile_size:
cfg.profile_size = size
rec = Recommender(cfg)
repo_size = rec.items_repository.get_doccount()
p_10 = []
f05_100 = []
for n in range(iterations):
# Fill sample profile
profile_len = len(user.pkg_profile)
item_score = {}
for pkg in user.pkg_profile:
item_score[pkg] = user.item_score[pkg]
sample = {}
sample_size = int(profile_len * 0.9)
for i in range(sample_size):
key = random.choice(item_score.keys())
sample[key] = item_score.pop(key)
iteration_user = User(item_score)
recommendation = rec.get_recommendation(
iteration_user, repo_size)
if hasattr(recommendation, "ranking"):
ranking = recommendation.ranking
real = RecommendationResult(sample)
predicted_10 = RecommendationResult(
dict.fromkeys(ranking[:10], 1))
evaluation = Evaluation(predicted_10, real, repo_size)
p_10.append(evaluation.run(Precision()))
predicted_100 = RecommendationResult(
dict.fromkeys(ranking[:100], 1))
evaluation = Evaluation(predicted_100, real, repo_size)
f05_100.append(evaluation.run(F_score(0.5)))
c_10[k][size] = c_10[k][size].union(
recommendation.ranking[:10])
c_100[k][size] = c_100[k][size].union(
recommendation.ranking[:100])
# save summary
if p_10:
p_10_summary[k][size].append(numpy.mean(p_10))
if f05_100:
f05_100_summary[k][size].append(numpy.mean(f05_100))
path = log_file + "-neighborhood%.3d-profile%.3d" % (k, size)
with open(path, 'a') as f:
f.write("%.4f\t\t%.4f\n" %
(numpy.mean(p_10), numpy.mean(f05_100)))
# back to main flow
coverage_10 = {}
coverage_100 = {}
for k in neighbor_size:
coverage_10[k] = {}
coverage_100[k] = {}
with open(comment_10[k], 'a') as f:
for size in profile_size:
coverage_10[k][size] = len(c_10[k][size]) / float(repo_size)
f.write("%3d\t\t%3d\t\t%.4f\t%.4f\t%.4f\n" %
(k, size, numpy.mean(p_10_summary[k][size]),
numpy.std(p_10_summary[k][size]),
coverage_10[k][size]))
with open(comment_100[k], 'a') as f:
for size in profile_size:
coverage_100[k][size] = len(c_100[k][size]) / float(repo_size)
f.write("%3d\t\t%3d\t\t%.4f\t%.4f\t%.4f\n" %
(k, size, numpy.mean(f05_100_summary[k][size]),
numpy.std(f05_100_summary[k][size]),
coverage_100[k][size]))
for k in neighbor_size:
# plot results summary
g = Gnuplot.Gnuplot()
g('set style data lines')
g('set yrange [0:1.0]')
g.xlabel('Profile size')
g.title("Setup: %s-neighborhood%3d (threshold 10)" % (cfg.strategy, k))
g.plot(Gnuplot.Data(sorted([[i, numpy.mean(p_10_summary[k][i]),
numpy.std(p_10_summary[k][i])]
for i in p_10_summary[k].keys(
)]), title="Precision"),
Gnuplot.Data(sorted([[i, numpy.mean(p_10_summary[k][i]),
numpy.std(p_10_summary[k][i])]
for i in p_10_summary[k].keys(
)]), title="Deviation",
with_="yerrorbar lt 2 pt 6"), Gnuplot.Data(
sorted(
[[i, coverage_10[k][i]] for i in coverage_10[k].keys()]),
title="Coverage"))
g.hardcopy(graph_10[k], terminal="png")
g = Gnuplot.Gnuplot()
g('set style data lines')
g('set yrange [0:1.0]')
g.xlabel('Profile size')
g.title("Setup: %s-neighborhood%3d (threshold 100)" %
(cfg.strategy, k))
g.plot(Gnuplot.Data(sorted([[i, numpy.mean(f05_100_summary[k][i]),
numpy.std(f05_100_summary[k][i])]
for i in f05_100_summary[k].keys(
)]), title="F05"),
Gnuplot.Data(sorted([[i, numpy.mean(f05_100_summary[k][i]),
numpy.std(f05_100_summary[k][i])]
for i in f05_100_summary[k].keys(
)]), title="Deviation",
with_="yerrorbar lt 2 pt 6"), Gnuplot.Data(
sorted(
[[i, coverage_100[k][i]] for i in coverage_100[k].keys()]),
title="Coverage"))
g.hardcopy(graph_100[k], terminal="png")
AppRecommender-0.7.5/apprecommender/experiments/misc-popcon.py 0000775 0000000 0000000 00000005467 13067513116 0024613 0 ustar 00root root 0000000 0000000 #! /usr/bin/env python
"""
misc_popcon - misc experiments with popcon data
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import Gnuplot
import xapian
def profile_population():
popcon = xapian.Database(
"/home/tassia/.app-recommender/popcon_desktopapps")
popcon_size = popcon.get_doccount()
print "User repository size: %d" % popcon_size
profiles_size = []
for n in range(1, popcon_size):
user = popcon.get_document(n)
profile = [t.term for t in user.termlist()]
profiles_size.append(len(profile))
profile_population = [(n, profiles_size.count(n))
for n in range(max(profiles_size) + 1)
if profiles_size.count(n) > 0]
ranges_population = []
ranges_percentage = []
ranges = range(0, 601, 50)
for maximum in ranges[1:]:
minimum = ranges[ranges.index(maximum) - 1]
valid = [x[1] for x in profile_population
if x[0] > minimum and x[0] <= maximum]
ranges_population.append((maximum, sum(valid)))
ranges_percentage.append((maximum, sum(valid) / float(popcon_size)))
g = Gnuplot.Gnuplot()
g('set style data points') # give gnuplot an arbitrary command
g.xlabel('Desktop profile size')
g.ylabel('Population size')
g.plot(profile_population)
g.hardcopy('profile_population.png', terminal="png")
g.hardcopy('profile_population.ps',
terminal="postscript", enhanced=1, color=1)
g.reset()
g.xlabel('Range Desktop profile size')
g.ylabel('Population size')
g.plot(ranges_population)
g.hardcopy('ranges_profile_population.png', terminal="png")
g.hardcopy('ranges_profile_population.ps',
terminal="postscript", enhanced=1, color=1)
g.reset()
g.xlabel('Range Desktop profile size')
g.ylabel('Population percentage')
g.plot(ranges_percentage)
g.hardcopy('ranges_profile_percentage.png', terminal="png")
g.hardcopy('ranges_profile_percentage.ps',
terminal="postscript", enhanced=1, color=1)
if __name__ == '__main__':
profile_population()
AppRecommender-0.7.5/apprecommender/experiments/popcon-population.py 0000775 0000000 0000000 00000005632 13067513116 0026044 0 ustar 00root root 0000000 0000000 #! /usr/bin/env python
"""
misc_popcon - misc experiments with popcon data
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import Gnuplot
import xapian
import os
def get_population_profile(popcon):
profiles_size = []
for n in range(1, popcon.get_doccount()):
user = popcon.get_document(n)
pkgs_profile = [
t.term for t in user.termlist() if t.term.startswith("XP")]
if len(pkgs_profile) < 10:
print "-- profile<10:", user.get_data()
profiles_size.append(len(pkgs_profile))
max_profile = max(profiles_size)
population_profile = [(n, profiles_size.count(n))
for n in range(max_profile + 1)
if profiles_size.count(n) > 0]
return population_profile, max_profile
def get_profile_ranges(population_profile, max_profile, popcon_size):
ranges = range(0, 251, 50)
ranges.append(max_profile)
ranges_population = []
ranges_percentage = []
for maximum in ranges[1:]:
minimum = ranges[ranges.index(maximum) - 1]
valid = [x[1] for x in population_profile
if x[0] > minimum and x[0] <= maximum]
ranges_population.append((maximum, sum(valid)))
ranges_percentage.append((maximum, sum(valid) / float(popcon_size)))
return ranges_population, ranges_percentage
def plot(data, xlabel, ylabel, output):
g = Gnuplot.Gnuplot()
g('set style data points')
g.xlabel(xlabel)
g.ylabel(ylabel)
g.plot(data)
g.hardcopy(output + ".png", terminal="png")
g.hardcopy(output + ".ps", terminal="postscript", enhanced=1, color=1)
if __name__ == '__main__':
popcon = xapian.Database(
os.path.expanduser("~/.app-recommender/popcon_desktopapps"))
print ("Popcon repository size: %d" % popcon.get_doccount())
profile_population, max_profile = get_population_profile(popcon)
ranges_population, ranges_percentage = get_profile_ranges(
profile_population, max_profile, popcon.get_doccount())
print "Population per profile range (up to index)"
print ranges_population
plot(profile_population, "Desktop profile size", "Population size",
"results/misc-popcon/profile_population")
AppRecommender-0.7.5/apprecommender/experiments/pure.py 0000775 0000000 0000000 00000022421 13067513116 0023324 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
profile-suite - experiment different profile sizes
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import os
import sys
import random
import Gnuplot
import numpy
from apprecommender.config import Config
from apprecommender.evaluation import Evaluation, Precision, F_score
from apprecommender.recommender import Recommender, RecommendationResult
from apprecommender.user import User, PopconSystem
if __name__ == '__main__':
if len(sys.argv) < 2:
print "Usage: pure strategy_category sample_file"
exit(1)
iterations = 20
profile_size = [10, 20, 40, 60, 80, 100, 140, 170, 200, 240]
neighbor_size = [3, 5, 10, 20, 30, 50, 70, 100, 150, 200]
content_strategies = [
'cb', 'cbt', 'cbd', 'cbh', 'cb_eset', 'cbt_eset',
'cbd_eset', 'cbh_eset']
collaborative_strategies = ['knn_eset', 'knn', 'knn_plus']
# iterations = 1
# profile_size = [10,20,30]
# neighbor_size = [3,5,10,20,30,50]
# content_strategies = ['cb']
# collaborative_strategies = ['knn']
strategy_category = sys.argv[1]
if strategy_category == "content":
strategies = content_strategies
sizes = profile_size
option_str = "profile"
elif strategy_category == "collaborative":
strategies = collaborative_strategies
sizes = neighbor_size
option_str = "neighborhood"
else:
print "Usage: profile-suite strategy_category sample_file"
exit(1)
cfg = Config()
population_sample = []
sample_file = sys.argv[2]
sample_str = sample_file.split('/')[-1]
with open(sample_file, 'r') as f:
for line in f.readlines():
user_id = line.strip('\n')
population_sample.append(
os.path.join(cfg.popcon_dir, user_id[:2], user_id))
sample_dir = ("results/%s/%s" %
(strategy_category, sample_str))
if not os.path.exists(sample_dir):
os.makedirs(sample_dir)
for strategy in strategies:
cfg.strategy = strategy
p_10_summary = {}
f05_100_summary = {}
c_10 = {}
c_100 = {}
log_file = os.path.join(sample_dir, sample_str + "-" + cfg.strategy)
graph_10 = log_file + "-10.png"
graph_100 = log_file + "-100.png"
graph_10_jpg = graph_10.strip(".png") + ".jpg"
graph_100_jpg = graph_100.strip(".png") + ".jpg"
comment_10 = graph_10_jpg + ".comment"
comment_100 = graph_100_jpg + ".comment"
with open(comment_10, 'w') as f:
f.write("# sample %s\n" % sample_str)
f.write("# strategy %s\n# threshold 10\n# iterations %d\n\n" %
(cfg.strategy, iterations))
f.write("# %s\tmean_p_10\tdev_p_10\tc_10\n\n" % option_str)
with open(comment_100, 'w') as f:
f.write("# sample %s\n" % sample_str)
f.write("# strategy %s\n# threshold 100\n# iterations %d\n\n" %
(cfg.strategy, iterations))
f.write("# %s\t\tmean_f05_100\t\tdev_f05_100\t\tc_100\n\n" %
option_str)
for size in sizes:
c_10[size] = set()
c_100[size] = set()
p_10_summary[size] = []
f05_100_summary[size] = []
with open(log_file + "-%s%.3d" % (option_str, size), 'w') as f:
f.write("# sample %s\n" % sample_str)
f.write("# strategy %s-%s%.3d\n\n" %
(cfg.strategy, option_str, size))
f.write("# p_10\tf05_100\n\n")
# main loop per user
for submission_file in population_sample:
user = PopconSystem(submission_file)
user.filter_pkg_profile(cfg.pkgs_filter)
user.maximal_pkg_profile()
for size in sizes:
cfg.profile_size = size
cfg.k_neighbors = size
rec = Recommender(cfg)
repo_size = rec.items_repository.get_doccount()
p_10 = []
f05_100 = []
for n in range(iterations):
# Fill sample profile
profile_len = len(user.pkg_profile)
item_score = {}
for pkg in user.pkg_profile:
item_score[pkg] = user.item_score[pkg]
sample = {}
sample_size = int(profile_len * 0.9)
for i in range(sample_size):
key = random.choice(item_score.keys())
sample[key] = item_score.pop(key)
iteration_user = User(item_score)
recommendation = rec.get_recommendation(
iteration_user, repo_size)
if hasattr(recommendation, "ranking"):
ranking = recommendation.ranking
real = RecommendationResult(sample)
predicted_10 = RecommendationResult(
dict.fromkeys(ranking[:10], 1))
evaluation = Evaluation(predicted_10, real, repo_size)
p_10.append(evaluation.run(Precision()))
predicted_100 = RecommendationResult(
dict.fromkeys(ranking[:100], 1))
evaluation = Evaluation(predicted_100, real, repo_size)
f05_100.append(evaluation.run(F_score(0.5)))
c_10[size] = c_10[size].union(
recommendation.ranking[:10])
c_100[size] = c_100[size].union(
recommendation.ranking[:100])
# save summary
if p_10:
p_10_summary[size].append(numpy.mean(p_10))
if f05_100:
f05_100_summary[size].append(numpy.mean(f05_100))
with open(log_file + "-%s%.3d" % (option_str, size), 'a') as f:
f.write("%.4f \t%.4f\n" %
(numpy.mean(p_10), numpy.mean(f05_100)))
# back to main flow
coverage_10 = {}
coverage_100 = {}
with open(comment_10, 'a') as f:
for size in sizes:
coverage_10[size] = len(c_10[size]) / float(repo_size)
f.write("%3d\t\t%.4f\t\t%.4f\t\t%.4f\n" %
(size, numpy.mean(p_10_summary[size]),
numpy.std(p_10_summary[size]), coverage_10[size]))
with open(comment_100, 'a') as f:
for size in sizes:
coverage_100[size] = len(c_100[size]) / float(repo_size)
f.write("%3d\t\t%.4f\t\t%.4f\t\t%.4f\n" %
(size, numpy.mean(f05_100_summary[size]),
numpy.std(f05_100_summary[size]), coverage_100[size]))
# plot results summary
g = Gnuplot.Gnuplot()
g('set style data lines')
g('set yrange [0:1.0]')
g.xlabel('%s size' % option_str.capitalize())
g.title("Setup: %s (threshold 10)" % cfg.strategy)
g.plot(Gnuplot.Data(sorted([[k, numpy.mean(p_10_summary[k]),
numpy.std(p_10_summary[k])]
for k in p_10_summary.keys(
)]), title="Precision"),
Gnuplot.Data(sorted([[k, numpy.mean(p_10_summary[k]),
numpy.std(p_10_summary[k])]
for k in p_10_summary.keys(
)]), title="Deviation",
with_="yerrorbar lt 2 pt 6"), Gnuplot.Data(
sorted(
[[k, coverage_10[k]]
for k in coverage_10.keys()]), title="Coverage"))
g.hardcopy(graph_10, terminal="png")
g = Gnuplot.Gnuplot()
g('set style data lines')
g('set yrange [0:1.0]')
g.xlabel('%s size' % option_str.capitalize())
g.title("Setup: %s (threshold 100)" % cfg.strategy)
g.plot(Gnuplot.Data(sorted([[k, numpy.mean(f05_100_summary[k]),
numpy.std(f05_100_summary[k])]
for k in f05_100_summary.keys(
)]), title="F05"),
Gnuplot.Data(sorted([[k, numpy.mean(f05_100_summary[k]),
numpy.std(f05_100_summary[k])]
for k in f05_100_summary.keys(
)]), title="Deviation",
with_="yerrorbar lt 2 pt 6"), Gnuplot.Data(
sorted(
[[k, coverage_100[k]]
for k in coverage_100.keys()]), title="Coverage"))
g.hardcopy(graph_100, terminal="png")
AppRecommender-0.7.5/apprecommender/experiments/roc-sample.py 0000775 0000000 0000000 00000022715 13067513116 0024421 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
recommender suite - recommender experiments suite
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import os
import sys
import random
import Gnuplot
import numpy
from apprecommender.config import Config
from apprecommender.recommender import Recommender, RecommendationResult
from apprecommender.user import User, PopconSystem
from apprecommender.evaluation import (Evaluation, Precision,
F_score, FPR, Recall)
def plot_roc(results, log_file, mean=0):
g = Gnuplot.Gnuplot()
g('set style data lines')
g.xlabel('False Positive Rate')
g.ylabel('True Positive Rate')
g('set xrange [0:1.0]')
g('set yrange [0:1.0]')
g.title("Setup: %s" % log_file.split("/")[-1])
g('set label "C %.4f" at 0.68,0.2' % results.coverage())
g('set label "AUC %.4f" at 0.68,0.15' % results.get_auc())
g('set label "P(10) %.2f +- %.2f" at 0.68,0.10' %
(numpy.mean(results.precision[10]), numpy.std(results.precision[10])))
g('set label "F05(100) %.2f +- %.2f" at 0.68,0.05' %
(numpy.mean(results.f05[100]), numpy.std(results.f05[100])))
if mean == 1:
g.plot(Gnuplot.Data(results.get_roc_points(), title="mean ROC"),
Gnuplot.Data([[0, 0], [1, 1]], with_="lines lt 7"))
g.hardcopy(log_file + "-roc-mean.png", terminal="png")
g.hardcopy(log_file + "-roc-mean.ps",
terminal="postscript", enhanced=1, color=1)
else:
g.plot(
Gnuplot.Data(
results.get_roc_points(), title="ROC", with_="xyerrorbars"),
Gnuplot.Data([[0, 0], [1, 1]], with_="lines lt 7"))
g.hardcopy(log_file + "-roc.png", terminal="png")
g.hardcopy(log_file + "-roc.ps",
terminal="postscript", enhanced=1, color=1)
def get_label(cfg):
label = {}
if cfg.strategy in content_based:
label["description"] = "strategy-profile"
label["values"] = ("%s-profile%.3d" %
(cfg.strategy, cfg.profile_size))
elif cfg.strategy in collaborative:
label["description"] = "strategy-knn"
label["values"] = ("%s-k%.3d" %
(cfg.strategy, cfg.k_neighbors))
elif cfg.strategy in hybrid:
label["description"] = "strategy-knn-profile"
label["values"] = ("%s-k%.3d-profile%.3d" %
(cfg.strategy, cfg.k_neighbors, cfg.profile_size))
return label
class ExperimentResults:
def __init__(self, repo_size):
self.repository_size = repo_size
self.precision = {}
self.recall = {}
self.fpr = {}
self.f05 = {}
self.recommended = {}
self.thresholds = [1] + range(10, self.repository_size, 10)
for size in self.thresholds:
self.precision[size] = []
self.recall[size] = []
self.fpr[size] = []
self.f05[size] = []
self.recommended[size] = set()
def add_result(self, ranking, sample):
for size in self.thresholds:
recommendation = ranking[:size]
self.recommended[size] = self.recommended[
size].union(recommendation)
predicted = RecommendationResult(dict.fromkeys(recommendation, 1))
real = RecommendationResult(sample)
evaluation = Evaluation(predicted, real, self.repository_size)
self.precision[size].append(evaluation.run(Precision()))
self.recall[size].append(evaluation.run(Recall()))
self.f05[size].append(evaluation.run(F_score(0.5)))
self.fpr[size].append(evaluation.run(FPR()))
def precision_summary(self):
return [[size, numpy.mean(self.precision[size])]
for size in self.thresholds]
def recall_summary(self):
return [[size, numpy.mean(self.recall[size])]
for size in self.thresholds]
def f05_summary(self):
return [[size, numpy.mean(self.f05[size])] for size in self.thresholds]
def coverage_summary(self):
return [[size, self.coverage(size)] for size in self.thresholds]
def coverage(self, size=0):
if not size:
size = self.thresholds[-1]
return len(self.recommended[size]) / float(self.repository_size)
def precision(self, size):
return numpy.mean(self.precision[size])
def get_auc(self):
roc_points = self.get_roc_points()
x_roc = [p[0] for p in roc_points]
y_roc = [p[1] for p in roc_points]
x_roc.insert(0, 0)
y_roc.insert(0, 0)
x_roc.append(1)
y_roc.append(1)
return numpy.trapz(y=y_roc, x=x_roc)
# Average ROC by threshold (= size of recommendation)
def get_roc_points(self):
points = []
for size in self.recall.keys():
tpr = self.recall[size]
fpr = self.fpr[size]
points.append(
[numpy.mean(fpr), numpy.mean(tpr), numpy.std(fpr),
numpy.std(tpr)])
return sorted(points)
def run_strategy(cfg, sample_file):
rec = Recommender(cfg)
repo_size = rec.items_repository.get_doccount()
results = ExperimentResults(repo_size)
label = get_label(cfg)
population_sample = []
sample_str = sample_file.split('/')[-1]
with open(sample_file, 'r') as f:
for line in f.readlines():
user_id = line.strip('\n')
population_sample.append(
os.path.join(cfg.popcon_dir, user_id[:2], user_id))
sample_dir = ("results/roc-sample/%s" % sample_str)
if not os.path.exists(sample_dir):
os.makedirs(sample_dir)
log_file = os.path.join(sample_dir, label["values"])
# n iterations per population user
for submission_file in population_sample:
user = PopconSystem(submission_file)
user.filter_pkg_profile(cfg.pkgs_filter)
user.maximal_pkg_profile()
for n in range(iterations):
# Fill sample profile
profile_len = len(user.pkg_profile)
item_score = {}
for pkg in user.pkg_profile:
item_score[pkg] = user.item_score[pkg]
sample = {}
sample_size = int(profile_len * 0.9)
for i in range(sample_size):
key = random.choice(item_score.keys())
sample[key] = item_score.pop(key)
iteration_user = User(item_score)
recommendation = rec.get_recommendation(iteration_user, repo_size)
if hasattr(recommendation, "ranking"):
results.add_result(recommendation.ranking, sample)
plot_roc(results, log_file)
plot_roc(results, log_file, 1)
with open(log_file + "-roc.jpg.comment", 'w') as f:
f.write("# %s\n# %s\n\n" %
(label["description"], label["values"]))
f.write("# roc AUC\n%.4f\n\n" % results.get_auc())
f.write(
"# threshold\tmean_fpr\tdev_fpr\t\tmean_tpr\tdev_tpr\t\tcoverage\n") # noqa
for size in results.thresholds:
f.write("%4d\t\t%.4f\t\t%.4f\t\t%.4f\t\t%.4f\t\t%.4f\n" %
(size, numpy.mean(results.fpr[size]),
numpy.std(results.fpr[size]),
numpy.mean(results.recall[size]),
numpy.std(results.recall[size]),
numpy.mean(results.coverage(size))))
def run_content(cfg, sample_file):
for size in profile_size:
cfg.profile_size = size
run_strategy(cfg, sample_file)
def run_collaborative(cfg, sample_file):
for k in neighbors:
cfg.k_neighbors = k
run_strategy(cfg, sample_file)
def run_hybrid(cfg, sample_file):
for k in neighbors:
cfg.k_neighbors = k
for size in profile_size:
cfg.profile_size = size
run_strategy(cfg, sample_file)
if __name__ == '__main__':
if len(sys.argv) < 2:
print "Usage: sample-roc strategy_str [popcon_sample_path]"
exit(1)
# iterations = 3
# content_based = ['cb']
# collaborative = ['knn_eset']
# hybrid = ['knnco']
# profile_size = [50,100]
# neighbors = [50]
iterations = 20
content_based = ['cb', 'cbt', 'cbd', 'cbh',
'cb_eset', 'cbt_eset', 'cbd_eset', 'cbh_eset']
collaborative = ['knn_eset', 'knn', 'knn_plus']
hybrid = ['knnco', 'knnco_eset']
profile_size = [10, 20, 50, 100, 200]
neighbors = [200]
# neighbors = [3,10,50,100,200]
# profile_size = [10,20,40,60,80,100,140,170,200,240]
# neighbors = [3,5,10,20,30,50,70,100,150,200]
cfg = Config()
cfg.strategy = sys.argv[1]
sample_file = sys.argv[2]
if cfg.strategy in content_based:
run_content(cfg, sample_file)
if cfg.strategy in collaborative:
run_collaborative(cfg, sample_file)
if cfg.strategy in hybrid:
run_hybrid(cfg, sample_file)
AppRecommender-0.7.5/apprecommender/experiments/roc-single.py 0000775 0000000 0000000 00000025121 13067513116 0024413 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
recommender suite - recommender experiments suite
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import os
import sys
import random
import Gnuplot
import numpy
import shutil
from apprecommender.config import Config
from apprecommender.recommender import Recommender, RecommendationResult
from apprecommender.evaluation import (Evaluation, Precision,
Recall, F_score, FPR)
from apprecommender.user import User, PopconSystem
def write_recall_log(label, n, sample, recommendation, profile_size, repo_size,
log_file):
# Write recall log
output = open(("%s-%.2d" % (log_file, n)), 'w')
output.write("# %s-n\n" % label["description"])
output.write("# %s-%.2d\n" % (label["values"], n))
output.write("\n# repository profile sample\n%d %d %d\n" %
(repo_size, profile_size, len(sample)))
if hasattr(recommendation, "ranking"):
notfound = []
ranks = []
for pkg in sample.keys():
if pkg in recommendation.ranking:
ranks.append(recommendation.ranking.index(pkg))
else:
notfound.append(pkg)
for r in sorted(ranks):
output.write(str(r) + "\n")
if notfound:
output.write("# out of recommendation:\n")
for pkg in notfound:
output.write(pkg + "\n")
output.close()
def plot_summary(results, log_file):
# Plot metrics summary
g = Gnuplot.Gnuplot()
g('set style data lines')
g('set yrange [0:1.0]')
g.xlabel('Threshold (recommendation size)')
g.title("Setup: %s" % log_file.split("/")[-1])
g.plot(Gnuplot.Data(results.precision_summary(), title="Precision"),
Gnuplot.Data(results.recall_summary(), title="Recall"),
Gnuplot.Data(results.f05_summary(), title="F05"),
Gnuplot.Data(results.coverage_summary(), title="Coverage"))
g.hardcopy(log_file + ".png", terminal="png")
g.hardcopy(log_file + ".ps", terminal="postscript", enhanced=1, color=1)
g('set logscale x')
g('replot')
g.hardcopy(log_file + "-logscale.png", terminal="png")
g.hardcopy(log_file + "-logscale.ps",
terminal="postscript", enhanced=1, color=1)
def plot_roc(results, log_file):
g = Gnuplot.Gnuplot()
g('set style data lines')
g.xlabel('False Positive Rate')
g.ylabel('True Positive Rate')
g('set xrange [0:1.0]')
g('set yrange [0:1.0]')
g.title("Setup: %s" % log_file.split("/")[-1])
g('set label "C %.2f" at 0.8,0.25' % results.coverage())
g('set label "AUC %.2f" at 0.8,0.2' % results.get_auc())
g('set label "P(10) %.2f" at 0.8,0.15' % numpy.mean(results.precision[10]))
g('set label "P(20) %.2f" at 0.8,0.10' % numpy.mean(results.precision[20]))
g('set label "F05(100) %.2f" at 0.8,0.05' % numpy.mean(results.f05[100]))
g.plot(Gnuplot.Data(results.get_roc_points(), title="ROC"),
Gnuplot.Data([[0, 0], [1, 1]], with_="lines lt 7"))
g.hardcopy(log_file + "-roc.png", terminal="png")
g.hardcopy(log_file + "-roc.ps",
terminal="postscript", enhanced=1, color=1)
def get_label(cfg):
label = {}
if cfg.strategy in content_based:
label["description"] = "strategy-profile"
label["values"] = ("%s-profile%.3d" %
(cfg.strategy, cfg.profile_size))
elif cfg.strategy in collaborative:
label["description"] = "strategy-knn"
label["values"] = ("%s-k%.3d" %
(cfg.strategy, cfg.k_neighbors))
elif cfg.strategy in hybrid:
label["description"] = "strategy-knn-profile"
label["values"] = ("%s-k%.3d-profile%.3d" %
(cfg.strategy, cfg.k_neighbors, cfg.profile_size))
return label
class ExperimentResults:
def __init__(self, repo_size):
self.repository_size = repo_size
self.precision = {}
self.recall = {}
self.fpr = {}
self.f05 = {}
self.recommended = {}
self.thresholds = [1] + range(10, self.repository_size, 10)
for size in self.thresholds:
self.precision[size] = []
self.recall[size] = []
self.fpr[size] = []
self.f05[size] = []
self.recommended[size] = set()
def add_result(self, ranking, sample):
for size in self.thresholds:
recommendation = ranking[:size]
self.recommended[size] = self.recommended[
size].union(recommendation)
predicted = RecommendationResult(dict.fromkeys(recommendation, 1))
real = RecommendationResult(sample)
evaluation = Evaluation(predicted, real, self.repository_size)
print evaluation.run(Precision())
self.precision[size].append(evaluation.run(Precision()))
self.recall[size].append(evaluation.run(Recall()))
self.f05[size].append(evaluation.run(F_score(0.5)))
self.fpr[size].append(evaluation.run(FPR()))
def precision_summary(self):
return [[size, numpy.mean(self.precision[size])]
for size in self.thresholds]
def recall_summary(self):
return [[size, numpy.mean(self.recall[size])]
for size in self.thresholds]
def f05_summary(self):
return [[size, numpy.mean(self.f05[size])] for size in self.thresholds]
def coverage_summary(self):
return [[size, self.coverage(size)] for size in self.thresholds]
def coverage(self, size=0):
if not size:
size = self.thresholds[-1]
return len(self.recommended[size]) / float(self.repository_size)
def precision(self, size):
return numpy.mean(self.precision[size])
def get_auc(self):
roc_points = self.get_roc_points()
x_roc = [p[0] for p in roc_points]
y_roc = [p[1] for p in roc_points]
x_roc.insert(0, 0)
y_roc.insert(0, 0)
x_roc.append(1)
y_roc.append(1)
return numpy.trapz(y=y_roc, x=x_roc)
# Average ROC by threshold (= size of recommendation)
def get_roc_points(self):
points = []
for size in self.recall.keys():
tpr = self.recall[size]
fpr = self.fpr[size]
points.append([sum(fpr) / len(fpr), sum(tpr) / len(tpr)])
return sorted(points)
def run_strategy(cfg, user):
rec = Recommender(cfg)
repo_size = rec.items_repository.get_doccount()
results = ExperimentResults(repo_size)
label = get_label(cfg)
user_dir = ("results/roc-suite/%s/%s" % (user.user_id[:8], cfg.strategy))
if not os.path.exists(user_dir):
os.makedirs(user_dir)
log_file = os.path.join(user_dir, label["values"])
for n in range(iterations):
# Fill sample profile
profile_len = len(user.pkg_profile)
item_score = {}
for pkg in user.pkg_profile:
item_score[pkg] = user.item_score[pkg]
sample = {}
sample_size = int(profile_len * 0.9)
for i in range(sample_size):
key = random.choice(item_score.keys())
sample[key] = item_score.pop(key)
iteration_user = User(item_score)
recommendation = rec.get_recommendation(iteration_user, repo_size)
write_recall_log(
label, n, sample, recommendation, profile_len, repo_size, log_file)
if hasattr(recommendation, "ranking"):
results.add_result(recommendation.ranking, sample)
with open(log_file + "-roc.jpg.comment", 'w') as f:
f.write("# %s\n# %s\n\n" %
(label["description"], label["values"]))
f.write("# roc AUC\n%.4f\n\n" % results.get_auc())
f.write("# threshold\tprecision\trecall\t\tf05\t\tcoverage\n")
for size in results.thresholds:
f.write("%4d\t\t%.4f\t\t%.4f\t\t%.4f\t\t%.4f\n" %
(size, numpy.mean(results.precision[size]),
numpy.mean(results.recall[size]),
numpy.mean(results.f05[size]),
numpy.mean(results.coverage(size))))
shutil.copy(log_file + "-roc.jpg.comment", log_file + ".jpg.comment")
shutil.copy(log_file + "-roc.jpg.comment",
log_file + "-logscale.jpg.comment")
plot_roc(results, log_file)
plot_summary(results, log_file)
def run_content(user, cfg):
for size in profile_size:
cfg.profile_size = size
run_strategy(cfg, user)
def run_collaborative(user, cfg):
for k in neighbors:
cfg.k_neighbors = k
run_strategy(cfg, user)
def run_hybrid(user, cfg):
for k in neighbors:
cfg.k_neighbors = k
for size in profile_size:
cfg.profile_size = size
run_strategy(cfg, user)
if __name__ == '__main__':
if len(sys.argv) < 2:
print "Usage: roc-suite strategy_str [popcon_submission_path]"
exit(1)
# iterations = 3
# content_based = ['cb']
# collaborative = ['knn_eset']
# hybrid = ['knnco']
# profile_size = [50,100]
# neighbors = [50]
iterations = 20
content_based = ['cb', 'cbt', 'cbd', 'cbh',
'cb_eset', 'cbt_eset', 'cbd_eset', 'cbh_eset']
collaborative = ['knn_eset', 'knn', 'knn_plus']
hybrid = ['knnco', 'knnco_eset']
profile_size = [10, 20, 40, 60, 80, 100, 140, 170, 200, 240]
neighbors = [3, 5, 10, 20, 30, 50, 70, 100, 150, 200]
cfg = Config()
cfg.strategy = sys.argv[1]
# user =
# PopconSystem("/root/.app-recommender/popcon-entries/4a/4a67a295ec14826db2aa1d90be2f1623")
user = PopconSystem(
"/root/.app-recommender/popcon-entries/8b/8b44fcdbcf676e711a153d5db09979d7") # noqa
# user = PopconSystem(sys.argv[1])
user.filter_pkg_profile(cfg.pkgs_filter)
user.maximal_pkg_profile()
if cfg.strategy in content_based:
run_content(user, cfg)
if cfg.strategy in collaborative:
run_collaborative(user, cfg)
if cfg.strategy in hybrid:
run_hybrid(user, cfg)
AppRecommender-0.7.5/apprecommender/experiments/sample-popcon-arch.py 0000775 0000000 0000000 00000002712 13067513116 0026042 0 ustar 00root root 0000000 0000000 #! /usr/bin/env python
"""
sample-popcon-arch - extract a sample of a specific arch
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import sys
from apprecommender.user import RandomPopcon
if __name__ == '__main__':
try:
size = int(sys.argv[1])
arch = sys.argv[2]
popcon_dir = sys.argv[3]
pkgs_filter = sys.argv[4]
except:
print "Usage: sample-popcon-arch size arch popcon_dir pkgs_filter"
exit(1)
sample_file = ("results/misc-popcon/sample-%s-%d" % (arch, size))
with open(sample_file, 'w') as f:
for n in range(1, size + 1):
user = RandomPopcon(popcon_dir, arch, pkgs_filter)
f.write(user.user_id + '\n')
print "sample", n
AppRecommender-0.7.5/apprecommender/experiments/sample-popcon.py 0000775 0000000 0000000 00000004056 13067513116 0025132 0 ustar 00root root 0000000 0000000 #! /usr/bin/env python
"""
sample-popcon - extract a sample from popcon population
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import xapian
import os
import sys
def extract_sample(size, popcon, min_profile, max_profile, output):
sample = []
for n in range(1, popcon.get_doccount() + 1):
user = popcon.get_document(n)
pkgs_profile = [
t.term for t in user.termlist() if t.term.startswith("XP")]
print len(pkgs_profile)
if(len(pkgs_profile) > min_profile and
len(pkgs_profile) <= max_profile):
sample.append(user.get_data())
print n, len(sample)
if len(sample) == size:
break
with open(("%s-%d-%d" % (output, min_profile, max_profile)), 'w') as f:
for s in sample:
f.write(s + '\n')
if __name__ == '__main__':
popcon = xapian.Database(
os.path.expanduser("~/.app-recommender/popcon_desktopapps"))
print ("Popcon repository size: %d" % popcon.get_doccount())
try:
min_profile = int(sys.argv[1])
max_profile = int(sys.argv[2])
size = int(sys.argv[3])
except:
print "Usage: sample-popcon min_profile max_profile sample_size"
exit(1)
sample_file = "results/misc-popcon/sample"
extract_sample(size, popcon, min_profile, max_profile, sample_file)
AppRecommender-0.7.5/apprecommender/generate_doc.sh 0000775 0000000 0000000 00000002246 13067513116 0022412 0 ustar 00root root 0000000 0000000 #!/bin/env bash
#
# generate_doc.sh - shell script to generate documentation using doxygen.
#
# Copyright (C) 2010 Tassia Camoes
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see .
# Get project version from git repository
TAG=$(git describe --tags --abbrev=0)
echo "Generating documentation for git tag $TAG"
sed -i "s/^PROJECT_NUMBER.*$/PROJECT_NUMBER\t\t= $TAG/" ../doc/doxy_config
rm -Rf ../doc/html
../doc/doxygen-1.7.3 ../doc/doxy_config
scp -r html/ tassia@eclipse.ime.usp.br:
echo "---> Remember to place doc in the right location on server side."
mv html/ ../doc/
AppRecommender-0.7.5/apprecommender/initialize.py 0000664 0000000 0000000 00000011343 13067513116 0022145 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import commands
import data
import datetime
import os
import shutil
import xapian
from apprecommender.config import Config
from apprecommender.decider import PkgInitDecider
class Initialize:
AXI_SAMPLES = ['sample', 'filter']
DEFAULT_AXI_PATH = "/var/lib/apt-xapian-index/index"
EXCLUDED_TAGS = ['culture::', 'devel::lang', 'hardware::',
'implemented-in::', 'interface::', 'iso15924::',
'made-of::', 'network::', 'protocol::', 'role::',
'scope::', 'secteam::', 'special::', 'uitoolkit::',
'x11::', 'TODO']
def __init__(self):
self.config = Config()
self.pkg_init_decider = PkgInitDecider()
def get_tags(self):
command = "cat /var/lib/debtags/vocabulary" \
"| grep 'Tag:'" \
"| egrep -v '%s'" \
"| awk '{print $2}'" % '|'.join(Initialize.EXCLUDED_TAGS)
tags = commands.getoutput(command).splitlines()
return tags
def get_axipkgs(self, axi_tag='XP', axi_path=DEFAULT_AXI_PATH):
axi = xapian.Database(axi_path)
all_terms = set()
for n in range(1, axi.get_lastdocid()):
doc = 0
try:
doc = axi.get_document(n)
except:
pass
if doc:
xp_terms = None
for t in doc.termlist():
if t.term.startswith(axi_tag):
xp_terms = t.term
break
if xp_terms:
pkg_name = xp_terms.lstrip(axi_tag)
if pkg_name.startswith('M'):
pkg_name = pkg_name.lstrip('M')
if self.pkg_init_decider(pkg_name):
all_terms.add(pkg_name)
return all_terms
def indexer_axi(self, axi_sample, filters_path, terms=[]):
axi_path = Initialize.DEFAULT_AXI_PATH
axi = xapian.Database(axi_path)
base_dir = self.config.base_dir
begin_time = datetime.datetime.now()
# axi sample based on the pkgs sample provided by command line
if axi_sample is 'sample':
with open(filters_path) as valid:
pkgs_list = [line.strip() for line in valid]
filter_str = 'axi_' + filters_path.split('/')[-1]
print"\nIndexing packages on xapian"
index = data.SampleAptXapianIndex(pkgs_list, axi,
os.path.join(base_dir,
filter_str))
print "Axi size: %d" % axi.get_doccount()
print "Packages list length: %d" % len(pkgs_list)
print "Sample index size: %d" % index.get_doccount()
# axi filtered by terms provided by command line
if axi_sample is "filter":
terms_str = "_".join([t.split("::")[-1] for t in terms])
index = data.FilteredXapianIndex(terms, axi,
os.path.join(base_dir,
"axi_" + terms_str))
print "Axi size: %d" % axi.get_doccount()
print "Terms filter: %s" % terms
print "Filtered index size: %d" % index.get_doccount()
end_time = datetime.datetime.now()
print "Indexing completed at %s" % end_time
delta = end_time - begin_time
print "Time elapsed: %d seconds." % delta.seconds
def prepare_data(self):
try:
if os.path.exists(self.config.base_dir):
if os.path.exists(self.config.user_data_dir):
shutil.rmtree(self.config.user_data_dir)
if os.path.exists(self.config.axi_desktopapps):
shutil.rmtree(self.config.axi_desktopapps)
if os.path.exists(self.config.filters_dir):
shutil.rmtree(self.config.filters_dir)
os.makedirs(self.config.filters_dir)
except OSError:
raise
tags = self.get_tags()
tags_path = "{}/debtags".format(self.config.filters_dir)
self.save_list(tags, tags_path)
pkgs = self.get_axipkgs()
pkgs_path = "{}/desktopapps".format(self.config.filters_dir)
self.save_list(pkgs, pkgs_path)
self.indexer_axi('sample', pkgs_path)
def get_role_program_pkgs(self):
command = "cat /var/lib/debtags/package-tags | " \
"grep 'role::program' | " \
"awk -F: '{ print $1}'"
pkgs = commands.getoutput(command).splitlines()
return pkgs
def save_list(self, elements, path):
with open(path, 'w') as text:
text.write('\n'.join(elements) + '\n')
AppRecommender-0.7.5/apprecommender/main/ 0000775 0000000 0000000 00000000000 13067513116 0020354 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/apprecommender/main/__init__.py 0000664 0000000 0000000 00000000000 13067513116 0022453 0 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/apprecommender/main/app_recommender.py 0000664 0000000 0000000 00000002306 13067513116 0024067 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import logging
import datetime
from apprecommender.recommender import Recommender
from apprecommender.user import LocalSystem
from apprecommender.config import Config
class AppRecommender:
def __init__(self):
self.recommender = Recommender()
self.config = Config()
def make_recommendation(self, reference_pkgs=None,
print_recommendation=True):
begin_time = datetime.datetime.now()
logging.info("Computation started at %s" % begin_time)
if not reference_pkgs:
reference_pkgs = []
user = LocalSystem(reference_pkgs)
recommendation_size = Config().num_recommendations
user_recommendation = (self.recommender.get_recommendation(
user, recommendation_size))
logging.info("Recommending applications for user %s" % user.user_id)
if print_recommendation:
print(user_recommendation)
end_time = datetime.datetime.now()
logging.info("Computation completed at %s" % end_time)
delta = end_time - begin_time
logging.info("Time elapsed: %d seconds." % delta.seconds)
return user_recommendation
AppRecommender-0.7.5/apprecommender/main/apt_run.py 0000664 0000000 0000000 00000006522 13067513116 0022403 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import argparse
import os
import shutil
import re
from apprecommender.config import Config
from apprecommender.main.app_recommender import AppRecommender
from apprecommender.user import LocalSystem
class AptRun:
def __init__(self):
apt_folder = os.path.join(Config().base_dir, "apt_run/")
self.set_folder(apt_folder)
def set_folder(self, folder_path):
folder_path = os.path.expanduser(folder_path)
folder_path = os.path.abspath(folder_path)
if folder_path[-1] != '/':
folder_path += '/'
self.apt_folder = folder_path
self.installed_pkgs_file = self.apt_folder + "installed_pkgs.txt"
def enable(self):
if not self.is_enable():
os.makedirs(self.apt_folder)
return True
return False
def disable(self):
if self.is_enable():
shutil.rmtree(self.apt_folder)
return True
return False
def is_enable(self):
return os.path.exists(self.apt_folder)
def get_user_pkgs(self):
user = LocalSystem()
user_pkgs = user.pkg_profile
return user_pkgs
def pre_install_pkgs(self):
if self.is_enable():
user_pkgs = self.get_user_pkgs()
with open(self.installed_pkgs_file, 'w') as text:
text.write("\n".join(user_pkgs))
def post_invoke(self):
if not self.is_enable():
return []
with open(self.installed_pkgs_file, 'r') as text:
pre_installed_pkgs = set([line.strip() for line in text])
pkgs = self.get_user_pkgs()
pos_installed_pkgs = set([pkg.strip() for pkg in pkgs])
installed_pkgs = list(pos_installed_pkgs - pre_installed_pkgs)
return installed_pkgs
def get_recommendation_pkgs(self, installed_pkgs):
app_recommender = AppRecommender()
app_recommender.recommender.set_strategy('cbpkg')
rec = app_recommender.make_recommendation(
reference_pkgs=installed_pkgs, print_recommendation=False)
pkgs_regex = re.compile(r'^\d:\s([^\s]+)', re.MULTILINE)
pkgs = pkgs_regex.findall(rec.__str__())
return pkgs
def make_recommendations(self, installed_pkgs):
if len(installed_pkgs) > 0:
pkgs = self.get_recommendation_pkgs(installed_pkgs)
if len(pkgs) > 0:
print '\nApprecommeder: The following packages are' \
' interesting'
for pkg in pkgs:
print ' - {}'.format(pkg)
def get_args():
aptrun_description = 'Integration of AppRecommender with apt'
parser = argparse.ArgumentParser(description=aptrun_description)
parser.add_argument(
'--pre-install-pkgs',
help='Indentify installed packages before install the new packages',
action='store_true')
parser.add_argument(
'--post-invoke',
help='Indentify the installed packages and makes recommendation',
action='store_true')
args = vars(parser.parse_args())
return args
def main():
args = get_args()
apt_run = AptRun()
if args['pre_install_pkgs']:
apt_run.pre_install_pkgs()
elif args['post_invoke']:
installed_pkgs = apt_run.post_invoke()
apt_run.make_recommendations(installed_pkgs)
if __name__ == '__main__':
main()
AppRecommender-0.7.5/apprecommender/main/cli.py 0000664 0000000 0000000 00000010064 13067513116 0021476 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import xapian
from apprecommender.main.app_recommender import AppRecommender
from apprecommender.config import Config
from apprecommender.initialize import Initialize
from apprecommender.strategy import (MachineLearning, MachineLearningBVA,
MachineLearningBOW,
MachineLearningTrainError)
from apprecommender.main import collect_user_data
from apprecommender.main import show_classifications
from apprecommender.main.apt_run import AptRun
from apprecommender.main.options import get_parser
SUCCESS = 0
ERROR_INIT = 1
ERROR_TRAIN = 2
PERMISSION_DENIED = 3
ERROR_INIT_TRAIN = 4
def parse_options(args, config):
if args['strategy']:
config.strategy = args['strategy']
if args['debug']:
config.debug = 1
if args['verbose']:
config.verbose = 1
if args['profile_size']:
config.profile_size = args['profile_size']
if args['because']:
config.because = True
if args['num_recommendations']:
config.num_recommendations = args['num_recommendations']
def run_apprecommender(reference_pkgs):
try:
app_recommender = AppRecommender()
app_recommender.make_recommendation(reference_pkgs)
return SUCCESS
except xapian.DatabaseOpeningError:
return ERROR_INIT
except IOError:
if "ml" in Config().strategy:
return ERROR_TRAIN
except OSError:
return PERMISSION_DENIED
def run_initialize():
print "Initializing AppRecommender"
initialize = Initialize()
try:
initialize.prepare_data()
except OSError:
return PERMISSION_DENIED
return SUCCESS
def run_train():
print "Training machine learning"
try:
MachineLearning.train(MachineLearningBVA)
MachineLearning.train(MachineLearningBOW)
except IOError:
return PERMISSION_DENIED
except MachineLearningTrainError:
return ERROR_INIT_TRAIN
return SUCCESS
def run(args):
if args['update']:
init_result = run_initialize()
if init_result != SUCCESS:
return init_result
train_result = run_train()
if train_result != SUCCESS:
return train_result
return SUCCESS
elif args['init']:
return run_initialize()
elif args['train']:
return run_train()
elif args['contribute']:
collect_user_data.main()
elif args['show_classifications']:
show_classifications.main()
elif args['enable_apt']:
try:
apt_run = AptRun()
if apt_run.enable():
print 'AppRecommender now makes recommendations when you ' \
' install new packages with apt'
else:
print 'This is already enabled'
return SUCCESS
except OSError:
return PERMISSION_DENIED
elif args['disable_apt']:
try:
apt_run = AptRun()
if apt_run.disable():
print 'AppRecommender now dont makes recommendations when' \
'you install new packages with apt'
else:
print 'This is already disabled'
return SUCCESS
except OSError:
return PERMISSION_DENIED
else:
config = Config()
parse_options(args, config)
reference_pkgs = args['packages']
return run_apprecommender(reference_pkgs)
def main():
parser = get_parser()
args = vars(parser.parse_args())
result = run(args)
if result is ERROR_INIT:
print "\n"
print "Please, Initialize AppRecommender"
print "Run: apprec.py --init"
elif result is ERROR_TRAIN:
print "\n"
print "Please, run Machine Learning Training"
print "Run: apprec.py --train"
elif result is PERMISSION_DENIED:
print "Please, run this command as sudo"
elif result is ERROR_INIT_TRAIN:
print 'Error: You need install more packages to use machine' \
' learning recommendations'
if __name__ == '__main__':
main()
AppRecommender-0.7.5/apprecommender/main/collect_user_data.py 0000664 0000000 0000000 00000030314 13067513116 0024403 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import binascii
import commands
import datetime as dt
import logging
import os
import re
import subprocess
import tarfile
import time
import xapian
from apprecommender.apt_cache import AptCache
from apprecommender.config import Config
from apprecommender.data import get_user_installed_pkgs
from apprecommender.data_classification import get_alternative_pkg
from apprecommender.main.app_recommender import AppRecommender
from apprecommender.main.ml_cross_validation import ml_cross_validation
from apprecommender.ml.data import MachineLearningData
from apprecommender.ml.pkg_time import PkgTime
from apprecommender.utils import print_progress_bar
LOG_PATH = os.path.expanduser('~/app_recommender_log')
SUFIX = dt.datetime.now().strftime('%Y%m%d%H%M')
LOG_PATH += SUFIX
ALL_INSTALLED_PKGS = LOG_PATH + '/all_pkgs.txt'
MANUAL_INSTALLED_PKGS_PATH = LOG_PATH + '/manual_installed_pkgs.txt'
PKGS_TIME_PATH = LOG_PATH + '/pkgs_time.txt'
HISTORY = LOG_PATH + '/user_history.txt'
PKGS_BINARY = LOG_PATH + '/pkgs_binary.txt'
RECOMMENDATION_PATH = LOG_PATH + '/{0}_recommendation.txt'
USER_PREFERENCES = LOG_PATH + '/user_preferences.txt'
POPCON_SUBMISSION = LOG_PATH + '/popcon-submission'
PC_INFORMATIONS = LOG_PATH + '/pc_informations.txt'
RECOMMENDATIONS_TIME = LOG_PATH + '/recommendations_time.txt'
PKGS_DEPENDENCIES = []
RE_MATCH = re.compile(r'^\d:\s([^\s]+)', re.MULTILINE)
def create_log_folder():
print "Creating log folder"
if not os.path.exists(LOG_PATH):
os.mkdir(LOG_PATH, 0755)
def create_file(file_path):
if not os.path.exists(file_path):
with open(file_path, 'a'):
os.utime(file_path, None)
return True
return False
def delete_file(file_path):
if os.path.exists(file_path):
os.remove(file_path)
def save_list(list_to_save, file_path):
delete_file(file_path)
create_file(file_path)
with open(file_path, 'w') as text:
for element in list_to_save:
text.write(str(element) + '\n')
def rename_file(original_name, new_name):
os.rename(original_name, new_name)
def get_submission_id(submission_header):
fields = submission_header.split(' ')
return fields[2][3:]
def create_popularity_contest_file():
if os.path.exists('popularity-contest.conf'):
return
host_id = binascii.hexlify(os.urandom(16))
with open('popularity-contest.conf', 'w') as text:
text.write('MY_HOSTID="{0}"\n'.format(host_id))
text.write('PARTICIPATE="no"\n')
text.write('USEHTTP="no"\n')
text.write('DAY="4"\n')
def collect_popcon_submission():
print "Collecting popularity-contest submission"
create_popularity_contest_file()
popcon = subprocess.Popen('./popularity-contest',
shell=False, stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
popcon_output = popcon.stdout.read()
popcon_parse = popcon_output.splitlines()
submission_id = get_submission_id(popcon_parse[0])
submission = [line for line in popcon_parse]
save_list(submission, POPCON_SUBMISSION)
file_name = LOG_PATH + "/" + submission_id + ".txt"
rename_file(POPCON_SUBMISSION, file_name)
def collect_manual_installed_pkgs():
print "Collecting manual installed pkgs"
if create_file(MANUAL_INSTALLED_PKGS_PATH):
packages = commands.getoutput('apt-mark showmanual')
packages = [pkg for pkg in packages.splitlines()]
save_list(packages, MANUAL_INSTALLED_PKGS_PATH)
def collect_all_user_pkgs():
print "Collecting all user packages"
if create_file(ALL_INSTALLED_PKGS):
packages = get_user_installed_pkgs()
save_list(packages, ALL_INSTALLED_PKGS)
def collect_pkgs_time():
print "Collecting packages time"
pkg_time = PkgTime()
if create_file(PKGS_TIME_PATH):
manual_pkgs = []
with open(MANUAL_INSTALLED_PKGS_PATH, 'r') as text:
manual_pkgs = [line.strip() for line in text]
pkgs_time = pkg_time.get_packages_time(manual_pkgs)
pkg_time.save_package_time(pkgs_time, PKGS_TIME_PATH)
def get_pkg_binary(pkg):
stat_command = "which {0}".format(pkg)
pkg_bin = commands.getoutput(stat_command.format(pkg))
if pkg_bin:
return pkg
return get_alternative_pkg(pkg)
def get_pkgs_of_recommendation(strategy):
app_recommender = AppRecommender()
app_recommender.recommender.set_strategy(strategy)
rec = app_recommender.make_recommendation(print_recommendation=False)
pkgs = RE_MATCH.findall(rec.__str__())
return pkgs
def collect_user_preferences():
Config().num_recommendations = 6
strategies = ['cb', 'cb_eset', 'cbtm', 'mlbva', 'mlbva_eset', 'mlbow',
'mlbow_eset']
recommendations = {}
recommendations_time = []
print "Preparing recommendations..."
len_strategies = len(strategies)
for index, strategy in enumerate(strategies):
first_time = int(round(time.time() * 1000))
recommendations[strategy] = get_pkgs_of_recommendation(strategy)
last_time = int(round(time.time() * 1000))
recommendations_time.append("{0}: {1}".format(strategy,
last_time - first_time))
print_progress_bar(index + 1, len_strategies)
all_recommendations = set(sum(recommendations.values(), []))
all_recommendations = sorted(list(all_recommendations))
index = 0
user_preferences = {}
all_rec_len = len(all_recommendations)
message = "\n\nPackage [{0}/{1}] - {2} \n"
message += "Description: {3}\n\n"
message += "Rank a package recommendation with 1-4\n"
message += "1 - Bad\n"
message += "2 - Redundant\n"
message += "3 - Useful\n"
message += "4 - Useful Surprise\n\n"
message += "exit - Cancel collect data\n\n"
message += "Rank: "
message_error = "\nPlease use digits 1-4 to rank a package: "
apt_cache = AptCache()
for i in range(len(all_recommendations)):
pkg = all_recommendations[i]
pkg_description = apt_cache[pkg].versions[0].description
rank = -1
raw_message = message.format((index + 1), all_rec_len, pkg,
pkg_description)
clear_prints()
print "\n\nCollecting user preferences"
if i > 0:
print "\n"
for j in range(i):
prev_pkg = all_recommendations[j]
print(
"[{0}/{1}] {2}, rank: {3}".format(j + 1, all_rec_len, prev_pkg,
user_preferences[prev_pkg]))
while rank < 1 or rank > 4:
try:
rank = raw_input(raw_message)
if rank == 'exit':
break
rank = int(rank)
except:
rank = -2
raw_message = message_error
if rank == 'exit':
exit(2)
user_preferences[pkg] = rank
index += 1
preferences_list = ["{0}:{1}".format(package, user_preferences[package])
for package in all_recommendations]
for rec_key, rec_value in recommendations.iteritems():
save_list(rec_value, RECOMMENDATION_PATH.format(rec_key))
save_list(preferences_list, USER_PREFERENCES)
save_list(recommendations_time, RECOMMENDATIONS_TIME)
def get_uninstalled_dependencies():
user_pkgs = []
unistalled_pkgs = []
user_pkgs = get_user_installed_pkgs()
for pkg in PKGS_DEPENDENCIES:
if pkg not in user_pkgs:
unistalled_pkgs.append(pkg)
return unistalled_pkgs
def check_dependencies():
unistalled_pkgs = get_uninstalled_dependencies()
unistalled_dependencies = ''
if len(unistalled_pkgs) > 0:
unistalled_dependencies = ''.join(str(pkg) + ' '
for pkg in unistalled_pkgs)
return unistalled_dependencies
def collect_pc_informations():
print "Collecting PC informations"
informations = []
linux_kernel_version = commands.getoutput('uname -s -r')
distribution_version = commands.getoutput('lsb_release -a')
distribution_version = distribution_version.splitlines()
processor = commands.getoutput("cat /proc/cpuinfo | grep 'model name'")
processor = processor.splitlines()[0].split(':')[1].strip()
processor = "Processor: {0}".format(processor)
informations.append(linux_kernel_version)
informations.extend(distribution_version)
informations.append(processor)
save_list(informations, PC_INFORMATIONS)
def collect_user_data():
collect_pc_informations()
collect_all_user_pkgs()
collect_manual_installed_pkgs()
collect_pkgs_time()
collect_popcon_submission()
def initial_prints():
experiment_agree = """
Participate in this package survey?
This survey has the objective to collect user feedback to allow the
comparison between some new strategies for AppRecommender. This survey
will them present a series of recommended packages that can
be evaluates as follows:
Bad: A package that you would not install on your system.
Redundant: You already have a package that provides the same
functionality as the one being recommended.
Useful: A package that you may install on you system.
Useful Surprise: A package that you would install on your system, but
the recommendatio also surprised you in a positive
way.
This survey will also collect the result of some cross validation
metrics generated for the AppRecommender strategies that use machine
learning.
Therefore, this survey will collect the following data:
* A list of your evaluations for the recommended packages.
* The result of the cross validation metrics for AppRecommender machine
learning strategies.
* The time in seconds that it took for each strategies to genarate its
recommendations
Once this process is over, a tar.gz file will be generated on your HOME
dir. After that just send this file over to:
lucas.moura128@gmail.com
lucianopcbr@gmail.com
terceiro@debian.org
"""
print experiment_agree
def user_accept_collect_data():
accept_message = "\nDo you want to participate on this survey ? [y, N]: "
accept_input = raw_input(accept_message)
return accept_input.lower() == 'y'
def clear_prints():
print '\n' * 80
os.system('clear')
def train_machine_learning():
try:
print "Training machine learning"
os.system("cp {} {}".format(
MachineLearningData.PKGS_CLASSIFICATIONS, LOG_PATH))
except xapian.DatabaseOpeningError:
print "\n\nPlease check if you prepared the AppRecommender data"
print "Try to run the following commands:"
print " $ cd .."
print " $ sudo apprec --init\n"
exit(1)
except IOError:
print "\n\nPlease run the train command before executing this script:"
print " $ sudo apprec --train\n"
def run_cross_validation():
print "Collecting cross validations"
strategies = ['mlbva', 'mlbva_eset', 'mlbow', 'mlbow_eset']
len_strategies = len(strategies)
for index, strategy in enumerate(strategies):
ml_cross_validation(LOG_PATH, strategy)
print_progress_bar(index + 1, len_strategies)
def make_tarfile(output_filename, source_dir):
with tarfile.open(output_filename, "w:gz") as tar:
tar.add(source_dir, arcname=os.path.basename(source_dir))
def main():
logging.getLogger().disabled = True
initial_prints()
if not user_accept_collect_data():
exit(1)
create_log_folder()
train_machine_learning()
run_cross_validation()
collect_user_preferences()
make_tarfile(LOG_PATH + '.tar.gz', LOG_PATH)
subprocess.call(['rm', '-rf', LOG_PATH])
print "\n\nFinished: All files and recommendations were collected"
print "Collect data folder: {0}.tar.gz\n".format(LOG_PATH)
print "Please, send this file to either one of these emails:\n"
print "lucas.moura128@gmail.com, lucianopcbr@gmail.com, terceiro@debian.org" # noqa
if __name__ == '__main__':
main()
AppRecommender-0.7.5/apprecommender/main/ml_cross_validation.py 0000664 0000000 0000000 00000007234 13067513116 0024767 0 ustar 00root root 0000000 0000000 #!/usr/bin/python
import datetime as dt
import logging
import os
import pickle
import sys
import getopt
from apprecommender.ml.cross_validation import (CrossValidationBVA,
CrossValidationBOW)
from apprecommender.evaluation import (SimpleAccuracy, Precision, Recall, FPR,
F_score)
from apprecommender.ml.data import MachineLearningData
from apprecommender.ml.bag_of_words import BagOfWords
from apprecommender.config import Config
BASE_DIR = Config().base_dir
CROSS_VALIDATION_FOLDER = BASE_DIR + '/cross_validation_data/'
def get_strategy(ml_strategy_str, pkg_data, partition_size, rounds,
metrics_list, labels):
if ml_strategy_str == 'bow':
return CrossValidationBOW(
pkg_data, partition_size, rounds, metrics_list,
labels)
else:
return CrossValidationBVA(
pkg_data, partition_size, rounds, metrics_list,
labels)
def get_pkg_data(ml_strategy_str, ml_data, labels):
if ml_strategy_str == 'bow':
path = BagOfWords.BAG_OF_WORDS_PKGS_CLASSIFICATION
else:
path = MachineLearningData.PKGS_CLASSIFICATIONS
with open(path, 'ra') as pkgs_classification:
return pickle.load(pkgs_classification)
def ml_cross_validation(folder_path, ml_strategy_str):
logger = logging.getLogger('')
logger.setLevel(logging.CRITICAL)
if folder_path[-1] != '/':
folder_path += '/'
if not os.path.exists(folder_path):
os.mkdir(folder_path)
partition_size = 0.8
rounds = 5
metrics_list = [SimpleAccuracy(), Precision(), Recall(), FPR(), F_score(1)]
labels = ['RU', 'U', 'NU']
ml_data = MachineLearningData()
pkg_data = get_pkg_data(ml_strategy_str, ml_data, labels)
ml_cross_validation = get_strategy(
ml_strategy_str, pkg_data, partition_size, rounds,
metrics_list, labels)
cross_validaton_file = 'cross_validation_result_{}_{}_{}_{}.txt'.format(
ml_strategy_str, rounds, partition_size,
dt.datetime.now().strftime('%Y%m%d%H%M'))
ml_cross_validation.run(None)
cross_validation_file_path = folder_path + cross_validaton_file
with open(cross_validation_file_path, 'w') as result:
result.write(ml_cross_validation.__str__())
return ml_cross_validation
def print_help():
print "\n"
print "Usage: get_axipkgs"
print " -h, --help \t\t Show this help"
print " -s, --strategy \t Set machine_learning of one strategy" \
", exemple: bva, bow"
print "\n"
print " [ strategy options ] "
print " bva - run cross_validation from mlbva, Binary Vector" \
" Approach"
print " bow - run cross_validation from mlbow, Bag of Words"
if __name__ == '__main__':
short_options = "hdvo:s:"
long_options = ["help", "strategy"]
valid_strategies = ['bva', 'bow']
ml_strategy_str = None
try:
opts, args = getopt.getopt(sys.argv[1:], short_options,
long_options)
except getopt.GetoptError as error:
logging.error("Bad syntax: %s" % str(error))
print_help()
sys.exit()
for option, param in opts:
if option in ("-h", "--help"):
print_help()
sys.exit()
elif option in ("-s", "--strategy"):
ml_strategy_str = param
else:
print_help()
assert False, "unhandled option"
if ml_strategy_str in valid_strategies:
print ml_cross_validation(CROSS_VALIDATION_FOLDER, ml_strategy_str)
print("Cross validation results saved on: %s" %
(CROSS_VALIDATION_FOLDER))
else:
print_help()
AppRecommender-0.7.5/apprecommender/main/options.py 0000664 0000000 0000000 00000004307 13067513116 0022425 0 ustar 00root root 0000000 0000000 import argparse
def get_parser():
apprec_description = 'Package recommender system for Debian (and derived) \
distros'
parser = argparse.ArgumentParser(description=apprec_description)
parser.add_argument(
'-s', '--strategy',
help='select strategy to run apprecommender (default: cb)',
type=str)
parser.add_argument(
'-d', '--debug',
help='run apprecommender on debug mode',
type=int)
parser.add_argument(
'-v', '--verbose',
help='run apprecommender on verbose mode',
type=int)
parser.add_argument(
'-z', '--profile-size',
help='set the profile size of an user on apprecommender',
type=int)
parser.add_argument(
'-i', '--init',
help='initialize apprecommender database',
action='store_true')
parser.add_argument(
'-t', '--train',
help='train machine learning algorithms',
action='store_true')
parser.add_argument(
'-b', '--because',
help="display which user's packages generated a recommendation",
action='store_true')
parser.add_argument(
'-n', '--num-recommendations',
help='set the number of packages that will be recommended',
type=int)
parser.add_argument(
'-c', '--contribute',
help='classify recommendations and help apprecommender to improve',
action='store_true')
parser.add_argument(
'-p', '--packages',
help="Add reference package for strategy 'cbpkg'",
type=str, nargs='+', default=[])
parser.add_argument(
'--show-classifications',
help='Show the user classifications for machine learning algorithms',
action='store_true')
parser.add_argument(
'-e', '--enable-apt',
help='Enable recommendations when install a package with apt',
action='store_true')
parser.add_argument(
'-r', '--disable-apt',
help='Disable recommendations when install a package with apt',
action='store_true')
parser.add_argument(
'--update',
help='Run both init and train commands',
action='store_true')
return parser
AppRecommender-0.7.5/apprecommender/main/show_classifications.py 0000664 0000000 0000000 00000002232 13067513116 0025143 0 ustar 00root root 0000000 0000000 #!/usr/bin/python
import os
import pickle
from apprecommender.ml.pkg_time import PkgTime
from apprecommender.config import Config
def main():
path = Config().user_data_dir + 'pkgs_classifications.txt'
if not os.path.exists(path):
print 'Could not find file pkgs_classification'
print 'Have you run apprec --train ?'
exit(-1)
pkg_time = PkgTime()
pkgs_times = pkg_time.get_package_data()
with open(path, 'ra') as data:
pkg_classification = pickle.load(data)
classifications = {'RU': [], 'U': [], 'NU': []}
for pkg, values in pkg_classification.iteritems():
classifications[values[-1]].append(pkg)
for classification, pkgs in classifications.iteritems():
print '\n'
print 'Classification: {}'.format(classification)
print '\n'
for pkg in sorted(pkgs):
pkg_text = '{} \t'
if len(pkg) < 15:
pkg_text += '\t'
if len(pkg) < 7:
pkg_text += '\t'
pkg_text += ' {}'
print pkg_text.format(pkg, pkgs_times[pkg][1].strip())
print '\nNum pkgs: {}'.format(len(pkg_classification))
AppRecommender-0.7.5/apprecommender/ml/ 0000775 0000000 0000000 00000000000 13067513116 0020040 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/apprecommender/ml/__init__.py 0000664 0000000 0000000 00000000000 13067513116 0022137 0 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/apprecommender/ml/bag_of_words.py 0000664 0000000 0000000 00000012177 13067513116 0023055 0 ustar 00root root 0000000 0000000 import os
import pickle
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import GaussianNB
from apprecommender.apt_cache import AptCache
from apprecommender.config import Config
from apprecommender.ml.data import MachineLearningData
class BagOfWords():
USER_DATA_DIR = Config().user_data_dir
BAG_OF_WORDS_DIR = USER_DATA_DIR + 'bag_of_words/'
BAG_OF_WORDS_MODEL = BAG_OF_WORDS_DIR + 'bag_of_words_model.pickle'
BAG_OF_WORDS_TERMS = BAG_OF_WORDS_DIR + 'bag_of_words_terms.pickle'
BAG_OF_WORDS_DEBTAGS = BAG_OF_WORDS_DIR + 'bag_of_words_debtags.pickle'
BAG_OF_WORDS_PKGS_CLASSIFICATION = BAG_OF_WORDS_DIR + \
'bow_pkgs_classification.pickle'
MODEL_ALREADY_CREATED = 1
CREATED_MODEL = 0
@staticmethod
def save(bag_of_words, file_path):
with open(file_path, 'wb') as text:
pickle.dump(bag_of_words, text)
@staticmethod
def load(file_path):
with open(file_path, 'rb') as text:
bag_of_words = pickle.load(text)
return bag_of_words
def __init__(self):
self.vectorizer = TfidfVectorizer(
max_df=0.8,
max_features=5000,
min_df=5,
stop_words='english',
use_idf=True)
def check_dir(self):
return os.path.exists(BagOfWords.BAG_OF_WORDS_DIR)
def combine_pkg_info(self, description, debtags, section):
description.extend(debtags)
description.append(section)
return description
def classify_pkg(self, attribute_vector, transform=True):
if transform:
pkg_feature = self.vectorizer.transform([attribute_vector])
pkg_feature = pkg_feature.toarray()
else:
pkg_feature = attribute_vector
label = self.classifier.predict(pkg_feature)
return label[0]
def create_pkg_data(self, pkg, axi, cache, ml_data):
description = self.get_pkg_description(pkg, cache, ml_data)
debtags = self.get_pkg_debtags(pkg, axi, ml_data)
section = self.get_pkg_section(pkg, cache, ml_data)
return ' '.join(self.combine_pkg_info(description, debtags, section))
def get_pkgs_classification(self, pkgs_list):
pkgs_classification = []
with open(MachineLearningData.PKGS_CLASSIFICATIONS) as pkgs:
pkgs_data = pickle.load(pkgs)
for pkg_name in pkgs_list:
pkgs_classification.append(pkgs_data[pkg_name][-1])
return pkgs_classification
def get_pkg_description(self, pkg, cache, ml_data):
return ml_data.get_pkg_terms(cache, pkg)
def get_pkg_debtags(self, pkg, axi, ml_data):
return map(lambda x: x.replace('::', '_'),
ml_data.get_pkg_debtags(axi, pkg))
def get_pkg_section(self, pkg, cache, ml_data):
return ml_data.get_pkg_section(cache, pkg)
def get_used_terms_and_debtags(self, features_lists):
terms, debtags = [], []
for feature in features_lists:
if '_' in feature:
debtags.append(feature.replace('_', '::'))
else:
terms.append(feature)
return terms, debtags
def prepare_data(self, pkg_list, axi, cache, ml_data):
pkgs_description = []
pkgs_classification = []
for pkg in pkg_list:
pkg_data = self.create_pkg_data(pkg, axi, cache, ml_data)
pkgs_description.append(pkg_data)
pkgs_classification = self.get_pkgs_classification(pkg_list)
return (pkgs_description, pkgs_classification)
def save_features(self, features, path):
if not self.check_dir():
os.mkdir(BagOfWords.BAG_OF_WORDS_DIR)
with open(path, 'wa') as feature_file:
pickle.dump(features, feature_file)
def save_pkgs_features(self, path, pkgs_list, features_array,
pkg_classification):
pkgs_classification = {}
for index, pkg in enumerate(pkgs_list):
value = features_array[index, :].tolist()
value.append(pkg_classification[index])
pkgs_classification[pkg] = value
with open(path, 'wa') as bow_pkgs_classification:
pickle.dump(pkgs_classification, bow_pkgs_classification)
def train_model(self, pkgs_list, axi, save_files=True):
cache = AptCache()
ml_data = MachineLearningData()
pkgs_description, pkg_classification = self.prepare_data(
pkgs_list, axi, cache, ml_data)
pkg_features = self.vectorizer.fit_transform(pkgs_description)
features_array = pkg_features.toarray()
terms, debtags = self.get_used_terms_and_debtags(
self.vectorizer.get_feature_names())
self.classifier = GaussianNB()
self.classifier.fit(features_array, pkg_classification)
path = BagOfWords.BAG_OF_WORDS_PKGS_CLASSIFICATION
if save_files:
self.save_features(terms, BagOfWords.BAG_OF_WORDS_TERMS)
self.save_features(debtags, BagOfWords.BAG_OF_WORDS_DEBTAGS)
self.save_pkgs_features(
path, pkgs_list, features_array, pkg_classification)
return BagOfWords.CREATED_MODEL
AppRecommender-0.7.5/apprecommender/ml/bayes_matrix.py 0000664 0000000 0000000 00000015247 13067513116 0023112 0 ustar 00root root 0000000 0000000 import pickle
import numpy as np
class BayesMatrix:
'''
This class contains the implementation of the naive bayes algorithm using
only matrix operations in order to perform its tasks.
'''
@staticmethod
def save(bayes_matrix, file_path):
with open(file_path, 'wb') as text:
pickle.dump(bayes_matrix, text)
@staticmethod
def load(file_path):
with open(file_path, 'rb') as text:
bayes_matrix = pickle.load(text)
return bayes_matrix
'''
data: A matrix on the format p x a, where p is the number
of packages that will be used to train the
algorithm and a is the number of features that a
package has.
classifications: A matrix that holds the temporal classification
labels for the packages found on the
attributes' matrix. This matrix is a column one,
where the number of lines represents the number of
packages used to train the algorithm.
order_of_classifications: A array with the possible classifications
on crescent order
labels: A column matrix that holds the possible labels that
can be used to classify a given package.
adjacency: A matrix in the format l x p, where l is the number
of possible classifications labels. This is a binary
matrix that holds which packages were classified
with some given label.
histogram: A column vector l x 1, where l is the possible
classification labels. This matrix holds the values
of how many packages p were classified with a
given label l.
label_probability: A column matrix that holds the individual
probability for each label l for the given
training data.
feature_per_label: A l x a matrix that hold for each label, the number
of times a feature a was present on the label.
prob: A l x a matrix that holds the probability of a given
feature a to be present in a given label l.
diag_histogram: A diagonal matrix for the histogram one.
attribute_vector: A vector 1 x a with the values of features to get a
classification for this vector based on the
training
'''
def __init__(self):
self.data = None
self.classifications = None
self.labels = None
self.adjacecy = None
self.histogram = None
self.label_probability = None
self.feature_per_label = None
self.prob = None
self.diag_histogram = None
def training(self, data_matrix, classifications,
order_of_classifications):
self.data = data_matrix.astype(float)
order_of_classifications = order_of_classifications[:]
self.used_order_of_classifications = (
self.get_used_order_of_classifications(classifications,
order_of_classifications))
self.labels = (self.convert_possible_labels_to_number(
self.used_order_of_classifications).astype(float))
num_packages = self.data.shape[0]
num_labels = self.labels.shape[0]
self.classifications = (self.convert_classifications_to_number(
classifications, self.used_order_of_classifications).astype(float))
self.adjacency = self.get_adjacent_matrix(num_labels,
num_packages).astype(float)
self.histogram = self.adjacency.dot(np.ones((num_packages, 1)))
self.label_probability = self.histogram / num_packages
self.feature_per_label = self.adjacency * self.data
self.diag_histogram = np.diag(np.array(self.histogram)[:, 0])
self.prob = np.linalg.inv(
self.diag_histogram) * self.feature_per_label
def get_classification(self, attribute_vector):
attribute_vector = attribute_vector.astype(float)
prob_vector = self.prob.copy()
label_probability_log = np.log(self.label_probability + 1)
indexes_one = np.matrix(np.where(attribute_vector == 1.0)[1])
indexes_one = indexes_one.tolist()[0]
indexes_zero = np.matrix(np.where(attribute_vector == 0.0)[1])
indexes_zero = indexes_zero.tolist()[0]
prob_vector[:, indexes_one] = 1 + prob_vector[:, indexes_one]
prob_vector[:, indexes_zero] = 2 - prob_vector[:, indexes_zero]
prob_vector = np.log(prob_vector)
prob_vector = prob_vector * np.ones((prob_vector.shape[1], 1))
prob_vector = label_probability_log + prob_vector
line, col = np.unravel_index(prob_vector.argmax(), prob_vector.shape)
best_prob_index = line
return self.used_order_of_classifications[best_prob_index]
def convert_possible_labels_to_number(self, order_of_classifications):
numbers = ""
for i in range(len(order_of_classifications)):
numbers += "{0};".format(i)
return np.matrix(numbers[0:-1])
def convert_classifications_to_number(self, classifications,
order_of_classifications):
numbers = ""
for i in range(len(classifications)):
number = order_of_classifications.index(classifications[i])
numbers += "{0};".format(number)
return np.matrix(numbers[0:-1])
def get_adjacent_matrix(self, num_labels, num_packages):
adjacent_matrix = np.zeros((num_labels, num_packages))
for i in range(len(self.classifications)):
index = int(self.classifications[i].item())
adjacent_matrix[index][i] = 1
return adjacent_matrix
def get_used_order_of_classifications(self, classifications,
order_of_classifications):
used_classifications = set()
num_possible_classifications = len(order_of_classifications)
for name in classifications:
if len(used_classifications) == num_possible_classifications:
return order_of_classifications
used_classifications.add(name[0].item())
for name in order_of_classifications[:]:
if name not in used_classifications:
index = order_of_classifications.index(name)
del order_of_classifications[index]
return order_of_classifications
AppRecommender-0.7.5/apprecommender/ml/cross_validation.py 0000664 0000000 0000000 00000030223 13067513116 0023755 0 ustar 00root root 0000000 0000000 import numpy as np
import xapian
from abc import ABCMeta
from collections import defaultdict
from apprecommender.evaluation import CrossValidation, Metric
from apprecommender.strategy import XAPIAN_DATABASE_PATH
from apprecommender.ml.bag_of_words import BagOfWords
from apprecommender.ml.bayes_matrix import BayesMatrix
from apprecommender.ml.utils import create_column_matrix, create_binary_matrix
NOT_NECESSARY = 1
class OverallAccuracy(Metric):
'''
Simple comparison between generated predictions and
real target values.
'''
def __init__(self):
self.desc = 'Overall_Accuracy'
def run(self, evaluation):
predicted = evaluation.predicted_results
target = evaluation.real_results
overall_accuracy = np.sum(predicted == target)
overall_accuracy /= float(predicted.shape[0])
return overall_accuracy * 100
class ConfusionMatrix():
def __init__(self, predicted_results, real_results):
self.predicted_results = predicted_results
self.real_results = real_results
self.repository_size = len(predicted_results)
self.true_positive_len = 0
self.true_negative_len = 0
self.false_positive_len = 0
self.false_negative_len = 0
self.predicted_relevant_len = 0
self.real_relevant_len = 0
self.real_negative_len = 0
def run(self):
matrix_values = np.zeros(shape=(2, 2))
num_classification = len(self.predicted_results)
for i in range(num_classification):
row = self.predicted_results[i][0]
column = self.real_results[i][0]
matrix_values[row][column] += 1
self.true_positive_len = matrix_values[1][1]
self.true_negative_len = matrix_values[0][0]
self.false_positive_len = matrix_values[1][0]
self.false_negative_len = matrix_values[0][1]
self.predicted_relevant_len = (self.true_positive_len +
self.false_positive_len)
self.real_relevant_len = (self.true_positive_len +
self.false_negative_len)
self.real_negative_len = self.repository_size - self.real_relevant_len
def __str__(self):
result = 'TP: {0}\nFP: {1}'.format(self.true_positive_len,
self.false_positive_len)
result += '\nFN: {0}\nTN: {1}\n'.format(self.false_negative_len,
self.true_negative_len)
return result
class Evaluation():
'''
:param predicted_results: The classifications generated by the machine
learning algorithm. This variable will be a
column matrix, where the number of lines
is equal to the number of data used to
validate the algorithm.
:param real_results: The real classifications of the data used to
test the machine learning algorithm. This
variable will be a column matrix, where the
number of lines is equal to the number of
data used to validate the algorithm.
:possible_classification: An list containg the labels that a input vector
can be classified upon.
'''
def __init__(self, predicted_results, real_results,
possible_classifications):
self.predicted_results = predicted_results
self.real_results = real_results
self.possible_classifications = possible_classifications
self.repository_size = predicted_results.shape[0]
self.classes_outputs = {}
self.create_classification_outputs()
def create_classification_outputs(self):
for classification in self.possible_classifications:
self.classes_outputs[classification] = (
self.create_confusion_matrix(classification))
def create_confusion_matrix(self, classification):
default_value = 0
# Create binary classifications using the one received as parameter
binary_predictions = create_binary_matrix(self.predicted_results,
classification,
default_value)
binary_real = create_binary_matrix(self.real_results, classification,
default_value)
confusion_matrix = ConfusionMatrix(binary_predictions, binary_real)
confusion_matrix.run()
return confusion_matrix
def run(self, metric):
results = {}
for label, confusion_matrix in self.classes_outputs.iteritems():
results[label] = metric.run(confusion_matrix)
return results
class CrossValidationMachineLearning(CrossValidation):
__metaclass__ = ABCMeta
def __init__(self, pkg_data, partition_proportion, rounds,
metrics_list, labels):
self.pkg_data = pkg_data
self.labels = labels
self.label_groups = {}
self.round_label_groups = []
self.round_num_data = []
self.evaluation = None
self.overall_accuracy = []
super(CrossValidationMachineLearning,
self).__init__(partition_proportion, rounds, None,
metrics_list, 0)
def __str__(self):
result_str = ''
metrics_mean = {}
num_data = len(self.pkg_data)
result_str += 'Model used {0}\n'.format(self.label)
result_str += 'Num data used: {0}\n'.format(num_data)
for label in self.labels:
result_str += 'Num of data marked as {0}: {1}\n'.format(
label, len(self.label_groups[label]))
result_str += '\n\n'
for r in range(self.rounds):
result_str += 'Round {0}:\n\n'.format(r)
result_str += 'Overall Accuracy: {0}\n'.format(
self.overall_accuracy[r])
result_str += 'Training data used: {0}\n'.format(
self.round_num_data[r])
for label in self.labels:
result_str += 'Data marked as {0}: {1}\n'.format(
label, len(self.round_label_groups[r][label]))
result_str += '\n'
result_str += '\n\n'
for metric in self.metrics_list:
result_str += '{0}:\n'.format(metric.desc)
metrics_mean[metric.desc] = 0
for r in range(self.rounds):
result_str += '\tRound {0}:\n'.format(r)
mean = 0
for label in self.labels:
result = self.cross_results[metric.desc][r][label]
mean += result
result_str += '\t\tClass {0}: {1}\n'.format(label, result)
mean /= len(self.labels)
result_str += '\t\tMean: {0}\n\n'.format(mean)
metrics_mean[metric.desc] += mean
metrics_mean[metric.desc] /= self.rounds
result_str += '\n\n'
result_str += 'Average results:\n'
result_str += '---------------\n'
for metric in self.metrics_list:
result_str += '{0}: {1}\n'.format(metric.desc,
metrics_mean[metric.desc])
return result_str
def create_labels_groups(self, data):
label_groups = {}
label_groups.fromkeys(self.labels)
label_groups = defaultdict(lambda: [], label_groups)
for input_vector in data.values():
label = input_vector[-1]
label_groups[label].append(input_vector)
return label_groups
def get_model(self, cross_item_score):
'''
This function should get the data that will be used as training data,
train the algorithm with this data and return the generated model
'''
self.round_num_data.append(len(cross_item_score))
self.round_label_groups.append(
self.create_labels_groups(cross_item_score))
def get_user_score(self, user):
user_score = self.pkg_data
self.label_groups = self.create_labels_groups(user_score)
return user_score
def get_real_results(self, round_partition):
'''
This method should return the real labels for the validation
set used on the algorithm.
'''
classifications = []
for input_vector in round_partition.values():
classifications.append(input_vector[-1])
# make classifications a column array
return create_column_matrix(classifications)
def get_result_size(self):
return NOT_NECESSARY
def get_evaluation(self, predicted_result, real_result):
self.evaluation = Evaluation(predicted_result, real_result,
self.labels)
return self.evaluation
def run_metrics(self, predicted_result, real_result):
super(CrossValidationMachineLearning, self).run_metrics(
predicted_result, real_result)
self.overall_accuracy.append(OverallAccuracy().run(self.evaluation))
class CrossValidationBVA(CrossValidationMachineLearning):
def __init__(self, pkg_data, partition_proportion, rounds,
metrics_list, labels):
super(CrossValidationBVA, self).__init__(
pkg_data, partition_proportion, rounds, metrics_list,
labels)
self.label = "Binary vector model"
def get_model(self, cross_item_score):
super(CrossValidationBVA, self).get_model(cross_item_score)
bayes_matrix = BayesMatrix()
all_matrix = (np.matrix(cross_item_score.values()))
data_matrix = all_matrix[0:, 0:-1]
classifications = all_matrix[0:, -1]
bayes_matrix.training(data_matrix, classifications,
self.labels)
return bayes_matrix
'''
:param round_user: The model created by the machine learning algorithm.
:param round_partition: The data that will be used to evaluate the
machine learning algorithm.
:param result_size: Not necessary for this context
'''
def get_predicted_results(self, round_user, round_partition,
result_size=0):
'''
This method should generate the predictions for the packages
received. It basically needs to used the generated model
and use it to generate the prediction.
'''
predicted_results = []
for pkg, input_vector in round_partition.iteritems():
input_vector = np.matrix(input_vector[:-1])
predicted_results.append(
round_user.get_classification(input_vector))
return create_column_matrix(predicted_results)
class CrossValidationBOW(CrossValidationMachineLearning):
def __init__(self, pkg_data, partition_proportion, rounds,
metrics_list, labels):
super(CrossValidationBOW, self).__init__(
pkg_data, partition_proportion, rounds, metrics_list,
labels)
self.axi = xapian.Database(XAPIAN_DATABASE_PATH)
self.label = "Bag of words model"
def get_model(self, cross_item_score):
super(CrossValidationBOW, self).get_model(cross_item_score)
pkgs_list = cross_item_score.keys()
bag_of_words = BagOfWords()
bag_of_words.train_model(pkgs_list, self.axi, save_files=False)
return bag_of_words
'''
:param round_user: The model created by the machine learning algorithm.
:param round_partition: The data that will be used to evaluate the
machine learning algorithm.
:param result_size: Not necessary for this context
'''
def get_predicted_results(self, round_user, round_partition,
result_size=0):
'''
This method should generate the predictions for the packages
received. It basically needs to used the generated model
and use it to generate the prediction.
'''
predicted_results = []
for pkg in round_partition.keys():
predicted_results.append(
round_user.classify_pkg(pkg)[0])
return create_column_matrix(predicted_results)
AppRecommender-0.7.5/apprecommender/ml/data.py 0000664 0000000 0000000 00000016152 13067513116 0021330 0 ustar 00root root 0000000 0000000 from os import path
from os import makedirs
import Stemmer
import pickle
import xapian
from apprecommender.apt_cache import AptCache
from apprecommender.config import Config
from apprecommender.decider import FilterTag, FilterDescription
from apprecommender.ml.pkg_time import PkgTime
class MachineLearningData():
XAPIAN_DATABASE_PATH = Config().axi_desktopapps
USER_DATA_DIR = Config().user_data_dir
BASE_DIR = Config().base_dir
PKG_DATA_PATH = USER_DATA_DIR + 'pkg_data.txt'
PKGS_CLASSIFICATIONS = USER_DATA_DIR + 'pkgs_classifications.txt'
MACHINE_LEARNING_TERMS = USER_DATA_DIR + 'machine_learning_terms.txt'
MACHINE_LEARNING_DEBTAGS = USER_DATA_DIR + 'machine_learning_debtags.txt'
MACHINE_LEARNING_TRAINING = USER_DATA_DIR + 'machine_learning_training.txt'
def __init__(self):
self.axi = xapian.Database(MachineLearningData.XAPIAN_DATABASE_PATH)
self.stemmer = Stemmer.Stemmer('english')
valid_tags = []
with open(path.join(Config().filters_dir, "debtags")) as tags:
valid_tags = [line.strip() for line in tags
if not line.startswith("#")]
self.filter_tag = FilterTag(valid_tags)
self.filter_description = FilterDescription()
def create_data(self, labels):
if not path.exists(MachineLearningData.USER_DATA_DIR):
makedirs(MachineLearningData.USER_DATA_DIR)
pkgs = self.get_pkgs_classification(labels)
cache = AptCache()
terms_name = self.get_terms_for_all_pkgs(cache, pkgs.keys())
debtags_name = self.get_debtags_for_all_pkgs(self.axi, pkgs.keys())
debtags_name = self.filter_debtags(debtags_name)
debtags_name = sorted(debtags_name)
terms_name = self.filter_terms(terms_name)
terms_name = sorted(terms_name)
pkgs_classifications = (
self.get_pkgs_table_classification(self.axi, pkgs,
cache, debtags_name,
terms_name))
try:
self.save_pkg_data(
terms_name, MachineLearningData.MACHINE_LEARNING_TERMS)
self.save_pkg_data(
debtags_name, MachineLearningData.MACHINE_LEARNING_DEBTAGS)
self.save_pkg_data(
pkgs_classifications, MachineLearningData.PKGS_CLASSIFICATIONS)
except IOError:
raise
return pkgs_classifications
def get_pkgs_classification(self, labels):
pkgs_percent = {}
pkgs_classification = {}
pkg_time = PkgTime()
pkg_data = pkg_time.get_package_data()
for name, time_values in pkg_data.iteritems():
_, access = time_values
pkgs_percent[name] = access
pkgs = pkgs_percent.keys()
pkgs = sorted(pkgs, key=lambda pkg: pkgs_percent[pkg])
pkgs = list(reversed(pkgs))
if len(pkgs) > len(labels):
size = len(pkgs) / len(labels)
for index, label in enumerate(labels):
index_begin = size * index
index_end = index_begin + size
classifications = dict.fromkeys(pkgs[index_begin:index_end],
label)
pkgs_classification.update(classifications)
index_begin = size * len(labels)
if index_begin < len(labels):
classifications = dict.fromkeys(pkgs[index_begin], label[-1])
pkgs_classification.update(classifications)
else:
for index, pkg in enumerate(pkgs):
pkgs_classification[pkg] = labels[index]
return pkgs_classification
def get_pkg_data(self, axi, pkg_name, data_type):
pkg_name = 'XP' + pkg_name
query = xapian.Query(xapian.Query.OP_OR, [pkg_name])
enquire = xapian.Enquire(axi)
enquire.set_query(query)
mset = enquire.get_mset(0, 10)
pkg_info = []
for pkg in mset:
for term in axi.get_document(pkg.docid).termlist():
pkg_term = term.term
if pkg_term.startswith(data_type):
pkg_info.append(pkg_term[len(data_type):])
elif data_type == 'term':
if pkg_term[0].islower():
pkg_info.append(pkg_term)
return pkg_info
def get_pkg_debtags(self, axi, pkg_name):
return self.get_pkg_data(axi, pkg_name, 'XT')
def get_pkg_terms(self, cache, pkg_name):
description = self.get_pkg_description(cache, pkg_name)
tokens = description.lower().split()
stems = [self.stemmer.stemWord(token) for token in tokens
if self.filter_description(token)]
return stems
def get_pkg_section(self, cache, pkg_name):
return cache[pkg_name].section
def get_pkg_description(self, cache, pkg_name):
if pkg_name not in cache:
return []
else:
description = cache[pkg_name].candidate.description
return description.strip()
def get_debtags_name(self, file_path):
with open(file_path, 'r') as text:
debtags_name = [debtag.strip() for debtag in text]
return debtags_name
def create_row_table_list(self, labels_name, pkg_elements):
row_list = []
for debtag in labels_name:
row_list.append(1 if debtag in pkg_elements else 0)
return row_list
def get_terms_for_all_pkgs(self, cache, pkgs):
pkg_terms = set()
for pkg in pkgs:
pkg_terms = pkg_terms | set(self.get_pkg_terms(cache, pkg))
return pkg_terms
def get_debtags_for_all_pkgs(self, axi, pkgs):
pkg_debtags = set()
for pkg in pkgs:
pkg_debtags = pkg_debtags | set(self.get_pkg_debtags(axi, pkg))
return pkg_debtags
def filter_terms(self, terms):
filtered_terms = []
for term in terms:
if self.filter_description(term):
filtered_terms.append(term)
return filtered_terms
def filter_debtags(self, debtags):
filtered_debtags = []
for tag in debtags:
if self.filter_tag('XT' + tag):
filtered_debtags.append(tag)
return filtered_debtags
def get_pkgs_table_classification(self, axi, pkgs, cache,
debtags_name, terms_name):
pkgs_classification = {}
for key, value in pkgs.iteritems():
pkgs_classification[key] = []
debtags = self.get_pkg_debtags(axi, key)
debtags = self.create_row_table_list(debtags_name, debtags)
pkgs_classification[key].extend(debtags)
terms = self.get_pkg_terms(cache, key)
terms = self.create_row_table_list(list(terms_name), terms)
pkgs_classification[key].extend(terms)
pkgs_classification[key].append(value)
return pkgs_classification
def save_pkg_data(self, pkg_data, file_path):
try:
ml_data = open(file_path, 'wb')
with ml_data:
pickle.dump(pkg_data, ml_data)
except IOError:
raise
AppRecommender-0.7.5/apprecommender/ml/pkg_time.py 0000664 0000000 0000000 00000006457 13067513116 0022225 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import commands
import os
import re
from apprecommender.config import Config
from apprecommender.data_classification import get_time_from_package
from apprecommender.user import LocalSystem
from apprecommender.utils import print_progress_bar
USER_DATA_DIR = Config().user_data_dir
class PkgTime:
def create_pkg_data(self):
user = LocalSystem()
user_pkgs = user.pkg_profile
pkgs_time = self.get_packages_time(user_pkgs)
self.save_package_time(pkgs_time)
return pkgs_time
def get_best_time(self, pkg):
valid_path_regex = re.compile(
r'/usr/bin/|/usr/game/|/usr/lib/.+/')
invalid_path_regex = re.compile(
r'(.+-){2,}|/usr/lib/mime/packages/|/init.d/|/media/')
invalid_files_regex = re.compile(
r'\.desktop|\.conf|\.egg-info|\.txt')
pkg_files = commands.getoutput('dpkg -L {}'.format(pkg))
bestatime, bestmtime = 0, 0
for pkg_file in pkg_files.splitlines():
if invalid_path_regex.search(pkg_file):
continue
if invalid_files_regex.search(pkg_file):
continue
if os.path.isdir(pkg_file):
continue
if valid_path_regex.search(pkg_file):
modify, access = get_time_from_package(pkg_file,
pkg_bin=False)
modify = int(modify) if modify else 0
access = int(access) if access else 0
if access > bestatime:
bestatime = access
bestmtime = modify
return (bestmtime, bestatime)
def get_package_data(self, file_path=USER_DATA_DIR + 'pkg_data.txt'):
if os.path.isfile(file_path):
pkgs_time = {}
with open(file_path, 'r') as pkg_data:
for pkg_line in pkg_data:
name, modify, access = pkg_line.split(' ')
pkgs_time[name] = [modify, access]
return pkgs_time
else:
return self.create_pkg_data()
def get_packages_time(self, pkgs, verbose=False):
pkgs_time = {}
len_pkgs = len(pkgs)
for index, pkg in enumerate(pkgs):
modify, access = self.get_best_time(pkg)
if modify and access:
if verbose:
print 'ADD: {}'.format(pkg)
pkgs_time[pkg] = []
pkgs_time[pkg].append(modify)
pkgs_time[pkg].append(access)
else:
if verbose:
print 'NOT: {} {} {}'.format(pkg, modify, access)
print_progress_bar(index + 1, len_pkgs)
return pkgs_time
def print_package_time(self, pkgs_time):
for key, value in pkgs_time.iteritems():
print "{} : Modify {}, Access {}".format(key, value[0], value[1])
def save_package_time(self, pkgs_time,
file_path=USER_DATA_DIR + 'pkg_data.txt'):
with open(file_path, 'w') as pkg_data:
pkg_str = "{pkg} {modify} {access}\n"
for pkg, times in pkgs_time.iteritems():
pkg_line = pkg_str.format(pkg=pkg, modify=times[0],
access=times[1])
pkg_data.write(pkg_line)
AppRecommender-0.7.5/apprecommender/ml/utils.py 0000664 0000000 0000000 00000000455 13067513116 0021556 0 ustar 00root root 0000000 0000000 from numpy import array, where
INVALID_PARAMETERS = -1
def create_column_matrix(matrix):
matrix = array(matrix)
matrix.shape = (len(matrix), 1)
return matrix
def create_binary_matrix(original_vector, value, default_value):
return where(original_vector == value, 1, default_value)
AppRecommender-0.7.5/apprecommender/recommender.py 0000664 0000000 0000000 00000014434 13067513116 0022310 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
recommender - python module for classes related to recommenders.
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import heapq
import inspect
import logging
import operator
import os
import xapian
from collections import namedtuple
from fuzzywuzzy import fuzz
from operator import attrgetter
import apprecommender.strategy
from apprecommender.apt_cache import AptCache
from apprecommender.config import Config
class RecommendationResult:
"""
Class designed to describe a recommendation result: items and scores.
"""
def __init__(self, item_score, ranking=0, limit=0, user_profile=None):
"""
Set initial parameters.
"""
self.item_score = item_score
self.size = len(item_score)
self.limit = limit
self.cache = AptCache()
self.pkg_descriptions = {}
if ranking:
self.ranking = ranking
if user_profile:
self.fill_pkg_descriptions(user_profile)
def fill_pkg_descriptions(self, user_profile):
for pkg in user_profile:
description = self.cache[pkg].candidate.description
self.pkg_descriptions[pkg] = description.lower()
def __str__(self):
"""
String representation of the object.
"""
result = self.get_prediction(self.limit)
rec_str = '\n'
index = 1
for pkg, _ in result:
summary = self.cache[pkg].candidate.summary
description = self.cache[pkg].candidate.description
rec_str += '{}: {} \t {}\n'.format(
index, pkg.ljust(20), summary)
if self.pkg_descriptions:
because_pkgs = self.get_because(description.lower())
rec_str += ' because you installed: \t {}\n\n'.format(
', '.join(because_pkgs))
index += 1
return rec_str
def get_because(self, rec_description):
because = []
PkgRatio = namedtuple('PkgRatio', ['pkg', 'ratio'])
for pkg, description in self.pkg_descriptions.iteritems():
ratio = fuzz.ratio(rec_description, description)
because.append(PkgRatio(pkg, ratio))
pkgs = heapq.nlargest(4, because, key=attrgetter('ratio'))
return [pkg for pkg, _ in pkgs]
def get_prediction(self, limit=0):
"""
Return prediction based on recommendation size (number of items).
"""
sorted_result = sorted(self.item_score.items(),
key=operator.itemgetter(1))
if not limit or limit > self.size:
limit = self.size
return list(reversed(sorted_result[-limit:]))
class Recommender:
"""
Class designed to play the role of recommender.
"""
def __init__(self):
"""
Set initial parameters.
"""
self.cfg = Config()
# Load xapian indexes
self.axi_desktopapps = xapian.Database(self.cfg.axi_desktopapps)
if self.cfg.popcon:
self.popcon_desktopapps = xapian.Database(
self.cfg.popcon_desktopapps)
# Load valid programs, desktopapps and tags
# format: one package or tag name per line
self.valid_desktopapps = []
self.valid_tags = []
logging.info("Loading recommender filters")
with open(os.path.join(self.cfg.filters_dir, "desktopapps")) as pkgs:
self.valid_desktopapps = [line.strip() for line in pkgs
if not line.startswith("#")]
with open(os.path.join(self.cfg.filters_dir, "debtags")) as tags:
self.valid_tags = [line.strip() for line in tags
if not line.startswith("#")]
# Set xapian index weighting scheme
if self.cfg.weight == "bm25":
self.weight = xapian.BM25Weight(self.cfg.bm25_k1, self.cfg.bm25_k2,
self.cfg.bm25_k3, self.cfg.bm25_b,
self.cfg.bm25_nl)
else:
self.weight = xapian.TradWeight()
self.set_strategy(self.cfg.strategy)
def get_all_strategies(self):
cls_members = inspect.getmembers(apprecommender.strategy,
inspect.isclass)
valid_strategies = {}
for name, obj in cls_members:
class_strategies = getattr(obj, 'get_valid_strategies', None)
if class_strategies:
valid_strategies = dict(class_strategies(), **valid_strategies)
return valid_strategies
def set_strategy(self, strategy_str, n=0):
"""
Set the recommendation strategy.
"""
valid_strategies = self.get_all_strategies()
profile_size = n if n else self.cfg.profile_size
self.items_repository = self.axi_desktopapps
self.valid_pkgs = self.valid_desktopapps
logging.info("Setting recommender strategy to \'%s\'" % strategy_str)
if strategy_str in valid_strategies:
strategy_type = valid_strategies[strategy_str]
class_name = strategy_type.class_name
content_type = strategy_type.content_type
self.strategy = eval(class_name)(
content_type, profile_size)
else:
logging.info("Strategy not defined.")
self.strategy = None
def get_recommendation(self, user, result_size=100):
"""
Produces recommendation using previously loaded strategy.
"""
if self.strategy is None:
return ""
return self.strategy.run(self, user, result_size)
AppRecommender-0.7.5/apprecommender/singleton.py 0000664 0000000 0000000 00000002360 13067513116 0022005 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
singleton - python class that implements singleton design pattern.
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
class Singleton(object):
"""
Base class for inheritance of only-one-instance classes.
Singleton design pattern.
"""
def __new__(cls, *args, **kwargs):
"""
Creates a new instance of the class only if none already exists.
"""
if '_inst' not in vars(cls):
cls._inst = object.__new__(cls)
return cls._inst
AppRecommender-0.7.5/apprecommender/strategy.py 0000664 0000000 0000000 00000040353 13067513116 0021651 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
strategy - python module for classes and methods related to recommendation
strategies.
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import collections
import logging
import operator
import pickle
import re
import recommender
import subprocess
import xapian
import numpy as np
from abc import ABCMeta, abstractmethod
from apprecommender.apt_cache import AptCache
from apprecommender.config import Config
from apprecommender.decider import (PkgMatchDecider,
PkgReverseDependeciesDecider)
from apprecommender.ml.bag_of_words import BagOfWords
from apprecommender.ml.bayes_matrix import BayesMatrix
from apprecommender.ml.data import MachineLearningData
from apprecommender.utils import get_class_and_module_name
XAPIAN_DATABASE_PATH = Config().axi_desktopapps
USER_DATA_DIR = Config().user_data_dir
PKGS_CLASSIFICATIONS_INDICES = (USER_DATA_DIR +
'pkgs_classifications_indices.txt')
StrategyType = collections.namedtuple('StrategyType',
'class_name content_type')
class RecommendationStrategy(object):
"""
Base class for recommendation strategies.
"""
def run(self, rec, user, recommendation_size):
raise NotImplementedError
class ContentBased(RecommendationStrategy):
"""
Content-based recommendation strategy based on Apt-xapian-index.
"""
@classmethod
def get_valid_strategies(cls):
class_name = get_class_and_module_name(cls)
valid_strategies = {'cb': StrategyType(class_name, 'mix'),
'cbt': StrategyType(class_name, 'tag'),
'cbd': StrategyType(class_name, 'desc'),
'cbh': StrategyType(class_name, 'half'),
'cbtm': StrategyType(class_name, 'time'),
'cb_eset': StrategyType(class_name, 'mix_eset'),
'cbt_eset': StrategyType(class_name, 'tag_eset'),
'cbd_eset': StrategyType(class_name, 'desc_eset'),
'cbh_eset': StrategyType(class_name, 'half_eset')}
return valid_strategies
def __init__(self, content, profile_size):
self.description = "Content-based"
self.content = content
self.profile_size = profile_size
def get_sugestion_from_profile(self, rec, user, profile,
recommendation_size, because=True,
pkg_decider=None):
query = xapian.Query(xapian.Query.OP_OR, profile)
enquire = xapian.Enquire(rec.items_repository)
enquire.set_weighting_scheme(rec.weight)
enquire.set_query(query)
user_profile = None
if pkg_decider is None:
pkg_decider = PkgMatchDecider(user.installed_pkgs)
# Retrieve matching packages
try:
mset = enquire.get_mset(0, recommendation_size, None,
pkg_decider)
except xapian.DatabaseError as error:
logging.critical("Content-based strategy: " + error.get_msg())
# Compose result dictionary
item_score = {}
ranking = []
for m in mset:
item_score[m.document.get_data()] = m.weight
ranking.append(m.document.get_data())
if because and Config().because:
user_profile = user.pkg_profile
result = recommender.RecommendationResult(
item_score, ranking, user_profile=user_profile)
return result
def run(self, rec, user, rec_size):
"""
Perform recommendation strategy.
"""
logging.debug("Composing user profile...")
profile = user.content_profile(rec.items_repository, self.content,
self.profile_size, rec.valid_tags)
logging.debug(profile)
result = self.get_sugestion_from_profile(rec, user, profile, rec_size)
return result
class PackageReference(ContentBased):
@classmethod
def get_valid_strategies(cls):
class_name = get_class_and_module_name(cls)
valid_strategies = {'cbpkg': StrategyType(class_name, 'mix')}
return valid_strategies
def __init__(self, content, profile_size):
ContentBased.__init__(self, content, profile_size)
self.content = content
self.description = 'Package-reference'
self.profile_size = profile_size
self.cache = AptCache()
self.pkgs_regex = re.compile(r'^\s+(?:\|)?(.+)$', re.MULTILINE)
def get_reverse_dependencies_pkgs(self, reference_pkgs):
reverse_dependencies_pkgs = []
for pkg in reference_pkgs:
command = 'apt-cache rdepends {}'.format(pkg)
rdepends = subprocess.check_output(command,
stderr=subprocess.STDOUT,
shell=True)
rdepends_pkgs = self.pkgs_regex.findall(rdepends)
reverse_dependencies_pkgs += rdepends_pkgs
return reverse_dependencies_pkgs
def content_profile_for_reference_pkgs(self, reference_pkgs):
content_profile = reference_pkgs[:]
for pkg in reference_pkgs:
content_profile += pkg.split('-')
content_profile = list(set(content_profile))
return content_profile
def run(self, rec, user, rec_size):
reference_pkgs = user.reference_pkgs
reverse_dependencies_pkgs = self.get_reverse_dependencies_pkgs(
reference_pkgs)
pkg_decider = PkgReverseDependeciesDecider(reverse_dependencies_pkgs,
user.installed_pkgs)
profile = user.content_profile(rec.items_repository, self.content,
self.profile_size, rec.valid_tags)
profile += self.content_profile_for_reference_pkgs(reference_pkgs)
rec.items_repository = xapian.Database(Config().axi)
result = self.get_sugestion_from_profile(
rec, user, profile, rec_size, pkg_decider=pkg_decider)
return result
class MachineLearningTrainError(Exception):
def __init__(self, value=''):
self.value = value
def __str__(self):
return repr(self.value)
class MachineLearning(ContentBased):
__metaclass__ = ABCMeta
PKGS_CLASSIFICATIONS = None
def __init__(self, content, profile_size, suggestion_size=200):
ContentBased.__init__(self, content, profile_size)
self.content = content
self.description = 'Machine-learning'
self.profile_size = profile_size
self.suggestion_size = suggestion_size
self.cache = AptCache()
self.ml_data = MachineLearningData()
self.axi = xapian.Database(XAPIAN_DATABASE_PATH)
def display_recommended_terms(self, terms_name, debtags_name, item_score,
rec_size):
sorted_result = sorted(item_score.items(), key=operator.itemgetter(1))
sorted_result = list(reversed(sorted_result))
sorted_result = [pkg[0] for pkg in sorted_result][0:rec_size]
sorted_result = list(reversed(sorted_result))
for pkg in sorted_result:
pkg_terms = self.ml_data.get_pkg_terms(self.cache, pkg)
pkg_debtags = self.ml_data.get_pkg_debtags(self.axi, pkg)
terms_match = []
for term in pkg_terms:
if term in terms_name:
terms_match.append(term)
debtags_match = []
for debtag in pkg_debtags:
if debtag in debtags_name:
debtags_match.append(debtag)
print "\n\n="
print "{0}".format(pkg)
print "debtags:"
print debtags_match
print "-"
print "terms:"
print terms_match
print "="
def get_item_score(self, pkgs_score, pkgs_classifications):
item_score = {}
order = ['RU', 'U', 'NU']
order_values = [0, 1000, 2000]
for pkg, classification in pkgs_classifications.iteritems():
item_score[pkg] = order_values[order.index(classification)]
item_score[pkg] += pkgs_score[pkg]
return item_score
def get_pkgs_and_scores(self, rec, user):
profile = user.content_profile(rec.items_repository, self.content,
self.suggestion_size, rec.valid_tags)
content_based = self.get_sugestion_from_profile(
rec, user, profile, self.suggestion_size, because=False)
pkgs, pkgs_score = [], {}
for pkg_line in str(content_based).splitlines()[1:]:
pkg = re.search(r'\d+:\s([\w-]+)', pkg_line)
if not pkg.groups():
continue
pkg = pkg.groups()[0]
pkg_score = int(pkg_line.split(':')[0].strip())
pkgs.append(pkg)
pkgs_score[pkg] = self.suggestion_size - pkg_score
return pkgs, pkgs_score
def get_pkgs_classifications(self, pkgs, terms_name, debtags_name):
ml_strategy = self.get_ml_strategy()
pkgs_classifications = {}
kwargs = {}
kwargs['terms_name'] = terms_name
kwargs['debtags_name'] = debtags_name
kwargs['ml_strategy'] = ml_strategy
for pkg in pkgs:
if pkg not in self.cache:
continue
attribute_vector = self.prepare_pkg_data(
pkg, **kwargs)
classification = self.get_pkg_classification(
ml_strategy, attribute_vector)
pkgs_classifications[pkg] = classification
return pkgs_classifications
def load_terms_and_debtags(self):
terms_name = []
debtags_name = []
terms_path = self.get_terms_path()
debtags_path = self.get_debtags_path()
with open(terms_path, 'rb') as terms:
terms_name = pickle.load(terms)
with open(debtags_path, 'rb') as debtags:
debtags_name = pickle.load(debtags)
return terms_name, debtags_name
@staticmethod
def train(cls):
ml_data = MachineLearningData()
labels = ['RU', 'U', 'NU']
try:
MachineLearning.PKGS_CLASSIFICATIONS = ml_data.create_data(
labels)
if len(MachineLearning.PKGS_CLASSIFICATIONS) >= 10:
cls.run_train(MachineLearning.PKGS_CLASSIFICATIONS)
else:
raise MachineLearningTrainError()
except IOError:
raise
@abstractmethod
def get_debtags_path(self):
raise NotImplementedError("Method not implemented.")
@abstractmethod
def get_ml_strategy(self):
raise NotImplementedError("Method not implemented.")
@abstractmethod
def get_pkg_classification(self, ml_strategy, attribute_vector):
raise NotImplementedError("Method not implemented.")
@abstractmethod
def get_terms_path(self):
raise NotImplementedError("Method not implemented.")
@abstractmethod
def prepare_pkg_data(self, pkg, **kwargs):
raise NotImplementedError("Method not implemented.")
@abstractmethod
def run_train(cls, pkgs_classifications):
raise NotImplementedError("Method not implemented.")
def run(self, rec, user, rec_size):
user_profile = None
terms_name, debtags_name = self.load_terms_and_debtags()
pkgs, pkgs_score = self.get_pkgs_and_scores(rec, user)
pkgs_classifications = self.get_pkgs_classifications(pkgs, terms_name,
debtags_name)
item_score = self.get_item_score(pkgs_score, pkgs_classifications)
if Config().because:
user_profile = user.pkg_profile
return recommender.RecommendationResult(
item_score, limit=rec_size, user_profile=user_profile)
class MachineLearningBVA(MachineLearning):
@classmethod
def get_valid_strategies(cls):
class_name = get_class_and_module_name(cls)
valid_strategies = {'mlbva': StrategyType(class_name, 'mlbva_mix'),
'mlbva_eset': StrategyType(class_name,
'mlbva_mix_eset')}
return valid_strategies
@classmethod
def run_train(cls, pkgs_classifications):
all_matrix = (np.matrix(pkgs_classifications.values()))
data_matrix = all_matrix[0:, 0:-1]
classifications = all_matrix[0:, -1]
order_of_classifications = ['NU', 'U', 'RU']
bayes_matrix = BayesMatrix()
bayes_matrix.training(data_matrix, classifications,
order_of_classifications)
BayesMatrix.save(bayes_matrix,
MachineLearningData.MACHINE_LEARNING_TRAINING)
def __init__(self, content, profile_size, suggestion_size=200):
super(MachineLearningBVA, self).__init__(
content, profile_size, suggestion_size)
self.description = "Machine-learning-binary-vector-approach"
def get_debtags_path(self):
return MachineLearningData.MACHINE_LEARNING_DEBTAGS
def get_ml_strategy(self):
return BayesMatrix.load(
MachineLearningData.MACHINE_LEARNING_TRAINING)
def get_pkg_classification(self, ml_strategy, attribute_vector):
return ml_strategy.get_classification(attribute_vector)
def get_terms_path(self):
return MachineLearningData.MACHINE_LEARNING_TERMS
def prepare_pkg_data(self, pkg, **kwargs):
terms_name = kwargs['terms_name']
debtags_name = kwargs['debtags_name']
pkg_terms = self.ml_data.get_pkg_terms(self.cache, pkg)
pkg_debtags = self.ml_data.get_pkg_debtags(self.axi, pkg)
debtags_attributes = self.ml_data.create_row_table_list(
debtags_name, pkg_debtags)
terms_attributes = self.ml_data.create_row_table_list(
terms_name, pkg_terms)
attribute_vector = terms_attributes + debtags_attributes
attribute_vector = np.matrix(attribute_vector)
return attribute_vector
class MachineLearningBOW(MachineLearning):
@classmethod
def get_valid_strategies(cls):
class_name = get_class_and_module_name(cls)
valid_strategies = {'mlbow': StrategyType(class_name, 'mlbow_mix'),
'mlbow_eset': StrategyType(class_name,
'mlbow_mix_eset')}
return valid_strategies
def __init__(self, content, profile_size, suggestion_size=200):
super(MachineLearningBOW, self).__init__(
content, profile_size, suggestion_size)
self.description = "Machine-learning-bag-of-words"
def get_debtags_path(self):
return BagOfWords.BAG_OF_WORDS_DEBTAGS
def get_ml_strategy(self):
return BagOfWords.load(
BagOfWords.BAG_OF_WORDS_MODEL)
def get_pkg_classification(self, ml_strategy, attribute_vector):
return ml_strategy.classify_pkg(attribute_vector)
def get_terms_path(self):
return BagOfWords.BAG_OF_WORDS_TERMS
def prepare_pkg_data(self, pkg, **kwargs):
ml_strategy = kwargs['ml_strategy']
attribute_vector = ml_strategy.create_pkg_data(
pkg, self.axi, self.cache, self.ml_data)
return attribute_vector
@classmethod
def run_train(cls, pkgs_classifications):
bag_of_words = BagOfWords()
pkgs_list = pkgs_classifications.keys()
axi = xapian.Database(XAPIAN_DATABASE_PATH)
bag_of_words.train_model(pkgs_list, axi)
BagOfWords.save(bag_of_words, BagOfWords.BAG_OF_WORDS_MODEL)
AppRecommender-0.7.5/apprecommender/tests/ 0000775 0000000 0000000 00000000000 13067513116 0020572 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/apprecommender/tests/test_apt_run.py 0000664 0000000 0000000 00000004230 13067513116 0023652 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import os
import shutil
import unittest
from mock import patch
from apprecommender.main.apt_run import AptRun
from apprecommender.user import LocalSystem
class AptRunTests(unittest.TestCase):
TEST_FOLDER = 'apprecommender/tests/.apt_run'
def setUp(self):
if os.path.exists(AptRunTests.TEST_FOLDER):
shutil.rmtree(AptRunTests.TEST_FOLDER)
def test_enable_apt_run(self):
apt_run = AptRun()
apt_run.set_folder(AptRunTests.TEST_FOLDER)
apt_run.enable()
result = apt_run.is_enable()
self.assertTrue(result)
def test_dont_enable_apt_run_if_already_enabled(self):
apt_run = AptRun()
apt_run.set_folder(AptRunTests.TEST_FOLDER)
apt_run.enable()
result = apt_run.enable()
self.assertFalse(result)
def test_disable_apt_run(self):
apt_run = AptRun()
apt_run.set_folder(AptRunTests.TEST_FOLDER)
os.makedirs(AptRunTests.TEST_FOLDER)
apt_run.disable()
result = apt_run.is_enable()
self.assertFalse(result)
def test_dont_disable_apt_run_if_already_disabled(self):
apt_run = AptRun()
apt_run.set_folder(AptRunTests.TEST_FOLDER)
result = apt_run.disable()
self.assertFalse(result)
def test_pre_install_pkgs(self):
apt_run = AptRun()
apt_run.set_folder(AptRunTests.TEST_FOLDER)
apt_run.enable()
apt_run.pre_install_pkgs()
user = LocalSystem()
assert_pkgs = user.pkg_profile
pkgs = []
with open(apt_run.installed_pkgs_file, 'r') as text:
pkgs = [line.strip() for line in text]
self.assertListEqual(assert_pkgs, pkgs)
@patch('apprecommender.main.apt_run.AptRun.get_user_pkgs')
def test_post_invoke(self, mock_user_pkgs):
mock_user_pkgs.return_value = ['vim', 'gedit']
apt_run = AptRun()
apt_run.set_folder(AptRunTests.TEST_FOLDER)
apt_run.enable()
with open(apt_run.installed_pkgs_file, 'w') as text:
text.write('gedit')
installed_pkgs = apt_run.post_invoke()
self.assertEqual(['vim'], installed_pkgs)
AppRecommender-0.7.5/apprecommender/tests/test_data.py 0000664 0000000 0000000 00000005415 13067513116 0023121 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
dataTests - Data classes test case
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import unittest
import xapian
from apprecommender.data import PopconSubmission, axi_search_pkg_tags
from apprecommender.config import Config
class AxiSearchTests(unittest.TestCase):
@classmethod
def setUpClass(self):
cfg = Config()
self.axi = xapian.Database(cfg.axi)
def test_search_pkg_tags(self):
tags = axi_search_pkg_tags(self.axi, 'vim')
self.assertEqual(set(tags), set(['XTscope::application',
'XTrole::program',
'XTimplemented-in::c',
'XTworks-with::unicode',
'XTworks-with::text',
'XTuse::editing',
'XTinterface::commandline',
'XTdevel::editor',
'XTuitoolkit::ncurses',
'XTinterface::text-mode']))
class PopconSubmissionTests(unittest.TestCase):
@classmethod
def setUpClass(self):
self.submission_path = "apprecommender/tests/test_data/test_popcon"
self.submission = PopconSubmission(self.submission_path)
def test_user_id(self):
with open(self.submission_path) as popcon_file:
user_id = popcon_file.readline().split()[2].lstrip("ID:")
self.assertEqual(self.submission.user_id, user_id)
def test_load(self):
with open(self.submission_path) as popcon_file:
size = len(popcon_file.readlines())
self.assertEqual(len(self.submission.packages), size - 2)
def test_str(self):
output = "\nPopularity-contest submission ID "
output += "8b44fcdbcf676e711a153d5db099test\n "
output += "dash: 1\n perl-base: 1\n libusbmuxd1: 1\n "
output += "libc6-i686: 1\n libc6: 1"
self.assertEqual(self.submission.__str__(), output)
AppRecommender-0.7.5/apprecommender/tests/test_data/ 0000775 0000000 0000000 00000000000 13067513116 0022542 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/apprecommender/tests/test_data/popcon_dir/ 0000775 0000000 0000000 00000000000 13067513116 0024676 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/apprecommender/tests/test_data/popcon_dir/test_popcon_0 0000664 0000000 0000000 00000000544 13067513116 0027400 0 ustar 00root root 0000000 0000000 POPULARITY-CONTEST-0 TIME:1309407492 ID:8b44fcdbcf676e711a153d5db0test_0 ARCH:i386 POPCONVER:1.52
1309407475 1303670994 gimp /usr/bin/perl
1309407451 1303670982 inkscape /lib/i686/cmov/libc-2.11.2.so
1309407450 1303670973 imagination /lib/ld-2.11.2.so
1309407434 1295654294 dash /bin/dash
0 0 libusbmuxd1
END-POPULARITY-CONTEST-0 TIME:1309407492
AppRecommender-0.7.5/apprecommender/tests/test_data/popcon_dir/test_popcon_1 0000664 0000000 0000000 00000000363 13067513116 0027400 0 ustar 00root root 0000000 0000000 POPULARITY-CONTEST-0 TIME:1309407492 ID:8b44fcdbcf676e711a153d5db0test_1 ARCH:i386 POPCONVER:1.52
1309407475 1303670994 gimp /usr/bin/perl
1309407434 1295654294 dash /bin/dash
0 0 libusbmuxd1
END-POPULARITY-CONTEST-0 TIME:1309407492
AppRecommender-0.7.5/apprecommender/tests/test_data/popcon_dir/test_popcon_2 0000664 0000000 0000000 00000000541 13067513116 0027377 0 ustar 00root root 0000000 0000000 POPULARITY-CONTEST-0 TIME:1309407492 ID:8b44fcdbcf676e711a153d5db0test_2 ARCH:i386 POPCONVER:1.52
1309407475 1303670994 iceweasel /usr/bin/perl
1309407451 1303670982 python /lib/i686/cmov/libc-2.11.2.so
1309407450 1303670973 libc6 /lib/ld-2.11.2.so
1309407434 1295654294 dash /bin/dash
0 0 libusbmuxd1
END-POPULARITY-CONTEST-0 TIME:1309407492
AppRecommender-0.7.5/apprecommender/tests/test_data/popcon_dir/test_popcon_3 0000664 0000000 0000000 00000000470 13067513116 0027401 0 ustar 00root root 0000000 0000000 POPULARITY-CONTEST-0 TIME:1309407492 ID:8b44fcdbcf676e711a153d5db0test_3 ARCH:i386 POPCONVER:1.52
1309407475 1303670994 eog /usr/bin/perl
1309407451 1303670982 nautilus /lib/i686/cmov/libc-2.11.2.so
1309407450 1303670973 libc6 /lib/ld-2.11.2.so
0 0 libusbmuxd1
END-POPULARITY-CONTEST-0 TIME:1309407492
AppRecommender-0.7.5/apprecommender/tests/test_data/popcon_dir/test_popcon_4 0000664 0000000 0000000 00000000473 13067513116 0027405 0 ustar 00root root 0000000 0000000 POPULARITY-CONTEST-0 TIME:1309407492 ID:8b44fcdbcf676e711a153d5db0test_4 ARCH:i386 POPCONVER:1.52
1309407475 1303670994 konqueror /usr/bin/perl
1309407451 1303670982 kedit /lib/i686/cmov/libc-2.11.2.so
1309407450 1303670973 libc6 /lib/ld-2.11.2.so
0 0 libusbmuxd1
END-POPULARITY-CONTEST-0 TIME:1309407492
AppRecommender-0.7.5/apprecommender/tests/test_data/popcon_dir/test_popcon_5 0000664 0000000 0000000 00000000370 13067513116 0027402 0 ustar 00root root 0000000 0000000 POPULARITY-CONTEST-0 TIME:1309407492 ID:8b44fcdbcf676e711a153d5db0test_5 ARCH:i386 POPCONVER:1.52
1309407475 1303670994 konqueror /usr/bin/perl
1309407434 1295654294 dash /bin/dash
0 0 libusbmuxd1
END-POPULARITY-CONTEST-0 TIME:1309407492
AppRecommender-0.7.5/apprecommender/tests/test_data/popcon_dir/test_popcon_6 0000664 0000000 0000000 00000000541 13067513116 0027403 0 ustar 00root root 0000000 0000000 POPULARITY-CONTEST-0 TIME:1309407492 ID:8b44fcdbcf676e711a153d5db0test_6 ARCH:i386 POPCONVER:1.52
1309407475 1303670994 perl-base /usr/bin/perl
1309407451 1303670982 eog /lib/i686/cmov/libc-2.11.2.so
1309407450 1303670973 nautilus /lib/ld-2.11.2.so
1309407434 1295654294 dash /bin/dash
0 0 libusbmuxd1
END-POPULARITY-CONTEST-0 TIME:1309407492
AppRecommender-0.7.5/apprecommender/tests/test_data/popcon_dir/test_popcon_7 0000664 0000000 0000000 00000000540 13067513116 0027403 0 ustar 00root root 0000000 0000000 POPULARITY-CONTEST-0 TIME:1309407492 ID:8b44fcdbcf676e711a153d5db0test_7 ARCH:i386 POPCONVER:1.52
1309407475 1303670994 apticron /usr/bin/perl
1309407451 1303670982 aptitude /lib/i686/cmov/libc-2.11.2.so
1309407450 1303670973 apt /lib/ld-2.11.2.so
1309407434 1295654294 dash /bin/dash
0 0 libusbmuxd1
END-POPULARITY-CONTEST-0 TIME:1309407492
AppRecommender-0.7.5/apprecommender/tests/test_data/popcon_dir/test_popcon_8 0000664 0000000 0000000 00000000540 13067513116 0027404 0 ustar 00root root 0000000 0000000 POPULARITY-CONTEST-0 TIME:1309407492 ID:8b44fcdbcf676e711a153d5db0test_8 ARCH:i386 POPCONVER:1.52
1309407475 1303670994 apticron /usr/bin/perl
1309407451 1303670982 eog /lib/i686/cmov/libc-2.11.2.so
1309407450 1303670973 nautilus /lib/ld-2.11.2.so
1309407434 1295654294 dash /bin/dash
0 0 libusbmuxd1
END-POPULARITY-CONTEST-0 TIME:1309407492
AppRecommender-0.7.5/apprecommender/tests/test_data/popcon_dir/test_popcon_9 0000664 0000000 0000000 00000000545 13067513116 0027412 0 ustar 00root root 0000000 0000000 POPULARITY-CONTEST-0 TIME:1309407492 ID:8b44fcdbcf676e711a153d5db0test_9 ARCH:i386 POPCONVER:1.52
1309407475 1303670994 perl-base /usr/bin/perl
1309407451 1303670982 libc6-i686 /lib/i686/cmov/libc-2.11.2.so
1309407450 1303670973 libc6 /lib/ld-2.11.2.so
1309407434 1295654294 dash /bin/dash
0 0 libusbmuxd1
END-POPULARITY-CONTEST-0 TIME:1309407492
AppRecommender-0.7.5/apprecommender/tests/test_data/tags_filter 0000664 0000000 0000000 00000012447 13067513116 0025000 0 ustar 00root root 0000000 0000000 accessibility::input
accessibility::ocr
accessibility::screen-magnify
accessibility::screen-reader
accessibility::speech
accessibility::speech-recognition
admin::accounting
admin::automation
admin::backup
admin::benchmarking
admin::boot
admin::cluster
admin::configuring
admin::file-distribution
admin::filesystem
admin::forensics
admin::hardware
admin::install
admin::issuetracker
admin::kernel
admin::logging
admin::login
admin::monitoring
admin::package-management
admin::power-management
admin::recovery
admin::user-management
admin::virtualization
biology::emboss
biology::format:aln
biology::format:fasta
biology::format:nexus
biology::nucleic-acids
biology::peptidic
devel::bugtracker
devel::buildtools
devel::code-generator
devel::compiler
devel::debian
devel::debugger
devel::doc
devel::docsystem
devel::ecma-cli
devel::editor
devel::examples
devel::ide
devel::interpreter
devel::i18n
devel::library
devel::machinecode
devel::modelling
devel::packaging
devel::prettyprint
devel::profiler
devel::rcs
devel::rpc
devel::runtime
devel::testing-qa
devel::ui-builder
devel::web
field::arts
field::astronomy
field::aviation
field::biology
field::biology:bioinformatics
field::biology:molecular
field::biology:structural
field::chemistry
field::computer-science
field::electronics
field::finance
field::genealogy
field::geography
field::geology
field::linguistics
field::mathematics
field::medicine
field::medicine:imaging
field::meteorology
field::physics
field::religion
field::statistics
game::adventure
game::arcade
game::board
game::board:chess
game::card
game::demos
game::fps
game::mud
game::platform
game::puzzle
game::rpg
game::rpg:rogue
game::simulation
game::sport
game::sport:racing
game::strategy
game::tetris
game::toys
game::typing
junior::arcade
junior::games-gl
junior::meta
mail::filters
mail::imap
mail::list
mail::notification
mail::pop
mail::smtp
mail::delivery-agent
mail::transport-agent
mail::user-agent
office::finance
office::groupware
office::presentation
office::project-management
office::spreadsheet
works-with::3dmodel
works-with::archive
works-with::audio
works-with::biological-sequence
works-with::bugs
works-with::calendar
works-with::db
works-with::dictionary
works-with::dtp
works-with::fax
works-with::file
works-with::font
works-with::graphs
works-with::im
works-with::logfile
works-with::mail
works-with::music-notation
works-with::network-traffic
works-with::people
works-with::pim
works-with::image
works-with::image:raster
works-with::image:vector
works-with::semantic-data
works-with::software:package
works-with::software:running
works-with::software:source
works-with::spreadsheet
works-with::text
works-with::unicode
works-with::vcs
works-with::video
works-with-format::bib
works-with-format::chm
works-with-format::diff
works-with-format::djvu
works-with-format::docbook
works-with-format::dvi
works-with-format::elf
works-with-format::epub
works-with-format::gif
works-with-format::xml:gpx
works-with-format::html
works-with-format::ical
works-with-format::info
works-with-format::iso9660
works-with-format::jpg
works-with-format::json
works-with-format::ldif
works-with-format::man
works-with-format::mp3
works-with-format::mpc
works-with-format::odf
works-with-format::oggtheora
works-with-format::oggvorbis
works-with-format::pdf
works-with-format::plaintext
works-with-format::png
works-with-format::po
works-with-format::postscript
works-with-format::rdf:json-ld
works-with-format::rdf:n3
works-with-format::rdf:ntriples
works-with-format::rdf:turtle
works-with-format::rdf:xml
works-with-format::sgml
works-with-format::svg
works-with-format::swf
works-with-format::tar
works-with-format::tex
works-with-format::tiff
works-with-format::vrml
works-with-format::wav
works-with-format::xml
works-with-format::xml:rss
works-with-format::xml:xslt
works-with-format::zip
system::cloud
system::embedded
system::laptop
system::mobile
system::server
system::virtual
security::antivirus
security::authentication
security::cryptography
security::firewall
security::forensics
security::ids
security::integrity
security::log-analyzer
security::privacy
sound::compression
sound::midi
sound::mixer
sound::player
sound::recorder
sound::sequencer
sound::speech
suite::apache
suite::bsd
suite::debian
suite::eclipse
suite::emacs
suite::gforge
suite::gimp
suite::gkrellm
suite::gnome
suite::gnu
suite::gnustep
suite::gpe
suite::kde
suite::mozilla
suite::netscape
suite::openoffice
suite::openstack
suite::opie
suite::postgresql
suite::roxen
suite::samba
suite::sugar
suite::webmin
suite::xfce
suite::xmms2
suite::zope
use::analysing
use::browsing
use::calculating
use::chatting
use::checking
use::comparing
use::compressing
use::configuring
use::converting
use::dialing
use::downloading
use::driver
use::editing
use::entertaining
use::filtering
use::gameplaying
use::learning
use::login
use::measuring
use::monitor
use::organizing
use::playing
use::printing
use::proxying
use::routing
use::searching
use::scanning
use::simulating
use::storing
use::synchronizing
use::timekeeping
use::transmission
use::typesetting
use::viewing
use::text-formatting
web::application
web::appserver
web::blog
web::browser
web::cms
web::cgi
web::commerce
web::forum
web::microblog
web::portal
web::scripting
web::search-engine
web::server
web::wiki
science::calculation
science::modelling
science::data-acquisition
science::plotting
science::bibliography
science::publishing
science::visualisation
AppRecommender-0.7.5/apprecommender/tests/test_data/test_popcon 0000664 0000000 0000000 00000000545 13067513116 0025026 0 ustar 00root root 0000000 0000000 POPULARITY-CONTEST-0 TIME:1309407492 ID:8b44fcdbcf676e711a153d5db099test ARCH:i386 POPCONVER:1.52
1309407475 1303670994 perl-base /usr/bin/perl
1309407451 1303670982 libc6-i686 /lib/i686/cmov/libc-2.11.2.so
1309407450 1303670973 libc6 /lib/ld-2.11.2.so
1309407434 1295654294 dash /bin/dash
0 0 libusbmuxd1
END-POPULARITY-CONTEST-0 TIME:1309407492
AppRecommender-0.7.5/apprecommender/tests/test_data_classification.py 0000664 0000000 0000000 00000000556 13067513116 0026175 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import unittest
from apprecommender.data_classification import linear_percent_function
class DataClassificationTests(unittest.TestCase):
def test_linear_percent_function(self):
modify, access, time_now = 100, 175, 200
percent = linear_percent_function(modify, access, time_now)
self.assertEqual(0.75, percent)
AppRecommender-0.7.5/apprecommender/tests/test_decider.py 0000664 0000000 0000000 00000007036 13067513116 0023610 0 ustar 00root root 0000000 0000000 import unittest
from apprecommender.decider import PkgInitDecider
class PkgInitDeciderTests(unittest.TestCase):
def setUp(self):
self.pkg_init_decider = PkgInitDecider()
def test_python_pkg_regex(self):
pkg = 'python-test'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'python3-test'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
def test_ruby_pkg_regex(self):
pkg = 'ruby-test'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
def test_texlive_pkg_regex(self):
pkg = 'texlive-test'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
def test_pkg_gir_regex(self):
pkg = 'gir1.2-test'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
def test_pkg_golang_regex(self):
pkg = 'golang-test'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
def test_pkg_data_regex(self):
pkg = 'test-data'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
def test_pkg_dev_regex(self):
pkg = 'test-dev'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
def test_pkg_utils_regex(self):
pkg = 'test-utils'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-utils-1.9'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
def test_pkg_common_regex(self):
pkg = 'test-common'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
def test_pkg_fonts_regex(self):
pkg = 'test-fonts'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
def test_pkg_png_regex(self):
pkg = 'test-png'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
def test_pkg_core_regex(self):
pkg = 'test-core'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
def test_pkg_default_regex(self):
pkg = 'test-default'
self.assertTrue(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
pkg = 'test-pkg'
self.assertFalse(self.pkg_init_decider.is_pkg_a_prefix_or_suffix(pkg))
AppRecommender-0.7.5/apprecommender/tests/test_evaluation.py 0000664 0000000 0000000 00000006771 13067513116 0024365 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
evaluationTests - Evaluation class test case
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import unittest
from apprecommender.evaluation import (Accuracy, Precision, Recall, Coverage,
Evaluation, CrossValidationRecommender)
from apprecommender.recommender import RecommendationResult
class MetricsTests(unittest.TestCase):
@classmethod
def setUpClass(self):
repository = ['apple', 'grape', 'pineaple', 'melon',
'watermelon', 'orange']
real = RecommendationResult(dict.fromkeys(['apple', 'grape',
'pineaple', 'melon'], 1))
predicted = RecommendationResult(dict.fromkeys(['apple', 'grape',
'orange'], 1))
self.evaluation = Evaluation(predicted, real, len(repository))
def test_class_accuracy(self):
accuracy = Accuracy().run(self.evaluation)
self.assertEqual(accuracy, 0.5)
def test_precision(self):
precision = Precision().run(self.evaluation)
self.assertEqual("%.2f" % precision, "0.67")
def test_recall(self):
recall = Recall().run(self.evaluation)
self.assertEqual(recall, 0.5)
def test_coverage(self):
evaluations_set = set()
evaluations_set.add(self.evaluation)
coverage = Coverage().run(evaluations_set)
self.assertEqual(coverage, 0.5)
def test_evaluation(self):
self.assertEqual(self.evaluation.true_positive, ['apple', 'grape'])
self.assertEqual(self.evaluation.false_positive, ['orange'])
self.assertEqual(self.evaluation.false_negative, ['pineaple', 'melon'])
class CrossValidationTests(unittest.TestCase):
def create_cross_validation(self, partition_proportion, rounds, rec,
metrics_list, result_proportion):
return CrossValidationRecommender(partition_proportion, rounds, rec,
metrics_list, result_proportion)
def test_get_partition_size(self):
cross_validation = self.create_cross_validation(0.7, 1, None, [], 1)
cross_item_score = {'item1': [1, 0, 1],
'item2': [1, 0, 1],
'item3': [1, 0, 1],
'item4': [1, 0, 1],
'item5': [1, 0, 1],
'item6': [1, 0, 1],
'item7': [1, 0, 1],
'item8': [1, 0, 1],
'item9': [1, 0, 1],
'item10': [1, 0, 1]}
expected_result = 7
self.assertEqual(expected_result,
cross_validation.get_partition_size(cross_item_score))
AppRecommender-0.7.5/apprecommender/tests/test_initialize.py 0000664 0000000 0000000 00000001057 13067513116 0024347 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import unittest
from apprecommender.initialize import Initialize
class InitializeTests(unittest.TestCase):
def test_get_tags(self):
initialize = Initialize()
tags = initialize.get_tags()
self.assertTrue(len(tags) > 0)
combined_tags = ''.join(tags)
for tag in Initialize.EXCLUDED_TAGS:
self.assertTrue(tag not in combined_tags)
def test_get_axipkgs(self):
initialize = Initialize()
pkgs = initialize.get_axipkgs()
self.assertTrue(len(pkgs) > 0)
AppRecommender-0.7.5/apprecommender/tests/test_ml/ 0000775 0000000 0000000 00000000000 13067513116 0022241 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/apprecommender/tests/test_ml/test_bayes_matrix.py 0000664 0000000 0000000 00000010427 13067513116 0026345 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import unittest
import numpy as np
from apprecommender.ml.bayes_matrix import BayesMatrix
class PkgClassificationTests(unittest.TestCase):
def setUp(self):
self.bayes_matrix = BayesMatrix()
def test_training(self):
data_matrix = np.matrix("1 0 1 0 1; 0 1 1 0 1; 1 0 0 1 1; 1 0 1 1 0;\
0 1 1 1 0")
classifications = np.matrix([[1], [2], [0], [1], [2]])
order_of_classifications = [0, 1, 2]
prob = np.matrix("1 0 0 1 1; 1 0 1 0.5 0.5; 0 1 1 0.5 0.5").astype(
float)
self.bayes_matrix = BayesMatrix()
self.bayes_matrix.training(data_matrix, classifications,
order_of_classifications)
self.assertTrue(np.allclose(prob, self.bayes_matrix.prob))
def test_get_classification(self):
self.bayes_matrix = BayesMatrix()
self.bayes_matrix.data = np.matrix(
"1 0 1 0 1; 0 1 1 0 1; 1 0 0 1 1; 1 0 1 1 0;\
0 1 1 1 0").astype(float)
self.bayes_matrix.labels = np.matrix("0; 1; 2").astype(float)
self.bayes_matrix.label_probability = (np.matrix(
"0.2; 0.4; 0.4").astype(float))
self.bayes_matrix.order_of_classifications = [0, 1, 2]
self.bayes_matrix.used_order_of_classifications = [0, 1, 2]
self.bayes_matrix.prob = np.matrix(
"1 0 0 1 1; 1 0 1 0.5 0.5; 0 1 1 0.5 0.5").astype(float)
attribute_vector = np.matrix("1 0 1 1 0")
self.assertEqual(
1, self.bayes_matrix.get_classification(attribute_vector))
def test_convert_classifications_to_numbers(self):
classifications = np.matrix([['M'], ['B'], ['G']])
order_of_classifications = ['B', 'M', 'G']
expected = np.array([[1], [0], [2]])
actual = self.bayes_matrix.convert_classifications_to_number(
classifications, order_of_classifications)
for index, label in enumerate(expected):
self.assertEqual(label[0], actual[index, 0])
def test_get_used_order_of_classifications(self):
classifications = np.matrix([['B'], ['G']])
order_of_classifications = ['B', 'M', 'G']
expected = ['B', 'G']
actual = self.bayes_matrix.get_used_order_of_classifications(
classifications, order_of_classifications)
self.assertEquals(expected, actual)
classifications = np.matrix([['M'], ['M'], ['B'], ['G']])
order_of_classifications = ['B', 'M', 'G']
expected = ['B', 'M', 'G']
actual = self.bayes_matrix.get_used_order_of_classifications(
classifications, order_of_classifications)
self.assertEquals(expected, actual)
classifications = np.matrix([['M'], ['G']])
order_of_classifications = ['B', 'M', 'G']
expected = ['M', 'G']
actual = self.bayes_matrix.get_used_order_of_classifications(
classifications, order_of_classifications)
classifications = np.matrix([['B']])
order_of_classifications = ['B', 'M', 'G']
expected = ['B']
actual = self.bayes_matrix.get_used_order_of_classifications(
classifications, order_of_classifications)
classifications = np.matrix([['B'], ['D'], ['F']])
order_of_classifications = ['A', 'B', 'C', 'D', 'E', 'F', 'G']
expected = ['B', 'D', 'F']
actual = self.bayes_matrix.get_used_order_of_classifications(
classifications, order_of_classifications)
self.assertEquals(expected, actual)
classifications = np.matrix([['B'], ['D'], ['F']])
order_of_classifications = ['A', 'B', 'C', 'D', 'E', 'F']
expected = ['B', 'D', 'F']
actual = self.bayes_matrix.get_used_order_of_classifications(
classifications, order_of_classifications)
self.assertEquals(expected, actual)
def test_dont_change_labels_when_run_training(self):
data_matrix = np.matrix("1 0 1 0 1; 0 1 1 0 1; 1 0 0 1 1; 1 0 1 1 0")
classifications = np.matrix([[1], [2], [1], [2]])
labels = [0, 1, 2]
self.bayes_matrix = BayesMatrix()
self.bayes_matrix.training(data_matrix, classifications,
labels)
expected_labels = [0, 1, 2]
self.assertEquals(expected_labels, labels)
AppRecommender-0.7.5/apprecommender/tests/test_ml/test_evaluation.py 0000664 0000000 0000000 00000026657 13067513116 0026041 0 ustar 00root root 0000000 0000000 import unittest
from collections import OrderedDict
from numpy import array, matrix
from apprecommender.ml.cross_validation import (ConfusionMatrix,
CrossValidationBVA,
Evaluation)
from apprecommender.evaluation import (SimpleAccuracy, Precision, Recall, FPR,
F_score, MCC, Accuracy)
class ConfusionMatrixTest(unittest.TestCase):
def test_confusion_matrix_values(self):
predicted_results = array([[1], [0], [1], [0]])
real_results = array([[1], [0], [0], [1]])
confusion_matrix = ConfusionMatrix(predicted_results, real_results)
confusion_matrix.run()
tp, tn, fp, fn = 1, 1, 1, 1
self.assertEqual(tp, confusion_matrix.true_positive_len)
self.assertEqual(tn, confusion_matrix.true_negative_len)
self.assertEqual(fp, confusion_matrix.false_positive_len)
self.assertEqual(fn, confusion_matrix.false_negative_len)
predicted_results = array([[1], [1], [1], [0]])
real_results = array([[1], [1], [0], [1]])
confusion_matrix = ConfusionMatrix(predicted_results, real_results)
confusion_matrix.run()
tp, tn, fp, fn = 2, 0, 1, 1
self.assertEqual(tp, confusion_matrix.true_positive_len)
self.assertEqual(tn, confusion_matrix.true_negative_len)
self.assertEqual(fp, confusion_matrix.false_positive_len)
self.assertEqual(fn, confusion_matrix.false_negative_len)
class EvaluationTest(unittest.TestCase):
def create_evaluation(self, predicted_values, real_values, labels):
return Evaluation(predicted_values, real_values, labels)
def test_evaluation_simple_accuracy(self):
labels = [1, 2, 3]
predicted_values = array([[1], [2], [3], [2], [1]])
real_values = array([[1], [2], [2], [1], [3]])
metric = SimpleAccuracy()
evaluation = self.create_evaluation(predicted_values, real_values,
labels)
results = evaluation.run(metric)
expected_label_1 = 0.6
expected_label_2 = 0.6
expected_label_3 = 0.6
self.assertEquals(expected_label_1, results[1])
self.assertEquals(expected_label_2, results[2])
self.assertEquals(expected_label_3, results[3])
class CrossValidationTests(unittest.TestCase):
def create_cross_validation_ml(self, pkg_data, partition_proportion,
rounds, metrics_list, labels):
return CrossValidationBVA(
pkg_data, partition_proportion, rounds, metrics_list,
labels)
def compare_column_matrix(self, expected_matrix, actual_matrix):
self.assertEquals(expected_matrix.shape, actual_matrix.shape)
num_itens = actual_matrix.shape[0]
for i in range(num_itens):
self.assertEqual(expected_matrix[i, 0], actual_matrix[i, 0])
def test_get_real_results(self):
cross_validation_ml = self.create_cross_validation_ml(
None, 0.1, 1, [], [])
test_data = {'test1': [1, 0, 1, 0, 'T'], 'test2': [1, 1, 1, 0, 'F']}
expected_result = array([['T'], ['F']])
actual_result = cross_validation_ml.get_real_results(test_data)
self.compare_column_matrix(expected_result, actual_result)
def test_cross_validation_process(self):
data_matrix = (('test1', [1, 0, 1, 0, 1, 1, 1, 1, 'G']),
('test2', [0, 1, 0, 1, 1, 1, 1, 0, 'M']),
('test3', [1, 0, 1, 0, 1, 0, 0, 1, 'B']),
('test4', [0, 0, 0, 0, 1, 1, 1, 1, 'M']),
('test5', [1, 1, 1, 1, 0, 0, 0, 1, 'G']),
('test6', [1, 1, 0, 0, 1, 1, 0, 0, 'M']),
('test7', [0, 0, 1, 0, 1, 0, 1, 1, 'B']),
('test8', [1, 1, 1, 0, 0, 0, 1, 0, 'G']),
('test9', [0, 1, 1, 0, 0, 1, 1, 1, 'B']),
('test10', [1, 1, 1, 0, 1, 0, 0, 1, 'G']))
data_matrix = OrderedDict(data_matrix)
partition_proportion = 0.7
rounds = 1
metrics_list = [SimpleAccuracy()]
labels = ['B', 'M', 'G']
cross_validation_ml = CrossValidationBVA(
data_matrix, partition_proportion, rounds, metrics_list,
labels)
self.assertEquals(len(data_matrix),
len(cross_validation_ml.get_user_score(None)))
expected_num_B = 3 # noqa
expected_num_G = 4 # noqa
expected_num_M = 3 # noqa
for label in labels:
self.assertEqual(eval('expected_num_{0}'.format(label)),
len(cross_validation_ml.label_groups[label]))
expected_num_data = 10
self.assertEqual(expected_num_data, len(cross_validation_ml.pkg_data))
bayes_model = cross_validation_ml.get_model(data_matrix)
data = array([[1, 0, 1, 0, 1, 1, 1, 1],
[0, 1, 0, 1, 1, 1, 1, 0],
[1, 0, 1, 0, 1, 0, 0, 1],
[0, 0, 0, 0, 1, 1, 1, 1],
[1, 1, 1, 1, 0, 0, 0, 1],
[1, 1, 0, 0, 1, 1, 0, 0],
[0, 0, 1, 0, 1, 0, 1, 1],
[1, 1, 1, 0, 0, 0, 1, 0],
[0, 1, 1, 0, 0, 1, 1, 1],
[1, 1, 1, 0, 1, 0, 0, 1]])
labels_number = matrix([[0], [1], [2]])
classifications_numbers = matrix([[2], [1], [0], [1], [2], [1], [0],
[2], [0], [2]])
histogram = array([[3], [3], [4]])
actual_labels_number = bayes_model.labels
actual_classifications_number = bayes_model.classifications
actual_histogram = bayes_model.histogram
self.assertEquals(data.shape, bayes_model.data.shape)
self.compare_column_matrix(labels_number, actual_labels_number)
self.compare_column_matrix(classifications_numbers,
actual_classifications_number)
self.compare_column_matrix(histogram, actual_histogram)
predictions = cross_validation_ml.get_predicted_results(bayes_model,
data_matrix)
expected_shape = (10, 1)
self.assertEquals(expected_shape, predictions.shape)
for i in range(len(predictions)):
self.assertIn(predictions[i, 0], labels)
real_results = cross_validation_ml.get_real_results(data_matrix)
cross_validation_ml.run_metrics(predictions, real_results)
class MetricsTest(unittest.TestCase):
def test_simple_accuracy(self):
predicted_results = array([[1], [1], [0], [0]])
actual_results = array([[1], [1], [0], [1]])
labels = [0, 1]
metric = SimpleAccuracy()
evaluation = Evaluation(predicted_results, actual_results, labels)
results = evaluation.run(metric)
expected_1 = 0.75
expected_0 = 0.75
self.assertEquals(expected_1, results[1])
self.assertEquals(expected_0, results[0])
def test_precision(self):
predicted_results = array([[1], [1], [0], [0]])
actual_results = array([[1], [1], [0], [1]])
labels = [0, 1]
metric = Precision()
evaluation = Evaluation(predicted_results, actual_results, labels)
results = evaluation.run(metric)
expected_1 = 1
expected_0 = 0.5
self.assertEquals(expected_1, results[1])
self.assertEquals(expected_0, results[0])
predicted_results = array([[0], [0], [0], [0]])
evaluation = Evaluation(predicted_results, actual_results, labels)
results = evaluation.run(metric)
expected_1 = 0
expected_0 = 0.25
self.assertEquals(expected_1, results[1])
self.assertEquals(expected_0, results[0])
def test_recall(self):
predicted_results = array([[1], [1], [0], [0], [1]])
actual_results = array([[1], [1], [0], [1], [1]])
labels = [0, 1]
metric = Recall()
evaluation = Evaluation(predicted_results, actual_results, labels)
results = evaluation.run(metric)
expected_1 = 0.75
expected_0 = 1
self.assertEquals(expected_1, results[1])
self.assertEquals(expected_0, results[0])
predicted_results = array([[0], [0], [1], [0], [0]])
actual_results = array([[1], [1], [0], [1], [1]])
evaluation = Evaluation(predicted_results, actual_results, labels)
results = evaluation.run(metric)
expected_1 = 0
expected_0 = 0
self.assertEquals(expected_1, results[1])
self.assertEquals(expected_0, results[0])
def test_fpr(self):
predicted_results = array([[1], [1], [0], [1], [0]])
actual_results = array([[0], [0], [0], [1], [0]])
labels = [0, 1]
metric = FPR()
evaluation = Evaluation(predicted_results, actual_results, labels)
results = evaluation.run(metric)
expected_1 = 0.5
expected_0 = 0
self.assertEquals(expected_1, results[1])
self.assertEquals(expected_0, results[0])
def test_f1_score(self):
predicted_results = array([[1], [1], [0], [1], [0], [1]])
actual_results = array([[1], [1], [1], [0], [1], [1]])
labels = [0, 1]
metric = F_score(1)
evaluation = Evaluation(predicted_results, actual_results, labels)
results = evaluation.run(metric)
expected_1 = 0.6666666666666666
expected_0 = 0
self.assertAlmostEqual(expected_1, results[1])
self.assertEqual(expected_0, results[0])
def test_f0_5_score(self):
predicted_results = array([[1], [1], [0], [1], [0], [1]])
actual_results = array([[1], [1], [1], [0], [1], [1]])
labels = [0, 1]
metric = F_score(0.5)
evaluation = Evaluation(predicted_results, actual_results, labels)
results = evaluation.run(metric)
expected_1 = 0.71428571
expected_0 = 0
self.assertAlmostEqual(expected_1, results[1])
self.assertEqual(expected_0, results[0])
def test_f2_score(self):
predicted_results = array([[1], [1], [0], [1], [0], [1]])
actual_results = array([[1], [1], [1], [0], [1], [1]])
labels = [0, 1]
metric = F_score(2)
evaluation = Evaluation(predicted_results, actual_results, labels)
results = evaluation.run(metric)
expected_1 = 0.625
expected_0 = 0
self.assertAlmostEqual(expected_1, results[1])
self.assertEqual(expected_0, results[0])
def test_mcc_score(self):
predicted_results = array([[1], [1], [0], [1], [0], [1]])
actual_results = array([[1], [1], [1], [0], [0], [1]])
labels = [0, 1]
metric = MCC()
evaluation = Evaluation(predicted_results, actual_results, labels)
results = evaluation.run(metric)
expected_1, expected_0 = 0.25, 0.25
self.assertAlmostEqual(expected_1, results[1])
self.assertEqual(expected_0, results[0])
def test_accuracy(self):
predicted_results = array([[1], [1], [0], [1], [0], [1]])
actual_results = array([[1], [1], [1], [0], [0], [1]])
labels = [0, 1]
metric = Accuracy()
evaluation = Evaluation(predicted_results, actual_results, labels)
results = evaluation.run(metric)
expected_1, expected_0 = 0.625, 0.625
self.assertAlmostEqual(expected_1, results[1])
self.assertEqual(expected_0, results[0])
AppRecommender-0.7.5/apprecommender/tests/test_ml/test_pkg_classification.py 0000664 0000000 0000000 00000005123 13067513116 0027507 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import unittest
import xapian
from mock import patch
from apprecommender.apt_cache import AptCache
from apprecommender.ml.data import MachineLearningData
class PkgClassificationTests(unittest.TestCase):
def setUp(self):
self.ml_data = MachineLearningData()
self.cache = AptCache()
def test_get_pkg_debtags(self):
vim_debtags = ['devel::editor', 'implemented-in::c',
'interface::commandline', 'interface::text-mode',
'role::program', 'scope::application',
'uitoolkit::ncurses', 'use::editing',
'works-with::text', 'works-with::unicode']
axi_path = "/var/lib/apt-xapian-index/index"
axi = xapian.Database(axi_path)
vim_debtags_result = self.ml_data.get_pkg_debtags(axi, 'vim')
for debtag in vim_debtags:
self.assertTrue(debtag in vim_debtags_result)
@patch('apprecommender.ml.data.MachineLearningData.get_pkg_description')
def test_get_pkg_terms(self, mock_description):
mock_description.return_value = 'Vim is an text editor written in C'
vim_terms = [u'vim', u'text', u'editor']
vim_terms_result = self.ml_data.get_pkg_terms(self.cache, 'vim')
for term in vim_terms:
self.assertTrue(term in vim_terms_result)
def test_create_row_table_list(self):
labels_name = ['devel::editor', 'implemented-in::c', 'complet',
'contain', 'syntax', 'unix', 'version']
pkg_elements = ['implemented-in::c', 'complet']
row_list_to_assert = [0, 1, 1, 0, 0, 0, 0]
row_list = self.ml_data.create_row_table_list(labels_name,
pkg_elements)
self.assertEqual(row_list_to_assert, row_list)
@patch('apprecommender.ml.data.MachineLearningData.get_pkg_description')
def test_get_pkg_classification(self, mock_description):
mock_description.return_value = 'vim is an text editor written in c'
axi_path = "/var/lib/apt-xapian-index/index"
axi = xapian.Database(axi_path)
pkgs = {'vim': 'EX'}
debtags_name = ['devel::editor', 'implemented-in::c',
'devel::interpreter', 'devel::lang:python']
terms_name = ['vim', 'editor', 'python']
assert_pkgs_classification = {'vim': [1, 1, 0, 0, 1, 1, 0, 'EX']}
pkgs_classification = self.ml_data.get_pkgs_table_classification(
axi, pkgs, self.cache, debtags_name, terms_name)
self.assertEqual(assert_pkgs_classification, pkgs_classification)
AppRecommender-0.7.5/apprecommender/tests/test_ml/test_pkg_time.py 0000664 0000000 0000000 00000007061 13067513116 0025455 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import unittest
from mock import patch
from apprecommender.ml.pkg_time import PkgTime
class PkgTimeTests(unittest.TestCase):
def setUp(self):
self.pkg_time = PkgTime()
@patch('commands.getoutput')
@patch('apprecommender.ml.pkg_time.get_time_from_package')
def test_invalid_paths_get_best_time(self, mock_time, mock_command):
mock_time.return_value = [10, 10]
mock_command.return_value = '/usr/lib/a-b-c/\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 0)
self.assertEqual(modify, 0)
mock_command.return_value = '/usr/lib/a-b-c-d/\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 0)
self.assertEqual(modify, 0)
mock_command.return_value = '/usr/lib/mime/packages/test\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 0)
self.assertEqual(modify, 0)
mock_command.return_value = '/etc/init.d/test\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 0)
self.assertEqual(modify, 0)
mock_command.return_value = '/media/files/test\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 0)
self.assertEqual(modify, 0)
@patch('commands.getoutput')
@patch('apprecommender.ml.pkg_time.get_time_from_package')
def test_valid_paths_get_best_time(self, mock_time, mock_command):
mock_time.return_value = [10, 10]
mock_command.return_value = '/usr/lib/a-b/\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 10)
self.assertEqual(modify, 10)
mock_time.return_value = [20, 20]
mock_command.return_value = '/usr/bin/test\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 20)
self.assertEqual(modify, 20)
mock_time.return_value = [30, 30]
mock_command.return_value = '/usr/game/test\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 30)
self.assertEqual(modify, 30)
mock_time.return_value = [40, 40]
mock_command.return_value = '/usr/lib/test/test\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 40)
self.assertEqual(modify, 40)
@patch('commands.getoutput')
@patch('apprecommender.ml.pkg_time.get_time_from_package')
def test_invalid_files_get_best_time(self, mock_time, mock_command):
mock_time.return_value = [10, 10]
mock_command.return_value = '/usr/bin/test.desktop\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 0)
self.assertEqual(modify, 0)
mock_time.return_value = [10, 10]
mock_command.return_value = '/usr/games/test.conf\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 0)
self.assertEqual(modify, 0)
mock_time.return_value = [10, 10]
mock_command.return_value = '/usr/lib/python/test.egg-info\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 0)
self.assertEqual(modify, 0)
mock_time.return_value = [10, 10]
mock_command.return_value = '/usr/lib/test/test.txt\n'
access, modify = self.pkg_time.get_best_time('test')
self.assertEqual(access, 0)
self.assertEqual(modify, 0)
AppRecommender-0.7.5/apprecommender/tests/test_ml/test_utils.py 0000664 0000000 0000000 00000001262 13067513116 0025013 0 ustar 00root root 0000000 0000000 import unittest
from numpy import array
from apprecommender.ml.utils import create_binary_matrix
class MLUtilsTest(unittest.TestCase):
def test_create_binary_matrix(self):
test_matrix = array([[2], [3], [2], [4]])
classification = 2
default_value = 0
expected_result = array([[1], [0], [1], [0]])
actual_result = create_binary_matrix(test_matrix, classification,
default_value)
self.assertEquals(expected_result.shape, actual_result.shape)
num_data = len(actual_result)
for i in range(num_data):
self.assertEqual(expected_result[i][0], actual_result[i][0])
AppRecommender-0.7.5/apprecommender/tests/test_recommender.py 0000664 0000000 0000000 00000007724 13067513116 0024515 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
recommenderTests - Recommender class test case
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import unittest
from apprecommender.recommender import RecommendationResult, Recommender
from apprecommender.user import User
from apprecommender.config import Config
from apprecommender.strategy import (ContentBased, MachineLearningBVA,
MachineLearningBOW, PackageReference)
class RecommendationResultTests(unittest.TestCase):
@classmethod
def setUpClass(self):
self.result = RecommendationResult({"gimp": 1.5, "inkscape": 3.0,
"eog": 1})
def test_str(self):
rec = '\n1: inkscape \t vector-based drawing program\n'
rec += '2: gimp \t GNU Image Manipulation Program\n' # noqa
rec += '3: eog \t Eye of GNOME graphics viewer program\n' # noqa
self.assertEqual(self.result.__str__(), rec)
def test_get_prediction(self):
prediction = [("inkscape", 3.0), ("gimp", 1.5), ("eog", 1)]
self.assertEqual(self.result.get_prediction(), prediction)
class RecommenderTests(unittest.TestCase):
@classmethod
def setUpClass(self):
cfg = Config()
cfg.popcon_index = "test_data/.sample_pxi"
cfg.popcon_dir = "test_data/popcon_dir"
cfg.clusters_dir = "test_data/clusters_dir"
cfg.popcon = 0
self.rec = Recommender()
def test_set_strategy(self):
self.rec.set_strategy("cb")
self.assertIsInstance(self.rec.strategy, ContentBased)
self.assertEqual(self.rec.strategy.content, "mix")
self.rec.set_strategy("cbt")
self.assertIsInstance(self.rec.strategy, ContentBased)
self.assertEqual(self.rec.strategy.content, "tag")
self.rec.set_strategy("cbd")
self.assertIsInstance(self.rec.strategy, ContentBased)
self.assertEqual(self.rec.strategy.content, "desc")
self.rec.set_strategy("cbtm")
self.assertIsInstance(self.rec.strategy, ContentBased)
self.assertEqual(self.rec.strategy.content, "time")
self.rec.set_strategy("mlbva")
self.assertIsInstance(self.rec.strategy, MachineLearningBVA)
self.assertEqual(self.rec.strategy.content, "mlbva_mix")
self.rec.set_strategy("mlbow")
self.assertIsInstance(self.rec.strategy, MachineLearningBOW)
self.assertEqual(self.rec.strategy.content, "mlbow_mix")
self.rec.set_strategy("mlbva_eset")
self.assertIsInstance(self.rec.strategy, MachineLearningBVA)
self.assertEqual(self.rec.strategy.content, "mlbva_mix_eset")
self.rec.set_strategy("mlbow_eset")
self.assertIsInstance(self.rec.strategy, MachineLearningBOW)
self.assertEqual(self.rec.strategy.content, "mlbow_mix_eset")
self.rec.set_strategy("cbpkg")
self.assertIsInstance(self.rec.strategy, PackageReference)
self.assertEqual(self.rec.strategy.content, "mix")
def test_get_recommendation(self):
user = User({"inkscape": 1, "gimp": 1, "eog": 1, "vim": 1})
result = self.rec.get_recommendation(user)
self.assertIsInstance(result, RecommendationResult)
self.assertGreater(len(result.item_score), 0)
AppRecommender-0.7.5/apprecommender/tests/test_run.py 0000664 0000000 0000000 00000002566 13067513116 0023020 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import unittest
import logging
import apprecommender.main.cli as apprec
from apprecommender.main.options import get_parser
from apprecommender.config import Config
from apprecommender.ml.data import MachineLearningData
class RunTests(unittest.TestCase):
def setUp(self):
logging.getLogger().disabled = True
self.axi_desktopapps = Config().axi_desktopapps
parser = get_parser()
self.args = vars(parser.parse_args(''))
def tearDown(self):
Config().axi_desktopapps = self.axi_desktopapps
def test_success_run_apprec(self):
logging.getLogger().disabled = False
result = apprec.run(self.args)
self.assertEqual(apprec.SUCCESS, result)
def test_error_init_on_run_apprec(self):
Config().axi_desktopapps = "asd"
result = apprec.run(self.args)
self.assertEqual(apprec.ERROR_INIT, result)
def test_error_train_on_run_apprec(self):
config = Config()
strategy = config.strategy
config.strategy = 'mlbva'
training_path = MachineLearningData.MACHINE_LEARNING_TRAINING
MachineLearningData.MACHINE_LEARNING_TRAINING = "error.txt"
result = apprec.run(self.args)
config.strategy = strategy
MachineLearningData.MACHINE_LEARNING_TRAINING = training_path
self.assertEqual(apprec.ERROR_TRAIN, result)
AppRecommender-0.7.5/apprecommender/tests/test_singleton.py 0000664 0000000 0000000 00000002133 13067513116 0024204 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
singletonTests - Singleton class test case
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import unittest
from apprecommender.singleton import Singleton
class SingletonTests(unittest.TestCase):
def test_creation(self):
object_1 = Singleton()
object_2 = Singleton()
self.assertEqual(id(object_1), id(object_2))
AppRecommender-0.7.5/apprecommender/tests/test_strategy.py 0000664 0000000 0000000 00000004013 13067513116 0024043 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
strategyTests - Recommendation strategies classes test case
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import unittest
import xapian
from apprecommender.decider import (PkgMatchDecider, PkgExpandDecider,
TagExpandDecider)
class PkgMatchDeciderTests(unittest.TestCase):
def setUp(self):
pkgs_list = ["gimp", "eog", "inkscape"]
self.decider = PkgMatchDecider(pkgs_list)
self.doc = xapian.Document()
def test_match(self):
self.doc.set_data("emacs")
self.assertTrue(self.decider(self.doc))
def test_no_match(self):
self.doc.set_data("gimp")
self.assertFalse(self.decider(self.doc))
class PkgExpandDeciderTests(unittest.TestCase):
def setUp(self):
pkgs_list = ["gimp", "eog", "inkscape"]
self.decider = PkgExpandDecider(pkgs_list)
def test_match(self):
self.assertTrue(self.decider("XPemacs"))
def test_no_match(self):
self.assertFalse(self.decider("XTgimp"))
class TagExpandDeciderTests(unittest.TestCase):
def setUp(self):
self.decider = TagExpandDecider()
def test_match(self):
self.assertTrue(self.decider("XTgimp"))
def test_no_match(self):
self.assertFalse(self.decider("gimp"))
AppRecommender-0.7.5/apprecommender/tests/test_user.py 0000664 0000000 0000000 00000014603 13067513116 0023165 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
userTests - User class test case
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import unittest
import xapian
from mock import patch
from apprecommender.user import User, LocalSystem, FilterTag, FilterDescription
from apprecommender.config import Config
from apprecommender.data import SampleAptXapianIndex
class FilterTagTests(unittest.TestCase):
@classmethod
def setUpClass(self):
tags_filter = "apprecommender/tests/test_data/tags_filter"
self.debtag = "use::searching"
with open(tags_filter) as valid_tags:
self.valid_tags = [line.strip() for line in valid_tags]
def test_call_true(self):
self.assertTrue(FilterTag(self.valid_tags)("XT" + self.debtag))
def test_call_false(self):
self.assertFalse(FilterTag(self.valid_tags)(self.debtag))
class FilterDescriptionTests(unittest.TestCase):
def test_call_true(self):
self.assertTrue(FilterDescription()("program"))
def test_call_false(self):
self.assertFalse(FilterDescription()("XTprogram"))
class UserTests(unittest.TestCase):
@classmethod
def setUpClass(self):
cfg = Config()
self.axi = xapian.Database(cfg.axi)
packages = ["gimp", "aaphoto", "eog", "emacs", "dia", "ferret",
"festival", "file", "inkscape", "xpdf"]
path = "apprecommender/tests/test_data/.sample_axi"
self.sample_axi = SampleAptXapianIndex(packages, self.axi, path)
self.user = User({"gimp": 1, "aaphoto": 1, "eog": 1, "emacs": 1})
def test_hash(self):
new_user = User(dict())
self.assertIsNotNone(new_user.id)
self.assertNotEqual(self.user.id, new_user.id)
def test_profile_default(self):
new_user = User(dict())
desktop = set(["x11", "accessibility", "game", "junior", "office",
"interface::x11"])
self.assertEqual(new_user.demographic_profile, desktop)
def test_profile_desktop(self):
self.user.set_demographic_profile(set(["desktop"]))
desktop = set(["x11", "accessibility", "game", "junior", "office",
"interface::x11"])
self.assertEqual(self.user.demographic_profile, desktop)
def test_profile_admin(self):
self.user.set_demographic_profile(set(["admin"]))
admin = set(["admin", "hardware", "mail", "protocol",
"network", "security", "web", "interface::web"])
self.assertEqual(self.user.demographic_profile, admin)
def test_profile_devel(self):
self.user.set_demographic_profile(set(["devel"]))
devel = set(["devel", "role::devel-lib", "role::shared-lib"])
self.assertEqual(self.user.demographic_profile, devel)
def test_profile_art(self):
self.user.set_demographic_profile(set(["art"]))
art = set(["field::arts", "sound"])
self.assertEqual(self.user.demographic_profile, art)
def test_profile_science(self):
self.user.set_demographic_profile(set(["science"]))
science = set(["science", "biology", "field::astronomy",
"field::aviation", "field::biology",
"field::chemistry", "field::eletronics",
"field::finance", "field::geography",
"field::geology", "field::linguistics",
"field::mathematics", "field::medicine",
"field::meteorology", "field::physics",
"field::statistics"])
self.assertEqual(self.user.demographic_profile, science)
def test_multi_profile(self):
self.user.set_demographic_profile(set(["devel", "art"]))
devel_art = set(["devel", "role::devel-lib", "role::shared-lib",
"field::arts", "sound"])
self.assertEqual(self.user.demographic_profile, devel_art)
self.user.set_demographic_profile(set(["art", "admin", "desktop"]))
desktop_art_admin = set(["x11", "accessibility", "game", "junior",
"office", "interface::x11", "field::arts",
"sound", "admin", "hardware", "mail",
"protocol", "network", "security", "web",
"interface::web"])
self.assertEqual(self.user.demographic_profile, desktop_art_admin)
def test_items(self):
self.assertEqual(set(self.user.items()),
set(["gimp", "aaphoto", "eog", "emacs"]))
def test_maximal_pkg_profile(self):
old_pkg_profile = self.user.items()
aaphoto_deps = ["libc6", "libgomp1", "libjasper1", "libjpeg62",
"libpng12-0"]
libc6_deps = ["libc-bin", "libgcc1"]
for pkg in aaphoto_deps + libc6_deps:
self.user.item_score[pkg] = 1
self.assertEqual(old_pkg_profile, self.user.maximal_pkg_profile())
class LocalSystemTest(unittest.TestCase):
def setUp(self):
self.user = LocalSystem()
@patch('commands.getoutput')
@patch('glob.glob', return_value=['test'])
def test_get_apt_installed_pkgs(self, mock_glob, mock_command):
mock_command.return_value = 'Commandline: apt install test1 test2\n'
apt_pkgs = self.user.get_apt_installed_pkgs()
self.assertEqual(2, len(apt_pkgs))
self.assertIn('test1', apt_pkgs)
self.assertIn('test2', apt_pkgs)
mock_command.return_value = 'Commandline: apt remove test1 test2\n'
apt_pkgs = self.user.get_apt_installed_pkgs()
self.assertEqual(0, len(apt_pkgs))
mock_command.return_value = 'Commandline: apt upgrade\n'
apt_pkgs = self.user.get_apt_installed_pkgs()
self.assertEqual(0, len(apt_pkgs))
AppRecommender-0.7.5/apprecommender/user.py 0000664 0000000 0000000 00000041236 13067513116 0020766 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
user - python module for classes and methods related
to recommenders' users.
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import commands
import datetime
import glob
import logging
import os
import pickle
import random
import re
import xapian
import apprecommender.data as data
from apprecommender.apt_cache import AptCache
from apprecommender.config import Config
from apprecommender.decider import (FilterTag, FilterDescription,
FilterTag_or_Description)
from apprecommender.error import Error
from apprecommender.singleton import Singleton
class DemographicProfile(Singleton):
def __init__(self):
self.admin = set(["admin", "hardware", "mail", "protocol",
"network", "security", "web", "interface::web"])
self.devel = set(["devel", "role::devel-lib", "role::shared-lib"])
self.desktop = set(["x11", "accessibility", "game", "junior", "office",
"interface::x11"])
self.art = set(["field::arts", "sound"])
self.science = set(["science", "biology", "field::astronomy",
"field::aviation", "field::biology",
"field::chemistry", "field::eletronics",
"field::finance", "field::geography",
"field::geology", "field::linguistics",
"field::mathematics", "field::medicine",
"field::meteorology", "field::physics",
"field::statistics"])
def __call__(self, profiles_set):
demographic_profile = set()
for profile in profiles_set:
profile_str = "self." + profile
demographic_profile = (demographic_profile | eval(profile_str, {},
{"self": self}))
return demographic_profile
class User:
"""
Define a user of a recommender.
"""
def __init__(self, item_score, user_id=0, arch=0, demo_profiles_set=0,
reference_pkgs=None):
"""
Set initial user attributes. pkg_profile gets the whole set of items,
a random user_id is set if none was provided and the demographic
profile defaults to 'desktop'.
"""
self.item_score = item_score
self.pkg_profile = self.items()
self.installed_pkgs = data.get_user_installed_pkgs()
self.arch = arch
self.reference_pkgs = reference_pkgs if reference_pkgs else []
if user_id:
self.user_id = user_id
else:
random.seed()
self.id = random.getrandbits(128)
if not demo_profiles_set:
profiles_set = set(["desktop"])
self.set_demographic_profile(profiles_set)
def items(self):
"""
Return the set of user items.
"""
return self.item_score.keys()
def set_demographic_profile(self, profiles_set):
"""
Set demographic profle based on labels in 'profiles_set'.
"""
self.demographic_profile = DemographicProfile()(profiles_set)
def content_profile(self, items_repository, content, size, valid_tags=0,
time_context=0):
"""
Get user profile for a specific type of content: packages tags,
description or both (mixed and half-half profiles)
"""
if content == "tag":
profile = self.tfidf_profile(items_repository, size,
FilterTag(valid_tags), time_context)
elif content == "desc":
profile = self.tfidf_profile(items_repository,
size, FilterDescription(),
time_context)
elif content == 'mlbow_mix' or content == 'mlbva_mix':
self.pkg_profile = self.get_most_usefull_pkgs()
profile = self.tfidf_profile(items_repository, size,
FilterTag_or_Description(valid_tags),
time_context)
elif content == "mix":
profile = self.tfidf_profile(items_repository, size,
FilterTag_or_Description(valid_tags),
time_context)
elif content == "half":
tag_profile = self.tfidf_profile(items_repository, size,
FilterTag(valid_tags),
time_context)
desc_profile = self.tfidf_profile(items_repository, size,
FilterDescription(),
time_context)
profile = tag_profile[:size / 2] + desc_profile[:size / 2]
elif content == "time":
tag_profile = self.tfidf_profile(items_repository, size,
FilterTag(valid_tags),
time_context=1)
desc_profile = self.tfidf_profile(items_repository, size,
FilterDescription(),
time_context=1)
profile = tag_profile[:size / 2] + desc_profile[:size / 2]
elif content == "tag_eset":
profile = self.eset_profile(items_repository, size,
FilterTag(valid_tags))
elif content == "desc_eset":
profile = self.eset_profile(items_repository, size,
FilterDescription())
elif content == 'mlbow_mix_eset' or content == 'mlbva_mix_eset':
self.pkg_profile = self.get_most_usefull_pkgs()
profile = self.eset_profile(items_repository, size,
FilterTag_or_Description(valid_tags))
elif content == "mix_eset":
profile = self.eset_profile(items_repository, size,
FilterTag_or_Description(valid_tags))
elif content == "half_eset":
tag_profile = self.eset_profile(items_repository, size,
FilterTag(valid_tags))
desc_profile = self.eset_profile(items_repository, size,
FilterDescription())
profile = tag_profile[:size / 2] + desc_profile[:size / 2]
else:
logging.debug("Unknown content type %s." % content)
raise Error
logging.debug("User %s profile: %s" % (content, profile))
return profile
def tfidf_profile(self, items_repository, size, content_filter,
time_context=0):
"""
Return the most relevant tags for the user list of packages based on
the sublinear tfidf weight of packages' tags.
"""
docs = data.axi_search_pkgs(items_repository, self.pkg_profile)
# weights = data.tfidf_plus(items_repository,docs,content_filter)
weights = data.tfidf_weighting(items_repository, docs, content_filter,
time_context=time_context)
# Eliminate duplicated stemmed term
profile = self._eliminate_duplicated([w[0] for w in weights], size)
return profile
def eset_profile(self, items_repository, size, content_filter):
"""
Return most relevant tags for a list of packages.
"""
# Store package documents in a relevant set
enquire = xapian.Enquire(items_repository)
docs = data.axi_search_pkgs(items_repository, self.pkg_profile)
rset_packages = xapian.RSet()
for d in docs:
rset_packages.add_document(d.docid)
# Get expanded query terms (statistically good differentiators)
eset_tags = enquire.get_eset(size * 2, rset_packages,
xapian.Enquire.INCLUDE_QUERY_TERMS,
1, content_filter)
# Eliminate duplicated stemmed term
profile = self._eliminate_duplicated([res.term for res in eset_tags],
size)
return profile
def _eliminate_duplicated(self, sorted_list, size):
profile = sorted_list[:size]
next_index = size
duplicate = 1
while duplicate:
duplicate = 0
for term in profile[:]:
if term.startswith("Z"):
for p in profile[:]:
if p.startswith(term.lstrip("Z")):
duplicate = 1
profile.remove(p)
if len(sorted_list) > next_index:
profile.append(sorted_list[next_index])
next_index += 1
return profile
def filter_pkg_profile(self, filter_list_or_file):
"""
Return list of packages from profile listed in the filter_file.
"""
if type(filter_list_or_file).__name__ == "list":
valid_pkgs = filter_list_or_file
elif type(filter_list_or_file).__name__ == "str":
try:
with open(filter_list_or_file) as valid:
valid_pkgs = [line.strip() for line in valid]
except IOError:
logging.critical("Could not open profile filter file: %s" %
filter_list_or_file)
raise Error
else:
logging.debug("No filter provided for user profiling.")
return self.pkg_profile
old_profile_size = len(self.pkg_profile)
for pkg in self.pkg_profile[:]: # iterate list copy
if pkg not in valid_pkgs:
self.pkg_profile.remove(pkg)
logging.debug("Discarded package %s during profile filtering"
% pkg)
profile_size = len(self.pkg_profile)
logging.debug("Filtered package profile: reduced packages profile size \
from %d to %d." % (old_profile_size, profile_size))
return self.pkg_profile
def maximal_pkg_profile(self):
"""
Return list of packages that are not dependence of any other package in
the list.
"""
cache = AptCache()
old_profile_size = len(self.pkg_profile)
for p in self.pkg_profile[:]: # iterate list copy
if p in cache:
pkg = cache[p]
if pkg.candidate:
for dep in pkg.candidate.dependencies:
for or_dep in dep.or_dependencies:
if or_dep.name in self.pkg_profile:
self.pkg_profile.remove(or_dep.name)
profile_size = len(self.pkg_profile)
logging.debug("Maximal package profile: reduced packages profile size \
from %d to %d." % (old_profile_size, profile_size))
return self.pkg_profile
def get_most_usefull_pkgs(self):
classification_path = Config().user_data_dir
classification_path += 'pkgs_classifications.txt'
if not os.path.exists(classification_path):
return -1
with open(classification_path, 'ra') as data:
pkg_classification = pickle.load(data)
classifications = {'RU': [], 'U': [], 'NU': []}
for pkg, values in pkg_classification.iteritems():
classifications[values[-1]].append(pkg)
return classifications['RU']
class RandomPopcon(User):
def __init__(self, submissions_dir, arch=0, pkgs_filter=0):
"""
Set initial parameters.
"""
len_profile = 0
match_arch = False
while len_profile < 100 or not match_arch:
path = random.choice([os.path.join(root, submission) for
root, dirs, files in os.walk(submissions_dir)
for submission in files])
user = PopconSystem(path)
print arch
print user.arch
if arch and user.arch == arch:
match_arch = True
print "match"
if pkgs_filter:
user.filter_pkg_profile(pkgs_filter)
len_profile = len(user.pkg_profile)
print "p", len_profile
submission = data.PopconSubmission(path)
User.__init__(self, submission.packages, submission.user_id,
submission.arch)
class PopconSystem(User):
def __init__(self, path, user_id=0):
"""
Set initial parameters.
"""
submission = data.PopconSubmission(path)
if not user_id:
user_id = submission.user_id
User.__init__(self, submission.packages, user_id, submission.arch)
class LocalSystem(User):
"""
Extend the class User to consider the packages installed on the local
system as the set of selected itens.
"""
def __init__(self, reference_pkgs=None):
"""
Set initial parameters.
"""
user_pkgs = self.get_manual_installed_pkgs()
item_score = {pkg: 1 for pkg in user_pkgs}
self.user_id = "local-" + str(datetime.datetime.now())
User.__init__(self, item_score, reference_pkgs=reference_pkgs)
def get_system_pkgs(self):
system_pkgs = []
all_pkgs = commands.getoutput("dpkg-query -Wf \
'${Package;-40}${Priority}\n'")
priority_terms = set(['important', 'required', 'standard'])
for line in all_pkgs.splitlines():
line_split = line.split(' ')
pkg_name = line_split[0]
pkg_priority = line_split[-1].strip()
if (pkg_priority in priority_terms):
system_pkgs.append(pkg_name)
return system_pkgs
def get_apt_installed_pkgs(self):
apt_pkgs = set()
apt_log = glob.glob('/var/log/apt/history.log*')
installed_pkgs_regex = re.compile(
r'^Commandline:.+apt.+install\s(.+)', re.MULTILINE)
no_remove_pkgs_regex = re.compile(r'--no-remove')
automatic_remove_regex = re.compile(r'APT::Get::AutomaticRemove=true')
apt_log = reversed(sorted(apt_log))
for log in apt_log:
command = 'zcat' if log.endswith('gz') else 'cat'
history_files = commands.getoutput(
'{} {} | grep "Commandline:"'.format(command, log))
'''
The log_files will hold packages with this format:
Commandline: apt install google-chrome-stable
Therefore it is necessary to perform another filter on it
to only get the package name.
'''
for apt_command in history_files.splitlines():
installed_pkgs = installed_pkgs_regex.search(apt_command)
no_remove_pkgs = no_remove_pkgs_regex.search(apt_command)
automatic_remove_pkgs = automatic_remove_regex.search(
apt_command)
if (installed_pkgs and not no_remove_pkgs and not
automatic_remove_pkgs):
pkgs = set(installed_pkgs.group(1).split())
apt_pkgs |= pkgs
return apt_pkgs
def __get_manual_marked_pkgs(self):
list_manual = commands.getoutput('apt-mark showmanual')
list_manual = list_manual.splitlines()
return set([pkg for pkg in list_manual])
def __remove_lib_packages(self, pkgs):
return set([pkg for pkg in pkgs if not re.match(r'^lib', pkg)])
def __remove_apt_packages(self, pkgs):
return set([pkg for pkg in pkgs if not re.match(r'^apt', pkg)])
def get_manual_installed_pkgs(self):
apt_pkgs = self.get_apt_installed_pkgs()
manual_pkgs = self.__get_manual_marked_pkgs()
system_pkgs = self.get_system_pkgs()
pkgs = apt_pkgs.intersection(manual_pkgs)
pkgs = self.__remove_lib_packages(pkgs)
pkgs = self.__remove_apt_packages(pkgs)
pkgs -= set(system_pkgs)
return pkgs
AppRecommender-0.7.5/apprecommender/utils.py 0000664 0000000 0000000 00000001623 13067513116 0021144 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
import sys
def print_progress_bar(number, n_numbers, message='Progress',
bar_length=40):
percent = float(number) / float(n_numbers)
hashes = '#' * int(round(percent * bar_length))
spaces = ' ' * (bar_length - len(hashes))
percent = int(round(percent * 100))
percent_message = ("\r{}: [{}] [{} / {}] {}%".format(message,
hashes + spaces,
number,
n_numbers,
percent))
sys.stdout.write(percent_message)
sys.stdout.flush()
if number == n_numbers:
print '\n'
def get_class_and_module_name(cls):
class_name = cls.__name__
module_name = cls.__module__
return module_name + '.' + class_name
AppRecommender-0.7.5/bin/ 0000775 0000000 0000000 00000000000 13067513116 0015177 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/bin/apprec.py 0000775 0000000 0000000 00000005451 13067513116 0017033 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
AppRecommender - A GNU/Linux application recommender
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import sys
import xapian
sys.path.insert(0, '../')
from apprecommender.app_recommender import AppRecommender
from apprecommender.initialize import Initialize
from apprecommender.load_options import LoadOptions
from apprecommender.config import Config
from apprecommender.strategy import (MachineLearning, MachineLearningBVA,
MachineLearningBOW)
from apprecommender.data_collect import collect_user_data
SUCCESS = 0
ERROR_INIT = 1
ERROR_TRAIN = 2
def check_for_flag(options, short_flag, long_flag):
for option, _ in options:
if option in (short_flag, long_flag):
return True
return False
def run_apprecommender(options):
try:
app_recommender = AppRecommender()
app_recommender.make_recommendation()
return SUCCESS
except xapian.DatabaseOpeningError:
return ERROR_INIT
except IOError:
if "ml" in Config().strategy:
return ERROR_TRAIN
def run():
load_options = LoadOptions()
load_options.load()
options = load_options.options
if check_for_flag(options, '-i', '--init'):
print "Initializing AppRecommender"
initialize = Initialize()
initialize.prepare_data()
return SUCCESS
elif check_for_flag(options, '-t', '--train'):
print "Training machine learning"
MachineLearning.train(MachineLearningBVA)
MachineLearning.train(MachineLearningBOW)
return SUCCESS
elif check_for_flag(options, '-c', '--contribute'):
collect_user_data.main()
else:
return run_apprecommender(load_options.options)
def main():
result = run()
if result is ERROR_INIT:
print "\n"
print "Please, Initialize AppRecommender"
print "Run: apprec.py --init"
elif result is ERROR_TRAIN:
print "\n"
print "Please, run Machine Learning Training"
print "Run: apprec.py --train"
if __name__ == '__main__':
main()
AppRecommender-0.7.5/bin/apt_config/ 0000775 0000000 0000000 00000000000 13067513116 0017310 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/bin/apt_config/__init__.py 0000664 0000000 0000000 00000000000 13067513116 0021407 0 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/bin/apt_config/add_apt_conf.py 0000775 0000000 0000000 00000002034 13067513116 0022265 0 ustar 00root root 0000000 0000000 #!/usr/bin/python
import os
import sys
sys.path.insert(0, "{0}/../../".format(os.path.dirname(__file__)))
APT_FOLDER = "/home/%s/.app-recommender/apt_conf" % os.getenv("SUDO_USER")
APT_CONF_FILE = '/etc/apt/apt.conf.d/99app-recommender'
FILES_FOLDER = os.path.dirname(os.path.abspath(__file__))
POST_INVOKE_PATH = FILES_FOLDER + "/post_invoke.py"
PRE_INSTALL_PKGS_PATH = FILES_FOLDER + "/pre_install_pkgs.py"
INSTALLED_PKGS_FILE = APT_FOLDER + "/installed_pkgs.txt"
def main():
post_invoke = 'DPkg::Post-Invoke {"python %s";};' % POST_INVOKE_PATH
pre_install_pkgs = ('DPkg::Pre-Install-Pkgs {"python %s";};' %
PRE_INSTALL_PKGS_PATH)
print("Creating folder: {}".format(APT_FOLDER))
if not os.path.exists(APT_FOLDER):
os.makedirs(APT_FOLDER)
print("Creating file: {}".format(APT_CONF_FILE))
with open(APT_CONF_FILE, 'w') as text:
text.write(post_invoke)
text.write('\n')
text.write(pre_install_pkgs)
text.write('\n')
if __name__ == "__main__":
main()
AppRecommender-0.7.5/bin/apt_config/clear_apt_conf.py 0000775 0000000 0000000 00000000317 13067513116 0022625 0 ustar 00root root 0000000 0000000 #!/usr/bin/python
import os
import shutil
from add_apt_conf import APT_FOLDER, APT_CONF_FILE
def main():
os.remove(APT_CONF_FILE)
shutil.rmtree(APT_FOLDER)
if __name__ == "__main__":
main()
AppRecommender-0.7.5/bin/apt_config/post_invoke.py 0000664 0000000 0000000 00000001113 13067513116 0022216 0 ustar 00root root 0000000 0000000 #!/usr/bin/python
import commands
from add_apt_conf import INSTALLED_PKGS_FILE
def main():
with open(INSTALLED_PKGS_FILE, 'r') as text:
pre_installed_pkgs = set([line.strip() for line in text])
pkgs = commands.getoutput("apt-mark showmanual")
pos_installed_pkgs = set([line.strip() for line in pkgs.splitlines()])
installed_pkgs = list(pos_installed_pkgs - pre_installed_pkgs)
if len(installed_pkgs) > 0:
print("\n\n")
print("installed packages: {}".format(installed_pkgs))
print("\n\n")
if __name__ == "__main__":
main()
AppRecommender-0.7.5/bin/apt_config/pre_install_pkgs.py 0000664 0000000 0000000 00000000326 13067513116 0023223 0 ustar 00root root 0000000 0000000 #!/usr/bin/python
import commands
from add_apt_conf import INSTALLED_PKGS_FILE
def main():
commands.getoutput("apt-mark showmanual > {}".format(INSTALLED_PKGS_FILE))
if __name__ == "__main__":
main()
AppRecommender-0.7.5/bin/cross_validation.py 0000775 0000000 0000000 00000005022 13067513116 0021116 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
CrossValidation - python module for classes and methods related to
recommenders evaluation.
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import os
import logging
import datetime
import sys
sys.path.insert(0, '../')
from apprecommender.config import Config
from apprecommender.evaluation import (CrossValidation, Precision, Recall,
F_score, FPR, Accuracy)
from apprecommender.recommender import Recommender
from apprecommender.user import PopconSystem
if __name__ == '__main__':
cfg = Config()
rec = Recommender()
# user = LocalSystem()
# user = RandomPopcon(cfg.popcon_dir)
# user = RandomPopcon(cfg.popcon_dir,os.path.join(cfg.filters_dir,
# "desktopapps"))
popcon_entries = "~/.app-recommender/popcon-entries/" \
"00/0001166d0737c6dffb083071e5ee69f5"
user = PopconSystem(os.path.expanduser(popcon_entries))
user.filter_pkg_profile(os.path.join(cfg.filters_dir, "desktopapps"))
user.maximal_pkg_profile()
begin_time = datetime.datetime.now()
metrics = []
metrics.append(Precision())
metrics.append(Recall())
metrics.append(F_score(0.5))
metrics.append(Accuracy())
metrics.append(FPR())
validation = CrossValidation(0.9, 20, rec, metrics, 0.005)
validation.run(user)
print validation
end_time = datetime.datetime.now()
delta = end_time - begin_time
logging.info("Cross-validation for user %s" % user.user_id)
logging.info("Recommender strategy: %s" % rec.strategy.description)
logging.debug("Cross-validation started at %s" % begin_time)
logging.debug("Cross-validation completed at %s" % end_time)
logging.info("Time elapsed: %d seconds." % delta.seconds)
AppRecommender-0.7.5/bin/demo_rec.py 0000775 0000000 0000000 00000010665 13067513116 0017341 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
"""
DemoRecommender - demonstration of a GNU/Linux application recommender.
"""
__author__ = "Tassia Camoes Araujo "
__copyright__ = "Copyright (C) 2011 Tassia Camoes Araujo"
__license__ = """
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
"""
import os
import commands
import re
import sys
sys.path.insert(0, '../')
import xapian
from debian import debtags # ???
from apprecommender.strategy import PkgMatchDecider
DB_PATH = "/var/lib/debtags/package-tags"
INDEX_PATH = os.path.expanduser("~/.app-recommender/debtags_index")
def load_debtags_db(path):
""" Load debtags database. """
debtags_db = debtags.DB()
tag_filter = re.compile(r"^special::.+$|^.+::TODO$")
try:
debtags_db.read(open(path, "r"), lambda x: not tag_filter.match(x))
except IOError:
print >> sys.stderr, ("IOError: could not open debtags file \'%s\'" %
path)
exit(1)
return debtags_db
def get_system_pkgs():
""" Return set of system packages. """
dpkg_output = commands.getoutput('/usr/bin/dpkg --get-selections')
return dpkg_output.replace('install', '\t').split()
def get_most_relevant_tags(debtags_db, pkgs_list):
""" Return most relevant tags considering a list of packages. """
relevant_db = debtags_db.choose_packages(pkgs_list)
relevance_index = debtags.relevance_index_function(debtags_db, relevant_db)
sorted_relevant_tags = sorted(relevant_db.iter_tags(),
lambda a, b: cmp(relevance_index(a),
relevance_index(b)))
return normalize_tags(' '.join(sorted_relevant_tags[-50:]))
def normalize_tags(string):
""" Normalize tag string so that it can be indexed and retrieved. """
return string.replace(':', '_').replace('-', '\'')
def create_debtags_index(debtags_db, index_path):
""" Create a xapian index for debtags info based on file 'debtags_db' and
place it at 'index_path'.
"""
if not os.path.exists(index_path):
os.makedirs(index_path)
print "Creating new debtags xapian index at \'%s\'" % index_path
debtags_index = xapian.WritableDatabase(index_path,
xapian.DB_CREATE_OR_OVERWRITE)
for pkg, tags in debtags_db.iter_packages_tags():
doc = xapian.Document()
doc.set_data(pkg)
for tag in tags:
doc.add_term(normalize_tags(tag))
print "indexing ", debtags_index.add_document(doc)
return debtags_index
def load_debtags_index(debtags_db, reindex):
""" Load an existing or new debtags index, based on boolean reindex. """
if not reindex:
try:
print ("Opening existing debtags xapian index at \'%s\'" %
INDEX_PATH)
debtags_index = xapian.Database(INDEX_PATH)
except xapian.DatabaseError:
print "Could not open debtags xapian index"
reindex = 1
if reindex:
debtags_index = create_debtags_index(debtags_db, INDEX_PATH)
return debtags_index
if __name__ == '__main__':
reindex = 0
if len(sys.argv) == 2:
DB_PATH = sys.argv[1]
reindex = 1
print "reindex true"
elif len(sys.argv) > 2:
print >> sys.stderr, ("Usage: %s [PATH_TO_DEBTAGS_DATABASE]" %
sys.argv[0])
sys.exit(1)
debtags_db = load_debtags_db(DB_PATH)
installed_pkgs = get_system_pkgs()
best_tags = get_most_relevant_tags(debtags_db, installed_pkgs)
debtags_index = load_debtags_index(debtags_db, reindex)
qp = xapian.QueryParser()
query = qp.parse_query(best_tags)
enquire = xapian.Enquire(debtags_index)
enquire.set_query(query)
mset = enquire.get_mset(0, 20, None, PkgMatchDecider(installed_pkgs))
for m in mset:
print "%2d: %s" % (m.rank, m.document.get_data())
AppRecommender-0.7.5/bin/example/ 0000775 0000000 0000000 00000000000 13067513116 0016632 5 ustar 00root root 0000000 0000000 AppRecommender-0.7.5/bin/example/dpkg_debian6 0000664 0000000 0000000 00000036756 13067513116 0021113 0 ustar 00root root 0000000 0000000 acpi 1.5-2
acpi-support-base 0.137-5
acpid 1:2.0.7-1
adduser 3.112+nmu2
alsa-base 1.0.23+dfsg-2
alsa-utils 1.0.23-3
anacron 2.3-14
apt 0.8.10.3
apt-listchanges 2.85.7
apt-utils 0.8.10.3
aptitude 0.6.3-3.2
at 3.1.12-1
base-files 6.0
base-passwd 3.5.22
bash 4.1-3
bash-completion 1:1.2-3
bc 1.06.95-2
bind9-host 1:9.7.2.dfsg.P3-1.1
bsd-mailx 8.1.2-0.20100314cvs-1
bsdmainutils 8.0.13
bsdutils 1:2.17.2-9
busybox 1:1.17.1-8
bzip2 1.0.5-6
capplets-data 1:2.30.1-2
console-setup 1.68
console-terminus 4.30-2
consolekit 0.4.1-4
coreutils 8.5-1
cpio 2.11-4
cpp 4:4.4.5-1
cpp-4.4 4.4.5-8
cron 3.0pl1-116
cups 1.4.4-7
cups-bsd 1.4.4-7
cups-client 1.4.4-7
cups-common 1.4.4-7
cups-ppdc 1.4.4-7
dash 0.5.5.1-7.4
dbus 1.2.24-4
dbus-x11 1.2.24-4
dc 1.06.95-2
debconf 1.5.36.1
debconf-i18n 1.5.36.1
debian-archive-keyring 2010.08.28
debian-faq 4.0.4+nmu1
debianutils 3.4
defoma 0.11.11
desktop-base 6.0.5
desktop-file-utils 0.15-2
diffutils 1:3.0-1
discover 2.1.2-5
discover-data 2.2010.10.18
dmidecode 2.9-1.2
dmsetup 2:1.02.48-5
dmz-cursor-theme 0.4.3
dnsutils 1:9.7.2.dfsg.P3-1.1
doc-debian 4.0.2
doc-linux-text 2008.08-1
docbook-xml 4.5-7
dosfstools 3.0.9-1
dpkg 1.15.8.10
e2fslibs 1.41.12-2
e2fsprogs 1.41.12-2
eject 2.1.5+deb1+cvs20081104-7.1
eog 2.30.2-1
epiphany-browser 2.30.6-1
epiphany-browser-data 2.30.6-1
esound-common 0.2.41-8
evince 2.30.3-2
evince-common 2.30.3-2
evolution 2.30.3-5
evolution-common 2.30.3-5
evolution-data-server 2.30.3-2
evolution-data-server-common 2.30.3-2
exim4 4.72-6
exim4-base 4.72-6
exim4-config 4.72-6
exim4-daemon-light 4.72-6
file 5.04-5
file-roller 2.30.2-2
findutils 4.4.2-1+b1
fontconfig 2.8.0-2.1
fontconfig-config 2.8.0-2.1
ftp 0.17-23
gcc-4.4-base 4.4.5-8
gconf2 2.28.1-6
gconf2-common 2.28.1-6
gdm3 2.30.5-6
gedit 2.30.4-1
gedit-common 2.30.4-1
gettext-base 0.18.1.1-3
ghostscript 8.71~dfsg2-9
gir1.0-clutter-1.0 1.2.12-3
gir1.0-freedesktop 0.6.14-1+b1
gir1.0-glib-2.0 0.6.14-1+b1
gir1.0-gstreamer-0.10 0.10.30-1
gir1.0-gtk-2.0 0.6.5-7
gir1.0-json-glib-1.0 0.10.2-2
gnome-about 2.30.2-2
gnome-applets 2.30.0-3
gnome-applets-data 2.30.0-3
gnome-control-center 1:2.30.1-2
gnome-core 1:2.30+7
gnome-desktop-data 2.30.2-2
gnome-disk-utility 2.30.1-2
gnome-doc-utils 0.20.1-1
gnome-icon-theme 2.30.3-2
gnome-js-common 0.1.1-1
gnome-keyring 2.30.3-5
gnome-menus 2.30.3-1
gnome-mime-data 2.18.0-1
gnome-panel 2.30.2-2
gnome-panel-data 2.30.2-2
gnome-power-manager 2.32.0-2
gnome-screensaver 2.30.0-2
gnome-session 2.30.2-3
gnome-session-bin 2.30.2-3
gnome-session-common 2.30.2-3
gnome-settings-daemon 2.30.2-2
gnome-terminal 2.30.2-1
gnome-terminal-data 2.30.2-1
gnome-themes 2.30.2-1
gnome-user-guide 2.30.1-1
gnupg 1.4.10-4
gpgv 1.4.10-4
grep 2.6.3-3
groff-base 1.20.1-10
grub-common 1.98+20100804-14
grub-pc 1.98+20100804-14
gsfonts 1:8.11+urwcyr1.0.7~pre44-4.2
gstreamer0.10-alsa 0.10.30-1
gstreamer0.10-plugins-base 0.10.30-1
gstreamer0.10-plugins-good 0.10.24-1
gstreamer0.10-x 0.10.30-1
gtk2-engines 1:2.20.1-1
gtk2-engines-pixbuf 2.20.1-2
gvfs 1.6.4-3
gvfs-backends 1.6.4-3
gzip 1.3.12-9
hdparm 9.32-1
hicolor-icon-theme 0.12-1
host 1:9.7.2.dfsg.P3-1.1
hostname 3.04
iceweasel 3.5.16-4
ifupdown 0.6.10
info 4.13a.dfsg.1-6
initramfs-tools 0.98.8
initscripts 2.88dsf-13.1
insserv 1.14.0-2
install-info 4.13a.dfsg.1-6
installation-report 2.44
iproute 20100519-3
iptables 1.4.8-3
iputils-ping 3:20100418-3
isc-dhcp-client 4.1.1-P1-15
isc-dhcp-common 4.1.1-P1-15
iso-codes 3.23-1
kbd 1.15.2-2
keyboard-configuration 1.68
klibc-utils 1.5.20-1
laptop-detect 0.13.7
less 436-1
libaa1 1.4p5-38
libacl1 2.2.49-4
libarchive1 2.8.4-1
libart-2.0-2 2.3.21-1
libasound2 1.0.23-2.1
libaspell15 0.60.6-4
libatasmart4 0.17+git20100219-2
libatk1.0-0 1.30.0-1
libattr1 1:2.4.44-2
libaudiofile0 0.2.6-8
libaudit0 1.7.13-1+b2
libavahi-client3 0.6.27-2
libavahi-common-data 0.6.27-2
libavahi-common3 0.6.27-2
libavahi-glib1 0.6.27-2
libavahi-gobject0 0.6.27-2
libavahi-ui0 0.6.27-2
libavc1394-0 0.5.3-1+b2
libbind9-60 1:9.7.2.dfsg.P3-1.1
libblas3gf 1.2-8
libblkid1 2.17.2-9
libbluetooth3 4.66-3
libbonobo2-0 2.24.3-1
libbonobo2-common 2.24.3-1
libbonoboui2-0 2.24.3-1
libbonoboui2-common 2.24.3-1
libboost-iostreams1.42.0 1.42.0-4
libbsd0 0.2.0-1
libbz2-1.0 1.0.5-6
libc-bin 2.11.2-10
libc6 2.11.2-10
libc6-i686 2.11.2-10
libcaca0 0.99.beta17-1
libcairo2 1.8.10-6
libcamel1.2-14 2.30.3-2
libcanberra-gtk0 0.24-1
libcanberra0 0.24-1
libcap2 1:2.19-3
libcdio-cdda0 0.81-4
libcdio-paranoia0 0.81-4
libcdio10 0.81-4
libcdparanoia0 3.10.2+debian-9
libck-connector0 0.4.1-4
libclutter-1.0-0 1.2.12-3
libcomerr2 1.41.12-2
libcpufreq0 007-1
libcroco3 0.6.2-1
libcups2 1.4.4-7
libcupscgi1 1.4.4-7
libcupsdriver1 1.4.4-7
libcupsimage2 1.4.4-7
libcupsmime1 1.4.4-7
libcupsppdc1 1.4.4-7
libcwidget3 0.5.16-3
libdatrie1 0.2.4-1
libdb4.6 4.6.21-16
libdb4.7 4.7.25-9
libdb4.8 4.8.30-2
libdbus-1-3 1.2.24-4
libdbus-glib-1-2 0.88-2.1
libdevmapper1.02.1 2:1.02.48-5
libdiscover2 2.1.2-5
libdjvulibre-text 3.5.23-3
libdjvulibre21 3.5.23-3
libdns69 1:9.7.2.dfsg.P3-1.1
libdrm-intel1 2.4.21-1~squeeze3
libdrm-nouveau1 2.4.21-1~squeeze3
libdrm-radeon1 2.4.21-1~squeeze3
libdrm2 2.4.21-1~squeeze3
libdv4 1.0.0-2.1
libebackend1.2-0 2.30.3-2
libebook1.2-9 2.30.3-2
libecal1.2-7 2.30.3-2
libedata-book1.2-2 2.30.3-2
libedata-cal1.2-7 2.30.3-2
libedataserver1.2-13 2.30.3-2
libedataserverui1.2-8 2.30.3-2
libedit2 2.11-20080614-2
libeggdbus-1-0 0.6-1
libegroupwise1.2-13 2.30.3-2
libenchant1c2a 1.6.0-1
libept1 1.0.4
libesd0 0.2.41-8
libevent-1.4-2 1.4.13-stable-1
libevince2 2.30.3-2
libevolution 2.30.3-5
libexempi3 2.1.1-1
libexif12 0.6.19-1
libexpat1 2.0.1-7
libfam0 2.7.0-17
libffi5 3.0.9-3
libfile-copy-recursive-perl 0.38-1
libflac8 1.2.1-2+b1
libfontconfig1 2.8.0-2.1
libfontenc1 1:1.0.5-2
libfreetype6 2.4.2-2.1
libfs6 2:1.0.2-1
libgail18 2.20.1-2
libgc1c2 1:6.8-1.2
libgcc1 1:4.4.5-8
libgconf2-4 2.28.1-6
libgcr0 2.30.3-5
libgcrypt11 1.4.5-2
libgdata-google1.2-1 2.30.3-2
libgdata1.2-1 2.30.3-2
libgdbm3 1.8.3-9
libgdu-gtk0 2.30.1-2
libgdu0 2.30.1-2
libgeoip1 1.4.7~beta6+dfsg-1
libgfortran3 4.4.5-8
libgirepository1.0-0 0.6.14-1+b1
libgl1-mesa-dri 7.7.1-4
libgl1-mesa-glx 7.7.1-4
libglade2-0 1:2.6.4-1
libglib2.0-0 2.24.2-1
libglib2.0-data 2.24.2-1
libglu1-mesa 7.7.1-4
libgmime-2.4-2 2.4.14-1+nmu1
libgmp3c2 2:4.3.2+dfsg-1
libgnome-desktop-2-17 2.30.2-2
libgnome-keyring0 2.30.1-1
libgnome-menu2 2.30.3-1
libgnome-window-settings1 1:2.30.1-2
libgnome2-0 2.30.0-1
libgnome2-common 2.30.0-1
libgnomecanvas2-0 2.30.1-1
libgnomecanvas2-common 2.30.1-1
libgnomekbd-common 2.30.2-2
libgnomekbd4 2.30.2-2
libgnomeui-0 2.24.3-1
libgnomeui-common 2.24.3-1
libgnomevfs2-0 1:2.24.3-1
libgnomevfs2-common 1:2.24.3-1
libgnutls26 2.8.6-1
libgp11-0 2.30.3-5
libgpg-error0 1.6-1
libgpgme11 1.2.0-1.2
libgphoto2-2 2.4.6-3
libgphoto2-port0 2.4.6-3
libgpm2 1.20.4-3.3
libgs8 8.71~dfsg2-9
libgsf-1-114 1.14.18-1
libgsf-1-common 1.14.18-1
libgssapi-krb5-2 1.8.3+dfsg-4
libgssglue1 0.1-4
libgssrpc4 1.8.3+dfsg-4
libgstreamer-plugins-base0.10-0 0.10.30-1
libgstreamer0.10-0 0.10.30-1
libgtk2.0-0 2.20.1-2
libgtk2.0-bin 2.20.1-2
libgtk2.0-common 2.20.1-2
libgtkhtml-editor-common 3.30.3-1
libgtkhtml-editor0 3.30.3-1
libgtkhtml3.14-19 3.30.3-1
libgtksourceview2.0-0 2.10.4-1
libgtksourceview2.0-common 2.10.4-1
libgtop2-7 2.28.1-1
libgtop2-common 2.28.1-1
libgucharmap7 1:2.30.3-1
libgudev-1.0-0 164-3
libgweather-common 2.30.3-1
libgweather1 2.30.3-1
libhal-storage1 0.5.14-3
libhal1 0.5.14-3
libhtml-parser-perl 3.66-1
libhtml-tagset-perl 3.20-2
libhtml-tree-perl 3.23-2
libhunspell-1.2-0 1.2.11-1
libical0 0.44-3
libice6 2:1.0.6-2
libicu44 4.4.1-7
libidl0 0.8.14-0.1
libidn11 1.15-2
libiec61883-0 1.2.0-0.1
libijs-0.35 0.35-7
libimobiledevice1 1.0.2-1
libisc62 1:9.7.2.dfsg.P3-1.1
libisccc60 1:9.7.2.dfsg.P3-1.1
libisccfg62 1:9.7.2.dfsg.P3-1.1
libjasper1 1.900.1-7+b1
libjbig2dec0 0.11-1
libjpeg62 6b1-1
libjson-glib-1.0-0 0.10.2-2
libk5crypto3 1.8.3+dfsg-4
libkadm5clnt-mit7 1.8.3+dfsg-4
libkadm5srv-mit7 1.8.3+dfsg-4
libkdb5-4 1.8.3+dfsg-4
libkeyutils1 1.4-1
libklibc 1.5.20-1
libkpathsea5 2009-8
libkrb5-3 1.8.3+dfsg-4
libkrb5support0 1.8.3+dfsg-4
liblapack3gf 3.2.1-8
liblcms1 1.18.dfsg-1.2+b3
libldap-2.4-2 2.4.23-7
liblocale-gettext-perl 1.05-6
liblockfile1 1.08-4
libltdl7 2.2.6b-2
liblua5.1-0 5.1.4-5
liblwres60 1:9.7.2.dfsg.P3-1.1
liblzma2 5.0.0-2
libmagic1 5.04-5
libmailtools-perl 2.06-1
libmetacity-private0 1:2.30.1-3
libmozjs2d 1.9.1.16-4
libmpfr4 3.0.0-2
libnautilus-extension1 2.30.1-2
libncurses5 5.7+20100313-5
libncursesw5 5.7+20100313-5
libnet-dbus-perl 0.33.6-2
libnewt0.52 0.52.11-1
libnfnetlink0 1.0.0-1
libnfsidmap2 0.23-2
libnotify1 0.5.0-2
libnspr4-0d 4.8.6-1
libnss3-1d 3.12.8-1
libogg0 1.2.0~dfsg-1
liboobs-1-4 2.30.1-1
libopenjpeg2 1.3+dfsg-4
liborbit2 1:2.14.18-0.1
liborc-0.4-0 1:0.4.6-2
libpam-modules 1.1.1-6.1
libpam-runtime 1.1.1-6.1
libpam0g 1.1.1-6.1
libpanel-applet2-0 2.30.2-2
libpango1.0-0 1.28.3-1+squeeze1
libpango1.0-common 1.28.3-1+squeeze1
libpaper1 1.1.24
libparted0debian1 2.3-5
libpci3 1:3.1.7-6
libpciaccess0 0.12.0-1
libpcre3 8.02-1.1
libpixman-1-0 0.16.4-1
libplist1 1.3-2
libpng12-0 1.2.44-1
libpolkit-agent-1-0 0.96-4
libpolkit-backend-1-0 0.96-4
libpolkit-gobject-1-0 0.96-4
libpolkit-gtk-1-0 0.96-3
libpoppler-glib4 0.12.4-1.2
libpoppler5 0.12.4-1.2
libpopt0 1.16-1
libproxy0 0.3.1-2
libpth20 2.0.7-16
libpython2.6 2.6.6-8+b1
librarian0 0.8.1-5
libraw1394-11 2.0.5-2
libreadline6 6.1-3
librpcsecgss3 0.19-2
librsvg2-2 2.26.3-1
librsvg2-common 2.26.3-1
libsasl2-2 2.1.23.dfsg1-7
libseed0 2.30.0-1+b1
libselinux1 2.0.96-1
libsepol1 2.0.41-1
libsgutils2-2 1.29-1
libshout3 2.2.2-5+b1
libsigc++-2.0-0c2a 2.2.4.2-1
libslab0a 2.30.0-1
libslang2 2.2.2-4
libslp1 1.2.1-7.8
libsm6 2:1.1.1-1
libsmbclient 2:3.5.6~dfsg-3
libsoup-gnome2.4-1 2.30.2-1
libsoup2.4-1 2.30.2-1
libspectre1 0.2.6-1
libspeex1 1.2~rc1-1
libsqlite3-0 3.7.3-1
libss2 1.41.12-2
libssl0.9.8 0.9.8o-4
libstartup-notification0 0.10-1
libstdc++6 4.4.5-8
libt1-5 5.1.2-3
libtag1-vanilla 1.6.3-1
libtag1c2a 1.6.3-1
libtalloc2 2.0.1-1
libtasn1-3 2.7-1
libtdb1 1.2.1-2+b1
libtext-charwidth-perl 0.04-6
libtext-iconv-perl 1.7-2
libtext-wrapi18n-perl 0.06-7
libthai-data 0.1.14-2
libthai0 0.1.14-2
libtheora0 1.1.1+dfsg.1-3
libtiff4 3.9.4-5
libtimedate-perl 1.2000-1
libtokyocabinet8 1.4.37-6
libtotem-plparser17 2.30.3-1
libudev0 164-3
libunique-1.0-0 1.1.6-1.1
libupower-glib1 0.9.5-5
liburi-perl 1.54-2
libusb-0.1-4 2:0.1.12-16
libusb-1.0-0 2:1.0.8-2
libusbmuxd1 1.0.4-1
libuuid-perl 0.02-4
libuuid1 2.17.2-9
libv4l-0 0.8.0-1
libvisual-0.4-0 0.4.0-3
libvorbis0a 1.3.1-1
libvorbisenc2 1.3.1-1
libvorbisfile3 1.3.1-1
libvte-common 1:0.24.3-2
libvte9 1:0.24.3-2
libwavpack1 4.60.1-1
libwbclient0 2:3.5.6~dfsg-3
libwebkit-1.0-2 1.2.6-2
libwebkit-1.0-common 1.2.6-2
libwnck-common 2.30.4-2
libwnck22 2.30.4-2
libwrap0 7.6.q-19
libwww-perl 5.836-1
libx11-6 2:1.3.3-4
libx11-data 2:1.3.3-4
libx11-xcb1 2:1.3.3-4
libxapian22 1.2.3-2
libxau6 1:1.0.6-1
libxaw7 2:1.0.7-1
libxcb-atom1 0.3.6-1
libxcb-aux0 0.3.6-1
libxcb-dri2-0 1.6-1
libxcb-event1 0.3.6-1
libxcb-render-util0 0.3.6-1
libxcb-render0 1.6-1
libxcb1 1.6-1
libxcomposite1 1:0.4.2-1
libxcursor1 1:1.1.10-2
libxdamage1 1:1.1.3-1
libxdmcp6 1:1.0.3-2
libxext6 2:1.1.2-1
libxfixes3 1:4.0.5-1
libxfont1 1:1.4.1-2
libxft2 2.1.14-2
libxi6 2:1.3-6
libxinerama1 2:1.1-3
libxkbfile1 1:1.0.6-2
libxklavier16 5.0-2
libxml-parser-perl 2.36-1.1+b1
libxml-twig-perl 1:3.34-1
libxml2 2.7.8.dfsg-2
libxml2-utils 2.7.8.dfsg-2
libxmu6 2:1.0.5-2
libxmuu1 2:1.0.5-2
libxpm4 1:3.5.8-1
libxrandr2 2:1.3.0-3
libxrender1 1:0.9.6-1
libxres1 2:1.0.4-1
libxslt1.1 1.1.26-6
libxss1 1:1.2.0-2
libxt6 1:1.0.7-1
libxtst6 2:1.1.0-3
libxv1 2:1.0.5-1
libxvmc1 2:1.0.5-1
libxxf86dga1 2:1.1.1-2
libxxf86vm1 1:1.1.0-2
linux-base 2.6.32-30
linux-image-2.6-686 2.6.32+29
linux-image-2.6.32-5-686 2.6.32-30
linux-sound-base 1.0.23+dfsg-2
locales 2.11.2-10
login 1:4.1.4.2+svn3283-2
logrotate 3.7.8-6
lsb-base 3.2-23.2squeeze1
lsof 4.81.dfsg.1-1
m4 1.4.14-3
man-db 2.5.7-8
manpages 3.27-1
mawk 1.3.3-15
menu 2.1.44
menu-xdg 0.5
metacity 1:2.30.1-3
metacity-common 1:2.30.1-3
mime-support 3.48-1
mlocate 0.22.2-1
module-init-tools 3.12-1
mount 2.17.2-9
mutt 1.5.20-9+squeeze1
nano 2.2.4-1
nautilus 2.30.1-2
nautilus-data 2.30.1-2
ncurses-base 5.7+20100313-5
ncurses-bin 5.7+20100313-5
ncurses-term 5.7+20100313-5
net-tools 1.60-23
netbase 4.45
netcat-traditional 1.10-38
nfs-common 1:1.2.2-4
notification-daemon 0.5.0-2
openssh-blacklist 0.4.1
openssh-client 1:5.5p1-6
openssl 0.9.8o-4
p7zip-full 9.04~dfsg.1-1
passwd 1:4.1.4.2+svn3283-2
patch 2.6-2
pciutils 1:3.1.7-6
perl 5.10.1-17
perl-base 5.10.1-17
perl-modules 5.10.1-17
policykit-1 0.96-4
policykit-1-gnome 0.96-3
poppler-utils 0.12.4-1.2
portmap 6.0.0-2
procmail 3.22-19
procps 1:3.2.8-9
psmisc 22.11-1
python 2.6.6-3+squeeze5
python-apt 0.7.100.1
python-apt-common 0.7.100.1
python-cairo 1.8.8-1+b1
python-central 0.6.16+nmu1
python-gconf 2.28.1-1
python-gmenu 2.30.3-1
python-gnome2 2.28.1-1
python-gobject 2.21.4+is.2.21.3-1
python-gtk2 2.17.0-4
python-gtksourceview2 2.10.1-1
python-libxml2 2.7.8.dfsg-2
python-minimal 2.6.6-3+squeeze5
python-numpy 1:1.4.1-5
python-pyorbit 2.24.0-6
python-reportbug 4.12.6
python-support 1.0.10
python2.6 2.6.6-8+b1
python2.6-minimal 2.6.6-8+b1
readline-common 6.1-3
reportbug 4.12.6
rsyslog 4.6.4-2
sed 4.2.1-7
sensible-utils 0.0.4
sgml-base 1.26+nmu1
sgml-data 2.0.4
shared-mime-info 0.71-4
ssl-cert 1.0.28
system-tools-backends 2.10.1-2
sysv-rc 2.88dsf-13.1
sysvinit 2.88dsf-13.1
sysvinit-utils 2.88dsf-13.1
tar 1.23-3
tasksel 2.88
tasksel-data 2.88
tcpd 7.6.q-19
telnet 0.17-36
texinfo 4.13a.dfsg.1-6
time 1.7-23.1
totem 2.30.2-6
totem-common 2.30.2-6
traceroute 1:2.0.15-1
ttf-dejavu-core 2.31-1
ttf-freefont 20090104-7
twm 1:1.0.4-2
tzdata 2010o-1
ucf 3.0025+nmu1
udev 164-3
udisks 1.0.1+git20100614-3
update-inetd 4.38+nmu1
upower 0.9.5-5
usbmuxd 1.0.4-1
usbutils 0.87-5
util-linux 2.17.2-9
vim-common 2:7.2.445+hg~cb94c42c0e1a-1
vim-tiny 2:7.2.445+hg~cb94c42c0e1a-1
w3m 0.5.2-9
wamerican 6-3
wget 1.12-2.1
whiptail 0.52.11-1
whois 5.0.10
x11-apps 7.5+5
x11-common 1:7.5+8
x11-session-utils 7.5+1
x11-utils 7.5+4
x11-xfs-utils 7.4+1
x11-xkb-utils 7.5+5
x11-xserver-utils 7.5+2
xauth 1:1.0.4-1
xfonts-100dpi 1:1.0.1
xfonts-75dpi 1:1.0.1
xfonts-base 1:1.0.1
xfonts-encodings 1:1.0.3-1
xfonts-scalable 1:1.0.1-1
xfonts-utils 1:7.5+2
xinit 1.2.0-2
xkb-data 1.8-2
xml-core 0.13
xorg 1:7.5+8
xorg-docs-core 1:1.5-1
xserver-common 2:1.7.7-11
xserver-xorg 1:7.5+8
xserver-xorg-core 2:1.7.7-11
xserver-xorg-input-all 1:7.5+8
xserver-xorg-input-evdev 1:2.3.2-6
xserver-xorg-input-synaptics 1.2.2-2
xserver-xorg-input-wacom 0.10.5+20100416-1
xserver-xorg-video-all 1:7.5+8
xserver-xorg-video-apm 1:1.2.2-2
xserver-xorg-video-ark 1:0.7.2-2
xserver-xorg-video-ati 1:6.13.1-2+squeeze1
xserver-xorg-video-chips 1:1.2.3-1
xserver-xorg-video-cirrus 1:1.3.2-2+squeeze1
xserver-xorg-video-fbdev 1:0.4.2-2
xserver-xorg-video-geode 2.11.9-7
xserver-xorg-video-i128 1:1.3.3-2
xserver-xorg-video-i740 1:1.3.2-2
xserver-xorg-video-intel 2:2.13.0-5
xserver-xorg-video-mach64 6.8.2-3
xserver-xorg-video-mga 1:1.4.11.dfsg-4+squeeze1
xserver-xorg-video-neomagic 1:1.2.4-3
xserver-xorg-video-nouveau 1:0.0.15+git20100329+7858345-5
xserver-xorg-video-nv 1:2.1.17-3
xserver-xorg-video-openchrome 1:0.2.904+svn842-2
xserver-xorg-video-r128 6.8.1-3
xserver-xorg-video-radeon 1:6.13.1-2+squeeze1
xserver-xorg-video-rendition 1:4.2.3-3
xserver-xorg-video-s3 1:0.6.3-2
xserver-xorg-video-s3virge 1:1.10.4-2
xserver-xorg-video-savage 1:2.3.1-2
xserver-xorg-video-siliconmotion 1:1.7.3-2
xserver-xorg-video-sis 1:0.10.3-1
xserver-xorg-video-sisusb 1:0.9.3-2
xserver-xorg-video-tdfx 1:1.4.3-2
xserver-xorg-video-trident 1:1.3.3-2
xserver-xorg-video-tseng 1:1.2.3-2
xserver-xorg-video-vesa 1:2.3.0-3
xserver-xorg-video-vmware 1:11.0.1-2
xserver-xorg-video-voodoo 1:1.2.3-2
xsltproc 1.1.26-6
xulrunner-1.9.1 1.9.1.16-4
xz-utils 5.0.0-2
yelp 2.30.1+webkit-1
zenity 2.30.0-1
zlib1g 1:1.2.3.4.dfsg-3 AppRecommender-0.7.5/bin/example/dpkg_debian6t 0000664 0000000 0000000 00000067671 13067513116 0021277 0 ustar 00root root 0000000 0000000 acpi 1.5-2
acpi-support-base 0.137-5
acpid 1:2.0.7-1
adduser 3.112+nmu2
alacarte 0.13.2-1
alsa-base 1.0.23+dfsg-2
alsa-utils 1.0.23-3
anacron 2.3-14
apache2.2-bin 2.2.16-6
app-install-data 2010.11.17
apt 0.8.10.3
apt-listchanges 2.85.7
apt-utils 0.8.10.3
apt-xapian-index 0.41
aptdaemon 0.31+bzr413-1.1
aptitude 0.6.3-3.2
aspell 0.60.6-4
aspell-en 6.0-0-6
at 3.1.12-1
at-spi 1.30.1-3
avahi-daemon 0.6.27-2+squeeze1
baobab 2.30.0-2
base-files 6.0
base-passwd 3.5.22
bash 4.1-3
bash-completion 1:1.2-3
bc 1.06.95-2
bind9-host 1:9.7.2.dfsg.P3-1.1
binfmt-support 1.2.18
bluez 4.66-3
bogofilter 1.2.2-2
bogofilter-bdb 1.2.2-2
bogofilter-common 1.2.2-2
brasero 2.30.3-2
brasero-common 2.30.3-2
browser-plugin-gnash 0.8.8-5
bsd-mailx 8.1.2-0.20100314cvs-1
bsdmainutils 8.0.13
bsdutils 1:2.17.2-9
bsh 2.0b4-12
bsh-gcj 2.0b4-12
busybox 1:1.17.1-8
bzip2 1.0.5-6
ca-certificates 20090814+nmu2
capplets-data 1:2.30.1-2
cdrdao 1:1.2.3-0.1
cheese 2.30.1-2
cheese-common 2.30.1-2
cli-common 0.7.1
console-setup 1.68
console-terminus 4.30-2
consolekit 0.4.1-4
coreutils 8.5-1
cpio 2.11-4
cpp 4:4.4.5-1
cpp-4.4 4.4.5-8
cpufrequtils 007-1
cron 3.0pl1-116
cups 1.4.4-7
cups-bsd 1.4.4-7
cups-client 1.4.4-7
cups-common 1.4.4-7
cups-driver-gutenprint 5.2.6-1
cups-pk-helper 0.1.0-2
cups-ppdc 1.4.4-7
dash 0.5.5.1-7.4
dasher 4.11-1
dasher-data 4.11-1
dbus 1.2.24-4
dbus-x11 1.2.24-4
dc 1.06.95-2
debconf 1.5.36.1
debconf-i18n 1.5.36.1
debian-archive-keyring 2010.08.28
debian-faq 4.0.4+nmu1
debianutils 3.4
defoma 0.11.11
deskbar-applet 2.32.0-1
desktop-base 6.0.5
desktop-file-utils 0.15-2
dictionaries-common 1.5.17
diffutils 1:3.0-1
discover 2.1.2-5
discover-data 2.2010.10.18
dmidecode 2.9-1.2
dmsetup 2:1.02.48-5
dmz-cursor-theme 0.4.3
dnsmasq-base 2.55-2+b1
dnsutils 1:9.7.2.dfsg.P3-1.1
doc-debian 4.0.2
doc-linux-text 2008.08-1
docbook-xml 4.5-7
dosfstools 3.0.9-1
dpkg 1.15.8.10
dvd+rw-tools 7.1-6
e2fslibs 1.41.12-2
e2fsprogs 1.41.12-2
eject 2.1.5+deb1+cvs20081104-7.1
ekiga 3.2.7-2
empathy 2.30.3-1
empathy-common 2.30.3-1
eog 2.30.2-1
epiphany-browser 2.30.6-1
epiphany-browser-data 2.30.6-1
epiphany-extensions 2.30.2-1
esound-common 0.2.41-8
espeak 1.43.03-2
espeak-data 1.43.03-2
evince 2.30.3-2
evince-common 2.30.3-2
evolution 2.30.3-5
evolution-common 2.30.3-5
evolution-data-server 2.30.3-2
evolution-data-server-common 2.30.3-2
evolution-exchange 2.30.2-1
evolution-plugins 2.30.3-5
evolution-webcal 2.28.1-1
exim4 4.72-6
exim4-base 4.72-6
exim4-config 4.72-6
exim4-daemon-light 4.72-6
fancontrol 1:3.1.2-6
file 5.04-5
file-roller 2.30.2-2
findutils 4.4.2-1+b1
fontconfig 2.8.0-2.1
fontconfig-config 2.8.0-2.1
foo2zjs 20090908dfsg-5.1
foomatic-db 20100630-1
foomatic-db-engine 4.0.4-3
foomatic-filters 4.0.5-6
foomatic-filters-ppds 1:4.0.4-3
freedesktop-sound-theme 0.7.dfsg-1
freeglut3 2.6.0-1
freepats 20060219-1
ftp 0.17-23
fuse-utils 2.8.4-1.1
gcalctool 5.30.2-2
gcc-4.4-base 4.4.5-8
gcj-4.4-jre-headless 4.4.5-2
gconf-defaults-service 2.28.1-6
gconf-editor 2.30.0-2
gconf2 2.28.1-6
gconf2-common 2.28.1-6
gdebi 0.6.4
gdebi-core 0.6.4
gdm3 2.30.5-6
gedit 2.30.4-1
gedit-common 2.30.4-1
gedit-plugins 2.30.0-1
genisoimage 9:1.1.11-1
geoclue 0.12.0-1
geoclue-hostip 0.12.0-1
geoclue-localnet 0.12.0-1
geoclue-manual 0.12.0-1
geoclue-yahoo 0.12.0-1
geoip-database 1.4.7~beta6+dfsg-1
gettext-base 0.18.1.1-3
ghostscript 8.71~dfsg2-9
ghostscript-cups 8.71~dfsg2-9
gir1.0-clutter-1.0 1.2.12-3
gir1.0-freedesktop 0.6.14-1+b1
gir1.0-glib-2.0 0.6.14-1+b1
gir1.0-gstreamer-0.10 0.10.30-1
gir1.0-gtk-2.0 0.6.5-7
gir1.0-json-glib-1.0 0.10.2-2
gksu 2.0.2-5
gnash 0.8.8-5
gnash-common 0.8.8-5
gnome 1:2.30+7
gnome-about 2.30.2-2
gnome-accessibility 1:2.30+7
gnome-accessibility-themes 2.30.2-1
gnome-applets 2.30.0-3
gnome-applets-data 2.30.0-3
gnome-backgrounds 2.32.0-1
gnome-bluetooth 2.30.0-2
gnome-cards-data 1:2.30.2-2
gnome-codec-install 0.4.7+nmu1
gnome-control-center 1:2.30.1-2
gnome-core 1:2.30+7
gnome-desktop-data 2.30.2-2
gnome-desktop-environment 1:2.30+7
gnome-dictionary 2.30.0-2
gnome-disk-utility 2.30.1-2
gnome-doc-utils 0.20.1-1
gnome-games 1:2.30.2-2
gnome-games-data 1:2.30.2-2
gnome-games-extra-data 2.30.0-1
gnome-icon-theme 2.30.3-2
gnome-js-common 0.1.1-1
gnome-keyring 2.30.3-5
gnome-mag 1:0.16.1-2
gnome-media 2.30.0-1
gnome-media-common 2.30.0-1
gnome-menus 2.30.3-1
gnome-mime-data 2.18.0-1
gnome-netstatus-applet 2.28.1-1
gnome-nettool 2.30.0-3
gnome-orca 2.30.2-2
gnome-panel 2.30.2-2
gnome-panel-data 2.30.2-2
gnome-power-manager 2.32.0-2
gnome-screensaver 2.30.0-2
gnome-screenshot 2.30.0-2
gnome-search-tool 2.30.0-2
gnome-session 2.30.2-3
gnome-session-bin 2.30.2-3
gnome-session-canberra 0.24-1
gnome-session-common 2.30.2-3
gnome-settings-daemon 2.30.2-2
gnome-system-log 2.30.0-2
gnome-system-monitor 2.28.1-1
gnome-system-tools 2.30.2-2
gnome-terminal 2.30.2-1
gnome-terminal-data 2.30.2-1
gnome-themes 2.30.2-1
gnome-themes-extras 2.22.0-3
gnome-themes-more 0.9.0.deb0.8
gnome-user-guide 2.30.1-1
gnome-user-share 2.30.1-1
gnome-utils-common 2.30.0-2
gnuchess 5.07-7
gnuchess-book 1.01-2
gnupg 1.4.10-4
gok 2.30.0-1
gpgv 1.4.10-4
grep 2.6.3-3
groff-base 1.20.1-10
grub-common 1.98+20100804-14
grub-pc 1.98+20100804-14
gsfonts 1:8.11+urwcyr1.0.7~pre44-4.2
gstreamer0.10-alsa 0.10.30-1
gstreamer0.10-ffmpeg 0.10.10-1
gstreamer0.10-fluendo-mp3 0.10.14.debian-1
gstreamer0.10-nice 0.0.12-1
gstreamer0.10-plugins-bad 0.10.19-2+b2
gstreamer0.10-plugins-base 0.10.30-1
gstreamer0.10-plugins-good 0.10.24-1
gstreamer0.10-plugins-ugly 0.10.15-1
gstreamer0.10-tools 0.10.30-1
gstreamer0.10-x 0.10.30-1
gtk2-engines 1:2.20.1-1
gtk2-engines-pixbuf 2.20.1-2
gtk2-engines-smooth 1:2.14.3+deb5
gucharmap 1:2.30.3-1
guile-1.8-libs 1.8.7+1-3
gvfs 1.6.4-3
gvfs-backends 1.6.4-3
gvfs-bin 1.6.4-3
gzip 1.3.12-9
hamster-applet 2.30.2-3
hdparm 9.32-1
hicolor-icon-theme 0.12-1
host 1:9.7.2.dfsg.P3-1.1
hostname 3.04
hp-ppd 0.9-0.1
hpijs 3.10.6-2
hplip 3.10.6-2
hplip-cups 3.10.6-2
hplip-data 3.10.6-2
hwdata 0.230-1
iceweasel 3.5.16-4
ifupdown 0.6.10
imagemagick 8:6.6.0.4-3
info 4.13a.dfsg.1-6
initramfs-tools 0.98.8
initscripts 2.88dsf-13.1
inkscape 0.47.0-2+b1
insserv 1.14.0-2
install-info 4.13a.dfsg.1-6
installation-report 2.44
iproute 20100519-3
iptables 1.4.8-3
iputils-ping 3:20100418-3
isc-dhcp-client 4.1.1-P1-15+squeeze1
isc-dhcp-common 4.1.1-P1-15+squeeze1
iso-codes 3.23-1
kbd 1.15.2-2
kerneloops 0.12+git20090217-1
keyboard-configuration 1.68
klibc-utils 1.5.20-1
laptop-detect 0.13.7
less 436-1
liba52-0.7.4 0.7.4-14
libaa1 1.4p5-38
libacl1 2.2.49-4
libao-common 1.0.0-5
libao4 1.0.0-5
libapache2-mod-dnssd 0.6-2
libapr1 1.4.2-6
libaprutil1 1.3.9+dfsg-5
libaprutil1-dbd-sqlite3 1.3.9+dfsg-5
libaprutil1-ldap 1.3.9+dfsg-5
libarchive1 2.8.4-1
libart-2.0-2 2.3.21-1
libart2.0-cil 2.24.1-6
libasound2 1.0.23-2.1
libaspell15 0.60.6-4
libass4 0.9.9-1
libasyncns0 0.3-1.1
libatasmart4 0.17+git20100219-2
libatk1.0-0 1.30.0-1
libatk1.0-data 1.30.0-1
libatspi1.0-0 1.30.1-3
libattr1 1:2.4.44-2
libaudiofile0 0.2.6-8
libaudit0 1.7.13-1+b2
libavahi-client3 0.6.27-2+squeeze1
libavahi-common-data 0.6.27-2+squeeze1
libavahi-common3 0.6.27-2+squeeze1
libavahi-core7 0.6.27-2+squeeze1
libavahi-glib1 0.6.27-2+squeeze1
libavahi-gobject0 0.6.27-2+squeeze1
libavahi-ui0 0.6.27-2+squeeze1
libavc1394-0 0.5.3-1+b2
libavcodec52 4:0.5.2-6
libavformat52 4:0.5.2-6
libavutil49 4:0.5.2-6
libbabl-0.0-0 0.0.22-1
libbind9-60 1:9.7.2.dfsg.P3-1.1
libblas3gf 1.2-8
libblkid1 2.17.2-9
libbluetooth3 4.66-3
libbonobo2-0 2.24.3-1
libbonobo2-common 2.24.3-1
libbonoboui2-0 2.24.3-1
libbonoboui2-common 2.24.3-1
libboost-date-time1.42.0 1.42.0-4
libboost-iostreams1.42.0 1.42.0-4
libboost-python1.42.0 1.42.0-4
libboost-thread1.42.0 1.42.0-4
libbrasero-media0 2.30.3-2
libbrlapi0.5 4.2-7
libbsd0 0.2.0-1
libburn4 0.8.0.pl00-2
libbz2-1.0 1.0.5-6
libc-bin 2.11.2-10
libc6 2.11.2-10
libc6-i686 2.11.2-10
libcaca0 0.99.beta17-1
libcairo-perl 1.070-1
libcairo2 1.8.10-6
libcairomm-1.0-1 1.8.4-3
libcamel1.2-14 2.30.3-2
libcanberra-gtk-module 0.24-1
libcanberra-gtk0 0.24-1
libcanberra0 0.24-1
libcap-ng0 0.6.4-1
libcap2 1:2.19-3
libcdaudio1 0.99.12p2-9
libcdio-cdda0 0.81-4
libcdio-paranoia0 0.81-4
libcdio10 0.81-4
libcdparanoia0 3.10.2+debian-9
libcdt4 2.26.3-5
libcelt0-0 0.7.1-1
libchamplain-0.4-0 0.4.6-2+b1
libchamplain-gtk-0.4-0 0.4.6-2+b1
libcheese-gtk18 2.30.1-2
libck-connector0 0.4.1-4
libclutter-1.0-0 1.2.12-3
libclutter-gtk-0.10-0 0.10.4-1
libcolamd2.7.1 1:3.4.0-2
libcolorblind0 0.0.1-1
libcomerr2 1.41.12-2
libcpufreq0 007-1
libcroco3 0.6.2-1
libcryptui0 2.30.1-2
libcups2 1.4.4-7
libcupscgi1 1.4.4-7
libcupsdriver1 1.4.4-7
libcupsimage2 1.4.4-7
libcupsmime1 1.4.4-7
libcupsppdc1 1.4.4-7
libcurl3-gnutls 7.21.0-1
libcwidget3 0.5.16-3
libdaemon0 0.14-2
libdatrie1 0.2.4-1
libdb4.6 4.6.21-16
libdb4.7 4.7.25-9
libdb4.7-java-gcj 4.7.25-9
libdb4.8 4.8.30-2
libdbus-1-3 1.2.24-4
libdbus-glib-1-2 0.88-2.1
libdc1394-22 2.1.2-3
libdca0 0.0.5-3
libdevmapper1.02.1 2:1.02.48-5
libdirac-encoder0 1.0.2-3
libdirectfb-1.2-9 1.2.10.0-4
libdiscid0 0.2.2-1
libdiscover2 2.1.2-5
libdjvulibre-text 3.5.23-3
libdjvulibre21 3.5.23-3
libdns69 1:9.7.2.dfsg.P3-1.1
libdrm-intel1 2.4.21-1~squeeze3
libdrm-nouveau1 2.4.21-1~squeeze3
libdrm-radeon1 2.4.21-1~squeeze3
libdrm2 2.4.21-1~squeeze3
libdv4 1.0.0-2.1
libdvdnav4 4.1.3-7
libdvdread4 4.1.3-10
libebackend1.2-0 2.30.3-2
libebook1.2-9 2.30.3-2
libecal1.2-7 2.30.3-2
libedata-book1.2-2 2.30.3-2
libedata-cal1.2-7 2.30.3-2
libedataserver1.2-13 2.30.3-2
libedataserverui1.2-8 2.30.3-2
libedit2 2.11-20080614-2
libeggdbus-1-0 0.6-1
libegroupwise1.2-13 2.30.3-2
libelf1 0.148-1
libenca0 1.13-3
libenchant1c2a 1.6.0-1
libepc-1.0-2 0.3.11-1
libepc-common 0.3.11-1
libepc-ui-1.0-2 0.3.11-1
libept1 1.0.4
libesd0 0.2.41-8
libespeak1 1.43.03-2
libevent-1.4-2 1.4.13-stable-1
libevince2 2.30.3-2
libevolution 2.30.3-5
libexempi3 2.1.1-1
libexif12 0.6.19-1
libexiv2-9 0.20-2
libexpat1 2.0.1-7
libfaad2 2.7-6
libfam0 2.7.0-17
libffi5 3.0.9-3
libfftw3-3 3.2.2-1
libfile-copy-recursive-perl 0.38-1
libflac8 1.2.1-2+b1
libflite1 1.4-release-2
libfont-afm-perl 1.20-1
libfont-freetype-perl 0.03-1
libfontconfig1 2.8.0-2.1
libfontenc1 1:1.0.5-2
libfreerdp-plugins-standard 0.7.4-1
libfreerdp0 0.7.4-1
libfreetype6 2.4.2-2.1
libfs6 2:1.0.2-1
libfuse2 2.8.4-1.1
libgail-common 2.20.1-2
libgail-gnome-module 1.20.3-1
libgail18 2.20.1-2
libgalago3 0.5.2-2
libgc1c2 1:6.8-1.2
libgcc1 1:4.4.5-8
libgcj-common 1:4.4.5-1
libgcj10 4.4.5-2
libgconf2-4 2.28.1-6
libgconf2.0-cil 2.24.1-6
libgcr0 2.30.3-5
libgcrypt11 1.4.5-2
libgd2-noxpm 2.0.36~rc1~dfsg-5
libgdata-common 0.6.4-2
libgdata-google1.2-1 2.30.3-2
libgdata1.2-1 2.30.3-2
libgdata7 0.6.4-2
libgdbm3 1.8.3-9
libgdict-1.0-6 2.30.0-2
libgdu-gtk0 2.30.1-2
libgdu0 2.30.1-2
libgee2 0.5.2-1
libgegl-0.0-0 0.0.22-2+b1
libgeoclue0 0.12.0-1
libgeoip1 1.4.7~beta6+dfsg-1
libgexiv2-0 0.1.0-1+b1
libgfortran3 4.4.5-8
libgif4 4.1.6-9
libgimp2.0 2.6.10-1
libgirepository1.0-0 0.6.14-1+b1
libgksu2-0 2.0.13~pre1-3
libgl1-mesa-dri 7.7.1-4
libgl1-mesa-glx 7.7.1-4
libglade2-0 1:2.6.4-1
libglade2.0-cil 2.12.10-1
libglib-perl 2:1.223-1
libglib2.0-0 2.24.2-1
libglib2.0-cil 2.12.10-1
libglib2.0-data 2.24.2-1
libglibmm-2.4-1c2a 2.24.2-1
libglu1-mesa 7.7.1-4
libgme0 0.5.5-2
libgmime-2.4-2 2.4.14-1+nmu1
libgmime2.4-cil 2.4.14-1+nmu1
libgmp3c2 2:4.3.2+dfsg-1
libgnome-bluetooth7 2.30.0-2
libgnome-desktop-2-17 2.30.2-2
libgnome-keyring0 2.30.1-1
libgnome-mag2 1:0.16.1-2
libgnome-media0 2.30.0-1
libgnome-menu2 2.30.3-1
libgnome-speech7 1:0.4.25-4
libgnome-vfs2.0-cil 2.24.1-6
libgnome-window-settings1 1:2.30.1-2
libgnome2-0 2.30.0-1
libgnome2-canvas-perl 1.002-2
libgnome2-common 2.30.0-1
libgnome2-perl 1.042-2
libgnome2-vfs-perl 1.081-1
libgnome2.24-cil 2.24.1-6
libgnomecanvas2-0 2.30.1-1
libgnomecanvas2-common 2.30.1-1
libgnomekbd-common 2.30.2-2
libgnomekbd4 2.30.2-2
libgnomepanel2.24-cil 2.26.0-3+b1
libgnomeui-0 2.24.3-1
libgnomeui-common 2.24.3-1
libgnomevfs2-0 1:2.24.3-1
libgnomevfs2-common 1:2.24.3-1
libgnomevfs2-extra 1:2.24.3-1
libgnutls26 2.8.6-1
libgomp1 4.4.5-8
libgp11-0 2.30.3-5
libgpg-error0 1.6-1
libgpgme11 1.2.0-1.2
libgphoto2-2 2.4.6-3
libgphoto2-port0 2.4.6-3
libgpm2 1.20.4-3.3
libgpod-common 0.7.93-0.3
libgpod4 0.7.93-0.3
libgraph4 2.26.3-5
libgraphite3 1:2.3.1-0.2
libgs8 8.71~dfsg2-9
libgsf-1-114 1.14.18-1
libgsf-1-common 1.14.18-1
libgsl0ldbl 1.14+dfsg-1
libgsm1 1.0.13-3
libgssapi-krb5-2 1.8.3+dfsg-4
libgssdp-1.0-2 0.8.0-2
libgssglue1 0.1-4
libgssrpc4 1.8.3+dfsg-4
libgstfarsight0.10-0 0.0.20-2
libgstreamer-plugins-base0.10-0 0.10.30-1
libgstreamer0.10-0 0.10.30-1
libgtk2-perl 2:1.222-1
libgtk2.0-0 2.20.1-2
libgtk2.0-bin 2.20.1-2
libgtk2.0-cil 2.12.10-1
libgtk2.0-common 2.20.1-2
libgtkglext1 1.2.0-1.1
libgtkhtml-editor-common 3.30.3-1
libgtkhtml-editor0 3.30.3-1
libgtkhtml3.14-19 3.30.3-1
libgtkimageview0 1.6.4-1
libgtkmm-2.4-1c2a 1:2.20.3-1
libgtksourceview2.0-0 2.10.4-1
libgtksourceview2.0-common 2.10.4-1
libgtkspell0 2.0.16-1
libgtop2-7 2.28.1-1
libgtop2-common 2.28.1-1
libgucharmap7 1:2.30.3-1
libgudev-1.0-0 164-3
libgupnp-1.0-3 0.14.0-2
libgupnp-igd-1.0-3 0.1.7-3
libgutenprint2 5.2.6-1
libgvc5 2.26.3-5
libgweather-common 2.30.3-1
libgweather1 2.30.3-1
libhal-storage1 0.5.14-3
libhal1 0.5.14-3
libhpmud0 3.10.6-2
libhsqldb-java-gcj 1.8.0.10-9
libhtml-format-perl 2.04-2
libhtml-parser-perl 3.66-1
libhtml-tagset-perl 3.20-2
libhtml-tree-perl 3.23-2
libhunspell-1.2-0 1.2.11-1
libhyphen0 2.5-1
libical0 0.44-3
libice6 2:1.0.6-2
libicu44 4.4.1-7
libid3tag0 0.15.1b-10
libidl0 0.8.14-0.1
libidn11 1.15-2
libiec61883-0 1.2.0-0.1
libieee1284-3 0.2.11-6
libijs-0.35 0.35-7
libilmbase6 1.0.1-3
libimobiledevice1 1.0.2-1
libiptcdata0 1.0.4-1+b1
libisc62 1:9.7.2.dfsg.P3-1.1
libisccc60 1:9.7.2.dfsg.P3-1.1
libisccfg62 1:9.7.2.dfsg.P3-1.1
libisofs6 0.6.32-2
libjack-jackd2-0 1.9.6~dfsg.1-2
libjasper1 1.900.1-7+b1
libjbig2dec0 0.11-1
libjpeg62 6b1-1
libjson-glib-1.0-0 0.10.2-2
libk5crypto3 1.8.3+dfsg-4
libkadm5clnt-mit7 1.8.3+dfsg-4
libkadm5srv-mit7 1.8.3+dfsg-4
libkate1 0.3.7-3
libkdb5-4 1.8.3+dfsg-4
libkeyutils1 1.4-1
libklibc 1.5.20-1
libkpathsea5 2009-8
libkrb5-3 1.8.3+dfsg-4
libkrb5support0 1.8.3+dfsg-4
liblapack3gf 3.2.1-8
liblcms1 1.18.dfsg-1.2+b3
libldap-2.4-2 2.4.23-7
liblircclient0 0.8.3-5
liblocale-gettext-perl 1.05-6
liblockfile1 1.08-4
liblouis-data 2.0.0-1
liblouis2 2.0.0-1
liblqr-1-0 0.4.1-1
libltdl7 2.2.6b-2
liblua5.1-0 5.1.4-5
liblwres60 1:9.7.2.dfsg.P3-1.1
liblzma2 5.0.0-2
libmad0 0.15.1b-5
libmagic1 5.04-5
libmagick++3 8:6.6.0.4-3
libmagickcore3 8:6.6.0.4-3
libmagickwand3 8:6.6.0.4-3
libmailtools-perl 2.06-1
libmetacity-private0 1:2.30.1-3
libmimic0 1.0.4-2+b2
libmms0 0.6-1
libmng1 1.0.10-1+b1
libmodplug1 1:0.8.8.1-1
libmono-addins-gui0.2-cil 0.4-8
libmono-addins0.2-cil 0.4-8
libmono-cairo2.0-cil 2.6.7-5
libmono-corlib2.0-cil 2.6.7-5
libmono-i18n-west2.0-cil 2.6.7-5
libmono-posix2.0-cil 2.6.7-5
libmono-security2.0-cil 2.6.7-5
libmono-sharpzip2.84-cil 2.6.7-5
libmono-system2.0-cil 2.6.7-5
libmozjs2d 1.9.1.16-4
libmpcdec6 2:0.1~r459-1
libmpeg2-4 0.4.1-3
libmpfr4 3.0.0-2
libmtp8 1.0.3-1
libmusicbrainz3-6 3.0.2-2
libmusicbrainz4c2a 2.1.5-4
libmythes-1.2-0 2:1.2.1-1
libnautilus-extension1 2.30.1-2
libncurses5 5.7+20100313-5
libncursesw5 5.7+20100313-5
libndesk-dbus-glib1.0-cil 0.4.1-3
libndesk-dbus1.0-cil 0.6.0-4
libneon27-gnutls 0.29.3-3
libnet-dbus-perl 0.33.6-2
libnet1 1.1.4-2
libnetpbm10 2:10.0-12.2+b1
libnewt0.52 0.52.11-1
libnfnetlink0 1.0.0-1
libnfsidmap2 0.23-2
libnice0 0.0.12-1
libnl1 1.1-6
libnm-glib-vpn1 0.8.1-6
libnm-glib2 0.8.1-6
libnm-util1 0.8.1-6
libnotify1 0.5.0-2
libnspr4-0d 4.8.6-1
libnss-mdns 0.10-3.1
libnss3-1d 3.12.8-1
libntfs-3g75 1:2010.3.6-1
libntfs10 2.0.0-1+b1
libofa0 0.9.3-3.1
libogg0 1.2.0~dfsg-1
liboil0.3 0.3.17-2
liboobs-1-4 2.30.1-1
libopal3.6.8 3.6.8~dfsg-2
libopencore-amrnb0 0.1.2-1
libopencore-amrwb0 0.1.2-1
libopenexr6 1.6.1-4.1
libopenjpeg2 1.3+dfsg-4
libopenobex1 1.5-2
libopenraw1 0.0.8-2
libopenspc0 0.3.99a-2
liborbit2 1:2.14.18-0.1
liborc-0.4-0 1:0.4.6-2
libpam-ck-connector 0.4.1-4
libpam-gnome-keyring 2.30.3-5
libpam-modules 1.1.1-6.1
libpam-runtime 1.1.1-6.1
libpam0g 1.1.1-6.1
libpanel-applet2-0 2.30.2-2
libpango-perl 1.221-2
libpango1.0-0 1.28.3-1+squeeze2
libpango1.0-common 1.28.3-1+squeeze2
libpangomm-1.4-1 2.26.2-1
libpaper-utils 1.1.24
libpaper1 1.1.24
libparted0debian1 2.3-5
libpathplan4 2.26.3-5
libpcap0.8 1.1.1-2
libpci3 1:3.1.7-6
libpciaccess0 0.12.0-1
libpcre3 8.02-1.1
libpcsclite1 1.5.5-4
libperl5.10 5.10.1-17
libpixman-1-0 0.16.4-1
libplist1 1.3-2
libplot2c2 2.5-4
libpng12-0 1.2.44-1
libpolkit-agent-1-0 0.96-4
libpolkit-backend-1-0 0.96-4
libpolkit-gobject-1-0 0.96-4
libpolkit-gtk-1-0 0.96-3
libpoppler-glib4 0.12.4-1.2
libpoppler5 0.12.4-1.2
libpopt0 1.16-1
libportaudio2 19+svn20071022-3.2
libpostproc51 4:0.5.2-6
libproxy0 0.3.1-2
libpstoedit0c2a 3.50-3+b1
libpt2.6.7 2.6.7-1
libpth20 2.0.7-16
libpulse-mainloop-glib0 0.9.21-3
libpulse0 0.9.21-3
libpython2.6 2.6.6-8+b1
libraptor1 1.4.21-2
librarian0 0.8.1-5
librasqal2 0.9.20-1
libraw1394-11 2.0.5-2
librdf0 1.0.10-3
libreadline6 6.1-3
librpcsecgss3 0.19-2
librpm1 4.8.1-6
librpmio1 4.8.1-6
librsvg2-2 2.26.3-1
librsvg2-common 2.26.3-1
libsane 1.0.21-9
libsane-extras 1.0.21.2
libsane-hpaio 3.10.6-2
libsasl2-2 2.1.23.dfsg1-7
libsasl2-modules 2.1.23.dfsg1-7
libschroedinger-1.0-0 1.0.9-2
libsdl1.2debian 1.2.14-6.1
libsdl1.2debian-alsa 1.2.14-6.1
libseed0 2.30.0-1+b1
libselinux1 2.0.96-1
libsensors4 1:3.1.2-6
libsepol1 2.0.41-1
libsgutils2-2 1.29-1
libshout3 2.2.2-5+b1
libsidplay1 1.36.59-5
libsigc++-2.0-0c2a 2.2.4.2-1
libslab0a 2.30.0-1
libslang2 2.2.2-4
libslp1 1.2.1-7.8
libslv2-9 0.6.6-5
libsm6 2:1.1.1-1
libsmbclient 2:3.5.6~dfsg-3squeeze2
libsndfile1 1.0.21-3
libsnmp-base 5.4.3~dfsg-2
libsnmp15 5.4.3~dfsg-2
libsoundtouch1c2 1.3.1-2
libsoup-gnome2.4-1 2.30.2-1
libsoup2.4-1 2.30.2-1
libspectre1 0.2.6-1
libspeex1 1.2~rc1-1
libspeexdsp1 1.2~rc1-1
libsqlite3-0 3.7.3-1
libsrtp0 1.4.4~dfsg-6
libss2 1.41.12-2
libssh-4 0.4.5-3
libssl0.9.8 0.9.8o-4squeeze1
libstartup-notification0 0.10-1
libstdc++6 4.4.5-8
libstlport4.6ldbl 4.6.2-7
libsvga1 1:1.4.3-29
libswscale0 4:0.5.2-6
libsysfs2 2.1.0+repack-1
libt1-5 5.1.2-3
libtag1-vanilla 1.6.3-1
libtag1c2a 1.6.3-1
libtalloc2 2.0.1-1
libtasn1-3 2.7-1
libtdb1 1.2.1-2+b1
libtelepathy-farsight0 0.0.14-2+b1
libtelepathy-glib0 0.11.11-1
libtext-charwidth-perl 0.04-6
libtext-iconv-perl 1.7-2
libtext-wrapi18n-perl 0.06-7
libthai-data 0.1.14-2
libthai0 0.1.14-2
libtheora0 1.1.1+dfsg.1-3
libtidy-0.99-0 20091223cvs-1
libtie-ixhash-perl 1.21-2
libtiff4 3.9.4-5
libtimedate-perl 1.2000-1
libtokyocabinet8 1.4.37-6
libtotem-plparser17 2.30.3-1
libtracker-client-0.8-0 0.8.17-1
libts-0.0-0 1.0-7
libtwolame0 0.3.12-1
libudev0 164-3
libunique-1.0-0 1.1.6-1.1
libupower-glib1 0.9.5-5
liburi-perl 1.54-2
libusb-0.1-4 2:0.1.12-16
libusb-1.0-0 2:1.0.8-2
libusbmuxd1 1.0.4-1
libutempter0 1.1.5-3
libuuid-perl 0.02-4
libuuid1 2.17.2-9
libv4l-0 0.8.0-1
libvisual-0.4-0 0.4.0-3
libvisual-0.4-plugins 0.4.0.dfsg.1-2
libvorbis0a 1.3.1-1
libvorbisenc2 1.3.1-1
libvorbisfile3 1.3.1-1
libvpx0 0.9.1-2
libvte-common 1:0.24.3-2
libvte9 1:0.24.3-2
libwavpack1 4.60.1-1
libwbclient0 2:3.5.6~dfsg-3squeeze2
libwebkit-1.0-2 1.2.6-2
libwebkit-1.0-common 1.2.6-2
libwildmidi1 0.2.3.2-2
libwmf0.2-7 0.2.8.4-6.1+b1
libwnck-common 2.30.4-2
libwnck22 2.30.4-2
libwpd8c2a 0.8.14-1
libwpg-0.1-1 0.1.3-1
libwps-0.1-1 0.1.2-1
libwrap0 7.6.q-19
libwww-perl 5.836-1
libx11-6 2:1.3.3-4
libx11-data 2:1.3.3-4
libx11-xcb1 2:1.3.3-4
libx86-1 1.1+ds1-6
libxapian22 1.2.3-2
libxau6 1:1.0.6-1
libxaw7 2:1.0.7-1
libxcb-atom1 0.3.6-1
libxcb-aux0 0.3.6-1
libxcb-dri2-0 1.6-1
libxcb-event1 0.3.6-1
libxcb-render-util0 0.3.6-1
libxcb-render0 1.6-1
libxcb1 1.6-1
libxcomposite1 1:0.4.2-1
libxcursor1 1:1.1.10-2
libxdamage1 1:1.1.3-1
libxdmcp6 1:1.0.3-2
libxdot4 2.26.3-5
libxext6 2:1.1.2-1
libxfixes3 1:4.0.5-1
libxfont1 1:1.4.1-2
libxft2 2.1.14-2
libxi6 2:1.3-6
libxinerama1 2:1.1-3
libxkbfile1 1:1.0.6-2
libxklavier16 5.0-2
libxml-parser-perl 2.36-1.1+b1
libxml-twig-perl 1:3.34-1
libxml-xpathengine-perl 0.12-2
libxml2 2.7.8.dfsg-2
libxml2-utils 2.7.8.dfsg-2
libxmu6 2:1.0.5-2
libxmuu1 2:1.0.5-2
libxpm4 1:3.5.8-1
libxrandr2 2:1.3.0-3
libxrender1 1:0.9.6-1
libxres1 2:1.0.4-1
libxslt1.1 1.1.26-6
libxss1 1:1.2.0-2
libxt6 1:1.0.7-1
libxtst6 2:1.1.0-3
libxv1 2:1.0.5-1
libxvmc1 2:1.0.5-1
libxxf86dga1 2:1.1.1-2
libxxf86vm1 1:1.1.0-2
libzbar0 0.10+doc-4
liferea 1.6.4-1
liferea-data 1.6.4-1
linux-base 2.6.32-30
linux-image-2.6-686 2.6.32+29
linux-image-2.6.32-5-686 2.6.32-30
linux-sound-base 1.0.23+dfsg-2
lm-sensors 1:3.1.2-6
locales 2.11.2-10
login 1:4.1.4.2+svn3283-2+squeeze1
logrotate 3.7.8-6
lsb-base 3.2-23.2squeeze1
lsb-release 3.2-23.2squeeze1
lsof 4.81.dfsg.1-1
m4 1.4.14-3
man-db 2.5.7-8
manpages 3.27-1
mawk 1.3.3-15
media-player-info 6-1
menu 2.1.44
menu-xdg 0.5
mesa-utils 7.7.1-4
metacity 1:2.30.1-3
metacity-common 1:2.30.1-3
mime-support 3.48-1
min12xxw 0.0.9-3
mlocate 0.22.2-1
mobile-broadband-provider-info 20101106-1
modemmanager 0.4+git.20100624t180933.6e79d15-2
module-init-tools 3.12-1
mono-2.0-gac 2.6.7-5
mono-gac 2.6.7-5
mono-runtime 2.6.7-5
mount 2.17.2-9
mousetweaks 2.30.2-1
mtools 4.0.12-1
mutt 1.5.20-9+squeeze1
myspell-en-us 1:3.2.1-2
nano 2.2.4-1
nautilus 2.30.1-2
nautilus-data 2.30.1-2
nautilus-sendto 2.28.4-2+b1
nautilus-sendto-empathy 2.30.3-1
ncurses-base 5.7+20100313-5
ncurses-bin 5.7+20100313-5
ncurses-term 5.7+20100313-5
net-tools 1.60-23
netbase 4.45
netcat-traditional 1.10-38
network-manager 0.8.1-6
network-manager-gnome 0.8.1-2
nfs-common 1:1.2.2-4
notification-daemon 0.5.0-2
ntfs-3g 1:2010.3.6-1
ntfsprogs 2.0.0-1+b1
obex-data-server 0.4.5-1+b1
obexd-client 0.28-1
openoffice.org-base 1:3.2.1-11+squeeze2
openoffice.org-calc 1:3.2.1-11+squeeze2
openoffice.org-common 1:3.2.1-11+squeeze2
openoffice.org-draw 1:3.2.1-11+squeeze2
openoffice.org-filter-binfilter 1:3.2.1-11+squeeze2
openoffice.org-gcj 1:3.2.1-11+squeeze2
openoffice.org-impress 1:3.2.1-11+squeeze2
openoffice.org-math 1:3.2.1-11+squeeze2
openoffice.org-thesaurus-en-us 1:3.2.1-2
openoffice.org-writer 1:3.2.1-11+squeeze2
openssh-blacklist 0.4.1
openssh-blacklist-extra 0.4.1
openssh-client 1:5.5p1-6
openssl 0.9.8o-4squeeze1
os-prober 1.42
p7zip-full 9.04~dfsg.1-1
passwd 1:4.1.4.2+svn3283-2+squeeze1
patch 2.6-2
pciutils 1:3.1.7-6
perl 5.10.1-17
perl-base 5.10.1-17
perl-modules 5.10.1-17
pkg-config 0.25-1.1
pm-utils 1.3.0-3
pnm2ppa 1.12-16.3
policykit-1 0.96-4
policykit-1-gnome 0.96-3
poppler-utils 0.12.4-1.2
portmap 6.0.0-2
powermgmt-base 1.31
ppp 2.4.5-4
procmail 3.22-19
procps 1:3.2.8-9
psmisc 22.11-1
python 2.6.6-3+squeeze5
python-apt 0.7.100.1
python-apt-common 0.7.100.1
python-aptdaemon 0.31+bzr413-1.1
python-aptdaemon-gtk 0.31+bzr413-1.1
python-axiom 0.6.0-2
python-beautifulsoup 3.1.0.1-2
python-brlapi 4.2-7
python-bugbuddy 2.30.0-4
python-cairo 1.8.8-1+b1
python-central 0.6.16+nmu1
python-chardet 2.0.1-1
python-clientform 0.2.10-2.1
python-coherence 0.6.6.2-5
python-configobj 4.7.2+ds-1
python-crypto 2.1.0-2
python-cups 1.9.48-1
python-cupshelpers 1.2.3-3
python-dbus 0.83.1-1
python-debian 0.1.18
python-epsilon 0.6.0-3
python-evolution 2.30.0-4
python-feedparser 4.1-14
python-gconf 2.28.1-1
python-gdata 2.0.8-1.1
python-gdbm 2.6.6-1
python-glade2 2.17.0-4
python-gmenu 2.30.3-1
python-gnome2 2.28.1-1
python-gnomeapplet 2.30.0-4
python-gnomedesktop 2.30.0-4
python-gnomekeyring 2.30.0-4
python-gnupginterface 0.3.2-9.1
python-gobject 2.21.4+is.2.21.3-1
python-gst0.10 0.10.19-1
python-gtk2 2.17.0-4
python-gtkglext1 1.1.0-5
python-gtksourceview2 2.10.1-1
python-httplib2 0.6.0-4
python-imaging 1.1.7-2
python-libxml2 2.7.8.dfsg-2
python-louie 1.1-1.1
python-louis 2.0.0-1
python-mako 0.3.4-5
python-markupsafe 0.9.2-3
python-mechanize 0.1.11-1.1
python-minimal 2.6.6-3+squeeze5
python-nevow 0.10.0-2
python-notify 0.1.1-2+b2
python-numpy 1:1.4.1-5
python-opengl 3.0.1~b2-1
python-openssl 0.10-1
python-pam 0.4.2-12.2
python-pexpect 2.3-1
python-pkg-resources 0.6.14-4
python-pyasn1 0.0.11a-1
python-pyatspi 1.30.1-3
python-pyorbit 2.24.0-6
python-pysqlite2 2.6.0-1
python-rdflib 2.4.2-1+b1
python-reportbug 4.12.6
python-serial 2.3-1
python-software-properties 0.60.debian-3
python-support 1.0.10
python-tagpy 0.94.7-2
python-twisted-bin 10.1.0-3
python-twisted-conch 1:10.1.0-1
python-twisted-core 10.1.0-3
python-twisted-web 10.1.0-1
python-utidylib 0.2-6
python-vte 1:0.24.3-2
python-webkit 1.1.7-1+b1
python-wnck 2.30.0-4
python-xapian 1.2.3-3
python-xdg 0.19-2
python-zope.interface 3.5.3-1+b1
python2.6 2.6.6-8+b1
python2.6-minimal 2.6.6-8+b1
radeontool 1.6.1-1
rarian-compat 0.8.1-5
readline-common 6.1-3
remmina 0.8.3-1
remmina-plugin-data 0.8.3-2
remmina-plugin-rdp 0.8.3-2
remmina-plugin-vnc 0.8.3-2
reportbug 4.12.6
rhythmbox 0.12.8-3
rhythmbox-plugin-cdrecorder 0.12.8-3
rhythmbox-plugins 0.12.8-3
rpm-common 4.8.1-6
rpm2cpio 4.8.1-6
rsyslog 4.6.4-2
sane-utils 1.0.21-9
seahorse 2.30.1-2
seahorse-plugins 2.30.1-3
sed 4.2.1-7
sensible-utils 0.0.4
sgml-base 1.26+nmu1
sgml-data 2.0.4
shared-mime-info 0.71-4
software-center 2.0.7debian7
software-properties-gtk 0.60.debian-3
sound-juicer 2.28.2-3
ssl-cert 1.0.28
sudo 1.7.4p4-2.squeeze.1
synaptic 0.70~pre1+b1
system-config-printer 1.2.3-3
system-config-printer-udev 1.2.3-3
system-tools-backends 2.10.1-2
sysv-rc 2.88dsf-13.1
sysvinit 2.88dsf-13.1
sysvinit-utils 2.88dsf-13.1
tar 1.23-3
tasksel 2.88
tasksel-data 2.88
tcl 8.4.16-2
tcl8.4 8.4.19-4
tcpd 7.6.q-19
tcptraceroute 1.5beta7+debian-4
telepathy-gabble 0.9.15-1+squeeze1
telepathy-mission-control-5 1:5.4.3-1
telepathy-salut 0.3.12-1
telnet 0.17-36
texinfo 4.13a.dfsg.1-6
time 1.7-23.1
tomboy 1.2.2-2
totem 2.30.2-6
totem-coherence 2.30.2-6
totem-common 2.30.2-6
totem-mozilla 2.30.2-6
totem-plugins 2.30.2-6
traceroute 1:2.0.15-1
transmission-common 2.03-2
transmission-gtk 2.03-2
tsconf 1.0-7
ttf-dejavu 2.31-1
ttf-dejavu-core 2.31-1
ttf-dejavu-extra 2.31-1
ttf-freefont 20090104-7
ttf-lyx 1.6.7-1
twm 1:1.0.4-2
tzdata 2010o-1
ucf 3.0025+nmu1
udev 164-3
udisks 1.0.1+git20100614-3
unattended-upgrades 0.62.2
uno-libs3 1.6.1+OOo3.2.1-11+squeeze2
unzip 6.0-4
update-inetd 4.38+nmu1
update-manager-core 0.200.5-1
update-manager-gnome 0.200.5-1
update-notifier 0.99.3debian8
update-notifier-common 0.99.3debian8
upower 0.9.5-5
ure 1.6.1+OOo3.2.1-11+squeeze2
usb-modeswitch 1.1.4-2
usb-modeswitch-data 20100826-1
usbmuxd 1.0.4-1
usbutils 0.87-5
util-linux 2.17.2-9
vbetool 1.1-2
vim-common 2:7.2.445+hg~cb94c42c0e1a-1
vim-tiny 2:7.2.445+hg~cb94c42c0e1a-1
vino 2.28.2-2
w3m 0.5.2-9
wamerican 6-3
wget 1.12-2.1
whiptail 0.52.11-1
whois 5.0.10
wpasupplicant 0.6.10-2.1
x-ttcidfont-conf 32
x11-apps 7.5+5
x11-common 1:7.5+8
x11-session-utils 7.5+1
x11-utils 7.5+4
x11-xfs-utils 7.4+1
x11-xkb-utils 7.5+5
x11-xserver-utils 7.5+2
xauth 1:1.0.4-1
xbase-clients 1:7.5+8
xbitmaps 1.1.0-1
xdg-user-dirs 0.13-2
xdg-user-dirs-gtk 0.8-1
xdg-utils 1.0.2+cvs20100307-2
xfonts-100dpi 1:1.0.1
xfonts-75dpi 1:1.0.1
xfonts-base 1:1.0.1
xfonts-encodings 1:1.0.3-1
xfonts-mathml 4
xfonts-scalable 1:1.0.1-1
xfonts-utils 1:7.5+2
xinit 1.2.0-2
xkb-data 1.8-2
xml-core 0.13
xorg 1:7.5+8
xorg-docs-core 1:1.5-1
xsane 0.997-2+b1
xsane-common 0.997-2
xserver-common 2:1.7.7-11
xserver-xephyr 2:1.7.7-11
xserver-xorg 1:7.5+8
xserver-xorg-core 2:1.7.7-11
xserver-xorg-input-all 1:7.5+8
xserver-xorg-input-evdev 1:2.3.2-6
xserver-xorg-input-mouse 1:1.5.0-2
xserver-xorg-input-synaptics 1.2.2-2
xserver-xorg-input-vmmouse 1:12.6.9-2
xserver-xorg-input-wacom 0.10.5+20100416-1
xserver-xorg-video-all 1:7.5+8
xserver-xorg-video-apm 1:1.2.2-2
xserver-xorg-video-ark 1:0.7.2-2
xserver-xorg-video-ati 1:6.13.1-2+squeeze1
xserver-xorg-video-chips 1:1.2.3-1
xserver-xorg-video-cirrus 1:1.3.2-2+squeeze1
xserver-xorg-video-fbdev 1:0.4.2-2
xserver-xorg-video-geode 2.11.9-7
xserver-xorg-video-i128 1:1.3.3-2
xserver-xorg-video-i740 1:1.3.2-2
xserver-xorg-video-intel 2:2.13.0-5
xserver-xorg-video-mach64 6.8.2-3
xserver-xorg-video-mga 1:1.4.11.dfsg-4+squeeze1
xserver-xorg-video-neomagic 1:1.2.4-3
xserver-xorg-video-nouveau 1:0.0.15+git20100329+7858345-5
xserver-xorg-video-nv 1:2.1.17-3
xserver-xorg-video-openchrome 1:0.2.904+svn842-2
xserver-xorg-video-r128 6.8.1-3
xserver-xorg-video-radeon 1:6.13.1-2+squeeze1
xserver-xorg-video-rendition 1:4.2.3-3
xserver-xorg-video-s3 1:0.6.3-2
xserver-xorg-video-s3virge 1:1.10.4-2
xserver-xorg-video-savage 1:2.3.1-2
xserver-xorg-video-siliconmotion 1:1.7.3-2
xserver-xorg-video-sis 1:0.10.3-1
xserver-xorg-video-sisusb 1:0.9.3-2
xserver-xorg-video-tdfx 1:1.4.3-2
xserver-xorg-video-trident 1:1.3.3-2
xserver-xorg-video-tseng 1:1.2.3-2
xserver-xorg-video-vesa 1:2.3.0-3
xserver-xorg-video-vmware 1:11.0.1-2
xserver-xorg-video-voodoo 1:1.2.3-2
xsltproc 1.1.26-6
xterm 261-1
xulrunner-1.9.1 1.9.1.16-4
xz-utils 5.0.0-2
yelp 2.30.1+webkit-1
zenity 2.30.0-1
zlib1g 1:1.2.3.4.dfsg-3 AppRecommender-0.7.5/bin/example/popcon_ubuntu1010t 0000664 0000000 0000000 00000223363 13067513116 0022154 0 ustar 00root root 0000000 0000000 POPULARITY-CONTEST-0 TIME:1312866727 ID:18c40ad842f4407ab153d9207e475498 ARCH:i386 POPCONVER:1.48ubuntu1
1312866727 1300530225 xserver-xorg-input-evdev /usr/lib/xorg/modules/input/evdev_drv.so
1312866727 1300530227 xserver-xorg-input-vmmouse /usr/lib/xorg/modules/input/vmmouse_drv.so
1312866727 1300530206 xserver-xorg-video-fbdev /usr/lib/xorg/modules/drivers/fbdev_drv.so
1312866727 1300530200 xserver-xorg-core /usr/bin/Xorg
1312866727 1300530223 xserver-xorg-video-vmware /usr/lib/xorg/modules/drivers/vmwlegacy_drv.so
1312866727 1300530221 xserver-xorg-video-vesa /usr/lib/xorg/modules/drivers/vesa_drv.so
1312866727 1300531937 wpasupplicant /sbin/wpa_supplicant
1312866727 1300528870 zlib1g /lib/libz.so.1.2.3.4
1312866726 1300528867 upstart /sbin/upstart-udev-bridge
1312866726 1300528866 udev /sbin/udevd
1312866726 1300531479 upower /usr/lib/upower/upowerd
1312866726 1300531882 ubuntuone-client-gnome /usr/lib/nautilus/extensions-2.0/libnautilus-ubuntuone.so
1312866726 1300530687 udisks /usr/lib/udisks/udisks-daemon
1312866726 1300528869 util-linux /sbin/getty
1312866725 1300529904 ttf-dejavu-core /usr/share/fonts/truetype/ttf-dejavu/DejaVuSansMono-Bold.ttf
1312866724 1300532122 totem /usr/lib/nautilus/extensions-2.0/libtotem-properties-page.so
1312866723 1300528854 python2.6-minimal /usr/bin/python2.6
1312866723 1300532061 rtkit /usr/lib/rtkit/rtkit-daemon
1312866723 1300529007 rsyslog /usr/lib/rsyslog/lmnet.so
1312866723 1300529006 python2.6 /usr/lib/python2.6/lib-dynload/pyexpat.so
1312866722 1300530633 python-gtk2 /usr/lib/pyshared/python2.6/gtk-2.0/pangocairo.so
1312866722 1300529924 python-gobject /usr/lib/pyshared/python2.6/gtk-2.0/gi/_gi.so
1312866722 1300531684 python-notify /usr/lib/pyshared/python2.6/gtk-2.0/pynotify/_pynotify.so
1312866721 1300530662 policykit-1 /usr/lib/policykit-1/polkitd
1312866721 1300532025 pulseaudio-module-x11 /usr/lib/pulse-0.9.21/modules/module-x11-xsmp.so
1312866721 1300532024 pulseaudio-module-gconf /usr/lib/pulse-0.9.21/modules/module-gconf.so
1312866721 1300530680 policykit-1-gnome /usr/lib/policykit-1-gnome/polkit-gnome-authentication-agent-1
1312866721 1300529923 python-cairo /usr/lib/pyshared/python2.6/cairo/_cairo.so
1312866721 1300532023 pulseaudio-esound-compat /usr/lib/pulse-0.9.21/modules/module-esound-protocol-unix.so
1312866721 1300532234 pulseaudio-module-bluetooth /usr/lib/pulse-0.9.21/modules/libbluetooth-util.so
1312866721 1300531787 pulseaudio /usr/lib/libpulsecore-0.9.21.so
1312866721 1300529909 python-dbus /usr/lib/pyshared/python2.6/_dbus_bindings.so
1312866721 1300532030 python-cups /usr/lib/pyshared/python2.6/cups.so
1312866720 1300528850 perl-base /usr/bin/perl
1312866720 1300532801 openssh-client /usr/bin/ssh-agent
1312866719 1300531940 network-manager /usr/lib/NetworkManager/libnm-settings-plugin-ifupdown.so
1312866719 1300531924 nautilus-sendto /usr/lib/nautilus/extensions-2.0/libnautilus-sendto.so
1312866719 1300531935 nautilus-share /usr/lib/nautilus/extensions-2.0/libnautilus-share.so
1312866719 1300528845 mountall /sbin/mountall
1312866719 1300531943 network-manager-gnome /usr/bin/nm-applet
1312866719 1300528837 login /bin/su
1312866719 1300531579 nautilus /usr/bin/nautilus
1312866719 1300531919 modemmanager /usr/lib/ModemManager/libmm-plugin-simtech.so
1312866719 1300531574 metacity /usr/bin/metacity
1312866710 1300530548 libvorbis0a /usr/lib/libvorbis.so.0.4.4
1312866710 1300530626 libvorbisenc2 /usr/lib/libvorbisenc.so.2.0.7
1312866710 1300530536 libxcursor1 /usr/lib/libXcursor.so.1.0.2
1312866710 1300529920 libx11-6 /usr/lib/libX11.so.6.3.0
1312866710 1300530586 libwnck22 /usr/lib/libwnck-1.so.22.3.30
1312866710 1300530629 libxtst6 /usr/lib/libXtst.so.6.1.0
1312866710 1300530172 libxkbfile1 /usr/lib/libxkbfile.so.1.0.2
1312866710 1300530549 libvorbisfile3 /usr/lib/libvorbisfile.so.3.3.2
1312866710 1300529921 libxcb-shm0 /usr/lib/libxcb-shm.so.0.0.0
1312866710 1300529915 libxau6 /usr/lib/libXau.so.6.0.0
1312866710 1300530535 libxcomposite1 /usr/lib/libXcomposite.so.1.0.0
1312866710 1300530188 libxfont1 /usr/lib/libXfont.so.1.4.1
1312866710 1300531130 libvte-common /usr/share/vte/termcap/xterm
1312866710 1300530617 libxklavier16 /usr/lib/libxklavier.so.16.0.0
1312866710 1300530565 libxcb-event1 /usr/lib/libxcb-event.so.1.0.0
1312866710 1300530622 libxss1 /usr/lib/libXss.so.1.0.0
1312866710 1300530537 libxinerama1 /usr/lib/libXinerama.so.1.0.0
1312866710 1300530628 libwrap0 /lib/libwrap.so.0.7.6
1312866710 1300529916 libxcb1 /usr/lib/libxcb.so.1.1.0
1312866710 1300530187 libxfixes3 /usr/lib/libXfixes.so.3.1.0
1312866710 1300530584 libxres1 /usr/lib/libXRes.so.1.0.0
1312866710 1300531131 libvte9 /usr/lib/libvte9/gnome-pty-helper
1312866710 1300530508 libxml2 /usr/lib/libxml2.so.2.7.7
1312866710 1300529916 libxdmcp6 /usr/lib/libXdmcp.so.6.0.0
1312866710 1300530189 libxi6 /usr/lib/libXi.so.6.1.0
1312866710 1300530537 libxdamage1 /usr/lib/libXdamage.so.1.1.0
1312866710 1300529922 libxrender1 /usr/lib/libXrender.so.1.3.0
1312866710 1300530167 libxext6 /usr/lib/libXext.so.6.4.0
1312866710 1300530209 libxcb-aux0 /usr/lib/libxcb-aux.so.0.0.0
1312866710 1300530538 libxrandr2 /usr/lib/libXrandr.so.2.2.0
1312866710 1300529920 libxcb-render0 /usr/lib/libxcb-render.so.0.0.0
1312866710 1300530186 libx11-xcb1 /usr/lib/libX11-xcb.so.1.0.0
1312866710 1300530565 libxcb-atom1 /usr/lib/libxcb-atom.so.1.0.0
1312866710 1300530963 libxxf86vm1 /usr/lib/libXxf86vm.so.1.0.0
1312866709 1300531172 libslp1 /usr/lib/libslp.so.1.0.1
1312866709 1300530511 libtasn1-3 /usr/lib/libtasn1.so.3.1.9
1312866709 1300531763 libspeexdsp1 /usr/lib/sse2/libspeexdsp.so.1.5.0
1312866709 1300530584 libpolkit-gobject-1-0 /usr/lib/libpolkit-gobject-1.so.0.0.0
1312866709 1300530581 libnotify1 /usr/lib/libnotify.so.1.2.3
1312866709 1300530535 libpango1.0-0 /usr/lib/pango/1.6.0/modules/pango-basic-fc.so
1312866709 1300528839 libudev0 /lib/libudev.so.0.9.1
1312866709 1300530630 libpulse-mainloop-glib0 /usr/lib/libpulse-mainloop-glib.so.0.0.4
1312866709 1300531169 libpaper1 /usr/lib/libpaper.so.1.1.2
1312866709 1300531473 libupower-glib1 /usr/lib/libupower-glib.so.1.0.1
1312866709 1300530608 librsvg2-2 /usr/lib/librsvg-2.so.2.32.0
1312866709 1300530630 libpulse0 /usr/lib/libpulsecommon-0.9.21.so
1312866709 1300530639 librsvg2-common /usr/lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-svg.so
1312866709 1300530575 libsoup2.4-1 /usr/lib/libsoup-2.4.so.1.3.0
1312866709 1300529914 libpixman-1-0 /usr/lib/libpixman-1.so.0.18.4
1312866709 1300530169 libsm6 /usr/lib/libSM.so.6.0.1
1312866709 1300528836 libplymouth2 /lib/libply-boot-client.so.2.0.0
1312866709 1300528834 libpam-modules /lib/security/pam_unix.so
1312866709 1300530517 libsasl2-2 /usr/lib/libsasl2.so.2.0.23
1312866709 1300531880 libsyncdaemon-1.0-1 /usr/lib/libsyncdaemon-1.0.so.1.0.0
1312866709 1300530566 libstartup-notification0 /usr/lib/libstartup-notification-1.so.0.0.0
1312866709 1300530582 libpanel-applet2-0 /usr/lib/libpanel-applet-2.so.0.2.68
1312866709 1300530678 libpolkit-agent-1-0 /usr/lib/libpolkit-agent-1.so.0.0.0
1312866709 1300531328 libnm-util1 /usr/lib/libnm-util.so.1.6.0
1312866709 1300531303 libtelepathy-glib0 /usr/lib/libtelepathy-glib.so.0.45.0
1312866709 1300528829 libncurses5 /lib/libncurses.so.5.7
1312866709 1300531591 libpangomm-1.4-1 /usr/lib/libpangomm-1.4.so.1.0.30
1312866709 1300531850 libpcsclite1 /lib/libpcsclite.so.1.0.0
1312866709 1300530627 libsndfile1 /usr/lib/libsndfile.so.1.0.21
1312866709 1300528826 libpng12-0 /lib/libpng12.so.0.44.0
1312866709 1300530657 libpolkit-backend-1-0 /usr/lib/libpolkit-backend-1.so.0.0.0
1312866709 1300531847 libpam-ck-connector /lib/security/pam_ck_connector.so
1312866709 1300528997 libsqlite3-0 /usr/lib/libsqlite3.so.0.8.6
1312866709 1300531762 libsamplerate0 /usr/lib/libsamplerate.so.0.1.7
1312866709 1300530598 libnspr4-0d /usr/lib/libplds4.so
1312866709 1300530182 libpciaccess0 /usr/lib/libpciaccess.so.0.10.8
1312866709 1300530185 libutouch-grail1 /usr/lib/libutouch-grail.so.1.0.0
1312866709 1300528825 libnih1 /lib/libnih.so.1.0.0
1312866709 1300530185 libtalloc2 /usr/lib/libtalloc.so.2.0.1
1312866709 1300529852 libsigc++-2.0-0c2a /usr/lib/libsigc-2.0.so.0.0.0
1312866709 1300528996 libpopt0 /lib/libpopt.so.0.0.0
1312866709 1300530602 libnss3-1d /usr/lib/libnss3.so
1312866709 1300530547 libogg0 /usr/lib/libogg.so.0.7.0
1312866709 1300530574 libproxy0 /usr/lib/libproxy.so.0.0.0
1312866709 1300528835 libpam0g /lib/libpam_misc.so.0.82.0
1312866709 1300528841 libuuid1 /lib/libuuid.so.1.3.0
1312866709 1300531474 libplist1 /usr/lib/libplist.so.1.1.3
1312866709 1300528826 libselinux1 /lib/libselinux.so.1
1312866709 1300528835 libpcre3 /lib/libpcre.so.3.12.1
1312866709 1300531476 libusb-1.0-0 /lib/libusb-1.0.so.0.0.0
1312866709 1300528987 libstdc++6 /usr/lib/libstdc++.so.6.0.14
1312866709 1300530621 libunique-1.0-0 /usr/lib/libunique-1.0.so.0.100.6
1312866709 1300528824 libnih-dbus1 /lib/libnih-dbus.so.1.0.0
1312866709 1300531848 libpam-gnome-keyring /lib/security/pam_gnome_keyring.so
1312866709 1300531212 libtotem-plparser17 /usr/lib/libtotem-plparser.so.17.0.0
1312866709 1300531160 libnl1 /usr/lib/libnl.so.1.1
1312866709 1300530507 liborbit2 /usr/lib/libORBitCosNaming-2.so.0.1.0
1312866709 1300528831 libssl0.9.8 /lib/libcrypto.so.0.9.8
1312866709 1300530548 libtdb1 /usr/lib/libtdb.so.1.2.1
1312866709 1300530576 libsoup-gnome2.4-1 /usr/lib/libsoup-gnome-2.4.so.1.3.0
1312866709 1300531329 libnm-glib2 /usr/lib/libnm-glib.so.2.4.1
1312866709 1300531475 libusbmuxd1 /usr/lib/libusbmuxd.so.1.0.4
1312866708 1300531652 libgphoto2-2 /usr/lib/libgphoto2.so.2.4.0
1312866708 1300530614 libindicator1 /usr/lib/libindicator.so.1.0.0
1312866708 1300531593 libgtkmm-2.4-1c2a /usr/lib/libgtkmm-2.4.so.1.1.0
1312866708 1300530546 libltdl7 /usr/lib/libltdl.so.7.2.1
1312866708 1300530615 libjson-glib-1.0-0 /usr/lib/libjson-glib-1.0.so.0.1000.2
1312866708 1300530168 libice6 /usr/lib/libICE.so.6.3.0
1312866708 1300530516 libgssapi-krb5-2 /usr/lib/libgssapi_krb5.so.2.2
1312866708 1300530513 libkrb5support0 /usr/lib/libkrb5support.so.0.1
1312866708 1300530517 libldap-2.4-2 /usr/lib/libldap_r-2.4.so.2.5.6
1312866708 1300530757 libgstreamer-plugins-base0.10-0 /usr/lib/libgstvideo-0.10.so.0.21.0
1312866708 1300531742 libgwibber0 /usr/lib/libgwibber.so.0.0.0
1312866708 1300530674 libgnomeui-0 /usr/lib/libgnomeui-2.so.0.2400.4
1312866708 1300531220 libnautilus-extension1 /usr/lib/libnautilus-extension.so.1.2.0
1312866708 1300530558 libgnomevfs2-0 /usr/lib/libgnomevfs-2.so.0.2400.3
1312866708 1300531649 libgphoto2-port0 /usr/lib/libgphoto2_port.so.0.8.0
1312866708 1300531326 libindicate4 /usr/lib/libindicate.so.4.0.3
1312866708 1300530179 libgpg-error0 /lib/libgpg-error.so.0.4.0
1312866708 1300530562 libgnomecanvas2-0 /usr/lib/libgnomecanvas-2.so.0.3000.2
1312866708 1300530542 libgtk2.0-0 /usr/lib/libgtk-x11-2.0.so.0.2200.0
1312866708 1300530513 libkeyutils1 /lib/libkeyutils.so.1.3
1312866708 1300531478 libimobiledevice1 /usr/lib/libimobiledevice.so.1.0.1
1312866708 1300530184 libmtdev1 /usr/lib/libmtdev.so.1.0.0
1312866708 1300531724 libibus2 /usr/lib/libibus.so.2.0.0
1312866708 1300530569 liblaunchpad-integration1 /usr/lib/liblaunchpad-integration.so.1.0.0
1312866708 1300530618 libgnomekbd4 /usr/lib/libgnomekbdui.so.4.2.0
1312866708 1300530581 libgweather1 /usr/lib/libgweather.so.1.6.10
1312866708 1300530692 libgp11-0 /usr/lib/libgp11.so.0.0.0
1312866708 1300530514 libk5crypto3 /usr/lib/libk5crypto.so.3.1
1312866708 1300530515 libkrb5-3 /usr/lib/libkrb5.so.3.3
1312866708 1300530689 libgvfscommon0 /usr/lib/libgvfscommon.so.0.0.0
1312866708 1300530599 libical0 /usr/lib/libicalvcal.so.0.44.0
1312866708 1300530756 libgstreamer0.10-0 /usr/lib/libgstreamer-0.10.so.0.26.0
1312866708 1300531743 libido-0.1-0 /usr/lib/libido-0.1.so.0.0.0
1312866708 1300530683 libgudev-1.0-0 /usr/lib/libgudev-1.0.so.0.0.1
1312866708 1300530525 libjpeg62 /usr/lib/libjpeg.so.62.0.0
1312866708 1300530512 libgnutls26 /usr/lib/libgnutls.so.26.14.12
1312866707 1300530560 libgnome2-0 /usr/lib/libgnome-2.so.0.3200.0
1312866707 1300529908 libdbus-glib-1-2 /usr/lib/libdbus-glib-1.so.2.1.0
1312866707 1300530242 libgl1-mesa-dri /usr/lib/dri/swrast_dri.so
1312866707 1300530613 libdbusmenu-gtk1 /usr/lib/libdbusmenu-gtk.so.1.0.17
1312866707 1300531300 libgee2 /usr/lib/libgee.so.2.0.0
1312866707 1300530561 libgail18 /usr/lib/libgailutil.so.18.0.1
1312866707 1300530607 libcroco3 /usr/lib/libcroco-0.6.so.3.0.1
1312866707 1300529913 libfreetype6 /usr/lib/libfreetype.so.6.6.0
1312866707 1300531783 libcanberra-gtk-module /usr/lib/gtk-2.0/modules/libcanberra-gtk-module.so
1312866707 1300528809 libc6 /lib/libpthread-2.12.1.so
1312866707 1300530612 libdbusmenu-glib1 /usr/lib/libdbusmenu-glib.so.1.0.17
1312866707 1300528812 libglib2.0-0 /lib/libglib-2.0.so.0.2600.0
1312866707 1300528994 libcap2 /lib/libcap.so.2.19
1312866707 1300530598 libedataserver1.2-13 /usr/lib/libedataserver-1.2.so.13.0.1
1312866707 1300531211 libgmime-2.4-2 /usr/lib/libgmime-2.4.so.2.4.14
1312866707 1300530479 libfuse2 /lib/libfuse.so.2.8.4
1312866707 1300531788 libcanberra-pulse /usr/lib/libcanberra-0.25/libcanberra-pulse.so
1312866707 1300530520 libgconf2-4 /usr/lib/libgconf-2.so.4.1.5
1312866707 1300530550 libcanberra0 /usr/lib/libcanberra.so.0.2.4
1312866707 1300530597 libcanberra-gtk0 /usr/lib/libcanberra-gtk.so.0.1.6
1312866707 1300530606 libgnome-menu2 /usr/lib/libgnome-menu.so.2.4.9
1312866707 1300528824 libdrm2 /lib/libdrm.so.2.4.0
1312866707 1300528811 libgcc1 /lib/libgcc_s.so.1
1312866707 1300531522 libgdu-gtk0 /usr/lib/libgdu-gtk.so.0.0.0
1312866707 1300529906 libgirepository1.0-1 /usr/lib/libgirepository-1.0.so.1.0.0
1312866707 1300530183 libfontenc1 /usr/lib/libfontenc.so.1.0.0
1312866707 1300528802 libcomerr2 /lib/libcom_err.so.2.1
1312866707 1300529905 libffi5 /usr/lib/libffi.so.5.0.10
1312866707 1300530524 libcups2 /usr/lib/libcups.so.2
1312866707 1300530625 libflac8 /usr/lib/libFLAC.so.8.2.0
1312866707 1300530180 libgcrypt11 /lib/libgcrypt.so.11.5.3
1312866707 1300528984 libexpat1 /lib/libexpat.so.1.5.2
1312866707 1300530528 libgdk-pixbuf2.0-0 /usr/lib/libgdk_pixbuf_xlib-2.0.so.0.2200.0
1312866707 1300530688 libgdu0 /usr/lib/libgdu.so.0.0.0
1312866707 1300529828 libgdbm3 /usr/lib/libgdbm.so.3.0.0
1312866707 1300531166 libcupsmime1 /usr/lib/libcupsmime.so.1
1312866707 1300531146 libdaemon0 /usr/lib/libdaemon.so.0.5.0
1312866707 1300530604 libebook1.2-9 /usr/lib/libebook-1.2.so.9.3.1
1312866707 1300530605 libedataserverui1.2-8 /usr/lib/libedataserverui-1.2.so.8.1.1
1312866707 1300531361 libevdocument3 /usr/lib/libevdocument.so.3.0.0
1312866707 1300531343 libexif12 /usr/lib/libexif.so.12.3.1
1312866707 1300530573 libgnome-keyring0 /usr/lib/libgnome-keyring.so.0.1.1
1312866707 1300529923 libcairo2 /usr/lib/libcairo.so.2.11000.0
1312866707 1300531588 libglibmm-2.4-1c2a /usr/lib/libglibmm-2.4.so.1.3.0
1312866707 1300529914 libfontconfig1 /usr/lib/libfontconfig.so.1.4.4
1312866707 1300530600 libecal1.2-7 /usr/lib/libecal-1.2.so.7.2.2
1312866707 1300530658 libck-connector0 /usr/lib/libck-connector.so.0.0.0
1312866707 1300530603 libcamel1.2-14 /usr/lib/libcamel-1.2.so.14.0.1
1312866707 1300528794 libdbus-1-3 /lib/libdbus-1.so.3.5.2
1312866707 1300530567 libgnome-desktop-2-17 /usr/lib/libgnome-desktop-2.so.17.1.4
1312866707 1300530542 libglade2-0 /usr/lib/libglade-2.0.so.0.0.7
1312866707 1300531590 libcairomm-1.0-1 /usr/lib/libcairomm-1.0.so.1.4.0
1312866707 1300530583 libeggdbus-1-0 /usr/lib/libeggdbus-1.so.0.0.0
1312866707 1300531342 libexempi3 /usr/lib/libexempi.so.3.2.1
1312866706 1300531741 indicator-application /usr/lib/indicators/4/libapplication.so
1312866706 1300530545 libasound2 /usr/lib/libasound.so.2.0.0
1312866706 1300528785 libattr1 /lib/libattr.so.1.1.0
1312866706 1300533901 language-pack-gnome-en-base /usr/share/locale-langpack/en/LC_MESSAGES/yelp.mo
1312866706 1300531148 libavahi-core7 /usr/lib/libavahi-core.so.7.0.0
1312866706 1300530615 libappindicator1 /usr/lib/libappindicator.so.1.0.0
1312866706 1300530510 libart-2.0-2 /usr/lib/libart_lgpl_2.so.2.3.21
1312866706 1300531219 libbrasero-media1 /usr/lib/libbrasero-media.so.1.2.0
1312866706 1300531745 indicator-messages /usr/lib/indicator-messages/indicator-messages-service
1312866706 1300530564 libbonoboui2-0 /usr/lib/libbonoboui-2.so.0.0.0
1312866706 1300530523 libavahi-common3 /usr/lib/libavahi-common.so.3.5.2
1312866706 1300530509 libbonobo2-0 /usr/lib/bonobo-activation/bonobo-activation-server
1312866706 1300530682 libatasmart4 /lib/libatasmart.so.4.0.3
1312866706 1300530551 libavahi-glib1 /usr/lib/libavahi-glib.so.1.0.2
1312866706 1300531521 libavahi-ui0 /usr/lib/libavahi-ui.so.0.1.2
1312866706 1300530523 libavahi-client3 /usr/lib/libavahi-client.so.3.2.7
1312866706 1300532225 indicator-sound /usr/lib/indicators/4/libsoundmenu.so
1312866706 1300531739 indicator-applet-session /usr/lib/indicator-applet/indicator-applet-session
1312866706 1300531739 indicator-applet /usr/lib/indicator-applet/indicator-applet
1312866706 1300530497 libatk1.0-0 /usr/lib/libatk-1.0.so.0.3209.1
1312866706 1300531728 ibus-gtk /usr/lib/gtk-2.0/2.10.0/immodules/im-ibus.so
1312866706 1300531746 indicator-session /usr/lib/indicators/4/libsession.so
1312866706 1300528790 libbz2-1.0 /lib/libbz2.so.1.0.4
1312866706 1300531744 indicator-me /usr/lib/indicator-me/indicator-me-service
1312866705 1300531496 gtk2-engines /usr/lib/gtk-2.0/2.10.0/engines/libclearlooks.so
1312866705 1300530690 gvfs /usr/lib/gvfs/gvfs-gdu-volume-monitor
1312866705 1300531567 gnome-screensaver /usr/bin/gnome-screensaver
1312866705 1300530632 gnome-settings-daemon /usr/lib/gnome-settings-daemon-2.0/libxsettings.so
1312866705 1300531598 gnome-terminal /usr/bin/gnome-terminal
1312866705 1300531670 gvfs-backends /usr/lib/gvfs/gvfs-gphoto2-volume-monitor
1312866705 1300531481 gnome-session-bin /usr/bin/gnome-session
1312866705 1300531671 gvfs-fuse /usr/lib/gvfs/gvfs-fuse-daemon
1312866704 1300531483 gdm /usr/sbin/gdm-binary
1312866704 1300530691 gnome-applets /usr/lib/gnome-applets/trashapplet
1312866704 1300529907 gir1.0-glib-2.0 /usr/lib/girepository-1.0/GLib-2.0.typelib
1312866704 1300530695 gnome-keyring /usr/bin/gnome-keyring-daemon
1312866704 1300530682 gnome-panel /usr/lib/gnome-panel/wnck-applet
1312866704 1300531564 gnome-power-manager /usr/bin/gnome-power-manager
1312866704 1300531525 gnome-disk-utility /usr/lib/nautilus/extensions-2.0/libnautilus-gdu.so
1312866703 1300531408 file-roller /usr/lib/nautilus/extensions-2.0/libnautilus-fileroller.so
1312866702 1300531366 evince /usr/lib/nautilus/extensions-2.0/libevince-properties-page.so
1312866702 1300528982 dhcp3-client /sbin/dhclient3
1312866701 1300530519 dbus /bin/dbus-daemon
1312866701 1300528981 cron /usr/sbin/cron
1312866701 1300530554 dbus-x11 /usr/bin/dbus-launch
1312866701 1300530660 consolekit /usr/sbin/console-kit-daemon
1312866701 1300531199 cups /usr/sbin/cupsd
1312866700 1300531149 avahi-daemon /usr/sbin/avahi-daemon
1312866700 1300530991 at /usr/sbin/atd
1312866700 1300531227 brasero /usr/lib/nautilus/extensions-2.0/libnautilus-brasero-extension.so
1312866700 1300528789 bash /bin/bash
1312866699 1300531079 anacron /usr/sbin/anacron
1312866699 1300531064 acpid /usr/sbin/acpid
1312866698 1300528800 dpkg /usr/bin/dpkg-query
1312866698 1300531045 popularity-contest /usr/sbin/popularity-contest
1312866669 1300528792 coreutils /usr/bin/groups
1312866662 1300529008 sudo /usr/bin/sudoedit
1312866642 1300528845 ncurses-base /lib/terminfo/x/xterm
1312866641 1300528991 less /usr/bin/lesspipe
1312866607 1300532099 system-config-printer-gnome /usr/bin/system-config-printer-applet
1312866590 1300531409 firefox-branding /usr/lib/firefox-3.6.10/icons/mozicon128.png
1312866589 1300530966 x11-utils /usr/bin/xprop
1312866588 1300530596 gnome-applets-data /usr/lib/bonobo/servers/GNOME_MailcheckApplet_Factory.server
1312866588 1300531921 mousetweaks /usr/lib/bonobo/servers/DwellClick_Factory.server
1312866587 1300532210 vinagre /usr/lib/bonobo/servers/GNOME_VinagreApplet.server
1312866587 1300532129 tsclient /usr/lib/bonobo/servers/GNOME_TSClientApplet.server
1312866587 1300530610 gnome-panel-data /usr/lib/bonobo/servers/GNOME_Wncklet_Factory.server
1312866587 1300530498 libbonobo2-common /usr/lib/bonobo/servers/Bonobo_CosNaming_NamingContext.server
1312866587 1300531145 at-spi /usr/lib/bonobo/servers/Accessibility_Registry.server
1312866586 1300532117 tomboy /usr/lib/bonobo/servers/GNOME_TomboyApplet.server
1312866586 1300531528 gnome-mag /usr/lib/bonobo/servers/GNOME_Magnifier.server
1312866585 1300530226 xserver-xorg-input-synaptics /usr/bin/syndaemon
1312866585 1300531520 gnome-dictionary /usr/lib/bonobo/servers/GNOME_DictionaryApplet.server
1312866585 1300531785 pulseaudio-utils /usr/bin/pactl
1312866581 1300531241 compiz-core /usr/bin/compiz
1312866581 1300531032 pciutils /usr/bin/lspci
1312866581 1300530964 libgl1-mesa-glx /usr/lib/mesa/libGL.so.1
1312866576 1300530479 fuse-utils /bin/fusermount
1312866576 1300531581 gnome-session /usr/bin/gnome-wm
1312866576 1300532184 xdg-user-dirs-gtk /usr/bin/xdg-user-dirs-gtk-update
1312866575 1300530555 gconf2 /usr/bin/gsettings-data-convert
1312866573 1300531063 x11-xserver-utils /usr/bin/xhost
1312866573 1300532183 xdg-user-dirs /usr/bin/xdg-user-dirs-update
1312866538 1300530178 x11-xkb-utils /usr/bin/xkbcomp
1312866537 1300532021 policykit-desktop-privileges /var/lib/polkit-1/localauthority/10-vendor.d/com.ubuntu.desktop.pkla
1312866536 1300531065 pm-utils /usr/bin/pm-is-supported
1312866534 1300531601 gtk2-engines-murrine /usr/lib/gtk-2.0/2.10.0/engines/libmurrine.so
1312866534 1300531599 gtk2-engines-pixbuf /usr/lib/gtk-2.0/2.10.0/engines/libpixmap.so
1312866531 1300528804 libc-bin /usr/bin/getent
1312866525 1300530230 xserver-xorg /usr/bin/X
1312866525 1300530197 xserver-common /usr/lib/xorg/protocol.txt
1312866521 1300528852 procps /usr/bin/pkill
1312866520 1300528844 module-init-tools /sbin/lsmod
1312866518 1300529001 ntpdate /usr/sbin/ntpdate
1312866518 1300528858 sysv-rc /usr/sbin/invoke-rc.d
1312866518 1300528992 lockfile-progs /usr/bin/lockfile-touch
1312866515 1300534256 grub-common /usr/bin/grub-editenv
1312866514 1300531016 hdparm /usr/lib/pm-utils/power.d/95hdparm-apm
1312866513 1300528858 sysvinit-utils /bin/pidof
1312866513 1300529260 linux-image-2.6.35-22-generic /lib/modules/2.6.35-22-generic/kernel/fs/binfmt_misc.ko
1312866513 1300531001 bind9-host /usr/bin/host
1312866513 1300528847 net-tools /sbin/route
1312866513 1300530838 openssl /usr/lib/ssl/openssl.cnf
1312866512 1300528989 iproute /sbin/ip
1312866512 1300528813 gzip /bin/gzip
1312866511 1300528797 debianutils /usr/bin/savelog
1312866511 1300531021 irqbalance /usr/sbin/irqbalance
1312866510 1300531044 plymouth-theme-ubuntu-text /lib/plymouth/themes/ubuntu-text/ubuntu-text.plymouth
1312866510 1300531765 libasound2-plugins /usr/lib/alsa-lib/libasound_module_conf_pulse.so
1312866510 1300528851 plymouth /lib/plymouth/renderers/vga16fb.so
1312866510 1300531078 alsa-utils /lib/udev/alsa-utils
1312866510 1300528998 lsb-release /usr/bin/lsb_release
1312866510 1300532019 plymouth-theme-ubuntu-logo /lib/plymouth/themes/ubuntu-logo/ubuntu-logo.plymouth
1312866509 1300528839 initscripts /lib/init/vars.sh
1312866509 1300532231 lm-sensors /usr/bin/sensors
1312866508 1300531059 update-manager-core /usr/lib/update-manager/check-new-release
1312866508 1300531108 python-lazr.restfulclient /usr/lib/python2.6/dist-packages/lazr.restfulclient-0.9.20-nspkg.pth
1312866508 1300529845 python-apt /usr/lib/python2.6/dist-packages/apt_pkg.so
1312866508 1300531676 python-imaging /usr/lib/python2.6/dist-packages/PIL.pth
1312866508 1300528854 python-minimal /usr/bin/python
1312866508 1300531938 update-notifier-common /usr/lib/update-notifier/update-motd-reboot-required
1312866508 1300528810 findutils /usr/bin/xargs
1312866508 1300711086 python-support /usr/lib/python2.6/dist-packages/python-support.pth
1312866508 1300528846 ncurses-bin /usr/bin/tput
1312866508 1300531513 python-gst0.10 /usr/lib/python2.6/dist-packages/pygst.pth
1312866508 1300531104 python-lazr.uri /usr/lib/python2.6/dist-packages/lazr.uri-1.0.2-nspkg.pth
1312866507 1300528822 lsb-base /lib/lsb/init-functions
1312866507 1300528991 kbd /bin/kbd_mode
1312866506 1300528855 sed /bin/sed
1312866506 1300531046 powermgmt-base /usr/bin/on_ac_power
1312866505 1300528811 gcc-4.4 /usr/lib/gcc/i686-linux-gnu/4.4/libgcc_s.so
1312866505 1300528813 grep /bin/egrep
1312866505 1300532101 system-config-printer-udev /lib/udev/udev-configure-printer
1312866505 1300528979 apt /usr/bin/apt-config
1312866505 1300528814 ifupdown /sbin/ifup
1312866504 1300528981 console-setup /lib/udev/console-setup-tty
1312866504 1300528843 mawk /usr/bin/mawk
1312866503 1300531274 cpu-checker /usr/bin/check-bios-nx
1312866502 1300528793 dash /bin/sh
1312866502 1300530977 apparmor /sbin/apparmor_parser
1312866502 1300531058 ufw /lib/ufw/ufw-init-functions
1312866501 1300528868 mount /bin/mount
1312866501 1300531075 linux-sound-base /lib/linux-sound-base/noOSS.modprobe.conf
1300873831 1300531885 libutempter0 /usr/lib/utempter/utempter
1300873829 1300532199 xterm /usr/bin/xterm
1300871547 1300531038 mlocate /usr/bin/updatedb.mlocate
1300871526 1300532430 man-db /usr/lib/man-db/mandb
1300871495 1300528993 logrotate /usr/sbin/logrotate
1300870738 1300530975 wget /usr/bin/wget
1300870341 1300528798 diffutils /usr/bin/cmp
1300713322 1300528817 klibc-utils /usr/lib/klibc/bin/resume
1300713304 1300528792 cpio /bin/cpio
1300713302 1300528993 dmsetup /sbin/dmsetup
1300713299 1300528789 busybox-initramfs /usr/lib/initramfs-tools/bin/busybox
1300713297 1300530485 ntfs-3g /sbin/mount.ntfs
1300713291 1300528802 e2fsprogs /sbin/dumpe2fs
1300713288 1300528815 initramfs-tools-bin /usr/lib/initramfs-tools/bin/wait-for-root
1300713257 1300529272 linux-firmware /lib/firmware/aic94xx-seq.fw
1300713215 1300528816 initramfs-tools /usr/sbin/mkinitramfs
1300713203 1300532013 pcmciautils /lib/udev/rules.d/85-pcmcia.rules
1300713203 1300529155 wireless-crda /lib/udev/rules.d/85-regulatory.rules
1300713203 1300531162 bluez /lib/udev/rules.d/97-bluetooth.rules
1300713203 1300531830 libgpod-common /lib/udev/rules.d/90-libgpod.rules
1300713203 1300531477 usbmuxd /lib/udev/rules.d/85-usbmuxd.rules
1300713202 1300531835 libmtp8 /lib/udev/rules.d/45-libmtp8.rules
1300713202 1300531511 gnome-bluetooth /lib/udev/rules.d/61-gnome-bluetooth-rfkill.rules
1300713202 1300532219 brltty /lib/udev/rules.d/85-brltty.rules
1300713202 1300531720 hplip /lib/udev/rules.d/56-hpmud_support.rules
1300713202 1300530228 xserver-xorg-input-wacom /lib/udev/rules.d/69-xserver-xorg-input-wacom.rules
1300713202 1300531429 foo2zjs /lib/udev/rules.d/85-hplj10xx.rules
1300713201 1300531916 media-player-info /lib/udev/rules.d/40-usb-media-players.rules
1300713201 1300530210 xserver-xorg-video-intel /lib/udev/rules.d/40-xserver-xorg-video-intel.rules
1300713201 1300531707 libsane /lib/udev/rules.d/40-libsane.rules
1300713201 1300531698 libhpmud0 /lib/udev/rules.d/40-hplip.rules
1300713201 1300528988 gnupg /lib/udev/rules.d/40-gnupg.rules
1300713201 1300532241 usb-modeswitch-data /lib/udev/rules.d/40-usb_modeswitch.rules
1300712291 1300528859 tar /bin/tar
1300712087 1300531934 samba-common-bin /usr/bin/net.samba3
1300711968 1300529010 ureadahead /sbin/ureadahead
1300711788 1300530616 libgnome-window-settings1 /usr/lib/libgnome-window-settings1/libmetacity.so
1300711780 1300530667 gnome-control-center /usr/bin/gnome-appearance-properties
1300711036 1300528797 debconf /usr/share/perl5/Debconf/Element.pm
1300711035 1300528814 hostname /bin/hostname
1300711030 1300531819 libgnome2-perl /usr/lib/perl5/Gnome2.pm
1300711030 1300531816 libgnome2-canvas-perl /usr/lib/perl5/auto/Gnome2/Canvas/Canvas.so
1300711030 1300531818 libgnome2-vfs-perl /usr/lib/perl5/auto/Gnome2/VFS/VFS.so
1300711029 1300531815 libgtk2-perl /usr/lib/perl5/Gtk2.pm
1300711029 1300531782 libcairo-perl /usr/lib/perl5/Cairo.pm
1300711029 1300531812 libpango-perl /usr/lib/perl5/Pango.pm
1300711029 1300531808 libglib-perl /usr/lib/perl5/Glib.pm
1300711017 1300530712 ure /usr/lib/ure/lib/invocadapt.uno.so
1300711005 1300530932 groff-base /usr/lib/mime/packages/groff-base
1300711005 1300532054 rhythmbox /usr/lib/mime/packages/rhythmbox
1300711005 1300529011 vim-common /usr/lib/mime/packages/vim-common
1300711005 1300528855 sensible-utils /usr/lib/mime/packages/sensible-utils
1300711005 1300531405 unzip /usr/lib/mime/packages/unzip
1300711005 1300532127 transmission-gtk /usr/lib/mime/packages/transmission-gtk
1300711005 1300528999 mime-support /usr/lib/mime/packages/mime-support
1300711005 1300531062 w3m /usr/lib/mime/packages/w3m
1300710994 1300528828 libtext-wrapi18n-perl /usr/share/perl5/Text/WrapI18N.pm
1300710994 1300529842 perl /usr/lib/perl/5.10.1/Encode/Config.pm
1300710994 1300528827 libtext-charwidth-perl /usr/lib/perl5/auto/Text/CharWidth/CharWidth.so
1300710994 1300528827 libtext-iconv-perl /usr/lib/perl5/auto/Text/Iconv/Iconv.so
1300710994 1300528824 liblocale-gettext-perl /usr/lib/perl5/auto/Locale/gettext/gettext.so
1300710068 1300530705 uno-libs3 /usr/lib/ure/lib/libuno_salhelpergcc3.so.3
1300710066 1300530544 libpython2.6 /usr/lib/python2.6/config/libpython2.6.so
1300710018 1300531944 pptp-linux /usr/sbin/pptpsetup
1300710018 1300531754 aptdaemon /usr/sbin/aptd
1300710018 1300531753 kerneloops-daemon /usr/sbin/kerneloops
1300710018 1300532105 tcpd /usr/sbin/safe_finger
1300710018 1300532064 sane-utils /usr/sbin/saned
1300710018 1300532167 cups-bsd /usr/sbin/lpc
1300710018 1300532118 toshset /usr/sbin/toshsat1800-irdasetup
1300710018 1300531951 ntfsprogs /sbin/mkfs.ntfs
1300710018 1300532049 radeontool /usr/sbin/avivotool
1300710018 1300532062 update-inetd /usr/sbin/update-inetd
1300710018 1300532089 software-center /usr/sbin/update-software-center
1300710018 1300531849 libpaper-utils /usr/sbin/paperconfig
1300710018 1300531845 system-tools-backends /usr/sbin/system-tools-backends
1300710017 1300531299 doc-base /usr/sbin/install-docs
1300710017 1300531255 computer-janitor-gtk /usr/sbin/computer-janitor-gtk
1300710017 1300531296 dnsmasq-base /usr/sbin/dnsmasq
1300710017 1300531280 cups-driver-gutenprint /usr/sbin/cups-genppdupdate
1300710017 1300531254 computer-janitor /usr/sbin/computer-janitor
1300710017 1300531188 cups-client /usr/sbin/cupsenable
1300710017 1300531229 wodim /usr/sbin/netscsid
1300710017 1300531189 ssl-cert /usr/sbin/make-ssl-cert
1300710017 1300531185 ghostscript /usr/sbin/update-gsfontmap
1300710016 1300531050 pppoeconf /usr/sbin/pppoeconf
1300710016 1300532828 uuid-runtime /usr/sbin/uuidd
1300710016 1300531134 synaptic /usr/sbin/synaptic
1300710016 1300531019 iptables /usr/sbin/ip6tables-apply
1300710016 1300530989 apparmor-utils /usr/sbin/aa-unconfined
1300710016 1300531140 dictionaries-common /usr/sbin/update-default-aspell
1300710016 1300531147 avahi-autoipd /usr/sbin/avahi-autoipd
1300710016 1300531054 tcpdump /usr/sbin/tcpdump
1300710016 1300531155 binfmt-support /usr/sbin/update-binfmts
1300710016 1300531048 ppp /usr/sbin/pppdump
1300710016 1300530976 foomatic-db-engine /usr/sbin/foomatic-extract-text
1300710016 1300531049 pppconfig /usr/sbin/pppconfig
1300710015 1300530486 xml-core /usr/sbin/update-xmlcatalog
1300710015 1300529816 dmidecode /usr/sbin/vpddecode
1300710015 1300530969 install-info /usr/sbin/update-info-dir
1300710015 1300528864 tzdata /usr/sbin/tzconfig
1300710015 1300530192 xfonts-utils /usr/sbin/update-fonts-scale
1300710015 1300528857 passwd /usr/sbin/vigr
1300710015 1300530529 defoma /usr/sbin/defoma-reconfigure
1300710015 1300529858 laptop-detect /usr/sbin/laptop-detect
1300710015 1300530485 sgml-base /usr/sbin/install-sgmlcatalog
1300710015 1300529521 usbutils /usr/sbin/update-usbids
1300710015 1300528821 locales /usr/sbin/locale-gen
1300710015 1300530753 ca-certificates /usr/sbin/update-ca-certificates
1300710015 1300530638 libgtk2.0-bin /usr/sbin/update-icon-caches
1300710015 1300530531 libpango1.0-common /usr/sbin/update-pangox-aliases
1300710015 1300529850 apt-xapian-index /usr/sbin/update-apt-xapian-index
1300710014 1300528816 insserv /usr/sbin/update-rc.d-insserv
1300710014 1300528834 libpam-runtime /usr/sbin/pam-auth-update
1300710014 1300528783 adduser /usr/sbin/delgroup
1300710014 1300532223 fancontrol /usr/sbin/pwmconfig
1300710014 1300532208 vbetool /usr/sbin/vbetool
1300710014 1300528787 base-passwd /usr/sbin/update-passwd
1300710014 1300534259 grub-pc /usr/sbin/upgrade-from-grub-legacy
1300710014 1300532242 usb-modeswitch /usr/sbin/usb_modeswitch_dispatcher
1300709887 1300532020 pnm2ppa /usr/bin/calibrate_ppa
1300709887 1300532070 seahorse /usr/bin/seahorse
1300709887 1300532177 update-notifier /usr/bin/update-notifier
1300709887 1300532095 syslinux-common /usr/bin/syslinux2ansi
1300709887 1300532170 gnome-system-tools /usr/bin/users-admin
1300709887 1300531870 xdg-utils /usr/bin/xdg-settings
1300709887 1300531959 onboard /usr/bin/onboard-settings
1300709887 1300532066 xscreensaver-data /usr/bin/xscreensaver-text
1300709887 1300532087 smbclient /usr/bin/smbtar
1300709887 1300531922 mtools /usr/bin/mxtar
1300709887 1300532097 syslinux /usr/bin/isohybrid
1300709887 1300532039 python-rdflib /usr/bin/rdfpipe
1300709887 1300532026 pxljr /usr/bin/ijs_pxljr
1300709886 1300531572 zenity /usr/bin/zenity
1300709886 1300531540 whois /usr/bin/mkpasswd
1300709886 1300531639 gstreamer0.10-plugins-base-apps /usr/bin/gst-visualise-0.10
1300709886 1300531679 python-mako /usr/bin/mako-render
1300709886 1300531687 gwibber-service /usr/bin/gwibber-service
1300709886 1300531752 jockey-gtk /usr/bin/jockey-gtk
1300709886 1300531569 gnome-screenshot /usr/bin/gnome-panel-screenshot
1300709886 1300531638 gstreamer0.10-tools /usr/bin/gst-typefind-0.10
1300709886 1300531690 gwibber /usr/bin/gwibber-error
1300709886 1300531539 gnome-media /usr/bin/gstreamer-properties
1300709885 1300531249 compiz-gnome /usr/bin/gtk-window-decorator
1300709885 1300531406 zip /usr/bin/zipnote
1300709885 1300531426 mscompress /usr/bin/msexpand
1300709885 1300531390 evolution /usr/bin/evolution-settings
1300709885 1300531353 espeak /usr/bin/espeak
1300709885 1300531262 erlang-base /usr/bin/erl
1300709885 1300531472 gdb /usr/bin/gdbtui
1300709885 1300531214 genisoimage /usr/bin/geteltorito
1300709885 1300531273 couchdb-bin /usr/bin/couchjs
1300709885 1300531238 checkbox-gtk /usr/bin/checkbox-gtk
1300709885 1300531215 dvd+rw-tools /usr/bin/dvd+rw-mediainfo
1300709885 1300531290 python-twisted-core /usr/bin/twistd
1300709885 1300531234 byobu /usr/bin/byobu-select-session
1300709885 1300531283 python-couchdb /usr/bin/couchdb-load
1300709884 1300531158 binutils /usr/bin/strip
1300709884 1300531115 apturl-common /usr/bin/apturl
1300709884 1300531141 aspell /usr/bin/aspell-import
1300709884 1300531014 gettext-base /usr/bin/envsubst
1300709884 1300530984 libwww-perl /usr/bin/lwp-mirror
1300709884 1300530993 bash-completion /usr/bin/dh_bash-completion
1300709884 1300531033 lshw /usr/bin/lshw
1300709884 1300531173 poppler-utils /usr/bin/pdftotext
1300709884 1300531052 rsync /usr/bin/rsync
1300709883 1300530962 x11-session-utils /usr/bin/xsm
1300709883 1300530961 x11-apps /usr/bin/xman
1300709883 1300530553 psmisc /usr/bin/peekfd
1300709883 1300530915 rarian-compat /usr/bin/rarian-example
1300709883 1300530933 bsdmainutils /usr/bin/ul
1300709883 1300530926 libxml2-utils /usr/bin/xmlcatalog
1300709883 1300530968 x11-xfs-utils /usr/bin/fstobdf
1300709883 1300530636 gnome-menus /usr/bin/gmenu-simple-editor
1300709882 1300529001 python-central /usr/bin/py3_compilefiles
1300709882 1300528984 file /usr/bin/file
1300709882 1300529857 aptitude /usr/bin/aptitude-run-state-bundle
1300709882 1300530181 xauth /usr/bin/xauth
1300709882 1300530183 intel-gpu-tools /usr/bin/intel_lid
1300709882 1300533996 fontconfig /usr/bin/fc-cache
1300709882 1300528870 xz-utils /usr/bin/xzless
1300709882 1300529863 tasksel /usr/bin/tasksel
1300709881 1300528840 bsdutils /usr/bin/scriptreplay
1300709880 1300532171 inputattach /usr/bin/inputattach
1300709880 1300532203 xscreensaver-gl /usr/bin/xscreensaver-gl-helper
1300709880 1300532207 usb-creator-gtk /usr/bin/usb-creator-gtk
1300709880 1300532200 xinput /usr/bin/xinput
1300709880 1300532194 xinit /usr/bin/xinit
1300709880 1300532176 update-manager /usr/bin/update-manager
1300709880 1300531205 bogofilter-bdb /usr/bin/bf_compact-bdb
1300709879 1300532071 simple-scan /usr/bin/simple-scan
1300709879 1300532051 rdesktop /usr/bin/rdesktop
1300709879 1300531956 nvidia-common /usr/bin/nvidia-detector
1300709879 1300532017 pitivi /usr/bin/pitivi
1300709878 1300531895 libxml-xpath-perl /usr/bin/xpath
1300709878 1300531908 lzma /usr/bin/unlzma
1300709878 1300531917 min12xxw /usr/bin/esc-m
1300709878 1300531907 lp-solve /usr/bin/lp_solve
1300709878 1300531879 ubuntuone-client /usr/bin/u1sdtool
1300709878 1300531906 lksctp-tools /usr/bin/withsctp