umegaya-0.13/ 0000755 0000000 0000000 00000000000 12132230762 007751 5 ustar umegaya-0.13/index.mdwn 0000644 0000000 0000000 00000001721 12132202421 011737 0 ustar Umegaya — MEtadata GAtherer using YAml
======================================
Umegaya is a gatherer of meta information about the software packaged in
Debian. This metadata is accumlated by the package maintainer in the version
control system containing the Debian souce package, in a file called
debian/upstream, and it is collected by the Umegaya web service
each time it is accessed. The Umegaya web service then aggregates information
about all the packages it knows, and outputs as tables or YAML records that can
be loaded in central information hubs like the Ultimate Debian Database.
Optionally, Umegaya will keep a local copy if the files it retrieves from the
VCS. See also [[!debwiki UpstreamMetadata]] on the Debian wiki.
Umegaya is distributed under the BOLA [[LICENSE]].
See the [[setup]] and [[man]] pages for more documentation, the [[todo]] list,
and of course the [source code](http://source.umegaya.branchable.com/?p=source.git;a=tree).
umegaya-0.13/setup.mdwn 0000644 0000000 0000000 00000006700 12132230406 011776 0 ustar Setting up Umegaya
==================
Programs needed
---------------
- Apache
- Berkeley DB
- The following Perl modules:
- LWP
- YAML
- YAML::Appconfig
YAML::Appconfig is ageing and generates deprecation warnings with recent
versions of Perl, but this has been corrected in Debian's 0.16-2 package.
Configuration
-------------
- Set up an Apache virtual host using the `apache.conf` file in
`/etc/umegaya/`.
- Set up a directory where apache can write the `umegaya.db` Berkeley DB. By
default it is `/var/lib/umegaya`.
- Make sure that the Umegaya administrator has also write access to the
directory.
- Umegaya finds the package's VCS using the `debcheckout` command of the
devscripts package, so a `deb-src` line tracking Sid is necessary for a full
coverage.
Triggering
----------
Umegaya is a passive system, that refreshes itself only when accessed. It
can be kept up to date by systems that will access it when some resources
are refreshed, for intance commit hooks.
To refresh Umegaya for each package that has been uploaded to Debian,
subscribe to the [debian-devel-changes][] mailing list and pipe its messages to
umegaya-ddc-ping(1).
[debian-devel-changes]: http://lists.debian.org/debian-devel-changes
Another way is to periodically access a list of selected packages. For
instance, the script `umegaya-refresh-blends(1)` refreshes Umegaya for all
packages in Debian Pure Blends. It is called by a daily cron job and can be
must be activated through the refresh_blends parameter in `/etc/umegaya/umegaya.conf`.
`umegaya-refresh-blends(1)` tries to guess the VCS URL packages that are not know
of debcheckout, using the layout policy of the blends.
The list of packages recommended by each blend overlap. To avoid unnecessary
requests, the default refresh delay is set to one hour. Alioth is never
queried when the last refresh was made within the refresh delay.
This (and below) assumes that the VCS name is the source package names.
This is true in theory, but in practice, a safeguard for errors may be
necessary.
To determine if a package managed with Git on Alioth contains a debian/upstream file.
for repo in /git/debian-med/*.git
do
(cd $repo ; git ls-tree master debian/ |
grep 'debian/upstream$' > /dev/null && echo "$repo")
done | sed -e 's|/git/debian-med/||' -e 's|.git$||'
Accumulating the files on the local disk
----------------------------------------
Via the _store_files_ and _debian_control_ and _debian_copyright_ options,
umegaya can keep a local copy of the upstream, control and copyright files of
the packages, in the _db_dir_ directory.
The Debian package provides a pooled directory structure in `/var/lib/umegaya`,
but it needs to be created if another path is chosen.
Pushing these files to a remote Subversion server
-------------------------------------------------
Create a passwordless key, and restrict it to run only svnserve, as [explained
in the red bean book][svn-ssh-tricks]. Check out the repository in umegaya's
_db_dir_ (--force). Commit from time to time.
[svn-ssh-tricks]: http://svnbook.red-bean.com/en/1.7/svn.serverconfig.svnserve.html#svn.serverconfig.svnserve.sshtricks
Notes about YAML
----------------
# Invalid
foo: bar:
# Valid
foo: "bar:"
# Also valid
foo: >
bar:
Also valid
foo: |
bar:
The difference between | and > is roughly the same as between the multi-line
and folded fields in Debian control files.
umegaya-0.13/debian/ 0000755 0000000 0000000 00000000000 12132231503 011165 5 ustar umegaya-0.13/debian/compat 0000644 0000000 0000000 00000000002 12074224361 012374 0 ustar 9
umegaya-0.13/debian/links 0000644 0000000 0000000 00000000163 12074224361 012241 0 ustar /etc/umegaya/robots.txt /usr/share/umegaya/htdocs/robots.txt
/etc/umegaya/apache.conf /etc/apache2/conf.d/umegaya
umegaya-0.13/debian/gbp.conf 0000644 0000000 0000000 00000000303 12074224361 012611 0 ustar # This source package is managed with git-buildpackage, but not pristine-tar.
[DEFAULT]
pristine-tar = False
[git-dch]
# Include 7 digits of the commit ID in the changelog entry.
id-length = 7
umegaya-0.13/debian/umegaya.lintian-overrides 0000644 0000000 0000000 00000000120 12074224361 016177 0 ustar # Umegaya is a recursive acronym.
umegaya: description-starts-with-package-name
umegaya-0.13/debian/dirs 0000644 0000000 0000000 00000002440 12074224361 012062 0 ustar var/lib/umegaya
var/lib/umegaya/packages-metadata/a
var/lib/umegaya/packages-metadata/b
var/lib/umegaya/packages-metadata/c
var/lib/umegaya/packages-metadata/d
var/lib/umegaya/packages-metadata/e
var/lib/umegaya/packages-metadata/f
var/lib/umegaya/packages-metadata/g
var/lib/umegaya/packages-metadata/h
var/lib/umegaya/packages-metadata/i
var/lib/umegaya/packages-metadata/j
var/lib/umegaya/packages-metadata/k
var/lib/umegaya/packages-metadata/l
var/lib/umegaya/packages-metadata/m
var/lib/umegaya/packages-metadata/n
var/lib/umegaya/packages-metadata/o
var/lib/umegaya/packages-metadata/p
var/lib/umegaya/packages-metadata/q
var/lib/umegaya/packages-metadata/r
var/lib/umegaya/packages-metadata/s
var/lib/umegaya/packages-metadata/t
var/lib/umegaya/packages-metadata/u
var/lib/umegaya/packages-metadata/v
var/lib/umegaya/packages-metadata/w
var/lib/umegaya/packages-metadata/x
var/lib/umegaya/packages-metadata/y
var/lib/umegaya/packages-metadata/z
var/lib/umegaya/packages-metadata/0
var/lib/umegaya/packages-metadata/1
var/lib/umegaya/packages-metadata/2
var/lib/umegaya/packages-metadata/3
var/lib/umegaya/packages-metadata/4
var/lib/umegaya/packages-metadata/5
var/lib/umegaya/packages-metadata/6
var/lib/umegaya/packages-metadata/7
var/lib/umegaya/packages-metadata/8
var/lib/umegaya/packages-metadata/9
umegaya-0.13/debian/copyright 0000644 0000000 0000000 00000002161 12132223461 013124 0 ustar Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Source: http://source.umegaya.branchable.com/?p=source.git;a=tags
Files: *
Copyright: none claimed
License: BOLA-1.1
BOLA - Buena Onda License Agreement (v1.1)
------------------------------------------
.
This work is provided 'as-is', without any express or implied warranty. In no
event will the authors be held liable for any damages arising from the use of
this work.
.
To all effects and purposes, this work is to be considered Public Domain.
.
.
However, if you want to be "buena onda", you should:
.
1. Not take credit for it, and give proper recognition to the authors.
2. Share your modifications, so everybody benefits from them.
3. Do something nice for the authors.
4. Help someone who needs it: sign up for some volunteer work or help your
neighbour paint the house.
5. Don't waste. Anything, but specially energy that comes from natural
non-renewable resources. Extra points if you discover or invent something
to replace them.
6. Be tolerant. Everything that's good in nature comes from cooperation.
umegaya-0.13/debian/changelog 0000644 0000000 0000000 00000006360 12132231432 013045 0 ustar umegaya (0.13) unstable; urgency=low
06dbcdc Hosting the source code on Branchable.
2c14a12 Normalised debian/control with 'cme fix dpkg-control'.
5586d64 Conforms to Policy 3.9.4.
bf975bf Distribute the index, setup and todo files as documentation.
d482ec0 umegaya-fields-stats: ranked list of fields from copyright files
-- Charles Plessy Sat, 13 Apr 2013 19:27:36 +0900
umegaya (0.12) unstable; urgency=low
2293ecd umegaya-query: a tool to avoid typing the long default URL.
4db19af umegaya-license-stats: rank licenses by frequency in copyright files.
bfc7689 Serve umegaya-license-stats output at /umegaya/license-stats.
-- Charles Plessy Sat, 12 Jan 2013 17:56:06 +0900
umegaya (0.11) unstable; urgency=low
0268e44 Assume 'foo' as the package name for 'foo/trunk' Svn URLs.
7361239 Guess path to debian/upstream in GitHub.
4d88db9 umegaya-self-clean: exit if umegaya-adm is not installed.
Closes: #689250
-- Charles Plessy Thu, 04 Oct 2012 07:12:40 +0900
umegaya (0.10) unstable; urgency=low
* Initial upload to Debian, closes: #657825.
c0c90f5 umegaya-ddc-ping: Removed -x flag, forgotten in by accident.
af2f6c7 umegaya-ddc-ping: `message` option to use syslog instead of exiting
on failure.
860eb57 umegaya-self-clean: a script that tidies the Umegaya database.
d4b581f Build-depend on perl-doc.
-- Charles Plessy Sat, 08 Sep 2012 14:35:01 +0900
umegaya (0.9) unreleased; urgency=low
* Deployed on debian-med.debian.net.
[ Charles Plessy ]
f8de3b3 Update daily push script to also push control files.
1b83a6b Allow the insertion of a local comment.
[ Andreas Tille ]
e06dafc Make sure a failed ping of a package will not let the script exit.
-- Charles Plessy Mon, 25 Jun 2012 09:30:16 +0900
umegaya (0.8) unreleased; urgency=low
* Download Debian source package control files (optional).
* Deployed on debian-med.debian.net.
-- Charles Plessy Wed, 23 May 2012 09:06:57 +0900
umegaya (0.7) unreleased; urgency=low
* Deployed locally for upload-driven refreshes.
-- Charles Plessy Sun, 22 Apr 2012 12:48:05 +0900
umegaya (0.6) unreleased; urgency=low
* Daily cron job to refresh packages recommended or suggested by Blends.
* Deployed on debian-med.debian.net.
-- Charles Plessy Sun, 08 Apr 2012 23:00:54 +0900
umegaya (0.5) unreleased; urgency=low
* Packages can have plusses in their names.
* Deployed on debian-med.debian.net.
-- Charles Plessy Mon, 02 Apr 2012 23:53:58 +0900
umegaya (0.4) unreleased; urgency=low
* Deployed on debian-med.debian.net.
-- Charles Plessy Sun, 04 Mar 2012 17:23:43 +0900
umegaya (0.3) unreleased; urgency=low
* Test release on a private machine.
-- Charles Plessy Thu, 01 Mar 2012 23:41:32 +0900
umegaya (0.2) unreleased; urgency=low
* Deployed on debian-med.debian.net.
-- Charles Plessy Thu, 01 Mar 2012 23:28:39 +0900
umegaya (0.1) unreleased; urgency=low
* Proof of principle that works on localhost.
-- Charles Plessy Sun, 19 Feb 2012 00:01:37 +0900
umegaya-0.13/debian/postinst 0000644 0000000 0000000 00000002314 12074224361 013004 0 ustar #!/bin/sh
# postinst script for umegaya
#
# see: dh_installdeb(1)
set -e
# summary of how this script can be called:
# * `configure'
# * `abort-upgrade'
# * `abort-remove' `in-favour'
#
# * `abort-remove'
# * `abort-deconfigure' `in-favour'
# `removing'
#
# for details, see http://www.debian.org/doc/debian-policy/ or
# the debian-policy package
case "$1" in
configure)
if [ -x /etc/init.d/apache2 ]
then
if which /usr/sbin/invoke-rc.d >/dev/null 2>&1
then
invoke-rc.d apache2 reload || true
else
/etc/init.d/apache2 reload || true
fi
fi
;;
abort-upgrade|abort-remove|abort-deconfigure)
;;
*)
echo "postinst called with unknown argument \`$1'" >&2
exit 1
;;
esac
# dh_installdeb will replace this with shell code automatically
# generated by other debhelper scripts.
#DEBHELPER#
exit 0
umegaya-0.13/debian/source/ 0000755 0000000 0000000 00000000000 12074224361 012476 5 ustar umegaya-0.13/debian/source/format 0000644 0000000 0000000 00000000015 12074224361 013705 0 ustar 3.0 (native)
umegaya-0.13/debian/rules 0000755 0000000 0000000 00000001027 12074224361 012256 0 ustar #!/usr/bin/make -f
export DH_VERBOSE=1
%:
dh $@
override_dh_auto_build:
perldoc -o nroff cgi-bin/umegaya > man/umegaya.1
perldoc -o nroff scripts/umegaya-adm > man/umegaya-adm.1
for file in man/*.1.mdwn ; do \
pandoc -s -w man $$file -o man/$$(basename $$file .mdwn) ; done
override_dh_clean:
$(RM) man/umegaya*.1
dh_clean
override_dh_fixperms:
dh_fixperms
chmod -R 2775 $(CURDIR)/debian/umegaya/var/lib/umegaya
chown -R www-data:www-data $(CURDIR)/debian/umegaya/var/lib/umegaya
umegaya-0.13/debian/docs 0000644 0000000 0000000 00000000076 12132223554 012052 0 ustar scripts/example-push-metadata
index.mdwn
setup.mdwn
todo.mdwn
umegaya-0.13/debian/manpages 0000644 0000000 0000000 00000000007 12074224361 012711 0 ustar man/*1
umegaya-0.13/debian/install 0000644 0000000 0000000 00000000616 12132231470 012564 0 ustar cgi-bin/umegaya usr/lib/cgi-bin
conf/apache.conf etc/umegaya
conf/robots.txt etc/umegaya
conf/umegaya.conf etc/umegaya
scripts/umegaya-adm usr/bin
scripts/umegaya-guess-url usr/bin
scripts/umegaya-ddc-ping usr/bin
scripts/umegaya-fields-stats etc/cron.daily
scripts/umegaya-license-stats etc/cron.daily
scripts/umegaya-refresh-blends etc/cron.daily
scripts/umegaya-self-clean etc/cron.daily
umegaya-0.13/debian/control 0000644 0000000 0000000 00000001741 12132223250 012573 0 ustar Source: umegaya
Maintainer: Charles Plessy
Section: database
Priority: optional
Build-Depends: debhelper (>= 9),
pandoc,
perl,
perl-doc
Standards-Version: 3.9.4
Vcs-Browser: http://source.umegaya.branchable.com/?p=source.git;a=tree
Vcs-Git: git://umegaya.branchable.com/
Homepage: http://umegaya.branchable.com
Package: umegaya
Architecture: all
Depends: devscripts,
libberkeleydb-perl,
libyaml-libyaml-perl,
libyaml-appconfig-perl,
libwww-perl,
subversion,
${perl:Depends},
${misc:Depends}
Recommends: curl
Description: Umegaya is a MEtadata GAtherer using YAml
Aggregator of meta information about the software packages. The central
concept is that the metadata is accumlated by the package maintainer in a
version control system containing the source package, in a file called
debian/upstream, and collected by the web aggregator each time it is accessed.
umegaya-0.13/debian/upstream 0000644 0000000 0000000 00000000162 12132201514 012746 0 ustar Archive: Alioth
Contact: Charles Plessy
Name: Umegaya
Homepage: http://umegaya.branchable.com
umegaya-0.13/scripts/ 0000755 0000000 0000000 00000000000 12132230177 011440 5 ustar umegaya-0.13/scripts/umegaya-refresh-blends 0000755 0000000 0000000 00000004652 12074224361 015731 0 ustar #!/bin/sh
set -e
# Exit if refresh_blends is null in umegaya.conf.
[ $(grep ^refresh_blends /etc/umegaya/umegaya.conf |
sed -r 's/refresh_blends:\s+//') = 0 ] && exit 0
# Exit if curl, debcheckout, umegaya-adm, umegaya-guess-url or svn are not installed.
# Exit code is zero as this script is intended as a cron job.
[ -x /usr/bin/curl ] || exit 0
[ -x /usr/bin/debcheckout ] || exit 0
[ -x /usr/bin/umegaya-adm ] || exit 0
[ -x /usr/bin/umegaya-guess-url ] || exit 0
[ -x /usr/bin/svn ] || exit 0
# Start with debichem med and science, but later let's fetch all the
# names from $(svn ls svn://svn.debian.org/blends/projects/)
BLENDS='debichem debian-med debian-science'
#set -x
for BLEND in $BLENDS
do
BLEND_URL=$(debcheckout -d $BLEND | grep url | cut -f2)
# Learn VCS URL for packages not yet in Debian.
for TASK in $(svn ls $BLEND_URL/tasks)
do
for VCS_URL in $(svn cat $BLEND_URL/tasks/$TASK | grep 'Vcs-[SG]' | cut -f2 -d' ')
do
PACKAGE=$(umegaya-guess-url $VCS_URL | cut -f1)
[ $PACKAGE ] &&
umegaya-adm -p $PACKAGE -k YAML-URL -v $VCS_URL
done
done
# Refresh Umegaya for each package.
PACKAGES=$(
svn cat $BLEND_URL/debian/control |
grep -e ^Recommends -e ^Suggests |
sed -e 's/,//g' -e 's/|//g' -e 's/Recommends://g' -e 's/Suggests://g' )
for PACKAGE in $PACKAGES
do
SOURCE=$(apt-cache showsrc $PACKAGE 2> /dev/null | grep Package -m1 | awk '{print $2}')
if [ "$SOURCE" != "" ] ; then
# make sure a failed ping will not interupt script
curl --silent "http://localhost/umegaya/$SOURCE/ping" || true
if [ $? != 0 ] ; then
echo "$0: Problem pinging $SOURCE ($?)"
fi
fi
done
done
# create downloadable archive of the fetched files
STOREFILES=`grep ^store_files /etc/umegaya/umegaya.conf | sed -r 's/store_files:\s+//'`
TARGETARCHIVE=`grep ^archive_for_bibref_gatherer /etc/umegaya/umegaya.conf | sed -r 's/archive_for_bibref_gatherer:\s+//'`
if [ $TARGETARCHIVE != "" -a $STOREFILES != "" ] ; then
STOREDIR=`dirname $STOREFILES`
if [ -d $STOREDIR ] ; then
cd $STOREDIR
BASESTOREDIR=`basename $STOREFILES`
if [ -d $BASESTOREDIR ] ; then
tmpignorezerolengthfiles=`mktemp`
find packages-metadata -size 0 >> $tmpignorezerolengthfiles
tar --exclude-vcs --exclude-from $tmpignorezerolengthfiles -cjf $TARGETARCHIVE $BASESTOREDIR
rm -f $tmpignorezerolengthfiles
fi
fi
fi
umegaya-0.13/scripts/umegaya-ddc-ping 0000755 0000000 0000000 00000001374 12074224361 014511 0 ustar #!/bin/sh -e
# Umegaya's debian-devel-changes pinger
#
# Piped emails from http://lists.debian.org/debian-devel-changes to this program
# trigger a ping for the corresponding package in an Umegaya instance.
URL=$(awk '/^ping_url:/ {print $2 ; nextfile}' /etc/umegaya/umegaya.conf)
PACKAGE=$(grep ^Subject -m1 | awk '{print $3}')
MESSAGES=$(awk '/^messages:/ {print $2 ; nextfile}' /etc/umegaya/umegaya.conf)
if [ "$PACKAGE" != '' ] ; then
curl --silent ${URL-http://localhost/umegaya}/$PACKAGE/ping > /dev/null
if [ "$MESSAGES" = 'syslog' ] ; then
logger -i umegaya-ddc-ping: pinged $PACKAGE at $URL
fi
exit 0
fi
if [ "$MESSAGES" = 'syslog' ] ; then
logger -i umegaya-ddc-ping: failed to find package name in standard input
else
exit 1
fi
exit 0
umegaya-0.13/scripts/umegaya-reshape 0000755 0000000 0000000 00000000472 12074224361 014451 0 ustar #!/usr/bin/r
write.table( file = "" ,
sep = "\t" ,
quote = FALSE ,
row.names = FALSE ,
reshape( data.frame(scan( "",
quiet = TRUE,
sep = "\t",
what = list(pack="", key="",val="")
)),
direction = "wide",
idvar = "pack",
timevar = "key"
)
)
umegaya-0.13/scripts/umegaya-adm 0000755 0000000 0000000 00000012205 12074224361 013560 0 ustar #!/usr/bin/perl
=head1 NAME
umegaya-adm - A tool to administrate the Umegaya database.
=head1 SYNOPSIS
B B<-p> package B<-k> key B<-v> value [B<--delete] [B<--verbose>]
B B<-h>, B<--help>
=head1 DESCRIPTION
C interrogates or modifies the Berkeley database in which
the gathered metadata is stored. If a combination of package, key and
value is given, it will set the value. Otherwise it will give
appropriate information using the of package or key provided as
argument.
For example:
$ umegaya-adm -p package -k key -v value
$ umegaya-adm -p package -k key
value
$ umegaya-adm -p package -k key2 -v value2
$ umegaya-adm -p package -k key2
value2
$ umegaya-adm -p package
key value
key2 value2
$ umegaya-adm -p package2 -k key -v value3
$ umegaya-adm -k key
package value
package2 value3
$ umegaya-adm -p package2 -k key --delete
$ umegaya-adm -k key
package value
=cut
use strict;
use warnings;
use 5.10.0;
use BerkeleyDB;
use Getopt::Long;
use YAML::AppConfig;
my $conf = YAML::AppConfig->new(file => "/etc/umegaya/umegaya.conf");
my $lib = $conf->get("db_dir");
my $berkeleydb = $lib . $conf->get("db_name");
my %stored;
umask 002;
tie (%stored, 'BerkeleyDB::Hash', -Filename => $berkeleydb, -Flags => DB_CREATE)
or die "Cannot open file $berkeleydb: $! $BerkeleyDB::Error\n" ;
=head1 OPTIONS
=over 8
=item B<-d>, B<--delete>
In combination with B<--key> or B<--package>, delete the value(s) instead of
displaying them.
=item B<-k>, B<--key>
If set alone, C will display all the package-values pairs for the
given key. If set together with B<--package>, C will display the
corresponding value.
=item B<-p>, B<--package>
If set alone, C will display all the key-values pairs for the given
package. If set together with B<--key>, C will display the
corresponding value.
=item B<--purge>
Deletes the umegaya database without asking for confirmation.
=item B<-v>, B<--value>
When this option is set together with B<--key> and B<--package>, C
sets the given value for the given key of the given package.
=item B<--verbose>
Returns the set value to the standard output when used in conjunction with
B<--key>, B<--package>, and B<--value>.
=back
=cut
my ($delete, $package, $purge, $help, $key, $value, $verbose);
GetOptions ( 'delete' => \$delete ,
'package|p=s' => \$package ,
'purge' => \$purge ,
'help' => \$help ,
'key=s' => \$key ,
'value|v=s' => \$value ,
'verbose' => \$verbose );
if ($help) {
print qx(perldoc $0);
exit;
}
if ($purge) {
die "$0: --purge can not be combined with other arguments.\n" if ($delete or $package or $key or $value);
untie %stored;
unlink $berkeleydb or die "$0: could not remove $berkeleydb: $!\n";
exit;
}
if ($value) {
die "$0: --value option needs --package and --key to be set\n" unless ($package and $key);
$stored{join(':', $package, $key)} = $value;
say $stored{join(':', $package, $key)} if $verbose;
} elsif ($key) {
if ($package) {
$delete ? delete $stored{join(':', $package, $key)} : say $stored{join(':', $package, $key)};
exit
}
foreach (keys (%stored)) {
next unless /:$key$/;
my $package = (split (':'))[0];
$delete ? delete $stored{$_} : say $package, "\t", $stored{$_};
}
} elsif ($package) {
foreach (keys (%stored)) {
next unless /^$package:/;
my $key = (split (':'))[1];
$delete ? delete $stored{$_} : say $key, "\t", $stored{$_};
}
} else {
die "Use ‘rm $berkeleydb’ to delete the whole file\n" if $delete;
say "Type umegaya-adm --help to read the documentation";
}
=head1 ADVANCED USAGE
use C to set special keys for the packages.
=head1 SEE ALSO
umegaya(7), upstream-metadata.debian.net(7)
=head1 AUTHOR
Charles Plessy
=head1 LICENCE
BOLA - Buena Onda License Agreement (v1.1)
------------------------------------------
This work is provided 'as-is', without any express or implied warranty. In no
event will the authors be held liable for any damages arising from the use of
this work.
To all effects and purposes, this work is to be considered Public Domain.
However, if you want to be "buena onda", you should:
1. Not take credit for it, and give proper recognition to the authors.
2. Share your modifications, so everybody benefits from them.
3. Do something nice for the authors.
4. Help someone who needs it: sign up for some volunteer work or help your
neighbour paint the house.
5. Don't waste. Anything, but specially energy that comes from natural
non-renewable resources. Extra points if you discover or invent something
to replace them.
6. Be tolerant. Everything that's good in nature comes from cooperation.
=head1 SOURCE
http://anonscm.debian.org/gitweb/?p=users/plessy/umegaya.git;a=blob;f=scripts/umegaya-adm
=cut
umegaya-0.13/scripts/umegaya-fields-stats 0000755 0000000 0000000 00000001474 12132230177 015424 0 ustar #!/bin/sh
set -e
# Exit if umegaya-adm is not installed.
# Exit code is zero as this script is intended as a cron job.
[ -x /usr/bin/umegaya-adm ] || exit 0
TMPFILE=$(mktemp -t umegaya-fields-stats.XXXXXXXXXX)
for file in $(grep -rl '^Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/$' /var/lib/umegaya/packages-metadata/*/*.copyright )
do
grep --only-matching --perl-regexp '^[^\s:]*' $file |
sort --ignore-case --unique
done |
sort --ignore-case |
uniq --ignore-case --count |
sort --reverse --numeric-sort > $TMPFILE
TMPFILE_USER=$(stat --format %u /var/lib/umegaya/)
TMPFILE_GROUP=$(stat --format %g /var/lib/umegaya/)
if [ $(whoami) = "root" ]
then
chown $TMPFILE_USER:$TMPFILE_GROUP $TMPFILE
chmod 664 $TMPFILE
fi
cp $TMPFILE /var/lib/umegaya/fields-stats
rm $TMPFILE
umegaya-0.13/scripts/example-push-metadata 0000644 0000000 0000000 00000002665 12132206122 015553 0 ustar #!/bin/sh -e
#
# This is the script used on blends.debian.net to push the collected files to
# the packages-metadata directory of the Subversion repositry 'collab-qa'.
#
# The repostitory has been checked out with the following command.
#
# SVN_SSH="ssh -i $HOME/.ssh/alioth-svn-commit_rsa" svn checkout --force svn+ssh://svn.debian.org/svn/collab-qa/packages-metadata/
#
# It is ugly and has many shortcomings. All the find commands should
# disappear.
#
# The SSH key is of course restricted to only run svnserve.
POOL=/var/lib/umegaya/packages-metadata/
STAMP= ${TMPDIR-/tmp}/umegaya-timestamp
rm -f $STAMP
touch $STAMP
find $POOL -empty | grep -v 'svn/' | xargs -L1 rm -f
find $POOL -name '*control' | xargs grep -l DOCT | xargs rm -f
find $POOL -name '*copyright' | xargs grep -l DOCT | xargs rm -f
find $POOL -name '*upstream' | xargs grep -l DOCT | xargs rm -f
find $POOL -name '*upstream' | xargs grep -l '^ $TMPFILE
TMPFILE_USER=$(stat --format %u /var/lib/umegaya/)
TMPFILE_GROUP=$(stat --format %g /var/lib/umegaya/)
if [ $(whoami) = "root" ]
then
chown $TMPFILE_USER:$TMPFILE_GROUP $TMPFILE
chmod 664 $TMPFILE
fi
cp $TMPFILE /var/lib/umegaya/license-stats
rm $TMPFILE
umegaya-0.13/scripts/biblio-for-UDD.sh 0000755 0000000 0000000 00000001175 12074224361 014444 0 ustar #!/bin/bash -e
UMEGAYA=/home/karl/upstream-metadata.debian.net/
rm -f $UMEGAYA/for_UDD/biblio.long
rm -f $UMEGAYA/for_UDD/biblio.wide
rm -f $UMEGAYA/for_UDD/biblio.yaml
for key in DOI PMID Reference-Author Reference-Eprint Reference-Journal Reference-Number Reference-Pages Reference-Title Reference-URL Reference-Volume Reference-Year References
do
curl http://upstream-metadata.debian.net/table/$key >> $UMEGAYA/for_UDD/biblio.long
curl http://upstream-metadata.debian.net/yaml/$key >> $UMEGAYA/for_UDD/biblio.yaml
done
$UMEGAYA/scripts/umegaya-reshape.r < $UMEGAYA/for_UDD/biblio.long | sed '1d' > $UMEGAYA/for_UDD/biblio.wide
umegaya-0.13/scripts/umegaya-query 0000755 0000000 0000000 00000000365 12074224361 014170 0 ustar #!/bin/sh
set -e
usage() {
echo "Usage: $0 key package|table|yaml"
exit 1
}
abort() {
echo $1
exit 1
}
[ -x /usr/bin/curl ] || abort "Aborted: please install curl."
BASEURL="http://upstream-metadata.debian.net"
curl $BASEURL/$2/$1
umegaya-0.13/scripts/umegaya-self-clean 0000755 0000000 0000000 00000000526 12074224361 015033 0 ustar #!/bin/sh
set -e
# Exit if umegaya-adm is not installed.
# Exit code is zero as this script is intended as a cron job.
[ -x /usr/bin/umegaya-adm ] || exit 0
# Delete from the Umegaya database every package whose URL is 'NA'
for package in $(umegaya-adm -k YAML-URL | awk '$2=="NA" { print $1}')
do
umegaya-adm -p $package -d
done
umegaya-0.13/scripts/umegaya-guess-url 0000755 0000000 0000000 00000004340 12074224361 014746 0 ustar #!/usr/bin/perl
use strict;
use warnings;
use 5.10.0;
my $package_url = shift;
my $package;
chomp $package_url;
given ($package_url) {
when (undef) {
die "Usage: umegaya-guess-url Vcs-URL\n";
}
when (/^svn/) {
# Debian Med
if (m|debian-med/trunk/packages|) {
if (m|svn://svn.debian.org/debian-med/trunk/packages.*/([a-z0-9\+\-\.]+)/trunk| ) {
$package = $1;
} else {
die <<"ERROR_MESSAGE"
Could not parse « $package_url ».
The package is located in Debian Med’s Subversion repository according to its
VCS URL, but it does not point at a trunk directory as it should. See the
following page for more information.
http://debian-med.alioth.debian.org/docs/policy.html#svn-repository-structure
ERROR_MESSAGE
}
}
# Debichem
elsif (m|svn://svn.debian.org/s?v?n?/?debichem/unstable/([a-z0-9\+\-\.]+)/|) {
$package = $1;
}
elsif (m|svn://svn.debian.org/s?v?n?/?debichem/experimental/([a-z0-9\+\-\.]+)/|) {
$package = $1;
}
elsif (m|svn://svn.debian.org/s?v?n?/?debichem/wnpp/([a-z0-9\+\-\.]+)/|) {
$package = $1;
}
# Debian Science
elsif (m|svn://svn.debian.org/s?v?n?/?debian-science/packages/R/([a-z0-9\+\-\.]+)/| ) {
$package = $1;
}
elsif (m|svn://svn.debian.org/s?v?n?/?debian-science/packages/([a-z0-9\+\-\.]+)/| ) {
$package = $1;
}
# Other repositories
elsif (m|svn://svn.debian.org/.*/([a-z0-9\+\-\.]+)/trunk| ) {
$package = $1;
}
else {
die "Could not detect package name for $package_url\n";
}
$package_url .= "debian/upstream";
}
when (/^git/) {
if (m|git://git.debian.org/.+/([a-z0-9\+\-\.]+).git|) {
$package = $1;
$package_url =~ s|^git://git.debian.org/g?i?t?/?(.*)|http://git.debian.org/?p=$1;a=blob_plain;f=debian/upstream;hb=HEAD|;
}
# GitHub (master branch only)
elsif (m|git://github.com/([A-Za-z0-9\+\-\.]+)/([a-z0-9\+\-\.]+).git$|) {
$package = $2;
$package_url = "https://raw.github.com/$1/$package/master/debian/upstream"
}
else {
die "Could not detect package name for $package_url\n";
}
}
default {
die "Only Subversion and Git URLs are supported"
}
}
say "$package\t$package_url";
umegaya-0.13/LICENSE 0000644 0000000 0000000 00000001606 12074224361 010764 0 ustar BOLA - Buena Onda License Agreement (v1.1)
------------------------------------------
This work is provided 'as-is', without any express or implied warranty. In no
event will the authors be held liable for any damages arising from the use of
this work.
To all effects and purposes, this work is to be considered Public Domain.
However, if you want to be "buena onda", you should:
1. Not take credit for it, and give proper recognition to the authors.
2. Share your modifications, so everybody benefits from them.
3. Do something nice for the authors.
4. Help someone who needs it: sign up for some volunteer work or help your
neighbour paint the house.
5. Don't waste. Anything, but specially energy that comes from natural
non-renewable resources. Extra points if you discover or invent something
to replace them.
6. Be tolerant. Everything that's good in nature comes from cooperation.
umegaya-0.13/cgi-bin/ 0000755 0000000 0000000 00000000000 12132222530 011253 5 ustar umegaya-0.13/cgi-bin/umegaya 0000755 0000000 0000000 00000020037 12132222530 012633 0 ustar #!/usr/bin/perl
=head1 NAME
Umegaya - Umegaya is a MEtadata GAtherer using YAml
=head1 SYNOPSIS
curl http://localhost/umegaya/B/B
/usr/lib/cgi-bin/umegaya package=B key=B
curl http://localhost/umegaya/table/B
/usr/lib/cgi-bin/umegaya table=1 key=B
curl http://localhost/umegaya/yaml/B
/usr/lib/cgi-bin/umegaya yaml=1 key=B
=head1 DESCRIPTION
B is a gatherer of meta information about the software packaged in
Debian. This metadata is accumlated by the package maintainer in the version
control system containing the Debian souce package, in a file called
C, and it is collected by the Umegaya web service each time it
is accessed. The Umegaya web service then aggregates information about all the
packages it knows, and outputs as tables or YAML records that can be loaded in
central information hubs like the Ultimate Debian Database.
Optionally, Umegaya will keep a local copy if the files it retrieves from the
VCS, if a path is indicated in the C configuration variable.
=head1 SPECIAL KEYS
YAML-ALL, YAML-REFRESH-DATE, YAML-URL
=head1 FILES
/etc/umegaya/umegaya.conf
=head1 SEE ALSO
http://wiki.debian.org/UpstreamMetadata
umegaya-adm(1)
=head1 SOURCE
http://umegaya.branchable.com/
=head1 LIMITATIONS
Only source packages stored in Subversion and Git are supported for the moment.
The CGI interface of Umegaya is currently read-only. But PUT and DELETE
operations are considered.
=cut
use strict;
use warnings;
use 5.10.0;
use BerkeleyDB;
use CGI;
use LWP::Simple qw(get);
use YAML::XS qw(Load Dump DumpFile);
use YAML::AppConfig;
my $conf = YAML::AppConfig->new(file => "/etc/umegaya/umegaya.conf");
my $lib = $conf->get("db_dir");
my $local_comment = $conf->get("local_comment");
my $berkeleydb = $lib . $conf->get("db_name");
my $delay = $conf->get("delay");
my $debug = $conf->get("debug");
my $store_files = $conf->get("store_files");
my $debian_control = $conf->get("debian_control") ;
my $debian_copyright = $conf->get("debian_copyright");
my %stored;
umask 002;
tie (%stored, 'BerkeleyDB::Hash', -Filename => $berkeleydb, -Flags => DB_CREATE)
or die "Cannot open file $berkeleydb: $! $BerkeleyDB::Error\n" ;
my $cgi = CGI->new;
my $package = $cgi->param("package") // 0 ;
my $help = $cgi->param("help") // 0 ;
my $table = $cgi->param("table") // 0 ;
my $yaml = $cgi->param("yaml") // 0 ;
my $key = $cgi->param("key") // 0 ;
$debug = $cgi->param("debug") // $debug ;
say <<"__DEBUG__" if $debug;
Package: $package
Table: $table
YAML: $yaml
Key: $key
__DEBUG__
# Default homepage.
help() if ($help);
help() unless ($table or $yaml or $package);
sub help {
my $help_text = qx(perldoc -o html umegaya);
$help_text =~ s//\n$local_comment/ if $local_comment;
print $cgi->header('text/html');
print $help_text;
exit;
}
print $cgi->header(-type=>'text/plain', -charset=>'utf-8');
say "Debug mode on; somebody is probably working on this site…" if $debug;
# Output for http://localhost/umegaya/table/key URLs.
if ($table | $yaml) {
foreach (keys (%stored)) {
next unless /:\Q$key\E$/i;
my $package = (split (/:/))[0];
say join ("\t", $package, $key, $stored{$_}) if $table;
say Dump ([$package, $key, $stored{$_}]) if $yaml;
}
exit;
}
# The URL of the 'upstream' file is never refreshed.
unless (defined ($stored{"$package:YAML-URL"})) {
$stored{"$package:YAML-URL"} = qx(debcheckout -d $package | grep url | cut -f2) || "NA";
chomp $stored{"$package:YAML-URL"};
}
# Refrain to refresh if the last update is still fresh, to avoid loading Alioth.
if (defined ($stored{"$package:YAML-REFRESH-DATE"})) {
say "Not updated since:", time - $stored{"$package:YAML-REFRESH-DATE"} if $debug;
$stored{"$package:YAML-REFRESH-DATE"} = refresh_from_url($stored{"$package:YAML-URL"}) if time - $stored{"$package:YAML-REFRESH-DATE"} > $delay;
} else {
say "$package is not yet in the database, trying to pull its metadata…" if $debug;
$stored{"$package:YAML-REFRESH-DATE"} = refresh_from_url($stored{"$package:YAML-URL"})
}
# Output for http://localhost/umegaya/package/key URL.
say "Looking for key $key of package ${package}." if $debug;
foreach (keys (%stored)) {
next unless /\Q${package}:$key\E$/i;
say "$_ is the key for $key and $package." if $debug;
say $stored{"$_"};
exit;
}
say "No $key key for package ${package}." if $debug;
# Reconstructs an URL to the a target file in the Debian directory.
# For Git repositories on Alioth, it relies on the Gitweb interface.
sub guess_file_url {
my $file_url = shift;
my $file_to_download = shift;
if ($file_url =~ /^svn/) {
$file_url .= '/' unless $file_url =~ m(/$);
$file_url .= "debian/$file_to_download";
}
$file_url =~ s|^git://git.debian.org/g?i?t?/?(.*)|http://git.debian.org/?p=$1;a=blob_plain;f=debian/$file_to_download;hb=HEAD| if $file_url =~ /^git/;
say "Will retreive $file_url for debian/$file_to_download." if $debug;
return $file_url;
}
# Refresh and return a time stamp; store files if asked.
sub refresh_from_url {
my $package_url = shift;
say "Refreshing data from $package_url" if $debug;
my $debian_upstream_url = guess_file_url($package_url, "upstream");
my $package_metadata_yaml;
$package_metadata_yaml = svn_export($debian_upstream_url) if $debian_upstream_url =~ /^svn/;
$package_metadata_yaml = get($debian_upstream_url ) if $debian_upstream_url =~ /^http/;
if ( $store_files ) {
$package =~ /^(.)/;
my $pool = $1;
open(my $debian_upstream, ">", "$store_files/$pool/${package}.upstream")
or die "Can not write $store_files/$pool/${package}.upstream: $!";
print $debian_upstream $package_metadata_yaml;
close($debian_upstream);
if ( $debian_control ) {
my $debian_control_url = guess_file_url($package_url, "control");
system(qq(svn cat "$debian_control_url" > $store_files/$pool/${package}.control)) if $debian_control_url =~ /^svn/;
system(qq(GET "$debian_control_url" > $store_files/$pool/${package}.control)) if $debian_control_url =~ /^http/;
}
if ( $debian_copyright ) {
my $debian_copyright_url = guess_file_url($package_url, "copyright");
system(qq(svn cat "$debian_copyright_url" > $store_files/$pool/${package}.copyright)) if $debian_copyright_url =~ /^svn/;
system(qq(GET "$debian_copyright_url" > $store_files/$pool/${package}.copyright)) if $debian_copyright_url =~ /^http/;
}
}
my $package_metadata;
eval { $package_metadata = Load ($package_metadata_yaml)};
say "Could not parse the YAML file for $package\n" and die if $@;
# Drops all previous keys.
foreach (keys (%stored)) {
next unless /^$package:/;
next if /:YAML-URL$/;
say "Deleting $_" if $debug;
undef $stored{$_};
}
# Loads the new keys.
foreach (keys (%$package_metadata)) {
my $key = $package_metadata->{$_};
my $keytype = ref($key);
say "Key type is $keytype." if $debug;
# Compound keywords are equivalent to nested mappings:
#
# Foo-Bar: baz
#
# is the same as:
#
# Foo:
# Bar: baz
#
if ( $keytype eq "HASH" ) {
my $left = $_;
foreach ( keys (%$key)) {
say "Setting ${package}:${left}-$_ to $key->{$_}" if $debug;
$stored{"${package}:${left}-$_"} = $key->{$_};
}
} elsif ( $keytype eq "ARRAY" ) {
say "Setting ${package}:$_ to $key" if $debug;
$stored{"${package}:$_"} = $key;
} else {
say "Setting ${package}:$_ to $key" if $debug;
$stored{"${package}:$_"} = $key;
}
}
# Keeps a backup of the 'upstream' file.
$stored{"${package}:YAML-ALL"} = $package_metadata_yaml;
return time;
}
# For the moment only SVN is supported.
sub svn_export {
my $url = shift;
return qx(svn cat $url);
}
umegaya-0.13/man/ 0000755 0000000 0000000 00000000000 12132231503 010516 5 ustar umegaya-0.13/man/umegaya-ddc-ping.1.mdwn 0000644 0000000 0000000 00000002136 12074224361 014672 0 ustar % UMEGAYA-DDC-PING(1) Umegaya User Manual
% Charles Plessy
% August 21, 2012
# NAME
umegaya-ddc-ping - Umegaya's debian-devel-changes pinger
# SYNOPSIS
umegaya-ddc-ping < email.txt
# DESCRIPTION
Piped emails from http://lists.debian.org/debian-devel-changes to this program
trigger a ping for the corresponding package in an Umegaya instance.
If `emailaddress` is subscribed to debian-devel-changes, the following
entry in `/etc/aliases` will forward emails to `umegaya-ddc-ping`.
emailaddress: |umegaya-ddc-ping
The name of the package to ping is searched in the third field of the first
line starting with `Subject`. In case of failure, the script returns with
an exit code of 1, unless the `messages` option is used.
# OPTIONS
`umegaya-ddc-ping` takes no option from the command line, but reads the
following from `/etc/umegaya/umegaya.conf`.
## `ping_url`
URL of an Umegaya instance. `http://localhost/umegaya` by default.
## `messages`
Log pings and failures to determine package name to syslog, and do not exit
with error in case of failure. Off by default.
# SEE ALSO
`umegaya.conf` (5).
umegaya-0.13/man/umegaya-query.1.mdwn 0000644 0000000 0000000 00000001266 12074224361 014355 0 ustar % UMEGAYA-QUERY(1) Umegaya User Manual
% Charles Plessy
% October 13, 2012
# NAME
umegaya-query - query a remote umegaya instance
# SYNOPSIS
umegaya-query key package|table|yaml
# DESCRIPTION
This is a convenience script to avoid typing the long URL
http://upstream-metadata.debian.net. All it does for the
moment is to call curl.
The name and the interface may change in the final implementation.
Later it will take its default URL form `umegaya.conf`, check
its command-line parameters, etc.
# OPTIONS
## key
First option is a key name.
## package|table|yaml
Second option is a package name, or a special command name such as "table" or "yaml".
# SEE ALSO
`umegaya.conf` (5).
umegaya-0.13/man/umegaya-fields-stats.1.mdwn 0000644 0000000 0000000 00000000710 12132230707 015577 0 ustar % UMEGAYA-FIELDS-STATS(1) Umegaya User Manual
% Charles Plessy
% April 13, 2013
# NAME
umegaya-fields-stats - ranked list of fields from collected copyright files
# SYNOPSIS
umegaya-fields-stats
# DESCRIPTION
Parses all the machine-readable Debian copyright files collected in the
repository `/var/lib/umegaya/packages-metadata`, and produces a list of
fields ranked by their frequency, reporting the number of files where the
fields has been found.
umegaya-0.13/man/umegaya-guess-url.1.mdwn 0000644 0000000 0000000 00000003314 12074224361 015132 0 ustar % UMEGAYA-GUESS-URL(1) Umegaya User Manual
% Charles Plessy
% April 8, 2012
# NAME
umegaya-guess-url - Converts a VCS URL in a URL to "debian/upstream"
# SYNOPSIS
umegaya-guess-url VCS_URL
# DESCRIPTION
VCS (version control system) URLs indicate where to clone or check out
anonymously source packages. `umegaya-guess-url` attempts to guess the package
name and convert the VCS URL in either a Subversion URL that can be used by
`svn cat`, or in a HTTP URL that can be used to retrieve `debian/upstream` via
Gitweb.
It recoginses VCS URLs from the following Debian Pure Blends.
svn://svn.debian.org/debian-med/trunk/packages/+?/([a-z0-9\+\-\.]+)/trunk
git://git.debian.org/debian-med/([a-z0-9\+\-\.]+).git
svn://svn.debian.org/s?v?n?/?debichem/unstable/([a-z0-9\+\-\.]+)/
svn://svn.debian.org/s?v?n?/?debichem/experimental/([a-z0-9\+\-\.]+)/
svn://svn.debian.org/s?v?n?/?debichem/wnpp/([a-z0-9\+\-\.]+)/
git://git.debian.org/g?i?t?/?debichem/packages/([a-z0-9\+\-\.]+).git
svn://svn.debian.org/s?v?n?/?debian-science/packages/R/([a-z0-9\+\-\.]+)/
svn://svn.debian.org/s?v?n?/?debian-science/packages/([a-z0-9\+\-\.]+)/
git://git.debian.org/g?i?t?/?debian-science/packages/([a-z0-9\+\-\.]+).git
# RETURNS
A package name, followed by a tabulation, followed by an URL.
# EXAMPLE
$ ./scripts/umegaya-guess-url $(debcheckout -d emboss | grep url | cut -f2)
emboss http://git.debian.org/?p=debian-med/emboss.git;a=blob_plain;f=debian/upstream;hb=HEAD
# SEE ALSO
`umegaya-adm(1)`, `umegaya-refresh-blend` (1), http://wiki.debian.org/UpstreamMetadata
# SOURCE
http://git.debian.org/?p=users/plessy/umegaya.git
# LIMITATIONS
Only source packages stored in Subversion and Git are supported for the moment.
umegaya-0.13/man/umegaya-license-stats.1.mdwn 0000644 0000000 0000000 00000000722 12074224361 015762 0 ustar % UMEGAYA-LICENSE-STATS(1) Umegaya User Manual
% Charles Plessy
% January 12, 2013
# NAME
umegaya-license-stats - ranked list of licenses from collected copyright files
# SYNOPSIS
umegaya-license-stats
# DESCRIPTION
Parses all the machine-readable Debian copyright files collected in the
repository `/var/lib/umegaya/packages-metadata`, and produces a list of
licenses ranked by their frequency, reporting the number of files where the
license has been found.
umegaya-0.13/conf/ 0000755 0000000 0000000 00000000000 12132230546 010676 5 ustar umegaya-0.13/conf/robots.txt 0000644 0000000 0000000 00000000032 12074224361 012745 0 ustar User-agent: *
Disallow: /
umegaya-0.13/conf/apache.conf 0000644 0000000 0000000 00000001253 12132230546 012767 0 ustar Alias /umegaya/fields-stats /var/lib/umegaya/fields-stats
Alias /umegaya/license-stats /var/lib/umegaya/license-stats
Alias /umegaya /usr/share/umegaya/htdocs
Options +FollowSymLinks
RewriteEngine on
RewriteRule ^$ /cgi-bin/umegaya?help=1
RewriteRule ^table/([\w_\-]+)$ /cgi-bin/umegaya?table=1;key=$1 [B]
RewriteRule ^yaml/([\w_\-]+)$ /cgi-bin/umegaya?yaml=1;key=$1 [B]
RewriteRule ^([a-z0-9\.\-\+]+)/([\w_\-]+)$ /cgi-bin/umegaya?package=$1;key=$2 [B]
#
# ServerName umegaya.example.com
# ServerAdmin umegaya@example.com
# DocumentRoot /usr/share/umegaya/htdocs
#
umegaya-0.13/conf/umegaya.conf 0000644 0000000 0000000 00000002615 12074224361 013204 0 ustar # Location for the Berkeley DB.
db_dir: /var/lib/umegaya/
# Name of the Berkeley DB.
db_name: umegaya.db
# Also download Debian control or copyright files if store_files is set.
#debian_control: 1
#debian_copyright: 1
# Numbers of seconds for which a new request will not trigger a refresh.
# Default: one hour.
delay: 3600
# Debug mode makes Umegaya scripts verbose.
debug: 0
# Local comment
#local_comment: >
# This Umegaya instance on upstream-metadata.debian.net is an experimental
# service that is not part of the official Debian infrastructure, which is why it
# is incubated on the debian.net domain instead of debian.org.
# When refresh_blends is set to 1, the daily cron job umegaya-refresh-blends
# will refresh Umegaya for all Debian Pure Blends. This means one request to
# alioth.debian.org for each package recommended or suggested by each Blend.
refresh_blends: 0
# umegaya-ddc-ping sends messages to syslog if the following is uncommented.
#messages: syslog
# Will store a copy of debian/upstream and debian/copyright if set to a location.
#store_files: /var/lib/umegaya/packages-metadata
# URLs for Umegaya
#local_url: http://umegaya.localhost
local_url: http://localhost/umegaya
ping_url: http://localhost/umegaya
# Archive with upstream files for UDD bibref gatherer
#archive_for_bibref_gatherer: /var/lib/gforge/chroot/home/groups/blends/htdocs/packages-metadata/packages-metadata.tar.bz2
umegaya-0.13/man.mdwn 0000644 0000000 0000000 00000000657 12132230762 011423 0 ustar Umegaya's manual pages
======================
* [[umegaya-ddc-ping.1]] — Umegaya's debian-devel-changes pinger
* [[umegaya-guess-url.1]] — Converts a VCS URL in a URL to `debian/upstream`
* [[umegaya-fields-stats.1]] — ranked list of fields from collected copyright files
* [[umegaya-license-stats.1]] — ranked list of licenses from collected copyright files
* [[umegaya-query.1]] — query a remote umegaya instance
umegaya-0.13/todo.mdwn 0000644 0000000 0000000 00000005054 12132175477 011624 0 ustar To do
=====
- Problem: make sure that Apache as the write permission on the pool. This
will not be if a user creates a file with a usual umask.
- Keep an eye on https://joinup.ec.europa.eu/asset/adms_foss/description
- umegaya-refresh-blends should log warnings instead of sending them
to root via the cron job.
- Make sure the cron job can work when the package is removed but not purged.
- Make umegaya-adm case-insensitive
- Safeguards to that umegaya only works on source package names.
- Cleanups to remove bogus files.
for package in $(for file in $(find . -name '*.upstream') ; do basename $file .upstream ; done) ; do echo -ne "$package\t" ; apt-cache showsrc $package | grep Package -m1 | awk '{print $2}'; done
The above prints the package name according to the file, and the source
package name. Mismatches should be errors, but there are false
positives, such as 'dialign-tx dialign-t'
- Prevent the creation of arbitrary file names (with spaces) that are
not package names.
- Do not create empty file if contents were not retrieved.
- Check if there is no buggy package names with spaces in the berkeley DB.
- Do not create 404 Gitweb files.
- See the example file 'example-push-metadata' on how to clean a package-metadata
repository before pushing
- Die gracefully if package is not found.
- Make sure two processes can run at the same time.
- Support HTTP DELETE method.
- Ignore safely the reserved YAML-* field names.
- Warn if debcheckout fails.
- User-Agent HTTP header.
- A Lintian check.
- dh_bibref ?
- Regression tests.
- Store YAML load errors.
- An example debian/upstream file, that illustrates YAML syntax.
- Use a proper installation script instead of debian/install.
- umegaya-adm: return package list when invoked with -k and -v
Useful for searching for NA YAML-URLs.
- Make help page of umegaya accessible when ran through a path (not
from the current directory).
- Sanitize input of the CGI program; it looks completely insecure.
- Read http://perldoc.perl.org/CGI.html#Avoiding-Denial-of-Service-Attacks
- URL-encode pacakge (and perhaps key) names.
- Warn when VCS URL does not point at a directory (lack a trailing slash).
- Integrate in the PTS.
- Detect all source packages that contain a debian/upstream file, for instance using
apt-file's "-a source" option (see "#632254").
- Provide an index of all packages in the Umegaya database (or with
upstream metadata in the UDD).
- Accept new debian/upstream files from the public.
- Integrate with http://duck.debian.net/