Create a Mirmon object from a config file found in $path,
or (by default) in the default list of possible config files.
Related objects (config, state) are created and initialised.
Mirmon always reports errors. Normally it only reports
changes (inserts/deletes) found in the mirror_list ;
in quiet mode, it doesn't. In verbose mode, it
reports progress: the startup and finishing of probes.
A Mirmon::Conf object represents a mirmon conguration.
It is normaly created by Mirmon::new().
A specified (or default) config file is read and interpreted.
A Mirmon::Mirror object represents the last known state of a mirror.
It is normaly created by Mirmon::new() from the state file,
as specified in the mirmon config file.
Mirmon::Mirror objects can be used to probe mirrors.
The probe history is a list of 's' (for success) and 'f' (for failure)
characters indicating the result of the probe. New results are appended
whenever the mirror is probed.
The state history consists of a timestamp, a '-' char, and a list of
chars indicating a past status: 's' (fresh), 'b' (oldish), 'f' (old),
'z' (bad) or 'x' (skip).
The timestamp indicates when the state history was last updated.
The current status of the mirror is determined by the mirror's age and
a few configuration parameters (min_sync, max_sync, max_poll).
The state history is updated when the mirror is probed.
If the last update of the history was less than 24 hours ago,
the last status is replaced by the current status.
If the last update of the history was more than 24 hours ago,
the current status is appended to the history.
One or more 'skip's are inserted, if the timestamp is two or more days old
(when mirmon hasn't run for more than two days).
Returns the mirror's age (in fractional days), based on the mirror's
timestamp as found by the last successful probe ; or based on the
length of the state history if no probe was ever successful.
Returns 'undef' if the mirror was never probed.
Start a probe for the mirror in non-blocking mode ;
returns the associated (IO::Handle) file handle.
The caller must maintain an association between
the handles and the mirror objects.
mirmon-2.10/mirmon.txt 0000644 0016337 0001750 00000042635 12373376740 014254 0 ustar penni101 staff MIRMON(1) User Contributed Perl Documentation MIRMON(1)
NNAAMMEE
mirmon - monitor the state of mirrors
SSYYNNOOPPSSIISS
mirmon [-v] [-q] [-t timeout] [-c conf] [-get all|update|url url]
OOPPTTIIOONNSS
--vv Be verbose ; mmiirrmmoonn normally only reports errors and changes in
the mirror list.
--qq Be quiet.
--tt _t_i_m_e_o_u_t
Set the timeout ; the default is _3_0_0.
--ggeett all | update | url
With aallll, probe all sites. With uuppddaattee, probe a selection of the
sites ; see option "max_poll" below. With uurrll, probe only the
given _u_r_l, which must appear in the mirror-list.
--cc _n_a_m_e
Use config file _n_a_m_e. The default list is
./mirmon.conf $HOME/.mirmon.conf /etc/mirmon.conf
UUSSAAGGEE
The program is intended to be run by cron every hour.
42 * * * * perl /path/to/mirmon -get update
It quietly probes a subset of the sites in a given list, writes the
results in the ’state’ file and generates a web page with the results.
The subset contains the sites that are new, bad and/or not probed for
a specified time.
When no ’get’ option is specified, the program just generates a new
web page from the last known state.
The program checks the mirrors by running a (user specified) program
on a pipe. A (user specified) number of probes is run in parallel
using nonblocking IO. When something can be read from the pipe, it
switches the pipe to blocking IO and reads one line from the pipe.
Then it flushes and closes the pipe. No attempt is made to kill the
probe.
The probe should return something that looks like
1043625600 ...
that is, a line of text starting with a timestamp. The exit status of
the probe is ignored.
CCOONNFFIIGG FFIILLEE
llooccaattiioonn
A config file can be specified with the -c option. If -c is not used,
the program looks for a config file in
· ..//mmiirrmmoonn..ccoonnff
· $$HHOOMMEE//..mmiirrmmoonn..ccoonnff
· //eettcc//mmiirrmmoonn..ccoonnff
ssyynnttaaxx
A config file looks like this :
+--------------------------------------------------
|# lines that start with '#' are comment
|# blank lines are ignored too
|# tabs are replaced by a space
|
|# the config entries are 'key' and 'value' pairs
|# a 'key' begins in column 1
|# the 'value' is the rest of the line
|somekey A_val B_val ...
|otherkey X_val Y_val ...
|
|# indented lines are glued
|# the next three lines mean 'somekey part1 part2 part3'
|somekey part1
| part2
| part3
|
|# lines starting with a '+' are concatenated
|# the next three lines mean 'somekey part1part2part3'
|somekey part1
|+ part2
|+ part3
|
|# lines starting with a '.' are glued too
|# don't use a '.' on a line by itself
|# 'somekey' gets the value "part1\n part2\n part3"
|somekey part1
|. part2
|. part3
+--------------------------------------------------
rreeqquuiirreedd eennttrriieess
project_name _n_a_m_e
Specify a short plaintext name for the project.
project_name Apache
project_name CTAN
project_url _u_r_l
Specify an url pointing to the ’home’ of the project.
project_url http://www.apache.org/
mirror_list _f_i_l_e_-_n_a_m_e
Specify the file containing the mirrors to probe.
mirror_list /path/to/mirror-list
If your mirror list is generated by a program, use
mirror_list /path/to/program arg1 ... |
Two formats are supported :
· plain : lines like
us http://www.tux.org/ [email] ...
nl http://apache.cs.uu.nl/dist/ [email] ...
nl rsync://archive.cs.uu.nl/apache-dist/ [email] ...
· apache : lines like those in the apache mirrors.list
ftp us ftp://ftp.tux.org/pub/net/apache/dist/ user@tux.org ...
http nl http://apache.cs.uu.nl/dist/ user@cs.uu.nl ...
Note that in style ’plain’ the third item is reserved for an
optional email address : the site’s contact address.
Specify the required format with option "list_style" (see below).
The default style is ’plain’.
web_page _f_i_l_e_-_n_a_m_e
Specify where the html report page is written.
icons _d_i_r_e_c_t_o_r_y_-_n_a_m_e
Specify the directory where the icons can be found, relative to
the _w_e_b___p_a_g_e, or relative to the DOCUMENTROOT of the web server.
If/when the _w_e_b___p_a_g_e lives in directory ".../mirmon/" and the
icons live in directory ".../mirmon/icons/", specify
icons icons
If/when the icons live in "/path/to/DOCUMENTROOT/icons/mirmon/",
specify
icons /icons/mirmon
probe _p_r_o_g_r_a_m _+ _a_r_g_u_m_e_n_t_s
Specify the program+args to probe the mirrors. Example:
probe /usr/bin/wget -q -O - -T %TIMEOUT% -t 1 %URL%TIME.txt
Before the program is started, %TIMEOUT% and %URL% are substituted
with the proper timeout and url values.
Here it is assumed that each hour the root server writes a
timestamp in /path/to/archive/TIME.txt, for instance with a
crontab entry like
42 * * * * perl -e 'print time, "\n"' > /path/to/archive/TIME.txt
Mirmon reads one line of output from the probe and interprets the
first word on that line as a timestamp ; for example :
1043625600
1043625600 Mon Jan 27 00:00:00 2003
1043625600 www.apache.org Mon Jan 27 00:00:00 2003
Mirmon is distributed with a program "probe" that handles ftp,
http and rsync urls.
state _f_i_l_e_-_n_a_m_e
Specify where the file containing the state is written.
The program reads this file on startup and writes the file when
mirrors are probed (-get is specified).
countries _f_i_l_e_-_n_a_m_e
Specify the file containing the country codes; The file should
contain lines like
us - United States
nl - Netherlands
The mirmon package contains a recent ISO list.
_F_a_k_e domains like _B_a_c_k_u_p, _M_a_s_t_e_r are allowed, and are listed first
in the report ; lowercase-first fake domains (like _b_a_c_k_u_p) are
listed last.
ooppttiioonnaall eennttrriieess
max_probes _n_u_m_b_e_r
Optionally specify the number of parallel probes (default 25).
timeout _s_e_c_o_n_d_s
Optionally specify the timeout for the probes (default 300).
After the last probe is started, the program waits for +
10 seconds, cleans up and exits.
project_logo _l_o_g_o
Optionally specify (the SRC of the IMG of) a logo to be placed top
right on the page.
project_logo /icons/apache.gif
project_logo http://www.apache.org/icons/...
htm_head _h_t_m_l
Optionally specify some HTML to be placed before .
htm_head
htm_top _h_t_m_l
Optionally specify some HTML to be placed near the top of the
page.
htm_top testing 1, 2, 3
htm_foot _h_t_m_l
Optionally specify HTML to be placed near the bottom of the page.
htm_foot
put_histo top|bottom|nowhere
Optionally specify where the age histogram must be placed. The
default is ’top’.
min_poll _t_i_m_e_-_s_p_e_c
For ’min_poll’ see next item. A _t_i_m_e_-_s_p_e_c is a number followed by
a unit ’s’ (seconds), or ’m’ (minutes), or ’h’ (hours), or ’d’
(days). For example ’3d’ (three days) or ’36h’ (36 hours).
max_poll _t_i_m_e_-_s_p_e_c
Optionally specify the maximum probe interval. When the program is
called with option ’-get update’, all sites are probed which are :
· new
the site appears in the list, but there is no known state
· bad
the last probe of the site was unsuccessful
· old
the last probe was more than ’max_poll’ ago.
Sites are not probed if the last probe was less than ’min_poll’
ago. So, if you specify
min_poll 4h
max_poll 12h
the ’reachable’ sites are probed twice daily and the ’unreachable’
sites are probed at most six times a day.
The default ’min_poll’ is ’1h’ (1 hour). The default ’max_poll’
is ’4h’ (4 hours).
min_sync _t_i_m_e_-_s_p_e_c
Optionally specify how often the mirrors are required to make an
update.
The default ’min_sync’ is ’1d’ (1 day).
max_sync _t_i_m_e_-_s_p_e_c
Optionally specify the maximum allowable sync interval.
Sites exceeding the limit will be considered ’old’. The default
’max_sync’ is ’2d’ (2 days).
always_get _r_e_g_i_o_n _._._.
Optionally specify a list of regions that must be probed always.
always_get Master Tier1
This is intended for _f_a_k_e _r_e_g_i_o_n_s like _M_a_s_t_e_r etc.
no_randomize
Mirmon tries to balance the probe load over the hourly mirmon
runs. If the current run has a below average number of mirrors to
probe, mirmon probes a few extra, randomly chosen mirrors, picked
from the runs that have the highest load.
If you don’t want this behaviour, use nnoo__rraannddoommiizzee.
no_add_slash
If the url part of a line in the mirror_list doesn’t end in a
slash (’/’), mirmon adds a slash and issues a warning unless it is
in quiet mode.
If you don’t want this behaviour, use nnoo__aadddd__ssllaasshh.
list_style plain|apache
Optionally specify the format (’plain’ or ’apache’) of the mirror-
list.
See the description of ’mirror_list’ above. The default
list_style is ’plain’.
site_url _s_i_t_e _u_r_l
Optionally specify a substitute url for a site.
When access to a site is restricted (in Australia, for instance),
another (sometimes secret) url can be used to probe the site. The
of an url is the part between ’://’ and the first ’/’.
env _k_e_y _v_a_l_u_e
Optionally specify an environment variable.
include _f_i_l_e_-_n_a_m_e
Optionally specify a file to include.
The specified file is processed ’in situ’. After the specified
file is read and processed, config processing is resumed in the
file where the "include" was encountered. The include depth is
unlimited. However, it is a fatal error to include a file twice
under the same name.
show
When the config processor encounters the ’show’ command, it dumps
the content of the current config to standout, if option "-v" is
specified. This is intented for debugging.
exit
When the config processor encounters the ’exit’ command, it
terminates the program. This is intented for debugging.
SSTTAATTEE FFIILLEE FFOORRMMAATT
The state file consists of lines; one line per site. Each line
consists of white space separated fields. The seven fields are :
· field 1 : url
The url as given in the mirror list.
· field 2 : age
The mirror’s timestamp found by the last successful probe, or
’undef’ if no probe was ever successful.
· field 3 : status last probe
The status of the last probe, or ’undef’ if the mirror was never
probed.
· field 4 : time last successful probe
The timestamp of the last successful probe or ’undef’ if the
mirror was never successfully probed.
· field 5 : probe history
The probe history is a list of ’s’ (for success) and ’f’ (for
failure) characters indicating the result of the probe. New
results are appended whenever the mirror is probed.
· field 6 : state history
The state history consists of a timestamp, a ’-’ char, and a list
of chars indicating a past status: ’s’ (fresh), ’b’ (oldish), ’f’
(old), ’z’ (bad) or ’x’ (skip). The timestamp indicates when the
state history was last updated. The current status of the mirror
is determined by the mirror’s age and a few configuration
parameters (min_sync, max_sync, max_poll). The state history is
updated when the mirror is probed. If the last update of the
history was less than 24 hours ago, the last status is replaced by
the current status. If the last update of the history was more
than 24 hours ago, the current status is appended to the history.
One or more ’skip’s is inserted, if the timestamp is two or more
days old (when mirmon hasn’t run for more than two days).
· field 7 : last probe
The timestamp of the last probe, or ’undef’ if the mirror was
never probed.
IINNSSTTAALLLLAATTIIOONN
ggeenneerraall
· Note: The (empty) state file must exist before mirmon runs.
· The mirmon repository is here :
https://svn.science.uu.nl/repos/project.mirmon/trunk/
· The mirmon tarball is here :
http://www.staff.science.uu.nl/~penni101/mirmon/mirmon.tar.gz
iinnssttaallllaattiioonn ssuuggggeessttiioonnss
To install and configure mirmon, take the following steps :
· First, make the webdir :
cd DOCUMENTROOT
mkdir mirmon
For _D_O_C_U_M_E_N_T_R_O_O_T, substitute the full pathname of the document root
of your webserver.
· Check out the mirmon repository :
cd /usr/local/src
svn checkout REPO mirmon
where
REPO = https://svn.science.uu.nl/repos/project.mirmon/trunk/
or download the package and unpack it.
· Chdir to directory mirmon :
cd mirmon
· Create the (empty) state file :
touch state.txt
· Install the icons in the webdir :
mkdir DOCUMENTROOT/mirmon/icons
cp icons/* DOCUMENTROOT/mirmon/icons
· Create a mirror list "mirror_list" ;
Use your favorite editor, or genererate the list from an existing
database.
nl http://archive.cs.uu.nl/your-project/ contact@cs.uu.nl
uk http://mirrors.this.org/your-project/ mirrors@this.org
us http://mirrors.that.org/your-project/ mirrors@that.org
The email addresses are optional.
· Create a mirmon config file "mirmon.conf" with your favorite editor.
# lines must start in the first column ; no leading white space
project_name ....
project_url ....
mirror_list mirror_list
state state.txt
countries countries.list
web_page DOCUMENTROOT/mirmon/index.html
icons /mirmon/icons
probe /usr/bin/wget -q -O - -T %TIMEOUT% -t 1 %URL%TIME.txt
This assumes the project’s timestamp is in file "TIME.txt".
· If you have rsync urls, change the probe line to :
probe perl /usr/local/src/mirmon/probe -t %TIMEOUT% %URL%TIME.txt
· Run mirmon :
perl mirmon -v -get all
The mirmon report should now be in ’DOCUMENTROOT/mirmon/index.html’
http://www.your.project.org/mirmon/
· If/when, at a later date, you want to upgrade mirmon :
cd /usr/local/src/mirmon
svn status -u
svn up
SSEEEE AALLSSOO
mirmon.pm(3)
AAUUTTHHOORR
(c) 2003-2014 Henk P. Penning
Faculty of Science, Utrecht University
http://www.staff.science.uu.nl/~penni101/ -- penning@uu.nl
mirmon-2.10 - Fri Aug 15 12:26:55 2014 ; henkp
perl v5.8.5 2014-08-15 MIRMON(1)
mirmon-2.10/LICENSE 0000644 0016337 0001750 00000002342 12373376740 013206 0 ustar penni101 staff # Copyright (c) 2003-2014 Henk Penning, all rights reserved.
# penning@uu.nl, http://www.staff.science.uu.nl/~penni101/
# Version 1.1 was donated to the Apache Software Foundation 2003 Jan 28.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
mirmon-2.10/VERSION 0000644 0016337 0001750 00000000014 12373376740 013243 0 ustar penni101 staff mirmon-2.10
mirmon-2.10/countries.list 0000644 0016337 0001750 00000010133 12373376740 015106 0 ustar penni101 staff # based on : http://www.iso.org/iso/list-en1-semic-3.txt
ad - Andorra
ae - United Arab Emirates
af - Afghanistan
ag - Antigua and Barbuda
ai - Anguilla
al - Albania
am - Armenia
ao - Angola
aq - Antarctica
ar - Argentina
as - American Samoa
at - Austria
au - Australia
aw - Aruba
ax - Åland Islands
az - Azerbaijan
ba - Bosnia and Herzegovina
bb - Barbados
bd - Bangladesh
be - Belgium
bf - Burkina Faso
bg - Bulgaria
bh - Bahrain
bi - Burundi
bj - Benin
bl - Saint Barthélemy
bm - Bermuda
bn - Brunei Darussalam
bo - Bolivia
bq - Bonaire, Sint Eustatius and Saba
br - Brazil
bs - Bahamas
bt - Bhutan
bv - Bouvet Island
bw - Botswana
by - Belarus
bz - Belize
ca - Canada
cc - Cocos (Keeling) Islands
cd - Congo
cf - Central African Republic
cg - Congo
ch - Switzerland
ci - Côte d'Ivoire
ck - Cook Islands
cl - Chile
cm - Cameroon
cn - China
co - Colombia
cr - Costa Rica
cu - Cuba
cv - Cabo Verde
cw - Curaçao
cx - Christmas Island
cy - Cyprus
cz - Czech Republic
de - Germany
dj - Djibouti
dk - Denmark
dm - Dominica
do - Dominican Republic
dz - Algeria
ec - Ecuador
ee - Estonia
eg - Egypt
eh - Western Sahara
er - Eritrea
es - Spain
et - Ethiopia
fi - Finland
fj - Fiji
fk - Falkland Islands (Malvinas)
fm - Micronesia, Federated States of
fo - Faroe Islands
fr - France
ga - Gabon
gd - Grenada
ge - Georgia
gf - French Guiana
gg - Guernsey
gh - Ghana
gi - Gibraltar
gl - Greenland
gm - Gambia
gn - Guinea
gp - Guadeloupe
gq - Equatorial Guinea
gr - Greece
gs - South Georgia and the South Sandwich Islands
gt - Guatemala
gu - Guam
gw - Guinea-Bissau
gy - Guyana
hk - Hong Kong
hm - Heard Island and Mcdonald Islands
hn - Honduras
hr - Croatia
ht - Haiti
hu - Hungary
id - Indonesia
ie - Ireland
il - Israel
im - Isle of Man
in - India
io - British Indian Ocean Territory
iq - Iraq
ir - Iran
is - Iceland
it - Italy
je - Jersey
jm - Jamaica
jo - Jordan
jp - Japan
ke - Kenya
kg - Kyrgyzstan
kh - Cambodia
ki - Kiribati
km - Comoros
kn - Saint Kitts and Nevis
kp - Korea, Democratic People's Republic of
kr - Korea, Republic of
kw - Kuwait
ky - Cayman Islands
kz - Kazakhstan
la - Lao People's Democratic Republic
lb - Lebanon
lc - Saint Lucia
li - Liechtenstein
lk - Sri Lanka
lr - Liberia
ls - Lesotho
lt - Lithuania
lu - Luxembourg
lv - Latvia
ly - Libya
ma - Morocco
mc - Monaco
md - Moldova
me - Montenegro
mf - Saint Martin (French Part)
mg - Madagascar
mh - Marshall Islands
mk - Macedonia
ml - Mali
mm - Myanmar
mn - Mongolia
mo - Macao
mp - Northern Mariana Islands
mq - Martinique
mr - Mauritania
ms - Montserrat
mt - Malta
mu - Mauritius
mv - Maldives
mw - Malawi
mx - Mexico
my - Malaysia
mz - Mozambique
na - Namibia
nc - New Caledonia
ne - Niger
nf - Norfolk Island
ng - Nigeria
ni - Nicaragua
nl - Netherlands
no - Norway
np - Nepal
nr - Nauru
nu - Niue
nz - New Zealand
om - Oman
pa - Panama
pe - Peru
pf - French Polynesia
pg - Papua New Guinea
ph - Philippines
pk - Pakistan
pl - Poland
pm - Saint Pierre and Miquelon
pn - Pitcairn
pr - Puerto Rico
ps - Palestine
pt - Portugal
pw - Palau
py - Paraguay
qa - Qatar
re - Réunion
ro - Romania
rs - Serbia
ru - Russian Federation
rw - Rwanda
sa - Saudi Arabia
sb - Solomon Islands
sc - Seychelles
sd - Sudan
se - Sweden
sg - Singapore
sh - Saint Helena, Ascension and Tristan da Cunha
si - Slovenia
sj - Svalbard and Jan Mayen
sk - Slovakia
sl - Sierra Leone
sm - San Marino
sn - Senegal
so - Somalia
sr - Suriname
ss - South Sudan
st - Sao Tome and Principe
sv - El Salvador
sx - Sint Maarten (Dutch Part)
sy - Syrian Arab Republic
sz - Swaziland
tc - Turks and Caicos Islands
td - Chad
tf - French Southern Territories
tg - Togo
th - Thailand
tj - Tajikistan
tk - Tokelau
tl - Timor-Leste
tm - Turkmenistan
tn - Tunisia
to - Tonga
tr - Turkey
tt - Trinidad and Tobago
tv - Tuvalu
tw - Taiwan
tz - Tanzania
ua - Ukraine
ug - Uganda
uk - United Kingdom
um - United States Minor Outlying Islands
us - United States
uy - Uruguay
uz - Uzbekistan
va - Holy See (Vatican City State)
vc - Saint Vincent and the Grenadines
ve - Venezuela
vg - Virgin Islands, British
vi - Virgin Islands, U.S.
vn - Viet Nam
vu - Vanuatu
wf - Wallis and Futuna
ws - Samoa
ye - Yemen
yt - Mayotte
za - South Africa
zm - Zambia
zw - Zimbabwe
mirmon-2.10/mirmon.pm.txt 0000644 0016337 0001750 00000017503 12373376740 014663 0 ustar penni101 staff Mirmon(3) User Contributed Perl Documentation Mirmon(3)
NNAAMMEE
Mirmon - OO interface for mirmon objects
SSYYNNOOPPSSIISS
use Mirmon ;
$m = Mirmon -> new ( [ $path-to-config ] )
$conf = $m -> conf ; # a Mirmon::Conf object
$state = $m -> state ; # the mirmon state
for my $url ( keys %$state )
{ $mirror = $state -> { $url } ; # a Mirmon::Mirror object
$mail = $mirror -> mail ; # contact address
$mirror -> age ( time ) ; # set mirror age
}
Many class and object methods can be used to get or set attributes :
$object -> attribute # get an atttibute
$object -> attribute ( $attr ) # set an atttibute
MMiirrmmoonn ccllaassss mmeetthhooddss
nneeww (( [[$$ppaatthh]] ))
Create a Mirmon object from a config file found in $path, or (by
default) in the default list of possible config files. Related
objects (config, state) are created and initialised.
verbosity
Mirmon always reports errors. Normally it only reports changes
(inserts/deletes) found in the mirror_list ; in _q_u_i_e_t mode, it
doesn’t. In _v_e_r_b_o_s_e mode, it reports progress: the startup and
finishing of probes.
Mirmon::verbose ( [ $bool ] ) # get/set verbose
Mirmon::quiet ( [ $bool ] ) # get/set quiet
Mirmon::debug ( [ $bool ] ) # get/set debug
MMiirrmmoonn oobbjjeecctt mmeetthhooddss
ccoonnff
Returns Mirmon’s Mirmon::Conf object.
ssttaattee
Returns a hashref "{ url => mirror, ... }", where _u_r_l is as
specified in the mirror list and _m_i_r_r_o_r is a Mirmon::Mirror
object.
rreeggiioonnss
Returns a hashref "{ country_code => country_name, ... }".
ccoonnffiigg__lliisstt
Returns the list of default locations for config files.
ggeett__ddaatteess (( $$ggeett [[,, _$$_UU_RR_LL]] ))
Probes all mirrors if $get is "all" ; or a subset if $get is
"update" ; or only _$_U_R_L if $get is "url".
MMiirrmmoonn::::CCoonnff oobbjjeecctt mmeetthhooddss
A Mirmon::Conf object represents a mirmon conguration. It is normaly
created by _M_i_r_m_o_n_:_:_n_e_w_(_). A specified (or default) config file is
read and interpreted.
attribute methods
For every config file entry, there is an attribute method :
wweebb__ppaaggee, ssttaattee, ccoouunnttrriieess, mmiirrrroorr__lliisstt, pprroobbee, pprroojjeecctt__nnaammee,
pprroojjeecctt__uurrll, iiccoonnss, pprroojjeecctt__llooggoo, ttiimmeeoouutt, mmaaxx__pprroobbeess, mmiinn__ppoollll,
mmaaxx__ppoollll, mmiinn__ssyynncc, mmaaxx__ssyynncc, lliisstt__ssttyyllee, ppuutt__hhiissttoo, rraannddoommiizzee,
aadddd__ssllaasshh.
rroooott
Returns the file name of (the root of) the configuration file(s).
ssiittee__uurrll
Returns a hashref "{ site => url, ... }", as specified in the
mirmon config file.
MMiirrmmoonn::::MMiirrrroorr oobbjjeecctt mmeetthhooddss
A Mirmon::Mirror object represents the last known state of a mirror.
It is normaly created by _M_i_r_m_o_n_:_:_n_e_w_(_) from the state file, as
specified in the mirmon config file. Mirmon::Mirror objects can be
used to probe mirrors.
aattttrriibbuuttee mmeetthhooddss
uurrll The url as given in the mirror list.
aaggee The mirror’s timestamp found by the last successful probe, or
’undef’ if no probe was ever successful.
llaasstt__ssttaattuuss
The status of the last probe, or ’undef’ if the mirror was never
probed.
llaasstt__ookk__pprroobbee
The timestamp of the last successful probe or ’undef’ if the
mirror was never successfully probed.
pprroobbee__hhiissttoorryy
The probe history is a list of ’s’ (for success) and ’f’ (for
failure) characters indicating the result of the probe. New
results are appended whenever the mirror is probed.
ssttaattee__hhiissttoorryy
The state history consists of a timestamp, a ’-’ char, and a list
of chars indicating a past status: ’s’ (fresh), ’b’ (oldish), ’f’
(old), ’z’ (bad) or ’x’ (skip). The timestamp indicates when the
state history was last updated. The current status of the mirror
is determined by the mirror’s age and a few configuration
parameters (min_sync, max_sync, max_poll). The state history is
updated when the mirror is probed. If the last update of the
history was less than 24 hours ago, the last status is replaced by
the current status. If the last update of the history was more
than 24 hours ago, the current status is appended to the history.
One or more ’skip’s are inserted, if the timestamp is two or more
days old (when mirmon hasn’t run for more than two days).
llaasstt__pprroobbee
The timestamp of the last probe, or ’undef’ if the mirror was
never probed.
oobbjjeecctt mmeetthhooddss
mmiirrmmoonn
Returns the parent Mirmon object.
ssttaattee__hhiissttoorryy__ttiimmee
Returns the _t_i_m_e part of the state_history attribute.
ssttaattee__hhiissttoorryy__hhiisstt
Returns the _h_i_s_t_o_r_y part of the state_history attribute.
ttyyppee, ssiittee, hhoommee
For an url like _f_t_p_:_/_/_w_w_w_._s_o_m_e_._o_r_g_/_p_a_t_h_/_t_o_/_h_o_m_e, the ttyyppee is _f_t_p,
the ssiittee is _w_w_w_._s_o_m_e_._o_r_g, and hhoommee is _f_t_p_:_/_/_w_w_w_._s_o_m_e_._o_r_g_/.
aaggee__iinn__ddaayyss
Returns the mirror’s age (in fractional days), based on the
mirror’s timestamp as found by the last successful probe ; or
based on the length of the state history if no probe was ever
successful. Returns ’undef’ if the mirror was never probed.
mmaaiill
Returns the mirror’s contact address as specified in the mirror
list.
rreeggiioonn
Returns the mirror’s country code as specified in the mirror list.
ssttaarrtt__pprroobbee
Start a probe for the mirror in non-blocking mode ; returns the
associated (IO::Handle) file handle. The caller must maintain an
association between the handles and the mirror objects.
ffiinniisshh__pprroobbee (( $$hhaannddllee ))
Sets the (IO::Handle) $$hhaannddllee to blocking IO ; reads a result from
the handle, and updates the state of the mirror.
SSEEEE AALLSSOO
mirmon(1)
AAUUTTHHOORR
(c) 2003-2014 Henk P. Penning
Faculty of Science, Utrecht University
http://www.staff.science.uu.nl/~penni101/ -- penning@uu.nl
mirmon-2.10 - Fri Aug 15 12:26:55 2014 ; henkp
perl v5.8.5 2014-08-15 Mirmon(3)
mirmon-2.10/mirmon.1 0000644 0016337 0001750 00000050562 12373376740 013573 0 ustar penni101 staff .\" Automatically generated by Pod::Man 2.26 (Pod::Simple 3.23)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings. \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote. \*(C+ will
.\" give a nicer C++. Capital omega is used to do unbreakable dashes and
.\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
. ds -- \(*W-
. ds PI pi
. if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
. if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch
. ds L" ""
. ds R" ""
. ds C` ""
. ds C' ""
'br\}
.el\{\
. ds -- \|\(em\|
. ds PI \(*p
. ds L" ``
. ds R" ''
. ds C`
. ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el .ds Aq '
.\"
.\" If the F register is turned on, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD. Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.nr rF 0
.if \n(.g .if rF .nr rF 1
.if (\n(rF:(\n(.g==0)) \{
. if \nF \{
. de IX
. tm Index:\\$1\t\\n%\t"\\$2"
..
. if !\nF==2 \{
. nr % 0
. nr F 2
. \}
. \}
.\}
.rr rF
.\"
.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
.\" Fear. Run. Save yourself. No user-serviceable parts.
. \" fudge factors for nroff and troff
.if n \{\
. ds #H 0
. ds #V .8m
. ds #F .3m
. ds #[ \f1
. ds #] \fP
.\}
.if t \{\
. ds #H ((1u-(\\\\n(.fu%2u))*.13m)
. ds #V .6m
. ds #F 0
. ds #[ \&
. ds #] \&
.\}
. \" simple accents for nroff and troff
.if n \{\
. ds ' \&
. ds ` \&
. ds ^ \&
. ds , \&
. ds ~ ~
. ds /
.\}
.if t \{\
. ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u"
. ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'
. ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'
. ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'
. ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'
. ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'
.\}
. \" troff and (daisy-wheel) nroff accents
.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'
.ds 8 \h'\*(#H'\(*b\h'-\*(#H'
.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#]
.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'
.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'
.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#]
.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#]
.ds ae a\h'-(\w'a'u*4/10)'e
.ds Ae A\h'-(\w'A'u*4/10)'E
. \" corrections for vroff
.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'
.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'
. \" for low resolution devices (crt and lpr)
.if \n(.H>23 .if \n(.V>19 \
\{\
. ds : e
. ds 8 ss
. ds o a
. ds d- d\h'-1'\(ga
. ds D- D\h'-1'\(hy
. ds th \o'bp'
. ds Th \o'LP'
. ds ae ae
. ds Ae AE
.\}
.rm #[ #] #H #V #F C
.\" ========================================================================
.\"
.IX Title "MIRMON 1"
.TH MIRMON 1 "2014-08-15" "perl v5.8.5" "User Contributed Perl Documentation"
.\" For nroff, turn off justification. Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
mirmon \- monitor the state of mirrors
.SH "SYNOPSIS"
.IX Header "SYNOPSIS"
.Vb 1
\& mirmon [\-v] [\-q] [\-t timeout] [\-c conf] [\-get all|update|url url]
.Ve
.SH "OPTIONS"
.IX Header "OPTIONS"
.IP "\fB\-v\fR" 4
.IX Item "-v"
Be verbose ; \fBmirmon\fR normally only reports
errors and changes in the mirror list.
.IP "\fB\-q\fR" 4
.IX Item "-q"
Be quiet.
.IP "\fB\-t\fR \fItimeout\fR" 4
.IX Item "-t timeout"
Set the timeout ; the default is \fI300\fR.
.IP "\fB\-get\fR all | update | url " 4
.IX Item "-get all | update | url "
With \fBall\fR, probe all sites.
With \fBupdate\fR, probe a selection of the sites ; see option \f(CW\*(C`max_poll\*(C'\fR below.
With \fBurl\fR, probe only the given \fIurl\fR, which must appear in the mirror-list.
.IP "\fB\-c\fR \fIname\fR" 4
.IX Item "-c name"
Use config file \fIname\fR. The default list is
.Sp
.Vb 1
\& ./mirmon.conf $HOME/.mirmon.conf /etc/mirmon.conf
.Ve
.SH "USAGE"
.IX Header "USAGE"
The program is intended to be run by cron every hour.
.PP
.Vb 1
\& 42 * * * * perl /path/to/mirmon \-get update
.Ve
.PP
It quietly probes a subset of the sites in a given list,
writes the results in the 'state' file and generates a web page
with the results. The subset contains the sites that are new, bad
and/or not probed for a specified time.
.PP
When no 'get' option is specified, the program just generates a
new web page from the last known state.
.PP
The program checks the mirrors by running a (user specified)
program on a pipe. A (user specified) number of probes is
run in parallel using nonblocking \s-1IO\s0. When something can be
read from the pipe, it switches the pipe to blocking \s-1IO\s0 and
reads one line from the pipe. Then it flushes and closes the
pipe. No attempt is made to kill the probe.
.PP
The probe should return something that looks like
.PP
.Vb 1
\& 1043625600 ...
.Ve
.PP
that is, a line of text starting with a timestamp. The exit status
of the probe is ignored.
.SH "CONFIG FILE"
.IX Header "CONFIG FILE"
.SS "location"
.IX Subsection "location"
A config file can be specified with the \-c option.
If \-c is not used, the program looks for a config file in
.IP "\(bu" 4
\&\fB./mirmon.conf\fR
.IP "\(bu" 4
\&\fB\f(CB$HOME\fB/.mirmon.conf\fR
.IP "\(bu" 4
\&\fB/etc/mirmon.conf\fR
.SS "syntax"
.IX Subsection "syntax"
A config file looks like this :
.PP
.Vb 10
\& +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
\& |# lines that start with \*(Aq#\*(Aq are comment
\& |# blank lines are ignored too
\& |# tabs are replaced by a space
\& |
\& |# the config entries are \*(Aqkey\*(Aq and \*(Aqvalue\*(Aq pairs
\& |# a \*(Aqkey\*(Aq begins in column 1
\& |# the \*(Aqvalue\*(Aq is the rest of the line
\& |somekey A_val B_val ...
\& |otherkey X_val Y_val ...
\& |
\& |# indented lines are glued
\& |# the next three lines mean \*(Aqsomekey part1 part2 part3\*(Aq
\& |somekey part1
\& | part2
\& | part3
\& |
\& |# lines starting with a \*(Aq+\*(Aq are concatenated
\& |# the next three lines mean \*(Aqsomekey part1part2part3\*(Aq
\& |somekey part1
\& |+ part2
\& |+ part3
\& |
\& |# lines starting with a \*(Aq.\*(Aq are glued too
\& |# don\*(Aqt use a \*(Aq.\*(Aq on a line by itself
\& |# \*(Aqsomekey\*(Aq gets the value "part1\en part2\en part3"
\& |somekey part1
\& |. part2
\& |. part3
\& +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
.Ve
.SS "required entries"
.IX Subsection "required entries"
.IP "project_name \fIname\fR" 4
.IX Item "project_name name"
Specify a short plaintext name for the project.
.Sp
.Vb 2
\& project_name Apache
\& project_name CTAN
.Ve
.IP "project_url \fIurl\fR" 4
.IX Item "project_url url"
Specify an url pointing to the 'home' of the project.
.Sp
.Vb 1
\& project_url http://www.apache.org/
.Ve
.IP "mirror_list \fIfile-name\fR" 4
.IX Item "mirror_list file-name"
Specify the file containing the mirrors to probe.
.Sp
.Vb 1
\& mirror_list /path/to/mirror\-list
.Ve
.Sp
If your mirror list is generated by a program, use
.Sp
.Vb 1
\& mirror_list /path/to/program arg1 ... |
.Ve
.Sp
Two formats are supported :
.RS 4
.IP "\(bu" 4
plain : lines like
.Sp
.Vb 3
\& us http://www.tux.org/ [email] ...
\& nl http://apache.cs.uu.nl/dist/ [email] ...
\& nl rsync://archive.cs.uu.nl/apache\-dist/ [email] ...
.Ve
.IP "\(bu" 4
apache : lines like those in the apache mirrors.list
.Sp
.Vb 2
\& ftp us ftp://ftp.tux.org/pub/net/apache/dist/ user@tux.org ...
\& http nl http://apache.cs.uu.nl/dist/ user@cs.uu.nl ...
.Ve
.RE
.RS 4
.Sp
Note that in style 'plain' the third item is reserved for an
optional email address : the site's contact address.
.Sp
Specify the required format with option \f(CW\*(C`list_style\*(C'\fR (see below).
The default style is 'plain'.
.RE
.IP "web_page \fIfile-name\fR" 4
.IX Item "web_page file-name"
Specify where the html report page is written.
.IP "icons \fIdirectory-name\fR" 4
.IX Item "icons directory-name"
Specify the directory where the icons can be found,
relative to the \fIweb_page\fR, or relative to the
\&\s-1DOCUMENTROOT\s0 of the web server.
.Sp
If/when the \fIweb_page\fR lives in directory \f(CW\*(C`.../mirmon/\*(C'\fR and
the icons live in directory \f(CW\*(C`.../mirmon/icons/\*(C'\fR,
specify
.Sp
.Vb 1
\& icons icons
.Ve
.Sp
If/when the icons live in \f(CW\*(C`/path/to/DOCUMENTROOT/icons/mirmon/\*(C'\fR, specify
.Sp
.Vb 1
\& icons /icons/mirmon
.Ve
.IP "probe \fIprogram + arguments\fR" 4
.IX Item "probe program + arguments"
Specify the program+args to probe the mirrors. Example:
.Sp
.Vb 1
\& probe /usr/bin/wget \-q \-O \- \-T %TIMEOUT% \-t 1 %URL%TIME.txt
.Ve
.Sp
Before the program is started, \f(CW%TIMEOUT\fR% and \f(CW%URL\fR% are
substituted with the proper timeout and url values.
.Sp
Here it is assumed that each hour the root server writes
a timestamp in /path/to/archive/TIME.txt, for instance with
a crontab entry like
.Sp
.Vb 1
\& 42 * * * * perl \-e \*(Aqprint time, "\en"\*(Aq > /path/to/archive/TIME.txt
.Ve
.Sp
Mirmon reads one line of output from the probe and interprets
the first word on that line as a timestamp ; for example :
.Sp
.Vb 3
\& 1043625600
\& 1043625600 Mon Jan 27 00:00:00 2003
\& 1043625600 www.apache.org Mon Jan 27 00:00:00 2003
.Ve
.Sp
Mirmon is distributed with a program \f(CW\*(C`probe\*(C'\fR that handles
ftp, http and rsync urls.
.IP "state \fIfile-name\fR" 4
.IX Item "state file-name"
Specify where the file containing the state is written.
.Sp
The program reads this file on startup and writes the
file when mirrors are probed (\-get is specified).
.IP "countries \fIfile-name\fR" 4
.IX Item "countries file-name"
Specify the file containing the country codes;
The file should contain lines like
.Sp
.Vb 2
\& us \- United States
\& nl \- Netherlands
.Ve
.Sp
The mirmon package contains a recent \s-1ISO\s0 list.
.Sp
\&\fIFake\fR domains like \fIBackup\fR, \fIMaster\fR are allowed,
and are listed first in the report ; lowercase-first
fake domains (like \fIbackup\fR) are listed last.
.SS "optional entries"
.IX Subsection "optional entries"
.IP "max_probes \fInumber\fR" 4
.IX Item "max_probes number"
Optionally specify the number of parallel probes (default 25).
.IP "timeout \fIseconds\fR" 4
.IX Item "timeout seconds"
Optionally specify the timeout for the probes (default 300).
.Sp
After the last probe is started, the program waits for
+ 10 seconds, cleans up and exits.
.IP "project_logo \fIlogo\fR" 4
.IX Item "project_logo logo"
Optionally specify (the \s-1SRC\s0 of the \s-1IMG\s0 of) a logo to be placed
top right on the page.
.Sp
.Vb 2
\& project_logo /icons/apache.gif
\& project_logo http://www.apache.org/icons/...
.Ve
.IP "htm_head \fIhtml\fR" 4
.IX Item "htm_head html"
Optionally specify some \s-1HTML\s0 to be placed before .
.Sp
.Vb 2
\& htm_head
\&
.Ve
.IP "htm_top \fIhtml\fR" 4
.IX Item "htm_top html"
Optionally specify some \s-1HTML\s0 to be placed near the top of the page.
.Sp
.Vb 1
\& htm_top testing 1, 2, 3
.Ve
.IP "htm_foot \fIhtml\fR" 4
.IX Item "htm_foot html"
Optionally specify \s-1HTML\s0 to be placed near the bottom of the page.
.Sp
.Vb 4
\& htm_foot
\&
\&
\&
.Ve
.IP "put_histo top|bottom|nowhere" 4
.IX Item "put_histo top|bottom|nowhere"
Optionally specify where the age histogram must be placed.
The default is 'top'.
.IP "min_poll \fItime-spec\fR" 4
.IX Item "min_poll time-spec"
For 'min_poll' see next item. A \fItime-spec\fR is a number followed by
a unit 's' (seconds), or 'm' (minutes), or 'h' (hours), or 'd' (days).
For example '3d' (three days) or '36h' (36 hours).
.IP "max_poll \fItime-spec\fR" 4
.IX Item "max_poll time-spec"
Optionally specify the maximum probe interval. When the program is
called with option '\-get update', all sites are probed which are :
.RS 4
.IP "\(bu" 4
new
.Sp
the site appears in the list, but there is no known state
.IP "\(bu" 4
bad
.Sp
the last probe of the site was unsuccessful
.IP "\(bu" 4
old
.Sp
the last probe was more than 'max_poll' ago.
.RE
.RS 4
.Sp
Sites are not probed if the last probe was less than 'min_poll' ago.
So, if you specify
.Sp
.Vb 2
\& min_poll 4h
\& max_poll 12h
.Ve
.Sp
the 'reachable' sites are probed twice daily and the 'unreachable'
sites are probed at most six times a day.
.Sp
The default 'min_poll' is '1h' (1 hour).
The default 'max_poll' is '4h' (4 hours).
.RE
.IP "min_sync \fItime-spec\fR" 4
.IX Item "min_sync time-spec"
Optionally specify how often the mirrors are required to make an update.
.Sp
The default 'min_sync' is '1d' (1 day).
.IP "max_sync \fItime-spec\fR" 4
.IX Item "max_sync time-spec"
Optionally specify the maximum allowable sync interval.
.Sp
Sites exceeding the limit will be considered 'old'.
The default 'max_sync' is '2d' (2 days).
.IP "always_get \fIregion ...\fR" 4
.IX Item "always_get region ..."
Optionally specify a list of regions that must be probed always.
.Sp
.Vb 1
\& always_get Master Tier1
.Ve
.Sp
This is intended for \fIfake regions\fR like \fIMaster\fR etc.
.IP "no_randomize" 4
.IX Item "no_randomize"
Mirmon tries to balance the probe load over the hourly mirmon runs.
If the current run has a below average number of mirrors to probe,
mirmon probes a few extra, randomly chosen mirrors, picked from the
runs that have the highest load.
.Sp
If you don't want this behaviour, use \fBno_randomize\fR.
.IP "no_add_slash" 4
.IX Item "no_add_slash"
If the url part of a line in the mirror_list doesn't end
in a slash ('/'), mirmon adds a slash and issues a warning
unless it is in quiet mode.
.Sp
If you don't want this behaviour, use \fBno_add_slash\fR.
.IP "list_style plain|apache" 4
.IX Item "list_style plain|apache"
Optionally specify the format ('plain' or 'apache') of the mirror-list.
.Sp
See the description of 'mirror_list' above.
The default list_style is 'plain'.
.IP "site_url \fIsite\fR \fIurl\fR" 4
.IX Item "site_url site url"
Optionally specify a substitute url for a site.
.Sp
When access to a site is restricted (in Australia, for instance),
another (sometimes secret) url can be used to probe the site.
The of an url is the part between '://' and the first '/'.
.IP "env \fIkey\fR \fIvalue\fR" 4
.IX Item "env key value"
Optionally specify an environment variable.
.IP "include \fIfile-name\fR" 4
.IX Item "include file-name"
Optionally specify a file to include.
.Sp
The specified file is processed 'in situ'. After the specified file is
read and processed, config processing is resumed in the file where the
\&\f(CW\*(C`include\*(C'\fR was encountered.
The include depth is unlimited. However, it is a fatal error to
include a file twice under the same name.
.IP "show" 4
.IX Item "show"
When the config processor encounters the 'show' command, it
dumps the content of the current config to standout, if option
\&\f(CW\*(C`\-v\*(C'\fR is specified. This is intented for debugging.
.IP "exit" 4
.IX Item "exit"
When the config processor encounters the 'exit' command, it
terminates the program. This is intented for debugging.
.SH "STATE FILE FORMAT"
.IX Header "STATE FILE FORMAT"
The state file consists of lines; one line per site.
Each line consists of white space separated fields.
The seven fields are :
.IP "\(bu" 4
field 1 : url
.Sp
The url as given in the mirror list.
.IP "\(bu" 4
field 2 : age
.Sp
The mirror's timestamp found by the last successful probe,
or 'undef' if no probe was ever successful.
.IP "\(bu" 4
field 3 : status last probe
.Sp
The status of the last probe, or 'undef' if the mirror was never probed.
.IP "\(bu" 4
field 4 : time last successful probe
.Sp
The timestamp of the last successful probe or 'undef'
if the mirror was never successfully probed.
.IP "\(bu" 4
field 5 : probe history
.Sp
The probe history is a list of 's' (for success) and 'f' (for failure)
characters indicating the result of the probe. New results are appended
whenever the mirror is probed.
.IP "\(bu" 4
field 6 : state history
.Sp
The state history consists of a timestamp, a '\-' char, and a list of
chars indicating a past status: 's' (fresh), 'b' (oldish), 'f' (old),
\&'z' (bad) or 'x' (skip).
The timestamp indicates when the state history was last updated.
The current status of the mirror is determined by the mirror's age and
a few configuration parameters (min_sync, max_sync, max_poll).
The state history is updated when the mirror is probed.
If the last update of the history was less than 24 hours ago,
the last status is replaced by the current status.
If the last update of the history was more than 24 hours ago,
the current status is appended to the history.
One or more 'skip's is inserted, if the timestamp is two or more days old
(when mirmon hasn't run for more than two days).
.IP "\(bu" 4
field 7 : last probe
.Sp
The timestamp of the last probe, or 'undef' if the mirror was never probed.
.SH "INSTALLATION"
.IX Header "INSTALLATION"
.SS "general"
.IX Subsection "general"
.IP "\(bu" 4
Note: The (empty) state file must exist before mirmon runs.
.IP "\(bu" 4
The mirmon repository is here :
.Sp
.Vb 1
\& https://svn.science.uu.nl/repos/project.mirmon/trunk/
.Ve
.IP "\(bu" 4
The mirmon tarball is here :
.Sp
.Vb 1
\& http://www.staff.science.uu.nl/~penni101/mirmon/mirmon.tar.gz
.Ve
.SS "installation suggestions"
.IX Subsection "installation suggestions"
To install and configure mirmon, take the following steps :
.IP "\(bu" 2
First, make the webdir :
.Sp
.Vb 2
\& cd DOCUMENTROOT
\& mkdir mirmon
.Ve
.Sp
For \fI\s-1DOCUMENTROOT\s0\fR, substitute the full pathname
of the document root of your webserver.
.IP "\(bu" 2
Check out the mirmon repository :
.Sp
.Vb 2
\& cd /usr/local/src
\& svn checkout REPO mirmon
.Ve
.Sp
where
.Sp
.Vb 1
\& REPO = https://svn.science.uu.nl/repos/project.mirmon/trunk/
.Ve
.Sp
or download the package and unpack it.
.IP "\(bu" 2
Chdir to directory mirmon :
.Sp
.Vb 1
\& cd mirmon
.Ve
.IP "\(bu" 2
Create the (empty) state file :
.Sp
.Vb 1
\& touch state.txt
.Ve
.IP "\(bu" 2
Install the icons in the webdir :
.Sp
.Vb 2
\& mkdir DOCUMENTROOT/mirmon/icons
\& cp icons/* DOCUMENTROOT/mirmon/icons
.Ve
.IP "\(bu" 2
Create a mirror list \f(CW\*(C`mirror_list\*(C'\fR ;
.Sp
Use your favorite editor, or genererate the list from an
existing database.
.Sp
.Vb 3
\& nl http://archive.cs.uu.nl/your\-project/ contact@cs.uu.nl
\& uk http://mirrors.this.org/your\-project/ mirrors@this.org
\& us http://mirrors.that.org/your\-project/ mirrors@that.org
.Ve
.Sp
The email addresses are optional.
.IP "\(bu" 2
Create a mirmon config file \f(CW\*(C`mirmon.conf\*(C'\fR with your favorite editor.
.Sp
.Vb 9
\& # lines must start in the first column ; no leading white space
\& project_name ....
\& project_url ....
\& mirror_list mirror_list
\& state state.txt
\& countries countries.list
\& web_page DOCUMENTROOT/mirmon/index.html
\& icons /mirmon/icons
\& probe /usr/bin/wget \-q \-O \- \-T %TIMEOUT% \-t 1 %URL%TIME.txt
.Ve
.Sp
This assumes the project's timestamp is in file \f(CW\*(C`TIME.txt\*(C'\fR.
.IP "\(bu" 2
If you have rsync urls, change the probe line to :
.Sp
.Vb 1
\& probe perl /usr/local/src/mirmon/probe \-t %TIMEOUT% %URL%TIME.txt
.Ve
.IP "\(bu" 2
Run mirmon :
.Sp
.Vb 1
\& perl mirmon \-v \-get all
.Ve
.Sp
The mirmon report should now be in 'DOCUMENTROOT/mirmon/index.html'
.Sp
.Vb 1
\& http://www.your.project.org/mirmon/
.Ve
.IP "\(bu" 2
If/when, at a later date, you want to upgrade mirmon :
.Sp
.Vb 3
\& cd /usr/local/src/mirmon
\& svn status \-u
\& svn up
.Ve
.SH "SEE ALSO"
.IX Header "SEE ALSO"
mirmon.pm(3)
.SH "AUTHOR"
.IX Header "AUTHOR"
(c) 2003-2014 Henk P. Penning
Faculty of Science, Utrecht University
http://www.staff.science.uu.nl/~penni101/ -- penning@uu.nl
mirmon-2.10 - Fri Aug 15 12:26:55 2014 ; henkp
mirmon-2.10/mirmon 0000644 0016337 0001750 00000212256 12373376740 013434 0 ustar penni101 staff #! /usr/bin/perl -w
# Copyright (c) 2003-2014 Henk Penning, all rights reserved.
# penning@uu.nl, http://www.staff.science.uu.nl/~penni101/
# Version 1.1 was donated to the Apache Software Foundation 2003 Jan 28.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
#
# Thanks to Klaus Heinz for sugestions ao htm_head ;
# Peter Pöml for MirrorBrain support ; Jeremy Olexa, Karl Berry, Roland
# Pelzer for suggestions regarding rsync support.
use strict ;
our $PRG = 'mirmon' ;
our $VER = "2.10" ;
our $DEF_TIMEOUT = 300 ;
our $HIST = 14 ;
our $TIM_PAT = '^(\d+)([smhd])$' ;
our %APA_TYPES = () ; $APA_TYPES { $_ } ++ for qw(backup ftp http rsync) ;
our %GET_OPTS = () ; $GET_OPTS { $_ } ++ for qw(all update url) ;
our $HIST_DELTA = 24 * 60 * 60 ;
our $APRX_DELTA = 300 ;
our $HOME = 'http://www.staff.science.uu.nl/~penni101/mirmon/' ;
package Base ; #####################################################
use base 'Exporter' ;
our ( @ISA, @EXPORT ) ;
BEGIN
{ @ISA = qw(Exporter) ;
@EXPORT =
qw(aprx_eq aprx_ge aprx_le aprx_gt aprx_lt
URL NAM SMA BLD NSS TAB BQ TR TH TD TDr RED GRN H1 H2 H3
s4tim pr_interval pr_diff
) ;
}
sub Version { "$PRG version $VER" ; }
sub version { "$PRG-$VER" ; }
sub DEF_TIMEOUT { $DEF_TIMEOUT ; }
sub is_get_opt { my $opt = shift ; exists $GET_OPTS { $opt } ; }
sub getset
{ my $self = shift ;
my $attr = shift ;
if ( @_ ) { $self -> { $attr } = shift ; }
die "no attr '$attr'" unless exists $self -> { $attr } ;
$self -> { $attr } ;
}
sub mk_method
{ my $self = shift ;
my $attr = shift ;
sprintf 'sub %s { my $self = shift ; $self -> getset ( "%s", @_ ) ; }'
, $attr, $attr ;
}
sub mk_methods
{ my $self = shift ;
join "\n", map { Base -> mk_method ( $_ ) ; } @_ ;
}
sub aprx_eq { my ( $t1, $t2 ) = @_ ; abs ( $t1 - $t2 ) < $APRX_DELTA ; }
sub aprx_ge { my ( $t1, $t2 ) = @_ ; $t1 > $t2 or aprx_eq $t1, $t2 ; }
sub aprx_le { my ( $t1, $t2 ) = @_ ; $t1 < $t2 or aprx_eq $t1, $t2 ; }
sub aprx_gt { my ( $t1, $t2 ) = @_ ; $t1 > $t2 and not aprx_eq $t1, $t2 ; }
sub aprx_lt { my ( $t1, $t2 ) = @_ ; $t1 < $t2 and not aprx_eq $t1, $t2 ; }
sub URL { sprintf '%s', $_[0], $_[1] ; }
sub NAM { sprintf '%s', $_[0], $_[1] ; }
sub SMA { sprintf "%s", $_[0] ; }
sub BLD { sprintf "%s", $_[0] ; }
sub NSS { sprintf SMA('%s site%s'), $_[0], ( $_[0] == 1 ? '' : 's' ) ; }
sub TAB { sprintf "
%s
", $_[0] ; }
sub BQ { sprintf "
\n%s\n
\n", $_[0] ; }
sub TR { sprintf "
%s
\n", $_[0] ; }
sub TH { sprintf "
%s
\n", $_[0] ; }
sub TD { sprintf "
%s
\n", $_[0] ; }
sub H1 { sprintf "
%s
\n", $_[0] ; }
sub H2 { sprintf "
%s
\n", $_[0] ; }
sub H3 { sprintf "
%s
\n", $_[0] ; }
sub TDr { sprintf "
%s
\n", $_[0] ; }
sub RED { sprintf "%s", $_[0] ; }
sub GRN { sprintf '%s', $_[0] ; }
sub s4tim
{ my $tim = shift ;
my %tab = ( 's' => 1, 'm' => 60, 'h' => 60 * 60, 'd' => 60 * 60 * 24 ) ;
die "wrong time '$tim'" unless $tim =~ /$TIM_PAT/o ;
my $m = $1 ; my $u = $2 ;
return $m * $tab { $u } ;
}
sub pr_interval
{ my $s = shift ;
my ( $magn, $unit ) ;
my $mins = $s / 60 ; my $m = int ( $mins + 0.5 ) ;
my $hours = $s / ( 60 * 60 ) ; my $h = int ( $hours + 0.5 ) ;
if ( $s < 50 )
{ $magn = $s ; $unit = 'second' ; }
elsif ( $m < 50 )
{ $magn = $m ; $unit = 'minute' ; }
elsif ( $h < 36 )
{ $magn = $h ; $unit = 'hour' ; }
else
{ $magn = sprintf "%.1f", $hours / 24 ; $unit = 'day' ; }
$unit .= 's' unless $magn == 1 ;
return "$magn $unit" ;
}
sub pr_diff
{ my $time = shift ;
my $max = shift ;
my $res ;
if ( $time == $^T )
{ $res = BLD 'renewed' ; }
else
{ $res = pr_interval $^T - $time ;
$res = BLD RED $res if aprx_lt $time, $max ;
}
return $res ;
}
sub exp_date
{ my @day = qw(Sun Mon Tue Wed Thu Fri Sat) ;
my @mon = qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec) ;
my @gmt = gmtime time + 3600 ;
sprintf "%s, %02d %s %4d %02d:%02d:%02d GMT"
, $day [ $gmt [ 6 ] ]
, $gmt [ 3 ]
, $mon [ $gmt [ 4 ] ]
, $gmt [ 5 ] + 1900
, @gmt [ 2, 1, 0 ]
;
}
sub htmlquote
{ my $x = shift ;
$x =~ s/&/&/g ;
$x =~ s//>/g ;
return $x ;
}
package Mirmon ; ###################################################
BEGIN { use base 'Base' ; Base -> import () ; }
use IO::Select ;
use Net::hostent ;
{ my %opt = ( v => 0 , d => 0 , q => 0 ) ;
sub _opt
{ my ( $key, $val ) = @_ ;
my $res ;
unless ( exists $opt { $key } )
{ warn "unknown Mirmon option '$key'\n" ; }
else
{ $res = $opt { $key } ;
$opt { $key } = $val if defined $val ;
}
$res ;
}
}
sub verbose { _opt ( 'v', shift ) ; }
sub quiet { _opt ( 'q', shift ) ; }
sub debug { _opt ( 'd', shift ) ; }
eval Base -> mk_methods ( qw(conf state regions) ) ;
sub config_list
{ my $self = shift ;
my $home = ( getpwuid $< ) [ 7 ] or die "can get homedir '$<' ($!)" ;
( 'mirmon.conf', "$home/.mirmon.conf", '/etc/mirmon.conf' ) ;
}
sub new
{ my $self = shift ;
my $path = shift ;
my $res = bless {}, $self ;
$res -> get_config ( $path ) ;
$res -> get_state ;
$res -> get_regions ;
$res ;
}
sub find_config
{ my $self = shift ;
my $arg = shift ;
my @LIST = $arg ? ( $arg ) : Mirmon -> config_list ;
for my $conf ( @LIST ) { return $conf if -r $conf and ! -d $conf ; }
die sprintf "can't find a config file :\n %s\n" , join "\n ", @LIST ;
}
sub get_config
{ my $self = shift ;
my $path = shift ;
my $file = $self -> find_config ( $path ) ; # or die
$self -> conf ( Mirmon::Conf -> new ( $file ) ) ;
}
sub get_state
{ my $self = shift ;
my $conf = $self -> conf ;
my $name = $conf -> project_name ;
my $state = $conf -> state ;
my $res = {} ;
open STATE, $state or die "can't open $state ($!)" ;
for my $line ( )
{ chop $line ;
my $mirror = Mirmon::Mirror -> new ( $self, $line ) ;
$res -> { $mirror -> url } = $mirror ;
}
close STATE ;
my $mlist = $conf -> mirror_list ;
my $style = $conf -> list_style ;
my %in_list = () ;
my $changes = '' ;
open MLIST, $mlist or die "can't open $mlist ($!)" ;
for my $line ( )
{ chop $line ;
next if $line =~ /^#/ ;
next if $line =~ /^\s*$/ ;
my ( $reg, $url, $mail ) ;
if ( $style eq 'plain' )
{ ( $reg, $url, $mail ) = split ' ', $line ; }
elsif ( $style eq 'apache' )
{ my $apache_type ;
( $apache_type, $reg, $url, $mail ) = split ' ', $line ;
unless ( defined $APA_TYPES { $apache_type } )
{ print "*** strange type in $url ($apache_type)\n"
unless Mirmon::quiet ;
next ;
}
}
if ( $conf -> add_slash and $url !~ m!/$! )
{ print "*** appended '/' to $url\n" unless Mirmon::quiet ;
$url .= '/' ;
}
$in_list { $url } ++ ;
unless ( exists $res -> { $url } )
{ $changes .= sprintf "added %s\n", $url unless Mirmon::quiet ;
$res -> { $url } = Mirmon::Mirror -> init ( $self, $url ) ;
}
my $mirror = $res -> { $url } ;
$mirror -> region ( $reg ) ;
$mirror -> mail ( $mail || '' ) ;
}
close MLIST ;
for my $url ( sort keys %$res )
{ # printf "%s\n", $res -> { $url } -> state ;
unless ( exists $in_list { $url } )
{ $changes .= sprintf "removed %s\n", $url unless Mirmon::quiet ;
delete $res -> { $url } ;
}
}
printf "changes in mirror-list for '%s':\n%s", $name, $changes
if $changes ;
$self -> state ( $res ) ;
}
sub put_state
{ my $self = shift ;
my $state = $self -> state ;
my $file = $self -> conf -> state ;
my $TMP = "$file.tmp" ;
open TMP, ">$TMP" or die "can't write '$TMP' ($!)" ;
for my $url ( sort keys %$state )
{ printf TMP "%s\n", $state -> { $url } -> state
or die "can't print $url to $TMP ($!)" ;
}
close TMP ;
if ( -z $TMP )
{ warn "wrote empty state file; keeping previous version" ; }
else
{ rename $TMP, $file or die "can't rename '$TMP', '$file' ($!)" ; }
}
sub get_regions
{ my $self = shift ;
my $file = $self -> conf -> countries ;
open REGS, $file or die "can't open countries '$file' ($!)" ;
while ( )
{ chop ;
next if /^#/ ;
my ( $code, $dash, $reg ) = split ' ', $_, 3 ;
$self -> { regions } { lc $code } = $reg ;
}
close REGS ;
}
sub _cmp_ccs
{ my $ccs = shift ;
my $x = shift ;
my $y = shift ;
my $xx = $ccs -> { $x } ;
my $yy = $ccs -> { $y } ;
if ( ! defined $xx and ! defined $yy )
{ $x cmp $y ; }
elsif ( ! defined $xx )
{ -1 ; }
elsif ( ! defined $yy )
{ +1 ; }
else
{ $xx cmp $yy ; }
}
sub _pr_round
{ my $x = shift ;
my $i = int $x ;
my $f = $x - $i ;
$i + ( rand 1 < $f ? 1 : 0 ) ;
}
sub _diag_qs
{ my $qs = shift ;
join ', ', map { sprintf "%s %s" , $_, scalar @{ $qs -> { $_ } } ; }
sort keys %$qs ;
}
sub _rpick
{ my $row = shift ;
die "_rpick : row empty" unless @$row ;
my $idx = int rand @$row ;
my $res = $row -> [ $idx ] ;
$row -> [ $idx ] = $row -> [ $#{$row} ] ;
pop @$row ;
$res ;
}
sub _buck_split
{ my $que = shift ;
my $tmp = [] ;
for my $mirr ( @$que )
{ my $lp = $mirr -> last_probe ;
my $hr = int ( ( $^T - $lp ) / 60 / 60 + 0.5 ) ;
push @{ $tmp -> [ $hr ] }, $mirr ;
}
[ grep defined $_, @$tmp ] ;
}
sub _buck_join
{ my $bucks = shift ;
my $res = [] ;
push @$res, @$_ for @$bucks ;
$res ;
}
sub _buck_pick
{ my $bucks = shift ;
die "buck_pick : bucks empty" unless @$bucks ;
my $buck = ( sort { @$b <=> @$a } @$bucks ) [ 0 ] ;
_rpick $buck ;
}
sub _randomize
{ my $ques = shift ;
my $poll = shift ;
my $hrs = int ( $poll / 60 / 60 + 0.5 ) ;
my $diag1 = _diag_qs $ques ;
my $todos = $ques -> { todo } ;
my $dones = $ques -> { done } ;
my $cnt = @$todos + @$dones ;
my $avg = $hrs ? $cnt / $hrs : 0 ;
my $iavg = _pr_round $avg ;
my $pick = 0 ;
my $bucks = _buck_split $dones ;
while ( @$todos < $iavg and $pick < @$dones )
{ push @$todos, _buck_pick $bucks ;
$pick ++ ;
}
$ques -> { done } = _buck_join $bucks ;
sprintf ''
. " hrs %s, %s\n"
. " avg %.2f -> %d , picked %d ; queued %s\n"
. " hrs %s, %s\n"
, $hrs, $diag1
, $avg, $iavg, $pick, scalar @$todos
, $hrs, _diag_qs ( $ques )
;
}
sub get_dates
{ my $self = shift ;
my $get = shift ;
my $URL = shift ;
my $state = $self -> state ;
my $conf = $self -> conf ;
my $CMD = $conf -> probe ;
my $PAR = $conf -> max_probes ;
my %m4h = () ;
my @QUE = () ;
my $GET = IO::Select -> new () ;
my $ques = {} ;
for my $col ( qw(new red grn xtr) )
{ $ques -> { $col } { $_ } = [] for qw(done todo) ; }
my $max_poll = s4tim $conf -> max_poll ;
my $min_poll = s4tim $conf -> min_poll ;
if ( Mirmon::verbose ) { printf "mirrors %d\n", scalar keys %$state ; }
if ( $get eq 'all' )
{ @QUE = sort { $a -> url cmp $b -> url } values %$state ; }
elsif ( $get eq 'url' )
{ @QUE = ( $state -> { $URL } ) ; }
elsif ( $get eq 'update' )
{ my $maxp = $^T - $max_poll ;
my $minp = $^T - $min_poll ;
if ( Mirmon::verbose )
{ printf "max_poll %s\n", scalar localtime $maxp ;
printf "min_poll %s\n", scalar localtime $minp ;
}
for my $url ( sort keys %$state )
{ my $mirror = $state -> { $url } ;
my $stat = $mirror -> last_status ;
my $vrfy = $mirror -> last_ok_probe ;
my $lprb = $mirror -> last_probe ;
my $col ;
my $que ;
if ( $stat eq 'undef' ) # never probed ; new mirror ; todo
{ $col = 'new' ; $que = 'todo' ; }
elsif ( $conf -> get_xtr ( $mirror -> region ) )
{ $col = 'xtr' ; $que = 'todo' ; }
else
{ my $poll = $stat eq 'ok' ? $maxp : $minp ;
$col = $stat eq 'ok' ? 'grn' : 'red' ;
$que = ( aprx_le $lprb, $poll ) ? 'todo' : 'done' ;
}
push @{ $ques -> { $col } { $que } }, $mirror ;
}
if ( $conf -> randomize )
{ my $msg = "randomize green\n" ;
$msg .= _randomize $ques -> { grn }, $max_poll ;
$msg .= "randomize red\n" ;
$msg .= _randomize $ques -> { red }, $min_poll ;
print $msg if Mirmon::verbose ;
}
@QUE =
( @{ $ques -> { new } { todo } }
, @{ $ques -> { red } { todo } }
, @{ $ques -> { grn } { todo } }
, @{ $ques -> { xtr } { todo } }
) ;
}
else
{ die "unknown opt_get '$get'" ; }
if ( Mirmon::verbose ) { printf "queued %d\n\n", scalar @QUE ; }
while ( @QUE )
{ my $started = 0 ;
while ( $GET -> count () < $PAR and @QUE )
{ my $mirror = shift @QUE ;
if ( gethost $mirror -> site )
{ my $handle = $mirror -> start_probe ;
$m4h { $handle } = $mirror ;
$GET -> add ( $handle ) ;
$started ++ ;
}
else
{ $mirror -> update ( 0, 'site_not_found', undef ) ; }
}
my @can_read = $GET -> can_read ( 0 ) ;
printf "queue %d, started %d, probes %d, can_read %d\n",
scalar @QUE, $started, $GET -> count (), scalar @can_read
if Mirmon::verbose ;
for my $handle ( @can_read )
{ # order is important ; wget's hang if/when actions are reversed
$GET -> remove ( $handle ) ;
$m4h { $handle } -> finish_probe ( $handle ) ;
}
sleep 1 ;
}
my $stop = time + $conf -> timeout + 10 ;
while ( $GET -> count () and time < $stop )
{ my @can_read = $GET -> can_read ( 0 ) ;
printf "wait %2d, probes %d, can_read %d\n",
$stop - scalar time, $GET -> count (), scalar @can_read
if Mirmon::verbose ;
for my $handle ( @can_read )
{ $GET -> remove ( $handle ) ;
$m4h { $handle } -> finish_probe ( $handle ) ;
}
sleep 10 ;
}
for my $handle ( $GET -> handles () )
{ $m4h { $handle } -> update ( 0, 'hangs', undef ) ; }
}
sub img_sf_cnt
{ my $self = shift ;
my $prf = shift ;
my $cnt = shift ;
my $res ;
if ( $prf eq 'x' )
{ sprintf
( ''
, $self -> conf -> icons
) x $cnt ;
}
else
{ sprintf ''
, $self -> conf -> icons, $prf, $cnt ;
}
}
sub img_sf { my $self = shift ; $self -> img_sf_cnt ( $_[0], 1 ) ; }
sub show_hist
{ my $self = shift ;
my $hst = shift ;
if ( $hst =~ /-(.*)$/ ) { $hst = $1 ; }
return '' unless $hst =~ m/^[sbfzx]+$/ ;
if ( length $hst == $HIST and $hst =~ /^(s*b)s*$/ )
{ return $self -> img_sf_cnt ( 'sb', length $1 ) ; }
elsif ( length $hst == $HIST and $hst =~ /^(s*f)s*$/ )
{ return $self -> img_sf_cnt ( 'sf', length $1 ) ; }
elsif ( length $hst == $HIST and $hst =~ /^(s*b)fs*$/ )
{ return $self -> img_sf_cnt ( 'sbf', length $1 ) ; }
my $res = '' ;
my $cnt = 1 ;
my $prf = substr $hst, 0, 1 ;
$hst = substr $hst, 1 ;
while ( $hst ne '' )
{ if ( substr ( $prf, 0, 1 ) eq substr ( $hst, 0, 1 ) )
{ $cnt ++ ;
$hst = substr $hst, 1 ;
}
else
{ $res .= $self -> img_sf_cnt ( $prf, $cnt ) ;
$prf = substr $hst, 0, 1 ;
$hst = substr $hst, 1 ;
$cnt = 1 ;
}
}
$res .= $self -> img_sf_cnt ( $prf, $cnt ) if $cnt ;
$res ;
}
sub gen_histogram_probes
{ my $self = shift ;
my $state = $self -> state ;
my %tab = () ;
my %bad = () ;
my $res = '' ;
my $s_cnt = 0 ;
my $f_cnt = 0 ;
my $hr_min ;
my $hr_max ;
for my $url ( keys %$state )
{ my $mirror = $state -> { $url } ;
my $lprb = $mirror -> last_probe ;
my $stat = $mirror -> last_status ;
next if $lprb eq 'undef' ;
my $hr = int ( ( $^T - $lprb ) / 3600 + 0.5 ) ;
$hr_min = $hr if ! defined $hr_min or $hr < $hr_min ;
$hr_max = $hr if ! defined $hr_max or $hr > $hr_max ;
if ( $stat eq 'ok' )
{ $tab { $hr } ++ ; $s_cnt ++ ; }
else
{ $bad { $hr } ++ ; $f_cnt ++ ; }
}
return BQ 'nothing yet' unless scalar keys %tab ;
$res = TR
( TH ( 'hours ago' )
. TH ( 'succ' )
. TH ( 'fail' )
. TH sprintf
( '%s %s, %s %s'
, $s_cnt , GRN ( 'successful' )
, $f_cnt , RED ( 'failed' )
)
) ;
my $max = 0 ;
for my $x ( keys %tab )
{ my $tot = $tab { $x } + ( $bad { $x } || 0 ) ;
$max = $tot if $max < $tot ;
}
return BQ "nothing yet" unless $max ;
for my $hr ( $hr_min .. $hr_max )
{ my $x = $tab { $hr } || 0 ;
my $y = $bad { $hr } || 0 ;
my $n = int ( $x / $max * $HIST ) ;
my $b = int ( $y / $max * $HIST ) ;
$res .= TR
( TDr ( $hr )
. TDr ( $x )
. TDr ( $y )
. TD
( ( $n ? $self -> img_sf_cnt ( 's', $n ) : '' )
. ( $b ? $self -> img_sf_cnt ( 'f', $b ) : '' )
. ( ( $n + $b ) ? '' : ' ' )
)
) ;
}
return BQ TAB $res ;
}
sub age_avg
{ my $self = shift ;
my $state = $self -> state ;
my @tab = () ;
for my $url ( keys %$state )
{ my $time = $state -> { $url } -> age ;
push @tab, $^T - $time if $time =~ /^\d+$/ ;
}
my $cnt = @tab ;
return undef if $cnt == 0 ;
@tab = sort { $a <=> $b } @tab ;
my $tot = 0 ;
for my $age ( @tab ) { $tot += $age ; }
my $mean = $tot / $cnt ;
my $median ;
if ( $cnt == 1 )
{ $median = $tab [ 0 ] ; }
elsif ( $cnt % 2 )
{ my $mid = int ( $#tab / 2 ) ;
$median = ( $tab [ $mid ] + $tab [ $mid + 1 ] ) / 2 ;
}
else
{ my $mid = int ( $#tab / 2 ) ;
$median = $tab [ $mid ] ;
}
if ( @tab < 2 )
{ return $mean, $median, undef ; }
my $sum = 0 ;
for my $age ( @tab )
{ $sum += ( $age - $mean ) ** 2 ; }
my $stddev = sqrt ( $sum / ( $cnt - 1 ) ) ;
return $mean, $median, $stddev ;
}
sub legend
{ my $self = shift ;
my $conf = $self -> conf ;
my $min_sync = $conf -> min_sync ;
my $max_sync = $conf -> max_sync ;
my $min_poll = $conf -> min_poll ;
my $max_poll = $conf -> max_poll ;
return <legend
project site -- home
project site is an url.
The href is the href for the site in the list of mirrors,
usually the root of the mirrored file tree.
The text is the site of that url.
home (represented by the @-symbol) is an url
pointing to the document root of the site. This pointer is
useful if the project site url is invalid,
possibly because the mirror site moved the archive.
type
Indicates the type (ftp or http) of
the project site and home urls.
mirror age, daily stats
The mirror age is based upon the last successful probe.
Once a day the status of a mirror site is determined.
The status (represented by a colored block) is appended
to the right of the status history (right
is recent). More precise, the status block is appended
if the last status block was appended 24 (or more) hours ago.
The status of a mirror depends on its age and a few
configuration parameters :
status
age
this project
in general
min
max
min
max
fresh
0
$min_sync + $max_poll
0
min_sync + max_poll
oldish
$min_sync + $max_poll
$max_sync + $max_poll
min_sync + max_poll
max_sync + max_poll
old
$max_sync + $max_poll
∞
max_sync + max_poll
∞
bad
the site or mirror tree was never found
last probe, probe stats
Last probe indicates when the last successful probe was made.
Probe stats gives the probe history (right is recent).
A probe is either a
success or a
failure.
last stat
Last stat gives the status of the last probe.
LEGENDA
}
sub _ths
{ return '' unless my $ths = shift ;
$ths == 1 ? TH '' : "
\n" ;
}
sub gen_histogram
{ my $self = shift ;
my $where = shift ;
my $conf = $self -> conf ;
my $state = $self -> state ;
return '' if $where ne $conf -> put_histo ;
my $MAX_H = $conf -> max_age1 ;
my $MAX_h = 1 +
( ( 20 * 3600 <= $MAX_H and $MAX_H <= 36 * 3600 )
? int ( $MAX_H / 3600 )
: 25
) ;
my $MAX_O = $conf -> max_age2 ;
my $MAX_o = int ( $MAX_O / 3600 + 0.5 ) ;
my $H = 18 ;
my %W = ( 'old' => 1, 'ded' => 1, 'bad' => 1 ) ;
my %Wmx = ( 'old' => 5, 'ded' => 3, 'bad' => 3 ) ;
my %tab ;
my %hst ;
my $res ;
for ( my $x = 0 ; $x < $MAX_h ; $x ++ ) { $tab { $x } = 0 ; }
$tab { old } = 0 ; $tab { ded } = 0 ; $tab { bad } = 0 ;
for my $url ( keys %$state )
{ my $time = $state -> { $url } -> age ;
if ( $time =~ /^\d+$/ )
{ my $s = $^T - $time ;
my $hr = int ( $s / $MAX_H * ( $MAX_h - 1 ) + 0.5 ) ;
if ( $s <= $MAX_H ) { $tab { $hr } ++ ; }
elsif ( $s <= $MAX_O ) { $tab { old } ++ ; }
else { $tab { ded } ++ ; }
}
else
{ $tab { bad } ++ ; }
}
my $max = 0 ;
for ( grep ! exists $Wmx { $_ }, keys %tab )
{ $max = $tab { $_ } if $tab { $_ } > $max ; }
my %bad ;
for my $aux ( keys %Wmx )
{ $bad { $aux } = $tab { $aux } ;
if ( $bad { $aux } > $max )
{ $W { $aux } = $Wmx { $aux } ;
my $d = int ( $bad { $aux } / $W { $aux } ) ;
for ( my $i = 1 ; $i < $W { $aux } ; $i++ )
{ $tab { $aux . $i } = $d ;
if ( $bad { $aux } % $Wmx { $aux } > $i )
{ $tab { $aux . $i } ++ ;
$tab { $aux } -- ;
}
}
$tab { $aux } -= ( $W { $aux } - 1 ) * $d ;
$max = $tab { $aux } if $max < $tab { $aux } ;
}
}
# if ( $opt{v} )
# { for my $hr ( keys %tab )
# { printf "tab '%s' = '%s'\n", $hr, $tab { $hr } ; }
# }
return 'nothing yet' unless $max ;
$H = $max if 8 <= $max and $max <= 26 ;
for ( keys %tab )
{ $hst { $_ } = int ( $H * $tab { $_ } / $max + 0.5 ) ; }
my @keys = sort { $a <=> $b } grep /^\d+$/, keys %hst ;
my $tab_hr = 0 ;
for my $hr ( @keys ) { $tab_hr += $tab { $hr } ; }
push @keys
, grep ( m/^old/, sort keys %tab )
, grep ( m/^ded/, sort keys %tab )
, grep ( m/^bad/, sort keys %tab )
;
my $img_bar = sprintf ''
, $conf -> icons ;
my %img = ( bar => $img_bar ) ;
for my $col ( qw(s b f z) ) { $img { $col } = $self -> img_sf ( $col ) ; }
for ( my $h = $H ; $h > 0 ; $h -- )
{ $res .= "
'
, $self -> conf -> icons
) x $cnt ;
}
else
{ sprintf '
'
, $self -> conf -> icons, $prf, $cnt ;
}
}
sub img_sf { my $self = shift ; $self -> img_sf_cnt ( $_[0], 1 ) ; }
sub show_hist
{ my $self = shift ;
my $hst = shift ;
if ( $hst =~ /-(.*)$/ ) { $hst = $1 ; }
return '' unless $hst =~ m/^[sbfzx]+$/ ;
if ( length $hst == $HIST and $hst =~ /^(s*b)s*$/ )
{ return $self -> img_sf_cnt ( 'sb', length $1 ) ; }
elsif ( length $hst == $HIST and $hst =~ /^(s*f)s*$/ )
{ return $self -> img_sf_cnt ( 'sf', length $1 ) ; }
elsif ( length $hst == $HIST and $hst =~ /^(s*b)fs*$/ )
{ return $self -> img_sf_cnt ( 'sbf', length $1 ) ; }
my $res = '' ;
my $cnt = 1 ;
my $prf = substr $hst, 0, 1 ;
$hst = substr $hst, 1 ;
while ( $hst ne '' )
{ if ( substr ( $prf, 0, 1 ) eq substr ( $hst, 0, 1 ) )
{ $cnt ++ ;
$hst = substr $hst, 1 ;
}
else
{ $res .= $self -> img_sf_cnt ( $prf, $cnt ) ;
$prf = substr $hst, 0, 1 ;
$hst = substr $hst, 1 ;
$cnt = 1 ;
}
}
$res .= $self -> img_sf_cnt ( $prf, $cnt ) if $cnt ;
$res ;
}
sub gen_histogram_probes
{ my $self = shift ;
my $state = $self -> state ;
my %tab = () ;
my %bad = () ;
my $res = '' ;
my $s_cnt = 0 ;
my $f_cnt = 0 ;
my $hr_min ;
my $hr_max ;
for my $url ( keys %$state )
{ my $mirror = $state -> { $url } ;
my $lprb = $mirror -> last_probe ;
my $stat = $mirror -> last_status ;
next if $lprb eq 'undef' ;
my $hr = int ( ( $^T - $lprb ) / 3600 + 0.5 ) ;
$hr_min = $hr if ! defined $hr_min or $hr < $hr_min ;
$hr_max = $hr if ! defined $hr_max or $hr > $hr_max ;
if ( $stat eq 'ok' )
{ $tab { $hr } ++ ; $s_cnt ++ ; }
else
{ $bad { $hr } ++ ; $f_cnt ++ ; }
}
return BQ 'nothing yet' unless scalar keys %tab ;
$res = TR
( TH ( 'hours ago' )
. TH ( 'succ' )
. TH ( 'fail' )
. TH sprintf
( '%s %s, %s %s'
, $s_cnt , GRN ( 'successful' )
, $f_cnt , RED ( 'failed' )
)
) ;
my $max = 0 ;
for my $x ( keys %tab )
{ my $tot = $tab { $x } + ( $bad { $x } || 0 ) ;
$max = $tot if $max < $tot ;
}
return BQ "nothing yet" unless $max ;
for my $hr ( $hr_min .. $hr_max )
{ my $x = $tab { $hr } || 0 ;
my $y = $bad { $hr } || 0 ;
my $n = int ( $x / $max * $HIST ) ;
my $b = int ( $y / $max * $HIST ) ;
$res .= TR
( TDr ( $hr )
. TDr ( $x )
. TDr ( $y )
. TD
( ( $n ? $self -> img_sf_cnt ( 's', $n ) : '' )
. ( $b ? $self -> img_sf_cnt ( 'f', $b ) : '' )
. ( ( $n + $b ) ? '' : ' ' )
)
) ;
}
return BQ TAB $res ;
}
sub age_avg
{ my $self = shift ;
my $state = $self -> state ;
my @tab = () ;
for my $url ( keys %$state )
{ my $time = $state -> { $url } -> age ;
push @tab, $^T - $time if $time =~ /^\d+$/ ;
}
my $cnt = @tab ;
return undef if $cnt == 0 ;
@tab = sort { $a <=> $b } @tab ;
my $tot = 0 ;
for my $age ( @tab ) { $tot += $age ; }
my $mean = $tot / $cnt ;
my $median ;
if ( $cnt == 1 )
{ $median = $tab [ 0 ] ; }
elsif ( $cnt % 2 )
{ my $mid = int ( $#tab / 2 ) ;
$median = ( $tab [ $mid ] + $tab [ $mid + 1 ] ) / 2 ;
}
else
{ my $mid = int ( $#tab / 2 ) ;
$median = $tab [ $mid ] ;
}
if ( @tab < 2 )
{ return $mean, $median, undef ; }
my $sum = 0 ;
for my $age ( @tab )
{ $sum += ( $age - $mean ) ** 2 ; }
my $stddev = sqrt ( $sum / ( $cnt - 1 ) ) ;
return $mean, $median, $stddev ;
}
sub legend
{ my $self = shift ;
my $conf = $self -> conf ;
my $min_sync = $conf -> min_sync ;
my $max_sync = $conf -> max_sync ;
my $min_poll = $conf -> min_poll ;
my $max_poll = $conf -> max_poll ;
return <' , $conf -> icons ; print <