calamaris-2.99.4.0/0000755000175000001440000000000010407322627012562 5ustar corduserscalamaris-2.99.4.0/CHANGES0000644000175000001440000012444210407330373013561 0ustar cordusers2006-03-19 18:59 cord * EXAMPLES.v3 (3.1): introducing EXAMPLES for Calamaris v3 2006-03-19 18:58 cord * calamaris.conf (3.1): * fixed typos * again reflecting all changes of Calamaris. 2006-03-19 18:52 cord * calamaris.1 (3.1): * reflecting the changes of Calamaris + HTML-Frame-Output + mention localtime in the timerange option + graphics-format + output-* + generate-index + omitting wrong s 2006-03-19 18:28 cord * calamaris (2.100): * Release V2.99.4.0 * don't use rcs-version-number anymore. now use an own variable * fixed error-messages on missing perl-modules * added rudimentary BlueCoat-Support (thanks to Michael Riedel ) * added --output-file-prefix -option * added %h (hostname) and %t (timerange) placeholders for --output-* -options. * added --generate-index -Option * updated HTML-Output from HTML 4.01 Trans to XHTML 1.0 Strict * added Jumbo-Patch by Michael Pophal + nicer HTML-Output including CSS (should be tidy(1)-clean) + more Reports + HTML-Frame-Output + GRaphic-Format depending possible in all formats libgd supports. + workaround a bug when Cachefiles contain -Entries. * fixed some division by zero issues * Timerange now in localtime (switched from gmt-time (reported by Scott Tregear )) * fixed a problem with Oops-Logfiles (reported by Sythos ) 2006-03-19 17:11 cord * TODO (3.1): no changes 2006-03-19 17:10 cord * BUGS (3.1): * added content-type bug. 2004-12-23 21:16 cord * calamaris.1 (3.0.0.2), calamaris.conf (3.0.0.4): * Patch by Michael Pophal: + reflect adding of the configuration-option for the graphics-format 2004-12-23 21:05 cord * calamaris (2.99.1.3): * Patch by Michial Pophal: + added configuration-option fpr the graphics-format + corrected the report_index for the new peak-report. + fixed handling of % in usernames. 2004-12-23 21:01 cord * INSTALL (3.1): clarified installation instructions. 2004-09-22 22:39 cord * calamaris.conf (3.0.0.3): * added $output_file * clarified $output_path 2004-09-22 22:37 cord * calamaris.1 (3.0.0.1): * added --output-file 2004-09-22 22:34 cord * calamaris (2.99.1.2): * fixed a bug in the benchmark-switch (reported by Chris Clemson * added a --output-file-switch to alter the filename for --output-path (suggested by Peter W. Osel ) * fixed a warning for usinf graph without --output-path 2004-09-22 22:27 cord * COPYRIGHT (3.1): * reflect the inclusion of Michael Pophals perl-modules which are dual-licensed GPL and Perl Artictic License. 2004-09-18 13:03 cord * calamaris.conf (3.0.0.2): * added Version Id. 2004-09-17 23:04 cord * calamaris.conf (3.0.0.1): * fixed a typo. 2004-09-17 23:03 cord * calamaris-cache-convert (0.2): * added bug-workaround from Calamaris v2 2004-09-17 23:02 cord * calamaris (2.99.1.1): * all errors now show which program caused them. * only try to graph if $xaxis is set. * only try 'most something' if $maxi and $xaxis[$maxi] is set. * only divide in removezerotime if divisor is != 0. * moved interval-sanity-check down behind interval-conversation. * define $opt_domain_report_limit if unset. 2004-09-15 23:51 cord * calamaris (2.99): * added Michael Pophal as Co-Author, he provided a jumbo patch which + adds graphics to Calamaris + adds a new report regarding duration distribution + adds a new report regarding server Response code distribution + extends information of the Requested extensions report + extends most other reports + adds an ipfilter-function + changes the cache-file-format + adds a config-file * removed old bug-workaround-options. * switched to long-options (short-options still work) * added a domain-report-limit (suggested by Franck Bourdonnec ) 2004-09-15 23:30 cord * calamaris.1 (3.0): * reflect the calamaris-changes in the manpage * new options added. * longoptions and configuration-file-options added. * tried to use a more sane grouping of the options. 2004-09-15 23:06 cord * calamaris.conf (3.0): Initial Calamaris-Config-File. 2004-09-15 23:02 cord * calAxestype.pm (3.2), calAxestype3d.pm (3.2), calBars3d.pm (3.2): Extended perl-Modul by Michael Pophal 2004-09-15 22:59 cord * calamaris-cache-convert (0.1): Initial revion of the conversion program 2004-09-15 22:58 cord * INSTALL (3.0): * splitted old README into 5 seperate files. * updated the suggested perl-Version. * reworked Installation-Instructions (thanks to Michael Pophal ) 2004-09-15 22:56 cord * TODO (3.0): * splitted old README into 5 seperate files. * added a note about an Installation-Routine * removed suggestion about limiting domain-reports. * added a comment to a suggestion for the Performance-Report. 2004-09-15 22:51 cord * BUGS (3.0): * splitted old README into 5 seperate files. * removed notes about old legacy-bugs. * added a note about the new cachefile-format. 2004-09-15 22:46 cord * README (3.0): * splitted old README into 5 seperate files. * extended the History of Calamaris * updated links to NetBSD and FreeBSD-Packages * removed dead link for rpm-Packages. 2004-09-15 22:41 cord * EXAMPLES (2.13): * added a note that the EXAMPLES-File isn't updated yet. 2004-06-06 18:29 cord * README (2.43, stable): * documented a problem with Logfiles from accelerated proxies. (reported by Pawel Worach ) * added a feature Request (requested by Ahjmad Kamal ) 2004-06-06 18:26 cord * calamaris (2.59, stable): * added 2004 to year-lines * changed Michael Pophals Mailadress * added sanity-check for time-interval-option (Michael Pophal ) * changed interval-display (Michael Pophal ) * added x-hiername for NetCache-Logparsing. (Peter W. Osel ) * added fix for iPlanet Web Proxy Server (Pawel Worach ) * moved CARP from HIT to MISS section (Michael Pophal ) * added a generation time stamp and reformatted Text-Output-Header. (Michael Pophal ) 2004-03-02 21:17 cord * EXAMPLES (2.12, stable): * added a suggestion from Alain Williams for a shorter and easier catting of Logfiles. * fixed Gottfried Hamms squidrep-example. Fixes sent in by Hanno 'Rince' Wagner . 2003-05-14 23:22 cord * calamaris.1 (1.28, stable): * added new -M-switch. * fixed some -m and -w examples, they are replaced by -F in the last version. * bumped refered Calamaris Version. 2003-05-14 23:13 cord * calamaris (2.58): * added a -M (Meta) switch to add free defined strings to html-. This switch may be useful to provide stylesheets to the html-report. (suggested by ycdtosa ? ) * Heavy tested and fixed problems with 'Division by zero'-errors (with broken logfiles caused by bugs of some Proxyservers.) (reported by Chris Knight ) While fixing those problems i removed the .00..001 workarounds with a correct solution. * found that .ru (Russia) now mainly uses 2nd-level domains, so i removed that from the 3rd-level-group of TopLevelDomains. * fixed warnings in HTML-Output, so now that will also be correct HTML. 2003-05-14 23:00 cord * README (2.42): * Clarified the RedHat-Bug a bit (triggered by a bug report from Massimo Carnevali ) 2003-02-09 01:06 cord * calamaris.1 (1.27): * added the new -F-switch and pulled the -w, -m, -W-switches. * cleaned up a little. * bumped referenced version to 2.57 2003-02-09 00:55 cord * calamaris (2.57): * Holger Marzen requested an output-format which he can use to parse afterwards. -F unformatted is the result. if you use -F unformatted along with -U, the byte values are calculated in the given Unit, and displayed without indication along with the numbers. the indication moves up to the header of the report. * as announced in the README i converted -m -w and -W also into the new -F-switch. so you now have to use -F mail, -F html and -F html-embed. For now the old switches will still work, but Calamaris puts out warnings about that. The switches defunct in the first release of '04. * I deleted (as scheduled) a warning about -N f which has been replaced by -N -1 * I pushed back removal schedules for Cachefile-Workarounds to the end of the year. * code-cleanup. 2003-02-09 00:06 cord * README (2.41): * removed note about pending changes in the output-options. 2003-02-08 23:57 cord * EXAMPLES (2.11): * changed the examples to the new output-switch syntax 2003-01-19 20:47 cord * calamaris (2.56): * fixed a bug in error-check for obsolete input-formats. 2003-01-19 17:58 cord * calamaris.1 (1.26): * reflect the chneges in input-formats in the manpage. * bumped refered Calamaris-Version-number * added 2003 to the copyright 2003-01-19 17:57 cord * calamaris (2.55): * replaced 'squid-mime' and 'squid-smartfilter' input formats with 'squid-extended' which is also usable for Cisco Content Engines Logfiles. (initial patch from Kenytt Avery ) * added an error for users of the replaced input formats. * fixed parsing of NetCache NetApp/5.2 Logfiles. (patch by SO Kwok Tsun ) * added 2003 to the copyright-notice * added/changed <> to all mailadresses 2003-01-19 17:52 cord * README (2.40): * added Cisco Content Engines * added <> to all mailadresses * modified the SmartFilter-comment * added a wishlist-item 'limit reports to a minimum number of transactions' requested by Franck Bourdonnec 2003-01-19 17:50 cord * EXAMPLES (2.10): * added <> to all mailadresses. 2002-11-23 18:53 cord * calamaris (2.54): * fixed small bug with new option. 2002-11-23 18:38 cord * calamaris.1 (1.25): * added new Time-Interval-Option. * bumped version-number for calamaris 2002-11-23 18:37 cord * README (2.39): * added a comment regarding a RedHat 8.0-Problem. * changed comment about SmartFilter. * removed wishlist-item Timerange. 2002-11-23 18:35 cord * calamaris (2.53): * added an Time-Range-Interval (first suggested by Steffen Sledz (sledz@zone42.org)) * rewrote internal-help-information. 2002-08-28 18:58 cord * calamaris (2.52): * re-added cache-bug workarounds. * added warnings for the reports, so everyone could know about problems with old cache-files. * fixed a perl-warning which appears in ascii-reports error-section (found by Michael Pophal (michael.pophal@siemens.com)) * moved the error-section to the top of the report. It now also produces a valid HTML-Table. 2002-08-28 18:54 cord * README (2.38): * spelling fixes. * added a note about (not) announcing spy-features. * re-added the statement about fixed cache-bugs. * added a workaround for broken SmartFilter-Logfiles. 2002-08-02 22:15 cord * calamaris (2.51): + fixed a bug in cache-file-writing. other-lines in host-related sub-reports in the requester-reports were put out with a missing field, which makes it impossible to read them in again. However: this part of the reports is not the default so this won't hit many users. I put a workaround in, which normally should recover 'broken' cachefiles without data-loss. (This one is reported by Radu - Eosif Mihailescu (rmihailescu@lumina.ro)) + removed some old bug-workarounds. + moved the Error-report to the Top of the reports. 2002-08-02 22:06 cord * README (2.37): + removed note about a workaround. which is now removed from the source. 2002-06-10 21:48 cord * calamaris.1 (1.24): + reflected change in -N-switch from f to -1 + bumped version Number to 2.50 2002-06-10 21:46 cord * calamaris (2.50): + fixed a problem with the new -N-switch. changed the f-value to -1 to make it more consistent to the other switches. + enabled IPv6-IPs. Calamaris now accepts IPv6-Ips and is able to resolve encapsulated IPv4-Ips. 2002-06-10 21:41 cord * README (2.36): + changed comment on IPv6: Calamaris cannot resolve real IPv6-IPs yet. 2002-06-05 19:46 cord * README (2.35): + added a statement about the net-yet supported IPv6 + added plan to change output-options. 2002-06-05 19:34 cord * calamaris.1 (1.23): + added new -W-option. + bumped calamaris-version-number. 2002-06-05 19:31 cord * calamaris (2.49): + fixed a bug with %-signs in the url, which Calamaris tried to expand internal. (reported by Sergey Zarubin (serge-home@yandex.ru)) + Helge Oldach (calamaris@oldach.net) submitted a patch which fixes a problem with syntactical-wrong html-output, and also added a -W-switch to produce HTML-Output without HTML-Headers, so it can used for Server-Side-Includes. 2002-05-29 22:13 cord * calamaris.1 (1.22): + changed 3rd-level to N-level-reports. + bumped calamaris-version to 2.48. 2002-05-29 22:10 cord * calamaris (2.48): + changed -3-switch to -N, which makes it possible for the user to define how many levels of URL-Hosts should be displayed. -Nf displays all Host-Components. (requested by Mark Güthling (privat@mague-pcservice.de)) + changed behaviour when wrong switches are submitted. Now there will be only a one-line hint will be displayed. 2002-05-23 22:26 cord * calamaris (2.47): + fixed problem with dynamic help, which happened if Calamris is run without -a or -P-option. (reported by Steve Snyder (swsnyder@home.com)) + fixed bug with -m -option. Mail-Subject wasn't printed out in the correct order. (reported by Philipp Frauenfelder (philipp.frauenfelder@swissonline.ch)) + found a problem with 'Incoming TCP-requests by status'. It wasn't displayed if all requestes were classified as ERROR. (found with the help of Alexey Markov (markov@crpi.ru)) + concentrated some Regexps. + cleaned even more up ;-) 2002-05-17 22:49 cord * calamaris.1 (1.21): + added documentation for the new -S and -3 options. + bumped referenced Calamaris-Version-Number. + cleaned up. 2002-05-17 22:47 cord * calamaris (2.46): + added third-level-switch (requested by Shawn Switenky (S.Switenky@telesat.ca)) + added seperate on/off-switching and sorting for all reports (requested by Jarkko Saloranta (jjs@kpo.fi) a long time ago, and Jigar Rasalawala (jrasalawala@fourelle.com)) + bugfixed Netscape-Proxy-Logfiles. The months were off-by-one. + cleaned up. made Calamaris better perl?! ;-) 2002-05-17 22:33 cord * README (2.34): + added Squid with SmartFilter-Patch to the supported Logfiles + added a statement about broken Squid-Smartfilterpatch-Logfiles. + added a statement for 0-0-bytes lines in the distribution histogram + removed wishlist item 'seperate on/off switching of incoming UDP/TCP-Requests' + removed wishlist item 'n-level requester report' 2002-05-17 22:30 cord * EXAMPLES (2.9): + cleaned up a little bit. 2002-01-25 22:25 cord * calamaris.1 (1.20): - added 'squid-smartfilter'-Input-Format. - bumped Version-Number to 2.45 2002-01-25 22:24 cord * calamaris (2.45): - added 'squid-smartfilter' input format (thanks to James Crocker (jcrocker@menasha.com)) - fixed/added the new Netcache V5.2-Logfile-Support (thanks to Enrico Ardizzoni (enrico@unife.it)) - added warnings and workarounds to the 'eating-all-memory'-problem in peak-reports (thanks to John Line (webadm@info.cam.ac.uk)) - corrected order of shorcuts in HTML-output. 2002-01-25 22:19 cord * README (2.33): - changed notes about peak-reports and unsorted input - removed note about Calamaris eating up all memory, as that problem is found and fixed. 2002-01-18 19:29 cord * calamaris.1 (1.19): - added documentation about the Debug-switch -L 2002-01-18 19:28 cord * calamaris (2.44): - fixed a bug in the new -D-option. now you should be able to set it yourself. (fix by John Line (webadm@info.cam.ac.uk)) 2002-01-18 19:27 cord * README (2.32): - removed the part about a probably buggy input-file guessing from the Bug-section. It's in there for months and nobody complained about it. - added a notice about a possible bug somewhere else, causing Calamaris to eat all your Memory. Reason unknown yet. Hints welcome. (reported by John Line (webadm@info.cam.ac.uk)) 2002-01-17 22:04 cord * calamaris.1 (1.18): * added -D-option. * clarified NetCache-Versions. * fixed typo. * bumped Version-Number for Calamaris * extended Copyright notice. 2002-01-17 21:55 cord * calamaris (2.43): * expanded Copyright-Notice. Hey! Calamaris is 5 years old! * added a new report about size-based Distribution of objects. (suggested by Gerold Meerkoetter (gerold@noc.fh-lippe.de)) * the -a-switch contains the new switch -D * clarified NetCache-Versions. (thanks to Stephane Lentz (Stephane.Lentz@ansf.alcatel.fr)) * added a -L-switch for debugging purposes (idea by Matt Hubbard (m.hubbard@ic.ac.uk)) * fixed 'gt' vs. > -bug in invalid counter (first reported by Gerold Meerkoetter (gerold@noc.fh-lippe.de)) * also fixed about 10 more similar bugs (maybe now i can remember when which is correct?) * added 'cs-username' as possible ident-content and cs(X-Forwarded-For) to ignore in elff-format. (patch by Matt Hubbard (m.hubbard@ic.ac.uk)) * quoted unquoted POST. 2002-01-17 20:46 cord * README (2.31): * added NetCache V5.x to the supported Proxies. * added wishlist-item about a timerange-option. (suggested by Steffen Sledz ) 2002-01-17 20:42 cord * COPYRIGHT (2.2, stable): The FSF (in person of Janet Casey) pointed out, that the 'How to Apply These Terms to Your New Programs' is a part of the GNU GPL. So i corrected this now. 2001-08-12 17:19 cord * calamaris (2.42): calamaris: * fixed a bug in Logfile-guessing 2001-03-24 22:40 cord * calamaris (2.41): * fixed bug in cache-reading routines witn unquoted keys. (noted by Elrond (elrond@Wunder-Nett.org)) * added support for NetApp NetCache V (thanks to Christian Niederdorfer (christian.niederdorfer@infineon.com) for providing logfiles, testing and also for the donmation. Thank You.) * fixed a bug in cache-reading of Requesters. (fixed by Klaus Brinkmeyer (Klaus_Brinkmeyer@inasys.de)) * corrected indentation * added 2001 to the copyright. (uh, since 1997. Calamaris is 4 now.) * added a warning to the output if more that 5% of the input lines are unparsable. 2001-03-24 22:27 cord * README (2.30): * changed Link to Inktomi Proxy * added 'NetApp NetCache V' to the supported Proxies. * overworked pointers to deb/rpm and *BSD-Ports. 2000-10-27 15:52 cord * calamaris.1 (1.17): * added 'nse' (Netscape Extended Logfile-Format) to the manpage. * added a workaround-hint for reusing cache-files * the manpage now described V2.40 and later of calamaris 2000-10-27 15:50 cord * calamaris (2.40): * improved Netscape Extended-1/2 Logfile Format support * fixed a seldom occuring problem with some perl-versions (reported by Stephen Welker (stephen.welker@nemostar.com.au)) 2000-10-27 15:48 cord * README (2.29): * added Netscape/iplanet Web Proxy Server to the supported servers. * added a pointer with a workaround to a problem with reusing cache-files. (problem noted by Clare Lahiff (clare@tarboosh.anu.edu.au)) 2000-10-27 15:42 cord * EXAMPLES (2.8): added an EXAMPLE for usage of caching taken from the Debian-package (written by Philipp Frauenfelder (pfrauenf@debian.org)) 2000-10-12 19:14 cord * calamaris (2.39): fixed the unit-bug again. 2000-10-12 19:03 cord * calamaris.1 (1.16): changed the pointer to the Calamaris Version. 2000-10-12 19:03 cord * calamaris (2.38): fixed a bug regarding units on byte values. 2000-09-18 19:25 cord * calamaris (2.37): * fixed a bug with undefined variable 2000-09-18 19:20 cord * calamaris (2.36), calamaris.1 (1.15): * added a -U -option to define the Unit used on the Byte-stats. (suggested by Steven Snyder (Steven_Snyder@3com.com)) 2000-09-14 19:15 cord * calamaris (2.35): * added Novell ICS to -f elff-help. * we get $log_url now from cs-uri-stem instead of cs-uri in elff-logs. * added cs(Referer)-field for elff. all reported and patched by Ram Cherukuri (ram@edgix.com) 2000-09-14 19:08 cord * README (2.28): added link for Novell Internet Caching System 2000-09-02 22:36 cord * calamaris (2.34): * fixed a bug in Logfile-guessing. * added alpha-quality support for Netscape-Extend-1/2 Logfiles. 2000-09-02 22:34 cord * README (2.27): added a statement. 2000-06-24 16:00 cord * calamaris.1 (1.14): * changed default for -a to -r 20 * added -R-Option. * added a new Section 'PRIVACY' 2000-06-24 14:56 cord * calamaris (2.33): * changed the one-line description. * changed the -a option to -r 20. * added -R-option to extend -r to report targets. * removed cts (Compaq Tasksmart)-Logformat. * added elff (Extended Logfile Format)-Logformat. * corrected help about -T * added another pointer to privacy-problems to help. 2000-06-24 14:44 cord * README (2.26): * checked referenced documents and corrected them where nessecary. * changed 'Which formats are supported?' to 'Which software can produce Calamaris-parseable Logfiles?' * changed 'How to use it?' from Squid-specific to a more general description. and added a pointer to Extended Logfile Format. * removed some pointers to known problems as nobody noted these for longer than a year. * added a harsh comment about (mis-?)using Calamaris to break the privacy of users. * removed the comment about not adding privacy-breaking functions. 2000-06-22 12:58 cord * calamaris.1 (1.13): * changed short description. * added all known Caches which can produce Calamaris-parseble logs. * changed the -f Default. * corrected -T description. 2000-06-22 12:06 cord * calamaris (2.32): * added support for 'Oops', a forking Squid-relative. * added support for Compaq Tasksmart. * changed guessing messages. * made guessing the default. 2000-06-22 12:00 cord * README (2.25): * added a new section 'Which formats are supported?' * changed the comment on Logfile-guessing. * removed 'Compaq Cache' from wishlist. 2000-05-25 19:44 cord * EXAMPLES (2.7): fixed a small bug in Thomas Wahyudis Example (noted by himself) 2000-05-24 23:34 cord * README (2.24): added a note about the to come support of Compaq Cache 2000-05-24 23:31 cord * calamaris (2.31): * added a new hierarchie-method of Inktomi Traffic Server --> EMPTY (patched by Warren Brown (wbrown@inktomi.com)) 2000-05-24 23:18 cord * EXAMPLES (2.6): * added another example provided by Thomas Wahyudi (thomas@home.unpar.ac.id) 2000-05-13 22:06 cord * calamaris.1 (1.12): * added descriptions for the new logfile formats. * added descriptions for -T, -c and -v * changed -c and -v to -C and -V. * manpage now reflects V2.30 and above. 2000-05-13 22:03 cord * calamaris (2.30): * corrected Dancer Vesperman's credit. * added a new logfile-type for Inktomi Traffic Server. (reported by Michael Smith (good_guy93@hotmail.com) * added a pseudo logfile type 'auto' which let's Calamaris guess what logfile is read. (still beta, please test) * added a verbose switch, (but it didn't do more yet as displaying which kind of logfile is guessed, maybe i should move the -b-switch in here?) * added a switch which enables correcting the Performance Report to a specific timezone. (noted by Holger Marzen (hm@sik-gmbh.de) * added a switch to make the file-extensions report case-insensitive. (requested by Steve Snyder (swsnyder@home.com) * changed the copyright switch from -c to -C. * changed the version switch from -v to -V. * changed the output of byte values: now all output is in Byte (not KByte anymore) if the space in the output would cut off something Calamaris will switch to KByte, MByte, GByte and TByte (I think this is enough ;-) indicating this by appending a K, M, G or T to the value. I saved a digit in the output here and gave it to the requests. (a problem with this is reported by Panagiotis Christias (P.Christias@noc.ntua.gr) 2000-05-13 21:01 cord * README (2.23): * Removed a hint for very old browsers. * Added a HELP REQUEST according the new Logfile-guessing. * Changed the comment on Common Logfile-Format. * Added a reply to the many requests about more user tracking with Calamaris. 2000-05-13 20:56 cord * EXAMPLES (2.5): * added an example provided by Gottfried Hamm (ghamm@ghks.de) 1999-12-16 23:56 cord * calamaris (2.29): * Bugfix to 2.28. 1999-09-25 14:33 cord * calamaris (2.28): * the benchmark option gives now hash-signs which were written to STDERR. (patch provided by Gerold Meerkoetter (gerold@noc.fh-lippe.de)) * all warn-calls were replaced by print to STDERR, so get rid of the automagically apended line-numbers. * now Calamaris checks the syntax of Logfilelines a little more, because lines such as 'if needed, or if running Squid for the first time.' lead to strange errors. A correct line has to have a date-field with the syntax \d+\.\d\d\d (reported by Gerold Meerkoetter (gerold@noc.fh-lippe.de)) * if ident-information parsing is activated, but there isn't an ident-information available now the ouput is without the leading '-@'. (patch by Chris Teakle (ccteakle@its.uq.edu.au)) * added new hierarchie-method 'ANY_PARENT'. * added a small invisible Y2K-fix. Now Calamaris uses internal the correct 4-digit-year. However Calamaris only prints 2-digit-years because i want to produce lines which were max. 79 chars long. (patch by Toni Andjelkovic (toni@telecom.at)) 1999-09-25 11:47 cord * README (2.22): * Feature request added: n-level requester report. (suggested by Jarkko Saloranta ) 1999-06-08 21:48 cord * calamaris (2.27): fixed parsing of squid-mime Logfiles. 1999-06-05 14:31 cord * calamaris (2.26): * changed the 'require 5' to 'require 5.002' because of vars.pm. (pointed out by Jonas Luster (jonas@nethammer.qad.org)) * fixed a bug regarding 'Outgoing requests by destination' where the approximate request time was calculated wrong for the Status-lines. (Clare Lahiff (Clare.Lahiff@anu.edu.au) reported this bug.) 1999-05-11 20:22 cord * calamaris (2.25), calamaris.1 (1.11): * added a new input-format 'squid-mime' for log_mime_hdrs-enhanced Squids. 1999-05-01 21:01 cord * calamaris (2.24): * added a new kind of ERROR from NetCache (reported by Ryan Donnelly) * changed the unknown log_hier_method -error, because many people simply sent the output to me, without thinking if the bug could be somewhere else (corrupted Logfile) or fixed in a later release. * cleaned up the HTML-Output (suggested and patch by Christos Cheretakis) because better HTML will be rendered faster (and weblint is now also happy ;-) 1999-05-01 20:55 cord * README (2.21): * clarified Mailinglist-Subcription. * changed link to FreeBSD-Port. * added a point to 'bugs and shortcomings regarding corrupted Logfiles. 1999-04-02 17:19 cord * README (2.20): * added a confirmation-request on wrong SSL-Request parsing. * added a suggestion from John Line ) about seperate on/off-switching for UDP/TCP-Requests-reports. 1999-04-02 17:17 cord * calamaris (2.23): * Copyright an Usage-Information will now also appear on STDOUT. * added new symbolic Protocol, Content-Type and extension for SSL-Requests. * all INVALID_URL-Requests will now be reported as symbolic in the TCP-Request report. * moved Nameserverlookups back into the main loop to avoid double reportings of Hosts, which were sometimes resolved by Squid, and sometimes not. (reported by John Line (webadm@info.cam.ac.uk)) * moved checking for -n also into the mainloop. * moved CARP-Requests from the 'FETCH from Parent Cache' to the 'HIT on Sibling or Parent Cache'-section of the Outgoing-request-status. (Hope this is correct, but the Reports provided by Emmanuel Adeline (emmanuel.adeline@mail.dotcom.fr) look to me like this is the correct section for CARP.) * the Performance-Report is now sorted correctly. * added some   to the HTML-Report to make the HTML-Report more readable. 1999-04-02 16:59 cord * EXAMPLES (2.4): added an Example of Matthew King. 1999-02-20 20:33 cord * calamaris (2.22): * bugfix for the old peak-calculation method. * added format-info to reports. 1999-02-20 18:59 cord * calamaris.1 (1.10): changed reference Id for Calamaris. 1999-02-20 18:45 cord * calamaris (2.21): * added new option -l (logo) for adding a customizable head to HTML-Report. * fixed (again) a small problem with -H option. * changed the hack to parse broken NetCache 3.2.x-Logfiles. The old hack broke some lines if a semicolon is the last char of a field. * all hostnames converted to lowercase. NuTsCaPe and nUtScApE as host is now recognized as the same. * fixed something with NetCaches ICP-Requests. * TIMEOUT_NONE is now recognized, Calamaris doesn't complain anymore. * added a HTML 4.0-header to the HTML-Report. (thanks to Iain Lea) * Performance-Report corrected. ICP-Request are not counted anymore. * Fixed a Y2K-Problem :-) Y2K has been reported as 100, which is correct, but this breaks my 80-chars text-width, and it is not that intuitive. Now Y2K will show up as 00. (Internal I calculate all with UNIX-epoch (seconds since 1.1.1970). A quick test shows that Calamaris should work 'til 19.1.19^H^H2038 ;-) However: Y2K-Statement: Calamaris comes with ABSOLUTELY NO WARRANTY. * added Homepage-Location to the Copyright-Footer. * added '-- ' to seperate footer and report in Mail-reports. * added links to the HTML-Copyright-footer. * cleaned up (printf --> print where possible.) 1999-02-20 18:12 cord * README (2.19): moved the Common-Log-Statement from todo to bugs&shortcomings. NetCache is now in late beta-state. Removed some warnings :-) cleaned up. 1999-02-20 18:07 cord * calamaris.1 (1.9): cleaned all up a bit (mainly in the groff-source) added new option -l 1999-02-06 22:08 cord * README (2.18), calamaris (2.20), calamaris.1 (1.8): NetCache-Support changed. removed the Support for Common-Log-style Logfiles. added a -f-option type for the default. 1999-02-04 23:39 cord * calamaris (2.19): fixed a bug regarding old squid logfile support. 1999-02-04 23:30 cord * calamaris.1 (1.7): chnaged a few things to reflect changes of V2.19 and later. 1999-02-04 23:29 cord * calamaris (2.18): * added new -f -option for selecting input logfile format * worked on NetCache-Support, now Calamaris should understand another format. (untested) * added support for pre-V1.1.beta26-Squid-Logfiles. * changed -p -option to select between old (2.8) and new (2.16) peak-measurement method * removed -p -option from the -a -switch. 1999-02-04 23:18 cord * README (2.17): changed statements regarding NetCache and peak mesurement. added Help Requests to these points. 1999-02-03 23:27 cord * README (2.16): Added pointer to the FreBSD-Calamris-Port. 1999-01-22 22:06 cord * README (2.15): added a todo. 1999-01-22 22:05 cord * calamaris (2.17): made the error message more helpful. 1999-01-22 22:02 cord * calamaris.1 (1.6): only a little reformatting. 1998-12-09 23:23 cord * README (2.14): looked through it and cleaned up again... 1998-12-09 23:21 cord * calamaris.1 (1.5): taken over the manpage... there is more work to be done... 1998-12-06 17:36 cord * README (2.13): added manpage to the installation-'manual' changed the comments on the new Squid-generation changed comments on peak-measurement added a comment about Common Logfileformat 1998-12-05 15:12 cord * calamaris (2.16): fixed small bug to handle (drop) negative reqtimes. 1998-12-05 14:18 cord * calamaris (2.15): reworked the measurement of peak-usage. Hour-data is now calculated in hour-steps, and a throughput peak is also there. The changes make Calamaris about 20% faster (on a 60k-line-file compared with V2.8). With bigger Logfiles the gain should be greater. However: The Peakmeasurement is the less efficient part of Calamaris, if you switch it off Calamaris will be 30% (with 60k-lines) or more faster. (I'm not glad with it, maybe someone has an idea how to build a routine which is fast AND more reliable?). made some changes in line-formating and some cosmetic changes in the code. included new adress for the Calamaris-Homepage. 1998-12-05 14:07 cord * README (2.12): Included new adress for the Calamaris-Homepage. 1998-12-05 14:06 cord * EXAMPLES (2.3): wrapped the long lines and added a warning. 1998-12-05 11:42 cord * calamaris (2.14): Added a few suggestions by Gerold Meerkoetter 1998-11-18 00:08 cord * calamaris.1 (1.4): A manpage for Calamaris. 1998-11-17 23:51 cord * README (2.11): Fixed some faults. Thanks to gerold Meerkoetter for pointing me on them. 1998-11-16 22:56 cord * calamaris.1 (1.3): New Manpage for Calamaris Many thanks to Philipp Frauenfelder 1998-11-16 22:52 cord * EXAMPLES (2.2), README (2.10): changed the name of calamris.pl to calamaris 1998-11-16 22:44 cord * calamaris (2.13): save the previous revisionnumbers from calamaris.pl 2.13 is the actual number of calamaris 1998-11-16 22:42 cord * calamaris (1.2), calamaris.pl (2.13): Renamed calamaris.pl to calamaris to make it more logically with the new manpage. 1998-11-16 22:25 cord * calamaris.1 (1.2): A manpage for Calamaris. Many thanks to Philipp Frauenfelder for it. 1998-10-23 00:37 cord * calamaris.pl (2.12): found another performance-thing... Now it is ~8% faster than 2.8 (last release) 1998-10-22 21:36 cord * calamaris.pl (2.11): moved $perf-initialisation to the TCP-Part. cleaned Calamaris up. Calamaris is now ~5% faster, if you use '-a' 1998-10-20 11:37 cord * README (2.9), calamaris.pl (2.10): Added new switch for chnaging the sort order in reports from number of requests to size of requests. 1998-10-19 00:06 cord * README (2.8): on Squid V2.0.x: I'm still looking for a report of a CARP-User how Calamaris works for him. I removed the todo 'rewrite Mainloop'. I have build it in, and now the Mainloop is build first and then run. But: It doesn't work as good as i hoped. On my measurements on my (slow) 386DX40 with a very short access.log it is slower, with a longer file (60k lines) it is faster by about 2% 1998-10-18 23:37 cord * calamaris.pl (2.9): Moved logfile-parsing part into a pre-run-part. Now the Mainloop for parsing is build before running it. Static things like switches are build in... 1998-10-14 22:56 cord * README (2.7): Added pointers for rpm and Debian-Packages 1998-10-12 22:03 cord * README (2.6), calamaris.pl (2.8): Had a better idea how to handle the broken data while showering :-) Now wrong data is printed as - in the Report. 1998-10-12 21:26 cord * calamaris.pl (2.7): fixed breaking old (buggy) cachefile. Now old and new Cachefile work without complaining... But with old cachefiles the performance-values of Cache-Hits are wrong... 1998-10-12 21:22 cord * README (2.5): added text about the Cachefile-breaking Bug. 1998-10-12 20:41 cord * calamaris.pl (2.6): found and removed no longer needed tcp_miss_neighbor-strings. moved caching for peakvalues into the $opt_P-Option. corrected (again) the measurement of transfered data in the Performance-Report 1998-10-12 20:37 cord * README (2.4): added a statement regarding perl 5.001 1998-10-10 20:51 cord * calamaris.pl (2.5): fixed more bugs regarding cached Performance... 1998-10-10 19:16 cord * calamaris.pl (2.4): added better handling of cached 'other'. 1998-10-10 15:39 cord * calamaris.pl (2.3): fixed bug regarding caching function. WARNING: This breaks cached Performance-data from old cachefiles. 1998-10-10 15:35 cord * README (2.3): added another todo 1998-10-07 19:41 cord * calamaris.pl (2.2): fixed bugs reported ans patched by Roar Smith 1998-10-07 19:40 cord * README (2.2): added wish for switch the sorting between requests and size 1998-10-05 20:50 cord * COPYRIGHT (2.1), EXAMPLES (2.1), README (2.1), calamaris.pl (2.1): no program changes. checked the spelling and rewrote a few parts of the README 'official' release of Calamaris V2 1998-10-03 14:33 cord * calamaris.pl (1.125): added CACHE_DIGEST and CARP 1998-09-30 19:46 cord * README (1.5): added a sentence to 'What Is It?' 1998-09-30 19:35 cord * calamaris.pl (1.124): changed location of Calamaris-Homepage. changed adress of Announcementlist 1998-09-30 19:33 cord * README (1.4): changed location of calamaris-Homepage. added link to the calamaris C++ port Seafood. 1998-09-25 21:02 cord * README (1.3): Cleaned up README. 1998-09-24 23:56 cord * EXAMPLES (1.2): Added Version-Information. 1998-09-24 23:52 cord * EXAMPLES (1.1): New File for Examples of calamaris usage. 1998-09-24 23:39 cord * README (1.2): Damned! CVS lost my first rewrite. OK, so here we go again. This is the text which was formerly at the beginning of calamaris.pl. Now it's more verbose, HTH 1998-09-24 20:07 cord * calamaris.pl (1.123): Fixed bug in Performancereport: Transfered Data is now real given in MBytes 1998-09-23 23:03 cord * COPYRIGHT (1.1), README (1.1), calamaris.pl (1.122): Moved READMEs out of calamaris into the New README file Added GPL-File 1998-09-22 23:43 cord * calamaris.pl (1.121): changed Performance-report: now zero-sized measures are printed as '-' instead of 0.00 1998-09-21 23:29 cord * calamaris.pl (1.120): added more usage-information. Hmmm... i think i'm going to wait a few weeks and then i release this as 'official' calamaris v2 1998-09-21 21:14 cord * calamaris.pl (1.119): added default-values for the -a -Option changed caching to add many cachefile at once fixed a bug in the Syntaxcheck of the input cachefiles. removed checks for -a-option 1998-09-19 00:30 cord * calamaris.pl (1.118): changed URL for calamaris added Announcement-Mailinglist added -P-switch for measuring performance values added new report for it removed transfervariable for peakmeasurement 1998-08-12 23:48 cord * calamaris.pl (1.117): added CLOSEST_PARENT method wrote a few lines on NetCache-Support 1998-08-12 23:04 cord * calamaris.pl (1.116): changed syntaxcheck of Logfile for parsing NetCache-Logs 1998-07-19 13:41 cord * calamaris.pl (1.115): fixed small bug in hostnamelookup 1998-07-18 23:52 cord * calamaris.pl (1.114): moved DNS-Lookups from the parsing part into the output part of calamaris 1998-07-09 21:56 cord * calamaris.pl (1.113): added support for parsing NetCache-Logfiles wrote a few lines in the README-Section 1998-07-09 21:04 cord * calamaris.pl (1.112): added reporting number of unique clients. fixed small bug with 1998-05-10 23:39 cord * calamaris.pl (1.111): fixed bug in content-type-limiting 1998-05-10 22:54 cord * calamaris.pl (1.110): added 3 todos changed usage line added Content-type for HTML-Mails *urgs* 1998-05-09 17:36 cord * calamaris.pl (1.109): fixed hostname output (again) removed bogus Debug-line corrected usage line 1998-04-15 23:24 cord * calamaris.pl (1.108): added Link for jumping back to the Top in HTML-Output fixed small bug in the content-type report (other content-types) wrapped long lines where possible. 1998-04-09 22:44 cord * calamaris.pl (1.107): changed output string. 1998-04-07 22:06 cord * calamaris.pl (1.106): now report ERR_INVALID_URL correct as error 1998-04-07 21:26 cord * calamaris.pl (1.105): added Referers into HTML-Output for indocument links. 1998-04-07 20:31 cord * calamaris.pl (1.104): fixed small bug with hostname-output. 1998-04-06 00:50 cord * calamaris.pl (1.103): changed short variables back into long ones. 1998-01-17 19:19 cord * calamaris.pl (1.102): at this point i remember tht there was 1.99.1.1, i put that changes in the 'official' devel branch. RCS simply fooled me ;-( replaced -f-option with -i and -o option moved -o-option into -H-option readded syntaxcheck of cachefiles 1998-01-15 23:34 cord * calamaris.pl (1.101): chnged behaviour of error reporting 1998-01-11 22:25 cord * calamaris.pl (1.100): This version is based on 1.99 added -o option for defining Hostname added more TLD's with functional subdomains. 1997-12-29 21:53 cord * calamaris.pl (1.99.1.1): changed the Cachefile-Option -f to two options -i and -o added syntaxcheck to cachefileparser fixed a few bugs 1997-12-28 22:51 cord * calamaris.pl (1.99): i choose this number to give the old calamaris room to grow shorten all variables to make calamaris shorter and faster added caching of data added -z option for only parsing cached data 1997-12-23 21:52 cord * calamaris.pl (1.1): cleaned up calamaris v1 added command-line-options made peak-measurement more effective added HTML-Output-option added new reports and changed some old reports moved diverse functions into subroutines This is the initial revision of calamaris v2 calamaris-2.99.4.0/calamaris.conf0000644000175000001440000010454610407330372015373 0ustar cordusers############################################################################### ################## CONFIGURATION FILE FOR CALAMARIS V3 #################### ############################################################################### # # Configuration file for calamaris V3.x # # This configuration file follows the perl syntax. To define variables, just # do it as you are used to do it. # $var = value; # undef($var); # same as $var = 0; # # To run calamaris with a configuration file, use: # cat access.log | ./calamaris --config-file calamaris.conf # # All command line arguments overwrites the configuration file items. # # Most of the configration items can be configured by commandline # arguments. Try # ./calamaris --help # # Other commandline arguments: # -L or --dump-loop: Loop (dumps the generated internal loop to STDERR for # debugging.) # # -C or --copyright copyright (prints the copyright) # -h or --help help (prints out this message) # -V or --version Version (prints version-info) # # On each section, you find a small description, the command line argument if # available, an example output (to give you an idea of this section), the # default value and how to use in this configuration file. # ############################################################################### # # IMPORTANT: # (*) These options break the privacy of your users. Please read the README # on this. # # Some items have to be configured in this configuration file, because no # command line options are available. Otherwise default values are taken! # ############################################################################### ############################################################################### ############################# REPORT SECTION ############################## ############################################################################### ############################################################################### # # show 'n' Top-level and 'n' second-level destinations, # -1 = unlimited # # command line argument: -d n | --domain-report n # # Example output: # # Request-destinations by toplevel-domain # destination request % Byte % hit-% # --------------------------------------- --------- ------ -------- ------ ------ # *.de 360753 51.58 4576770K 47.41 22.48 # *.com 202536 28.96 4684139K 48.52 18.88 # # Request-destinations by 2nd-level-domain # destination request % Byte % hit-% # --------------------------------------- --------- ------ -------- ------ ------ # *.plaxo.com 36 10.00 84835 8.00 0.80 # *.ebay.com 20 7.40 44735 5.50 1.10 # # # Default: # undef($domain_report); # (no top- and second-level destinations are # reported) # # Usage: # $domain_report = n; # ############################################################################### # # limit the display of lines to those with a minimum of requests. # # command line argument: --domain-report-limit n # # Default : # undef($domain_report_limit); # no limit. # # Usage: # $domain_report_limit = n; ############################################################################### # # change all 2nd-level-reports to N-level-reports. 'n' can be # any number from 2 up. -1 means full report. # This is only useful, if $domain_report is set (see above). # # command line argument: -N n | --domain-report-n-level n # # # Default: # undef($domain_report_n_level); # (reporting of second-level destinations) # # Usage: # $domain_report_n_level = n; # (reporting of n-level destinations) ############################################################################### # # Error code distribution # # command line argument: --errorcode-distribution-report # # Example output: # # TCP Response code distribution # status-code request % Byte % # ----------------------------------- --------- ------ -------- ------ # 000 (Used mostly with UDP traffic) 727633 2.93 3625190K 1.97 # 100 (Continue) 4 0.00 687 0.00 # 200 (OK) 20145830 81.19 171223M 95.12 # # # Default: # undef($errorcode_distribution_report); # (no reporting) # # Usage: # $errorcode_distribution_report = [0|1]; ############################################################################### # # Object freshness report # Here you can find information about the freshness of objects in your cache. # # Calamaris looks for freshness tags like 'TCP_HIT', 'TCP_REFRESH_MISS', ... # and make statistics on it. With this information you can optimize the # caching behaviour of your cache depending on the objects content type. # E.g. squid admins could use this information to configure the # refresh_pattern. # # Default: # undef($object_freshness_report); # (no reporting) # # Usage: # $object_freshness_report = [0|1]; # Here you have to define the 'TRANSACTION HEADER' for your cache software! # # Note: the variable $refresh_tags is not used at the moment # Usage: the hash keys have to be one of this (see --input-format): # squid, squid-old, nc, elff, its or nse # Please adjust the 'TRANSACTION HEADER' according your cache manual. # # $xxx_tags{'hash keys'} = [( 'TRANSACTION HEADER 1', # 'TRANSACTION HEADER 2', # 'TRANSACTION HEADER 3' ... )]; # # $fresh_tags: all fresh hits # $stale_tags: all stale misses # $refresh_tags: not used at the moment # $mod_tags: hits which have been modified after an IMS # $unmod_tags: hits which have not been modified after an IMS # # Explanation: # all hits = $fresh_tags + $stale_tags # $stale_tags = $mod_tags + $unmod_tags # # The following list is not complete (as you can see) and may be incorrect. # If there is a need of improvement please notify. #Tags for squid $fresh_tags{'squid'} = [( 'TCP_HIT', 'TCP_MEM_HIT', 'TCP_IMS_HIT', 'TCP_IMS_MISS' )]; $stale_tags{'squid'} = [( 'TCP_REFRESH_HIT', 'TCP_REFRESH_MISS', 'TCP_REF_FAIL_HIT' )]; $refresh_tags{'squid'} = [( 'TCP_CLIENT_REFRESH' )]; $mod_tags{'squid'} = [( 'TCP_REFRESH_MISS' )]; $unmod_tags{'squid'} = [( 'TCP_REFRESH_HIT' )]; #Tags for squid-old $fresh_tags{'squid-old'} = [( 'TCP_HIT', 'TCP_MEM_HIT', 'TCP_IMS_HIT', 'TCP_IMS_MISS' )]; $stale_tags{'squid-old'} = [( 'TCP_REFRESH_HIT', 'TCP_REFRESH_MISS', 'TCP_REF_FAIL_HIT' )]; $refresh_tags{'squid-old'} = [( 'TCP_CLIENT_REFRESH' )]; $mod_tags{'squid-old'} = [( 'TCP_REFRESH_MISS' )]; $unmod_tags{'squid-old'} = [( 'TCP_REFRESH_HIT' )]; #Tags for NetCache 5.5 $fresh_tags{'nc'} = [( 'TCP_HIT', 'TCP_HIT_ACCESS_DENIED', 'TCP_HIT_EJECT', 'TCP_HIT_HIT_PARTIAL', 'TCP_HIT_HIT_VERIFY' )]; $stale_tags{'nc'} = [( 'TCP_HIT_IMS_NOTMOD','TCP_HIT_PRECONDITION_FAILED', 'TCP_MISS_CACHE_VERSION', 'TCP_MISS_VERIFY', 'TCP_REF_FAIL_HIT' )]; $refresh_tags{'nc'} = [( 'TCP_MISS_RELOAD' )]; $mod_tags{'nc'} = [( 'TCP_MISS_VERIFY' )]; $unmod_tags{'nc'} = [( 'TCP_HIT_IMS_NOTMOD' )]; $fresh_tags{'elff'} = [( 'TCP_HIT', 'TCP_HIT_ACCESS_DENIED', 'TCP_HIT_EJECT', 'TCP_HIT_HIT_PARTIAL', 'TCP_HIT_HIT_VERIFY' )]; $stale_tags{'elff'} = [( 'TCP_HIT_IMS_NOTMOD','TCP_HIT_PRECONDITION_FAILED', 'TCP_MISS_CACHE_VERSION', 'TCP_MISS_VERIFY', 'TCP_REF_FAIL_HIT' )]; $refresh_tags{'elff'} = [( 'TCP_MISS_RELOAD' )]; $mod_tags{'elff'} = [( 'TCP_MISS_VERIFY' )]; $unmod_tags{'elff'} = [( 'TCP_HIT_IMS_NOTMOD' )]; ############################################################################### # # measure peak requests # # command line argument: -p [new|old] | --peak-report [new|old] # # Example output: # # Incoming request peak per protocol # prt sec peak begins at min peak begins at hour peak begins at # --- ---- ------------------ ----- ------------------ ------- ------------------ # UDP 0 0 0 # TCP 348 04.Apr 04 04:33:58 3059 04.Apr 04 02:01:18 82479 04.Apr 04 23:00:00 # --- ---- ------------------ ----- ------------------ ------- ------------------ # ALL 348 04.Apr 04 04:33:58 3059 04.Apr 04 02:01:18 82479 04.Apr 04 23:00:00 # # Incoming transfer volume per protocol # proto kB/hour peak begins at # ----- -------- ------------------ # UDP 0 # TCP 1964368 04.Apr 04 22:00:00 # ----- -------- ------------------ # ALL 1964368 04.Apr 04 22:00:00 # # # Default: # undef($peak_report); # (no peak measurement is reported) # # Usage: # $peak_report = ['new'|'old']; # old = make old request-peak mesurement # 'Incoming transfer volume per protocol' is not calculated! # new = make new request&byte-peak measurement # (both slow Calamaris significantly down.) ############################################################################### # # Performance: show throughput data for every 'n' minutes # # command line argument: -P n | --performance-report n # # Example output: # # Performance in 60 minute steps # incomin hit miss direct sibling fetch # date request Byte kB/sec kB/sec kB/sec kB/sec kB/sec kB/sec # --------------- --------- ----- ------- ------- ------- ------- ------- ------- # 04.Apr 04 00:00 20217 211M 8.69 53.23 6.35 7.38 4.03 1.11 # 04.Apr 04 01:00 14685 169M 7.93 77.43 5.16 5.64 2.79 13.18 # 04.Apr 04 02:00 23450 388M 15.45 44.42 13.13 16.09 5.76 1.14 # 04.Apr 04 03:00 15092 284M 10.50 3.76 8.52 8.15 16.98 0.39 # # # Default: # undef($performance_report); # (no performance is reported) # # Usage: # $performance_report = n; ############################################################################### # # Time # adjust the Performance-Report in minutes # # command line argument: -T | --performance-report-adjust # # # Default: # undef($performance_report_adjust); # don't adjust the Performance-Report # # Usage: # $performance_report_adjust = [0|1]; ############################################################################### # # requester: show 'n' Requesters/User, -1 = unlimited # # command line argument: -r n | --requester-report n # # Example output: # # Incoming TCP-requests by host # host request hit-% Byte hit-% sec kB/sec # --------------------------------- --------- ------ -------- ------ ---- ------- # 10.1.1.1 186643 3.80 6366926K 0.22 1 55.18 # 10.1.4.2 86331 0.00 26654655 0.00 0 130.18 # 10.1.1.3 48482 7.43 13294970 20.33 0 53.73 # 10.1.5.10 48474 7.43 13289730 20.34 0 57.84 # # # Default: # undef($requester_report); # (no requester is be monitored) # # Usage: # $requester_report = n; ############################################################################### # # no-lookup # don't look IP-Numbers up # # command line argument: -n | --requester-report-no-dns-lookup # # # Default: # undef($requester_report_no_dns_lookup); # lookup IP-Numbers # # Usage: # $requester_report_no_dns_lookup= [0|1]; ############################################################################### # # use ident information if available (*) # # command line argument: -u | --requester-report-use-user-info # # # Default: # undef($requester_report_use_user_info); # don't use login information # # Usage: # $requester_report_use_user_info= [0|1]; ############################################################################### # # targets for requester: show 'n' targets for every Requester, # -1 = unlimited), implies $requester_report = n (*) # If $requester_report is not set, $requester_report is set to # $requester_report_with_targets. # # command line argument: -R n | --requester-report-with-targets n # # Example: # # Incoming TCP-requests by host # host / target request hit-% Byte hit-% sec kB/sec # --------------------------------- --------- ------ -------- ------ ---- ------- # 10.101.93.111 917 8.07 1529952 12.26 1 1.94 # *.the-arena.de 665 10.68 882366 14.49 0 35.11 # *.gmx.net 93 3.23 397865 15.00 0 66.17 # *.ebay.de 54 0.00 53779 0.00 0 24.69 # # # Default: # undef($requester_report_with_targets); # (no requester is monitored) # # Usage: # $requester_report_with_targets = n; ############################################################################### # # Response Time Diagram: # sum up the time distribution over all objects. # # command line argument: --response-time-report # # Example output: # # TCP-Request duration distribution in msec # time request % Byte % # --------------- --------- ------ -------- ------ # <= 0.1 88936 0.36 80003410 0.04 # <= 0.2 88936 0.36 80003410 0.04 # <= 0.5 88936 0.36 80003410 0.04 # <= 1 482528 1.94 339898K 0.18 # # # Default: # undef($response_time_report); # (no reporting) # # Usage: # $response_time_report = [0|1]; ############################################################################### # # Response Time Interval: # This array defines the time steps, which should be reported in the # 'Response Time Diagram', see above. # # # Default: # @response_time_report_interval = qw(0.001 0.01 0.02 0.05 0.1 0.2 0.5 1 2 5 10 20 50 100 200 500 1000 2000 5000 10000 20000 50000 100000 200000 500000 1000000 1e10); # # Usage: # @response_time_report_interval = qw(0.001 0.1 0.2 0.5 1 2 5 10 20 50 100 200 500 1000 2000 5000 10000 20000 50000 1e10); ############################################################################### # # Response Time Limit: # Calamaris calculates an average response time value, which is shown in the cache statistics overview. # Very slow requests (web server) can be skipped, by setting this value. # This value is only recognized, when $response_time_report is set. # # # Default: # $response_time_limit = $response_time_report_interval[$#response_time_report_interval]; # # Usage: # $response_time_limit = 10000; $response_time_limit = 2000; ############################################################################### # # Size-Distribution-Report: # shows size-based distribution of requested # objects, smaller numbers result in more verbose reports. # choose 2, 10 or 100 for useful output. (You can also play # with this ;-)) # # command line argument: -D [2|10|100] | --size-distribution-report [2|10|100] # # Example output: # # object-size (bytes) request hit-% Byte hit-% sec kB/sec # -------------------- --------- ------- ------- ------ ------- ------ # 0-0 138146 33.70 0 0.00 4 0.00 # 1-9 7 0.00 35 0.00 0 0.01 # 10-99 38240 0.00 2772033 0.00 10 0.01 # # # Default: # undef($size_distribution_report); # (no reporting) # # Usage: # $size_distribution_report = [2|10|100]; ############################################################################### # # status: show verbose status reports # # command line argument: -s | --status-report # # Example output: # # Summary # lines parsed: 699415 # invalid lines: 0 # unique hosts/users: 363 # parse time (sec): 136 # # # Default: # undef($status_report); # (no status reporting) # # Usage: # $status_report = [0|1]; ############################################################################### # # show 'n' content-type, 'n' extensions and requested protocols, # -1 = unlimited # # command line argument: -t n | --type-report n # # Example output: # # Requested content-type # content-type request % Byte % hit-% # --------------------------------------- --------- ------ -------- ------ ------ # text/html 223479 31.95 4600042K 47.65 11.02 # text/plain 116291 16.63 1460336K 15.13 0.74 # # Requested extensions # extensions request % Byte % hit-% # --------------------------------------- --------- ------ -------- ------ ------ # 202347 28.93 4720924K 48.90 4.65 # 139819 19.99 1480113K 15.33 5.59 # gif 115411 16.50 186475K 1.93 62.50 # # # Default: # undef($type_report); # (no reporting) # # Usage: # $type_report = n; ############################################################################### # # switch to case-insensitive reporting (useful for extensions-report) # # command line argument: -c | --type-report-ignore-case # # # Default: # undef($type_report_ignore_case); # make the reports case sensitive # # Usage: # $type_report_ignore_case= [0|1]; ############################################################################### ############################# INPUT SECTION ############################### ############################################################################### ############################################################################### # # Logformat type: # sets the type of input logfiles # auto = tries to guess the input format # (This is the Default) # squid = Native-Logfile derived from Squid V1.1.beta26-V2.x # squid-extended = Native-Logfile with log_mime_hdrs enabled # derived from Squid V1.1.beta26-V2.x (*) # or Cisco Content Engines (*) # or Squid with SmartFilter-patch (*) # squid-old = Native-Logfile derived from Squid # V1.1.alpha1-V1.1.beta25 # nc = Squid-style Logfiles derived from NetCache V?? (<5.x) # its = Logfiles derived from Inktomi Traffic Server # elff = Extended Logfile Format (i.e Compaq Tasksmart, Novell # Internet Caching System, NetCache 5.x) # nse = Netscape Extended-1/2 Logfile Format # # command line argument: -f [options] | --input-format [options] # # # Default: # $input_format = 'auto'; # auto detection # # Usage: # $input_format = ['auto'|'squid'|'squid-extended'|'squid-old'|'nc'|'its'|'elff'|'nse']; ############################################################################### # # IP-Filters: # # This item needs the perl Module NetAddr::IP! # The IP-list which should be excluded from reporting or which only should be # monitored (included) is separated by ':' # There are two kind of filters: exclude and include: # # exclude: # all IPs are reported, except IP/range. (*) # command line argument: --ip-filter-exclude List # # include: # no IPs are analyzed, except IP/range. (*) # command line argument: --ip-filter-include List # # List-Format: 1.1.1.1/32:1.1.2.0/24 # 1.1.1.1/255.255.255.255:1.1.2.0/255.255.255.0 # # # command line argument: # --ipfilter-exclude IP/range # --ipfilter-include IP/range # # Default: # undef($ipfilter_exclude); # no IPs are excluded # undef($ipfilter_include); # no IPs are included # # Usage: # $ipfilter_exclude = '1.1.1.1/255.255.255.255:1.1.2.0/255.255.255.0'; # $ipfilter_include = '1.1.1.1/255.255.255.255:1.1.2.0/255.255.255.0'; ############################################################################### # # no input via STDIN # This is useful when reading cache files. # # command line argument: -z | --no-input # # # Default: # undef($no_input); # (calamaris expects logfile via STDIN) # # Usage: # $no_input = [0|1]; ############################################################################### # # Interval # defines which time-interval should be parsed # t has to be the format yyyymmddhhmmss (localtime) # omitting the beginning or ending is allowed. # # command line argument: -I t1-t2 | --time-interval t1-t2 # # # Default: # undef($time_interval); # # Usage: # $time_interval= 't1-t2'; ############################################################################### ############################# OUTPUT SECTION ############################## ############################################################################### ############################################################################### # # Graph colours: # These values cannot be configured by commandline arguments! # If you want to change the default values, you have to use this # configuration items. # # Define the colours of the columns and text/axis/legend/labels. # The colours for Hit-% are automatically darkend. # # Default: # $column1_color = '#6699cc'; # $column2_color = '#ff9900'; # $text_color = '#222266'; ############################################################################### # # image type # Sets the image type to gif, png, jpeg, gd or gd2. Only usefull when # --output-format graph is set. The supported images types are dependend # on your GD::Graph installation. # Calamaris tells you which formats are supported, if an error occurs. # # # Default: # $image_type = 'png'; # # Usage: # $image_type = ['gif'|'png'|'jpeg'|'gd'|'gd2']; ############################################################################### # # output format of the tables # # Better don't play with this ;-), no warranty what happen with the output, if # not used correctly # Please consider: # - you can not change the order of the table columns. # - you can switch off the output of single columns by using the keyword 'off'. # - don't change the numbers or '%'. Do you know, what you are doing? # - you can change the output of Request/sec and Byte/sec by using the # following keywords: # # kbps = kByte/sec # spkb = sec/kByte # bps = Byte/sec # spb = sec/Byte # rps = req/sec # spr = sec/req # rpms = req/msec # mspr = msec/req # off # % # # Default: the output is optimized to 79 character per line. # # This units (keywords) are not influenced by $unit, see below. # Incoming requests by method $formats[3] = [ 30, 9, '%', 'spr', 8, '%', 'kbps' ]; # Incoming UDP-requests by status $formats[4] = [ 30, 9, '%', 'mspr', 8, '%', 'kbps' ]; # Incoming TCP-requests by status $formats[5] = [ 30, 9, '%', 'spr', 8, '%', 'kbps' ]; # Outgoing requests by status $formats[6] = [ 30, 9, '%', 'spr', 8, '%', 'kbps' ]; # Outgoing requests by destination $formats[7] = [ 30, 9, '%', 'spr', 8, '%', 'kbps' ]; # Request-destinations by ${N}-level-domain $formats[8] = [ 26, 9, '%', '%', 'spr', 8, '%', '%', 'kbps' ]; # Request-destinations by toplevel-domain $formats[9] = [ 16, 9, '%', '%', 'spr', 8, '%', '%', 'kbps' ]; # TCP-Request-protocol $formats[10] = [ 16, 9, '%', '%', 'spr', 8, '%', '%', 'kbps' ]; # Requested content-type $formats[11] = [ 26, 9, '%', '%', 'spr', 8, '%', '%', 'kbps' ]; # Requested extensions $formats[12] = [ 16, 9, '%', '%', 'spr', 8, '%', '%', 'kbps', 11, 11 ]; # Incoming UDP-requests by host $formats[13] = [ 16, 9, '%', '%', 'spr', 8, '%', '%', 'kbps' ]; # Incoming TCP-requests by host $formats[14] = [ 16, 9, '%', '%', 'spr', 8, '%', '%', 'kbps' ]; # Distribution Histogram $formats[15] = [ 16, 9, '%', '%', 'spr', 8, '%', '%', 'kbps' ]; # Performance in $P steps $formats[16] = [ 15, 9, '%', 5, '%', 6, 'kbps', 'kbps', 'kbps', 'kbps', 'kbps', 'kbps' ]; # UDP-Request duration distribution in msec $formats[17] = [ 16, 9, '%', '%', 'mspr', 8, '%', '%', 'kbps' ]; # TCP-Request duration distribution in msec $formats[18] = [ 16, 9, '%', '%', 'mspr', 8, '%', '%', 'kbps' ]; # UDP Response code distribution $formats[19] = [ 36, 9, '%', '%', 'spr', 8, '%', '%', 'kbps' ]; # TCP Response code distribution $formats[20] = [ 36, 9, '%', '%', 'spr', 8, '%', '%', 'kbps' ]; ############################################################################### # # Hostname # Define the name for the Report Output # 'lookup' issues a lookup for the current host # FQDN means, give an full qualified domain name or any name you want. # # command line argument: -H ['FQDN|'lookup'] | --hostname ['FQDN|'lookup'] # # # Default: # undef($hostname); # The report output headline is without any hostname # # Usage: # $hostname= ['FQDN|'lookup']; ############################################################################### # # logo # Here you can define a HTML-string which should be included into the report # head. You can also define a filename, then the file is included into the # report head. It works only in combination with # $output_format = 'html' or 'html,graph'. # # command line argument: -l [string|filename] | --logo [string|filename] # # # Default: # undef($logo); # no logo is included into the html head # # Usage: # $logo= ' HTML-TEXT '; # define string # $logo= './calamaris_head.html'; # or define file ############################################################################### # # Meta # includes the given strings in html-. You can also give a filename, then # the file is included in html-. It works only in combination with # $output_format = 'html' or 'html,graph'. # This is useful to include CSS or JavaScript. It's up to you, to include some # more CSS Tags in the calamaris output. You have to edit the source code. # # command line argument: -M [string|filename] | --meta [string|filename] # # # Default: # undef($meta); # (no meta information is included) # # Usage: # $meta = ''; # define string # $meta ='./calamaris.css'; # or define CSS-file $meta = ' '; ############################################################################### # # output format (comma-seperated list) # (Default is plain formatted text) # mail = mail format # html = HTML format # html-embed = HTML format without HTML-headers # html-frame = HTML format with Frames # graph = enable graphics, needs GD::Graph, only useful with # html, html-embed or html-frame # (see also $output_path) # unformatted = plain unformatted output # # command line argument: -F [options] | --output-format [options] # # # Default: # undef($output_format); # Default is plain formatted text # # Usage: # $output_format = ['mail'|'html'|'html-embed'|'html-frame'|'graph'|'unformatted']; ############################################################################### # # output path # output file # output file prefix # # defines where the output of calamaris is written. # # output_path defines the path where to write, # output_file contains the filename, where the data is stored. # output_file_prefix prepends a pattern to the filename. # # In case of html-embed,graph; html-frame,graph or html,graph output, # the graphics destination is the value of $output_path. If $output_path # is not given, all graphics are written to the working directory. './'. # # If $output_file is not given the Filename is index.html for html-output and # calamaris.txt for plain-output. # # If $output_file_prefix the pattern is prepended to all written files. # # You may use the special pattern # %h (value of $hostname) # %t (the calculated report-timerange) # %% (use this to get a single %) # in all three configuration-directives. # # command line argument: --output-path 'path' # command line argument: --output-file 'filename' # command line argument: --output-file-prefix 'prefix' # # Default: # undef($output_path); # Default is output path is working directory './' # undef($output_file); # Default is index.html or calamaris.txt # undef($output_file_prefix); # Default is no prefix # # Usage: # $output_path= '/path'; # $output_file= 'filename'; # $output_file_prefix= 'prefix'; ############################################################################### # # list Show # Shows only the defined reports (comma-seperated list) in the # specified order. The following numbers are defined (see ./calamaris -h): # 0 Summary # 1 Incoming request peak per protocol # 2 Incoming transfer volume peak per protocol # 3 Incoming requests by method # 4 Incoming UDP-requests by status # 5 Incoming TCP-requests by status # 6 Outgoing requests by status # 7 Outgoing requests by destination # 8 Request-destinations by 2nd-level-domain # 9 Request-destinations by toplevel-domain # 10 TCP-Request-protocol # 11 Requested content-type # 12 Requested extensions # 13 Incoming UDP-requests by host # 14 Incoming TCP-requests by host # 15 Size Distribution Diagram # 16 Performance in n minute steps # 17 UDP-Request duration distribution in msec # 18 TCP-Request duration distribution in msec # 19 UDP Response code distribution # 20 TCP Response code distribution # # Note: only putting out one report does not speed up Calamaris # as the internal operations were done based on the # report-switches. Default: Reports are displayed based on # activated reports. # # command line argument: # -S comma-separated-list | --show-reports comma-separated-list # # # Default: # undef($show_reports); # sort request size # # Usage: # $show_reports = 0,1,2,4,6; ############################################################################### # # Sort Order # changes the sort order in the reports to request size, # default is sorting by number of requests. # # command line argument: -O | --sort-order # # # Default: # undef($sort_order); # sort request size # # Usage: # $sort_order = [0|1]; ############################################################################### # # define the Unit for the Byte-values, else it will be auto # K(ilo), M(ega), G(iga), T(era) # # command line argument: -U option | --unit option # # # Default: # undef($unit); # Default is yoto formatting of unit. # # Usage: # $unit = [K|M|G|T]; ############################################################################### # # Define the graph size in pixel. # The image ratio (width : height) is 3 : 2. # # Default: # $width = 600; # the graph has a size of 600x400 Pixel # Usage: # $width = 300; ############################################################################### # # How many datasets should be drawn on the graph. # $x_scale = 30 is a good value, play with this -> no warranty what happens # if $x_scale is too big! # # Default: # $x_scale = 30; # Usage: # $x_scale = 10; ############################################################################### # # Generate an index-page for all reports that match the # output_*-Directives. # # Default: # undef($generate_index); # sort request size # # Usage: # $generate_index = [0|1]; ############################################################################### ############################# CACHE SECTION ############################### ############################################################################### ############################################################################### # # input-file # input-datafile for caching, to add many files separate them with a ':'. # # command line argument: # -i 'inputfile.dat' | --cache-input-file 'inputfile.dat' # # # Default: # undef($cache_input_file); # no input from cache file # # Usage: # $cache_input_file= 'file1:file2:file3'; ############################################################################### # # output-file # output-datafile for caching, can be the same as $cache_input_file # # command line argument: -o filename.dat | --cache-output-file filename.dat # # # Default: # undef($cache_output_file); # no cachefile will be written # # Usage: # $cache_output_file = 'cache.dat'; ############################################################################### ############################## MISC SECTION ############################### ############################################################################### ############################################################################### # # benchmark # prints a hash-sign (#) to STDERR for each n lines # processed # # command line argument: -b n | --benchmark n # # # Default: # undef($benchmark); # don't show hashes (#) # # Usage: # $benchmark = n; ############################################################################### ############################# DEBUG SECTION ############################### ############################################################################### ############################################################################### # # make some small tests (only for programmer) #$test = 1; ############################################################################### # # verbose # print information what Calamaris is doing. Useful for debugging. # # command line argument: -v | --verbose # # # Default: # undef($verbose); # don't write debug information # # Usage: # $verbose = [0|1]; ############################################################################### # #Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 # Cord Beermann. Authors: Cord Beermann and Michael Pophal #Calamaris comes with ABSOLUTELY NO WARRANTY. It is free software, and you are #welcome to redistribute it under certain conditions. See source for details. #Calamaris-Homepage: http://Calamaris.Cord.de/ # ############################################################################### # $Id: calamaris.conf,v 3.1 2006-03-19 17:58:25 cord Exp $ calamaris-2.99.4.0/calamaris.10000644000175000001440000004142210407330372014577 0ustar cordusers.TH "CALAMARIS" "1" "$Date: 2006-03-19 17:52:48 $" "Handmade" "Calamaris Manual" .SH "NAME" .B calamaris - generate text and graphical statistics out of log files from Proxy-Cache-Servers .SH "SYNOPSIS" cat log | .B calamaris [ --config-file .../calamaris.conf ] [ switches ] .SH "DESCRIPTION" .B Calamaris is used to produce statistical output from \fBSquid\fR, \fBNetCache\fR, \fBInktomi Traffic Server\fR, \fBOops! proxy server\fR, \fBCompaq Tasksmart\fR, \fBCisco Content Engines\fR or related Proxy log files. The resulting output can be ascii or html with or without graphic and with or without frames. It is possible to cache calculated data in a file to use them in later runs. This manual page describes the options of \fBCalamaris\fR and gives a few examples. .SH "OPTIONS" .SS Configuration File .TP .I --config-file file Not all reports and modification can be made through command-line-switches. To use all options of Calamaris you\'ll have to use the configuration file. You'll find the configuration-directives below, always inside of braces. Examples are in the calamaris.conf which should come with this package. .SS Reports .TP .I --all-useful-reports|-a extracts all useful reports available, .I --all-useful-reports equals .I --size-distribution-report 10 .I --domain-report 20 .I --performance-report 60 .I --requester-report 20 .I --status-report .I --type-report 20 .I --response-time-report .I --errorcode-distribution-report . .TP .I --domain-report|-d n ($domain_report) switches the top level and the second level report on. The data is derived from the URL. The output is limited by n. (-1 is unlimited) .TP .I --domain-report-limit n ($domain_report_limit) limit the domain-report to lines which have n or more requests. .TP .I --domain-report-n-level|-N n ($domain_report_n_level) All URL-Host reports will be switched from 2nd-level to n-level-reports. (-1 shows a full urlhost-report) .B Note: This option is only useful with activated domain-report. .TP .I --errorcode-distribution-report ($errorcode_distribution_report) shows the Response code distribution over all objects .TP .I ($object_freshness_report) shows the freshness of objects in your cache. .B Calamaris looks for freshness tags like 'TCP_HIT', 'TCP_REFRESH_MISS', ... and make statistics on it. With this information you can optimize the caching behaviour of your cache depending on the objects content type. E.g. squid admins could use this information to configure the refresh_pattern. This option needs more configuration in the configuration-file. .TP .I --peak-report|-p type ($peak_report) Measures the peaks of the Proxy usage in requests per second, per minute and per hour. It reports peaks for TCP, UDP and ALL requests. If set to .I 'old' these values were calculated with the old slow method, if set to .I 'new' the new faster (but still slow) method is used. .TP .I --performance-report|-P n ($performance_report) Shows the throughput of TCP requests for every n minutes. .TP .I --performance-report-adjust|-T n ($performance_report_adjust) Time: Adjust the Performancereport in minutes for non GMT-Timezoners. .TP .I --requester-report|-r n ($requester_report) Switches the UDP and TCP requester reports on. The output is limited by n. (-1 is unlimited) .TP .I --requester-report-no-dns-lookup|-n ($requester_report_no_dns_lookup) Switches the IP lookup for the requesters off. .TP .I --requester-report-use-user-info|-u ($requester_report_use_user_info) Switches the usage of eventually available ident information for requester reports on. .B Warning: This breaks the privacy of your users! (see PRIVACY-Section below) .TP .I --requester-report-with-targets|-R n ($requester_report_with_targets) adds to each line of the requester report the requested URLs. The output is limited by n. (-1 is unlimited, and can result in very very long reports.) .B Warning: Using this option breaks the privacy of your users! (see PRIVACY-Section below) .TP .I --response-time-report ($response_time_report) sums up the time distribution over all objects .TP .I (@response_time_report_interval) This array defines the time steps, which should be reported in the response-time-report. .TP .I --size-distribution-report|-D n ($size_distribution_report) shows size-based distribution of requested objects, smaller numbers result in more verbose reports. (choose 2, 10 or 100 for useful output.) .TP .I --status-report|-s ($status_report) alters the default behaviour of .B Calamaris and makes the status reports more verbose. .TP .I --type-report|-t n ($type_report) switches the content type and the file extension report on. The output is limited by n. (-1 is unlimited) .TP .I --type-report-ignore-case|-c ($type_report_ignore_case) Switch to case-insensitive. This is useful for the 'Requested extensions' report. .SS Input .TP .I --input-format|-f type ($input_format) sets the type of input logfiles. If set to .I 'auto' .B Calamaris tries to guess the input file format. This is the Default. .B Note: If the first line of your input file is corrupted, .B Calamaris will stop with an error. .I 'squid' .B Calamaris expects native logfile derived from .B Squid V1.1.beta26-V2.x or .B OOPS. .I 'squid-extended' .B Calamaris expects native logfile derived from .B Squid V1.1.alpha1-V2.x with .I log_mime_hdrs enabled or .B Squid with .B Smartfilter-Patch or squid-style logfiles out of .B Cisco Content Engines. (This only enables parsing of these kind of logfile, the additional data will be ignored.) (Logging of MIME-Headers breaks the privacy of your users! (see PRIVACY-Section below) .I 'squid-old' .B Calamaris expects native logfile derived from .B Squid V1.1.alpha1-V1.1.beta25. .I 'nc' .B Calamaris expects Logfiles from .B NetCache up to V4.x. (Please see the README on this.) .I 'its' .B Calamaris expects Logfiles from .B Inktomi Traffic Server. .I 'elff' .B Calamaris expects Logfiles in Extended Logfile Format (i.e. from .B Compaq Tasksmart, .B Novell Internet Caching System or .B NetCache V5.x) .I 'nse' .B Calamaris expects Logfiles in Netscape Extended-1 or Netscape Extended-2 Logfile Format (from .B Netscape/iPlanet/SunOne Proxy-Server ) .TP .I --ipfilter-exclude IP/range ($ipfilter_exclude) all IPs are analyzed, except IP/range. Format: 1.1.1.1/32:1.1.2.0/24 1.1.1.1/255.255.255.255:1.1.2.0/255.255.255.0 IP list separated by ':' This switch needs the perl Module NetAddr::IP. .B Warning: This breaks the privacy of your users! (see PRIVACY-Section below) .TP .I --ipfilter-include IP/range ($ipfilter_include) no IPs are analyzed, except IP/range. Format: see --ipfilter-exclude .B Warning: This breaks the privacy of your users! (see PRIVACY-Section below) .TP .I --no-input|-z ($no_input) Switches reading from standard input off. You can use this to merge many cache files to one (see .I --cache-input-file and .I --cache-output-file) or to generate a report out of cache files. .TP .I --time-interval|-I t-t ($time_interval) defines which time-interval should be parsed. t has to be the format yyyymmddhhmmss (localtime) .B Note: omitting the beginning or ending date is allowed. .SS Output Standard output format is plain ascii with 80 chars width. .TP .I ($column1_color) .I ($column2_color) defines the colors for the columns in graphics. (only useful with .I --output-format graph) .TP .I ($formats[n]) Through the config-file you are able to modify the width of the report and alter the culomns that are displayed in the reports. n is the number of the report, as displayed by --help in the --show-reports-option. .TP .I --hostname|-H name ($hostname) The name for the title or subject of the output. If set to .I 'lookup' .B Calamaris looks up the host name of the system its been run on. .TP .I --image-type ($image_type) Sets the image type to gif, png, jpeg, gd or gd2. Only useful when .I --output-format graph is set. The available images types are dependend on your GD::Graph installation. Default is 'png'. .TP .I --logo|-l string ($logo) add a custom string to a HTML-Report. It'll be added to a table on the top of the output. .I -l 'Cord' will add my logo with a link to the Report. .B Note: .I --logo works only in combination with .I --output-format html or html-frame .TP .I --meta|-M string ($meta) Meta: adds a custom string or the content of a file into the of a HTML-Report. Useful if you want to add Stylesheets or something to the Report. .B Note: .I --meta works only in combination with .I --output-format html or html-frame .TP .I --output-format|-F type[,type[,type[,...]]] ($output_format) Format: sets the format of the output-report. If set to .I 'mail' adds a subject header to the beginning of the report. .I 'html' all output is given in html with tables. Can be combined with .I 'mail' to send html mails. .I 'html-frame' all output is given in html frames with tables. .I 'html-embed' all output is given in html with tables without HTML-Headers. Useful for Server-Side-Includes. .I 'graph' enables graphics for html, html-embed or html-frame. .I 'unformatted' gives out the raw numbers seperated by spaces. Useful for re-using the output in other scripts. If you use this along with .I -U, the byte values are calculated in the given Unit, and displayed without indication along with the numbers. the indication moves up to the header of the report. .TP .I --output-path ($output_path) output calamaris statistics to /path. In case of graph output, the graphics destination is /path and the filename is index.html, else it is calamaris.txt. If .I --output-path is not given, all graphics are written to the working directory. .TP .I --output-file ($output_file) alter the filename of --output-path. .TP .I --output-file-prefix ($output_file_prefix) adds a prefix to .I --output-file . .I %t is replaced by the timerange of the report, .I %h by the hostname (see .I --hostname ) .TP .I --show-reports|-S n[,n[,n[,...]]] ($show_reports) Show: Shows only the defined reports in the specified order. Default is to display the reports as they are defined through the report-switches above. The following numbers are defined: .RS 0 Summary .RE .RS 1 Incoming request peak per protocol .RE .RS 2 Incoming transfer volume peak per protocol .RE .RS 3 Incoming requests by method .RE .RS 4 Incoming UDP-requests by status .RE .RS 5 Incoming TCP-requests by status .RE .RS 6 Outgoing requests by status .RE .RS 7 Outgoing requests by destination .RE .RS 8 Request-destinations by 2ndlevel-domain .RE .RS 9 Request-destinations by toplevel-domain .RE .RS 10 TCP-Request-protocol .RE .RS 11 Requested content-type .RE .RS 12 Requested extensions .RE .RS 13 Incoming UDP-requests by host .RE .RS 14 Incoming TCP-requests by host .RE .RS 15 Size Distribution Diagram .RE .RS 16 Performance in n minute steps .RE .RS 17 UDP-Request duration distribution in msec .RE .RS 18 TCP-Request duration distribution in msec .RE .RS 19 UDP Response code distribution .RE .RS 20 TCP Response code distribution .RE .B Note: Using this doesn't make .B Calamaris any faster, the internal calculations will be done as the report-switches were set (see above). .TP .I --sort-order|-O ($sort_order) Changes the sort order in the reports to request size, default is sorting by number of requests. .TP .I ($text_color) defines the colors for text/axis/legend/labels in graphics. (only useful with .I --output-format graph ) .TP .I --unit|-U string ($unit) You can define this to K(ilo), M(ega), G(iga) or T(era) for the Byte-units. .TP .I ($width) defines the width of the graphics. height is calculated from this with a 3:2-ratio. (only useful with .I --output-format graph ) .TP .I ($x_scale) defines how many datasets should be drawn on the graph. 30 is a good value, but you can play with this. if $x_scale gets to big, you're on your own ;-) .TP .I --generate-index ($generate_index) generates an index for all reports that match .I --output-file-prefix. .SS Caching .TP .I --cache-input-file|-i file ($cache_input_file) You can reuse a cache file generated with .I --cache-output-file file to add old data to a new report. Several files can be seperated with a ':'. .B Note: if you use more than one cache file, make sure they are chronologicaly ordered (oldest first). .B Note: if you reuse cache-files, which were not created with .I -d -1 -r -1 -t -1 -R -1 the number of 'others' would be wrong everywhere. In this case the number of 'others' are omitted. .TP .I --cache-output-file|-o file ($cache_output_file) .B Calamaris stores a summary of the computed information in .I file and you can reuse it at a later time with .I --cache-input-file . .B Note: The output file can be the same as the input file: it is simply overwritten after reading the data. It is not recommended to change the options between different runs if you include older data as this can result in strange measurements. .SS Misc .TP .I --benchmark|-b n ($benchmark) benchmark: A switch for the impatient as it prints a '#' for every n parsed lines. .TP .I --copyright|-C Prints the copyright information of .B Calamaris . .TP .I --help|-h Prints a brief description of the command line options. .TP .I --version|-V Prints out the Version-Number. .SS Debug .TP .I --dump-loop|-L prints the internal loop to STDERR. (for Debugging) .TP .I ($test) activates some small tests for the programmer. .TP .I --verbose|-v ($verbose) print more information about what is Calamaris is doing and believing. .SH "EXAMPLES" .TP This example mails the daily statistics to root: cat /var/log/squid/access.log | nice -39 .B calamaris .I --all-useful-reports .I --hostname 'daily worf' .I --output-format mail | mail root .TP This one only caches a summary for later use: cat /var/log/squid/access.log | .B calamaris --all-useful-reports --cache-output-file daily.`date +"%w"` > /dev/null .TP You can then use the caches to have weekly statistics: if [ $DAYOFWEEK = "0" ]; then .B calamaris --all-useful-reports --cache-input-file daily.1:daily.2:daily.3:daily.4:daily.5:daily.6:daily.0 --no-input --output-format mail --hostname "weekly worf" | mail root ; fi .SH "BUGS" If you have a problem with .B Calamaris , please make sure that you use the recent version of .B Calamaris (see VERSION below). Also check that your proxy works correctly and doesn't produce invalid Logfiles. (see the README for buglist and pointers.) If you're sure that you've encountered a bug in .B Calamaris please report it to Calamaris-bug@cord.de. This also applies if .B Calamaris itself says 'please report this'. .SH "PRIVACY" .B Calamaris can be (mis-)used to track what users are requesting. So please read the following and think about it, before using .B Calamaris to be the .I Big Brother. .TP - If you don't trust your users than there is something more wrong than the loss of productivity. .TP - Squid has some nice acl-mechanisms. If you think that your users don't use the net properly, don't let them use it. (You can also open the net at specific times or to specific sites, if you want.) .TP - If you still want to use .B Calamaris that way, let your vict^Wusers know, that they'll be monitored. (in Germany you have to let them know!) .SH "SEE ALSO" .B squid(8) .SH "AUTHOR" Cord Beermann , Michael Pophal . There are also a lot of people who contributed code, gave ideas or requested features. Look them up in the executable. This man page was written by Philipp Frauenfelder , maintainer of the Debian package. Maintenance is now taken over by Cord Beermann. .SH "VERSION" Version of this manpage: $Id: calamaris.1,v 3.1 2006-03-19 17:52:48 cord Exp $ It describes the usage of Calamaris V3.0 and later. Information about new releases, mailing lists, and other related issues can be found from the .B Calamaris home page at .URL http://Calamaris.Cord.de/ .SH "WARRANTY" .B Calamaris comes with "absolutely no warranty". .SH "COPYRIGHT" Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004 Cord Beermann This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. (If you modify and want to publish it under the name .B Calamaris , please ask me. I don't want to confuse the 'audience' with many different versions of the same name and/or Version number. (This is not part of the license, it is only a favour i asked of you.)) This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. calamaris-2.99.4.0/calamaris-cache-convert0000755000175000001440000000774310407330372017172 0ustar cordusers#!/usr/bin/perl -w # # $Id: calamaris-cache-convert,v 0.2 2004/09/17 21:03:50 cord Exp $ # # DESCRIPTION: calamaris-cache-convert - # convert old Calamaris-Caches to new ones. # # Copyright (C) 2004 Cord Beermann # # URL: http://Calamaris.Cord.de/ # Announcement-Mailing-list: send Mail with 'subscribe' in the Mail-Body to # Calamaris-announce-request@Cord.de # # AUTHOR: Cord Beermann # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the Free # Software Foundation; either version 2 of the License, or (at your option) # any later version. # (If you modify and want to publish it under the name 'Calamaris', please ask # me. I don't want to confuse the 'audience' with many different versions of # the same name and/or Version number. (This is not part of the license, it # is only a favour i asked of you.)) # This program is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # You should have received a copy of the GNU General Public License along with # this program; if not, write to the Free Software Foundation, Inc., 59 Temple # Place - Suite 330, Boston, MA 02111-1307, USA. # A Perl script is "correct" if it gets the job done before your boss fires # you. # -- 'Programming Perl Second Edition' use Switch; $count = $error_count = 0; unless ($#ARGV == 1) { print "Usage: $0 old-file new-file\n"; exit 255; } open( OLD, "$ARGV[0]" ) or die ("$0: can't open $ARGV[0] for reading: $!\n"); open( NEW, "> $ARGV[1]" ) or die ("$0: can't open $ARGV[1] for writing: $!\n"); while () { chomp; next if m#^$#; $count++; @cache = split 'µ'; switch ($cache[0]) { case 'A' { $cache[0] = 0; } case 'B' { $cache[0] = 1 if $#cache == 18; $cache[0] = 2 if $#cache == 24; } case 'C' { $cache[0] = 3; } case 'D' { $cache[0] = 4.1; } case 'E' { $cache[0] = 4.2; } case 'F' { $cache[0] = 5.1; } case 'G' { $cache[0] = 5.2; } case 'H' { $cache[0] = 5.3; } case 'I' { $cache[0] = 6.1; } case 'J' { $cache[0] = 6.2; } case 'K' { $cache[0] = 6.3; } case 'L' { $cache[0] = 7.1; } case 'M' { $cache[0] = 7.2; } case 'N' { $cache[0] = 8; push(@cache, 0, 0); $error{8} = 'Missing data in Request-destinations by 2ndlevel-domain - empty data inserted'; } case 'O' { $cache[0] = 9; push(@cache, 0, 0); $error{9} = 'Missing data in Request-destinations by toplevel-domain - empty data inserted'; } case 'P' { $cache[0] = 10; push(@cache, 0, 0); $error{10} = 'Missing data in TCP-Request-protocol - empty data inserted'; } case 'Q' { $cache[0] = 11; push(@cache, 0, 0); $error{11} = 'Missing data in Requested content-type - empty data inserted'; } case 'R' { $cache[0] = 12; push(@cache, 0, 0, 0, 0, 0, 0, 0); $error{12} = 'Missing data in Requested extensions - empty data inserted'; } case 'S' { $cache[0] = 13.1; } case 'V' { $cache[0] = 13.2; } case 'T' { $cache[0] = 14.1; } case 'W' { $cache[0] = 14.2; } case 'X' { $cache[0] = 15; } case 'U' { $cache[0] = 16; @cache = ($cache[0], $cache[1], $cache[2], $cache[3], 0, 0, $cache[5], $cache[6], $cache[7], $cache[8], $cache[9], $cache[10], $cache[11], $cache[12], $cache[13]) if $#cache == 13;; push(@cache, 0, 0); $error{16} = 'Missing data in Performance in n minute steps - empty data inserted'; } else { $error_count++; $error{99} = "$error_count lines couldn't be converted."; print "$cache[0] $cache[1]\n"; } } print NEW join('µ', @cache) . "\n"; } close (NEW); close (OLD); foreach $error (sort keys %error) { print "$error{$error}\n"; } print "$count lines successfully converted.\n"; calamaris-2.99.4.0/calamaris0000755000175000001440000062031610407330372014450 0ustar cordusers#!/usr/bin/perl -w # # $Id: calamaris,v 2.100 2006-03-19 17:28:02 cord Exp $ # # we now ignore the rcs-revision-Number $VERSION = '2.99.4.0'; # # DESCRIPTION: calamaris - statistic package for diverse Proxy-Cache-Servers # # Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 # Cord Beermann # # URL: http://Calamaris.Cord.de/ # Announcement-Mailing-list: send Mail with 'subscribe' in the Mail-Body to # Calamaris-announce-request@Cord.de # # AUTHORS: Cord Beermann # Michael Pophal # # Thanks to these contributors, bug reporters, and feature requesters: # John Heaton # Andreas Lamprecht # Kenny Ng # Claus Langhans # Andreas Jung # Ernst Heiri # Shamil R. Yahin # Thoralf Freitag # Marco Paganini # Michael Riedel # Kris Boulez # Mark Visser # Gary Palmer # Stefan Watermann # Roar Smith # Bernd Lienau # Gary Lindstrom # Jost Krieger # Gerd Michael Hoffmann # Gerold Meerkoetter # Iain Lea # Emmanuel Adeline # John Line # Christos Cheretakis # Ryan Donnelly # Richard Vaughan # Jonas Luster # Clare Lahiff # Toni Andjelkovic # Chris Teakle # Dancer Vesperman # Vincent ? # Elrond ? # Holger Marzen # Panagiotis Christias # Patrik Rak # Steve Snyder # Michael Copeland # Warren Brown # Andy Nik # Frank Roechter # Antonio Casado Rodríguez # Pavol Adamec # Ram Cherukuri # Marco Koch # Stephen Welker # Christian Niederdorfer # Klaus Brinkmeyer # Filip ? # Matt Hubbard # James Crocker # Enrico Ardizzoni # Shawn Switenky # Jarkko Saloranta # Jigar Rasalawala # Philipp Frauenfelder # Alexey Markov # Mark Güthling # Sergey Zarubin # Helge Oldach # Michael R. Schwarzbach # Radu - Eosif Mihailescu # Steffen Sledz # Kenytt Avery # SO Kwok Tsun # Chris Knight # ycdtosa ? # Peter W. Osel # Pawel Worach # Franck Bourdonnec # Chris Clemson # Scott Tregear # Sythos ? # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the Free # Software Foundation; either version 2 of the License, or (at your option) # any later version. # (If you modify and want to publish it under the name 'Calamaris', please ask # me. I don't want to confuse the 'audience' with many different versions of # the same name and/or Version number. (This is not part of the license, it # is only a favour i asked of you.)) # This program is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # You should have received a copy of the GNU General Public License along with # this program; if not, write to the Free Software Foundation, Inc., 59 Temple # Place - Suite 330, Boston, MA 02111-1307, USA. # A Perl script is "correct" if it gets the job done before your boss fires # you. # -- 'Programming Perl Second Edition' # by Larry Wall, Tom Christiansen & Randal L. Schwartz # If you have to remove this, read the README! require 5.002; use Getopt::Long; use Sys::Hostname; use lib '/usr/local'; Getopt::Long::Configure("bundling"); GetOptions( "all-useful-reports|a" => \$opt_a, "benchmark|b=i" => \$opt_b, "cache-input-file|i=s" => \$opt_i, "cache-output-file|o=s" => \$opt_o, "config-file=s" => \$opt_config_file, "copyright|C" => \$opt_C, "domain-report|d=i" => \$opt_d, "domain-report-limit=i" => \$opt_domain_report_limit, "domain-report-n-level|N=i" => \$opt_N, "dump-loop|L" => \$opt_L, "errorcode-distribution-report" => \$opt_errorcode_distribution, "generate-index" => \$opt_generate_index, "help|h" => \$opt_h, "hostname|H=s" => \$opt_H, "input-format|f=s" => \$opt_f, "image-type=s" => \$opt_image_type, "ipfilter-exclude=s" => \$opt_ipfilter_exclude, "ipfilter-include=s" => \$opt_ipfilter_include, "logo|l=s" => \$opt_l, "meta|M=s" => \$opt_M, "no-input|z" => \$opt_z, "output-file=s" => \$opt_output_file, "output-file-prefix=s" => \$opt_output_file_prefix, "output-format|F=s" => \$opt_F, "output-path=s" => \$opt_output_path, "peak-report|p=s" => \$opt_p, "performance-report|P=i" => \$opt_P, "performance-report_adjust|T=i" => \$opt_T, "requester-report|r=i" => \$opt_r, "requester-report-no-dns-lookup|n" => \$opt_n, "requester-report-use-user-info|u" => \$opt_u, "requester-report-with-targets|R=i" => \$opt_R, "response-time-report" => \$opt_response_time, "show-reports|S=s" => \$opt_S, "size-distribution-report|D=i" => \$opt_D, "sort-order|O" => \$opt_O, "status-report|s" => \$opt_s, "time-interval|I=s" => \$opt_I, "type-report|t=i" => \$opt_t, "type-report-ignore-case|c" => \$opt_c, "unit|U=s" => \$opt_U, "verbose|v" => \$opt_v, "version|V" => \$opt_V, ); readconfig(); print "$USAGE\n\n" if $opt_h; print "Calamaris $VERSION\n\n" if $opt_V; print "$COPYRIGHT\n\n$LICENSE\n\n$HOMEPAGE\n" if $opt_C or $opt_h or $opt_V; exit 0 if $opt_h or $opt_C or $opt_V; if ($usage_err) { print STDERR "run '$0 -h' for help.\n\n"; exit 1; } # initialize variables $test_string = "\nTest results:\n\n" if $test; $counter = $hier = $hier_direct = $hier_direct_size = $hier_direct_time = $hier_parent = $hier_parent_size = $hier_parent_time = $hier_sibling = $hier_sibling_size = $hier_sibling_time = $hier_size = $hier_time = $invalid = $ordered_tcp_req_time_max = $ordered_tcp_req_time_max_interval = $ordered_udp_req_time_max = $ordered_udp_req_time_max_interval = $peak_all_hour = $peak_all_hour_time = $peak_all_min = $peak_all_min_time = $peak_all_sec = $peak_all_sec_time = $peak_tcp_hour = $peak_tcp_hour_time = $peak_tcp_min = $peak_tcp_min_time = $peak_tcp_sec = $peak_tcp_sec_time = $peak_udp_hour = $peak_udp_hour_time = $peak_udp_min = $peak_udp_min_time = $peak_udp_sec = $peak_udp_sec_time = $size = $skipped = $tcp = $tcp_hit = $tcp_hit_size = $tcp_hit_time = $tcp_miss = $tcp_miss_none = $tcp_miss_none_size = $tcp_miss_none_time = $tcp_miss_size = $tcp_miss_time = $tcp_size = $tcp_time = $time = $time_end = $time_run = $udp = $udp_hit = $udp_hit_size = $udp_hit_time = $udp_miss = $udp_miss_size = $udp_miss_time = $udp_size = $udp_time = 0; $time_begin = 1e10; @months = qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec); undef(%udp_reqtime_size); undef(%udp_hit_reqtime_size); undef(%udp_hit_reqtime); undef(%tcp_reqtime_size); undef(%tcp_hit_reqtime_size); undef(%tcp_hit_reqtime); undef(%perf_ip); readcache() if ($opt_i); unless ($opt_z) { while ( defined( $line = <> ) ) { if ( not defined $opt_f or $opt_f eq 'auto' ) { if ( $line =~ m#^\d+\.\d+\s+\d+\s+[\w\-\.:]+\s+\w+/\d+\s+\d+\s+\w+\s+\S+\s+\S+\s+\w+/\S+\s+\S+$# ) { $opt_f = 'squid'; print STDERR "guessing... using '-f squid'\n" if ($opt_v); last; } elsif ( $line =~ m#^\d+\s+\d+\s+[\w\-\.:]+\s+\w+/\d+\s+\d+\s+\w+\s+\S+\s+\S+\s+\w+/\S+\s+\S+$# ) { $opt_f = 'its'; print STDERR "guessing... using '-f its'\n" if ($opt_v); last; } elsif ( $line =~ m#^\d+\.\d+\s+\d+\s+[\w\-\.:]+\s+\w+/\d+\s+\d+\s+\w+\s+\S+\s+\S+\s+\w+/\S+$# ) { $opt_f = 'squid-old'; print STDERR "guessing... using '-f squid-old'\n" if ($opt_v); last; } elsif ( $line =~ m#^\d+\.\d+\s+\d+\s+[\w\-\.:]+\s+\w+/\d+\s+\d+\s+\w+\s+\S+\s+\S+\s+\w+/\S+\s+(\S|; c)+\s+\S+$# ) { $opt_f = 'nc'; print STDERR "guessing... using '-f nc'\n" if ($opt_v); last; } elsif ( $line =~ s/^\s*#\s*Fields:\s*// ) { $opt_f = 'elff'; print STDERR "guessing... using '-f elff'\n" if ($opt_v); last; } elsif ( $line =~ s/^\s*format=\s*// ) { $opt_f = 'nse'; print STDERR "guessing... using '-f nse'\n" if ($opt_v); last; } elsif ( $line =~ m#^\d+\.\d+\s+\d+\s+[\w\-\.:]+\s+\w+/\-?\d+\s+\d+\s+\w+\s+\S+\s+\S+\s+\w+/\S+\s+\S+\s+# ) { $opt_f = 'squid-extended'; print STDERR "guessing... using '-f squid-extended'\n" if ($opt_v); last; } elsif ( $line =~ m#^[\w\-\.:]+\s+\S+\s+\S+\s+\[.+\]\s+\"\w+\s+\S+\s+\S+\"\s+\d+\s+\d+\s+\S+(\s+\[.*\]\s+\[.*\])?$# ) { print STDERR "$0: The first line of the input looks to me as if you switched 'emulate_httpd_log' to on. I can't parse that format. Please read the README on this.\n\n"; exit(1); } elsif ( $line =~ m/^\s*(#|$)/ ) { print STDERR "skipping: $line\n" if ($opt_v); next; } else { print STDERR "$0: I don't know this input format. Please check the input. If you\'re sure that the following line is NOT corrupt and the error also occurs with the recent version of Calamaris (see the README for pointers and known bugs) then report it with the following line to . Thank You.\n\n$line\n\n"; exit(1); } } else { last; } } if ( not defined $opt_f or $opt_f eq 'auto' ) { print "\nno requests found\n"; exit(0); } # IP Filter my %filter_ip; if ($opt_ipfilter_exclude or $opt_ipfilter_include) { die( "$0: Couldn't load package NetAddr::IP, maybe it is not installed: $!\n" ) unless (eval "require NetAddr::IP"); my $to_split = ($opt_ipfilter_exclude) ? $opt_ipfilter_exclude : $opt_ipfilter_include; foreach ( split /:/, $to_split ) { $ip = new NetAddr::IP( (split /\//, $_) ); foreach ( $ip->split('32') ) { $filter_ip{$_->addr()} = 1; } } } print STDERR "print a hash-sign for each $opt_b lines:\n" if ($opt_b); $loop = ' for ( ; $line; $line = <> ) {'; if ( $opt_f eq 'squid' ) { $loop .= ' ( $log_date, $log_reqtime, $log_requester, $log_status, $log_size, $log_method, $log_url, $log_ident, $log_hier, $log_content, $foo ) = split ( /\s+/, $line ); if ( not defined $foo or not defined $log_content or $foo ne \'\' or $log_content eq \'\' or $log_reqtime < 0 or $log_date !~ m#^\d+\.\d{3}$# ) {'; } elsif ( $opt_f eq 'squid-extended' ) { $loop .= ' $line =~ s/ \[[^\[\]]*\]//g; ( $log_date, $log_reqtime, $log_requester, $log_status, $log_size, $log_method, $log_url, $log_ident, $log_hier, $log_content) = split ( /\s+/, $line, 10 ); chomp($log_content); if ( not defined $log_content or $log_content eq \'\' or $log_reqtime < 0 or $log_date !~ m#^\d+\.\d{3}$# ) {'; } elsif ( $opt_f eq 'its' ) { $loop .= ' ( $log_date, $log_reqtime, $log_requester, $log_status, $log_size, $log_method, $log_url, $log_ident, $log_hier, $log_content, $foo ) = split ( /\s+/, $line ); if ( not defined $foo or not defined $log_content or $foo ne \'\' or $log_content eq \'\' or $log_reqtime < 0 or $log_date !~ m#^\d+$# ) {'; } elsif ( $opt_f eq 'squid-old' ) { $loop .= ' ( $log_date, $log_reqtime, $log_requester, $log_status, $log_size, $log_method, $log_url, $log_ident, $log_hier, $foo ) = split ( /\s+/, $line ); unless ( not defined $foo or not defined $log_hier or $foo ne \'\' or $log_hier eq \'\' or $log_reqtime < 0 or $log_date !~ m#^\d+\.\d{3}$# ) {'; } elsif ( $opt_f eq 'nc' ) { $loop .= ' $line =~ s#\; c#\;_c#og; # Hack to handle buggy logfiles of NetCache V3.2.x ( $log_date, $log_reqtime, $log_requester, $log_status, $log_size, $log_method, $log_url, $log_ident, $log_hier, $log_content, $log_abort, $foo ) = split ( /\s+/, $line ); if ( not defined $foo or not defined $log_abort or $foo ne \'\' or $log_abort eq \'\' or $log_reqtime < 0 or $log_date !~ m#^\d+\.\d{3}$# ) {'; } elsif ( $opt_f eq 'elff' ) { @fields = split ( /\s+/, $line ); $loop .= ' use Time::Local; if ( $line =~ m#^'; foreach (@fields) { $tmpline1 .= '\s+' if ($tmpline1); if ( $_ eq 'date' ) { $tmpline1 .= '(\d+)-(\d+)-(\d+)'; $tmpline2 .= ' $log_year = $' . ++$offset . '; $log_month = $' . ++$offset . '; $log_day = $' . ++$offset . ';'; } elsif ( $_ eq 'time' ) { $tmpline1 .= '(\d+):(\d+):(\d+)'; $tmpline2 .= ' $log_hour = $' . ++$offset . '; $log_min = $' . ++$offset . '; $log_sec = $' . ++$offset . ';'; } elsif ( $_ eq 'x-timestamp' ) { $tmpline1 .= '(\d+\.\d+)'; $tmpline2 .= ' $log_date = $' . ++$offset . ';'; $log_date = 0; } elsif ( $_ eq 'c-ip' ) { $tmpline1 .= '([\w\-\.:]+)'; $tmpline2 .= ' $log_requester = $' . ++$offset . ';'; } elsif ( $_ eq 'cs-authname' or $_ eq 'x-remote-id' or $_ eq 'x-username' or $_ eq 'cs-username' ) { $tmpline1 .= '(\S+)'; $tmpline2 .= ' $log_ident = $' . ++$offset . ';'; } elsif ( $_ eq 's-ip' or $_ eq 's-sitename' or $_ eq 'sc-filter-result' or $_ eq 'sc-filter-category' or $_ eq 'x-virus-id' ) { $tmpline1 .= '[\w\-\.]+'; } elsif ( $_ eq 'cs-method' ) { $tmpline1 .= '([\w\-]+)'; $tmpline2 .= ' $log_method = $' . ++$offset . ';'; } elsif ( $_ eq 'cs-uri' ) { $tmpline1 .= '(\S+)'; $tmpline2 .= ' $log_url = $' . ++$offset . ';'; } elsif ( $_ eq 'cs-uri-scheme' ) { $tmpline1 .= '(\S+)'; $tmpline2 .= ' $log_cs_uri_scheme = $' . ++$offset . ';'; } elsif ( $_ eq 'cs-host' ) { $tmpline1 .= '(\S+)'; $tmpline2 .= ' $log_cs_host = $' . ++$offset . ';'; } elsif ( $_ eq 'cs-uri-path' ) { $tmpline1 .= '(\S+)'; $tmpline2 .= ' $log_cs_uri_path = $' . ++$offset . ';'; } elsif ( $_ eq 'cs-uri-stem' or $_ eq 'cs-uri-query' or $_ eq 'x-note' ) { $tmpline1 .= '\S+'; } elsif ( $_ eq 'c-version' ) { $tmpline1 .= '\w+/[\d\.]+'; } elsif ( $_ eq 'sc-status' ) { $tmpline1 .= '(\d+)'; $tmpline2 .= ' $log_code = $' . ++$offset . ';'; } elsif ( $_ eq 'sc-bytes' or $_ eq 'bytes' ) { $tmpline1 .= '(\d+)'; $tmpline2 .= ' $log_size = $' . ++$offset . ';'; } elsif ( $_ eq 'cs-bytes' ) { $tmpline1 .= '\d+'; } elsif ( $_ eq 'x-elapsed-milliseconds' ) { $tmpline1 .= '(\d+)'; $tmpline2 .= ' $log_reqtime = $' . ++$offset . ';'; } elsif ( $_ eq 'time-taken' ) { $tmpline1 .= '([\d\.]+)'; $tmpline2 .= ' $log_reqtime = $' . ++$offset . ' * 1000;'; } elsif ( $_ eq 'cs(User-Agent)' or $_ eq 'cs(Cookie)' or $_ eq 'cs(Referer)' or $_ eq 'sc(Referer)' or $_ eq 'cs(X-Forwarded-For)' or $_ eq 'x-hiername') { $tmpline1 .= '(\"[^\"]*\"|-)'; $tmpline2 .= ' ++$offset;'; } elsif ( $_ eq 'cs(Content-Type)' ) { $tmpline1 .= '(\"[^\"]*\"|-)'; $tmpline2 .= ' $log_content = $' . ++$offset . ';'; } elsif ( $_ eq 'cached' ) { $tmpline1 .= '(\d+)'; $tmpline2 .= ' $log_cached = $' . ++$offset . ';'; } elsif ( $_ eq 'x-transaction' ) { $tmpline1 .= '(\w+\/[\d\-]+)'; $tmpline2 .= ' $log_status = $' . ++$offset . ';'; $log_status = 0; } elsif ( $_ eq 'x-fill-proxy-ip' ) { $tmpline1 .= '([\w\-\.]+)'; $tmpline2 .= ' $log_proxy_ip = $' . ++$offset . ';'; } elsif ( $_ eq 'x-origin-ip' ) { $tmpline1 .= '([\w\-\.]+)'; $tmpline2 .= ' $log_origin_ip = $' . ++$offset . ';'; } elsif ( $_ eq 'x-hiercode' ) { $tmpline1 .= '([\w\-\.\/]+)'; $tmpline2 .= ' $log_hier = $' . ++$offset . ';'; $log_hier = 0; } elsif ( $_ eq 's-hierarchy' ) { $tmpline1 .= '([\w\-\.\/]+)'; $tmpline2 .= ' $log_hierarchy = $' . ++$offset . ';'; } elsif ( $_ eq 's-action' ) { $tmpline1 .= '([\w\-\.\/]+)'; $tmpline2 .= ' $log_s_action = $' . ++$offset . ';'; } elsif ( $_ eq 's-supplier-name' ) { $tmpline1 .= '([\w\-\.\/]+)'; $tmpline2 .= ' $log_supplier_name = $' . ++$offset . ';'; } elsif ( $_ eq 'rs(Content-Type)' ) { $tmpline1 .= '\"[^\"]*\"'; } else { print STDERR "$0: I don't know this input format. Please check the input. If you\'re sure that the following is NOT corrupt and the error also occurs with the recent version of Calamaris (see the README for pointers and known bugs) then report it with the following line to . Thank You.\n\n$_\n\n"; exit(1); } } foreach $pattern ( qw(date time c-ip cs-method cs-uri sc-status sc-bytes time-taken cached x-fill-proxy-ip x-origin-ip) ) { unless ( grep $pattern, @fields ) { print STDERR "$0: Your input file format is missing at least the field \'$pattern\'. I can\'t parse it. Sorry. If you think that this field isn't important to you, please report this error to . Thank You.\n\n"; exit(1); } } foreach $pattern (qw(cs-authname)) { unless ( grep $pattern, @fields ) { if ( $pattern eq 'cs-authname' ) { $tmpline2 .= ' $log_ident = "-";'; } } } $loop .= $tmpline1 . '.?$# ) {' . $tmpline2; $loop .= ' $log_date = timegm( $log_sec, $log_min, $log_hour, $log_day, $log_month - 1, $log_year - 1900 );' unless defined $log_date; $loop .= ' if ($log_s_action) { $log_status = "$log_s_action/$log_code"; } else { $log_status = "$log_cached/$log_code"; }' unless defined $log_status; $loop .= ' if ( $log_hierarchy and $log_supplier_name ) { $log_hier = "$log_hierarchy/$log_supplier_name"; } elsif ( $log_origin_ip ne \'-\' ) { $log_hier = "DIRECT/$log_origin_ip"; } elsif ( $log_proxy_ip ne \'-\' ) { if ( $log_cached eq \'0\' ) { $log_hier = "PARENT_MISS/$log_proxy_ip"; } else { $log_hier = "PARENT_HIT/$log_proxy_ip"; } } else { $log_hier = "NONE/-"; }' unless defined $log_hier; $loop .= ' } else {'; while ( defined( $line = <> ) ) { last unless $line =~ /^\s*#/; } } elsif ( $opt_f eq 'nse' ) { $line =~ s#^format=##og; @fields = split ( /\s+/, $line ); $loop .= ' use Time::Local; if ( $line =~ m#^'; foreach (@fields) { $tmpline1 .= '\s+' if ($tmpline1); if ( $_ eq '[%SYSDATE%]' ) { $tmpline1 .= '\[(\d+)/(\w+)/(\d+):(\d+):(\d+):(\d+)\s+\S+\]'; $tmpline2 .= ' $log_day = $' . ++$offset . '; $log_month = $' . ++$offset . '; $log_year = $' . ++$offset . '; $log_hour = $' . ++$offset . '; $log_min = $' . ++$offset . '; $log_sec = $' . ++$offset . ';'; } elsif ( $_ eq '%Ses->client.ip%' ) { $tmpline1 .= '([\w\-\.:]+)'; $tmpline2 .= ' $log_requester = $' . ++$offset . ';'; } elsif ( $_ eq '%Req->vars.pauth-user%' ) { $tmpline1 .= '(\S+)'; $tmpline2 .= ' $log_ident = $' . ++$offset . ';'; } elsif ( $_ eq '%Req->vars.remote-status%' or $_ eq '%Req->vars.r2p-cl%' or $_ eq '%Req->vars.cli-status%' or $_ eq '%Req->vars.svr-status%' or $_ eq '%Req->vars.cch-status%' ) { $tmpline1 .= '[\w\-\.]+'; } elsif ( $_ eq '"%Req->reqpb.proxy-request%"' ) { $tmpline1 .= '\"(\w+)\s+([^\"]+)\s*\S*\"'; $tmpline2 .= ' $log_method = $' . ++$offset . '; $log_url = $' . ++$offset . ';'; } elsif ( $_ eq '%Req->srvhdrs.clf-status%' ) { $tmpline1 .= '([\d\-]+)'; $tmpline2 .= ' $log_code = $' . ++$offset . ';'; } elsif ( $_ eq '%Req->vars.p2c-cl%' ) { $tmpline1 .= '([\d\-]+)'; $tmpline2 .= ' $log_size = $' . ++$offset . ';'; } elsif ( $_ eq '%Req->headers.content-length%' or $_ eq '%Req->vars.p2r-cl%' or $_ eq '%Req->vars.c2p-hl%' or $_ eq '%Req->vars.p2c-hl%' or $_ eq '%Req->vars.p2r-hl%' or $_ eq '%Req->vars.r2p-hl%' ) { $tmpline1 .= '[\d\-]+'; } elsif ( $_ eq '%Req->vars.xfer-time%' ) { $tmpline1 .= '([\d\.]+)'; $tmpline2 .= ' $log_reqtime = $' . ++$offset . ' * 1000;'; } elsif ( $_ eq '%route%' ) { $tmpline1 .= '(\d+)'; $tmpline2 .= ' $log_cached = $' . ++$offset . ';'; } elsif ( $_ eq '%Req->vars.actual-route%' ) { $tmpline1 .= '(\S+)'; $tmpline2 .= ' $log_hier = $' . ++$offset . '; if ( $log_hier =~ m#[\(\)]# ) { $log_hier =~ s#^(\w+)(\((\S+)\))?$#$1/$3#; $log_status = \'TCP_MISS/-\'; } elsif ( $log_hier =~ m#-# ) { $log_hier = $log_hier . \'/-\'; $log_status = \'TCP_HIT/-\'; } else { $log_hier = $log_hier . \'/-\'; $log_status = \'TCP_MISS/-\'; }'; } elsif ( $_ eq '-' ) { $tmpline1 .= '-'; } else { print STDERR "$0: I don't know this input format. Please check the input. If you\'re sure that the following is NOT corrupt and the error also occurs with the recent version of Calamaris (see the README for pointers and known bugs) then report it with the following line to . Thank You.\n\n$_\n\n"; exit(1); } } foreach $pattern ( qw(%Ses->client.ip% [%SYSDATE%] %Req->reqpb.proxy-request% %Req->srvhdrs.clf-status% %Req->vars.p2c-cl% %Req->vars.xfer-time%) ) { unless ( grep $pattern, @fields ) { print STDERR "$0: Your input file format is missing at least the field \'$pattern\'. I can\'t parse it. Sorry. If you think that this field isn't important to you, please report this error to . Thank You.\n\n"; exit(1); } } foreach $pattern (qw(%Req->vars.pauth-user% %Req->vars.actual-route%)) { unless ( grep /$pattern/, @fields ) { if ( $pattern eq '%Req->vars.pauth-user%' ) { $tmpline2 .= ' $log_ident = "-";'; } elsif ( $pattern eq '%Req->vars.actual-route%' ) { $tmpline2 .= ' $log_hier = \'-/-\'; $log_status = \'-/-\';'; } } } $loop .= $tmpline1 . '.?$# ) {' . $tmpline2 . ' $monthcount = -1; foreach $month (@months) { $monthcount++; last if ( $month eq $log_month ); } $log_date = timegm( $log_sec, $log_min, $log_hour, $log_day, $monthcount, $log_year - 1900 ); } else {'; while ( defined( $line = <> ) ) { last unless $line =~ /^\s*#/; } } else { print STDERR "$0: unknown value at -f -option: \"$opt_f\"\n\n$USAGE\n\n"; exit 1; } $loop .= ' chomp($line);'; $loop .= ' warn(\'invalid line: "\' . $line . "\"\n");' if $opt_v; $loop .= ' $invalid++; next; }'; $loop .= ' if (($log_date < $interval_begin) or ($log_date > $interval_end)) { $skipped++; next; } ' if $opt_I; $loop .= ' if($filter_ip{$log_requester}) { $skipped++; next; } ' if $opt_ipfilter_exclude; $loop .= ' if(! $filter_ip{$log_requester}) { $skipped++; next; } ' if $opt_ipfilter_include; $loop .= ' $log_reqtime = .1 if $log_reqtime == 0; ( $log_hitfail, $log_code ) = split \'/\', $log_status; $log_size = 0 if ( $log_size eq \'-\' ); $log_url = "$log_cs_uri_scheme://$log_cs_host/$log_cs_uri_path" unless ($log_url); $log_url =~ s/\?.*$/?/; @url = split m#[/\\\]#o, $log_url; ( $urlprot, $urlhost, $urlext ) = (@url)[ 0, 2, $#url ]; $urlext = \'.\' if $#url <= 2; if ( $#url <= -1 ) { $urlext = \'.\'; $urlprot = $urlhost = \'\'; } $urlext = \'.\' if ( $urlext =~ m#[\?;&\$,!@=|%]#o or $log_method eq \'POST\' ); unless ( defined $urlhost ) { $urlhost = $urlprot;'; $loop .= ' $log_content = ' unless $opt_f eq 'squid-old' or $opt_f eq 'elff' or $opt_f eq 'nse'; $loop .= ' $urlprot = \'\'; $urlext = \'.\'; } $urlhost =~ s#^.*@##o; $urlhost =~ s#[:\?].*$##o; $urlhost =~ tr/A-Z/a-z/; @urlext = split \'\.\', $urlext; $urlext = (@urlext)[$#urlext]; $urlext = \'\' if $#urlext <= 0;'; $loop .= ' $urlext =~ tr/A-Z/a-z/;' if $opt_c; $loop .= ' if ( $urlhost =~ m#^(([0-9][0-9]{0,2}\.){3})[0-9][0-9]{0,2}$#o ) {'; $loop .= ' $urlhost = $1 . \'*\';' unless $opt_N == -1; $loop .= ' $urltld = \'\'; } elsif ( $urlhost =~ m#^(.*\.([^\.]+\.)?)?([^\.]+\.([^\.]+))\.?$#o ) { @list = split \'\.\', $urlhost; $urltld = pop @list;'; if ( $opt_N != -1 ) { $loop .= ' $urlhost = $urltld;'; for ( $i = $opt_N; $i != 1; $i-- ) { $loop .= ' $urlhost = pop (@list) . \'.\' . $urlhost if $#list >= 0;'; } $loop .= ' $urlhost = pop (@list) . \'.\' . $urlhost if ( $urltld =~ m#^(a[rtu]|br|c[no]|hk|i[dlm]|jp|kr|l[by]|m[oxy]|nz|p[elnry]|sg|t[hrw]|u[aks]|ve|yu|za)$#o and $#list >= 0 ); $urlhost = \'*.\' . $urlhost if $#list >= 0; $urltld = \'*.\' . $urltld;'; } $loop .= ' } elsif ( $urlhost =~ m#([!a-z0-9\.\-]|\.\.)#o ) { $urlhost = $urltld = $urlext = $urlprot = \'\'; } else { $urltld = $urlhost; }'; if ($opt_n) { $loop .= ' $requester = $log_requester;'; } else { $loop .= ' $requester = getfqdn($log_requester);'; } $loop .= ' $requester = $log_ident . \'@\' . $requester if $log_ident ne \'-\';' if $opt_u; $loop .= ' ( $log_hier_method, $log_hier_host ) = ( split \'/\', $log_hier )[ 0, 1 ];'; $loop .= ' $log_content = \'\' if $log_content eq \'-\'; $log_content =~ tr/A-Z/a-z/; $log_content =~ s#\s+.*##; $log_content = ' unless $opt_f eq 'squid-old' or $opt_f eq 'elff' or $opt_f eq 'nse'; $loop .= ' $urlhost = $urltld = $urlext = $urlprot = \'\' if ( $log_code =~ m#^[45]#o );'; $loop .= " print STDERR '#' if (0 == (\$counter % $opt_b));" if $opt_b; $loop .= ' $counter++; $size += $log_size; $time += $log_reqtime; $method{$log_method} = $method_size{$log_method} = $method_time{$log_method} = 0 unless defined $method{$log_method}; $method{$log_method}++; $method_size{$log_method} += $log_size; $method_time{$log_method} += $log_reqtime; $time_begin = $log_date if $log_date < $time_begin; $time_end = $log_date if $log_date > $time_end;'; if ( defined $opt_p ) { $loop .= ' if ( defined(@peak_all) ) { if ( $log_date < $peak_all[$#peak_all] ) { $peak_warn = \'Peak values are most likely wrong due to unsorted input!\'; undef(@peak_all); undef(@peak_udp); undef(@peak_tcp); $peak_all_min_pointer = $peak_all_sec_pointer = $peak_tcp_min_pointer = $peak_tcp_sec_pointer = $peak_udp_min_pointer = $peak_udp_sec_pointer = 0; chomp($line); warn( \'unsorted input: "\' . $line . "\"\n" ) if $opt_v; } }'; if ( $opt_p eq 'old' ) { $loop .= ' $peak_all_sec_pointer++; $peak_all_min_pointer++; unshift ( @peak_all, $log_date ); $peak_all_sec_pointer-- while $peak_all[ $peak_all_sec_pointer - 1 ] < ( $log_date - 1 ); $peak_all_min_pointer-- while $peak_all[ $peak_all_min_pointer - 1 ] < ( $log_date - 60 ); pop (@peak_all) while $peak_all[$#peak_all] < ( $log_date - 3600 ); if ( $peak_all_hour < @peak_all ) { $peak_all_hour = @peak_all; $peak_all_hour_time = $log_date - 3600; } if ( $peak_all_min < $peak_all_min_pointer ) { $peak_all_min = $peak_all_min_pointer; $peak_all_min_time = $log_date - 60; } if ( $peak_all_sec < $peak_all_sec_pointer ) { $peak_all_sec = $peak_all_sec_pointer; $peak_all_sec_time = $log_date - 1; }'; } elsif ( $opt_p eq 'new' ) { $loop .= ' $date_hour = int( ( $log_date - ( $log_reqtime / 1000 ) ) / 3600 ) * 3600; foreach $i ( ( $date_hour / 3600 ) .. int( $log_date / 3600 ) ) { $peak_all_hour{ $i * 3600 } = $peak_all_hour_size{ $i * 3600 } = 0 unless defined $peak_all_hour{ $i * 3600 }; $peak_all_hour{ $i * 3600 }++; $peak_all_hour_size{ $i * 3600 } += $log_size / int( $log_reqtime / 3600000 + 1 ); } $peak_all_sec_pointer++; unshift ( @peak_all, $log_date ); $peak_all_sec_pointer-- while $peak_all[ $peak_all_sec_pointer - 1 ] < ( $log_date - 1 ); pop (@peak_all) while $peak_all[$#peak_all] < ( $log_date - 60 ); if ( $peak_all_min < @peak_all ) { $peak_all_min = @peak_all; $peak_all_min_time = $log_date - 60; } if ( $peak_all_sec < $peak_all_sec_pointer ) { $peak_all_sec = $peak_all_sec_pointer; $peak_all_sec_time = $log_date - 1; }'; } else { print STDERR "$0: unknown value at -p -option: \"$opt_p\"\n\n$USAGE\n\n"; exit 1; } } $loop .= ' if ( ( $log_method =~ m#^ICP_#o ) or ( $log_status =~ m#^ICP#o ) ) { $udp++; $udp_size += $log_size; $udp_time += $log_reqtime;'; $loop .= ' $udp_reqtime{$log_reqtime} = $udp_hit_reqtime{$log_reqtime} = $udp_reqtime_size{$log_reqtime} = $udp_hit_reqtime_size{$log_reqtime} = 0 unless defined $udp_reqtime{$log_reqtime}; $udp_reqtime{$log_reqtime}++; $udp_reqtime_size{$log_reqtime} += $log_size;' if $opt_response_time; $loop .= ' $udp_code{$log_code} = $udp_code_time{$log_code} = $udp_code_size{$log_code} = $udp_hit_code{$log_code} = $udp_hit_code_size{$log_code} = 0 unless defined $udp_code{$log_code}; $udp_code{$log_code}++; $udp_code_time{$log_code} += $log_reqtime; $udp_code_size{$log_code} += $log_size;' if $opt_errorcode_distribution; $loop .= ' $udp_requester{$requester} = $udp_requester_size{$requester} = $udp_requester_time{$requester} = $udp_hit_requester{$requester} = $udp_hit_requester_size{$requester} = 0 unless defined $udp_requester{$requester}; $udp_requester{$requester}++; $udp_requester_size{$requester} += $log_size; $udp_requester_time{$requester} += $log_reqtime;' if ($opt_r); $loop .= ' $udp_requester_urlhost{$requester}{$urlhost} = $udp_requester_urlhost_size{$requester}{$urlhost} = $udp_requester_urlhost_time{$requester}{$urlhost} = $udp_hit_requester_urlhost{$requester}{$urlhost} = $udp_hit_requester_urlhost_size{$requester}{$urlhost} = 0 unless defined $udp_requester_urlhost{$requester}{$urlhost}; $udp_requester_urlhost{$requester}{$urlhost}++; $udp_requester_urlhost_size{$requester}{$urlhost} += $log_size; $udp_requester_urlhost_time{$requester}{$urlhost} += $log_reqtime;' if ($opt_R); if ( not defined $opt_p ) { } elsif ( $opt_p eq 'old' ) { $loop .= ' $peak_udp_sec_pointer++; $peak_udp_min_pointer++; unshift ( @peak_udp, $log_date ); $peak_udp_sec_pointer-- while $peak_udp[ $peak_udp_sec_pointer - 1 ] < ( $log_date - 1 ); $peak_udp_min_pointer-- while $peak_udp[ $peak_udp_min_pointer - 1 ] < ( $log_date - 60 ); pop @peak_udp while $peak_udp[$#peak_udp] < ( $log_date - 3600 ); if ( $peak_udp_hour < @peak_udp ) { $peak_udp_hour = @peak_udp; $peak_udp_hour_time = $log_date - 3600; } if ( $peak_udp_min < $peak_udp_min_pointer ) { $peak_udp_min = $peak_udp_min_pointer; $peak_udp_min_time = $log_date - 60; } if ( $peak_udp_sec < $peak_udp_sec_pointer ) { $peak_udp_sec = $peak_udp_sec_pointer; $peak_udp_sec_time = $log_date - 1; }'; } elsif ( $opt_p eq 'new' ) { $loop .= ' foreach $i ( ( $date_hour / 3600 ) .. int( $log_date / 3600 ) ) { $peak_udp_hour{ $i * 3600 } = $peak_udp_hour_size{ $i * 3600 } = 0 unless defined $peak_udp_hour{ $i * 3600 }; $peak_udp_hour{ $i * 3600 }++; $peak_udp_hour_size{ $i * 3600 } += $log_size / int( $log_reqtime / 3600000 + 1 ); } $peak_udp_sec_pointer++; unshift ( @peak_udp, $log_date ); $peak_udp_sec_pointer-- while $peak_udp[ $peak_udp_sec_pointer - 1 ] < ( $log_date - 1 ); pop @peak_udp while $peak_udp[$#peak_udp] < ( $log_date - 60 ); if ( $peak_udp_min < @peak_udp ) { $peak_udp_min = @peak_udp; $peak_udp_min_time = $log_date - 60; } if ( $peak_udp_sec < $peak_udp_sec_pointer ) { $peak_udp_sec = $peak_udp_sec_pointer; $peak_udp_sec_time = $log_date - 1; }'; } $loop .= ' if ( $log_hitfail =~ m#^(UDP|ICP)_HIT#o ) { $udp_hit++; $udp_hit_size += $log_size; $udp_hit_time += $log_reqtime;'; $loop .= ' $udp_hit_reqtime{$log_reqtime}++; $udp_hit_reqtime_size{$log_reqtime} += $log_size;' if $opt_response_time; $loop .= ' $udp_hit_code{$log_code}++; $udp_hit_code_size{$log_code} += $log_size;' if $opt_errorcode_distribution; $loop .= ' $udp_hit_requester{$requester}++; $udp_hit_requester_size{$requester} += $log_size;' if $opt_r; $loop .= ' $udp_hit_requester_urlhost{$requester}{$urlhost}++; $udp_hit_requester_urlhost_size{$requester}{$urlhost} += $log_size;' if ($opt_R); $loop .= ' $udp_hit{$log_hitfail} = $udp_hit_size{$log_hitfail} = $udp_hit_time{$log_hitfail} = 0 unless defined $udp_hit{$log_hitfail}; $udp_hit{$log_hitfail}++; $udp_hit_size{$log_hitfail} += $log_size; $udp_hit_time{$log_hitfail} += $log_reqtime;' if ($opt_s); $loop .= ' } else { $udp_miss++; $udp_miss_size += $log_size; $udp_miss_time += $log_reqtime;'; $loop .= ' $udp_miss{$log_hitfail} = $udp_miss_size{$log_hitfail} = $udp_miss_time{$log_hitfail} = 0 unless defined $udp_miss{$log_hitfail}; $udp_miss{$log_hitfail}++; $udp_miss_size{$log_hitfail} += $log_size; $udp_miss_time{$log_hitfail} += $log_reqtime;' if ($opt_s); $loop .= ' } } else { $tcp++; $tcp_size += $log_size; $tcp_time += $log_reqtime;'; $loop .= ' $tcp_reqtime{$log_reqtime} = $tcp_hit_reqtime{$log_reqtime} = $tcp_reqtime_size{$log_reqtime} = $tcp_hit_reqtime_size{$log_reqtime} = 0 unless defined $tcp_reqtime{$log_reqtime}; $tcp_reqtime{$log_reqtime}++; $tcp_reqtime_size{$log_reqtime} += $log_size;' if $opt_response_time; $loop .= ' $tcp_code{$log_code} = $tcp_code_time{$log_code} = $tcp_code_size{$log_code} = $tcp_hit_code_size{$log_code} = $tcp_hit_code{$log_code} = 0 unless defined $tcp_code{$log_code}; $tcp_code{$log_code}++; $tcp_code_time{$log_code} += $log_reqtime; $tcp_code_size{$log_code} += $log_size;' if $opt_errorcode_distribution; $loop .= ' $perf_date = ( int( ( $log_date + ' . "( $opt_T * 60 ) ) / ( 60 * $opt_P ) ) * 60 * $opt_P ) - ( $opt_T * 60 );" . ' unless ( defined $perf_counter{$perf_date} ) { $perf_counter{$perf_date} = $perf_size{$perf_date} = $perf_tcp_hit_size{$perf_date} = $perf_tcp_miss_size{$perf_date} = $perf_hier_direct_size{$perf_date} = $perf_tcp_hit{$perf_date} = $perf_hier_sibling_size{$perf_date} = $perf_hier_parent_size{$perf_date} = 0; $perf_time{$perf_date} = $perf_tcp_hit_time{$perf_date} = $perf_tcp_miss_time{$perf_date} = $perf_hier_direct_time{$perf_date} = $perf_hier_sibling_time{$perf_date} = $perf_hier_parent_time{$perf_date} = .0000000001; } $perf_counter{$perf_date}++; $perf_ip{$perf_date}{$log_requester} = 1; $perf_size{$perf_date} += $log_size; $perf_time{$perf_date} += $log_reqtime;' if ($opt_P); $loop .= ' $tcp_requester{$requester} = $tcp_requester_size{$requester} = $tcp_requester_time{$requester} = $tcp_hit_requester{$requester} = $tcp_hit_requester_size{$requester} = 0 unless defined $tcp_requester{$requester}; $tcp_requester{$requester}++; $tcp_requester_size{$requester} += $log_size; $tcp_requester_time{$requester} += $log_reqtime;' if ($opt_r); $loop .= ' $distribution = $log_size ? int( log($log_size) / log($opt_D) ) : -1; $tcp_distribution{$distribution} = $tcp_distribution_size{$distribution} = $tcp_distribution_time{$distribution} = $tcp_hit_distribution{$distribution} = $tcp_hit_distribution_size{$distribution} = 0 unless defined $tcp_distribution{$distribution}; $tcp_distribution{$distribution}++; $tcp_distribution_size{$distribution} += $log_size; $tcp_distribution_time{$distribution} += $log_reqtime;' if ($opt_D); $loop .= ' $tcp_requester_urlhost{$requester}{$urlhost} = $tcp_requester_urlhost_size{$requester}{$urlhost} = $tcp_requester_urlhost_time{$requester}{$urlhost} = $tcp_hit_requester_urlhost{$requester}{$urlhost} = $tcp_hit_requester_urlhost_size{$requester}{$urlhost} = 0 unless defined $tcp_requester_urlhost{$requester}{$urlhost}; $tcp_requester_urlhost{$requester}{$urlhost}++; $tcp_requester_urlhost_size{$requester}{$urlhost} += $log_size; $tcp_requester_urlhost_time{$requester}{$urlhost} += $log_reqtime;' if ($opt_R); $loop .= ' $tcp_urlhost{$urlhost} = $tcp_urlhost_size{$urlhost} = $tcp_urlhost_time{$urlhost} = $tcp_hit_urlhost{$urlhost} = $tcp_hit_urlhost_size{$urlhost} = 0 unless defined $tcp_urlhost{$urlhost}; $tcp_urlhost{$urlhost}++; $tcp_urlhost_size{$urlhost} += $log_size; $tcp_urlhost_time{$urlhost} += $log_reqtime; $tcp_urltld{$urltld} = $tcp_urltld_size{$urltld} = $tcp_urltld_time{$urltld} = $tcp_hit_urltld{$urltld} = $tcp_hit_urltld_size{$urltld} = 0 unless defined $tcp_urltld{$urltld}; $tcp_urltld{$urltld}++; $tcp_urltld_time{$urltld} += $log_reqtime; $tcp_urltld_size{$urltld} += $log_size;' if ($opt_d); $loop .= ' $tcp_urlprot{$urlprot} = $tcp_urlprot_size{$urlprot} = $tcp_hit_urlprot{$urlprot} = $tcp_hit_urlprot_size{$urlprot} = 0 unless defined $tcp_urlprot{$urlprot}; $tcp_urlprot{$urlprot}++; $tcp_urlprot_time{$urlprot} += $log_reqtime; $tcp_urlprot_size{$urlprot} += $log_size;' if ($opt_t); if ( not defined $opt_p ) { } elsif ( $opt_p eq 'old' ) { $loop .= ' $peak_tcp_sec_pointer++; $peak_tcp_min_pointer++; unshift ( @peak_tcp, $log_date ); $peak_tcp_sec_pointer-- while $peak_tcp[ $peak_tcp_sec_pointer - 1 ] < ( $log_date - 1 ); $peak_tcp_min_pointer-- while $peak_tcp[ $peak_tcp_min_pointer - 1 ] < ( $log_date - 60 ); pop (@peak_tcp) while $peak_tcp[$#peak_tcp] < ( $log_date - 3600 ); if ( $peak_tcp_hour < @peak_tcp ) { $peak_tcp_hour = @peak_tcp; $peak_tcp_hour_time = $log_date - 3600; } if ( $peak_tcp_min < $peak_tcp_min_pointer ) { $peak_tcp_min = $peak_tcp_min_pointer; $peak_tcp_min_time = $log_date - 60; } if ( $peak_tcp_sec < $peak_tcp_sec_pointer ) { $peak_tcp_sec = $peak_tcp_sec_pointer; $peak_tcp_sec_time = $log_date - 1; }'; } elsif ( $opt_p eq 'new' ) { $loop .= ' foreach $i ( $date_hour / 3600 .. int( $log_date / 3600 ) ) { $peak_tcp_hour{ $i * 3600 } = $peak_tcp_hour_size{ $i * 3600 } = 0 unless defined $peak_tcp_hour{ $i * 3600 }; $peak_tcp_hour{ $i * 3600 }++; $peak_tcp_hour_size{ $i * 3600 } += $log_size / int( $log_reqtime / 3600000 + 1 ); } $peak_tcp_sec_pointer++; unshift ( @peak_tcp, $log_date ); $peak_tcp_sec_pointer-- while $peak_tcp[ $peak_tcp_sec_pointer - 1 ] < ( $log_date - 1 ); pop (@peak_tcp) while $peak_tcp[$#peak_tcp] < ( $log_date - 60 ); if ( $peak_tcp_min < @peak_tcp ) { $peak_tcp_min = @peak_tcp; $peak_tcp_min_time = $log_date - 60; } if ( $peak_tcp_sec < $peak_tcp_sec_pointer ) { $peak_tcp_sec = $peak_tcp_sec_pointer; $peak_tcp_sec_time = $log_date - 1; }'; } $loop .= ' $tcp_content{$log_content} = $tcp_content_size{$log_content} = $tcp_content_time{$log_content} = $tcp_hit_content{$log_content} = $tcp_hit_content_size{$log_content} = 0 unless defined $tcp_content{$log_content}; $tcp_content{$log_content}++; $tcp_content_time{$log_content} += $log_reqtime; $tcp_content_size{$log_content} += $log_size;' unless $opt_f eq 'squid-old' or $opt_f eq 'elff' or $opt_f eq 'nse'; $loop .= ' $tcp_urlext{$urlext} = $tcp_urlext_size{$urlext} = $tcp_urlext_time{$urlext} = $tcp_hit_urlext{$urlext} = $tcp_hit_urlext_size{$urlext} = $tcp_urlext_fresh{$urlext} = $tcp_urlext_stale{$urlext} = $tcp_urlext_refresh{$urlext} = $tcp_urlext_mod{$urlext} = $tcp_urlext_unmod{$urlext} = 0 unless defined $tcp_urlext{$urlext}; $tcp_urlext{$urlext}++; $tcp_urlext_time{$urlext} += $log_reqtime; $tcp_urlext_size{$urlext} += $log_size;' if ($opt_t); $loop .= ' $tcp_urlext_fresh{$urlext}++ if grep { /^$log_hitfail$/ } @{$fresh_tags{$opt_f}}; $tcp_urlext_stale{$urlext}++ if grep { /^$log_hitfail$/ } @{$stale_tags{$opt_f}}; $tcp_urlext_refresh{$urlext}++ if grep { /^$log_hitfail$/ } @{$refresh_tags{$opt_f}}; $tcp_urlext_mod{$urlext}++ if grep { /^$log_hitfail$/ } @{$mod_tags{$opt_f}}; $tcp_urlext_unmod{$urlext}++ if grep { /^$log_hitfail$/ } @{$unmod_tags{$opt_f}};' if ($object_freshness_report and $opt_t); if ( $opt_f eq 'elff' ) { $loop .= ' if ( $log_hitfail eq \'1\' or $log_hitfail =~ m#^(TCP_HIT|HIT_)#o ) {'; } else { $loop .= ' if ( $log_hitfail =~ m#^TCP_\w*HIT#o ) {'; } $loop .= ' $tcp_hit++; $tcp_hit_size += $log_size; $tcp_hit_time += $log_reqtime;'; $loop .= ' $tcp_hit_reqtime{$log_reqtime}++; $tcp_hit_reqtime_size{$log_reqtime} += $log_size;' if $opt_response_time; $loop .= ' $tcp_hit_code{$log_code}++; $tcp_hit_code_size{$log_code} += $log_size;' if $opt_errorcode_distribution; $loop .= ' $perf_tcp_hit{$perf_date}++; $perf_tcp_hit_size{$perf_date} += $log_size; $perf_tcp_hit_time{$perf_date} += $log_reqtime;' if ($opt_P); $loop .= ' $tcp_hit{$log_hitfail} = $tcp_hit_size{$log_hitfail} = $tcp_hit_time{$log_hitfail} = 0 unless defined $tcp_hit{$log_hitfail}; $tcp_hit{$log_hitfail}++; $tcp_hit_size{$log_hitfail} += $log_size; $tcp_hit_time{$log_hitfail} += $log_reqtime;' if ($opt_s) and $opt_f ne 'elff'; $loop .= ' $tcp_hit_requester{$requester}++; $tcp_hit_requester_size{$requester} += $log_size;' if ($opt_r); $loop .= ' $tcp_hit_requester_urlhost{$requester}{$urlhost}++; $tcp_hit_requester_urlhost_size{$requester}{$urlhost} += $log_size;' if ($opt_R); $loop .= ' $tcp_hit_distribution{$distribution}++; $tcp_hit_distribution_size{$distribution} += $log_size;' if ($opt_D); $loop .= ' $tcp_hit_urlhost{$urlhost}++; $tcp_hit_urlhost_size{$urlhost} += $log_size; $tcp_hit_urltld{$urltld}++; $tcp_hit_urltld_size{$urltld} += $log_size;' if ($opt_d); $loop .= ' $tcp_hit_content{$log_content}++; $tcp_hit_content_time{$log_content} += $log_reqtime; $tcp_hit_content_size{$log_content} += $log_size;' unless $opt_f eq 'squid-old' or $opt_f eq 'elff' or $opt_f eq 'nse'; $loop .= ' $tcp_hit_urlext{$urlext}++; $tcp_hit_urlext_size{$urlext} += $log_size; $tcp_hit_urlprot{$urlprot}++; $tcp_hit_urlprot_size{$urlprot} += $log_size;' if ($opt_t); $loop .= ' } elsif ( $log_hier_method =~ m#EMPTY|NONE|NULL|UNKNOWN|^\-$#o or $log_hitfail =~ m#^ERR_#o ) { $tcp_miss_none++; $tcp_miss_none_size += $log_size; $tcp_miss_none_time += $log_reqtime;' if $opt_f ne 'elff'; $loop .= ' $tcp_miss_none{$log_hitfail} = $tcp_miss_none_size{$log_hitfail} = $tcp_miss_none_time{$log_hitfail} = 0 unless defined $tcp_miss_none{$log_hitfail}; $tcp_miss_none{$log_hitfail}++; $tcp_miss_none_size{$log_hitfail} += $log_size; $tcp_miss_none_time{$log_hitfail} += $log_reqtime;' if ($opt_s) and $opt_f ne 'elff'; $loop .= ' } else { $tcp_miss++; $tcp_miss_size += $log_size; $tcp_miss_time += $log_reqtime;'; $loop .= ' $perf_tcp_miss_size{$perf_date} += $log_size; $perf_tcp_miss_time{$perf_date} += $log_reqtime;' if ($opt_P); $loop .= ' $tcp_miss{$log_hitfail} = $tcp_miss_size{$log_hitfail} = $tcp_miss_time{$log_hitfail} = 0 unless defined $tcp_miss{$log_hitfail}; $tcp_miss{$log_hitfail}++; $tcp_miss_size{$log_hitfail} += $log_size; $tcp_miss_time{$log_hitfail} += $log_reqtime;' if ($opt_s) and $opt_f ne 'elff'; $loop .= ' $tcp_miss_requester{$requester} = $tcp_miss_requester_size{$requester} = 0 unless defined $tcp_miss_requester{$requester}; $tcp_miss_requester{$requester}++; $tcp_miss_requester_size{$requester} += $log_size;' if ($opt_r); $loop .= ' } if ( $log_hier_method !~ m#EMPTY|NONE|NULL|UNKNOWN|^\-$#o ) { $hier++; $hier_size += $log_size; $hier_time += $log_reqtime; if ( $log_hier_method =~ m#DIRECT|SOURCE_FASTEST#o ) { $hier_direct++; $hier_direct_size += $log_size; $hier_direct_time += $log_reqtime;'; $loop .= ' $perf_hier_direct_size{$perf_date} += $log_size; $perf_hier_direct_time{$perf_date} += $log_reqtime;' if ($opt_P); $loop .= ' $hier_direct{$log_hier_method} = $hier_direct_size{$log_hier_method} = $hier_direct_time{$log_hier_method} = 0 unless defined $hier_direct{$log_hier_method}; $hier_direct{$log_hier_method}++; $hier_direct_size{$log_hier_method} += $log_size; $hier_direct_time{$log_hier_method} += $log_reqtime;' if ($opt_s) and $opt_f ne 'elff'; $loop .= ' } elsif ( $log_hier_method =~ m#(CACHE_DIGEST|NEIGHBOR|PARENT|SIBLING)_\w*HIT|SIBLING#o ) { $hier_sibling++; $hier_sibling_size += $log_size; $hier_sibling_time += $log_reqtime;'; $loop .= ' $perf_hier_sibling_size{$perf_date} += $log_size; $perf_hier_sibling_time{$perf_date} += $log_reqtime;' if ($opt_P); $loop .= ' $hier_sibling{$log_hier_method} = $hier_sibling_size{$log_hier_method} = $hier_sibling_time{$log_hier_method} = 0 unless defined $hier_sibling{$log_hier_method}; $hier_sibling{$log_hier_method}++; $hier_sibling_size{$log_hier_method} += $log_size; $hier_sibling_time{$log_hier_method} += $log_reqtime;' if ($opt_s) and $opt_f ne 'elff'; $loop .= ' $hier_neighbor{$log_hier_host} = $hier_neighbor_size{$log_hier_host} = $hier_neighbor_time{$log_hier_host} = 0 unless defined $hier_neighbor{$log_hier_host}; $hier_neighbor{$log_hier_host}++; $hier_neighbor_size{$log_hier_host} += $log_size; $hier_neighbor_time{$log_hier_host} += $log_reqtime;'; $loop .= ' $hier_neighbor_status{$log_hier_host}{$log_hier_method} = $hier_neighbor_status_size{$log_hier_host}{$log_hier_method} = $hier_neighbor_status_time{$log_hier_host}{$log_hier_method} = 0 unless defined $hier_neighbor_status{$log_hier_host}{$log_hier_method}; $hier_neighbor_status{$log_hier_host}{$log_hier_method}++; $hier_neighbor_status_size{$log_hier_host}{$log_hier_method} += $log_size; $hier_neighbor_status_time{$log_hier_host}{$log_hier_method} += $log_reqtime;' if ($opt_s); $loop .= ' } elsif ( $log_hier_method =~ m#(ANY|CLOSEST|DEFAULT|FIRST_UP|PASSTHROUGH|ROUNDROBIN|SINGLE)_PARENT|CARP|PARENT_MISS|PARENT|PROXY#o ) { $hier_parent++; $hier_parent_size += $log_size; $hier_parent_time += $log_reqtime;'; $loop .= ' $perf_hier_parent_size{$perf_date} += $log_size; $perf_hier_parent_time{$perf_date} += $log_reqtime;' if ($opt_P); $loop .= ' $hier_parent{$log_hier_method} = $hier_parent_size{$log_hier_method} = $hier_parent_time{$log_hier_method} = 0 unless defined $hier_parent{$log_hier_method}; $hier_parent{$log_hier_method}++; $hier_parent_size{$log_hier_method} += $log_size; $hier_parent_time{$log_hier_method} += $log_reqtime;' if ($opt_s) and $opt_f ne 'elff'; $loop .= ' $hier_neighbor{$log_hier_host} = $hier_neighbor_size{$log_hier_host} = $hier_neighbor_time{$log_hier_host} = 0 unless defined $hier_neighbor{$log_hier_host}; $hier_neighbor{$log_hier_host}++; $hier_neighbor_size{$log_hier_host} += $log_size; $hier_neighbor_time{$log_hier_host} += $log_reqtime;'; $loop .= ' $hier_neighbor_status{$log_hier_host}{$log_hier_method} = $hier_neighbor_status_size{$log_hier_host}{$log_hier_method} = $hier_neighbor_status_time{$log_hier_host}{$log_hier_method} = 0 unless defined $hier_neighbor_status{$log_hier_host}{$log_hier_method}; $hier_neighbor_status{$log_hier_host}{$log_hier_method}++; $hier_neighbor_status_size{$log_hier_host}{$log_hier_method} += $log_size; $hier_neighbor_status_time{$log_hier_host}{$log_hier_method} += $log_reqtime;' if ($opt_s); $loop .= ' } else { chomp($log_hier_method); unless ( defined $errormsg ) { print STDERR " Please check the following error(s). If you\'re sure that the offending line(s) are NOT corrupt and the error also occurs with the recent version of Calamaris (see the README for pointers and known bugs) then report them. Don\'t send me thousands of similar errors. use . Thank You. " unless $errormsg; $errormsg = 1; } warn( " unknown log_hier_method: \"$log_hier_method\" found in line $counter of input: $line" ); } } } }'; $time_run = time - $time_run; print STDERR "$loop\n" if $opt_L; eval $loop; die $@ if $@; $time_run = time - $time_run; } ### Yea! File read. Now for something completely different ;-) if ( $counter == 0 ) { print "\nno requests found\n"; exit(0); } $loginterval = convertdate($time_begin) . ' - ' . convertdate($time_end); $timestamp = convertdate($time_begin,1) . '-' . convertdate($time_end,1); if ($path) { $path =~ s#%t#$timestamp#g if $timestamp; $path =~ s#%h#$host_name#g if $host_name; $path =~ s#%%#%#g; } if ($filename) { $filename =~ s#%t#$timestamp#g if $timestamp; $filename =~ s#%h#$host_name#g if $host_name; $filename =~ s#%%#%#g; } if ($file_prefix) { $file_prefix =~ s#%t#$timestamp#g if $timestamp; $file_prefix =~ s#%h#$host_name#g if $host_name; $file_prefix =~ s#%%#%#g; } open( CACHE, ">$opt_o" ) or die ("$0: can't open $opt_o for writing: $!\n") if ($opt_o); # Summary $report_index = 0; @format = ( 57, 14, 6 ); writecache( $report_index, $time_begin, $time_end, $counter, $size, $time, $invalid, $time_run, $udp, $udp_size, $udp_time, $udp_hit, $udp_hit_size, $udp_hit_time, $udp_miss, $udp_miss_size, $udp_miss_time, $tcp, $tcp_size, $tcp_time, $tcp_hit, $tcp_hit_size, $tcp_hit_time, $tcp_miss, $tcp_miss_size, $tcp_miss_time, $tcp_miss_none, $tcp_miss_none_size, $tcp_miss_none_time, $hier, $hier_size, $hier_time, $hier_direct, $hier_direct_size, $hier_direct_time, $hier_sibling, $hier_sibling_size, $hier_sibling_time, $hier_parent, $hier_parent_size, $hier_parent_time ); outstart($report_index); outheader( $report_index, 'Calamaris statistics', '', '' ); outseperator($report_index); outline( $report_index, '', 'lines parsed:', 'lines', $counter ); outline( $report_index, '', 'invalid lines:', 'lines', $invalid ); outline( $report_index, '', 'skipped lines:', 'lines', $skipped ) if ($opt_I or $opt_ipfilter_exclude or $opt_ipfilter_include); outline( $report_index, '', 'parse time:', 'sec', $time_run ); outline( $report_index, '', 'parse speed:', 'lines/sec', $time_run ? ( $counter + $invalid + $skipped ) / $time_run : ( $counter + $invalid + $skipped ) ); outseperator($report_index); outheader( $report_index, 'Proxy statistics', '', '' ); outseperator($report_index); outline( $report_index, '', 'Total amount:', 'requests', $tcp + $udp); outline( $report_index, '', 'unique hosts/users:', 'hosts', scalar keys %tcp_requester ) if $opt_r; outline( $report_index, '', 'Total Bandwidth:', 'Byte', kilomegagigatera( $tcp_size + $udp_size, $format[2] ) ); @format = ( 57, 14, '%' ); $peak_all_hour_size_time = ( sort { $peak_all_hour_size{$b} <=> $peak_all_hour_size{$a} } keys(%peak_all_hour_size) )[0] or $peak_all_hour_size_time = $peak_all_hour_size{0} = 0; outline( $report_index, '', 'Max. Bandwidth usage:', 'MBit/sec', $peak_all_hour_size{$peak_all_hour_size_time} * 8 / ( 1024**2 * 3600 ) ) if $opt_p; outline( $report_index, '', 'Proxy efficiency (HIT [kB/sec] / DIRECT [kB/sec]):', 'factor', ($tcp_hit_time and ($tcp_miss_size + $tcp_miss_none_size)) ? $tcp_hit_size/$tcp_hit_time * ($tcp_miss_none_time + $tcp_miss_time) / ($tcp_miss_size + $tcp_miss_none_size) : 0 ); outline( $report_index, '', 'Average speed increase:', '%', ($tcp_miss_size + $tcp_miss_none_size) ? 100 * ( -1 + $tcp_size/$tcp_time * ($tcp_miss_none_time + $tcp_miss_time) / ($tcp_miss_size + $tcp_miss_none_size)) : 0 ); @format = ( 57, 14, 6 ); if ($opt_response_time) { # average response time $reqtime_sum = $reqtime_num = $avg_udp_reqtime = $avg_tcp_reqtime = 0; my $skipped = 0; foreach $reqtime ( keys %udp_reqtime ) { if ($reqtime > $response_time_limit) { $skipped += $tcp_reqtime{$reqtime}; next; } $reqtime_sum += $reqtime * $udp_reqtime{$reqtime}; $reqtime_num += $tcp_reqtime{$reqtime} if $tcp_reqtime{$reqtime}; } $avg_udp_reqtime = ($reqtime_num) ? $reqtime_sum / $reqtime_num : '0'; my $time_skipped = ($response_time_limit == $response_time_report_interval[$#response_time_report_interval]) ? '' : " (requests > $response_time_limit msec skipped)"; my $skipped_percent = (($skipped+$reqtime_num) ? round(100*$reqtime_num/($skipped+$reqtime_num),2) : 100 ) . ' percent'; outline( $report_index, '', "UDP response time of $skipped%% requests$time_skipped:", 'msec', kilomegagigatera( $avg_udp_reqtime, $format[2] ) ) if ($avg_udp_reqtime); $skipped = 0; $reqtime_sum = $reqtime_num = 0; foreach $reqtime ( keys %tcp_reqtime ) { if ($reqtime > $response_time_limit) { $skipped += $tcp_reqtime{$reqtime}; next; } $reqtime_sum += $reqtime * $tcp_reqtime{$reqtime}; $reqtime_num += $tcp_reqtime{$reqtime}; } $avg_tcp_reqtime = ($reqtime_num) ? $reqtime_sum / $reqtime_num : '0'; $skipped_percent = (($skipped+$reqtime_num) ? round(100*$reqtime_num/($skipped+$reqtime_num),2) : 100 ) . '%%'; outline( $report_index, '', "TCP response time of $skipped_percent requests$time_skipped:", 'msec', kilomegagigatera( $avg_tcp_reqtime, $format[2] ) ) if ($avg_tcp_reqtime); } outseperator($report_index); outheader( $report_index, 'Cache statistics', '', ''); outseperator($report_index); outline( $report_index, '', 'Total amount cached:', 'requests', $tcp_hit + $udp_hit ); @format = ( 57, 14, '%' ); outline( $report_index, '', 'Request hit rate:', '%', ($tcp + $udp) ? 100 * ($tcp_hit + $udp_hit) / ($tcp + $udp) : 0 ); @format = ( 57, 14, 6 ); outline( $report_index, '', 'Bandwidth savings:', 'Byte', kilomegagigatera( $tcp_hit_size + $udp_hit_size, $format[2] ) ); @format = ( 57, 14, '%' ); outline( $report_index, '', 'Bandwidth savings in Percent (Byte hit rate):', '%', ($tcp_size + $udp_size) ? 100 * ($tcp_hit_size + $udp_hit_size) / ($tcp_size + $udp_size) : 0 ); @format = ( 57, 14, 6 ); if ($opt_D) { # Object sizes $cached_obj_size = $cached_obj_num = $obj_size = $obj_num = $avg_cached_obj_size = $avg_direct_obj_size = $avg_all_obj_size = 0; foreach $distribution ( sort { $a <=> $b } keys(%tcp_distribution) ) { $cached_obj_size += $tcp_hit_distribution_size{$distribution}; $cached_obj_num += $tcp_hit_distribution{$distribution}; $obj_size += $tcp_distribution_size{$distribution}; $obj_num += $tcp_distribution{$distribution}; } $avg_cached_obj_size = $cached_obj_size / $cached_obj_num if $cached_obj_num; $avg_direct_obj_size = ($obj_size - $cached_obj_size) / ($obj_num - $cached_obj_num) if $obj_num - $cached_obj_num; $avg_all_obj_size = $obj_size / $obj_num if $obj_num; outline( $report_index, '', 'Average cached object size:', 'Byte', kilomegagigatera( $avg_cached_obj_size, $format[2] ) ); outline( $report_index, '', 'Average direct object size:', 'Byte', kilomegagigatera( $avg_direct_obj_size, $format[2] ) ); outline( $report_index, '', 'Average object size:', 'Byte', kilomegagigatera( $avg_all_obj_size, $format[2] ) ); } $max_value[$report_index] = '- - -'; outseperator($report_index); outstop($report_index); # Incoming request peak per protocol $report_index = 1; @format = ( 3, 4, 18, 5, 18, 7, 18 ); if ( not defined $opt_p ) { } elsif ( $opt_p eq 'old' ) { writecache( $report_index, $peak_udp_sec, $peak_udp_sec_time, $peak_udp_min, $peak_udp_min_time, $peak_udp_hour, $peak_udp_hour_time, $peak_tcp_sec, $peak_tcp_sec_time, $peak_tcp_min, $peak_tcp_min_time, $peak_tcp_hour, $peak_tcp_hour_time, $peak_all_sec, $peak_all_sec_time, $peak_all_min, $peak_all_min_time, $peak_all_hour, $peak_all_hour_time ); outstart($report_index); outheader( $report_index, 'prt', ' sec', 'peak begins at', ' min', 'peak begins at', ' hour', 'peak begins at' ); outseperator($report_index); outline( $report_index, '', 'UDP', $peak_udp_sec, convertdate($peak_udp_sec_time), $peak_udp_min, convertdate($peak_udp_min_time), $peak_udp_hour, convertdate($peak_udp_hour_time) ); outline( $report_index, '', 'TCP', $peak_tcp_sec, convertdate($peak_tcp_sec_time), $peak_tcp_min, convertdate($peak_tcp_min_time), $peak_tcp_hour, convertdate($peak_tcp_hour_time) ); outseperator($report_index); outline( $report_index, '', 'ALL', $peak_all_sec, convertdate($peak_all_sec_time), $peak_all_min, convertdate($peak_all_min_time), $peak_all_hour, convertdate($peak_all_hour_time) ); $max_value[$report_index] = '- - -'; outstop($report_index); } elsif ( $opt_p eq 'new' ) { $peak_udp_hour_time = ( sort { $peak_udp_hour{$b} <=> $peak_udp_hour{$a} } keys(%peak_udp_hour) )[0] or $peak_udp_hour_time = $peak_udp_hour{0} = 0; $peak_udp_hour_size_time = ( sort { $peak_udp_hour_size{$b} <=> $peak_udp_hour_size{$a} } keys(%peak_udp_hour_size) )[0] or $peak_udp_hour_size_time = $peak_udp_hour_size{0} = 0; $peak_tcp_hour_time = ( sort { $peak_tcp_hour{$b} <=> $peak_tcp_hour{$a} } keys(%peak_tcp_hour) )[0] or $peak_tcp_hour_time = $peak_tcp_hour{0} = 0; $peak_tcp_hour_size_time = ( sort { $peak_tcp_hour_size{$b} <=> $peak_tcp_hour_size{$a} } keys(%peak_tcp_hour_size) )[0] or $peak_tcp_hour_size_time = $peak_tcp_hour_size{0} = 0; $peak_all_hour_time = ( sort { $peak_all_hour{$b} <=> $peak_all_hour{$a} } keys(%peak_all_hour) )[0] or $peak_all_hour_time = $peak_all_hour{0} = 0; $peak_all_hour_size_time = ( sort { $peak_all_hour_size{$b} <=> $peak_all_hour_size{$a} } keys(%peak_all_hour_size) )[0] or $peak_all_hour_size_time = $peak_all_hour_size{0} = 0; writecache( $report_index, $peak_udp_sec, $peak_udp_sec_time, $peak_udp_min, $peak_udp_min_time, $peak_udp_hour{$peak_udp_hour_time}, $peak_udp_hour_time, $peak_udp_hour_size{$peak_udp_hour_size_time}, $peak_udp_hour_size_time, $peak_tcp_sec, $peak_tcp_sec_time, $peak_tcp_min, $peak_tcp_min_time, $peak_tcp_hour{$peak_tcp_hour_time}, $peak_tcp_hour_time, $peak_tcp_hour_size{$peak_tcp_hour_size_time}, $peak_tcp_hour_size_time, $peak_all_sec, $peak_all_sec_time, $peak_all_min, $peak_all_min_time, $peak_all_hour{$peak_all_hour_time}, $peak_all_hour_time, $peak_all_hour_size{$peak_all_hour_size_time}, $peak_all_hour_size_time ); outstart($report_index); outheader( $report_index, 'prt', ' sec', 'peak begins at', ' min', 'peak begins at', ' hour', 'peak begins at' ); outseperator($report_index); outline( $report_index, '', 'UDP', $peak_udp_sec, convertdate($peak_udp_sec_time), $peak_udp_min, convertdate($peak_udp_min_time), $peak_udp_hour{$peak_udp_hour_time}, convertdate($peak_udp_hour_time) ); outline( $report_index, '', 'TCP', $peak_tcp_sec, convertdate($peak_tcp_sec_time), $peak_tcp_min, convertdate($peak_tcp_min_time), $peak_tcp_hour{$peak_tcp_hour_time}, convertdate($peak_tcp_hour_time) ); outseperator($report_index); outline( $report_index, '', 'ALL', $peak_all_sec, convertdate($peak_all_sec_time), $peak_all_min, convertdate($peak_all_min_time), $peak_all_hour{$peak_all_hour_time}, convertdate($peak_all_hour_time) ); $max_value[$report_index] = '- - -'; outstop($report_index); # Incoming transfer volume per protocol $report_index = 2; @format = ( 5, 8, 18 ); outstart($report_index); outheader( $report_index, 'proto', ' kB/hour', 'peak begins at' ); outseperator($report_index); outline( $report_index, '', 'UDP', $peak_udp_hour_size{$peak_udp_hour_size_time} / 1024, convertdate($peak_udp_hour_size_time) ); outline( $report_index, '', 'TCP', $peak_tcp_hour_size{$peak_tcp_hour_size_time} / 1024, convertdate($peak_tcp_hour_size_time) ); outseperator($report_index); outline( $report_index, '', 'ALL', $peak_all_hour_size{$peak_all_hour_size_time} / 1024, convertdate($peak_all_hour_size_time) ); $max_value[$report_index] = '- - -'; outstop($report_index); } # Incoming requests by method $report_index = 3; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 30, 9, '%', 'spr', 8, '%', 'kbps' ); outstart($report_index); if ( $counter == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); outimg($report_index) if ($outtype_graph); outheader( $report_index, 'method', ' request', '% ', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'auto' ); outseperator($report_index); foreach $method ( sort { ${"method$sortorder"}{$b} <=> ${"method$sortorder"}{$a} } keys(%method) ) { push @xaxis, $method; push @yaxis1, $method{$method}; push @yaxis2, $method_size{$method}; writecache( $report_index, $method, $method{$method}, $method_size{$method}, $method_time{$method} ); outline( $report_index, 'toggle', $method, $method{$method}, 100 * $method{$method} / $counter, $method_time{$method} / ( 1000 * $method{$method} ), kilomegagigatera( $method_size{$method}, $format[4] ), $size ? 100 * $method_size{$method} / $size : 0, $method_size{$method} / ( 1.024 * $method_time{$method} ) ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $counter, 100, $time / ( $counter * 1000 ), kilomegagigatera( $size, $format[4] ), 100, $size / ( 1.024 * $time ) ); outgraph($report_index, \@graph_legend, $x_scale, \@xaxis, \@yaxis1, \@yaxis2) if ($outtype_graph and $xaxis[0]); $max_value[$report_index] = 'most requested method ' . htmlescape($xaxis[0]) . ' ' . ($opt_O ? kilomegagigatera($yaxis2[0], 6) . ' Byte' : $yaxis1[0] . ' Requests' ); } outstop($report_index); # Incoming UDP-requests by status $report_index = 4; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 30, 9, '%', 'mspr', 8, '%', 'kbps' ); outstart($report_index); if ( $udp == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); outimg($report_index) if ($outtype_graph); outheader( $report_index, 'status', ' request', '% ', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'auto' ); outseperator($report_index); if ( $udp_hit == 0 ) { outline( $report_index, '', 'HIT', 0, 0, 0, 0, 0, 0 ); push @xaxis, 'HIT'; push @yaxis1, 0; push @yaxis2, 0; } else { push @xaxis, 'HIT'; push @yaxis1, $udp_hit; push @yaxis2, $udp_hit_size; outline( $report_index, '', 'HIT', $udp_hit, 100 * $udp_hit / $udp, $udp_hit_time / (1000 * $udp_hit), kilomegagigatera( $udp_hit_size, $format[4] ), $udp_size ? 100 * $udp_hit_size / $udp_size : 0, $udp_hit_size / ( 1.024 * $udp_hit_time ) ); foreach $hitfail ( sort { ${"udp_hit$sortorder"}{$b} <=> ${"udp_hit$sortorder"}{$a} } keys(%udp_hit) ) { writecache( "$report_index.1", $hitfail, $udp_hit{$hitfail}, $udp_hit_size{$hitfail}, $udp_hit_time{$hitfail} ); outline( $report_index, '', ' ' . $hitfail, $udp_hit{$hitfail}, 100 * $udp_hit{$hitfail} / $udp, $udp_hit_time{$hitfail} / (1000 * $udp_hit{$hitfail}), kilomegagigatera( $udp_hit_size{$hitfail}, $format[4] ), $udp_size ? 100 * $udp_hit_size{$hitfail} / $udp_size : 0, $udp_hit_size{$hitfail} / ( 1.024 * $udp_hit_time{$hitfail} ) ); } } if ( $udp_miss == 0 ) { outline( $report_index, '', 'MISS', 0, 0, 0, 0, 0, 0 ); push @xaxis, 'MISS'; push @yaxis1, $udp_miss; push @yaxis2, $udp_miss_size; } else { push @xaxis, 'MISS'; push @yaxis1, $udp_miss; push @yaxis2, $udp_miss_size; outline( $report_index, '', 'MISS', $udp_miss, 100 * $udp_miss / $udp, $udp_miss_time / (1000 * $udp_miss), kilomegagigatera( $udp_miss_size, $format[4] ), $udp_size ? 100 * $udp_miss_size / $udp_size : 0, $udp_miss_size / ( 1.024 * $udp_miss_time ) ); foreach $hitfail ( sort { ${"udp_miss$sortorder"}{$b} <=> ${"udp_miss$sortorder"}{$a} } keys(%udp_miss) ) { writecache( "$report_index.2", $hitfail, $udp_miss{$hitfail}, $udp_miss_size{$hitfail}, $udp_miss_time{$hitfail} ); outline( $report_index, '', ' ' . $hitfail, $udp_miss{$hitfail}, 100 * $udp_miss{$hitfail} / $udp, $udp_miss_time{$hitfail} / (1000 * $udp_miss{$hitfail}), kilomegagigatera( $udp_miss_size{$hitfail}, $format[4] ), $udp_size ? 100 * $udp_miss_size{$hitfail} / $udp_size : 0, $udp_miss_size{$hitfail} / ( 1.024 * $udp_miss_time{$hitfail} ) ); } } outseperator($report_index); outline( $report_index, '2', 'Sum', $udp, 100, $udp_time / (1000 * $udp), kilomegagigatera( $udp_size, $format[4] ), 100, $udp_size / ( 1.024 * $udp_time ) ); outgraph( $report_index, \@graph_legend, $x_scale, \@xaxis, \@yaxis1, \@yaxis2) if ($outtype_graph and $xaxis[0]); # find max value my ($max, $maxi) = ($opt_O) ? maxi(@yaxis2) : maxi(@yaxis1); if (defined($maxi) and defined($xaxis[$maxi])) { $max_value[$report_index] = 'most incoming request by status to ' . htmlescape($xaxis[$maxi]) . ' ' . ($opt_O ? kilomegagigatera($yaxis2[$maxi], 6) . ' Byte' : $yaxis1[$maxi] . ' Requests'); } } outstop($report_index); # Incoming TCP-requests by status $report_index = 5; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 30, 9, '%', 'spr', 8, '%', 'kbps' ); outstart($report_index); if ( $tcp_hit + $tcp_miss + $tcp_miss_none == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); outimg($report_index) if ($outtype_graph); outheader( $report_index, 'status', ' request', '% ', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'auto' ); outseperator($report_index); if ( $tcp_hit == 0 ) { outline( $report_index, '', 'HIT', 0, 0, 0, 0, 0, 0 ); push @xaxis, 'HIT'; push @yaxis1, 0; push @yaxis2, 0; } else { push @xaxis, 'HIT'; push @yaxis1, $tcp_hit; push @yaxis2, $tcp_hit_size; outline( $report_index, 'toggle', 'HIT', $tcp_hit, 100 * $tcp_hit / $tcp, $tcp_hit_time / ( 1000 * $tcp_hit ), kilomegagigatera( $tcp_hit_size, $format[4] ), $tcp_size ? 100 * $tcp_hit_size / $tcp_size : 0, $tcp_hit_size / ( 1.024 * $tcp_hit_time ) ); foreach $hitfail ( sort { ${"tcp_hit$sortorder"}{$b} <=> ${"tcp_hit$sortorder"}{$a} } keys(%tcp_hit) ) { writecache( "$report_index.1", $hitfail, $tcp_hit{$hitfail}, $tcp_hit_size{$hitfail}, $tcp_hit_time{$hitfail} ); outline( $report_index, 'toggle', ' ' . $hitfail, $tcp_hit{$hitfail}, 100 * $tcp_hit{$hitfail} / $tcp, $tcp_hit_time{$hitfail} / ( 1000 * $tcp_hit{$hitfail} ), kilomegagigatera( $tcp_hit_size{$hitfail}, $format[4] ), $tcp_size ? 100 * $tcp_hit_size{$hitfail} / $tcp_size : 0, $tcp_hit_size{$hitfail} / ( 1.024 * $tcp_hit_time{$hitfail} ) ); } } if ( $tcp_miss == 0 ) { outline( $report_index, '', 'MISS', 0, 0, 0, 0, 0, 0 ); push @xaxis, 'MISS'; push @yaxis1, 0; push @yaxis2, 0; } else { push @xaxis, 'MISS'; push @yaxis1, $tcp_miss; push @yaxis2, $tcp_miss_size; outline( $report_index, 'toggle', 'MISS', $tcp_miss, 100 * $tcp_miss / $tcp, $tcp_miss_time / ( 1000 * $tcp_miss ), kilomegagigatera( $tcp_miss_size, $format[4] ), $tcp_size ? 100 * $tcp_miss_size / $tcp_size : 0, $tcp_miss_size / ( 1.024 * $tcp_miss_time ) ); foreach $hitfail ( sort { ${"tcp_miss$sortorder"}{$b} <=> ${"tcp_miss$sortorder"}{$a} } keys(%tcp_miss) ) { writecache( "$report_index.2", $hitfail, $tcp_miss{$hitfail}, $tcp_miss_size{$hitfail}, $tcp_miss_time{$hitfail} ); outline( $report_index, 'toggle', ' ' . $hitfail, $tcp_miss{$hitfail}, 100 * $tcp_miss{$hitfail} / $tcp, $tcp_miss_time{$hitfail} / ( 1000 * $tcp_miss{$hitfail} ), kilomegagigatera( $tcp_miss_size{$hitfail}, $format[4] ), $tcp_size ? 100 * $tcp_miss_size{$hitfail} / $tcp_size : 0, $tcp_miss_size{$hitfail} / ( 1.024 * $tcp_miss_time{$hitfail} ) ); } } if ( $tcp_miss_none == 0 ) { outline( $report_index, '', 'ERROR', 0, 0, 0, 0, 0, 0 ); push @xaxis, 'ERROR'; push @yaxis1, 0; push @yaxis2, 0; } else { push @xaxis, 'ERROR'; push @yaxis1, $tcp_miss_none; push @yaxis2, $tcp_miss_none_size; outline( $report_index, 'toggle', 'ERROR', $tcp_miss_none, 100 * $tcp_miss_none / $tcp, $tcp_miss_none_time / ( 1000 * $tcp_miss_none ), kilomegagigatera( $tcp_miss_none_size, $format[4] ), $tcp_size ? 100 * $tcp_miss_none_size / $tcp_size : 0, $tcp_miss_none_size / ( 1.024 * $tcp_miss_none_time ) ); foreach $hitfail ( sort { ${"tcp_miss_none$sortorder"}{$b} <=> ${"tcp_miss_none$sortorder"}{$a} } keys(%tcp_miss_none) ) { writecache( "$report_index.3", $hitfail, $tcp_miss_none{$hitfail}, $tcp_miss_none_size{$hitfail}, $tcp_miss_none_time{$hitfail} ); outline( $report_index, 'toggle', ' ' . $hitfail, $tcp_miss_none{$hitfail}, 100 * $tcp_miss_none{$hitfail} / $tcp, $tcp_miss_none_time{$hitfail} / ( 1000 * $tcp_miss_none{$hitfail} ), kilomegagigatera( $tcp_miss_none_size{$hitfail}, $format[4] ), $tcp_size ? 100 * $tcp_miss_none_size{$hitfail} / $tcp_size : 0, $tcp_miss_none_size{$hitfail} / ( 1.024 * $tcp_miss_none_time{$hitfail} ) ); } } outseperator($report_index); outline( $report_index, '2', 'Sum', $tcp, 100, $tcp_time / ( 1000 * $tcp ), kilomegagigatera( $tcp_size, $format[4] ), 100, $tcp_size / ( 1.024 * $tcp_time ) ); outgraph( $report_index, \@graph_legend, $x_scale, \@xaxis, \@yaxis1, \@yaxis2) if ($outtype_graph and $xaxis[0]); # find max value my ($max, $maxi) = ($opt_O) ? maxi(@yaxis2) : maxi(@yaxis1); if (defined($maxi) and defined($xaxis[$maxi])) { $max_value[$report_index] = 'most incoming request by status to ' . htmlescape($xaxis[$maxi]) . ' ' . ($opt_O ? kilomegagigatera($yaxis2[$maxi], 6) . " Byte" : $yaxis1[$maxi]. " Requests" ); } } outstop($report_index); # Outgoing requests by status $report_index = 6; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 30, 9, '%', 'spr', 8, '%', 'kbps' ); outstart($report_index); if ( $hier == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); outimg($report_index) if ($outtype_graph); outheader( $report_index, 'status', ' request', '% ', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'auto' ); outseperator($report_index); if ( $hier_direct == 0 ) { outline( $report_index, '', 'DIRECT', 0, 0, 0, 0, 0, 0 ); push @xaxis, 'DIRECT'; push @yaxis1, 0; push @yaxis2, 0; } else { push @xaxis, 'DIRECT Fetch from Source'; push @yaxis1, $hier_direct; push @yaxis2, $hier_direct_size; outline( $report_index, 'toggle', 'DIRECT Fetch from Source', $hier_direct, 100 * $hier_direct / $hier, $hier_direct_time / ( 1000 * $hier_direct ), kilomegagigatera( $hier_direct_size, $format[4] ), $hier_size ? 100 * $hier_direct_size / $hier_size : 0, $hier_direct_size / ( 1.024 * $hier_direct_time ) ); foreach $hitfail ( sort { ${"hier_direct$sortorder"}{$b} <=> ${"hier_direct$sortorder"}{$a} } keys(%hier_direct) ) { writecache( "$report_index.1", $hitfail, $hier_direct{$hitfail}, $hier_direct_size{$hitfail}, $hier_direct_time{$hitfail} ); outline( $report_index, 'toggle', ' ' . $hitfail, $hier_direct{$hitfail}, 100 * $hier_direct{$hitfail} / $hier, $hier_direct_time{$hitfail} / ( 1000 * $hier_direct{$hitfail} ), kilomegagigatera( $hier_direct_size{$hitfail}, $format[4] ), $hier_size ? 100 * $hier_direct_size{$hitfail} / $hier_size : 0, $hier_direct_size{$hitfail} / ( 1.024 * $hier_direct_time{$hitfail} ) ); } } if ( $hier_sibling == 0 ) { outline( $report_index, '', 'SIBLING', 0, 0, 0, 0, 0, 0 ); push @xaxis, 'SIBLING'; push @yaxis1, 0; push @yaxis2, 0; } else { push @xaxis, 'HIT on Sibling or Parent Cache'; push @yaxis1, $hier_sibling; push @yaxis2, $hier_sibling_size; outline( $report_index, 'toggle', 'HIT on Sibling or Parent Cache', $hier_sibling, 100 * $hier_sibling / $hier, $hier_sibling_time / ( 1000 * $hier_sibling ), kilomegagigatera( $hier_sibling_size, $format[4] ), $hier_size ? 100 * $hier_sibling_size / $hier_size : 0, $hier_sibling_size / ( 1.024 * $hier_sibling_time ) ); foreach $hitfail ( sort { ${"hier_sibling$sortorder"}{$b} <=> ${"hier_sibling$sortorder"}{$a} } keys(%hier_sibling) ) { writecache( "$report_index.2", $hitfail, $hier_sibling{$hitfail}, $hier_sibling_size{$hitfail}, $hier_sibling_time{$hitfail} ); outline( $report_index, 'toggle', ' ' . $hitfail, $hier_sibling{$hitfail}, 100 * $hier_sibling{$hitfail} / $hier, $hier_sibling_time{$hitfail} / ( 1000 * $hier_sibling{$hitfail} ), kilomegagigatera( $hier_sibling_size{$hitfail}, $format[4] ), $hier_size ? 100 * $hier_sibling_size{$hitfail} / $hier_size : 0, $hier_sibling_size{$hitfail} / ( 1.024 * $hier_sibling_time{$hitfail} ) ); } } if ( $hier_parent == 0 ) { outline( $report_index, '', 'PARENT', 0, 0, 0, 0, 0, 0 ); push @xaxis, 'PARENT'; push @yaxis1, 0; push @yaxis2, 0; } else { push @xaxis, 'FETCH from Parent Cache'; push @yaxis1, $hier_parent; push @yaxis2, $hier_parent_size; outline( $report_index, 'toggle', 'FETCH from Parent Cache', $hier_parent, 100 * $hier_parent / $hier, $hier_parent_time / ( 1000 * $hier_parent ), kilomegagigatera( $hier_parent_size, $format[4] ), $hier_size ? 100 * $hier_parent_size / $hier_size : 0, $hier_parent_size / ( 1.024 * $hier_parent_time ) ); foreach $hitfail ( sort { ${"hier_parent$sortorder"}{$b} <=> ${"hier_parent$sortorder"}{$a} } keys(%hier_parent) ) { writecache( "$report_index.3", $hitfail, $hier_parent{$hitfail}, $hier_parent_size{$hitfail}, $hier_parent_time{$hitfail} ); outline( $report_index, 'toggle', ' ' . $hitfail, $hier_parent{$hitfail}, 100 * $hier_parent{$hitfail} / $hier, $hier_parent_time{$hitfail} / ( 1000 * $hier_parent{$hitfail} ), kilomegagigatera( $hier_parent_size{$hitfail}, $format[4] ), $hier_size ? 100 * $hier_parent_size{$hitfail} / $hier_size : 0, $hier_parent_size{$hitfail} / ( 1.024 * $hier_parent_time{$hitfail} ) ); } } outseperator($report_index); outline( $report_index, '2', 'Sum', $hier, 100, $hier_time / ( 1000 * $hier ), kilomegagigatera( $hier_size, $format[4] ), 100, $hier_size / ( 1.024 * $hier_time ) ); outgraph($report_index, \@graph_legend, $x_scale, \@xaxis, \@yaxis1, \@yaxis2) if ($outtype_graph and $xaxis[0]); # find max value my ($max, $maxi) = ($opt_O) ? maxi(@yaxis2) : maxi(@yaxis1); if (defined($maxi) and defined($xaxis[$maxi])) { $max_value[$report_index] = 'most outgoing request to ' . htmlescape($xaxis[$maxi]) . ' ' . ($opt_O ? kilomegagigatera($yaxis2[$maxi], 6) . ' Byte' : $yaxis1[$maxi]. ' Requests' ); } } outstop($report_index); # Outgoing requests by destination $report_index = 7; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 30, 9, '%', 'spr', 8, '%', 'kbps' ); outstart($report_index); if ( $hier == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); outimg($report_index) if ($outtype_graph); outheader( $report_index, 'neighbor type', ' request', '% ', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'auto' ); outseperator($report_index); outline( $report_index, 'toggle', 'DIRECT', $hier_direct, 100 * $hier_direct / $hier, $hier_direct_time / ( 1000 * $hier_direct ), kilomegagigatera( $hier_direct_size, $format[4] ), $hier_size ? 100 * $hier_direct_size / $hier_size : 0, $hier_direct_size / ( 1.024 * $hier_direct_time ) ) unless $hier_direct == 0; push @xaxis, 'DIRECT' unless $hier_direct == 0; push @yaxis1, $hier_direct unless $hier_direct == 0; push @yaxis2, $hier_direct_size unless $hier_direct == 0; foreach $neighbor ( sort { ${"hier_neighbor$sortorder"}{$b} <=> ${"hier_neighbor$sortorder"}{$a} } keys(%hier_neighbor) ) { push @xaxis, $neighbor; push @yaxis1, $hier_neighbor{$neighbor}; push @yaxis2, $hier_neighbor_size{$neighbor}; writecache( "$report_index.1", $neighbor, $hier_neighbor{$neighbor}, $hier_neighbor_size{$neighbor}, $hier_neighbor_time{$neighbor} ); outline( $report_index, 'toggle', $neighbor, $hier_neighbor{$neighbor}, 100 * $hier_neighbor{$neighbor} / $hier, $hier_neighbor_time{$neighbor} / ( 1000 * $hier ), kilomegagigatera( $hier_neighbor_size{$neighbor}, $format[4] ), $hier_size ? 100 * $hier_neighbor_size{$neighbor} / $hier_size : 0, $hier_neighbor_size{$neighbor} / ( 1.024 * $hier_neighbor_time{$neighbor} ) ); foreach $status ( sort { ${"hier_neighbor_status$sortorder"}{$neighbor}{$b} <=> ${"hier_neighbor_status$sortorder"}{$neighbor}{$a} } keys(%{$hier_neighbor_status{$neighbor} } ) ) { writecache( "$report_index.2", $neighbor, $status, $hier_neighbor_status{$neighbor}{$status}, $hier_neighbor_status_size{$neighbor}{$status}, $hier_neighbor_status_time{$neighbor}{$status} ); outline( $report_index, 'toggle', ' ' . $status, $hier_neighbor_status{$neighbor}{$status}, 100 * $hier_neighbor_status{$neighbor}{$status} / $hier, $hier_neighbor_status_time{$neighbor}{$status} / ( 1000 * $hier_neighbor_status{$neighbor}{$status} ), kilomegagigatera( $hier_neighbor_status_size{$neighbor}{$status}, $format[4] ), $hier_size ? 100 * $hier_neighbor_status_size{$neighbor}{$status} / $hier_size : 0, $hier_neighbor_status_size{$neighbor}{$status} / ( 1.024 * $hier_neighbor_status_time{$neighbor}{$status} ) ); } } outseperator($report_index); outline( $report_index, '2', 'Sum', $hier, 100, $hier_time / ( 1000 * $hier ), kilomegagigatera( $hier_size, $format[4] ), 100, $hier_size / ( 1.024 * $hier_time ) ); outgraph( $report_index, \@graph_legend, $x_scale, \@xaxis, \@yaxis1, \@yaxis2) if ($outtype_graph and $xaxis[0]); # find max value my ($max, $maxi) = ($opt_O) ? maxi(@yaxis2) : maxi(@yaxis1); if (defined($maxi) and defined($xaxis[$maxi])) { $max_value[$report_index] = 'most requested destination ' . htmlescape($xaxis[$maxi]) . ' ' . ($opt_O ? kilomegagigatera($yaxis2[$maxi], 6) . ' Byte' : $yaxis1[$maxi] . ' Requests'); } } outstop($report_index); # Request-destinations by ${N}-level-domain $report_index = 8; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 32, 9, '%', '%', 'off', 8, '%', '%', 'off' ); if ($opt_d) { outstart($report_index); if ( $tcp == 0 ) { outline( $report_index, '', 'no matching requests' ); outstop($report_index); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); $max_x_data = ($opt_d < $x_scale and $opt_d != -1) ? $opt_d : $x_scale; $i = 0; outimg($report_index) if ($outtype_graph); @counter = keys %tcp_urlhost; $other_urlhost = $#counter + 1; $other = $tcp; $other_size = $tcp_size; $other_hit = $tcp_hit; $other_time = $tcp_time; $other_hit_size = $tcp_hit_size; $other_count = $opt_d; outheader( $report_index, 'destination', ' request', '% ', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'hit-%', 'auto' ); outseperator($report_index); foreach $urlhost ( sort { ${"tcp_urlhost$sortorder"}{$b} <=> ${"tcp_urlhost$sortorder"}{$a} } keys(%tcp_urlhost) ) { next if $urlhost eq ''; $other_urlhost--; $other -= $tcp_urlhost{$urlhost}; $other_size -= $tcp_urlhost_size{$urlhost}; $other_time -= $tcp_urlhost_time{$urlhost}; $other_hit -= $tcp_hit_urlhost{$urlhost}; $other_hit_size -= $tcp_hit_urlhost_size{$urlhost}; $i++; push @xaxis, $urlhost if ($i < $max_x_data); push @yaxis1, $tcp_urlhost{$urlhost} if ($i < $max_x_data); push @yaxis2, $tcp_urlhost_size{$urlhost} if ($i < $max_x_data); push @yaxis3, $tcp_urlhost{$urlhost} ? $tcp_hit_urlhost{$urlhost} / $tcp_urlhost{$urlhost} : 0 if ($i < $max_x_data); push @yaxis4, $tcp_urlhost_size{$urlhost} ? $tcp_hit_urlhost_size{$urlhost} / $tcp_urlhost_size{$urlhost} : 0 if ($i < $max_x_data); writecache( $report_index, $urlhost, $tcp_urlhost{$urlhost}, $tcp_urlhost_size{$urlhost}, $tcp_hit_urlhost{$urlhost}, $tcp_hit_urlhost_size{$urlhost}, $tcp_urlhost_time{$urlhost} ); outline( $report_index, 'toggle', $urlhost, $tcp_urlhost{$urlhost}, 100 * $tcp_urlhost{$urlhost} / $tcp, 100 * $tcp_hit_urlhost{$urlhost} / $tcp_urlhost{$urlhost}, $tcp_urlhost_time{$urlhost} / (1000 * $tcp_urlhost{$urlhost}), kilomegagigatera( $tcp_urlhost_size{$urlhost}, $format[5] ), $tcp_size ? 100 * $tcp_urlhost_size{$urlhost} / $tcp_size : 0, $tcp_urlhost_size{$urlhost} ? 100 * $tcp_hit_urlhost_size{$urlhost} / $tcp_urlhost_size{$urlhost} : 0, $tcp_urlhost_time{$urlhost} ? $tcp_urlhost_size{$urlhost} / ( 1.024 * $tcp_urlhost_time{$urlhost} ) : 0 ); last if ( ( --$other_count == 0 and $other != 1 ) or ($tcp_urlhost{$urlhost} < $opt_domain_report_limit) ); } if ($other) { push @xaxis, ''; push @yaxis1, $other; push @yaxis2, $other_size; push @yaxis3, $other ? $other_hit / $other : 0; push @yaxis4, $other_size ? $other_hit_size / $other_size : 0; $max_x_data = ( $opt_d < $x_scale and $opt_d != -1 ) ? $opt_d + 1 : $x_scale + 1; writecache( $report_index, '', $other, $other_size, $other_hit, $other_hit_size, $other_time ); $other_urlhost = '' unless $show_other_tcp_urlhost; outline( $report_index, '', 'other: ' . $other_urlhost . " $N-level-domains", $other, 100 * $other / $tcp, 100 * $other_hit / $other, $other_time / (1000 * $other), kilomegagigatera( $other_size, $format[5] ), $tcp_size ? 100 * $other_size / $tcp_size : 0, $other_size ? 100 * $other_hit_size / $other_size : 0, $other_size ? $other_size/ (1.024 * $other_time ) : 0 ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $tcp, 100, 100 * $tcp_hit / $tcp, $tcp_time / (1000 * $tcp), kilomegagigatera( $tcp_size, $format[5] ), 100, $tcp_size ? 100 * $tcp_hit_size / $tcp_size : 0, $tcp_time ? $tcp_size / ( 1.024 * $tcp_time ) : 0 ); outgraph($report_index, \@graph_legend, $max_x_data, \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4) if ($outtype_graph and $xaxis[0]); $max_value[$report_index] = "most requested $N-level-domain " . htmlescape($xaxis[0]) . ' ' . ($opt_O ? kilomegagigatera($yaxis2[0], 6) . ' Byte' : $yaxis1[0] . ' Requests'); test( $report_index, \%tcp_urlhost, \%tcp_urlhost_size, \%tcp_urlhost_time, 'tcp') if $test; outstop($report_index); # Request-destinations by toplevel-domain $report_index = 9; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 32, 9, '%', '%', 'off', 8, '%', '%', 'off' ); outstart($report_index); @xaxis = @yaxis1 = @yaxis2 = @yaxis3 = @yaxis4 = (); $max_x_data = ($opt_d < $x_scale and $opt_d != -1) ? $opt_d : $x_scale; $i = 0; outimg($report_index) if ($outtype_graph); outheader( $report_index, 'destination', ' request', '% ', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'hit-%', 'auto' ); outseperator($report_index); @counter = keys %tcp_urltld; $other_tld = $#counter + 1; $other = $tcp; $other_size = $tcp_size; $other_hit = $tcp_hit; $other_time = $tcp_time; $other_hit_size = $tcp_hit_size; $other_count = $opt_d; foreach $urltld ( sort { ${"tcp_urltld$sortorder"}{$b} <=> ${"tcp_urltld$sortorder"}{$a} } keys(%tcp_urltld) ) { next if $urltld eq ''; $other_tld--; $other -= $tcp_urltld{$urltld}; $other_size -= $tcp_urltld_size{$urltld}; $other_time -= $tcp_urltld_time{$urltld}; $other_hit -= $tcp_hit_urltld{$urltld}; $other_hit_size -= $tcp_hit_urltld_size{$urltld}; $i++; push @xaxis, $urltld if ($i < $max_x_data); push @yaxis1, $tcp_urltld{$urltld} if ($i < $max_x_data); push @yaxis2, $tcp_urltld_size{$urltld} if ($i < $max_x_data); push @yaxis3, $tcp_urltld{$urltld} ? $tcp_hit_urltld{$urltld} / $tcp_urltld{$urltld} : 0 if ($i < $max_x_data); push @yaxis4, $tcp_urltld_size{$urltld} ? $tcp_hit_urltld_size{$urltld} / $tcp_urltld_size{$urltld} : 0 if ($i < $max_x_data); writecache( $report_index, $urltld, $tcp_urltld{$urltld}, $tcp_urltld_size{$urltld}, $tcp_hit_urltld{$urltld}, $tcp_hit_urltld_size{$urltld}, $tcp_urltld_time{$urltld} ); outline( $report_index, 'toggle', $urltld, $tcp_urltld{$urltld}, 100 * $tcp_urltld{$urltld} / $tcp, 100 * $tcp_hit_urltld{$urltld} / $tcp_urltld{$urltld}, $tcp_urltld_time{$urltld} / (1000 * $tcp_urltld{$urltld}), kilomegagigatera( $tcp_urltld_size{$urltld}, $format[5] ), $tcp_size ? 100 * $tcp_urltld_size{$urltld} / $tcp_size : 0, $tcp_urltld_size{$urltld} ? 100 * $tcp_hit_urltld_size{$urltld} / $tcp_urltld_size{$urltld} : 0, $tcp_urltld_time{$urltld} ? $tcp_urltld_size{$urltld} / ( 1.024 * $tcp_urltld_time{$urltld} ) : 0 ); last if ( ( --$other_count == 0 and $other != 1 ) or ($tcp_urltld{$urltld} < $opt_domain_report_limit) ); } if ($other) { push @xaxis, ''; push @yaxis1, $other; push @yaxis2, $other_size; push @yaxis3, $other ? $other_hit / $other : 0; push @yaxis4, $other_size ? $other_hit_size / $other_size : 0; $max_x_data = ( $opt_d < $x_scale and $opt_d != -1 ) ? $opt_d + 1 : $x_scale + 1; writecache( $report_index, '', $other, $other_size, $other_hit, $other_hit_size, $other_time ); $other_tld = '' unless $show_other_tcp_urltld; outline( $report_index, '', 'other: ' . $other_tld . ' top-level-domains', $other, 100 * $other / $tcp, 100 * $other_hit / $other, $other_time / (1000 * $other), kilomegagigatera( $other_size, $format[5] ), $tcp_size ? 100 * $other_size / $tcp_size : 0, $other_size ? 100 * $other_hit_size / $other_size : 0, $other_size ? $other_size/ (1.024 * $other_time ) : 0 ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $tcp, 100, 100 * $tcp_hit / $tcp, $tcp_time / (1000 * $tcp), kilomegagigatera( $tcp_size, $format[5] ), 100, $tcp_size ? 100 * $tcp_hit_size / $tcp_size : 0, $tcp_time ? $tcp_size / ( 1.024 * $tcp_time ) : 0 ); outgraph($report_index, \@graph_legend, $max_x_data, \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4) if ($outtype_graph and $xaxis[0]); outstop($report_index); test( $report_index, \%tcp_urltld, \%tcp_urltld_size, \%tcp_urltld_time, 'tcp') if $test; $max_value[$report_index] = 'most requested toplevel-domain ' . htmlescape($xaxis[0]) . ' ' . ($opt_O ? kilomegagigatera($yaxis2[0], 6) . " Byte" : $yaxis1[0]. " Requests"); } } # TCP-Request-protocol $report_index = 10; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 32, 9, '%', '%', 'off', 8, '%', '%', 'off' ); if ($opt_t) { outstart($report_index); if ( $tcp == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); $max_x_data = ($opt_t < $x_scale and $opt_t != -1) ? $opt_t : $x_scale; $i = 0; outimg($report_index) if ($outtype_graph); outheader( $report_index, 'protocol', ' request', '% ', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'hit-%', 'auto' ); outseperator($report_index); foreach $urlprot ( sort { ${"tcp_urlprot$sortorder"}{$b} <=> ${"tcp_urlprot$sortorder"}{$a} } keys(%tcp_urlprot) ) { push @xaxis, $urlprot; push @yaxis1, $tcp_urlprot{$urlprot}; push @yaxis2, $tcp_urlprot_size{$urlprot}; push @yaxis3, $tcp_urlprot{$urlprot} ? $tcp_hit_urlprot{$urlprot} / $tcp_urlprot{$urlprot} : 0; push @yaxis4, $tcp_urlprot_size{$urlprot} ? $tcp_hit_urlprot_size{$urlprot} / $tcp_urlprot_size{$urlprot} : 0; writecache( $report_index, $urlprot, $tcp_urlprot{$urlprot}, $tcp_urlprot_size{$urlprot}, $tcp_hit_urlprot{$urlprot}, $tcp_hit_urlprot_size{$urlprot}, $tcp_urlprot_time{$urlprot} ); outline( $report_index, 'toggle', $urlprot, $tcp_urlprot{$urlprot}, 100 * $tcp_urlprot{$urlprot} / $tcp, 100 * $tcp_hit_urlprot{$urlprot} / $tcp_urlprot{$urlprot}, $tcp_urlprot_time{$urlprot} / (1000 * $tcp_urlprot{$urlprot}), kilomegagigatera( $tcp_urlprot_size{$urlprot}, $format[5] ), $tcp_size ? 100 * $tcp_urlprot_size{$urlprot} / $tcp_size : 0, $tcp_urlprot_size{$urlprot} ? 100 * $tcp_hit_urlprot_size{$urlprot} / $tcp_urlprot_size{$urlprot} : 0, $tcp_urlprot_time{$urlprot} ? $tcp_urlprot_size{$urlprot} / ( 1.024 * $tcp_urlprot_time{$urlprot} ) : 0 ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $tcp, 100, 100 * $tcp_hit / $tcp, $tcp_time / (1000 * $tcp), kilomegagigatera( $tcp_size, $format[5] ), 100, $tcp_size ? 100 * $tcp_hit_size / $tcp_size : 0, $tcp_size / $tcp_time ); outgraph($report_index, \@graph_legend, $max_x_data, \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4) if ($outtype_graph and $xaxis[0]); test( $report_index, \%tcp_urlprot, \%tcp_urlprot_size, \%tcp_urlprot_time, 'tcp') if $test; # find max value my ($max, $maxi) = ($opt_O) ? maxi(@yaxis2) : maxi(@yaxis1); if (defined($maxi) and defined($xaxis[$maxi])) { $max_value[$report_index] = 'most requested protocol ' . htmlescape($xaxis[$maxi]) . ' ' . ($opt_O ? kilomegagigatera($yaxis2[$maxi], 6) . ' Byte' : $yaxis1[0] . ' Requests'); } } outstop($report_index); # Requested content-type $report_index = 11; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 32, 9, '%', '%', 'off', 8, '%', '%', 'off' ); if ( defined(%tcp_content) ) { outstart($report_index); if ( $tcp == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); $max_x_data = ($opt_t < $x_scale and $opt_t != -1) ? $opt_t : $x_scale; $i = 0; outimg($report_index) if ($outtype_graph); @counter = keys %tcp_content; $other_content = $#counter + 1; $other = $tcp; $other_size = $tcp_size; $other_time = $tcp_time; $other_hit = $tcp_hit; $other_hit_size = $tcp_hit_size; $other_count = $opt_t; outheader( $report_index, 'content-type', ' request', '% ', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'hit-%', 'auto' ); outseperator($report_index); foreach $content ( sort { ${"tcp_content$sortorder"}{$b} <=> ${"tcp_content$sortorder"}{$a} } keys(%tcp_content) ) { next if $content eq ''; $other_content--; $other -= $tcp_content{$content}; $other_size -= $tcp_content_size{$content}; $other_time -= $tcp_content_time{$content}; $other_hit -= $tcp_hit_content{$content}; $other_hit_size -= $tcp_hit_content_size{$content}; $i++; push @xaxis, $content if ($i < $max_x_data); push @yaxis1, $tcp_content{$content} if ($i < $max_x_data); push @yaxis2, $tcp_content_size{$content} if ($i < $max_x_data); push @yaxis3, $tcp_content{$content} ? $tcp_hit_content{$content} / $tcp_content{$content} : 0 if ($i < $max_x_data); push @yaxis4, $tcp_content_size{$content} ? $tcp_hit_content_size{$content} / $tcp_content_size{$content} : 0 if ($i < $max_x_data); writecache( $report_index, $content, $tcp_content{$content}, $tcp_content_size{$content}, $tcp_hit_content{$content}, $tcp_hit_content_size{$content}, $tcp_content_time{$content} ); outline( $report_index, 'toggle', $content, $tcp_content{$content}, 100 * $tcp_content{$content} / $tcp, 100 * $tcp_hit_content{$content} / $tcp_content{$content}, $tcp_content_time{$content} / (1000 * $tcp_content{$content}), kilomegagigatera( $tcp_content_size{$content}, $format[5] ), $tcp_size ? 100 * $tcp_content_size{$content} / $tcp_size : 0, $tcp_content_size{$content} ? 100 * $tcp_hit_content_size{$content} / $tcp_content_size{$content} : 0, $tcp_content_time{$content} ? $tcp_content_size{$content} / ( 1.024 * $tcp_content_time{$content} ) : 0 ); last if ( --$other_count == 0 and $other != 1 ); } if ($other) { push @xaxis, ''; push @yaxis1, $other; push @yaxis2, $other_size; push @yaxis3, $other ? $other_hit / $other : 0; push @yaxis4, $other_size ? $other_hit_size / $other_size : 0; $max_x_data = ($opt_t < $x_scale and $opt_t != -1) ? $opt_t + 1 : $x_scale + 1; writecache( $report_index, '', $other, $other_size, $other_hit, $other_hit_size, $other_time ); $other_content = '' unless $show_other_tcp_content; outline( $report_index, '', 'other: ' . $other_content . ' content-types', $other, 100 * $other / $tcp, 100 * $other_hit / $other, $other_time / (1000 * $other), kilomegagigatera( $other_size, $format[5] ), $tcp_size ? 100 * $other_size / $tcp_size : 0, $other_size ? 100 * $other_hit_size / $other_size : 0, $other_size ? $other_size/ (1.024 * $other_time ) : 0 ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $tcp, 100, 100 * $tcp_hit / $tcp, $tcp_time / (1000 * $tcp), kilomegagigatera( $tcp_size, $format[5] ), 100, $tcp_size ? 100 * $tcp_hit_size / $tcp_size : 0, $tcp_time ? $tcp_size / ( 1.024 * $tcp_time ) : 0 ); outgraph($report_index, \@graph_legend, $max_x_data, \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4) if ($outtype_graph and $xaxis[0]); test( $report_index, \%tcp_content, \%tcp_content_size, \%tcp_content_time, 'tcp') if $test; $max_value[$report_index] = 'most requested content-type ' . htmlescape($xaxis[0]) . ' ' . ($opt_O ? kilomegagigatera($yaxis2[0], 6) . ' Byte' : $yaxis1[0] . ' Requests'); } outstop($report_index); } # Requested extensions $report_index = 12; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 32, 9, '%', '%', 'off', 8, '%', '%', 'off', 'off', 'off' ); outstart($report_index); if ( $tcp == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); $max_x_data = ($opt_t < $x_scale and $opt_t != -1) ? $opt_t : $x_scale; $i = 0; outimg($report_index) if ($outtype_graph); @counter = keys %tcp_urlext; $other_urlext = $#counter + 1; $other = $tcp; $other_size = $tcp_size; $other_time = $tcp_time; $other_hit = $tcp_hit; $other_hit_size = $tcp_hit_size; $other_count = $opt_t; $sum_tcp_urlext_fresh = 0; $sum_tcp_urlext_stale = 0; $sum_tcp_urlext_refresh = 0; $sum_tcp_urlext_mod = 0; $sum_tcp_urlext_unmod = 0; foreach $urlext ( sort {${"tcp_urlext$sortorder"}{$b} <=> ${"tcp_urlext$sortorder"}{$a} } keys(%tcp_urlext) ) { $sum_tcp_urlext_fresh += $tcp_urlext_fresh{$urlext}; $sum_tcp_urlext_stale += $tcp_urlext_stale{$urlext}; $sum_tcp_urlext_refresh += $tcp_urlext_refresh{$urlext}; $sum_tcp_urlext_mod += $tcp_urlext_mod{$urlext}; $sum_tcp_urlext_unmod += $tcp_urlext_unmod{$urlext}; } $other_fresh = $sum_tcp_urlext_fresh; $other_stale = $sum_tcp_urlext_stale; $other_mod = $sum_tcp_urlext_mod; $other_unmod = $sum_tcp_urlext_unmod; $other_refresh = $sum_tcp_urlext_refresh; outheader( $report_index, 'extensions', ' request', '% ', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'hit-%', 'auto', 'fresh/stale', 'unmod/mod' ); outseperator($report_index); foreach $urlext ( sort {${"tcp_urlext$sortorder"}{$b} <=> ${"tcp_urlext$sortorder"}{$a} } keys(%tcp_urlext) ) { $sum_tcp_urlext_fresh += $tcp_urlext_fresh{$urlext}; $sum_tcp_urlext_stale += $tcp_urlext_stale{$urlext}; $sum_tcp_urlext_refresh += $tcp_urlext_refresh{$urlext}; $sum_tcp_urlext_mod += $tcp_urlext_mod{$urlext}; $sum_tcp_urlext_unmod += $tcp_urlext_unmod{$urlext}; } foreach $urlext ( sort {${"tcp_urlext$sortorder"}{$b} <=> ${"tcp_urlext$sortorder"}{$a} } keys(%tcp_urlext) ) { next if $urlext eq ''; $other_urlext--; $other -= $tcp_urlext{$urlext}; $other_size -= $tcp_urlext_size{$urlext}; $other_time -= $tcp_urlext_time{$urlext}; $other_hit -= $tcp_hit_urlext{$urlext}; $other_hit_size -= $tcp_hit_urlext_size{$urlext}; $other_fresh -= $tcp_urlext_fresh{$urlext}; $other_stale -= $tcp_urlext_stale{$urlext}; $other_mod -= $tcp_urlext_mod{$urlext}; $other_unmod -= $tcp_urlext_unmod{$urlext}; $other_refresh -= $tcp_urlext_refresh{$urlext}; $i++; push @xaxis, $urlext if ($i < $max_x_data); push @yaxis1, $tcp_urlext{$urlext} if ($i < $max_x_data); push @yaxis2, $tcp_urlext_size{$urlext} if ($i < $max_x_data); push @yaxis3, $tcp_urlext{$urlext} ? $tcp_hit_urlext{$urlext} / $tcp_urlext{$urlext} : 0 if ($i < $max_x_data); push @yaxis4, $tcp_urlext_size{$urlext} ? $tcp_hit_urlext_size{$urlext} / $tcp_urlext_size{$urlext} : 0 if ($i < $max_x_data); writecache( $report_index, $urlext, $tcp_urlext{$urlext}, $tcp_urlext_size{$urlext}, $tcp_hit_urlext{$urlext}, $tcp_hit_urlext_size{$urlext}, $tcp_urlext_time{$urlext}, $tcp_urlext_fresh{$urlext}, $tcp_urlext_stale{$urlext}, $tcp_urlext_refresh{$urlext}, $tcp_urlext_mod{$urlext}, $tcp_urlext_unmod{$urlext} ); outline( $report_index, 'toggle', $urlext, $tcp_urlext{$urlext}, 100 * $tcp_urlext{$urlext} / $tcp, 100 * $tcp_hit_urlext{$urlext} / $tcp_urlext{$urlext}, $tcp_urlext_time{$urlext} / ( 1000 * $tcp_urlext{$urlext} ), kilomegagigatera( $tcp_urlext_size{$urlext}, $format[5] ), $tcp_size ? 100 * $tcp_urlext_size{$urlext} / $tcp_size : 0, $tcp_urlext_size{$urlext} ? 100 * $tcp_hit_urlext_size{$urlext} / $tcp_urlext_size{$urlext} : 0, $tcp_urlext_time{$urlext} ? $tcp_urlext_size{$urlext} / ( 1.024 * $tcp_urlext_time{$urlext} ) : 0, join ('/', gcd($tcp_urlext_fresh{$urlext}, $tcp_urlext_stale{$urlext})), join ('/', gcd($tcp_urlext_unmod{$urlext}, $tcp_urlext_mod{$urlext})) ); last if ( --$other_count == 0 and $other != 1 ); } if ($other) { push @xaxis, ''; push @yaxis1, $other; push @yaxis2, $other_size; push @yaxis3, $other ? $other_hit / $other : 0; push @yaxis4, $other_size ? $other_hit_size / $other_size : 0; $max_x_data = ( $opt_t < $x_scale and $opt_t != -1 ) ? $opt_t + 1 : $x_scale + 1; writecache( $report_index, '', $other, $other_size, $other_hit, $other_hit_size, $other_time, $other_fresh, $other_stale, $other_refresh, $other_mod, $other_unmod ); $other_urlext = '' unless $show_other_tcp_urlext; outline( $report_index, '', 'other: ' . $other_urlext . ' extensions', $other, 100 * $other / $tcp, 100 * $other_hit / $other, $other_time / (1000 * $other), kilomegagigatera( $other_size, $format[5] ), $tcp_size ? 100 * $other_size / $tcp_size : 0, $other_size ? 100 * $other_hit_size / $other_size : 0, $other_size ? $other_size/ (1.024 * $other_time ) : 0, join ('/', gcd($other_fresh, $other_stale)), join ('/', gcd($other_unmod, $other_mod)) ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $tcp, 100, 100 * $tcp_hit / $tcp, $tcp_time / (1000 * $tcp), kilomegagigatera( $tcp_size, $format[5] ), 100, $tcp_size ? 100 * $tcp_hit_size / $tcp_size : 0, $tcp_time ? $tcp_size / ( 1.024 * $tcp_time ) : 0, join ('/', gcd($sum_tcp_urlext_fresh, $sum_tcp_urlext_stale)), join ('/', gcd($sum_tcp_urlext_mod, $sum_tcp_urlext_unmod)) ); outgraph($report_index, \@graph_legend, $max_x_data, \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4) if ($outtype_graph and $xaxis[0]); test( $report_index, \%tcp_urlext, \%tcp_urlext_size, \%tcp_urlext_time, 'tcp') if $test; $max_value[$report_index] = 'most requested extension ' . htmlescape($xaxis[0]) . ' ' . ($opt_O ? kilomegagigatera($yaxis2[0], 6) . ' Byte' : $yaxis1[0]. ' Requests'); } outstop($report_index); } if ($opt_r) { # Incoming UDP-requests by host $report_index = 13; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 38, 9, 'off', '%', 'off', 8, 'off', '%', 'kbps' ); outstart($report_index); if ( $udp == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); $max_x_data = ($opt_t < $x_scale and $opt_t != -1) ? $opt_t : $x_scale; $i = 0; outimg($report_index) if ($outtype_graph); if ($opt_R) { outheader( $report_index, 'host / target', ' request', '%', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '%', 'hit-%', 'auto' ); } else { outheader( $report_index, 'host', ' request', '%', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '%', 'hit-%', 'auto' ); } outseperator($report_index); @counter = keys %udp_requester; $other_requester = $#counter + 1; $other = $udp; $other_size = $udp_size; $other_time = $udp_time; $other_hit = $udp_hit; $other_hit_size = $udp_hit_size; $other_count = $opt_r; foreach $requester ( sort { ${"udp_requester$sortorder"}{$b} <=> ${"udp_requester$sortorder"}{$a} } keys(%udp_requester) ) { next if $requester eq ''; $other_requester--; $other -= $udp_requester{$requester}; $other_size -= $udp_requester_size{$requester}; $other_time -= $udp_requester_time{$requester}; $other_hit -= $udp_hit_requester{$requester}; $other_hit_size -= $udp_hit_requester_size{$requester}; $i++; push @xaxis, $requester if ($i < $max_x_data); push @yaxis1, $udp_requester{$requester} if ($i < $max_x_data); push @yaxis2, $udp_requester_size{$requester} if ($i < $max_x_data); push @yaxis3, $udp_requester{$requester} ? $udp_hit_requester{$requester} / $udp_requester{$requester} : 0 if ($i < $max_x_data); push @yaxis4, $udp_requester_size{$requester} ? $udp_hit_requester_size{$requester} / $udp_requester_size{$requester} : 0 if ($i < $max_x_data); writecache( "$report_index.1", $requester, $udp_requester{$requester}, $udp_requester_size{$requester}, $udp_requester_time{$requester}, $udp_hit_requester{$requester}, $udp_hit_requester_size{$requester} ); outline( $report_index, '', $requester, $udp_requester{$requester}, $udp ? 100 * $udp_requester{$requester} / $udp : 0, 100 * $udp_hit_requester{$requester} / $udp_requester{$requester}, $udp_requester{$requester} ? $udp_requester_time{$requester} / ( 1000 * $udp_requester{$requester} ) : 0, kilomegagigatera( $udp_requester_size{$requester}, $format[5] ), $udp_size ? 100 * $udp_requester_size{$requester} / $udp_size : 0, $udp_requester_size{$requester} ? 100 * $udp_hit_requester_size{$requester} / $udp_requester_size{$requester} : 0, $udp_requester_size{$requester} / ( 1.024 * $udp_requester_time{$requester} ) ); if ($opt_R) { @counter2 = keys( %{ $udp_requester_urlhost{$requester} } ); $other2_requester_urlhost = $#counter2 + 1; $other2 = $udp_requester{$requester}; $other2_size = $udp_requester_size{$requester}; $other2_time = $udp_requester_time{$requester}; $other2_hit = $udp_hit_requester{$requester}; $other2_hit_size = $udp_hit_requester_size{$requester}; $other2_count = $opt_R; foreach $urlhost ( sort { ${"udp_requester_urlhost$sortorder"}{$requester}{$b} <=> ${"udp_requester_urlhost$sortorder"}{$requester}{$a} } keys( %{ $udp_requester_urlhost{$requester} } ) ) { next if $urlhost eq ''; $other2_requester_urlhost--; $other2 -= $udp_requester_urlhost{$requester}{$urlhost}; $other2_size -= $udp_requester_urlhost_size{$requester}{$urlhost}; $other2_time -= $udp_requester_urlhost_time{$requester}{$urlhost}; $other2_hit -= $udp_hit_requester_urlhost{$requester}{$urlhost}; $other2_hit_size -= $udp_hit_requester_urlhost_size{$requester}{$urlhost}; writecache( "$report_index.2", $requester, $urlhost, $udp_requester_urlhost{$requester}{$urlhost}, $udp_requester_urlhost_size{$requester}{$urlhost}, $udp_requester_urlhost_time{$requester}{$urlhost}, $udp_hit_requester_urlhost{$requester}{$urlhost}, $udp_hit_requester_urlhost_size{$requester}{$urlhost} ); outline( $report_index, 'toggle', ' ' . $urlhost, $udp_requester_urlhost{$requester}{$urlhost}, '', 100 * $udp_hit_requester_urlhost{$requester}{$urlhost} / $udp_requester_urlhost{$requester}{$urlhost}, $udp_requester_urlhost_time{$requester}{$urlhost} / (1000 * $udp_requester_urlhost{$requester}{$urlhost}), kilomegagigatera( $udp_requester_urlhost_size{$requester}{$urlhost}, $format[5] ), '', $udp_requester_urlhost_size{$requester}{$urlhost} ? 100 * $udp_hit_requester_urlhost_size{$requester}{$urlhost} / $udp_requester_urlhost_size{$requester}{$urlhost} : 0, $udp_requester_urlhost_size{$requester}{$urlhost} / ( 1.024 * $udp_requester_urlhost_time{$requester}{$urlhost} ) ); last if ( --$other2_count == 0 and $other2 != 1 ); } if ($other2) { writecache( "$report_index.2", $requester, '', $other2, $other2_size, $other2_time, $other2_hit, $other2_hit_size ); outline( $report_index, '', ' other: ' . $other2_requester_urlhost . ' requested urlhosts', $other2, '', 100 * $other2_hit / $other2, $other2_time / ( 1000 * $other2_requester_urlhost ), kilomegagigatera( $other2_size, $format[5] ), '', $other2_size ? 100 * $other2_hit_size / $other2_size : 0, $other2_size / ( 1.024 * $other2_time ) ); } } last if ( --$other_count == 0 and $other != 1 ); } if ($other) { push @xaxis, ''; push @yaxis1, $other; push @yaxis2, $other_size; push @yaxis3, $other ? $other_hit / $other : 0; push @yaxis4, $other_size ? $other_hit_size / $other_size : 0; $max_x_data = ( $opt_r < $x_scale and $opt_r != -1 ) ? $opt_r + 1 : $x_scale + 1; writecache( "$report_index.1", '', $other, $other_size, $other_time, $other_hit, $other_hit_size ); $other_requester = '' unless $show_other_udp_requester; outline( $report_index, '', 'other: ' . $other_requester . ' requesting hosts', $other, '', 100 * $other_hit / $other, $other_time / ( 1000 * $udp ), kilomegagigatera( $other_size, $format[5] ), '', 100 * $other_hit_size / $other_size, $other_size / ( 1.024 * $udp_time ) ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $udp, 100, 100 * $udp_hit / $udp, $udp_time / (1000 * $udp), kilomegagigatera( $udp_size, $format[5] ), 100, $udp_size ? 100 * $udp_hit_size / $udp_size : 0, $udp_time ? $udp_size / ( 1.024 * $udp_time ) : 0 ); outgraph( $report_index, \@graph_legend, $max_x_data, \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4 ) if ($outtype_graph and $xaxis[0]); test( $report_index, \%udp_requester, \%udp_requester_size, \%udp_requester_time, 'udp') if $test; $max_value[$report_index] = 'most active host ' . htmlescape($xaxis[0]) . ' ' . ($opt_O ? kilomegagigatera($yaxis2[0], 6) . ' Byte' : $yaxis1[0] . ' Requests'); } outstop($report_index); # Incoming TCP-requests by host $report_index = 14; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 30, 9, 'off', '%', 'spr', 8, 'off', '%', 'kbps' ); outstart($report_index); if ( $tcp == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); $max_x_data = ($opt_r < $x_scale and $opt_r != -1) ? $opt_r : $x_scale; $i = 0; outimg($report_index) if ($outtype_graph); if ($opt_R) { outheader( $report_index, 'host / target', ' request', '%', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '%', 'hit-%', 'auto' ); } else { outheader( $report_index, 'host', ' request', '%', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '%', 'hit-%', 'auto' ); } outseperator($report_index); @counter = keys %tcp_requester; $other_requester = $#counter + 1; $other = $tcp; $other_size = $tcp_size; $other_time = $tcp_time; $other_hit = $tcp_hit; $other_hit_size = $tcp_hit_size; $other_count = $opt_r; foreach $requester ( sort { ${"tcp_requester$sortorder"}{$b} <=> ${"tcp_requester$sortorder"}{$a} } keys(%tcp_requester) ) { next if $requester eq ''; $other_requester--; $other -= $tcp_requester{$requester}; $other_size -= $tcp_requester_size{$requester}; $other_time -= $tcp_requester_time{$requester}; $other_hit -= $tcp_hit_requester{$requester}; $other_hit_size -= $tcp_hit_requester_size{$requester}; $i++; push @xaxis, $requester if ($i < $max_x_data); push @yaxis1, $tcp_requester{$requester} if ($i < $max_x_data); push @yaxis2, $tcp_requester_size{$requester} if ($i < $max_x_data); push @yaxis3, $tcp_requester{$requester} ? $tcp_hit_requester{$requester} / $tcp_requester{$requester} : 0 if ($i < $max_x_data); push @yaxis4, $tcp_requester_size{$requester} ? $tcp_hit_requester_size{$requester} / $tcp_requester_size{$requester} : 0 if ($i < $max_x_data); writecache( "$report_index.1", $requester, $tcp_requester{$requester}, $tcp_requester_size{$requester}, $tcp_requester_time{$requester}, $tcp_hit_requester{$requester}, $tcp_hit_requester_size{$requester} ); outline( $report_index, 'toggle', uri_unescape($requester), $tcp_requester{$requester}, $tcp ? 100 * $tcp_requester{$requester} / $tcp : 0, 100 * $tcp_hit_requester{$requester} / $tcp_requester{$requester}, $tcp_requester_time{$requester} / ( 1000 * $tcp_requester{$requester} ), kilomegagigatera( $tcp_requester_size{$requester}, $format[5] ), $tcp_size ? 100 * $tcp_requester_size{$requester} / $tcp_size : 0, $tcp_requester_size{$requester} ? 100 * $tcp_hit_requester_size{$requester} / $tcp_requester_size{$requester} : 0, $tcp_requester_size{$requester} / ( 1.024 * $tcp_requester_time{$requester} ) ); if ($opt_R) { @counter2 = keys( %{ $tcp_requester_urlhost{$requester} } ); $other2_requester_urlhost = $#counter2 + 1; $other2 = $tcp_requester{$requester}; $other2_size = $tcp_requester_size{$requester}; $other2_time = $tcp_requester_time{$requester}; $other2_hit = $tcp_hit_requester{$requester}; $other2_hit_size = $tcp_hit_requester_size{$requester}; $other2_count = $opt_R; foreach $urlhost ( sort { ${"tcp_requester_urlhost$sortorder"}{$requester}{$b} <=> ${"tcp_requester_urlhost$sortorder"}{$requester}{$a} } keys( %{ $tcp_requester_urlhost{$requester} } ) ) { next if $urlhost eq ''; $other2_requester_urlhost--; $other2 -= $tcp_requester_urlhost{$requester}{$urlhost}; $other2_size -= $tcp_requester_urlhost_size{$requester}{$urlhost}; $other2_time -= $tcp_requester_urlhost_time{$requester}{$urlhost}; $other2_hit -= $tcp_hit_requester_urlhost{$requester}{$urlhost}; $other2_hit_size -= $tcp_hit_requester_urlhost_size{$requester}{$urlhost}; writecache( "$report_index.2", $requester, $urlhost, $tcp_requester_urlhost{$requester}{$urlhost}, $tcp_requester_urlhost_size{$requester}{$urlhost}, $tcp_requester_urlhost_time{$requester}{$urlhost}, $tcp_hit_requester_urlhost{$requester}{$urlhost}, $tcp_hit_requester_urlhost_size{$requester}{$urlhost} ); outline( $report_index, 'toggle', ' ' . $urlhost, $tcp_requester_urlhost{$requester}{$urlhost}, '', 100 * $tcp_hit_requester_urlhost{$requester}{$urlhost} / $tcp_requester_urlhost{$requester}{$urlhost}, $tcp_requester_urlhost_time{$requester}{$urlhost} / ( 1000 * $tcp_requester_urlhost{$requester}{$urlhost} ), kilomegagigatera( $tcp_requester_urlhost_size{$requester}{$urlhost}, $format[5] ), '', $tcp_requester_urlhost_size{$requester}{$urlhost} ? 100 * $tcp_hit_requester_urlhost_size{$requester}{$urlhost} / $tcp_requester_urlhost_size{$requester}{$urlhost} : 0, $tcp_requester_urlhost_size{$requester}{$urlhost} / ( 1.024 * $tcp_requester_urlhost_time{$requester}{$urlhost} ) ); last if ( --$other2_count == 0 and $other2 != 1 ); } if ($other2) { writecache( "$report_index.2", $requester, '', $other2, $other2_size, $other2_time, $other2_hit, $other2_hit_size ); outline( $report_index, '', ' other: ' . $other2_requester_urlhost . ' requested urlhosts', $other2, '', 100 * $other2_hit / $other2, $other2_time / ( 1000 * $other2_requester_urlhost ), kilomegagigatera( $other2_size, $format[5] ), '', $other2_size ? 100 * $other2_hit_size / $other2_size : 0, $other2_size / ( 1.024 * $other2_time ) ); } } last if ( --$other_count == 0 and $other != 1 ); } if ($other) { push @xaxis, ''; push @yaxis1, $other; push @yaxis2, $other_size; push @yaxis3, $other ? $other_hit / $other : 0; push @yaxis4, $other_size ? $other_hit_size / $other_size : 0; $max_x_data = ( $opt_r < $x_scale and $opt_r != -1 ) ? $opt_r + 1 : $x_scale + 1; writecache( "$report_index.1", '', $other, $other_size, $other_time, $other_hit, $other_hit_size ); $other_requester = '' unless $show_other_tcp_requester; outline( $report_index, '', 'other: ' . $other_requester . ' requesting hosts', $other, '', 100 * $other_hit / $other, $other_time / ( 1000 * $tcp ), kilomegagigatera( $other_size, $format[5] ), '', $other_size ? 100 * $other_hit_size / $other_size : 0, $other_size / ( 1.024 * $tcp_time ) ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $tcp, 100, 100 * $tcp_hit / $tcp, $tcp_time / (1000 * $tcp), kilomegagigatera( $tcp_size, $format[5] ), 100, $tcp_size ? 100 * $tcp_hit_size / $tcp_size : 0, $tcp_time ? $tcp_size / ( 1.024 * $tcp_time ) : 0 ); outgraph( $report_index, \@graph_legend, $max_x_data, \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4 ) if ($outtype_graph and $xaxis[0]); test( $report_index, \%tcp_requester, \%tcp_requester_size, \%tcp_requester_time, 'tcp') if $test; $max_value[$report_index] = 'most active host ' . htmlescape($xaxis[0]) . ' ' . ($opt_O ? kilomegagigatera($yaxis2[0], 6) . ' Byte' : $yaxis1[0] . ' Requests'); } outstop($report_index); } # Size Distribution Diagram $report_index = 15; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 30, 9, 'off', '%', 'spr', 8, 'off', '%', 'kbps' ); if ($opt_D) { outstart($report_index); if ( $tcp == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); outimg($report_index) if ($outtype_graph); outheader( $report_index, 'object-size (bytes)', ' request', '%', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '%', 'hit-%', 'auto' ); outseperator($report_index); foreach $distribution ( sort { $a <=> $b } keys(%tcp_distribution) ) { push @xaxis, int( $opt_D**$distribution ) . '-' . int( $opt_D**( $distribution + 1 ) - 1 ); push @yaxis1, $tcp_distribution{$distribution}; push @yaxis2, $tcp_distribution_size{$distribution}; push @yaxis3, $tcp_distribution{$distribution} ? $tcp_hit_distribution{$distribution} / $tcp_distribution{$distribution} : 0; push @yaxis4, $tcp_distribution_size{$distribution} ? $tcp_hit_distribution_size{$distribution} / $tcp_distribution_size{$distribution} : 0; writecache( $report_index, $distribution, $tcp_distribution{$distribution}, $tcp_distribution_size{$distribution}, $tcp_distribution_time{$distribution}, $tcp_hit_distribution{$distribution}, $tcp_hit_distribution_size{$distribution} ); outline( $report_index, '', int( $opt_D**$distribution ) . '-' . int( $opt_D**( $distribution + 1 ) - 1 ), $tcp_distribution{$distribution}, 100 * $tcp_distribution{$distribution} / $tcp, 100 * $tcp_hit_distribution{$distribution} / $tcp_distribution{$distribution}, $tcp_distribution_time{$distribution} / ( 1000 * $tcp_distribution{$distribution} ), kilomegagigatera( $tcp_distribution_size{$distribution}, $format[5] ), $tcp_size ? 100 * $tcp_distribution_size{$distribution} / $tcp_size : 0, $tcp_distribution_size{$distribution} ? 100 * $tcp_hit_distribution_size{$distribution} / $tcp_distribution_size{$distribution} : 0, $tcp_distribution_size{$distribution} / ( 1.024 * $tcp_distribution_time{$distribution} ) ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $tcp, 100, 100 * $tcp_hit / $tcp, $tcp_time / (1000 * $tcp), kilomegagigatera( $tcp_size, $format[5] ), 100, $tcp_size ? 100 * $tcp_hit_size / $tcp_size : 0, $tcp_time ? $tcp_size / ( 1.024 * $tcp_time ) : 0 ); outgraph( $report_index, \@graph_legend, $x_scale, \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4 ) if ($outtype_graph and $xaxis[0]); # find max value my ($max, $maxi) = ($opt_O) ? maxi(@yaxis2) : maxi(@yaxis1); if (defined($maxi) and defined($xaxis[$maxi])) { $max_value[$report_index] = 'most requested object_size ' . htmlescape($xaxis[$maxi]) . ' ' . ($opt_O ? kilomegagigatera($max, 6) . ' Byte' : $max . ' Requests'); } test( $report_index, \%tcp_distribution, \%tcp_distribution_size, \%tcp_distribution_time, 'tcp') if $test; } outstop($report_index); } # Performance in $P steps $report_index = 16; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 15, 9, 'off', 5, 'off', 'off', 'kbps', 'kbps', 'kbps', 'kbps', 'kbps', 'kbps' ); if ($opt_P) { outstart($report_index); if ( $tcp == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); outimg($report_index) if ($outtype_graph); outheader( $report_index, '', '', '', '', '', '', 'incomin', ' hit', ' miss', ' direct', 'sibling', ' fetch' ); outheader( $report_index, 'date', ' request', 'hit-%', ' Byte', 'hit-%', 'IPs', 'auto', 'auto', 'auto', 'auto', 'auto', 'auto'); outseperator($report_index); foreach $perf_date ( sort { $a <=> $b } keys(%perf_counter) ) { $perf_requester{$perf_date} = scalar keys %{$perf_ip{$perf_date}}; push @xaxis, substr( convertdate($perf_date), 0, 15 ); push @yaxis1, $perf_counter{$perf_date}; push @yaxis2, $perf_size{$perf_date}; push @yaxis3, $perf_counter{$perf_date} ? $perf_tcp_hit{$perf_date} / $perf_counter{$perf_date} : 0; push @yaxis4, $tcp_size ? $perf_tcp_hit_size{$perf_date} / $tcp_size : 0; writecache( $report_index, $perf_date, $perf_counter{$perf_date}, $perf_size{$perf_date}, $perf_time{$perf_date}, $perf_tcp_hit_size{$perf_date}, $perf_tcp_hit_time{$perf_date}, $perf_tcp_miss_size{$perf_date}, $perf_tcp_miss_time{$perf_date}, $perf_hier_direct_size{$perf_date}, $perf_hier_direct_time{$perf_date}, $perf_hier_sibling_size{$perf_date}, $perf_hier_sibling_time{$perf_date}, $perf_hier_parent_size{$perf_date}, $perf_hier_parent_time{$perf_date}, $perf_requester{$perf_date}, $perf_tcp_hit{$perf_date} ); outline( $report_index, 'toggle', substr( convertdate($perf_date), 0, 15 ), $perf_counter{$perf_date}, $perf_counter{$perf_date} ? 100 * $perf_tcp_hit{$perf_date} / $perf_counter{$perf_date} : 0, kilomegagigatera( $perf_size{$perf_date}, $format[3] ), $tcp_size ? 100 * $perf_tcp_hit_size{$perf_date} / $tcp_size : 0, $perf_requester{$perf_date}, removezerotimes( $perf_size{$perf_date}, $perf_time{$perf_date} ), removezerotimes( $perf_tcp_hit_size{$perf_date}, $perf_tcp_hit_time{$perf_date} ), removezerotimes( $perf_tcp_miss_size{$perf_date}, $perf_tcp_miss_time{$perf_date} ), removezerotimes( $perf_hier_direct_size{$perf_date}, $perf_hier_direct_time{$perf_date} ), removezerotimes( $perf_hier_sibling_size{$perf_date}, $perf_hier_sibling_time{$perf_date} ), removezerotimes( $perf_hier_parent_size{$perf_date}, $perf_hier_parent_time{$perf_date} ) ); } outseperator($report_index); outline( $report_index, '2', 'overall', $tcp, 100 * $tcp_hit / $tcp, kilomegagigatera( $tcp_size, $format[3] ), $tcp_size ? 100 * $tcp_hit_size / $tcp_size : 0, scalar keys %tcp_requester, removezerotimes( $tcp_size, $tcp_time ), removezerotimes( $tcp_hit_size, $tcp_hit_time ), removezerotimes( $tcp_miss_size, $tcp_miss_time ), removezerotimes( $hier_direct_size, $hier_direct_time ), removezerotimes( $hier_sibling_size, $hier_sibling_time ), removezerotimes( $hier_parent_size, $hier_parent_time ) ); outgraph( $report_index, \@graph_legend, 31, \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4 ) if ($outtype_graph and $xaxis[0]); test( $report_index, \%perf_counter, \%perf_size, \%perf_time, 'tcp') if $test; my ($max, $maxi) = ($opt_O) ? maxi(@yaxis2) : maxi(@yaxis1); if (defined($maxi) and defined($xaxis[$maxi])) { $max_value[$report_index] = 'most active day ' . htmlescape($xaxis[$maxi]) . ' ' . ($opt_O ? kilomegagigatera($max, 6) . ' Byte' : $max . ' Requests'); } } outstop($report_index); } # UDP-Request duration distribution in msec $report_index = 17; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 16, 9, '%', '%', 'spr', 8, '%', '%', 'kbps' ); if ($opt_response_time) { outstart($report_index); if ( $udp == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my $max = 0; my $max_interval = 0; foreach $reqtime ( keys %udp_reqtime ) { for ($i = 0; $i <= $#response_time_report_interval; $i++) { if ($reqtime <= $response_time_report_interval[$i]) { if (${"udp_reqtime$sortorder"}{$reqtime} > $max) { $max = ${"udp_reqtime$sortorder"}{$reqtime}; $max_interval = "<= $response_time_report_interval[$i]"; } $ordered_udp_req_time_time{"<= $response_time_report_interval[$i]"} += $reqtime * $udp_reqtime{$reqtime}; $ordered_udp_req_time{"<= $response_time_report_interval[$i]"} += $udp_reqtime{$reqtime}; $ordered_udp_req_time_size{"<= $response_time_report_interval[$i]"} += $udp_reqtime_size{$reqtime}; $ordered_udp_hit_req_time{"<= $response_time_report_interval[$i]"} += $udp_hit_reqtime{$reqtime}; $ordered_udp_hit_req_time_size{"<= $response_time_report_interval[$i]"} += $udp_hit_reqtime_size{$reqtime}; if ( ${"ordered_udp_req_time$sortorder"}{"<= $response_time_report_interval[$i]"} - (($i == 0 or !defined(${"ordered_udp_req_time$sortorder"}{"<= $response_time_report_interval[$i-1]"})) ? 0 : ${"ordered_udp_req_time$sortorder"}{"<= $response_time_report_interval[$i-1]"}) > $max) { $max = ${"ordered_udp_req_time$sortorder"}{"<= $response_time_report_interval[$i]"} - (($i == 0 or !defined(${"ordered_udp_req_time$sortorder"}{"<= $response_time_report_interval[$i-1]"})) ? 0 : ${"ordered_udp_req_time$sortorder"}{"<= $response_time_report_interval[$i-1]"}); $max_interval = "<= $response_time_report_interval[$i]"; } } } } my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); outimg($report_index) if ($outtype_graph); outheader( $report_index, 'time', ' request', '% ', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'hit-%', 'auto'); outseperator($report_index); writecache( $report_index, $ordered_udp_req_time_max_interval, $ordered_udp_req_time_max ); foreach $time_interval ( sort { $a =~ m/^(>|<=)\s*(\S+)$/; my $a1 = $2; $b =~ m/^(>|<=)\s*(\S+)$/; my $b1 = $2; return $a1 <=> $b1 } keys(%ordered_udp_req_time) ) { push @xaxis, $time_interval; push @yaxis1, $ordered_udp_req_time{$time_interval}; push @yaxis2, $ordered_udp_req_time_size{$time_interval}; push @yaxis3, $ordered_udp_req_time{$time_interval} ? $ordered_udp_hit_req_time{$time_interval} / $ordered_udp_req_time{$time_interval} : 0; push @yaxis4, $ordered_udp_req_time_size{$time_interval} ? $ordered_udp_hit_req_time_size{$time_interval} / $ordered_udp_req_time_size{$time_interval} : 0; writecache( $report_index, $time_interval, $ordered_udp_req_time{$time_interval}, $ordered_udp_req_time_size{$time_interval}, $ordered_udp_hit_req_time{$time_interval}, $ordered_udp_hit_req_time_size{$time_interval}, $ordered_udp_req_time_time{$time_interval}); outline( $report_index, 'toggle', $time_interval, $ordered_udp_req_time{$time_interval}, 100 * $ordered_udp_req_time{$time_interval} / $udp, 100 * $ordered_udp_hit_req_time{$time_interval} / $ordered_udp_req_time{$time_interval}, $ordered_udp_req_time{$time_interval} ? $ordered_udp_req_time_time{$time_interval} / (1000 * $ordered_udp_req_time{$time_interval}) : 0, kilomegagigatera( $ordered_udp_req_time_size{$time_interval}, $format[5] ), $udp_size ? 100 * $ordered_udp_req_time_size{$time_interval} / $udp_size : 0, $ordered_udp_req_time_size{$time_interval} ? 100 * $ordered_udp_hit_req_time_size{$time_interval} / $ordered_udp_req_time_size{$time_interval} : 0, $ordered_udp_req_time_time{$time_interval} ? $ordered_udp_req_time_size{$time_interval} / (1.024 * $ordered_udp_req_time_time{$time_interval}) : 0 ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $udp, 100, 100 * $udp_hit / $udp, $udp_time / (1000 * $udp), kilomegagigatera( $udp_size, $format[5] ), 100, $udp_size ? 100 * $udp_hit_size / $udp_size : 0, $udp_time ? $udp_size / ( 1.024 * $udp_time ) : 0); outgraph( $report_index, \@graph_legend, scalar(@response_time_report_interval), \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4 ) if ($outtype_graph and $xaxis[0]); test( $report_index, \%udp_reqtime, \%udp_reqtime_size, 0, 'udp') if $test; $max_value[$report_index] = 'most frequent response time ' . htmlescape($max_interval) . ' ' . ($opt_O ? kilomegagigatera($max, 6) . ' Byte' : $max . ' Requests'); } outstop($report_index); # TCP-Request duration distribution in msec $report_index = 18; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 16, 9, '%', '%', 'spr', 8, '%', '%', 'kbps' ); outstart($report_index); if ( $tcp == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { foreach $reqtime ( keys %tcp_reqtime ) { for ( $i = 0; $i <= $#response_time_report_interval; $i++ ) { if ($reqtime <= $response_time_report_interval[$i]) { $ordered_tcp_req_time_time{"<= $response_time_report_interval[$i]"} += $reqtime * $tcp_reqtime{$reqtime}; $ordered_tcp_req_time{"<= $response_time_report_interval[$i]"} += $tcp_reqtime{$reqtime}; $ordered_tcp_req_time_size{"<= $response_time_report_interval[$i]"} += $tcp_reqtime_size{$reqtime}; $ordered_tcp_hit_req_time{"<= $response_time_report_interval[$i]"} += $tcp_hit_reqtime{$reqtime}; $ordered_tcp_hit_req_time_size{"<= $response_time_report_interval[$i]"} += $tcp_hit_reqtime_size{$reqtime}; if ( ${"ordered_tcp_req_time$sortorder"}{"<= $response_time_report_interval[$i]"} - (( $i == 0 or !defined(${"ordered_tcp_req_time$sortorder"}{"<= $response_time_report_interval[$i-1]"})) ? 0 : ${"ordered_tcp_req_time$sortorder"}{"<= $response_time_report_interval[$i-1]"}) > $ordered_tcp_req_time_max) { $ordered_tcp_req_time_max = ${"ordered_tcp_req_time$sortorder"}{"<= $response_time_report_interval[$i]"} - (( $i == 0 or !defined(${"ordered_tcp_req_time$sortorder"}{"<= $response_time_report_interval[$i-1]"})) ? 0 : ${"ordered_tcp_req_time$sortorder"}{"<= $response_time_report_interval[$i-1]"}); $ordered_tcp_req_time_max_interval = "<= $response_time_report_interval[$i]"; } } } } my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); outimg($report_index) if ($outtype_graph); outheader( $report_index, 'time', ' request', '% ', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'hit-%', 'auto'); outseperator($report_index); writecache( $report_index, $ordered_tcp_req_time_max_interval, $ordered_tcp_req_time_max ); foreach $time_interval ( sort { $a =~ m/^(>|<=)\s*(\S+)$/; my $a1 = $2; $b =~ m/^(>|<=)\s*(\S+)$/; my $b1 = $2; return $a1 <=> $b1 } keys(%ordered_tcp_req_time) ) { push @xaxis, $time_interval; push @yaxis1, $ordered_tcp_req_time{$time_interval}; push @yaxis2, $ordered_tcp_req_time_size{$time_interval}; push @yaxis3, $ordered_tcp_req_time{$time_interval} ? $ordered_tcp_hit_req_time{$time_interval} / $ordered_tcp_req_time{$time_interval} : 0; push @yaxis4, $ordered_tcp_req_time_size{$time_interval} ? $ordered_tcp_hit_req_time_size{$time_interval} / $ordered_tcp_req_time_size{$time_interval} : 0; writecache( $report_index, $time_interval, $ordered_tcp_req_time{$time_interval}, $ordered_tcp_req_time_size{$time_interval}, $ordered_tcp_hit_req_time{$time_interval}, $ordered_tcp_hit_req_time_size{$time_interval}, $ordered_tcp_req_time_time{$time_interval}); outline( $report_index, 'toggle', $time_interval, $ordered_tcp_req_time{$time_interval}, 100 * $ordered_tcp_req_time{$time_interval} / $tcp, 100 * $ordered_tcp_hit_req_time{$time_interval} / $ordered_tcp_req_time{$time_interval}, $ordered_tcp_req_time{$time_interval} ? $ordered_tcp_req_time_time{$time_interval} / (1000 * $ordered_tcp_req_time{$time_interval}) : 0, kilomegagigatera( $ordered_tcp_req_time_size{$time_interval}, $format[5] ), $tcp_size ? 100 * $ordered_tcp_req_time_size{$time_interval} / $tcp_size : 0, $ordered_tcp_req_time_size{$time_interval} ? 100 * $ordered_tcp_hit_req_time_size{$time_interval} / $ordered_tcp_req_time_size{$time_interval} : 0, $ordered_tcp_req_time_time{$time_interval} ? $ordered_tcp_req_time_size{$time_interval} / (1.024 * $ordered_tcp_req_time_time{$time_interval}) : 0 ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $tcp, 100, 100 * $tcp_hit / $tcp, $tcp_time / (1000 * $tcp), kilomegagigatera( $tcp_size, $format[5] ), 100, $tcp_size ? 100 * $tcp_hit_size / $tcp_size : 0, $tcp_time ? $tcp_size / ( 1.024 * $tcp_time ) : 0); outgraph( $report_index, \@graph_legend, scalar(@response_time_report_interval), \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4 ) if ($outtype_graph and $xaxis[0]); test( $report_index, \%tcp_reqtime, \%tcp_reqtime_size, 0, 'tcp') if $test; $max_value[$report_index] = 'most frequent response time ' . htmlescape($ordered_tcp_req_time_max_interval) . ' ' . ($opt_O ? kilomegagigatera($ordered_tcp_req_time_max, 6) . ' Byte' : $ordered_tcp_req_time_max . ' Requests'); } outstop($report_index); } # UDP Response code distribution $report_index = 19; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 46, 9, '%', 'off', 'off', 8, '%', 'off', 'off' ); if ($opt_errorcode_distribution) { %err_code = geterrcode(); outstart($report_index); if ( $udp == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); outimg($report_index) if ($outtype_graph); outheader( $report_index, 'status-code', ' request', '% ', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'hit-%', 'auto' ); outseperator($report_index); foreach $code ( sort { $a cmp $b } keys %udp_code ) { push @xaxis, $code; push @yaxis1, $udp_code{$code}; push @yaxis2, $udp_code_size{$code}; push @yaxis3, $udp_code{$code} ? $udp_hit_code{$code} / $udp_code{$code} : 0; push @yaxis4, $udp_code_size{$code} ? $udp_hit_code_size{$code} / $udp_code_size{$code} : 0; writecache( $report_index, $code, $udp_code{$code}, $udp_code_size{$code}, $udp_hit_code{$code}, $udp_hit_code_size{$code}, $udp_code_time{$code}); outline( $report_index, 'toggle', defined($err_code{$code}) ? "$code ($err_code{$code})" : "$code (unknown)", $udp_code{$code}, $udp ? 100 * $udp_code{$code} / $udp : 0, $udp_code{$code} ? 100 * $udp_hit_code{$code} / $udp_code{$code} : 0, $udp_code{$code} ? $udp_code_time{$code} / (1000 * $udp_code{$code}) : 0, kilomegagigatera( $udp_code_size{$code}, $format[5] ), $udp_size ? 100 * $udp_code_size{$code} / $udp_size : 0, $udp_code_size{$code} ? 100 * $udp_hit_code_size{$code} / $udp_code_size{$code} : 0, $udp_code_time{$code} ? $udp_hit_code_size{$code} / (1.024 * $udp_code_time{$code}) : 0 ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $udp, 100, 100 * $udp_hit / $udp, $udp_time / (1000 * $udp), kilomegagigatera( $udp_size, $format[5] ), 100, $udp_size ? 100 * $udp_hit_size / $udp_size : 0, $udp_time ? $udp_size / ( 1.024 * $udp_time ) : 0); outgraph( $report_index, \@graph_legend, $x_scale, \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4 ) if ($outtype_graph and $xaxis[0]); test( $report_index, \%udp_code, \%udp_code_size, \%udp_code_time, 'udp') if $test; my ($max, $maxi) = ($opt_O) ? maxi(@yaxis2) : maxi(@yaxis1); if (defined($maxi) and defined($xaxis[$maxi])) { $max_value[$report_index] = 'most frequent response code ' . htmlescape($xaxis[$maxi]) . ' ' . ($opt_O ? kilomegagigatera($max, 6) . ' Byte' : $max . ' Requests' ); } } outstop($report_index); # TCP Response code distribution $report_index = 20; @format = ref($formats[$report_index]) ? @{$formats[$report_index]} : ( 46, 9, '%', 'off', 'off', 8, '%', 'off', 'off' ); outstart($report_index); if ( $tcp == 0 ) { outline( $report_index, '', 'no matching requests' ); } else { my (@xaxis, @yaxis1, @yaxis2, @yaxis3, @yaxis4); outimg($report_index) if ($outtype_graph); outheader( $report_index, 'status-code', ' request', '% ', 'hit-%', 'auto', $outtype_unformatted ? " ${opt_U}Byte" : ' Byte', '% ', 'hit-%', 'auto' ); outseperator($report_index); foreach $code ( sort { $a cmp $b } keys %tcp_code ) { push @xaxis, $code; push @yaxis1, $tcp_code{$code}; push @yaxis2, $tcp_code_size{$code}; push @yaxis3, $tcp_code{$code} ? $tcp_hit_code{$code} / $tcp_code{$code} : 0; push @yaxis4, $tcp_code_size{$code} ? $tcp_hit_code_size{$code} / $tcp_code_size{$code} : 0; writecache( $report_index, $code, $tcp_code{$code}, $tcp_code_size{$code}, $tcp_hit_code{$code}, $tcp_hit_code_size{$code}, $tcp_code_time{$code}); outline( $report_index, 'toggle', defined($err_code{$code}) ? "$code ($err_code{$code})" : "$code (unknown)", $tcp_code{$code}, $tcp ? 100 * $tcp_code{$code} / $tcp : 0, $tcp_code{$code} ? 100 * $tcp_hit_code{$code} / $tcp_code{$code} : 0, $tcp_code{$code} ? $tcp_code_time{$code} / (1000 * $tcp_code{$code}) : 0, kilomegagigatera( $tcp_code_size{$code}, $format[5] ), $tcp_size ? 100 * $tcp_code_size{$code} / $tcp_size : 0, $tcp_code_size{$code} ? 100 * $tcp_hit_code_size{$code} / $tcp_code_size{$code} : 0, $tcp_code_time{$code} ? $tcp_hit_code_size{$code} / (1.024 * $tcp_code_time{$code}) : 0 ); } outseperator($report_index); outline( $report_index, '2', 'Sum', $tcp, 100, 100 * $tcp_hit / $tcp, $tcp_time / (1000 * $tcp), kilomegagigatera( $tcp_size, $format[5] ), 100, $tcp_size ? 100 * $tcp_hit_size / $tcp_size : 0, $tcp_time ? $tcp_size / ( 1.024 * $tcp_time ) : 0); outgraph( $report_index, \@graph_legend, $x_scale, \@xaxis, \@yaxis1, \@yaxis2, \@yaxis3, \@yaxis4) if ($outtype_graph and $xaxis[0]); test( $report_index, \%tcp_code, \%tcp_code_size, \%tcp_code_time, 'tcp') if $test; my ($max, $maxi) = ($opt_O) ? maxi(@yaxis2) : maxi(@yaxis1); if (defined($maxi) and defined($xaxis[$maxi])) { $max_value[$report_index] = 'most frequent response code ' . htmlescape($xaxis[$maxi]) . ' ' . ($opt_O ? kilomegagigatera($max, 6) . ' Byte' : $max . ' Requests'); } } outstop($report_index); } close(CACHE); ################################################## # now print it out. $generated = convertdate(time); $out_head = ''; $out_head .= "MIME-Version: 1.0\nContent-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit\n" if ( $outtype_mail and $outtype_html ); $out_head .= 'Subject: ' . ( $host_name ? "$host_name " : '') . "Proxy Report ($loginterval)\n\n" if ($outtype_mail); if ( $outtype_html or $outtype_htmlembed or outtype_htmlframe ) { $out_head .= ' ' . ( $host_name ? "$host_name " : '') . 'Proxy Report (' . $loginterval . ') ' . ( $opt_M ? " $opt_M\n" : '' ) . ' ' if ($outtype_html or $outtype_htmlframe); $out_index_head .= ' ' if ($outtype_html or $outtype_htmlembed or $outtype_htmlframe); if ($opt_l) { $out_index_head .= ' '; } $out_index_head .= '
' . $opt_l . '

Proxy Report

' . $host_name . '
Report period: ' . $loginterval . '
Generated at: ' . $generated . '
' if ($outtype_html or $outtype_htmlembed or $outtype_htmlframe); } else { $out_index_head .= "\n${host_name} Proxy-Report\n Report period: $loginterval\nGenerated at: $generated\n"; } @format = (60); if ( ( $invalid / $counter ) > .05 and $counter > 1000 ) { outstart('E'); outline( 'E', '', '' ); outline( 'E', '', 'More than 5% discarded logfile-lines.' ); outline( 'E', '', 'Please check your logfile with calamaris -v' ) unless ($opt_v); outstop('E'); } if ( defined($peak_warn) ) { outstart('E'); outline( 'E', '', '' ); outline( 'E', '', "$peak_warn" ); outline( 'E', '', 'Please read the README on unsorted input' ); outline( 'E', '', 'To find out which line caused this, try calamaris -v' ) unless ($opt_v); outstop('E'); } if ( defined($cache_warn) ) { outstart('E'); outline( 'E', '', '' ); outline( 'E', '', 'with Calamaris V3.x the Cache-File-Format is completely' ); outline( 'E', '', 'changed.' ); outline( 'E', '', '' ); outline( 'E', '', 'To re-use your old Cachefiles you\'ll first have to' ); outline( 'E', '', 'convert them with' ); outline( 'E', '', 'calamaris-cache-convert old.cache new.cache' ); outstop('E'); } if ( $opt_I and $opt_i ) { outstart('E'); outline( 'E', '', '' ); outline( 'E', '', 'You have run Calamaris with the -I (Interval) and the -i' ); outline( 'E', '', '(input cache) option. This is normally not useful,' ); outline( 'E', '', 'because the time-interval cannot be applied to' ); outline( 'E', '', 'cache-files.' ); outstop('E'); } if ($opt_ipfilter_exclude) { outstart('E'); outline( 'E', '', '' ); outline( 'E', '', 'The following IP adresses have been ignored:' ); foreach ( split ( ':', $opt_ipfilter_exclude ) ) { outline( 'E', '', " $_" ); } outstop('E'); } if ($opt_ipfilter_include) { outstart('E'); outline( 'E', '', '' ); outline( 'E', '', 'Only the following IP adresses have been recognized:' ); foreach ( split ( ':', $opt_ipfilter_include ) ) { outline( 'E', '', " $_" ); } outstop('E'); } if ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe ) { $out_index_body .= ' '; @reportlist = ($opt_S) ? ( split ( ",", $opt_S ) ) : (0 .. $#reports); foreach ( @reportlist ) { outref($_) if $outref{$_}; } $out_index_body .= '
Table of Content / Overview
'; } if ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe ) { my $year = (localtime)[5] + 1900; $out_tail = "
$HTMLCOPYRIGHT"; $out_tail .= ' ' if ($outtype_html or $outtype_htmlframe); } else { $out_tail .= "\n\n"; $out_tail .= "-- \n" if ($outtype_mail); $out_tail .= "Calamaris $VERSION\n$COPYRIGHT\n$LICENSE\n$HOMEPAGE\n"; } my $fh = *STDOUT; if ($opt_output_path or $opt_output_file or $opt_output_file_prefix) { print STDERR "writing output to $path/$file_prefix$filename\n" if $opt_v; if ($outtype_htmlframe) { open( OUT, ">$path/${file_prefix}overview.html" ) or die ("$0: can't open $path/${file_prefix}overview.html for writing: $!\n"); } else { open( OUT, ">$path/$file_prefix$filename" ) or die ("$0: can't open $path/$file_prefix$filename for writing: $!\n"); } $fh = *OUT; } print $fh $out_head; print $fh $out_index_head if $out_index_head; print $fh $out_index_body if $out_index_body; print $fh $out_tail if $outtype_htmlframe; if ($opt_output_path and $outtype_htmlframe) { # print frameset open( FRAME, ">$path/${file_prefix}framehead.html" ) or die ("$0: can't open $path/${file_prefix}framehead.html for writing: $!\n"); $frame = *FRAME; print $frame $out_head; print $frame $out_index_head; print $frame ' '; close($frame); open( FRAME, ">$path/$file_prefix$filename" ) or die ("$0: can't open $path/$file_prefix$filename for writing: $!\n"); $frame = *FRAME; print $frame ' ' . ($host_name ? "$host_name " : '') . 'Proxy Report (' . $loginterval . ') ' . ( $opt_M ? " $opt_M\n" : '' ) . ' <body> <p> Your browser is not able to view frames! Please follow this <a href="' . $file_prefix . 'toc.html">link</a> to get noframes. </p> <hr /> ' . $HTMLCOPYRIGHT . ' </body> '; close($frame); # print table of content open( TOC, ">$path/${file_prefix}toc.html" ) or die ("$0: can't open $path/${file_prefix}toc.html for writing: $!\n"); my $toc = *TOC; print $toc $out_head; outtoc(); print $toc $out_toc; print $toc ' '; close($toc); } @reportlist = ($opt_S) ? ( split ( ",", $opt_S ) ) : (0 .. $#reports); foreach $index ( 'E', @reportlist ) { if ( $outtype_htmlframe ) { open( OUT, ">$path/${file_prefix}$index.html" ) or die ("$0: can't open $path/${file_prefix}$index.html for writing: $!\n"); $fh = *OUT; print $fh $out_head; } if ($outref{$index}) { if ( not defined( $out_body{$index} ) and $index ne 'E' ) { outstart($index); outline( $index, 'no matching requests' ); outstop($index); } print $fh $out_body{$index} if defined( $out_body{$index} ); } if ($outtype_htmlframe) { print $fh $out_tail; close($fh); } } print $fh $out_tail unless $outtype_htmlframe; close($fh); print "$test_string\n" if $test; if ( $opt_generate_index and $opt_output_path and $opt_output_file_prefix) { opendir DIR, $opt_output_path or die("$0: can't opendir $opt_output_path: $!\n"); $pattern = $opt_output_file_prefix . $filename; $pattern =~ s#%h#(\\S+)#; $pattern =~ s#%t#(\\d{14}-\\d{14})#; $pattern =~ s#%%#%#; @files = grep { m#^$pattern# && -f "$opt_output_path/$_" } readdir(DIR); closedir DIR; $filename = 'index.html' unless $opt_output_file; open (INDEX, ">$opt_output_path/$filename") or die ("$0: can't open $opt_output_path/$filename for writing: $!\n"); print INDEX ' Proxy Report Index ' . ( $opt_M ? " $opt_M\n" : '' ) . '

Proxy Report Index

'; foreach (@files) { m#^$pattern#; if ($1 =~ m#\d{14}-\d{14}#) { $timestamp = $1; $host = $2; } else { $host = $1; $timestamp = $2; } ($startdate, $enddate) = split ('-', $timestamp) if $timestamp; print INDEX " \n"; } print INDEX '
Hostname Startdate Enddate
$host $startdate $enddate
' . $HTMLCOPYRIGHT . ' '; close(INDEX); } sub kilomegagigatera { my ($value) = shift (@_); my ($length) = $outtype_unformatted ? 999 : shift (@_); if ( $value > 10**( $length + 8 ) or $opt_U eq 'T' ) { return sprintf( '%d%s', int( ( $value / 1024**4 ) + .5 ), $outtype_unformatted ? '' : 'T' ); } elsif ( $value > 10**( $length + 5 ) or $opt_U eq 'G' ) { return sprintf( '%d%s', int( ( $value / 1024**3 ) + .5 ), $outtype_unformatted ? '' : 'G' ); } elsif ( $value > 10**( $length + 2 ) or $opt_U eq 'M' ) { return sprintf( '%d%s', int( ( $value / 1024**2 ) + .5 ), $outtype_unformatted ? '' : 'M' ); } elsif ( $value > 10**($length) or $opt_U eq 'K' ) { return sprintf( '%d%s', int( ( $value / 1024 ) + .5 ), $outtype_unformatted ? '' : 'K' ); } else { return $value; } } sub removezerotimes { my ($size) = shift (@_); my ($time) = shift (@_); if ( $size == 0 or $time == 0 ) { return '-'; } else { return $size / ( 1.024 * $time ); } } sub getfqdn { my ($host) = @_; if ( $host =~ m#^([^@]+@)?(::ffff:)?(([0-9][0-9]{0,2}\.){3}[0-9][0-9]{0,2}$)#o ) { $hostcache{$3} = addtonam($3) unless defined $hostcache{$3}; return $1 . $hostcache{$3} if defined $1; return $hostcache{$3}; } else { return $host; } } sub addtonam { my ($address) = shift (@_); my (@octets); my ( $host_name, $aliases, $type, $len, $addr ); my ($ip_number); @octets = split '\.', $address; if ( $#octets != 3 ) { undef; } $ip = pack( "CCCC", @octets[ 0 .. 3 ] ); ( $host_name, $aliases, $type, $len, $addr ) = gethostbyaddr( $ip, 2 ); if ($host_name) { $host_name; } else { $address; } } sub convertdate { my $date = shift (@_); my $type = shift (@_); if ($date) { my ( $sec, $min, $hour, $mday, $mon, $year ) = ( localtime($date) )[ 0, 1, 2, 3, 4, 5, 6 ]; my $month = $months[$mon]; $year += 1900; my $retdate = $type ? sprintf( "%04d%02d%02d%02d%02d%02d\n", $year, $mon+1, $mday, $hour, $min, $sec ) : sprintf( "%02d.%s %02d %02d:%02d:%02d\n", $mday, $month, substr( $year, -2 ), $hour, $min, $sec ); chomp($retdate); return $retdate; } else { return ' '; } } sub outtoc { $out_toc .= " "; @reportlist = ($opt_S) ? ( split ( ",", $opt_S ) ) : (0 .. $#reports); foreach ( @reportlist ) { $out_toc .= " "; } $out_toc .= '

Table of Content

Overview
Warnings
$reports[$_]
'; } sub outref { my $name = shift (@_); $link = ( $outtype_htmlframe ) ? "${file_prefix}${name}.html" : "#${name}"; if (defined($max_value[$name])) { $out_index_body .= " $reports[$name] $max_value[$name] "; } else { $out_index_body .= " $reports[$name] - - no requests found "; } } sub outstart { my $index = shift (@_); if ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe ) { $out_body{$index} .= '
'; if ( $index eq 'E' ) { $out_body{$index} .= ' '; } else { $out_body{$index} .= '
'; } } else { $out_body{$index} .= "\n# $reports[$index]\n" unless ( $index eq 'E' ); } } sub outheader { my $index = shift (@_); my $print; my $no = 0; $out_body{$index} .= "\n " if ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe ); foreach (@_) { $p = $_; $p = 'kB/sec' if ($format[$no] eq 'kbps' and $p eq 'auto'); $p = 'sec/kB' if ($format[$no] eq 'spkb' and $p eq 'auto'); $p = 'req/sec' if ($format[$no] eq 'rps' and $p eq 'auto'); $p = 'sec/req' if ($format[$no] eq 'spr' and $p eq 'auto'); $p = 'Byte/sec' if ($format[$no] eq 'bps' and $p eq 'auto'); $p = 'sec/Byte' if ($format[$no] eq 'spb' and $p eq 'auto'); $p = 'req/msec' if ($format[$no] eq 'rpms' and $p eq 'auto'); $p = 'msec/req' if ($format[$no] eq 'mspr' and $p eq 'auto'); if ($format[$no] eq 'off') { # do nothing } elsif ($outtype_unformatted) { $out_body{$index} .= "$p "; } elsif ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe ) { $p =~ s# +# #go; $p =~ s#(^ | $)##go; $p = ' ' if ( $p eq '' ); $out_body{$index} .= " "; } elsif ( $format[$no] eq '%' ) { $out_body{$index} .= sprintf( ' ' x ( 6 - length($p) ) ) . substr( $p, 0, 6 ) . ' '; } elsif ( $format[$no] =~ m/[bps|spb|spkb|rps|rpms|spr]$/ ) { $out_body{$index} .= sprintf( substr( $p, 0, 7 ) . ' ' x ( 7 - length($p) ) . ' ' ); } else { $out_body{$index} .= sprintf( substr( $p, 0, $format[$no] ) . ' ' x ( $format[$no] - length($p) ) . ' ' ); } $no++; } $out_body{$index} .= "\n"; $out_body{$index} .= ' ' if ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe ); } sub outimg { my $index = shift; $out_body{$index} .= " "; } sub outgraph { my $index = shift; my $legend_ref = shift; my $max_x_data = shift; my $xaxis_ref = shift; my $yaxis1_ref = shift; my $yaxis2_ref = shift; my $yaxis3_ref = shift; my $yaxis4_ref = shift; my ($factor0, $factor1, $unit0, $unit1, $min_x, $max_x); die( "$0: Couldn't load package calamaris::calBars3d, maybe it is not installed: $!\n" ) unless (eval "require calamaris::calBars3d"); if ($max_x_data < 0) { # show last $max_x_data Values $min_x = ($#{@$xaxis_ref} + $max_x_data < 0) ? 0 : $#{@$xaxis_ref} + $max_x_data + 1; $max_x = $#{@$xaxis_ref}; } else { # show first $max_x_data Values $min_x = 0; $max_x = ($#{@$xaxis_ref} > $max_x_data) ? $max_x_data : $#{@$xaxis_ref}; } my $graph = calamaris::calBars3d->new($width, int($width/3*2)); # check image-type if (! $verified) { @img_format = $graph->export_format; foreach ( @img_format ) { $format{$_} = 1; } if ( ! defined($format{$opt_image_type}) ) { # Image type not supported print STDERR "$0: $opt_image_type is not supported by GD::Graph!\n Please use one of the supported image types: @img_format \n\n"; exit(1); } $verified = 1; # don't do it on every 'sub outgraph' call } # 1 axis graph my @data = ([@$xaxis_ref[$min_x..$max_x]], [@$yaxis1_ref[$min_x..$max_x]]); ($factor0, $unit0) = getfactor(max(@$yaxis1_ref[$min_x..$max_x]),9); $yaxis1_ref = reformatarray($factor0, $yaxis1_ref) if ($factor0 > 1); my %graph_label = (x_label => '', y_label => "$unit0 ${@$legend_ref}[0]", title => '', two_axes => '0', x_labels_vertical => '1', y_long_ticks => '1', y_tick_number => '5', x_ticks => '0', box_axis => '0', show_values => '0', values_vertical => '0', bar_spacing => '6', set_spacing => '0', shadowclr => 'lgray', shadow_depth => '0', boxclr => "$bg_color", fgclr => "$text_color", labelclr => "$text_color", axislabelclr => "$text_color", legendclr => "$text_color", valuesclr => "$text_color", textclr => "$text_color" ); $graph->set(%graph_label) or die $graph->error; # 2 axis graph if (scalar(@$legend_ref) > 1) { ($factor1, $unit1) = getfactor(max(@$yaxis2_ref[$min_x..$max_x]),5); $yaxis2_ref = reformatarray($factor1, $yaxis2_ref) if ($factor1 > 1); push @data, [@$yaxis2_ref[$min_x..$max_x]]; push @data, [@$yaxis3_ref[$min_x..$max_x]] if ref($yaxis3_ref); push @data, [@$yaxis4_ref[$min_x..$max_x]] if ref($yaxis4_ref); %graph_label = ( y1_label => "$unit0 ${@$legend_ref}[0]", y2_label => "$unit1 ${@$legend_ref}[1]", two_axes => '1', bar_spacing => '0', set_spacing => '6', ); } $graph->set(%graph_label) or die $graph->error; $graph->set_text_clr("$text_color"); $graph->set_legend(@$legend_ref); $graph->set( dclrs => [$column2_color, $column1_color] ); open(IMG, ">$path/${file_prefix}${index}.${opt_image_type}") or die ("$0: can't open $path/${file_prefix}${index}.${opt_image_type} for writing: $!\n"); binmode IMG; print IMG $graph->plot(\@data)->$opt_image_type; close(IMG); } sub maxi { my $max = -10e10; # an absolute small number my @array = @_; for ( $i = 0; $i <= $#array; $i++ ) { if ($array[$i] > $max ) { $max = $array[$i]; $maxi = $i; } } return $max, $maxi; } sub max { my $max = -10e10; # an absolute small number foreach (@_) { $max = $_ if $_ > $max } return $max; } sub reformatarray { my $factor = shift; my $array_ref = shift; my @array; foreach ( @$array_ref ) { push @array, $_/$factor } return \@array; } sub getfactor { my ($value) = shift (@_); my ($length) = $outtype_unformatted ? 999 : shift (@_); my ($factor, $unit); if ( $value > 10**( $length + 8 ) or $opt_U eq 'T' ) { $factor = 1024**4; $unit = 'Tera'; } elsif ( $value > 10**( $length + 5 ) or $opt_U eq 'G' ) { $factor = 1024**3; $unit = 'Giga'; } elsif ( $value > 10**( $length + 2 ) or $opt_U eq 'M' ) { $factor = 1024**2; $unit = 'Mega'; } elsif ( $value > 10**($length) or $opt_U eq 'K' ) { $factor = 1024; $unit = 'Kilo'; } else { $factor = 1; $unit = ''; } return $factor, $unit; } sub outline { my $index = shift (@_); my $linecolor = shift (@_); my $print; my $no = 0; $out_body{$index} .= "\n " if ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe ); $color = 0 unless $linecolor; if ($linecolor eq 'toggle') { $color = $color ? 0 : 1; } elsif ($linecolor) { $color = $linecolor; } foreach (@_) { $print = $_; if ($format[$no] eq 'off') { # do nothing } elsif ($outtype_unformatted) { $out_body{$index} .= "$print "; } elsif ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe ) { $print =~ s# +# #go; $print =~ s# $##go; $print =~ s#<#\<\;#go; $print =~ s#>#\>\;#go; if ( $no == 0 ) { unless ( $print =~ s#^ ##go ) { $out_body{$index} .= " "; } else { $out_body{$index} .= " "; } } elsif ( $format[$no] eq '%' ) { if ( $print eq '' or $print eq '-' ) { $out_body{$index} .= " "; } else { $out_body{$index} .= sprintf( ' ', $color, $print ); } } elsif ( $format[$no] =~ m/[bps|spb|spkb|rps|rpms|spr]$/ ) { my $factor = 1; $factor = 1000 if ($format[$no] =~ m#^rpms$# or $format[$no] =~ m#^mspr$#); $factor = 1024 if ($format[$no] =~ m#^bps$# or $format[$no] =~ m#^spb$#); if ( $print eq '' or $print eq '-' ) { $out_body{$index} .= " "; # kByte/sec } elsif ($format[$no] =~ m/bps/) { $out_body{$index} .= sprintf( ' ', $color, $print * $factor ); # sec/kByte } elsif ($format[$no] =~ m/spkb/ or $format[$no] =~ m/spb/) { $out_body{$index} .= sprintf( ' ', $color, 1 / ($factor * $print) ); # req/[m]sec } elsif ($format[$no] =~ m/rps/ or $format[$no] =~ m/rpms/) { $out_body{$index} .= sprintf( ' ', $color, 1 / ($print * $factor) ); # [m]sec/req } elsif ($format[$no] =~ m/spr/) { $out_body{$index} .= sprintf( ' ', $color, $print * $factor ); # % } else { $out_body{$index} .= sprintf( ' ', $color, $print ); } } elsif ( $no == 1 and $print !~ m#^[\d\.e\-\+]+$#o ) { $out_body{$index} .= sprintf( ' ', $color, $print ); } elsif ( $no == 1 or $print =~ m#^[\d\.e\-\+]+$#o ) { $out_body{$index} .= sprintf( ' ', $color, $print ); } else { if ($print) { $out_body{$index} .= " "; } else { $out_body{$index} .= " "; } } } else { if ( $no == 0 ) { if ( length($print) > $format[$no] ) { $out_body{$index} .= sprintf( $print . "\n" . ' ' x $format[$no] . ' ' ); } else { $out_body{$index} .= sprintf( $print . ' ' x ( $format[$no] - length($print) ) . ' ' ); } } elsif ( $format[$no] eq '%' ) { if ( $print eq '' or $print eq '-' ) { $out_body{$index} .= ' ' x 7; } else { $out_body{$index} .= sprintf( '%6.2f ', $print ); } } elsif ( $format[$no] =~ m/[bps|spb|spkb|rps|rpms|spr]$/ ) { my $factor = 1; $factor = 1000 if ($format[$no] =~ m#^rpms$# or $format[$no] =~ m#^mspr$#); $factor = 1024 if ($format[$no] =~ m#^bps$# or $format[$no] =~ m#^spb$#); if ( $print eq '-' ) { $out_body{$index} .= ' - '; # [k]Byte/sec } elsif ($format[$no] =~ m/bps/) { if ( $print >= 10000 ) { $out_body{$index} .= sprintf( '%7.0f ', $factor * $print ); } else { $out_body{$index} .= sprintf( '%7.2f ', $factor * $print ); } # sec/[k]Byte } elsif ($format[$no] =~ m/spkb/ or $format[$no] =~ m/spb/) { if ( $print >= 1000 ) { $out_body{$index} .= sprintf( '%7.0f ', 1 / ($factor * $print) ); } else { $out_body{$index} .= sprintf( '%7.3f ', 1 / ($factor * $print) ); } # req/[m]sec } elsif ($format[$no] =~ m/rps/ or $format[$no] =~ m/rpms/) { if ( $print >= 10000 ) { $out_body{$index} .= sprintf( '%7.0f ', 1 / ($factor * $print) ); } else { $out_body{$index} .= sprintf( '%7.2f ', 1 / ($factor * $print) ); } # [m]sec/req } elsif ($format[$no] =~ m/spr/) { if ( $print >= 10000 ) { $out_body{$index} .= sprintf( '%7.0f ', $factor * $print ); } else { $out_body{$index} .= sprintf( '%7.2f ', $factor * $print ); } } } else { $print = sprintf( '%d', $print + .5 ) if $print =~ m#^[\d\.e\-\+]+$#o; $out_body{$index} .= sprintf( ' ' x ( $format[$no] - length($print) ) ) . substr( $print, 0, $format[$no] ) . ' '; } } $no++; } $out_body{$index} .= "\n"; $out_body{$index} .= ' ' if ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe); } sub outseperator { my $index = shift (@_); my $print; $out_body{$index} .= "\n " if ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe ); foreach $print (@format) { next if $print eq 'off'; if ($outtype_unformatted) { $out_body{$index} .= "--- "; } elsif ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe ) { $out_body{$index} .= ' '; } elsif ( $print eq '%' ) { $out_body{$index} .= sprintf( '-' x 6 . ' ' ); } elsif ( $print =~ m/[bps|spb|spkb|rps|rpms|spr]$/ ) { $out_body{$index} .= sprintf( '-' x 7 . ' ' ); } else { $out_body{$index} .= sprintf( '-' x $print . ' ' ); } } $out_body{$index} .= "\n"; $out_body{$index} .= ' ' if ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe ); } sub outstop { my $index = shift (@_); my $link = ($outtype_htmlframe) ? "$file_prefix$filename" : '#top'; if ( $outtype_html or $outtype_htmlembed or $outtype_htmlframe ) { $out_body{$index} .= '
' . $reports[$index] . '
$p
\"Graphic:
$print   $print   %.2f   %.2f %.2f %.2f %.2f %.2f %s %d $print  
'; $out_body{$index} .= '

Back to Top

' unless $outtype_htmlframe; } else { $out_body{$index} .= "\n"; } } sub writecache { print CACHE join ( 'µ', @_ ) . "\n" if $opt_o; } sub geterrcode { my %err_code; $err_code{'000'} = 'Used mostly with UDP traffic'; $err_code{'100'} = 'Continue'; $err_code{'101'} = 'Switching Protocols'; $err_code{'102'} = 'Processing'; $err_code{'200'} = 'OK'; $err_code{'201'} = 'Created'; $err_code{'202'} = 'Accepted'; $err_code{'203'} = 'Non-Authoritative Information'; $err_code{'204'} = 'No Content'; $err_code{'205'} = 'Reset Content'; $err_code{'206'} = 'Partial Content'; $err_code{'207'} = 'Multi Status'; $err_code{'300'} = 'Multiple Choices'; $err_code{'301'} = 'Moved Permanently'; $err_code{'302'} = 'Moved Temporarily'; $err_code{'303'} = 'See Other'; $err_code{'304'} = 'Not Modified'; $err_code{'305'} = 'Use Proxy'; $err_code{'307'} = 'Temporary Redirect'; $err_code{'400'} = 'Bad Request'; $err_code{'401'} = 'Unauthorized'; $err_code{'402'} = 'Payment Required'; $err_code{'403'} = 'Forbidden'; $err_code{'404'} = 'Not Found'; $err_code{'405'} = 'Method Not Allowed'; $err_code{'406'} = 'Not Acceptable'; $err_code{'407'} = 'Proxy Authentication Required'; $err_code{'408'} = 'Request Timeout'; $err_code{'409'} = 'Conflict'; $err_code{'410'} = 'Gone'; $err_code{'411'} = 'Length Required'; $err_code{'412'} = 'Precondition Failed'; $err_code{'413'} = 'Request Entity Too Large'; $err_code{'414'} = 'Request URI Too Large'; $err_code{'415'} = 'Unsupported Media '; $err_code{'416'} = 'Request Range Not Satisfiable'; $err_code{'417'} = 'Expectation Failed'; $err_code{'424'} = 'Locked'; $err_code{'424'} = 'Failed Dependency'; $err_code{'433'} = 'Unprocessable Entity'; $err_code{'500'} = 'Internal Server Error'; $err_code{'501'} = 'Not Implemented'; $err_code{'502'} = 'Bad Gateway'; $err_code{'503'} = 'Service Unavailable'; $err_code{'504'} = 'Gateway Timeout'; $err_code{'505'} = 'HTTP Version Not Supported'; $err_code{'507'} = 'Insufficient Storage'; $err_code{'600'} = 'Squid header parsing error'; return %err_code; } sub readconfig { # Default values undef($benchmark); undef($cache_input_file); undef($cache_output_file); undef($domain_report); undef($domain_report_limit); undef($domain_report_n_level); undef($errorcode_distribution_report); undef($generate_index); undef($hostname); undef($input_format); undef($ipfilter_exclude); undef($ipfilter_include); undef($logo); undef($meta); undef($no_input); undef($object_freshness_report); undef($output_file); undef($output_file_prefix); undef($output_format); undef($output_path); undef($peak_report); undef($performance_report); undef($performance_report_adjust); undef($requester_report); undef($requester_report_no_dns_lookup); undef($requester_report_use_user_info); undef($requester_report_with_targets); undef($response_time_report); undef($show_reports); undef($size_distribution_report); undef($sort_order); undef($status_report); undef($time_interval); undef($type_report); undef($type_report_ignore_case); undef($unit); undef($verbose); @response_time_report_interval = qw( 0.001 0.01 0.02 0.05 0.1 0.2 0.5 1 2 5 10 20 50 100 200 500 1000 2000 5000 10000 20000 50000 100000 200000 500000 1000000 1e10 ); $response_time_limit = $response_time_report_interval[$#response_time_report_interval]; # show $show_other_tcp_urlhost = 1; $show_other_tcp_urltld = 1; $show_other_tcp_content = 1; $show_other_tcp_urlext = 1; $show_other_udp_requester = 1; $show_other_tcp_requester = 1; # GRAPH SECTION $column1_color = '#6699cc'; $column2_color = '#ff9900'; $text_color = '#222266'; $bg_color = '#ffffcc'; $x_scale = 30; $image_type = 'png'; $width = 600; $test = 0; $verified = 0; @graph_legend = ('Requests', 'Byte', 'Request Hit Rate', 'Byte Hit Rate'); # HTML SECTION if ($opt_config_file) { my $file = $opt_config_file; my $return; unless ($return = do $file) { warn "$0: Parsing of $file failed: $@" if $@; warn "$0: do on $file failed: $!" unless defined $return; warn "$0: couldn't execute $file" unless $return; } } $opt_b = $benchmark unless $opt_b; $opt_i = $cache_input_file unless $opt_i; $opt_o = $cache_output_file unless $opt_o; $opt_d = $domain_report unless $opt_d; $opt_domain_report_limit = $domain_report_limit unless $opt_domain_report_limit; $opt_N = $domain_report_n_level unless $opt_N; $opt_errorcode_distribution = $errorcode_distribution_report unless $opt_errorcode_distribution; $opt_H = $hostname unless $opt_H; $opt_c = $type_report_ignore_case unless $opt_c; $opt_f = $input_format unless $opt_f; $opt_image_type = $image_type unless $opt_image_type; $opt_generate_index = $generate_index unless $opt_generate_index; $opt_ipfilter_exclude = $ipfilter_exclude unless $opt_ipfilter_exclude; $opt_ipfilter_include = $ipfilter_include unless $opt_ipfilter_include; $opt_l = $logo unless $opt_l; $opt_M = $meta unless $opt_M; $opt_z = $no_input unless $opt_z; $opt_output_file = $output_file unless $opt_output_file; $opt_output_file_prefix = $output_file_prefix unless $opt_output_file_prefix; $opt_F = $output_format unless $opt_F; $opt_output_path = $output_path unless $opt_output_path; $opt_p = $peak_report unless $opt_p; $opt_P = $performance_report unless $opt_P; $opt_T = $performance_report_adjust unless $opt_T; $opt_r = $requester_report unless $opt_r; $opt_n = $requester_report_no_dns_lookup unless $opt_n; $opt_u = $requester_report_use_user_info unless $opt_u; $opt_R = $requester_report_with_targets unless $opt_R; $opt_response_time = $response_time_report unless $opt_response_time; undef($opt_response_time) unless scalar(@response_time_report_interval); $opt_S = $show_reports unless $opt_S; $opt_D = $size_distribution_report unless $opt_D; $opt_O = $sort_order unless $opt_O; $opt_s = $status_report unless $opt_s; $opt_I = $time_interval unless $opt_I; $opt_t = $type_report unless $opt_t; $opt_U = $unit unless $opt_U; $opt_v = $verbose unless $opt_v; # -M: if it's a file, read it and store it in $opt_M if ( defined($opt_M) and $opt_M !~ /\n/ and -e $opt_M ) { open(IN, "<$opt_M") or die "$0: Couldn't open file $opt_M for reading: $!\n"; $opt_M = ''; while () { $opt_M .= $_ } } # -l: if it's a file, read it and store it in $opt_l if ( defined($opt_l) and $opt_l !~ /\n/ and -e $opt_l ) { open(IN, "<$opt_l") or die "$0: Couldn't open file $opt_l for reading: $!\n"; $opt_l = ''; while () { $opt_l .= $_ } } if ( $opt_b and $opt_b < 1 ) { print STDERR "$0: wrong value at -b -option: \"$opt_b\"\n\n"; $usage_err = 1; } else { $| = 1; } if ($opt_U) { unless ( $opt_U =~ m#^[KMGT]$# ) { print STDERR "$0: wrong value at -U -option: \"$opt_U\"\n\n"; $usage_err = 1; } } else { $opt_U = ''; } if ($opt_D) { if ( $opt_D <= 1 ) { print STDERR "$0: wrong value at -D -option: \"$opt_D\"\n\n"; $usage_err = 1; } } if ($opt_H) { if ( $opt_H eq '1' or $opt_H eq 'lookup' ) { $host_name = hostname(); } else { $host_name = $opt_H; } } else { $host_name = ''; } if ( $opt_N and $opt_N != -1 and $opt_N < 2 ) { print STDERR "$0: wrong value at -N -option: \"$opt_N\"\n\n"; $usage_err = 1; } if ($opt_I) { use Time::Local; ( $interval_begin, $interval_end ) = split ( '-', $opt_I ); if ( $interval_begin =~ m#^(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})$# ) { $interval_begin = timelocal( $6, $5, $4, $3, $2 - 1, $1 - 1900 ); } elsif ( $interval_begin eq '' ) { $interval_begin = 0; } else { print STDERR "$0: wrong value at -I -option: \"$opt_I\"\n\n"; $usage_err = 1; } if ( $interval_end =~ m#^(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})$# ) { $interval_end = timelocal( $6, $5, $4, $3, $2 - 1, $1 - 1900 ); } elsif ( $interval_end eq '' ) { $interval_end = 9999999999; } else { print STDERR "$0: wrong value at -I -option: \"$opt_I\"\n\n"; $usage_err = 1; } if ($interval_begin > $interval_end) { print STDERR "$0: wrong value at -I -option: \"$opt_I\". Interval begin newer than interval end!\n\n"; $usage_err = 1; } } $path = $opt_output_path ? $opt_output_path : '.'; $filename = $opt_output_file ? $opt_output_file : 'calamaris.txt'; $file_prefix = $opt_output_file_prefix ? $opt_output_file_prefix : ''; if ($opt_F) { $outtype_htmlembed = $outtype_htmlframe = $outtype_html = 0; foreach $output ( split ( /\s*,\s*/, $opt_F ) ) { if ($output eq 'mail') { $outtype_mail = 1; }elsif ($output eq 'html') { $outtype_html = 1; } elsif ($output eq 'html-embed') { $outtype_htmlembed = 1; } elsif ($output eq 'html-frame') { $outtype_htmlframe = 1; } elsif ( $output eq 'graph' ) { $outtype_graph = 1; } elsif ( $output eq 'unformatted' ) { $outtype_unformatted = 1; } else { print STDERR "$0: unknown output-format: $output\n\n"; $usage_err = 1; } } if ( ( $outtype_htmlembed + $outtype_htmlframe + $outtype_html ) > 1 ) { print STDERR "$0: only one of 'html', 'html-embed' or 'html-frame' can be used: $opt_F\n\n"; $usage_err = 1; } if ( $outtype_mail and ($outtype_htmlembed or $outtype_htmlframe or $outtype_graph) ) { print STDERR "$0: 'mail' together with 'html-embed', 'html-frame' or 'graph' won't work: $opt_F\n\n"; $usage_err = 1; } print STDERR "$0: please use --output-path /path, when using -F 'graph'\n\n" if ( $outtype_graph and not $opt_output_path ); if ($outtype_graph and not ($outtype_html or $outtype_htmlembed or $outtype_htmlframe) ) { print STDERR "$0: -F 'graph' is kinda useless without 'html', 'html-embed' or 'html-frame'.\n\n"; } } if ( $opt_generate_index and not $opt_output_path and not $opt_output_file_prefix) { print STDERR "$0: --generate-index needs --output-path and --output-file-prefix\n"; } if ($outtype_htmlembed or $outtype_html or $outtype_htmlframe) { $filename = $opt_output_file ? $opt_output_file : 'index.html'; } $sortorder = $opt_O ? '_size' : ''; if ($opt_a) { $opt_response_time = 1; $opt_errorcode_distribution = 1; $opt_D = 10 unless $opt_D; $opt_P = 60 unless $opt_P; $opt_d = 20 unless $opt_d; $opt_r = 20 unless $opt_r; $opt_s = 1; $opt_t = 20 unless $opt_t; } $opt_domain_report_limit = 0 unless $opt_domain_report_limit; $opt_N = 2 unless $opt_N; $opt_T = 0 unless $opt_T; $opt_r = $opt_R unless $opt_r; if ($object_freshness_report) { $opt_t = 20 unless $opt_t; } $P = $opt_P ? "$opt_P minute" : '60 minute'; $P = $opt_P/60 . ' hour' if ( defined($opt_P) and ($opt_P % 60) == 0 and $opt_P >= 60 ); $P = $opt_P/1440 . ' day' if ( defined($opt_P) and ($opt_P % 1440) == 0 and $opt_P >= 1440 ); if ( $opt_N == -1 or $opt_N > 2 ) { if ( $opt_N == 3 ) { $N = '3rd'; } elsif ( $opt_N == -1 ) { $N = 'all'; } else { $N = $opt_N . 'th'; } } else { $N = '2nd'; } $outref{E} = 1; $outref{0} = 1; $outref{1} = ($opt_p) ? 1 : 0; $outref{2} = ($opt_p and $opt_p eq 'new') ? 1 : 0; $outref{3} = 1; $outref{4} = 1; $outref{5} = 1; $outref{6} = 1; $outref{7} = 1; $outref{8} = ($opt_d) ? 1 : 0; $outref{9} = ($opt_d) ? 1 : 0; $outref{10} = ($opt_t) ? 1 : 0; $outref{11} = ($opt_t) ? 1 : 0; $outref{12} = ($opt_t) ? 1 : 0; $outref{13} = ($opt_r) ? 1 : 0; $outref{14} = ($opt_r) ? 1 : 0; $outref{15} = ($opt_D) ? 1 : 0; $outref{16} = ($opt_P) ? 1 : 0; $outref{17} = ($opt_response_time) ? 1 : 0; $outref{18} = ($opt_response_time) ? 1 : 0; $outref{19} = ($opt_errorcode_distribution) ? 1 : 0; $outref{20} = ($opt_errorcode_distribution) ? 1 : 0; @reports = ( 'Summary', 'Incoming request peak per protocol', 'Incoming transfer volume peak per protocol', 'Incoming requests by method', 'Incoming UDP-requests by status', 'Incoming TCP-requests by status', 'Outgoing requests by status', 'Outgoing requests by destination', "Request-destinations by ${N}-level-domain", 'Request-destinations by toplevel-domain', 'TCP-Request-protocol', 'Requested content-type', 'Requested extensions', 'Incoming UDP-requests by host', 'Incoming TCP-requests by host', 'Size Distribution Diagram', "Performance in $P steps", 'UDP-Request duration distribution in msec', 'TCP-Request duration distribution in msec', 'UDP Response code distribution', 'TCP Response code distribution' ); $LICENSE = 'Calamaris comes with ABSOLUTELY NO WARRANTY. It is free software, and you are welcome to redistribute it under certain conditions. See source for details.'; $COPYRIGHT = 'Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Cord Beermann. Authors: Cord Beermann and Michael Pophal.'; $HOMEPAGE = 'http://Calamaris.Cord.de/'; $HTMLCOPYRIGHT = '
Calamaris ' . $VERSION . ', Copyright © 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Cord Beermann. Authors: Cord Beermann and Michael Pophal.

' . $LICENSE . '

'; $USAGE = ' Usage: cat log | ' . $0 . ' --config-file /path/to/calamaris.conf [switches] or cat log | ' . $0 . ' [switches] --config-file file Not all reports and modification can be made through command-line-switches. To use all options of Calamaris you\'ll have to use the configuration file. see the manpage for the configuration-file syntax. Reports: --all-useful-reports|-a extracts all useful reports available, --all-useful-reports equals --size-distribution-report 10 \ --domain-report 20 \ --performance-report 60 \ --requester-report 20 \ --status-report \ --type-report 20 \ --response-time-report \ --errorcode-distribution-report --domain-report|-d n show n Top-level and n second-level destinations, -1 = unlimited --domain-report-limit n limit display of domains to those with n requests or more. --domain-report-n-level|-N n change all 2nd-level-reports to n-level-reports. n can be any number from 2 up. -1 means full report. --errorcode-distribution-report shows the Response code distribution over all objects --peak-report|-p type measure peak requests old = make old request-peak mesurement new = make new request&byte-peak measurement (both slow Calamaris significantly down.) --performance-report|-P n show throughput data for every n minutes --performance-report-adjust|-T n adjust the Performance-Report in minutes --requester-report|-r n show n Requesters, -1 = unlimited --requester-report-no-dns-lookup|-n don\'t look IP-Numbers up --requester-report-use-user-info|-u use ident information if available (*) --requester-report-with-targets|-R n show n targets for every Requester, -1 = unlimited), implies --requester-report (*) --response-time-report shows the time distribution over all objects --size-distribution-report|-D n shows size-based distribution of requested objects, smaller numbers result in more verbose reports. choose 2, 10 or 100 for useful output. (You can also play with this ;-)) --status-report|-s show verbose status reports --type-report|-t n show n content-type, n extensions and requested protocols, -1 = unlimited --type-report-ignore-case|-c switch to case-insensitive (useful for extensions-report) More reports and report-modifications are available via the configuration-file. see manpage. Input: --input-format|-f type sets the type of input logfiles auto = tries to guess the input format (This is the Default) squid = Native-Logfile derived from Squid V1.1.beta26-V2.x squid-extended = Native-Logfile with log_mime_hdrs enabled derived from Squid V1.1.beta26-V2.x (*) or Cisco Content Engines (*) or Squid with SmartFilter-patch (*) squid-old = Native-Logfile derived from Squid V1.1.alpha1-V1.1.beta25 nc = Squid-style Logfiles derived from NetCache V?? (<5.x) its = Logfiles derived from Inktomi Traffic Server elff = Extended Logfile Format (i.e Compaq Tasksmart, Novell Internet Caching System, NetCache 5.x, BlueCoat) nse = Netscape Extended-1/2 Logfile Format --ipfilter-exclude IP/range all IPs are analyzed, except IP/range. (*) Format: 1.1.1.1/32:1.1.2.0/24 1.1.1.1/255.255.255.255:1.1.2.0/255.255.255.0 IP list separated by \':\' This switch needs the perl Module NetAddr::IP. --ipfilter-include IP/range no IPs are analyzed, except IP/range. (*) Format: see --ipfilter-exclude --no-input|-z no input via stdin --time-interval|-I t-t defines which time-interval should be parsed. t has to be the format yyyymmddhhmmss (localtime). omitting the beginning or ending is allowed. Output: (Default is plain formatted text) --hostname|-H name a name for the Output, -H \'lookup\' issues a lookup for the current host --image-type Sets the image type to gif, png, jpeg, gd or gd2. Only usefull when --output-format graph is set. The supported images types are dependend on your GD::Graph installation. --logo|-l string add this string to the head of the report. works only in combination with --output-format html or html-frame --meta|-M string/file includes the given strings in html-. If a file is given, the file is included in html-. works only in combination with --output-format html or html-frame --output-format|-F type output format (comma-seperated list) mail = mail format, disables graph output. html = HTML format html-frame = HTML frames, disables mail output. html-embed = HTML format without HTML-headers. graph = enable graphics, needs GD::Graph, only useful with html or html-embed, see --output-path. Disables mail output. unformatted = plain unformatted output --output-path /path --output-file-prefix string --output-file filename output calamaris statistics to /path. In case of graph output, the graphics destination is /path and the filename is index.html, else it is calamaris.txt. If --output-path is not given, all graphics are written to the working directory. %h will be expanded to the hostname, %t to a timerange, %% to a single %. --generate-index Generates an index-file of all matching reports if output-path and output-file-prefix is set --show-reports|-S list Shows only the defined reports (comma-seperated list) in the specified order. The following numbers are defined: '; foreach ( 0 .. $#reports ) { $USAGE .= "\t $_\t$reports[$_]\n"; } $USAGE .= ' Note: only putting out one report does not speed up Calamaris as the internal operations were done based on the report-switches. Default: Reports are displayed based on activated reports. --sort-order|-O changes the sort order in the reports to request size, default is sorting by number of requests --unit|-U string define the Unit for the Byte-values, else it will be auto. K(ilo), M(ega), G(iga), T(era) There are more options to modify the output of Calamaris. Please see the man-page. Caching: --cache-input-file|-i file input-datafile for caching, to add many files separate them with a \':\') --cache-output-file|-o file output-datafile for caching, can be the same as --cache-input-file Misc: --benchmark|-b n prints a hash-sign (#) to stderr for each n lines processed --copyright|-C prints the copyright --help|-h prints out this message --version|-V prints version-info Debug: --verbose|-v print information what Calamaris is doing. Useful for debugging. --dump-loop|-L dumps the generated internal loop to STDERR for debugging. (*) These options break the privacy of your users. Please read the README on this.'; } sub readcache { ### Read Cache. foreach $file ( split ':', $opt_i ) { open( CACHE, "$file" ) or die ("$0: can't open $file for reading: $!\n"); while () { chomp; next if m#^$#; @cache = split 'µ'; $x = shift (@cache); next unless ($x ne ''); if ( $x eq '0' and $#cache == 39 ) { $time_begin = $cache[0] if $cache[0] < $time_begin; $time_end = $cache[1] if $cache[1] > $time_end; $counter += $cache[2]; $size += $cache[3]; $time += $cache[4]; $invalid += $cache[5]; $time_run += $cache[6]; $udp += $cache[7]; $udp_size += $cache[8]; $udp_time += $cache[9]; $udp_hit += $cache[10]; $udp_hit_size += $cache[11]; $udp_hit_time += $cache[12]; $udp_miss += $cache[13]; $udp_miss_size += $cache[14]; $udp_miss_time += $cache[15]; $tcp += $cache[16]; $tcp_size += $cache[17]; $tcp_time += $cache[18]; $tcp_hit += $cache[19]; $tcp_hit_size += $cache[20]; $tcp_hit_time += $cache[21]; $tcp_miss += $cache[22]; $tcp_miss_size += $cache[23]; $tcp_miss_time += $cache[24]; $tcp_miss_none += $cache[25]; $tcp_miss_none_size += $cache[26]; $tcp_miss_none_time += $cache[27]; $hier += $cache[28]; $hier_size += $cache[29]; $hier_time += $cache[30]; $hier_direct += $cache[31]; $hier_direct_size += $cache[32]; $hier_direct_time += $cache[33]; $hier_sibling += $cache[34]; $hier_sibling_size += $cache[35]; $hier_sibling_time += $cache[36]; $hier_parent += $cache[37]; $hier_parent_size += $cache[38]; $hier_parent_time += $cache[39]; } elsif ( $x eq '1' and $#cache == 17 ) { unless ( $peak_udp_sec == 0 ) { warn("multiple cache files.\n") if $opt_v; $peak_warn = 'Peak values are possibly wrong!'; } if ( $peak_udp_sec < $cache[0] ) { $peak_udp_sec = $cache[0]; $peak_udp_sec_time = $cache[1]; } if ( $peak_udp_min < $cache[2] ) { $peak_udp_min = $cache[2]; $peak_udp_min_time = $cache[3]; } $peak_udp_hour{ $cache[5] } = 0 unless defined $peak_udp_hour{ $cache[5] }; $peak_udp_hour{ $cache[5] } += $cache[4]; if ( $peak_tcp_sec < $cache[6] ) { $peak_tcp_sec = $cache[6]; $peak_tcp_sec_time = $cache[7]; } if ( $peak_tcp_min < $cache[8] ) { $peak_tcp_min = $cache[8]; $peak_tcp_min_time = $cache[9]; } $peak_tcp_hour{ $cache[11] } = 0 unless defined $peak_tcp_hour{ $cache[11] }; $peak_tcp_hour{ $cache[11] } += $cache[10]; if ( $peak_all_sec < $cache[12] ) { $peak_all_sec = $cache[12]; $peak_all_sec_time = $cache[13]; } if ( $peak_all_min < $cache[14] ) { $peak_all_min = $cache[14]; $peak_all_min_time = $cache[15]; } $peak_all_hour{ $cache[17] } = 0 unless defined $peak_all_hour{ $cache[17] }; $peak_all_hour{ $cache[17] } += $cache[16]; } elsif ( $x eq '1' and $#cache == 23 ) { unless ( $peak_udp_sec == 0 ) { warn("multiple cache files.\n") if $opt_v; $peak_warn = 'Peak values are possibly wrong!'; } if ( $peak_udp_sec < $cache[0] ) { $peak_udp_sec = $cache[0]; $peak_udp_sec_time = $cache[1]; } if ( $peak_udp_min < $cache[2] ) { $peak_udp_min = $cache[2]; $peak_udp_min_time = $cache[3]; } $peak_udp_hour{ $cache[5] } = 0 unless defined $peak_udp_hour{ $cache[5] }; $peak_udp_hour{ $cache[5] } += $cache[4]; $peak_udp_hour_size{ $cache[7] } = 0 unless defined $peak_udp_hour_size{ $cache[7] }; $peak_udp_hour_size{ $cache[7] } += $cache[6]; if ( $peak_tcp_sec < $cache[8] ) { $peak_tcp_sec = $cache[8]; $peak_tcp_sec_time = $cache[9]; } if ( $peak_tcp_min < $cache[10] ) { $peak_tcp_min = $cache[10]; $peak_tcp_min_time = $cache[11]; } $peak_tcp_hour{ $cache[13] } = 0 unless defined $peak_tcp_hour{ $cache[13] }; $peak_tcp_hour{ $cache[13] } += $cache[12]; $peak_tcp_hour_size{ $cache[15] } = 0 unless defined $peak_tcp_hour_size{ $cache[15] }; $peak_tcp_hour_size{ $cache[15] } += $cache[14]; if ( $peak_all_sec < $cache[16] ) { $peak_all_sec = $cache[16]; $peak_all_sec_time = $cache[17]; } if ( $peak_all_min < $cache[18] ) { $peak_all_min = $cache[18]; $peak_all_min_time = $cache[19]; } $peak_all_hour{ $cache[21] } = 0 unless defined $peak_all_hour{ $cache[21] }; $peak_all_hour{ $cache[21] } += $cache[20]; $peak_all_hour_size{ $cache[23] } = 0 unless defined $peak_all_hour_size{ $cache[23] }; $peak_all_hour_size{ $cache[23] } += $cache[22]; } elsif ( $x eq '3' and $#cache == 3 ) { $y = shift (@cache); $method{$y} = $method_size{$y} = $method_time{$y} = 0 unless defined $method{$y}; $method{$y} += $cache[0]; $method_size{$y} += $cache[1]; $method_time{$y} += $cache[2]; } elsif ( $x eq '4.1' and $#cache == 3 ) { $y = shift (@cache); $udp_hit{$y} = $udp_hit_size{$y} = $udp_hit_time{$y} = 0 unless defined $udp_hit{$y}; $udp_hit{$y} += $cache[0]; $udp_hit_size{$y} += $cache[1]; $udp_hit_time{$y} += $cache[2]; } elsif ( $x eq '4.2' and $#cache == 3 ) { $y = shift (@cache); $udp_miss{$y} = $udp_miss_size{$y} = $udp_miss_time{$y} = 0 unless defined $udp_miss{$y}; $udp_miss{$y} += $cache[0]; $udp_miss_size{$y} += $cache[1]; $udp_miss_time{$y} += $cache[2]; } elsif ( $x eq '5.1' and $#cache == 3 ) { $y = shift (@cache); $tcp_hit{$y} = $tcp_hit_size{$y} = $tcp_hit_time{$y} = 0 unless defined $tcp_hit{$y}; $tcp_hit{$y} += $cache[0]; $tcp_hit_size{$y} += $cache[1]; $tcp_hit_time{$y} += $cache[2]; } elsif ( $x eq '5.2' and $#cache == 3 ) { $y = shift (@cache); $tcp_miss{$y} = $tcp_miss_size{$y} = $tcp_miss_time{$y} = 0 unless defined $tcp_miss{$y}; $tcp_miss{$y} += $cache[0]; $tcp_miss_size{$y} += $cache[1]; $tcp_miss_time{$y} += $cache[2]; } elsif ( $x eq '5.3' and $#cache == 3 ) { $y = shift (@cache); $tcp_miss_none{$y} = $tcp_miss_none_size{$y} = $tcp_miss_none_time{$y} = 0 unless defined $tcp_miss_none{$y}; $tcp_miss_none{$y} += $cache[0]; $tcp_miss_none_size{$y} += $cache[1]; $tcp_miss_none_time{$y} += $cache[2]; } elsif ( $x eq '6.1' and $#cache == 3 ) { $y = shift (@cache); $hier_direct{$y} = $hier_direct_size{$y} = $hier_direct_time{$y} = 0 unless defined $hier_direct{$y}; $hier_direct{$y} += $cache[0]; $hier_direct_size{$y} += $cache[1]; $hier_direct_time{$y} += $cache[2]; } elsif ( $x eq '6.2' and $#cache == 3 ) { $y = shift (@cache); $hier_sibling{$y} = $hier_sibling_size{$y} = $hier_sibling_time{$y} = 0 unless defined $hier_sibling{$y}; $hier_sibling{$y} += $cache[0]; $hier_sibling_size{$y} += $cache[1]; $hier_sibling_time{$y} += $cache[2]; } elsif ( $x eq '6.3' and $#cache == 3 ) { $y = shift (@cache); $hier_parent{$y} = $hier_parent_size{$y} = $hier_parent_time{$y} = 0 unless defined $hier_parent{$y}; $hier_parent{$y} += $cache[0]; $hier_parent_size{$y} += $cache[1]; $hier_parent_time{$y} += $cache[2]; } elsif ( $x eq '7.1' and $#cache == 3 ) { $y = shift (@cache); $hier_neighbor{$y} = $hier_neighbor_size{$y} = $hier_neighbor_time{$y} = 0 unless defined $hier_neighbor{$y}; $hier_neighbor{$y} += $cache[0]; $hier_neighbor_size{$y} += $cache[1]; $hier_neighbor_time{$y} += $cache[2]; } elsif ( $x eq '7.2' and $#cache == 4 ) { $y = shift (@cache); $z = shift (@cache); $hier_neighbor_status{$y}{$z} = $hier_neighbor_status_size{$y}{$z} = $hier_neighbor_status_time{$y}{$z} = 0 unless defined $hier_neighbor_status{$y}{$z}; $hier_neighbor_status{$y}{$z} += $cache[0]; $hier_neighbor_status_size{$y}{$z} += $cache[1]; $hier_neighbor_status_time{$y}{$z} += $cache[2]; } elsif ( $x eq '8' and $#cache == 5 ) { $y = shift (@cache); $tcp_urlhost{$y} = $tcp_urlhost_size{$y} = $tcp_urlhost_time{$y} = $tcp_hit_urlhost{$y} = $tcp_hit_urlhost_size{$y} = 0 unless defined $tcp_urlhost{$y}; $show_other_tcp_urlhost = 0 if defined($tcp_urlhost{''}); $tcp_urlhost{$y} += $cache[0]; $tcp_urlhost_size{$y} += $cache[1]; $tcp_hit_urlhost{$y} += $cache[2]; $tcp_hit_urlhost_size{$y} += $cache[3]; $tcp_urlhost_time{$y} += $cache[4]; } elsif ( $x eq '9' and $#cache == 5 ) { $y = shift (@cache); $tcp_urltld{$y} = $tcp_urltld_size{$y} = $tcp_urltld_time{$y} = $tcp_hit_urltld{$y} = $tcp_hit_urltld_size{$y} = 0 unless defined $tcp_urltld{$y}; $show_other_tcp_urltld = 0 if defined($tcp_urltld{''}); $tcp_urltld{$y} += $cache[0]; $tcp_urltld_size{$y} += $cache[1]; $tcp_hit_urltld{$y} += $cache[2]; $tcp_hit_urltld_size{$y} += $cache[3]; $tcp_urltld_time{$y} += $cache[4]; } elsif ( $x eq '10' and $#cache == 5 ) { $y = shift (@cache); $tcp_urlprot{$y} = $tcp_urlprot_size{$y} = $tcp_urlprot_time{$y} = $tcp_hit_urlprot{$y} = $tcp_hit_urlprot_size{$y} = 0 unless defined $tcp_urlprot{$y}; $tcp_urlprot{$y} += $cache[0]; $tcp_urlprot_size{$y} += $cache[1]; $tcp_hit_urlprot{$y} += $cache[2]; $tcp_hit_urlprot_size{$y} += $cache[3]; $tcp_urlprot_time{$y} += $cache[4]; } elsif ( $x eq '11' and $#cache == 5 ) { $y = shift (@cache); $tcp_content{$y} = $tcp_content_size{$y} = $tcp_content_time{$y} = $tcp_hit_content{$y} = $tcp_hit_content_size{$y} = 0 unless defined $tcp_content{$y}; $show_other_tcp_content = 0 if defined($tcp_content{''}); $tcp_content{$y} += $cache[0]; $tcp_content_size{$y} += $cache[1]; $tcp_hit_content{$y} += $cache[2]; $tcp_hit_content_size{$y} += $cache[3]; $tcp_content_time{$y} += $cache[4]; } elsif ( $x eq '12' and $#cache == 10 ) { $y = shift (@cache); $tcp_urlext{$y} = $tcp_urlext_size{$y} = $tcp_urlext_time{$y} = $tcp_hit_urlext{$y} = $tcp_hit_urlext_size{$y} = 0 unless defined $tcp_urlext{$y}; $show_other_tcp_urlext = 0 if defined($tcp_urlext{''}); $tcp_urlext{$y} += $cache[0]; $tcp_urlext_size{$y} += $cache[1]; $tcp_hit_urlext{$y} += $cache[2]; $tcp_hit_urlext_size{$y} += $cache[3]; $tcp_urlext_time{$y} += $cache[4]; $tcp_urlext_fresh{$y} += $cache[5]; $tcp_urlext_stale{$y} += $cache[6]; $tcp_urlext_refresh{$y} += $cache[7]; $tcp_urlext_mod{$y} += $cache[8]; $tcp_urlext_unmod{$y} += $cache[9]; } elsif ( $x eq '13.1' and $#cache == 5 ) { $y = shift (@cache); $udp_requester{$y} = $udp_requester_size{$y} = $udp_requester_time{$y} = $udp_hit_requester{$y} = $udp_hit_requester_size{$y} = 0 unless defined $udp_requester{$y}; $show_other_udp_requester = 0 if defined($udp_requester{''}); $udp_requester{$y} += $cache[0]; $udp_requester_size{$y} += $cache[1]; $udp_requester_time{$y} += $cache[2]; $udp_hit_requester{$y} += $cache[3]; $udp_hit_requester_size{$y} += $cache[4]; } elsif ( $x eq '14.1' and $#cache == 5 ) { $y = shift (@cache); $tcp_requester{$y} = $tcp_requester_size{$y} = $tcp_requester_time{$y} = $tcp_hit_requester{$y} = $tcp_hit_requester_size{$y} = 0 unless defined $tcp_requester{$y}; $show_other_tcp_requester = 0 if defined($tcp_requester{''}); $tcp_requester{$y} += $cache[0]; $tcp_requester_size{$y} += $cache[1]; $tcp_requester_time{$y} += $cache[2]; $tcp_hit_requester{$y} += $cache[3]; $tcp_hit_requester_size{$y} += $cache[4]; } elsif ( $x eq '16' and $#cache == 15 ) { $y = shift (@cache); $perf_counter{$y} += $cache[0]; $perf_size{$y} += $cache[1]; $perf_time{$y} += $cache[2]; $perf_tcp_hit_size{$y} += $cache[3]; $perf_tcp_hit_time{$y} += $cache[4]; $perf_tcp_miss_size{$y} += $cache[5]; $perf_tcp_miss_time{$y} += $cache[6]; $perf_hier_direct_size{$y} += $cache[7]; $perf_hier_direct_time{$y} += $cache[8]; $perf_hier_sibling_size{$y} += $cache[9]; $perf_hier_sibling_time{$y} += $cache[10]; $perf_hier_parent_size{$y} += $cache[11]; $perf_hier_parent_time{$y} += $cache[12]; $perf_requester{$y} += $cache[13]; $perf_tcp_hit{$y} += $cache[14]; } elsif ( $x eq '13.2' and $#cache == 6 ) { $y = shift (@cache); $z = shift (@cache); $udp_requester_urlhost{$y}{$z} = $udp_requester_urlhost_size{$y}{$z} = $udp_requester_urlhost_time{$y}{$z} = $udp_hit_requester_urlhost{$y}{$z} = $udp_hit_requester_urlhost_size{$y}{$z} = 0 unless defined $udp_requester_urlhost{$y}{$z}; $udp_requester_urlhost{$y}{$z} += $cache[0]; $udp_requester_urlhost_size{$y}{$z} += $cache[1]; $udp_requester_urlhost_time{$y}{$z} += $cache[2]; $udp_hit_requester_urlhost{$y}{$z} += $cache[3]; $udp_hit_requester_urlhost_size{$y}{$z} += $cache[4]; } elsif ( $x eq '14.2' and $#cache == 6 ) { $y = shift (@cache); $z = shift (@cache); $tcp_requester_urlhost{$y}{$z} = $tcp_requester_urlhost_size{$y}{$z} = $tcp_requester_urlhost_time{$y}{$z} = $tcp_hit_requester_urlhost{$y}{$z} = $tcp_hit_requester_urlhost_size{$y}{$z} = 0 unless defined $tcp_requester_urlhost{$y}{$z}; $tcp_requester_urlhost{$y}{$z} += $cache[0]; $tcp_requester_urlhost_size{$y}{$z} += $cache[1]; $tcp_requester_urlhost_time{$y}{$z} += $cache[2]; $tcp_hit_requester_urlhost{$y}{$z} += $cache[3]; $tcp_hit_requester_urlhost_size{$y}{$z} += $cache[4]; } elsif ( $x eq '15' and $#cache == 5 ) { $y = shift (@cache); $tcp_distribution{$y} = $tcp_distribution_size{$y} = $tcp_distribution_time{$y} = $tcp_hit_distribution{$y} = $tcp_hit_distribution_size{$y} = 0 unless defined $tcp_distribution{$y}; $tcp_distribution{$y} += $cache[0]; $tcp_distribution_size{$y} += $cache[1]; $tcp_distribution_time{$y} += $cache[2]; $tcp_hit_distribution{$y} += $cache[3]; $tcp_hit_distribution_size{$y} += $cache[4]; # read max values } elsif ( $x eq '18' and $#cache == 1 ) { $ordered_tcp_req_time_max_interval = $cache[0]; $ordered_tcp_req_time_max = $cache[1]; } elsif ( $x eq '18' and $#cache == 5 ) { $y = shift (@cache); $ordered_tcp_req_time{$y} = $ordered_tcp_req_time_size{$y} = $ordered_tcp_hit_req_time{$y} = $ordered_tcp_hit_req_time_size{$y} = $ordered_tcp_req_time_time{$y} = 0 unless defined $ordered_tcp_req_time{$y}; $ordered_tcp_req_time{$y} += $cache[0]; $ordered_tcp_req_time_size{$y} += $cache[1]; $ordered_tcp_hit_req_time{$y} += $cache[2]; $ordered_tcp_hit_req_time_size{$y} += $cache[3]; $ordered_tcp_req_time_time{$y} += $cache[4]; # read max values } elsif ( $x eq '17' and $#cache == 1 ) { $ordered_udp_req_time_max_interval = $cache[0]; $ordered_udp_req_time_max = $cache[1]; } elsif ( $x eq '17' and $#cache == 5 ) { $y = shift (@cache); $ordered_udp_req_time{$y} = $ordered_udp_req_time_size{$y} = $ordered_udp_hit_req_time{$y} = $ordered_udp_hit_req_time_size{$y} = $ordered_udp_req_time_time{$y} = 0 unless defined $ordered_udp_req_time{$y}; $ordered_udp_req_time{$y} += $cache[0]; $ordered_udp_req_time_size{$y} += $cache[1]; $ordered_udp_hit_req_time{$y} += $cache[2]; $ordered_udp_hit_req_time_size{$y} += $cache[3]; $ordered_udp_req_time_time{$y} += $cache[4]; } elsif ( $x eq '19' and $#cache == 5 ) { $y = shift (@cache); $udp_code{$y} = $udp_code_size{$y} = 0 unless defined $udp_code{$y}; $udp_code{$y} += $cache[0]; $udp_code_size{$y} += $cache[1]; $udp_hit_code{$y} += $cache[2]; $udp_hit_code_size{$y} += $cache[3]; $udp_code_time{$y} += $cache[4]; } elsif ( $x eq '20' and $#cache == 5 ) { $y = shift (@cache); $tcp_code{$y} = $tcp_code_size{$y} = 0 unless defined $tcp_code{$y}; $tcp_code{$y} += $cache[0]; $tcp_code_size{$y} += $cache[1]; $tcp_hit_code{$y} += $cache[2]; $tcp_hit_code_size{$y} += $cache[3]; $tcp_code_time{$y} += $cache[4]; } else { print STDERR "can't parse cache-line: \"$x @cache\"\n"; $cache_warn = 1 if ( $x =~ m#^[A-Z]$# ); } } close(CACHE); } } sub htmlescape { my $toencode = shift; return undef unless defined($toencode); $toencode=~s/&/&/gso; $toencode=~s/\"/"/gso; $toencode=~s/>/>/gso; $toencode=~s/= $b) ? $a : $b; my $next = ($a >= $b) ? $b : $a; return $a, $b if !$gcd; if ($next) { foreach (1..3) { # reduces accuracy, but increases readability :-) my $r = $gcd % $next; $r += $next if $r < 0; # fix for % in integer mode $gcd = $next; $next = $r; last if ($next == 0); } } return $a/$gcd, $b/$gcd; } sub uri_unescape { # Note from RFC1630: "Sequences which start with a percent sign # but are not followed by two hexadecimal characters are reserved # for future extension" my $str = shift; if (@_ && wantarray) { # not executed for the common case of a single argument my @str = ($str, @_); # need to copy foreach (@str) { s/%([0-9A-Fa-f]{2})/chr(hex($1))/eg } return @str; } $str =~ s/%([0-9A-Fa-f]{2})/chr(hex($1))/eg; $str; } calamaris-2.99.4.0/calBars3d.pm0000644000175000001440000002714510407330372014723 0ustar cordusers#========================================================================== # Module: calamaris::calBars3d.pm # # Copyright # # calBars3d.pm Copyright (c) 2004 Michael Pophal based on # GD::Graph Copyright (c) 1995-1999 Martien Verbruggen. # (http://search.cpan.org/~mverb/GDGraph-1.43/Graph.pm) # # All rights reserved. This package is free software; you can redistribute # it and/or modify it under the same terms as Perl itself. # # Acknowledgements # # Thanks to Martien Verbruggen's ingenious tool GD::Graph, which is basically # the same, except some small changes needed for calamaris. #========================================================================== # # Adapted to calamaris by: (c) 2004 Michael Pophal # # Based on: # GD::Graph::bars3d.pm,v 1.21 2000/04/15 08:59:36 mgjv # http://search.cpan.org/~mverb/GDGraph-1.43/Graph.pm # Copyright (c) 1995-1998 Martien Verbruggen # #-------------------------------------------------------------------------- # # Name: # calamaris::calBars3d.pm # # $Id: calBars3d.pm,v 3.2 2004/09/15 21:02:13 cord Exp $ # #-------------------------------------------------------------------------- # Date Modification Author # ------------------------------------------------------------------------- # 2004AUG30 Adapted to calamaris staehler #========================================================================== package calamaris::calBars3d; use strict; use lib '/usr/local'; use calamaris::calAxestype3d; use GD::Graph::bars; use GD::Graph::utils qw(:all); use GD::Graph::colour qw(:colours); @calamaris::calBars3d::ISA = qw(calamaris::calAxestype3d); $calamaris::calBars3d::VERSION = '0.63'; use constant PI => 4 * atan2(1,1); my %Defaults = ( # Spacing between the bars bar_spacing => 0, # The 3-d extrusion depth of the bars bar_depth => 10, ); sub initialise { my $self = shift; my $rc = $self->SUPER::initialise(); $self->set(correct_width => 1); while( my($key, $val) = each %Defaults ) { $self->{$key} = $val } # end while return $rc; } # end initialise sub set { my $s = shift; my %args = @_; $s->{_set_error} = 0; for (keys %args) { /^bar_depth$/ and do { $s->{bar_depth} = $args{$_}; delete $args{$_}; next; }; } return $s->SUPER::set(%args); } # CONTRIB Jeremy Wadsack # This is a complete overhaul of the original GD::Graph::bars # design, because all versions (overwrite = 0, 1, 2) # require that the bars be drawn in a loop of point over sets sub draw_data { my $self = shift; my $g = $self->{graph}; my $bar_s = _round($self->{bar_spacing}/2); my $set_s = _round($self->{set_spacing}/2); my (%lset_space, %rset_space); for (1..2) { $lset_space{$_} = 0; $rset_space{$_} = 0; }; if ($self->{set_spacing}) { $lset_space{1} = $set_s; $lset_space{2} = 0; $rset_space{2} = $set_s; $rset_space{1} = 0; } my $zero = $self->{zeropoint}; my $i; my @iterate = (0 .. $self->{_data}->num_points()); for $i ($self->{rotate_chart} ? reverse(@iterate) : @iterate) { my ($xp, $t); my $overwrite = 0; $overwrite = $self->{overwrite} if defined $self->{overwrite}; my $j; my $num_sets = ($self->{_data}->num_sets() >= 2) ? 2 : $self->{_data}->num_sets(); my @iterate = (1 .. $num_sets); for $j (($self->{rotate_chart} && $self->{cumulate} == 0) ? reverse(@iterate) : @iterate) { my $value = $self->{_data}->get_y( $j, $i ); my $factor = ($self->{_data}->num_sets() > 2) ? $self->{_data}->get_y( $j+2, $i ) : 0; next unless defined $value; my $bottom = $self->_get_bottom($j, $i); $value = $self->{_data}->get_y_cumulative($j, $i) if ($self->{cumulate}); # Pick a data colour, calc shading colors too, if requested # cycle_clrs option sets the color based on the point, not the dataset. my @rgb; if( $self->{cycle_clrs} ) { @rgb = $self->pick_data_clr( $i + 1 ); } else { @rgb = $self->pick_data_clr( $j ); } # end if my $dsci = $self->set_clr( @rgb ); if( $self->{'3d_shading'} ) { $self->{'3d_highlights'}[$dsci] = $self->set_clr( $self->_brighten( @rgb ) ); $self->{'3d_shadows'}[$dsci] = $self->set_clr( $self->_darken( @rgb ) ); $self->{'3d_dshadows'}[$dsci] = $self->set_clr( $self->_darken($self->_darken( @rgb )) ); $self->{'3d_ddshadows'}[$dsci] = $self->set_clr( $self->_darken($self->_darken($self->_darken( @rgb ))) ); } # end if # contrib "Bremford, Mike" my $brci; if( $self->{cycle_clrs} > 1 ) { $brci = $self->set_clr($self->pick_data_clr($i + 1)); } else { $brci = $self->set_clr($self->pick_border_clr($j)); } # end if # get coordinates of top and center of bar ($xp, $t) = $self->val_to_pixel($i + 1, $value, $j); # calculate offsets of this bar my $x_offset = 0; my $y_offset = 0; if( $overwrite == 1 ) { # $x_offset = $self->{bar_depth} * ($self->{_data}->num_sets() - $j); # $y_offset = $self->{bar_depth} * ($self->{_data}->num_sets() - $j); $x_offset = $self->{bar_depth} * ($num_sets - $j); $y_offset = $self->{bar_depth} * ($num_sets - $j); } $t -= $y_offset; # calculate left and right of bar my ($l, $r); if ($self->{rotate_chart}) { $l = $bottom; ($r) = $self->val_to_pixel($i + 1, $value, $j); } if( (ref $self eq 'GD::Graph::mixed') || ($overwrite >= 1) ) { if ($self->{rotate_chart}) { $bottom = $t + $self->{x_step}/2 - $bar_s + $x_offset; $t = $t - $self->{x_step}/2 + $bar_s + $x_offset; } else { $l = $xp - $self->{x_step}/2 + $bar_s + $x_offset; $r = $xp + $self->{x_step}/2 - $bar_s + $x_offset; } } else { if ($self->{rotate_chart}) { warn "base is $t"; #$bottom = $t - $self->{x_step}/2 # + ($j) * $self->{x_step}/$self->{_data}->num_sets() # + $bar_s + $x_offset; #$t = $t - $self->{x_step}/2 # + ($j-1) * $self->{x_step}/$self->{_data}->num_sets() # - $bar_s + $x_offset; $bottom = $t - $self->{x_step}/2 + ($j) * $self->{x_step}/$num_sets + $bar_s + $x_offset; $t = $t - $self->{x_step}/2 + ($j-1) * $self->{x_step}/$num_sets - $bar_s + $x_offset; warn "top bottom is ($t, $bottom)"; } else { #$l = $xp # - $self->{x_step}/2 # + ($j - 1) * $self->{x_step}/$self->{_data}->num_sets() # + $bar_s + $x_offset; #$r = $xp # - $self->{x_step}/2 # + $j * $self->{x_step}/$self->{_data}->num_sets() # - $bar_s + $x_offset; $l = $xp - $self->{x_step}/2 + ($j - 1) * $self->{x_step}/$num_sets + $bar_s + $lset_space{$j} + $x_offset; $r = $xp - $self->{x_step}/2 + $j * $self->{x_step}/$num_sets - $bar_s - $rset_space{$j} + $x_offset; } } if ($value >= 0) { # draw the positive bar $self->draw_bar( $g, $l, $t, $r, $bottom-$y_offset, $dsci, $brci, 0, $factor ); } else { # draw the negative bar $self->draw_bar( $g, $l, $bottom-$y_offset, $r, $t, $dsci, $brci, -1, $factor ); } # end if } # end for } # end for # redraw the 'zero' axis, front and right if( $self->{zero_axis} ) { $g->line( $self->{left}, $self->{zeropoint}, $self->{right}, $self->{zeropoint}, $self->{fgci} ); $g->line( $self->{right}, $self->{zeropoint}, $self->{right}+$self->{depth_3d}, $self->{zeropoint}-$self->{depth_3d}, $self->{fgci} ); } # end if # redraw the box face if ( $self->{box_axis} ) { # Axes box $g->rectangle($self->{left}, $self->{top}, $self->{right}, $self->{bottom}, $self->{fgci}); $g->line($self->{right}, $self->{top}, $self->{right} + $self->{depth_3d}, $self->{top} - $self->{depth_3d}, $self->{fgci}); $g->line($self->{right}, $self->{bottom}, $self->{right} + $self->{depth_3d}, $self->{bottom} - $self->{depth_3d}, $self->{fgci}); } # end if return $self; } # end draw_data # CONTRIB Jeremy Wadsack # This function draws a bar at the given # coordinates. This is called in all three # overwrite modes. sub draw_bar { my $self = shift; my $g = shift; my( $l, $t, $r, $b, $dsci, $brci, $neg, $factor ) = @_; my $height = ($b - $t)*$factor; # get depth of the bar my $depth = $self->{bar_depth}; # get the bar shadow depth and color my $bsd = $self->{shadow_depth}; my $bsci = $self->set_clr(_rgb($self->{shadowclr})); my( $xi ); # shadow if( $bsd > 0 ) { my $sb = $b - $depth; my $st = $t - $depth + $bsd; if( $neg != 0 ) { $st -= $bsd; if( $self->{zero_axis_only} ) { $sb += $bsd; } else { $sb = _min($b-$depth+$bsd, $self->{bottom}-$depth); } # end if } # end if # ** If this isn't the back bar, then no side shadow should be # drawn or else the top should be lowered by # ($bsd * dataset_num), it should be drawn on the back surface, # and a shadow should be drawn behind the front bar if the # bar is positive and the back is negative. $g->filledRectangle($l+$depth+$bsd, $st, $r+$depth+$bsd, $sb, $bsci); # Only draw bottom shadow if at the bottom and has bottom # axis. Always draw top shadow if( ($neg == 0) || ($sb >= $self->{bottom}-$depth) ) { my $poly = new GD::Polygon; $poly->addPt( $r, $b ); $poly->addPt( $r+$bsd, $b ); $poly->addPt( $r+$depth+$bsd, $b-$depth ); $poly->addPt( $r+$depth, $b-$depth ); $g->filledPolygon( $poly, $bsci ); } # end if } # end if # side top my $poly = new GD::Polygon; $poly->addPt( $r, $b ); $poly->addPt( $r+$depth, $b-$depth ); $poly->addPt( $r+$depth, $b-$depth-$height ); $poly->addPt( $r, $b-$height ); if( $self->{'3d_shading'} ) { $g->filledPolygon( $poly, $self->{'3d_ddshadows'}[$dsci] ); } else { $g->filledPolygon( $poly, $dsci ); } # end if # side bottom $poly = new GD::Polygon; $poly->addPt( $r, $b-$height ); $poly->addPt( $r+$depth, $b-$depth-$height ); $poly->addPt( $r+$depth, $t-$depth ); $poly->addPt( $r, $t ); if( $self->{'3d_shading'} ) { $g->filledPolygon( $poly, $self->{'3d_shadows'}[$dsci] ); } else { $g->filledPolygon( $poly, $dsci ); } # end if $g->polygon( $poly, $brci ); # top # -- only draw negative tops if the bar starts at zero if( ($neg == 0) || ($t <= $self->{zeropoint}) ) { $poly = new GD::Polygon; $poly->addPt( $l, $t ); $poly->addPt( $l+$depth, $t-$depth ); $poly->addPt( $r+$depth, $t-$depth ); $poly->addPt( $r, $t ); if( $self->{'3d_shading'} ) { $g->filledPolygon( $poly, $self->{'3d_highlights'}[$dsci] ); } else { $g->filledPolygon( $poly, $dsci ); } # end if $g->polygon( $poly, $brci ); } # end if # face $g->filledRectangle( $l, $t, $r, $b-$height, $dsci ); # face top $g->filledRectangle( $l, $b-$height, $r, $b, $self->{'3d_dshadows'}[$dsci] ); # face bottom $g->rectangle( $l, $t, $r, $b-$height, $brci ); # frame face top $g->rectangle( $l, $b-$height, $r, $b, $brci ); # frame face bottom } # end draw_bar # [JAW] Overrides axestype's set_max_min. # Go through the parent's process then adjust the baseline to 0 for bar graphs. sub set_max_min { my $self = shift; $self->SUPER::set_max_min( @_ ); # This code is taken from Martien's axestype.pm for my $i (1..($self->{two_axes} ? 2 : 1)) { # If at the same side of the zero axis if( $self->{y_max}[$i] && $self->{y_min}[$i]/$self->{y_max}[$i] > 0 ) { $self->{y_min}[$i] > 0 ? $self->{y_min}[$i] = 0 : $self->{y_max}[$i] = 0 ; } # end if } # end for return $self; } # end set_max_min # [JW] Just use the one in GD::Graph::bars sub draw_values { return &GD::Graph::bars::draw_values( @_ ); } 1; calamaris-2.99.4.0/calAxestype3d.pm0000644000175000001440000005552510407330372015641 0ustar cordusers#========================================================================== # Module: calamaris::calAxestype3d.pm # # Copyright # # calAxestype3d.pm Copyright (c) 2004 Michael Pophal based on # GD::Graph Copyright (c) 1999 Martien Verbruggen. # (http://search.cpan.org/~mverb/GDGraph-1.43/Graph.pm) # # All rights reserved. This package is free software; you can redistribute # it and/or modify it under the same terms as Perl itself. # # Acknowledgements # # Thanks to Martien Verbruggen's ingenious tool GD::Graph, which is basically # the same, except some small changes needed for calamaris. #========================================================================== # # Adapted to calamaris by: (c) 2004 Michael Pophal # # Based on: # GD::Graph::axestype3d.pm,v 1.21 2000/04/15 08:59:36 mgjv # http://search.cpan.org/~mverb/GDGraph-1.43/Graph.pm # Copyright (c) 1995-1998 Martien Verbruggen # #-------------------------------------------------------------------------- # # Name: # calamaris::calAxestype3d.pm # # $Id: calAxestype3d.pm,v 3.2 2004/09/15 21:02:13 cord Exp $ # #-------------------------------------------------------------------------- # Date Modification Author # ------------------------------------------------------------------------- # 2004AUG30 Adapted to calamaris staehler #========================================================================== package calamaris::calAxestype3d; use strict; use lib '/usr/local'; use GD::Graph; use calamaris::calAxestype; use GD::Graph::utils qw(:all); use GD::Graph::colour qw(:colours); use Carp; @calamaris::calAxestype3d::ISA = qw(calamaris::calAxestype); $calamaris::calAxestype3d::VERSION = '0.63'; # Commented inheritance from GD::Graph::axestype unless otherwise noted. use constant PI => 4 * atan2(1,1); my %Defaults = ( depth_3d => 20, '3d_shading' => 1, # the rest are inherited ); # Inherit _has_default # Can't inherit initialise, because %Defaults is referenced file- # specific, not class specific. sub initialise { my $self = shift; my $rc = $self->SUPER::initialise(); while( my($key, $val) = each %Defaults ) { $self->{$key} = $val } # end while return $rc; } # end initialise # PUBLIC # Inherit plot # Inherit set # Inherit setup_text # Inherit set_x_label_font # Inherit set_y_label_font # Inherit set_x_axis_font # Inherit set_y_axis_font # Inherit set_legend # Inherit set_legend_font # ---------------------------------------------------------- # Sub: init_graph # # Args: (None) # # Description: # Override GD::Graph::init_graph to add 3d shading colors, # if requested # # [From GD::Graph] # Initialise the graph output canvas, setting colours (and # getting back index numbers for them) setting the graph to # transparent, and interlaced, putting a logo (if defined) # on there. # ---------------------------------------------------------- # Date Modification Author # ---------------------------------------------------------- # 20Aug2000 Added to support 3d graph extensions JW # ---------------------------------------------------------- sub init_graph { my $self = shift; # Sets up the canvas and color palette $self->SUPER::init_graph( @_ ); # Now create highlights and showdows for each color # in the palette if( $self->{'3d_shading'} ) { $self->{'3d_highlights'} = []; $self->{'3d_shadows'} = []; $self->{'3d_highlights'}[$self->{bgci}] = $self->set_clr( $self->_brighten( _rgb($self->{bgclr}) ) ); $self->{'3d_shadows'}[$self->{bgci}] = $self->set_clr( $self->_darken( _rgb($self->{bgclr}) ) ); $self->{'3d_highlights'}[$self->{fgci}] = $self->set_clr( $self->_brighten( _rgb($self->{fgclr}) ) ); $self->{'3d_shadows'}[$self->{fgci}] = $self->set_clr( $self->_darken( _rgb($self->{fgclr}) ) ); $self->{'3d_highlights'}[$self->{tci}] = $self->set_clr( $self->_brighten( _rgb($self->{textclr}) ) ); $self->{'3d_shadows'}[$self->{tci}] = $self->set_clr( $self->_darken( _rgb($self->{textclr}) ) ); $self->{'3d_highlights'}[$self->{lci}] = $self->set_clr( $self->_brighten( _rgb($self->{labelclr}) ) ); $self->{'3d_shadows'}[$self->{lci}] = $self->set_clr( $self->_darken( _rgb($self->{labelclr}) ) ); $self->{'3d_highlights'}[$self->{alci}] = $self->set_clr( $self->_brighten( _rgb($self->{axislabelclr}) ) ); $self->{'3d_shadows'}[$self->{alci}] = $self->set_clr( $self->_darken( _rgb($self->{axislabelclr}) ) ); $self->{'3d_highlights'}[$self->{acci}] = $self->set_clr( $self->_brighten( _rgb($self->{accentclr}) ) ); $self->{'3d_shadows'}[$self->{acci}] = $self->set_clr( $self->_darken( _rgb($self->{accentclr}) ) ); $self->{'3d_highlights'}[$self->{valuesci}] = $self->set_clr( $self->_brighten( _rgb($self->{valuesclr}) ) ); $self->{'3d_shadows'}[$self->{valuesci}] = $self->set_clr( $self->_darken( _rgb($self->{valuesclr}) ) ); $self->{'3d_highlights'}[$self->{legendci}] = $self->set_clr( $self->_brighten( _rgb($self->{legendclr}) ) ); $self->{'3d_shadows'}[$self->{legendci}] = $self->set_clr( $self->_darken( _rgb($self->{legendclr}) ) ); if( $self->{boxclr} ) { $self->{'3d_highlights'}[$self->{boxci}] = $self->set_clr( $self->_brighten( _rgb($self->{boxclr}) ) ); $self->{'3d_shadows'}[$self->{boxci}] = $self->set_clr( $self->_darken( _rgb($self->{boxclr}) ) ); } # end if } # end if return $self; } # end init_graph # PRIVATE # ---------------------------------------------------------- # Sub: _brighten # # Args: $r, $g, $b # $r, $g, $b The Red, Green, and Blue components of a color # # Description: Brightens the color by adding white # ---------------------------------------------------------- # Date Modification Author # ---------------------------------------------------------- # 21AUG2000 Created to build 3d highlights table JW # ---------------------------------------------------------- sub _brighten { my $self = shift; my( $r, $g, $b ) = @_; my $p = ($r + $g + $b) / 70; $p = 3 if $p < 3; my $f = _max( $r / $p, _max( $g / $p, $b / $p ) ); $r = _min( 255, int( $r + $f ) ); $g = _min( 255, int( $g + $f ) ); $b = _min( 255, int( $b + $f ) ); return( $r, $g, $b ); } # end _brighten # ---------------------------------------------------------- # Sub: _darken # # Args: $r, $g, $b # $r, $g, $b The Red, Green, and Blue components of a color # # Description: Darkens the color by adding black # ---------------------------------------------------------- # Date Modification Author # ---------------------------------------------------------- # 21AUG2000 Created to build 3d shadows table JW # ---------------------------------------------------------- sub _darken { my $self = shift; my( $r, $g, $b ) = @_; my $p = ($r + $g + $b) / 70; $p = 3 if $p < 3; my $f = _max( $r / $p, _max( $g / $p, $b / $p) ); $r = _max( 0, int( $r - $f ) ); $g = _max( 0, int( $g - $f ) ); $b = _max( 0, int( $b - $f ) ); return( $r, $g, $b ); } # end _darken # inherit check_data from GD::Graph # [JAW] Setup boundaries as parent, the adjust for 3d extrusion sub _setup_boundaries { my $self = shift; $self->SUPER::_setup_boundaries(); # adjust for top of 3-d extrusion $self->{top} += $self->{depth_3d}; return $self->_set_error('Vertical size too small') if $self->{bottom} <= $self->{top}; # adjust for right of 3-d extrusion $self->{right} -= $self->{depth_3d}; return $self->_set_error('Horizontal size too small') if $self->{right} <= $self->{left}; return $self; } # end _setup_boundaries # [JAW] Determine 3d-extrusion depth, then call parent sub setup_coords { my $self = shift; # Calculate the 3d-depth of the graph # Note this sets a minimum depth of ~20 pixels # if (!defined $self->{x_tick_number}) { my $depth = _max( $self->{bar_depth}, $self->{line_depth} ); if( $self->{overwrite} == 1 ) { $depth *= $self->{_data}->num_sets(); } # end if $self->{depth_3d} = _max( $depth, $self->{depth_3d} ); # } # end if $self->SUPER::setup_coords(); return $self; } # end setup_coords # Inherit create_y_labels # Inherit get_x_axis_label_height # Inherit create_x_labels # inherit open_graph from GD::Graph # Inherit draw_text # [JAW] Draws entire bounding cube for 3-d extrusion sub draw_axes { my $s = shift; my $g = $s->{graph}; my ($l, $r, $b, $t) = ( $s->{left}, $s->{right}, $s->{bottom}, $s->{top} ); my $depth = $s->{depth_3d}; if ( $s->{box_axis} ) { # -- Draw a bounding box if( $s->{boxci} ) { # -- Fill the box with color # Back box $g->filledRectangle($l+$depth+1, $t-$depth+1, $r+$depth-1, $b-$depth-1, $s->{boxci}); # Left side my $poly = new GD::Polygon; $poly->addPt( $l, $t ); $poly->addPt( $l + $depth, $t - $depth ); $poly->addPt( $l + $depth, $b - $depth ); $poly->addPt( $l, $b ); if( $s->{'3d_shading'} ) { $g->filledPolygon( $poly, $s->{'3d_shadows'}[$s->{boxci}] ); } else { $g->filledPolygon( $poly, $s->{boxci} ); } # end if # Bottom $poly = new GD::Polygon; $poly->addPt( $l, $b ); $poly->addPt( $l + $depth, $b - $depth ); $poly->addPt( $r + $depth, $b - $depth ); $poly->addPt( $r, $b ); if( $s->{'3d_shading'} ) { $g->filledPolygon( $poly, $s->{'3d_highlights'}[$s->{boxci}] ); } else { $g->filledPolygon( $poly, $s->{boxci} ); } # end if } # end if # -- Draw the box frame # Back box $g->rectangle($l+$depth, $t-$depth, $r+$depth, $b-$depth, $s->{fgci}); # Connecting frame $g->line($l, $t, $l + $depth, $t - $depth, $s->{fgci}); $g->line($r, $t, $r + $depth, $t - $depth, $s->{fgci}); $g->line($l, $b, $l + $depth, $b - $depth, $s->{fgci}); $g->line($r, $b, $r + $depth, $b - $depth, $s->{fgci}); # Front box $g->rectangle($l, $t, $r, $b, $s->{fgci}); } else { if( $s->{boxci} ) { # -- Fill the background box with color # Back box $g->filledRectangle($l+$depth+1, $t-$depth+1, $r+$depth-1, $b-$depth-1, $s->{boxci}); # Left side my $poly = new GD::Polygon; $poly->addPt( $l, $t ); $poly->addPt( $l + $depth, $t - $depth ); $poly->addPt( $l + $depth, $b - $depth ); $poly->addPt( $l, $b ); if( $s->{'3d_shading'} ) { $g->filledPolygon( $poly, $s->{'3d_shadows'}[$s->{boxci}] ); } else { $g->filledPolygon( $poly, $s->{boxci} ); } # end if # Bottom $poly = new GD::Polygon; $poly->addPt( $l, $b ); $poly->addPt( $l + $depth, $b - $depth ); $poly->addPt( $r + $depth, $b - $depth ); $poly->addPt( $r, $b ); if( $s->{'3d_shading'} ) { $g->filledPolygon( $poly, $s->{'3d_highlights'}[$s->{boxci}] ); } else { $g->filledPolygon( $poly, $s->{boxci} ); } # end if } # end if # -- Draw the frame only for back & sides # Back box $g->rectangle($l + $depth, $t - $depth, $r + $depth, $b - $depth, $s->{fgci}); # Y axis my $poly = new GD::Polygon; $poly->addPt( $l, $t ); $poly->addPt( $l, $b ); $poly->addPt( $l + $depth, $b - $depth ); $poly->addPt( $l + $depth, $t - $depth ); $g->polygon( $poly, $s->{fgci} ); # X axis if( !$s->{zero_axis_only} ) { $poly = new GD::Polygon; $poly->addPt( $l, $b ); $poly->addPt( $r, $b ); $poly->addPt( $r + $depth, $b - $depth ); $poly->addPt( $l + $depth, $b - $depth ); $g->polygon( $poly, $s->{fgci} ); } # end if # Second Y axis if( $s->{two_axes} ){ $poly = new GD::Polygon; $poly->addPt( $r, $b ); $poly->addPt( $r, $t ); $poly->addPt( $r + $depth, $t - $depth ); $poly->addPt( $r + $depth, $b - $depth ); $g->polygon( $poly, $s->{fgci} ); } # end if } # end if # Zero axis if ($s->{zero_axis} or $s->{zero_axis_only}) { my ($x, $y) = $s->val_to_pixel(0, 0, 1); my $poly = new GD::Polygon; $poly->addPt( $l, $y ); $poly->addPt( $r, $y ); $poly->addPt( $r + $depth, $y - $depth ); $poly->addPt( $l + $depth, $y - $depth); $g->polygon( $poly, $s->{fgci} ); } # end if } # end draw_axes # [JAW] Draws ticks and values for y axes in 3d extrusion # Modified from MVERB source sub draw_y_ticks { my $self = shift; for my $t (0 .. $self->{y_tick_number}) { for my $a (1 .. ($self->{two_axes} + 1)) { my $value = $self->{y_values}[$a][$t]; my $label = $self->{y_labels}[$a][$t]; my ($x, $y) = $self->val_to_pixel(0, $value, $a); $x = ($a == 1) ? $self->{left} : $self->{right}; # CONTRIB Jeremy Wadsack # Draw on the back of the extrusion $x += $self->{depth_3d}; $y -= $self->{depth_3d}; if ($self->{y_long_ticks}) { $self->{graph}->line( $x, $y, $x + $self->{right} - $self->{left}, $y, $self->{fgci} ) unless ($a-1); # CONTRIB Jeremy Wadsack # Draw conector ticks $self->{graph}->line( $x - $self->{depth_3d}, $y + $self->{depth_3d}, $x, $y, $self->{fgci} ) unless ($a-1); } else { $self->{graph}->line( $x, $y, $x + (3 - 2 * $a) * $self->{y_tick_length}, $y, $self->{fgci} ); # CONTRIB Jeremy Wadsack # Draw conector ticks $self->{graph}->line( $x - $self->{depth_3d}, $y + $self->{depth_3d}, $x - $self->{depth_3d} + (3 - 2 * $a) * $self->{y_tick_length}, $y + $self->{depth_3d} - (3 - 2 * $a) * $self->{y_tick_length}, $self->{fgci} ); } next if $t % ($self->{y_label_skip}) || ! $self->{y_plot_values}; $self->{gdta_y_axis}->set_text($label); $self->{gdta_y_axis}->set_align('center', $a == 1 ? 'right' : 'left'); $x -= (3 - 2 * $a) * $self->{axis_space}; # CONTRIB Jeremy Wadsack # Subtract 3-d extrusion width from left axis label # (it was added for ticks) $x -= (2 - $a) * $self->{depth_3d}; # CONTRIB Jeremy Wadsack # Add 3-d extrusion height to label # (it was subtracted for ticks) $y += $self->{depth_3d}; $self->{gdta_y_axis}->draw($x, $y); } # end foreach } # end foreach return $self; } # end draw_y_ticks # [JAW] Darws ticks and values for x axes wih 3d extrusion # Modified from MVERB source sub draw_x_ticks { my $self = shift; for (my $i = 0; $i < $self->{_data}->num_points; $i++) { my ($x, $y) = $self->val_to_pixel($i + 1, 0, 1); $y = $self->{bottom} unless $self->{zero_axis_only}; # CONTRIB Damon Brodie for x_tick_offset next if (!$self->{x_all_ticks} and ($i - $self->{x_tick_offset}) % $self->{x_label_skip} and $i != $self->{_data}->num_points - 1 ); # CONTRIB Jeremy Wadsack # Draw on the back of the extrusion $x += $self->{depth_3d}; $y -= $self->{depth_3d}; if ($self->{x_ticks}) { if ($self->{x_long_ticks}) { # CONTRIB Jeremy Wadsack # Move up by 3d depth $self->{graph}->line( $x, $self->{bottom} - $self->{depth_3d}, $x, $self->{top} - $self->{depth_3d}, $self->{fgci}); # CONTRIB Jeremy Wadsack # Draw conector ticks $self->{graph}->line( $x - $self->{depth_3d}, $y + $self->{depth_3d}, $x, $y, $self->{fgci} ); } else { $self->{graph}->line( $x, $y, $x, $y - $self->{x_tick_length}, $self->{fgci} ); # CONTRIB Jeremy Wadsack # Draw conector ticks $self->{graph}->line( $x - $self->{depth_3d}, $y + $self->{depth_3d}, $x - $self->{depth_3d} + $self->{x_tick_length}, $y + $self->{depth_3d} - $self->{x_tick_length}, $self->{fgci} ); } } # CONTRIB Damon Brodie for x_tick_offset next if ($i - $self->{x_tick_offset}) % ($self->{x_label_skip}) and $i != $self->{_data}->num_points - 1; $self->{gdta_x_axis}->set_text($self->{_data}->get_x($i)); # CONTRIB Jeremy Wadsack # Subtract 3-d extrusion width from left label # Add 3-d extrusion height to left label # (they were changed for ticks) $x -= $self->{depth_3d}; $y += $self->{depth_3d}; my $yt = $y + $self->{axis_space}; if ($self->{x_labels_vertical}) { $self->{gdta_x_axis}->set_align('center', 'right'); $self->{gdta_x_axis}->draw($x, $yt, PI/2); } else { $self->{gdta_x_axis}->set_align('top', 'center'); $self->{gdta_x_axis}->draw($x, $yt); } } # end for return $self; } # end draw_x_ticks # CONTRIB Scott Prahl # Assume x array contains equally spaced x-values # and generate an appropriate axis # #### # 'True' numerical X axis addition # From: Gary Deschaines # # These modification to draw_x_ticks_number pass x-tick values to the # val_to_pixel subroutine instead of x-tick indices when ture[sic] numerical # x-axis mode is detected. Also, x_tick_offset and x_label_skip are # processed differently when true numerical x-axis mode is detected to # allow labeled major x-tick marks and un-labeled minor x-tick marks. # # For example: # # x_tick_number => 14, # x_ticks => 1, # x_long_ticks => 1, # x_tick_length => -4, # x_min_value => 100, # x_max_value => 800, # x_tick_offset => 2, # x_label_skip => 2, # # # ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ # | | | | | | | | | | | | | # 1 -| | | | | | | | | | | | | # | | | | | | | | | | | | | # 0 _|_________|____|____|____|____|____|____|____|____|____|____|_________| # | | | | | | | | | | | # 200 300 400 500 600 700 #### # [JAW] Added commented items for 3d rendering # Based on MVERB source sub draw_x_ticks_number { my $self = shift; for my $i (0 .. $self->{x_tick_number}) { my ($value, $x, $y); if (defined($self->{x_min_value}) && defined($self->{x_max_value})) { next if ($i - $self->{x_tick_offset}) < 0; next if ($i + $self->{x_tick_offset}) > $self->{x_tick_number}; $value = $self->{x_values}[$i]; ($x, $y) = $self->val_to_pixel($value, 0, 1); } else { $value = ($self->{_data}->num_points - 1) * ($self->{x_values}[$i] - $self->{true_x_min}) / ($self->{true_x_max} - $self->{true_x_min}); ($x, $y) = $self->val_to_pixel($value + 1, 0, 1); } $y = $self->{bottom} unless $self->{zero_axis_only}; # Draw on the back of the extrusion $x += $self->{depth_3d}; $y -= $self->{depth_3d}; if ($self->{x_ticks}) { if ($self->{x_long_ticks}) { # XXX This mod needs to be done everywhere ticks are # drawn if ( $self->{x_tick_length} >= 0 ) { # Move up by 3d depth $self->{graph}->line( $x, $self->{bottom} - $self->{depth_3d}, $x, $self->{top} - $self->{depth_3d}, $self->{fgci}); } else { $self->{graph}->line( $x, $self->{bottom} - $self->{x_tick_length}, $x, $self->{top}, $self->{fgci}); } # CONTRIB Jeremy Wadsack # Draw conector ticks $self->{graph}->line( $x - $self->{depth_3d}, $y + $self->{depth_3d}, $x, $y, $self->{fgci} ); } else { $self->{graph}->line($x, $y, $x, $y - $self->{x_tick_length}, $self->{fgci} ); # CONTRIB Jeremy Wadsack # Draw conector ticks $self->{graph}->line( $x - $self->{depth_3d}, $y + $self->{depth_3d}, $x, - $self->{depth_3d} + $self->{tick_length}, $y, + $self->{depth_3d} - $self->{tick_length}, $self->{fgci} ); } # end if -- x_long_ticks } # end if -- x_ticks # If we have to skip labels, we'll do it here. # Make sure to always draw the last one. next if $i % $self->{x_label_skip} && $i != $self->{x_tick_number}; $self->{gdta_x_axis}->set_text($self->{x_labels}[$i]); # CONTRIB Jeremy Wadsack # Subtract 3-d extrusion width from left label # Add 3-d extrusion height to left label # (they were changed for ticks) $x -= $self->{depth_3d}; $y += $self->{depth_3d}; if ($self->{x_labels_vertical}) { $self->{gdta_x_axis}->set_align('center', 'right'); my $yt = $y + $self->{text_space}/2; $self->{gdta_x_axis}->draw($x, $yt, PI/2); } else { $self->{gdta_x_axis}->set_align('top', 'center'); my $yt = $y + $self->{text_space}/2; $self->{gdta_x_axis}->draw($x, $yt); } # end if } # end for return $self; } # end draw_x_tick_number # Inherit draw_ticks # Inherit draw_data # Inherit draw_data_set # Inherit set_max_min # Inherit get_max_y # Inherit get_min_y # Inherit get_max_min_y_all # Inherit _get_bottom # Inherit val_to_pixel # Inherit setup_legend # [JW] Override draw_legend and reverse the drawing order # if cumulate is enabled so legend matches data on chart sub draw_legend { my $self = shift; return unless defined $self->{legend}; my $xl = $self->{lg_xs} + $self->{legend_spacing}; my $y = $self->{lg_ys} + $self->{legend_spacing} - 1; # If there's a frame, offset by the size and margin $xl += $self->{legend_frame_margin} + $self->{legend_frame_size} if $self->{legend_frame_size}; $y += $self->{legend_frame_margin} + $self->{legend_frame_size} if $self->{legend_frame_size}; my $i = 0; my $row = 1; my $x = $xl; # start position of current element my @legends = @{$self->{legend}}; my $i_step = 1; # If we are working in cumulate mode, then reverse the drawing order if( $self->{cumulate} ) { @legends = reverse @legends; $i = scalar(@legends); $i = $self->{_data}->num_sets if $self->{_data}->num_sets < $i; $i++; $i_step = -1; } # end if foreach my $legend (@legends) { $i += $i_step; # Legend for Pie goes over first set, and all points # Works in either direction last if $i > $self->{_data}->num_sets; last if $i < 1; my $xe = $x; # position within an element next unless defined($legend) && $legend ne ""; $self->draw_legend_marker($i, $xe, $y); $xe += $self->{legend_marker_width} + $self->{legend_spacing}; my $ys = int($y + $self->{lg_el_height}/2 - $self->{lgfh}/2); $self->{gdta_legend}->set_text($legend); $self->{gdta_legend}->draw($xe, $ys); $x += $self->{lg_el_width}; if (++$row > $self->{lg_cols}) { $row = 1; $y += $self->{lg_el_height}; $x = $xl; } } # If there's a frame, draw it now if( $self->{legend_frame_size} ) { $x = $self->{lg_xs} + $self->{legend_spacing}; $y = $self->{lg_ys} + $self->{legend_spacing} - 1; for $i ( 0 .. $self->{legend_frame_size} - 1 ) { $self->{graph}->rectangle( $x + $i, $y + $i, $x + $self->{lg_x_size} + 2 * $self->{legend_frame_margin} - $i - 1, $y + $self->{lg_y_size} + 2 * $self->{legend_frame_margin} - $i - 1, $self->{acci}, ); } # end for } # end if } # Inherit draw_legend_marker 1; calamaris-2.99.4.0/calAxestype.pm0000644000175000001440000016562310407330372015413 0ustar cordusers#========================================================================== # Module: calamaris::calAxestype.pm # # Copyright # # calAxestype.pm Copyright (c) 2004 Michael Pophal based on # GD::Graph Copyright (c) 1995-1999 Martien Verbruggen. # (http://search.cpan.org/~mverb/GDGraph-1.43/Graph.pm) # # All rights reserved. This package is free software; you can redistribute # it and/or modify it under the same terms as Perl itself. # # Acknowledgements # # Thanks to Martien Verbruggen's ingenious tool GD::Graph, which is basically # the same, except some small changes needed for calamaris. #========================================================================== # # Adapted to calamaris by: (c) 2004 Michael Pophal # # Based on: # GD::Graph::axestype.pm,v 1.21 2000/04/15 08:59:36 mgjv # http://search.cpan.org/~mverb/GDGraph-1.43/Graph.pm # Copyright (c) 1995-1998 Martien Verbruggen # #-------------------------------------------------------------------------- # # Name: # calamaris::calAxestype.pm # # $Id: calAxestype.pm,v 3.2 2004/09/15 21:02:13 cord Exp $ # #-------------------------------------------------------------------------- # Date Modification Author # ------------------------------------------------------------------------- # 2004AUG30 Adapted to calamaris staehler #========================================================================== package calamaris::calAxestype; ($axestype::VERSION) = '$Revision: 3.2 $' =~ /\s([\d.]+)/; use strict; use lib '/usr/local'; use GD::Graph; use GD::Graph::utils qw(:all); use Carp; @calamaris::calAxestype::ISA = qw(GD::Graph); use constant PI => 4 * atan2(1,1); my %Defaults = ( # Set the length for the 'short' ticks on the axes. x_tick_length => 4, y_tick_length => 4, # Do you want ticks to span the entire width of the graph? x_long_ticks => 0, y_long_ticks => 0, # Number of ticks for the y axis y_tick_number => 5, x_tick_number => undef, # CONTRIB Scott Prahl x_tick_offset => 0, # CONTRIB Damon Brodi # Skip every nth label. if 1 will print every label on the axes, # if 2 will print every second, etc.. x_label_skip => 1, y_label_skip => 1, # Do we want ticks on the x axis? x_ticks => 1, x_all_ticks => 0, # Where to place the x and y labels x_label_position => 3/4, y_label_position => 1/2, # vertical printing of x labels x_labels_vertical => 0, # Draw axes as a box? (otherwise just left and bottom) box_axis => 1, # Disable axes? # undef -> all axes, 0 -> Only line for bars, other -> no axes at all. no_axes => undef, # Use two different axes for the first and second dataset. The first # will be displayed using the left axis, the second using the right # axis. You cannot use more than two datasets when this option is on. two_axes => 0, # Which axis to use for each dataset. This only is in effect when # two_axes is true. The axis number will wrap around, just like # the dclrs array. use_axis => [1, 2], # Print values on the axes? x_plot_values => 1, y_plot_values => 1, # Space between axis and text axis_space => 4, # Do you want bars to be drawn on top of each other, or side by side? overwrite => 0, # This will replace 'overwrite = 2'. For now, it is hardcoded to set # overwrite to 2 cumulate => 0, # Do you want me to correct the width of the graph, so that bars are # always drawn with a nice integer number of pixels? # # The GD::Graph::bars::initialise sub will switch this on. # Do not set this to anything else than undef! correct_width => undef, # XXX The following two need to get better defaults. Maybe computed. # Draw the zero axis in the graph in case there are negative values zero_axis => 0, # Draw the zero axis, but do not draw the bottom axis, in case # box-axis == 0 # This also moves the x axis labels to the zero axis zero_axis_only => 0, # Size of the legend markers legend_marker_height => 8, legend_marker_width => 12, legend_spacing => 4, legend_placement => 'BC', # '[BR][LCR]' lg_cols => undef, # Display the y values above the bar or point in the graph. show_values => undef, values_vertical => undef, # vertical? values_space => 4, # extra spacing values_format => undef, # how to format the value # Draw the X axis left and the y1 axis at the bottom (y2 at top) rotate_chart => undef, # CONTRIB Edwin Hildebrand # How narrow is a dataset allowed to become before we drop the # accents? accent_treshold => 4, # Format of the numbers on the x and y axis y_number_format => undef, x_number_format => undef, # CONTRIB Scott Prahl # and some attributes without default values x_label => undef, y_label => undef, y1_label => undef, y2_label => undef, x_min_value => undef, x_max_value => undef, y_min_value => undef, y1_min_value => undef, y2_min_value => undef, y_max_value => undef, y1_max_value => undef, y2_max_value => undef, y_min_range => undef, # CONTRIB Ben Tilly y1_min_range => undef, y2_min_range => undef, borderclrs => undef, # XXX # Multiple inheritance (linespoints and mixed) finally bit me. The # _has_defaults and set methods can only work correctly when the # spot where the defaults are kept are in a mutual parent, which # would be this. The odd implementation of SUPER doesn't help # XXX points # The size of the marker to use in the points and linespoints graphs # in pixels marker_size => 4, # attributes with no default markers => undef, # XXX lines # The width of the line to use in the lines and linespoints graphs # in pixels line_width => 1, # Set the scale of the line types line_type_scale => 8, # Which line types to use line_types => [1], # Skip undefined values, and don't draw them at all skip_undef => 0, # XXX bars # Spacing between the bars bar_width => undef, bar_spacing => 0, set_spacing => 0, # cycle through colours per data point, not set cycle_clrs => 0, # colour of the shadow shadowclr => 'dgray', shadow_depth => 0, # XXX mixed default_type => 'lines', types => undef, ); sub _has_default { my $self = shift; my $attr = shift || return; exists $Defaults{$attr} || $self->SUPER::_has_default($attr); } sub initialise { my $self = shift; $self->SUPER::initialise(); while (my($key, $val) = each %Defaults) { $self->{$key} = $val } $self->set_x_label_font(GD::gdSmallFont); $self->set_y_label_font(GD::gdSmallFont); $self->set_x_axis_font(GD::gdTinyFont); $self->set_y_axis_font(GD::gdTinyFont); $self->set_legend_font(GD::gdTinyFont); $self->set_values_font(GD::gdTinyFont); } # PUBLIC sub plot { my $self = shift; my $data = shift; $self->check_data($data) or return; $self->init_graph() or return; $self->setup_text() or return; $self->setup_legend(); $self->setup_coords() or return; $self->draw_text(); unless (defined $self->{no_axes}) { $self->draw_axes(); $self->draw_ticks() or return; } $self->draw_data() or return; $self->draw_values() or return; $self->draw_legend(); return $self->{graph} } sub set { my $self = shift; my %args = @_; for (keys %args) { /^tick_length$/ and do { $self->{x_tick_length} = $self->{y_tick_length} = $args{$_}; delete $args{$_}; next; }; /^long_ticks$/ and do { $self->{x_long_ticks} = $self->{y_long_ticks} = $args{$_}; delete $args{$_}; next; }; /^overwrite$/ and do { $self->{cumulate} = 1 if $args{$_} == 2; $self->{overwrite} = $args{$_}; delete $args{$_}; next; }; /^cumulate$/ and do { $self->{cumulate} = $args{$_}; # XXX And for now $self->{overwrite} = 2 if $args{$_}; delete $args{$_}; next; }; } return $self->SUPER::set(%args); } sub setup_text { my $self = shift; $self->{gdta_x_label}->set(colour => $self->{lci}); $self->{gdta_y_label}->set(colour => $self->{lci}); $self->{xlfh} = $self->{gdta_x_label}->get('height'); $self->{ylfh} = $self->{gdta_y_label}->get('height'); $self->{gdta_x_axis}->set(colour => $self->{alci}); $self->{gdta_y_axis}->set(colour => $self->{alci}); $self->{xafh} = $self->{gdta_x_axis}->get('height'); $self->{yafh} = $self->{gdta_x_axis}->get('height'); $self->{gdta_title}->set(colour => $self->{tci}); $self->{gdta_title}->set_align('top', 'center'); $self->{tfh} = $self->{gdta_title}->get('height'); $self->{gdta_legend}->set(colour => $self->{legendci}); $self->{gdta_legend}->set_align('top', 'left'); $self->{lgfh} = $self->{gdta_legend}->get('height'); $self->{gdta_values}->set(colour => $self->{valuesci}); unless ($self->{rotate_chart}) { if ($self->{values_vertical}) { $self->{gdta_values}->set_align('center', 'left'); } else { $self->{gdta_values}->set_align('bottom', 'center'); } } else { if ($self->{values_vertical}) { $self->{gdta_values}->set_align('top', 'center'); } else { $self->{gdta_values}->set_align('center', 'left'); } } return $self; } sub set_x_label_font # (fontname) { my $self = shift; $self->_set_font('gdta_x_label', @_); } sub set_y_label_font # (fontname) { my $self = shift; $self->_set_font('gdta_y_label', @_); } sub set_x_axis_font # (fontname) { my $self = shift; $self->_set_font('gdta_x_axis', @_); } sub set_y_axis_font # (fontname) { my $self = shift; $self->_set_font('gdta_y_axis', @_); } sub set_values_font { my $self = shift; $self->_set_font('gdta_values', @_); } sub set_legend # List of legend keys { my $self = shift; $self->{legend} = [@_]; } sub set_legend_font # (font name) { my $self = shift; $self->_set_font('gdta_legend', @_); } sub get_hotspot { my $self = shift; my $ds = shift; # Which data set my $np = shift; # Which data point? if (defined $np && defined $ds) { return @{$self->{_hotspots}->[$ds]->[$np]}; } elsif (defined $ds) { return @{$self->{_hotspots}->[$ds]}; } else { return @{$self->{_hotspots}}; } } sub _set_feature_coords { my $self = shift; my $feature = shift; my $type = shift; $self->{_feat_coords}->{$feature} = [ $type, @_ ]; } sub _set_text_feature_coords { my $self = shift; my $feature = shift; $self->_set_feature_coords($feature, "rect", @_[0,1,4,5]); } sub get_feature_coordinates { my $self = shift; my $feature = shift; if ($feature) { $self->{_feat_coords}->{$feature}; } else { $self->{_feat_coords}; } } # PRIVATE # inherit check_data from GD::Graph # # calculate the bottom of the bounding box for the graph # sub setup_bottom_boundary { my $self = shift; $self->{bottom} = $self->{height} - $self->{b_margin} - 1; if (! $self->{rotate_chart}) { # X label $self->{bottom} -= $self->{xlfh} + $self->{text_space} if $self->{xlfh}; # X axis tick labels $self->{bottom} -= $self->{x_label_height} + $self->{axis_space} if $self->{xafh}; } else { # Y1 label $self->{bottom} -= $self->{ylfh} + $self->{text_space} if $self->{y1_label}; # Y1 axis labels $self->{bottom} -= $self->{y_label_height}[1] + $self->{axis_space} if $self->{y_label_height}[1]; } } # # Calculate the top of the bounding box for the graph # sub setup_top_boundary { my $self = shift; $self->{top} = $self->{t_margin}; # Chart title $self->{top} += $self->{tfh} + $self->{text_space} if $self->{tfh}; if (! $self->{rotate_chart}) { # Make sure the text for the y axis tick markers fits on the canvas $self->{top} = $self->{yafh}/2 if $self->{top} == 0; } else { if ($self->{two_axes}) { # Y2 label $self->{top} += $self->{ylfh} + $self->{text_space} if $self->{y2_label}; # Y2 axis labels $self->{top} += $self->{y_label_height}[2] + $self->{axis_space} if $self->{y_label_height}[2]; } } } # # calculate the left of the bounding box for the graph # sub setup_left_boundary { my $self = shift; $self->{left} = $self->{l_margin}; if (! $self->{rotate_chart}) { # Y1 label $self->{left} += $self->{ylfh} + $self->{text_space} if $self->{y1_label}; # Y1 axis labels $self->{left} += $self->{y_label_len}[1] + $self->{axis_space} if $self->{y_label_len}[1]; } else { # X label $self->{left} += $self->{xlfh} + $self->{text_space} if $self->{x_label}; # X axis labels $self->{left} += $self->{x_label_width} + $self->{axis_space} if $self->{x_label_width}; } } # # calculate the right of the bounding box for the graph # sub setup_right_boundary { my $self = shift; $self->{right} = $self->{width} - $self->{r_margin} - 1; if (! $self->{rotate_chart}) { if ($self->{two_axes}) { # Y2 label $self->{right} -= $self->{ylfh} + $self->{text_space} if $self->{y2_label}; # Y2 axis label $self->{right} -= $self->{y_label_len}[2] + $self->{axis_space} if $self->{y_label_len}[2]; } } else { # Adjust right margin to allow last label of y axes. Only do # this when the right margin doesn't have enough space # already. # # TODO Don't assume rightmost label is the same as the # longest label (stored in y_label_len) The worst that can # happen now is that we reserve too much space. my $max_len = $self->{y_label_len}[1]; if ($self->{two_axes}) { $max_len = $self->{y_label_len}[2] if $self->{y_label_len}[2] > $max_len; } $max_len = int ($max_len/2); if ($self->{right} + $max_len >= $self->{width} - $self->{r_margin}) { $self->{right} -= $max_len; } } } sub _setup_boundaries { my $self = shift; $self->setup_bottom_boundary(); $self->setup_top_boundary(); $self->setup_left_boundary(); $self->setup_right_boundary(); if ($self->correct_width && !$self->{x_tick_number}) { if (! $self->{rotate_chart}) { # Make sure we have a nice integer number of pixels $self->{r_margin} += ($self->{right} - $self->{left}) % ($self->{_data}->num_points + 1); $self->setup_right_boundary(); } else { # Make sure we have a nice integer number of pixels $self->{b_margin} += ($self->{bottom} - $self->{top}) % ($self->{_data}->num_points + 1); $self->setup_bottom_boundary(); } } return $self->_set_error('Vertical size too small') if $self->{bottom} <= $self->{top}; return $self->_set_error('Horizontal size too small') if $self->{right} <= $self->{left}; return $self; } # This method should return 1 if the width of the graph needs to be # corrected to whole integers, and 0 if not. The default behaviour is to # not correct the width. Individual classes should override this by # setting the $self->{correct_width} attribute in their initialise # method. Only in complex cases (see mixed.pm) should this method be # overridden sub correct_width { $_[0]->{correct_width} } sub setup_x_step_size_v { my $s = shift; # calculate the step size for x data # CONTRIB Changes by Scott Prahl if (defined $s->{x_tick_number}) { my $delta = ($s->{right} - $s->{left})/($s->{x_max} - $s->{x_min}); # 'True' numerical X axis addition # From: Gary Deschaines if (defined($s->{x_min_value}) && defined($s->{x_max_value})) { $s->{x_offset} = $s->{left}; $s->{x_step} = $delta; } else { $s->{x_offset} = ($s->{true_x_min} - $s->{x_min}) * $delta + $s->{left}; $s->{x_step} = ($s->{true_x_max} - $s->{true_x_min}) * $delta/($s->{_data}->num_points - 1); } } else { $s->{x_step} = ($s->{right} - $s->{left})/($s->{_data}->num_points + 1); $s->{x_offset} = $s->{left}; } } sub setup_x_step_size_h { my $s = shift; # calculate the step size for x data # CONTRIB Changes by Scott Prahl if (defined $s->{x_tick_number}) { my $delta = ($s->{bottom} - $s->{top})/($s->{x_max} - $s->{x_min}); # 'True' numerical X axis addition # From: Gary Deschaines if (defined($s->{x_min_value}) && defined($s->{x_max_value})) { $s->{x_offset} = $s->{top}; $s->{x_step} = $delta; } else { $s->{x_offset} = ($s->{true_x_min} - $s->{x_min}) * $delta + $s->{top}; $s->{x_step} = ($s->{true_x_max} - $s->{true_x_min}) * $delta/($s->{_data}->num_points - 1); } } else { $s->{x_step} = ($s->{bottom} - $s->{top})/($s->{_data}->num_points + 1); $s->{x_offset} = $s->{top}; } } sub setup_coords { my $s = shift; # Do some sanity checks $s->{two_axes} = 0 if $s->{_data}->num_sets < 2 || $s->{two_axes} < 0; $s->{two_axes} = 1 if $s->{two_axes} > 1; delete $s->{y_label2} unless $s->{two_axes}; # Set some heights for text $s->{tfh} = 0 unless $s->{title}; $s->{xlfh} = 0 unless $s->{x_label}; # Make sure the y1 axis has a label if there is one set for y in # general $s->{y1_label} = $s->{y_label} if !$s->{y1_label} && $s->{y_label}; # Set axis tick text heights and widths to 0 if they don't need to # be plotted. $s->{xafh} = 0, $s->{xafw} = 0 unless $s->{x_plot_values}; $s->{yafh} = 0, $s->{yafw} = 0 unless $s->{y_plot_values}; # Calculate minima and maxima for the axes $s->set_max_min() or return; # Create the labels for the axes, and calculate the max length $s->create_y_labels(); $s->create_x_labels(); # CONTRIB Scott Prahl # Calculate the boundaries of the chart $s->_setup_boundaries() or return; # CONTRIB Scott Prahl # make sure that we can generate valid x tick marks undef($s->{x_tick_number}) if $s->{_data}->num_points < 3; undef($s->{x_tick_number}) if !defined $s->{x_max} || !defined $s->{x_min} || $s->{x_max} == $s->{x_min}; $s->{rotate_chart} ? $s->setup_x_step_size_h() : $s->setup_x_step_size_v(); # get the zero axis level my ($zl, $zb) = $s->val_to_pixel(0, 0, 1); $s->{zeropoint} = $s->{rotate_chart} ? $zl : $zb; # More sanity checks $s->{x_label_skip} = 1 if $s->{x_label_skip} < 1; $s->{y_label_skip} = 1 if $s->{y_label_skip} < 1; $s->{y_tick_number} = 1 if $s->{y_tick_number} < 1; return $s; } sub create_y_labels { my $self = shift; # XXX This should really be y_label_width $self->{y_label_len}[$_] = 0 for 1, 2; $self->{y_label_height}[$_] = 0 for 1, 2; for my $t (0 .. $self->{y_tick_number}) { # XXX Ugh, why did I ever do it this way? How bloody obscure. for my $axis (1 .. ($self->{two_axes} + 1)) { my $label = $self->{y_min}[$axis] + $t * ($self->{y_max}[$axis] - $self->{y_min}[$axis]) / $self->{y_tick_number}; $self->{y_values}[$axis][$t] = $label; if (defined $self->{y_number_format}) { $label = ref $self->{y_number_format} eq 'CODE' ? &{$self->{y_number_format}}($label) : sprintf($self->{y_number_format}, $label); } $self->{gdta_y_axis}->set_text($label); my $len = $self->{gdta_y_axis}->get('width'); $self->{y_labels}[$axis][$t] = $label; # TODO Allow vertical y labels $self->{y_label_len}[$axis] = $len if $len > $self->{y_label_len}[$axis]; $self->{y_label_height}[$axis] = $self->{yafh}; } } } sub get_x_axis_label_length { my $self = shift; my @values = $self->{x_tick_number} ? @{$self->{x_values}} : $self->{_data}->x_values; my $maxlen = 0; foreach my $label (@values) { $self->{gdta_x_axis}->set_text($label); my $len = $self->{gdta_x_axis}->get('width'); $maxlen = $len if $maxlen < $len; } return $maxlen; } # CONTRIB Scott Prahl sub create_x_labels { my $self = shift; my $maxlen = 0; $self->{x_label_height} = 0; $self->{x_label_width} = 0; if (defined $self->{x_tick_number}) { # We want to emulate numerical x axes foreach my $t (0..$self->{x_tick_number}) { my $label = $self->{x_min} + $t * ($self->{x_max} - $self->{x_min})/$self->{x_tick_number}; $self->{x_values}[$t] = $label; if (defined $self->{x_number_format}) { $label = ref $self->{x_number_format} eq 'CODE' ? &{$self->{x_number_format}}($label) : sprintf($self->{x_number_format}, $label); } $self->{gdta_x_label}->set_text($label); my $len = $self->{gdta_x_label}->get('width'); $self->{x_labels}[$t] = $label; $maxlen = $len if $len > $self->{x_label_height}; } } else { $maxlen = $self->get_x_axis_label_length; } $self->{x_label_height} = $self->{x_labels_vertical} ? $maxlen : $self->{xafh}; $self->{x_label_width} = $self->{x_labels_vertical} ? $self->{xafh} : $maxlen; } # # The drawing of labels for the axes. This is split up in the four # positions a label can appear in, depending on a few settings. These # settings are all dealt with in the draw_x_labels and draw_y_labels # subroutines, which in turn call the approriate directional label # drawer # sub draw_left_label { my ($self, $label, $align) = @_; $label->set_align('top', 'left'); my $tx = $self->{l_margin}; my $ty = $self->{bottom} - $align * ($self->{bottom} - $self->{top}) + $align * $label->get('width'); $label->draw($tx, $ty, PI/2); } sub draw_bottom_label { my ($self, $label, $align) = @_; $label->set_align('bottom', 'left'); my $tx = $self->{left} + $align * ($self->{right} - $self->{left}) - $align * $label->get('width'); my $ty = $self->{height} - $self->{b_margin}; $label->draw($tx, $ty, 0); } sub draw_top_label { my ($self, $label, $align) = @_; $label->set_align('top', 'left'); my $tx = $self->{left} + $align * ($self->{right} - $self->{left}) - $align * $label->get('width'); my $ty = $self->{t_margin}; $ty += $self->{tfh} + $self->{text_space} if $self->{tfh}; $label->draw($tx, $ty, 0); } sub draw_right_label { my ($self, $label, $align) = @_; $label->set_align('bottom', 'left'); my $tx = $self->{width} - $self->{r_margin}; my $ty = $self->{bottom} - $align * ($self->{bottom} - $self->{top}) + $align * $label->get('width'); $label->draw($tx, $ty, PI/2); } sub draw_x_label { my $self = shift; my ($tx, $ty, $a); my @coords; # coordinates of the label drawn return unless $self->{x_label}; $self->{gdta_x_label}->set_text($self->{x_label}); if ($self->{rotate_chart}) { @coords = $self->draw_left_label($self->{gdta_x_label}, $self->{x_label_position}); } else { @coords = $self->draw_bottom_label($self->{gdta_x_label}, $self->{x_label_position}); } $self->_set_text_feature_coords("x_label", @coords); } sub draw_y_labels { my $self = shift; my @coords; # coordinates of the labels drawn if (defined $self->{y1_label}) { $self->{gdta_y_label}->set_text($self->{y1_label}); if ($self->{rotate_chart}) { @coords = $self->draw_bottom_label($self->{gdta_y_label}, $self->{y_label_position}); } else { @coords = $self->draw_left_label($self->{gdta_y_label}, $self->{y_label_position}); } $self->_set_text_feature_coords("y1_label", @coords); $self->_set_text_feature_coords("y_label", @coords); } if ( $self->{two_axes} && defined $self->{y2_label} ) { $self->{gdta_y_label}->set_text($self->{y2_label}); if ($self->{rotate_chart}) { @coords = $self->draw_top_label($self->{gdta_y_label}, $self->{y_label_position}); } else { @coords = $self->draw_right_label($self->{gdta_y_label}, $self->{y_label_position}); } $self->_set_text_feature_coords("y2_label", @coords); } } sub draw_text { my $self = shift; if ($self->{title}) { my $xc = $self->{left} + ($self->{right} - $self->{left})/2; $self->{gdta_title}->set_align('top', 'center'); $self->{gdta_title}->set_text($self->{title}); my @coords = $self->{gdta_title}->draw($xc, $self->{t_margin}); $self->_set_text_feature_coords("title", @coords); } $self->draw_x_label(); $self->draw_y_labels(); } sub draw_axes { my $self = shift; my ($l, $r, $b, $t) = ( $self->{left}, $self->{right}, $self->{bottom}, $self->{top} ); # Sanity check for zero_axis and zero_axis_only unless ($self->{y_min}[1] < 0 && $self->{y_max}[1] > 0) { $self->{zero_axis} = 0; $self->{zero_axis_only} = 0; } if ( $self->{box_axis} ) { $self->{graph}->filledRectangle($l+1, $t+1, $r-1, $b-1, $self->{boxci}) if $self->{boxci}; $self->{graph}->rectangle($l, $t, $r, $b, $self->{fgci}); } else { $self->{graph}->line($l, $t, $l, $b, $self->{fgci}); $self->{graph}->line($l, $b, $r, $b, $self->{fgci}) unless ($self->{zero_axis_only}); $self->{graph}->line($r, $b, $r, $t, $self->{fgci}) if ($self->{two_axes}); } if ($self->{zero_axis} or $self->{zero_axis_only}) { my ($x, $y) = $self->val_to_pixel(0, 0, 1); $self->{graph}->line($l, $y, $r, $y, $self->{fgci}); } $self->_set_feature_coords("axes", "rect", $l, $b, $r, $t); } # # Ticks and values for y axes # sub draw_y_ticks_h { my $self = shift; for my $t (0 .. $self->{y_tick_number}) { for my $axis (1 .. ($self->{two_axes} + 1)) { my $value = $self->{y_values}[$axis][$t]; my $label = $self->{y_labels}[$axis][$t]; my ($x, $y) = $self->val_to_pixel(0, $value, $axis); $y = ($axis == 1) ? $self->{bottom} : $self->{top}; if ($self->{y_long_ticks}) { $self->{graph}->line( $x, $self->{bottom}, $x, $self->{top}, $self->{fgci} ) unless ($axis-1); } else { $self->{graph}->line( $x, $y, $x, $y - $self->{y_tick_length}, $self->{fgci} ); } next if $t % ($self->{y_label_skip}) || ! $self->{y_plot_values}; $self->{gdta_y_axis}->set_text($label); if ($axis == 1) { $self->{gdta_y_axis}->set_align('top', 'center'); $y += $self->{axis_space}; } else { $self->{gdta_y_axis}->set_align('bottom', 'center'); $y -= $self->{axis_space}; } $self->{gdta_y_axis}->draw($x, $y); } } return $self; } sub draw_y_ticks_v { my $self = shift; for my $t (0 .. $self->{y_tick_number}) { # XXX Ugh, why did I ever do it this way? How bloody obscure. for my $axis (1 .. ($self->{two_axes} + 1)) { my $value = $self->{y_values}[$axis][$t]; my $label = $self->{y_labels}[$axis][$t]; my ($x, $y) = $self->val_to_pixel(0, $value, $axis); $x = ($axis == 1) ? $self->{left} : $self->{right}; if ($self->{y_long_ticks}) { $self->{graph}->line( $x, $y, $x + $self->{right} - $self->{left}, $y, $self->{fgci} ) unless ($axis-1); } else { $self->{graph}->line( $x, $y, $x + (3 - 2 * $axis) * $self->{y_tick_length}, $y, $self->{fgci} ); } next if $t % ($self->{y_label_skip}) || ! $self->{y_plot_values}; $self->{gdta_y_axis}->set_text($label); if ($axis == 1) { $self->{gdta_y_axis}->set_align('center', 'right'); $x -= $self->{axis_space}; } else { $self->{gdta_y_axis}->set_align('center', 'left'); $x += $self->{axis_space}; } $self->{gdta_y_axis}->draw($x, $y); } } return $self; } sub draw_y_ticks { #TODO Clean this up! $_[0]->{rotate_chart} ? goto &draw_y_ticks_h : goto &draw_y_ticks_v; } # # Ticks and values for x axes # sub draw_x_ticks_h { my $self = shift; for (my $i = 0; $i < $self->{_data}->num_points; $i++) { my ($x, $y) = $self->val_to_pixel($i + 1, 0, 1); $x = $self->{left} unless $self->{zero_axis_only}; # CONTRIB Damon Brodie for x_tick_offset next if (!$self->{x_all_ticks} and ($i - $self->{x_tick_offset}) % $self->{x_label_skip} and $i != $self->{_data}->num_points - 1 ); if ($self->{x_ticks}) { if ($self->{x_long_ticks}) { $self->{graph}->line($self->{left}, $y, $self->{right}, $y, $self->{fgci}); } else { $self->{graph}->line( $x, $y, $x + $self->{x_tick_length}, $y, $self->{fgci}); } } # CONTRIB Damon Brodie for x_tick_offset next if ($i - $self->{x_tick_offset}) % ($self->{x_label_skip}) and $i != $self->{_data}->num_points - 1; $self->{gdta_x_axis}->set_text($self->{_data}->get_x($i)); my $angle = 0; if ($self->{x_labels_vertical}) { $self->{gdta_x_axis}->set_align('bottom', 'center'); $angle = PI/2; } else { $self->{gdta_x_axis}->set_align('center', 'right'); } $self->{gdta_x_axis}->draw($x - $self->{axis_space}, $y, $angle); } return $self; } sub draw_x_ticks_v { my $self = shift; for (my $i = 0; $i < $self->{_data}->num_points; $i++) { my ($x, $y) = $self->val_to_pixel($i + 1, 0, 1); $y = $self->{bottom} unless $self->{zero_axis_only}; # CONTRIB Damon Brodie for x_tick_offset next if (!$self->{x_all_ticks} and ($i - $self->{x_tick_offset}) % $self->{x_label_skip} and $i != $self->{_data}->num_points - 1 ); if ($self->{x_ticks}) { if ($self->{x_long_ticks}) { $self->{graph}->line($x, $self->{bottom}, $x, $self->{top}, $self->{fgci}); } else { $self->{graph}->line($x, $y, $x, $y - $self->{x_tick_length}, $self->{fgci}); } } # CONTRIB Damon Brodie for x_tick_offset next if ($i - $self->{x_tick_offset}) % ($self->{x_label_skip}) and $i != $self->{_data}->num_points - 1; $self->{gdta_x_axis}->set_text($self->{_data}->get_x($i)); my $angle = 0; if ($self->{x_labels_vertical}) { $self->{gdta_x_axis}->set_align('center', 'right'); $angle = PI/2; } else { $self->{gdta_x_axis}->set_align('top', 'center'); } $self->{gdta_x_axis}->draw($x, $y + $self->{axis_space}, $angle); } return $self; } sub draw_x_ticks { #TODO Clean this up! $_[0]->{rotate_chart} ? goto &draw_x_ticks_h : goto &draw_x_ticks_v; } # CONTRIB Scott Prahl # Assume x array contains equally spaced x-values # and generate an appropriate axis # #### # 'True' numerical X axis addition # From: Gary Deschaines # # These modification to draw_x_ticks_number pass x-tick values to the # val_to_pixel subroutine instead of x-tick indices when ture numerical # x-axis mode is detected. Also, x_tick_offset and x_label_skip are # processed differently when true numerical x-axis mode is detected to # allow labeled major x-tick marks and un-labeled minor x-tick marks. # # For example: # # x_tick_number => 14, # x_ticks => 1, # x_long_ticks => 1, # x_tick_length => -4, # x_min_value => 100, # x_max_value => 800, # x_tick_offset => 2, # x_label_skip => 2, # # # ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ # | | | | | | | | | | | | | # 1 -| | | | | | | | | | | | | # | | | | | | | | | | | | | # 0 _|_________|____|____|____|____|____|____|____|____|____|____|_________| # | | | | | | | | | | | # 200 300 400 500 600 700 sub draw_x_ticks_number { my $self = shift; for my $i (0 .. $self->{x_tick_number}) { my ($value, $x, $y); if (defined($self->{x_min_value}) && defined($self->{x_max_value})) { next if ($i - $self->{x_tick_offset}) < 0; next if ($i + $self->{x_tick_offset}) > $self->{x_tick_number}; $value = $self->{x_values}[$i]; ($x, $y) = $self->val_to_pixel($value, 0, 1); } else { $value = ($self->{_data}->num_points - 1) * ($self->{x_values}[$i] - $self->{true_x_min}) / ($self->{true_x_max} - $self->{true_x_min}); ($x, $y) = $self->val_to_pixel($value + 1, 0, 1); } $y = $self->{bottom} unless $self->{zero_axis_only}; if ($self->{x_ticks}) { if ($self->{x_long_ticks}) { # XXX This mod needs to be done everywhere ticks are # drawn if ( $self->{x_tick_length} >= 0 ) { $self->{graph}->line($x, $self->{bottom}, $x, $self->{top}, $self->{fgci}); } else { $self->{graph}->line( $x, $self->{bottom} - $self->{x_tick_length}, $x, $self->{top}, $self->{fgci}); } } else { $self->{graph}->line($x, $y, $x, $y - $self->{x_tick_length}, $self->{fgci} ); } } # If we have to skip labels, we'll do it here. # Make sure to always draw the last one. next if $i % $self->{x_label_skip} && $i != $self->{x_tick_number}; $self->{gdta_x_axis}->set_text($self->{x_labels}[$i]); if ($self->{x_labels_vertical}) { $self->{gdta_x_axis}->set_align('center', 'right'); my $yt = $y + $self->{text_space}/2; $self->{gdta_x_axis}->draw($x, $yt, PI/2); } else { $self->{gdta_x_axis}->set_align('top', 'center'); my $yt = $y + $self->{text_space}/2; $self->{gdta_x_axis}->draw($x, $yt); } } return $self; } sub draw_ticks { my $self = shift; $self->draw_y_ticks() or return; return $self unless $self->{x_plot_values}; if (defined $self->{x_tick_number}) { $self->draw_x_ticks_number() or return; } else { $self->draw_x_ticks() or return; } return $self; } sub draw_data { my $self = shift; # Calculate bar_spacing from bar_width if ($self->{bar_width}) { my $chart_width = $self->{rotate_chart} ? $self->{right} - $self->{left} : $self->{bottom} - $self->{top}; my $n_bars = $self->{_data}->num_points; my $n_sets = $self->{_data}->num_sets; my $bar_space = $chart_width/($n_bars + 1) / ($self->{overwrite} ? 1 : $n_sets); $self->{bar_spacing} = $bar_space - $self->{bar_width}; $self->{bar_spacing} = 0 if $self->{bar_spacing} < 0; } # XXX is this comment still pertinent? # The drawing of 'cumulated' sets needs to be done in reverse, # for area and bar charts. This is mainly because of backward # compatibility for (my $dsn = 1; $dsn <= $self->{_data}->num_sets; $dsn++) { $self->draw_data_set($dsn) or return; } return $self } # # Draw the values of the data point with the bars, lines or markers sub draw_values { my $self = shift; return $self unless $self->{show_values}; my $text_angle = $self->{values_vertical} ? PI/2 : 0; for (my $dsn = 1; $dsn <= $self->{_data}->num_sets; $dsn++) { my @values = $self->{_data}->y_values($dsn) or return $self->_set_error("Impossible illegal data set: $dsn", $self->{_data}->error); my @display = $self->{show_values}->y_values($dsn) or next; for (my $i = 0; $i < @values; $i++) { next unless defined $display[$i]; my ($xp, $yp); if (defined($self->{x_min_value}) && defined($self->{x_max_value})) { ($xp, $yp) = $self->val_to_pixel( $self->{_data}->get_x($i), $values[$i], $dsn); } else { ($xp, $yp) = $self->val_to_pixel($i+1, $values[$i], $dsn); } $yp -= $self->{values_space}; my $value = $display[$i]; if (defined $self->{values_format}) { $value = ref $self->{values_format} eq 'CODE' ? &{$self->{values_format}}($value) : sprintf($self->{values_format}, $value); } $self->{gdta_values}->set_text($value); $self->{gdta_values}->draw($xp, $yp, $text_angle); } } return $self } # # draw_data_set is in sub classes # sub draw_data_set { # ABSTRACT my $self = shift; $self->die_abstract( "sub draw_data missing, ") } # # This method corrects the minimum and maximum y values for chart # types that need to always include a zero point. # This is supposed to be called before the methods that pick # good-looking values. # # Input: current minimum and maximum. # Output: new minimum and maximum. # sub _correct_y_min_max { my $self = shift; my ($min, $max) = @_; # Make sure bars and area always have a zero offset # Only bars and areas need return ($min, $max) unless $self->isa("GD::Graph::bars") or $self->isa("GD::Graph::area"); # If either $min or $max are 0, we can return return ($min, $max) if $max == 0 or $min == 0; # If $min and $max on opposite end of zero axis, no work needed return ($min, $max) unless $min/$max > 0; if ($min > 0) { $min = 0; } else { $max = 0; } return ($min, $max); } # # Figure out the maximum values for the vertical exes, and calculate # a more or less sensible number for the tops. # sub set_max_min { my $self = shift; # XXX fix to calculate min and max for each data set # independently, and store in an array. Then, based on use_axis, # pick the minimust and maximust for each axis, and use those. # First, calculate some decent values if ( $self->{two_axes} ) { my $min_range_1 = defined($self->{min_range_1}) ? $self->{min_range_1} : $self->{min_range}; my $min_range_2 = defined($self->{min_range_2}) ? $self->{min_range_2} : $self->{min_range}; ( $self->{y_min}[1], $self->{y_max}[1], $self->{y_min}[2], $self->{y_max}[2], $self->{y_tick_number} ) = _best_dual_ends( $self->_correct_y_min_max($self->{_data}->get_min_max_y(1)), $min_range_1, $self->_correct_y_min_max($self->{_data}->get_min_max_y(2)), $min_range_2, $self->{y_tick_number} ); } else { my ($y_min, $y_max); if ($self->{cumulate}) { my $data_set = $self->{_data}->copy(); $data_set->cumulate; ($y_min, $y_max) = $data_set->get_min_max_y($data_set->num_sets); } else { ($y_min, $y_max) = $self->{_data}->get_min_max_y_all; } ($y_min, $y_max) = $self->_correct_y_min_max($y_min, $y_max); ($self->{y_min}[1], $self->{y_max}[1], $self->{y_tick_number}) = _best_ends($y_min, $y_max, @$self{'y_tick_number','y_min_range'}); } if (defined($self->{x_tick_number})) { if (defined($self->{x_min_value}) && defined($self->{x_max_value})) { $self->{true_x_min} = $self->{x_min_value}; $self->{true_x_max} = $self->{x_max_value}; } else { ($self->{true_x_min}, $self->{true_x_max}) = $self->{_data}->get_min_max_x; ($self->{x_min}, $self->{x_max}, $self->{x_tick_number}) = _best_ends($self->{true_x_min}, $self->{true_x_max}, @$self{'y_tick_number','y_min_range'}); } } # Overwrite these with any user supplied ones $self->{y_min}[1] = $self->{y_min_value} if defined $self->{y_min_value}; $self->{y_min}[2] = $self->{y_min_value} if defined $self->{y_min_value}; $self->{y_max}[1] = $self->{y_max_value} if defined $self->{y_max_value}; $self->{y_max}[2] = $self->{y_max_value} if defined $self->{y_max_value}; $self->{y_min}[1] = $self->{y1_min_value} if defined $self->{y1_min_value}; $self->{y_max}[1] = $self->{y1_max_value} if defined $self->{y1_max_value}; $self->{y_min}[2] = $self->{y2_min_value} if defined $self->{y2_min_value}; $self->{y_max}[2] = $self->{y2_max_value} if defined $self->{y2_max_value}; $self->{x_min} = $self->{x_min_value} if defined $self->{x_min_value}; $self->{x_max} = $self->{x_max_value} if defined $self->{x_max_value}; if ($self->{two_axes}) { # If we have two axes, we need to make sure that the zero is at # the same spot. # And we need to change the number of ticks on the axes my $l_range = $self->{y_max}[1] - $self->{y_min}[1]; my $r_range = $self->{y_max}[2] - $self->{y_min}[2]; my $l_top = $self->{y_max}[1]/$l_range; my $r_top = $self->{y_max}[2]/$r_range; my $l_bot = $self->{y_min}[1]/$l_range; my $r_bot = $self->{y_min}[2]/$r_range; if ($l_top > $r_top) { $self->{y_max}[2] = $l_top * $r_range; $self->{y_min}[1] = $r_bot * $l_range; $self->{y_tick_number} *= 1 + abs $r_bot - $l_bot; } else { $self->{y_max}[1] = $r_top * $l_range; $self->{y_min}[2] = $l_bot * $r_range; $self->{y_tick_number} *= 1 + abs $r_top - $l_top; } } # Check to see if we have sensible values if ($self->{two_axes}) { for my $i (1 .. 2) { my ($min, $max) = $self->{_data}->get_min_max_y($i); return $self->_set_error("Minimum for y" . $i . " too large") if $self->{y_min}[$i] > $min; return $self->_set_error("Maximum for y" . $i . " too small") if $self->{y_max}[$i] < $max; } } return $self; } # CONTRIB Scott Prahl # # Calculate best endpoints and number of intervals for an axis and # returns ($nice_min, $nice_max, $n), where $n is the number of # intervals and # # $nice_min <= $min < $max <= $nice_max # # Usage: # ($nmin,$nmax,$nint) = _best_ends(247, 508); # ($nmin,$nmax) = _best_ends(247, 508, 5); # use 5 intervals # ($nmin,$nmax,$nint) = _best_ends(247, 508, [4..7]); # best of 4,5,6,7 intervals # ($nmin,$nmax,$nint) = _best_ends(247, 508, 'auto'); # best of 3,4,5,6 intervals # ($nmin,$nmax,$nint) = _best_ends(247, 508, [2..5]); # best of 2,3,4,5 intervals sub _best_ends { my ($min, $max, $n_ref, $min_range) = @_; # Adjust for the min range if need be ($min, $max) = _fit_vals_range($min, $max, $min_range); my ($best_min, $best_max, $best_num) = ($min, $max, 1); # mgjv - Sometimes, for odd values, and only one data set, this will be # necessary _after_ the previous step, not before. Data sets of one # long with negative values were causing infinite loops later on. ($min, $max) = ($max, $min) if ($min > $max); # Check that min and max are not the same, and not 0 ($min, $max) = ($min) ? ($min * 0.5, $min * 1.5) : (-1,1) if ($max == $min); my @n = ref($n_ref) ? @$n_ref : $n_ref; if (@n <= 0) { @n = (3..6); } else { @n = map { ref($_) ? @$_ : /(\d+)/i ? $1 : (3..6) } @n; } my $best_fit = 1e30; my $range = $max - $min; # create array of interval sizes my $s = 1; while ($s < $range) { $s *= 10 } while ($s > $range) { $s /= 10 } my @step = map {$_ * $s} (0.2, 0.5, 1, 2, 5); for my $n (@n) { # Try all numbers of intervals next if ($n < 1); for my $step (@step) { next if ($n != 1) && ($step < $range/$n); # $step too small my ($nice_min, $nice_max, $fit) = _fit_interval($min, $max, $n, $step); next if $best_fit <= $fit; $best_min = $nice_min; $best_max = $nice_max; $best_fit = $fit; $best_num = $n; } } return ($best_min, $best_max, $best_num) } # CONTRIB Ben Tilly # # Calculate best endpoints and number of intervals for a pair of axes # where it is trying to line up the scale of the two intervals. It # returns ($nice_min_1, $nice_max_1, $nice_min_2, $nice_max_2, $n), # where $n is the number of intervals and # # $nice_min_1 <= $min_1 < $max_1 <= $nice_max_1 # $nice_min_2 <= $min_2 < $max_2 <= $nice_max_2 # # and 0 will appear at the same point on both axes. # # Usage: # ($nmin_1,$nmax_1,$nmin_2,$nmax_2,$nint) = _best_dual_ends(247, 508, undef, -1, 5, undef, [2..5]); # etc. (The usage of the last arguments just parallels _best_ends.) # sub _best_dual_ends { my ($min_1, $max_1) = _fit_vals_range(splice @_, 0, 3); my ($min_2, $max_2) = _fit_vals_range(splice @_, 0, 3); my @rem_args = @_; # Fix the situation where both min_1 and max_1 are 0, which makes it # loop forever ($min_1, $max_1) = (0, 1) unless $min_1 or $max_1; my $scale_1 = _max(abs($min_1), abs($max_1)); my $scale_2 = _max(abs($min_2), abs($max_2)); $scale_1 = defined($scale_2) ? $scale_2 : 1 unless defined($scale_1); $scale_2 = $scale_1 unless defined($scale_2); my $ratio = $scale_1 / ($scale_2 || 1); my $fact_1 = my $fact_2 = 1; while ($ratio < sqrt(0.1)) { $ratio *= 10; $fact_2 *= 10; } while ($ratio > sqrt(10)) { $ratio /= 10; $fact_1 *= 10; } my ($best_min_1, $best_max_1, $best_min_2, $best_max_2, $best_n, $best_fit) = ($min_1, $max_1, $min_2, $max_2, 1, 1e10); # Now try all of the ratios of "simple numbers" in the right size-range foreach my $frac ( [1,1], [1,2], [1,3], [2,1], [2,3], [2,5], [3,1], [3,2], [3,4], [3,5], [3,8], [3,10], [4,3], [4,5], [5,2], [5,3], [5,4], [5,6], [5,8], [6,5], [8,3], [8,5], [10,3] ) { my $bfact_1 = $frac->[0] * $fact_1; my $bfact_2 = $frac->[1] * $fact_2; my $min = _min( $min_1/$bfact_1, $min_2/$bfact_2 ); my $max = _max( $max_1/$bfact_1, $max_2/$bfact_2 ); my ($bmin, $bmax, $n) = _best_ends($min, $max, @rem_args); my ($bmin_1, $bmax_1) = ($bfact_1*$bmin, $bfact_1*$bmax); my ($bmin_2, $bmax_2) = ($bfact_2*$bmin, $bfact_2*$bmax); my $fit = _measure_interval_fit($bmin_1, $min_1, $max_1, $bmax_1) + _measure_interval_fit($bmin_2, $min_2, $max_2, $bmax_2); next if $best_fit < $fit; ( $best_min_1, $best_max_1, $best_min_2, $best_max_2, $best_n, $best_fit ) = ( $bmin_1, $bmax_1, $bmin_2, $bmax_2, $n, $fit ); } return ($best_min_1, $best_max_1, $best_min_2, $best_max_2, $best_n); } # Takes $min, $max, $step_count, $step_size. Assumes $min <= $max and both # $step_count and $step_size are positive. Returns the fitted $min, $max, # and a $fit statistic (where smaller is better). Failure to fit the # interval results in a poor fit statistic. :-) sub _fit_interval { my ($min, $max, $step_count, $step_size) = @_; my $nice_min = $step_size * int($min/$step_size); $nice_min -= $step_size if ($nice_min > $min); my $nice_max = ($step_count == 1) ? $step_size * int($max/$step_size + 1) : $nice_min + $step_count * $step_size; my $fit = _measure_interval_fit($nice_min, $min, $max, $nice_max); # Prevent division by zero errors further up return ($min, $max, 0) if ($step_size == 0); return ($nice_min, $nice_max, $fit); } # Takes 2 values and a minimum range. Returns a min and max which holds # both values and is at least that minimum size sub _fit_vals_range { my ($min, $max, $min_range) = @_; ($min, $max) = ($max, $min) if $max < $min; if (defined($min_range) and $min_range > $max - $min) { my $nice_min = $min_range * int($min/$min_range); $nice_min = $nice_min - $min_range if $min < $nice_min; my $nice_max = $max < $nice_min + $min_range ? $nice_min + $min_range : $max; ($min, $max) = ($nice_min, $nice_max); } return ($min, $max); } # Takes $bmin, $min, $max, $bmax and returns a fit statistic for how well # ($bmin, $bmax) encloses the interval ($min, $max). Smaller is better, # and failure to fit will be a very bad fit. Assumes that $min <= $max # and $bmin < $bmax. sub _measure_interval_fit { my ($bmin, $min, $max, $bmax) = @_; return 1000 if $bmin > $min or $bmax < $max; my $range = $max - $min; my $brange = $bmax - $bmin; return $brange < 10 * $range ? ($brange / $range) : 10; } sub _get_bottom { my $self = shift; my ($ds, $np) = @_; my $bottom = $self->{zeropoint}; if ($self->{cumulate} && $ds > 1) { my $left; my $pvalue = $self->{_data}->get_y_cumulative($ds - 1, $np); ($left, $bottom) = $self->val_to_pixel($np + 1, $pvalue, $ds); $bottom = $left if $self->{rotate_chart}; } return $bottom; } # # Convert value coordinates to pixel coordinates on the canvas. # TODO Clean up all the rotate_chart stuff # sub val_to_pixel # ($x, $y, $i) in real coords ($Dataspace), { # return [x, y] in pixel coords my $self = shift; my ($x, $y, $i) = @_; # XXX use_axis my $y_min = ($self->{two_axes} && $i == 2) ? $self->{y_min}[2] : $self->{y_min}[1]; my $y_max = ($self->{two_axes} && $i == 2) ? $self->{y_max}[2] : $self->{y_max}[1]; my $y_step = $self->{rotate_chart} ? abs(($self->{right} - $self->{left})/($y_max - $y_min)) : abs(($self->{bottom} - $self->{top})/($y_max - $y_min)); my $ret_x; my $origin = $self->{rotate_chart} ? $self->{top} : $self->{left}; if (defined($self->{x_min_value}) && defined($self->{x_max_value})) { $ret_x = $origin + ($x - $self->{x_min}) * $self->{x_step}; } else { $ret_x = ($self->{x_tick_number} ? $self->{x_offset} : $origin) + $x * $self->{x_step}; } my $ret_y = $self->{rotate_chart} ? $self->{left} + ($y - $y_min) * $y_step : $self->{bottom} - ($y - $y_min) * $y_step; return $self->{rotate_chart} ? (_round($ret_y), _round($ret_x)) : (_round($ret_x), _round($ret_y)); } # # Legend # sub setup_legend { my $self = shift; return unless defined $self->{legend}; my $maxlen = 0; my $num = 0; # Save some variables $self->{r_margin_abs} = $self->{r_margin}; $self->{b_margin_abs} = $self->{b_margin}; foreach my $legend (@{$self->{legend}}) { if (defined($legend) and $legend ne "") { $self->{gdta_legend}->set_text($legend); my $len = $self->{gdta_legend}->get('width'); $maxlen = ($maxlen > $len) ? $maxlen : $len; $num++; } last if $num >= $self->{_data}->num_sets; } $self->{lg_num} = $num; # calculate the height and width of each element my $legend_height = _max($self->{lgfh}, $self->{legend_marker_height}); $self->{lg_el_width} = $maxlen + $self->{legend_marker_width} + 3 * $self->{legend_spacing}; $self->{lg_el_height} = $legend_height + 2 * $self->{legend_spacing}; my ($lg_pos, $lg_align) = split(//, $self->{legend_placement}); if ($lg_pos eq 'R') { # Always work in one column $self->{lg_cols} = 1; $self->{lg_rows} = $num; # Just for completeness, might use this in later versions $self->{lg_x_size} = $self->{lg_cols} * $self->{lg_el_width}; $self->{lg_y_size} = $self->{lg_rows} * $self->{lg_el_height}; # Adjust the right margin for the rest of the graph $self->{r_margin} += $self->{lg_x_size}; # Set the x starting point $self->{lg_xs} = $self->{width} - $self->{r_margin}; # Set the y starting point, depending on alignment if ($lg_align eq 'T') { $self->{lg_ys} = $self->{t_margin}; } elsif ($lg_align eq 'B') { $self->{lg_ys} = $self->{height} - $self->{b_margin} - $self->{lg_y_size}; } else # default 'C' { my $height = $self->{height} - $self->{t_margin} - $self->{b_margin}; $self->{lg_ys} = int($self->{t_margin} + $height/2 - $self->{lg_y_size}/2) ; } } else # 'B' is the default { # What width can we use my $width = $self->{width} - $self->{l_margin} - $self->{r_margin}; (!defined($self->{lg_cols})) and $self->{lg_cols} = int($width/$self->{lg_el_width}); $self->{lg_cols} = _min($self->{lg_cols}, $num); $self->{lg_rows} = int($num / $self->{lg_cols}) + (($num % $self->{lg_cols}) ? 1 : 0); $self->{lg_x_size} = $self->{lg_cols} * $self->{lg_el_width}; $self->{lg_y_size} = $self->{lg_rows} * $self->{lg_el_height}; # Adjust the bottom margin for the rest of the graph $self->{b_margin} += $self->{lg_y_size}; # Set the y starting point $self->{lg_ys} = $self->{height} - $self->{b_margin}; # Set the x starting point, depending on alignment if ($lg_align eq 'R') { $self->{lg_xs} = $self->{width} - $self->{r_margin} - $self->{lg_x_size}; } elsif ($lg_align eq 'L') { $self->{lg_xs} = $self->{l_margin}; } else # default 'C' { $self->{lg_xs} = int($self->{l_margin} + $width/2 - $self->{lg_x_size}/2); } } } sub draw_legend { my $self = shift; return unless defined $self->{legend}; my $xl = $self->{lg_xs} + $self->{legend_spacing}; my $y = $self->{lg_ys} + $self->{legend_spacing} - 1; my $i = 0; my $row = 1; my $x = $xl; # start position of current element foreach my $legend (@{$self->{legend}}) { $i++; last if $i > $self->{_data}->num_sets; my $xe = $x; # position within an element next unless defined($legend) && $legend ne ""; $self->draw_legend_marker($i, $xe, $y); $xe += $self->{legend_marker_width} + $self->{legend_spacing}; my $ys = int($y + $self->{lg_el_height}/2 - $self->{lgfh}/2); $self->{gdta_legend}->set_text($legend); $self->{gdta_legend}->draw($xe, $ys); $x += $self->{lg_el_width}; if (++$row > $self->{lg_cols}) { $row = 1; $y += $self->{lg_el_height}; $x = $xl; } } } # # This will be virtual; every sub class should define their own # if this one doesn't suffice # sub draw_legend_marker # data_set_number, x, y { my $s = shift; my $n = shift; my $x = shift; my $y = shift; my $g = $s->{graph}; my $ci = ($n<=2) ? $s->set_clr($s->pick_data_clr($n)) : $s->set_clr($s->_darken($s->_darken($s->pick_data_clr($n)))); return unless defined $ci; $y += int($s->{lg_el_height}/2 - $s->{legend_marker_height}/2); $g->filledRectangle( $x, $y, $x + $s->{legend_marker_width}, $y + $s->{legend_marker_height}, $ci ); $g->rectangle( $x, $y, $x + $s->{legend_marker_width}, $y + $s->{legend_marker_height}, $s->{acci} ); } "Just another true value"; calamaris-2.99.4.0/TODO0000644000175000001440000000347210407330372013254 0ustar cordusersWhat will happen next? ---------------------- I think that Calamaris v3 is nearly finished. (except for bugs, that maybe were not found yet.) But if you have an idea what is still missing in a software for parsing proxy log-files, let me know. --> . I'll will build it in, or add it to the wish-list below :-) * adding a Makefile or some other installation-routine, so fixing lib-paths isn't nessecary anymore. (Help wanted) * I suggest that the '-P' option has a option, like all other analyses. So the time step would be something like = / . (Ahmad Kamal ) Comment: As I cumulate the Performance-Data in the Mainloop, when i don't know which timerange i get, i don't have at the time i'd need it, so i don't think that i can implement this without a major rewrite. * rewrite peak-measurement. The new calculation method is very time intensive and slows down Calamaris by 30 or more percent, but it is faster than the old way. However: i'm not really satisfied with it, so i put it out of the -a -option. You'll have to add a '-p (old|new)' -option to get old or new peak-statistic. HELP REQUEST: If someone has an idea how to build an efficient AND fast method to work it out... let me know! * try 'use integer'. This can result in a less memory-hungry, but faster version of Calamaris. (idea by Gerold Meerkoetter) * make Calamaris faster. see above. If someone wants to rewrite Calamaris in a faster language: Feel Free! (But respect the GNU-License) It would be nice if you drop me a line about it, I'll mention it in the README. And please please please don't use the name 'Calamaris' for it without asking me! Version of the TODO ------------------- $Id: TODO,v 3.1 2006-03-19 16:11:25 cord Exp $ calamaris-2.99.4.0/README0000644000175000001440000000664310407330372013447 0ustar cordusersCalamaris Version 3 What is it? ----------- Calamaris is a Perl script, which was first intended as demo for a statistical software for Squid. I started it at 13 January 1997 (Version 1.1) as a rewrite of my old Squid-Analysis-Script weekly.pl (which was in German language). I announced it (Version 1.16) to the public at 28 Feb 1997. (see http://www.squid-cache.org/mail-archive/squid-users/199702/0551.html for the Original-Announcement) Since then it is used by people all around the world, and i decided to build a new improved version of it. Calamaris V2 was a nearly complete rewrite, with changed and more reports. 2004 Michael Pophal send me a jumbo-patch against Calamaris 2.59, which adds two more reports, and the long wanted graphics to make your mangle^H^H^Hagement happy. Which software can produce Calamaris-parseable Log-files? --------------------------------------------------------- * Squid V1.1.alpha1-V2.x (http://www.squid-cache.org/) * NetCache V??? (http://www.netapp.com/products/netcache/) * Inktomi Traffic Server V??? (http://www.inktomi.com/products/network/products/) * Oops! proxy server V??? (http://zipper.paco.net/~igor/oops.eng/) * Compaq Tasksmart (http://www.compaq.com/tasksmart/) * Novell Internet Caching System (http://www.novell.com/products/ics/) * Netscape/iPlanet/SunONE Web Proxy Server (http://www.iplanet.com/downloads/download/detail_14_13.html) * Squid with the SmartFilter-patch * Cisco Content Engines (http://www.cisco.com/en/US/products/hw/contnetw/index.html) Where to get Calamaris? ----------------------- The Calamaris-Home-page is located at http://Calamaris.Cord.de/ There is also an Announcement-Mailing-list. To subscribe send mail with 'subscribe your@mail.adr.ess' in the Mail-Body to . Subscribers will get a mail on every new release, including a list of the changes. --> low traffic. Philipp Frauenfelder has build a Debian Package, which can be found via http://packages.debian.org/calamaris . There is a port for FreeBSD, which can be found at http://www.freebsd.org/cgi/ports.cgi?query=calamaris . A package for NetBSD is here: ftp://ftp.netbsd.org/pub/NetBSD/packages/pkgsrc/www/calamaris/README.html rpms are also available from various people. You can search for them via http://rpmfind.net/linux/rpm2html/search.php?query=calamaris . Is there anything else? ----------------------- Ernst Heiri has build a spin-off of my Calamaris V1, which can be found *where?* There is also a C++-port of Ernst Heiri's Calamaris available. It is (according to the author Jens-S. Voeckler ) five times faster than the Perl-variant. check http://www.cache.dfn.de/DFN-Cache/Development/seafood.html for this. more Squid-logfile-Analysers can be found via the Squid-Home-page at http://www.squid-cache.org/Scripts/ Thank You! ---------- * The developers and contributors of Squid. * The developers and contributors of Perl. * The contributors, feature requesters and bug-reporters of Calamaris. * Gerold 'Nimm Perl' Meerkoetter. * Massimo Carnevali * Michael Pophal Not happy yet? -------------- Drop me a line to and tell me what is missing or wrong or not clear or whatever. You are welcome (especially if you read this file that far :-) Version of the README --------------------- $Id: README,v 3.0 2004/09/15 20:46:31 cord Exp $ calamaris-2.99.4.0/INSTALL0000644000175000001440000000555610407330372013622 0ustar cordusersCalamaris Version 3 How to use it? -------------- * You'll need Perl Version 5 (see http://www.Perl.com/). Calamaris is reported to work with Perl 5.001 (maybe you have to remove the '-w' from the first line and comment out the 'use vars'-line), but it is highly recommended (especially for security of your computer) that you use a recent version (>=5.8.4) of it. * You'll also need one of the noted log-files: + Squid V1.1.alpha1-V1.1.beta25 Native log-files + Squid V1.1.beta26-V2.x Native log-files + Squid V1.1.beta26-V2.x Native log-files with log_mime_hdrs enabled + NetCache V??? Squid-style Log-files + NetCache V5.x Default Log-file-Format (Extended Log-file-Format) + Inktomi Traffic Server V??? Log-files + OOPS V??? Native log-files + Extended Log-file-Format + NetApp Default Log-file-Format (some kind of Extended Log-file-Format) + NetApps understanding of Squid Native log-files + Squid with SmartFilter-Patch Log-files + Cisco Content Engines If Calamaris can't parse the input, check your log-file format. + Squid-Log-files: http://www.squid-cache.org/Doc/FAQ/FAQ-6.html + Extended Log-file-Format: http://www.w3.org/TR/WD-logfile * You'll need GD::Graph package to get graphical html output. For some features you need also NetAddr::IP. Calamaris runs without both perl packages, but most of the new V3.x features won't work without them. * Installation: + get GD::Graph and NetAddr::IP from http://www.cpan.org and install them. (if your system has packages for these available, you should use them.) + cd /usr/local + tar -xzvf /path/to/calamaris-3.0.xxx.tgz + ln -s /usr/local/calamaris-3.0.xxx /usr/local/calamaris + Note: calamaris looks in /usr/local/ for needed calamaris perl modules. If you want to change this path, you have to change the 'use lib' directive in the calamaris code and perl modules. (still looking for a nicier solution for this, suggestions/patches welcome) + Maybe (if your Perl isn't located at /usr/bin/perl) you'll have to change the first line of Calamaris to point to your copy of Perl. + There is also a man-page for Calamaris. You should copy it to an appropriate place like /usr/local/man/man1, where your man(1) can find it. * Use it! 'cat access.log.1 access.log.0 | /usr/local/calamaris/calamaris' Calamaris by default generates by a brief ASCII report of incoming and outgoing requests. NOTE: If you pipe more than one log-file into Calamaris, make sure that they are chronologically ordered (oldest file first), else some reports can return wrong values. You can alter Calamaris' behaviour with switches. Start Calamaris with '-h' or check the man-page. You should also take a look at the EXAMPLES-File, for 'Real-Life'-usage-examples of Calamaris. Version of the INSTALL --------------------- $Id: INSTALL,v 3.1 2004/12/23 20:01:03 cord Exp $ calamaris-2.99.4.0/EXAMPLES.v30000644000175000001440000001012110407330372014241 0ustar cordusersI want to give the 'audience' some examples for using Calamaris. So if you build some Scripts, crontabs or else around Calamaris, please mail and describe them to . I'll probably add them to this file. Thank You. Pavel Malakhov first sends in a new Example. ----------------------------------------------------------------------- calam_rep.sh ============ #!/bin/bash # Script for calamaris 3 to generate reports for squid # # "Usage: calam_rep.sh [today|yesterday|week|month]" # # Pavel Malakhov 28.03.05 # 12.05.05 fixed yesterday date format, new vars CALAM_DIR='/etc/calamaris'; # where calamaris is SQUID_LOG_DIR='/usr/local/squid/var/logs'; # where the squid logs are # where to store reports. Root dir, all the other will be created by this script REP_PATH_PREFIX='/var/www/html/reports/squid'; CACHE_DIR='/etc/calamaris/cache'; # where to store cache files for every week YESTD=`date -d yesterday +%d`; # yesterday's day of month YESTW=`date -d yesterday +%V`; # yesterday's week of year YESTM=`date -d yesterday +%m_%B`; # yesterday's month number and name # ------------ You don't need to edit anything below --------------------------- # ------------------ [but welcome to analyze!] --------------------------------- # Check for dir for report.Create, if does not exist. function checkdir { if [ ! -e "$REPPATH" ];then echo -n `date +%c` "Dir \"$REPPATH\" is not created. Creating... "; mkdir -p $REPPATH; echo "Done."; fi; } # Check for parameter if [ "$1" = "" ]; then echo "Usage: calam_rep.sh [today|yesterday|week|month]" exit 1 fi # check for cache dir if [ ! -e "$CACHE_DIR" ];then echo -n `date +%c` "Cache dir \"$CACHE_DIR\" is not created. Creating... "; mkdir -p $CACHE_DIR; echo "Done."; fi; case "$1" in "today" ) REPPATH=$REP_PATH_PREFIX'/today'; checkdir; cd $CALAM_DIR; echo -n `date +%c` "Processing data for today... "; cat $SQUID_LOG_DIR/access.log | ./calamaris.pl --config-file ./calamaris.conf --output-path $REPPATH; ;; "yesterday" ) REPPATH=$REP_PATH_PREFIX'/days/'$YESTD; checkdir; cd $CALAM_DIR; echo -n `date +%c` "Processing data for yesterday... "; cat $SQUID_LOG_DIR/access.log.0 | ./calamaris.pl --config-file ./calamaris.conf --output-path $REPPATH --cache-output-file $CACHE_DIR/day.$YESTD; ;; "week" ) REPPATH=$REP_PATH_PREFIX'/weeks/'$YESTW; checkdir; cd $SQUID_LOG_DIR; echo -n `date +%c` "Processing data for week... "; cat access.log.6 access.log.5 access.log.4 access.log.3 access.log.2 access.log.1 access.log.0 | $CALAM_DIR/calamaris.pl --config-file $CALAM_DIR/calamaris.conf --output-path $REPPATH --cache-output-file $CACHE_DIR/week.$YESTW; ;; "month" ) REPPATH=$REP_PATH_PREFIX'/months/'$YESTM; checkdir; cd $CACHE_DIR; CACHEFILES=""; for ((i=1; i<=31; i++)); do FILE='day.'$i; if [ -e "$FILE" ]; then if ["$CACHEFILES" = ""]; then CACHEFILES=$FILE; else CACHEFILES=$CACHEFILES':'$FILE; fi fi done echo 'files to process '$CACHEFILES; echo -n `date +%c` "Processing data for month... "; $CALAM_DIR/calamaris.pl --config-file $CALAM_DIR/calamaris.conf --cache-input-file $CACHEFILES --no-input --output-path $REPPATH; echo "Done"; # clean up cache directory at the start of a month # delete only cached days, leave cached weeks DD=`date +%d`; if [ "$DD" = "01" ]; then echo -n `date +%c` "Cleaning up cache dir... "; rm -f $CACHE_DIR/day.*; fi ;; esac echo "Done"; echo `date +%c` "---Everything is done" exit 0 ----------------------------------------------------- Everyone who really uses while testing has to send me a postcard! ----------------------------------------------------------------------- Version of the EXAMPLES.v3 -------------------------- $Id: EXAMPLES.v3,v 3.1 2006-03-19 17:59:03 cord Exp $ calamaris-2.99.4.0/EXAMPLES0000644000175000001440000002623410407330372013726 0ustar cordusersNOTE: This file hasn't updated yet and reflects the (still working) Calamaris V2-Usage. New/Updated Examples welcome ;-) I want to give the 'audience' some examples for using Calamaris. So if you build some Scripts, crontabs or else around Calamaris, please mail and describe them to . I'll probably add them to this file. Thank You. Philipp Frauenfelder added this to run Calamaris automagically in Debian: ----------------------------------------------------------------------- /etc/cron.daily/calamaris ========================= #! /bin/sh set -e # calamaris: daily cron script. # This script should be run before the one for squid. According to the # man page of run-parts this is okay: squid comes after calamaris in the # alphabet. # Date: 1998-10-07 CONFFILE=/etc/calamaris.conf CALAMARIS=/usr/bin/calamaris if [ ! -x /usr/bin/calamaris ]; then exit 0 fi CALAMARISOPTIONS=-a ME=/etc/cron.daily/calamaris WEEKFILES=daily.1:daily.2:daily.3:daily.4:daily.5:daily.6:daily.0 SQUIDLOGDIR=/var/log/squid cd $SQUIDLOGDIR || exit 1 if [ ! -r access.log ]; then exit 0 fi LOGDIR=/var/log/calamaris cd $LOGDIR || exit 1 # today DAYOFWEEK=`date +"%w"` # read configuration file: /etc/calamaris.conf # is there a more elegant way to do this? DAYMAIL=`awk -F: '(!/¨#/) && ($1 == "daily") { print $2; }' $CONFFILE` DAYWEB=`awk -F: '(!/¨#/) && ($1 == "daily") { print $3; }' $CONFFILE` DAYDO=`awk -F: '(!/¨#/) && ($1 == "daily") { print $4; }' $CONFFILE` DAYTITLE=`awk -F: '(!/¨#/) && ($1 == "daily") { print $5; }' $CONFFILE` WEEKMAIL=`awk -F: '(!/¨#/) && ($1 == "weekly") { print $2; }' $CONFFILE` WEEKWEB=`awk -F: '(!/¨#/) && ($1 == "weekly") { print $3; }' $CONFFILE` WEEKDO=`awk -F: '(!/¨#/) && ($1 == "weekly") { print $4; }' $CONFFILE` WEEKTITLE=`awk -F: '(!/¨#/) && ($1 == "weekly") { print $5; }' $CONFFILE` # perhaps sometimes I will do this, # but as weekends and ends of months don't meet always... # MONTHMAIL=`awk -F: '(!/¨#/) && ($1 == "monthly") { print $2; }' $CONFFILE` # MONTHWEB=`awk -F: '(!/¨#/) && ($1 == "monthly") { print $3; }' $CONFFILE` # MONTHDO=`awk -F: '(!/¨#/) && ($1 == "monthly") { print $4; }' $CONFFILE` # MONTHTITLE=`awk -F: '(!/¨#/) && ($1 == "monthly") { print $5; }' $CONFFILE` # if we need monthly or weekly reports save a summary if [ $WEEKDO != "nothing" ]; then CALAMARISOPTIONSOLD="$CALAMARISOPTIONS" CALAMARISOPTIONS="$CALAMARISOPTIONS -o daily.$DAYOFWEEK" fi # do the daily report case $DAYDO in nothing) if [ $WEEKDO != "nothing" ]; then cat $SQUIDLOGDIR/access.log | \ nice -39 $CALAMARIS $CALAMARISOPTIONS > /dev/null fi ;; mail) ( echo "To: $DAYMAIL" cat $SQUIDLOGDIR/access.log | \ nice -39 $CALAMARIS $CALAMARISOPTIONS -F mail -H "$DAYTITLE" ) | /usr/lib/sendmail -t ;; web) cat $SQUIDLOGDIR/access.log | \ nice -39 $CALAMARIS $CALAMARISOPTIONS -F html -H "$DAYTITLE" > $DAYWEB ;; both) cat $SQUIDLOGDIR/access.log | \ nice -39 $CALAMARIS $CALAMARISOPTIONS -F html -H "$DAYTITLE" > $DAYWEB ( echo "To: $DAYMAIL" cat $SQUIDLOGDIR/access.log | \ nice -39 $CALAMARIS $CALAMARISOPTIONS -F mail -H "$DAYTITLE" ) | /usr/lib/sendmail -t ;; *) echo "the 'todo' for the daily Squid report in $CONFFILE" echo -n "is '$DAYDO' instead of one out of " echo "(nothing, mail, web, both)." >&2 exit 1 ;; esac # do the weekly report on Sunday <=> $DAYOFWEEK==0 if [ -n "$CALAMARISOPTIONSOLD" ]; then CALAMARISOPTIONS="$CALAMARISOPTIONSOLD" fi if [ $DAYOFWEEK = "0" ]; then case $WEEKDO in nothing) ;; mail) ( echo "To: $WEEKMAIL" nice -39 $CALAMARIS $CALAMARISOPTIONS -i $WEEKFILES \ -zH "$WEEKTITLE" -F mail ) | /usr/lib/sendmail -t ;; web) nice -39 $CALAMARIS $CALAMARISOPTIONS -i $WEEKFILES \ -zH "$WEEKTITLE" -F html > $WEEKWEB ;; both) nice -39 $CALAMARIS $CALAMARISOPTIONS -i $WEEKFILES \ -zH "$WEEKTITLE" -F html > $WEEKWEB ( echo "To: $WEEKMAIL" nice -39 $CALAMARIS $CALAMARISOPTIONS -i $WEEKFILES \ -zH "$WEEKTITLE" -F mail ) | /usr/lib/sendmail -t ;; *) echo "the 'todo' for the weekly Squid report in $CONFFILE" echo -n "is '$WEEKDO' instead of one out of " echo "(nothing, mail, web, both)." >&2 exit 1 ;; esac # if [ $MONTHDP != "nothing" ]; then # nice -39 $CALAMARIS $CALAMARISOPTIONS -i $WEEKFILES -z -o weekly.$WEEKWHAT > /dev/null # fi fi # do the monthly report and rotate the monthly logs: nothing to do :-) exit 0 /etc/calamaris.conf =================== # configuration file for calamaris # by Philipp Frauenfelder # 1998-10-09 # there are three categories: daily, weekly and monthly. For each of these # one line is responsible. There must be a line for each category but only # one. # cat: [daily|weekly|monthly] # mailto: mailaddress, eg. root # webto: path incl. file name, eg. /var/www/daily.html. The script does # currently not check wether the directory exists and # fail with a rather ugly error. # todo: [nothing|mail|web|both] # title: try it :-) # cat:mailto:webto:todo:title daily:root:/var/www/calamaris/daily.html:both:'Squid daily' weekly:root:/var/www/calamaris/weekly.html:both:'Squid weekly' # monthly does not work right now. #monthly:root:/var/www/monthly.html:both:'Squid monthly' # how many months of calamaris logs should be kept: integer #monthstokeep 2 ----------------------------------------------------------------------- Matthew King squidreport.cron: ----------------------------------------------------------------------- #!/bin/sh # SquidReport Script by Matthew King # Last update: 27-03-99. # This script will remove the current Squid HTML report, and will replace # it with a fresh one. The report will include all available squid access # log files.. Roughly 7 days worth. The report will then be dumped into # /home/httpd/html/ to be viewed via a web browser. # Remove the current report! cd /home/httpd/html/ rm -f squidreport.html echo > squidreport.html cd / # Create the new report and place it into the /home/httpd/html/ dir.. cd /var/log/squid/ cat access.log.7 access.log.6 access.log.5 access.log.4 access.log.3 \ access.log.2 access.log.1 access.log \ | /usr/local/bin/calamaris.pl -a -F html > /home/httpd/html/squidreport.html # Phew! It is done :) 60 odd seconds later :) ----------------------------------------------------------------------- Alain Williams suggested to make the above long line with all access.log-Files easier (and shorter): ----------------------------------------------------------------------- ( zcat $( ls -tr access.log.*.gz ) cat access.log ) | calamaris -a -F html > /var/www/html/calamaris.html Gottfried Hamm idea of using calamaris for reporting: (thanks to Hanno 'Rince' Wagner for bugfixing.) ----------------------------------------------------------------------- squid.conf ========== [...] logfile_rotate 7 [...] crontab ======= [...] 0 0 * * * /usr/local/squid/bin/squidrep [...] /usr/local/squid/bin/squidrep ============================= #!/bin/sh # SquidReport by Gottfried Hamm # Created 07.01.2000 # Updated BINDIR=/usr/local/squid/bin LOGDIR=/usr/local/squid/logs DAYOFWEEK=`date +"%w"` DATE=`date +"%Y%m%d"` ## Rotate the logs # $BINDIR/squid -k rotate sleep 300 ## Daily report via mail to webmaster # cd $LOGDIR cat access.log.0 | \ $BINDIR/calamaris -aH 'Daily' -F mail | \ mailx -s "Daily Proxy-Report `date`" squidmaster@cord.de ## Weekly report will be stored as web page # if [ $DAYOFWEEK = "0" ]; then cat access.log.? | $BINDIR/calamaris -a -F html -l 'GHKS' > \ /data/www/ghks.de/admin/proxy/report-$DATE.html fi exit 0 ----------------------------------------------------------------------- Thomas Wahyudi method: ----------------------------------------------------------------------- #!/bin/sh # SquidReport Script by Matthew King # rewriten by Thomas Wahyudi # Last update: 15-05-2000. # Sample Crontab # 0 0 * * * /cache/squid/bin/squid -k rotate # 30 0 * * * /cache/squid/bin/squidreport ################################################## dailyreport="/www/htdocs/Squid/report/daily/" weeklyreport="/www/htdocs/Squid/report/weekly/" programpath="/cache/squid/bin/calamaris" squidlog="/cache/squid/logs" hef1=`/bin/date | /usr/bin/awk '{print $2}'` hef2=`/bin/date | /usr/bin/awk '{print $3}'` hef3=`/bin/date | /usr/bin/awk '{print $1}'` hef4=`/bin/date | /usr/bin/awk '{print $6}'` # Convert date if less then 10 ############################## if [ "$hef2" -lt "10" ] then hef2="0$hef2" fi # Create the new report and place it into the report location # report in file ex. Mar.28.Tue.2000.htm will contain yesterday performance ########################################################################### cat $squidlog/access.log.0 | nice -39 $programpath -aH 'Yesterday worf' -F mail,html > "$dailyreport$hef1.$hef2.$hef3.$hef4.htm" # This script will remove the outdated Squid HTML report, # The report will include all available squid access log files.. # Roughly 31 days worth. The report will then be dumped into # report location to be viewed via a web browser. ################################################################ totalreport=`/bin/ls $dailyreport | /usr/bin/wc | /usr/bin/awk '{print $1}'` if [ $totalreport -gt 31 ] then get1file=`/bin/ls $dailyreport | /usr/bin/head -n 1` /bin/rm $dailyreport$get1file fi # Every Monday will create report for weekly report ################################################### totalreport=`/bin/ls $weeklyreport | /usr/bin/wc | /usr/bin/awk '{print $1}'` if [ "$hef3" = "Mon" ] then cat $squidlog/access.log.6 $squidlog/access.log.5 $squidlog/access.log.4 $squidlog/access.log.3 \ $squidlog/access.log.2 $squidlog/access.log.1 $squidlog/access.log.0 \ | $programpath -aH 'Weekly worf' -F mail,html > "$weeklyreport$hef1.$hef2.$hef3.$hef4.htm" if [ $totalreport -gt 5 ] then get1file=`/bin/ls $weeklyreport | /usr/bin/head -n 1` /bin/rm $weeklyreport$get1file fi fi ----------------------------------------------------------------------- My method: ----------------------------------------------------------------------- squid.conf: ----------------------------------------------------- [...] logfile_rotate 7 [...] ----------------------------------------------------- crontab: ----------------------------------------------------- 0 0 * * * /usr/local/squid/bin/squid -k rotate 30 0 * * * cat /var/log/squid/access.log.0 | \ nice -39 /usr/local/squid/bin/calamaris -aH 'daily worf' -F mail | \ mail Squidmaster@Cord.de 0 3 * * 7 (cd /var/log/squid/; cat access.log.6 access.log.5 \ access.log.4 access.log.3 access.log.2 access.log.1 \ access.log.0) | \ nice -39 /usr/local/squid/bin/calamaris -aH 'weekly worf' -F mail | \ mail Squidmaster@Cord.de ----------------------------------------------------- Everyone who really uses while testing has to send me a postcard! ----------------------------------------------------------------------- Version of the EXAMPLES ----------------------- $Id: EXAMPLES,v 2.13 2004/09/15 20:41:58 cord Exp $ calamaris-2.99.4.0/COPYRIGHT0000644000175000001440000005766710407330372014076 0ustar cordusersCalamaris itself is licensed under the GPL V2 or later. The Perl-Modules are dual-licensed under the GPL V1 or later and the Perl Artistic License ----------------------------------------------------------------------------- GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Library General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. , 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Library General Public License instead of this License. ----------------------------------------------------------------------------- ----------------------------------------------------------------------------- The "Artistic License" Preamble The intent of this document is to state the conditions under which a Package may be copied, such that the Copyright Holder maintains some semblance of artistic control over the development of the package, while giving the users of the package the right to use and distribute the Package in a more-or-less customary fashion, plus the right to make reasonable modifications. Definitions: "Package" refers to the collection of files distributed by the Copyright Holder, and derivatives of that collection of files created through textual modification. "Standard Version" refers to such a Package if it has not been modified, or has been modified in accordance with the wishes of the Copyright Holder as specified below. "Copyright Holder" is whoever is named in the copyright or copyrights for the package. "You" is you, if you're thinking about copying or distributing this Package. "Reasonable copying fee" is whatever you can justify on the basis of media cost, duplication charges, time of people involved, and so on. (You will not be required to justify it to the Copyright Holder, but only to the computing community at large as a market that must bear the fee.) "Freely Available" means that no fee is charged for the item itself, though there may be fees involved in handling the item. It also means that recipients of the item may redistribute it under the same conditions they received it. 1. You may make and give away verbatim copies of the source form of the Standard Version of this Package without restriction, provided that you duplicate all of the original copyright notices and associated disclaimers. 2. You may apply bug fixes, portability fixes and other modifications derived from the Public Domain or from the Copyright Holder. A Package modified in such a way shall still be considered the Standard Version. 3. You may otherwise modify your copy of this Package in any way, provided that you insert a prominent notice in each changed file stating how and when you changed that file, and provided that you do at least ONE of the following: a) place your modifications in the Public Domain or otherwise make them Freely Available, such as by posting said modifications to Usenet or an equivalent medium, or placing the modifications on a major archive site such as uunet.uu.net, or by allowing the Copyright Holder to include your modifications in the Standard Version of the Package. b) use the modified Package only within your corporation or organization. c) rename any non-standard executables so the names do not conflict with standard executables, which must also be provided, and provide a separate manual page for each non-standard executable that clearly documents how it differs from the Standard Version. d) make other distribution arrangements with the Copyright Holder. 4. You may distribute the programs of this Package in object code or executable form, provided that you do at least ONE of the following: a) distribute a Standard Version of the executables and library files, together with instructions (in the manual page or equivalent) on where to get the Standard Version. b) accompany the distribution with the machine-readable source of the Package with your modifications. c) give non-standard executables non-standard names, and clearly document the differences in manual pages (or equivalent), together with instructions on where to get the Standard Version. d) make other distribution arrangements with the Copyright Holder. 5. You may charge a reasonable copying fee for any distribution of this Package. You may charge any fee you choose for support of this Package. You may not charge a fee for this Package itself. However, you may distribute this Package in aggregate with other (possibly commercial) programs as part of a larger (possibly commercial) software distribution provided that you do not advertise this Package as a product of your own. You may embed this Package's interpreter within an executable of yours (by linking); this shall be construed as a mere form of aggregation, provided that the complete Standard Version of the interpreter is so embedded. 6. The scripts and library files supplied as input to or produced as output from the programs of this Package do not automatically fall under the copyright of this Package, but belong to whoever generated them, and may be sold commercially, and may be aggregated with this Package. If such scripts or library files are aggregated with this Package via the so-called "undump" or "unexec" methods of producing a binary executable image, then distribution of such an image shall neither be construed as a distribution of this Package nor shall it fall under the restrictions of Paragraphs 3 and 4, provided that you do not represent such an executable image as a Standard Version of this Package. 7. C subroutines (or comparably compiled subroutines in other languages) supplied by you and linked into this Package in order to emulate subroutines and variables of the language defined by this Package shall not be considered part of this Package, but are the equivalent of input as in Paragraph 6, provided these subroutines do not change the language in any way that would cause it to fail the regression tests for the language. 8. Aggregation of this Package with a commercial distribution is always permitted provided that the use of this Package is embedded; that is, when no overt attempt is made to make this Package's interfaces visible to the end user of the commercial distribution. Such use shall not be construed as a distribution of this Package. 9. The name of the Copyright Holder may not be used to endorse or promote products derived from this software without specific prior written permission. 10. THIS PACKAGE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE. The End calamaris-2.99.4.0/BUGS0000644000175000001440000001363610407330372013252 0ustar cordusersCalamaris Version 3 Are there known bugs or other problems? --------------------------------------- * The Content-Type-stats for some elff-logging-proxies are broken, i have to check why. * RedHat 8.0 has set the LANG-Variable to en_US.UTF-8, which caused Calamaris to crash with 'Split loop at (eval 1) line 21, <> line 1', maybe due to a perl-bug. (Investigation needed). You can workaround this problem by unsetting LANG. Please see https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=77437 . * There is a problem if you parse Logfiles from accelerating proxies. In iPlanet Web Proxy Server there is only an 'unqualified' URL in the Logfiles, which confuses all reports that rely on that field. (reported by Pawel Worach ) * Calamaris can't resolve IPv6-IPs to DNS-Names yet. If you can tell me how i can do it in perl, let me know ;-) * the byte-histogram sometimes displays a 0-0-byte line. this is correct. the requests added there are really logged as 0-byte-sized in the Logfile. Note that empty byte-ranges as 1-9 (which is impossible because of protocol-overhead) are skipped and not displayed in the report, so you can end up with 0-0 followed by 10-99 in the report. (noted by Reagan Blundell through Debian-Bug-Tracking) * there were many requests to add something that enables Calamaris to track down who is using the Cache to get what. I added these with stomach-ache, because it breaks the privacy of the users. So please read the following and think about it, before using Calamaris to be the 'Big Brother': - If you don't trust your users than there is something more wrong than the loss of productivity. - Squid has some nice acl-mechanisms. If you think that your users don't use the net properly, don't let them use it. (You can also open the net at specific times or to specific sites, if you want.) If you still want to use Calamaris that way, let your vict^Wusers know that they'll be monitored. (in Germany you have to let them know!) After some discussion in a newsgroup i was 'accused' to not announce the spy-features of Calamaris. THIS IS INTENTIONAL. This is MY program, it can spy on users, because there are many people who want to have that feature, but what i write on the feature-sheet is MY decision. Don't annoy me, you might damage my motivation to maintain this FREE (as in speech) piece of software. * In Calamaris V3 I use a new Cache-File-Format, which is incompatible with the old V2-Format. You can convert V2-cache to V3-cache with the calamaris-convert-cache-script, but it will only make the old data readable for the new version. You'll notice that the missing data, which wasn't calculated in the old version will result in sometimes strange results in the output. * If you reuse a cache-file, which is not created with '-d -1 -r -1 -t -1 -R -1' the number of 'others' is likely wrong everywhere. (reported by Clare Lahiff ) If i store the number of 'others' somewhere i still don't know which data is ment there, and in the next run (if i sum up) the number of others is to high (if the number of occurrences is below the threshold) or the summed up data misses the occurrences of the last run (if the number of occurances is above the threshold). i think i can't fix this... * If you want to parse more than one logfile (i.e. from the 'logfilerotate') or want to use more than one input-cache-file you have to put them in chronological sorted order (oldest first), else you get wrong peak values. However: If you use the caching function the peak-values can be wrong, because peaks occurring during log-rotate-time can't be detected. Calamaris will add a warning to the report if it recognises unsorted input. * Squid with SmartFilter-Patch and Cisco Content Engines have the ability to block or allow requests by checking against a database, and write this to the Logfiles. I will not add a report to give an overview about the usage of the Categories. (see first point of this chapter.) * Squid doesn't log outgoing UDP-Requests, so i can't put them into the statistics without parsing squid.conf and the cache.log-file. (Javier Puche asked for this), but i don't think that i should put this into Calamaris... * Squid and NetCache also support some kind of 'Common Logfile-format'. I won't support that, because Common Log is missing some very important data i.e. the request-time and the hierarchie-information. If you're still stuck with that format, i recommend the 'analog'-software by Steven Turner. Other way round: change logging to 'native' and convert it to 'common'. There is software for that available, i.e. my shrimp.pl. This also applies for the Common-style Log-files which NetCache produces. * If you use Calamaris at UNIX-epoch-date 2147483648 or later (~19.Jan 2038) you might get wrong dates on 32bit-systems. (I just added this to delight the people who really read this ;-) and to make a statement on this... on Y2K they found many systems which wasn't expected to run in that year. If you read this while checking if this package is Y2K038-compatible, then this is probably a really old system ;-) Y2038-Statement: Calamaris is as buggy as the used perl-version. * It is written in Perl. Yea, Perl is a great language for something like this (also it is the only one I'm able to write something like this in ;-). Calamaris was first intended as demo for what i expect from a statistical software. (OK, it is fun to write it, and it is even more fun to recognise that many people use the script). For my Caches with about 150MB logfile per week it is OK, but for those people on a heavy loaded Parent-cache it is possibly to slow. How does it perform with the perlcc coming in Perl5.6 or Perl6? Version of the BUGS ------------------- $Id: BUGS,v 3.1 2006-03-19 16:10:53 cord Exp $