pax_global_header00006660000000000000000000000064150007423320014506gustar00rootroot0000000000000052 comment=54eed453e63d76a842f141bf4c3fc3ae0370abf1 xmltv-1.4.0/000077500000000000000000000000001500074233200126625ustar00rootroot00000000000000xmltv-1.4.0/COPYING000066400000000000000000000432541500074233200137250ustar00rootroot00000000000000 GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Lesser General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. , 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. xmltv-1.4.0/Changes000066400000000000000000000421401500074233200141560ustar00rootroot000000000000001.4.0 2025-04-17 - tv_grab_fi_sv: disable grabber (upstream site changes) - tv_grab_fr: disable grabber (upstream Terms and Conditions) - tv_grab_huro: disable Romanian listings (upstream site gone) - tv_grab_it: disable grabber (upstream site changes) * tv_grab_fi: improvements to episode/season handling and upstream channel availability * tv_grab_pt_vodafone: migrate to new upstream API and improvements to quality of programme data * tv_grab_uk_freeview: improvements to programme retrieval and handling, web page cache is now used by default * tv_grab_zz_sdjson: improvements to episode/season handling * tv_grab_zz_sdjson_sqlite: adds deaf-signed subtitles element support and improves database handling 1.3.0 2024-02-24 + tv_grab_uk_freeview: new grabber for UK Freeview schedules - tv_grab_eu_epgdata: disable grabber (upstream site gone) - tv_grab_na_dtv: disable grabber (upstream site changes) - tv_grab_pt_meo: disable grabber (upstream site changes) - tv_grab_uk_guide: disable grabber (upstream site changes) * tv_grab_ch_search: fix setting of configured channels * tv_grab_fi: updates for STAR Chanel/FOXTV rename, and YLE data source (API key needs) * tv_grab_zz_sdjson: improvements to episode/season handling * tv_grab_zz_sdjson_sqlite: improvements to episode/season handling, 3rd-party metadata, lineups handling, and more 1.2.1 2023-02-23 - tv_grab_ar: disable grabber * tv_grab_fi_sv: update UserAgent to work with upstream changes 1.2.0 2023-02-19 - tv_grab_tr: disable grabber * tv_grab_fi: improvements to handle upstream changes * tv_grab_fr: improvements to channel name handling and updates for upstream changes * tv_grab_na_dd: add some debug info * tv_grab_uk_tvguide: bug fixes and improvements * tv_grab_zz_sdjson: support Schedules Direct redirection response * tv_grab_zz_sdjson_sqlite: improve rating agency data validation and update documentation * tv_merge: minor change to output handling 1.1.2 2022-04-18 * tv_grab_fi_sv: refresh internal channel map * tv_grab_fr: improvements to channel name handling ignore programmes when title is missing * tv_grab_uk_tvguide: add alternative method to retrieve available channels improvements to GMT/BST changeover handling improvements to documentation and examples 1.1.1 2022-02-19 * tv_grab_uk_tvguide: improvements to channel-id handling * QuickStart: documentation updates 1.1.0 2022-01-31 - tv_grab_eu_xmltvse: disable grabber + tv_grab_pt_meo: new grabber for Portugal (MEO) + tv_tmdb: new utility to augment listings with TMDB data * tv_grab_fi: foxtv: improvements to programme data telsu: disable listings source * tv_grab_fi_sv: improved channel handling avoids downloading data for unconfigured channels * tv_grab_uk_tvguide: enhance data with programme URLs and images * tv_grab_zz_sdjson_sqlite: improvements to programme data * tv_imdb: improvements to film and series matching * xmltv.dtd: add 'system' attribute to 'url' element add 'image' element to 'programme' add 'image' and 'url' as sub-elements to credits 1.0.0 2021-02-07 + tv_grab_ch_search: re-enable fixed grabber * tv_grab_eu_xmltvse: fetch listings over SSL * tv_grab_fi: many improvements to listings parsers * tv_grab_na_dtv: fetch listings over SSL * tv_grab_pt_vodafone: more reliable SSL conections using recent OpenSSL versions * tv_grab_uk_tvguide: improvements to XMLTV ID compliance - tv_grab_dk_dr: disable grabber (source site gone) - tv_grab_uk_bleb: disable grabber (source site gone) * tv_grep: allow regex filtering on channel ID * tv_imdb: significant reductions in memory consumption * Windows build: migrate to PAR::Packer 0.6.3 2020-08-22 - tv_grab_ch_search: disable deprecated grabber 0.6.2 2020-08-21 - tv_grab_dotmedia: disable deprecated grabber - tv_grab_se_tvzon: disable deprecated grabber - tv_grab_dtv_la: disable broken grabber - tv_grab_il: disable broken grabber - tv_grab_pt_meo: disable broken grabber - tv_grab_se_swedb: disable broken grabber * XMLTV.pm: update handling of reading from STDIN due to XML::Parser adopting 3-arg open * tv_grab_ch_search: handle upstream cookies * tv_grab_eu_epgdata: various fixes and improvements * tv_grab_fi: various fixes and improvements * tv_grab_fr: update grabber due to upstream changes * tv_grab_huro: use https source site URLs * tv_grab_it: fix overlapping/duplicate programmes * tv_grab_na_dd: use https source site URLs * tv_grab_na_dtv: various fixes and improvements * tv_grab_pt_vodafone: various fixes and improvements * tv_grab_uk_tvguide: various fixes and improvements * tv_grab_zz_sdjson_sqlite: many fixes and improvements 0.6.1 2019-02-21 * IMPORTANT * tv_grab_eu_dotmedia and tv_grab_se_tvzon are deprecated and will be removed in the next release of XMLTV. Please switch to the new tv_grab_eu_xmltvse grabber as soon as possible. + tv_grab_eu_xmltvse: new grabber for Europe + tv_grab_pt_vodafone: new grabber for Portugal - tv_grab_es_laguiatv: disable broken grabber - tv_grab_fr_kazer: disable broken grabber - tv_grab_in_toi: disable broken grabber - tv_grab_nl: disable broken grabber * tv_grab_eu_epgdata: include fanart URLs in output * tv_grab_fi: add new ampparit and telsu sources * tv_grab_il: update grabber due to upstream changes * tv_grab_is: now only provides RUV channels * tv_grab_zz_sdjson_sqlite: improvements to lineup management add support for TheTVDB metadata * tv_augment: new rules to improve episode numbering logging must now be enabled explicitly * tv_count/tv_merge: mandatory command line options for files * tv_imdb: migrate to new URL for archived IMDB data 0.6.0 2019-02-18 * TRIAL RELEASE * - tv_grab_pt_vodafone: new grabber for Portugal - tv_grab_es_laguiatv: disable broken grabber - tv_grab_fr_kazer: disable broken grabber - tv_grab_in_toi: disable broken grabber - tv_grab_nl: disable broken grabber - tv_grab_eu_epgdata: include fanart URLs in output - tv_grab_fi: add new ampparit and telsu sources - tv_grab_il: update grabber due to upstream changes - tv_grab_is: now only provides RUV channels - tv_grab_zz_sdjson_sqlite: improvements to lineup management add support for TheTVDB metadata - tv_augment: new rules to improve episode numbering - tv_imdb: migrate to new URL for archived data 0.5.70 2017-11-28 - tv_grab_eu_egon: removed broken grabber - tv_grab_fi_sv: re-enable Swidish language linstings of Finnish TV channels - tv_grab_sd_json: renamed to tv_grab_zz_sdjson (not country specific) 0.5.69 2017-01-24 - tv_grab_zz_sdjson_sqlite: add additional grabber for Schedule Direct's fee-based SD-JSON service. Supports 50+ countries. http://www.schedulesdirect.org/regions - tv_grab_hr: removed broken grabber - tv_grab_pt: removed broken grabber - tv_grab_uk_atlas: removed due to new target site rules - tv_grab_fi: major changes, need to run configure again - tv_grab_sd_json/tv_grab_zz_sdjson: major speed improvements. This grabber is replicated as tv_grab_zz_sdjson and will only be available via the new name in the next release. Please switch to the new name. 0.5.68 2016-06-02 - tv_grab_sd_json: new grabber for ScheduleDirect.org's SD-JSON service ($$) includes coverage for 50+ countries 0.5.67 2015-08-25 - tv_grab_tr: new grabber for Turkey - tv_grab_za: broken grabber removed - tv_augment: new tool to assist in augmenting/tweaking listings data 0.5.66 2014-10-21 - tv_garb_na_tvmedia: new grabber for US/Canada - tv_grab_ar: adjustments for site changes - tv_grab_is: adjustments for site changes - tv_grab_na_dd: adjustments for source changes - tv_grab_na_dtv: now uses parallel processes - tv_grab_pt: adjustments for site changes - tv_sort: major performance fixes 0.5.65 2014-05-09 - tv_grab_dk_dr: fixed for source site changes - tv_grab_dtv_la (AR,CL,CO,DO,EC,PE,PR,UY,VE): reinstate grabber after fixes for source site changes - tv_grab_es_laguiatv: fixes for source site changes - tv_grab_fi: improve series subtitles - tv_grab_fi_sv: reinstate grabber after fixes for source site changes - tv_grab_huro (HU,RO,CZ,SK): bug fixes & fixed grabber to work on all sites - tv_grab_is: fixes for validation errors - tv_grab_pt: fixed for source site changes - tv_grab_pt_meo: bug fixes and performance improvements - tv_grab_nl: reinstate grabber after fixes for source site changes - tv_grab_uk_atlas: minor bug fix & changes - tv_grab_uk_guardian: minor bug fixes & improvements - tv_grab_uk_tvguide: minor bug fixes & improvements - tv_cat: concatenate files with dissimilar character encodings - tv_imdb: fix character encoding of merged data. Add keywords. Add Plot Summary - tv_to_text & tv_to_latex: add optional output of programme description 0.5.64 2013-12-23 - tv_grab_huro: fixes for source site changes - tv_count: new utility - tv_merge: new utility - tv_grab_uk_guardian: new grabber - tv_grab_uk_tvguide: new grabber - tv_grab_uk_atlas: new grabber - tv_grab_na_icon: removed due to source site changes - tv_grab_dr_dk: removed due to source site changes 0.5.63 2012-06-14 - tv_grab_uk_rt: fix bug in 0.5.62 release - tv_grab_dtv_la: remove broken grabber - tv_grab_ee: remove broken grabber - tv_grab_es_miguiatv: remove broken grabber - tv_grab_nl: remove broken grabber - tv_grab_pt: remove broken grabber - tv_grab_uk_rt: addition of lineups support for easier configuration, improved unicode handling, and 6-7x performance increase with changed date/time handling - tv_augment_tz: new filter to convert floating time to explicit time 0.5.62 2012-06-10 - xmltv.dtd: add a lang attribute to review elements - tv_grab_uk_rt: improved unicode handling - tv_grab_pt_meo: added back to xmltv.exe - tv_grab_eu-epgdata: added back to xmltv.exe 0.5.61 2011-06-22 - tv_grab_ar: replace switch statements to maintain backwards compatibility - tv_grab_is: replace switch statements to maintain backwards compatibility 0.5.60 2011-06-21 - tv_grab_fr_kazer: new graber for kazer.org - tv_grab_ar: rewrite! back in distro - tv_grab_fi: rewrite! back in distro - tv_grab_re: disable broken grabber - tv_grab_na_dtv: disable broken grabber 0.5.59 2010-11-22 - tv_grab_pt_meo: new grabber (MEO from SAPO in Portugal) - tv_grab_fi_sv: new grabber (listings for Finland in Swedish) 0.5.58 2010-09-07 - tv_grab_in: new grabber for India (now live) - tv_grab_ar: removed due to target site changes (will re-add when fixed) - find_grabbers: now runs much faster 0.5.57 2010-04-21 - tv_grab_in: new grabber for India (experimental) - tv_grab_il: new grabber for Israel - tv_grab_nl: new grabber for Netherlands (old grabber reactivated, works fine) - tv_grab_dk_dr: re-release of Danish grabber after re-write - tv_grab_es_laguiatv: channel icons added - tv_grab_fr: major code cleanup with new features added - tv_grab_uk_rt: many changes, upgrade strongly recommended - xmltv.dtd: add support for composer and editor credit roles; add review element; add channel "transport-id" element added - Miscellaneous: fixes for POD syntax errors fixes for new versions (6.00+) of Date::Manip module and DST handling 0.5.56 2009-08-10 - tv_grab_it_dvb: new grabber for Italian DVB-S streams - tv_grab_huro: add Slovakian episode parsing - tv_grab_uk_rt: improve UTF8 support, improve actor support - tv_grab_za: South African grabber fixed 0.5.55 2009-03-14 - tv_grab_huro: add Czech and Slovenia support - tv_grab_is: added support for timeshifted channels. - tv_grab_it: lots of fixes - tv_grab_pt: bugfix for latest site changes. - tv_grab_uk_rt: improve title and credits processing - tv_grab_br_net: disable broken grabber - tv_grab_es: disable broken grabber - tv_grab_jp: disable broken grabber - tv_grab_za: disable broken grabber 0.5.54 2009-01-14 - tv_grab_be: removed due to source site blocking - tv_grab_eu_epgdata: should now work on win32 - tv_grab_id: add 3 more backends - tv_grab_na_dd: add Movie and Sports category - tv_grab_uk_rt: add support for Sky Arts 2 - tv_check: add --season-reset switch to change everything to 'title-only' 0.5.53 2008-09-02 - tv_grab_dk_dr: new grabber for Denmark - tv_grab_ar: now supports 2 data sources - tv_grab_dk: remove broken grabber - tv_grab_eu_epgdata: added category support, stop times, age-rating, more *NOTE* fixes to tv_grab_eu_epgdata episode-num will break duplicate detection in MythTV - tv_grab_huro: fix --offset problem and data source changes - tv_grab_is: fixed for upstream changes; minor fix to midnight handling - tv_grab_na_dd: add auto-config config file option - tv_grab_na_icons: adjust to site changes - tv_grab_uk_rt: improve episode numbering 0.5.52 2008-07-14 - tv_grab_il: removed broken grabber - tv_grab_nl_wolf: removed broken grabber - tv_grab_be: responding to source site changes - tv_grab_fr: better title detection - tv_grab_huro: now generates stop times - tv_grab_na_dtv: responding to source site changes - tv_grab_pt: responding to source site changes - tv_grab_re: better season/episode number handling - tv_grab_uk_rt: lots of improvements - tv_grab_za: improve episode tags 0.5.51 2008-02-18 - tv_grab_es_miguiatv: new grabber for Spain - tv_grab_eu_epgdata: new European grabber ($) - tv_grab_nc: remove broken grabber - tv_grab_nl: remove broken grabber - tv_grab_no: remove broken grabber - tv_grab_br_net: adapted for changes on the grabbed website - tv_grab_fi: misc fixes for site changes - tv_grab_fr: update for site changes - tv_grab_it: add new data source and speed improvment - tv_grab_na_dtv: misc fixes - tv_grab_re: misc fixes - tv_grab_uk_rt: add title/subtitle processing for consistant programme info, add support for timeshifted and part-time channels, improve support for non-english characters and bad characters - XMLTV.pm: add Programme->Video->Quality attribute - IMDB.pm: misc fixes 0.5.50 2007-11-05 - tv_grab_eu_epgdata: new grabber for the commercial epgdata.com service ($) - tv_grab_na_dtv: new grabber for North America Direct TV users - tv_grab_fr: improves Color/B&W detection, autocorrect, actors, director field parsing - tv_grab_uk_rt: improved bad character handling. Added "recommended" and "deaf-signed" notations. - tv_grab_il: replace Locale::Hebrew module to Text::Bidi due licensing problems - tv_check: skip --myreplaytv as MyReplayTV.com has been discontinued. - XMLTV::Supplement: add new capability to many grabbers. Those grabbers will now check an xmltv.org web server for channel-id and other supplemental information. This ensures more current versions and should make new channels available sooner for affected grabbers. - xmltv.dtd: add "system" attribute to now allows multiple instances xmltv-1.4.0/MANIFEST000066400000000000000000002362221500074233200140220ustar00rootroot00000000000000COPYING Changes MANIFEST Makefile.PL README.md README.cygwin Uninstall.pm authors.txt xmltv-lineups.xsd xmltv.dtd xmltv_logo.ico xmltv_logo.png analyse_tvprefs/README analyse_tvprefs/analyse_tvprefs analyse_tvprefs/bnc_freq.txt choose/tv_check/README.tv_check choose/tv_check/tv_check choose/tv_check/tv_check_doc.html choose/tv_check/tv_check_doc.jpg choose/tv_pick/tv_pick_cgi choose/tv_pick/merge_tvprefs doc/COPYING doc/README-Windows.md doc/QuickStart doc/exe_build.html doc/code/coding_standards filter/Grep.pm filter/augment/augment.conf filter/augment/augment.rules filter/tv_augment filter/tv_augment_tz filter/tv_cat filter/tv_count filter/tv_extractinfo_ar filter/tv_extractinfo_en filter/tv_grep.PL filter/tv_grep.in filter/tv_imdb filter/tv_merge filter/tv_remove_some_overlapping filter/tv_sort filter/tv_split filter/tv_tmdb filter/tv_to_latex filter/tv_to_potatoe filter/tv_to_text grab/Get_nice.pm grab/Grab_XML.pm grab/Memoize.pm grab/DST.pm grab/Config_file.pm grab/Mode.pm grab/ch_search/test.conf grab/ch_search/tv_grab_ch_search.PL grab/ch_search/tv_grab_ch_search.in grab/combiner/test.conf grab/combiner/tv_grab_combiner grab/fi/fi/common.pm grab/fi/fi/day.pm grab/fi/fi/programme.pm grab/fi/fi/programmeStartOnly.pm grab/fi/fi/source/iltapulu.pm grab/fi/fi/source/star.pm grab/fi/fi/source/telkku.pm grab/fi/fi/source/telsu.pm grab/fi/fi/source/yle.pm grab/fi/get_latest_version.sh grab/fi/merge.PL grab/fi/test.conf grab/fi/test.sh grab/fi/tv_grab_fi.pl grab/fi_sv/test.conf grab/fi_sv/tv_grab_fi_sv grab/fr/test.conf grab/fr/tv_grab_fr grab/huro/catmap.hu grab/huro/jobmap grab/huro/test.conf grab/huro/tv_grab_huro.PL grab/huro/tv_grab_huro.in grab/is/test.conf grab/is/tv_grab_is grab/is/category_map grab/it_dvb/channel_ids grab/it_dvb/sky_it.dict grab/it_dvb/sky_it.themes grab/it_dvb/tv_grab_it_dvb.PL grab/it_dvb/tv_grab_it_dvb.in grab/na_dd/tv_grab_na_dd.PL grab/na_dd/tv_grab_na_dd.in grab/na_tvmedia/test.conf grab/na_tvmedia/tv_grab_na_tvmedia grab/pt_vodafone/channel.list grab/pt_vodafone/test.conf grab/pt_vodafone/tv_grab_pt_vodafone grab/test_grabbers grab/uk_freeview/test.conf grab/uk_freeview/tv_grab_uk_freeview grab/zz_sdjson/tv_grab_zz_sdjson grab/zz_sdjson_sqlite/fixups.txt grab/zz_sdjson_sqlite/tv_grab_zz_sdjson_sqlite lib/Ask.pm lib/Ask/Term.pm lib/Ask/Tk.pm lib/Augment.pm lib/Capabilities.pm lib/Clumps.pm lib/Configure.pm lib/Configure/Writer.pm lib/Data/Recursive/Encode.pm lib/Date.pm lib/Description.pm lib/GUI.pm lib/Gunzip.pm lib/IMDB.pm lib/Options.pm lib/PreferredMethod.pm lib/ProgressBar.pm lib/ProgressBar/None.pm lib/ProgressBar/Term.pm lib/ProgressBar/Tk.pm lib/Summarize.pm lib/Supplement.pm.PL lib/Supplement.pm.in lib/TMDB.pm lib/TMDB/API.pm lib/TMDB/API/Config.pm lib/TMDB/API/Movie.pm lib/TMDB/API/Person.pm lib/TMDB/API/Tv.pm lib/TZ.pm lib/Usage.pm lib/ValidateFile.pm lib/ValidateGrabber.pm lib/Version.pm lib/XMLTV.pm.PL lib/XMLTV.pm.in lib/exe_opt.pl lib/xmltv.pl lib/set_share_dir.pl t/README t/parallel_test t/add_time_info t/test_filters.t t/test_library.t t/test_tv_split.t t/test_icon.t t/test_dst.t t/data/attrs.xml t/data/amp.xml t/data/clump.xml t/data/clump_extract.xml t/data/clump_extract_1.xml t/data/dups.xml t/data/empty.xml t/data/length.xml t/data/overlap.xml t/data/simple.xml t/data/sort.xml t/data/sort1.xml t/data/sort2.xml t/data/test.xml t/data/test_empty.xml t/data/test_livre.xml t/data/test_sort_by_channel.xml t/data/tv_cat_all_UTF8.expected t/data/tv_cat_amp_xml.expected t/data/tv_cat_amp_xml_amp_xml.expected t/data/tv_cat_amp_xml_clump_xml.expected t/data/tv_cat_amp_xml_dups_xml.expected t/data/tv_cat_amp_xml_empty_xml.expected t/data/tv_cat_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_cat_attrs_xml.expected t/data/tv_cat_clump_extract_1_xml.expected t/data/tv_cat_clump_extract_xml.expected t/data/tv_cat_clump_xml.expected t/data/tv_cat_clump_xml_amp_xml.expected t/data/tv_cat_clump_xml_clump_xml.expected t/data/tv_cat_clump_xml_dups_xml.expected t/data/tv_cat_clump_xml_empty_xml.expected t/data/tv_cat_dups_xml.expected t/data/tv_cat_dups_xml_amp_xml.expected t/data/tv_cat_dups_xml_clump_xml.expected t/data/tv_cat_dups_xml_dups_xml.expected t/data/tv_cat_dups_xml_empty_xml.expected t/data/tv_cat_empty_xml.expected t/data/tv_cat_empty_xml_amp_xml.expected t/data/tv_cat_empty_xml_clump_xml.expected t/data/tv_cat_empty_xml_dups_xml.expected t/data/tv_cat_empty_xml_empty_xml.expected t/data/tv_cat_length_xml.expected t/data/tv_cat_overlap_xml.expected t/data/tv_cat_simple_xml.expected t/data/tv_cat_simple_xml_x_whatever_xml.expected t/data/tv_cat_sort_xml.expected t/data/tv_cat_test_empty_xml.expected t/data/tv_cat_test_livre_xml.expected t/data/tv_cat_test_sort_by_channel_xml.expected t/data/tv_cat_test_xml.expected t/data/tv_cat_test_xml_test_xml.expected t/data/tv_cat_whitespace_xml.expected t/data/tv_cat_x_whatever_xml.expected t/data/tv_extractinfo_en_all_UTF8.expected t/data/tv_extractinfo_en_amp_xml.expected t/data/tv_extractinfo_en_amp_xml_amp_xml.expected t/data/tv_extractinfo_en_amp_xml_clump_xml.expected t/data/tv_extractinfo_en_amp_xml_dups_xml.expected t/data/tv_extractinfo_en_amp_xml_empty_xml.expected t/data/tv_extractinfo_en_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_extractinfo_en_attrs_xml.expected t/data/tv_extractinfo_en_clump_extract_1_xml.expected t/data/tv_extractinfo_en_clump_extract_xml.expected t/data/tv_extractinfo_en_clump_xml.expected t/data/tv_extractinfo_en_clump_xml_amp_xml.expected t/data/tv_extractinfo_en_clump_xml_clump_xml.expected t/data/tv_extractinfo_en_clump_xml_dups_xml.expected t/data/tv_extractinfo_en_clump_xml_empty_xml.expected t/data/tv_extractinfo_en_dups_xml.expected t/data/tv_extractinfo_en_dups_xml_amp_xml.expected t/data/tv_extractinfo_en_dups_xml_clump_xml.expected t/data/tv_extractinfo_en_dups_xml_dups_xml.expected t/data/tv_extractinfo_en_dups_xml_empty_xml.expected t/data/tv_extractinfo_en_empty_xml.expected t/data/tv_extractinfo_en_empty_xml_amp_xml.expected t/data/tv_extractinfo_en_empty_xml_clump_xml.expected t/data/tv_extractinfo_en_empty_xml_dups_xml.expected t/data/tv_extractinfo_en_empty_xml_empty_xml.expected t/data/tv_extractinfo_en_length_xml.expected t/data/tv_extractinfo_en_overlap_xml.expected t/data/tv_extractinfo_en_simple_xml.expected t/data/tv_extractinfo_en_simple_xml_x_whatever_xml.expected t/data/tv_extractinfo_en_sort_xml.expected t/data/tv_extractinfo_en_test_empty_xml.expected t/data/tv_extractinfo_en_test_livre_xml.expected t/data/tv_extractinfo_en_test_sort_by_channel_xml.expected t/data/tv_extractinfo_en_test_xml.expected t/data/tv_extractinfo_en_test_xml_test_xml.expected t/data/tv_extractinfo_en_whitespace_xml.expected t/data/tv_extractinfo_en_x_whatever_xml.expected t/data/tv_grep_a_all_UTF8.expected t/data/tv_grep_a_amp_xml.expected t/data/tv_grep_a_amp_xml_amp_xml.expected t/data/tv_grep_a_amp_xml_clump_xml.expected t/data/tv_grep_a_amp_xml_dups_xml.expected t/data/tv_grep_a_amp_xml_empty_xml.expected t/data/tv_grep_a_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_a_attrs_xml.expected t/data/tv_grep_a_clump_extract_1_xml.expected t/data/tv_grep_a_clump_extract_xml.expected t/data/tv_grep_a_clump_xml.expected t/data/tv_grep_a_clump_xml_amp_xml.expected t/data/tv_grep_a_clump_xml_clump_xml.expected t/data/tv_grep_a_clump_xml_dups_xml.expected t/data/tv_grep_a_clump_xml_empty_xml.expected t/data/tv_grep_a_dups_xml.expected t/data/tv_grep_a_dups_xml_amp_xml.expected t/data/tv_grep_a_dups_xml_clump_xml.expected t/data/tv_grep_a_dups_xml_dups_xml.expected t/data/tv_grep_a_dups_xml_empty_xml.expected t/data/tv_grep_a_empty_xml.expected t/data/tv_grep_a_empty_xml_amp_xml.expected t/data/tv_grep_a_empty_xml_clump_xml.expected t/data/tv_grep_a_empty_xml_dups_xml.expected t/data/tv_grep_a_empty_xml_empty_xml.expected t/data/tv_grep_a_length_xml.expected t/data/tv_grep_a_overlap_xml.expected t/data/tv_grep_a_simple_xml.expected t/data/tv_grep_a_simple_xml_x_whatever_xml.expected t/data/tv_grep_a_sort_xml.expected t/data/tv_grep_a_test_empty_xml.expected t/data/tv_grep_a_test_livre_xml.expected t/data/tv_grep_a_test_sort_by_channel_xml.expected t/data/tv_grep_a_test_xml.expected t/data/tv_grep_a_test_xml_test_xml.expected t/data/tv_grep_a_whitespace_xml.expected t/data/tv_grep_a_x_whatever_xml.expected t/data/tv_grep_category_b_all_UTF8.expected t/data/tv_grep_category_b_amp_xml.expected t/data/tv_grep_category_b_amp_xml_amp_xml.expected t/data/tv_grep_category_b_amp_xml_clump_xml.expected t/data/tv_grep_category_b_amp_xml_dups_xml.expected t/data/tv_grep_category_b_amp_xml_empty_xml.expected t/data/tv_grep_category_b_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_category_b_attrs_xml.expected t/data/tv_grep_category_b_clump_extract_1_xml.expected t/data/tv_grep_category_b_clump_extract_xml.expected t/data/tv_grep_category_b_clump_xml.expected t/data/tv_grep_category_b_clump_xml_amp_xml.expected t/data/tv_grep_category_b_clump_xml_clump_xml.expected t/data/tv_grep_category_b_clump_xml_dups_xml.expected t/data/tv_grep_category_b_clump_xml_empty_xml.expected t/data/tv_grep_category_b_dups_xml.expected t/data/tv_grep_category_b_dups_xml_amp_xml.expected t/data/tv_grep_category_b_dups_xml_clump_xml.expected t/data/tv_grep_category_b_dups_xml_dups_xml.expected t/data/tv_grep_category_b_dups_xml_empty_xml.expected t/data/tv_grep_category_b_empty_xml.expected t/data/tv_grep_category_b_empty_xml_amp_xml.expected t/data/tv_grep_category_b_empty_xml_clump_xml.expected t/data/tv_grep_category_b_empty_xml_dups_xml.expected t/data/tv_grep_category_b_empty_xml_empty_xml.expected t/data/tv_grep_category_b_length_xml.expected t/data/tv_grep_category_b_overlap_xml.expected t/data/tv_grep_category_b_simple_xml.expected t/data/tv_grep_category_b_simple_xml_x_whatever_xml.expected t/data/tv_grep_category_b_sort_xml.expected t/data/tv_grep_category_b_test_empty_xml.expected t/data/tv_grep_category_b_test_livre_xml.expected t/data/tv_grep_category_b_test_sort_by_channel_xml.expected t/data/tv_grep_category_b_test_xml.expected t/data/tv_grep_category_b_test_xml_test_xml.expected t/data/tv_grep_category_b_whitespace_xml.expected t/data/tv_grep_category_b_x_whatever_xml.expected t/data/tv_grep_category_e_and_title_f_all_UTF8.expected t/data/tv_grep_category_e_and_title_f_amp_xml.expected t/data/tv_grep_category_e_and_title_f_amp_xml_amp_xml.expected t/data/tv_grep_category_e_and_title_f_amp_xml_clump_xml.expected t/data/tv_grep_category_e_and_title_f_amp_xml_dups_xml.expected t/data/tv_grep_category_e_and_title_f_amp_xml_empty_xml.expected t/data/tv_grep_category_e_and_title_f_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_category_e_and_title_f_attrs_xml.expected t/data/tv_grep_category_e_and_title_f_clump_extract_1_xml.expected t/data/tv_grep_category_e_and_title_f_clump_extract_xml.expected t/data/tv_grep_category_e_and_title_f_clump_xml.expected t/data/tv_sort_amp_xml.expected t/data/tv_grep_category_e_and_title_f_clump_xml_amp_xml.expected t/data/tv_grep_category_e_and_title_f_clump_xml_clump_xml.expected t/data/tv_grep_category_e_and_title_f_clump_xml_dups_xml.expected t/data/tv_grep_category_e_and_title_f_clump_xml_empty_xml.expected t/data/tv_grep_category_e_and_title_f_dups_xml.expected t/data/tv_grep_category_e_and_title_f_dups_xml_amp_xml.expected t/data/tv_grep_category_e_and_title_f_dups_xml_clump_xml.expected t/data/tv_grep_category_e_and_title_f_dups_xml_dups_xml.expected t/data/tv_grep_category_e_and_title_f_dups_xml_empty_xml.expected t/data/tv_grep_category_e_and_title_f_empty_xml.expected t/data/tv_grep_category_e_and_title_f_empty_xml_amp_xml.expected t/data/tv_grep_category_e_and_title_f_empty_xml_clump_xml.expected t/data/tv_grep_category_e_and_title_f_empty_xml_dups_xml.expected t/data/tv_grep_category_e_and_title_f_empty_xml_empty_xml.expected t/data/tv_grep_category_e_and_title_f_length_xml.expected t/data/tv_grep_category_e_and_title_f_overlap_xml.expected t/data/tv_grep_category_e_and_title_f_simple_xml.expected t/data/tv_grep_category_e_and_title_f_simple_xml_x_whatever_xml.expected t/data/tv_grep_category_e_and_title_f_sort_xml.expected t/data/tv_grep_category_e_and_title_f_test_empty_xml.expected t/data/tv_grep_category_e_and_title_f_test_livre_xml.expected t/data/tv_grep_category_e_and_title_f_test_sort_by_channel_xml.expected t/data/tv_grep_category_e_and_title_f_test_xml.expected t/data/tv_grep_category_e_and_title_f_test_xml_test_xml.expected t/data/tv_grep_category_e_and_title_f_whitespace_xml.expected t/data/tv_grep_category_e_and_title_f_x_whatever_xml.expected t/data/tv_grep_category_g_or_title_h_all_UTF8.expected t/data/tv_grep_category_g_or_title_h_amp_xml.expected t/data/tv_grep_category_g_or_title_h_amp_xml_amp_xml.expected t/data/tv_grep_category_g_or_title_h_amp_xml_clump_xml.expected t/data/tv_grep_category_g_or_title_h_amp_xml_dups_xml.expected t/data/tv_grep_category_g_or_title_h_amp_xml_empty_xml.expected t/data/tv_grep_category_g_or_title_h_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_category_g_or_title_h_attrs_xml.expected t/data/tv_grep_category_g_or_title_h_clump_extract_1_xml.expected t/data/tv_grep_category_g_or_title_h_clump_extract_xml.expected t/data/tv_grep_category_g_or_title_h_clump_xml.expected t/data/tv_grep_category_g_or_title_h_clump_xml_amp_xml.expected t/data/tv_grep_category_g_or_title_h_clump_xml_clump_xml.expected t/data/tv_grep_category_g_or_title_h_clump_xml_dups_xml.expected t/data/tv_grep_category_g_or_title_h_clump_xml_empty_xml.expected t/data/tv_grep_category_g_or_title_h_dups_xml.expected t/data/tv_grep_category_g_or_title_h_dups_xml_amp_xml.expected t/data/tv_grep_category_g_or_title_h_dups_xml_clump_xml.expected t/data/tv_grep_category_g_or_title_h_dups_xml_dups_xml.expected t/data/tv_grep_category_g_or_title_h_dups_xml_empty_xml.expected t/data/tv_grep_category_g_or_title_h_empty_xml.expected t/data/tv_grep_category_g_or_title_h_empty_xml_amp_xml.expected t/data/tv_grep_category_g_or_title_h_empty_xml_clump_xml.expected t/data/tv_grep_category_g_or_title_h_empty_xml_dups_xml.expected t/data/tv_grep_category_g_or_title_h_empty_xml_empty_xml.expected t/data/tv_grep_category_g_or_title_h_length_xml.expected t/data/tv_grep_category_g_or_title_h_overlap_xml.expected t/data/tv_grep_category_g_or_title_h_simple_xml.expected t/data/tv_grep_category_g_or_title_h_simple_xml_x_whatever_xml.expected t/data/tv_grep_category_g_or_title_h_sort_xml.expected t/data/tv_grep_category_g_or_title_h_test_empty_xml.expected t/data/tv_grep_category_g_or_title_h_test_livre_xml.expected t/data/tv_grep_category_g_or_title_h_test_sort_by_channel_xml.expected t/data/tv_grep_category_g_or_title_h_test_xml.expected t/data/tv_grep_category_g_or_title_h_test_xml_test_xml.expected t/data/tv_grep_category_g_or_title_h_whitespace_xml.expected t/data/tv_grep_category_g_or_title_h_x_whatever_xml.expected t/data/tv_grep_new_all_UTF8.expected t/data/tv_grep_channel_id_channel4_com_all_UTF8.expected t/data/tv_grep_channel_id_channel4_com_amp_xml.expected t/data/tv_grep_channel_id_channel4_com_amp_xml_amp_xml.expected t/data/tv_grep_channel_id_channel4_com_amp_xml_clump_xml.expected t/data/tv_grep_channel_id_channel4_com_amp_xml_dups_xml.expected t/data/tv_grep_channel_id_channel4_com_amp_xml_empty_xml.expected t/data/tv_grep_channel_id_channel4_com_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_channel_id_channel4_com_attrs_xml.expected t/data/tv_grep_channel_id_channel4_com_clump_extract_1_xml.expected t/data/tv_grep_channel_id_channel4_com_clump_extract_xml.expected t/data/tv_grep_channel_id_channel4_com_clump_xml.expected t/data/tv_grep_channel_id_channel4_com_clump_xml_amp_xml.expected t/data/tv_grep_channel_id_channel4_com_clump_xml_clump_xml.expected t/data/tv_grep_channel_id_channel4_com_clump_xml_dups_xml.expected t/data/tv_grep_channel_id_channel4_com_clump_xml_empty_xml.expected t/data/tv_grep_channel_id_channel4_com_dups_xml.expected t/data/tv_grep_channel_id_channel4_com_dups_xml_amp_xml.expected t/data/tv_grep_channel_id_channel4_com_dups_xml_clump_xml.expected t/data/tv_grep_channel_id_channel4_com_dups_xml_dups_xml.expected t/data/tv_grep_channel_id_channel4_com_dups_xml_empty_xml.expected t/data/tv_grep_channel_id_channel4_com_empty_xml.expected t/data/tv_grep_channel_id_channel4_com_empty_xml_amp_xml.expected t/data/tv_grep_channel_id_channel4_com_empty_xml_clump_xml.expected t/data/tv_grep_channel_id_channel4_com_empty_xml_dups_xml.expected t/data/tv_grep_channel_id_channel4_com_empty_xml_empty_xml.expected t/data/tv_grep_channel_id_channel4_com_length_xml.expected t/data/tv_grep_channel_id_channel4_com_overlap_xml.expected t/data/tv_grep_channel_id_channel4_com_simple_xml.expected t/data/tv_grep_channel_id_channel4_com_simple_xml_x_whatever_xml.expected t/data/tv_grep_channel_id_channel4_com_sort_xml.expected t/data/tv_grep_channel_id_channel4_com_test_empty_xml.expected t/data/tv_grep_channel_id_channel4_com_test_livre_xml.expected t/data/tv_grep_channel_id_channel4_com_test_sort_by_channel_xml.expected t/data/tv_grep_channel_id_channel4_com_test_xml.expected t/data/tv_grep_channel_id_channel4_com_test_xml_test_xml.expected t/data/tv_grep_channel_id_channel4_com_whitespace_xml.expected t/data/tv_grep_channel_id_channel4_com_x_whatever_xml.expected t/data/tv_grep_channel_name_d_all_UTF8.expected t/data/tv_grep_channel_name_d_amp_xml.expected t/data/tv_grep_channel_name_d_amp_xml_amp_xml.expected t/data/tv_grep_channel_name_d_amp_xml_clump_xml.expected t/data/tv_grep_channel_name_d_amp_xml_dups_xml.expected t/data/tv_grep_channel_name_d_amp_xml_empty_xml.expected t/data/tv_grep_channel_name_d_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_channel_name_d_attrs_xml.expected t/data/tv_grep_channel_name_d_clump_extract_1_xml.expected t/data/tv_grep_channel_name_d_clump_extract_xml.expected t/data/tv_grep_channel_name_d_clump_xml.expected t/data/tv_grep_channel_name_d_clump_xml_amp_xml.expected t/data/tv_grep_channel_name_d_clump_xml_clump_xml.expected t/data/tv_grep_channel_name_d_clump_xml_dups_xml.expected t/data/tv_grep_channel_name_d_clump_xml_empty_xml.expected t/data/tv_grep_channel_name_d_dups_xml.expected t/data/tv_grep_channel_name_d_dups_xml_amp_xml.expected t/data/tv_grep_channel_name_d_dups_xml_clump_xml.expected t/data/tv_grep_channel_name_d_dups_xml_dups_xml.expected t/data/tv_grep_channel_name_d_dups_xml_empty_xml.expected t/data/tv_grep_channel_name_d_empty_xml.expected t/data/tv_grep_channel_name_d_empty_xml_amp_xml.expected t/data/tv_grep_channel_name_d_empty_xml_clump_xml.expected t/data/tv_grep_channel_name_d_empty_xml_dups_xml.expected t/data/tv_grep_channel_name_d_empty_xml_empty_xml.expected t/data/tv_grep_channel_name_d_length_xml.expected t/data/tv_grep_channel_name_d_overlap_xml.expected t/data/tv_grep_channel_name_d_simple_xml.expected t/data/tv_grep_channel_name_d_sort_xml.expected t/data/tv_grep_channel_name_d_simple_xml_x_whatever_xml.expected t/data/tv_grep_channel_name_d_test_empty_xml.expected t/data/tv_grep_channel_name_d_test_livre_xml.expected t/data/tv_grep_channel_name_d_test_sort_by_channel_xml.expected t/data/tv_grep_channel_name_d_test_xml.expected t/data/tv_grep_channel_name_d_test_xml_test_xml.expected t/data/tv_grep_channel_name_d_whitespace_xml.expected t/data/tv_grep_channel_name_d_x_whatever_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_all_UTF8.expected t/data/tv_grep_channel_xyz_or_channel_b_amp_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_amp_xml_amp_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_amp_xml_clump_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_amp_xml_dups_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_amp_xml_empty_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_attrs_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_clump_extract_1_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_clump_extract_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_clump_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_clump_xml_amp_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_clump_xml_clump_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_clump_xml_dups_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_clump_xml_empty_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_dups_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_dups_xml_amp_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_dups_xml_clump_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_dups_xml_dups_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_dups_xml_empty_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_empty_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_empty_xml_amp_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_empty_xml_clump_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_empty_xml_dups_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_empty_xml_empty_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_length_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_overlap_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_simple_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_simple_xml_x_whatever_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_sort_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_test_empty_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_test_livre_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_test_sort_by_channel_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_test_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_test_xml_test_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_whitespace_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_x-whatever_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_x_whatever_xml.expected t/data/tv_grep_eval_scalar_keys_5_all_UTF8.expected t/data/tv_grep_eval_scalar_keys_5_amp_xml.expected t/data/tv_grep_eval_scalar_keys_5_amp_xml_amp_xml.expected t/data/tv_grep_eval_scalar_keys_5_amp_xml_clump_xml.expected t/data/tv_grep_eval_scalar_keys_5_amp_xml_dups_xml.expected t/data/tv_grep_eval_scalar_keys_5_amp_xml_empty_xml.expected t/data/tv_grep_eval_scalar_keys_5_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_eval_scalar_keys_5_attrs_xml.expected t/data/tv_grep_eval_scalar_keys_5_clump_extract_1_xml.expected t/data/tv_grep_eval_scalar_keys_5_clump_extract_xml.expected t/data/tv_grep_eval_scalar_keys_5_clump_xml.expected t/data/tv_grep_eval_scalar_keys_5_clump_xml_amp_xml.expected t/data/tv_grep_eval_scalar_keys_5_clump_xml_clump_xml.expected t/data/tv_grep_eval_scalar_keys_5_clump_xml_dups_xml.expected t/data/tv_grep_eval_scalar_keys_5_clump_xml_empty_xml.expected t/data/tv_grep_eval_scalar_keys_5_dups_xml.expected t/data/tv_grep_eval_scalar_keys_5_dups_xml_amp_xml.expected t/data/tv_grep_eval_scalar_keys_5_sort_xml.expected t/data/tv_grep_eval_scalar_keys_5_dups_xml_clump_xml.expected t/data/tv_grep_eval_scalar_keys_5_dups_xml_dups_xml.expected t/data/tv_grep_eval_scalar_keys_5_dups_xml_empty_xml.expected t/data/tv_grep_eval_scalar_keys_5_empty_xml.expected t/data/tv_grep_eval_scalar_keys_5_empty_xml_amp_xml.expected t/data/tv_grep_eval_scalar_keys_5_empty_xml_clump_xml.expected t/data/tv_grep_eval_scalar_keys_5_empty_xml_dups_xml.expected t/data/tv_grep_eval_scalar_keys_5_empty_xml_empty_xml.expected t/data/tv_grep_eval_scalar_keys_5_length_xml.expected t/data/tv_grep_eval_scalar_keys_5_overlap_xml.expected t/data/tv_grep_eval_scalar_keys_5_simple_xml.expected t/data/tv_grep_eval_scalar_keys_5_simple_xml_x_whatever_xml.expected t/data/tv_grep_eval_scalar_keys_5_test_empty_xml.expected t/data/tv_grep_eval_scalar_keys_5_test_livre_xml.expected t/data/tv_grep_eval_scalar_keys_5_test_sort_by_channel_xml.expected t/data/tv_grep_eval_scalar_keys_5_test_xml.expected t/data/tv_grep_eval_scalar_keys_5_test_xml_test_xml.expected t/data/tv_grep_eval_scalar_keys_5_whitespace_xml.expected t/data/tv_grep_eval_scalar_keys_5_x_whatever_xml.expected t/data/tv_grep_i_category_i_title_h_all_UTF8.expected t/data/tv_grep_i_category_i_title_h_amp_xml.expected t/data/tv_grep_i_category_i_title_h_amp_xml_amp_xml.expected t/data/tv_grep_i_category_i_title_h_amp_xml_clump_xml.expected t/data/tv_grep_i_category_i_title_h_amp_xml_dups_xml.expected t/data/tv_grep_i_category_i_title_h_amp_xml_empty_xml.expected t/data/tv_grep_i_category_i_title_h_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_i_category_i_title_h_attrs_xml.expected t/data/tv_grep_i_category_i_title_h_clump_extract_1_xml.expected t/data/tv_grep_i_category_i_title_h_clump_extract_xml.expected t/data/tv_grep_i_category_i_title_h_clump_xml.expected t/data/tv_grep_i_category_i_title_h_clump_xml_amp_xml.expected t/data/tv_grep_i_category_i_title_h_clump_xml_clump_xml.expected t/data/tv_grep_i_category_i_title_h_clump_xml_dups_xml.expected t/data/tv_grep_i_category_i_title_h_clump_xml_empty_xml.expected t/data/tv_grep_i_category_i_title_h_dups_xml.expected t/data/tv_grep_i_category_i_title_h_dups_xml_amp_xml.expected t/data/tv_grep_i_category_i_title_h_dups_xml_clump_xml.expected t/data/tv_grep_i_category_i_title_h_dups_xml_dups_xml.expected t/data/tv_grep_i_category_i_title_h_dups_xml_empty_xml.expected t/data/tv_grep_i_category_i_title_h_empty_xml.expected t/data/tv_grep_i_category_i_title_h_empty_xml_amp_xml.expected t/data/tv_grep_i_category_i_title_h_empty_xml_clump_xml.expected t/data/tv_grep_i_category_i_title_h_empty_xml_dups_xml.expected t/data/tv_grep_i_category_i_title_h_empty_xml_empty_xml.expected t/data/tv_grep_i_category_i_title_h_length_xml.expected t/data/tv_grep_i_category_i_title_h_overlap_xml.expected t/data/tv_grep_i_category_i_title_h_simple_xml.expected t/data/tv_grep_i_category_i_title_h_simple_xml_x_whatever_xml.expected t/data/tv_grep_i_category_i_title_h_sort_xml.expected t/data/tv_grep_i_category_i_title_h_test_empty_xml.expected t/data/tv_grep_i_category_i_title_h_test_livre_xml.expected t/data/tv_grep_i_category_i_title_h_test_sort_by_channel_xml.expected t/data/tv_grep_i_category_i_title_h_test_xml.expected t/data/tv_grep_i_category_i_title_h_test_xml_test_xml.expected t/data/tv_grep_i_category_i_title_h_whitespace_xml.expected t/data/tv_grep_i_category_i_title_h_x_whatever_xml.expected t/data/tv_grep_i_category_i_title_j_all_UTF8.expected t/data/tv_grep_i_category_i_title_j_amp_xml.expected t/data/tv_grep_i_category_i_title_j_amp_xml_amp_xml.expected t/data/tv_grep_i_category_i_title_j_amp_xml_clump_xml.expected t/data/tv_grep_i_category_i_title_j_amp_xml_dups_xml.expected t/data/tv_grep_i_category_i_title_j_amp_xml_empty_xml.expected t/data/tv_grep_i_category_i_title_j_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_i_category_i_title_j_attrs_xml.expected t/data/tv_grep_i_category_i_title_j_clump_extract_1_xml.expected t/data/whitespace.xml t/data/tv_grep_i_category_i_title_j_clump_extract_xml.expected t/data/tv_grep_i_category_i_title_j_clump_xml.expected t/data/tv_grep_i_category_i_title_j_clump_xml_amp_xml.expected t/data/tv_grep_i_category_i_title_j_clump_xml_clump_xml.expected t/data/tv_grep_i_category_i_title_j_clump_xml_dups_xml.expected t/data/tv_grep_i_category_i_title_j_clump_xml_empty_xml.expected t/data/tv_grep_i_category_i_title_j_dups_xml.expected t/data/tv_grep_i_category_i_title_j_dups_xml_amp_xml.expected t/data/tv_grep_i_category_i_title_j_dups_xml_clump_xml.expected t/data/tv_grep_i_category_i_title_j_dups_xml_dups_xml.expected t/data/tv_grep_i_category_i_title_j_dups_xml_empty_xml.expected t/data/tv_grep_i_category_i_title_j_empty_xml.expected t/data/tv_grep_i_category_i_title_j_empty_xml_amp_xml.expected t/data/tv_grep_i_category_i_title_j_empty_xml_clump_xml.expected t/data/tv_grep_i_category_i_title_j_empty_xml_dups_xml.expected t/data/tv_grep_i_category_i_title_j_empty_xml_empty_xml.expected t/data/tv_grep_i_category_i_title_j_length_xml.expected t/data/tv_grep_i_category_i_title_j_overlap_xml.expected t/data/tv_grep_i_category_i_title_j_simple_xml.expected t/data/tv_grep_i_category_i_title_j_simple_xml_x_whatever_xml.expected t/data/tv_grep_i_category_i_title_j_sort_xml.expected t/data/tv_grep_i_category_i_title_j_test_empty_xml.expected t/data/tv_grep_i_category_i_title_j_test_livre_xml.expected t/data/tv_grep_i_category_i_title_j_test_sort_by_channel_xml.expected t/data/tv_grep_i_category_i_title_j_test_xml.expected t/data/tv_grep_i_category_i_title_j_test_xml_test_xml.expected t/data/tv_grep_i_category_i_title_j_whitespace_xml.expected t/data/tv_grep_i_category_i_title_j_x_whatever_xml.expected t/data/tv_grep_i_last_chance_c_all_UTF8.expected t/data/tv_grep_i_last_chance_c_amp_xml.expected t/data/tv_grep_i_last_chance_c_amp_xml_amp_xml.expected t/data/tv_grep_i_last_chance_c_amp_xml_clump_xml.expected t/data/tv_grep_i_last_chance_c_amp_xml_dups_xml.expected t/data/tv_grep_i_last_chance_c_amp_xml_empty_xml.expected t/data/tv_grep_i_last_chance_c_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_i_last_chance_c_attrs_xml.expected t/data/tv_grep_i_last_chance_c_clump_extract_1_xml.expected t/data/tv_grep_i_last_chance_c_clump_extract_xml.expected t/data/tv_grep_i_last_chance_c_clump_xml.expected t/data/tv_grep_i_last_chance_c_clump_xml_amp_xml.expected t/data/tv_grep_i_last_chance_c_clump_xml_clump_xml.expected t/data/tv_grep_i_last_chance_c_clump_xml_dups_xml.expected t/data/tv_grep_i_last_chance_c_clump_xml_empty_xml.expected t/data/tv_grep_i_last_chance_c_dups_xml.expected t/data/tv_grep_i_last_chance_c_dups_xml_amp_xml.expected t/data/tv_grep_i_last_chance_c_dups_xml_clump_xml.expected t/data/tv_grep_i_last_chance_c_dups_xml_dups_xml.expected t/data/tv_grep_i_last_chance_c_dups_xml_empty_xml.expected t/data/tv_grep_i_last_chance_c_empty_xml.expected t/data/tv_grep_i_last_chance_c_empty_xml_amp_xml.expected t/data/tv_grep_i_last_chance_c_empty_xml_clump_xml.expected t/data/tv_grep_i_last_chance_c_empty_xml_dups_xml.expected t/data/tv_grep_i_last_chance_c_empty_xml_empty_xml.expected t/data/tv_grep_i_last_chance_c_length_xml.expected t/data/tv_grep_i_last_chance_c_overlap_xml.expected t/data/tv_grep_i_last_chance_c_simple_xml.expected t/data/tv_grep_i_last_chance_c_simple_xml_x_whatever_xml.expected t/data/tv_grep_i_last_chance_c_sort_xml.expected t/data/tv_grep_i_last_chance_c_test_empty_xml.expected t/data/tv_grep_i_last_chance_c_test_livre_xml.expected t/data/tv_grep_i_last_chance_c_test_sort_by_channel_xml.expected t/data/tv_grep_i_last_chance_c_test_xml.expected t/data/tv_grep_i_last_chance_c_test_xml_test_xml.expected t/data/tv_grep_i_last_chance_c_whitespace_xml.expected t/data/tv_grep_i_last_chance_c_x_whatever_xml.expected t/data/tv_grep_new_amp_xml.expected t/data/tv_grep_new_amp_xml_amp_xml.expected t/data/tv_grep_new_amp_xml_clump_xml.expected t/data/tv_grep_new_amp_xml_dups_xml.expected t/data/x-whatever.xml t/data/tv_grep_new_amp_xml_empty_xml.expected t/data/tv_grep_new_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_new_attrs_xml.expected t/data/tv_grep_new_clump_extract_1_xml.expected t/data/tv_grep_new_clump_extract_xml.expected t/data/tv_grep_new_clump_xml.expected t/data/tv_grep_new_clump_xml_amp_xml.expected t/data/tv_grep_new_clump_xml_clump_xml.expected t/data/tv_grep_new_clump_xml_dups_xml.expected t/data/tv_grep_new_clump_xml_empty_xml.expected t/data/tv_grep_new_dups_xml.expected t/data/tv_grep_new_dups_xml_amp_xml.expected t/data/tv_grep_new_dups_xml_clump_xml.expected t/data/tv_grep_new_dups_xml_dups_xml.expected t/data/tv_grep_new_dups_xml_empty_xml.expected t/data/tv_grep_new_empty_xml.expected t/data/tv_grep_new_empty_xml_amp_xml.expected t/data/tv_grep_new_empty_xml_clump_xml.expected t/data/tv_grep_new_empty_xml_dups_xml.expected t/data/tv_grep_new_empty_xml_empty_xml.expected t/data/tv_grep_new_length_xml.expected t/data/tv_grep_new_overlap_xml.expected t/data/tv_grep_new_simple_xml.expected t/data/tv_grep_new_simple_xml_x_whatever_xml.expected t/data/tv_grep_new_sort_xml.expected t/data/tv_grep_new_test_empty_xml.expected t/data/tv_grep_new_test_livre_xml.expected t/data/tv_grep_new_test_sort_by_channel_xml.expected t/data/tv_grep_new_test_xml.expected t/data/tv_grep_new_test_xml_test_xml.expected t/data/tv_grep_new_whitespace_xml.expected t/data/tv_grep_new_x_whatever_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_all_UTF8.expected t/data/tv_grep_on_after_2002_02_05_UTC_amp_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_amp_xml_amp_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_amp_xml_clump_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_amp_xml_dups_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_amp_xml_empty_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_attrs_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_clump_extract_1_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_clump_extract_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_clump_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_clump_xml_amp_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_clump_xml_clump_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_clump_xml_dups_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_clump_xml_empty_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_dups_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_dups_xml_amp_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_dups_xml_clump_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_dups_xml_dups_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_dups_xml_empty_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_empty_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_empty_xml_amp_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_empty_xml_clump_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_empty_xml_dups_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_empty_xml_empty_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_length_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_overlap_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_simple_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_simple_xml_x_whatever_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_sort_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_test_empty_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_test_livre_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_test_sort_by_channel_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_test_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_test_xml_test_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_whitespace_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_x_whatever_xml.expected t/data/tv_grep_premiere_all_UTF8.expected t/data/tv_grep_premiere_amp_xml.expected t/data/tv_grep_premiere_amp_xml_amp_xml.expected t/data/tv_grep_premiere_amp_xml_clump_xml.expected t/data/tv_grep_premiere_amp_xml_dups_xml.expected t/data/tv_grep_premiere_amp_xml_empty_xml.expected t/data/tv_grep_premiere_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_premiere_attrs_xml.expected t/data/tv_grep_premiere_clump_extract_1_xml.expected t/data/tv_grep_premiere_clump_extract_xml.expected t/data/tv_grep_premiere_clump_xml.expected t/data/tv_grep_premiere_clump_xml_amp_xml.expected t/data/tv_grep_premiere_clump_xml_clump_xml.expected t/data/tv_grep_premiere_clump_xml_dups_xml.expected t/data/tv_grep_premiere_clump_xml_empty_xml.expected t/data/tv_grep_premiere_dups_xml.expected t/data/tv_grep_premiere_dups_xml_amp_xml.expected t/data/tv_grep_premiere_dups_xml_clump_xml.expected t/data/tv_grep_premiere_dups_xml_dups_xml.expected t/data/tv_grep_premiere_dups_xml_empty_xml.expected t/data/tv_grep_premiere_empty_xml.expected t/data/tv_grep_premiere_empty_xml_amp_xml.expected t/data/tv_grep_premiere_empty_xml_clump_xml.expected t/data/tv_grep_premiere_empty_xml_dups_xml.expected t/data/tv_grep_premiere_empty_xml_empty_xml.expected t/data/tv_grep_premiere_length_xml.expected t/data/tv_grep_premiere_overlap_xml.expected t/data/tv_grep_premiere_simple_xml.expected t/data/tv_grep_premiere_simple_xml_x_whatever_xml.expected t/data/tv_grep_premiere_sort_xml.expected t/data/tv_grep_premiere_test_empty_xml.expected t/data/tv_grep_premiere_test_livre_xml.expected t/data/tv_grep_premiere_test_sort_by_channel_xml.expected t/data/tv_grep_premiere_test_xml.expected t/data/tv_grep_premiere_test_xml_test_xml.expected t/data/tv_grep_premiere_whitespace_xml.expected t/data/tv_grep_premiere_x_whatever_xml.expected t/data/tv_grep_previously_shown_all_UTF8.expected t/data/tv_grep_previously_shown_amp_xml.expected t/data/tv_grep_previously_shown_amp_xml_amp_xml.expected t/data/tv_grep_previously_shown_amp_xml_clump_xml.expected t/data/tv_grep_previously_shown_amp_xml_dups_xml.expected t/data/tv_grep_previously_shown_amp_xml_empty_xml.expected t/data/tv_grep_previously_shown_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_previously_shown_attrs_xml.expected t/data/tv_grep_previously_shown_clump_extract_1_xml.expected t/data/tv_grep_previously_shown_clump_extract_xml.expected t/data/tv_grep_previously_shown_clump_xml.expected t/data/tv_grep_previously_shown_clump_xml_amp_xml.expected t/data/tv_grep_previously_shown_clump_xml_clump_xml.expected t/data/tv_grep_previously_shown_clump_xml_dups_xml.expected t/data/tv_grep_previously_shown_clump_xml_empty_xml.expected t/data/tv_grep_previously_shown_dups_xml.expected t/data/tv_grep_previously_shown_dups_xml_amp_xml.expected t/data/tv_grep_previously_shown_dups_xml_clump_xml.expected t/data/tv_grep_previously_shown_dups_xml_dups_xml.expected t/data/tv_grep_previously_shown_dups_xml_empty_xml.expected t/data/tv_grep_previously_shown_empty_xml.expected t/data/tv_grep_previously_shown_empty_xml_amp_xml.expected t/data/tv_grep_previously_shown_empty_xml_clump_xml.expected t/data/tv_grep_previously_shown_empty_xml_dups_xml.expected t/data/tv_grep_previously_shown_empty_xml_empty_xml.expected t/data/tv_grep_previously_shown_length_xml.expected t/data/tv_grep_previously_shown_overlap_xml.expected t/data/tv_grep_previously_shown_simple_xml.expected t/data/tv_grep_previously_shown_simple_xml_x_whatever_xml.expected t/data/tv_grep_previously_shown_sort_xml.expected t/data/tv_grep_previously_shown_test_empty_xml.expected t/data/tv_grep_previously_shown_test_livre_xml.expected t/data/tv_grep_previously_shown_test_sort_by_channel_xml.expected t/data/tv_grep_previously_shown_test_xml.expected t/data/tv_grep_previously_shown_test_xml_test_xml.expected t/data/tv_grep_previously_shown_whitespace_xml.expected t/data/tv_grep_previously_shown_x_whatever_xml.expected t/data-tv_augment/configs/tv_augment_automatic_type_1.xml.conf t/data-tv_augment/configs/tv_augment_automatic_type_2.xml.conf t/data-tv_augment/configs/tv_augment_automatic_type_3.xml.conf t/data-tv_augment/configs/tv_augment_automatic_type_4.xml.conf t/data-tv_augment/configs/tv_augment_automatic_type_5-0a.xml.conf t/data-tv_augment/configs/tv_augment_automatic_type_5-0b.xml.conf t/data-tv_augment/configs/tv_augment_automatic_type_5-1.xml.conf t/data-tv_augment/configs/tv_augment_automatic_type_5-2.xml.conf t/data-tv_augment/configs/tv_augment_automatic_type_5-3.xml.conf t/data-tv_augment/configs/tv_augment_automatic_type_5-4.xml.conf t/data-tv_augment/configs/tv_augment_user_type_1.xml.conf t/data-tv_augment/configs/tv_augment_user_type_10.xml.conf t/data-tv_augment/configs/tv_augment_user_type_11.xml.conf t/data-tv_augment/configs/tv_augment_user_type_12.xml.conf t/data-tv_augment/configs/tv_augment_user_type_13.xml.conf t/data-tv_augment/configs/tv_augment_user_type_14.xml.conf t/data-tv_augment/configs/tv_augment_user_type_15.xml.conf t/data-tv_augment/configs/tv_augment_user_type_16.xml.conf t/data-tv_augment/configs/tv_augment_user_type_2.xml.conf t/data-tv_augment/configs/tv_augment_user_type_3.xml.conf t/data-tv_augment/configs/tv_augment_user_type_4.xml.conf t/data-tv_augment/configs/tv_augment_user_type_5.xml.conf t/data-tv_augment/configs/tv_augment_user_type_6.xml.conf t/data-tv_augment/configs/tv_augment_user_type_7.xml.conf t/data-tv_augment/configs/tv_augment_user_type_8.xml.conf t/data-tv_augment/configs/tv_augment_user_type_9.xml.conf t/data-tv_augment/rules/test_tv_augment.rules t/data-tv_augment/tv_augment_automatic_type_1.xml t/data-tv_augment/tv_augment_automatic_type_1.xml-expected t/data-tv_augment/tv_augment_automatic_type_2.xml t/data-tv_augment/tv_augment_automatic_type_2.xml-expected t/data-tv_augment/tv_augment_automatic_type_3.xml t/data-tv_augment/tv_augment_automatic_type_3.xml-expected t/data-tv_augment/tv_augment_automatic_type_4.xml t/data-tv_augment/tv_augment_automatic_type_4.xml-expected t/data-tv_augment/tv_augment_automatic_type_5-0a.xml t/data-tv_augment/tv_augment_automatic_type_5-0a.xml-expected t/data-tv_augment/tv_augment_automatic_type_5-0b.xml t/data-tv_augment/tv_augment_automatic_type_5-0b.xml-expected t/data-tv_augment/tv_augment_automatic_type_5-1.xml t/data-tv_augment/tv_augment_automatic_type_5-1.xml-expected t/data-tv_augment/tv_augment_automatic_type_5-2.xml t/data-tv_augment/tv_augment_automatic_type_5-2.xml-expected t/data-tv_augment/tv_augment_automatic_type_5-3.xml t/data-tv_augment/tv_augment_automatic_type_5-3.xml-expected t/data-tv_augment/tv_augment_automatic_type_5-4.xml t/data-tv_augment/tv_augment_automatic_type_5-4.xml-expected t/data-tv_augment/tv_augment_user_type_1.xml t/data-tv_augment/tv_augment_user_type_1.xml-expected t/data-tv_augment/tv_augment_user_type_10.xml t/data-tv_augment/tv_augment_user_type_10.xml-expected t/data-tv_augment/tv_augment_user_type_11.xml t/data-tv_augment/tv_augment_user_type_11.xml-expected t/data-tv_augment/tv_augment_user_type_12.xml t/data-tv_augment/tv_augment_user_type_12.xml-expected t/data-tv_augment/tv_augment_user_type_13.xml t/data-tv_augment/tv_augment_user_type_13.xml-expected t/data-tv_augment/tv_augment_user_type_14.xml t/data-tv_augment/tv_augment_user_type_14.xml-expected t/data-tv_augment/tv_augment_user_type_15.xml t/data-tv_augment/tv_augment_user_type_15.xml-expected t/data-tv_augment/tv_augment_user_type_16.xml t/data-tv_augment/tv_augment_user_type_16.xml-expected t/data-tv_augment/tv_augment_user_type_2.xml t/data-tv_augment/tv_augment_user_type_2.xml-expected t/data-tv_augment/tv_augment_user_type_3.xml t/data-tv_augment/tv_augment_user_type_3.xml-expected t/data-tv_augment/tv_augment_user_type_4.xml t/data-tv_augment/tv_augment_user_type_4.xml-expected t/data-tv_augment/tv_augment_user_type_5.xml t/data-tv_augment/tv_augment_user_type_5.xml-expected t/data-tv_augment/tv_augment_user_type_6.xml t/data-tv_augment/tv_augment_user_type_6.xml-expected t/data-tv_augment/tv_augment_user_type_7.xml t/data-tv_augment/tv_augment_user_type_7.xml-expected t/data-tv_augment/tv_augment_user_type_8.xml t/data-tv_augment/tv_augment_user_type_8.xml-expected t/data-tv_augment/tv_augment_user_type_9.xml t/data-tv_augment/tv_augment_user_type_9.xml-expected t/test_tv_augment.t t/data-tv_imdb/lists/actors.list t/data-tv_imdb/lists/actresses.list t/data-tv_imdb/lists/directors.list t/data-tv_imdb/lists/genres.list t/data-tv_imdb/lists/keywords.list t/data-tv_imdb/lists/movies.list t/data-tv_imdb/lists/plot.list t/data-tv_imdb/lists/ratings.list t/data-tv_imdb/After-data-freeze.xml t/data-tv_imdb/After-data-freeze.xml-expected t/data-tv_imdb/Cast-actor-with-generation.xml t/data-tv_imdb/Cast-actor-with-generation.xml-expected t/data-tv_imdb/Cast-actors-and-actresses.xml t/data-tv_imdb/Cast-actors-and-actresses.xml-expected t/data-tv_imdb/Cast-billing.xml t/data-tv_imdb/Cast-billing.xml-expected t/data-tv_imdb/Cast-duplicate.xml t/data-tv_imdb/Cast-duplicate.xml-expected t/data-tv_imdb/Cast-host-or-narrator.xml t/data-tv_imdb/Cast-host-or-narrator.xml-expected t/data-tv_imdb/Cast-name-with-suffix.xml t/data-tv_imdb/Cast-name-with-suffix.xml-expected t/data-tv_imdb/Cast-role.xml t/data-tv_imdb/Cast-role.xml-expected t/data-tv_imdb/Director-multiple-and-duplicate-directors.xml t/data-tv_imdb/Director-multiple-and-duplicate-directors.xml-expected t/data-tv_imdb/Director-name-with-suffix.xml t/data-tv_imdb/Director-name-with-suffix.xml-expected t/data-tv_imdb/Director-with-generation.xml t/data-tv_imdb/Director-with-generation.xml-expected t/data-tv_imdb/Genres-duplicate.xml t/data-tv_imdb/Genres-duplicate.xml-expected t/data-tv_imdb/Genres-multiple.xml t/data-tv_imdb/Genres-multiple.xml-expected t/data-tv_imdb/Genres-single.xml t/data-tv_imdb/Genres-single.xml-expected t/data-tv_imdb/Movie1.xml t/data-tv_imdb/Movie1.xml-expected t/data-tv_imdb/Movie1-case-insensitive.xml t/data-tv_imdb/Movie1-case-insensitive.xml-expected t/data-tv_imdb/Movie1-movies-only.xml t/data-tv_imdb/Movie1-movies-only.xml-expected t/data-tv_imdb/Movie3-and-amp.xml t/data-tv_imdb/Movie3-and-amp.xml-expected t/data-tv_imdb/Movie5-ignore-punc.xml t/data-tv_imdb/Movie5-ignore-punc.xml-expected t/data-tv_imdb/Movie5-with-punc.xml t/data-tv_imdb/Movie5-with-punc.xml-expected t/data-tv_imdb/Movie6-articles.xml t/data-tv_imdb/Movie6-articles.xml-expected t/data-tv_imdb/Movie21-accents.xml t/data-tv_imdb/Movie21-accents.xml-expected t/data-tv_imdb/Movie22-dots.xml t/data-tv_imdb/Movie22-dots.xml-expected t/data-tv_imdb/Movie100-years.xml t/data-tv_imdb/Movie100-years.xml-expected t/data-tv_imdb/Movie101-movie-and-tv.xml t/data-tv_imdb/Movie101-movie-and-tv.xml-expected t/data-tv_imdb/Movie-same-year-movie-and-series.xml t/data-tv_imdb/Movie-same-year-movie-and-series.xml-expected t/data-tv_imdb/Movie-startswith-hyphen.xml t/data-tv_imdb/Movie-startswith-hyphen.xml-expected t/data-tv_imdb/Movie-two-in-same-year.xml t/data-tv_imdb/Movie-two-in-same-year.xml-expected t/data-tv_imdb/Movie-with-aka.xml t/data-tv_imdb/Movie-with-aka.xml-expected t/data-tv_imdb/Movie-with-unknown-year.xml t/data-tv_imdb/Movie-with-unknown-year.xml-expected t/data-tv_imdb/Ratings.xml t/data-tv_imdb/Ratings.xml-expected t/data-tv_imdb/Show1.xml t/data-tv_imdb/Show1.xml-expected t/data-tv_imdb/Show1-movies-only.xml t/data-tv_imdb/Show1-movies-only.xml-expected t/test_tv_imdb.t t/data-tv_tmdb/Cast-actors-and-actresses.xml t/data-tv_tmdb/Cast-actors-and-actresses.xml-expected t/data-tv_tmdb/Cast-actor-with-generation.xml t/data-tv_tmdb/Cast-actor-with-generation.xml-expected t/data-tv_tmdb/Cast-billing.xml t/data-tv_tmdb/Cast-billing.xml-expected t/data-tv_tmdb/Cast-duplicate.xml t/data-tv_tmdb/Cast-duplicate.xml-expected t/data-tv_tmdb/Cast-host-or-narrator.xml t/data-tv_tmdb/Cast-host-or-narrator.xml-expected t/data-tv_tmdb/Cast-image-url.xml t/data-tv_tmdb/Cast-image-url.xml-expected t/data-tv_tmdb/Cast-merge.xml t/data-tv_tmdb/Cast-merge.xml-expected t/data-tv_tmdb/Cast-multiple-role.xml t/data-tv_tmdb/Cast-multiple-role.xml-expected t/data-tv_tmdb/Cast-role.xml t/data-tv_tmdb/Cast-role.xml-expected t/data-tv_tmdb/configs/Cast-actors-and-actresses.conf t/data-tv_tmdb/configs/Cast-actor-with-generation.conf t/data-tv_tmdb/configs/Cast-billing.conf t/data-tv_tmdb/configs/Cast-duplicate.conf t/data-tv_tmdb/configs/Cast-host-or-narrator.conf t/data-tv_tmdb/configs/Cast-image-url.conf t/data-tv_tmdb/configs/Cast-merge.conf t/data-tv_tmdb/configs/Cast-multiple-role.conf t/data-tv_tmdb/configs/Cast-role.conf t/data-tv_tmdb/configs/Content-id.conf t/data-tv_tmdb/configs/Director-multiple-and-duplicate-directors.conf t/data-tv_tmdb/configs/Director-with-generation.conf t/data-tv_tmdb/configs/Genres-duplicate.conf t/data-tv_tmdb/configs/Genres-multiple.conf t/data-tv_tmdb/configs/Genres-single.conf t/data-tv_tmdb/configs/Movie-and-tv.conf t/data-tv_tmdb/configs/Movie.conf t/data-tv_tmdb/configs/Movie-icon-and-url.conf t/data-tv_tmdb/configs/Movie-same-year-movie-and-series.conf t/data-tv_tmdb/configs/Movies-only.conf t/data-tv_tmdb/configs/Movie-two-in-same-year.conf t/data-tv_tmdb/configs/Movie-with-aka.conf t/data-tv_tmdb/configs/Movie-with-unknown-year.conf t/data-tv_tmdb/configs/Movie-years.conf t/data-tv_tmdb/configs/Ratings.conf t/data-tv_tmdb/configs/Show.conf t/data-tv_tmdb/configs/Show-movies-only.conf t/data-tv_tmdb/configs/Show-two-in-same-year.conf t/data-tv_tmdb/configs/Title-and-amp.conf t/data-tv_tmdb/configs/Title-articles.conf t/data-tv_tmdb/configs/Title-case-insensitive.conf t/data-tv_tmdb/configs/Title-dots.conf t/data-tv_tmdb/configs/Title-ignore-punc.conf t/data-tv_tmdb/configs/Title-startswith-hyphen.conf t/data-tv_tmdb/configs/Title-with-accent.conf t/data-tv_tmdb/configs/Title-with-not-az.conf t/data-tv_tmdb/configs/Title-with-punc.conf t/data-tv_tmdb/configs/Title-year-from-title.conf t/data-tv_tmdb/configs/Utf8.conf t/data-tv_tmdb/Content-id.xml t/data-tv_tmdb/Content-id.xml-expected t/data-tv_tmdb/Director-multiple-and-duplicate-directors.xml t/data-tv_tmdb/Director-multiple-and-duplicate-directors.xml-expected t/data-tv_tmdb/Director-with-generation.xml t/data-tv_tmdb/Director-with-generation.xml-expected t/data-tv_tmdb/Genres-duplicate.xml t/data-tv_tmdb/Genres-duplicate.xml-expected t/data-tv_tmdb/Genres-multiple.xml t/data-tv_tmdb/Genres-multiple.xml-expected t/data-tv_tmdb/Genres-single.xml t/data-tv_tmdb/Genres-single.xml-expected t/data-tv_tmdb/Movie-and-tv.xml t/data-tv_tmdb/Movie-and-tv.xml-expected t/data-tv_tmdb/Movie-icon-and-url.xml t/data-tv_tmdb/Movie-icon-and-url.xml-expected t/data-tv_tmdb/Movie-same-year-movie-and-series.xml t/data-tv_tmdb/Movie-same-year-movie-and-series.xml-expected t/data-tv_tmdb/Movies-only.xml t/data-tv_tmdb/Movies-only.xml-expected t/data-tv_tmdb/Movie-two-in-same-year.xml t/data-tv_tmdb/Movie-two-in-same-year.xml-expected t/data-tv_tmdb/Movie-with-aka.xml t/data-tv_tmdb/Movie-with-aka.xml-expected t/data-tv_tmdb/Movie-with-unknown-year.xml t/data-tv_tmdb/Movie-with-unknown-year.xml-expected t/data-tv_tmdb/Movie.xml t/data-tv_tmdb/Movie.xml-expected t/data-tv_tmdb/Movie-years.xml t/data-tv_tmdb/Movie-years.xml-expected t/data-tv_tmdb/Ratings.xml t/data-tv_tmdb/Ratings.xml-expected t/data-tv_tmdb/Show-movies-only.xml t/data-tv_tmdb/Show-movies-only.xml-expected t/data-tv_tmdb/Show-two-in-same-year.xml t/data-tv_tmdb/Show-two-in-same-year.xml-expected t/data-tv_tmdb/Show.xml t/data-tv_tmdb/Show.xml-expected t/data-tv_tmdb/Title-and-amp.xml t/data-tv_tmdb/Title-and-amp.xml-expected t/data-tv_tmdb/Title-articles.xml t/data-tv_tmdb/Title-articles.xml-expected t/data-tv_tmdb/Title-case-insensitive.xml t/data-tv_tmdb/Title-case-insensitive.xml-expected t/data-tv_tmdb/Title-dots.xml t/data-tv_tmdb/Title-dots.xml-expected t/data-tv_tmdb/Title-ignore-punc.xml t/data-tv_tmdb/Title-ignore-punc.xml-expected t/data-tv_tmdb/Title-startswith-hyphen.xml t/data-tv_tmdb/Title-startswith-hyphen.xml-expected t/data-tv_tmdb/Title-with-accent.xml t/data-tv_tmdb/Title-with-accent.xml-expected t/data-tv_tmdb/Title-with-not-az.xml t/data-tv_tmdb/Title-with-not-az.xml-expected t/data-tv_tmdb/Title-with-punc.xml t/data-tv_tmdb/Title-with-punc.xml-expected t/data-tv_tmdb/Title-year-from-title.xml t/data-tv_tmdb/Title-year-from-title.xml-expected t/data-tv_tmdb/Utf8.xml t/data-tv_tmdb/Utf8.xml-expected t/test_tv_tmdb.t t/data/tv_sort_all_UTF8.expected t/data/tv_sort_amp_xml_amp_xml.expected t/data/tv_sort_amp_xml_clump_xml.expected t/data/tv_sort_amp_xml_dups_xml.expected t/data/tv_sort_amp_xml_empty_xml.expected t/data/tv_sort_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_sort_attrs_xml.expected t/data/intervals.xml t/data/tv_sort_by_channel_all_UTF8.expected t/data/tv_sort_by_channel_amp_xml.expected t/data/tv_sort_by_channel_amp_xml_amp_xml.expected t/data/tv_sort_by_channel_amp_xml_clump_xml.expected t/data/tv_sort_by_channel_amp_xml_dups_xml.expected t/data/tv_sort_by_channel_amp_xml_empty_xml.expected t/data/tv_sort_by_channel_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_sort_by_channel_attrs_xml.expected t/data/tv_sort_by_channel_clump_extract_1_xml.expected t/data/tv_sort_by_channel_clump_extract_xml.expected t/data/tv_sort_by_channel_clump_xml.expected t/data/tv_sort_by_channel_clump_xml_amp_xml.expected t/data/tv_sort_by_channel_clump_xml_clump_xml.expected t/data/tv_sort_by_channel_clump_xml_dups_xml.expected t/data/tv_sort_by_channel_clump_xml_empty_xml.expected t/data/tv_sort_by_channel_dups_xml.expected t/data/tv_sort_by_channel_dups_xml_amp_xml.expected t/data/tv_sort_by_channel_dups_xml_clump_xml.expected t/data/tv_sort_by_channel_dups_xml_dups_xml.expected t/data/tv_sort_by_channel_dups_xml_empty_xml.expected t/data/tv_sort_by_channel_empty_xml.expected t/data/tv_sort_by_channel_empty_xml_amp_xml.expected t/data/tv_sort_by_channel_empty_xml_clump_xml.expected t/data/tv_sort_by_channel_empty_xml_dups_xml.expected t/data/tv_sort_by_channel_empty_xml_empty_xml.expected t/data/tv_sort_by_channel_length_xml.expected t/data/tv_sort_by_channel_overlap_xml.expected t/data/tv_sort_by_channel_simple_xml.expected t/data/tv_sort_by_channel_simple_xml_x_whatever_xml.expected t/data/tv_sort_by_channel_sort_xml.expected t/data/tv_sort_by_channel_test_empty_xml.expected t/data/tv_sort_by_channel_test_livre_xml.expected t/data/tv_sort_by_channel_test_sort_by_channel.expected t/data/tv_sort_by_channel_test_sort_by_channel_xml.expected t/data/tv_sort_by_channel_test_xml.expected t/data/tv_sort_by_channel_test_xml_test_xml.expected t/data/tv_sort_by_channel_whitespace_xml.expected t/data/tv_sort_by_channel_x_whatever_xml.expected t/data/tv_sort_clump_extract_1_xml.expected t/data/tv_sort_clump_extract_xml.expected t/data/tv_sort_clump_xml.expected t/data/tv_sort_clump_xml_amp_xml.expected t/data/tv_sort_clump_xml_clump_xml.expected t/data/tv_sort_clump_xml_dups_xml.expected t/data/tv_sort_clump_xml_empty_xml.expected t/data/tv_sort_dups_xml.expected t/data/tv_sort_dups_xml_amp_xml.expected t/data/tv_sort_dups_xml_clump_xml.expected t/data/tv_sort_dups_xml_dups_xml.expected t/data/tv_sort_dups_xml_empty_xml.expected t/data/tv_sort_empty_xml.expected t/data/tv_sort_empty_xml_amp_xml.expected t/data/tv_sort_empty_xml_clump_xml.expected t/data/tv_sort_empty_xml_dups_xml.expected t/data/tv_sort_empty_xml_empty_xml.expected t/data/tv_sort_length_xml.expected t/data/tv_sort_overlap_xml.expected t/data/tv_sort_overlap_xml.expected_err t/data/tv_sort_simple_xml.expected t/data/tv_sort_simple_xml_x_whatever_xml.expected t/data/tv_sort_sort_xml.expected t/data/tv_sort_test_empty_xml.expected t/data/tv_sort_test_livre_xml.expected t/data/tv_sort_test_sort_by_channel_xml.expected t/data/tv_sort_test_xml.expected t/data/tv_sort_test_xml_test_xml.expected t/data/tv_sort_whitespace_xml.expected t/data/tv_sort_x_whatever_xml.expected t/data/tv_to_latex_all_UTF8.expected t/data/tv_to_latex_amp_xml.expected t/data/tv_to_latex_amp_xml_amp_xml.expected t/data/tv_to_latex_amp_xml_clump_xml.expected t/data/tv_to_latex_amp_xml_dups_xml.expected t/data/tv_to_latex_amp_xml_empty_xml.expected t/data/tv_to_latex_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_to_latex_attrs_xml.expected t/data/tv_to_latex_clump_extract_1_xml.expected t/data/tv_to_latex_clump_extract_xml.expected t/data/tv_to_latex_clump_xml.expected t/data/tv_to_latex_clump_xml_amp_xml.expected t/data/tv_to_latex_clump_xml_clump_xml.expected t/data/tv_to_latex_clump_xml_dups_xml.expected t/data/tv_to_latex_clump_xml_empty_xml.expected t/data/tv_to_latex_dups_xml.expected t/data/tv_to_latex_dups_xml_amp_xml.expected t/data/tv_to_latex_dups_xml_clump_xml.expected t/data/tv_to_latex_dups_xml_dups_xml.expected t/data/tv_to_latex_dups_xml_empty_xml.expected t/data/tv_to_latex_empty_xml.expected t/data/tv_to_latex_empty_xml_amp_xml.expected t/data/tv_to_latex_empty_xml_clump_xml.expected t/data/tv_to_latex_empty_xml_dups_xml.expected t/data/tv_to_latex_empty_xml_empty_xml.expected t/data/tv_to_latex_length_xml.expected t/data/tv_to_latex_overlap_xml.expected t/data/tv_to_latex_simple_xml.expected t/data/tv_to_latex_simple_xml_x_whatever_xml.expected t/data/tv_to_latex_sort_xml.expected t/data/tv_to_latex_test_empty_xml.expected t/data/tv_to_latex_test_livre_xml.expected t/data/tv_to_latex_test_sort_by_channel_xml.expected t/data/tv_to_latex_test_xml.expected t/data/tv_to_latex_test_xml_test_xml.expected t/data/tv_to_latex_whitespace_xml.expected t/data/tv_to_latex_x_whatever_xml.expected t/data/tv_to_text_all_UTF8.expected t/data/tv_to_text_amp_xml.expected t/data/tv_to_text_amp_xml_amp_xml.expected t/data/tv_to_text_amp_xml_clump_xml.expected t/data/tv_to_text_amp_xml_dups_xml.expected t/data/tv_to_text_amp_xml_empty_xml.expected t/data/tv_to_text_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_to_text_attrs_xml.expected t/data/tv_to_text_clump_extract_1_xml.expected t/data/tv_to_text_clump_extract_xml.expected t/data/tv_to_text_clump_xml.expected t/data/tv_to_text_clump_xml_amp_xml.expected t/data/tv_to_text_clump_xml_clump_xml.expected t/data/tv_to_text_clump_xml_dups_xml.expected t/data/tv_to_text_clump_xml_empty_xml.expected t/data/tv_to_text_dups_xml.expected t/data/tv_to_text_dups_xml_amp_xml.expected t/data/tv_to_text_dups_xml_clump_xml.expected t/data/tv_to_text_dups_xml_dups_xml.expected t/data/tv_to_text_dups_xml_empty_xml.expected t/data/tv_to_text_empty_xml.expected t/data/tv_to_text_empty_xml_amp_xml.expected t/data/tv_to_text_empty_xml_clump_xml.expected t/data/tv_to_text_empty_xml_dups_xml.expected t/data/tv_to_text_empty_xml_empty_xml.expected t/data/tv_to_text_length_xml.expected t/data/tv_to_text_overlap_xml.expected t/data/tv_to_text_simple_xml.expected t/data/tv_to_text_simple_xml_x_whatever_xml.expected t/data/tv_to_text_sort_xml.expected t/data/tv_to_text_test_empty_xml.expected t/data/tv_to_text_test_livre_xml.expected t/data/tv_to_text_test_sort_by_channel_xml.expected t/data/tv_to_text_test_xml.expected t/data/tv_to_text_test_xml_test_xml.expected t/data/tv_to_text_whitespace_xml.expected t/data/tv_to_text_x_whatever_xml.expected t/data/tv_cat_intervals_xml.expected t/data/tv_extractinfo_en_intervals_xml.expected t/data/tv_grep_a_intervals_xml.expected t/data/tv_grep_category_b_intervals_xml.expected t/data/tv_grep_category_e_and_title_f_intervals_xml.expected t/data/tv_grep_category_g_or_title_h_intervals_xml.expected t/data/tv_grep_channel_id_channel4_com_intervals_xml.expected t/data/tv_grep_channel_name_d_intervals_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_intervals_xml.expected t/data/tv_grep_eval_scalar_keys_5_intervals_xml.expected t/data/tv_grep_i_category_i_title_h_intervals_xml.expected t/data/tv_grep_i_category_i_title_j_intervals_xml.expected t/data/tv_grep_i_last_chance_c_intervals_xml.expected t/data/tv_grep_new_intervals_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_intervals_xml.expected t/data/tv_grep_premiere_intervals_xml.expected t/data/tv_grep_previously_shown_intervals_xml.expected t/data/tv_sort_by_channel_intervals_xml.expected t/data/tv_sort_intervals_xml.expected t/data/tv_to_latex_intervals_xml.expected t/data/tv_to_text_intervals_xml.expected t/data/tv_grep_on_after_200302161330_UTC_all_UTF8.expected t/data/tv_grep_on_after_200302161330_UTC_amp_xml.expected t/data/tv_grep_on_after_200302161330_UTC_amp_xml_amp_xml.expected t/data/tv_grep_on_after_200302161330_UTC_amp_xml_clump_xml.expected t/data/tv_grep_on_after_200302161330_UTC_amp_xml_dups_xml.expected t/data/tv_grep_on_after_200302161330_UTC_amp_xml_empty_xml.expected t/data/tv_grep_on_after_200302161330_UTC_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_on_after_200302161330_UTC_attrs_xml.expected t/data/tv_grep_on_after_200302161330_UTC_clump_extract_1_xml.expected t/data/tv_grep_on_after_200302161330_UTC_clump_extract_xml.expected t/data/tv_grep_on_after_200302161330_UTC_clump_xml.expected t/data/tv_grep_on_after_200302161330_UTC_clump_xml_amp_xml.expected t/data/tv_grep_on_after_200302161330_UTC_clump_xml_clump_xml.expected t/data/tv_grep_on_after_200302161330_UTC_clump_xml_dups_xml.expected t/data/tv_grep_on_after_200302161330_UTC_clump_xml_empty_xml.expected t/data/tv_grep_on_after_200302161330_UTC_dups_xml.expected t/data/tv_grep_on_after_200302161330_UTC_dups_xml_amp_xml.expected t/data/tv_grep_on_after_200302161330_UTC_dups_xml_clump_xml.expected t/data/tv_grep_on_after_200302161330_UTC_dups_xml_dups_xml.expected t/data/tv_grep_on_after_200302161330_UTC_dups_xml_empty_xml.expected t/data/tv_grep_on_after_200302161330_UTC_empty_xml.expected t/data/tv_grep_on_after_200302161330_UTC_empty_xml_amp_xml.expected t/data/tv_grep_on_after_200302161330_UTC_empty_xml_clump_xml.expected t/data/tv_grep_on_after_200302161330_UTC_empty_xml_dups_xml.expected t/data/tv_grep_on_after_200302161330_UTC_empty_xml_empty_xml.expected t/data/tv_grep_on_after_200302161330_UTC_intervals_xml.expected t/data/tv_grep_on_after_200302161330_UTC_length_xml.expected t/data/tv_grep_on_after_200302161330_UTC_overlap_xml.expected t/data/tv_grep_on_after_200302161330_UTC_simple_xml.expected t/data/tv_grep_on_after_200302161330_UTC_simple_xml_x_whatever_xml.expected t/data/tv_grep_on_after_200302161330_UTC_sort_xml.expected t/data/tv_grep_on_after_200302161330_UTC_test_empty_xml.expected t/data/tv_grep_on_after_200302161330_UTC_test_livre_xml.expected t/data/tv_grep_on_after_200302161330_UTC_test_sort_by_channel_xml.expected t/data/tv_grep_on_after_200302161330_UTC_test_xml.expected t/data/tv_grep_on_after_200302161330_UTC_test_xml_test_xml.expected t/data/tv_grep_on_after_200302161330_UTC_whitespace_xml.expected t/data/tv_grep_on_after_200302161330_UTC_x_whatever_xml.expected t/data/tv_grep_on_before_200302161330_UTC_all_UTF8.expected t/data/tv_grep_on_before_200302161330_UTC_amp_xml.expected t/data/tv_grep_on_before_200302161330_UTC_amp_xml_amp_xml.expected t/data/tv_grep_on_before_200302161330_UTC_amp_xml_clump_xml.expected t/data/tv_grep_on_before_200302161330_UTC_amp_xml_dups_xml.expected t/data/tv_grep_on_before_200302161330_UTC_amp_xml_empty_xml.expected t/data/tv_grep_on_before_200302161330_UTC_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_on_before_200302161330_UTC_attrs_xml.expected t/data/tv_grep_on_before_200302161330_UTC_clump_extract_1_xml.expected t/data/tv_grep_on_before_200302161330_UTC_clump_extract_xml.expected t/data/tv_grep_on_before_200302161330_UTC_clump_xml.expected t/data/tv_grep_on_before_200302161330_UTC_clump_xml_amp_xml.expected t/data/tv_grep_on_before_200302161330_UTC_clump_xml_clump_xml.expected t/data/tv_grep_on_before_200302161330_UTC_clump_xml_dups_xml.expected t/data/tv_grep_on_before_200302161330_UTC_clump_xml_empty_xml.expected t/data/tv_grep_on_before_200302161330_UTC_dups_xml.expected t/data/tv_grep_on_before_200302161330_UTC_dups_xml_amp_xml.expected t/data/tv_grep_on_before_200302161330_UTC_dups_xml_clump_xml.expected t/data/tv_grep_on_before_200302161330_UTC_dups_xml_dups_xml.expected t/data/tv_grep_on_before_200302161330_UTC_dups_xml_empty_xml.expected t/data/tv_grep_on_before_200302161330_UTC_empty_xml.expected t/data/tv_grep_on_before_200302161330_UTC_empty_xml_amp_xml.expected t/data/tv_grep_on_before_200302161330_UTC_empty_xml_clump_xml.expected t/data/tv_grep_on_before_200302161330_UTC_empty_xml_dups_xml.expected t/data/tv_grep_on_before_200302161330_UTC_empty_xml_empty_xml.expected t/data/tv_grep_on_before_200302161330_UTC_intervals_xml.expected t/data/tv_grep_on_before_200302161330_UTC_length_xml.expected t/data/tv_grep_on_before_200302161330_UTC_overlap_xml.expected t/data/tv_grep_on_before_200302161330_UTC_simple_xml.expected t/data/tv_grep_on_before_200302161330_UTC_simple_xml_x_whatever_xml.expected t/data/tv_grep_on_before_200302161330_UTC_sort_xml.expected t/data/tv_grep_on_before_200302161330_UTC_test_empty_xml.expected t/data/tv_grep_on_before_200302161330_UTC_test_livre_xml.expected t/data/tv_grep_on_before_200302161330_UTC_test_sort_by_channel_xml.expected t/data/tv_grep_on_before_200302161330_UTC_test_xml.expected t/data/tv_grep_on_before_200302161330_UTC_test_xml_test_xml.expected t/data/tv_grep_on_before_200302161330_UTC_whitespace_xml.expected t/data/tv_grep_on_before_200302161330_UTC_x_whatever_xml.expected t/data/tv_cat_sort1_xml.expected t/data/tv_cat_sort2_xml.expected t/data/tv_extractinfo_en_sort1_xml.expected t/data/tv_extractinfo_en_sort2_xml.expected t/data/tv_grep_a_sort1_xml.expected t/data/tv_grep_a_sort2_xml.expected t/data/tv_grep_category_b_sort1_xml.expected t/data/tv_grep_category_b_sort2_xml.expected t/data/tv_grep_category_e_and_title_f_sort1_xml.expected t/data/tv_grep_category_e_and_title_f_sort2_xml.expected t/data/tv_grep_category_g_or_title_h_sort1_xml.expected t/data/tv_grep_category_g_or_title_h_sort2_xml.expected t/data/tv_grep_channel_id_channel4_com_sort1_xml.expected t/data/tv_grep_channel_id_channel4_com_sort2_xml.expected t/data/tv_grep_channel_name_d_sort1_xml.expected t/data/tv_grep_channel_name_d_sort2_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_sort1_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_sort2_xml.expected t/data/tv_grep_eval_scalar_keys_5_sort1_xml.expected t/data/tv_grep_eval_scalar_keys_5_sort2_xml.expected t/data/tv_grep_i_category_i_title_h_sort1_xml.expected t/data/tv_grep_i_category_i_title_h_sort2_xml.expected t/data/tv_grep_i_category_i_title_j_sort1_xml.expected t/data/tv_grep_i_category_i_title_j_sort2_xml.expected t/data/tv_grep_i_last_chance_c_sort1_xml.expected t/data/tv_grep_i_last_chance_c_sort2_xml.expected t/data/tv_grep_new_sort1_xml.expected t/data/tv_grep_new_sort2_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_sort1_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_sort2_xml.expected t/data/tv_grep_on_after_200302161330_UTC_sort1_xml.expected t/data/tv_grep_on_after_200302161330_UTC_sort2_xml.expected t/data/tv_grep_on_before_200302161330_UTC_sort1_xml.expected t/data/tv_grep_on_before_200302161330_UTC_sort2_xml.expected t/data/tv_grep_premiere_sort1_xml.expected t/data/tv_grep_premiere_sort2_xml.expected t/data/tv_grep_previously_shown_sort1_xml.expected t/data/tv_grep_previously_shown_sort2_xml.expected t/data/tv_sort_by_channel_sort1_xml.expected t/data/tv_sort_by_channel_sort2_xml.expected t/data/tv_sort_sort1_xml.expected t/data/tv_sort_sort2_xml.expected t/data/tv_to_latex_sort1_xml.expected t/data/tv_to_latex_sort2_xml.expected t/data/tv_to_text_sort1_xml.expected t/data/tv_to_text_sort2_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_all_UTF8.expected t/data/tv_grep_channel_xyz_or_not_channel_b_amp_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_amp_xml_amp_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_amp_xml_clump_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_amp_xml_dups_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_amp_xml_empty_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_attrs_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_clump_extract_1_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_clump_extract_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_clump_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_clump_xml_amp_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_clump_xml_clump_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_clump_xml_dups_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_clump_xml_empty_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_dups_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_dups_xml_amp_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_dups_xml_clump_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_dups_xml_dups_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_dups_xml_empty_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_empty_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_empty_xml_amp_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_empty_xml_clump_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_empty_xml_dups_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_empty_xml_empty_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_intervals_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_length_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_overlap_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_simple_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_simple_xml_x_whatever_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_sort1_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_sort2_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_sort_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_test_empty_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_test_livre_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_test_sort_by_channel_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_test_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_test_xml_test_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_whitespace_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_x_whatever_xml.expected t/data/tv_grep_not_channel_id_channel4_com_all_UTF8.expected t/data/tv_grep_not_channel_id_channel4_com_amp_xml.expected t/data/tv_grep_not_channel_id_channel4_com_amp_xml_amp_xml.expected t/data/tv_grep_not_channel_id_channel4_com_amp_xml_clump_xml.expected t/data/tv_grep_not_channel_id_channel4_com_amp_xml_dups_xml.expected t/data/tv_grep_not_channel_id_channel4_com_amp_xml_empty_xml.expected t/data/tv_grep_not_channel_id_channel4_com_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_not_channel_id_channel4_com_attrs_xml.expected t/data/tv_grep_not_channel_id_channel4_com_clump_extract_1_xml.expected t/data/tv_grep_not_channel_id_channel4_com_clump_extract_xml.expected t/data/tv_grep_not_channel_id_channel4_com_clump_xml.expected t/data/tv_grep_not_channel_id_channel4_com_clump_xml_amp_xml.expected t/data/tv_grep_not_channel_id_channel4_com_clump_xml_clump_xml.expected t/data/tv_grep_not_channel_id_channel4_com_clump_xml_dups_xml.expected t/data/tv_grep_not_channel_id_channel4_com_clump_xml_empty_xml.expected t/data/tv_grep_not_channel_id_channel4_com_dups_xml.expected t/data/tv_grep_not_channel_id_channel4_com_dups_xml_amp_xml.expected t/data/tv_grep_not_channel_id_channel4_com_dups_xml_clump_xml.expected t/data/tv_grep_not_channel_id_channel4_com_dups_xml_dups_xml.expected t/data/tv_grep_not_channel_id_channel4_com_dups_xml_empty_xml.expected t/data/tv_grep_not_channel_id_channel4_com_empty_xml.expected t/data/tv_grep_not_channel_id_channel4_com_empty_xml_amp_xml.expected t/data/tv_grep_not_channel_id_channel4_com_empty_xml_clump_xml.expected t/data/tv_grep_not_channel_id_channel4_com_empty_xml_dups_xml.expected t/data/tv_grep_not_channel_id_channel4_com_empty_xml_empty_xml.expected t/data/tv_grep_not_channel_id_channel4_com_intervals_xml.expected t/data/tv_grep_not_channel_id_channel4_com_length_xml.expected t/data/tv_grep_not_channel_id_channel4_com_overlap_xml.expected t/data/tv_grep_not_channel_id_channel4_com_simple_xml.expected t/data/tv_grep_not_channel_id_channel4_com_simple_xml_x_whatever_xml.expected t/data/tv_grep_not_channel_id_channel4_com_sort1_xml.expected t/data/tv_grep_not_channel_id_channel4_com_sort2_xml.expected t/data/tv_grep_not_channel_id_channel4_com_sort_xml.expected t/data/tv_grep_not_channel_id_channel4_com_test_empty_xml.expected t/data/tv_grep_not_channel_id_channel4_com_test_livre_xml.expected t/data/tv_grep_not_channel_id_channel4_com_test_sort_by_channel_xml.expected t/data/tv_grep_not_channel_id_channel4_com_test_xml.expected t/data/tv_grep_not_channel_id_channel4_com_test_xml_test_xml.expected t/data/tv_grep_not_channel_id_channel4_com_whitespace_xml.expected t/data/tv_grep_not_channel_id_channel4_com_x_whatever_xml.expected t/data/tv_grep_not_channel_name_d_all_UTF8.expected t/data/tv_grep_not_channel_name_d_amp_xml.expected t/data/tv_grep_not_channel_name_d_amp_xml_amp_xml.expected t/data/tv_grep_not_channel_name_d_amp_xml_clump_xml.expected t/data/tv_grep_not_channel_name_d_amp_xml_dups_xml.expected t/data/tv_grep_not_channel_name_d_amp_xml_empty_xml.expected t/data/tv_grep_not_channel_name_d_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_not_channel_name_d_attrs_xml.expected t/data/tv_grep_not_channel_name_d_clump_extract_1_xml.expected t/data/tv_grep_not_channel_name_d_clump_extract_xml.expected t/data/tv_grep_not_channel_name_d_clump_xml.expected t/data/tv_grep_not_channel_name_d_clump_xml_amp_xml.expected t/data/tv_grep_not_channel_name_d_clump_xml_clump_xml.expected t/data/tv_grep_not_channel_name_d_clump_xml_dups_xml.expected t/data/tv_grep_not_channel_name_d_clump_xml_empty_xml.expected t/data/tv_grep_not_channel_name_d_dups_xml.expected t/data/tv_grep_not_channel_name_d_dups_xml_amp_xml.expected t/data/tv_grep_not_channel_name_d_dups_xml_clump_xml.expected t/data/tv_grep_not_channel_name_d_dups_xml_dups_xml.expected t/data/tv_grep_not_channel_name_d_dups_xml_empty_xml.expected t/data/tv_grep_not_channel_name_d_empty_xml.expected t/data/tv_grep_not_channel_name_d_empty_xml_amp_xml.expected t/data/tv_grep_not_channel_name_d_empty_xml_clump_xml.expected t/data/tv_grep_not_channel_name_d_empty_xml_dups_xml.expected t/data/tv_grep_not_channel_name_d_empty_xml_empty_xml.expected t/data/tv_grep_not_channel_name_d_intervals_xml.expected t/data/tv_grep_not_channel_name_d_length_xml.expected t/data/tv_grep_not_channel_name_d_overlap_xml.expected t/data/tv_grep_not_channel_name_d_simple_xml.expected t/data/tv_grep_not_channel_name_d_simple_xml_x_whatever_xml.expected t/data/tv_grep_not_channel_name_d_sort1_xml.expected t/data/tv_grep_not_channel_name_d_sort2_xml.expected t/data/tv_grep_not_channel_name_d_sort_xml.expected t/data/tv_grep_not_channel_name_d_test_empty_xml.expected t/data/tv_grep_not_channel_name_d_test_livre_xml.expected t/data/tv_grep_not_channel_name_d_test_sort_by_channel_xml.expected t/data/tv_grep_not_channel_name_d_test_xml.expected t/data/tv_grep_not_channel_name_d_test_xml_test_xml.expected t/data/tv_grep_not_channel_name_d_whitespace_xml.expected t/data/tv_grep_not_channel_name_d_x_whatever_xml.expected t/data/test_remove_some_overlapping.xml t/data/tv_cat_test_remove_some_overlapping_xml.expected t/data/tv_extractinfo_en_test_remove_some_overlapping_xml.expected t/data/tv_grep_a_test_remove_some_overlapping_xml.expected t/data/tv_grep_category_b_test_remove_some_overlapping_xml.expected t/data/tv_grep_category_e_and_title_f_test_remove_some_overlapping_xml.expected t/data/tv_grep_category_g_or_title_h_test_remove_some_overlapping_xml.expected t/data/tv_grep_channel_id_channel4_com_test_remove_some_overlapping_xml.expected t/data/tv_grep_channel_name_d_test_remove_some_overlapping_xml.expected t/data/tv_grep_channel_xyz_or_channel_b_test_remove_some_overlapping_xml.expected t/data/tv_grep_channel_xyz_or_not_channel_b_test_remove_some_overlapping_xml.expected t/data/tv_grep_eval_scalar_keys_5_test_remove_some_overlapping_xml.expected t/data/tv_grep_i_category_i_title_h_test_remove_some_overlapping_xml.expected t/data/tv_grep_i_category_i_title_j_test_remove_some_overlapping_xml.expected t/data/tv_grep_i_last_chance_c_test_remove_some_overlapping_xml.expected t/data/tv_grep_new_test_remove_some_overlapping_xml.expected t/data/tv_grep_not_channel_id_channel4_com_test_remove_some_overlapping_xml.expected t/data/tv_grep_not_channel_name_d_test_remove_some_overlapping_xml.expected t/data/tv_grep_on_after_2002_02_05_UTC_test_remove_some_overlapping_xml.expected t/data/tv_grep_on_after_200302161330_UTC_test_remove_some_overlapping_xml.expected t/data/tv_grep_on_before_200302161330_UTC_test_remove_some_overlapping_xml.expected t/data/tv_grep_premiere_test_remove_some_overlapping_xml.expected t/data/tv_grep_previously_shown_test_remove_some_overlapping_xml.expected t/data/tv_grep_channel_id_exp_sat_all_UTF8.expected t/data/tv_grep_channel_id_exp_sat_amp_xml.expected t/data/tv_grep_channel_id_exp_sat_amp_xml_amp_xml.expected t/data/tv_grep_channel_id_exp_sat_amp_xml_clump_xml.expected t/data/tv_grep_channel_id_exp_sat_amp_xml_dups_xml.expected t/data/tv_grep_channel_id_exp_sat_amp_xml_empty_xml.expected t/data/tv_grep_channel_id_exp_sat_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_grep_channel_id_exp_sat_attrs_xml.expected t/data/tv_grep_channel_id_exp_sat_clump_extract_1_xml.expected t/data/tv_grep_channel_id_exp_sat_clump_extract_xml.expected t/data/tv_grep_channel_id_exp_sat_clump_xml.expected t/data/tv_grep_channel_id_exp_sat_clump_xml_amp_xml.expected t/data/tv_grep_channel_id_exp_sat_clump_xml_clump_xml.expected t/data/tv_grep_channel_id_exp_sat_clump_xml_dups_xml.expected t/data/tv_grep_channel_id_exp_sat_clump_xml_empty_xml.expected t/data/tv_grep_channel_id_exp_sat_dups_xml.expected t/data/tv_grep_channel_id_exp_sat_dups_xml_amp_xml.expected t/data/tv_grep_channel_id_exp_sat_dups_xml_clump_xml.expected t/data/tv_grep_channel_id_exp_sat_dups_xml_dups_xml.expected t/data/tv_grep_channel_id_exp_sat_dups_xml_empty_xml.expected t/data/tv_grep_channel_id_exp_sat_empty_xml.expected t/data/tv_grep_channel_id_exp_sat_empty_xml_amp_xml.expected t/data/tv_grep_channel_id_exp_sat_empty_xml_clump_xml.expected t/data/tv_grep_channel_id_exp_sat_empty_xml_dups_xml.expected t/data/tv_grep_channel_id_exp_sat_empty_xml_empty_xml.expected t/data/tv_grep_channel_id_exp_sat_intervals_xml.expected t/data/tv_grep_channel_id_exp_sat_length_xml.expected t/data/tv_grep_channel_id_exp_sat_overlap_xml.expected t/data/tv_grep_channel_id_exp_sat_simple_xml.expected t/data/tv_grep_channel_id_exp_sat_simple_xml_x_whatever_xml.expected t/data/tv_grep_channel_id_exp_sat_sort1_xml.expected t/data/tv_grep_channel_id_exp_sat_sort2_xml.expected t/data/tv_grep_channel_id_exp_sat_sort_xml.expected t/data/tv_grep_channel_id_exp_sat_test_empty_xml.expected t/data/tv_grep_channel_id_exp_sat_test_livre_xml.expected t/data/tv_grep_channel_id_exp_sat_test_remove_some_overlapping_xml.expected t/data/tv_grep_channel_id_exp_sat_test_sort_by_channel_xml.expected t/data/tv_grep_channel_id_exp_sat_test_xml.expected t/data/tv_grep_channel_id_exp_sat_test_xml_test_xml.expected t/data/tv_grep_channel_id_exp_sat_whitespace_xml.expected t/data/tv_grep_channel_id_exp_sat_x_whatever_xml.expected t/data/tv_remove_some_overlapping_all_UTF8.expected t/data/tv_remove_some_overlapping_amp_xml.expected t/data/tv_remove_some_overlapping_amp_xml_amp_xml.expected t/data/tv_remove_some_overlapping_amp_xml_clump_xml.expected t/data/tv_remove_some_overlapping_amp_xml_dups_xml.expected t/data/tv_remove_some_overlapping_amp_xml_empty_xml.expected t/data/tv_remove_some_overlapping_amp_xml_empty_xml_empty_xml_clump_xml.expected t/data/tv_remove_some_overlapping_attrs_xml.expected t/data/tv_remove_some_overlapping_clump_extract_1_xml.expected t/data/tv_remove_some_overlapping_clump_extract_xml.expected t/data/tv_remove_some_overlapping_clump_xml.expected t/data/tv_remove_some_overlapping_clump_xml_amp_xml.expected t/data/tv_remove_some_overlapping_clump_xml_clump_xml.expected t/data/tv_remove_some_overlapping_clump_xml_dups_xml.expected t/data/tv_remove_some_overlapping_clump_xml_empty_xml.expected t/data/tv_remove_some_overlapping_dups_xml.expected t/data/tv_remove_some_overlapping_dups_xml_amp_xml.expected t/data/tv_remove_some_overlapping_dups_xml_clump_xml.expected t/data/tv_remove_some_overlapping_dups_xml_dups_xml.expected t/data/tv_remove_some_overlapping_dups_xml_empty_xml.expected t/data/tv_remove_some_overlapping_empty_xml.expected t/data/tv_remove_some_overlapping_empty_xml_amp_xml.expected t/data/tv_remove_some_overlapping_empty_xml_clump_xml.expected t/data/tv_remove_some_overlapping_empty_xml_dups_xml.expected t/data/tv_remove_some_overlapping_empty_xml_empty_xml.expected t/data/tv_remove_some_overlapping_intervals_xml.expected t/data/tv_remove_some_overlapping_length_xml.expected t/data/tv_remove_some_overlapping_overlap_xml.expected t/data/tv_remove_some_overlapping_simple_xml.expected t/data/tv_remove_some_overlapping_simple_xml_x_whatever_xml.expected t/data/tv_remove_some_overlapping_sort1_xml.expected t/data/tv_remove_some_overlapping_sort2_xml.expected t/data/tv_remove_some_overlapping_sort_xml.expected t/data/tv_remove_some_overlapping_test_empty_xml.expected t/data/tv_remove_some_overlapping_test_livre_xml.expected t/data/tv_remove_some_overlapping_test_remove_some_overlapping_xml.expected t/data/tv_remove_some_overlapping_test_sort_by_channel_xml.expected t/data/tv_remove_some_overlapping_test_xml.expected t/data/tv_remove_some_overlapping_test_xml_test_xml.expected t/data/tv_remove_some_overlapping_whitespace_xml.expected t/data/tv_remove_some_overlapping_x_whatever_xml.expected t/data/tv_sort_by_channel_test_remove_some_overlapping_xml.expected t/data/tv_sort_test_remove_some_overlapping_xml.expected t/data/tv_to_latex_test_remove_some_overlapping_xml.expected t/data/tv_to_text_test_remove_some_overlapping_xml.expected tools/tv_find_grabbers tools/tv_validate_file.PL tools/tv_validate_file.in tools/tv_validate_grabber.PL tools/tv_validate_grabber.in xmltv-1.4.0/MANIFEST.SKIP000066400000000000000000000002551500074233200145620ustar00rootroot00000000000000(^|/)\.\# (^|/)do_not_dist(/|$) ^MANIFEST\.SKIP$ ^\.git/ ^\.github/ ^\.gitattributes ^\.gitignore ^ci/ ^\.dockerignore ^\.travis.yml ^mkdist$ ^todo/ ^#.*#$ \.cache$ \.orig$ xmltv-1.4.0/Makefile.PL000066400000000000000000001210531500074233200146360ustar00rootroot00000000000000#!/usr/bin/perl use warnings; use 5.008003; use ExtUtils::MakeMaker; use Config; use File::Basename (); use File::Find; # Don't use ':config pass_through' because that requires Getopt::Long # 2.24 or later, and we don't have a clean way to require that. # use Getopt::Long; Getopt::Long::Configure('pass_through'); # Suppress 'isn't numeric' warnings from MakeMaker - see # . # $SIG{__WARN__} = sub { for (my $msg = shift) { $_ = "warning: something's wrong" if not defined; warn $_ unless /Argument .+ isn.t numeric in numeric lt .+MakeMaker.pm/; } }; # ExtUtils::MakeMaker before 6.21 or so has a bug when PREFIX is not # given explicitly - # . # unless ($ExtUtils::MakeMaker::VERSION ge 6.21) { if (not grep /^PREFIX=/, @ARGV) { warn "You may want to explicitly give PREFIX to work around MakeMaker bugs.\n"; } } use strict; sub test_module( $$ ); sub check_date_manip(); sub test_special( $$ ); sub targets( $ ); # A couple of undocumented options to help with building the Windows # distribution even on hosts that don't have all the necessary # modules. By default, we only warn if dependencies are missing. # If --strict-deps is passed in, then the whole process fails. # my $opt_strictdeps = 0; my $opt_yes = 0; my $opt_default = 0; my $opt_components; my $opt_exclude; GetOptions('strict-deps' => \$opt_strictdeps, # be strict about dependencies yes => \$opt_yes, # answer yes to all questions default => \$opt_default, # answer default to all questions 'components=s' => \$opt_components, 'exclude=s' => \$opt_exclude, ); our $VERSION; $VERSION = '1.4.0'; # Fragment of Makefile text to give the directory where files should # be installed. The extra '.' in the middle of the path is to avoid # beginning with '//', which means a network share on Cygwin. # my $location = '$(DESTDIR)/./$(PREFIX)'; our %extra_constants; %extra_constants = (INST_PLAINDOC => 'blib/doc', INSTALLPLAINDOC => "$location/share/doc/xmltv-$::VERSION", INST_SHARE => 'blib/share', INSTALLSHARE => "$location/share/xmltv", # Manual page constants, shouldn't really be needed, but work # around bugs and make sure this stuff is the same across # MakeMaker versions. INSTALLMAN1DIR => "$location/share/man/man1", INSTALLMAN3DIR => "$location/share/man/man3", MAN3EXT => '3', # Directory to install into when making Windows binary dist. WINDOWS_DIST => "xmltv-$VERSION-win64", VERSION => "$VERSION", ); # The following lists of dependencies and files to be installed may # get modified later depending on what the user chooses. # # Documentation files to be installed. This is a global variable # because it is accessed by some code we add to MakeMaker. # our @docs; @docs = qw(doc/COPYING doc/QuickStart doc/README-Windows.md README.md); # Executables to be installed. my @exes = qw(filter/tv_augment_tz filter/tv_extractinfo_en filter/tv_extractinfo_ar filter/tv_grep filter/tv_sort filter/tv_to_latex filter/tv_to_text filter/tv_to_potatoe filter/tv_cat filter/tv_split filter/tv_imdb filter/tv_tmdb filter/tv_remove_some_overlapping filter/tv_count filter/tv_merge filter/tv_augment tools/tv_validate_grabber tools/tv_validate_file tools/tv_find_grabbers ); # Libraries to be installed. my %pm = ('lib/XMLTV.pm' => '$(INST_LIBDIR)/XMLTV.pm', 'lib/TZ.pm' => '$(INST_LIBDIR)/XMLTV/TZ.pm', 'lib/Clumps.pm' => '$(INST_LIBDIR)/XMLTV/Clumps.pm', 'lib/Usage.pm' => '$(INST_LIBDIR)/XMLTV/Usage.pm', 'lib/Date.pm' => '$(INST_LIBDIR)/XMLTV/Date.pm', 'lib/Version.pm' => '$(INST_LIBDIR)/XMLTV/Version.pm', 'lib/Ask.pm' => '$(INST_LIBDIR)/XMLTV/Ask.pm', 'lib/Ask/Tk.pm' => '$(INST_LIBDIR)/XMLTV/Ask/Tk.pm', 'lib/Ask/Term.pm' => '$(INST_LIBDIR)/XMLTV/Ask/Term.pm', 'lib/GUI.pm' => '$(INST_LIBDIR)/XMLTV/GUI.pm', 'lib/ProgressBar.pm' => '$(INST_LIBDIR)/XMLTV/ProgressBar.pm', 'lib/ProgressBar/None.pm' => '$(INST_LIBDIR)/XMLTV/ProgressBar/None.pm', 'lib/ProgressBar/Term.pm' => '$(INST_LIBDIR)/XMLTV/ProgressBar/Term.pm', 'lib/ProgressBar/Tk.pm' => '$(INST_LIBDIR)/XMLTV/ProgressBar/Tk.pm', 'lib/Summarize.pm' => '$(INST_LIBDIR)/XMLTV/Summarize.pm', 'lib/Supplement.pm' => '$(INST_LIBDIR)/XMLTV/Supplement.pm', 'lib/IMDB.pm' => '$(INST_LIBDIR)/XMLTV/IMDB.pm', 'lib/TMDB.pm' => '$(INST_LIBDIR)/XMLTV/TMDB.pm', 'lib/TMDB/API.pm' => '$(INST_LIBDIR)/XMLTV/TMDB/API.pm', 'lib/TMDB/API/Config.pm' => '$(INST_LIBDIR)/XMLTV/TMDB/API/Config.pm', 'lib/TMDB/API/Movie.pm' => '$(INST_LIBDIR)/XMLTV/TMDB/API/Movie.pm', 'lib/TMDB/API/Person.pm' => '$(INST_LIBDIR)/XMLTV/TMDB/API/Person.pm', 'lib/TMDB/API/Tv.pm' => '$(INST_LIBDIR)/XMLTV/TMDB/API/Tv.pm', 'lib/Gunzip.pm' => '$(INST_LIBDIR)/XMLTV/Gunzip.pm', 'lib/Capabilities.pm' => '$(INST_LIBDIR)/XMLTV/Capabilities.pm', 'lib/Description.pm' => '$(INST_LIBDIR)/XMLTV/Description.pm', 'lib/Configure.pm' => '$(INST_LIBDIR)/XMLTV/Configure.pm', 'lib/Configure/Writer.pm' => '$(INST_LIBDIR)/XMLTV/Configure/Writer.pm', 'lib/Options.pm' => '$(INST_LIBDIR)/XMLTV/Options.pm', 'lib/ValidateFile.pm' => '$(INST_LIBDIR)/XMLTV/ValidateFile.pm', 'lib/ValidateGrabber.pm' => '$(INST_LIBDIR)/XMLTV/ValidateGrabber.pm', 'lib/PreferredMethod.pm' => '$(INST_LIBDIR)/XMLTV/PreferredMethod.pm', 'lib/Data/Recursive/Encode.pm' => '$(INST_LIBDIR)/XMLTV/Data/Recursive/Encode.pm', 'lib/Augment.pm' => '$(INST_LIBDIR)/XMLTV/Augment.pm', 'filter/Grep.pm' => '$(INST_LIBDIR)/XMLTV/Grep.pm', 'grab/Memoize.pm' => '$(INST_LIBDIR)/XMLTV/Memoize.pm', 'grab/Grab_XML.pm' => '$(INST_LIBDIR)/XMLTV/Grab_XML.pm', 'grab/DST.pm' => '$(INST_LIBDIR)/XMLTV/DST.pm', 'grab/Config_file.pm' => '$(INST_LIBDIR)/XMLTV/Config_file.pm', 'grab/Get_nice.pm' => '$(INST_LIBDIR)/XMLTV/Get_nice.pm', 'grab/Mode.pm' => '$(INST_LIBDIR)/XMLTV/Mode.pm', ); # Modules required to install. my %prereqs = ( 'Date::Manip' => 5.42, 'File::Slurp' => 0, 'JSON' => 0, 'LWP' => 5.65, 'LWP::UserAgent'=> 0, 'LWP::Protocol::https' => 0, 'HTTP::Request' => 0, 'HTTP::Response'=> 0, 'URI' => 0, 'Memoize' => 0, 'Storable' => 2.04, 'Term::ReadKey' => 0, # XML::Parser is required by XML::Twig, but older versions have a # bug when $SIG{__DIE__} is set (well, could be a perl bug, but # anyway it doesn't appear with 2.34). # 'XML::Parser' => 2.34, 'XML::TreePP' => 0, 'XML::Twig' => 3.28, 'XML::Writer' => 0.600, ); # Files which are run to generate source code. my %pl_files = ('filter/tv_grep.PL' => 'filter/tv_grep', 'tools/tv_validate_file.PL' => 'tools/tv_validate_file', 'tools/tv_validate_grabber.PL' => 'tools/tv_validate_grabber', 'lib/XMLTV.pm.PL' => 'lib/XMLTV.pm', 'lib/Supplement.pm.PL' => 'lib/Supplement.pm', ); # Some tools which are generated from .PL files need the share/ # directory passed as an extra argument. # my @need_share = ('tools/tv_validate_file', 'tools/tv_validate_grabber', 'lib/Supplement.pm', ); # Files to be installed in the system-wide share/ directory. my %share_files = ('xmltv.dtd' => 'xmltv.dtd', 'xmltv-lineups.xsd' => 'xmltv-lineups.xsd', 'filter/augment/augment.conf' => 'tv_augment/augment.conf', 'filter/augment/augment.rules' => 'tv_augment/augment.rules', ); # Files which 'make clean' should remove, but doesn't by default, so # we have to patch it. # my @to_clean = ('filter/tv_grep', 'tools/tv_validate_file', 'tools/tv_validate_grabber', 'lib/XMLTV.pm', 'lib/Supplement.pm', ); # Extra dependencies to add to the Makefile. my @deps = ('filter/tv_grep' => [ qw(filter/tv_grep.in pm_to_blib) ], 'tools/tv_validate_file' => [ qw(tools/tv_validate_file.in) ], 'tools/tv_validate_grabber' => [ qw(tools/tv_validate_grabber.in) ], 'lib/XMLTV.pm' => [ 'lib/XMLTV.pm.in' ], 'lib/Supplement.pm' => [ 'lib/Supplement.pm.in' ], ); # Some grabbers which are generated from .PL files need the share/ # directory passed as an extra argument. # my @grab_need_share; # 'Recommended but not required'. It isn't currently handled to have # the same module in both sets. # my %recommended = ( 'Compress::Zlib' => 0, 'Lingua::Preferred' => '0.2.4', 'Term::ProgressBar' => 2.03, 'Unicode::String' => 0, ); # And Log::TraceMessages is 'suggested' but we don't warn about that. if ($opt_yes) { *ask = sub { print "$_[0] yes\n"; 1 }; } elsif ($opt_default) { *ask = sub { print "$_[0] $_[2]\n"; $_[2] }; } else { require './lib/Ask/Term.pm'; *ask = \&XMLTV::Ask::Term::ask_boolean; } # Weird shit happens when you change things like PREFIX without # rebuilding everything explicitly. # if (-e 'Makefile') { warn < 'tv_grab_ch_search', blurb => 'Grabber for Switzerland', exes => [ 'grab/ch_search/tv_grab_ch_search' ], deps => [ 'grab/ch_search/tv_grab_ch_search' => [ 'grab/ch_search/tv_grab_ch_search.in' ] ], pl_files => { 'grab/ch_search/tv_grab_ch_search.PL' => 'grab/ch_search/tv_grab_ch_search' }, to_clean => [ 'grab/ch_search/tv_grab_ch_search' ], grab_need_share => [ 'ch_search' ], prereqs => { 'HTML::Entities' => 1.27, 'HTML::TreeBuilder' => 0, 'HTTP::Cookies' => 0, 'URI::Escape' => 0, 'URI::URL' => 0, }, }, { name => 'tv_grab_fi', blurb => 'Grabber for Finland', exes => [ 'grab/fi/tv_grab_fi' ], deps => [ 'grab/fi/tv_grab_fi' => [ 'grab/fi/tv_grab_fi.pl', 'grab/fi/fi/common.pm', 'grab/fi/fi/day.pm', 'grab/fi/fi/programme.pm', 'grab/fi/fi/programmeStartOnly.pm', 'grab/fi/fi/source/iltapulu.pm', 'grab/fi/fi/source/star.pm', 'grab/fi/fi/source/telkku.pm', 'grab/fi/fi/source/telsu.pm', 'grab/fi/fi/source/yle.pm', ], ], pl_files => { 'grab/fi/merge.PL' => 'grab/fi/tv_grab_fi' }, to_clean => [ 'grab/fi/tv_grab_fi' ], prereqs => { 'HTML::TreeBuilder' => 0, 'LWP::Protocol::https' => 0, 'URI::Escape' => 0, }, }, # { name => 'tv_grab_fi_sv', # blurb => 'Grabber for Finland (Swedish)', # exes => [ 'grab/fi_sv/tv_grab_fi_sv' ], # prereqs => { 'DateTime' => 0, # 'HTML::TreeBuilder' => 0, # 'IO::Scalar' => 0, }, # }, # { name => 'tv_grab_fr', # blurb => 'Grabber for France (TeleStar)', # exes => [ 'grab/fr/tv_grab_fr' ], # prereqs => { 'DateTime' => 0, # 'DateTime::Duration' => 0, # 'DateTime::TimeZone' => 0, # 'HTML::Entities' => 1.27, # 'HTML::TreeBuilder' => 0, # 'HTTP::Cache::Transparent' => 1.0, }, # }, { name => 'tv_grab_huro', blurb => 'Grabber for Hungary (port.hu)', exes => [ 'grab/huro/tv_grab_huro' ], pl_files => { 'grab/huro/tv_grab_huro.PL' => 'grab/huro/tv_grab_huro' }, share_files => { 'grab/huro/jobmap' => 'tv_grab_huro/jobmap', 'grab/huro/catmap.hu' => 'tv_grab_huro/catmap.hu', }, to_clean => [ 'grab/huro/tv_grab_huro' ], deps => [ 'grab/huro/tv_grab_huro' => [ 'grab/huro/tv_grab_huro.in' ] ], grab_need_share => [ 'huro' ], prereqs => { 'HTML::Entities' => 0, 'HTML::TreeBuilder' => 0, 'LWP::Protocol::https' => 0, 'Time::Piece' => 0, 'Time::Seconds' => 0, }, }, # { name => 'tv_grab_il', # blurb => 'Grabber for Israel', # exes => [ 'grab/il/tv_grab_il' ], # prereqs => { 'DateTime' => 0, }, # }, { name => 'tv_grab_is', blurb => 'Grabber for Iceland', exes => [ 'grab/is/tv_grab_is' ], share_files => { 'grab/is/category_map' => 'tv_grab_is/category_map' }, prereqs => { 'HTML::Entities' => 0, 'HTML::TreeBuilder' => 0, 'URI' => 0, 'XML::DOM' => 0, 'XML::LibXSLT' => 0, }, }, # { name => 'tv_grab_it', # blurb => 'Grabber for Italy', # exes => [ 'grab/it/tv_grab_it' ], # pl_files => { 'grab/it/tv_grab_it.PL' => 'grab/it/tv_grab_it' }, # share_files => { 'grab/it/channel_ids' => 'tv_grab_it/channel_ids' }, # to_clean => [ 'grab/it/tv_grab_it', 'grab/it/tv_grab_it.in2' ], # deps => [ 'grab/it/tv_grab_it' => [ 'grab/it/tv_grab_it.in' ] ], # grab_need_share => [ 'it' ], # prereqs => { 'HTML::Entities' => 0, # 'HTML::Parser' => 0, # 'URI::Escape' => 0, }, # }, { name => 'tv_grab_it_dvb', blurb => 'Grabber for Italy from DVB-S stream', exes => [ 'grab/it_dvb/tv_grab_it_dvb' ], pl_files => { 'grab/it_dvb/tv_grab_it_dvb.PL' => 'grab/it_dvb/tv_grab_it_dvb' }, share_files => { 'grab/it_dvb/channel_ids' => 'tv_grab_it_dvb/channel_ids', 'grab/it_dvb/sky_it.dict' => 'tv_grab_it_dvb/sky_it.dict', 'grab/it_dvb/sky_it.themes' => 'tv_grab_it_dvb/sky_it.themes', }, to_clean => [ 'grab/it_dvb/tv_grab_it_dvb' ], deps => [ 'grab/it_dvb/tv_grab_it_dvb' => [ 'grab/it_dvb/tv_grab_it_dvb.in' ] ], grab_need_share => [ 'it_dvb' ], prereqs => { 'Data::Dump' => 0, 'HTML::Entities' => 0, 'HTML::Parser' => 0, 'IO::Select' => 0, 'Linux::DVB' => 0, 'Time::HiRes' => 0, 'URI::Escape' => 0, }, }, { name => 'tv_grab_na_dd', blurb => '$$ Grabber for North America-schedulesdirect.org', exes => [ 'grab/na_dd/tv_grab_na_dd' ], pl_files => { 'grab/na_dd/tv_grab_na_dd.PL' => 'grab/na_dd/tv_grab_na_dd' }, deps => [ 'grab/na_dd/tv_grab_na_dd' => [ 'grab/na_dd/tv_grab_na_dd.in' ] ], to_clean => [ 'grab/na_dd/tv_grab_na_dd' ], prereqs => { 'SOAP::Lite' => 0.67, }, grab_need_share => [ 'na_dd' ], }, # 2023-09-04 edenr tv_grab_na_dtv removed due to source site changes # { name => 'tv_grab_na_dtv', # blurb => 'Grabber for North America (DirecTV)', # exes => [ 'grab/na_dtv/tv_grab_na_dtv' ], # prereqs => { # 'DateTime' => 0, # 'HTTP::Cookies' => 0, # 'URI' => 0, # 'URI::Escape' => 0, }, # }, { name => 'tv_grab_na_tvmedia', blurb => 'Grabber for North America (TVMedia)', exes => [ 'grab/na_tvmedia/tv_grab_na_tvmedia' ], prereqs => { 'XML::LibXML' => 0, }, }, ## { name => 'tv_grab_pt_meo', ## blurb => 'Grabber for Portugal (MEO)', ## exes => [ 'grab/pt_meo/tv_grab_pt_meo' ], ## prereqs => { 'DateTime' => 0, ## 'Encode' => 0, ## 'JSON' => 0, ## 'IO::File' => 0, ## 'File::Path' => 0, ## 'File::Basename' => 0, }, ## }, { name => 'tv_grab_pt_vodafone', blurb => 'Grabber for Portugal (Vodafone)', exes => [ 'grab/pt_vodafone/tv_grab_pt_vodafone' ], share_files => { 'grab/pt_vodafone/channel.list' => 'tv_grab_pt_vodafone/channel.list', }, prereqs => { 'DateTime' => 0, 'URI::Escape' => 0, 'XML::LibXML' => 0, 'DateTime::Format::Strptime' => 0, 'URI::Encode' => 0, 'Text::Unidecode' => 0, }, }, { name => 'tv_grab_uk_freeview', blurb => 'Grabber for UK using Freeview website', exes => [ 'grab/uk_freeview/tv_grab_uk_freeview' ], prereqs => { 'DateTime' => 0, 'Encode' => 0, 'JSON' => 0, 'IO::File' => 0, 'File::Path' => 0, 'File::Basename' => 0, }, }, ## { name => 'tv_grab_uk_tvguide', ## blurb => 'Grabber for UK and Ireland using TV Guide website', ## exes => [ 'grab/uk_tvguide/tv_grab_uk_tvguide' ], ## share_files => { ## 'grab/uk_tvguide/tv_grab_uk_tvguide.map.conf' ## => 'tv_grab_uk_tvguide/tv_grab_uk_tvguide.map.conf', ## }, ## prereqs => { 'Date::Parse' => 0, ## 'DateTime' => 0, ## 'HTML::TreeBuilder' => 0, ## 'HTTP::Cache::Transparent' => 1.0, ## 'HTTP::Cookies' => 0, ## 'URI::Escape' => 0, }, ## }, { name => 'tv_grab_zz_sdjson', blurb => '$$ Grabber for schedulesDirect.org SD-JSON service (many countries)', exes => [ 'grab/zz_sdjson/tv_grab_zz_sdjson' ], prereqs => { 'DateTime' => 0, 'Digest::SHA' => 0, 'HTTP::Message' => 0, 'LWP::Protocol::https' => 0, 'Try::Tiny' => 0, }, }, { name => 'tv_grab_zz_sdjson_sqlite', blurb => '$$ Grabber for schedulesDirect.org SD-JSON service (many countries, using sqlite)', exes => [ 'grab/zz_sdjson_sqlite/tv_grab_zz_sdjson_sqlite' ], prereqs => { 'DateTime' => 0, 'DateTime::Format::ISO8601' => 0, 'DateTime::Format::SQLite' => 0, 'DateTime::TimeZone' => 0, 'DBD::SQLite' => 0, 'DBI' => 0, 'Digest::SHA' => 0, 'File::HomeDir' => 0, 'File::Which' => 0, 'List::MoreUtils' => 0, 'LWP::Protocol::https' => 0, 'LWP::UserAgent::Determined' => 0, }, }, { name => 'tv_check', blurb => 'Program to report exceptions and changes in a schedule', exes => [ 'choose/tv_check/tv_check' ], docs => [ qw(choose/tv_check/README.tv_check choose/tv_check/tv_check_doc.html choose/tv_check/tv_check_doc.jpg ) ], prereqs => { 'Tk' => 0, 'Tk::TableMatrix' => 0, } }, { name => 'tv_grab_combiner', blurb => 'Grabber that combines data from other grabbers', exes => [ 'grab/combiner/tv_grab_combiner' ], prereqs => { 'XML::LibXML' => 0, }, }, { name => 'tv_pick_cgi', blurb => 'CGI program to filter listings (to install manually)', prereqs => { 'CGI' => 0, 'CGI::Carp' => 0, }, type => 'run', }, ); # Now we need to prompt about each optional component. The style of # prompting, though not the code, is based on SOAP::Lite. I would # like to add '--noprompt' and '--with tv_grab_nl' options to help # automated package building, but I haven't implemented that yet. # # For each component work out whether its prereqs are installed and # store the result in {missing} - either false, or a hashref. # foreach my $info (@opt_components) { my $name = $info->{name}; my %modules_missing; our %module_prereqs; local *module_prereqs = $info->{prereqs} || {}; foreach (sort keys %module_prereqs) { my $ver = $module_prereqs{$_}; next if test_module($_, $ver)->[0] eq 'OK'; warn "strange, module prereq $_ mentioned twice" if defined $modules_missing{$_}; $modules_missing{$_} = $ver; } our @special_prereqs; my %special_missing; local *special_prereqs = $info->{special_prereqs} || {}; foreach (@special_prereqs) { my ($sub, $name, $ver, $friendly_ver) = @$_; next if test_special($sub, $ver)->[0] eq 'OK'; warn "strange, special prereq $name mentioned twice" if defined $special_missing{$name}; $special_missing{$name} = $friendly_ver; } my %missing = (%modules_missing, %special_missing); if (not keys %missing) { $info->{missing} = 0; } else { $info->{missing} = \%missing; } } if (not defined $opt_components) { # Generate a default configuration that installs as much as possible. print STDERR <{name}} = $_; $_->{exclude} = 0; # default if not mentioned } if (defined $opt_exclude) { foreach (split /,/, $opt_exclude) { my $i = $by_name{$_}; die "unknown component $_\n" if not $i; $i->{exclude} = 1; } } my $width = 0; foreach my $info (@opt_components) { my $w = length("$info->{blurb} ($info->{name})"); $width = $w if $w > $width; } foreach my $info (@opt_components) { my $missing = $info->{missing}; my $s = "$info->{blurb} ($info->{name})"; # Guess a default value for {install} based on whether # prerequisites were found. # $info->{install} = (not $info->{exclude}) && ($opt_yes || not $info->{missing}); print STDERR ($s, ' ' x (1 + $width - length $s), $info->{install} ? '[yes]' : '[no]', "\n"); } print STDERR "\n"; if (not ask(0, 'Do you want to proceed with this configuration?', 1)) { # Need to set {install} for each component by prompting. foreach my $info (@opt_components) { my $missing = $info->{missing}; my $name = $info->{name}; print STDERR "\n* $info->{blurb} ($name)\n\n"; if ($missing) { print STDERR "These dependencies are missing for $name:\n\n"; foreach (sort keys %$missing) { print STDERR "$_"; my $min_ver = $missing->{$_}; if ($min_ver) { print STDERR " (version $min_ver or higher)"; } print STDERR "\n"; } print STDERR "\n"; } my $msg; my $type = $info->{type}; if (not defined $type or $type eq 'install') { $msg = "Do you wish to install $name?"; } elsif ($type eq 'run') { $msg = "Do you plan to run $name?"; } else { die; } $info->{install} = ask(0, $msg, not $missing); } } } else { my @to_install = split /\s+/, $opt_components; my %by_name; foreach (@opt_components) { $by_name{$_->{name}} = $_; $_->{install} = 0; # default if not mentioned } foreach (@to_install) { my $i = $by_name{$_}; die "unknown component $_\n" if not $i; $i->{install} = 1; } } foreach my $info (@opt_components) { next if not $info->{install}; push @exes, @{$info->{exes}} if $info->{exes}; push @docs, @{$info->{docs}} if $info->{docs}; %pm = (%pm, %{$info->{pm}}) if $info->{pm}; %prereqs = (%prereqs, %{$info->{prereqs}}) if $info->{prereqs}; %pl_files = (%pl_files, %{$info->{pl_files}}) if $info->{pl_files}; %share_files = (%share_files, %{$info->{share_files}}) if $info->{share_files}; push @to_clean, @{$info->{to_clean}} if $info->{to_clean}; push @deps, @{$info->{deps}} if $info->{deps}; push @grab_need_share, @{$info->{grab_need_share}} if $info->{grab_need_share}; } my $warned_uninstall_broken = 1; # Test the installed version of a module. # # Parameters: # Name of module # Version required, or 0 for don't care # # Returns a tuple of two scalars: the first scalar is one of # # OK - a recent enough version is installed. # NOT_INSTALLED - the module is not installed. # FAILED - the second scalar contains an error message. # TOO_OLD - the second scalar contains the version found. # sub test_module( $$ ) { my ($mod, $minver) = @_; die if not defined $mod; die if not defined $minver; eval "require $mod"; if ($@) { # This if-test is separate to suppress spurious 'Use of # uninitialized value in numeric lt (<)' warning. # if ($@ ne '') { if ($@ =~ /^Can\'t locate \S+\.pm in \@INC/) { return [ 'NOT_INSTALLED', undef ]; } else { chomp (my $msg = $@); return [ 'FAILED', $msg ]; } } } my $ver = $mod->VERSION; if ($minver ne '0') { return [ 'TOO_OLD', undef ] if not defined $ver; return [ 'TOO_OLD', $ver ] if $ver lt $minver; } return [ 'OK', undef ]; } # Run a subroutine and check that its output has the correct version. # # Parameters: # code reference to run # minumum version # # The code ref should return undef meaning 'package not present' or # else a version number. # # Returns as for test_module() (but 'FAILED' not an option). # sub test_special( $$ ) { my ($sub, $minver) = @_; my $ver = $sub->(); return [ 'NOT_INSTALLED', undef ] if not defined $ver; if ($minver ne '0') { return [ 'TOO_OLD', undef ] if not defined $ver; return [ 'TOO_OLD', $ver ] if $ver lt $minver; } return [ 'OK', undef ]; } # MakeMaker's warning message can be intimidating, check ourselves # first. We warn about missing 'recommended' modules but don't abort # because of them. # my $err = 0; foreach my $p ((sort keys %prereqs), (sort keys %recommended)) { my $required = (defined $prereqs{$p}); my $verbed = $required ? 'required' : 'recommended'; my $Verbed = uc(substr($verbed, 0, 1)) . substr($verbed, 1); my $minver = $required ? $prereqs{$p} : $recommended{$p}; die "bad minver for $p" if not defined $minver; my ($r, $more) = @{test_module($p, $minver)}; if ($r eq 'OK') { # Installed and recent enough. } elsif ($r eq 'NOT_INSTALLED') { print STDERR "Module $p seems not to be installed.\n"; print(($minver ? "$p $minver" : $p), " is $verbed.\n"); ++ $err if $required; } elsif ($r eq 'FAILED') { print STDERR "$Verbed module $p failed to load: $more\n"; print(($minver ? "$p $minver" : $p), " is $verbed.\n"); ++ $err if $required; } elsif ($r eq 'TOO_OLD') { if (defined $more) { print STDERR "$p-$minver is $verbed, but $more is installed\n"; } else { print STDERR "$p-$minver is $verbed, but an unknown version is installed\n"; } ++ $err if $required; } else { die } } if ($err) { if ($opt_strictdeps) { die "Required modules missing. Makefile will not be created.\n"; } else { warn "Required modules missing, 'make' is unlikely to work\n"; } } WriteMakefile ( 'NAME' => 'XMLTV', # No VERSION_FROM, it's set in this file 'EXE_FILES' => \@exes, 'PL_FILES' => \%pl_files, 'PM' => \%pm, 'PREREQ_PM' => \%prereqs, # No special parameters for 'make clean' or 'make dist' ); sub MY::constants { package MY; my $inherited = shift->SUPER::constants(@_); die if not keys %::extra_constants; foreach (sort keys %::extra_constants) { $inherited .= "$_ = $::extra_constants{$_}\n"; } return $inherited; } sub MY::install { package MY; my $inherited = shift->SUPER::install(@_); # Decided that 'plaindoc_install' should be directly under # 'install', not under the misleadingly named 'doc_install'. # my %extra_deps = (install => [ 'plaindoc_install', 'share_install' ]); foreach my $t (keys %extra_deps) { foreach my $d (@{$extra_deps{$t}}) { $inherited =~ s/^(\s*$t\s+::\s.+)/$1 $d/m or die; } } foreach (qw(plaindoc share)) { my $target = $_ . '_install'; my $uc = uc; my $inst_var = "INST_$uc"; my $extra = <{$_}; my @t = @{$targets->{$_}}; # make a copy my $done = 0; foreach (@t) { if (s/\@\$\(MOD_INSTALL\)/\$(PERL) -I. -MUninstall -e "uninstall(\@ARGV)"/) { $done = 1; last; } s/Installing contents of (\S+) into (\S+)/Removing contents of $1 from $2/; } if (not $done) { print STDERR "couldn't find \@\$(MOD_INSTALL) in target $_, uninstall may not work\n" unless $warned_uninstall_broken; } (my $new_target = $_) =~ s/install$/uninstall/ or die; foreach ("\n\n$new_target ::\n", @t) { $inherited .= $_; } } $inherited .= 'pure_uninstall :: pure_$(INSTALLDIRS)_uninstall' . "\n"; $inherited .= 'uninstall :: all pure_uninstall plaindoc_uninstall share_uninstall' . "\n"; # Add a target for a Windows distribution. Note this is # singlequoted and then we substitute one variable by hand! # $inherited .= q{ xmltv.exe :: $(EXE_FILES) lib/xmltv.pl lib/xmltv32.pl lib/exe_opt.pl echo $(EXE_FILES) >exe_files.txt perl lib/exe_opt.pl $(VERSION) >exe_opt.txt pp_autolink -o xmltv.exe --cachedeps=pp.cache --reusable @exe_opt.txt lib/xmltv.pl lib/xmltv32.pl $(EXE_FILES) $(RM_F) exe_files.txt $(RM_F) exe_opt.txt windows_dist :: @perl -e "if (-e '$location') { print STDERR qq[To build a Windows distribution, please rerun Makefile.PL with\nPREFIX set to a new (nonexistent) directory then 'make windows_dist'.\n(Remember that only absolute paths work properly with MakeMaker.)\n]; exit 1 }" @perl -e 'print "Have you updated doc/README-Windows.md for this release? "; exit 1 unless =~ /^[yY]/' $(MAKE) install perl -MExtUtils::Command -e mv $(INSTALLPLAINDOC) $location/doc/ perl -MExtUtils::Command -e rm_r $location/share/doc perl -MExtUtils::Command -e mkpath $location/doc/man # Generate plain text documentation from pod. perl -e "chdir 'blib/script' or die; foreach (<*>) { system qq'pod2text <\$$_ >$location/doc/man/\$$_.txt' }" # Remove 'real' manual pages, not needed on Windows. perl -MExtUtils::Command -e rm_rf $location/man $location/share/man # My MakeMaker creates this dud directory. perl -MExtUtils::Command -e rm_rf $location/5.8.0 rmdir $location/share/doc # Generate Date::Manip docs by filtering perldoc output. The # use of temp files instead of pipes is so set -e works properly. # echo Extracting part of Date::Manip manual page into $location/doc/man/date_formats.txt echo "This is an extract from the documentation of Perl's Date::Manip module," >>$location/doc/man/date_formats.txt echo "describing the different format strings that may be used for dates." >>$location/doc/man/date_formats.txt echo "Bear in mind that depending on your Windows version you will need to" >>$location/doc/man/date_formats.txt echo "quote the % characters on the command line somehow (see README-Windows.md)." >>$location/doc/man/date_formats.txt echo "" >>$location/doc/man/date_formats.txt perldoc -u Date::Manip >$location/doc/man/date_formats.txt.tmp perl -ne "BEGIN { print qq'\n=pod\n\n' } print if (/^The format options are:/ .. /^=/) and not /^=/" <$location/doc/man/date_formats.txt.tmp >$location/doc/man/date_formats.txt.tmp.1 pod2text <$location/doc/man/date_formats.txt.tmp.1 >>$location/doc/man/date_formats.txt perl -MExtUtils::Command -e rm_f $location/doc/man/date_formats.txt.tmp* # Don't use $(INSTALLBIN), it seems to disregard PREFIX passed # to 'make'. # perl -MExtUtils::Command -e rm_rf $location/bin/ $location/lib/ $(INSTMANDIR) $(INSTALLMAN3DIR) perl -MExtUtils::Command -e cp xmltv.dtd $location perl -MExtUtils::Command -e cp xmltv-lineups.xsd $location perl -MExtUtils::Command -e cp ChangeLog $location/ChangeLog.txt # The following command will not be necessary when the source # tree was checked out on a DOSish system. It may not even # work properly when run on a DOSish system - should check. # # (Simulation in perl of find | xargs; there's probably a # better way but I'm too lazy to find it.) # perl -MFile::Find -e "find(sub { print qq[\\$$File::Find::name\n] if -f and not /[.]jpg/ }, '$location')" | perl -e 'chomp(@ARGV = (@ARGV, )); exec @ARGV' perl -i -pe 'BEGIN { binmode STDIN } s/\r*\n*$$/\r\n/' perl -MExtUtils::Command -e mv $location/doc/README* $location perl -MExtUtils::Command -e mv $location/README-Windows.md $location/README.txt @echo @echo Part of a Windows distribution tree has been made in $location/. @echo Now copy in the executables! }; $inherited =~ s/\$location/$location/g or die; return $inherited; } # Extend installbin() to put doc and share under blib/. sub MY::installbin { package MY; my $inherited = shift->SUPER::installbin(@_); # Add a target for each documentation file. my %doc_files; foreach (@::docs) { $doc_files{$_} = File::Basename::basename($_); } my %new_filetypes = (plaindoc => \%doc_files, share => \%share_files); my %seen_dir; foreach my $filetype (sort keys %new_filetypes) { my $uc = uc $filetype; our %files; local *files = $new_filetypes{$filetype}; foreach my $src (sort keys %files) { my $inst_pos = $files{$src}; my $extra = ''; # The directory containing this file in blib/ needs to be created. my @dirs = split m!/!, $inst_pos; pop @dirs; foreach (0 .. $#dirs) { my $dir = join('/', @dirs[0 .. $_]); my $parent = join('/', @dirs[0 .. $_-1]); next if $seen_dir{$dir}++; die if (length $parent and not $seen_dir{$parent}); my $parent_exists = "\$(INST_$uc)/$parent/.exists"; $parent_exists =~ tr!/!/!s; $extra .= <SUPER::clean(@_); $inherited =~ s/\s+$//; $inherited .= "\n\t-\$(RM_F) $_\n" foreach @to_clean; return $inherited; } sub MY::processPL { package MY; my $inherited = shift->SUPER::processPL(@_); # Add some exra dependencies. my ($k, $v); while (@deps) { ($k, $v, @deps) = @deps; $inherited =~ s!^(\s*$k\s+::\s.+)!"$1 " . join(' ', @$v)!me or die "no $k in: $inherited"; } # And some of the .in generators need the share/ directory passed # as an extra argument. This is the location in the installed # system, not that where files are being copied, so $(PREFIX) but # no $(DESTDIR). # foreach (@grab_need_share) { $inherited =~ s<(grab/$_/tv_grab_$_.PL grab/$_/tv_grab_$_)\s*$> <$1 \$(PREFIX)/share/xmltv>m or die "no call to $_.PL in: $inherited"; } foreach (@need_share) { $inherited =~ s<($_.PL $_)\s*$> <$1 \$(PREFIX)/share/xmltv>m or die "no call to $_.PL in: $inherited"; } return $inherited; } sub MY::makefile { package MY; my $inherited = shift->SUPER::makefile(@_); return $inherited; } # Fix filename of some generated manpages. sub MY::manifypods { package MY; for (my $inherited = shift->SUPER::manifypods(@_)) { foreach my $s (qw(Augment Grab_XML Configure::Writer Configure Data::Recursive::Encode Date GUI Gunzip Options PreferredMethod Summarize Supplement ValidateFile ValidateGrabber Version)) { s!\$\(INST_MAN3DIR\)/(?:grab::|)$s[.]\$\(MAN3EXT\)!"\$(INST_MAN3DIR)/XMLTV::$s.\$(MAN3EXT)"!; s!\$\(INSTALLMAN3DIR\)/$s.\$\(MAN3EXT\)!"\$(INSTALLMAN3DIR)/XMLTV::$s.\$(MAN3EXT)"!; } return $_; } } # Split a section of makefile into targets. sub targets( $ ) { my @lines = split /\n/, shift; $_ .= "\n" foreach @lines; my %r; my $curr_target; foreach (@lines) { if (/^(\S+)\s+:/) { # Beginning of a new target. my $name = $1; die "target $name seen twice" if defined $r{$name}; $r{$name} = $curr_target = []; } elsif (/^\s+/ and defined $curr_target) { # Commands for the target. push @$curr_target, $_; } elsif (/^$/) { # Blank lines are legal in a target definition } elsif (/^\s*(?:\#.*)?$/) { undef $curr_target; } else { chomp; die "bad makefile line: '$_'"; } } return \%r; } xmltv-1.4.0/README.cygwin000066400000000000000000000327521500074233200150520ustar00rootroot00000000000000 HOWTO: install XMLTV under Cygwin on Windows ------------------------------------------- Introduction ------------ XMLTV works mostly fine under the Cygwin unix environment for Windows... This allows you to use the normal Perl Source code XMLTV installation without the need for the Windows executable version, giving the following advantages: * faster download of new XMLTV versions (source dist is 1/10th the size of the Win32 version) * ability to develop, debug and test new grabbers * ability to quickly patch bugs in grabbers without waiting for the new Win32 EXE version.. * ability to use grabbers that are not yet part of the Win32 EXE The disadvantages are: - Cygwin gives a unix command line environment that is alien to most windows users, - the initial download is very large -- Cygwin itself with the required packages is about 40Mb and the XMLTV prereq file is another 4Mb. - Cygwin uses a large amount of disk space. Also cygwin itself has a few odd features that can cause some issues with XMLTV, and TK based XMLTV GUI apps do not work in cygwin (tv_check) WARNING: I have not tested every XMLTV app and grabber under Cygwin... WARNING: Unix command lines can be very powerful and provide very little hand-holding (such as 'Are you sure' messages when you want to delete things). For this reason it is relatively easy to delete files unintentionally and to render your system non-operational.. Take care with command lines and make regular backups! Scope ----- This HOWTO only deals with how to get XMLTV running on Cygwin. It does not deal in any way with how to use Cygwin itself other than installing the required packages fro XMLTV, so knowledge of how to work using the unix-style command line shell will be required. Advanced usage, such as applying patch files and working with CVS checkouts is not covered by this HOWTO. Instructions ------------ 1) Install Cygwin: ------------------ Download and run Cygwin's installer. http://www.cygwin.com/setup.exe In the installer, select: Install from Internet -> Next Select a suitable installation directory, All Users, Unix file type -> Next Select Local Package directory = some temporary folder... there will be a folder created - named after the download server - which you can delete after installation. -> Next Select the correct HTTP/FTP proxy settings -> Next Select a nearby mirror server -> Next The package list will now be downloaded and shown in a categorised tree view. -- Click on the (+) text button to expand / contract a branch -- Click on the '() Skip' text-button on a package line to select a version to download. It will change from 'Skip' to the version number. If the package is already installed, the additional options 'Keep', 'Reinstall', 'Uninstall' will also be shown Apart from the default packages that are pre-selected by the installer, you need to ensure that the following packages are selected Base/ diffutils tar Devel/ expat gcc make Interpreters/ perl Libs/ libiconv libiconv2 libxml2 ** select version 2.5.11 or 2.6.7 or later ** see below Utils/ bzip2 patch patchutils rebase Web/ wget Other useful packages to install are editors: Editor/ vim (vi clone) nano (pico clone) If you want to have access to the pre-release, under-development versions of XMLTV then the CVS package is also useful Devel/ cvs ** Note Re libxml2: The Perl XML::LibXML package is known not to work with certain versions of libxml2. This package is required for some of the grabbers (tv_grab_se) At time of writing, libxml2 version 2.5.11 is known to be OK, as is 2.6.7. At time of writing, Cygwin's default version is 2.6.4, which is known not to work For more information on working versions see README.md in xml_prereq_x/XML-LibXML-xxx. For this reason you must make sure that the correct version is selected here by clicking several times on the () 'Skip'/'Version' button... After a lot of downloading, (about 40Mb) and some installing you should now have cygwin in your start menu... Running it gives you a command window with a unix-style prompt: username@hostname WorkingDir $ Tips: The Tab key will complete the remains of a filename/directory name if unique. If not unique, it will complete as much as possible, and a second Tab will indicate the possible matches. ~/ is shorthand for your home directory. You can always go to this directory with cd ~/ Your PC's drives now appear with unix paths as: /cygdrive/c/ but the cd command accepts cd c: as a shortcut. The cygwin installation directory: (c:/cygwin by default) is mapped to the cygwin 'root' directory: '/' Note: If your PC does not have access to a broadband internet connection, and you don't want to spend several hours downloading 40Mb of Cygwin, but you have access to another PC that does have broadband, the cygwin installer has the option 'Download from Internet' to just download the required packages into the 'Local Package Directory'. You then write the contents of the directory to CD or a Flash disk, then use the 'Install from Local Directory' option on your target PC. 2) Download and unpack XMLTV source ----------------------------------- Now download the XMLTV source distribution, and the _prereq files from sourceforge http://files.xmltv.org Save them in your Cygwin Home directory: c:/cygwin/home/username At the Cygwin command prompt, Unpack the 2 tar.bz2 files using the command lines: bunzip2 -c xmltv-x.x.x.tar.bz2 | tar -xvf - bunzip2 -c xmltv_prereq-x.tar.bz2 | tar -xvf - Where x is replaced by the version numbers of the files you downloaded 3) Install the XMLTV prereq packages: ------------------------------------- Change directory to the prereq directory created by unpacking the tar.bz2 file and list the packages therein. cd xmltv_prereq-x ls For more info check the README.md: less README.md Note that the packages with TK cannot be installed in the current version of Cygwin, as PERL/TK is not yet part of Cygwin... This is not important, and only means that the XMLTV tv_check command cannot be installed The packages need to be installed in alphanumeric order -- ie all packages with 00 first, then 01 then A-Z then a-z So, For each directory starting with 00 (apart from 00_tk*) do: cd directory perl Makefile.PL; make test install cd .. Do the same for each directory starting with 01, then for each of the remaining directories (apart from those with Tk*)... Note: Some of the prerequisite modules (such as PerlIO-gzip, and libwww-perl) might fail some of its tests and will not install with the above command. These will need to be installed by running make install in the package directory. I have not found the failed tests to cause a problem yet. Note: When installing XML-LibXML I got an error: C:\cygwin\bin\perl.exe: *** unable to remap C:\cygwin\bin\cygiconv-2.dll to same address as parent... If this is the case, close all other Cygwin windows and run the command: rebaseall -v then try again. Note: When installing HTML-Parser, answer No to the question: Do you want decoding on unicode entities? [no] no I have had problems with decoded Unicode characters in Perl/Cygwin. Note: When installing libwww-perl, answer NO to the questions: Do you want to install the GET alias? [y] n Do you want to install the HEAD alias? [y] n Do you want to install the POST alias? [y] n this is because installing these aliases failed on my system! Note: you will not be able to install Text-Kakasi (required for Japan grabber) without downloading, building and installing the Kakasi library. See README in the xmltv_prereq directory for more details. If you don't need this, just skip the package. cd back to your home directory when complete cd ~/ 4) Configure TimeZone --------------------- XMLTV (actually the PERL Date::Manip module) needs the TZ variable set correctly for the time and date handling for summer and winter time to work. Date::Manip has internal knowledge of some time zone values. If the TZ variable is not set, some XMLTV grabbers and the tv_split command will not work. Cygwin has its own idea of the time zone (from windows), and overriding the time zone with TZ can show confusing output from some Cygwin commands that display the time and date... eg: No TZ set, my time zone in windows is CET == GMT+1, the current local time is 11:15) $ date Tue Mar 9 11:15:06 RST 2004 Setting TZ to CET $ export TZ=CET $ date Tue Mar 9 10:15:25 GMT 2004 as you can see setting a TZ variable makes Cygwin report all times as GMT... NOTE Setting TZ does not change the date of the system, merely the *display* of the dates... To avoid this confusion, it is best only to set the TZ variable when you are running the XMLTV tools. To set your time zone, run. export TZ=EST or whatever your time zone is... This will set the time zone for all future commands run in this Cygwin window. If you use another window, you will need to rerun this command... To see the list of the time zones understood by Date::Manip: http://search.cpan.org/~sbeck/DateManip-5.42a/Manip.pod#TIMEZONES or if a newer version has been released, go to http://search.cpan.org/~sbeck click on Date::Manip Click on Date::Manip in the Modules list to see the docs Click on TIMEZONES to see the TZ list To check that this TZ works with Date::Manip, run the command: $ perl -e 'use Date::Manip; print Date_TimeZone;' If all is OK, and Date::Manip recognised your TZ, this should print out the time zone. If not you will get some errors such as "ERROR: Date::Manip unable to determine TimeZone." The actual value of the time zone should not affect the results of most of the XMLTV commands, grabbers and tools *EXCEPT* for tv_grab_na -- which requires the time zone to be correctly set. 5) Install XMLTV ---------------- cd to the unpacked xmltv-x.x.x directory and run perl Makefile.PL The defaults will list all the grabbers that you can install and that are compatible with the Perl modules on your system. Either accept the defaults, or customise the grabbers/tools that you want to install.. Note tv_check will not work due to TK not being available. Build and test: make test Normally, all tests should pass successfully. Install: make install That?s it. You can now delete the downloaded xmltv files and unpacked directories if you want to. You now can use XMLTV like you would normally on unix: eg tv_grab_uk_rt --configure tv_grab_uk_rt --days 1 > uk_today.xml 6) Automation of grabbing (optional) ------------------------------------ If you need to automate the grabbing, you can use a Windows scheduled task to run a Bash shell script as follows: Create a file in your Cygwin home directory called xmltv_scheduled_grab.sh containing: -- START OF FILE -- CUT-HERE -- #!/usr/bin/bash # IMPORTANT! previous line must be first line of file! # # Set up environment . /etc/profile . ~/.bash_profile # cd to home directory cd # Set up TZ variable # CHANGE THIS TO MATCH YOUR TIME ZONE # export TZ=CET # Run grabber get 5 days listings, sending log messages to a logfile # CHANGE THE GRABBER COMMAND LINE TO WHAT YOU REQUIRE # tv_grab_uk_rt --days 5 --output uk_rt_xmltv.xml | tee grab_uk_rt_xmltv.log 2>&1 -- END OF FILE -- CUT-HERE -- (Obviously, as indicated, you need to change the grabber command line and the setting of TZ to what you require...) Make this command executable: chmod +x xmltv_scheduled_grab.sh Test this command at the cygwin command line xmltv_scheduled_grab.sh If it is OK, you should get 2 files in your cygwin home uk_rt_xmltv.xml containing the XMLTV output grab_uk_rt_xmltv.log containing the log messages from the grabber You now need to create a Windows scheduled task to run this script (Win XP: Start -> Control Panel -> Performance and Maintenance -> Scheduled tasks -> Add Scheduled task Win 2000: Start -> Control Panel -> Scheduled tasks -> Add Scheduled task) Select ANY program from the list (you will change the actual command line later). Select a schedule (you can also change this later) Ignore the username/password screen for the moment. Select 'Open advanced properties for this task' before clicking OK to allow you set the real command line in the Task Properties window... The problem with this command is that it will want to create a console window that will pop up on top of everything else when it runs... You can either create this window minimised (via a transient window that will pop up and disappear -- also annoying, but less so) using the scheduled command: cmd /c start /min c:\cygwin\bin\bash.exe ~/xmltv_scheduled_grab.sh or if you download and install the program runhide from: http://www.extendingflash.com/utilities/runhide.html and then use the scheduled command line runhide.exe c:\cygwin\bin\bash.exe ~/xmltv_scheduled_grab.sh then the window will never appear... You can test these command lines by pasting them into the start->Run box. Enter the required command line in the Scheduled Task Properties window. Click the 'Set password' button, and enter your password (if you have one). Check the Schedule and Settings tabs. Finally click OK. You can test the scheduled task by right-clicking it and selecting Run. ====================================================================== HISTORY ------- 09/03/2004 Niel Markwick initial version 11/03/2004 Niel Markwick added more information on Time Zones, corrected typo's xmltv-1.4.0/README.md000066400000000000000000000227571500074233200141560ustar00rootroot00000000000000

# XMLTV 1.4.0 ## Table of Contents - [XMLTV](#xmltv) * [Description](#description) * [Changes](#changes) * [Installation (Package)](#installation-package) + [Linux](#linux) + [Windows](#windows) + [MacOS](#macos) * [Installation (Source)](#installation-source) + [Getting Source Code](#getting-source-code) + [Building](#building) + [Required distributions/modules](#required-distributionsmodules) + [Recommended distributions/modules](#recommended-distributionsmodules) + [JSON libraries](#json-libraries) + [CPAN](#cpan) + [Proxy servers](#proxy-servers) * [Known issues](#known-issues) * [License](#license) * [Authors](#authors) * [Resources](#resources) ## Description The XMLTV project provides a suite of software to gather television listings, process listings data, and help organize your TV viewing. XMLTV listings use a mature XML file format for storing TV listings, which is defined and documented in the [XMLTV DTD](xmltv.dtd). In addition to the many "grabbers" that provide listings for large parts of the world, there are also several tools to process and filter these listings. Please see our [QuickStart](doc/QuickStart) documentation for details on what each program does. ## Changes To see what has changed in the current XMLTV release please check the [Changes](Changes) file. ## Installation (Package) ### Linux XMLTV is packaged for most major Linux distributions and FreeBSD. It is recommended that users install XMLTV using their preferred package manager. [![Packaging status](https://repology.org/badge/vertical-allrepos/xmltv.svg?minversion=1.0.0&columns=3)](https://repology.org/project/xmltv/versions) #### Debian/Ubuntu ```bash % sudo apt install xmltv ``` #### Fedora/CentOS Stream/Rocky Linux (via RPM Fusion) ```bash % dnf install xmltv ``` ### Windows Windows users are strongly advised to use the [pre-built binary](http://alpha-exe.xmltv.org/) as installing all prerequisites is non-trivial. Please also check the Github release page for a pre-built release binary. For those who want to give it a go, please read the [EXE build instructions](doc/exe_build.html). The instructions can be used for both building xmltv.exe as well as a local install. ### MacOS XMLTV is packaged for MacOS in the [MacPorts Project](https://ports.macports.org/port/xmltv/) ## Installation (Source) ### Getting Source Code #### Tarball/Zipfile The source code for the current release can be downloaded as a tarball (or zipfile) from [GitHub](https://github.com/XMLTV/xmltv/releases/latest) and extracted to a preferred location. #### Git The source code for all previous, current and future releases is available in our GitHub repository: ```bash % git clone https://github.com/XMLTV/xmltv.git ``` ### Building To build from source please ensure all required modules are available (see below). Change to the directory containing the XMLTV source: ```bash % perl Makefile.PL % make % make test % make install ``` To install to a custom directory, update the first line to provide a suitable `PREFIX` location: ``` % perl Makefile.PL PREFIX=/opt/xmltv/ ``` The system requirements are Perl 5.8.3 or later, and a few Perl modules. You will be asked about some optional components; if you choose not to install them then there are fewer dependencies. Please note that in addition to the specific modules listed below, the `tv_grab_zz_sdjson_sqlite` grabber requires Perl 5.16 to be installed. ### Required distributions/modules Required distributions/modules for XMLTV's core libraries are: ```perl Date::Manip 5.42a File::Slurp JSON (see note below) HTTP::Request HTTP::Response LWP 5.65 LWP::UserAgent LWP::Protocol::https Term::ReadKey URI XML::LibXML XML::Parser 2.34 XML::TreePP XML::Twig 3.28 XML::Writer 0.6.0 ``` Required modules for grabbers/utilities are: ``` CGI (tv_pick_cgi, core module until 5.20.3, part of CGI) CGI::Carp (tv_pick_cgi, core module until 5.20.3, part of CGI) Compress::Zlib (for some of the grabbers, core module since 5.9.3, part of IO::Compress) Data::Dump (for tv_grab_it_dvb) Date::Format (for some of the grabbers, part of TimeDate) DateTime (for several of the grabbers) DateTime::Format::ISO8601 (tv_grab_zz_sdjson_sqlite) DateTime::Format::SQLite (tv_grab_zz_sdjson_sqlite) DateTime::TimeZone (tv_grab_fr) DBD::SQLite (tv_grab_zz_sdjson_sqlite) DBI (tv_grab_zz_sdjson_sqlite) Digest::SHA (tv_grab_zz_sdjson{,_sqlite}, core module since 5.9.3) File::HomeDir (tv_grab_zz_sdjson_sqlite) File::Which (tv_grab_zz_sdjson_sqlite) HTML::Entities 1.27 (for several of the grabbers, part of HTML::Parser 3.34) HTML::Parser 3.34 (tv_grab_it_dvb, part of HTML::Parser 3.34) HTML::Tree (for many of the grabbers, part of HTML::Tree) HTML::TreeBuilder (for many of the grabbers, part of HTML::Tree) HTTP::Cache::Transparent 1.0 (for several of the grabbers) HTTP::Cookies (for several of the grabbers) IO::Scalar (for some of the grabbers, part of IO::Stringy) List::MoreUtils (tv_grab_zz_sdjson_sqlite) LWP::Protocol::https (tv_grab_fi, tv_grab_huro, tv_grab_zz_sdjson) LWP::UserAgent::Determined (tv_grab_zz_sdjson_sqlite) SOAP::Lite 0.67 (tv_grab_na_dd) Text::Unidecode (tv_grab_pt_vodafone) Time::Piece (tv_grab_huro, core module since 5.9.5) Time::Seconds (tv_grab_huro, core module since 5.9.5) Tk (tv_check) Tk::TableMatrix (tv_check) URI (for some of the grabbers, part of URI) URI::Encode (tv_grab_pt_vodafone) URI::Escape (for some of the grabbers, part of URI) XML::DOM (tv_grab_is) XML::LibXSLT (tv_grab_is) ``` When building XMLTV, any missing modules that are required for the selected grabbers/utilities will be reported. ### Recommended distributions/modules The following modules are recommended but XMLTV works without them installed: ``` File::chdir (testing grabbers) JSON::XS (faster JSON handling, see note below) Lingua::Preferred 0.2.4 (helps with multilingual listings) Log::TraceMessages (useful for debugging, not needed for normal use) PerlIO::gzip (can make tv_imdb a bit faster) Term::ProgressBar (displays pretty progress bars) Unicode::String (improved character handling in tv_to_latex) URI::Escape::XS (faster URI handling) ``` ### JSON libraries By default, libraries and grabbers that need to handle JSON data should specify the JSON module. This module is a wrapper for JSON::XS-compatible modules and supports the following JSON modules: ``` JSON::XS JSON::PP Cpanel::JSON::XS ``` JSON will use JSON::XS if available, falling back to JSON::PP (a core module since 5.14.0) if JSON::XS is not available. Cpanel::JSON::XS can be used as an explicit alternative by setting the PERL_JSON_BACKEND environment variable (please refer to the JSON module's documentation for details). ### CPAN All required modules can be quickly installed from CPAN using the `cpanm` utility. For example: ```bash % cpanm XML::Twig ``` Please note that you may find it easier to search for packaged versions of required modules, as sources which distribute a packaged version of XMLTV also provide packaged versions of required modules. ### Proxy servers Proxy server support is provide by the LWP modules. You can define a proxy server via the HTTP_PROXY environment variable. ```bash % HTTP_PROXY=http://somehost.somedomain:port ``` For more information, see this [article](http://search.cpan.org/~gaas/libwww-perl-5.803/lib/LWP/UserAgent.pm#$ua->env_proxy) ## Known issues If a full HTTP URL to the XMLTV.dtd is provided in the DOCTYPE declaration of an XMLTV document, please be aware that it is possible for the link to instead redirect to a page for accepting cookies. Such cookie-acceptance pages are more common in Europe, and can result in applications being unable to parse the file. ## License XMLTV is free software, distributed under the GNU General Public License, version 2. Please see [COPYING](COPYING) for more details. ## Authors There have been many contributors to XMLTV. Where possible they are credited in individual source files and in the [authors](authors.txt) mapping file. ## Resources ### GitHub Our [GitHub project](https://github.com/XMLTV/xmltv) contains all source code, issues and Pull Requests. ### Project Wiki We have a project [web page and wiki](http://www.xmltv.org) ### Mailing Lists We run the following mailing lists: - [xmltv-users](https://sourceforge.net/projects/xmltv/lists/xmltv-users): for users to ask questions and report problems with XMLTV software - [xmltv-devel](https://sourceforge.net/projects/xmltv/lists/xmltv-devel): for development discussion and support - [xmltv-announce](https://sourceforge.net/projects/xmltv/lists/xmltv-announce): announcements of new XMLTV releases ### IRC We run an IRC channel #xmltv on Libera Chat. Please join us! -- Nick Morrott, knowledgejunkie@gmail.com, 2025-04-17 xmltv-1.4.0/Uninstall.pm000066400000000000000000000030601500074233200151700ustar00rootroot00000000000000# Supplement to ExtUtils::MakeMaker to add back some rudimentary # uninstall functionality. This needs to be called with an extra # 'uninstall' target in the Makefile, for which you will need to # modify your Makefile.PL. I kept well away from the deprecated and # no-longer-working uninstall stuff in MakeMaker itself. package Uninstall; use strict; use base 'Exporter'; our @EXPORT = qw(uninstall); use File::Find; sub uninstall( % ) { my %h = @_; my %seen_pl; foreach (qw(read write)) { my $pl = delete $h{$_}; if (defined $pl and not $seen_pl{$pl}++) { warn "ignoring packlist $pl\n"; } } foreach my $from (keys %h) { next if not -e $from; my $to = $h{$from}; print "uninstalling contents of $from from $to\n"; find(sub { for ($File::Find::name) { # return if not -f; # why doesn't this work? # The behaviour of File::Find seems different # under 5.005. # s!^\Q$from\E/*!! or ($[ < 5.006) or die "filename '$_' doesn't start with $from/"; return if not length; # skip directory itself return if m!(?:/|^)\.exists!; my $inside_to = "$to/$_"; if (-e $inside_to) { if (-f $inside_to) { print "unlinking $inside_to\n"; unlink $inside_to or warn "cannot unlink $inside_to: $!"; } elsif (-d $inside_to) { print "not removing directory $inside_to\n"; } else { print "not removing special file $inside_to\n"; } } else { print "$inside_to is not installed\n"; } } }, $from); } } xmltv-1.4.0/analyse_tvprefs/000077500000000000000000000000001500074233200160675ustar00rootroot00000000000000xmltv-1.4.0/analyse_tvprefs/README000066400000000000000000000002731500074233200167510ustar00rootroot00000000000000A silly little tool to work out what things interest you by processing the 'tvprefs' file generated by tv_pick_cgi. Please see . xmltv-1.4.0/analyse_tvprefs/analyse_tvprefs000077500000000000000000000074701500074233200212320ustar00rootroot00000000000000#!/usr/bin/perl # # Tool for finding frequency of words in programme titles, see # . use warnings; use strict; my $opt_noprefs = 0; if (@ARGV) { if ($ARGV[0] eq '--noprefs') { $opt_noprefs = 1; shift @ARGV; } else { die "usage: $0 [--noprefs]"; } } my %type_scores = (never => -2, no => -1, yes => 1, always => 2); my (%pos_points, %neg_points); my ($total_pos, $total_neg); while (<>) { s/\#.*//; s/^\s+//; s/\s+$//; next unless /\S/; s/^(\w+):\s*//; unless ($opt_noprefs) { my $type = $1; die if not defined $type; my $score = $type_scores{$type}; if (not defined $score) { die "$ARGV:$.: unknown type $type"; } while (s/(\w+)//) { my $word = lc $1; if ($score > 0) { $pos_points{$word} += $score; # Add zero just to make sure the entry gets there. So # no undefined value warnings later on. # $neg_points{$word} += 0; $total_pos += $score; } elsif ($score < 0) { $neg_points{$word} += -$score; $pos_points{$word} += 0; $total_neg += -$score; } } } else { while (s/(\w+)//) { my $word = lc $1; ++ $pos_points{$word}; $neg_points{$word} = 0; ++ $total_pos; $total_neg = 0; } } } # Normalize - first, if we are doing preferences, by rebalancing # positive and negative scores. (Normally, there will be many more # 'never' programmes than any other kind, so without normalizing # there'd be huge negative scores and a few small positive ones.) # unless ($opt_noprefs) { if ($total_pos == 0) { die "no programmes at all had a positive score\n"; } elsif ($total_neg == 0) { die "no programmes at all had a negative score\n"; } my $norm_factor = $total_neg / $total_pos; foreach (keys %pos_points) { $pos_points{$_} *= $norm_factor; } } # Now we've got +ve and -ve on the same sort of scale, we can add them # to get an overall score for each word. # my %score; foreach (keys %pos_points) { $score{$_} = $pos_points{$_} - $neg_points{$_}; } # Now divide by the total number of points allocated, so we get # reasonably similar-sized numbers no matter how large or small the # prefs file. # # Finally, divide each word's score by its frequency in English: so we # don't end up with a big negative score for 'the', outweighing # everything else, for example. # my $total = $total_pos + $total_neg; foreach (keys %score) { $score{$_} /= $total; $score{$_} /= word_freq($_); } foreach (sort { $score{$a} <=> $score{$b} || $a cmp $b } keys %score) { printf "%s:\t%.3g\n", $_, $score{$_}; } # Given a word, return its frequency in English. We have a file of # the top 3000; we assume that these are the only words in English, # but that lower-ranking words all have the same frequency as the # 3000th. (Nobody is checking whether the frequencies add up to 1.) # # The file is distilled from the 1996 version of the British National # Corpus; I have mirrored it at # . # my %bnc_freq; my $default_freq; sub word_freq { my $w = shift; if (not defined $default_freq) { open(BNC_FREQ, 'bnc_freq.txt') or die "cannot open bnc_freq.txt: $!"; my $total = 0; while () { chomp; s/\r$//; next unless /\S/; my @F = split; die "bnc_freq.txt:$.: bad line" if @F != 3; # Add to existing entry, if it exists, since we aren't # interested in parts of speech distinctions. # $bnc_freq{$F[0]} += $F[1]; $total += $F[1]; } die "no frequencies found in bnc_freq.txt" if $total == 0; my $least; foreach (keys %bnc_freq) { $bnc_freq{$_} /= $total; if (not defined $least or $least > $bnc_freq{$_}) { $least = $bnc_freq{$_}; } } $default_freq = $least; } defined $bnc_freq{$w} && return $bnc_freq{$w}; return $default_freq; } xmltv-1.4.0/analyse_tvprefs/bnc_freq.txt000066400000000000000000002607421500074233200204220ustar00rootroot00000000000000the 6187267 at0 of 2941444 prf and 2682863 cjc a 2126369 at0 in 1812609 prp to 1620850 to0 it 1089186 pnp is 998389 vbz was 923948 vbd to 917579 prp i 884599 pnp for 833360 prp you 695498 pnp he 681255 pnp be 662516 vbi with 652027 prp on 647344 prp that 628999 cjt by 507317 prp at 478162 prp are 470943 vbb not 462486 xx0 this 461945 dt0 but 454096 cjc 's 442545 pos they 433441 pnp his 426896 dps from 413532 prp had 409012 vhd she 380257 pnp which 372031 dtq or 370808 cjc we 358039 pnp an 343063 at0 n't 332839 xx0 's 325048 vbz were 322824 vbd that 286913 dt0 been 268723 vbn have 268490 vhb their 260919 dps has 259431 vhz would 255188 vm0 what 249466 dtq will 244822 vm0 there 239460 ex0 if 237089 cjs can 234386 vm0 all 227737 dt0 her 218258 dps as 208623 cjs who 205432 pnq have 205195 vhi do 196635 vdb that 194800 cjt-dt0 one 190499 crd said 185277 vvd them 173414 pnp some 171174 dt0 could 168387 vm0 him 165014 pnp into 163469 prp its 163081 dps then 160652 av0 two 156111 crd when 155417 avq-cjs up 154288 avp time 153679 nn1 my 152619 dps out 150958 avp so 147324 av0 did 143405 vdd about 142118 prp your 138334 dps now 137801 av0 me 137151 pnp no 137026 at0 more 134029 av0 other 129451 aj0 just 125465 av0 these 125442 dt0 also 124884 av0 people 123916 nn0 any 123655 dt0 first 118699 ord only 115994 av0 new 114655 aj0 may 113024 vm0 very 111538 av0 should 111236 vm0 as 111083 cjs-prp like 108988 prp her 108710 pnp than 108618 cjs as 106427 prp how 101508 avq well 96080 av0 way 95313 nn1 our 95001 dps as 91583 av0 between 90541 prp years 90173 nn2 er 89838 unc many 89659 dt0 those 88862 dt0 there 88490 av0 've 87706 vhb being 87105 vbg because 85183 cjs do 83661 vdi 're 83504 vbb yeah 83382 itj three 79759 crd down 76798 avp such 75524 dt0 back 75494 avp good 74839 aj0 where 74609 avq-cjs year 73757 nn1 through 73166 prp 'll 72591 vm0 must 72059 vm0 still 71717 av0 even 70962 av0 know 70238 vvb too 70164 av0 here 69947 av0 get 69636 vvi own 69459 dt0 does 68725 vdz oh 68413 itj last 68063 ord no 67999 itj more 67198 dt0 'm 65789 vbb going 64163 vvg so 64028 cjs erm 62603 unc after 62570 prp us 62350 pnp government 62163 nn0 might 61446 vm0 same 61402 dt0 much 60838 av0 see 60628 vvi yes 60592 itj go 59772 vvi make 59664 vvi day 58863 nn1 man 58769 nn1 another 58182 dt0 world 58070 nn1 see 57933 vvb got 57451 vvn work 57401 nn1 however 57150 av0 life 56592 nn1 again 56229 av0 against 56208 prp think 56091 vvb never 55899 av0 under 55498 prp one 55038 crd-pni most 54966 av0 old 54595 aj0 over 53619 prp know 53034 vvi something 52452 pni mr 52382 np0 take 51486 vvi why 50877 avq each 50732 dt0 while 50548 cjs part 50195 nn1 on 49967 avp number 49286 nn1 out_of 49038 prp made 48397 vvn different 48373 aj0 really 48062 av0 went 48028 vvd ' 47416 pos came 46919 vvd after 46739 cjs-prp children 46577 nn2 always 46228 av0 four 46115 crd without 45867 prp one 45845 pni within 45042 prp system 44651 nn1 local 44220 aj0 during 44013 prp most 43792 dt0 although 43635 cjs next 43139 ord small 43008 aj0 case 42924 nn1 great 42691 aj0 things 42409 nn2 social 41605 aj0 end 41468 nn1 say 41291 vvi quite 41169 av0 both 41162 dt0 group 41030 nn0 five 40738 crd about 40710 av0 every 40114 at0 women 39886 nn2 says 39773 vvz party 39561 nn0 's 39316 vhz important 39265 aj0 place 39201 nn1 house 39116 nn1 took 38831 vvd before 38670 prp information 38641 nn1 1 38384 crd men 38324 nn2 per_cent 38205 nn0 often 37640 av0 seen 37541 vvn money 37355 nn1 school 37331 nn1 national 37211 aj0 fact 36970 nn1 night 36521 nn1 had 36378 vhn given 36297 vvn second 35922 ord point 35876 nn1 off 35628 avp done 35473 vdn having 35274 vhg thing 35203 nn1 british 35174 aj0 london 35142 np0 taken 35108 vvn think 35104 vvi area 35096 nn1 perhaps 35039 av0 company 35026 nn1 in 34946 avp all 34704 av0 family 34455 nn0 2 34394 crd hand 34357 nn1 'd 34314 vm0 already 34292 av0 possible 34178 aj0 over 34115 avp-prp nothing 34064 pni when 33947 cjs mm 33893 itj home 33783 nn1 large 33768 aj0 yet 33658 av0 business 33573 nn1 in 33413 avp-prp water 33287 nn1 side 33240 nn1 whether 33169 cjs days 33048 nn2 used 32968 vvn john 32813 np0 'd 32393 vhd development 32276 nn1 week 32216 nn1 state 32067 nn1 such_as 32060 prp give 32016 vvi head 31925 nn1 want 31871 vvb ca 31730 vm0 right 31638 av0 almost 31588 av0 country 31408 nn1 much 31284 dt0 himself 31082 pnx find 31052 vvi council 31032 nn0 power 30962 nn1 of_course 30942 av0 come 30909 vvi thought 30871 vvd young 30831 aj0 room 30794 nn1 able 30410 aj0 political 30366 aj0 six 30355 crd later 30326 av0 until 29994 cjs get 29852 vvb members 29660 nn2 eyes 29653 nn2 public 29475 aj0 use 29306 nn1 service 29146 nn1 mean 29089 vvb problem 28991 nn1 though 28801 cjs high 28698 aj0 go 28605 vvb towards 28600 prp anything 28321 pni up 28276 avp-prp others 28255 nn2 war 27842 nn1 car 27798 nn1 both 27644 av0 doing 27632 vdg police 27508 nn2 problems 27492 nn2 interest 27436 nn1 probably 27303 av0 available 27184 aj0 where 26814 avq on 26791 avp-prp say 26109 vvb saw 26076 vvd full 26053 aj0 knew 26018 vvd actually 25990 av0 education 25973 nn1 policy 25941 nn1 ever 25895 av0 am 25884 vbb made 25871 vvd-vvn best 25846 ajs a_few 25781 dt0 today 25775 av0 face 25737 nn1 at_least 25713 av0 times 25677 nn2 enough 25635 av0 office 25587 nn1 looking 25477 vvg door 25360 nn1 since 25352 cjs voice 25206 nn1 before 25178 cjs-prp less 25147 av0 britain 25090 np0 form 25074 nn1 body 25043 nn1 3 25040 crd person 24982 nn1 together 24960 av0 research 24923 nn1 when 24913 avq want 24858 vvi services 24816 nn2 months 24809 nn2 up_to 24704 prp big 24684 aj0 only 24641 aj0 name 24592 nn1 little 24572 aj0 million 24550 crd health 24527 nn1 law 24510 nn1 question 24487 nn1 book 24479 nn1 long 24468 aj0 words 24399 nn2 mother 24396 nn1 using 24395 vvg child 24385 nn1 come 24303 vvb period 24290 nn1 making 24264 vvg got 24146 vvd court 24077 nn1 main 24054 aj0 several 24002 dt0 for_example 23829 av0 society 23776 nn0 market 23719 nn1 itself 23712 pnx themselves 23673 pnx like 23664 vvi began 23576 vvd economic 23484 aj0 upon 23409 prp areas 23353 nn2 away 23303 avp major 23282 aj0 therefore 23218 av0 woman 23201 nn1 england 23142 np0 over 23117 av0 including 23082 prp help 22985 vvi among 22864 prp gave 22838 vvd right 22835 aj0 real 22815 aj0 job 22784 nn1 likely 22592 aj0 position 22581 nn1 process 22570 nn1 effect 22509 nn1 known 22432 vvn a 22367 zz0 far 22350 av0 line 22333 nn1 half 22232 dt0 staff 22227 nn0 became 22188 vvd moment 22146 nn1 community 22146 nn0 difficult 22033 aj0 action 22018 nn1 particularly 22002 av0 special 21966 aj0 particular 21937 aj0 international 21929 aj0 kind 21900 nn1 father 21892 nn1 report 21889 nn1 look 21879 vvi age 21781 nn1 across 21763 prp management 21733 nn0 idea 21702 nn1 certain 21626 aj0 so_that 21513 cjs taking 21445 vvg evidence 21364 nn1 rather 21341 av0 looked 21318 vvd off 21315 avp-prp minister 21241 nn1 tell 21200 vvi third 21197 ord seems 21194 vvz view 21150 nn1 morning 21045 nn1 found 21030 vvn else 20919 av0 sense 20860 nn1 use 20822 vvi behind 20694 prp back 20532 nn1 sometimes 20517 av0 thus 20488 av0 level 20398 nn1 getting 20355 vvg better 20343 ajc ten 20268 crd shall 20234 vm0 need 20209 vvb over 20205 avp seemed 20152 vvd table 20142 nn1 further 20138 av0 made 20118 vvd death 20061 nn1 4 20017 crd mrs 19837 np0 whose 19833 dtq industry 19817 nn1 ago 19808 av0 century 19696 nn1 take 19694 vvb control 19623 nn1 sort 19539 nn1 gone 19529 vvn church 19500 nn1 yesterday 19459 av0 clear 19437 aj0 range 19428 nn1 black 19375 aj0 general 19349 aj0 word 19313 nn1 groups 19309 nn2 work 19297 vvi history 19296 nn1 free 19295 aj0 keep 19275 vvi road 19251 nn1 order 19196 nn1 few 19160 dt0 usually 19151 av0 make 19128 vvb wanted 19116 vvd hundred 19108 crd centre 19073 nn1 put 18997 vvi food 18985 nn1 study 18967 nn1 programme 18936 nn1 result 18902 nn1 air 18901 nn1 hours 18884 nn2 committee 18864 nn0 indeed 18858 av0 team 18851 nn0 experience 18843 nn1 change 18830 nn1 course 18772 nn1 language 18771 nn1 white 18701 aj0 someone 18681 pni everything 18675 pni certainly 18647 av0 mind 18643 nn1 hands 18620 nn2 10 18570 crd home 18540 av0 rate 18532 nn1 section 18517 nn1 said 18496 vvn similar 18432 aj0 trade 18414 nn1 minutes 18374 nn2 reason 18350 nn1 authority 18260 nn1 cases 18213 nn2 role 18210 nn1 data 18188 nn0 working 18144 vvg europe 18142 np0 before 18142 cjs necessary 18107 aj0 nature 18099 nn1 class 18076 nn1 felt 18060 vvd central 17947 aj0 because_of 17812 prp well 17809 itj english 17770 aj0 companies 17763 nn2 rather_than 17759 prp simply 17756 av0 early 17746 aj0 department 17698 nn0 especially 17694 av0 asked 17691 vvd called 17637 vvn personal 17622 aj0 value 17586 nn1 5 17557 crd member 17481 nn1 around 17471 prp patients 17330 nn2 seven 17319 crd uk 17315 np0 paper 17303 nn1 land 17301 nn1 eight 17298 crd systems 17261 nn2 true 17252 aj0 herself 17197 pnx told 17191 vvd support 17156 nn1 act 17137 nn1 type 17120 nn1 wife 17102 nn1 once 17032 av0 city 16982 nn1 former 16973 dt0 common 16938 aj0 friend 16846 nn1 changes 16840 nn2 countries 16769 nn2 care 16728 nn1 sure 16633 aj0 decision 16628 nn1 financial 16622 aj0 single 16620 aj0 terms 16563 nn2 price 16522 nn1 provide 16467 vvi stage 16409 nn1 matter 16375 nn1 parents 16333 nn2 club 16331 nn0 practice 16325 nn1 based 16293 vvn trying 16275 vvg as_well_as 16257 cjc private 16253 aj0 cos 16247 cjs foreign 16234 aj0 god 16194 np0 saying 16179 vvg wo 16164 vm0 further 16135 ajc need 16129 nn1 whole 16057 aj0 town 16027 nn1 makes 16027 vvz open 15991 aj0 situation 15978 nn1 comes 15968 vvz soon 15903 av0 strong 15898 aj0 b 15879 zz0 bed 15854 nn1 recent 15812 aj0 us 15783 np0 girl 15755 nn1 according_to 15722 prp david 15714 np0 higher 15676 ajc twenty 15641 crd as_if 15633 cjs better 15626 av0 quality 15617 nn1 either 15599 av0 european 15544 aj0 various 15502 aj0 to 15488 avp-prp conditions 15369 nn2 clearly 15349 av0 at_all 15348 av0 ground 15322 nn1 weeks 15296 nn2 down 15296 avp-prp long 15288 av0 may 15242 np0 tax 15230 nn1 look 15172 vvb found 15170 vvd away 15153 av0 poor 15124 aj0 production 15091 nn1 friends 15081 nn2 shown 15029 vvn month 15010 nn1 na 14995 to0 music 14993 nn1 1990 14982 crd anyone 14956 pni game 14930 nn1 ways 14927 nn2 schools 14855 nn2 issue 14796 nn1 mr. 14756 np0 royal 14711 aj0 put 14705 vvd april 14702 np0 workers 14695 nn2 goes 14692 vvz american 14639 aj0 students 14592 nn2 despite 14592 prp knowledge 14586 nn1 june 14580 np0 art 14508 nn1 let 14489 vvb hair 14443 nn1 basis 14420 nn1 subject 14396 nn1 p. 14361 nn0 series 14348 nn0 natural 14293 aj0 coming 14211 vvg come 14203 vvn bad 14196 aj0 bank 14182 nn1 since 14105 cjs-prp feet 14100 nn2 greater 14087 ajc south 14053 np0 simple 13936 aj0 lot 13910 nn0 pay 13873 vvi west 13871 np0 rest 13852 nn1 secretary 13828 nn1 security 13821 nn1 manager 13799 nn1 hospital 13790 nn1 cost 13780 nn1 heart 13729 nn1 structure 13700 nn1 attention 13684 nn1 story 13661 nn1 & 13629 cjc means 13627 vvz nine 13626 crd letter 13589 nn1 questions 13584 nn2 need 13570 vvi chapter 13569 nn1 field 13554 nn1 short 13547 aj0 become 13542 vvi studies 13519 nn2 per 13502 prp movement 13492 nn1 6 13425 crd union 13395 nn0 success 13392 nn1 figure 13388 nn1 united 13381 np0 1989 13376 crd everyone 13337 pni near 13248 prp whatever 13236 dtq analysis 13232 nn1 news 13223 nn1 chance 13218 nn1 evening 13209 nn1 population 13181 nn1 boy 13158 nn1 modern 13144 aj0 20 13139 crd following 13127 aj0 theory 13117 nn1 legal 13103 aj0 computer 13089 nn1 scotland 13084 np0 approach 13076 nn1 books 13073 nn2 final 13058 aj0 wrong 13023 aj0 finally 13014 av0 ask 13011 vvi give 12997 vvb performance 12993 nn1 human 12989 aj0 off 12972 prp authorities 12971 nn2 environment 12966 nn1 please 12962 av0 fire 12955 nn1 rights 12944 nn2 whom 12926 pnq above 12889 prp relationship 12888 nn1 in_order 12878 av0 told 12826 vvn 1991 12798 crd feel 12791 vvi growth 12734 nn1 agreement 12719 nn1 parties 12699 nn2 account 12695 nn1 found 12692 vvd-vvn nice 12689 aj0 size 12663 nn1 son 12644 nn1 held 12629 vvn space 12606 nn1 amount 12550 nn1 property 12539 nn1 example 12496 nn1 project 12487 nn1 normal 12451 aj0 myself 12444 pnx nor 12442 cjc gon 12442 vvg meeting 12398 nn1 set 12383 vvn quickly 12381 av0 feel 12379 vvb record 12368 nn1 france 12331 np0 behaviour 12316 nn1 left 12315 vvn c 12297 zz0 previous 12286 aj0 only 12268 aj0-av0 peter 12263 np0 giving 12254 vvg recently 12249 av0 serious 12232 aj0 anyway 12232 av0 couple 12202 nn1 energy 12195 nn1 sir 12192 np0 term 12176 nn1 either 12167 dt0 director 12148 nn1 current 12141 aj0 st 12134 np0 east 12110 np0 12 12102 crd love 12093 nn1 significant 12073 aj0 put 12051 vvn income 12050 nn1 leave 12002 vvi building 11987 nn1 as_well 11985 av0 pressure 11949 nn1 right 11946 aj0-nn1 levels 11894 nn2 treatment 11892 nn1 july 11892 np0 north 11872 np0 prime 11853 aj0 throughout 11810 prp model 11798 nn1 suddenly 11795 av0 pounds 11765 nn2 choice 11759 nn1 away_from 11756 prp results 11755 nn2 scheme 11730 nn1 try 11693 vvi fine 11685 aj0 details 11675 nn2 takes 11674 vvz bring 11669 vvi design 11657 nn1 happy 11649 aj0 list 11644 nn1 defence 11636 nn1 parts 11621 nn2 bit 11616 nn1 points 11610 nn2 red 11605 aj0 loss 11601 nn1 industrial 11599 aj0 got 11598 vvd-vvn activities 11597 nn2 low 11591 aj0 floor 11545 nn1 generally 11537 av0 issues 11527 nn2 used 11513 vm0 activity 11508 nn1 rates 11484 nn2 nearly 11484 av0 become 11473 vvn other 11468 aj0-nn1 march 11443 np0 sorry 11435 aj0 30 11413 crd paul 11400 np0 mean 11378 vvi forward 11378 av0 army 11365 nn0 talking 11350 vvg difference 11335 nn1 hour 11334 nn1 labour 11324 aj0 summer 11302 nn1 specific 11300 aj0 outside 11276 prp numbers 11255 nn2 concerned 11238 aj0 husband 11222 nn1 lord 11218 np0 relations 11202 nn2 following 11190 vvg dr 11175 np0 understand 11156 vvi contract 11150 nn1 turned 11146 vvd product 11111 nn1 ideas 11096 nn2 george 11087 np0 material 11084 nn1 wall 11060 nn1 meet 11055 vvi believe 11041 vvb thank 11033 vvb obviously 11014 av0 unless 11011 cjs appropriate 11010 aj0 teachers 11005 nn2 dead 11005 aj0 military 10998 aj0 season 10948 nn1 15 10915 crd look 10911 nn1 unit 10905 nn1 arms 10895 nn2 easy 10894 aj0 lower 10882 ajc show 10874 vvi basic 10860 aj0 find 10833 vvb reasons 10805 nn2 successful 10803 aj0 colour 10779 nn1 play 10772 vvi aware 10764 aj0 technology 10762 nn1 each_other 10759 pnx effects 10753 nn2 yourself 10746 pnx exactly 10729 av0 beyond 10705 prp figures 10686 nn2 style 10683 nn1 7 10676 crd round 10664 avp date 10651 nn1 germany 10632 np0 october 10617 np0 products 10601 nn2 popular 10600 aj0 hear 10585 vvi window 10578 nn1 science 10558 nn1 forces 10555 nn2 showed 10547 vvd new 10544 np0 more_than 10524 av0 resources 10522 nn2 less 10522 dt0 once 10511 cjs long 10511 aj0-av0 sea 10485 nn1 looked 10484 vvd-vvn maybe 10472 av0 events 10452 nn2 gives 10435 vvz advice 10431 nn1 circumstances 10417 nn2 above 10408 av0 september 10400 np0 de 10400 np0 plan 10398 nn1 present 10397 aj0 event 10387 nn1 hon. 10375 aj0 thousand 10373 crd set 10349 nn1 training 10336 nn1-vvg stood 10334 vvd picture 10333 nn1 sales 10326 nn2 1988 10322 crd village 10310 nn1 original 10301 aj0 america 10297 np0 called 10263 vvd-vvn 1992 10240 crd investment 10224 nn1 oil 10217 nn1 thinking 10179 vvg cup 10177 nn1 lines 10172 nn2 january 10163 np0 believe 10145 vvi james 10143 np0 okay 10140 av0 goods 10134 nn2 blood 10133 nn1 opportunity 10120 nn1 like 10115 vvb useful 10071 aj0 prices 10067 nn2 professional 10055 aj0 conference 10054 nn1 follows 10039 vvz extent 10036 nn1 interests 10032 nn2 application 10027 nn1 page 10020 nn1 operation 10001 nn1 film 9995 nn1 richard 9993 np0 in_terms_of 9993 prp response 9982 nn1 right 9981 nn1 ones 9971 pni majority 9965 nn0 rules 9957 nn2 shop 9952 nn1 effective 9951 aj0 press 9932 nn0 ah 9930 itj written 9907 vvn york 9903 np0 used 9892 vvd-vvn move 9890 vvi hard 9890 aj0 sat 9878 vvd easily 9877 av0 degree 9865 nn1 wrote 9861 vvd statement 9854 nn1 around 9844 avp risk 9838 nn1 force 9838 nn1 miles 9836 nn2 capital 9834 nn1 economy 9831 nn1 means 9823 nn0 traditional 9819 aj0 site 9817 nn1 glass 9817 nn1 ready 9812 aj0 died 9809 vvd president 9781 nn1 house 9781 np0 street 9763 np0 costs 9755 nn2 earlier 9751 av0 playing 9738 vvg fish 9733 nn0 stop 9729 vvi scottish 9724 aj0 8 9713 crd importance 9712 nn1 remember 9708 vvi individual 9687 aj0 test 9663 nn1 complete 9660 aj0 jobs 9656 nn2 immediately 9652 av0 standards 9640 nn2 work 9634 nn1-vvb until 9626 cjs-prp talk 9624 vvi considerable 9614 aj0 girls 9611 nn2 ireland 9610 np0 asked 9607 vvn parliament 9601 nn1 labour 9601 nn1 interesting 9582 aj0 physical 9564 aj0 thought 9561 vvn species 9555 nn0 little 9553 dt0 thought 9538 nn1 garden 9537 nn1 title 9533 nn1 michael 9516 np0 start 9513 vvi street 9511 nn1 eye 9509 nn1 access 9499 nn1 help 9479 nn1 brought 9474 vvn employment 9464 nn1 buy 9451 vvi fell 9446 vvd daughter 9443 nn1 remember 9437 vvb responsible 9429 aj0 competition 9428 nn1 tell 9427 vvb around 9418 avp-prp about 9415 avp left 9410 vvd-vvn held 9392 vvd-vvn november 9390 np0 december 9367 np0 sunday 9364 np0 plans 9334 nn2 medical 9334 aj0 purpose 9317 nn1 mouth 9316 nn1 piece 9294 nn1 wide 9268 aj0 heavy 9262 aj0 answer 9254 nn1 tomorrow 9243 av0 leaving 9240 vvg 11 9238 crd thirty 9234 crd use 9214 vvb needs 9205 nn2 put 9204 vvb task 9188 nn1 worth 9183 prp wales 9183 np0 responsibility 9179 nn1 doctor 9179 nn1 later 9170 ajc arm 9155 nn1 eventually 9138 av0 ability 9135 nn1 highly 9132 av0 hotel 9125 nn1 looks 9118 vvz pattern 9096 nn1 method 9091 nn1 source 9088 nn1 existing 9074 aj0 television 9071 nn1 lost 9069 vvn election 9069 nn1 ensure 9059 vvi baby 9057 nn1 charles 9043 np0 a_lot 9042 pni region 9027 nn1 suppose 9025 vvb early 9023 aj0-av0 total 9018 aj0 seem 9002 vvi change 9002 vvi surface 8993 nn1 best 8990 av0 required 8978 vvn older 8970 ajc heard 8954 vvn methods 8953 nn2 future 8952 nn1 left 8938 vvd campaign 8934 nn1 equipment 8924 nn1 fully 8911 av0 disease 8905 nn1 machine 8902 nn1 lack 8898 nn1 independent 8893 aj0 slightly 8873 av0 software 8851 nn1 hot 8849 aj0 peace 8839 nn1 charge 8839 nn1 types 8837 nn2 policies 8834 nn2 let 8833 vvi houses 8829 nn2 no_longer 8828 av0 due_to 8814 prp hardly 8801 av0 direct 8801 aj0 otherwise 8798 av0 left 8791 aj0 directly 8784 av0 even_if 8757 cjs windows 8755 nn2 stay 8748 vvi teacher 8739 nn1 radio 8728 nn1 forms 8724 nn2 provision 8720 nn1 factors 8702 nn2 direction 8702 nn1 trouble 8686 nn1 ran 8676 vvd beautiful 8670 aj0 leader 8669 nn1 civil 8660 aj0 officers 8655 nn2 status 8654 nn1 places 8641 nn2 sound 8639 nn1 allow 8631 vvi ii 8629 crd character 8626 nn1 wants 8623 vvz variety 8619 nn1 considered 8617 vvn light 8616 nn1 continue 8615 vvi safety 8609 nn1 completely 8605 av0 tea 8599 nn1 box 8589 nn1 below 8587 av0 fifty 8579 crd sector 8577 nn1 animals 8568 nn2 oxford 8555 np0 around 8553 av0 culture 8548 nn1 weight 8537 nn1 a_little 8518 av0 top 8512 aj0-nn1 obvious 8510 aj0 i 8509 crd late 8506 aj0 1987 8488 crd increase 8470 nn1 include 8453 vvb context 8453 nn1 on_to 8448 prp station 8447 nn1 since 8444 prp needs 8441 vvz none 8440 pni turned 8437 vvd-vvn sale 8430 nn1 afternoon 8420 nn1 knows 8419 vvz produce 8403 vvi william 8389 np0 february 8378 np0 positive 8371 aj0 king 8369 nn1 top 8366 nn1 provides 8364 vvz miss 8361 np0 essential 8342 aj0 extra 8341 aj0 training 8335 nn1 live 8327 vvi condition 8322 nn1 set 8318 vvd families 8318 nn2 works 8315 nn0 try 8313 vvb saturday 8310 np0 university 8309 nn1 appeal 8307 nn1 along 8307 prp trees 8306 nn2 sex 8300 nn1 allowed 8298 vvn argument 8267 nn1 normally 8265 av0 kitchen 8244 nn1 brother 8238 nn1 keep 8237 vvb demand 8212 nn1 library 8206 nn1 alone 8183 aj0-av0 principle 8182 nn1 run 8179 vvi let's 8161 vm0 north 8150 nn1 for 8143 cjs-prp pupils 8141 nn2 start 8138 nn1 chairman 8138 nn1 cash 8125 nn1 version 8124 nn1 expected 8116 vvn states 8113 np0 hope 8107 vvb sun 8104 nn1 league 8103 nn1 as_to 8086 prp involved 8081 vvd-vvn duty 8079 nn1 county 8077 nn1 18 8077 crd rule 8057 nn1 concern 8052 nn1 environmental 8050 aj0 presence 8048 nn1 boys 8047 nn2 truth 8045 nn1 16 8044 crd dog 8030 nn1 french 8022 aj0 board 8015 nn1 courses 8014 nn2 individuals 8009 nn2 seem 7989 vvb blue 7987 aj0 general 7982 aj0-nn1 media 7977 nn0 exchange 7959 nn1 14 7955 crd 9 7954 crd relevant 7950 aj0 august 7941 np0 balance 7926 nn1 robert 7921 np0 turn 7919 vvi 1986 7919 crd slowly 7914 av0 relatively 7912 av0 close 7911 aj0 players 7909 nn2 discussion 7906 nn1 huge 7901 aj0 paid 7892 vvn sure 7883 aj0-av0 letters 7880 nn2 heard 7865 vvd budget 7863 nn1 protection 7859 nn1 marriage 7858 nn1 road 7849 np0 i 7840 zz0 collection 7837 nn1 speech 7828 nn1 south 7828 nn1 smith 7821 np0 africa 7820 np0 born 7812 vvn differences 7803 nn2 high 7800 aj0-av0 effort 7792 nn1 gets 7787 vvz attempt 7781 nn1 nuclear 7776 aj0 survey 7764 nn1 latter 7762 dt0 failure 7762 nn1 annual 7754 aj0 practical 7745 aj0 career 7733 nn1 horse 7729 nn1 features 7729 nn2 light 7719 aj0-nn1 chair 7717 nn1 becomes 7717 vvz 25 7706 crd matters 7696 nn2 apparently 7696 av0 turn 7685 nn1 started 7681 vvd subjects 7675 nn2 shares 7672 nn2 text 7665 nn1 appears 7658 vvz consider 7657 vvi student 7645 nn1 separate 7641 aj0 larger 7639 ajc cells 7635 nn2 future 7616 aj0-nn1 accept 7609 vvi function 7602 nn1 merely 7596 av0 plus 7592 prp memory 7588 nn1 studio 7585 nn1 involved 7572 vvn regular 7561 aj0 13 7556 crd provided 7555 vvn a_little 7535 dt0 decisions 7529 nn2 little 7525 av0 call 7520 vvi opposition 7516 nn1 facilities 7514 nn2 commercial 7511 aj0 running 7505 vvg bar 7504 nn1 cars 7497 nn2 skills 7496 nn2 firms 7496 nn2 post 7495 nn1 mhm 7495 itj after 7486 cjs edge 7478 nn1 built 7476 vvn image 7461 nn1 politics 7456 nn0 reference 7455 nn1 aid 7446 nn1 show 7444 nn1 organisation 7441 nn1 instead 7437 av0 putting 7433 vvg expression 7422 nn1 award 7417 nn1 influence 7403 nn1 quarter 7398 nn1 no_one 7390 pni association 7389 nn0 award 7388 nn1-vvb public 7377 nn0 opinion 7367 nn1 additional 7364 aj0 1985 7359 crd benefit 7355 nn1 largely 7350 av0 for_instance 7343 av0 advantage 7338 nn1 late 7333 aj0-av0 benefits 7317 nn2 somebody 7316 pni western 7314 aj0 happen 7309 vvi smaller 7301 ajc pain 7299 nn1 gas 7291 nn1 aspects 7291 nn2 active 7290 aj0 names 7282 nn2 scale 7281 nn1 university 7278 np0 early 7274 av0 officer 7271 nn1 clothes 7271 nn2 stock 7261 nn1 plant 7261 nn1 affairs 7258 nn2 views 7254 nn2 impact 7246 nn1 nevertheless 7236 av0 leaders 7234 nn2 regional 7231 aj0 base 7221 nn1 deal 7217 nn1 complex 7214 aj0 powerful 7213 aj0 possibly 7211 av0 speaker 7203 nn1 where 7192 cjs share 7190 nn1 carefully 7180 av0 corner 7177 nn1 plants 7176 nn2 described 7175 vvn lives 7172 nn2 told 7171 vvd-vvn length 7171 nn1 race 7170 nn1 telephone 7161 nn1 round 7161 avp-prp foot 7141 nn1 values 7136 nn2 river 7131 nn1 mary 7121 np0 possibility 7103 nn1 followed 7103 vvn drawn 7099 vvn speak 7097 vvi strength 7077 nn1 examples 7077 nn2 tried 7072 vvd mainly 7065 av0 carry 7064 vvi commission 7062 nn0 visit 7060 nn1 round 7055 prp insurance 7055 nn1 units 7053 nn2 currently 7050 av0 ball 7045 nn1 inside 7043 prp impossible 7036 aj0 somewhere 7026 av0 sitting 7021 vvg skin 7019 nn1 confidence 7002 nn1 develop 7001 vvi carried 7000 vvn wind 6985 nn1 whole 6971 aj0-nn1 legislation 6965 nn1 bodies 6963 nn2 message 6958 nn1 hold 6958 vvi enough 6958 dt0 write 6957 vvi shows 6952 vvz earlier 6948 ajc holiday 6941 nn1 spoke 6937 vvd 100 6936 crd railway 6926 nn1 stone 6924 nn1 smiled 6924 vvd sent 6915 vvn sexual 6910 aj0 edward 6907 np0 thought 6906 vvd-vvn network 6899 nn1 attack 6899 nn1 ordinary 6893 aj0 for 6893 avp-prp entirely 6892 av0 expect 6890 vvi lay 6875 vvd previously 6873 av0 tonight 6856 av0 actual 6849 aj0 cause 6845 nn1 difficulties 6844 nn2 proposals 6828 nn2 whole 6827 nn1 extremely 6814 av0 fresh 6810 aj0 scene 6804 nn1 ministers 6796 nn2 materials 6793 nn2 speed 6782 nn1 solution 6776 nn1 dark 6769 aj0 in_particular 6767 av0 article 6764 nn1 credit 6762 nn1 includes 6759 vvz changed 6757 vvn thomas 6752 np0 distance 6752 nn1 very 6747 aj0-av0 50 6745 crd banks 6743 nn2 customers 6732 nn2 include 6730 vvi items 6729 nn2 band 6722 nn1 internal 6721 aj0 suggests 6720 vvz excellent 6717 aj0 fairly 6702 av0 technical 6701 aj0 interested 6700 aj0 domestic 6695 aj0 a_bit 6687 av0 his 6678 pnp produced 6677 vvn beginning 6676 nn1 lead 6672 vvi working 6669 aj0 crime 6666 nn1 avoid 6666 vvi sources 6659 nn2 a_bit 6657 pni traffic 6655 nn1 hard 6652 av0 introduction 6634 nn1 increasingly 6634 av0 explain 6634 vvi forty 6632 crd administration 6631 nn0 animal 6629 nn1 sir 6628 nn1 sister 6607 nn1 mum 6607 nn1 step 6595 nn1 planning 6593 nn1-vvg existence 6592 nn1 executive 6592 nn0 twelve 6576 crd added 6572 vvd remains 6570 vvz steps 6561 nn2 soviet 6558 aj0 equally 6553 av0 phone 6548 nn1 records 6547 nn2 brought 6546 vvd-vvn cultural 6543 aj0 winter 6540 nn1 create 6537 vvi users 6536 nn2 remained 6535 vvd offer 6534 vvi apart_from 6534 prp becoming 6527 vvg institutions 6525 nn2 in_front_of 6523 prp rich 6522 aj0 lady 6514 nn1 male 6511 aj0 legs 6507 nn2 proper 6504 aj0 powers 6498 nn2 24 6495 crd famous 6485 aj0 religious 6476 aj0 spirit 6473 nn1 tv 6465 nn1 primary 6464 aj0 trial 6461 nn1 developed 6461 vvn relief 6452 nn1 progress 6444 nn1 straight 6442 av0 programmes 6438 nn2 cold 6438 aj0 formal 6436 aj0 sight 6432 nn1 s 6432 zz0 move 6431 nn1 coffee 6427 nn1 past 6426 prp prison 6421 nn1 follow 6421 vvi review 6420 nn1 concept 6415 nn1 watching 6410 vvg elements 6404 nn2 accident 6399 nn1 will 6392 nn1 look 6392 nn1-vvb earth 6390 nn1 reality 6386 nn1 leave 6386 vvb appear 6384 vvi hard 6383 aj0-av0 ask 6382 vvb tree 6371 nn1 strange 6371 aj0 read 6362 vvi papers 6361 nn2 unable 6359 aj0 warm 6358 aj0 surely 6352 av0 firm 6340 nn1 p 6338 zz0 difficulty 6333 nn1 latest 6331 ajs run 6327 vvn past 6326 nn1 17 6324 crd support 6316 vvi sides 6316 nn2 d 6314 zz0 assessment 6314 nn1 ahead 6309 av0 proportion 6306 nn1 dinner 6294 nn1 distribution 6293 nn1 before 6285 av0 win 6282 vvi weekend 6280 nn1 meaning 6274 nn1 half 6271 nn0 lovely 6263 aj0 wine 6260 nn1 usual 6260 aj0 worse 6258 ajc rural 6258 aj0 buildings 6258 nn2 wish 6255 vvb twice 6254 av0 applications 6254 nn2 henry 6253 np0 factor 6253 nn1 equal 6252 aj0 inc 6246 aj0 to 6245 avp path 6245 nn1 fund 6237 nn1 whereas 6236 cjs stuff 6231 nn1 nobody 6229 pni &formula 6226 unc substantial 6222 aj0 funds 6222 nn2 northern 6219 np0 reasonable 6209 aj0 onto 6208 prp learn 6204 vvi aircraft 6200 nn0 games 6197 nn2 background 6170 nn1 officials 6164 nn2 strategy 6158 nn1 works 6157 vvz prepared 6154 vvn achieve 6148 vvi soft 6145 aj0 president 6145 np0 top 6142 aj0 transport 6133 nn1 battle 6131 nn1 smile 6124 nn1 joint 6124 aj0 asked 6122 vvd-vvn contrast 6113 nn1 sort_of 6110 av0 needed 6110 vvn communication 6109 nn1 immediate 6107 aj0 grounds 6106 nn2 seat 6101 nn1 play 6100 nn1 stand 6094 vvi fourth 6094 ord paris 6091 np0 safe 6090 aj0 client 6087 nn1 everybody 6084 pni debate 6082 nn1 suitable 6077 aj0 ought 6077 vm0 future 6073 aj0 set 6067 vvi e 6064 zz0 manner 6063 nn1 return 6061 nn1 homes 6060 nn2 german 6060 aj0 prevent 6057 vvi french 6052 aj0-nn1 moving 6050 vvg measures 6047 nn2 classes 6047 nn2 freedom 6042 nn1 operations 6039 nn2 selection 6038 nn1 human 6038 aj0-nn1 while 6037 nn1 read 6033 vvb requirements 6032 nn2 pair 6032 nn0 danger 6016 nn1 walls 6014 nn2 user 6013 nn1 felt 6011 vvd-vvn compared 6001 vvn relationships 5978 nn2 edinburgh 5974 np0 due 5974 aj0 afraid 5967 aj0 japan 5966 np0 feeling 5964 nn1 attitude 5957 nn1 led 5951 vvn pound 5948 nn1 sign 5946 nn1 elections 5944 nn2 goal 5936 nn1 caused 5936 vvn techniques 5932 nn2 train 5929 nn1 sufficient 5917 aj0 claim 5917 nn1 even_though 5894 cjs sell 5885 vvi states 5884 nn2 democratic 5880 aj0 through 5874 avp match 5868 nn1 unemployment 5867 nn1 ec 5867 np0 reduce 5866 vvi capacity 5855 nn1 provide 5854 vvb achieved 5854 vvn fig. 5852 nn1 along 5852 avp-prp detailed 5847 aj0 expensive 5846 aj0 together_with 5844 prp = 5844 unc construction 5841 nn1 all_right 5841 av0 whilst 5839 cjs showing 5838 vvg managers 5837 nn2 happens 5836 vvz video 5835 nn1 employees 5833 nn2 supply 5832 nn1 increase 5828 vvi patterns 5827 nn2 present 5822 aj0-nn1 join 5820 vvi 1984 5820 crd christian 5817 aj0 purposes 5813 nn2 opportunities 5812 nn2 asking 5812 vvg totally 5807 av0 procedure 5807 nn1 mark 5807 np0 mental 5803 aj0 scientific 5799 aj0 frequently 5799 av0 writing 5798 vvg football 5796 nn1 individual 5792 aj0-nn1 weather 5790 nn1 beside 5785 prp absolutely 5782 av0 arrangements 5780 nn2 absence 5780 nn1 sentence 5779 nn1 largest 5778 ajs damage 5778 nn1 park 5777 np0 district 5777 nn1 fingers 5776 nn2 sites 5768 nn2 received 5764 vvn detail 5763 nn1 critical 5763 aj0 past 5761 aj0 below 5748 prp established 5743 vvn open 5740 vvb decided 5731 vvd dangerous 5730 aj0 minute 5727 nn1 courts 5725 nn2 college 5721 nn1 card 5721 nn1 senior 5720 aj0 exercise 5718 nn1 1983 5712 crd understanding 5711 nn1 rise 5711 nn1 educational 5710 aj0 reported 5709 vvn by 5708 avp-prp partly 5706 av0 reports 5705 nn2 familiar 5704 aj0 seriously 5699 av0 principles 5695 nn2 keeping 5693 vvg elsewhere 5684 av0 fear 5682 nn1 recognition 5681 nn1 return 5675 vvi walked 5671 vvd necessarily 5670 av0 live 5668 vvb properly 5667 av0 kept 5665 vvn 1993 5660 crd track 5659 nn1 output 5655 nn1 crisis 5653 nn1 40 5653 crd held 5652 vvd unlikely 5649 aj0 met 5648 vvn happened 5648 vvn fields 5646 nn2 right 5645 aj0-av0 1981 5643 crd correct 5641 aj0 led 5633 vvd-vvn commitment 5631 nn1 regarded 5628 vvn 21 5622 crd rock 5618 nn1 rain 5617 nn1 victory 5616 nn1 living 5615 vvg hall 5611 nn1 widely 5610 av0 carrying 5601 vvg meeting 5595 nn1-vvg efforts 5594 nn2 element 5593 nn1 silence 5589 nn1 holding 5581 vvg offer 5577 nn1 though 5576 av0 perfect 5574 aj0 target 5573 nn1 notes 5571 nn2 increasing 5564 vvg darlington 5564 np0 labour 5563 nn0 pieces 5560 nn2 colleagues 5552 nn2 bill 5551 nn1 friday 5540 np0 pp. 5530 nn2 apply 5528 vvi closely 5521 av0 rooms 5518 nn2 cell 5518 nn1 liverpool 5517 np0 expenditure 5509 nn1 birds 5500 nn2 charges 5497 nn2 telling 5480 vvg audience 5480 nn0 reaction 5478 nn1 neck 5477 nn1 thousands 5471 nn2 build 5469 vvi conversation 5466 nn1 historical 5463 aj0 seeing 5460 vvg tiny 5457 aj0 note 5457 nn1 improve 5454 vvi congress 5449 nn0 shape 5447 nn1 consideration 5447 nn1 city 5442 np0 bright 5442 aj0 appearance 5437 nn1 1979 5436 crd killed 5435 vvn conflict 5432 nn1 option 5424 nn1 profits 5422 nn2 no_doubt 5421 av0 processes 5417 nn2 board 5416 nn0 as_far_as 5414 cjs estate 5410 nn1 appeared 5407 vvd-vvn started 5404 vvd-vvn threat 5399 nn1 king 5394 np0 boat 5393 nn1 violence 5391 nn1 profit 5390 nn1 thanks 5387 nn2 upper 5382 aj0 empty 5382 aj0 rose 5376 vvd as_though 5368 cjs deal 5367 vvi turning 5363 vvg key 5363 aj0 volume 5361 nn1 feature 5360 nn1 meetings 5349 nn2 emphasis 5347 nn1 urban 5344 aj0 assembly 5344 nn1 feelings 5337 nn2 become 5336 vvb organization 5335 nn1 leg 5333 nn1 fifteen 5332 crd 22 5331 crd designed 5329 vvn ian 5327 np0 dark 5324 aj0-nn1 introduced 5322 vvn obtained 5320 vvn 19 5315 crd contribution 5313 nn1 apparent 5306 aj0 happened 5303 vvd decided 5300 vvn quiet 5295 aj0 procedures 5293 nn2 reach 5292 vvi dry 5291 aj0 martin 5290 np0 monday 5286 np0 contact 5284 nn1 described 5275 vvd-vvn arts 5275 nn2 occasion 5274 nn1 second 5271 nn1 facts 5271 nn2 requires 5270 vvz except 5270 cjs player 5269 nn1 membership 5268 nn0 younger 5264 ajc route 5256 nn1 needed 5254 vvd-vvn shook 5251 vvd payment 5250 nn1 front 5249 aj0-nn1 institute 5246 nn1 spend 5242 vvi pleasure 5241 nn1 past 5237 aj0-nn1 1980 5232 crd t 5227 zz0 neither 5227 av0 cabinet 5226 nn0 alan 5226 np0 murder 5221 nn1 deep 5218 aj0 careful 5218 aj0 bus 5218 nn1 remain 5217 vvi replied 5211 vvd literature 5210 nn1 leading 5210 vvg front 5208 nn1 walking 5207 vvg offered 5206 vvn branch 5204 nn1 heat 5196 nn1 show 5187 vvb faith 5184 nn1 hope 5183 nn1 island 5181 nn1 wait 5180 vvi dad 5178 nn1 farm 5172 nn1 set 5171 vvd-vvn aye 5166 itj liberal 5164 aj0 1982 5162 crd driver 5159 nn1 entry 5156 nn1 along 5155 avp eat 5154 vvi attractive 5152 aj0 limited 5146 aj0 call 5146 nn1 investigation 5143 nn1 planning 5137 nn1 needed 5136 vvd breath 5131 nn1 continued 5130 vvd-vvn subject_to 5128 prp fair 5127 aj0 pictures 5124 nn2 birth 5123 nn1 general 5121 np0 met 5117 vvd description 5117 nn1 agreed 5116 vvn prove 5115 vvi wood 5111 nn1 developments 5110 nn2 tend 5107 vvb send 5106 vvi projects 5103 nn2 sit 5100 vvi receive 5100 vvi claimed 5100 vvd vital 5098 aj0 spring 5097 nn1 greatest 5092 ajs accounts 5091 nn2 curriculum 5090 nn1 created 5090 vvn belief 5088 nn1 italy 5087 np0 conclusion 5087 nn1 turned 5081 vvn coal 5073 nn1 along_with 5072 prp 31 5067 crd ltd 5066 aj0 growing 5060 aj0 heads 5058 nn2 lead 5049 nn1 thin 5048 aj0 effectively 5048 av0 etc 5047 av0 notice 5040 nn1 moved 5036 vvd gentleman 5036 nn1 west 5034 nn1 irish 5034 aj0 lips 5032 nn2 hell 5032 nn1 required 5031 vvd-vvn document 5031 nn1 division 5025 nn0 newspaper 5017 nn1 copy 5016 nn1 far 5013 aj0-av0 object 5012 nn1 exhibition 5012 nn1 manchester 5011 np0 shops 5008 nn2 work 5007 vvb treaty 5007 nn1 engine 5005 nn1 usa 5004 np0 owner 5000 nn1 jack 5000 np0 tradition 4998 nn1 published 4998 vvn lunch 4998 nn1 package 4996 nn1 flowers 4992 nn2 v. 4991 prp major 4989 np0 russian 4983 aj0 standard 4982 aj0 tony 4981 np0 depends 4980 vvz 28 4975 crd elderly 4973 aj0 broke 4973 vvd australia 4964 np0 level 4962 aj0-nn1 sky 4956 nn1 company 4956 nn0 tom 4955 np0 anybody 4952 pni external 4946 aj0 green 4943 aj0-nn1 capable 4943 aj0 united 4942 aj0 streets 4941 nn2 models 4941 nn2 orders 4934 nn2 reform 4925 nn1 generation 4918 nn1 beneath 4917 prp waiting 4914 vvg organisations 4912 nn2 desire 4910 nn1 starting 4907 vvg spent 4907 vvn providing 4899 vvg wider 4898 ajc kept 4896 vvd busy 4890 aj0 ancient 4890 aj0 wonderful 4884 aj0 horses 4882 nn2 functions 4882 nn2 decide 4882 vvi place 4880 nn1-vvb broad 4879 aj0 about 4872 avp-prp treated 4869 vvn raise 4864 vvi god 4862 nn1-np0 clients 4861 nn2 agreed 4861 vvd-vvn e.g. 4860 av0 reached 4858 vvn stephen 4857 np0 iii 4853 crd begin 4845 vvi through 4844 avp-prp tour 4843 nn1 forced 4843 vvn plenty 4841 pni and_so_on 4840 av0 actions 4840 nn2 said 4837 vvd-vvn longer 4834 ajc narrow 4833 aj0 entire 4832 aj0 bag 4831 nn1 aim 4831 nn1 secondary 4829 aj0 definition 4827 nn1 acid 4826 nn1 bringing 4824 vvg atmosphere 4824 nn1 welfare 4822 nn1 doubt 4821 nn1 standing 4820 vvg moved 4820 vvd-vvn laws 4816 nn2 typical 4815 aj0 meanwhile 4813 av0 examination 4813 nn1 troops 4812 nn2 doors 4806 nn2 demands 4800 nn2 partner 4798 nn1 debt 4798 nn1 call 4798 vvb very 4796 aj0 milk 4787 nn1 middle 4783 nn1 address 4783 nn1 x 4781 zz0 dealing 4780 vvg hence 4778 av0 attitudes 4777 nn2 ibm 4775 np0 at_first 4772 av0 lose 4771 vvi 23 4771 crd reduction 4768 nn1 ended 4768 vvd-vvn criticism 4765 nn1 stories 4757 nn2 billion 4753 crd teeth 4752 nn2 explanation 4751 nn1 alright 4751 av0 included 4747 vvd-vvn centres 4745 nn2 schemes 4739 nn2 nodded 4738 vvd housing 4738 nn1-vvg visitors 4736 nn2 nineteen 4736 crd worked 4735 vvn under 4734 av0 potential 4734 aj0 as_soon_as 4734 cjs challenge 4732 nn1 governments 4731 nn2 included 4729 vvn passed 4721 vvn official 4720 aj0 goals 4719 nn2 vast 4718 aj0 india 4718 np0 creation 4711 nn1 intention 4709 nn1 master 4708 nn1 little 4705 aj0-av0 christmas 4702 np0 forest 4700 nn1 journey 4695 nn1 chosen 4695 vvn r 4694 zz0 form 4694 vvi northern 4692 aj0 bill 4690 np0 christ 4688 np0 screen 4687 nn1 readers 4687 nn2 leadership 4686 nn1 establish 4686 vvi flight 4685 nn1 tests 4681 nn2 thoughts 4676 nn2 kinds 4675 nn2 surprise 4673 nn1 hall 4673 np0 engineering 4669 nn1 farmers 4667 nn2 fall 4666 vvi significance 4663 nn1 control 4662 nn1-vvb o'clock 4656 av0 rare 4651 aj0 via 4647 prp least 4642 dt0 in_relation_to 4639 prp somewhat 4638 av0 support 4637 nn1-vvb drew 4635 vvd key 4633 aj0-nn1 biggest 4632 ajs museum 4631 nn1 sections 4628 nn2 phase 4628 nn1 terrible 4624 aj0 inside 4622 av0 read 4621 vvd metal 4619 nn1 technique 4616 nn1 motion 4616 nn1 moved 4615 vvn unfortunately 4612 av0 put 4611 vvd-vvn save 4609 vvi severe 4607 aj0 factory 4606 nn1 wild 4597 aj0 agree 4596 vvi speaking 4595 vvg suggest 4594 vvi contains 4593 vvz strongly 4590 av0 east 4590 nn1 laughed 4585 vvd v 4583 prp bloody 4580 aj0 park 4579 nn1 corporate 4579 aj0 brain 4574 nn1 i.e. 4573 av0 ooh 4565 itj court 4561 np0 sold 4560 vvn knowing 4560 vvg tall 4558 aj0 rapidly 4558 av0 protect 4555 vvi southern 4553 aj0 low 4553 aj0-av0 harry 4552 np0 amongst 4552 prp worst 4544 ajs afterwards 4544 av0 signs 4543 nn2 criminal 4541 aj0 jones 4539 np0 injury 4535 nn1 chief 4535 aj0 discuss 4531 vvi permanent 4530 aj0 appear 4529 vvb respect 4527 nn1 act 4523 nn1-vvb leeds 4522 np0 ourselves 4515 pnx desk 4513 nn1 used 4510 vvd unions 4507 nn2 shoulder 4507 nn1 brief 4506 aj0 hole 4505 nn1 break 4505 vvi fundamental 4504 aj0 similarly 4502 av0 implications 4501 nn2 associated 4498 vvd-vvn originally 4492 av0 brian 4492 np0 funny 4490 aj0 wondered 4489 vvd carried 4488 vvd-vvn noise 4486 nn1 suggested 4485 vvn somehow 4483 av0 payments 4483 nn2 double 4482 aj0 andrew 4481 np0 worked 4480 vvd grew 4479 vvd odd 4478 aj0 customer 4474 nn1 inner 4471 aj0 broken 4471 vvn cold 4467 aj0-nn1 affected 4467 vvn un 4464 np0 spain 4462 np0 crucial 4462 aj0 reached 4461 vvd machines 4461 nn2 cover 4461 vvi structures 4460 nn2 that 4459 av0 independence 4459 nn1 chris 4458 np0 bridge 4457 nn1 front 4454 aj0 seats 4453 nn2 constant 4449 aj0 combination 4449 nn1 in_addition 4445 av0 won 4441 vvn star 4441 nn1 perfectly 4439 av0 maintain 4435 vvi settlement 4431 nn1 at_last 4424 av0 justice 4423 nn1 beginning 4421 vvg consequences 4416 nn2 played 4415 vvn run 4413 nn1 lot 4409 nn1 leading 4409 aj0 session 4407 nn1 wednesday 4406 np0 add 4406 vvi finding 4405 vvg act 4403 vvi lying 4401 vvg appointment 4399 nn1 virtually 4398 av0 green 4398 aj0 started 4392 vvn doctors 4390 nn2 movements 4388 nn2 sports 4385 nn2 anne 4384 np0 yorkshire 4382 np0 placed 4381 vvn fashion 4380 nn1 temperature 4379 nn1 drugs 4379 nn2 academic 4379 aj0 26 4375 crd societies 4374 nn2 consumer 4373 nn1 nation 4372 nn1 finance 4372 nn1 initial 4371 aj0 drink 4366 nn1 search 4363 nn1 motor 4363 nn1 afford 4363 vvi accepted 4361 vvn massive 4360 aj0 pool 4358 nn1 dogs 4358 nn2 transfer 4356 nn1 offices 4356 nn2 feeling 4353 vvg youth 4352 nn1 unique 4352 aj0 steve 4352 np0 aspect 4352 nn1 sixty 4351 crd objectives 4346 nn2 interpretation 4346 nn1 author 4346 nn1 agent 4346 nn1 understand 4345 vvb vote 4344 nn1 sample 4344 nn1 brought 4344 vvd back 4344 aj0-nn1 assistance 4344 nn1 patient 4343 nn1 file 4339 nn1 directors 4339 nn2 parliamentary 4337 aj0 lights 4336 nn2 joe 4334 np0 unknown 4333 aj0 walk 4329 vvi grey 4329 aj0 badly 4329 av0 moreover 4327 av0 industries 4327 nn2 27 4321 crd suggest 4320 vvb russia 4318 np0 quick 4318 aj0 jim 4318 np0 crowd 4309 nn0 that_is 4308 av0 subsequent 4303 aj0 built 4303 vvd-vvn sport 4300 nn1 finished 4300 vvn other_than 4297 prp covered 4297 vvn iron 4294 nn1 objects 4292 nn2 meant 4285 vvd-vvn pages 4284 nn2 minor 4279 aj0 simon 4276 np0 total 4269 aj0-nn1 contemporary 4269 aj0 nose 4267 nn1 meal 4263 nn1 cause 4263 vvi ship 4258 nn1 reduced 4257 vvn practices 4257 nn2 learning 4257 vvg colours 4254 nn2 alive 4254 aj0 negative 4252 aj0 female 4252 aj0 agency 4251 nn1 involving 4250 vvg wearing 4248 vvg departments 4241 nn2 begun 4241 vvn shoulders 4240 nn2 italian 4240 aj0 yours 4239 pnp designed 4238 vvd-vvn opened 4236 vvd-vvn bedroom 4234 nn1 proceedings 4233 nn2 guilty 4233 aj0 beauty 4233 nn1 pretty 4232 av0 o 4229 zz0 late 4229 av0 angry 4226 aj0 jane 4225 np0 sequence 4223 nn1 building 4223 nn1-vvg impression 4222 nn1 called 4221 vvd union 4212 np0 tone 4212 nn1 eastern 4212 aj0 choose 4211 vvi graham 4209 np0 cities 4203 nn2 sheet 4202 nn1 regulations 4202 nn2 inflation 4201 nn1 glasgow 4200 np0 documents 4199 nn2 discussed 4199 vvn cut 4198 vvn enormous 4196 aj0 enterprise 4194 nn1 identify 4192 vvi fuel 4192 nn1 rarely 4190 av0 mike 4190 np0 employers 4190 nn2 improvement 4189 nn1 average 4187 aj0 tape 4186 nn1 ninety 4186 crd trust 4185 nn1 allowed 4183 vvd-vvn regions 4182 nn2 significantly 4179 av0 j. 4177 np0 determined 4174 vvn naturally 4172 av0 involvement 4166 nn1 increased 4164 aj0 kids 4158 nn2 tears 4157 nn2 increased 4155 vvn contracts 4153 nn2 pick 4151 vvi accommodation 4149 nn1 mine 4148 pnp roof 4147 nn1 communities 4143 nn2 face 4142 vvi quietly 4140 av0 provisions 4140 nn2 unix 4139 nn1 grass 4139 nn1 statements 4136 nn2 nations 4136 nn2 allows 4132 vvz presented 4129 vvn shock 4127 nn1 60 4127 crd in_general 4126 av0 christmas 4121 nn1 fruit 4119 nn0 unusual 4118 aj0 revolution 4118 nn1 gold 4115 nn1 teaching 4113 nn1-vvg raised 4113 vvn pension 4112 nn1 n 4109 xx0 draw 4109 vvi stages 4108 nn2 claims 4108 nn2 properties 4107 nn2 anywhere 4105 av0 mind 4104 vvi long-term 4103 aj0 code 4101 nn1 date 4098 nn1-vvb seconds 4097 nn2 measure 4096 nn1 vehicle 4092 nn1 spot 4092 nn1 pay 4092 nn1 stared 4087 vvd distinction 4086 nn1 bbc 4083 np0 release 4080 nn1 heavily 4079 av0 extensive 4078 aj0 lots 4077 pni bottle 4077 nn1 love 4074 vvb revenue 4072 nn1 h 4072 zz0 consider 4070 vvb bob 4070 np0 glad 4068 aj0 forget 4066 vvi vision 4063 nn1 start 4063 vvb friendly 4058 aj0 artist 4058 nn1 costs 4055 nn2-vvz sum 4053 nn1 plastic 4052 nn1 pass 4051 vvi highest 4051 ajs row 4050 nn1 cover 4049 nn1 breakfast 4049 nn1 artists 4047 nn2 crown 4046 nn1 housing 4045 nn1 m 4044 zz0 won 4042 vvd opened 4042 vvd continued 4042 vvd at_once 4041 av0 writing 4040 nn1-vvg proposal 4040 nn1 answer 4039 vvi association 4038 nn1 clean 4037 aj0 lost 4033 vvd-vvn location 4030 nn1 gently 4030 av0 market 4028 nn1-vvb relative 4021 aj0 cut 4021 vvi imagine 4020 vvi tried 4019 vvn slow 4016 aj0 open 4016 vvi watch 4012 vvi soviet 4009 np0 lucky 4008 aj0 29 4006 crd periods 4004 nn2 maintenance 4001 nn1 hundreds 4001 nn2 persons 3999 nn2 display 3999 nn1 establishment 3997 nn0 end 3995 vvi song 3994 nn1 jesus 3992 np0 worry 3990 vvi college 3990 np0 towns 3986 nn2 efficient 3984 aj0 encourage 3980 vvi outside 3979 av0 index 3979 nn1 decided 3979 vvd-vvn continues 3978 vvz weapons 3972 nn2 bottom 3972 nn1 passage 3970 nn1 total 3969 nn1 wilson 3966 np0 occasionally 3963 av0 philip 3961 np0 firmly 3960 av0 formation 3959 nn1 drug 3959 nn1 comfortable 3957 aj0 believes 3952 vvz identity 3950 nn1 abroad 3941 av0 countryside 3933 nn1 obtain 3931 vvi added 3929 vvn mostly 3928 av0 sudden 3924 aj0 brown 3923 np0 soil 3921 nn1 write 3920 vvb ref 3920 nn1 approval 3919 nn1 concentration 3918 nn1 autumn 3917 nn1 teams 3916 nn2 conventional 3916 aj0 criteria 3915 nn2 magazine 3912 nn1 standard 3910 nn1 brown 3908 aj0 enable 3906 vvi european 3901 aj0-nn1 enjoy 3900 vvi times 3897 np0 willing 3896 aj0 followed 3896 vvd markets 3895 nn2 voluntary 3894 aj0 cards 3893 nn2 selling 3889 vvg editor 3888 nn1 characters 3885 nn2 thinks 3884 vvz valuable 3883 aj0 followed 3881 vvd-vvn division 3880 nn1 theatre 3879 nn1 starts 3878 vvz situations 3878 nn2 short 3877 aj0-av0 program 3873 nn1 regularly 3871 av0 living 3871 aj0-vvg stars 3870 nn2 spokesman 3868 nn1 returned 3867 vvd led 3867 vvd remaining 3866 aj0 stopped 3864 vvd tried 3863 vvd-vvn household 3863 nn1 occasions 3862 nn2 appeared 3859 vvd agriculture 3858 nn1 replaced 3857 vvn duties 3852 nn2 announced 3849 vvd-vvn touch 3846 nn1 reader 3846 nn1 guide 3844 nn1 losses 3841 nn2 judge 3841 nn1 talks 3840 nn2 intended 3840 vvn error 3839 nn1 reported 3837 vvd-vvn dramatic 3837 aj0 produced 3835 vvd-vvn golden 3835 aj0 temporary 3833 aj0 arguments 3833 nn2 living 3831 aj0 close 3830 av0 african 3829 aj0 shortly 3828 av0 lies 3828 vvz read 3825 vvn decline 3825 nn1 initially 3824 av0 eighty 3823 crd firm 3820 aj0-nn1 pub 3816 nn1 framework 3816 nn1 recovery 3815 nn1 loan 3815 nn1 add 3815 vvb as_long_as 3805 cjs japanese 3801 aj0 cat 3801 nn1 rail 3800 nn1 margaret 3800 np0 chest 3799 nn1 silent 3798 aj0 require 3793 vvi control 3793 vvi theme 3792 nn1 eleven 3788 crd bring 3786 vvb video-taped 3782 aj0 remain 3778 vvb coming 3777 aj0-vvg th 3775 unc married 3774 aj0 item 3774 nn1 characteristics 3774 nn2 watched 3770 vvd yellow 3766 aj0 options 3765 nn2 keen 3762 aj0 liability 3760 nn1 clubs 3760 nn2 electricity 3758 nn1 specifically 3755 av0 light 3755 aj0 agents 3754 nn2 owners 3752 nn2 reports 3751 nn2-vvz meant 3751 vvd statutory 3748 aj0 federal 3745 aj0 failed 3745 vvd-vvn overall 3744 aj0 religion 3743 nn1 positions 3743 nn2 paid 3742 vvd-vvn deeply 3741 av0 writer 3740 nn1 wealth 3736 nn1 felt 3736 vvn coast 3736 nn1 received 3731 vvd-vvn lewis 3730 np0 1978 3730 crd height 3728 nn1 parent 3727 nn1 decade 3724 nn1 out 3723 avp-prp interview 3723 nn1 thursday 3722 np0 care 3722 nn1-vvb applied 3721 vvn publication 3720 nn1 expected 3720 aj0-vvn v 3719 zz0 rather_than 3719 cjs bread 3719 nn1 resistance 3717 nn1 ice 3716 nn1 chief 3716 aj0-nn1 cancer 3715 nn1 representatives 3714 nn2 eggs 3713 nn2 reputation 3711 nn1 cope 3710 vvi democracy 3706 nn1 bank 3706 np0 capital 3704 aj0-nn1 roads 3702 nn2 lifespan 3698 nn1 stress 3697 nn1 used 3695 aj0-vvn offence 3695 nn1 subject 3694 aj0-nn1 conservative 3694 aj0 wages 3693 nn2 recession 3693 nn1 expect 3693 vvb panel 3688 nn1 tasks 3687 nn2 stations 3687 nn2 dress 3679 nn1 subsequently 3677 av0 struggle 3677 nn1 border 3676 nn1 care 3675 vvi forgotten 3673 vvn 1977 3672 crd n 3669 zz0 paying 3668 vvg bought 3668 vvn tells 3667 vvz deputy 3666 nn1 completed 3666 vvn yards 3665 nn2 kill 3665 vvi candidate 3665 nn1 agencies 3661 nn2 alternative 3660 aj0 allowing 3660 vvg trip 3659 nn1 theories 3659 nn2 candidates 3659 nn2 now_that 3656 cjs gradually 3654 av0 grown 3652 vvn china 3652 np0 negotiations 3651 nn2 german 3651 aj0-nn1 notion 3648 nn1 lived 3648 vvd co-operation 3648 nn1 initiative 3647 nn1 acceptable 3647 aj0 competitive 3646 aj0 offering 3645 vvg cambridge 3645 np0 stairs 3644 nn2 grow 3644 vvi shoes 3642 nn2 outcome 3639 nn1 sensitive 3636 aj0 foundation 3635 nn1 caught 3633 vvn representation 3631 nn1 essentially 3631 av0 drove 3631 vvd businesses 3630 nn2 turn 3629 vvb taylor 3627 np0 easier 3624 ajc supporters 3622 nn2 belfast 3621 np0 ta 3618 to0 containing 3612 vvg assets 3611 nn2 visit 3610 vvi elizabeth 3610 np0 delivery 3609 nn1 seek 3608 vvi global 3608 aj0 announced 3608 vvn once_again 3607 av0 aside 3607 av0 chain 3606 nn1 ring 3605 nn1 g 3605 zz0 ministry 3602 nn0 fall 3602 nn1 meat 3598 nn1 sent 3594 vvd-vvn victim 3593 nn1 provided 3592 cjs expansion 3592 nn1 supported 3590 vvn flow 3590 nn1 et_al 3588 av0 dream 3588 nn1 lee 3587 np0 teaching 3583 nn1 sit 3583 vvb emotional 3583 aj0 agree 3583 vvb individual 3582 nn1 1976 3582 crd fifth 3581 ord drive 3579 nn1 identified 3576 vvn parish 3575 nn1 advance 3575 nn1 weak 3571 aj0 adequate 3571 aj0 resolution 3570 nn1 writers 3569 nn2 hill 3569 np0 use 3568 nn1-vvb tuesday 3568 np0 instructions 3568 nn2 partners 3567 nn2 closed 3566 vvd-vvn alright 3566 aj0 centuries 3563 nn2 images 3558 nn2 computers 3558 nn2 facing 3555 vvg sharp 3553 aj0 pale 3552 aj0 in_favour_of 3552 prp figure 3552 nn1-vvb acting 3551 vvg furniture 3549 nn1 bigger 3549 ajc soldiers 3548 nn2 administrative 3548 aj0 wooden 3547 aj0 efficiency 3546 nn1 healthy 3544 aj0 frank 3540 np0 block 3540 nn1 sees 3535 vvz adam 3533 np0 enter 3532 vvi luke 3531 np0 formed 3531 vvn serve 3530 vvi pocket 3530 nn1 hello 3530 itj easy 3530 aj0-av0 fallen 3528 vvn arrived 3528 vvd newspapers 3527 nn2 remarkable 3525 aj0 mistake 3525 nn1 request 3524 nn1 lost 3523 vvd seeking 3519 vvg talk 3516 nn1 scientists 3516 nn2 map 3515 nn1 used 3514 aj0 extension 3510 nn1 restaurant 3508 nn1 churches 3508 nn2 played 3507 vvd listening 3506 vvg apart 3506 av0 worker 3504 nn1 brilliant 3498 aj0 diet 3497 nn1 tired 3496 aj0 connection 3496 nn1 200 3496 crd link 3494 nn1 worked 3492 vvd-vvn licence 3492 nn1 surprising 3490 aj0 removed 3489 vvn strike 3488 nn1 eh 3488 itj precisely 3486 av0 convention 3486 nn1 content 3486 nn1 self 3484 nn1 mountain 3484 nn1 gun 3484 nn1 communications 3481 nn2 awareness 3480 nn1 absolute 3480 aj0 setting 3477 vvg kingdom 3476 nn1 bear 3475 vvi die 3473 vvi inquiry 3472 nn1 sarah 3471 np0 average 3471 aj0-nn1 white 3468 np0 involves 3468 vvz store 3466 nn1 councils 3464 nn2 regime 3461 nn1 residents 3457 nn2 agreed 3457 vvd comments 3454 nn2 falling 3449 vvg ages 3448 nn2 corporation 3447 nn0 focus 3446 nn1 arrival 3445 nn1 potential 3444 aj0-nn1 passed 3444 vvd-vvn wish 3443 vvi plane 3443 nn1 bird 3443 nn1 sad 3439 aj0 across 3439 av0 occur 3437 vvi bill 3434 nn1-np0 birmingham 3432 np0 dependent 3429 aj0 consequence 3428 nn1 fault 3427 nn1 expressed 3425 vvn in_addition_to 3424 prp electronic 3422 aj0 published 3418 vvd-vvn rapid 3417 aj0 golf 3415 nn1 comprehensive 3415 aj0 met 3414 vvd-vvn hoping 3414 vvg affect 3413 vvi experiences 3406 nn2 enemy 3402 nn0 pollution 3401 nn1 constitution 3399 nn1 copies 3397 nn2 scope 3396 nn1 database 3396 nn1 pure 3394 aj0 tim 3393 np0 bought 3393 vvd nigel 3391 np0 indian 3390 aj0 heard 3390 vvd-vvn taste 3389 nn1 minority 3389 nn1 metres 3387 nn2 literary 3385 aj0 queen 3382 nn1 1975 3381 crd looked 3376 vvn xmltv-1.4.0/authors.txt000066400000000000000000000047721500074233200151220ustar00rootroot00000000000000alewando=Adam Lewandowski allena28=Adam Allen atirc=Alberto González Rodríguez attila_nagy=Attila Nagy axis3x3=Andy Balaam b4max=Max Becker betlit=Daniel Bittel bhaak=Patric Mueller bilbo_uk=Geoff Westcott candu_sf=Chris Owen car_unlp=Christian A. Rodriguez christianw=Christian Wattengård chunkygoodness=Andre Renaud cpicton=Chris Picton crispygoth=Chris Butler dekarl=Karl Dietz dubman=Mike Dubman eborn=Eric Bus ecastelnau=Eric Castelnau eggertthor=Eggert Thorlacius epaepa=Ed Avis fgouget=Francois Gouget gawen=Bruno Tavares gtb=Gary Buhrmaster igitur=Francois Botha jskov=Jesper Skov jtoft=Jesper Toft jveldhuis=Jerry Veldhuis kgroeneveld1=Kevin Groeneveld knowledgejunkie=Nick Morrott komoriya=Takeru Komoriya lightpriest=Or Cohen ma_begaj=Ma Begaj marianok=Mariano Cosentino <4XMLTV@marianok.com.ar> mattiasholmlund=Mattias Holmlund mbdev=Balazs Molnar mihaas=Michael Haas mnbjhguyt=Davide Chiarini mtoledo=Marcelo Toledo ngarratt=Neil Garratt nielm=Niel Markwick perlundberg=Per Lundberg pingel=Søren Pingel Dalsgaard pronovic=Kenneth Pronovici ramonroca=Ramon Roca reudeudeu=Sylvain Fabre rmeden=Robert Eden staffanmalmgren=Staffan Malmgren stefanb2=Stefan Becker stesie=Stefan Siegl sunsetsystems=Rod Roark thh=Thomas Horsten va1210=Ville Ahonen yunosh=Jan Schneider zcougar=Cougar xmltv-1.4.0/choose/000077500000000000000000000000001500074233200141425ustar00rootroot00000000000000xmltv-1.4.0/choose/tv_check/000077500000000000000000000000001500074233200157305ustar00rootroot00000000000000xmltv-1.4.0/choose/tv_check/README.tv_check000066400000000000000000000074101500074233200203770ustar00rootroot00000000000000This file describes the tv_check module of XMLTV. TV-CHECK is a Perl script that reads in a file with show information and checks it against a TV guide listing, alerting you to unexpected episodes or schedule changes. Questions/Comments/Suggestions/Thanks are, of course welcome. XMLTV - users xmltv-users@lists.sourceforge.net. XMLTV - developers xmltv-devel@lists.sourceforge.net tv_check author reden@cpan.org Documentation is available in "tv_check_doc.html" Quick-start guide ( run tv-check w/o parameters for options ) ------------------------------------------------------------- The default show file name is shows.xml The default guide file name is guide.xml Windows EXE users should add "xmltv.exe" to the beginning of each tv_ command 1. set timezone if not set ( this should not be necessary if using the EXE version) set TZ=CST6CDT 2. download listings using any XMLTV grabber, for example using tv_grab_fi for Finnish listings: tv_grab_fi --configure tv_grab_fi --output guide.xml 3. use tv_check to create a "show file" tv_check --configure Pick some shows to track, Click Add to add to the list. exit tv_check 4. use tv_check to scan the guide tv_check --scan --html --out=a.html Sample shows.xml file ======================= en_US Parameters ==================== lang : preferred language. Optional, can have multiple, in order of preferences shows : list of shows you're interested in Show attributes: ==================== day : day of the week and Sun,Mon,Tue,Wed,Thu,Fri,Sat channel: channelID used by the guide title : *EXACT* title of show. hhmm : start time of show (hhmm) len : expected length of show in minutes ( this may become optional ) device : device that records the show. Reports recording conflicts. : If the device contains "replay" takes into account ReplayTV's fuzzy recording logic Optional attributes: ==================== chanonly: if 1, episode scan ignores shows on other channels dayonly : if 1, episode scan ignores shows on other days timeonly: if 1, episode scan ignores shows at other times neartime: if 1, episode scan only reports shows within a 3 hour window Notes: ---------------------- "No Guide Info" means no guide information was found for that channel at that time. something may be wrong with the channel ID,listing fetch, or the guide file is just old. A "--html" option is available that outputs HTML output with color flagging. After processing shows, a check is made for recording conflicts. Only shows with the same "device" are checked for conflicts. VCR's probably would never give a conflict, but ReplayTV tries to find your show if it moves. If the device contains the string "REPLAY" (case insensitive) during the title scan, a recording is expected if the show within 1 timeslot of the requested time. This mimic's ReplayTV's recording logic and detects conflicts due to a show moving. Episodes before today and more than a week old are excluded from the title scan. Known bugs: --------------------- If you select a title in the title selection list and then try and select a blank day or channel the field doesn't blank. I have no clue. The variable doesn't match the displayed value... looks like a TK bug to me. Sometimes the GUI doesn't take focus at startup. Not sure why, and what can be done. if it shows up minimized, just click to bring it up. When using the "compiled" Windows EXE code on windows 98, the command window minimizes. No idea why. Seems to work fine on Win2k. xmltv-1.4.0/choose/tv_check/tv_check000077500000000000000000001601321500074233200174470ustar00rootroot00000000000000#!/usr/bin/perl -w # # tv_check # # This script searches a channel GUIDE for shows in a show list and alerts when # a listed show is missing from its time slot, or shows up at other days or times. # # The show list is a custom XML format. # The channel guide needs to be in XMLTV format. # # for details, see Usage below # # (C)2001 - Robert Eden, free to use under the GNU License. # # Robert Eden - reden@cpan.org # # See cvs logs entries for module history # # =pod =head1 NAME tv_check - Check TV guide listings =head1 SYNOPSIS tv_check --configure|--scan [other options] =head1 DESCRIPTIONS tv_check is a Perl script that reads in a file with show information and checks it against a TV guide listing, reporting on upcoming episodes and alerting you to unexpected episodes or schedule changes. =head1 OPTIONS B<--configure> Run configuration GUI. Either this option or --scan must be provided. B<--season-reset> special --configure option to remove everything but the title to help new season setup. The idea is to keep everything a "title-only" search until seasons begin. Then you update the details including record device. *expirimental* B<--scan> Scan TV listings. Either this option or --configure must be provided. B<--myreplaytv=UNIT,USERNAME,PASSWORD> ** Feature removed ** This option used to auto-populate a config file based on myreplaytv.com. B<--shows=FILE> Specify the name of XML shows file (default: shows.xml). B<--guide=FILE>, B<--listings=FILE> Specify the name of XML guide file (default: guide.xml). B<--html> Generate output in HTML format. B<--bluenew> Highlights new episodes in blue (helpful back when there was an off-season) B<--output=FILE> Write to FILE rather than standard output B<--help> Provide a usage/help listing. =head1 SEE ALSO L. =head1 AUTHOR Robert Eden; manpage by Kenneth J. Pronovici. =cut use strict; use XMLTV qw(best_name); use XMLTV::Version "$XMLTV::VERSION"; use Tk; use Tk::TableMatrix; use XML::Twig; use Date::Manip; use Time::Local; use Data::Dumper; use Getopt::Long; ## use HTTP::Cookies; ## use HTTP::Request::Common qw(POST GET); ## use LWP::UserAgent; use XMLTV::Date; use XMLTV::Usage ' tv_check "$XMLTV::VERSION" ' . < xml files with show info (default shows.xml ) --listings xml files with guide info (default guide.xml ) --configure run configuration GUI instead of checking listings --html scan output is in HTML format --ddmm prints DDMM date instead of MMDD in reports --days n process n days (default 7) --notruncate don't exclude episodes before today in extra-episode scans don't exclude episodes after '--days' days in extra-episode scans --season-reset special --configure option to remove everything but the title to help new season setup. The idea is to keep everything a "title-only" search until its season begins, then add the details including recording device. *experimental* END ; # # Define constants # select STDERR; $|=1; select STDOUT; $|=1; $ENV{TZ}='UTC' unless exists $ENV{TZ}; my @WEEKDAY = qw (Sun Mon Tue Wed Thu Fri Sat); my $WEEKDAY = "SunMonTueWedThuFriSat "; my $R_ON = ""; # used for HTML output my $G_ON = ""; # used for HTML output my $B_ON = ""; my $N_ON = ""; my $OFF = ""; # COL_TYPE 1:List 2:Entry 3:checkbox my @COL = qw(device day channel hhmm len title chanonly dayonly timeonly neartime ); my %COL; $COL{$COL[$_]}=$_ foreach (0..$#COL); # populate $COL reverse hash my @COL_TYPE = qw(1 1 1 2 2 1 3 3 3 3 ); my $CONFIGURE= 0; my $HTML = 0; my $DDMM = 0; my $DAYS = 7; my $NOTRUNCATE = 0; my $BLUENEW = 0; my $SEASON_RESET =0; my $GUIDE_XML= 'guide.xml'; my $SHOW_XML = 'shows.xml'; my $OUTPUT_FILE = undef; my $TODAY = $WEEKDAY[(localtime())[6]]; (my $TODAY_MMDD)= UnixDate( "Now", "%Y%m%d"); (my $WEEK_MMDD) = UnixDate( "$DAYS days later", "%Y%m%d"); (my $TWOM_MMDD) = UnixDate( "2 months ago", "%Y%m%d"); # # Global Vars/Databases # my @SHOWS = (); # raw show data my $SHOW_TABLE = ""; # stores pointer to SHOW_TABLE my @SHOW_DATA = (); # pointer to raw by SHOW_TABLE row my %SHOW_DATA = (); # data for SHOW_TABLE my %SHOW_WIDTH = (); # column widths for SHOW_TABLE my %SHOW_TIME; # order of shows for report my %OLD_SHOW; # {old_title}=[show entryies] my %MIDNIGHTS = (); # {day}[] Holds midnights for each future day of the week my @MYREPLAY_LIST = (); my $MYREPLAY_UNIT = ""; # parameters for MYREPLAY fetch my $MYREPLAY_USER = ""; my $MYREPLAY_PASS = ""; my $MYREPLAY_NONG = ""; my $MYREPLAY_DEBUG = ""; # 0=ignore, 1=save to replay.html, 2=load from replay.html my $SHOW_CHANGED = 0; # updd if show needs to be saved my $SHOW_SORT = $COL{title}; # column to sort SHOW_TABLE my $SHOW_ROW = 0; # last selected row # # Episode data is comes from XMLTV, but data is added to the hash # for our own use. Since we never write out the Episode XLM, this is ok. # The following non XMLTV fields are used # {prev} = pointer to previous episode on channel # {next} = pointer to next episode on channel # {device} = device that will record this episode # {hhmm} = start time ( computed on demand or if $CONFIGURE) # {day} = start day ( computed on demand or if $CONFIGURE) # {mmdd} = start date ( computed on demand or if $CONFIGURE) # {len } = episode length ( computed on demand or if $CONFIGURE) my @GUIDE = (); # episode list my %GUIDE = (); # episode indexes # # Episode Indexes ( CAPS are constants ) # # $GUIDE{ALL}{title}=[ep...] # $GUIDE{chan}{binstart}=$ep # $GUIDE{starts}{chan}=[all-start-times]; # # The following indexes are only used by configure mode # array=[day,channel,hhmm,len] # $GUIDE{TITLE}{title} =[ [day,chan,hhmm,len]...] # $GUIDE{CHAN}{chan}{title}=[ [day,chan,hhmm,len]...] # $GUIDE{DAY}{day}{title} =[ [day,chan,hhmm,len]...] # $GUIDE{day}{chan}{title} =[ [day,chan,hhmm,len]...] This works since day!=chan. I hope :) # my $ENCODING; # character encoding for listings data my @CHAN = (); # channel list (sorted) my %CHAN = (); # channel list ( channel-id key ) my %CHAN_NAME = (); # channel list ( display-name key ) my %SELECT = (); # array of selector widgits my %RECORD = (); # hash of shows to record (conflict check) my %DEVICE = (); # list of recording devices ( hash to avoid dupes ) my $ADD_BUTTON; my $DELETE_BUTTON; my $UPDATE_BUTTON; my $CLEAR_BUTTON; my $TOP; my @LANG = (); # preferred languages my @COL_VALUE=(); $COL_VALUE[$_] = "" foreach (0..$#COL); # # Step 1, Parse Parameters ------------------------------------------------------- # # First lets check to see if someone asked for help. # this is easier to do here than later. { my $scan=0; my $help=0; my $myreplayargs; GetOptions('configure' => \$CONFIGURE, 'scan' => \$scan, 'myreplaytv=s' => \@MYREPLAY_LIST, 'html' => \$HTML, 'shows=s' => \$SHOW_XML, 'output=s' => \$OUTPUT_FILE, 'guide|listings=s' => \$GUIDE_XML, 'ddmm' => \$DDMM, 'days=i' => \$DAYS, 'notruncate' => \$NOTRUNCATE, 'bluenew' => \$BLUENEW, 'season-reset' => \$SEASON_RESET, 'help' => \$help) or usage(); usage(1) if $help; die "Please select either --scan, --configure, or --help\n" if ($CONFIGURE+$scan != 1); if (defined $OUTPUT_FILE) { print STDERR "Sending output to $OUTPUT_FILE\n"; open(STDOUT,">$OUTPUT_FILE") or die "Can't open for output $OUTPUT_FILE\n"; } foreach (@MYREPLAY_LIST) { ($MYREPLAY_UNIT,$MYREPLAY_USER,$MYREPLAY_PASS,$MYREPLAY_NONG,$MYREPLAY_DEBUG)=split(/,/,$_); die "MYREPLAY UNIT not specified\n" unless length($MYREPLAY_UNIT)>0; die "MYREPLAY USER not specified\n" unless length($MYREPLAY_USER)>0; die "MYREPLAY PASS not specified\n" unless length($MYREPLAY_PASS)>0; } } # get params load_guide($GUIDE_XML); load_shows($SHOW_XML); ### ---------------------------------------- ### do we need to get shows from MYREPLAYTV? ### ### disabled, since myreplaytv.com doesn't exist any more! ### ### ##if (@MYREPLAY_LIST) { ## print STDERR "**WARNING** Replay has discontinued the MyReplayTV service. Ignoring -myreplay\n"; ##} ### foreach (@MYREPLAY_LIST) { ##if (0) { ## $MYREPLAY_UNIT=$MYREPLAY_USER=$MYREPLAY_PASS=$MYREPLAY_NONG=$MYREPLAY_DEBUG=undef; ## ($MYREPLAY_UNIT,$MYREPLAY_USER,$MYREPLAY_PASS,$MYREPLAY_NONG,$MYREPLAY_DEBUG)=split(/,/,$_); ## $MYREPLAY_NONG=0 unless defined $MYREPLAY_NONG; ## $MYREPLAY_DEBUG=0 unless defined $MYREPLAY_DEBUG; ## ## my $html=""; ## my $device="MyReplayTV$MYREPLAY_UNIT"; ## ### ### remove existing MYREPLAY_UNIT entries (they will be loaded fresh later) ### ## for my $show (@SHOWS) ## { ## if (defined $MYREPLAY_UNIT and $show->{device} eq "MyReplayTV$MYREPLAY_UNIT") ## { ## push @{$OLD_SHOW{$show->{title}}},$show; # quick hack to save previous options ## $show->{title}=''; ## } ## } ## ## print STDERR "Fetching shows from $device\n"; ## ##if ($MYREPLAY_DEBUG != 2) ##{ ### ### create user agent ### ## my $ua = LWP::UserAgent->new; ## $ua->cookie_jar( HTTP::Cookies->new); ## $ua->agent("tv_check/1.0" . $ua->agent); ## ### ### login to MyReplayTV ### ### print STDERR "MyReplayTV logging in\n"; ## my $res = $ua->request(POST 'http://my.replaytv.com/servlet/Login', ## [ username => $MYREPLAY_USER, ## password => $MYREPLAY_PASS, ## savePassword => '', ## ]); ## ## unless ( $res->is_success && $res->title eq 'ReplayGuideRecordings' ) ## { ## open(FILE,">error.html") && print(FILE $res -> as_string); ## die "MyReplayTV login error!. Debug info in 'error.html'\n"; ## } ## ### ### get MyReplayTV show info ### ## sleep 5; ### print STDERR "MyReplayTV getting Replay Channels\n"; ## $res = $ua->request( GET('http://my.replaytv.com/servlet/ReplayGuideRequests', ## HTTP::Headers->new( ## Referer => 'http://my.replaytv.com/servlet/ReplayGuideRecordings' ## ))); ## ## unless ($res->is_success && $res->title eq 'Replay Guide Shows') ## { ## open(FILE,">error.html") && print(FILE $res -> as_string); ## die "MyReplayTV show fetch error. Debug info in 'error.html'\n"; ## } ## ### ### debug save (to make things faster and not overload Replay's servers during debug) ### ## if ($MYREPLAY_DEBUG == 1) ## { ## open(FILE,">replay_$MYREPLAY_UNIT.html"); ## print FILE $res -> as_string; ## close FILE; ## } ## $html=$res->as_string; ##} ##else ##{ ## open(FILE,"); ## close FILE; ##} # quick debug hack ## ### ### Got the listings... find our shows ### ##foreach (split(/\n/,$html)) ##{ ## s/\s+/ /g; ## next unless length($_)>5; ## next if /was scheduled to record/; ## next if /Nothing else is scheduled to record/; ## ## if (my @a= / This show.+current episode.s. of (.+) occurring every \((.+)\) on Channel (\d+)\((.+)\).+ (\d+):(\d+)(\w). - (\d+):(\d+)(\w).+\. (.+) at /) ## { ## ## $a[4] = "0" if ($a[4]==12 and $a[6] eq 'A'); # midnight -> 00; ## $a[7] = "0" if ($a[7]==12 and $a[9] eq 'A'); # midnight -> 00; ## ## my $title = $a[0]; $title =~ s/\x92/'/g; # fix illegal character in Replay Feed ' ## my $days = $a[1]; ## my $chan = "$a[2] $a[3]"; ## my $hhmm = sprintf("%02d%02d",(($a[6] eq 'P') && ($a[4] != 12) ? $a[4]+12 : $a[4]),$a[5]); ## my $stop = sprintf("%02d%02d",(($a[9] eq 'P') && ($a[7] != 12) ? $a[7]+12 : $a[7]),$a[8]); ## my $guar = ( $a[10] =~ /^Not/ ? 0 : 1 ); ## ## next unless $guar || $MYREPLAY_NONG; ## ## my $len = hhmm_min($stop) - hhmm_min($hhmm); ## $len += 24*60 if $len < 0; ## ## ##print STDERR "\nMyReplay looking for ",join("|",$title,$chan,$hhmm,$len,$days),"\n" if ($MYREPLAY_DEBUG == 2); ## ### ### convert channel ID to new format if ncessary ### ## if ( ! exists $CHAN{$chan} && exists $CHAN_NAME{$chan} ) ## { ## $chan=$CHAN_NAME{$chan}; ## } ## ### ### Check Channel ### ## unless ( exists $CHAN{$chan}) ## { ## print STDERR "MyReplayTV Channel '$chan' not in guide\n"; ## $CHAN{$chan}{'display-name'}[0][0]=$chan; ## } ## ### ### if Replay expects our show on a specific day, we can just add it ### ## if (length($days) == 3) ## { ## add_myreplaytv_show($title,$chan,$hhmm,$len,$days); ## next; ## } ## ### ### Now this gets tricky. MyReplayTV tells us the time of a show, but not ### the day. We can't assume the show is available for all days listed ### because that would cause too many false alarms in tv_check ### ### We can't use any day the show is on because of syndication. A 2am ### Daily showing of a weekly show would also cause false alarms. ### ### So, the solution is to find the episode 2 slots back and 2 slots forward. ### If the MyReplay hhmm start time is between these values, record the day. ### ### This will cause problems around midnight. I don't have a good solution there ### ### Personally, I now set all shows to record on a single day on the Replay, and ### if you specify a single day, this check isn't done... there's you're work-around! ### ## my $found=""; ## for my $ep (@{$GUIDE{all}{lc($title)}}) ## { ## gen_episode_dates($ep) unless $ep->{day}; ## my $day = $ep->{day}; ## ## next if $chan ne $ep->{channel}; ## next if $days !~ /$day/; # episode on of myreplay's days? ## next if $found =~ /:$day/; # already got this day? ## ### ### get start time 2 slots back ### ## my ($ep1,$ep2,$wstart,$wstop); ## $ep1= $ep; ## $ep1 =$ep1->{prev} if $ep1->{prev}; ## gen_episode_dates($ep1) unless $ep1->{day}; ## ## $ep2= $ep1; ## $ep2 =$ep2->{prev} if $ep2->{prev}; ## gen_episode_dates($ep2) unless $ep2->{day}; ## ## $wstart=$ep ->{hhmm}; ## $wstart=$ep1->{hhmm} if $ep1->{day} eq $day; ## $wstart=$ep2->{hhmm} if $ep2->{day} eq $day; ## ## ### ### Now start time 2 slots forward ### ## $ep1= $ep; ## $ep1 =$ep1->{next} if $ep1->{next}; ## gen_episode_dates($ep1) unless $ep1->{day}; ## ## $ep2= $ep1; ## $ep2 =$ep2->{next} if $ep2->{next}; ## gen_episode_dates($ep2) unless $ep2->{day}; ## ## $wstop=$ep ->{hhmm}; ## $wstop=$ep1->{hhmm} if $ep1->{day} eq $day; ## $wstop=$ep2->{hhmm} if $ep2->{day} eq $day; ## ## ##printf STDERR "day search: %s: %s<%s<%s\n",$title,$wstart,$hhmm,$wstop if $MYREPLAY_DEBUG > 1; ## ### ### record the day if MyReplay start time is between these times ### ## next if $hhmm lt $wstart; ## next if $hhmm gt $wstop; ## ### ### guess it's a hit... mark it ### ## add_myreplaytv_show($title,$chan,$hhmm,$len,$day); ### ### not sure why we're marking this here. It prevents display when a show moves! ### ### $ep->{device} = $device; ## $found .= ":$day"; ## ## } # myreplay day search ## ### ### add it as an unknown if not found ### ## unless ($found) ## { ## $days="*" if $days eq "Sun, Mon, Tue, Wed, Thu, Fri, Sat"; ## ## unless (add_myreplaytv_show($title,$chan,$hhmm,$len,"")) ## { ## print STDERR " Can't guess day, using title scan for ",join("|",$title,$chan,$hhmm,$days),"\n"; ## } ## } ## } # show entry match ##} # listing loop ## ##load_show_table(); # build indexes ##} # MYREPLAY # # is it time to CONFIGURE? -------------------------------------------------------- # if ($CONFIGURE) { if ($SEASON_RESET) { # season-reset is an experiemtnal way to reset for a new season for my $show (@SHOWS) { for my $key (keys %$show) { next if $key eq 'title'; next if $key eq 'channel'; delete $show->{$key}; } #key loop } # show loop load_show_table(); # build indexes } #SEASON-RESET # # create main window! # $TOP = MainWindow->new; $TOP->focusmodel("active"); # # configure menu bar # { my $menubar = $TOP->Menu(-type => 'menubar'); $TOP->OnDestroy( sub{ return if changed_check(1); $TOP -> destroy(); } ); $TOP->configure(-menu => $menubar ); my $f = $menubar->cascade(-label => '~File', -tearoff => 0); $f->command(-label => 'New', -underline => 0, -command => sub { $SHOW_XML=''; @SHOWS=(); load_show_table(); }); $f->command(-label => 'Open...', -underline => 0, -command => sub { return if changed_check(); my $file = $TOP->getOpenFile(-filetypes => [["XML Files",".xml"]], -title => 'Open Show File'); load_shows($file) if defined $file; }); $f->command(-label => 'Save', -underline => 0, -command => \&Save_shows ); $f->command(-label => 'Save As...', -underline => 5, -command => sub { my $file = $TOP->getSaveFile( -filetypes => [["XML Files",".xml"]], -title => 'Save show file'); if (defined $file) { $SHOW_XML=$file; Save_shows(); } }); $f->command(-label => 'Listings...', -underline => 0, -command => sub { my $file = $TOP->getOpenFile(-filetypes => [["XML Files",".xml"]], -title => 'Open Listing File' ); load_guide($file) if defined ($file); }); $f->command(-label => 'Exit', -underline => 1, -command => sub { return if changed_check(); $TOP -> destroy(); }); my $h = $menubar->cascade(-label => '~Help', -tearoff => 0); $h->command(-label => 'Help', -underline => 0, -command => \&help_popup ); $h->command(-label => 'About', -underline => 0, -command => \&help_about ); } # menu bar # # create show table # $SHOW_TABLE = $TOP->Scrolled('TableMatrix', -cols => ($#COL+1), -rows => ($#SHOWS > 8 ? $#SHOWS+2 : 10 ), -height => 10, -titlerows => 1, -variable => \%SHOW_DATA, -roworigin => 0, -colorigin => 0, -colstretchmode => 'all', -selecttype => 'row', -sparsearray => 1, -state => 'disabled', -anchor => 'w', -exportselection => 0, ); $SHOW_TABLE->colWidth( %SHOW_WIDTH ); $SHOW_TABLE->pack(-expand => 1, -fill => 'both'); $SHOW_TABLE->bind('<1>', sub { my $w = shift; my $Ev = $w->XEvent; my $row = $w->index('@'.$Ev->x.",".$Ev->y,"row"); my $col = $w->index('@'.$Ev->x.",".$Ev->y,"col"); $w->selectionClear('all'); $SHOW_ROW=0; $UPDATE_BUTTON -> configure ( -state => "disabled" ); $DELETE_BUTTON -> configure ( -state => "disabled" ); if ($row) { return unless $SHOW_DATA{"$row,$COL{title}"}; # title must exist $SHOW_ROW=$row; $UPDATE_BUTTON -> configure ( -state => "normal" ); $DELETE_BUTTON -> configure ( -state => "normal" ); $w->selectionSet("$row,0","$row,".($#COL+1)); for $col (0..$#COL) # load selection pane { $COL_VALUE[$col] = $SHOW_DATA{"$row,$col"}; } } else { $SHOW_SORT = ($SHOW_SORT == $col ? -$col : $col); load_show_table(); } }); # show table click bind my $selframe = $TOP->Frame->pack(-side => 'bottom'); # # Control Buttons # { my $frame=$selframe->Frame()->pack( -side => 'left' ); $CLEAR_BUTTON = $frame->Button( -text => "Clear Selection", -command => sub{ $SHOW_ROW=0; $SHOW_TABLE->selectionClear('all'); $UPDATE_BUTTON -> configure ( -state => "disabled" ); $DELETE_BUTTON -> configure ( -state => "disabled" ); $COL_VALUE[$_]='' foreach (0..$#COL); load_selection_items(); }) -> pack(-fill => 'x'); $ADD_BUTTON = $frame->Button( -text => "Add Selection", -command => sub{ $SHOW_ROW=0; $SHOW_TABLE->selectionClear('all'); $UPDATE_BUTTON -> configure ( -state => "disabled" ); $DELETE_BUTTON -> configure ( -state => "disabled" ); return unless $COL_VALUE[$COL{title}]; my $row = $#SHOWS+1; validate_col_value(); $SHOWS[$row]{$COL[$_]}=$COL_VALUE[$_] foreach (0..$#COL); load_show_table(); $SHOW_CHANGED=1; $COL_VALUE[$COL{title}]=''; }) -> pack(-fill => 'x'); $UPDATE_BUTTON = $frame->Button( -text => "Update Show", -state => "disabled", -command => sub{ return unless $SHOW_ROW; return unless $COL_VALUE[$COL{title}]; validate_col_value(); $SHOW_DATA[$SHOW_ROW]->{$COL[$_]}=$COL_VALUE[$_] foreach (0..$#COL); $SHOW_CHANGED=1; load_show_table(); }) -> pack(-fill => 'x'); $DELETE_BUTTON = $frame->Button( -text => "Delete Show", -state => "disabled", -command => sub{ return unless $SHOW_ROW; $SHOW_DATA[$SHOW_ROW]{title}=''; load_show_table(); $SHOW_CHANGED=1; }) -> pack(-fill => 'x'); } # control buttons # # Selector Widgets # Type 1 ( listbox ) # for my $col (0..$#COL) { next unless $COL_TYPE[$col] == 1; my $frame =$selframe->Frame()->pack( -side => 'left' ); my $label =$frame->Label(-text => $COL[$col])->pack(); my $entry =$frame->Entry(-textvariable => \$COL_VALUE[$col])->pack(); my $list =$frame->Scrolled('Listbox', -setgrid => 1, -height =>12, -selectmode => 'row', -exportselection => 0, -scrollbars => 'w'); $list -> {SubWidget} -> {scrolled} -> privateData('Entry') -> {Entry} = $entry; $list -> {SubWidget} -> {scrolled} -> privateData('Entry') -> {Col} = $col; $list -> pack(qw/-side left -expand yes -fill both/); $list -> bind('' => sub { my $w = shift; my $entry = $w->privateData('Entry') -> {Entry}; my $col = $w->privateData('Entry') -> {Col}; my $val = $w->get('active'); #print STDERR "Storing ($val) into $col\n"; $COL_VALUE[$col]=$val; load_selection_items(); }); $SELECT{$COL[$col]}= { frame => $frame, label => $label, entry => $entry, list => $list }; } # type 1 selectors # # Selector Widgets # Type 2 ( entry ) # Note: Type 2 and Type 3 share a frame # my $selframe2 =$selframe->Frame()->pack( -side => 'left' ); for my $col (0..$#COL) { next unless $COL_TYPE[$col] == 2; my $frame = $selframe2; my $label =$frame->Label(-text => $COL[$col])->pack(); my $entry =$frame->Entry(-textvariable => \$COL_VALUE[$col])->pack(); $frame->Label(-text => " ")->pack(); $SELECT{$COL[$col]}= { frame => $frame, label => $label, entry => $entry, }; } # type 2 selectors # # Selector Widgets # Type 3 ( checkbox ) # Note: Type 2 and Type 3 share a frame # for my $col (0..$#COL) { next unless $COL_TYPE[$col] == 3; my $frame = $selframe2; my $check = $frame->Checkbutton( -text => $COL[$col], -variable => \$COL_VALUE[$col], ) -> pack(); $SELECT{$COL[$col]}= { frame => $frame, check => $check, }; } # type 3 selectors load_selection_items(); # # let the games begin! # print STDERR "GUI running\n"; Tk::MainLoop; } # CONFIGURE # # Step 3, do an actual tv check -------------------------------------------------------- # else { # # Print HTML Banner # if ($HTML) { $R_ON = ""; $G_ON = ""; $B_ON = ""; $N_ON = ""; $OFF = ""; my $now = localtime(); # Make the output in the same encoding as the programme data. We # assume this is a superset of ASCII. # print < TV-CHECK report

TV-CHECK

$now | $SHOW_XML | $GUIDE_XML 

END
;}


#
# Build list of midnight bintimes
#
{
   my $noon=timelocal(0,0,12,substr($TODAY_MMDD,6,2),substr($TODAY_MMDD,4,2)-1,substr($TODAY_MMDD,0,4)-1900);
   foreach (0..($DAYS-1))
   {
      my $day=$WEEKDAY[(localtime($noon))[6]];
      my $midnight=$noon - 12*3600;   # by using this midnight, DST day show times will be off from 0-2am. oh well.
      unshift @{$MIDNIGHTS{$day}},$midnight;

      printf "WARNING: DST change detected on $day\n" if ((localtime($midnight))[2] != 0);
      $noon=timelocal(0,0,12,(localtime($noon+24*3600))[3,4,5]);

   }
}

#
# Build show_time index
#
print STDERR "Computing show time index\n";
my $unique=1;
for my $show (@SHOW_DATA)
{
    $show->{channel}="" unless exists $show->{channel};
    $show->{day}=""     unless exists $show->{day};

    if (exists $MIDNIGHTS{$show->{day}})  # deal with shows on a specific day
    {
        my $time_of_day=substr($show->{hhmm},0,2)*3600+substr($show->{hhmm},2,2)*60;

        for my $midnight (@{$MIDNIGHTS{$show->{day}}})
        {
             $show->{start} = $midnight + $time_of_day;
             my @date       = localtime($show->{start});
                              $date[4]++; $date[5]+=1900;
             $show->{mmdd}  = sprintf("%04d%02d%02d",@date[5,4,3]);

             if (exists $SHOW_TIME{$show->{start}}
             and exists $SHOW_TIME{$show->{start}}{$show->{channel}.$show->{title}} ) {
                     $show->{dupe}=1; # start day,time,title matches.. mark dupe
                     $SHOW_TIME{$show->{start}}{$show->{channel}.$show->{title}.($unique++)} = {%$show};
             }
             else { $SHOW_TIME{$show->{start}}{$show->{channel}.$show->{title}} = {%$show}; }
        }
     }
     else
     {
        $show->{mmdd} = "";
        $show->{day}  = "";
        $SHOW_TIME{"Z".($unique++)}{$show->{channel}} = $show;
     }

} #build SHOW_TIME index

#
# let the games begin... process shows!
#
print STDERR "Processing shows\n\n";
for my $start (sort keys %SHOW_TIME)
{
    for my $key (sort keys %{$SHOW_TIME{$start}})
    {
        my $show = $SHOW_TIME{$start}{$key};
        my $chan = $show->{channel};
        my $ep_desc = "";
  	    next unless $show->{title};

        $CHAN{$chan}{'display-name'}[0][0]=$chan unless exists $CHAN{$chan};

#
# See what episode is on at that time
#
    if ( $show -> {mmdd} ) # this phase only gets shows with a mmdd
    {
        my $ep = find_episode($show);

#
# look for close episode matches
#
        $ep=$ep->{prev} if ($ep && $ep->{prev}
                                && !($ep->{prev}->{displayed})  # don't flag shows already hit
                                && lc(get_text($ep->{title}      )) ne lc($show->{title})
                                && lc(get_text($ep->{prev}{title})) eq lc($show->{title}));

        $ep=$ep->{next} if ($ep && $ep->{next}
                                && !($ep->{next}->{displayed})  # don't flag shows already hit
                                && lc(get_text($ep->{title}      )) ne lc($show->{title})
                                && lc(get_text($ep->{next}{title})) eq lc($show->{title}));
#
# display results
#
        if (!defined $ep)
        {
           printf "${R_ON}%-60s **** NO GUIDE DATA ****${OFF}\n",sh_summary($show);
        }
        elsif ( lc(get_text($ep->{title})) ne lc($show->{title}) )
        {
           printf "${R_ON}%-50s **** wrong show in slot ****\n",sh_summary($show);
           print " "x10,ep_summary($ep),"${OFF}\n";
        }
        else # ( guess we got what we wanted )
        {
            if (length($show->{device})
                && ! $ep->{displayed}  )# don't flag shows already hit)
            {
                push @{$RECORD{$show->{device}}},$ep;
                $ep->{device}=$show->{device};
            }

            $ep->{displayed}=$show;
            print $B_ON if $BLUENEW && !$ep->{"previously-shown"};
            print ep_summary($ep),opt_summary($show),"\n";
            print $OFF  if $BLUENEW && !$ep->{"previously-shown"};
            if ( $show->{hhmm} ne $ep->{hhmm} )
            {
                print "${R_ON}     ***** Start Time Alert ***** Expected $show->{hhmm} got $ep->{hhmm}${OFF}\n";
            }
            if ( $show->{len} && $ep->{len} && $show->{len} ne $ep->{len} )
            {
                print "${R_ON}     ***** LENGTH ALERT ***** Expected $show->{len} got $ep->{len}${OFF}\n";
            }
            $ep_desc = get_text($ep ->{"sub-title"}); # use this later
        }
    }
    else
    {
       print sh_summary($show)."\n";
    }

#
# See if the show is on at other times
#
    for my $ep ( @{$GUIDE{all}{lc($show->{title})}})
    {
        gen_episode_dates($ep)    unless $ep->{day};
        next if !$NOTRUNCATE && $ep->{mmdd} lt $TODAY_MMDD;  # ignore shows before today
        next if !$NOTRUNCATE && $ep->{mmdd} ge $WEEK_MMDD ;  # ignore shows more than a week away
        next if $ep->{displayed} eq $show;
        next if length($ep->{device}) >0 && ($ep->{device} eq $show->{device}); #skip if already recording

        gen_episode_dates($ep) unless $ep->{day};


# check channel
#
        next if ( $show->{chanonly} && $chan ne $ep->{channel} );


#
# check day
#
        next if ( $show->{dayonly}  && $show->{day} ne $ep->{day});

#
# check time
#
        next if ( $show->{timeonly} && $show->{hhmm} ne $ep->{hhmm});
        if ( $show -> {neartime})
        {
            my $delta = abs( substr($show->{hhmm},0,2) -
                             substr(  $ep->{hhmm},0,2) );
            next unless $delta < 2;
        }

#
# ok, guess we're interested in it, print it
#
#   highlight new bonus episodes in green, otherwise gray
#
        my $tmp=get_text($ep ->{"sub-title"}) || "";
        if ( $ep_desc && $tmp &&
            $ep_desc ne $tmp  &&
            !$ep->{"previously-shown"} )
        {
            print " "x5,$N_ON,ep_summary($ep,1),"$OFF\n";
        }
        else
        {
            print " "x5,$G_ON,ep_summary($ep,1),"$OFF\n";
        }

#
# special hack to for ReplayTV's "smart" record
#
        if ($show->{device} =~ /^REPLAY/i )
#
# let's try leaving out ReplayTV's "smart" record hack
# for MYREPLAY shows.  It should be caught by the MYREPLAY
# code as an episode on that day
#
#            or $show->{device} =~ /^MYREPLAY/i )
        {
          next unless length($show->{day} ); # don't record title-only scans
          next unless length($show->{hhmm}); # this should never happen
          next unless $ep->{channel} eq $show->{channel}; # Replay is channel specific

#
# check show two show slots forward + back (one slot caught by start-time search)
#
          my $hit=undef;
          my $epp=undef;

          $epp = $ep->{prev} if defined $ep;
          $epp = $ep->{prev} if defined $epp;
          $hit = $epp if lc(get_text($epp->{title})) eq lc($show->{title});
          $hit = undef if $epp->{device} eq $show->{device};

          $epp = $ep->{next} if defined $ep;
          $epp = $ep->{next} if defined $epp;
          $hit = $epp if !$hit && lc(get_text($epp->{title})) eq lc($show->{title});
          $hit = undef if $epp->{device} eq $show->{device};

          if ($hit)
          {
              $epp->{device}=$show->{device};
              push @{$RECORD{$show->{device}}},$epp;
          }
        } # replay conflict check
    } # extra episode scan

#
# if the title conains a "*" character, do a full search
#
    if ( $show->{title} =~ /\*/ )
    {
        my $key=$show->{title};
        $key =~ s/\*/.\*/g;	# replace * wildcard with .*

    	for my $ep_title ( keys %{$GUIDE{all}} )
    	{
    		next unless $ep_title =~ /^$key$/i;
    		for my $ep ( @{$GUIDE{all}{$ep_title}} )
    	    {
                next if ( $show->{chanonly} && $chan ne $ep->{channel} );
                next if ( $show->{dayonly}  && $show->{day} ne $ep->{day});
                next if ( $show->{timeonly} && $show->{hhmm} ne $ep->{hhmm});
                if ( $show -> {neartime})
                {
                    my $delta = abs( substr($show->{hhmm},0,2) -
                                     substr(  $ep->{hhmm},0,2) );
                    next unless $delta < 2;
                }

                print " "x10,ep_summary($ep)."\n";
    		}
    	}
    } # wildcard scan

  print "\n";
  } # show chan loop
} # show time loop

#
# Now check for recording conflicts
#
for my $dev_name (sort keys %RECORD)
{
    my @shows = @{$RECORD{$dev_name}};
    for my $ep1 ( 0..($#shows-1) )
    {
        my $start = $shows[$ep1] -> {start};
        my $stop  = $shows[$ep1] -> {stop};
        my $header = 0;

        for my $ep2 ( ($ep1+1)..$#shows )
        {
            next if ( $shows[$ep2]->{stop}  le $start);
            next if ( $shows[$ep2]->{start} ge $stop);
            unless ($header)
            {
                delete $shows[$ep1]{device}; # don't need device print anymore
                print "${R_ON}**** recording conflict for device $dev_name\n";
                print " "x5,ep_summary($shows[$ep1]),"\n";
                $header=1;
            }
            delete $shows[$ep2]{device}; # don't need device print anymore
            print " "x5,ep_summary($shows[$ep2]),"\n";
        } # show2 loop
        print "$OFF\n" if $header;
    } # show1 loop
} # recording device loop

#
# Now check for deleted shows
#
if (defined $MYREPLAY_LIST[0] )
{
    for my $title (sort keys %OLD_SHOW)
    {
        for my $show (@{$OLD_SHOW{$title}})
        {
            next if $show->{title} ne "";     # already used?
            $show->{title}=$title;
            printf "${R_ON}** DELETED ** %-60s ${OFF}\n",sh_summary($show);
            $show->{title}="";
        }
    }
}

if ($HTML)
{
    print "
\n"; } # # If we're doing a MyReplayTV scan, save show file # (we can't do this earlier, due to null cleanup breaking scan) # Save_shows() if ($MYREPLAY_USER ne '' ); } # tv check scan # # That's it, have a nice day # print STDERR "Exiting\n"; exit 0; # # Support subroutines ------------------------------------------------------- # sub opt_summary { my $show=shift; my @options=(); foreach (0..$#COL) { next unless $COL_TYPE[$_] == 3; push @options,$COL[$_] if $show->{$COL[$_]}; } push @options,'*DUPE*' if exists $show->{dupe}; return '{'.join(",",@options).'}' if @options; return ""; } #opt_summary # # ep_summary # # Print a one-line summary of the specified episode ( in a subroutine to make changes easier ) # sub ep_summary { my $ep = shift || die "ep_summary, how about a episode fella!"; my $flag = shift || 0; gen_episode_dates($ep) unless $ep->{day}; # # XMLTV format does some wierd things (IMHO) for multi-part episodes. let's deal with it # my $desc = get_text($ep ->{"sub-title"}) || get_text($ep->{desc}) || ""; my @parts; foreach (@{$ep->{"episode-num"}}) { my $text = $_->[0]; if ($text =~ m!Part *(\d+) *of *(\d+)!i) { push @parts, "$1/$2"; } elsif ($text =~ m!(\d+)/(\d+)$!) { push @parts, ($1+1)."/$2"; } else { # Ignore episode-nums that aren't understood. FIXME do properly. } } my $part; if (not @parts) { $part = ""; } else { $part = shift @parts; foreach (@parts) { warn "discarding part $_, doesn't match $part" if $_ ne $part; } } gen_episode_dates($ep) unless $ep->{day}; return join(" ",$ep->{day}, mmdd_swap($ep->{mmdd}), "$ep->{hhmm}/$ep->{len}", get_text($CHAN{ $ep->{channel}}->{'display-name'}), ($flag ? "" : get_text( $ep->{title} ) ), "\"$desc\" $part", ($ep->{"previously-shown"} ? "(R)" : "" ), ($ep->{device} ? "[$ep->{device}] " : "" )); } # ep_summary # # sh_summary # # Print a one-line summary of the specified show ( in a subroutine to make changes easier ) # sub sh_summary { my $show = shift; my $val=""; $val = $show->{title}." (title-scan)" unless $show->{day}; $val = $show->{day} if $show->{day}; $val .= " ".mmdd_swap($show->{mmdd}) if $show->{mmdd}; $val .= " ".$show->{hhmm} if $show->{hhmm}; $val .= "/".$show->{len} if $show->{len}; $val .= " ".get_text($CHAN{$show->{channel}}->{'display-name'}); $val .= " ".$show->{title} if $show->{day}; $val .= " [".$show->{device}."]" if $show->{device}; $val .= " ".opt_summary($show); return $val; } #sh_summary # # find_episode # # given a pointer to a show ( with channel/date/time info) see what's playing then. # # we have a ordered binary date array # Returns undef if no episodes are found (or all are greater, see above) This is signifies no guide info # sub find_episode { my $show = shift || die "find_episode(show), show to match please"; my $chan = $show->{channel}; my $time = $show->{start}; # # first let's search for a direct match! # my $ep=$GUIDE{$chan}{$time}; return $ep if defined $ep; # # now let's do a binary search # my $times = $GUIDE{starts}{$chan}; return unless defined $times; # channel not found! my $low = 0; my $high = @$times; while ($low < $high ) { my $mid=int(($high+$low)/2); last if $mid == $low; $low =$mid if $time >= $times->[$mid]; $high=$mid if $time < $times->[$mid]; } # # ok we may have found our show. # $ep=$GUIDE{$chan}{$times->[$low]}; gen_episode_dates($ep) unless $ep->{day}; # # we have a miss if result has ended before our start time. # return undef if $time > $ep->{binstart}+($ep->{len}*60); # # guess we have a hit # return $GUIDE{$chan}{$times->[$low]}; } # find_episode # # get_text # # Given a pointer to an array of [text,lang] pairs, return the best value for our langauge # Note, if more than one value exists for a language, only the first is returned. # # @LANG should point to a list of languages in order of preferences # sub get_text { my $val = (best_name(\@LANG, $_[0]))[0]; $val = $val->[0] if ref($val); return $val||""; } #################################################################### sub load_show_table { %SHOW_DATA=(); %SHOW_WIDTH=(); # # Table headings # for my $col (0..$#COL) { $SHOW_DATA{"0,$col"}=(abs($SHOW_SORT) == $col ? uc("_$COL[$col]_") : lc($COL[$col])); $SHOW_WIDTH{$col} = length($COL[$col]); } # # build sort key of table data # my %sort_keys=(); for my $show (@SHOWS) { next unless length($show->{title}); # skip deleted records my $key = $show->{$COL[abs($SHOW_SORT)]} || 0; # # special sort... by day # if ( $COL[abs($SHOW_SORT)] eq 'day' ) { $key=index($WEEKDAY,$key)/3; $key=9 if $key < 0; $key=int($key); } # # special sort.. channel # elsif ( $COL[abs($SHOW_SORT)] eq 'chan' ) { $key=sprintf("%03d",$1) if $key =~ /^(\d+)/; } # # save value # push @{$sort_keys{lc($key)}},$show; } # build sort keys # # display table data sorted by key # my $row=0; my @keys=sort keys %sort_keys; @keys = reverse @keys if $SHOW_SORT<0; for my $key (@keys) { for my $show (@{$sort_keys{$key}}) { $row++; $SHOW_DATA[$row]=$show; for my $col (0..$#COL) { my $val = $show->{$COL[$col]}; $val="" unless defined $val; next unless length($val); $DEVICE{$val}=1 if ($COL[$col] eq 'device'); # help build device list $SHOW_DATA{"$row,$col"}= $val; $SHOW_WIDTH{$col} = length($val) if ($SHOW_WIDTH{$col} configure (-rows => ($#SHOWS > 8 ? $#SHOWS+2 : 10 )); $SHOW_TABLE -> clearCache if $SHOW_TABLE; $SHOW_TABLE -> selectionClear('all'); $TOP->title("tv_check config -".( $SHOW_XML || '(untitled)' )); $SHOW_ROW=0; $UPDATE_BUTTON -> configure ( -state => "disabled" ); $DELETE_BUTTON -> configure ( -state => "disabled" ); } load_selection_items() if $SELECT{day}; # in case device list has changed. } # load_show_table # # load selection values # sub load_selection_items { # # load Device list # $SELECT{device}{list} -> delete(0,"end"); $SELECT{device}{list} -> insert(0,"",sort keys %DEVICE); # # load Day list # $SELECT{day}{list} -> delete(0,"end"); $SELECT{day}{list} -> insert(0,"",@WEEKDAY); # # load Channel list # $SELECT{channel}{list} -> delete(0,"end"); $SELECT{channel}{list} -> insert(0,"",@CHAN); my $day = $COL_VALUE[$COL{day} ]; my $chan = $COL_VALUE[$COL{channel}]; my $title = $COL_VALUE[$COL{title} ]; my $match = undef; $day = "" unless defined $day; $chan = "" unless defined $chan; $title = "" unless defined $title; $day =~ s/^\s+|\s+$//g; $chan =~ s/^\s+|\s+$//g; $title =~ s/^\s+|\s+$//g; # # load Title list ( also fill hhmm and day if known ) # $SELECT{title}{list} -> delete(0,"end"); if (length($day) && length($chan)) { $SELECT{title}{list} -> insert(0,"",sort keys %{$GUIDE{$day}{$chan}}); $match = $GUIDE{$day}{$chan}{$title}; } elsif (length($day)) { $SELECT{title}{list} -> insert(0,"",sort keys %{$GUIDE{day}{$day}} ); $match=$GUIDE{day}{$day}{$title}; } elsif (length($chan)) { $SELECT{title}{list} -> insert(0,"",sort keys %{$GUIDE{chan}{$chan}} ); $match=$GUIDE{chan}{$chan}{$title}; } else { $SELECT{title}{list} -> insert(0,"",sort keys %{$GUIDE{all}} ); $match=$GUIDE{title}{$title}; } # # if we have a match, fill all fields # if ($match) { $COL_VALUE[$COL{day} ] = $match->[0]->[0] || ""; $COL_VALUE[$COL{channel}] = $match->[0]->[1] || ""; $COL_VALUE[$COL{hhmm} ] = $match->[0]->[2] || ""; $COL_VALUE[$COL{len} ] = $match->[0]->[3] || ""; } } #load_selection_items # # help popup # sub help_popup { my $help = MainWindow->new; $help->title("tv_check help"); $help->Label(-wraplength => '4i' , -justify => 'left', -text => " This is a program to create/maintain a show XML file for use with tv_check. I hope it's fairly intuitive. One thing that can get you is the aggressive nature of the autofill of the selection fields. The good news is the routine only kicks off when you click a listbox. Don't click in a listbox and you can edit the raw data all like. Don't forget to check out README.tv_check Good Luck! Robert Eden rmeden\@cpan.org ")->pack(); } # help_popup sub help_about { my $help = MainWindow->new; $help->title("tv_check about"); $help->Label(-wraplength => '4i' , -justify => 'left', -text => " tv_check $XMLTV::VERSION (C) 2002 Robert Eden reden\@cpan.org This program can be used/distributed on the same terms as the XMLTV distribution. https://github.com/XMLTV/xmltv ")->pack; } # help_about # # Error popup # sub error_popup { my $msg = shift; print STDERR "\nerror: $msg\n"; $TOP->messageBox( -icon => 'error', -type => 'ok', -title => 'TV-Check error', -message => $msg) if $TOP; } #error popup # # load show array # sub load_shows { my $file = shift; unless (-e $file) { print STDERR "\nWarning: show file not found ($file)\n"; return; } $SHOW_XML = $file; print STDERR "Loading xml show info ($SHOW_XML)\n"; my $twig = new XML::Twig(TwigHandlers => { shows => sub { my ($twig, $show) =@_; push @SHOWS,$show->atts; }, lang => sub { my ($twig, $lang) =@_; push @LANG,$lang->text; }, }); $twig->parsefile($SHOW_XML); printf STDERR "Loaded xml show file ($SHOW_XML) (%d/%d)\n",$#SHOWS+1,$#LANG+1; # # fix show entry # for my $show (@SHOWS) { # # UTF-8 encoding seems to *BREAK* display! go figure # utf8::downgrade($show->{title}); # # ensure no null values # for my $col ( keys %COL ) { $show->{$col} = '' unless defined $show->{$col}; } # # convert channel ID to new format if ncessary # if ( ! exists $CHAN{$show->{channel}} && exists $CHAN_NAME{$show->{channel}} ) { printf STDERR "Converting Show File Channel ID %10s to %25s\n",$show->{channel},$CHAN_NAME{$show->{channel}}; $show->{channel}=$CHAN_NAME{$show->{channel}}; } # # convert numeric date if needed. # # next unless length($show->{day}); $show->{day}=$WEEKDAY[$1] if $show->{day} =~ /^(\d+)/; } # fix entries unless (@SHOWS) { error_popup("$SHOW_XML does not appear to be a show xml file"); } load_show_table(); if ($SHOW_TABLE) { $SHOW_TABLE->pack('forget'); $SHOW_TABLE->pack(-side => 'top', -expand => 1, -fill => 'both'); } $SHOW_CHANGED=0; } #load_show # # load channel guide # sub load_guide { my $file = shift; unless (-e $file) { error_popup("Guide file not found ($file)"); return; } my $st=time(); my $c=0; $GUIDE_XML = $file; print STDERR "Loading xml guide info ($file) "; my $xml = XMLTV::parsefile($file); $ENCODING = $xml->[0]; %CHAN = %{$xml->[2] }; @GUIDE = @{$xml->[3] }; %GUIDE = (); print STDERR $#GUIDE+1," recs / ",(time()-$st)," secs\n"; unless (@GUIDE) { error_popup("Listings file ($file) invalid or empty"); } # # Build indexes for Episode Data # $st=time(); $c=0; print STDERR "Building Episode Indexes "; for my $ep (@GUIDE) { print STDERR "." unless $c++ % 1000; my $title = lc(get_text($ep->{title})); my $chan = $ep->{channel} || "" ; $CHAN{$chan}{'display-name'}[0][0]=$chan unless exists $CHAN{$chan}; if (! exists $ep->{start}) { warn "\n No start time for $title\n"; next; } # # convert XMLTV time to binary # $ep->{stop}=$ep->{start} unless exists $ep->{stop}; $ep->{binstart} = UnixDate($ep->{start},"%s"); # # don't consider a show a repeat if it has been shown in the past 2 months. # delete $ep->{"previously-shown"} if exists $ep->{"previously-shown"} and exists $ep->{"previously-shown"}{start} and $ep->{"previously-shown"}{start} gt $TWOM_MMDD; $ep->{displayed}=""; $ep->{device}=""; # # build general indexes (--scan + --configure) # push @{$GUIDE{all}{$title}},$ep; # all titles $GUIDE{$chan}{$ep->{binstart}}=$ep; # chan, datetime # # build --configure only indexes # if ($CONFIGURE) { gen_episode_dates($ep); my $array = [$ep->{day},$ep->{channel},$ep->{hhmm},$ep->{len}]; push @{$GUIDE{title} {$title}} ,$array; # titles by chan push @{$GUIDE{chan} {$chan} {$title}} ,$array; # titles by chan push @{$GUIDE{day} {$ep->{day}} {$title}} ,$array; # titles by day push @{$GUIDE{$ep->{day}}{$chan} {$title}} ,$array; # titles by chan by day } } # building guide indexes # # Now compute next/prev episodes and start time array # for my $chan (keys %GUIDE) { $GUIDE{starts}{$chan}=[sort keys %{$GUIDE{$chan}}]; # start time array my $prev=undef; next if $chan eq 'chan'; # skip special indexes next if $chan eq 'day'; next if $chan eq 'all'; next if $chan eq 'starts'; next unless exists $CHAN{$chan}; for my $date ( @{$GUIDE{starts}{$chan}} ) { my $ep=$GUIDE{$chan}{$date}; $ep ->{prev}=$prev; $prev->{next}=$ep if defined $prev; $prev =$ep; } #date $prev->{next}=undef if defined $prev; } #chan print STDERR " $c recs / ",time()-$st,"secs \n"; error_popup("guide file $GUIDE_XML does not appear to be valid") unless @GUIDE; # # Build channel sort # my %sorting; foreach (keys %CHAN ) { my $key = $_; $key=sprintf("%03d",$1) if /^(\d+)/; $sorting{$key}=$_; $CHAN_NAME{get_text($CHAN{$_}->{'display-name'})}=$_, } @CHAN=(); map { push @CHAN,$sorting{$_}; } sort keys %sorting; load_selection_items() if $SELECT{day}; } #load_guide # # Generate XML to save current show array # sub Save_shows { unless ($SHOW_XML) { error_popup("no show file defined, data will be lost, aborting"); return 1; } # # recreate show array dropping deleted elements # my @newshow; for my $show (@SHOWS) { next unless $show -> {title}; for my $item ( keys %$show ) { if ( exists $COL{$item} ) { delete $show -> {$item} unless $show->{$item}; #no null values } else { delete $show -> {$item}; # no "extra" values } } push @newshow,$show; } # # dump xml # print STDERR "saving shows to $SHOW_XML\n"; my $output = new IO::File(">$SHOW_XML"); my $writer = new XML::Writer(OUTPUT=>$output, DATA_MODE=>1, DATA_INDENT=>2); $writer->xmlDecl("ISO-8859-1"); $writer->startTag('tv_check'); $writer->emptyTag('lang' ,%$_) foreach (@LANG); $writer->emptyTag('shows',%$_) foreach (@newshow); $writer->endTag('tv_check'); $writer->end; $SHOW_CHANGED=0; } # Save_shows # # give chance to save file before losing changes # sub changed_check { my $nocan = shift || 0; if ($SHOW_CHANGED) { my $button = lc($TOP->messageBox( -icon => 'warning', -type => ( $nocan ? 'YesNo' : 'YesNoCancel'), -title => 'File Change Warning', -message => "Show data changed. Do you want to save?")); if ($button eq 'yes') { Save_shows(); } elsif ($button eq 'cancel' ) { return 1; } elsif ($button ne 'no' ) { die "Button returned unexpected value <$button>\n"}; $SHOW_CHANGED=0; # prevent 2nd warning } return 0; } # changed_check # # Note, Date::Manip doesn't deal with DST switch correctly. We need to use localtime # sub gen_episode_dates { my $ep = shift || die "empty episode "; my @d=localtime($ep->{binstart}); $d[4]++; $d[5]+=1900; $ep->{day} = $WEEKDAY[$d[6]]; $ep->{hhmm} = sprintf("%02d%02d",@d[2,1]); $ep->{mmdd} = sprintf("%4d%02d%02d",@d[5,4,3]); $ep->{len} = Delta_Format( DateCalc( $ep->{start},$ep->{stop}), 0,"%mh"); } # gen_episode_dates # # # sub validate_col_value { for my $col (0..$#COL) { $_ = $COL_VALUE[$col]; $_ = '' unless defined $_; next unless length($_) ; s/^\s+|\s+$//g; if ($COL[$col] eq 'len') { $_ = '' unless /^\d+/; } if ($COL_TYPE[$col] == 3) { $_ = ( $_ ? 1 : ''); } $COL_VALUE[$col] = $_; } } # validate_col_value sub add_myreplaytv_show { print STDERR " adding myreplaytv: @_\n" if ($MYREPLAY_DEBUG == 2); my $show; my $title = shift || ''; my $chan = shift || ''; my $start = shift || ''; my $len = shift || ''; my $day = shift || ''; my $foundit = 0; #used to supress message on auto-theme printf STDERR "want <%s>/<%s>/<%s>\n",$chan,$start,$day if ($MYREPLAY_DEBUG == 2); for my $old (@{$OLD_SHOW{$title}}) # capture settings from pre-existing show { next if $old->{title} ne ""; # already used? printf STDERR " got <%s>/<%s>.<%s>\n",$old->{channel},$old->{hhmm},$old->{day} if ($MYREPLAY_DEBUG == 2); if ( ( $old->{channel} eq $chan #use old show if chan/time match and $old->{hhmm} eq $start) || ( !$day && #use old show if old and new are title only ( !exists $old->{day} or $old->{day} eq '' )) ) { print STDERR "Found old $title\n" if ($MYREPLAY_DEBUG == 2); $foundit=1; $show=$old; $show->{day} = $day if $day; #only change day if we know what it is! last; } } # old show check unless ($show) # build a new show entry { print STDERR "Make new $title\n" if ($MYREPLAY_DEBUG == 2); $show->{$_}='' foreach (0..$#COL); # initialize to blanks $show->{device} ="MyReplayTV$MYREPLAY_UNIT"; # set initial values $show->{chanonly}=1; $show->{day}=$day; push @SHOWS,$show; } $show->{title} = $title; $show->{channel}= $chan; $show->{hhmm} = $start; $show->{len} = $len; return $foundit; } #add_myreplaytv_show # # quick routine to compute minute of day from hhmm # sub hhmm_min { my $hh=substr($_[0],0,2); my $mm=substr($_[0],2,2); return ($hh*60+$mm) } # # quick routine for mmdd->ddmm for our users across the pond # sub mmdd_swap { my $mm=substr($_[0],4,2); my $dd=substr($_[0],6,2); return $dd.$mm if $DDMM; return $mm.$dd; } xmltv-1.4.0/choose/tv_check/tv_check_doc.html000066400000000000000000000161151500074233200212350ustar00rootroot00000000000000

TV-CHECK

Summary:

TV-CHECK compares a listing of your favorite TV shows against an actual XMLTV broadcast schedule, and reports changes in your favorite shows as well as any extra episodes.

The TV-CHECK script has two modes, a configure mode, which is used to build a show file and a scan mode to actually do the check.

To use the script, first run a grabber (for example tv_grab_fi) to collect a schedule listing, then run TV-CHECK in configure mode to build a list of shows. Finally, run TV-CHECK in scan mode to produce your report. Once your show list is built, you typically just grab the listings and run the scan.

Configure Mode

tv_check --configure ---shows=show_filename ---listing=listing_filename

show_filename defaults to 'shows.xml'

listing_filename defaults to 'tv.xml'

Configure mode brings up a GUI used to create the show file.It uses actual schedule information to assist in choosing your shows.

The application has two major frames. The upper frame contains the shows you want to check, the bottom frame is a selector used to add and modify rows in the upper frame.

Clicking on a column label sorts the table by that column. Clicking again, reversed the sort order.

Clicking on a row populates the selector frame with the contents of that row, to allow you to edit or delete that record.

The selector frame contains entry fields for each column. If you click on a value in a listbox, the associated entry field is automatically populated. If a day and/or channel is selected, the title listbox will contain only titles that match the selected day and/or channel. If you click on a value in a title listbox, all entry fields are populated with details for an episode of that show. There may be other episodes available, so if the one you wanted isn't chosen, correct the entry fields before adding the record.

If the day field is left blank, only a title-scan is performed (see below)

The following options are available (typically used to prevent false alarms due to syndication)

TIMEONLY

The title-scan only returns shows on the selected channel.

DAYONLY

The title-scan only returns shows on the selected day.

TIMEONLY

The title-scan only returns shows at the specific time.

NEARTIME

The title-scan only returns shows within a few hours of the specified time.

The buttons to the left of the selection frames do the following:

CLEAR SELECTION

Blanks out all entry fields.

ADD SELECTION

Takes all the entry fields and adds a record to the show table.

UPDATE SHOW

Replaces the last selected row in the show table with the values in the selection fields.

DELETE SHOW

Deletes the last selected row in the show table. (selection fields are not used)

The show file can be saved by an option under the File menu. The File menu also contains selections to open new listing and guide files, as well as exit the program.

SCAN Mode

tv_check --scan ---shows=show_filename ---listing=listing_filename --output=output_file --html --DDMM

show_filename defaults to 'shows.xml'.

listing_filename defaults to 'tv.xml'.

output_file defaults to standard out.

--html causes the output be in HTML (color). Default is text.

--bluenew causes html output to highlight new episodes in blue (useful during repeat season)

--DDMM causes Day/Month format in reports. Default is Month/Day (MMDD).

--notruncate will prevent episodes more than 7 days away from being included in the extra-episode scan.

This command does the actual scan. The scan uses three phases.

In the first phase, if a specific day/time is provided for the show, that timeslot is checked to make sure it contains the expected show. If it does, the sub-title is given. If the timeslot doesn't contain the show, a warning is given and you're told what's in your shows place.

After the timeslot is checked, a title-scan is performed that reports any other episodes matching your show's title. This title-scan can be limited using the options described above.

Lastly, if the title contains an asterisk (*), a title wildcard scan is performed.  The * can match any character. A title of "*love*" will hit on any episodes in which "love" appears in the title, insensitive to case.

Notes:

This script was created to deal with the TV networks moving everything around.

If this script is helpful, why not drop the author or user email list a note of thanks!

Author

Robert Eden reden@cpan.org

User Mailing List

xmltv-users@lists.sourceforge.net.

Sourceforge Web Page

http://sourceforge.net/projects/xmltv/

Project Web Page

http://xmltv.org/

 

xmltv-1.4.0/choose/tv_check/tv_check_doc.jpg000066400000000000000000001034421500074233200210510ustar00rootroot00000000000000JFIF``Software: Microsoft OfficeC    #%$""!&+7/&)4)!"0A149;>>>%.DIC;C  ;("(;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;?"Q  !1AQST"ar#23RUq456Bbs$CD%tc$QR!1Aa( ?ݦ+5QS%?nYb=-)?(\cڱqek_  {{]^im+O{fiV)鹇P 840dNudۿՌa,~H4؂*rx^9<&_K3. ݞQbxܳYIIeČ{:Wo\}uSJ۫J]WpLsIM# fy\!4?ՌW>iH~5P͍PQ~YﯰOV9nMC.y$Mr;6[)џWto3sOSJc<5kE>u^AG+X1YdlRaYp.rKXBRɋhD,, m|[kRJ~گW|һ✂V%qutNCI)CQ '40K ||\6]!b|V %lSq>CI+f%ȩ9||\6]Ĺ/5t8"`J ||\6]"殏tՎ A>>./`?\GRLGV<%lr:njrJnjȄ !)_a()pZIMJsVP%lWuw%欰J  滒WF+5?5fPMl?ܚ7k[ɩ41ZO-+y5?5w:OOYF>/F7cYaYJ  򾳓]򶳓Yя 殍UJ  }U_5wʊ&j A4c/cu\\к1V|6_Ո'BWB A4lM#ъM#%ӏxo4.Jo4(A()eM+ %ӏVs`QBRieX3IZpF0S2V::)Ϙ6F='ᵚ:Z_r:kpٌ6J^4:2X5 ;<vt6 .&f~ju\1I{5`uj,[j) eHK Mn`01Į6O.5T+?+H[KYH[!Z$N4 |  &8NI >bqVӊw !;kfVt±篊G8[!n_ygw#-ϵ] 7[[ sHUVZFO@$f-JeUkm,ۍyg [h$0|Ti\=7Z3Yv;@f7 wdToۙqhlrhcz53p;|J_L­lsO1aӪmIcx/-fB^m/x-$e|*܁x[YwmM><[sD3qth x:@mii'ڧxмo=MsD1ZT-xߐ%J<8zRM><[s,+AҕBn(<8zR"N&^&j-ȼ+F`L)WM><[7'HW|:B}x?#~*N!Rt]7dToHT!]K Bn)x:B%0*N&j-\++Ka|\!VM><[G_y]c * 'jx&p-ſUnNo~M[N[Z!6+<øyVHM:xayV(M:xP|G~#X!6 x;QZ/qSЛrto|ME(=w8MoQG+)1 ->+˾+)hMo)0pHBmxc?ZC~m<[> |.Pox֍|N!6ż#BD&xgI؏ἂob=żC>)&%xG8r_6]l!=żC>)&*}>夎Z,Y~K{SV4c]#@ku : _;Rk*dzk\;VW+jdKY/9fqmeV mYvy]flo򞗈TB*!b.t'XTAS4.6|P%OKMvz^"oRVѲͳ](iO=/7ڏ)x՘VTP`/lvPYc}_:EDgj<&;Uu6 K"W294euȱ<[L4ido )xyOKMvQ Ӓ]gu*W3ZG Dgj<&;Va O=/7ڏ)x՘N@p\.>OKMvz^"oBO`Zwع. O0d2ɰu"=/7ڏ)xYUAR7ɚ<εR}J"O=/7ڏ)x՛3]Oҙz\fU|&;Q=/7ڳPFc.,5 O=/7ڏ)x՘]@yOKMvz^"oW3 z/S1sQ2C>a=/7ڏ)xX[e5䌬㭻NV4S}򞗈Y$ ?Dgj<&;Va O=/7ڏ)x՘B@OKMvz^"of4S}򞗈Y$ ?Dgj<&;Va O=/7ڏ)x[QBf{#1i2?WڻKR%{b8k. jI>S}򞗈U*Llc527Ҹ.Ӥ>\ yXEDgj<&;Un/CLXX~TA򞗈GDgj' 7W8iIY h<&;Q=/7ڡɅ53B-15m}6&#Ɠ+CmrNWR(,<&;Q=/7ڡGS42:( ݙ.( kˮ\u[?JEDgj<&;Va O=/7ڏ)xICƠH -[xhJ>]#[$6Z8,('OKMvz^"o@){$ cH.!~_br =KD-E)xyOKMvJ6SE!$4CHOKMvz^"of4S}򞗈UT9F$.氶bSpx_$7Y#/ͬ"z^"oS}+0gs7!n2%cᢑBP/ } oku&+3S^ `ÿҒ,P40mqm${&0i*j7fE+10Xz\ݗme}Tתҵ>-{owEŏ'q=jВ2N>z竛8 ZA 1;+@pjn$A-EӁbܹ p㒚j\2wصn\s׻sYPw̯8E=K2aGɰym"8M瓸ŏ76+6 dĒ|FICga+xhip7h \ p,Sf:z_$ ̭{Icwm’$b& 6Z3tĸ^J?ҷhs'YBF!-#icKmuww: v#K uNe[4o#M5-wT'/ocZ7P"Kn^lPx@9st@.7_r N|NvVov5?X Jw!cZFSCGŏhIO'q=N>zա$e<>,|y;|XVw!cZFSCGk iwZ&)#20,Q k j ⑛dBH ZpLnM̜;V #16 OI9S~N>zա$e<>,|y?^Y4;SYjLB3,,lD|͜,\uFkM^KP|3C,K_m\ {9nIf%ŶX$ Rw!cZFS˶mh6RCr[Zf<i#/LT9.f.= 6{թBH́sɞA`Hw֭ #)!czsݣmo;i.ZD֏cRF``87פؔ0LXBFX`X$4n}i@FZ6ZMR;\u_N>zա$e<>,|RICO)s fjUGd#HKYg44տeɊ>S]\1뿅[ҒxKljUe- MBs!ڠę_QU; AMCv(c6$\2 Fܝ#;gb|KR qy|wnC wҠjgӿ\5jVs;p,p'jlR6x,̍F:# Enn srl\k(`dl eVph3_WS~}#;GrwZ#HNxIs ~:-nZflAa`emؗf2xFRp,RU \3lC\C WTo9%Iww~)48r0Li1k el܍-vt "x֍#;c|fIϦsY%8YLTVqs&w~W(qm!HNxFܝ𲕘Ok"Db- ypB`5smZ|3é%4J4eć-'a ^ܝ#;cUx asgXmڬh?;вi;ЁGrw4焴 Fܝ#;-w'<$y3DMܝ#;WfҹRfѹkHp@#;GrwZ#HNxFܝ;KτpqM5kru#%߻o <#HNxKBi;ЁGrw4焴 h=yo ^cR7 K@#;GrwZ#HNxFܝ;f{;Ώtju"G #;GrwZ#HNxFܝ;w'<%4K}<+G>x L_ Ujj/buQQ9#hֶ ׽-,o >iq} gmv_Ҫ)qiM;ft -UuZXR6[8]?v*,u,1Ӷ*[v`Z I>Zޛ-{; `?BVw^ ob';?<${KWi5uۅdoƯdd.4ɞm#hh| =\qL1EfK8rvWA/q74A؁#`Yj'Ob2jpҺ745u FGsĴ F&G} ԉE(B0%_.A^REO@+[!tm eQjkq<Oc9{ΕH/6]|;6qcCYK wHNzAثpD\i;ˢ|g/nP5#ݞ7i.enRnRnFqb~K/6K[m>FREZcr6ΈNqWnZi'= FE_4;c2E麚,x0;^j>t@x5h zd f9Y,mP6bݖpB㽕oU]h[j2Yk_]kVwko b͝b0BB2, dasmTl/m<5tz/ .kZ.-k;jmk86"&OnO[زuUlpَH)~bZl@1RXsfxy5 4;3Hx%iH;FtҶ!==HFHMԹKa;UbGsiH;ЁGsiH;ЁGsiH;ЁGsiH;ЃH;f{<ׄν ?:?zeFw^ H;'= KBiH;'= KBiH;'= KBiH;K*IГxH5kw:ݷGsĴ FGsĴ FGsĴ h=y >sc} ZGs7 K@#9b4h@#9b4h@#9bLq}#Ώo'R'޴?v#HNzAؖ?v#HNzAؖ?v#HNzAؖ?v*]ޛqkUmA^=~jbW'SQ|+VO w+`<CE}T30>7nY:6Y=G].F=%:jPi3loMQ Kl3L˘}ѺEwѬ53ixo=:o‰'j>8Ki${'p7U`okWXҶ߾++Eo³y7,1:qCFH\ 1vw~54+1s:oq;m{fg}æ(IɇMV.*hR>B:=n$fsvJnQS%-{OCNg׿JKəqNW+B2MI ux(M݃^Lΰ{Kəqg$ L$Q9>v=җ[IEWIB)䊱yQ"4Iy3:#I/&gX=H اi[9qì2~m0˪i-p>I/&gX=i%R4fuIg?VC@7V^Lΰ{Kəqc)SH0ia35a ~Wo`6Ff #J ob;^AKəqIy3:*[]ncOcytq)>d H3t ݫMV^Lΰ{[4W8?>R%833Jsl!v/6:Iy3:#I/&gX=3 59=.eA|dv5e 1]?3ޏq:aY4n: 54c7Hsd~jfuF^Lΰ{/7KP%4psɥy\k[ULRD̚Lƻ$Cx3u =$`3)_WOƓ$9 ݗ$l׬ WӠF^Lΰ{Kəq-i%$`ЁIy3:#I/&gX=Ĵ F^Lΰ{Kəq-34fuKBi%34^hα5 ԉ&(I/&gX=i%R4fuF^Lΰ{h@$`3Z#I/&gX=Ē<$bmƚ֝Hwr KəqIy3:%$`3Z#I/&gX=i%އ'5svky$`߇o5-eHKəqIy3:%$`3Z#I/&gX=ę!}h󣷻^ԝHwzj KəqIy3:%$`3Z#I/&gX=i%KəqRǒ,oچT{"51|+V}>ELsqY$l&Kkw웮%?:{\F^Jΰ{KYq-|kuo$`?mYb+drRc|}F5py#I4VuYxɰִMZcXf&ۈ? *ਙ| .0jI/%gX=i%$2vZdqh Eu@$`ZHKYq&Gɞ]:-n$4 "V^Jΰ{KYqdttUaҵi>i{N8wxwh/&| Z3^6Iy+:#I/%gX=Yh2xO/ˣ-ߺ@ԏ<'{O앤n6/,m @C$u V5.V8 oAKYqIy+:(.X,z]%f/{/ӵ#mG{ IItLz?q:aYT 54zC'f5zIy+:#I/%gX=ő~)Mƙj qsbE~bSA`tMߢqh;wz5:Iy+:#I/%gX=Śq=oц.{؋}G*tCd0+6HmH5Iy+:#I/%gX=E⸆ Ո:8/ڵAשTQɍG@};;Ei' i%$`vM%Mspӱ^kd$vVRNi[kl$lui%$`ЊF^Jΰ{KYq-4VuKBi%34^3Ou"ɦ KYqIy+:%$`Z#I/%gX=i%KYq$ARHi&&imnɯNmi%$`ЊF^Jΰ{KYq-4VuKBjvo^i%<hi%$`ЊF^Jΰ{KYq-LϐӾoFԉ'wƢ?%$`вIy+:#I/%gX=Ĵ F^Jΰ{KYq-'s$e7UmC}^=~}_ Uꢦg9c6m-WtfuD'=KP#I/&gX=i% L ;[&JKəqN՛0'8䥌#"Ij'\j4Iy3:#I/&gX=ş; *jDk Zgk7w7'jn,A5e>UQ+qІ DkԃI34fuY\ -x.vP@I;}+JIy3:#I/&gX=Ĵ F^Lΰ{2>Lozu1]4wϒ\vKəqIy3:,)-59C,󁨿\{= U"I`4FNiϚm-˷~7Iy3:#I/&gX=YCk m#.;Uٿul&xNi/JKəq7YDܗ͖L{:H멩M#r2!t'ik34fuQpQ;p,/ 4FsImrlw°ĮJr "s"A8`AKəqIy3:(x-\՘xw1nfۃz~ žIy3:#I/&gX=Ĵ F^Lΰ{Kəq-34fuKBi%34^hα5 ԉ&( $`3Z#I/&gX=i%KəqIy3:%4fuI/Mi&&imn˯Nm34fuKBi%$`ЁIy3:#I/&gX=Ĵ h>A<ցu7JV^Lΰ{o7Iy3:#I/&gX=Ĵ F^Lΰ{Kəq-3LϐӾ4yݯtoNO;h534fuKBi%$`ЁIy3:#I/&gX=Ĵ F^Lΰ{wXsfw痈gL{t>z\F(D.[cڪDؼ5#$>kIlm xBS^!1i"pvW߼s8 ԠFyxtǺ u-^!13t-Jg*u5PZu!6k53=ggL{&)+(&F=3MfZO|6o{t1SQƷ¤/sXs} 6][/d u [\i#k(SzfYG9FZjGImZS7i%n8% @pH)J̆퀈!cŞp^.8ms3=ggL{. فa'm,bL6Q{nAt,#I?3GǺwwC0HNx"ї8;ξB] /ΘQ^!1qlPYǡres\FV]n/hL'#Lu4}zlcbl3o85KIψԲwӖ6'IJLI`?6b:UKk袷b3Œlu c3yk令7oF y]T2vC6?>ͣR 9cb3Ĵ"n&>$3tl9?+.c)FKp/m)YC 8Xdb*[QN ۂ}ꩴc '/AŇHbb.Po8]G##ko=.+նFK2IM8J]m \䧉x-sK͈;TǢ+`OԳS<{#Ldiv'q LFZG4Zga䛝9Ph+.盉= }dR R4ґ&`l,nEآIMl\aiihA|j NyC،q1gilD͢iCNlǷ@^\EǞ۽iP#%3g=h@q1$O 6qBu#%߻oȃ$ ;o:?An&>#*]qŨz};'D\VŪs,U\n&Oo&E٨#lFrabsZ VO_IE"(d6ϡGS&I[\ȌŝFs_{M'{؍}gGB #ɤb4GI$h&=M'{t ؍}gGB #ɤb4GItlukp :`R4GI$h&=M'{t ؍}gGB #ɤbL"{<-tHM'{أ}g#ɤb4GIđҹMm ^_24GIh&=:HM'{أ̀O!wj%h&=:HM'{أ}g&H ِS.擷ydHM'{أ}g4!:w-]ۼ; di>زۻLm$N;7}LwiM $A[/Rdֻ1iގ~OPmOo&ӓ|'hU4uJ])$,V6&߭40?0ݷڬŸ3t,GqfYجŸ3t,GqfYجŸ3t,GqfYجŸ3t,GqfYجŸ3t,GqfYجŸ3t,GqfYجŸ3t,GqfYجuҺJKwmAחn[dhwŸ3t,YUG5TM^M#"7ƼnkbuH#v {j"gqfY؏,гY![C噺v#Ÿ3t,VHAxfnn! ;ZVfnn! ;Vfnn! ;Vfnn! ;jGG;;cqcmCA`3ZqfYت-_UYM$13KHlÐ]mKv_Y:}Lb5Ѻ6hu/&x"Ÿ3t,\8}{m|nQsaxV:eMk4ώM;oRo-km5wblj1Af `rZ].[|7Bxfn,г->YgbB ,г->YgbB ,г->YgbB ,г->YgbB ,г->YgbB  ܷ"hZ׹ "fKS$1AMHվWHmca5`詣6hn`苿,г->YgbB*Ÿ3t,GqfYجŸ3t,GqfYجŸ3t,GqfYجŸ3t,GqfYجŸ3t,GqfYجU51DeWHŅpDBn! ;C噺v)RtU\ ,г->YgbB*Ÿ3t,GqfYجŸ3t,\ mS*]JckmombB"Ÿ3t,T{ßGVM-S$H9fkwRI[ -(BBBBB\^L Y{I![wwMa_Ut8VTUx##~gnZ, g 4pP63O1%Nmg`E2qWqWk-lq L36@u%`eSn3 b܍W׼*QQVZA͊Vb -ΨͮV¿STeSn3 bW9X$څJR3Bf$]p߳J>W9G|$3r2v87Rؑkf{kb N:*hؙs=^ZMu>W9G|$ <'~ZZ!dn؅ UUo`ԮG|#Oq^M9}'8\rڇ7°QEi/veoDH^&i/;YQwKVS &lʨ/bto`[8hpիm?0|:ǘuR>:FM%K1Ŷ{Xa5-467yvoBI~Iz4GSUEDZ$ ̂70_6aAGt14fcL<o|i/8Aƒz aCG<J?#]PdSѷ=c*Q4@[UH7͵4GI~Iz(#!t̠:μS`ѹ)E#sTWGwXUn:2 j>YLk3׏U$3; Z'vtlJ4:f5 A:6#2aEv[Hִ>u" C칿݈%!EBBBBHI7Uf//+9}䞳}Vb?rb#9-&Zx-Ke &U)EQD L}״ĝjtX UK]OѺyKn. uWRa2[8]XۋPWN*ڧHKOW .^ؼ٭mjCL[S,TRI`.sZDvWk ,*NeLlI.#H|a}Sxp"NO*@>])'5@"VbZN֫E 9k ,>`RQB?ýuetqC,18IavdHjVz kW׍7 TB8#/(C)If;)Ҽw2+rR]ɞ[7#.ۋfv.d{]4sRvp#+b}koROS+\B 7kp(t؍TEL _LӴs4J]M 5p> D9:9HiaTT69k{v0cf:viILm=;ukcۨzN G%@n ; ǜܥ/`'f&$teeCi%ytn`F3;+2!h0p6D"u8v]td4-s؏8l֞+I=;#,%To}7Ҏ 95FF]1]ye)R<<#PWUn5.tQ {Z.gsr.z. M#R|I{^ǁvsUP"iCosd9$tX׹6!}:tfUdc2I;r4p\/..=:bU `{dpsml^R`Ts4F| w,;`.uNg JmCm~/K$]9xBlF8՘S(Oyqm; kUL=vu3oH%ǰ)1y4639.af)MP^ MK\Ʊ\Fŧ; kQvue #6Gamw[jA&P; kQvue #6pഘS\wN3m{:M,?F=@M`M&Oo&NKQQ*#sPK D*JLB0F] @~`3nӫ^"j[L_a{[m~3&)G029ts $evgbF:W1xA{Mj7aCA 2Q$9e9ϱ{ ׼B9ãss ':])X[h"V15}O3EvFpq/qsqak3NNjX祧c_SѴL:b~.RTӊ3r<1}닙Wif =SQ/Ѭ?L٪[L1vFS n;x*xhiNO9rm\\n;x&w;YpsfGj_la཯h 0*ڊ?MR$xլ|j$jSSR4$MkP{ (($9 {P:!!@!@!@!A"_y'aUzܿجzYgt᱾$.uTy$ N5S͂A$uM,`{8&6 d6GJ2&9ځ6^6O !ľx+5mih'ڃ?%dӾ;qϤgH}_IWQ㖫'4js{ a6`%ͽJ񛠠X)7 4g($2x,0J3c 4?QAFKB qsH nR-NBB?ýw?ϵc) ii<+ߕ%;-~Kb&1+@lm.q9wTg": ~be&FmemmֻNeg$,>ole$~ж41VSxI-$b$F~eLjS($Dآ.q$Xi QF&u_sIw\:ɨڟlm,S36w)&`󎣫m*v %,A!N[! &A ,ON6t{}]MFS@4i"CdtE{vHX k|xnx)*檎]og8<:Bevy^6>6 >x(%ѣf݋V:錞$06;#iAǫ0cI+6浤9sqbUT:6k:m^^\~A ESo"h-A<  JiY ̆<-۰i*ȽƳg8*-4cixI <XG~Ⲱ9S9ṛknHiu1s]V;Aڼ)ᬕJ]#i|v7jP䪩SԲiRQ65$IfWM3ѷ19cxu+AVUY&h]P>{v&׾A f#|*) t|kbOƭ]^c}>-]LCo$kCrknb. ]Mª殟A)#33A0}]!dn~8pD$.@.o憂*癑5:ЊqқN?cSh!qlqGh$u ,*~zkGuj_3ڴ ?ѩxj#i[ -G%OP(S+(`il $޶rbP ц|I,cG<I#l8_\ZK&-Y~qsZl6B^gMr nPq{,NmkXNcR_ ` pq~uMr nQ97Y(@2 Z4'Tӫ,st\}X>u61)Fg(}65&18`7>=njjgGwXM`t`{{e=)gERO6xe]k?:|$ L酝pJ؟+i_O5lU!Q]YQ 8:6&h*!!@!@!@!A"_y'aUzܿجzYg| sɖXdD@snFq&G-=!hi Mۗ.5[QC2&>ydOEm C|Dr]լ%ps~ـP^ńQ Dsm^q}@ɚzgI> mS2ls>hϫ/_eZZInh \l%èj=$T -v׳.H(r\I-7"!!!6Pu M{In9#>56s\Xmkb=6}#-;* 8c$ڥxqxsmsjrZV9ƨ3F#68#aFY(}Z\qՒ<xS# ґFi0Al '`TNIzwn=PQEa2\F(G j7K_[SRV9rߜ7Ī⩧C$¡啦f8yga6 !ehGUV HP9>pbHB1)?6Q͕_!-0mpJ젊hmkcs8F'(JC^7<)Z+Ev#U{9cvfp5zFТϺXwLt^ sb/ke-f*RZ#3hկR][M;l4|Mch'~7Z􉡎C+xv [6!JiRdm͎}\c45 QBj_3ڴ ?ѩxj#I7 }6?E9/zG}slj}*Dhtm!ޛv q<{s6*0 ;@>˰ kq 7:蓮46ٮmA`HL[r7nXX"3InIgo/eѺC%4 Բӓ#`1q .GuT")ewci[b\|Ln%2ƶKI@ݶ[P z,My`$)*+'kikҤjh"`coYBB?ýue*jGc2F\YN<|Z>ՕwC14L|N< 6ǤCOk#teC] _kt01H9%+#:6* 96bo1Ѱ pwDEE7kPE;Bܥֹ^IAO]Kw2?<p3Η.襖yo+_;\o7z KI -cwZ"֏͌I3#<.phsPA;wy4Sf55$W:hN!Q;C$Vaz\0>iy#skk4i!ywGeC3;lavg -.}a4kdi@{2վu~6 4OQ)>$xR'^ݗ&CKC @lybtfhn\|ā8䯒wRT1mKkֵ=7V4xT45Kpz(&E8M1)7cXrMGIHbd'sIm)S LPV66LI\ 31 iV9{5 뾾 }4A6[ :)hIǜH͵Y_N5๱6\Hqp?PHB6!j_3ڈM`M&Oo&Bqߓw&ӎF3׏?0|43HBB$! -'ǵ8-TaM lIpFGXhװ2zV38lɗF0Noo9CY k(٫ lQ:GԘuC⥨d26W_)  U+^7f!Iqp l)c: ?I5tDu P678I##]@޺n,@dY@u>:1 t̞GaJ6!WFI\-|R2h,nc7أʼnQ$q(.0e#6Cg}^Х "!!6Pu M{`:J!ߔ,О*loge1m{Em>=R4O,M-˜sN7+3[06It|ѾAot%Zul3H^^ {TCZΦtFcr/\#_խ& KT9av繑y kA$.{ZEFeF3]AWSC!\9- ;1ukkLq3lL~g{l~- !% uLg1#c9'S}vmAM&1H0^փw=X [ZBKTX*1E!cZXelV-pZ34xF!Q fyYa`<Ѵk+LYK Ca|hh>vk|HEPۋѻ}|f?FIRrb1)?6BP>$i넺[, ܓlQP컘tSӾI+ܿ;bՔ} ;]$Rk.ɻOϘ Zw-<ű96wg@pY6&7RE7̅o6SkoX (nC)EB ?ѩxj,?F=$ -rom!@!@!@!@';=g{m1>QUY̓"[zD=g~aEGŰ*L=ig-{,&9IJ43B     %DF}&+qP\b -66<8Z(Kd pov_{yXQ: CLY >-0*:ŵuQIOjG84lh gIŪ媠G [p2f.u| ' &5ͫYN-p>PGEDȃ|0s]n}LK-DJWAK<+#c'P6^j ֱ;Nmd7վ8>/,ƭϥJwAXnuY+0]r0D':872*tnm! 1As+XXmNc] A9i aVs%- DY̒,qlvk PVOT[t͐Km3`vaGcZddF%jxkä'fjP(f蠨K4qI%OS Jm!'ЄpG xFb\:lZ`NOi6Ч:S1I.m&\Ƚ\xlL{[`3Y~JEDY36?Z= q.7 :qNC%i-yuj67]Bܵ%TI#);nELn:qtĆ3ZѰ]5G˼kHqOxtvprefs.merged'. use strict; my %wanted; while (<>) { s/\#.*//; s/^\s+//; s/\s+$//; next if $_ eq ''; if (/^(never|no|yes|always): (.+)$/) { my ($pref, $prog) = ($1, $2); $wanted{$prog} = $pref; } else { die "$ARGV:$.: bad line (remnant is $_)\n" } } foreach (sort keys %wanted) { my $pref = $wanted{$_}; print "$pref: $_\n"; } xmltv-1.4.0/choose/tv_pick/tv_pick_cgi000077500000000000000000000421361500074233200200160ustar00rootroot00000000000000#!/usr/bin/perl -w # # FIXME -T should be on but Date::Manip is currently broken under # taint mode. # # tv_pick.cgi # # Web page for the user to pick which programmes he wants to watch. # # The idea is to get TV listings for the next few days and store them # as XML in the file $LISTINGS. Then 'run' this program (install it # as a CGI script and view it in a web browser, or use Lynx's # CGI emulation) to pick which programmes you want to watch. # # Your preferences will be stored in the file $PREFS_FILE, and if a # programme title is listed in there, you won't be asked about it. So # although you may get hundreds of programmes to wade through the # first time, the second time round most of them will be listed in the # preferences file and you'll be asked only about new ones. # # So to use this CGI script to plan your TV viewing, here's what # you'll typically need to do: # # - Get listings for the next few days using the appropriate backend, # for example if you want British listings do: # # % tv_grab_uk_rt >tv.xml # # - Optionally, filter these listings to remove programmes which have # already been broadcast: # # % filter_shown tmp; mv tmp tv.xml # # - Install this file as a CGI script, and make sure that the # Configuration section below points to the correct filenames. # # - View the page from a web browser, and choose your preferences for # the shows listed. If you choose 'never' or 'always' as your # preference, you won't be asked about that programme ever again, so # 'no' or 'yes' would be a more cautious choice, since that will mean # you are asked again next time. # # - Submit the form and go on to the next page. Repeat until you have # got to the end of the listings ('Finished'). You can now download # an XMLTV file with the programmes you want to watch. You might want # to print out this XML file: # # % tv_to_latex towatch.tex # % latex towatch.tex # % dvips towatch.dvi # % lpr towatch.ps # # - Also look at $PREFS_FILE to see all the programmes you have # killfiled (including those you 'always' want to see without # prompting). This list can only get bigger, there's currently no way # to unkill a programme except by editing the file by hand. # # The first time you do this, you might find that you accidentally say # 'never' to a programme you wanted to watch. So it would be best to # print out a full copy of the TV listings from tv.xml and # double-check that everything you want is listed in towatch.xml. # Remember, once you've said 'never' to watch a programme, it becomes # as if it does not exist at all! # # -- Ed Avis, ed@membled.com # # If taint checking were turned on (which used to be the case, and is # planned for future versions) we'd need these lines. # # Keep the taint checking happy (the Cwd module runs pwd(1)) #$ENV{PATH} = '/bin:/usr/bin'; # # The PERL5LIB environment variable is ignored, so you might need #use lib '.'; use strict; use XMLTV qw; use Fcntl ':flock'; use Date::Manip; use File::Copy; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); $Log::TraceMessages::CGI = 1; } } sub ordinate { for ($_[0]) { /1$/ && return $_ . 'st'; /2$/ && return $_ . 'nd'; /3$/ && return $_ . 'rd'; return $_ . 'th'; } } # Load CGI last of all so that harmless failures in loading # not-really-needed modules don't produce errors. # use CGI qw<:standard -newstyle_urls>; use CGI::Carp qw; BEGIN { carpout(\*STDOUT) } ######## # Configuration # Maximum number of programmes to display in a single page. my $CHUNK_SIZE = 100; # Input file containing all TV listings. my $LISTINGS = 'tv.xml'; # Scratch file for storage between requests (this should really be # done with form data or cookies). # my $TOWATCH = 'towatch.tmp'; # Final output file my $OUTPUT = 'towatch.xml'; # Input file containing preferences (killfiled programmes, etc). my $PREFS_FILE = 'tvprefs'; # Preferred languages - if information is available in several # languages, the ones in this list are used if possible. List in # order of preference. Passed to best_name(). # # FIXME should find this out from HTTP headers. # my @PREF_LANGS; # Hopefully the environment variable $LANG will be set my $el = $ENV{LANG}; if (defined $el and $el =~ /\S/) { $el =~ s/\..+$//; # remove character set @PREF_LANGS = ($el); } else { @PREF_LANGS = ('en'); # change for your language - or just set $LANG } ######## # End of configuration # Prototype declarations sub store_prefs($$); sub display_form($); sub print_date_for($;$); sub clumpidx_to_english($); sub download_xml(); # Load data into globals $data and @programmes. my $data = XMLTV::parsefile($LISTINGS); my $encoding = $data->[0]; my @programmes = @{$data->[3]}; if (url_param('download')) { download_xml(); exit(); } # Newer versions of CGI.pm have support for stuff. # But for the moment, we'll keep compatibility with older ones. # # We assume the encoding used for listings data is a superset of # ASCII. # print header({ expires => 'now', 'Content-Type' => "text/html; charset=$encoding" }); print < TV listings END ; # %wanted # # Does the user wish to watch a programme? # # Maps title to: # undef - this programme is not known # 'never' - no, the user never watches this programme # 'no' - probably not, but ask # 'yes' - probably, but ask # 'always' - yes, the user always watches this programme # # Read in from the file $PREFS_FILE. # my %wanted = (); # Open for 'appending' - but really we just want to create an empty # file if needed. # open(PREFS, "+>>$PREFS_FILE") or die "cannot open $PREFS_FILE: $!"; flock(PREFS, LOCK_SH); seek PREFS, 0, 0; while () { s/\#.*//; s/^\s+//; s/\s+$//; next if $_ eq ''; # t("got line from $PREFS_FILE: " . d($_)); if (/^(never|no|yes|always): (.+)$/) { my ($pref, $prog) = ($1, $2); $wanted{$prog} = $pref; } else { die "$PREFS_FILE:$.: bad line (remnant is $_)\n" } } #t('\%wanted=' . d(\%wanted)); my ($skip, $next) = (url_param('skip'), url_param('next')); foreach ($skip, $next) { die "bad URL parameter $_" if defined and tr/0-9//c; } #t('$skip=' . d($skip) . ', $next=', d($next)); if (defined $skip and defined $next) { # Must be that the user has submitted some preferences. store_prefs($skip, $next); } elsif (defined $skip and not defined $next) { # This is one of the form pages, skipping some programmes already # seen. # close PREFS; display_form($skip); } elsif (not defined $skip and not defined $next) { # Initial page, corresponding to skip=0. if (-e $TOWATCH) { if (-M _ < -M $LISTINGS) { print p <>$TOWATCH") or die "cannot append to $TOWATCH: $!"; print TOWATCH <{title})->[0]; print "$title: $val
\n"; my $found = 0; foreach (qw[never no yes always]) { if ($val eq $_) { $wanted{$title} = $val; $found = 1; last; } } die "bad preference '$val' for prog$i" unless $found; } } # Update $PREFS_FILE with preferences. 'yes' or 'no' preferences # are still worth storing because they let us pick the default # radio button next time. # copy($PREFS_FILE, "$PREFS_FILE.old") or die "cannot copy $PREFS_FILE to $PREFS_FILE.old: $!"; flock(PREFS, LOCK_EX); truncate PREFS, 0 or die "cannot truncate $PREFS_FILE: $!"; print PREFS <>$TOWATCH") or die "cannot append to $TOWATCH: $!"; flock(TOWATCH, LOCK_EX); for (my $i = $skip; $i < $next; $i++) { my $val = param("prog$i"); my $title = best_name(\@PREF_LANGS, $programmes[$i]->{title})->[0]; if ((defined $wanted{$title} and $wanted{$title} eq 'always') or (defined $val and $val eq 'yes') ) { print TOWATCH "$LISTINGS/$i\n"; print br(), "Planning to watch $title\n"; } } close TOWATCH; print p(strong("List of programme numbers to watch added to $TOWATCH")); my $url = url(-relative => 1); if ($next >= @programmes) { write_output(); print p <an XML file of the programmes to watch. END ; } else { print a({ href => "$url?skip=$next" }, "Next page"); } print end_html(); exit(); } # display_form() # # Parameters: # number of programmes to skip at start of @programmes # sub display_form($) { die 'usage: display_form(skip)' if @_ != 1; my $skip = shift; my @nums_to_show = (); my $i; for ($i = $skip; $i < @programmes and @nums_to_show < $CHUNK_SIZE; $i++ ) { my $prog = $programmes[$i]; my $title = best_name(\@PREF_LANGS, $prog->{title})->[0]; for ($wanted{$title}) { if (not defined or $_ eq 'no' or $_ eq 'yes') { push @nums_to_show, $i; } elsif ($_ eq 'never' or $_ eq 'always') { # Don't bother the user with this programme } else { die } } } # Now actually print the things, we had to leave it until now # because we didn't know what the new 'skip' would be. # print start_form(-action => url(-relative => 1) . "?skip=$skip;next=$i"); print '', "\n"; my $prev; foreach my $n (@nums_to_show) { my %h = %{$programmes[$n]}; my ($start, $stop, $channel) = @h{qw(start stop channel)}; $stop = '' if not defined $stop; my $title = best_name(\@PREF_LANGS, $h{title})->[0]; my $display_title = $title; $display_title .= " ($h{date})" if defined $h{date}; my $category = best_name(\@PREF_LANGS, $h{category})->[0] if $h{category}; my $sub_title = best_name(\@PREF_LANGS, $h{'sub-title'})->[0] if $h{'sub-title'}; my $desc = best_name(\@PREF_LANGS, $h{desc})->[0] if $h{desc}; if (defined $prev) { print_date_for(\%h, $prev); } else { print_date_for(\%h); } print "\n"; print "\n"; my $default; for ($wanted{$title}) { if (not defined) { $default = 'never'; # Pessmistic! } elsif ($_ eq 'yes' or $_ eq 'no') { $default = $_; } else { die "bad pref for $title: $wanted{$title}"; } } foreach (qw) { print "\n"; } print "\n"; $prev = \%h; } print "
\n"; print "$display_title\n"; print "", ucfirst($category), "\n" if defined $category; print "
$sub_title\n" if defined $sub_title; print "

\n$desc\n

\n" if defined $desc; if ($h{credits}) { # XMLTV.pm returns a hash mapping job to list of people. our %credits; local *credits = $h{credits}; print "\n"; foreach (sort keys %credits) { print ''; print td({ class => 'job' }, ucfirst($_)); print join('', map { td({ class => 'person' }, $_) } @{$credits{$_}} ); print "\n"; } print "
\n"; } if (defined $h{clumpidx}) { print "", clumpidx_to_english($h{clumpidx}), "
\n"; } t d \%h; print "		\n"; my $checked = ($_ eq $default) ? 'checked' : ''; print qq[$_\n]; print "
\n"; print submit(); print end_form(); print end_html(); } # print_date_for() # # Print the date for a programme as part of the form, so that the # reader will have some idea of when the programmes will be shown. # # Printing the date ends the current table, prints the date, and then # starts a new table. But it won't happen unless it is needed, ie the # date has changed since the previous programme. # # Parameters: # (ref to) programme to print # (optional) (ref to) previous programme # # If the previous programme is not given, the date will always be # printed. # # Printing the date also (at least ATM) ends the current HTML table # and begins a new one after the date. # sub print_date_for($;$) { # local $Log::TraceMessages::On = 0; die 'usage: print_date_for(programme, [prev programme])' unless 1 <= @_ and @_ < 3; my ($prog, $prev) = @_; t('$prog=' . d($prog)); t('$prev=' . d($prev)); my $DAY_FMT = '%A'; # roughly as for date(1) my $day = UnixDate($prog->{start}, $DAY_FMT); my $prev_day = defined $prev ? UnixDate($prev->{start}, $DAY_FMT) : undef; t('$day=' . d($day)); t('$prev_day=' . d($prev_day)); if ((not defined $prev_day) or ($day ne $prev_day)) { print "\n"; print h1($day); print '', "\n"; } } # clumpidx_to_english() # # Convert a series-episode-part number like '2/3 . 4/10 . 0/2' to an # English description like '3rd series of 3; 5th episode of 10; 1st # part of 2'. # sub clumpidx_to_english($) { local $_ = shift; s/\s+//g; my @bits = split /\./; unshift @bits, undef until @bits >= 3; my ($series, $episode, $part) = @bits; sub of($$) { my $name = shift; local $_ = shift; if (m!^(\d+)/(\d+)$!) { return ordinate($1 + 1) . " $name of $2"; } elsif (m!^\d+$!) { return ordinate($_ + 1); } else { die "bad number-of-number $_"; } } my @r; push @r, of('series', $series) if defined $series; push @r, of('episode', $part) if defined $episode; push @r, of('part', $part) if defined $part; return join('; ', @r); } # write_output() # # After all the programmes have been picked, convert the 'towatch' # file (which is really just a list of numbers) to an XML document for # the user to download. # sub write_output() { die 'usage: write_output()' if @_; # Find programme numbers to keep my %nums; open(TOWATCH, $TOWATCH) or die "cannot open $TOWATCH: $!"; while () { s/\#.*//; s/^\s*//; s/\s*$//; next if $_ eq ''; m!^\Q$LISTINGS\E/(\d+)$! or die "$TOWATCH:$.: bad line $_"; $nums{$1}++ && die "$TOWATCH:$.: already seen number $1"; } # We read the original XML file and weed out elements. my @new_programmes; for (my $i = 0; $i < @programmes; $i++) { push @new_programmes, $programmes[$i] if $nums{$i}; } my $output = new IO::File(">$OUTPUT"); die "cannot write to $OUTPUT" if not $output; XMLTV::write_data([ $data->[0], $data->[1], $data->[2], \@new_programmes ], OUTPUT => $output); } xmltv-1.4.0/doc/000077500000000000000000000000001500074233200134275ustar00rootroot00000000000000xmltv-1.4.0/doc/COPYING000066400000000000000000000432541500074233200144720ustar00rootroot00000000000000 GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Lesser General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. , 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. xmltv-1.4.0/doc/QuickStart000066400000000000000000000244541500074233200154550ustar00rootroot00000000000000To get the most out of the XMLTV package you will probably want to use a grabber to get some listings, and then perhaps pipe them through some filter programs and then use one of the two chooser programs to select your viewing for the next week. * Grabbers These are programs which retrieve TV listings data and output them in XMLTV format. Grabbers are included for the following countries: Finland tv_grab_fi, tv_grab_fi_sv France tv_grab_fr Hungary tv_grab_huro Iceland tv_grab_is Italy tv_grab_it, tv_grab_it_dvb Portugal tv_grab_pt_vodafone Switzerland tv_grab_ch_search US and Canada tv_grab_na_dd, tv_grab_na_tvmedia Grabbers are included for the following larger geographic areas: Europe/US/Canda/Latin America/Caribbean tv_grab_zz_sdjson, tv_grab_zz_sdjson_sqlite Contributions from other countries are welcome, of course. Most grabbers have a configuration stage: once you've decided what grabber to use, first run it with --configure, for example: % tv_grab_fi --configure If the grabber does not need configuration it will tell you so, otherwise you will be able to choose what channels to download listings for (fewer is usually faster). By default, grabbers print listings to standard output (STDOUT) but you will probably want to redirect output to a file, for example: % tv_grab_fi --output fi.xml The default is to grab listings for the longest time period possible (one or two weeks) but to speed up downloading you may want to specify '--days 1' for one day only. * Filters There are some Unix-style filter programs which perform processing on XMLTV listings files. In particular the grabber output is not guaranteed to be in any particular order and you probably want to sort it. Each filter is normally run with both input and output redirected, for example: % tv_sort fi_sorted.xml Please see the programs' manual pages for more detailed instructions; also each supports the --help convention to print a usage message. * * tv_sort: sort listings into date order Programmes are sorted based on their start time and date, and if those are the same then their end time. tv_sort also adds in the 'stop time' field of a programme, if it is missing, by looking at when the next thing starts on the same channel. It also performs some sanity checks such as checking that two programmes on the same channel do not overlap. * * tv_grep: filter listings by regexp matching This is a tool to extract all programmes or channels that match a given regular expression. You can use tv_grep as a quick and dirty way to filter for your favourite shows, but it is easier to use tv_pick_cgi or tv_check for that. The simple usage is with a single regular expression, for example 'tv_grep Countdown'. It is also possible to match individual fields of a programme, for example: % tv_grep --ignore-case --category drama How effective this is depends on how well the listings data is organized into different fields. There are also some tests which don't quite count as 'grepping': --on-after TIME will remove all programmes which would already have finished at time TIME. So 'tv_grep --on-after now' will filter out shows you have already missed. You can combine tests with --or and --and. * * tv_split: split listings into separate files If you want separate XML files for different time periods or channels, pipe the listings into tv_split with a suitable template for the output filename. For example: % tv_split --output %channel-%Y%m%d.xml will generate one output file for each channel/date combination. * * tv_extractinfo_en: extract info from English-language programmes Often the listings source being grabbed won't be as machine-readable as it could be. For example instead of storing the director of a film in a separate field, it may simply write 'Directed by William Wyler'. Or separate programmes on at different times may be combined into a single entry. tv_extractinfo_en attempts to sort out this mess using heuristics and regular expressions that match English-language descriptions. It was written for the UK listings, and some of the things corrected (such as multipart programmes) are specific to that source. But it should work for any anglophone listings source. The North American programme descriptions are too terse to extract much information, but tv_extractinfo_en occasionally manages to get the name of a presenter. Lots of details that could be handled are not, because any heuristic for this kind of thing must occasionally get the wrong answer. It's more important to minimize the number of false positives. You should run a week's listings through tv_extractinfo_en and diff the results against the original to decide whether you trust the program enough to use it. * * tv_imdb: enrich listings with data from the Internet Movie Database tv_imdb is a filter program which tries to look up programmes in the publicly available imdb data and add information to them before writing them out again. At present it requires you to download some (rather large) data files from the imdb ftp site. See the manual page for more details. * * tv_tmdb: enrich listings with data from The Movie Database This filter program tries to look up programmes in the publicly available tmdb data and add information to them before writing them out again. Registration with tmdb is required to obtain a license key. See the manual page for more details. * * tv_to_latex: convert listings to LaTeX source To print out listings in a concise format run them through tv_to_latex and then LaTeX, for example: % tv_to_latex tv.tex % latex tv.tex % dvips tv.dvi and then print tv.ps if it hasn't printed already. You may want to do this on the full sorted listings for a complete TV guide (which will run to many pages), or on the output from tv_pick_cgi or tv_grep for a personal TV guide. Tools exist to convert XMLTV data to HTML or to PDFs, but they are not included in this release. * * tv_to_text: convert TV listings to plain text This filter generates a plain text summary of listings. The information included in the summary is the same as with tv_to_latex. * Choosers The real point of getting a TV guide in machine-readable form is to let the computer do the work of looking through it finding things for you to watch. Two programs are distributed to do this. tv_check is a GUI-based program where you select some shows and then generate a printed report which flags any deviations from the normal weekly schedule. tv_pick_cgi is a Web-based program which takes a different approach: it shows you all the programmes that are on and asks what to do with each one, then generates a personal TV guide with the shows chosen. However preferences are remembered for the next run, so next time you'll only be asked about new programmes. See README.tv_check for instructions on using tv_check. To use tv_pick_cgi, you will need an environment for running CGI scripts. If you're lucky enough to have a web server handy, copy the file as tv_pick.cgi to a directory somewhere, copy XMLTV.pm and the XMLTV/ directory (which contains more Perl modules) to the same place, and copy a listings file there as tv.xml. It is best for the listings to be sorted. If you have no web server, you can still run tv_pick_cgi using the 'CGI emulation' mode of the Lynx text-based web browser. Run 'lynx lynxcgi:tv_pick_cgi'. This assumes your Lynx has the CGI emulation compiled in - if not, suggest it to your vendor. Quick guide to Lynx: move between radio buttons using up-arrow and down-arrow. Press right-arrow to select a radio button, to press an on-screen button like 'Submit' move the highlight to it and press Enter. You should now be presented with a list of all programmes you haven't seen before (on the first run, this will be everything). For each programme there are four choices: never - no, I don't want to watch it, and don't ask me about programmes with this title ever again. no - I won't watch it this time, but ask me again next time. yes - I might watch it (put it in the output listings), but ask me again next time. always - whenever a programme with this title appears, always put it in the output without asking. The default option for unrecognized titles is 'never', reflecting the fact that most things on TV are rubbish. Because something marked as 'never' is effectively censored from all future sessions with tv_pick_cgi, you should be sure to change this for any programme you might want to watch in the future. Saying 'no' is a safe choice for things you don't want to watch. when you've chosen your preferences for everything on the page, press 'Submit' and a page will appear confirming your preferences and listing which of the programmes will appear in the output ('planning to watch'). There will probably be several pages of listings, so go to 'Next page' and repeat as necessary. Take comfort in the thought that you'll never have to deal with most of these shows ever again :-). At the end a personal listings file towatch.xml is generated, which you can download with your browser if you want, and your preferences are stored for next time in the file tvprefs. It is worth checking this file after your first use of pick_cgi in case you accidentally marked something as 'never'. * Using the tools together It's probably easiest, once you get used to the tools, to run them together in a pipeline. For example: % tv_grab_fi | tv_sort | tv_extractinfo_en \ | tv_sort | tv_grep --on-after now >guide.xml This gets listings, sorts them, munges them through tv_extractinfo_en to see what it finds (in this case it will probably break up 'Open University' into subprogrammes, among other things), sorts again and filters out those programmes already missed. The first sorting is needed to add stop times to programmes to give tv_extractinfo_en the most information to work on; the second sorting because tv_extractinfo_en does not necessarily produce fully sorted output. Most of the XMLTV tools do not strictly require that the input be sorted, but they tend to work a bit better if it is. Then run 'tv_check --scan' or use tv_pick_cgi to generate a text report or a personal TV listing. xmltv-1.4.0/doc/README-Windows.md000066400000000000000000000140171500074233200163410ustar00rootroot00000000000000# XMLTV 1.4.0 Windows Release ## Table of Contents - [XMLTV](#xmltv) * [Description](#description) * [64-bit and 32-bit Builds](#64-bit-and-32-bit-builds) * [Changes](#changes) * [Installation](#installation) * [General Notes](#general-notes) * [Known Issues](#known-issues) + [Proxy Servers](#proxy-servers) * [License](#license) * [Authors](#authors) * [Resources](#resources) ## Description The XMLTV project provides a suite of software to gather television listings, process listings data, and help organize your TV viewing. XMLTV listings use a mature XML file format for storing TV listings, which is defined and documented in the [XMLTV DTD](xmltv.dtd). In addition to the many "grabbers" that provide listings for large parts of the world, there are also several tools to process and filter these listings. Please see our [QuickStart](doc/QuickStart) documentation for details on what each program does. This is a release of the software as a single Windows binary (xmltv.exe), generated from the Perl source code linked from . ## 64-bit and 32-bit Builds Please keep an eye on our [releases page](https://github.com/XMLTV/xmltv/releases) for 64-bit and 32-bit builds of our current releases when available. All current releases of XMLTV for Windows are built for 64-bit and 32-bit Windows by default. Download the one relevant to your version of Windows - see [How Do I Know if I’m Running 32-bit or 64-bit Windows?](https://www.howtogeek.com/21726/) - or, if in doubt, simply download the 32-bit version (as it will probably work in all cases). If using the 32-bit version then substitute xmltv32.exe for xmltv.exe in the following instructions. To build and run a current version yourself you will need to run Cygwin, or Strawberry Perl. [Some instructions are available in the XMLTV Wiki](http://wiki.xmltv.org/index.php/XMLTVWindowsBuild) ## Changes Major Changes in this release | Grabber | Change | | ---------- | --------- | | tv_grab_fi_sv | disable grabber (upstream site changes) | | tv_grab_fr | disable grabber (upstream Terms and Conditions) | | tv_grab_huro | disable Romanian listings (upstream site gone) | | tv_grab_it | disable grabber (upstream site changes) | | tv_grab_fi | improvements to episode/season handling and upstream channel availability | | tv_grab_pt_vodafone | migrate to new upstream API and improvements to quality of programme data | | tv_grab_uk_freeview | improvements to programme retrieval and handling, web page cache is now used by default | | tv_grab_zz_sdjson | improvements to episode/season handling | | tv_grab_zz_sdjson_sqlite | adds deaf-signed subtitles element support and improves database handling | Please see the git log for full details of changes in this release. ## Installation There is no installer - unpack the zipfile into a directory such as C:\xmltv. If you are reading this you've probably already done that. All the different programs are combined into a single executable. For example, instead of running 'tv_grab_na --days 2 >na.xml' you would run ```bash c:\xmltv\xmltv.exe tv_grab_na_dd --days 2 --output a.xml ``` Apart from the extra 'xmltv.exe' at the front of each command line, the usage should be the same as the Unix version. Some programs make use of a "share" directory. That directory is assumed be named "share" at the same location as the exe. If you just keep everything where you unzipped it, everything should be fine. If you must move xmltv.exe, you may need to specify a --share option with some programs. xmltv.exe will try and guess a timezone. This usually works fine. If it doesn't, you can set a TZ variable just like on Unix. ## General Notes Spaces in filenames may cause problems with some programs. Directories with spaces (i.e. C:\program files\xmltv) are not recommended. C:\xmltv is better. Some of the programs allow you pass a date format on the command line. This uses % followed by a letter to specify a component of a date, for example %Y gives a four digit year. This can cause problems on windows since % is used as a shell escape character. To get around this, use %% to pass a % to the application. (ex. %%Y%%M ) If you *DO* want to insert a shell variable, you can do so by surrounding it with percents. (ex %HOME% ) ## Known Issues The first time xmltv.exe is run, it can take a while... up to 5 minutes as it prepares some files in %TEMP%. This is normal. Subsequent runs are fast. Due to prerequisite problems, EXE support is not currently available for tv_grab_is and tv_grab_it_dvb, If you need one of those you'll need to install Perl and the necessary modules and use the full distribution. ## Proxy Servers Proxy server support is provided by the LWP modules. You can define a proxy server via the HTTP_PROXY environment variable: ```bash http_proxy=http://somehost.somedomain:port ``` For more information, see the [LWP::UserAgent documentation](https://metacpan.org/pod/LWP::UserAgent#PROXY-ATTRIBUTES) ## License XMLTV is free software, distributed under the GNU General Public License, version 2. Please see [COPYING](COPYING) for more details. ## Authors There have been many contributors to XMLTV. Where possible they are credited in individual source files and in the [authors](authors.txt) mapping file. ## Resources ### GitHub Our [GitHub project](https://github.com/XMLTV/xmltv) contains all source code, issues and Pull Requests. ### Project Wiki We have a project [web page and wiki](http://www.xmltv.org) ### Mailing Lists We run the following mailing lists: - [xmltv-users](https://sourceforge.net/projects/xmltv/lists/xmltv-users): for users to ask questions and report problems with XMLTV software - [xmltv-devel](https://sourceforge.net/projects/xmltv/lists/xmltv-devel): for development discussion and support - [xmltv-announce](https://sourceforge.net/projects/xmltv/lists/xmltv-announce): announcements of new XMLTV releases -- Nick Morrott, knowledgejunkie@gmail.com, 2025-04-17 xmltv-1.4.0/doc/code/000077500000000000000000000000001500074233200143415ustar00rootroot00000000000000xmltv-1.4.0/doc/code/coding_standards000066400000000000000000000074041500074233200175770ustar00rootroot00000000000000* Coding standards for XMLTV Mostly the coding standards are: follow the existing convention. There is code written by a variety of authors and since it is mostly self-contained scripts there wasn't any great effort to enforce a uniform style. So if you are changing a file just conform to the style that's already there. New code should use the preferred style for the language in which it is written. In C use the style given by K&R (second edition), in C++ imitate the examples in 'The C++ Programming Language' (third edition), in Java copy Sun's coding style and in Perl read the perlstyle(1) manual page. (This doesn't 100% match my own preferences, but I think it is a good basis for a common standard among several contributors.) For grabbing listings from websites you need to write very defensively and handle every possible condition. For example, if ($role eq 'actor') { # handle actor } elsif ($role eq 'director') { # handle director } else { warn "unknown role $role"; } Even if the third case has never happened, you should detect it and warn about it. The format of a website will undergo small and not-so-small changes without warning. Another way of phrasing this principle is *do not throw away any information*. In the worst case you should print data that isn't understood to stderr so that the user sees it; nothing should be thrown away. (Okay so you will have to strip away lots of banner ads, navigation bars and other cruft from typical pages - but in the listings themselves try to do something meaningful with every last character.) A useful technique is to store details in a hash and then when handling the hash to make a copy and delete keys from it. At the end, if there are any keys left, there's something you didn't handle. my %h = %in; # make a copy if (defined(my $val = delete $p{stop})) { # handle 'stop' attribute } if (defined(my $val = delete $p{post})) { # handle 'post' attribute } # I think that's all, just check... foreach (keys %h) { warn "unknown key in hash: $_"; } Most of this file remains to be written, until then follow the general advice above. * git log messages For the XMLTV changelog messages, I have been using the style of describing what I did in the past simple tense: for example, 'Added a flag to stop excessive grunking when fully formed pompoms are not available.' as opposed to 'Add a flag...' For consistency, I think we should keep the same style. So please when committing to git write meaningful log messages and describe what you did and why, in the past tense. For small changes you can describe the new program behaviour, and for this use the present tense: 'Read the whole file first before checking for markers.' In other words, use the present tense to describe what the program now _does_; use the past tense to describe what you just _did_. * Subroutines (and functions, and methods) All subroutines should have a comment at the top saying _what_ the routine does and how to call it. Mostly it will suffice to describe the meaning of the different parameters and the return value. If you want to describe _how_ a subroutine works, do so with comments inside the subroutine body. Using documentation systems (such as Perldoc and Javadoc) is necessary if you're building a library which can be installed by itself, but otherwise it's not strictly required. Having said that, we probably should be using Perldoc more than we are currently. * DTD Additions to the DTD are mainly based on three criteria: - does a listings source being scraped include this data? - is it something you'd expect to find in printed TV listings? - does any application need it? The more of these are true, the more likely something is to go in. xmltv-1.4.0/doc/exe_build.html000066400000000000000000000007501500074233200162570ustar00rootroot00000000000000

Instructions for building XMLTV from source on Windows is in the Wiki: http://wiki.xmltv.org/index.php/XMLTVWindowsBuild

Instructions for building XMLTV.EXE using Par::Packer is in the Wiki: http://wiki.xmltv.org/index.php/XMLTVexeBuild

xmltv-1.4.0/filter/000077500000000000000000000000001500074233200141475ustar00rootroot00000000000000xmltv-1.4.0/filter/Grep.pm000066400000000000000000000105131500074233200154020ustar00rootroot00000000000000# This is intended mostly as a helper library for tv_grep and not for # general purpose use (yet). package XMLTV::Grep; use strict; use XMLTV; use base 'Exporter'; our @EXPORT_OK; @EXPORT_OK = qw(get_matcher); my %key_type = %{XMLTV::list_programme_keys()}; # Parameters: # key found in programme hashes # ignore-case flag # # Returns: # extra argument type needed to filter on this key: # undef: no extra argument required # 'regexp': extra argument should be regexp # 'empty': extra argument must be the empty string, and is ignored # # subroutine which may take an argument (depending on whether # argument type is 'regexp'), and matches a programme hash in $_. # sub get_matcher( $$ ) { my ($key, $ignore_case) = @_; my ($handler, $mult) = @{$key_type{$key}}; if ($handler eq 'presence') { die "bad multiplicity $mult for 'presence'" if $mult ne '?'; return [ undef, sub { exists $_->{$key} } ]; } elsif ($handler eq 'scalar') { if ($mult eq '?') { return [ 'regexp', sub { my $regexp = shift; return 0 if not exists $_->{$key}; return 1 if $regexp eq ''; if ($ignore_case) { return $_->{$key} =~ /$regexp/i; } else { return $_->{$key} =~ /$regexp/; } } ]; } elsif ($mult eq '1') { return [ 'regexp', sub { my $regexp = shift; die if not exists $_->{$key}; return 1 if $regexp eq ''; if ($ignore_case) { return $_->{$key} =~ /$regexp/i; } else { return $_->{$key} =~ /$regexp/; } } ]; } elsif ($mult eq '*') { return [ 'regexp', sub { my $regexp = shift; # It is possible (though unusual) for the key # to exist but be an empty list. # return 0 if not exists $_->{$key} or not @{$_->{$key}}; return 1 if $regexp eq ''; foreach (@{$_->{$key}}) { return 1 if ($ignore_case ? /$regexp/i : /$regexp/); } return 0; } ]; } elsif ($mult eq '+') { return [ 'regexp', sub { my $regexp = shift; die if not @{$_->{$key}}; return 1 if $regexp eq ''; foreach (@{$_->{$key}}) { return 1 if ($ignore_case ? /$regexp/i : /$regexp/); } return 0; } ]; } else { die } } elsif ($handler =~ m!^with-lang(?:/[a-z]*)?$!) { if ($mult eq '?') { return [ 'regexp', sub { my $regexp = shift; return 0 if not exists $_->{$key}; return 1 if $regexp eq ''; for ($_->{$key}->[0]) { return 0 if not defined; if ($ignore_case) { return /$regexp/i; } else { return /$regexp/; } } } ]; } elsif ($mult eq '1') { return [ 'regexp', sub { my $regexp = shift; die if not exists $_->{$key}; return 1 if $regexp eq ''; for ($_->{$key}->[0]) { if (not defined) { warn "undef text for $key"; return 0; } if ($ignore_case) { return /$regexp/i; } else { return /$regexp/; } } } ]; } elsif ($mult eq '*') { return [ 'regexp', sub { my $regexp = shift; return 0 if not exists $_->{$key} or not @{$_->{$key}}; return 1 if $regexp eq ''; foreach (map { $_->[0] } @{$_->{$key}}) { if (not defined) { warn "undef text for $key"; next; } return 1 if ($ignore_case ? /$regexp/i : /$regexp/); } return 0; } ]; } elsif ($mult eq '+') { return [ 'regexp', sub { my $regexp = shift; die if not @{$_->{$key}}; return 1 if $regexp eq ''; foreach (map { $_->[0] } @{$_->{$key}}) { if (not defined) { warn "undef text for $key"; next; } return 1 if ($ignore_case ? /$regexp/i : /$regexp/); } return 0; } ]; } else { die } } elsif ($handler eq 'icon' or $handler eq 'credits' or $handler eq 'length' # TODO or $handler eq 'episode-num' # TODO or $handler eq 'video' or $handler eq 'audio' or $handler eq 'previously-shown' or $handler eq 'subtitles' or $handler eq 'rating' # TODO or $handler eq 'star-rating' # TODO or $handler eq 'review' # TODO or $handler eq 'url' # TODO or $handler eq 'image' # TODO ) { # Cannot query on this except for presence. But empty string # argument for future expansion. # return [ 'empty', sub { exists $_->{$key} } ]; } else { die } } 1; xmltv-1.4.0/filter/augment/000077500000000000000000000000001500074233200156075ustar00rootroot00000000000000xmltv-1.4.0/filter/augment/augment.conf000066400000000000000000000114731500074233200201240ustar00rootroot00000000000000# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_tvguide supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # To enable a rule set it 1, to disable set it to 0. # # Note: if the 'enable_all_options' is set to 1 then the following lines # will be ignored (i.e. they will be run as if they were set to 1). # # Rule #A1 - Remove "New $title :" from . remove_duplicated_new_title_in_ep = 1 # Rule #A2 - Remove duplicated programme title *and* episode from . remove_duplicated_title_and_ep_in_ep = 1 # Rule #A3 - Remove duplicated programme title from . remove_duplicated_title_in_ep = 1 # Rule #A4 - update_premiere_repeat_flags_from_desc = 1 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 1 # Update existing programme numbering if extracted from title/episode/part update_existing_numbering = 0 # Rule #A5.1 - Extract series/episode numbering found in . # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 1 # Rule #A5.4 - If no <sub-title> then make one from "part" numbers. # (requires #A5 enabled) make_episode_from_part_numbers = 1 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # # You could disable a rule simply by not having any entries in the rules file, # but this provides a convenient way to switch on/off any particular rule. # # The corollary is, to action an particular rule type, you must have some # relevant rules of the type in the rules file, *and* you must enable the # option below (or enable the 'all' option above) # # Note: if the 'enable_all_options' is set to 1 then the following lines # will be ignored (i.e. they will be run as if they were set to 1). # # Rule #user - Process programme against user-defined fixups # If you disable this option (by setting to 0) then none of the user rules # will be actioned (irrespective of whether you have them set to 1 or 0) # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 1 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 1 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 1 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 1 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 1 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 1 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 1 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 1 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 1 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 1 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 1 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 1 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 1 # Rule #14 - Replace specified categories with another value process_translate_genres = 1 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 1 # Rule #16 - Remove episode numbering from a programme title process_remove_numbering_from_programmes = 1 �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/augment/augment.rules������������������������������������������������������������0000664�0000000�0000000�00000026504�15000742332�0020332�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample "rules" file for use with tv_augment # ############################################################################### # # This file contains the rules used by XMLTV::Augment. # # The objective is to fix errors and inconsistencies in the incoming data from # a grabber, and to enhance the programme xml where certain data are missing. # # For example: # Some programme entries in the listings data may contain subtitle/episode # information in the title field, others may contain the programme title # and subtitle reversed, and yet more may contain 'episode' information that # should be in the episode-num field. # # Rules are divided into a number of 'types' according to what they do. # Individual types (rule sets) can be switched off in the augmentation config # file. # # Matching is usually (but not exclusively) done by comparing the incoming # <title> against the title specified in the rule below. # # A 'rule' definition consists of the rule 'type' separated rom the rule # content by a pipe character ('|'). The rule content has a variable (but fixed # for any given rule type) number of fields separated by tilde characters ('~') # # The action taken depends on the rule type: # # 1) non_title_info # The content is text that is to be removed from any programme titles where # this text occurs at the beginning of the <title> element followed by # any of : ; or , # e.g. # 1|Action Heroes Season # "Action Heroes Season: Rambo" --> "Rambo" # # 2) mixed_title_subtitle # The content is the desired title of a programme when the incoming title # contains both the programme's title *and* episode separated by : ; or - # ($title:$episode). We reassign the episode information to the <episode> # element, leaving only the programme title in the <title> element. # e.g. # 2|Blackadder II # "Blackadder II: Potato / " --> "Blackadder II / Potato" # # 3) mixed_subtitle_title # The content is the desired title of a programme when the incoming title # contains both the programme's episode *and* title separated by : ; or - # ($episode:$title). We reassign the episode information to the <episode> # element, leaving only the programme title in the <title> element. # e.g. # 3|Storyville # "Kings of Pastry: Storyville / " --> "Storyville / Kings of Pastry" # # 4) reversed_title_subtitle # The content is the desired title of a programme which is listed as the # programme's episode (i.e. the title and episode details have been # reversed). We therefore reverse the <title> and <episode> elements. # e.g. # 4|Storyville # "Kings of Pastry / Storyville" --> "Storyville / Kings of Pastry" # # 5) replacement_titles # The content contains two programme titles, separated by a # tilde (~). The first title is replaced by the second in the listings # output. # This is useful to fix inconsistent naming (e.g. "Law and Order" vs. # "Law & Order") or inconsistent grammar ("xxxx's" vs. "xxxxs'") # e.g. # 5|A Time Team Special~Time Team # "A Time Team Special / Doncaster" --> "Time Team / Doncaster" # # 6) replacement_genres # The content contains a programme title and a programme category(-ies) # (genres), separated by tilde (~). Categories can be assigned to # uncategorised programmes (which can be seen in the stats log). # (Note that *all* categories are replaced for the title.) # e.g. # 6|Antiques Roadshow~Entertainment~Arts~Shopping # "Antiques Roadshow / " category "Reality" --> # "Antiques Roadshow / " category "Entertainment" + "Arts" + "Shopping" # # 7) replacement_episodes # The content contains a programme title, an old episode value and a new # episode value, separated by tildes (~). # e.g. # 7|Time Team~Time Team Special: Doncaster~Doncaster # "Time Team / Time Team Special: Doncaster" --> "Time Team / Doncaster" # # 8) replacement_titles_episodes # The content contains an old programme title, an old episode value, a new # programme title and a new episode value. The old and new titles MUST be # given, the episode fields can be left empty but the field itself must be # present. # e.g. # 8|Top Gear USA Special~Detroit~Top Gear~USA Special # "Top Gear USA Special / Detroit" --> "Top Gear / USA Special" # # 8|Top Gear USA Special~~Top Gear~USA Special # "Top Gear USA Special / " --> "Top Gear / USA Special" # # 9) replacement_ep_from_desc # The content contains a programme title, a new episode value to update, # and a description (or at least the start of it) to match against. When # title matches incoming data and the incoming description startswith the # text given then the episode value will be replaced. # e.g. # 9|Heroes of Comedy~The Goons~The series celebrating great British # comics pays tribute to the Goons. # "Heroes of Comedy / " desc> = "The series celebrating great British # comics pays tribute to the Goons." # --> "Heroes of Comedy / The Goons" # Should be used with care; e.g. # "Heroes of Comedy / Spike Milligan" desc> = "The series celebrating # great British comics pays tribute to the Goons." # would *also* become # "Heroes of Comedy / The Goons" # this may not be what you want! # # 10) replacement_titles_desc # The content contains an old programme title, an old episode value, a new # programme title, a new episode value and the episode description (or at # least the start of it) to match against. # The old and new titles and description MUST be given, the episode fields # can be left empty but the field itself must be present. # This is useful to fix episodes where the series is unknown but can be # pre-determined from the programme's description. # e.g. # 10|Which Doctor~~Gunsmoke~Which Doctor~Festus and Doc go fishing, but # are captured by a family that is feuding with the Haggens. # "Which Doctor / " desc> = "Festus and Doc go fishing, but are captured # by a family that is feuding with the Haggens. [...]" # --> "Gunsmoke / Which Doctor" # # 11) demoted_titles # The content contains a programme 'brand' and a new title to be extracted # from subtitle field and promoted to programme title, replacing the brand # title. # In other words, if title matches, and sub-title starts with text then # remove the matching text from sub-title and move it into the title. # Any text after 'separator' (any of .,:;-) in the sub-title is preserved. # e.g. # 11|Blackadder~Blackadder II # "Blackadder / Blackadder II: Potato" --> "Blackadder II / Potato" # # 12) replacement_film_genres # The content contains a film title and a category (genre) or categories, # separated by a tilde (~). # If title matches the rule's text and the prog has category "Film" or # "Films", then use the replacement category(-ies) supplied. # Use case: some film-related programmes are incorrectly flagged as films # and should to be re-assigned to a more suitable category. # (Note ALL categories are replaced, not just "Film") # e.g. # 12|The Hobbit Special~Entertainment~Interview # "The Hobbit Special / " category "Film" + "Drama" --> # "The Hobbit Special / " category "Entertainment" + "Interview" # # 13) subtitle_remove_text # The content contains a programme title and arbitrary text to # be removed from the start/end of the programme's subtitle. If the text to # be removed precedes or follows a "separator" (any of .,:;-), the # separator is removed also. # e.g. # 13|Time Team~A Time Team Special # "Time Team / Doncaster : A Time Team Special" --> # "Time Team / Doncaster" # # 14) process_replacement_genres # The content contains a category (genre) value followed by replacement # category(-ies) separated by a tilde (~). # Use case: useful if your PVR doesn't understand some of the category # values in the incoming data; you can translate them to another value. # e.g. # 14|Adventure/War~Action Adventure~War # "The Guns of Navarone" category "Adventure/War" --> # "The Guns of Navarone" category "Action Adventure" + "War" # # 15) process_add_genres_to_channel # The content contains a channel id followed by replacement # category(-ies) separated by a tilde (~). # Use case: this can add a category if data from your supplier is always # missing; e.g. add "News" to a news channel, or "Music" to a music # vid channel. # e.g. # 15|travelchannel.co.uk~Travel # "World's Greatest Motorcycle Rides" category "Motoring" --> # "World's Greatest Motorcycle Rides" category "Motoring" + "Travel" # 15|cnbc.com~News~Business # "Investing in India" category "" --> # "Investing in India" category "News" + "Business" # You should be very careful with this one as it will add the category you # specify to EVERY programme broadcast on that channel. This may not be what # you always want (e.g. Teleshopping isn't really "music" even if it is on MTV!) # # 16) process_remove_numbering_from_programmes # The content contains a title value, followed by an optional channel # (separated by a tilde (~)). # Use case: can remove programme numbering from a specific title if it # is regularly wrong or inconsistent over time. # e.g. # 16|Bedtime Story # "CBeebies Bedtime Story" episode-num ".700." --> # "CBeebies Bedtime Story" episode-num "" # 16|CBeebies Bedtime Story~cbeebies.bbc.co.uk # "CBeebies Bedtime Story" episode-num ".700." --> # "CBeebies Bedtime Story" episode-num "" # Remember to specify the optional channel limiter if you have good # programme numbering for a given programme title on some channels but # not others. # # ############################################################################### # # Some sample rules follow; obviously you should delete these and replace with # your own! # 1|Action Heroes Season 1|Python Night 1|Western Season 2|Blackadder II 2|Comic Relief 2|Old Grey Whistle Test 3|Storyville 4|Storyville 4|Timewatch 5|A Time Team Special~Time Team 5|Cheaper By the Dozen~Cheaper by the Dozen 5|Later - with Jools Holland~Later... with Jools Holland 6|Antiques Roadshow~Entertainment~Arts~Shopping 6|Deal or No Deal~Game show 6|Men Behaving Badly~Sitcom 7|Time Team~Time Team Special: Doncaster~Doncaster 8|Top Gear USA Special~Detroit~Top Gear~USA Special 9|Heroes of Comedy~The Goons~The series celebrating great British comics pays tribute to the Goons. 10|Which Doctor~~Gunsmoke~Which Doctor~Festus and Doc go fishing, but are captured by a family that is feuding with the Haggens. 11|Blackadder~Blackadder II 11|Formula One~Live Formula One 11|Man on Earth~Man on Earth with Tony Robinson 12|Hell on Wheels~Drama 12|The Hobbit Special~Entertainment~Interview 13|Time Team~A Time Team Special 13|World's Busiest~World's Busiest 14|Adventure/War~Action Adventure~War 14|NFL~American Football 14|Soccer~Football 15|smashhits.net~Music 15|travelchannel.co.uk~Travel 16|Antiques Roadshow~bbc1.bbc.co.uk 16|Classic Antiques Roadshow # # (the sample rules shown here are in sorted order but that is not necessary # in your live file) ############################################################################### ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_augment�����������������������������������������������������������������������0000664�0000000�0000000�00000012515�15000742332�0016247�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w eval 'exec /usr/bin/perl -w -S $0 ${1+"$@"}' if 0; # not running under some shell =pod =head1 NAME tv_augment - Augment XMLTV listings files with automatic and user-defined rules. =head1 SYNOPSIS tv_augment [--rule <file>] [--config <file>] [--input <file>] [--output <file>] [--log <file>] [--nostats] [--debug <level>] tv_augment [-r <file>] [-c <file>] [-i <file>] [-o <file>] [-l <file>] [-n] [-d <level>] =head1 DESCRIPTION Augment an XMLTV xml file by applying corrections ("fixups") to programmes matching defined criteria ("rules"). Two types of rules are actioned: (i) automatic, (ii) user-defined. Automatic rules use pre-programmed input and output to modify the input programmes. E.g. removing a "title" where it is repeated in a "sub-title" (e.g. "Horizon" / "Horizon: Star Wars"), or trying to identify and extract series/episode numbers from the programme title, sub-title or description. User-defined rules use the content of a "rules" file which allows programmes matching certain user-defined criteria to be corrected/enhanced with the user data supplied (e.g. adding/changing categories for all episodes of "Horizon", or fixing misspellings in titles, etc.) (see "perldoc XMLTV::Augment" for more details) B<--input FILE> read from FILE rather than standard input. B<--output FILE> write to FILE rather than standard output. B<--rule FILE> file containing the user-defined rules. B<--config FILE> configuration file containing a list of which rules you want to run. B<--nostats> do not print the summary log of actions performed, or list of suggested fixups. B<--log FILE> output the stats to this FILE. B<--debug LEVEL> print debug info to STDERR (debug level > 3 is not likely to be of much use (it generates a lot of output)) =head1 SEE ALSO L<xmltv(5)> =head1 AUTHOR Geoff Westcott, honir.at.gmail.dot.com, Dec. 2014. =cut use strict; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use Data::Dumper; use Getopt::Long; use XMLTV::Data::Recursive::Encode; # simplify testing by also looking for package in current directory eval 'use XMLTV::Augment'; if ($@ ne '') { eval 'use Augment'; } use XMLTV::Usage <<END $0: Augment programme listings with automatic and user-defined rules $0 [--rule <file>] [--config <file>] [--input <file>] [--output <file>] [--nostats] [--log <file>] [--debug (1-10)] $0 [-r <file>] [-c <file>] [-i <file>] [-o <file>] [-n] [-l <file>] [-d (1-10)] END ; my ( $opt_help, $opt_input, $opt_output, $opt_rule, $opt_config, $opt_nostats, $opt_log, $opt_debug, $opt_do, ); GetOptions( 'h|help' => \$opt_help, 'i|input=s' => \$opt_input, 'o|output=s' => \$opt_output, 'r|rule=s' => \$opt_rule, 'c|config|config-file=s' => \$opt_config, 'n|nostats' => \$opt_nostats, 'l|log:s' => \$opt_log, 'd|debug:i' => \$opt_debug, 'do:i' => \$opt_do, ) or usage(0); usage(1) if $opt_help; #rule is now optional if using Supplement via config # usage(0) if !$opt_rule; $opt_input = '-' if ( !defined($opt_input) ); #$opt_output = 'STDOUT' if ( !defined($opt_output) ); $opt_debug = 0 if ( !defined($opt_debug) ); my $opt_stats = ( !defined($opt_nostats) ? 1 : !$opt_nostats ); # object construction & open log file my $augment = new XMLTV::Augment( 'rule' => $opt_rule, 'config' => $opt_config, 'debug' => $opt_debug, 'stats' => $opt_stats, 'log' => $opt_log, ) || eval { print STDERR "Failed to create XMLTV::Augment object \n"; exit 1; }; my %w_args = (); if (defined $opt_output) { my $fh = new IO::File ">$opt_output"; die "cannot write to $opt_output\n" if not $fh; %w_args = (OUTPUT => $fh); } # our XMLTV::Writer object my $w; # store the input file's encoding my $encoding; # count of input records my $in_count = 0; # parsefiles_callback needs an array my @files = ( $opt_input ); XMLTV::parsefiles_callback(\&encoding_cb, \&credits_cb, \&channel_cb, \&programme_cb, @files); # note: we only get a Writer if the encoding callback gets called if ( $w ) { $w->end(); } # log the stats $augment->printInfo(); # close the log file $augment->end(); exit(0); # callbacks used by parsefiles_callback # sub encoding_cb( $ ) { die if defined $w; $encoding = shift; # callback returns the file's encoding $w = new XMLTV::Writer(%w_args, encoding => $encoding); $augment->setEncoding($encoding); } # sub credits_cb( $ ) { $w->start(shift); } # sub channel_cb( $ ) { my $ch = shift; # store the channel details $augment->inputChannel( $ch ); # write the channel element to the output xml $w->write_channel($ch); } # sub programme_cb( $ ) { my $prog = shift; $in_count++; # developer's option to only process a few records in input file and then stop if ( defined($opt_do) && $in_count > $opt_do ) { return; } # decode the incoming programme $prog = XMLTV::Data::Recursive::Encode->decode($encoding, $prog); # augmentProgramme will now do any requested processing of the input xml $prog = $augment->augmentProgramme( $prog ); # re-code the modified programme back to original encoding $prog = XMLTV::Data::Recursive::Encode->encode($encoding, $prog); # output the augmented programme $w->write_programme($prog); } # �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_augment_tz��������������������������������������������������������������������0000775�0000000�0000000�00000006376�15000742332�0016777�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl =pod =head1 NAME tv_augment_tz - Convert floating time to explicit time. =head1 SYNOPSIS tv_augment_tz [--help] [--output FILE] [--tz TIMEZONE] [FILE...] =head1 DESCRIPTION Read XMLTV data and augment all times with the correct offset. Times that are already explicit will be converted to TIMEZONE as well. B<--output FILE> write to FILE rather than standard output B<--tz TIMEZONE> use TIMEZONE (e.g. Europe/Berlin) rather then the default of UTC =head1 SEE ALSO L<xmltv(5)>. =head1 AUTHOR Karl Dietz, <dekarl@spaetfruehstuecken.org> =head1 BUGS Guessing the right time when local time is abigous doesn't work properly. =cut use strict; use warnings; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use Encode; # used to convert 'perl strings' into 'utf-8 strings' use DateTime; use Getopt::Long; use XML::LibXML; use XMLTV::Usage <<END $0: Convert floating local time to explicit local time. usage: $0 [--help] [--output FILE] [--tz TIMEZONE] [FILE...] END ; my ($opt_help, $opt_output, $opt_tz); GetOptions('help' => \$opt_help, 'output=s' => \$opt_output, 'tz=s' => \$opt_tz ) or usage(0); usage(1) if $opt_help; @ARGV = ('-') if not @ARGV; if (!defined ($opt_tz)) { $opt_tz = 'UTC'; } sub parse_date ( $ ) { my $raw = shift; my ($datetime, $tz) = split (/\s/, $raw); my ($year, $month, $day, $hour, $minute, $second) = (0, 0, 0, 0, 0, 0); if (length ($datetime) == 4+2+2+2+2+2) { ($year, $month, $day, $hour, $minute, $second) = ($datetime =~ m|(\d\d\d\d)(\d\d)(\d\d)(\d\d)(\d\d)(\d\d)|); } elsif (length ($datetime) == 4+2+2+2+2) { ($year, $month, $day, $hour, $minute) = ($datetime =~ m|(\d\d\d\d)(\d\d)(\d\d)(\d\d)(\d\d)|); } elsif (length ($datetime) == 4+2+2+2) { ($year, $month, $day, $hour, $minute) = ($datetime =~ m|(\d\d\d\d)(\d\d)(\d\d)(\d\d)|); } elsif (length ($datetime) == 4+2+2) { ($year, $month, $day, $hour, $minute) = ($datetime =~ m|(\d\d\d\d)(\d\d)(\d\d)|); } elsif (length ($datetime) == 4+2) { ($year, $month, $day, $hour, $minute) = ($datetime =~ m|(\d\d\d\d)(\d\d)|); } elsif (length ($datetime) == 4) { ($year, $month, $day, $hour, $minute) = ($datetime =~ m|(\d\d\d\d)|); } my $dt = DateTime->new ( year => $year, month => $month, day => $day, hour => $hour, minute => $minute, second => $second ); if ($tz) { $dt->set_time_zone ($tz); } return $dt; } # load XML if ($ARGV[0]) { open *STDIN, '<', $ARGV[0]; } binmode *STDIN; # drop all PerlIO layers possibly created by a use open pragma my $doc = XML::LibXML->load_xml(IO => *STDIN); my $starttimes = $doc->findnodes ('/tv/programme[@start]'); foreach my $node ($starttimes->get_nodelist ()) { my $time = $node->getAttribute ('start'); $time = parse_date ($time)->set_time_zone ($opt_tz)->strftime ('%Y%m%d%H%M%S %z'); $node->setAttribute('start', $time); } my $stoptimes = $doc->findnodes ('/tv/programme[@stop]'); foreach my $node ($stoptimes->get_nodelist ()) { my $time = $node->getAttribute ('stop'); $time = parse_date ($time)->set_time_zone ($opt_tz)->strftime ('%Y%m%d%H%M%S %z'); $node->setAttribute('stop', $time); } # save modified XML if ($opt_output) { open *STDOUT, '>', $opt_output; } binmode *STDOUT; # as above $doc->toFH(*STDOUT); ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_cat���������������������������������������������������������������������������0000775�0000000�0000000�00000005203�15000742332�0015355�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w =pod =head1 NAME tv_cat - Concatenate XMLTV listings files. =head1 SYNOPSIS tv_cat [--help] [--utf8] [--output FILE] [FILE...] =head1 DESCRIPTION Read one or more XMLTV files and write a file to standard output whose programmes are the concatenation of the programmes in the input files, and whose channels are the union of the channels in the input files. B<--output FILE> write to FILE rather than standard output The treatment of programmes and channels is slightly different because for programmes, the ordering is important (typically programmes are processed or displayed in the same order as they appear in the input) whereas channels are just a set indexed by channel id. There is a warning if channel details clash for the same id. One more wrinkle is the credits (source, generator and so on), they are taken from one of the files and then thereE<39>s a warning if the other files differ. If two input files have different character encodings, then it is not meaningful to combine their data (without recoding or other processing) and tv_cat die with an error message. But if you do want to combine multiple input files with different character encodings then you can use the --utf8 command-line parameter which will create a combined output file in UTF-8 format. (Note: it is not safe to combine mixed encodings to anything other than UTF-8 so no other output encoding is provided for.) (Note 2: any input file which does not have an encoding specified in the xml tag is not only naughty but is assumed to already be UTF-8, which may not be true!) This tool is rather useless, but it makes a good testbed for the XMLTV module. =head1 SEE ALSO L<xmltv(5)>. =head1 AUTHOR Ed Avis, ed@membled.com =cut use strict; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use Getopt::Long; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } } use XMLTV::Usage <<END $0: concatenate listings, merging channels, (optional: utf-8 output file) usage: $0 [--help] [--utf8] [--output FILE] [FILE...] END ; my ($opt_help, $opt_output, $opt_utf8); GetOptions('help' => \$opt_help, 'output=s' => \$opt_output, 'utf8' => \$opt_utf8) or usage(0); usage(1) if $opt_help; @ARGV = ('-') if not @ARGV; my %w_args = (); if (defined $opt_output) { my $fh = new IO::File ">$opt_output"; die "cannot write to $opt_output\n" if not $fh; %w_args = (OUTPUT => $fh); } if (defined $opt_utf8) { $w_args{'UTF8'} = 1; } XMLTV::catfiles(\%w_args, @ARGV); ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_count�������������������������������������������������������������������������0000664�0000000�0000000�00000005152�15000742332�0015736�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl =pod =head1 NAME tv_count - Count (and print) the number of channels and programmes in an XMLTV file. =head1 SYNOPSIS tv_count -i FILE =head1 DESCRIPTION Read XMLTV listings and print a count of the number of channels and number of programmes in the file. =head1 EXAMPLE To print counts of all channels and programmes in the file called F<listings.xml>: tv_count -i listings.xml =head1 SEE ALSO L<XMLTV(3)> =head1 AUTHOR Copyright Geoff Westcott, February 2013. This code is distributed under the GNU General Public License v2 (GPLv2) . =cut use warnings; use strict; use XML::TreePP; use Date::Parse; use POSIX qw(strftime); use Getopt::Std; $Getopt::Std::STANDARD_HELP_VERSION = 1; use Data::Dumper; # Process command line arguments my %opts = (); getopts("dDqi:hv", \%opts); my $input_file = ($opts{'i'}) ? $opts{'i'} : ""; my $debug = ($opts{'d'}) ? $opts{'d'} : ""; my $debugmore = ($opts{'D'}) ? $opts{'D'} : ""; my $quiet_mode = ($opts{'q'}) ? $opts{'q'} : ""; if ($opts{'h'}) { VERSION_MESSAGE(); HELP_MESSAGE(); exit; } if ($opts{'v'}) { VERSION_MESSAGE(); exit; } if ( !%opts || ! defined $opts{'i'} || ! -r $opts{'i'} ) { HELP_MESSAGE(); exit; } if ($debugmore) { $debug++; } # Parse the XMLTV file my $tpp = XML::TreePP->new(); $tpp->set( force_array => [ 'channel', 'programme' ] ); # force array ref for some fields my $xmltv = $tpp->parsefile( $input_file ); if ($debugmore) { print Dumper($xmltv); } my $ccount = scalar @{ $xmltv->{'tv'}->{'channel'} }; my $ccount_plural = $ccount == 1 ? "" : "s"; my $pcount = scalar @{ $xmltv->{'tv'}->{'programme'} }; my $pcount_plural = $pcount == 1 ? "" : "s"; if (!$quiet_mode) { print "Count : $ccount channel$ccount_plural $pcount programme$pcount_plural \n"; } ############################################################################### ############################################################################### sub VERSION_MESSAGE { use XMLTV; my $VERSION = $XMLTV::VERSION; print "XMLTV module version $XMLTV::VERSION\n"; print "This program version $VERSION\n"; } sub HELP_MESSAGE { # print usage message my $filename = (split(/\//,$0))[-1]; print << "EOF"; Count (and print) the number of channels and programmes in an XMLTV file Usage: $filename [-dDq] -i FILE -i FILE : input XMLTV file -d : print debugging messages -D : print even more debugging messages -q : quiet mode (no STDOUT messages) -h, --help : print help message and exit -v, --version : print version and exit Example: $filename -i listings.xml EOF } ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_extractinfo_ar����������������������������������������������������������������0000664�0000000�0000000�00000034410�15000742332�0017615�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w eval 'exec /usr/bin/perl -w -S $0 ${1+"$@"}' if 0; # not running under some shell =pod =head1 NAME tv_extractinfo_ar - read Spanish (Argentinean) language listings and extract info from programme descriptions. =head1 SYNOPSIS tv_extractinfo_ar [--help] [--output FILE] [FILE...] =head1 DESCRIPTION Read XMLTV data and attempt to extract information from Spanish-language programme descriptions, putting it into machine-readable form. For example the human-readable text '(Repeticion)' in a programme description might be replaced by the XML element <previously-shown>. B<--output FILE> write to FILE rather than standard output This tool also attempts to split multipart programmes into their constituents, by looking for a description that seems to contain lots of times and titles. But this depends on the description following one particular style and is useful only for some listings sources (Ananova). If some text is marked with the 'lang' attribute as being some language other than Spanish ('es'), it is ignored. =head1 SEE ALSO L<xmltv(5)>. =head1 AUTHOR Mariano Cosentino, Mok@marianok.com.ar =head1 BUGS Trying to parse human-readable text is always error-prone, more so with the simple regexp-based approach used here. But because TV listing descriptions usually conform to one of a few set styles, tv_extractinfo_en does reasonably well. It is fairly conservative, trying to avoid false positives (extracting 'information' which isnE<39>t really there) even though this means some false negatives (failing to extract information and leaving it in the human-readable text). However, the leftover bits of text after extracting information may not form a meaningful Spanish sentence, or the punctuation may be wrong. On the two listings sources currently supported by the XMLTV package, this program does a reasonably good job. But it has not been tested with every source of anglophone TV listings. This Spanish Version is heavily customized for the XML results from tv_grab_ar (developed by Christian A. Rodriguez and postriorily updated by Mariano S. Cosentino). This file should probably be called tv_extractinfo_es, but I have not tested it with any other spanish grabbers, so I don't want to be presumptious. =cut use strict; sub MSC_Extract(); use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use XMLTV::Usage <<END $0: read Spanish-language listings and extract info from programme descriptions usage: $0 [--help] [--output FILE] [FILE...] END ; use XML::LibXML; use Getopt::Long; my ($opt_help, $opt_output); GetOptions('help' => \$opt_help, 'output=s' => \$opt_output) or usage(0); usage(1) if $opt_help; @ARGV = ('-') if not @ARGV; my $parser = XML::LibXML->new; my $archivo = @ARGV; my $doc = $parser->parse_file("@ARGV") or die "can't parse xmltv file: $@"; my $root = $doc->documentElement(); MSC_Extract(); if (defined $opt_output) { my $status = $doc->toFile("$opt_output"); die "cannot write to $opt_output\n" if not $status; } else { print $doc->toString; } sub MSC_Extract() { my @nodeList = $doc->getElementsByTagName('programme'); foreach my $programa (@nodeList) { my @oldTitle = $programa->getChildrenByTagName('title'); my @oldDesc = $programa->getChildrenByTagName('desc'); my $oldDesc=$oldDesc[0]->textContent; my $curProgname = $oldTitle[0]->textContent; # my $parentNode=$programa->parent; $curProgname =~ s/^\s+//; $curProgname =~ s/\s+$//; my $Progname = $curProgname; # $curProgname =~ s/^Disney.s\s//i; $curProgname =~ s/^Disney.\s//i; $curProgname =~ s/^\s+//; $curProgname =~ s/\s+$//; if ($curProgname eq "") { $curProgname=$Progname; } # else { # $Progname = $curProgname; # } my $progTitle=""; my $progsubtitle=""; my $episodio=""; my $Temporada=""; my $newNodo=""; my $newNodo_PS=""; my $newNodo_EN=""; my $bChanged=0; # print "$Progname\n"; if ($curProgname =~ /(.*)\(Repetici.n\)$/) { $newNodo_PS = XML::LibXML::Element->new('previously-shown'); $curProgname =$1; # =~ s,$1,,; $curProgname =~ s/\(\)//g; } if ($curProgname =~ /(.*)\(Repetici.n\)(.*)/) { $newNodo_PS = XML::LibXML::Element->new('previously-shown'); $curProgname =$1 . $3; # =~ s,$1,,; $curProgname =~ s/\(\)//g; } if ($curProgname =~ /(.*)\(diferido\)$/) { $curProgname =$1; # =~ s,$1,,; $curProgname =~ s/\(\)//g; } if ($curProgname =~ /(.*)\(diferido\)(.*)/) { $curProgname =$1 . $3; # =~ s,$1,,; $curProgname =~ s/\(\)//g; } if ($curProgname =~ /(.*)\(grabado\)$/) { $curProgname =$1; # =~ s,$1,,; $curProgname =~ s/\(\)//g; } if ($curProgname =~ /(.*)\(grabado\)(.*)/) { $curProgname =$1 . $3; # =~ s,$1,,; $curProgname =~ s/\(\)//g; } if ($curProgname =~ /(.*)\(vivo\)$/) { $curProgname =$1; # =~ s,$1,,; $curProgname =~ s/\(\)//g; } if ($curProgname =~ /(.*)\(vivo\)(.*)/) { $curProgname =$1 . $3; # =~ s,$1,,; $curProgname =~ s/\(\)//g; } if ($curProgname =~ /(.*)\(doblad.\)$/) { $curProgname =$1; # =~ s,$1,,; $curProgname =~ s/\(\)//g; } if ($curProgname =~ /(.*)\(doblad.\)(.*)/) { $curProgname =$1 . $3; # =~ s,$1,,; $curProgname =~ s/\(\)//g; } if ($curProgname =~ /(.*)\(subtitulad..?\)$/) { $curProgname =$1; # =~ s,$1,,; $curProgname =~ s/\(\)//g; } if ($curProgname =~ /(.*)\(subtitulad..?\)(.*)/) { $curProgname =$1 . $3; # =~ s,$1,,; $curProgname =~ s/\(\)//g; } if ($curProgname =~ /(.*)?(.?\sEpisodio\s(\d+))(.*)?/) { $episodio=$3; #$2; $curProgname =$1 . $4; # =~ s,$1,,; $curProgname =~ s/^\s+//; $curProgname =~ s/\s+$//; $curProgname =~ s/ //g; $curProgname =~ s/\.\./\./g; } else { $episodio=""; } if ($curProgname =~ /(.*)?(.?\sTemporada\s(\d+))(.*)?/) { $Temporada=$3; #$2; $curProgname = $1 . $4; # =~ s,'$1',,; $curProgname =~ s/^\s+//; $curProgname =~ s/\s+$//; $curProgname =~ s/ //g; $curProgname =~ s/\.\./\./g; } else { $Temporada=""; } if ("$Temporada.$episodio" ne ".") { $newNodo_EN = XML::LibXML::Element->new('episode-num'); $newNodo_EN->addChild ( $doc->createTextNode( "$Temporada.$episodio." ) ); $newNodo_EN->addChild( $doc->createAttribute('system','xmltv_ns') ); $newNodo=""; } if ($curProgname =~ /([^.]*)\. (.*)/) { $progTitle = $1; $progsubtitle = $2; } else { $progTitle = $curProgname; $progsubtitle = ""; } if ($progTitle ne "") { $progTitle =~ s/^\s+//; $progTitle =~ s/\s+$//; $progTitle =~ s/ //g; $progTitle =~ s/\.\./\./g; $progTitle =~ s/\.+$//g; } if ($progsubtitle ne "") { $progsubtitle =~ s/^\s+//; $progsubtitle =~ s/\s+$//; $progsubtitle =~ s/ //g; $progsubtitle =~ s/\.\./\./g; $progsubtitle =~ s/\.+$//g; } #title+ if ($Progname ne $progTitle) { if ($progTitle ne "") { $newNodo = XML::LibXML::Element->new('title'); $newNodo->addChild ( $doc->createTextNode( $progTitle ) ); $newNodo->addChild( $doc->createAttribute('lang','es') ); $programa->replaceChild ( $newNodo, $oldTitle[0]); $bChanged=1; $newNodo=""; } } my @TempList; my $TempItem; #Título original: Drive if ($oldDesc =~ /Título original:\s(.*)(\n|\r)?/i) { $TempItem=$1; my $TempOtherTitle=""; $TempItem=~ s/^\s//; $TempItem=~ s/\s$//; $bChanged=1; if ($TempItem =~ /(.*)\s\(A.?K.?A.?\s(.*)\)/i) { $TempItem=$1; $TempOtherTitle=$2; $TempItem=~ s/^\s//; $TempItem=~ s/\s$//; $TempOtherTitle=~ s/^\s//; $TempOtherTitle=~ s/\s$//; } $newNodo = XML::LibXML::Element->new('title'); $newNodo->addChild ( $doc->createTextNode( "$TempItem" ) ); $newNodo->addChild( $doc->createAttribute('lang','en') ); $programa->addChild ( $newNodo ); $newNodo=""; if ($TempOtherTitle ne "") { $newNodo = XML::LibXML::Element->new('title'); $newNodo->addChild ( $doc->createTextNode( "$TempOtherTitle" ) ); $newNodo->addChild( $doc->createAttribute('lang','en') ); $programa->addChild ( $newNodo ); $newNodo=""; } } #sub-title* if ($progsubtitle ne "") { $bChanged=1; my $itemnodo = $programa->addNewChild('', 'sub-title'); $itemnodo->addChild( $doc->createTextNode($progsubtitle) ); } # desc* if ($bChanged) { foreach my $tempNode_desc (@oldDesc) { $programa->removeChild($tempNode_desc); $programa->addChild ( $tempNode_desc ); } } my $bCredits=0; my $CreditsNodo = XML::LibXML::Element->new('credits'); # <desc lang="es">Directores: Ed Friedman, Mark Glamack if ($oldDesc =~ /Director(es)?:\s(.*)(\n|\r)?/i) { @TempList=split(",",$2); if (scalar(@TempList)<2) { @TempList=$2; } foreach $TempItem (@TempList) { $TempItem=~ s/^\s//; $TempItem=~ s/\s$//; $newNodo = XML::LibXML::Element->new('director'); $newNodo->addChild ( $doc->createTextNode( "$TempItem" ) ); $bCredits=1; $CreditsNodo->addChild ( $newNodo ); $newNodo=""; } } # <desc lang="es">Protagonista: Kiefer Sutherland # <desc lang="es">Protagonistas: Antonio Aguilar, Elda Peralta, Chela Pastor, Agustín Isunza if ($oldDesc =~ /Protagonistas?:\s(.*)(\n|\r)?/i) { @TempList=split(",",$1); if (scalar(@TempList)<2) { @TempList=$1; } foreach $TempItem (@TempList) { # warn "\nP: $TempItem"; $TempItem=~ s/^\s//; $TempItem=~ s/\s$//; $newNodo = XML::LibXML::Element->new('actor'); $newNodo->addChild ( $doc->createTextNode( "$TempItem" ) ); $bCredits=1; $CreditsNodo->addChild ( $newNodo ); } $newNodo=""; } # Conductor: Adriana Betancour # Conductores: Adam Corrolla, Jimmy Kimmel # Conductor: Fernando Merino if ($oldDesc =~ /Conductor(es)?:\s(.*)(\n|\r)?/i) { @TempList=split(",",$2); if (scalar(@TempList)<2) { @TempList=$2; } foreach $TempItem (@TempList) { $TempItem=~ s/^\s//; $TempItem=~ s/\s$//; $newNodo = XML::LibXML::Element->new('presenter'); $newNodo->addChild ( $doc->createTextNode( "$TempItem" ) ); $bCredits=1; $CreditsNodo->addChild ( $newNodo ); $newNodo=""; } } # credits? if ($bCredits ) { $programa->addChild ( $CreditsNodo ); } $CreditsNodo=""; # date? # Año: 1934 if ($oldDesc =~ /año:\s(.*)(\n|\r)?/i) { $TempItem= $1; $TempItem=~ s/^\s//; $TempItem=~ s/\s$//; $newNodo = XML::LibXML::Element->new('date'); $newNodo->addChild ( $doc->createTextNode( "$TempItem" ) ); $programa->addChild ( $newNodo ); $newNodo=""; } # category* # <desc lang="es">Género: Agro</desc> if ($oldDesc =~ /Género:\s(.*)(\n|\r)?/i) { @TempList=split(",",$1); if (scalar(@TempList)<2) { @TempList=$1; } foreach $TempItem (@TempList) { $TempItem=~ s/^\s//; $TempItem=~ s/\s$//; # warn "Género: $TempItem\n"; $newNodo = XML::LibXML::Element->new('category'); $newNodo->addChild ( $doc->createTextNode( "$TempItem" ) ); $programa->addChild ( $newNodo ); $newNodo=""; } } # language? # orig-language? # length? # Duración: 102 minutos if ($oldDesc =~ /Duración:\s(\d*)\s(.*)(\n|\r)?/i) { my $TempLength=$1; my $tempunit=$2; $tempunit=~ s/^\s//; $tempunit=~ s/\s$//; $tempunit=~ s/minutos?/minutes/; $tempunit=~ s/horas?/hours/; $tempunit=~ s/segundos?/seconds/; $newNodo = XML::LibXML::Element->new('length'); $newNodo->addChild ( $doc->createTextNode( "$TempLength" ) ); $newNodo->addChild( $doc->createAttribute('units',"$tempunit") ); $programa->addChild ( $newNodo ); $newNodo=""; } # icon* # url* # country* #País: Argentina if ($oldDesc =~ /País(es)?:\s(.*)(\n|\r)?/i) { @TempList=split("-",$2); if (scalar(@TempList)<2) { @TempList=$2; } foreach $TempItem (@TempList) { $TempItem=~ s/^\s//; $TempItem=~ s/\s$//; $newNodo = XML::LibXML::Element->new('country'); $newNodo->addChild ( $doc->createTextNode( "$TempItem" ) ); $newNodo->addChild( $doc->createAttribute('lang','es') ); $programa->addChild ( $newNodo ); $newNodo=""; } } # episode-num* if ($newNodo_EN ne "") { $programa->addChild ($newNodo_EN ); } # my $TempLength=""; # my $tempunit=""; # video? # audio? # previously-shown? if ($newNodo_PS ne "") { $programa->addChild ($newNodo_PS ); } # premiere? # last-chance? # new? # subtitles* # rating* # Clasificación: Apta para todo público</desc> # Clasificación: Sólo apta para mayores de 13 años</desc> # Clasificación: Sólo apta para mayores de 16 años</desc> # Clasificación: Sólo apta para mayores de 18 años</desc> if ($oldDesc =~ /Clasificación:\s(.*)(\n|\r)?/i) { $TempItem=$1; $TempItem=~ s/^\s//; $TempItem=~ s/\s$//; $newNodo = XML::LibXML::Element->new('rating'); $newNodo->addChild( $doc->createAttribute('system',"Argentina") ); my $newSubNodo = XML::LibXML::Element->new('value'); $newSubNodo->addChild ( $doc->createTextNode( "$TempItem" ) ); $newNodo->addChild( $newSubNodo); $programa->addChild ( $newNodo ); $newNodo=""; } # star-rating* } } #/(.*)\. (.*)/) { # print "o: $curProgname\n"; # print "T:$1\n"; # print "St:$2\n"; # $progName = $curProgname; ## my $itemnodo = $programa->addNewChild('', 'title'); ## $itemnodo->addChild( $doc->createTextNode($progTitle) ); ## $itemnodo->addChild( $doc->createAttribute('lang','es') ); # my $itemnodo = $programa->addNewChild('', 'title'); # $itemnodo->addChild( $doc->createTextNode($progTitle) ); # $itemnodo->addChild( $doc->createAttribute('lang','es') ); #find($programa->nodePath . "/title"); # print $oldNodo[0]->nodePath; # print $programa->nodePath; ## $programa = $progTitle; #->textContent # print "$progName [$progsubtitle - $Temporada.$episodio]\n"; # pretty print #sub addSubElm($$$) { # my ($pet, $name, $body) = @_; # my $subElm = $pet->addNewChild('', $name); # $subElm->addChild( $doc->createTextNode($body) ); #} #sub addField($$$$) { # my ($type, $name, $dob, $price) = @_; # # addNewChild is non-compliant; could use addSibling instead # my $pet = $root->addNewChild('', $type); # addSubElm ( $pet, 'name', $name ); # addSubElm ( $pet, 'dob', $dob ); # addSubElm ( $pet, 'price', $price ); #} # my $newPrice = sprintf "%6.2f", $curPrice * 1.2; # my $parent = $programa->parentNode; # my $newNodo = XML::LibXML::Element->new('programme'); # $newNodo->addChild ( $doc->createTextNode( $newPrice ) ); # $parent->replaceChild ( $newPriceNode, $priceNode ); #print $doc->toString(1); # pretty print #mv /home/notroot/xmltv2tivo/20080430.slice.gz /home/notroot/TivoEmulator/tivo-service/static/listings/BA01235_13999-14006.slice.gz ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_extractinfo_en����������������������������������������������������������������0000775�0000000�0000000�00000165763�15000742332�0017640�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w =pod =head1 NAME tv_extractinfo_en - read English-language listings and extract info from programme descriptions. =head1 SYNOPSIS tv_extractinfo_en [--help] [--output FILE] [FILE...] =head1 DESCRIPTION Read XMLTV data and attempt to extract information from English-language programme descriptions, putting it into machine-readable form. For example the human-readable text '(repeat)' in a programme description might be replaced by the XML element <previously-shown>. B<--output FILE> write to FILE rather than standard output This tool also attempts to split multipart programmes into their constituents, by looking for a description that seems to contain lots of times and titles. But this depends on the description following one particular style and is useful only for some listings sources (Ananova). If some text is marked with the 'lang' attribute as being some language other than English ('en'), it is ignored. =head1 SEE ALSO L<xmltv(5)>. =head1 AUTHOR Ed Avis, ed@membled.com =head1 BUGS Trying to parse human-readable text is always error-prone, more so with the simple regexp-based approach used here. But because TV listing descriptions usually conform to one of a few set styles, tv_extractinfo_en does reasonably well. It is fairly conservative, trying to avoid false positives (extracting 'information' which isnE<39>t really there) even though this means some false negatives (failing to extract information and leaving it in the human-readable text). However, the leftover bits of text after extracting information may not form a meaningful English sentence, or the punctuation may be wrong. On the two listings sources currently supported by the XMLTV package, this program does a reasonably good job. But it has not been tested with every source of anglophone TV listings. =cut use strict; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use XMLTV::Date; use Date::Manip; use Carp; use Getopt::Long; BEGIN { if (int(Date::Manip::DateManipVersion) >= 6) { Date::Manip::Date_Init("SetDate=now,UTC"); } else { Date::Manip::Date_Init("TZ=UTC"); } } # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } } # Use Term::ProgressBar if installed. use constant Have_bar => eval { require Term::ProgressBar; 1 }; use XMLTV::TZ qw(gettz offset_to_gmt); use XMLTV::Clumps qw(clump_relation relatives fix_clumps nuke_from_rel); use XMLTV::Usage <<END $0: read English-language listings and extract info from programme descriptions usage: $0 [--help] [--output FILE] [FILE...] END ; # There are some seeming bugs in Perl which corrupt the stop time of # programmes. They are less in 5.6.1 than 5.6.0 but still there. The # debugging assertions cst() and no_shared_scalars() have the effect # of stopping the problem (it's a Heisenbug). So one way of making # stop times correct is to call this routines regularly. # # Alternatively, we can limit the script's functionality to work # around the bug. It seems to affect stop times, so if we just don't # add stop times things should be okay. # # This flag decides which of the two to pick: slow but with maximum # information about stop times, or fast without them. (Stop times can # easily be added back in by tv_sort, and they weren't that good # anyway, so you should probably leave this off.) # my $SLOW = 0; warn "this version has debugging calls, will run slowly\n" if $SLOW; # It might turn out that a particular version of perl is needed. # BEGIN { # eval { require 5.6.1 }; # if ($@) { # for ($@) { # chomp; # s/, stopped at .+$//; # warn "$_, continuing but output may be wrong\n"; # } # } # } sub list_names( $ ); sub check_same_channel( $ ); sub special_category( $ ); sub special_multipart( $ ); sub special_credits( $ ); sub special_radio4( $ ); sub special_split_title( $ ); sub special_film( $ ); sub special_new_series( $ ); sub special_year( $ ); sub special_tv_movie( $ ); sub special_teletext_subtitles( $ ); sub special_useless( $ ); sub check_time_fits( $$ ); sub cst( $ ); sub no_shared_scalars( $ ); sub has( $$@ ); sub hasp( $$$ ); sub pd( $ ); sub ud( $ ); sub nd( $ ); sub bn( $ ); sub munge( $ ); sub multipart_split_desc( $$ ); sub clocks_poss( $ ); sub time12to24( $ ); sub add( $$$ ); sub scrub_empty( @ ); sub set_stop_time( $$ ); sub dump_pseudo_programme( $ ); # --no-create-sub-titles is an undocumented switch, affecting the # splitting of multipart programmes only, to not break a title # containing colons into title and sub-title, but always keep it as a # single title containing a colon. This is for consistency with some # data sources that do this. # my ($opt_help, $opt_output, $opt_no_create_sub_titles); GetOptions('help' => \$opt_help, 'output=s' => \$opt_output, 'no-create-sub-titles' => \$opt_no_create_sub_titles) or usage(0); usage(1) if $opt_help; @ARGV = ('-') if not @ARGV; #### # Language selection stuff. # my $LANG = 'en'; # bn(): wrapper for XMLTV::best_name(). Does some memoizing (so # assumes that the languages in a list of pairs will not change). # my %bn; sub bn( $ ) { my $pairs = shift; return undef if not defined $pairs; die 'bn(): expected ref to list of [text,lang] pairs' if ref $pairs ne 'ARRAY'; for ($bn{$pairs}) { return $_ if defined; foreach (@$pairs) { carp "found bad [text,lang] pair: $_" if ref ne 'ARRAY'; } return $_ = XMLTV::best_name([ $LANG ], $pairs); } } # pair_ok(): returns whether a [ text, lang ] pair is usable. sub pair_ok( $ ) { not defined $_->[1] or $_->[1] =~ /^$LANG(?:_\w+)?$/o; } #### # Human name stuff. # # Regular expression to match a name my $UC = '[A-Z]'; # upper case my $LC = "[a-z]"; # lower case my $AC_P = "[\'A-Za-z-]"; # any case with punctuation my $NAME_RE; { # One word of a name. Uppercase, anycase then ending in at least # two lowercase. Alternatively, uppercase then lowercase (eg # 'Lee'), all uppercase ('DJ') or just uppercase and an optional dot (for # initials). # my $name_comp_re = "(?:$UC(?:(?:$AC_P+$LC$LC)|(?:$LC+)|(?:$UC+)|\\.?))"; foreach ('Simon', 'McCoy') { die "cannot match name component $_" unless /^$name_comp_re$/o; } foreach ("Valentine's") { die "wrongly matched name component $_" if /^$name_comp_re$/o; } # Additional words valid in the middle of names. my $name_join_re = "(?:von|van|de|di|da|van\\s+den|bin|ben|al)"; # A name must have at least two components. This excludes those # celebrities known only by first name but it's a reasonable # heuristic for distinguishing between the names of actors and the # names of characters. # my $name_re = "(?:$name_comp_re\\s+(?:(?:(?:$name_comp_re)|$name_join_re)\\s+)*$name_comp_re)"; foreach ('Simon McCoy', 'Annie Wu') { die "cannot match $_" unless /^$name_re$/o; } # Special handling for some titles. But others beginning 'the' # are specifically excluded (to avoid 'the Corornation Street # star' parsing as '$NAME_RE star'). # $NAME_RE = "(?<!the\\s)(?:(?:[Tt]he\\s+Rev(?:\\.|erend)\\s+)?$name_re)"; } # Regexp to match a list of names: 'Tom, Dick, and Harry' my $NAMES_RE = "(?:$NAME_RE(?:(?:\\s*,\\s*$NAME_RE)*(?:\\s*,?\\s*\\band\\s+$NAME_RE))?(?!\\s*(?:and\\s+$UC|from|[0-9])))"; # Subroutine to extract the names from this list sub list_names( $ ) { die 'usage: list_names(English string listing names)' if @_ != 1; local $_ = shift; die if not defined; t 'list_names() processing string: ' . d $_; my @r; s/^($NAME_RE)\s*// or die "bad 'names' '$_'"; push @r, $1; while (s/^,?\s*(?:and\s+)?($NAME_RE)\s*//) { push @r, $1; } die "unmatched bit of names $_" unless $_ eq ''; return @r; } my @tests = ( [ 'Richard Whiteley and Carol Vorderman', [ 'Richard Whiteley', 'Carol Vorderman' ] ], [ 'show presented by Jonathan Ross, with', [ 'Jonathan Ross' ] ], [ 'Shane Richie, Michael Starke and Scott Wright', [ 'Shane Richie', 'Michael Starke', 'Scott Wright' ] ], [ 'Basil Brush,Barney Harwood and Ugly Yetty present', [ 'Basil Brush', 'Barney Harwood', 'Ugly Yetty'] ], ); foreach (@tests) { my ($in, $expected) = @$_; for ($in) { /($NAMES_RE)/o or die "$in doesn't match \$NAMES_RE"; my @out = list_names($1); local $Log::TraceMessages::On = 1; if (d(\@out) ne d($expected)) { die "$in split into " . d(\@out); } } } #### # Date handling stuff. # # This loses any information on partial dates (FIXME). # sub pd( $ ) { for ($_[0]) { return undef if not defined; return parse_date($_); } } sub ud( $ ) { for ($_[0]) { return undef if not defined; return UnixDate($_, '%q'); } } sub nd( $ ) { for ($_[0]) { return undef if not defined; return ud(pd($_)); } } # Memoize some subroutines if possible. FIXME commonize to # XMLTV::Memoize. # eval { require Memoize }; unless ($@) { foreach (qw(parse_date UnixDate Date_Cmp clocks_poss time12to24)) { Memoize::memoize($_) or die "cannot memoize $_: $!"; } } my $encoding; my $credits; my %ch; my @progs; XMLTV::parsefiles_callback(sub( $ ) { $encoding = shift }, sub( $ ) { $credits = shift }, sub( $ ) { my $c = shift; $ch{$c->{id}} = $c }, sub( $ ) { push @progs, shift }, @ARGV); # Assume encoding is a superset of ASCII, and that Perl's regular # expressions work with it in the current locale. # my $related = clump_relation(\@progs); # Apply all handlers. We just haphazardly # run one after the other; when a programme has been run # through all of them in sequence without any changes, we # move it to @progs_done. # # The reason for using _lists_ is that some handlers turn # a single programme into several. # my @progs_done = (); my $bar = new Term::ProgressBar('munging programmes', scalar @progs) if Have_bar; while (@progs) { # Deal with one more programme from the input, it may transform # itself into one or more programmes which need processing in # turn. When all the offspring are dealt with we have finally # finished with that input and can update the progress bar. # no_shared_scalars(\@progs) if $SLOW; push @progs_done, munge(shift @progs); update $bar if Have_bar; } if ($SLOW) { cst $_ foreach @progs_done } my %w_args = (); if (defined $opt_output) { my $fh = new IO::File ">$opt_output"; die "cannot write to $opt_output\n" if not $fh; %w_args = (OUTPUT => $fh); } XMLTV::write_data([ $encoding, $credits, \%ch, \@progs_done ], %w_args); exit(); # Take a programme, munge it and return a list of programmes (empty if # the programme was deleted). Uses the global $related to fiddle with # other programmes in the same clump. # sub munge( $ ) { # local $Log::TraceMessages::On = 1; t 'munge() ENTRY'; my @todo = (shift); my @done; t 'todo list initialized to: ' . d \@todo; t 'done list initialized to: ' . d \@done; t 'relatives of todo programme: ' . d relatives($related, $todo[0]); # Special-case mungers for various programme types. Each of these # should take a single programme and return: a reference to a list of # programmes, if successful; undef, if the programme is to be left # alone. Most special-case handlers will not break up a programme # into several others, so the returned list will have only one # element. # # A handler may modify the programme passed in iff it returns a list # of munged programmes. # # Ones earlier in the list get the chance to run first, so in general # things like splitting programmes (which may be relied on by other # handlers) should come at the top and last-chance guesswork (such as # parsing English text) at the bottom. # my @special_handlers = ( \&special_multipart, \&special_category, \&special_credits, \&special_new_series, \&special_year, \&special_tv_movie, \&special_teletext_subtitles, \&special_useless, # There are three handlers specific to Ananova / UK listings. I # haven't yet decided what to do with them: should they be in this # program and enabled with a special flag, or moved into the # Ananova grabber? # # They haven't been ported to the new XMLTV.pm data structures, so # leave them commented for now. # # \&special_radio4, # \&special_split_title, # \&special_film, ); PROG: while (@todo) { my $prog = shift @todo; t('running handlers for prog: ' . d($prog)); my $prog_length; if (defined $prog->{stop}) { # Get the programme length in seconds. my $delta = DateCalc($prog->{start}, $prog->{stop}); $prog_length = Delta_Format($delta, 0, '%st') if defined $delta; } foreach (@special_handlers) { t('running handler: ' . d($_)); my $out = $_->($prog); if (defined $out) { t('gave new list of progs: ' . d($out)); die "handler didn't return list of progs" if ref($out) ne 'ARRAY'; if ($SLOW) { cst $_ foreach @$out } check_time_fits($prog, $out); if ($SLOW) { cst $_ foreach @$out } fix_clumps($prog, $out, $related); foreach (@$out) { cst $_ if $SLOW; # Sanity check that length hasn't increased. if (defined $_->{stop}) { my $delta = DateCalc($_->{start}, $_->{stop}); if (defined $prog_length) { my $new_length = Delta_Format($delta, 0, '%st'); if ($new_length > $prog_length) { local $Log::TraceMessages::On = 1; t 'original programme (after handlers run): ' . d $prog; t 'split into: ' . d $out; t 'offending result: ' . d $_; t 'length of result: ' . d $new_length; t 'length of original programme: ' . d $prog_length; die 'split into programme longer than the original'; } } } } push @todo, @$out; next PROG; } t('gave undef'); } cst $prog if $SLOW; t 'none of the handlers fired, finished with this prog'; cst $prog if $SLOW; push @done, $prog; } return @done; } # All the special handlers # special_category() # # Some descriptions have the last word as the category: 'blah blah # blah. Western' (or 'Western series'). Remove this to the <category> # element. # # Also look for magic words like 'news' or 'interview' and add them as # categories. This is mostly so that other handlers can then fire. # sub special_category( $ ) { t 'special_category() ENTRY'; my $p = shift; my $changed = 0; # First, non-destructively look for 'news' in title or desc. foreach (qw(title desc)) { foreach my $pair (grep { pair_ok($_) } @{$p->{$_}}) { t "pair for $_: " . d $pair; if ($pair->[0] =~ /\bnews/i) { t 'matches "news"'; if (hasp($p, 'category', sub { $_[0] =~ /\b(?:soap|drama|game show)\b/i })) { t '...but clearly not a news programme'; } else { $changed |= add($p, 'category', 'news'); cst $p if $SLOW; } } if ($pair->[0] =~ /\binterviews\b/i) { t 'matches "interviews"'; $changed |= add($p, 'category', 'talk'); cst $p if $SLOW; } } } # Now try the last-word-of-desc munging. my $replacement = sub( $$$$ ) { my ($punct, $adj, $country, $genre) = @_; $changed |= add($p, 'category', lc $genre); if (length $adj or length $country) { return "$punct $adj$country$genre"; } else { $changed = 1; return $punct; } }; foreach (grep { pair_ok($_) } @{$p->{desc}}) { # 'Western' -> '' # 'Western series' -> '' # 'Classic Western' -> 'Classic Western' # etc. # $_->[0] =~ s/(^|\.|\?)\s* (Classic\s+|Award-winning\s+|) (Australian\s+|) ([aA]dventure|[aA]nimation|[bB]iopic|[cC]hiller |[cC]omedy|[dD]ocumentary|[dD]rama|[fF]antasy |[hH]eadlines|[hH]ighlights|[hH]orror|[mM]agazine |[mM]elodrama|[mM]usical|[mM]ystery|[oO]mnibus |[qQ]uiz|[rR]omance|[sS]itcom|[tT]earjerker |[tT]hriller|[wW]estern)\s*(?:series\s*|)$/$replacement->($1, $2, $3, $4)/xe; } if ($changed) { t 'some categories found, programme: ' . d $p; scrub_empty($p->{title}, $p->{desc}); t 'after removing empty titles and descs, programme: ' . d $p; return [ $p ]; } else { return undef; } } # special_multipart() # # Often TV listings contain several programmes stuffed into one entry, # which might have made sense for a printed guide to save space, but # is stupid for electronic data. This special handler looks at the # programme description and haphazardly attempts to split the # programme into its components. # # Parameters: a 'programme' hash reference # Returns: reference to list of sub-programmes, or undef if programme # was not split # # We find the title using bn(), in other words we look only at # the first title. Similarly we use only the first description. But # it should work. FIXME should split the secondary title as well! # sub special_multipart( $ ) { # local $Log::TraceMessages::On = 1; die "usage: special_multipart(hashref of programme details)" if @_ != 1; my $p = shift; cst $p if $SLOW; t 'special_multipart() ENTRY'; t 'checking programme descs: ' . d $p->{desc}; my $best_desc = bn($p->{desc}); t 'got best desc: ' . d $best_desc; return undef if not $best_desc; my ($desc, $desc_lang) = @$best_desc; t 'testing description for multipart: ' . d $desc; local $_ = $desc; my @words = split; my @poss_times = split /[ ,;-]/; my @r; my ($p_start, $p_stop) = (pd($p->{start}), pd($p->{stop})); # Assume that the timezone for every time listed in the # description is the same as the timezone for the programme's # start. FIXME will fail when timezone changes partway through. # my $tz = gettz($p->{start}); my $day; if (defined $tz) { # Find the base day taking into account timezones. Eg if a # programme starts at 00:45 BST on the 20th and then lists # times as '01:00' etc, the base date for these times is the # 20th, even though the real start time is 23:45 UTC on the # 19th. # $day = pd(UnixDate(Date_ConvTZ($p_start, 'UTC', offset_to_gmt($tz)), '%Q')); } else { $day = pd(UnixDate($p_start, '%q')); } t "day is $day"; # FIXME won't be correct when split programme spans days. # Sanity check for a time, that it is within the main programme's # timespan. # my $within_time_period = sub { my $t = shift; t("checking whether $t is in time period $p_start.." . (defined $p_stop ? $p_stop : '')); if (Date_Cmp($t, $p_start) < 0) { # Before start of programme, that makes no sense. return 0; } if (defined $p_stop and Date_Cmp($p_stop, $t) < 0) { # After end of programme, likewise. return 0; } return 1; }; # Three different ways of interpreting a time. Return undef if # not valid under that system, a 24 hour hh:mm otherwise. # # FIXME doesn't handle multiparts bridging noon or midnight. # my $as_12h_am = sub { my $w = shift; $w =~ s/[,;.]$//; t "trying $w as 12 hour am time"; clocks_poss($w)->[0] || return undef; return time12to24("$w am"); }; my $as_12h_pm = sub { my $w = shift; $w =~ s/[,;.]$//; t "trying $w as 12 hour pm time"; clocks_poss($w)->[0] || return undef; return time12to24("$w pm"); }; my $as_24h = sub { my $w = shift; $w =~ s/[,;.]$//; t "trying $w as 24 hour time"; clocks_poss($w)->[1] || return undef; $w =~ tr/./:/; return $w; }; if (defined $tz) { t "using timezone $tz for interpreting times" } else { t "interpreting times with no timezone (ie UTC)" } my ($best_interp, $best_count, $best_first_word_is_time, $best_including_at_time); INTERP: foreach my $interp ($as_24h, $as_12h_am, $as_12h_pm) { t 'testing an interpretation of times'; my $count = 0; my $first_word_is_time = 0; my $including_at_time = 0; my $prev; for (my $pos = 0; $pos < @poss_times; $pos++) { t "testing word $poss_times[$pos] at position $pos"; my $w = $poss_times[$pos]; t "word is '$w'"; my $i = $interp->($w); if (not defined $i) { t "doesn't parse to a time with this interp."; next; } warn "bad 24h returned time: $i" unless $i =~ /^\d?\d:\d\d$/; t "found a time that interprets: $i"; my $t = Date_SetTime($day, $i); die if not defined $t; t "taken as day $day, gets time $t"; $t = Date_ConvTZ($t, offset_to_gmt($tz), 'UTC') if defined $tz; t "after converting to UTC, $t"; if (not $within_time_period->($t)) { # Obviously wrong. One bad time is enough to abandon # this whole interpretation and try another. # t "not within time period, whole interpretation wrong"; next INTERP; } # Don't insist that times be in order, this isn't the case # for all listings (eg 'News at 0700 and 0730; Weather at # 0715'). # $prev = $t; ++ $count; if ($pos == 0) { $first_word_is_time = 1; } if ($pos >= 2 and $poss_times[$pos - 2] =~ /^[Ii]ncluding$/ and $poss_times[$pos - 1] eq 'at') { $including_at_time = 1; t 'previous words are "including at", setting $including_at_time true'; } } t "found $count matching times and nothing badly wrong"; if (not defined $best_interp or $count > $best_count) { t 'best so far'; $best_interp = $interp; $best_count = $count; $best_first_word_is_time = $first_word_is_time; $best_including_at_time = $including_at_time; } } if (defined $best_interp) { t "best result found: count $best_count"; t "first word? $best_first_word_is_time"; t "best includes 'at time'? $best_including_at_time"; } else { t "couldn't find any interpretation that worked at all"; } # Heuristic. We require at least three valid times to split - or # when the programme description begins with a time, that's also # good enough. Also when the description contains 'including at' # followed by a time. # return undef if not defined $best_interp; return undef unless ($best_count >= 3 or $best_first_word_is_time or $best_including_at_time); # local $Log::TraceMessages::On = 1; t 'looks reasonable, proceed'; t 'calling multipart_split_desc() with words and interpretation fn'; my $split = multipart_split_desc(\@words, $best_interp); t 'got result from multipart_split_desc(): ' . d $split; die if not defined $split->[0]; die if not defined $split->[2]; our @pps; local *pps = $split->[0]; t 'got list of pseudo-programmes: ' . d \@pps; if (not @pps) { warn "programme looked like a multipart, but couldn't grok it"; return undef; } if (@pps == 1) { # Didn't really split, perhaps it wasn't a multipart. t 'split into only one, leave unchanged'; return undef; } foreach (@pps) { die if not defined; die if not keys %$_; } my $common = $split->[1]; our @errors; local *errors = $split->[2]; # We split the first description, and only after checking it did # look like a plausible multipart. So if anything went wrong we # should warn about it. # foreach (@errors) { warn $_; } # What was returned is a list of pseudo-programmes, these have # main_desc instead of real [text, lang] descriptions, and hh:mm # 'time' instead of real start time+date. # # At most one of them is allowed to have time undefined; this is # the 'rump' of the parent programme. Whether such a rump exists # depends on what kind of splitting was done. # my $seen_rump = 0; foreach (@pps) { my $time = delete $_->{time}; die if not defined $time and $seen_rump++; if (defined $time) { my $start = Date_SetTime($day, $time); die if not defined $start; $start = Date_ConvTZ($start, offset_to_gmt($tz), 'UTC') if defined $tz; if (Date_Cmp($start, $p->{start}) < 0) { my $dump = dump_pseudo_programme($_); die "subprogramme ($dump, has 'time' $time) " . "starts before main programme ($p->{start}, $p->{title}->[0]->[0])"; } if (defined $p->{stop} and Date_Cmp($p->{stop}, $start) < 0) { my $dump = dump_pseudo_programme($_); die "subprogramme ($dump, has 'time' $time) starts after main one stops"; } # Now we store the time in the official 'start' key. But # convert back to the original timezone to look nice. # if (defined $tz) { $_->{start} = ud(Date_ConvTZ($start, 'UTC', offset_to_gmt($tz))) . " $tz"; } else { $_->{start} = ud($start); } } else { $_->{start} = $p->{start}; } if (not defined $_->{main_title}) { # A title is needed, normally splitting will find one, but # in case it didn't... # $_->{title} = $p->{title}; } # Now deal with each of the main_X fields turning them into # real X. # foreach my $key (qw(desc title sub-title)) { my $v = delete $_->{"main_$key"}; next if not defined $v; $_->{$key} = [ [ $v, $desc_lang ] ]; } if (defined $common) { # Add the common text to this programme. So far it has at # most one description in language $desc_lang. # for ($_->{desc}->[0]->[0]) { if (defined and length) { $_ .= '. ' if $_ !~ /[.?!]\s*$/; $_ .= " $common"; } else { $_ = $common; } } $_->{desc}->[0]->[1] = $desc_lang; } $_->{channel} = $p->{channel}; t "set channel of split programme to $_->{channel}"; } # The last subprogramme should stop at the same time as the # multipart programme stopped. # if (defined $p->{stop}) { t "setting stop time of last subprog to stop time of main prog ($p->{stop})"; set_stop_time($pps[-1], $p->{stop}); } else { t 'main prog had no stop time, not adding to last subprog' } # And similarly, the first should start at the same time as the # multipart programme. Add a dummy prog to fill the gap if # necessary. # my $first_sub_start = $pps[0]->{start}; my $cmp = Date_Cmp(pd($first_sub_start), $p_start); if ($cmp < 0) { # Should have caught this already. die 'first subprogramme starts _before_ main programme'; } elsif ($cmp == 0) { # Okay. } elsif ($cmp > 0) { my $dummy = { title => $p->{title}, channel => $p->{channel}, start => $p->{start}, stop => $first_sub_start }; t 'inserting dummy subprogramme: ' . d $dummy; cst $dummy if $SLOW; unshift @pps, $dummy; } else { die } if ($SLOW) { cst $_ foreach @pps } scrub_empty($_->{title}, $_->{"sub-title"}, $_->{desc}) foreach @pps; t 'returning new list of programmes: ' . d \@pps; return \@pps; } # Given a programme description split into a list of words, and a # subroutine to interpret times, return a list of the subprogrammes # (assuming it is a multipart). # # Returns [pps, common, errs] where pps is a list of 'pseudo-programmes', # hashes containing some of: # # time: 24h time within the main programme's day, # main_title, main_desc, main_sub-title: text in the same language as # the desc passed in, # # and where common is text which belongs to the description of every # subprogramme, and errs is a list of errors found (probably quite # large if the description was not multipart). # sub multipart_split_desc( $$ ) { our @words; local *words = shift; my $interp = shift; # We need to decide what style of multipart listing this is. # There's the kind that has time - title - description for each # subprogramme. There's the kind that has 'News at time0, time1, # time2; Weather at time3, time4'. And then something more like a # normal English sentence, which of course is the hardest to # parse. We use some heuristics to work out which it is and call # the appropriate 'parsing' routine. # t "testing for 'Including at'"; foreach my $i (0 .. $#words - 1) { t "looking at pos $i, word is $words[$i]"; if ($words[$i] =~ /^[Ii]ncluding$/ and $words[$i + 1] eq 'at') { t 'yup, calling multipart_split_desc_including_at()'; return multipart_split_desc_including_at(\@words, $interp); } } t "testing for 'With X at T0, T1; ...'"; if (@words >= 4 and $words[0] =~ /^with$/i) { my $first_lc_word; foreach (@words) { if (not tr/[A-Z]//) { $first_lc_word = $_; last; } } if (defined $first_lc_word and $first_lc_word eq 'at') { return multipart_split_desc_rt(\@words, $interp); } } t "looking for two times in a row, or separated only by 'and'"; my $prev_was_time = 0; foreach (@words) { if (defined $interp->($_)) { # Found a time. if ($prev_was_time) { t 'found two times in a row, using multipart_split_desc_simple()'; return multipart_split_desc_simple(\@words, $interp); } $prev_was_time = 1; } elsif ($_ eq 'and') { # Skip. } else { $prev_was_time = 0; } } t "looking for pairs of times 'from-to'"; foreach (@words) { if (/^([0-9.:]+)-([0-9.:]+)$/) { my ($from, $to) = ($1, $2); if (defined $interp->($from) and defined $interp->($to)) { return multipart_split_desc_fromto(\@words, $interp); } } } t "must be old style of 'time title. description'"; return multipart_split_desc_ananova(\@words, $interp); } # And these routines handle the different styles. sub multipart_split_desc_ananova( $$ ) { our @words; local *words = shift; my $interp = shift; my @r; my @errors; # First extract any 'common text' at the start of the programme, # before any sub-programmes. # my $common; while (@words) { my $first = shift @words; if (defined $interp->($first)) { unshift @words, $first; last; } if (defined $common and length $common) { $common .= " $first"; } else { $common = $first; } } t 'common text: ' . d $common; while (@words > 1) { # At least one thing after the time my $time = shift @words; my $i = $interp->($time); if (defined $i) { my (@title_words, @desc_words); # Build up a current 'pseudo-programme' with title, # description and time. It's up to our caller to # manipulate these simple data structures into real # programmes. # my $curr_pp; $curr_pp->{time} = $i; my $done_title = 0; my @words_orig = @words; while (@words) { my $word = shift @words; if (defined $interp->($word)) { # Finished this bit of multipart. unshift @words, $word; last; } elsif (not $done_title) { if ($word =~ s/[.]$// or $word =~ s/([!?])$/$1/) { # Finished the title, move on to description. $done_title = 1; } push @title_words, $word; } else { push @desc_words, $word; } } if (not @title_words) { warn "trouble finding title in multipart"; if (not @desc_words) { warn "cannot find title or description in multipart"; @title_words = ('???'); } else { # Use the description so far as the title. if ($desc_words[-1] eq 'at') { pop @desc_words; } @title_words = @desc_words; @desc_words = (); } } # The title sometimes looks like 'History in Action: Women # in the 20th Century'; this should be broken into main # title and secondary title. But not 'GNVQ: Is It For You # 2'. So arbitrarily we check that the main title has at # least two words. # if (@title_words) { my (@main_title_words, @sub_title_words); while (@title_words) { my $word = shift @title_words; my $main_title_length = @main_title_words + 1; # Split at colon, sometimes if ((not $opt_no_create_sub_titles) and $main_title_length >= 2 and $word =~ s/:$//) { push @main_title_words, $word; @sub_title_words = @title_words; last; } else { push @main_title_words, $word; } } $curr_pp->{main_title} = join(' ', @main_title_words); $curr_pp->{'main_sub-title'} = join(' ', @sub_title_words) if @sub_title_words; } $curr_pp->{main_desc} = join(' ', @desc_words) if @desc_words; t 'built sub-programme: ' . d $curr_pp; push @r, $curr_pp; } else { push @errors, "expected time in multipart description, got $time"; # Add it to the previous programme, so it doesn't get lost if (@r) { my $prev = $r[-1]; $prev->{main_desc} = '' if not defined $prev->{main_desc}; $prev->{main_desc} .= $time; } else { # Cannot happen. If @r is empty, this must be the # first word. # warn 'first word of desc is not time, but checked this before'; # Not worthy of @errors, this is a bug in the code. } } } foreach (@r) { die if not keys %$_; die if not defined $_->{main_title}; } t 'returning list of pseudo-programmes: ' . d \@r; t '...and common text: ' . d $common; t '...and errors: ' . d \@errors; return [\@r, $common, \@errors]; } sub multipart_split_desc_rt( $$ ) { our @words; local *words = shift; my $interp = shift; my @errors; my $with = shift @words; die if not defined $with; die if $with !~ /^with$/i; my @got; my @title = (); my @times = (); my $done_title = 0; while (@words) { my $w = shift @words; if ($w eq 'at') { $done_title = 1; next; } my $i = $interp->($w); if (defined $i) { # It's a time. if (not $done_title) { warn "unexpected time $w in multipart description, before 'at'"; push @errors, $w; } else { push @times, $i; } if ($w =~ /[.;]$/) { # End of the list of times for this programme. push @got, [ [ @title ], [ @times ] ]; @title = (); @times = (); $done_title = 0; } elsif ($w =~ /,$/) { # List continues. } else { warn "strange time $w"; } next; } # Not a time, should be part of the title. if ($done_title) { warn "strange word $w in multipart description, expected a time"; push @errors, $w; } else { push @title, $w; } } my @r; foreach (@got) { my ($title, $times) = @$_; foreach (@$times) { push @r, { main_title => join(' ', @$title), time => $_ }; } } # There is no 'common text' with this splitter. return [\@r, undef, \@errors]; } # Split the programme by looking for times, but each new programme has # the same words (except times). # sub multipart_split_desc_simple( $$ ) { our @words; local *words = shift; my $interp = shift; my @common; my @times; foreach (@words) { die if not defined; my $i = $interp->($_); if (defined $i) { push @times, $i; if (@common and ($common[-1] eq 'at' or $common[-1] eq 'and')) { pop @common; } } else { push @common, $_; } } my @r; foreach (@times) { die if not defined; push @r, { time => $_ }; } # No 'errors' but lots of 'common text'. return [ \@r, join(' ', @common), [] ]; } sub multipart_split_desc_fromto( $$ ) { our @words; local *words = shift; my $interp = shift; my @r; my @errors; # This routine is limited a bit because it's expected to return # hashes with just 'time'. But we know more than that, we know # both start time and stop time for each subprogramme. That # information would be thrown away. # # For now, it seems that this kind of multipart programme always # has one part beginning when the previous one ended, so we can # just check that this property holds. Then there will be no loss # of stop-time information. # my ($last_start, $last_stop); my @title = (); my $done_title = 0; my @desc = (); foreach (@words) { if (/^([0-9.:]+)-([0-9.:]+)$/ and defined(my $istart = $interp->($1)) and defined(my $istop = $interp->($2))) { # It's a pair of times. if (defined $last_start) { # Deal with the previous subprogramme. warn "mismatch between stop time $last_stop and start time $istart" if $last_stop ne $istart; my %p = ( time => $last_start, main_title => join(' ', @title) ); $p{main_desc} = join(' ', @desc) if @desc; push @r, \%p; } ($last_start, $last_stop) = ($istart, $istop); @title = (); $done_title = 0; @desc = (); } elsif (/:$/) { # A colon ends the title. if (not $done_title) { (my $tmp = $_) =~ s/:$//; push @title, $tmp; $done_title = 1; } else { warn "seen colon in description: '$_'"; push @desc, $_; } } elsif ($_ eq 'with') { # Also 'with' can end a title, as in 'News with...'. This # is probably the only time I've seen a use for the # convention that words in titles should be capitalized. # # The 'with' stuff goes into the description, where some # other handler can pick it up. # $done_title = 1; push @desc, $_; } else { if (not $done_title) { push @title, $_; } else { push @desc, $_; } } } if (defined $last_start) { my %p = ( time => $last_start, main_title => join(' ', @title) ); $p{main_desc} = join(' ', @desc) if @desc; push @r, \%p; } return [ \@r, undef, [] ]; } # Really an 'including at' programme should be sandwiched in the # middle of its parent, but the format doesn't allow that so for # simplicity we treat as a multipart. # sub multipart_split_desc_including_at( $$ ) { our @words; local *words = shift; my $interp = shift; my @r; my @rump; while (@words) { my $t; if (@words >= 4 and $words[0] =~ /^[Ii]ncluding$/ and $words[1] eq 'at' and defined ($t = $interp->($words[2])) and $words[3] =~ /^[A-Z]/) { shift @words; shift @words; shift @words; my @title; while (@words and $words[0] =~ /^[A-Z]/) { my $w = shift @words; if ($w =~ s/[.,;]$//) { push @title, $w; last; } else { push @title, $w; } } push @r, { time => $t, main_title => join(' ', @title) }; } else { push @rump, shift @words; } } unshift @r, { main_desc => join(' ', @rump) }; return [ \@r, '', [] ]; } # Is a time string using the 12 hour or 24 hour clock? Returns a pair # of two booleans: the first means it could be 12h, the seecond that # it could be 24h. Expects an h.mm or hh.mm time string. If the # string is not a valid time under either clock, returns [0, 0]. # # Allows eg '5.30' to be a 24 hour time (05:30). # sub clocks_poss( $ ) { local $_ = shift; if (not /^(\d\d?)\.(\d\d)$/) { return [0, 0]; } my ($hh, $mm) = ($1, $2); return [0, 0] if $mm > 59; return [0, 1] if $hh =~ /^0/; return [1, 1] if 1 <= $hh && $hh < 13; return [0, 1] if 13 <= $hh && $hh < 24; # Do not accept '24:00', '24:01' etc - not until it's proved we # need to. # return [0, 0]; } # Debugging stringification. sub dump_pseudo_programme( $ ) { my @r; my $pp = shift; foreach (qw(time main_title main_desc)) { push @r, $pp->{$_} if defined $pp->{$_}; } return join(' ', @r); } # time12to24() # # Convert a 12 hour time string to a 24 hour one, without anything too # fancy. In particular the timezone is passed through unchanged. # sub time12to24( $ ) { die 'usage: time12to24(12 hour time string)' if @_ != 1; local $_ = shift; die if not defined; # Remove the timezone and stick it back on afterwards. my $tz = gettz($_); s/\Q$tz\E// if defined $tz; s/\s+//; my ($hours, $mins, $ampm) = /^(\d\d?)[.:]?(\d\d)\s*(am|pm)$/ or die "bad time $_"; if ($ampm eq 'am') { if (1 <= $hours and $hours < 12) { $hours = $hours; # 5am = 05:00 } elsif ($hours == 12) { $hours = 0; # 12am = 00:00 } else { die "bad number of hours $hours" } } elsif ($ampm eq 'pm') { if ($hours == 12) { $hours = 12; # 12pm = 12:00 } elsif (1 <= $hours and $hours < 12) { $hours = 12 + $hours; # 5pm = 17:00 } else { die "bad number of hours $hours" } } else { die } my $r = sprintf('%02d:%02d', $hours, $mins); $r .= " $tz" if defined $tz; return $r; } # special_credits() # # Try to sniff out presenter, actor or guest info from the start of the # description and put it into the credits section instead. # # Parameters: one programme (hashref) # Returns: [ modified programme ], or undef # # May modify the programme passed in, if return value is not undef. # But that's okay for a special-case handler. # sub special_credits( $ ) { # local $Log::TraceMessages::On = 1; die 'usage: special_credits(programme hashref)' if @_ != 1; my $prog = shift; t 'special_credits() ENTRY'; # Caution: we need to make sure $_ is 'live' so updates to it # change the programme, when calling the extractors. # foreach my $pair (grep { pair_ok($_) } @{$prog->{desc}}) { die if not defined; t "testing desc: $pair->[0]"; if (not length $pair->[0]) { local $Log::TraceMessages::On = 1; t 'programme with empty desc:' . d $prog; } if (s/\b([pP])resenteed\b/$1resented/g) { t "fixing spelling mistake!"; return [ $prog ]; } # Regexps to apply to the description (currently only the # first English-language description is matched). The first # element is a subroutine which should alter $_ and return a # name or string of names if it succeeds, undef if it fails to # match. # # The first argument of the subroutine is the programme # itself, but this usually isn't used. In any case, it should # not be modified except by altering $_. # my @extractors = ( # Definitely presenter [ sub { s{(\b[a-z]\w+\s+)(?:(?:guest|virtual|new\s+)?presenters?)\s+($NAMES_RE)}{$1$2}o && return $2; s{((?:^|\.|\?)\s*)($NAMES_RE)\s+(?:(?:presents?)|(?:rounds?\s+up)|(?:introduces?))\b\s*(\.|,|\w|\Z)} {$1 . uc $3}oe && return $2; s{Presenters?\s+($NAMES_RE)}{$1}o && return $1; s{,?\s*[cC]o-?presenters?\s+($NAMES_RE)}{}o && return $1; s{,?\s*[pP]resented by\s+($NAMES_RE)\b\s*(.|,?\s+\w|\Z)}{uc $2}oe && return $1; s{^\s*([hH]eadlines?(?:\s+\S+)?),?\s*[wW]ith\s+($NAMES_RE)\b(?:\.\s*)?}{$1}o && return $2; s{,?\s*(?:[iI]ntroduced|[cC]haired)\s+by\s+($NAMES_RE)(?:\.\s*)?}{}o && return $1; # This last one is special: it adds 'Last in series' # which some other handler might pick up. # s{((?:^|\.|\?)\s*)($NAMES_RE)\s+concludes?\s+the\s+series\b\s*(?:with\b\s*)?(\.|,|\w|\Z)} {$1 . 'Last in series. ' . uc $3}oe && return $2; return undef; }, 'presenter' ], # Leave 'virtual presenter', 'aquatic presenter', # 'new presenter' alone for now # # Might be presenter depending on type of show [ sub { if (hasp($_[0], 'category', sub { $_[0] =~ /\b(?:comedy|drama|childrens?)\b/i }) and not $prog->{credits}->{presenter}) { return undef; } s{^\s*,?\s*[wW]ith\s+($NAMES_RE)\b(?:(?:\.\s*)?$)?}{}o && return $1; s{^\s*(?:[hH]ost\s+)?($NAME_RE) (?:introduces|conducts) (\w)(?![^.,;:!?]*\bto\b)} {uc $2}oe && return $1; s{^\s*(?:[hH]ost\s+)?($NAME_RE)\s+(?:explores|examines)\s*}{}o && return $1; return undef; }, 'presenter' ], [ sub { s{((?:^|\.|\?)\s*)($NAME_RE)\s+interviews\b\s*(\.|,|\w|\Z)}{$1 . uc $3}oe && return $2; return undef; }, 'presenter' ], # FIXME should be 'host' or 'interviewer' # 'with' in quiz shows is guest (maybe) [ sub { return undef unless hasp($_[0], 'category', sub { $_[0] =~ /\b(?:quiz|sports?)\b/i }); s{((?:^|,|\.|\?)\s*)[wW]ith\s*($NAMES_RE)\b(?!\s+among)(\.\s*\S)} {$1 ne ',' ? "$1$2" : $2}oe && return $2; s{((?:^|,|\.|\?)\s*)[wW]ith\s*($NAMES_RE)\b(?!\s+among)(?:\.\s*$)?} {$1 ne ',' ? $1 : ''}oe && return $2; return undef; }, 'guest' ], # 'with' in news/children shows is presenter (equally # dubious). Also a 'with' in a talk show might be # presenter or might be guest, but at least we know it's # not actor. # [ sub { return undef unless hasp($_[0], 'category', sub { $_[0] =~ /\b(?:news|business|economics?|political|factual|talk|childrens?|game show)\b/i }); s{(?:^|,|\.|\?)\s*[wW]ith\s*($NAMES_RE)\b(?:\.\s*)?}{}o && return $1; return undef; }, 'presenter' ], [ sub { # Anything with a 'presenter' does not have actors. return undef if $prog->{credits}->{presenter}; s{(?:[Ww]ith\s+)?[gG]uest\s+star\s+($NAMES_RE)\b\s*[,;.]?\s*}{}o && return $1; s{^($NAMES_RE) (?:co-)?stars? in\s+(?:this\s+)?}{uc $2}oe && return $1; s{\s*($NAMES_RE) (?:co-)?stars?\.?\s*$}{}o && return $1; s{(?:^|\.|\?)\s*($NAMES_RE)\s+(?:co-)?stars?\s+as\s*$}{}o && return $1; return undef; }, 'actor' ], [ sub { # A discussion of 'a film starring Robin Williams' # does not itself feature that actor. # return undef if $prog->{credits}->{presenter}; return undef if hasp($_[0], 'category', sub { $_[0] =~ /\barts\b/i }); s{(?:^|,|\.|\?)\s*[wW]ith\s*($NAMES_RE)\b(?:,|\.|;|$)?}{}o && return $1; s{,?\s*(?:(?:[Aa]lso|[Aa]nd)\s+)?(?:[Cc]o-|[Gg]uest-|[Gg]uest\s+)?[Ss]tarring\s+($NAMES_RE)\s*$}{}o && return $1; return undef; }, 'actor' ], [ sub { s{,?\s*[wW]ith\s+guests?\s+($NAMES_RE)\b(?:\.\s*)?}{}o && return $1; s{((?:^|\.|!|\?)\s*)($NAME_RE)\s+guests(?:$|(?:\s+)|(?:.\s*))}{$1}o && return $2; return undef; }, 'guest' ], [ sub { s{(?:^|\.|!|\?|,)(?:[Ww]ritten\s+)?\s*by\s+($NAMES_RE)\b($|\.)}{$2}o && return $1; return undef; }, 'writer' ], ); # Run our own little hog-butchering algorithm to match each of # the subroutines in turn. # my $matched = 0; EXTRACTORS: foreach my $e (@extractors) { my ($sub, $person) = @$e; t "running extractor for role $person"; my $old_length = length $pair->[0]; my $match; for ($pair->[0]) { $match = $sub->($prog) } if (defined $match) { # Found one or more $person called $match. We add them to # the list unless they're already in there. We use a # per-programme cache of this information to avoid # going through the list each time (basically because # hashes are more Perlish). # t "got list of people: $match"; my @names = list_names($match); t 'that is, names: ' . d \@names; t 'by shortening desc, programme updated to: ' . d $prog; for my $credits ($prog->{credits}) { my %seen; if (lc $person eq 'guest') { # Impossible for someone to be guest as well # as another part, so don't add it if already # listed anywhere. # foreach (keys %$credits) { $seen{$_}++ foreach @{$credits->{$_}}; } } else { # Cannot add this person if they are already # given in the same job, or as a guest. # foreach (@{$credits->{$person}}, @{$credits->{guest}}) { $seen{$_}++ && warn "person $_ seen twice"; } } t 'people already known (or ineligible): ' . d \%seen; foreach (@names) { t "maybe adding $_ as a $person"; push @{$credits->{$person}}, $_ unless $seen{$_}++; } t '$credits->{$person}=' . d $credits->{$person}; } if (length $pair->[0] >= $old_length) { warn "extractor failed to shorten text: now $pair->[0]"; } t 'by adding people, programme updated to: ' . d $prog; $matched = 1; goto EXTRACTORS; # start again from beginning of loop } } if ($matched) { t 'some handlers matched, programme: ' . d $prog; scrub_empty($prog->{desc}); t 'after removing empty things, programme: ' . d $prog; return [ $prog ]; } } return undef; } # has() # # Check whether some attribute of a programme matches a particular # string. For example, does the programme have the category 'quiz'? # This means checking all categories of acceptable language. # # has($programme, 'category', 'quiz'); # sub has( $$@ ) { # local $Log::TraceMessages::On = 1; my ($p, $attr, @allowed) = @_; t 'testing whether programme: ' . d $p; t "has attribute $attr in the list: " . d \@allowed; my $list = $p->{$attr}; t 'all [text, lang] pairs for this attr: ' . d $list; return 0 if not defined $list; foreach (grep { pair_ok($_) } @$list) { my ($text, $lang) = @$_; foreach (@allowed) { t "testing if $text matches $_ (nocase)"; return 1 if lc $text eq $_; } } t 'none of them matched, returning false'; return 0; } # hasp() # # Like has() but instead of a list of strings to compare against, # takes a subroutine reference. This subroutine will be run against # all the text strings of suitable language in turn until it matches # one, when true is returned. If none match, returns false. # # Parameters: # ref to programme hash # name of key to look under # subroutine to apply to each value of key with acceptable language # # Returns: whether subroutine gives true for at least one value. # # The subroutine will get the text value passed in $_[0]. # sub hasp( $$$ ) { # local $Log::TraceMessages::On = 1; my ($p, $attr, $sub) = @_; die "expected programme hash as first argument, not $p" if ref $p ne 'HASH'; t 'testing whether programme: ' . d $p; t "has a value for attribute $attr that makes sub return true"; # FIXME commonize this with has(). my $list = $p->{$attr}; t 'all [text, lang] pairs for this attr: ' . d $list; return 0 if not defined $list; foreach (grep { pair_ok($_) } @$list) { my ($text, $lang) = @$_; t "testing if $text matches"; return 1 if $sub->($text); } t 'none of them matched, returning false'; return 0; } # special_new_series() # # Contrary to first appearances, the <new /> element in the XML isn't # to indicate a new series - it means something stronger, a whole new # show (not a new season of an existing show). But you can represent # part of the meaning of 'new series' within the episode-num # structure, because obviously a new series means that this is the # first episode of the current series. # # This handler is mostly here to get rid of the 'New series' text at # the start of descriptions, to try and make output from different # grabbers look the same. # sub special_new_series( $ ) { die 'usage: special_new_series(programme)' if @_ != 1; my $p = shift; # Just assume that if it contains 'New series' at the start then # it's English. # my $is_new_series = 0; foreach (@{$p->{desc}}) { for ($_->[0]) { if (s/^New series(?:\.\s*|$)// or s/^New series (?:of (?:the )?)?(\w)/uc $1/e ) { $is_new_series = 1; } } } return undef if not $is_new_series; if (defined $p->{'episode-num'}) { foreach (@{$p->{'episode-num'}}) { my ($content, $system) = @$_; next unless $system eq 'xmltv_ns'; $content =~ m!^\s*(\d+/\d+|\d+|)\s*\.\s*(\d+/\d+|\d+|)\s*\.\s*(\d+/\d+|\d+|)\s*$! or warn("badly formed xmltv_ns episode-num: $content"), return [ $p ]; my ($season, $episode, $part) = ($1, $2, $3); if ($episode ne '' and $episode !~ /^0/) { warn "new series, but episode number $episode"; } elsif ($episode eq '') { # We now know the information that this is the first # episode of the series. # $episode = '0'; $content = "$season . $episode . $part"; $_ = [ $content, $system ]; last; } } } else { # Make a dummy episode num which says nothing other than # this is the first episode of the series. # $p->{'episode-num'} = [ [ ' . 0 . ', 'xmltv_ns' ] ]; } scrub_empty($p->{desc}); return [ $p ]; } # special_year(): take a year at the start of the description and move # it to the 'date' field. # sub special_year( $ ) { die 'usage: special_new_series(programme)' if @_ != 1; my $p = shift; my $year; foreach (@{$p->{desc}}) { if ($_->[0] =~ s/^(\d{4})\s+//) { my $got = $1; if (defined $year and $got ne $year) { warn "found different years $year and $got"; return [ $p ]; } $year = $got; } } return undef if not defined $year; if (defined $p->{date}) { if ($p->{date} !~ /^\s*$year/) { warn "found year $year in programme description, but date $p->{date}"; } } else { $p->{date} = $year; } scrub_empty($p->{desc}); return [ $p ]; } # 'TVM' at start of description means TV movie. sub special_tv_movie( $ ) { die 'usage: special_tv_movie(programme)' if @_ != 1; my $p = shift; my $is_tv_movie = 0; foreach (@{$p->{desc}}) { my $lang = $_->[1]; if (not defined $lang or $lang =~ /^en/) { if ($_->[0] =~ s/^TVM\b\s*//) { $is_tv_movie = 1; } } } return undef if not $is_tv_movie; add($p, 'category', 'TV movie'); scrub_empty($p->{desc}); return [ $p ]; } # '(T)' in description means teletext subtitles. But this should run # after doing any splitting and other stuff. # sub special_teletext_subtitles( $ ) { die 'usage: special_teletext_subtitles(programme)' if @_ != 1; my $p = shift; my $has_t = 0; foreach (@{$p->{desc}}) { if ($_->[0] =~ s/\s*\(T\)\s*$//) { $has_t = 1; } } return undef if not $has_t; if (defined $p->{subtitles}) { foreach (@{$p->{subtitles}}) { return [ $p ] if defined $_->{type} and $_->{type} eq 'teletext'; } } push @{$p->{subtitles}}, { type => 'teletext' }; scrub_empty($p->{desc}); return [ $p ]; } # Remove stock phrases that have no meaning. sub special_useless( $ ) { die 'usage: special_useless(programme)' if @_ != 1; my $p = shift; # FIXME need to commonize hog-butchering with special_credits(). my $changed = 0; foreach (@{$p->{desc}}) { for ($_->[0]) { $changed |= s/^(?:a\s+|)round-up\s+of\s+(\w)/uc $1/ie; $changed |= s/^(\w+[^s])\s+round-up\.?\s*$/$1 . 's'/ie; $changed |= s/((?:^|\.|!|\?)\s*)Coverage\s+of\s+(\w)/$1 . uc $2/e; } } return [ $p ] if $changed; return undef; } # special_radio4() # # Split Radio 4 into FM and LW. # sub special_radio4( $ ) { die 'usage: special_radio4(programme)' if @_ != 1; my $p = shift; return undef if $p->{channel} ne 'radio4'; for ($p->{title}) { if (s/^\(FM\)\s+//) { $p->{channel} = 'radio4-fm'; return [ $p ]; } if (s/^\(LW\)\s+//) { $p->{channel} = 'radio4-lw'; return [ $p ]; } my %fm = ( %$p, channel => 'radio4-fm' ); my %lw = ( %$p, channel => 'radio4-lw' ); return [ \%fm, \%lw ]; } } # special_split_title() # # In addition to the 'programme tacked onto the end of another' # handled by add_clumpidx, you also sometimes see two programmes # totally sharing an entry. For example 'News; Shipping Forecast'. # sub special_split_title( $ ) { die 'usage: special_split_title(programme)' if @_ != 1; my $p = shift; return undef if $p->{title} !~ tr/;//; # Split the title at ; and make N identical programmes one with # each title. The programme details are given to only the last of # the programmes - in the listings data we're getting, normally # the insignificant programme comes first with the main feature # last, as in 'News; Radio 3 Lunchtime Concert'. # # List of elements which are meta-data and should be kept for all # the programmes we split into - the rest are given only to the # last programme. # my %meta = (start => 1, stop => 1, 'pdc-start' => 1, 'vps-start' => 1, showview => 1, videoplus => 1, channel => 1); # but not clumpidx! my %p_meta; foreach (grep { $meta{$_} } keys %$p) { $p_meta{$_} = $p->{$_}; } my @r; my @titles = split /\s*;+\s*/, $p->{title}; for (my $i = 0; $i < @titles - 1; $i++) { push @r, { %p_meta, title => $titles[$i], clumpidx => ( "$i/" . scalar @titles ) }; } push @r, { %$p, title => $titles[-1], clumpidx => ("$#titles/" . scalar @titles) }; return \@r; } # special_film() # sub special_film( $ ) { die 'usage: special_film(programme)' if @_ != 1; my $p = shift; if (not defined $p->{'sub-title'} or $p->{'sub-title'} ne '(Film)') { return undef; } warn "replacing category $p->{category} with 'film'" if defined $p->{category}; $p->{category} = 'film'; undef $p->{'sub-title'}; if (defined $p->{desc} and $p->{desc} =~ s/^(\d{4})\s+//) { warn "found year $1 in description, replacing date $p->{date}" if defined $p->{date}; $p->{date} = $1; } return [ $p ]; } # add() # # Add a piece of human-readable information to a particular slot, but # only if it isn't there already. For example add the category # 'music', but only if that category isn't already set. This is for # keys that take multiple values and each value is a [ text, lang ] # pair. The language is assumed to be English. # # Parameters: # programme hash to add to # name of key # textual value to add # # Returns: whether the programme was altered. # sub add( $$$ ) { my ($p, $k, $v) = @_; if (defined $p->{$k}) { foreach (@{$p->{$k}}) { return 0 if $_->[0] eq $v; } } push @{$p->{$k}}, [ $v, $LANG ]; return 1; } # scrub_empty(): remove empty strings from a list of [text, lang] # pairs. # # Parameters: zero or more listrefs # # Modifies lists passed in, removing all [ '', whatever ] pairs. # sub scrub_empty( @ ) { foreach (@_) { @$_ = grep { length $_->[0] } @$_; } } # Make sure that a programme altered by a special handler does not # spill outside its alotted timespan. This is just a sanity check # before fix_clumps() does its stuff. In a future version we might # remove this restriction and allow special handlers to move # programmes outside their original timeslot. # # Parameters: # original programme # ref to list of new programmes # sub check_time_fits( $$ ) { my $orig = shift; my @new = @{shift()}; my $o_start = pd($orig->{start}); die if not defined $o_start; my $o_stop = pd($orig->{stop}); foreach (@new) { my $start = pd($_->{start}); die if not defined $start; if (Date_Cmp($start, $o_start) < 0) { die "programme starting at $o_start was split into one starting at $start"; } if (defined $o_stop) { my $stop = pd($_->{stop}); if (defined $stop and Date_Cmp($o_stop, $stop) < 0) { die "programme stopping at $o_stop was split into one stopping at $stop"; } } } } # Another sanity check. sub check_same_channel( $ ) { my $progs = shift; my $ch; foreach my $prog (@$progs) { for ($prog->{channel}) { if (not defined) { t 'no channel! ' . d $prog; croak 'programme has no channel'; } if (not defined $ch) { $ch = $_; } elsif ($ch eq $_) { # Okay. } else { # Cannot use croak() due to this error message: # # Bizarre copy of ARRAY in aassign at /usr/lib/perl5/5.6.0/Carp/Heavy.pm line 79. # local $Log::TraceMessages::On = 1; t 'same clump, different channels: ' . d($progs->[0]) . ' and ' . d($prog); die "programmes in same clump have different channels: $_, $ch"; } } } } # There is a very hard to track down bug where stop times mysteriously # get set to something ridiculous. It varies from one perl version to # another (hence the version check at the top) but still occurs even # with 5.6.1. To track it down I have isolated all code that sets # stop times in this subroutine. # sub set_stop_time( $$ ) { my $p = shift; my $s = shift; if ($SLOW) { # Another mysterious-bug-preventing line, see no_shared_scalars(). my $dummy = "$s"; $p->{stop} = $s; } else { # Don't set stop times at all. delete $p->{stop}; } } # More debugging aids. sub cst( $ ) { my $p = shift; croak "prog $p->{title}->[0]->[0] has bogus stop time" if exists $p->{stop} and $p->{stop} eq 'boogus FIXME XXX'; } sub no_shared_scalars( $ ) { my %seen; foreach my $h (@{$_[0]}) { foreach my $k (keys %$h) { my $ref = \ ($h->{$k}); my $addr = "$ref"; $seen{$addr}++ && die "scalar $addr seen twice"; } } } �������������xmltv-1.4.0/filter/tv_grep.PL�����������������������������������������������������������������������0000664�0000000�0000000�00000002756�15000742332�0016064�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Generate tv_grep from tv_grep.in. This processing is necessary for # the pod documentation, which should be partly autogenerated from # information provided by the Grep.pm library. # # We could avoid this step if we had the documentation in a separate # file - but then we'd need a separate step for producing # documentation. The input file tv_grep.in is also a legal Perl # program, so debugging should not be any harder than it was. use IO::File; use XMLTV; require './filter/Grep.pm'; my $out = shift @ARGV; die "no output file given" if not defined $out; my $in = 'filter/tv_grep.in'; my $in_fh = new IO::File "< $in" or die "cannot read $in: $!"; my $out_fh = new IO::File "> $out" or die "cannot write to $out: $!"; while (<$in_fh>) { if (/^\s*\@PROGRAMME_CONTENT_TESTS\s*$/) { # Query XMLTV.pm to find out what keys of programme exist. # This is rather a duplicate of the usage message in tv_grep # itself: should unify one day. # my %key_type = %{XMLTV::list_programme_keys()}; foreach (sort keys %key_type) { my ($arg, $matcher) = @{XMLTV::Grep::get_matcher($_)}; if (not defined $arg) { print $out_fh "B<--$_>\n\n"; } elsif ($arg eq 'regexp') { print $out_fh "B<--$_> REGEXP\n\n"; } elsif ($arg eq 'empty') { print $out_fh "B<--$_> ''\n\n"; } else { die "bad arg type from get_matcher(): $arg"; } } } else { print $out_fh $_; } } close $out_fh or die "cannot close $out: $!"; close $in_fh or die "cannot close $in: $!"; ������������������xmltv-1.4.0/filter/tv_grep.in�����������������������������������������������������������������������0000775�0000000�0000000�00000062574�15000742332�0016166�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w =pod =head1 NAME tv_grep - Filter programmes and channels from an XMLTV listings file. =head1 SYNOPSIS C<tv_grep [--help] [--output FILE] [--ignore-case|-i] (EXPR | REGEXP) [FILE...]> =head1 DESCRIPTION Reads XMLTV listings data and writes out data containing some of the programmes and channels from the original. Which programmes and channels are left in the output is controlled by the regexp or Boolean expression given. Simple usage is B<tv_grep REGEXP [FILE...]>, where B<REGEXP> is a Perl 5 regular expression (see L<perlre(1)>). This finds all <programme> elements containing text matching the regexp. The channels are left unchanged, that is, all the <channel> elements are output. For more advanced searches, you can specify a Boolean expression (which loosely follows the style of find(1)). There are many tests for matching programme content against a regular expression, a few for matching channels and programmes on those channels, and a few special tests. =head1 OPTIONS B<--output FILE> write to FILE rather than standard output. B<--ignore-case>, B<-i> treat all regular expression matches as case insensitive. =head1 EXPRESSIONS =head2 PROGRAMME CONTENT TESTS The tests for programme content match against particular attributes or subelements of the <programme> element in the XML data. Each test is named the same as the attribute or element it matches. Those which take a regexp as an argument match if the programme contains at least one attribute or element of the same name whose content matches the regexp. Those which do not take a regexp match if the programme simply contains one or more attributes or elements of that name. Some elements may or may not have content - they may just be empty. The regular expression '' (the empty string) matches any element, even one with empty content, while a nonempty regular expression matches only those with content. For example, B<--desc Racing> matches a programme if the programme has at least one <desc> element whose content contains 'Racing'. B<--stop ''> (the second argument is the empty string) matches a programme if the programme gives a stop time. There are some elements where only yes/no matching is possible, where you cannot give a regexp to query the elementE<39>s content. For these the second B<''> argument is mandatory. For example B<--previously-shown ''> will match programmes which have that element, but a test of B<--previously-shown foo> will give an error because querying the content of previously-shown is not implemented. The additional empty-string argument is to leave room for future expansion. The content tests are generated from the XMLTV file format. The current set of programme content tests is: @PROGRAMME_CONTENT_TESTS While every attribute and subelement of <programme> elements is included in the above list, for some of them it is normally more convenient to use the special tests described below. =head2 CHANNEL TESTS There are three tests for channels. These filter both <programme> and <channel> elements: if a channel is filtered out then all programmes on that channel are too. B<--channel-name REGEXP> True if the channel has a <name> whose content matches REGEXP. B<--channel-id CHANNEL_ID> True if the channelE<39>s XMLTV id is exactly equal to CHANNEL_ID. B<--channel-id-exp REGEXP> True if the channel has a <id> whose content matches REGEXP. =head2 TIME TESTS Normally you donE<39>t want to test time strings with a regular expression but rather compare them with some other time. There are two tests for this. B<--on-after DATE> True if the programme will be broadcast at or after DATE, or will be part of the way through broadcasting at DATE. (Note: a programme is considered to be broadcasting from its start time, up to but not including its stop time.) DATE can be given in any sane date format; but if you donE<39>t specify the timezone then UTC is assumed. To remove all the programmes you have already missed, try B<--on-after now>. B<--on-before DATE> True if the programme will be broadcast wholly before DATE, or if it will be part of the way through broadcasting at DATE. To remove all the programmes that havenE<39>t yet begun broadcasting, try B<--on-before now>. You can use B<--on-before> and B<--on-after> together to find all programmes which are broadcasting at a certain time. Another way of thinking about these two tests is that B<--on-after now> gives 'all programmes you could possibly still watch, although perhaps only catching the end'. B<--on-before now> gives 'all programmes you could possibly have seen, even if only the start'. B<--eval CODE> Evaluate CODE as Perl code, use the return value to decide whether to keep the programme. The Perl code will be given the programme data in $_ in XMLTV.pm hash format (see L<XMLTV>). The code can actually modify the programme passed in, which can be used for quick fixups. This option is not intended for normal use, but as an escape in case none of the existing tests is what you want. If you develop any useful bits of code, please submit them to be included as new tests. =head2 LOGICAL OPERATORS B<EXPR1 --and EXPR2>, B<EXPR1 -and EXPR2>, B<EXPR1 EXPR2> B<EXPR1 --or EXPR2>, B<EXPR1 -or EXPR2> B<--not EXPR>, B<-not EXPR>, B<! EXPR> Of these, 'not' binds tightest, affecting the following predicate only. 'and' is next, and 'or' binds loosest. =head1 SEE ALSO L<xmltv(5)>, L<perl(1)>, L<XMLTV(3)>. =head1 AUTHOR Ed Avis, ed@membled.com =head1 BUGS The --on-after test cannot be totally accurate when the input data did not give a stop time for a programme. In this case we assume the stop time is equal to the start time. This filters out more programmes than if the stop time were given. There will be a warning if this happens more than once on any single channel. It could be worthwhile to filter the listings data through L<tv_sort(1)> beforehand to add stop times. Similar remarks apply to --on-before: if the stop time is missing we assume it is equal to the start time, and this can mean leaving in a programme which, if it had a stop time, would be removed. The assumption of UTC for dates without timezones could be considered a bug. Perhaps the user input should be interpreted according to the local timezone. OTOH, if the data has no timezones and neither does the user input, then things will work as expected. The simple usage is the only way to match against all a programmeE<39>s content because some things (like <credits>) do not have programme content tests defined. It actually works by stringifying the whole programme and regexp matching that, which means that it could give wrong results for regular expressions containing quote characters or some punctuation symbols. This is not particularly likely to happen in practice. Some listings sources generate timeslots containing two or more programmes in succession. This is represented in XMLTV with the 'clumpidx' attribute. If tv_grep selects only some of the programmes from a clump, then it will alter the clumpidx of those remaining to make it consistent. This is maybe not ideal, perhaps the clumpidx should be left unchanged so itE<39>s obvious that something is missing, but at least it prevents complaints from other XMLTV tools about badly formed clumps. The clumpidx handling does mean that tv_grep is not always idempotent. =cut use strict; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use XMLTV::Clumps qw(clump_relation fix_clumps); use XMLTV::Grep qw(get_matcher); use XMLTV::TZ qw(parse_local_date); use XMLTV::Date; use Data::Dumper; use Date::Manip; # We will call Date_Init() later on, but to start with, parse # command-line arguments in the local timezone. # # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } } sub usage( ;$ ); # too complex for XMLTV::Usage sub all_text( $$ ); sub abbrev( $$ ); sub on_after( $ ); sub on_before( $ ); sub whole_programme_regexp( $ ); my $ignore_case = 0; # global flag my %key_type = %{XMLTV::list_programme_keys()}; # Tests to apply. We store them as a disjunction of conjunctions, for # example (a && b && c) || (d && e) || (f && g). # # We keep tests separately for programmes and channels: but really the # channel tests are just extras and not important. When we add a # programme test referring to channels, we add a channel test to go # with it so that the list of channels in the output is trimmed. But # remember that the tests primarily are there to filter programmes. # my (@prog_conjs, @curr_prog_conj); my (@chan_conjs, @curr_chan_conj); # Hash mapping regexp -> channel id -> true/undef (see later) my %ch_name; my @ch_regexps; # regexps to populate %ch_name with my @chid_regexps; # regexps for matching with channel id # Prepare an OptionAbbrev object with all the long options we expect # to find. # my $oa = new OptionAbbrev(qw(--ignore-case --help --output --channel-id --channel-name --channel-id-exp --on-after --on-before --eval --and --or --not)); # Add the autogenerated options. We add even those which aren't # supported just so we can annoy the user with an error message. # $oa->add(map { "--$_" } keys %key_type); # Secret debugging option. if (@ARGV and $ARGV[0] eq '--echo') { print "arguments enclosed by '':\n"; print "'$_'\n" foreach @ARGV; exit(); } my ($output, $regexp, $ended_options, @filenames); my $not = 0; # next arg expected to be a predicate, and negated while (@ARGV) { my $arg = shift @ARGV; t 'processing arg: ' . d $arg; unless ($ended_options) { if ($arg eq '--') { $ended_options = 1; next; } my @long_opts = $oa->match($arg); my $lo; if (@long_opts >= 2) { die "option $arg ambiguous: could be any of @long_opts\n"; } elsif (@long_opts == 1) { $lo = $long_opts[0]; die unless $lo =~ /^--/; } elsif (@long_opts == 0) { die "unknown long option $arg\n" if $arg =~ /^--/; # Otherwise okay. } else { die } if (defined $lo and $lo eq '--help') { usage(1); } if (defined $lo and $lo eq '--output') { if (defined $output) { die "option --output can be given only once\n"; } $output = shift @ARGV; if (not defined $output) { die "option --output requires an argument, a filename\n"; } next; } if ($arg eq '-i' or (defined $lo and $lo eq '--ignore-case')) { $ignore_case = 1; # no warning if given twice next; } # Logical operators --and, --or and --not. --not binds the # most tightly, and affects only the following predicate. # --and is next and binds together predicates in a # conjunction. --or binds loosest and joins together two # conjunctions. # if ($arg eq '-and' or (defined $lo and $lo eq '--and')) { next; } elsif ($arg eq '-o' or $arg eq '-or' or (defined $lo and $lo eq '--or')) { # Finished with this conjunction, start a new one. The # final test is a disjunction of all conjunctions. # # Won't be this easy if we ever implement ( and ). # if (not @curr_prog_conj) { warn "nothing to the left of $arg, should use as EXPR1 $arg EXPR2\n"; usage(); } push @prog_conjs, [ @curr_prog_conj ]; # make a copy @curr_prog_conj = (); # And the same for the channel predicates (if any). push @chan_conjs, [ @curr_chan_conj ]; @curr_chan_conj = (); next; } elsif ($arg eq '!' or $arg eq '-not' or (defined $lo and $lo eq '--not')) { $not = 1; die "$arg requires a predicate following" if not @ARGV; next; } # Called to add a predicate to the current conjunction, taking # account of any preceding 'not'. # my $add_to_prog_conj = sub( $ ) { my $pred = shift; if ($not) { push @curr_prog_conj, sub { not $pred->(@_) }; } else { push @curr_prog_conj, $pred; } }; # Similarly for channel filtering. my $add_to_chan_conj = sub( $ ) { my $pred = shift; if ($not) { push @curr_chan_conj, sub { not $pred->(@_) }; } else { push @curr_chan_conj, $pred; } }; # See if it's a predicate. if ($arg eq '-e' or (defined $lo and $lo eq '--eval')) { my $code = shift @ARGV; die "-e requires an argument, a snippet of Perl code" if not defined $code; my $pred = eval "sub { $code }"; if ($@) { die "-e $code: $@\n"; } if (not defined $pred) { # Shouldn't happen, I think. die "-e $code failed for some reason"; } $add_to_prog_conj->($pred); $not = 0; next; } if (defined $lo and $lo eq '--on-after') { my $date = shift @ARGV; die "--on-after requires an argument, a date and time" if not defined $date; my $pd = parse_local_date($date); die "--on-after $date: invalid date\n" if not defined $pd; t 'parsed date argument: ' . d $pd; $add_to_prog_conj->(sub { on_after($pd) }); $not = 0; next; } if (defined $lo and $lo eq '--on-before') { my $date = shift @ARGV; die "--on-before requires an argument, a date and time" if not defined $date; my $pd = parse_local_date($date); die "--on-before $date: invalid date\n" if not defined $pd; t 'parsed date argument: ' . d $pd; $add_to_prog_conj->(sub { on_before($pd) }); $not = 0; next; } if (defined $lo and $lo eq '--channel-id') { my $id = shift @ARGV; die "--channel-id requires an argument, an XMLTV internal channel id\n" if not defined $id; # We know every programme has 'channel' and every channel # has 'id'. # $add_to_prog_conj->(sub { $_->{channel} eq $id }); $add_to_chan_conj->(sub { $_->{id} eq $id }); $not = 0; next; } if (defined $lo and $lo eq '--channel-id-exp') { my $regexp = shift @ARGV; die "--channel-id-exp requires an argument, a Perl regular expression\n" if not defined $regexp; # reuses some --channel-name processing # $add_to_prog_conj->(sub { $ch_name{$regexp}->{$_->{channel}} }); $add_to_chan_conj->(sub { $ch_name{$regexp}->{$_->{id}} }); $not = 0; push @chid_regexps, $regexp; next; } if (defined $lo and $lo eq '--channel-name') { my $regexp = shift @ARGV; die "--channel name requires an argument, a Perl regular expression\n" if not defined $regexp; # The matchers check against a global hash mapping # channel-name regexps to channel ids to true/undef. We # must remember to create this hash later when we've read # in the channels. # $add_to_prog_conj->(sub { $ch_name{$regexp}->{$_->{channel}} }); $add_to_chan_conj->(sub { $ch_name{$regexp}->{$_->{id}} }); $not = 0; push @ch_regexps, $regexp; next; } if (defined $lo) { # Must be one of the autogenerated ones like --title. $lo =~ /^--(.+)/ or die "matched long option $lo, no --"; my $key = $1; t "getting matcher for key $key"; my ($arg_type, $matcher) = @{get_matcher($key, $ignore_case)}; t 'expects extra argument: ' . d $arg_type; my $s; if (not defined $arg_type) { t 'no extra argument wanted'; $s = $matcher; } elsif ($arg_type eq 'regexp') { t 'expects a regexp'; my $arg = shift @ARGV; t 'got arg: ' . d $arg; die "$lo requires an argument, a Perl regular expression\n" if not defined $arg; $s = sub { $matcher->($arg) }; } elsif ($arg_type eq 'empty') { t 'expects empty string'; my $arg = shift @ARGV; t 'got arg: ' . d $arg; die "$lo requires an argument, which currently must be the empty string\n" if $arg ne ''; $s = $matcher; } else { die "bad arg type $arg_type" } $add_to_prog_conj->($s); $not = 0; next; } # It wasn't a predicate. Just check that the previous option # wasn't --not, since that requires a predicate to follow. # die "argument '$arg' follows 'not', but isn't a predicate" if $not; } # It wasn't an option, see if it's a regexp or filename. if (not $ended_options and $arg =~ /^-/) { die "bad option $arg\n"; } # A regular expression is allowed only in the simple case where we # haven't got any of the fancy boolean tests. # if (not defined $regexp and not @prog_conjs and not @curr_prog_conj) { $regexp = $arg; next; } else { push @filenames, $arg; next; } } push @prog_conjs, \@curr_prog_conj if @curr_prog_conj; push @chan_conjs, \@curr_chan_conj if @curr_chan_conj; if (not @prog_conjs and not defined $regexp) { warn "neither boolean tests nor regexp given\n"; usage(); } elsif (not @prog_conjs and defined $regexp) { t "no predicates, but regexp $regexp"; @prog_conjs = ([ sub { whole_programme_regexp($regexp) } ]); } elsif (@prog_conjs and not defined $regexp) { t 'predicates given, not simple regexp'; } elsif (@prog_conjs and defined $regexp) { warn "bad argument $regexp\n"; usage(); } t '\@prog_conjs=' . d \@prog_conjs; t '\@chan_conjs=' . d \@chan_conjs; # No test for @chan_conjs since there is no test which weeds out # channels but does not weed out programmes. (How could there be?) # # Now we have finished parsing dates in arguments, go to UTC mode to # parse the files. # t 'setting Date::Manip timezone to UTC'; if (int(Date::Manip::DateManipVersion) >= 6) { Date_Init("SetDate=now,UTC"); } else { Date_Init("TZ=UTC"); } @filenames = ('-') if not @filenames; my ($encoding, $credits, $ch, $progs) = @{XMLTV::parsefiles(@filenames)}; #local $Log::TraceMessages::On = 1; # Prepare the channel name lookup. my %seen_ch_id; foreach my $ch_id (keys %$ch) { $seen_ch_id{$ch_id}++ && die "duplicate channel id $ch_id\n"; my $ch = $ch->{$ch_id}; die if not defined $ch; my %seen_re; foreach my $re (@ch_regexps) { next if $seen_re{$re}++; my $matched = 0; if (exists $ch->{'display-name'}) { foreach (map { $_->[0] } @{$ch->{'display-name'}}) { if ($re eq '' or ($ignore_case ? /$re/i : /$re/)) { $matched = 1; last; } } } if ($matched) { $ch_name{$re}->{$ch_id}++ && die; } } } # Prepare the channel id lookup. my %seen_chid_id; foreach my $ch_id (keys %$ch) { $seen_chid_id{$ch_id}++ && die "duplicate channel id $ch_id\n"; my $ch = $ch->{$ch_id}; die if not defined $ch; my %seen_re; foreach my $re (@chid_regexps) { next if $seen_re{$re}++; my $matched = 0; if ($re eq '' or ($ignore_case ? $ch_id =~ /$re/i : $ch_id =~ /$re/)) { $matched = 1; } if ($matched) { $ch_name{$re}->{$ch_id}++ && die; } } } # Filter channels. This has an effect only for the --channel-id and # --channel-name predicates; we do not drop channels simply because no # programmes remained on them after filtering. # my %new_ch; if (@chan_conjs) { CH: foreach my $ch_id (keys %$ch) { local $_ = $ch->{$ch_id}; CONJ: foreach my $conj (@chan_conjs) { foreach my $test (@$conj) { # Every test in the conjunction must succeed. next CONJ if not $test->(); } # They all succeeded, the channel should be kept. $new_ch{$ch_id} = $_; next CH; } # All the conjunctions failed, won't write. } } else { # No tests specifically affecting channels, keep the full listing. %new_ch = %$ch; } # Filter programmes. my @new_progs; my $related = clump_relation($progs); PROG: foreach (@$progs) { t 'filtering prog: ' . d $_; CONJ: foreach my $conj (@prog_conjs) { t 'testing against all of conjunction: ' . d $conj; foreach my $test (@$conj) { t 'testing condition: ' . d $test; if ($test->()) { t 'passed, onto next condition in conj (if any)'; } else { t 'failed, so failed this conj'; next CONJ; } } t 'passed all tests in conj, finished with prog'; push @new_progs, $_; next PROG; } t 'failed at least one test in all conjs, not keeping'; fix_clumps($_, [], $related); } # All done, write the new programmes and channels. t 'finished grepping, writing'; my %w_args = (); if (defined $output) { my $fh = new IO::File ">$output"; die "cannot write to $output\n" if not $fh; %w_args = (OUTPUT => $fh); } XMLTV::write_data([ $encoding, $credits, \%new_ch, \@new_progs ], %w_args); exit(); # Parameter: if true, write 'help message' rather than 'usage # message', ie write to stdout and exit successfully. # sub usage( ;$ ) { my $is_help = shift; $is_help = 0 if not defined $is_help; my $msg = <<END usage: $0 [--help] [--output FILE] [--ignore-case|-i] (EXPR | REGEXP) [FILE] where EXPR may consist of (programme content matches) END ; foreach (sort keys %key_type) { # (Assume ignore-case flag does not affect argument syntax.) my $arg_type = get_matcher($_, 0)->[0]; if (not defined $arg_type) { $msg .= " --$_\n"; } elsif ($arg_type eq 'regexp') { $msg .= " --$_ REGEXP\n"; } elsif ($arg_type eq 'empty') { # Can query on this only for presence. $msg .= " --$_ ''\n"; } else { die } } $msg .= <<END (channel matches) --channel-name REGEXP --channel-id CHANNEL_ID --channel-id-exp REGEXP (special tests) --on-after DATE --on-before DATE --eval PERL_CODE (logical operators) --not EXPR EXPR1 [--and|-and] EXPR2 EXPR1 [--or|-or|-o] EXPR2 --and is implicit and may be omitted. END ; if ($is_help) { print $msg; exit(0); } else { print STDERR $msg; exit(1); } } # all_text() # # Get all pieces of text for a particular programme attribute. # # Parameters: # programme hashref # attribute name, eg 'title', 'desc' # # Returns: list of text strings for that attribute # # I wrote Lingua::Preferred::acceptable_lang() especially for this # routine but then realized that when grepping you probably don't care # about viewing only those strings applicable to the current language. # sub all_text( $$ ) { my ($p, $key) = @_; return () if not $p->{$key}; return map { $_->[0] } @{$p->{$key}}; } #### # Boolean tests. These work on the programme $_ and return true or # false. Their behaviour should be affected, if appropriate, by the # global $ignore_case. # my %warned_no_stop; sub on_after( $ ) { my $cutoff = shift; # local $Log::TraceMessages::On = 1; t "testing on-after $cutoff"; my $stop = $_->{stop}; t 'stop time: ' . d $stop; if (not defined $stop) { # We use the start time instead, that will lose some shows # crossing the boundary but is mostly accurate. # my $start = $_->{start}; t 'no stop time, using start time: ' . d $start; my $pd = parse_date($start); t 'parsed to: ' . d $pd; my $result = (Date_Cmp($cutoff, $pd) < 0); t 'cutoff before start: ' . $result; if (not $result) { # This programme was dropped, but maybe it wouldn't have # been if it had a stop time. # # We should warn about this: but have an allowance of one # programme per channel without stop time, because you # can reasonably expect that from sorted listings. # unless ($warned_no_stop{$_->{channel}}++) { warn "not all programmes have stop times, " . "cannot accurately filter those on after a certain time\n" . "(maybe filter through tv_sort to add stop times)\n"; } } return $result; } else { my $pd = parse_date($stop); t 'parsed stop time: ' . d $pd; my $r = Date_Cmp($cutoff, $pd) < 0; t 'cutoff before stop: ' . d $r; return $r; } } sub on_before( $ ) { my $cutoff = shift; my $start = $_->{start}; my $pd = parse_date($start); return (Date_Cmp($cutoff, $pd) >= 0); } sub whole_programme_regexp( $ ) { my $re = shift; # Stringify the whole darn thing and match against that. local $_ = Dumper($_); # t 'testing stringified whole programme: ' . d $_; return 1 if $re eq ''; return $ignore_case ? /$re/i : /$re/; } # Class for long option abbreviation. You tell it all the options # you're going to use, and then it will tell you whether a (possibly # abbreviated) argument matches an option unambiguously, ambiguously # could match several options, or matches none. # # Having to roll my own Getopt::Long is getting annoying. I wonder # how much of this code could be shared? # package OptionAbbrev; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } # Constructor. Give a list of long options and/or add() them later. sub new { my $proto = shift; my $class = (ref $proto) || $proto; # The representation of an object is a list of long options known # about. # my $self = []; bless $self, $class; $self->add(@_); return $self; } sub add { my $self = shift; foreach (@_) { die 'long options start with --' unless /^--/; foreach my $already (@$self) { die "option $_ already added" if $_ eq $already; } push @$self, $_; } return $self; } # match() returns a list of possible long options matched. So if the # list has no elements, no match; one element is the unambiguous # match; two or more elements mean ambiguity. # sub match { my ($self, $arg) = @_; t "matching arg $arg against list: " . d $self; return () unless $arg =~ /^--\w/; t 'begins with --, continue'; foreach (@$self) { t "testing for exact match: '$arg' against '$_'"; return ($_) if $arg eq $_; } t 'no exact match, try initial substring'; my @r; foreach (@$self) { t "testing if $arg is initial substring of $_"; push @r, $_ if index($_, $arg) == 0; } t 'returning list of matches: ' . d \@r; return @r; } ������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_imdb��������������������������������������������������������������������������0000775�0000000�0000000�00000034110�15000742332�0015520�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl =pod =head1 NAME tv_imdb - Augment XMLTV listings files with imdb.com data. =head1 SYNOPSIS tv_imdb --imdbdir <dir> [--help] [--quiet] [--download] [--movies-only] [--filesort] [--nosystemsort] [--prepStage (1-9,all)] tv_imdb --imdbdir <dir> [--help] [--quiet] [--with-keywords] [--with-plot] [--movies-only] [--actors NUMBER] [--stats] [--debug] [--output FILE] [FILE...] tv_imdb --imdbdir <dir> --validate-title 'movie title' --validate-year 2004 [--with-keywords] [--with-plot] [--debug] =head1 DESCRIPTION tv_imdb is very similar to tv_cat in semantics (see tv_cat), except whenever a programme appears with "date" entry the title and date are used to look up extra data using XMLTV::IMDB. B<--output FILE> write to FILE rather than standard output. B<--with-keywords> include IMDb keywords in the output file. B<--with-plot> include IMDb plot summary in the output file. B<--actors NUMBER> number of actors from IMDb to add (default=3). B<--quiet> disable all status messages (that normally appear on stderr). B<--download> try to download data files if they are missing (in --prepStage). B<--stats> output grab stats (stats output disabled in --quiet mode). B<--debug> output info from movie matching B<--movies-only> only augment programs that look like movie listings (4 digit E<39>dateE<39> field). All programs are checked against imdb.com data (unless --movies-only is used). For the purposes of tv_imdb, an "exact" match is defined as a case insensitive match against imdb.com data (which may or may not include the transformation of E<39>&E<39> to E<39>andE<39> and vice-versa. If the program includes a 4 digit E<39>dateE<39> field the following matches are attempted, with the first successful match being used: B<1.> an "exact" title/year match against movie titles is done B<2.> an "exact" title match against tv series (and tv mini series) B<3.> an "exact" title match against movie titles with production dates within 2 years of the E<39>dateE<39> value. Unless --movies-only is used, if the program does not include a 4 digit E<39>dateE<39> field the following matches are attempted, the first succeeding match is used: B<1.> an "exact" title match against tv series (and tv mini series) When a match is found in the imdb.com data the following is applied: B<1.> the E<39>titleE<39> field is set to match exactly the title from the imdb.com data. This includes modification of the case to match and any transformations mentioned above. B<2.> if the match is a movie, the E<39>dateE<39> field is set to imdb.com 4 digit year of production. B<3.> the type of match found (Movie, TV Movie, Video Movie, TV Series, or TV Mini Series) is placed in the E<39>categoriesE<39> field. B<4.> the url to the www.imdb.com page is added B<5.> the director is added if the match was a movie or if only one director is listed in the imdb.com data (because some tv series have > 30 directors) B<6.> the top 3 billing actors are added (use -actors [num] to adjust). B<7.> genres added to E<39>categoriesE<39> field (current list of genres are Short, Drama, Comedy, Documentary, Animation, Adult, Action, Family, Romance, Crime, Thriller, Musical, Adventure, Western, Horror, Sci-Fi, Fantasy, Mystery, War, Film-Noir, Music B<8.> imdb user ratings added to E<39>star-ratingsE<39> field. B<9.> imdb keywords added to E<39>keywordE<39> fields (if --with-keywords used). B<10.> imdb plot summary is added (if --with-plot used). =head1 HOWTO In order to use tv_imdb, you need: B<1.> choose a directory location to use for the tv_imdb database (youE<39>ll need about 1 GB of free space), B<2a.> run E<39>tv_imdb --imdbdir <dir> --prepStage all --downloadE<39> to download the list files from imdb.com. Or, B<2b> If you have a slow network connection you may prefer to omit the '--download' flag and be prompted for what you need to download by hand. See <http://www.imdb.com/interfaces> for the download sites. Then once you have the files rerun without '--download'. Note: '--prepStage' requires up to 520MB of memory. This can be reduced a little by running each prepStage separately, using --prepStage with each of the stages individually (see --help for details). Memory use can be reduced further by using --filesort option when building the database. This will try to use the operating system to sort the interim data files rather than sorting in memory. If this system sort does not work for you then you can use the File::Sort package if it is installed on your system, by also adding the option --nosystemsort (however this method of sorting is very slow). If you specify neither option then Perl will sort the files in memory. If you are only interested in movies, you can reduce the memory required and the size of the database by passing the --movies-only option to the database build, which will exclude tv-series from the database. B<3.> Once you have the database loaded try E<39>cat tv.xml | tv_imdb --imdbdir <dir> > tv1.xmlE<39>. Feel free to report any problems with these steps at https://github.com/XMLTV/xmltv/issues. =head1 TESTING The --validate-title and --validate-year flags can be used to validate the information in the tv_imdb database. For exmple: tv_imdb --imdbdir . --validate-title 'Army of Darkness' --validate-year 1994 =head1 BUGS Could use a --configure step just like the grabbers so you do not have to specify the --imdbdir on the command line every time. Also this could step you through the prep stages with more description of what is being done and what is required. Configure could also control the number of actors to add (since some movies have an awful lot), currently we are adding the top 3. How and what to look up needs to be option driven. Needs some more controls for fine tuning "close" matches. For instance, currently it looks like the North America grabber only has date entries for movies, but the imdb.com data contains made for video movies as well as as real movies, so itE<39>s possible to get the wrong data to be inserted. In this case we may want to say "ignore tv series" and "ignore tv mini series". Along with this, weE<39>d want to define what a "close" match is. For instance does a movie by the same title with a date out by 1 year or 2 years considered a match (currently weE<39>re using 2). Nice to haves include: verification/addition of programme MPAA/VCHIP ratings, addition of imdb.com user ratings (by votes) to programmes. Potentially we could expand to include "country of origin", "description", "writer" and "producer" credits, maybe even "commentator". Heh, if the XMLTV.dtd supported it, we could even include urls to head shots of the actors :) =head1 SEE ALSO L<xmltv(5)> =head1 AUTHOR Jerry Veldhuis, jerry@matilda.com =cut use strict; use warnings; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use Data::Dumper; use Getopt::Long; use XMLTV::Data::Recursive::Encode; use XMLTV::Usage <<END $0: augment listings with data from imdb.com $0 --imdbdir <dir> [--help] [--quiet] [--download] [--filesort] [--prepStage (1-9,all)] $0 --imdbdir <dir> [--help] [--quiet] [--download] [--with-keywords] [--with-plot] [--movies-only] [--actors NUMBER] [--stats] [--debug] [--output FILE] [FILE...] END ; use XMLTV::IMDB; my ($opt_help, $opt_output, $opt_prepStage, $opt_imdbDir, $opt_quiet, $opt_download, $opt_stats, $opt_debug, $opt_movies_only, $opt_with_keywords, $opt_with_plot, $opt_num_actors, $opt_validate_title, $opt_validate_year, $opt_sample, $opt_filesort, $opt_systemsort, ); GetOptions('help' => \$opt_help, 'output=s' => \$opt_output, 'prepStage=s' => \$opt_prepStage, 'imdbdir=s' => \$opt_imdbDir, 'with-keywords' => \$opt_with_keywords, 'with-plot' => \$opt_with_plot, 'movies-only' => \$opt_movies_only, 'actors=s' => \$opt_num_actors, 'quiet' => \$opt_quiet, 'download' => \$opt_download, 'stats' => \$opt_stats, 'debug+' => \$opt_debug, 'validate-title=s' => \$opt_validate_title, 'validate-year=s' => \$opt_validate_year, 'sample=s' => \$opt_sample, 'filesort!' => \$opt_filesort, 'systemsort!' => \$opt_systemsort, ) or usage(0); usage(1) if $opt_help; usage(1) if ( not defined($opt_imdbDir) ); $opt_with_keywords=0 if ( !defined($opt_with_keywords) ); $opt_with_plot=0 if ( !defined($opt_with_plot) ); $opt_num_actors=3 if ( !defined($opt_num_actors) ); $opt_movies_only=0 if ( !defined($opt_movies_only) ); $opt_debug=0 if ( !defined($opt_debug) ); $opt_sample=0 if ( !defined($opt_sample) ); $opt_filesort=0 if ( !defined($opt_filesort) ); $opt_systemsort=1 if ( !defined($opt_systemsort) ); $opt_quiet=(defined($opt_quiet)); if ( !defined($opt_stats) ) { $opt_stats=!$opt_quiet; } else { $opt_stats=(defined($opt_stats)); } $opt_debug=0 if $opt_quiet; if ( defined($opt_prepStage) ) { if ( ! $opt_quiet ) { print STDERR <<END; Building data files. The IMDb data files used by XMLTV were frozen by Amazon in December 2017. No updates will be made to the data files after this date. No new films will be added. If you have a successful build of these data files then there is no reason to build them again unless you suspect your existing files are corrupt. END if ($opt_prepStage eq 'all') { print STDERR <<END; Do you wish to continue with a new IMDb data build? (y/n) END my $yn = <>; # ask for user input chomp($yn); exit(1) if (lc($yn) ne "y"); } } my %options = ('imdbDir' => $opt_imdbDir, 'verbose' => !$opt_quiet, 'showProgressBar' => !$opt_quiet, 'stageToRun' => $opt_prepStage, 'downloadMissingFiles' => $opt_download, 'sample' => $opt_sample, 'filesort' => $opt_filesort, 'systemsort' => $opt_systemsort, 'moviesonly' => $opt_movies_only, ); if ( $opt_prepStage eq "all" ) { my $n=new XMLTV::IMDB::Crunch(%options); if ( !$n ) { exit(1); } for (my $stage=1 ; $stage <= 9 ; $stage++ ) { my $ret=$n->crunchStage($stage); if ( $ret != 0 ) { exit($ret); } } print STDERR "database load complete, let the games begin !\n" if ( !$opt_quiet); exit(0); } else { my $n=new XMLTV::IMDB::Crunch(%options); if ( !$n ) { exit(1); } my $ret=$n->crunchStage(int($opt_prepStage)); if ( $ret == 0 && int($opt_prepStage) == 9 ) { print STDERR "database load complete, let the games begin !\n" if ( !$opt_quiet); } exit($ret); } } elsif ( $opt_download ) { my %options = ('imdbDir' => $opt_imdbDir, 'verbose' => !$opt_quiet, 'showProgressBar' => !$opt_quiet, 'stageToRun' => 'all', 'downloadMissingFiles' => $opt_download, ); my $n=new XMLTV::IMDB::Crunch(%options); if ( !$n ) { exit(1); } exit(0); } my $imdb=new XMLTV::IMDB('imdbDir' => $opt_imdbDir, 'verbose' => $opt_debug, 'cacheLookups' => 1, 'cacheLookupSize' => 1000, 'updateKeywords' => $opt_with_keywords, 'updatePlot' => $opt_with_plot, 'numActors' => $opt_num_actors, ); #$imdb->{verbose}++; if ( my $errline=$imdb->sanityCheckDatabase() ) { print STDERR "$errline"; print STDERR "tv_imdb: you need to use --prepStage to rebuild\n"; exit(1); } if ( !$imdb->openMovieIndex() ) { print STDERR "tv_imdb: open database failed\n"; exit(1); } if ( defined($opt_validate_title) != defined($opt_validate_year) ) { print STDERR "tv_imdb: both --validate-title and --validate-year must be used together\n"; exit(1); } if ( defined($opt_validate_title) && defined($opt_validate_year) ) { my $prog; $prog->{title}->[0]->[0]=$opt_validate_title; $prog->{date}=$opt_validate_year; $imdb->{updateTitles}=0; #print Dumper($prog); my $n=$imdb->augmentProgram($prog, $opt_movies_only); if ( $n ) { $Data::Dumper::Sortkeys = 1; # ensure consistent order of dumped hash #my $encoding; #my $w = new XMLTV::Writer((), encoding => $encoding); #$w->start(shift); #$w->write_programme($n); print Dumper($n); #$w->end(); } $imdb->closeMovieIndex(); exit(0); } # test that movie database works okay my %w_args = (); if (defined $opt_output) { my $fh = new IO::File ">$opt_output"; die "cannot write to $opt_output\n" if not $fh; %w_args = (OUTPUT => $fh); } my $numberOfSeenChannels=0; my $w; my $encoding; # store encoding of input file sub encoding_cb( $ ) { die if defined $w; $encoding = shift; # callback returns the file's encoding $w = new XMLTV::Writer(%w_args, encoding => $encoding); } sub credits_cb( $ ) { $w->start(shift); } my %seen_ch; sub channel_cb( $ ) { my $c = shift; my $id = $c->{id}; $Data::Dumper::Sortkeys = 1; # ensure consistent order of dumped hash if (not defined $seen_ch{$id}) { $w->write_channel($c); $seen_ch{$id} = $c; $numberOfSeenChannels++; } elsif (Dumper($seen_ch{$id}) eq Dumper($c)) { # They're identical, okay. } else { warn "channel $id may differ between two files, " . "picking one arbitrarily\n"; } } sub programme_cb( $ ) { my $prog=shift; # The database made by IMDB.pm is read as iso-8859-1. The xml file may be different (e.g. utf-8). # IMDB::augmentProgram does not re-encode the data it adds, so the output file has invalid characters (bug #440). my $orig_prog = $prog; if (lc($encoding) ne 'iso-8859-1') { # decode the incoming programme $prog = XMLTV::Data::Recursive::Encode->decode($encoding, $prog); } # augmentProgram will now add imdb data as iso-8859-1 my $nprog=$imdb->augmentProgram($prog, $opt_movies_only); if ( $nprog ) { if (lc($encoding) ne 'iso-8859-1') { # re-code the modified programme back to original encoding $nprog = XMLTV::Data::Recursive::Encode->encode($encoding, $nprog); } $prog=$nprog; } else { $prog = $orig_prog; } # we only add movie information to programmes # that have a 'date' element defined (since we need # a year to work with when verifing we got the correct # hit in the imdb data) $w->write_programme($prog); } @ARGV = ('-') if not @ARGV; XMLTV::parsefiles_callback( \&encoding_cb, \&credits_cb, \&channel_cb, \&programme_cb, @ARGV ); # we only get a Writer if the encoding callback gets called if ( $w ) { $w->end(); } if ( $opt_stats ) { print STDERR $imdb->getStatsLines($numberOfSeenChannels); } $imdb->closeMovieIndex(); exit(0); ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_merge�������������������������������������������������������������������������0000664�0000000�0000000�00000033006�15000742332�0015704�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl =pod =head1 NAME tv_merge - Merge (combine) two XMLTV files. =head1 SYNOPSIS tv_merge [-t] -i FILE -m FILE -o FILE =head1 DESCRIPTION Read XMLTV listings from two files and merge them together. Unlike tv_cat (which just joins files) this will update (add/replace/delete) the original XMLTV file with channels and programmes contained in the second file. It works with multiple channels, and will insert any new programmes and delete any overlapping programmes. B<IMPORTANT> The input files must be pre-sorted into datetime within channel order by using the "--by-channel" option to tv_sort: tv_sort --by-channel --output FILE FILE All programmes must have start and stop times. (Note: programmes in the merged-in file I<replace> any in the master file, i.e. data are not updated I<within> programmes) This program uses XML::TreePP which doesn't write <DOCTYPE> definitions in the output file. If you need to add a suitable <DOCTYPE> tag then use the optional -t parameter: tv_merge -t -i FILE -m FILE -o FILE =head1 EXAMPLE To merge all channels/programmes in F<newadditions.xml> into F<master.xml>, and write the output to F<newmaster.xml> with a DOCTYPE tag: tv_merge -t -i master.xml -m newadditions.xml -o newmaster.xml =head1 SEE ALSO L<tv_sort(1)>, L<XMLTV(3)> =head1 AUTHOR Copyright Geoff Westcott, February 2013. This code is distributed under the GNU General Public License v2 (GPLv2). =cut # # Inspired by xmltvmerger.py ( http://niels.dybdahl.dk/xmltvdk/index.php/Xmltvmerger.py ) with bug fixes and enhancements. # # IMPORTANT: the input files must be pre-sorted into datetime within channel order by using the "--by-channel" option to tv_sort # e.g. tv_sort --by-channel --output FILE FILE # # Help: tv_merge -h # use warnings; use strict; use XML::TreePP; use Date::Parse; use POSIX qw(strftime); use Getopt::Std; $Getopt::Std::STANDARD_HELP_VERSION = 1; use Data::Dumper; # Process command line arguments my %opts = (); # hash to store input args getopts("dDqti:o:m:hv", \%opts); # load the args into %opts my $input_file = ($opts{'i'}) ? $opts{'i'} : ""; # main file my $merge_file = ($opts{'m'}) ? $opts{'m'} : ""; # file to merge in my $outfile = ($opts{'o'}) ? $opts{'o'} : ""; my $debug = ($opts{'d'}) ? $opts{'d'} : ""; my $debugmore = ($opts{'D'}) ? $opts{'D'} : ""; my $quiet_mode = ($opts{'q'}) ? $opts{'q'} : ""; my $doctype = ($opts{'t'}) ? $opts{'t'} : ""; if ($opts{'h'}) { VERSION_MESSAGE(); HELP_MESSAGE(); exit; } if ($opts{'v'}) { VERSION_MESSAGE(); exit; } if ( !%opts ) { HELP_MESSAGE(); exit 1; } # Check input file if (! defined $opts{'i'} || ! -r $opts{'i'}) { print "Error: Please provide a valid input file\n"; HELP_MESSAGE(); exit 1; } # Check input file if (! defined $opts{'m'} || ! -r $opts{'m'}) { print "Error: Please provide a valid merge file\n"; HELP_MESSAGE(); exit 1; } # Check ouput file if (! defined $opts{'o'}) { print "Error: Please specify an output file\n"; HELP_MESSAGE(); exit 1; } if ($debugmore) { $debug++; } # Parse the XMLTV file my $tpp = XML::TreePP->new(); $tpp->set( force_array => [ 'channel', 'programme' ] ); # force array ref for some fields $tpp->set( indent => 2 ); $tpp->set( first_out => [ 'channel', 'programme', 'title', 'sub-title', 'desc', 'credits', 'date', 'category', 'keyword', 'language', 'orig-language', 'length', 'icon', 'url', 'country', 'episode-num', 'video', 'audio', 'previously-shown', 'premiere', 'last-chance', 'new', 'subtitles', 'rating', 'star-rating', 'review', 'image' ] ); my $xmltv = $tpp->parsefile( $input_file ); if ($debugmore) { print Dumper($xmltv); } my $xmltv_merge = $tpp->parsefile( $merge_file ); if ($debugmore) { print Dumper($xmltv_merge); } my %xmltv_new_channels; my @xmltv_merged_channels; my @xmltv_merged_progs; # Merge the channels &merge_channels(); # Merge the programmes &merge_programmes(); # format and output the data &write_newxml(); ############################################################################### ############################################################################### sub merge_programmes { # Process the XMLTV data structure # # Assumes all data has start and stop times (note: this isn't a requirement of XMLTV DTD) # # If the files contain >1 channel then they must be pre-sorted into datetime within channel # e.g. tv_sort --by-channel --output FILE FILE # # Rules: # 1. always use the new programme details (todo: bit a bit more clever with merging) # my $prog1 = $xmltv->{tv}->{programme}; my $prog2 = $xmltv_merge->{tv}->{programme}; if ($debugmore) { print Dumper("Incoming programmes :", $prog1 ); } if ($debugmore) { print Dumper("Merging programmes :", $prog2 ); } my ($p1_channel, $p1_start, $p1_stop) = ('', 0 , 0); my ($p2_channel, $p2_start, $p2_stop) = ('', 0 , 0); my $i = 0; my ($thischannel, $lastchannel) = ('', ''); my $p2count = scalar (@{ $prog2 }); for my $p (@{ $prog1 }) { $thischannel = $p->{-channel} if $thischannel eq ''; $p1_channel = $p->{-channel}; $p1_start = to_timestamp( $p->{-start} ); $p1_stop = to_timestamp( $p->{-stop} ); if ($p2_stop == -1) { # 'new' file channel has changed # copy any remaining 'old' records for this channel, then insert the new channel's programmes if ($p1_channel eq $lastchannel) { push @xmltv_merged_progs, $p; # copy the old record next; } # old channel has changed = so we have finished copying old records and can now insert new channel my $insertchannel = $p2_channel; while ($p2_channel eq $insertchannel) { push @xmltv_merged_progs, $prog2->[$i]; $i++; if ($i < $p2count) { $p2_channel = $prog2->[$i]->{-channel}; } else { $p2_channel = ''; # no more new records, end the loop } } $p2_stop = 0; } if ($p2_stop != 0 && $p1_start < $p2_stop && $p1_channel eq $p2_channel) { # skip old recs until start time >= last new rec's stop time if ($debug) { print "skippy $p1_start $p2_stop\n"; } next; } if ($i < $p2count) { $p2_channel = $prog2->[$i]->{-channel}; $p2_start = to_timestamp( $prog2->[$i]->{-start} ); $p2_stop = to_timestamp( $prog2->[$i]->{-stop} ); if ($debug) { print "$p1_start $p2_start || $p1_stop $p2_stop \n"; } if ($p1_channel eq $p2_channel) { $lastchannel = $thischannel; $thischannel = $p1_channel; # remember the 'current' channel if ($p1_start == $p2_start) { # either # (a) progs identical - replace old with new # (b) new prog is longer - replace old with new & delete all old progs until new stop time # (c) new prog is shorter - insert new prog # push @xmltv_merged_progs, $prog2->[$i]; $i++; next; } elsif ($p1_start < $p2_start) { # get here either when new schedule hasn't commenced yet, or when there is a gap in the new schedule # keep current old prog # unless this would cause an overlap if ($p1_stop > $p2_start) { # uh oh overlap - we could possibly set the stop time to equal new prog's start, but that # is just making up schedules! On balance I think a hole is better than an overlap: # skip this old prog next; } $p2_stop = 0; # ensure we *don't* skip the next old record } elsif ($p1_start > $p2_start) { # get here when additional prog in new schedule that's not in the old # insert new prog & keep current old prog push @xmltv_merged_progs, $prog2->[$i]; $i++; redo; } } else { # channel has changed on one (or both) of the files if ($p1_channel ne $thischannel) { # 'old' file channel has changed # copy any remaining 'new' records for this channel while ($p2_channel eq $thischannel) { push @xmltv_merged_progs, $prog2->[$i]; $i++; if ($i < $p2count) { $p2_channel = $prog2->[$i]->{-channel}; } else { $p2_channel = ''; # no more new records, end the loop } } $lastchannel = $thischannel; $thischannel = $p1_channel; # remember the new 'current' channel redo; } elsif ($p2_channel ne $thischannel) { # 'new' file channel has changed # copy any remaining 'old' records for this channel # then, if this is a totally new channel, insert the new channel's programmes if (defined $xmltv_new_channels{$p2_channel}) { $p2_stop = -1; redo; } else { $p2_stop = 0; } } } } # copy the old record push @xmltv_merged_progs, $p; } # no more old progs # addend any remaining new progs while ($i < $p2count) { push @xmltv_merged_progs, $prog2->[$i]; $i++; } if ($debugmore) { print Dumper("Merged programmes :", @xmltv_merged_progs); } } sub merge_channels { # Merge the channels in the two input files # # - always use the newer channel details # my $chan1 = $xmltv->{tv}->{channel}; my $chan2 = $xmltv_merge->{tv}->{channel}; if ($debugmore) { print Dumper("Incoming channels :", $chan1 ); } if ($debugmore) { print Dumper("Merging channels :", $chan2 ); } my ($c1_id); my ($c2_id); my %channels; for my $c1 (@{ $chan1 }) { $c1_id = $c1->{-id}; $channels{$c1_id} = $c1; } for my $c2 (@{ $chan2 }) { $c2_id = $c2->{-id}; $xmltv_new_channels{$c2_id} = 1 if !defined $channels{$c2_id}; # array of new channels not in current file $channels{$c2_id} = $c2; } foreach my $key ( keys %channels ) { push @xmltv_merged_channels, $channels{$key}; } if ($debugmore) { print Dumper("Merged channels :", @xmltv_merged_channels); } } sub to_timestamp { my ($ts) = @_; use DateTime::Format::Strptime; my $format = DateTime::Format::Strptime->new( pattern => '%Y%m%d%H%M%S %z', # "20130320060000 +0000" time_zone => 'UTC', # (better than 'local' - e.g. handles DST) on_error => 'croak', ); return ($format->parse_datetime($ts))->epoch(); } sub write_newxml { # Write the output xml files # # create an xml container my %xml = (); $xml{'tv'}{'-generator-info-name'} = $xmltv->{tv}->{'-generator-info-name'} if $xmltv->{tv}->{'-generator-info-name'}; $xml{'tv'}{'-generator-info-url'} = $xmltv->{tv}->{'-generator-info-url'} if $xmltv->{tv}->{'-generator-info-url'}; $xml{'tv'}{'-source-info-name'} = $xmltv->{tv}->{'-source-info-name'} if $xmltv->{tv}->{'-source-info-name'}; $xml{'tv'}{'-source-info-url'} = $xmltv->{tv}->{'-source-info-url'} if $xmltv->{tv}->{'-source-info-url'}; # add the <channel> elements $xml{'tv'}{'channel'} = \@xmltv_merged_channels; # add the <programme> elements $xml{'tv'}{'programme'} = \@xmltv_merged_progs; # my $ccount = scalar @{ $xml{'tv'}{'channel'} }; my $ccount_plural = $ccount == 1 ? "" : "s"; my $pcount = scalar @{ $xml{'tv'}{'programme'} }; my $pcount_plural = $pcount == 1 ? "" : "s"; if (!$quiet_mode) { print "Writing : $ccount channel$ccount_plural $pcount programme$pcount_plural \n"; } # write the output xml file if ($debugmore) { print Dumper(\%xml); } if ($doctype) { # TreePP doesn't write a DOCTYPE - add one if the user requests it my $xmlout = $tpp->write( \%xml, 'UTF-8' ); $xmlout =~ s/^(<\?xml.*>)/$1\n<!DOCTYPE tv SYSTEM "xmltv.dtd">\n/; open OUT, "> $outfile" or die "Failed to open $outfile for writing"; printf OUT $xmlout; close OUT; } else { $tpp->writefile( $outfile, \%xml, 'UTF-8' ); } } sub VERSION_MESSAGE { use XMLTV; our $VERSION = $XMLTV::VERSION; print "XMLTV module version $XMLTV::VERSION\n"; print "This program version $VERSION\n"; } sub HELP_MESSAGE { # print usage message my $filename = (split(/\//,$0))[-1]; print STDERR << "EOF"; Merge (combine) two XMLTV files Files should be sorted into datetime within channel order e.g. tv_sort --by-channel --output FILE FILE Assumes all data has start and stop times Usage: $filename [-dDq] [-t] -i FILE -m FILE -o FILE -i FILE : input XMLTV file -m FILE : merge XMLTV file -o FILE : output XMLTV file -t : add DOCTYPE to output file -d : print debugging messages -D : print even more debugging messages -q : quiet mode (no STDOUT messages) -h, --help : print help message and exit -v, --version : print version and exit Example: $filename -i master.xml -m newadditions.xml -o newmaster.xml EOF } ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_remove_some_overlapping�������������������������������������������������������0000775�0000000�0000000�00000014533�15000742332�0021542�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w =pod =head1 NAME tv_remove_some_overlapping - Remove some overlapping programmes from XMLTV data. =head1 SYNOPSIS tv_remove_some_overlapping [--help] [--output FILE] [FILE...] =head1 DESCRIPTION Read one or more XMLTV files and write a file to standard output containing the same data, except that some 'magazine' programmes which seem to contain two or more other programmes are removed. For example, if 'Schools TV' runs from 10:00 to 12:00, and there are two programmes 'History' from 10:00 to 11:00 and 'Geography' from 11:00 to 12:00 on the same channel, then 'Schools TV' could be removed. A programme is removed only if there are two or more other programmes which partition its timeslot, which implies that it and these other programmes must have stop times specified. To avoid throwing away any real programmes, no programme will be discarded if it has content data other than title and URL. Filtering this tool won't remove all overlapping programmes but it will deal with the 'big magazine programme containing smaller programmes' data commonly seen from listings sources. B<--output FILE> write to FILE rather than standard output =head1 SEE ALSO L<xmltv(5)>. =head1 AUTHOR Ed Avis, ed@membled.com =cut use strict; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use XMLTV::Date; use Getopt::Long; use Date::Manip; BEGIN { if (int(Date::Manip::DateManipVersion) >= 6) { Date::Manip::Date_Init("SetDate=now,UTC"); } else { Date::Manip::Date_Init("TZ=UTC"); } } # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } } use XMLTV::Usage <<END $0: remove some programmes which seem to be mere containers for others usage: $0 [--help] [--output FILE] [FILE...] END ; # Memoize some subroutines if possible eval { require Memoize }; unless ($@) { foreach (qw/Date_Cmp pd interesting/) { Memoize::memoize($_) or die "cannot memoize $_: $!"; } } sub pd( $ ); sub exists_partition( $$$$$ ); sub interesting( $ ); sub should_write( $ ); # Keys of a programme hash which don't indicate any data we're # especially concerned to preserve. Poke around inside XMLTV.pm to # find the list of attributes. # my %boring_programme_key = (title => 1); $boring_programme_key{$_} = 1 foreach map { $_->[0] } @XMLTV::Programme_Attributes; $boring_programme_key{url} = 1; # common in some sources, and boring my ($opt_help, $opt_output); GetOptions('help' => \$opt_help, 'output=s' => \$opt_output) or usage(0); usage(1) if $opt_help; @ARGV = ('-') if not @ARGV; my %w_args = (); if (defined $opt_output) { my $fh = new IO::File ">$opt_output"; die "cannot write to $opt_output\n" if not $fh; %w_args = (OUTPUT => $fh); } # Unfortunately we need to load the whole file before processing. I # don't want to require the input to be sorted, since tv_sort adds # guessed stop times which could cause this program to remove too many # programmes. This is another reason to eventually change tv_sort not # to add stop times (move into tv_guess_stop_times or whatever). # my ($encoding, $credits, $ch, $progs) = @{XMLTV::parsefiles(@ARGV)}; my $w = new XMLTV::Writer(%w_args, encoding => $encoding); $w->start($credits); $w->write_channels($ch); # Since zero-length and unknown-length programmes are always written # unchanged, we could write them immediately and discard them. But # it's a bit nicer to write the output in the same order as the input. # However, we don't bother to index these programmes, they cannot / # should not be used in looking for partitionings. # my %by_channel_and_start; foreach (@$progs) { push @{$by_channel_and_start{$_->{channel}}{pd $_->{start}}}, $_ if interesting $_; } $w->write_programme($_) foreach grep { should_write($_) } @$progs; $w->end(); exit(); # Given that %by_channel_and_start and %boring_programme_key have been # set up, should a programme (with start and stop time) be written? # sub should_write( $ ) { my $p = shift; # Always write zero length and unknown-length programmes. return 1 if not interesting $p; # If this programme cannot be partitioned by at least two others, # definitely write it. # return 1 unless exists_partition(pd $p->{start}, pd $p->{stop}, $p->{channel}, 2, { $p => 1 }); foreach (keys %$p) { if (not $boring_programme_key{$_}) { warn <<END not filtering programme at $p->{start} on $p->{channel} because it has $_ END ; return 1; } } return 0; } # We process only programmes with stop time and nonzero length. sub interesting( $ ) { my $p = shift; my $stop = $p->{stop}; return 0 if not defined $stop; my $cmp = Date_Cmp(pd $p->{start}, pd $stop); if ($cmp < 0) { # start < stop, okay. return 1; } elsif ($cmp == 0) { # Zero length, won't consider. return 0; } elsif ($cmp > 0) { warn "programme on $p->{channel} " . "with start time ($p->{start}) " . "before stop time ($stop)\n"; return 0; } else { die } } # Does there exist a sequence of programmes hopping from $start to # $stop, where none of the programmes is in $used and the sequence is # at least $min_length long? # # $start and $stop are Date::Manip objects, $used a hash whose keys # are used programmes and whose values are true. # sub exists_partition( $$$$$ ) { my ($start, $stop, $channel, $min_length, $used) = @_; # local $Log::TraceMessages::On = 1; t "seeking at least $min_length $start to $stop on $channel"; t '... not including: ' . d $used; my $cmp = Date_Cmp($start, $stop); if ($cmp < 0) { t 'start before stop, okay'; my @poss = grep { not $used->{$_} } @{$by_channel_and_start{$channel}{$start}}; t 'possible first steps of path: ' . d \@poss; --$min_length if $min_length; foreach my $p (@poss) { return 1 if exists_partition(pd $p->{stop}, $stop, $channel, $min_length, { %$used, $p => 1 }); } t 'no paths found'; return 0; } elsif ($cmp == 0) { t 'zero length, so path of length zero'; return not $min_length; } elsif ($cmp > 0) { t 'stop < start, so no path'; return 0; } else { die } } # Lift parse_date() to handle undef. sub pd( $ ) { for ($_[0]) { return undef if not defined; return parse_date($_); } } exit 1; ���������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_sort��������������������������������������������������������������������������0000775�0000000�0000000�00000035211�15000742332�0015577�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w =pod =head1 NAME tv_sort - Sort XMLTV listings files by date, and add stop times. =head1 SYNOPSIS tv_sort [--help] [--by-channel] [--output FILE] [FILE...] =head1 DESCRIPTION Read XMLTV data and write out the same data sorted in date order. Where stop times of programmes are missing, guess them from the start time of the next programme on the same channel. For the last programme of a channel, no stop time can be added. Tv_sort also performs some sanity checks such as making sure no two programmes on the same channel overlap. B<--output FILE> write to FILE rather than standard output B<--by-channel> sort first by channel id, then by date within each channel. B<--duplicate-error> If the input contains the same programme more than once, consider this as an error. Default is to silently ignore duplicate entries. The time sorting is by start time, then by stop time. Without B<--by-channel>, if start times and stop times are equal then two programmes are sorted by internal channel id. With B<--by-channel>, channel id is compared first and then times. You can think of tv_sort as converting XMLTV data into a canonical form, useful for diffing two files. =head1 EXAMPLES At a typical Unix shell or Windows command prompt: =over =item tv_sort <in.xml >out.xml =item tv_sort in.xml --output out.xml =back These are different ways of saying the same thing. =head1 AUTHOR Ed Avis, ed@membled.com =cut use strict; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use XMLTV::Date; use Date::Manip; use Getopt::Long; # We use Storable to do 'deep equality' of data structures; this # requires setting canonical mode. # use Storable qw(freeze); $Storable::canonical = 1; BEGIN { if (int(Date::Manip::DateManipVersion) >= 6) { Date::Manip::Date_Init("SetDate=now,UTC"); } else { Date::Manip::Date_Init("TZ=UTC"); } } # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } } use XMLTV::Usage <<END $0: sort listings by time, sanity-check and add stop time to programmes usage: $0 [--help] [--by-channel] [--duplicate-error] [--output FILE] [FILE...] END ; # Memoize some subroutines if possible eval { require Memoize }; unless ($@) { foreach (qw/Date_Cmp pd programme_eq/) { Memoize::memoize($_) or die "cannot memoize $_: $!"; } # clumpidx_cmp() isn't memoized, since it uses undef arguments and # they cause warnings. # } # Prototype declarations sub programme_cmp(); sub chan_cmp( $$ ); sub clumpidx_cmp( $$ ); sub overlap( $$ ); sub add_stop( $ ); sub programme_eq( $$ ); sub pd( $ ); my ($opt_help, $opt_output, $opt_by_channel); my $opt_duplicate_error = 0; GetOptions('help' => \$opt_help, 'output=s' => \$opt_output, 'by-channel' => \$opt_by_channel, 'duplicate-error' => \$opt_duplicate_error ) or usage(0); usage(1) if $opt_help; @ARGV = ('-') if not @ARGV; my ($encoding, $credits, $channels, $progs) = @{XMLTV::parsefiles(@ARGV)}; my @progs = @$progs; # We really want the sort to be stable, so that tv_sort can be # idempotent. Since the manual page claims that tv_sort produces a # 'canonical form', it would be embarrassing otherwise. Okay, it's # not really important what to do with clearly stupid listings having # two different programmes on at exactly the same time on the same # channel, but since the XMLTV format still allows this we should do # something sensible. # # Accordingly, we use the original ordering of programmes as a # comparison of last resort. # # TODO: use sort 'stable'; pragma with perl 5.8. # # This function takes a reference to a list of elements, and a # comparison function f. It returns a comparison function f' which # agrees with f, except that where f would return 0 for two elements, # f' orders them according to their original position in the list. In # other words you can turn any sort into a stable sort. (Expects the # sort function to use $a and $b, not function parameters.) # sub make_stable_sort_fn( $$ ) { our @orig; local *orig = shift; my $f = shift; my %orig_order; for (my $i = 0; $i < @orig; $i++) { $orig_order{$orig[$i]} = $i; } return sub() { my $r = &$f; return $r if $r; return $orig_order{$a} <=> $orig_order{$b}; }; } # Check that a list is sorted according to a given comparison # function. Used for debugging. # use Carp; sub check_sorted( $$ ) { my $f = shift; die if ref $f ne 'CODE'; die if ref $_[0] ne 'ARRAY'; our @l; local *l = shift; our ($a, $b); foreach my $i (0 .. @l - 2) { ($a, $b) = ($l[$i], $l[$i + 1]); if ($f->() > 0) { # local $Log::TraceMessages::On = 1; t 'not sorted elements: ' . d($a); t '...and: ' . d($b); confess 'failed to sort correctly'; } } } # Split up programmes according to channel my %chan; foreach (@progs) { push @{$chan{$_->{channel}}}, $_; } # Sort each channel individually, and guess stop times. foreach (keys %chan) { our @ps; local *ps = $chan{$_}; my $f = make_stable_sort_fn(\@ps, \&programme_cmp); @ps = sort { $f->() } @ps; check_sorted(\&programme_cmp, \@ps); add_stop(\@ps); check_sorted(\&programme_cmp, \@ps); } # Remove duplicates and sanity-check that there is no overlap on a # channel. # foreach (sort keys %chan) { my $progs = $chan{$_}; my @new_progs; die if not @$progs; # Sanity check that no programme starts after it begins. As with # the 'overlapping programmes' check below, this should really be # moved into a separate tv_semantic_check or whatever. # foreach (@$progs) { if (not defined $_->{stop}) { delete $_->{stop}; # sometimes gets set undef, don't know why next; } if (Date_Cmp(pd($_->{start}), pd($_->{stop})) > 0) { warn <<END programme on channel $_->{channel} stops before it starts: $_->{start}, $_->{stop} END ; } } my $first = shift @$progs; while (@$progs) { my $second = shift @$progs; die if not defined $first or not defined $second; t 'testing consecutive programmes to see if the same'; t 'first: ' . d $first; t 'second: ' . d $second; if (programme_eq($first, $second)) { if ($opt_duplicate_error) { print STDERR "Duplicate program found:\n" . " $first->{title}->[0]->[0]\t" . "at $first->{start}-|$first->{stop}\n"; } next; } else { if (overlap($first, $second)) { local $^W = 0; warn <<END overlapping programmes on channel $_: $first->{title}->[0]->[0]\tat $first->{start}-|$first->{stop} and $second->{title}->[0]->[0]\tat $second->{start}-|$second->{stop} END ; } } push @new_progs, $first; $first = $second; } # Got to the last element. push @new_progs, $first; $chan{$_} = \@new_progs; check_sorted(\&programme_cmp, $chan{$_}); } # Combine the channels back into a big list. @progs = (); foreach (sort keys %chan) { push @progs, @{$chan{$_}}; } unless ($opt_by_channel) { # Sort again. (Could use merge sort.) my $f = make_stable_sort_fn(\@progs, \&programme_cmp); @progs = sort { $f->() } @progs; check_sorted(\&programme_cmp, \@progs); } # Write out the new document t 'writing out data'; t 'new programmes list: ' . d \@progs; my %w_args = (); if (defined $opt_output) { my $fh = new IO::File ">$opt_output"; die "cannot write to $opt_output\n" if not $fh; %w_args = (OUTPUT => $fh); } XMLTV::write_data([ $encoding, $credits, $channels, \@progs ], %w_args); exit(); # Compare two programme hashes. sub programme_cmp() { my $xa = $a; my $xb = $b; my $r; # Assume that {start} is always there, as it should be. my ($a_start, $b_start) = (pd($xa->{start}), pd($xb->{start})); $r = Date_Cmp($a_start, $b_start); t "compare start times: " . d $r; return $r if $r; # {stop} is optional and a programme without stop comes before one # with (assuming they have the same start). I did try comparing # stop only if both programmes had it, but this made the sort # function inconsistent, eg # # (0, 1) <= (0, undef) <= (0, 0). # my ($a_stop_u, $b_stop_u) = ($xa->{stop}, $xb->{stop}); if (not defined $a_stop_u and not defined $b_stop_u) { # Go on to to compare other things. } elsif (not defined $a_stop_u and defined $b_stop_u) { return -1; } elsif (defined $a_stop_u and not defined $b_stop_u) { return 1; } elsif (defined $a_stop_u and defined $b_stop_u) { my ($a_stop, $b_stop) = (pd($a_stop_u), pd($b_stop_u)); $r = Date_Cmp($a_stop, $b_stop); t "compare stop times: " . d $r; return $r if $r; } else { die } # Channel. Ideally would sort on pretty name, but no big deal. $r = $xa->{channel} cmp $xb->{channel}; t "compare channels: " . d $r; return $r if $r; $r = clumpidx_cmp($xa->{clumpidx}, $xb->{clumpidx}); t "compare clumpidxes: " . d $r; return $r if $r; t 'do not sort'; return 0; } # Compare indexes-within-clump sub clumpidx_cmp( $$ ) { my ($A, $B) = @_; if (not defined $A and not defined $B) { return 0; # equal } elsif ((not defined $A and defined $B) or (defined $A and not defined $B)) { warn "mismatching clumpidxs: one programme has, one doesn't"; return undef; } elsif (defined $A and defined $B) { $A =~ m!^(\d+)/(\d+)$! or die "bad clumpidx $A"; my ($ai, $num_in_clump) = ($1, $2); $B =~ m!^(\d+)/(\d+)$! or die "bad clumpidx $B"; my $bi = $1; if ($2 != $num_in_clump) { warn "clumpidx's $A and $B don't match"; return undef; } return $ai <=> $bi; } else { die } } # Test whether two programmes overlap in time. This takes account of # start time and stop time, and clumpidx (so two programmes with the same # start and stop times, but different places within the clump, are not # considered to overlap). # sub overlap( $$ ) { my ($xa, $xb) = @_; my ($xa_start, $xb_start) = (pd($xa->{start}), pd($xb->{start})); my $xa_stop = pd($xa->{stop}) if exists $xa->{stop}; my $xb_stop = pd($xb->{stop}) if exists $xb->{stop}; die if exists $xa->{stop} and not defined $xa->{stop}; die if exists $xb->{stop} and not defined $xb->{stop}; # If we don't know the stop times we can't do an overlap test; if # we know only one stop time we can do only one half of the # test. We assume no overlap if we can't prove otherwise. # # However, obviously two _identical_ start times on the same # channel must overlap, except for zero length. # { local $^W = 0; t "xa: $xa_start -| $xa_stop"; t "xb: $xb_start -| $xb_stop" } if (not defined $xa_stop and not defined $xb_stop) { # Cannot prove overlap, even if they start at the same time. return 0; } elsif (not defined $xa_stop and defined $xb_stop) { return (Date_Cmp($xa_start, $xb_start) > 0 and Date_Cmp($xa_start, $xb_stop) < 0); # (Cannot prove overlap if A and B start at same time, # or A starts before B.) # } elsif (defined $xa_stop and not defined $xb_stop) { return (Date_Cmp($xb_start, $xa_start) > 0 and Date_Cmp($xb_start, $xa_stop) < 0); # (Cannot prove overlap if A and B start at same time, # or A starts before B.) # } elsif (defined $xa_stop and defined $xb_stop) { if (Date_Cmp($xa_stop, $xb_start) <= 0) { # A finishes before B starts. return 0; } elsif (Date_Cmp($xa_start, $xb_start) < 0 and Date_Cmp($xa_stop, $xb_start) > 0) { # A starts before B starts, finishes after. return 1; } elsif (Date_Cmp($xa_start, $xb_start) == 0 and Date_Cmp($xa_start, $xa_stop) < 0 and Date_Cmp($xb_start, $xb_stop) < 0) { # They start at the same time and neither is zero length. my $cmp = clumpidx_cmp($xa->{clumpidx}, $xb->{clumpidx}); if (not defined $cmp) { # No clumpidxes, so must overlap. (Also happens if # the two indexes were not comparable - but that will # have been warned about already.) # t 'no clumpidxes, overlap'; return 1; } t 'compared clumpidxes: same? ' . not $cmp; return not $cmp; } elsif (Date_Cmp($xa_start, $xb_start) > 0 and Date_Cmp($xa_start, $xb_stop) < 0) { # B starts before A starts, finishes after. return 1; } elsif (Date_Cmp($xa_start, $xb_stop) >= 0) { # B finishes before A starts. return 0; } else { die } } } # Add 'stop time' to a list of programmes (hashrefs). # The stop time of a programme is the start time of the next. # # Parameters: reference to list of programmes, sorted by date, to be # shown consecutively (except for 'clumps'). # # Modifies the list passed in. # # Precondition: the list of programmes is sorted. Postcondition: it's # still sorted. # sub add_stop( $ ) { die 'usage: add_stop(ref to list of programme hashrefs)' if @_ != 1; our @l; local *l = shift; # We make several passes over the programmes, stopping when no # further stop times can be added. # PASS: t 'doing a pass through list of programmes: ' . d \@l; my $changed = 0; my $p = undef; for (my $i = 0; $i < @l - 1; $i++) { my ($last_start, $last_stop); if ($p) { $last_start = $p->{start}; $last_stop = $p->{stop}; } $p = $l[$i]; next if defined $p->{stop}; t 'programme without stop time: ' . d $p; my $f = $l[$i + 1]; if (not defined $f) { t 'this is the last programme, cannot pick following'; next; } t 'look at following: ' . d $f; my $cmp = Date_Cmp(pd($f->{start}), pd($p->{start})); if ($cmp < 0) { die 'strange, programmes not sorted in add_sort()'; } elsif ($cmp == 0) { # The following programme has the same start time as # this one. Don't use it as a stop time, that would # make this one be zero length. # # If the following programme has a stop time we can use it # and still have this <= following. # if (defined $f->{stop}) { t 'following has stop time, use it'; $p->{stop} = $f->{stop}; $changed = 1; } } elsif ($cmp > 0) { t 'found programme with later start time, use that as stop time'; # Since the list was sorted we know that this # programme is the last with its start time. So we # can set the stop time and it will still be the last. # t 'following has later start than our start, use it as stop'; $p->{stop} = $f->{start}; $changed = 1; } t 'doing next programme'; } goto PASS if $changed; } sub programme_eq( $$ ) { # local $Log::TraceMessages::On = 1; t 'comparing programmes ' . d($_[0]) . ' and ' . d($_[1]); return freeze($_[0]) eq freeze($_[1]); } # Lift parse_date() to handle undef. sub pd( $ ) { for ($_[0]) { return undef if not defined; return parse_date($_); } } ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_split�������������������������������������������������������������������������0000775�0000000�0000000�00000011657�15000742332�0015753�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w =pod =head1 NAME tv_split - Split XMLTV listings into separate files by date and channel. =head1 SYNOPSIS tv_split --output TEMPLATE [FILE...] =head1 DESCRIPTION Read XMLTV listings and split them into some number of output files. The output file chosen for each programme is given by substitutions on the filename template supplied. You can split listings by time and by channel. The TEMPLATE is a filename but substitutions are applied: first C<%channel> is replaced with the id of a programmeE<39>s channel, and then L<Date::Manip> substitutions (which broadly follow L<date(1)>) are applied based on the start time of each programme. In this way each programme is written to a particular output file. When an output file is created it will also contain all the channel elements from the input. One or more input files can be given; if more than one then they are concatenated in the same way as L<tv_cat>. If no input files are given then standard input is read. =head1 EXAMPLE Use C<tv_split --output %channel-%Y%m%d.xml> to separate standard input into separate files for each day and channel. The files will be created with names like B<bbc1.bbc.co.uk-20020330.xml>. =head1 SEE ALSO L<Date::Manip(3)>. =head1 AUTHOR Ed Avis, ed@membled.com. =cut use strict; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use Data::Dumper; use Getopt::Long; use Date::Manip; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } } use XMLTV::Usage <<END $0: concatenate listings, merging channels usage: $0 [--help] [--output FILE] [FILE...] END ; sub new_writer( $$ ); my ($opt_help, $opt_output); GetOptions('help' => \$opt_help, 'output=s' => \$opt_output) or usage(0); usage(1) if $opt_help; usage(1) if not defined $opt_output; @ARGV = ('-') if not @ARGV; # Whether we are splitting by channel - if so, write just one # <channel> to each output file. # my $by_channel = ($opt_output =~ /%channel/); my ($encoding, $credits, %channels); sub encoding_cb( $ ) { $encoding = shift } sub credits_cb( $ ) { $credits = shift } my %seen_unstripped; sub channel_cb( $ ) { my $c = shift; for ($c->{id}) { my $old = $_; if (tr/%//d) { warn "stripping % characters from channel id '$old' (which is not RFC2838 anyway)"; if (defined $seen_unstripped{$old} and $seen_unstripped{$old} ne $_) { die "two channel ids ('$old' and '$seen_unstripped{$old}') not unique after stripping %"; } $seen_unstripped{$old} = $_; } $channels{$_} = $c; } } my %writers; # map filename to XMLTV::Writer objects my %todo; # map filename to programmes, in case too many open files my $too_many = 0; sub programme_cb( $ ) { my $p = shift; my $ch = $p->{channel}; my $filename = $opt_output; for ($filename) { s/%channel/$ch/g; $_ = UnixDate($p->{start}, $_) if tr/%//; } if (not defined $writers{$filename} and not $too_many) { my $w = new_writer($filename, $by_channel ? $ch : undef); if ($w) { $writers{$filename} = $w; } else { if ($! =~ /[Tt]oo many open files/) { warn "too many open files, storing programmes in memory\n"; $too_many = 1; } else { die "cannot write to $filename: $!, aborting"; } } } if (defined $writers{$filename}) { $writers{$filename}->write_programme($p); } else { # Can't write it now, do it later. push @{$todo{$filename}}, $p; } } XMLTV::parsefiles_callback(\&encoding_cb, \&credits_cb, \&channel_cb, \&programme_cb, @ARGV); # Now finish up (write </tv>, etc). END { # First those which have XML::Writer objects to hand. foreach my $f (keys %writers) { my $w = $writers{$f}; my $todo = delete $todo{$f}; if ($todo) { $w->write_programme($_) foreach @$todo; } $w->end(); } # We've freed up some filehandles so we can write the remaining # todo programmes, if any. # foreach my $f (keys %todo) { my @ps = @{$todo{$f}}; die if not @ps; my $w = new_writer($f, $by_channel ? $ps[0]->{channel} : undef); die "cannot write to $f: $!, aborting" if not $w; $w->write_programme($_) foreach @ps; $w->end(); } } # Create a new XMLTV::Writer and get it ready to write programmes # (using the global variables set above). Returns undef if failure. # # Parameters: filename, channel id that will go into this file (or # undef for all channels) # sub new_writer( $$ ) { my ($f, $ch) = @_; my $fh = new IO::File ">$f" or return undef; my $w = new XMLTV::Writer(OUTPUT => $fh, encoding => $encoding) or return undef; $w->start($credits); if (defined $ch) { # Write this one <channel> if we have it. (If it wasn't in # the input, do without.) # for ($channels{$ch}) { $w->write_channel($_) if defined; } } else { $w->write_channels(\%channels) } return $w; } ���������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_tmdb��������������������������������������������������������������������������0000775�0000000�0000000�00000035324�15000742332�0015543�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl =pod =head1 NAME tv_tmdb - Augment XMLTV listings files with themoviedb.org data. =head1 SYNOPSIS tv_tmdb [--help] [--quiet] [--configure] [--config-file FILE] [--apikey KEY] [--with-keywords] [--with-plot] [--movies-only] [--actors NUMBER] [--reviews NUMBER] [--stats] [--debug NUMBER] [--output FILE] [FILE...] =head1 PURPOSE tv_tmdb reads your xml file of tv programmes and attempts to find a matching entry for the programme title in The Movie Database community-built movie and TV database. Access to the TMDB system uses their API interface to access data in realtime. Therefore you must be online to be able to augment your listings using tv_tmdb. Using the TMDB system requires an API key which you can get from The Movie Database website (https://www.themoviedb.org/). The key is free for non-commercial use. You will need to get this API key B<before> you can start using tv_tmdb. =head1 PARAMETERS B<--apikey KEY> your TMDB API key. B<--output FILE> write to FILE rather than standard output. B<--with-keywords> include tmdb keywords in the output file. B<--with-plot> include tmdb plot summary in the output file. B<--actors NUMBER> number of actors from tmdb to add (default=3). B<--reviews NUMBER> number of reviews from tmdb to add (default=0). B<--movies-only> only augment programs that look like movie listings (have a 4 digit E<39>dateE<39> field). B<--quiet> disable all status messages (that normally appear on stderr). B<--stats> output grab stats (stats output disabled in --quiet mode). B<--configure> store frequent parameters in a config file (apikey, actors). B<--config-file FILE> specify your own file location instead of XMLTV default. B<--debug NUMBER> output info from movie matching (optional value to increase debug level: 2 is probably the max you will find useful). =head1 DESCRIPTION All programs are checked against themoviedb.org (TMDB) data (unless --movies-only is used). For the purposes of tv_tmdb, an "exact" match is defined as a case insensitive match against themoviedb.org data (which may or may not include the transformation of E<39>&E<39> to E<39>andE<39> and vice-versa). If the program includes a 4 digit E<39>dateE<39> field the following matches are attempted, with the first successful match being used: 1. an "exact" title/year match against movie titles is done 2. an "exact" title match against tv series 3. an "exact" title match against movie titles with production dates within 2 years of the E<39>dateE<39> value. Unless --movies-only is used, if the program does not include a 4 digit E<39>dateE<39> field the following matches are attempted, the first succeeding match is used: 1. an "exact" title match against tv series When a match is found in the themoviedb.org data the following is applied: 1. the E<39>titleE<39> field is set to match exactly the title from the themoviedb.org data. This includes modification of the case to match and any transformations mentioned above. 2. if the match is a movie, the E<39>dateE<39> field is set to themoviedb.org 4 digit year of production. 3. the type of match found (Movie, or TV Series) is placed in the E<39>categoriesE<39> field. 4. a url to the program on www.imdb.com is added. 5. the director is added if the match was a movie or if only one director is listed in the themoviedb.org data (because some tv series have. 30 directors). 6. the top 3 billing actors are added (use --actors [num] to adjust). 7. genres are added to E<39>categoriesE<39> field. 8. TMDB user-ratings added to E<39>star-ratingsE<39> field. 9. TMDB keywords are added to E<39>keywordE<39> fields (if --with-keywords used). 10. TMDB plot summary is added (if --with-plot used). 11. The top TMDB reviews are added (use --reviews [num] to adjust). =head1 HOWTO 1. In order to use tv_tmdb, you need an API key from themoviedb.org. These are free for Personal use. You need to create a log-in with themoviedb.org and then click on the API link on the Settings page. (See https://www.themoviedb.org/documentation/api ) 2. run E<39>tv_tmdb --apikey <key> --output myxmlout.xml myxmlin.xmlE<39> or E<39>cat tv.xml | tv_tmdb --apikey <key> tv1.xmlE<39> or etc. 3. To use a config file to avoid entering your apikey on the commandline, run E<39>tv_tmdb --configureE<39> and follow the prompts. Feel free to report any problems with these steps at https://github.com/XMLTV/xmltv/issues. =head1 BACKGROUND Like the original (pre Amazon) IMDb, "The Movie Database" (TMDB) (https://www.themoviedb.org/) is a community effort, and relies on people adding the movies. Note TMDB is I<not> IMDB...but it's getting there! As at December 2021, TMDB has over 700,000 movies and 123,000 TV shows while IMDB has approx 770,000 movies and 217,000 TV series. However there are bound to be some films/TV programmes on IMDb which are not on TMDB. So if you can't find a film that you can find manually on IMDb then you might consider signing up to TMDB and adding it yourself. =head1 BUGS We only add movie information to programmes that have a 'date' element defined (since we need a year to work with when verifing we got the correct hit in the TMDB data). A date is required for a movie to be augmented. (If no date is found in the incoming data then it is assumed the program is a tv series/episode.) For movies we look for matches on title plus release-year within two years of the program date. We could check other data such as director or top 3 actors to help identify the correct match. Headshots of the actors are possible with the TMDB data, but the XMLTV.dtd does not currently support them. =head1 DISCLAIMER This product uses the TMDB API but is not endorsed or certified by TMDB. It is B<YOUR> responsibility to comply with TMDB's Terms of Use of their API. In particular your attention is drawn to TMDB's restrictions on Commercial Use. Your use is deemed to be Commercial if I<any> of: 1. Users are charged a fee for your product or a 3rd party's product or service or a 3rd party's service that includes some sort of integration using the TMDB APIs. 2. You sell services using TMDb's APIs to bring users' TMDB content into your service. 3. Your site is a "destination" site that uses TMDB content to drive traffic and generate revenue. 4. Your site generates revenue by charging users for access to content related to TMDB content such as movies, television shows and music. If any of these events are true then you cannot use TMDB data in any part of your product or service without a commercial license. =head1 SEE ALSO L<xmltv(5)> =head1 AUTHOR Geoff Westcott, Jerry Veldhuis =cut use strict; use warnings; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use Data::Dumper; use Getopt::Long; use XMLTV::Data::Recursive::Encode; use XMLTV::Usage <<END $0: augment listings with data from themoviedb.org $0 --apikey <key> [--help] [--quiet] [--with-keywords] [--with-plot] [--movies-only] [--actors NUMBER] [--reviews NUMBER] [--stats] [--debug] [--output FILE] [FILE...] END ; use XMLTV::TMDB; my ($opt_help, $opt_output, $opt_quiet, $opt_stats, $opt_debug, $opt_movies_only, $opt_with_keywords, $opt_with_plot, $opt_num_actors, $opt_num_reviews, $opt_apikey, $opt_configure, $opt_configfile, ); GetOptions('help' => \$opt_help, 'output=s' => \$opt_output, 'quiet' => \$opt_quiet, 'stats' => \$opt_stats, 'debug:i' => \$opt_debug, 'movies-only' => \$opt_movies_only, 'with-keywords' => \$opt_with_keywords, 'with-plot' => \$opt_with_plot, 'actors=i' => \$opt_num_actors, 'reviews=i' => \$opt_num_reviews, 'apikey:s' => \$opt_apikey, 'configure' => \$opt_configure, 'config-file=s' => \$opt_configfile, ) or usage(0); usage(1) if $opt_help; $opt_debug=1 if ( defined($opt_debug) && $opt_debug==0); $opt_debug=0 if ( !defined($opt_debug) ); $opt_quiet=(defined($opt_quiet)); if ( !defined($opt_stats) ) { $opt_stats=!$opt_quiet; } else { $opt_stats=(defined($opt_stats)); } $opt_debug=0 if $opt_quiet; # undocumented option: for use by the test harness my $test_opts = {}; my @_opts = qw/ updateDates updateTitles updateCategories updateCategoriesWithGenres updateKeywords updateURLs updateDirectors updateActors updatePresentors updateCommentators updateGuests updateStarRatings updateRatings updatePlot updateRuntime updateContentId updateImage numActors updateActorRole updateCastImage updateCastUrl getYearFromTitles removeYearFromTitles updateReviews numReviews /; my %t_opts = map {$_ => 1 } @_opts; # the --configure and --config-file options allow the storing of # apikey, movies-only, actors, with-plot, with-keywords # if (defined($opt_configure)) { # configure mode # store apikey, num_actors # get configuration file name require XMLTV::Config_file; my $file = XMLTV::Config_file::filename( $opt_configfile, 'tv_tmdb', $opt_quiet ); XMLTV::Config_file::check_no_overwrite( $file ); # open configuration file. Assume UTF-8 encoding open(my $fh, ">:utf8", $file) or die "$0: can't open configuration file '$file': $!"; print $fh "# config file for tv_tmdb # \n"; # get apikey my $apikey = XMLTV::Ask::ask( 'Enter your TMDB api key' ); chomp($apikey); # # write configuration file print $fh "apikey=$apikey\n"; # get movies-only my $moviesonly = XMLTV::Ask::ask_boolean( 'Movies only?', 0 ); print $fh "movies-only=$moviesonly\n"; # get num_actors my $actors = XMLTV::Ask::ask( 'Enter number of actors from TMDB to add (default=3)' ); chomp($actors); $actors = 3 if ( $actors eq '' || ($actors !~ /^\d+$/) ); print $fh "actors=$actors\n"; # get with-plot my $withplot = XMLTV::Ask::ask_boolean( 'Add plot from TMDB?', 0 ); print $fh "with-plot=$withplot\n"; # get with-keywords my $withkeywords = XMLTV::Ask::ask_boolean( 'Add keywords from TMDB?', 0 ); print $fh "with-keywords=$withkeywords\n"; # check for write errors close($fh) or die "$0: can't write to configuration file '$file': $!"; print "Configuration completed ok \n"; exit(0); } # load config file if we have one if ( 1 ) { # read configuration # read apikey, movies-only, actors, with-plot, with-keywords # get configuration file name require XMLTV::Config_file; my $file = XMLTV::Config_file::filename( $opt_configfile, 'tv_tmdb', $opt_quiet ); # does file exist? if (-f -r $file) { # read configuration file open(my $fh, "<:utf8", $file) or die "$0: can't open configuration file '$file': $!"; # read config file while (<$fh>) { # comment removal, white space trimming and compressing s/\#.*//; s/^\s+//; s/\s+$//; s/\s+/ /g; s/\s+=\s+/=/; next unless length; # skip empty lines # process a line my($k, $v) = /^(.*)=(.*)$/; # use config values unless given as opts # commndline overrides config file if ( $k eq 'apikey' ) { $opt_apikey = $v unless defined $opt_apikey; } if ( $k eq 'actors' ) { $opt_num_actors = $v unless defined $opt_num_actors; } if ( $k eq 'movies-only' ) { $opt_movies_only = $v unless defined $opt_movies_only; } if ( $k eq 'with-plot' ) { $opt_with_plot = $v unless defined $opt_with_plot; } if ( $k eq 'with-keywords' ) { $opt_with_keywords = $v unless defined $opt_with_keywords; } # is it one of the tester options? if ( exists($t_opts{$k}) ) { $test_opts->{$k} = $v; } } close($fh); } } # set some defaults $opt_with_keywords=0 if ( !defined($opt_with_keywords) ); $opt_with_plot=0 if ( !defined($opt_with_plot) ); $opt_num_actors=3 if ( !defined($opt_num_actors) ); $opt_num_reviews=0 if ( !defined($opt_num_reviews) ); $opt_movies_only=0 if ( !defined($opt_movies_only) ); # check we have an api key if ( !defined($opt_apikey) || $opt_apikey eq '' ) { print STDERR <<END; In order to use tv_tmdb, you need an API key from themoviedb.org ( https://www.themoviedb.org/ ) These are free for Personal use. You need to create a log-in with themoviedb.org and then click on the API link on the Settings page ( https://www.themoviedb.org/settings/api ) END exit(1) } # create package options from user input my $tmdb_opts = {'apikey' => $opt_apikey, 'verbose' => $opt_debug, 'updateKeywords' => $opt_with_keywords, 'updatePlot' => $opt_with_plot, 'numActors' => $opt_num_actors, 'numReviews' => $opt_num_reviews, 'moviesonly' => $opt_movies_only, }; # merge in the test params my $tmdb_params = { %$tmdb_opts, %$test_opts }; # invoke a TMDB object my $tmdb=new XMLTV::TMDB( %$tmdb_params ); # check the API works (e.g. apikey is valid) if ( my $errline=$tmdb->sanityCheckDatabase() ) { print STDERR "$errline"; exit(1); } # instantiate an API object if ( !$tmdb->openMovieIndex() ) { print STDERR "tv_tmdb: open api client failed\n"; exit(1); } # open the output file my %w_args = (); if (defined $opt_output) { my $fh = new IO::File ">$opt_output"; die "cannot write to $opt_output\n" if not $fh; %w_args = (OUTPUT => $fh); } my $numberOfSeenChannels=0; #------------------------------------------------------------------------ # callback function definitions, and file processing my $w; my $encoding; # store encoding of input file sub encoding_cb( $ ) { die if defined $w; $encoding = shift; # callback returns the file's encoding $w = new XMLTV::Writer(%w_args, encoding => $encoding); } sub credits_cb( $ ) { $w->start(shift); } my %seen_ch; sub channel_cb( $ ) { my $c = shift; my $id = $c->{id}; $Data::Dumper::Sortkeys = 1; # ensure consistent order of dumped hash if (not defined $seen_ch{$id}) { $w->write_channel($c); $seen_ch{$id} = $c; $numberOfSeenChannels++; } elsif (Dumper($seen_ch{$id}) eq Dumper($c)) { # They're identical, okay. } else { warn "channel $id may differ between two files, " . "picking one arbitrarily\n"; } } sub programme_cb( $ ) { my $prog=shift; # The data from TMDB is encoded as utf-8. # The xml file may be different (e.g. iso-8859-1). my $orig_prog = $prog; # decode the incoming programme $prog = XMLTV::Data::Recursive::Encode->decode($encoding, $prog); # augmentProgram will now add tmdb data as utf-8 my $nprog = $tmdb->augmentProgram($prog, $opt_movies_only); if ( $nprog ) { # re-code the modified programme back to original encoding $nprog = XMLTV::Data::Recursive::Encode->encode($encoding, $nprog); $prog = $nprog; } else { $prog = $orig_prog; } $w->write_programme($prog); } @ARGV = ('-') if not @ARGV; XMLTV::parsefiles_callback( \&encoding_cb, \&credits_cb, \&channel_cb, \&programme_cb, @ARGV ); if ( $w ) { # we only get a Writer if the encoding callback gets called $w->end(); } #------------------------------------------------------------------------ # print some stats if ( $opt_stats ) { print STDERR $tmdb->getStatsLines($numberOfSeenChannels); } # destroy the API object $tmdb->closeMovieIndex(); exit(0); ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_to_latex����������������������������������������������������������������������0000775�0000000�0000000�00000013537�15000742332�0016436�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w =pod =head1 NAME tv_to_latex - Convert XMLTV listings to LaTeX source. =head1 SYNOPSIS tv_to_latex [--help] [--with-desc] [--output FILE] [FILE...] =head1 DESCRIPTION Read XMLTV data and output LaTeX source for a summary of listings. The programme titles, subtitles, times and channels are shown. B<--with-desc> include programme description in output B<--output FILE> write to FILE rather than standard output =head1 SEE ALSO L<xmltv(5)>. =head1 AUTHOR Ed Avis, ed@membled.com =head1 BUGS The LaTeX source generated is not perfect, it sometimes produces spurious blank lines in the output. =cut use strict; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use IO::File; use Getopt::Long; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } } # Use Unicode::String if installed, else kludge. my $warned_strip = 0; sub my_u8_to_latin1( $ ) { local $_ = shift; tr/\000-\177//cd && (not $warned_strip++) && warn "stripping non-ASCII characters, install Unicode::String\n"; return $_; } BEGIN { eval { require Unicode::String }; if ($@) { *u8_to_latin1 = \&my_u8_to_latin1; } else { *u8_to_latin1 = sub { Unicode::String::utf8($_[0])->latin1() }; } } use XMLTV::Summarize qw(summarize); use XMLTV::Usage <<END $0: convert listings to LaTeX source for printing usage: $0 [--help] [--with-desc] [--output FILE] [FILE...] END ; sub quote( $ ); my ($opt_help, $opt_output, $opt_withdesc); GetOptions('help' => \$opt_help, 'output=s' => \$opt_output, 'with-desc' => \$opt_withdesc) or usage(0); usage(1) if $opt_help; @ARGV = ('-') if not @ARGV; if (defined $opt_output) { open(STDOUT, ">$opt_output") or die "cannot write to $opt_output: $!"; } $opt_withdesc = 0 if !defined $opt_withdesc; ######## # Configuration # # Width of programme title my $WIDTH = '0.7\textwidth'; $WIDTH = '0.35\textwidth' if $opt_withdesc; # adjust column width if outputting description # Number of programmes in each table (should fit onto a page) my $CHUNK_SIZE = 30; # at least 1, please ######## # End of configuration # # FIXME maybe memoize some stuff here # Print the start of the LaTeX document. print <<'END'; \documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} END # Read input and work out how to handle its encoding. my ($encoding, $credits, $ch, $progs) = @{XMLTV::parsefiles(@ARGV)}; my $to_latin1; for ($encoding) { START: if (not defined) { warn "encoding of input unknown, assuming ISO-8859-1\n"; $_ = 'ISO-8859-1'; goto START; } elsif (/^UTF-?8$/) { $to_latin1 = \&u8_to_latin1; } elsif (/^(?:US-?)ASCII$/ or /^[Ll]atin-?1/ or /^ISO-?8859-?1$/) { $to_latin1 = sub { $_[0] }; # identity } else { warn "unknown encoding $_, assuming ISO-8859-1\n"; $_ = 'ISO-8859-1'; goto START; } } my $table_size = 0; # 0 means no table currently open sub maybe_end_table() { return if not $table_size; print '\end{tabular} \\\\ ', "\n"; $table_size = 0; } # Pass in LaTeX source. sub print_row( $ ) { my $tex = shift; if ($table_size == $CHUNK_SIZE) { maybe_end_table(); # back to 0 } print '\begin{tabular}{r@{--}lp{', $WIDTH, '}r', ($opt_withdesc ? "p{$WIDTH}" : ''), '} ', "\n" if not $table_size++; print $tex; } my @summ = summarize($ch, $progs); foreach (@summ) { die if not defined; if (not ref) { # Heading for a new day. maybe_end_table(); print '\section*{\sf ', quote($to_latin1->($_)), "}\n"; } else { my ($start, $stop, $title, $sub_title, $channel, $desc) = map { defined() ? quote($to_latin1->($_)) : undef } @$_; die if not defined $start; die if not defined $title; die if not defined $channel; # Apparently, you have to put \smallskip _before_ each line # (even the first) in order to get consistent spacing. The # blank line after $title is to explicitly end the paragraph, # so that \raggedright takes effect. # $stop = '' if not defined $stop; $title .= " // $sub_title" if defined $sub_title; $desc = '' if not defined $desc; my $row; if ($opt_withdesc) { $row = <<END \\smallskip $start & $stop & { \\small \\raggedright $title } & $channel & { \\small \\raggedright $desc } \\\\ END } else { $row = <<END \\smallskip $start & $stop & { \\small \\raggedright $title } & $channel \\\\ END } print_row($row); } } maybe_end_table(); print "\\end{flushleft}\n"; # Acknowledgements my $g = $credits->{'generator-info-name'}; $g =~ s!/(\d)! $1! if defined $g; my $s = $credits->{'source-info-name'}; if (not defined $g and not defined $s) { # No acknowledgement since unknown source. } elsif (not defined $g and defined $s) { print 'Generated from \textbf{', quote($to_latin1->($s)), "}.\n"; } elsif (defined $g and not defined $s) { print 'Generated by \textbf{', quote($to_latin1->($g)), "}.\n"; } elsif (defined $g and defined $s) { print 'Generated from \textbf{', quote($to_latin1->($s)), '} by \textbf{', quote($to_latin1->($g)), "}.\n"; } else { die } print "\\end{document}\n"; # quote() # # Quote at least some characters which do funny things in LaTeX. # # Parameters: string to quote # Returns: quoted version # # Copied from <http://membled.com/work/apps/dtd2latex/>; # should put something like this into a 'LaTeX' module some day. # sub quote( $ ) { die 'usage: quote(string)' if @_ != 1; local $_ = shift; # Quote characters s/\\/\\(\\backslash\\)/g; foreach my $ch ('_', '#', '%', '{', '}', '&', '|') { s/\Q$ch\E/\\$ch/g; } s/\$/\\\$/g; foreach my $ch ('<', '>') { s/$ch/\\($ch\\)/g; } s/~/\\(\\sim\\)/g; s/\^/\\(\\hat{}\\)/g; s//\|/g; # Lines of dots s/\.{3,}\s*$/\\dotfill/mg; return $_; } �����������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_to_potatoe��������������������������������������������������������������������0000775�0000000�0000000�00000011575�15000742332�0016774�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2, or (at your option) # any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program; if not, you can either send email to this # program's maintainer or write to: The Free Software Foundation, # Inc.; 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. =pod =head1 NAME tv_to_potatoe - Convert XMLTV listings to potatoe format. =head1 SYNOPSIS tv_to_potatoe [--help] [--outdir PATH] [--lang LANGUAGE] [FILE...] =head1 DESCRIPTION Read XMLTV data and output potatoe files to either the current working directory or the specified one. B<--outdir PATH> write to PATH rather than current working directory B<--lang LANGUAGE> the LANGUAGE you prefer. This argument may be specified multiple times. If no B<--lang> arguments are provided, German is used as the language of choice, followed by English. =head1 SEE ALSO L<xmltv(5)>. =head1 AUTHOR Stefan Siegl, ssiegl@gmx.de =cut use strict; use XMLTV qw(best_name); use XMLTV::Version "$XMLTV::VERSION"; use IO::File; use Date::Manip; use Getopt::Long; sub lisp_quote( $ ); sub get_best_name( $$ ); BEGIN { if (int(Date::Manip::DateManipVersion) >= 6) { Date::Manip::Date_Init("SetDate=now,UTC"); } else { Date::Manip::Date_Init("TZ=UTC"); } } # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } } use XMLTV::Summarize qw(summarize); use XMLTV::Usage <<END $0: Convert XMLTV listings to potatoe format usage: $0 [--help] [--outdir PATH] [--lang LANGUAGE] [FILE...] END ; my @default_langs = ("de", "en"); my $langs = []; my $opt_help = 0; my $opt_outdir = "."; GetOptions('help' => \$opt_help, 'outdir=s' => \$opt_outdir, 'lang=s' => \@$langs ) or usage(0); usage(1) if $opt_help; # use default languages unless at least one was specified by the user push @$langs, @default_langs unless(@$langs); @ARGV = ('-') if not @ARGV; my ($encoding, $credits, $ch, $progs) = @{XMLTV::parsefiles(@ARGV)}; my %channels; $channels{$_->{q(id)}} = get_best_name($_, "display-name") foreach(values %$ch); my %split_by_date; foreach(@$progs) { push @{$split_by_date{substr($_->{q(start)}, 0, 8)}}, $_; } foreach my $date (keys(%split_by_date)) { my ($year, $month, $day) = $date =~ m/([12]...)(..)(..)/; my $filename = $opt_outdir . "/tv-$year-$month-$day"; open HANDLE, ">$filename" or die "cannot open file '$filename' for writing"; # write out the header print HANDLE ";;; -*- lisp -*-\n\n(\n"; foreach(@{$split_by_date{$date}}) { print HANDLE "["; print HANDLE "$year $month $day "; # category my $category = get_best_name($_, "category"); if(defined($category)) { print HANDLE lisp_quote($category), " "; } else { print HANDLE "\"\" "; } # write empty "" to keep potatoe.el # from writing out 'nil' as category. # sorry for the hack ;-) print HANDLE substr($_->{q(start)}, 8, 2), " "; # hour print HANDLE substr($_->{q(start)}, 10, 2), " "; # minute print HANDLE lisp_quote(get_best_name($_, "title")), " "; print HANDLE lisp_quote(undef), " "; # url print HANDLE lisp_quote($channels{$_->{q(channel)}}), " "; print HANDLE lisp_quote($_->{q(showview)}), " "; # vps start time in 'hh:mm' format (I suppose) if(defined $_->{q(vps-start)}) { print HANDLE substr($_->{q(vps-start)}, 8, 2), ":"; # hour print HANDLE substr($_->{q(vps-start)}, 10, 2), " "; # minute } else { print HANDLE "() "; } # what shall we write out as 'aux'? # please vote what you'd like to have here ;) print HANDLE lisp_quote(get_best_name($_, "desc")); # finish line print HANDLE "]\n"; } # finishing file ... print HANDLE ")\n"; close HANDLE or die "unable to close file '$filename'"; } # quote string to be written out into lisp source sub lisp_quote( $ ) { my ($str) = @_; # return nil if either not defined or zero length ... return "()" unless(defined($str) && length($str)); $str =~ s/\"/\\\"/g; # quote! return "\"$str\""; } # get the bestname from programme hash in given field sub get_best_name( $$ ) { my ($prog, $field) = @_; my $bestname = XMLTV::best_name($langs, $prog->{$field}); return undef unless(defined($bestname)); return $bestname->[0]; # return the value from bestname pair } �����������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/filter/tv_to_text�����������������������������������������������������������������������0000775�0000000�0000000�00000005003�15000742332�0016272�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w =pod =head1 NAME tv_to_text - Convert XMLTV listings to text. =head1 SYNOPSIS tv_to_text [--help] [--with-desc] [--output FILE] [FILE...] =head1 DESCRIPTION Read XMLTV data and output a summary of listings. The programme titles, subtitles, times and channels are shown. B<--with-desc> include programme description in output B<--output FILE> write to FILE rather than standard output =head1 SEE ALSO L<xmltv(5)>. =head1 AUTHOR Ed Avis, ed@membled.com =cut use strict; use warnings; use XMLTV qw(best_name); use XMLTV::Version "$XMLTV::VERSION"; use IO::File; use Date::Manip; use Getopt::Long; BEGIN { if (int(Date::Manip::DateManipVersion) >= 6) { Date::Manip::Date_Init("SetDate=now,UTC"); } else { Date::Manip::Date_Init("TZ=UTC"); } } # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } } use XMLTV::Summarize qw(summarize); use XMLTV::Usage <<END $0: convert listings to plain text usage: $0 [--help] [--with-desc] [--output FILE] [FILE...] END ; my ($opt_help, $opt_output, $opt_withdesc); GetOptions('help' => \$opt_help, 'output=s' => \$opt_output, 'with-desc' => \$opt_withdesc) or usage(0); usage(1) if $opt_help; @ARGV = ('-') if not @ARGV; if (defined $opt_output) { open(STDOUT, ">$opt_output") or die "cannot write to $opt_output: $!"; } $opt_withdesc = 0 if !defined $opt_withdesc; # FIXME maybe memoize some stuff here my ($encoding, $credits, $ch, $progs) = @{XMLTV::parsefiles(@ARGV)}; my $wrote_prog = 0; foreach (summarize($ch, $progs)) { if (not ref) { print "\n" if $wrote_prog; print "$_\n\n"; next; } my ($start, $stop, $title, $sub_title, $channel, $desc) = @$_; $stop = '' if not defined $stop; $title .= " // $sub_title" if defined $sub_title; print "$start--$stop\t$title\t$channel". ( $opt_withdesc && defined $desc ? "\t$desc" : '' ) . "\n"; $wrote_prog = 1; } # Acknowledgements my $g = $credits->{'generator-info-name'}; $g =~ s!/(\d)! $1! if defined $g; my $s = $credits->{'source-info-name'}; if (not defined $g and not defined $s) { # No acknowledgement since unknown source. } elsif (not defined $g and defined $s) { print "\nGenerated from $s.\n"; } elsif (defined $g and not defined $s) { print "\nGenerated by $g.\n"; } elsif (defined $g and defined $s) { print "\nGenerated from $s by $g.\n"; } else { die } �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/�����������������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0013575�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/Config_file.pm���������������������������������������������������������������������0000664�0000000�0000000�00000005621�15000742332�0016343�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������package XMLTV::Config_file; use strict; use XMLTV::Ask; # First argument is an explicit config filename or undef. The second # argument is the name of the current program (probably best not to # use $0 for this). Returns the config filename to use (the file may # not necessarily exist). Third argument is a 'quiet' flag (default # false). # # May do other magic things like migrating a config file to a new # location; you can specify the old program name as an optional fourth # argument if your program has recently been renamed. # sub filename( $$;$$ ) { my ($explicit, $progname, $quiet, $old_progname) = @_; return $explicit if defined $explicit; $quiet = 0 if not defined $quiet; my $home = $ENV{HOME}; $home = '.' if not defined $home; my $conf_dir = "$home/.xmltv"; (-d $conf_dir) or mkdir($conf_dir, 0777) or die "cannot mkdir $conf_dir: $!"; my $new = "$conf_dir/$progname.conf"; my @old; for ($old_progname) { push @old, "$conf_dir/$_.conf" if defined } foreach (@old) { if (-f and not -e $new) { warn "migrating config file $_ -> $new\n"; rename($_, $new) or die "cannot rename $_ to $new: $!"; last; } } print STDERR "using config filename $new\n" unless $quiet; return $new; } # If the given file exists, ask for confirmation of overwriting it; # exit if no. # sub check_no_overwrite( $ ) { my $f = shift; if (-s $f) { if (not ask_boolean <<END A nonempty configuration file $f already exists. There is currently no support for altering an existing configuration: you have to reconfigure from scratch. Do you wish to overwrite the old configuration? END , 0) { say( "Exiting since you don't want to overwrite the old configuration." ); exit 0; } } } # Take a filename and return a list of lines with comments and # leading/trailing whitespace stripped. Blank lines are returned as # undef, so the number of lines returned is the same as the original # file. # # Dies ('run --configure') if the file doesn't exist. # # Arguments: # filename # # (optional, default false) whether the file is created at xmltv # installation. This controls the message given when it's not # found. If false, you need to run --configure; if true, xmltv # was not correctly installed. # sub read_lines( $;$ ) { my ($f, $is_installed) = @_; $is_installed = 0 if not defined $is_installed; local *FH; if (not -e $f) { if ($is_installed) { die "cannot find $f, xmltv was not installed correctly\n"; } else { die "config file $f does not exist, run me with --configure\n"; } } open(FH, $f) or die "cannot read $f: $!\n"; my @r; while (<FH>) { s/\#.*//; s/^\s+//; s/\s+$//; undef $_ if not length; push @r, $_; } close FH or die "cannot close $f: $!\n"; die "config file $f is empty, please delete and run me with --configure\n" if not @r; return @r; } 1; ���������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/DST.pm�����������������������������������������������������������������������������0000664�0000000�0000000�00000030027�15000742332�0014567�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Timezone stuff, including routines to guess timezones in European # (and other) countries that have daylight saving time. # # Warning: this might break if Date::Manip is initialized to some # timezone other than UTC: best to call Date_Init('TZ=+0000') first. package XMLTV::DST; use strict; use Carp qw(croak); use Date::Manip; # no Date_Init(), that can be done by the app use XMLTV::TZ qw(gettz tz_to_num offset_to_gmt); use XMLTV::Date; # Three modes: # eur (default): Europe and elsewhere # na: US (most states) and Canada # none: places that don't observe DST # our $Mode = 'eur'; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } # Memoize some subroutines if possible. FIXME commonize to # XMLTV::Memoize. We are memoizing our own routines plus gettz() from # XMLTV::TZ, that too needs sorting out. # eval { require Memoize }; unless ($@) { foreach (qw(parse_local_date date_to_local dst_dates parse_date UnixDate DateCalc Date_Cmp gettz)) { Memoize::memoize($_) or die "cannot memoize $_: $!"; } } use base 'Exporter'; our @EXPORT = qw(parse_local_date date_to_local utc_offset); # parse_local_date() # # Wrapper for parse_date() that tries to guess what timezone a date is # in. You must pass in the 'base' timezone as the second argument: # this base timezone gives winter time, and summer time is one hour # ahead. So the base will be UTC for Britain, Ireland and Portugal, # UTC+1 for many other countries. # # If the date already has a timezone it is left alone, but undef is # returned if the explicit timezone doesn't match winter or # summer time for the base passed in. # # The switchover from winter to summer time gives a one hour window of # 'impossible' times when the clock goes forward; those give undef. # Putting the clocks back in autumn gives one hour of ambiguous times; # we assume summer time for those. # # Parameters: # unparsed date from some country following EU DST conventions # base timezone giving winter time in that country # # Returns: parsed date. Throws exception if error. # sub parse_local_date($$) { # local $Log::TraceMessages::On = 1; my ($date, $base) = @_; croak 'usage: parse_local_date(unparsed date, base timeoffset)' if @_ != 2 or not defined $date or not defined $base; croak 'second parameter must be a time offset (+xxxx,-xxxx)' if( $base !~ /^[-+]\d{4}$/ ); my $winter_tz = $base; my $summer_tz = sprintf('%+05d', $winter_tz + 100); # 'one hour' my $got_tz = gettz($date); # t "got timezone $got_tz from date $date"; if (defined $got_tz) { # Need to work out whether the timezone is one of the two # allowable values (or UTC, that's always okay). # # I don't remember the reason for this check... perhaps it is # just paranoia. # my $got_tz_num = tz_to_num($got_tz); croak "got timezone $got_tz from $date, but it's not $winter_tz, $summer_tz or UTC\n" if $got_tz_num ne $winter_tz and $got_tz_num ne $summer_tz and $got_tz_num ne '+0000'; # One thing we don't check is that the explicit timezone makes # sense for this time of year. So you can specify summer # time even in January if you want. # # OK, the timezone is there and it looks sane, continue. return parse_date($date); } t 'no timezone present, we need to guess'; my $dp = parse_date($date); t "parsed date string $date into: " . d $dp; # Start and end of summer time in that year, in UTC my $year = UnixDate($dp, '%Y'); t "year of date is $year"; die "cannot convert Date::Manip object $dp to year" if not defined $year; # Start and end dates of DST in local winter time. my ($start_dst, $end_dst); if ($Mode eq 'eur') { ($start_dst, $end_dst) = @{dst_dates($year)}; } elsif ($Mode eq 'na') { ($start_dst, $end_dst) = @{dst_dates_na($year, $winter_tz)}; } elsif ($Mode eq 'none') { return Date_ConvTZ($dp, offset_to_gmt($winter_tz), 'UTC'); } else { die } foreach ($start_dst, $end_dst) { $_ = Date_ConvTZ($_, 'UTC', offset_to_gmt($winter_tz)); } # The clocks shift backwards and forwards by one hour. my $clock_shift = "1 hour"; # The times that the clocks go forward to in spring (local time) my $start_dst_skipto = DateCalc($start_dst, "+ $clock_shift"); # The local time when the clocks go back my $end_dst_backfrom = DateCalc($end_dst, "+ $clock_shift"); my $summer; if (Date_Cmp($dp, $start_dst) < 0) { # Before the start of summer time. $summer = 0; } elsif (Date_Cmp($dp, $start_dst) == 0) { # Exactly _at_ the start of summer time. Really such a date # should not exist since the clocks skip forward an hour at # that point. But we tolerate this fencepost error. # $summer = 0; } elsif (Date_Cmp($dp, $start_dst_skipto) < 0) { # This date is impossible, since the clocks skip forwards an # hour from $start_dst to $start_dst_skipto. But some # listings sources seem to use it. Assume it means winter # time. # $summer = 0; } elsif (Date_Cmp($dp, $end_dst) < 0) { # During summer time. $summer = 1; } elsif (Date_Cmp($dp, $end_dst_backfrom) < 0) { # warn("$date is ambiguous " # . "(clocks go back from $end_dst_backfrom $summer_tz to $end_dst $winter_tz), " # . "assuming $summer_tz" ); $summer = 1; } else { # Definitely after the end of summer time. $summer = 0; } if ($summer) { t "summer time, converting $dp from $summer_tz to UTC"; return Date_ConvTZ($dp, offset_to_gmt($summer_tz), 'UTC'); } else { t "winter time, converting $dp from $winter_tz to UTC"; return Date_ConvTZ($dp, offset_to_gmt($winter_tz), 'UTC'); } } # date_to_local() # # Take a date in UTC and convert it to one of two timezones, depending # on when during the year it is. # # Parameters: # date in UTC (from parse_date()) # base timezone (winter time) # # Returns ref to list of # new date # timezone of new date # # For example, date_to_local with a date of 13:00 on June 10th 2000 and # a base timezone of UTC would be be 14:00 +0100 on the same day. The # input and output date are both in Date::Manip internal format. # sub date_to_local( $$ ) { my ($d, $base_tz) = @_; croak 'date_to_local() expects a Date::Manip object as first argument' if (not defined $d) or ($d !~ /\S/); my $year = UnixDate($d, '%Y'); if ((not defined $year) or ($year !~ tr/0-9//)) { croak "cannot get year from '$d'"; } # Find the start and end dates of summer time. my ($start_dst, $end_dst); if ($Mode eq 'eur') { ($start_dst, $end_dst) = @{dst_dates($year)}; } elsif ($Mode eq 'na') { ($start_dst, $end_dst) = @{dst_dates_na($year, $base_tz)}; } elsif ($Mode eq 'none') { return [ Date_ConvTZ($d, 'UTC', offset_to_gmt($base_tz)), $base_tz ]; } else { die } my $use_tz; if (Date_Cmp($d, $start_dst) < 0) { # Before the start of summer time. $use_tz = $base_tz; } elsif (Date_Cmp($d, $end_dst) < 0) { # During summer time. my $base_tz_num = tz_to_num($base_tz); $use_tz = sprintf('%+05d', $base_tz_num + 100); # one hour } else { # After summer time. $use_tz = $base_tz; } die if not defined $use_tz; return [ Date_ConvTZ($d, 'UTC', offset_to_gmt($use_tz)), $use_tz ]; } # utc_offset() # # Given a date/time string in a parse_date() compatible format # (preferably YYYYMMDDhhmmss) and a 'base' timezone (eg '+0100'), # return this time string with UTC offset appended. The 'base' # timezone should be the non-DST timezone for the country ('winter # time'). This function figures out (through parse_local_date() and # date_to_local()) whether DST is in effect for the specified date, and # adjusts the UTC offset appropriately. # sub utc_offset( $$ ) { my ($indate, $basetz) = @_; croak "empty date" if not defined $indate; croak "empty base TZ" if not defined $basetz; $basetz = tz_to_num( $basetz ) if $basetz !~ /^[-+]\d{4}$/; my $d = date_to_local(parse_local_date($indate, $basetz), $basetz); return UnixDate($d->[0],"%Y%m%d%H%M%S") . " " . $d->[1]; } # dst_dates() # # Return the dates (in UTC) when summer starts and ends in a given # year. Private. # # According to <http://www.rog.nmm.ac.uk/leaflets/summer/summer.html>, # summer time starts at 01:00 on the last Sunday in March, and ends at # 01:00 on the last Sunday in October. That's 01:00 UTC in both # cases, irrespective of what the winter and summer timezones are. # This has been the case throughout the European Union since 1998, and # some other countries such as Norway follow the same rules. # # Parameters: year (only 1998 or later works) # # Returns: ref to list of # start time and date of summer time (in UTC) # end time and date of summer time (in UTC) # sub dst_dates( $ ) { die "usage: dst_dates(year), got args: @_" if @_ != 1; my $year = shift; die "don't know about DST before 1998" if $year < 1998; my ($start_dst, $end_dst); foreach (25 .. 31) { my $mar = "$year-03-$_" . ' 01:00:00 +0000'; my $mar_d = parse_date($mar); $start_dst = $mar_d if UnixDate($mar_d, "%A") =~ /Sunday/; # A time between '00:00' and '01:00' just before the last # Sunday in October is ambiguous. # my $oct = "$year-10-$_" . ' 01:00:00 +0000'; my $oct_d = parse_date($oct); $end_dst = $oct_d if UnixDate($oct_d, "%A") =~ /Sunday/; } die if not defined $start_dst or not defined $end_dst; return [ $start_dst, $end_dst ]; } sub dst_dates_na( $$ ) { die "usage: dst_dates(year, winter_tz), got args: @_" if @_ != 2; my ($year, $winter_tz) = @_; die "don't know about DST before 1988" if $year < 1988; return dst_dates_na_old($year, $winter_tz) if $year < 2007; return dst_dates_na_new($year, $winter_tz); } # Old North American daylight saving time, used before 2007. sub dst_dates_na_old( $$ ) { my ($year, $winter_tz) = @_; $winter_tz =~ /^\s*-\s*(\d\d)(?:00)?\s*$/ or die "bad North American winter time zone $winter_tz"; my $hours = $1; my ($start_dst, $end_dst); foreach (1 .. 31) { if (not defined $start_dst and $_ < 31) { my $date = "$year-04-$_"; my $day = UnixDate(parse_date($date), '%A'); if ($day =~ /Sunday/) { # First Sunday in April. DST starts at 02:00 local # standard time. # $start_dst = Date_ConvTZ(parse_date("$date 02:00"), offset("-$winter_tz"), 'UTC'); } } my $date = "$year-10-$_"; my $day = UnixDate(parse_date($date), '%A'); next unless $day =~ /Sunday/; # A Sunday in October (and the last one we see will be the # last Sunday). DST ends at 01:00 local standard time. # $end_dst = Date_ConvTZ(parse_date("$date 01:00"), offset_to_gmt("-$winter_tz"), 'UTC'); } die if not defined $start_dst or not defined $end_dst; return [ $start_dst, $end_dst ]; } # New US daylight saving time from 2007, also followed by most # Canadian provinces. # sub dst_dates_na_new( $$ ) { my ($year, $winter_tz) = @_; $winter_tz =~ /^\s*-\s*(\d\d)(?:00)?\s*$/ or die "bad North American winter time zone $winter_tz"; my $hours = $1; my ($start_dst, $end_dst); my $seen_Sunday_in_March = 0; foreach (1 .. 31) { if (not defined $start_dst) { my $date = "$year-03-$_"; my $day = UnixDate(parse_date($date), '%A'); if ($day =~ /Sunday/) { if ($seen_Sunday_in_March) { # Second Sunday in March. DST starts at 02:00 # local standard time. # $start_dst = Date_ConvTZ(parse_date("$date 02:00"), offset_to_gmt("-$winter_tz"), 'UTC'); } else { $seen_Sunday_in_March = 1; } } } next if defined $end_dst; my $date = "$year-11-$_"; my $day = UnixDate(parse_date($date), '%A'); next unless $day =~ /Sunday/; # A Sunday in November (and the first one we see). DST ends # at 01:00 local standard time. # $end_dst = Date_ConvTZ(parse_date("$date 01:00"), offset_to_gmt("-$winter_tz"), 'UTC'); } die if not defined $start_dst or not defined $end_dst; return [ $start_dst, $end_dst ]; } 1; ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/Get_nice.pm������������������������������������������������������������������������0000664�0000000�0000000�00000017510�15000742332�0015654�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Library to wrap LWP::UserAgent to put in a random delay between # requests and set the User-Agent string. We really should be using # LWP::RobotUI but this is better than nothing. # # If you're sure your app doesn't need a random delay (because it is # fetching from a site designed for that purpose) then set # $XMLTV::Get_nice::Delay to zero, or a value in seconds. This is the # maximum delay - on average the sleep will be half that. # #This random delay will be between 0 and 5 ($Delay) seconds. This means # some sites will complain you're grabbing too fast (since 20% of your # grabs will be less than 1 second apart). To introduce a minimum delay # set $XMLTV::Get_nice::MinDelay to a value in seconds. # This will be added to $Delay to derive the actual delay used. # E.g. Delay = 5 and MinDelay = 3, then the actual delay will be # between 3 and 8 seconds, # # get_nice() is the function to call, however # XMLTV::Get_nice::get_nice_aux() is the one to cache with # XMLTV::Memoize or whatever. If you want an HTML::Tree object use # get_nice_tree(). # Alternatively, get_nice_json() will get you a JSON object, # or get_nice_xml() will get a XML::Parser 'Tree' object use strict; package XMLTV::Get_nice; # use version number for feature detection: # 0.005065 : new methods get_nice_json(), get_nice_xml() # 0.005065 : add decode option to get_nice_tree() # 0.005065 : expose the LWP response object ($Response) # 0.005066 : support unknown tags in HTML::TreeBuilder ($IncludeUnknownTags) # 0.005067 : new method post_nice_json() # 0.005070 : skip get_nice sleep for cached pages # 0.005070 : support passing HTML::TreeBuilder options via a hashref our $VERSION = 0.005070; use base 'Exporter'; our @EXPORT = qw(get_nice get_nice_tree get_nice_xml get_nice_json post_nice_json error_msg); use Encode qw(decode); use LWP::UserAgent; use XMLTV; our $Delay = 5; # in seconds our $MinDelay = 0; # in seconds our $FailOnError = 1; # Fail on fetch error our $Response; # LWP response object our $IncludeUnknownTags = 0; # add support for HTML5 tags which are unknown to older versions of TreeBuilder (and therfore ignored by it) our $ua = LWP::UserAgent->new; $ua->agent("xmltv/$XMLTV::VERSION"); $ua->env_proxy; our %errors = (); sub error_msg($) { my ($url) = @_; $errors{$url}; } sub get_nice( $ ) { # This is to ensure scalar context, to work around weirdnesses # with Memoize (I just can't figure out how SCALAR_CACHE and # LIST_CACHE relate to each other, with or without MERGE). # return scalar get_nice_aux($_[0]); } # Fetch page and return as HTML::Tree object. # Optional arguments: # i) a function to put the page data through (eg, to clean up bad characters) # before parsing. # ii) convert incoming page to UNICODE using this codepage (use "UTF-8" for # strict utf-8) # iii) a hashref containing options to configure the HTML::TreeBuilder object # before parsing # sub get_nice_tree( $;$$$ ) { my ($uri, $filter, $codepage, $htb_opts) = @_; require HTML::TreeBuilder; my $content = get_nice $uri; $content = $filter->($content) if $filter; if ($codepage) { $content = decode($codepage, $content); } else { $content = decode('UTF-8', $content); } my $t = HTML::TreeBuilder->new(); $t->ignore_unknown(!$IncludeUnknownTags); if (ref $htb_opts eq 'HASH') { $t->$_($htb_opts->{$_}) foreach (keys %$htb_opts); } $t->parse($content) or die "cannot parse content of $uri\n"; $t->eof; return $t; } # Fetch page and return as XML::Parser 'Tree' object. # Optional arguments: # i) a function to put the page data through (eg, to clean up bad # characters) before parsing. # ii) convert incoming page to UNICODE using this codepage (use "UTF-8" for strict utf-8) # sub get_nice_xml( $;$$ ) { my ($uri, $filter, $codepage) = @_; require XML::Parser; my $content = get_nice $uri; $content = $filter->($content) if $filter; if ($codepage) { $content = decode($codepage, $content); } else { $content = decode('UTF-8', $content); } my $t = XML::Parser->new(Style => 'Tree')->parse($content) or die "cannot parse content of $uri\n"; return $t; } # Fetch page and return as JSON object. # Optional arguments: # i) a function to put the page data through (eg, to clean up bad # characters) before parsing. # ii) convert incoming UTF-8 to UNICODE # sub get_nice_json( $;$$ ) { my ($uri, $filter, $utf8) = @_; require JSON; my $content = get_nice $uri; $content = $filter->($content) if $filter; $utf8 = defined $utf8 ? 1 : 0; my $t = JSON->new()->utf8($utf8)->decode($content) or die "cannot parse content of $uri\n"; return $t; } my $last_get_time; my $last_get_from_cache; sub get_nice_aux( $ ) { my $url = shift; if (defined $last_get_time && (defined $last_get_from_cache && !$last_get_from_cache) ) { # A page has already been retrieved recently. See if we need # to sleep for a while before getting the next page - being # nice to the server. # my $next_get_time = $last_get_time + (rand $Delay) + $MinDelay; my $sleep_time = $next_get_time - time(); sleep $sleep_time if $sleep_time > 0; } my $r = $ua->get($url); # Then start the delay from this time on the next fetch - so we # make the gap _between_ requests rather than from the start of # one request to the start of the next. This punishes modem users # whose individual requests take longer, but it also punishes # downloads that take a long time for other reasons (large file, # slow server) so it's about right. # $last_get_time = time(); # expose the response object for those grabbers which need to process the headers, status code, etc. $Response = $r; # Set flag if last fetch was from local HTTP::Cache::Transparent cache. # Check for presence of both x-content-unchanged and x-cached headers. $last_get_from_cache = (defined $r->{'_headers'}{'x-content-unchanged'} && defined $r->{'_headers'}{'x-cached'} && $r->{'_headers'}{'x-cached'} == 1); if ($r->is_error) { # At the moment download failures seem rare, so the script dies if # any page cannot be fetched. We could later change this routine # to return undef on failure. But dying here makes sure that a # failed page fetch doesn't get stored in XMLTV::Memoize's cache. # die "could not fetch $url, error: " . $r->status_line . ", aborting\n" if $FailOnError; $errors{$url} = $r->status_line; return undef; } else { return $r->content; } } # Fetch page via a JSON object in the Content and return as a JSON object. # Arguments: # URI to post to # JSON object with the AJAX data to be posted e.g. "{ 'programId':'123456', 'channel':'BBC'}" # sub post_nice_json( $$ ) { my $url = shift; my $json = shift; require JSON; if (defined $last_get_time) { # A page has already been retrieved recently. See if we need # to sleep for a while before getting the next page # my $next_get_time = $last_get_time + (rand $Delay) + $MinDelay; my $sleep_time = $next_get_time - time(); sleep $sleep_time if $sleep_time > 0; } my $r = $ua->post($url, 'Content_Type' => 'application/json; charset=utf-8', 'Content' => $json); $last_get_time = time(); # expose the response object for those grabbers which need to process the headers, status code, etc. $Response = $r; if ($r->is_error) { die "could not fetch $url, error: " . $r->status_line . ", aborting\n" if $FailOnError; $errors{$url} = $r->status_line; return undef; } else { my $content = JSON->new()->utf8(1)->decode($r->content) or die "cannot parse content of $url\n"; return $content; } } 1; ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/Grab_XML.pm������������������������������������������������������������������������0000664�0000000�0000000�00000024662�15000742332�0015540�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������package XMLTV::Grab_XML; use strict; use Getopt::Long; use Date::Manip; use XMLTV; use XMLTV::Usage; use XMLTV::Memoize; use XMLTV::ProgressBar; use XMLTV::Ask; use XMLTV::TZ qw(parse_local_date); use XMLTV::Get_nice qw(); use XMLTV::Date; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } } =pod =head1 NAME XMLTV::Grab_XML - Perl extension to fetch raw XMLTV data from a site =head1 SYNOPSIS package Grab_XML_rur; use base 'XMLTV::Grab_XML'; sub urls_by_date( $ ) { my $pkg = shift; ... } sub country( $ ) { my $pkg = shift; return 'Ruritania' } # Maybe override a couple of other methods as described below... Grab_XML_rur->go(); =head1 DESCRIPTION This module helps to write grabbers which fetch pages in XMLTV format from some website and output the data. It is not used for grabbers which scrape human-readable sites. It consists of several class methods (package methods). The way to use it is to subclass it and override some of these. =head1 METHODS =over =item XMLTV::Grab_XML->date_init() Called at the start of the program to set up Date::Manip. You might want to override this with a method that sets the timezone. =cut sub date_init( $ ) { my $pkg = shift; Date_Init(); } =pod =item XMLTV::Grab_XML->urls_by_date() Returns a hash mapping YYYYMMDD dates to a URL where listings for that date can be downloaded. This method is abstract, you must override it. Arguments: the command line options for --config-file and --quiet. =cut sub urls_by_date( $$$ ) { my $pkg = shift; die 'abstract class method: override in subclass'; } =pod =item XMLTV::Grab_XML->xml_from_data(data) Given page data for a particular day, turn it into XML. The default implementation just returns the data unchanged, but you might override it if you need to decompress the data or patch it up. =cut sub xml_from_data( $$ ) { my $pkg = shift; t 'Grab_XML::xml_from_data()'; return shift; # leave unchanged } =pod =item XMLTV::Grab_XML->configure() Configure the grabber if needed. Arguments are --config-file option (or undef) and --quiet flag (or undef). This method is not provided in the base class; if you don't provide it then attempts to --configure will give a message that configuration is not necessary. =item XMLTV::Grab_XML->nextday(day) Bump a YYYYMMDD date by one. You probably shouldnE<39>t override this. =cut sub nextday( $$ ) { my $pkg = shift; my $d = shift; $d =~ /^\d{8}$/ or die; my $p = parse_date($d); my $n = DateCalc($p, '+ 1 day'); die if not defined $n; return UnixDate($n, '%Q'); } =item XMLTV::Grab_XML->country() Return the name of the country youE<39>re grabbing for, used in usage messages. Abstract. =cut sub country( $ ) { my $pkg = shift; die 'abstract class method: override in subclass'; } =item XMLTV::Grab_XML->usage_msg() Return a command-line usage message. This calls C<country()>, so you probably need to override only that method. =cut sub usage_msg( $ ) { my $pkg = shift; my $country = $pkg->country(); if ($pkg->can('configure')) { return <<END $0: get $country television listings in XMLTV format usage: $0 --configure [--config-file FILE] $0 [--output FILE] [--days N] [--offset N] [--quiet] [--config-file FILE] $0 --help END ; } else { return <<END $0: get $country television listings in XMLTV format usage: $0 [--output FILE] [--days N] [--offset N] [--quiet] $0 --help END ; } } =item XMLTV::Grab_XML->get() Given a URL, fetch the content at that URL. The default implementation calls XMLTV::Get_nice::get_nice() but you might want to override it if you need to do wacky things with http requests, like cookies. Note that while this method fetches a page, C<xml_from_data()> does any further processing of the result to turn it into XML. =cut sub get( $$ ) { my $pkg = shift; my $url = shift; return XMLTV::Get_nice::get_nice($url); } =item XMLTV::Grab_XML->go() The main program. Parse command line options, fetch and write data. Most of the options are fairly self-explanatory but this routine also calls the XMLTV::Memoize module to look for a B<--cache> argument. The functions memoized are those given by the C<cachables()> method. =cut sub go( $ ) { my $pkg = shift; XMLTV::Memoize::check_argv($pkg->cachables()); my ($opt_days, $opt_help, $opt_output, $opt_share, $opt_gui, $opt_offset, $opt_quiet, $opt_configure, $opt_config_file, $opt_list_channels, ); $opt_offset = 0; # default $opt_quiet = 0; # default GetOptions('days=i' => \$opt_days, 'help' => \$opt_help, 'output=s' => \$opt_output, 'share=s' => \$opt_share, # undocumented 'gui:s' => \$opt_gui, 'offset=i' => \$opt_offset, 'quiet' => \$opt_quiet, 'configure' => \$opt_configure, 'config-file=s' => \$opt_config_file, 'list-channels' => \$opt_list_channels, ) or usage(0, $pkg->usage_msg()); die 'number of days must not be negative' if (defined $opt_days && $opt_days < 0); usage(1, $pkg->usage_msg()) if $opt_help; usage(0, $pkg->usage_msg()) if @ARGV; XMLTV::Ask::init($opt_gui); if ($opt_share) { if ($pkg->can('set_share_dir')) { $pkg->set_share_dir($opt_share); } else { print STDERR "share directory not in use\n"; } } my $has_config = $pkg->can('configure'); if ($opt_configure) { if ($has_config) { $pkg->configure($opt_config_file, $opt_quiet); } else { print STDERR "no configuration necessary\n"; } exit; } for ($opt_config_file) { warn("this grabber has no configuration, so ignoring --config-file\n"), undef $_ if defined and not $has_config; } # Need to call parse_local_date() before any resetting of # Date::Manip's timezone. # my $now = DateCalc(parse_local_date('now'), "$opt_offset days"); die if not defined $now; $pkg->date_init(); my $today = UnixDate($now, '%Q'); my %urls = $pkg->urls_by_date($opt_config_file, $opt_quiet); t 'URLs by date: ' . d \%urls; my @to_get; if ($opt_list_channels) { # We won't bother to do an exhaustive check for every option # that is ignored with --list-channels. # die "useless to give --days or --offset with --list-channels\n" if defined $opt_days or $opt_offset != 0; # For now, assume that the upstream site doesn't provide any # way to get just the channels, so we'll have to pick a # listings file and then discard most of it. # my @dates = sort keys %urls; die 'no dates found on site' if not @dates; my $latest = $dates[-1]; @to_get = $urls{$latest}; } else { # Getting programme listings. my $days_left = $opt_days; t '$days_left starts at ' . d $days_left; t '$today=' . d $today; for (my $day = $today; defined $urls{$day}; $day = $pkg->nextday($day)) { t "\$urls{$day}=" . d $urls{$day}; if (defined $days_left and $days_left-- == 0) { t 'got to last day'; last; } push @to_get, $urls{$day}; } if (defined $days_left and $days_left > 0) { warn "couldn't get all of $opt_days days, only " . ($opt_days - $days_left) . "\n"; } elsif (not @to_get) { warn "couldn't get any listings from the site for today or later\n"; } } my $bar = new XMLTV::ProgressBar('downloading listings', scalar @to_get) if not $opt_quiet; my @listingses; foreach my $url (@to_get) { my $xml; # Set error handlers. Strange bugs if you call warn() or # die() inside these, at least I have seen such bugs in # XMLTV.pm, so I'm avoiding it here. # local $SIG{__WARN__} = sub { my $msg = shift; $msg = "warning: something's wrong" if not defined $msg; print STDERR "$url: $msg\n"; }; local $SIG{__DIE__} = sub { my $msg = shift; $msg = 'died' if not defined $msg; print STDERR "$url: $msg, exiting\n"; exit(1); }; my $got = $pkg->get($url); if (not defined $got) { warn 'failed to download, skipping'; next; } $xml = $pkg->xml_from_data($got); t 'got XML: ' . d $xml; if (not defined $xml) { warn 'could not get XML from page, skipping'; next; } push @listingses, XMLTV::parse($xml); update $bar if not $opt_quiet; } $bar->finish() if not $opt_quiet; my %w_args = (); if (defined $opt_output) { my $fh = new IO::File ">$opt_output"; die "cannot write to $opt_output\n" if not $fh; %w_args = (OUTPUT => $fh); } if ($opt_list_channels) { die if @listingses != 1; my $l = $listingses[0]; undef $l->[3]; # blank out programme data XMLTV::write_data($l, %w_args); } else { XMLTV::write_data(XMLTV::cat(@listingses), %w_args); } } =item XMLTV::Grab_XML->cachables() Returns a list of names of functions which could reasonably be memoized between runs. This will normally be whatever function fetches the web pages - you memoize that to save on repeated downloads. A subclass might want to add things to this list if it has its own way of fetching web pages. =cut sub cachables( $ ) { my $pkg = shift; return ('XMLTV::Get_nice::get_nice_aux'); } =pod =item XMLTV::Grab_XML->remove_early_stop_times() Checks each stop time and removes it if it's before the start time. Argument: the XML to correct Returns: the corrected XML =cut my $warned_bad_stop_time = 0; sub remove_early_stop_times( $$ ) { my $pkg = shift; my @lines = split /\n/, shift; foreach (@lines) { if (/<programme/) { # First change to numeric timezones. s{(start|stop)="(\d+) ([A-Z]+)"} {qq'$1="$2 ' . tz_to_num($3) . '"'}eg; # Now remove stop times before start. Only worry about # cases where the timezone is the same - we hope the # upstream data will be fixed by the next TZ changeover. # /start="(\d+) (\S+)"/ or next; my ($start, $tz) = ($1, $2); /stop="(\d+) \Q$tz\E"/ or next; my $stop = $1; if ($stop lt $start) { warn "removing stop time before start time: $_" unless $warned_bad_stop_time++; s/stop="[^""]+"\s*// or die; } } } return join("\n", @lines); } =pod =back =head1 AUTHOR Ed Avis, ed@membled.com =head1 SEE ALSO L<perl(1)>, L<XMLTV(3)>. =cut 1; ������������������������������������������������������������������������������xmltv-1.4.0/grab/Memoize.pm�������������������������������������������������������������������������0000664�0000000�0000000�00000006530�15000742332�0015544�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Just some routines related to the Memoize module that are used in # more than one place in XMLTV. But not general enough to merge back # into Memoize. package XMLTV::Memoize; use strict; use File::Basename; use Getopt::Long; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } # Add an undocumented option to cache things in a DB_File database. # You need to decide which subroutines should be cached: see # XMLTV::Get_nice for how to memoize web page fetches. Call like # this: # # if (check_argv('fred', 'jim')) { # # The subs fred() and jim() are now memoized. # } # # If the user passed a --cache option to your program, this will be # removed from @ARGV and caching will be turned on. The optional # argument to --cache gives the filename to use. # # Currently it is assumed that the function gives the same result in # both scalar and list context. # # Note that the Memoize module is not loaded unless --cache options # are found. # # Returns a ref to a list of code references for the memoized # versions, if memoization happened (but does install the memoized # versions under the original names too). Returns undef if no # memoization was wanted. # sub check_argv( @ ) { # local $Log::TraceMessages::On = 1; my $yes = 0; my $p = new Getopt::Long::Parser(config => ['passthrough']); die if not $p; my $opt_cache; my $opt_quiet = 0; my $result = $p->getoptions('cache:s' => \$opt_cache, 'quiet' => \$opt_quiet ); die "failure processing --cache option" if not $result; unshift @ARGV, "--quiet" if $opt_quiet; return undef if not defined $opt_cache; my $filename; if ($opt_cache eq '') { # --cache given, but no filename. Guess one. my $basename = File::Basename::basename($0); $filename = "$basename.cache"; } else { $filename = $opt_cache; } print STDERR "using cache $filename\n" unless $opt_quiet; require POSIX; require Memoize; require DB_File; # Annoyingly tie(%cache, @tie_args) doesn't work #my @tie_args = ('DB_File', $filename, # POSIX::O_RDWR() | POSIX::O_CREAT(), 0666); # $from_caller is a sub which converts a function name into one # seen from the caller's namespace. Namespaces do not nest, so if # it already has :: it should be left alone. # my $caller = caller(); t "caller: $caller"; my $from_caller = sub( $ ) { for (shift) { return $_ if /::/; return "${caller}::$_"; } }; # Annoyingly tie(%cache, @tie_args) doesn't work my %cache; tie %cache, 'DB_File', $filename, POSIX::O_RDWR() | POSIX::O_CREAT(), 0666; my @r; foreach (@_) { my $r = Memoize::memoize($from_caller->($_), SCALAR_CACHE => [ HASH => \%cache ], # # Memoize 1.03 broke tied SCALAR_CACHE with # together with LIST_CACHE => 'MERGE'. See # bug report on CPAN: # # https://rt.cpan.org/Public/Bug/Display.html?id=91927 # # As no user of this module calls memoized # functions in list context, we can simply # replace it with 'FAULT'. # #LIST_CACHE => 'MERGE'); LIST_CACHE => 'FAULT'); die "could not memoize $_" if not $r; push @r, $r; } return \@r; } 1; ������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/Mode.pm����������������������������������������������������������������������������0000664�0000000�0000000�00000001636�15000742332�0015025�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# A simple library to handle mutually exclusive choices. For example # # my $mode = XMLTV::Mode::mode('eat', # default # $opt_walk => 'walk', # $opt_sleep => 'sleep', # ); # # Only one of the choices can be active and mode() will die() with an # error message if $opt_walk and $opt_sleep are both set. It will # otherwise return one of the strings 'eat', 'walk' or 'sleep'. # # TODO find some way of getting this cleanly into Getopt::Long. package XMLTV::Mode; sub mode( $@ ) { my $default = shift; die 'usage: mode(default, [COND => MODE, ...])' if @_ % 2; my $got_mode; my ($cond, $mode); while (@_) { ($cond, $mode, @_) = @_; next if not $cond; die "cannot both $got_mode and $mode\n" if defined $got_mode; $got_mode = $mode; } $got_mode = $default if not defined $got_mode; return $got_mode; } 1; ��������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/ch_search/�������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0015514�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/ch_search/test.conf����������������������������������������������������������������0000775�0000000�0000000�00000005401�15000742332�0017345�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������channel 13thstreet #13TH STREET channel 3plus #3+ channel 3sat #3sat #channel alpha #ALPHA #channel animalplanet #ANIMAL PLANET #channel ard #ARD #channel arte #Arte #channel atv #ATV #channel bbc #BBC #channel bbcent #BBC Ent #channel belgien1 #Belgien 1 #channel bfs #BR #channel bibeltv #Bibel TV #channel bloombergtv #BloombergTV #channel classica #CLASSICA #channel cnbc #CNBC #channel cnn #CNN #channel dasvierte #DAS VIERTE #channel discoverychannel #DISCOVERY #channel disneychannel #DISNEY CHANNEL #channel disneycinemagic #DISNEY CINEMAGIC #channel disneyxd #DISNEY XD #channel dr1 #DR1 #channel dr2 #DR2 #channel dwtv #DW-tv #channel einsextra #EinsExtra #channel einsfestival #EinsFestival #channel euronews #EURONEWS #channel eurosp #Eurosport #channel eurosporthd #EUROSPORTHD #channel extremesportschannel #Extreme Sports #channel finelivingnetwork #Fine Living #channel fr2 #France 2 #channel fr4 #France 4 #channel fr5 #France 5 #channel france #France 3 #channel goldstartv #GOLDSTAR TV #channel gotv #gotv #channel heimatkanal #HEIMATKANAL #channel hf #HF #channel hh1 #HH1 #channel historyhd #HISTORY HD #channel junior #JUNIOR #channel k1 #Kabel 1 #channel k1ch #K1CH #channel kka #Kinderkanal #channel m6 #M6 #channel mdr #MDR #channel motorvision #MOTORVISION #channel mtv #MTV #channel mtvch #MTVCH #channel n24 #N24 #channel n3 #NDR #channel neunlive #NEUN LIVE #channel nickch #NICK/VIVA #channel nl1 #NL1 #channel nl2 #NL2 #channel nl3 #NL3 #channel ntv #NTV #channel orf1 #ORF 1 #channel orf2 #ORF 2 #channel phoenix #PHOENIX #channel planet #PLANET #channel pro7 #Pro 7 #channel pro7ch #Pro 7 CH #channel qvc #QVC #channel rai1 #Rai Uno #channel rai2 #RAI 2 #channel rai3 #Rai Tre #channel rbb #rbb #channel rnfplus #RNFplus #channel rtl #RTL #channel rtl2 #RTL 2 #channel rtl9 #RTL 9 #channel rtlch #RTLCH #channel rtls #S RTL DE #channel sat1ch #Sat 1 CH #channel scifi #SCI FI #channel sf1 #SF 1 #channel sf2 #SF 2 #channel sfinfo #SF Info #channel spiegelgeschichte #GESCHICHTE #channel sport1 #Sport1 #channel spreekanal #Spreekanal #channel srtlch #Super RTL #channel ssf #SSF #channel startv #Star TV #channel swr #SWR #channel tccine #Teleclub Cinema #channel tele5 #Tele 5 #channel teleba #Telebasel #channel telebe #Tele Brn #channel telem1 #Tele M1 #channel teletell #Tele 1 #channel teleto #Tele Top #channel telezu #Tele Zri #channel tf1 #TF 1 #channel tsi1 #RSI LA1 #channel tsi2 #RSI LA2 #channel tsr1 #TSR 1 #channel tsr2 #TSR 2 #channel tv5 #TV5 #channel tvfranken #TV Franken #channel tvslovenija1 #TV Slovenija 1 #channel tvslovenija2 #TV Slovenija 2 #channel tw1 #TW1 #channel viva1 #VIVA1 #channel vox #Vox #channel voxch #VOXCH #channel wdr #WDR #channel zdf #ZDF #channel zdf%20tivi #ZDF TIVI #channel zdfinfo #ZDF Info #channel zdftheaterkanal #ZDF Theaterkanal ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/ch_search/tv_grab_ch_search.PL�����������������������������������������������������0000775�0000000�0000000�00000001320�15000742332�0021373�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Generate tv_grab_ch_search from tv_grab_ch_search.in. This is to set the path to # the files in /usr/local/share/xmltv or wherever. # # The second argument is the share directory for the final # installation. use IO::File; my $out = shift @ARGV; die "no output file given" if not defined $out; my $share_dir = shift @ARGV; die "no final share/ location given" if not defined $share_dir; my $in = 'grab/ch_search/tv_grab_ch_search.in'; my $in_fh = new IO::File "< $in" or die "cannot read $in: $!"; my $out_fh = new IO::File "> $out" or die "cannot write to $out: $!"; my $seen = 0; while (<$in_fh>) { print $out_fh $_; } close $out_fh or die "cannot close $out: $!"; close $in_fh or die "cannot close $in: $!"; ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/ch_search/tv_grab_ch_search.in�����������������������������������������������������0000775�0000000�0000000�00000045741�15000742332�0021505�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl =pod =head1 NAME tv_grab_ch_search - Grab TV listings for Switzerland (from tv.search.ch webpage). =head1 SYNOPSIS tv_grab_ch_search --help tv_grab_ch_search [--config-file FILE] --configure [--gui OPTION] tv_grab_ch_search [--config-file FILE] [--output FILE] [--quiet] [--days N] [--offset N] tv_grab_ch_search --list-channels tv_grab_ch_search --capabilities tv_grab_ch_search --version =head1 DESCRIPTION Output TV listings for several channels available in Switzerland and (partly) central Europe. The data comes from tv.search.ch. The grabber relies on parsing HTML so it might stop working at any time. First run B<tv_grab_ch_search --configure> to choose, which channels you want to download. Then running B<tv_grab_ch_search> with no arguments will output listings in XML format to standard output. B<--configure> Ask for each available channel whether to download and write the configuration file. B<--config-file FILE> Set the name of the configuration file, the default is B<~/.xmltv/tv_grab_ch_search.conf>. This is the file written by B<--configure> and read when grabbing. B<--gui OPTION> Use this option to enable a graphical interface to be used. OPTION may be 'Tk', or left blank for the best available choice. Additional allowed values of OPTION are 'Term' for normal terminal output (default) and 'TermNoProgressBar' to disable the use of Term::ProgressBar. B<--output FILE> Write to FILE rather than standard output. B<--days N> Grab N days. The default is fourteen. B<--offset N> Start N days in the future. The default is to start from now on (= zero). B<--quiet> Suppress the progress messages normally written to standard error. B<--list-channels> Write output giving <channel> elements for every channel available (ignoring the config file), but no programmes. B<--capabilities> Show which capabilities the grabber supports. For more information, see L<http://wiki.xmltv.org/index.php/XmltvCapabilities> B<--version> Show the version of the grabber. B<--help> print a help message and exit. =head1 SEE ALSO L<xmltv(5)>. =head1 AUTHORS Daniel Bittel <betlit@gmx.net>. Inspired by tv_grab_ch by Stefan Siegl. Patric Mueller <bhaak@gmx.net>. Markus Keller <markus.kell.r@gmail.com>. =head1 BUGS If you happen to find a bug, you're requested to send a mail to one of the XMLTV mailing lists, see webpages at http://sourceforge.net/projects/xmltv/. =cut use warnings; use strict; use Encode; use DateTime; use LWP::Simple; use HTTP::Cookies; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use XMLTV::Capabilities qw/baseline manualconfig cache/; use XMLTV::Description 'Switzerland (tv.search.ch)'; use XMLTV::Supplement qw/GetSupplement/; use Getopt::Long; use HTML::TreeBuilder; use HTML::Entities; use URI::Escape; use URI::URL; use XMLTV::Ask; use XMLTV::ProgressBar; use XMLTV::DST; use XMLTV::Config_file; use XMLTV::Mode; use XMLTV::Get_nice; use XMLTV::Memoize; use XMLTV::Usage<<END $0: get Swiss television listings from tv.search.ch in XMLTV format To configure: $0 --configure [--config-file FILE] [--gui OPTION] To grab data: $0 [--config-file FILE] [--output FILE] [--quiet] [--days N] [--offset N] Channel List: $0 --list-channels To show capabilities: $0 --capabilities To show version: $0 --version END ; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } ## our own prototypes first ... sub get_channels(); sub channel_id($); sub get_page($); ## attributes of xmltv root element my $head = { 'source-data-url' => 'https://search.ch/tv/channels', 'source-info-url' => 'https://search.ch/tv', 'generator-info-name' => 'XMLTV', 'generator-info-url' => 'http://xmltv.org/', }; ## the timezone search.ch lives in is, CET/CEST my constant $TZ = "+0100"; my constant $lang = "de"; ## Parse argv now. First do undocumented --cache option. XMLTV::Memoize::check_argv('XMLTV::Get_nice::get_nice_aux'); my $opt_configure; my $opt_config_file; my $opt_gui; my $opt_output; my $opt_days = 14; my $opt_offset = 0; my $opt_quiet = 0; my $opt_slow = 0; my $opt_list_channels; my $opt_help; GetOptions( 'configure' => \$opt_configure, 'config-file=s' => \$opt_config_file, 'gui:s' => \$opt_gui, 'output=s' => \$opt_output, 'days=i' => \$opt_days, 'offset=i' => \$opt_offset, 'quiet' => \$opt_quiet, 'slow' => \$opt_slow, 'list-channels' => \$opt_list_channels, 'help' => \$opt_help, ) or usage(0); usage(1) if $opt_help; XMLTV::Ask::init($opt_gui); ## make sure offset+days arguments are within range die "neither offset nor days may be negative" if($opt_offset < 0 || $opt_days < 0); ## calculate global start/stop times ... my $grab_start = DateTime->now(time_zone => 'Europe/Zurich')->add( days => $opt_offset ); my $grab_stop = DateTime->now(time_zone => 'Europe/Zurich')->add ( days => $opt_offset + $opt_days ); my $mode = XMLTV::Mode::mode('grab', # default value $opt_configure => 'configure', $opt_list_channels => 'list-channels', ); ## initialize config file support my $config_file = XMLTV::Config_file::filename($opt_config_file, 'tv_grab_ch_search', $opt_quiet); my @config_lines; if($mode eq 'configure') { XMLTV::Config_file::check_no_overwrite($config_file); } elsif($mode eq 'grab' || $mode eq 'list-channels') { @config_lines = XMLTV::Config_file::read_lines($config_file); } else { die("never heard of XMLTV mode $mode, sorry :-(") } ## initialize user agent, so that cookie jar can be filled and passed on my $ua = LWP::UserAgent->new(keep_alive => 300); my $cookies = HTTP::Cookies->new(); $ua->cookie_jar($cookies); $ua->agent("xmltv/$XMLTV::VERSION"); $ua->env_proxy; ## hey, we can't live without channel data, so let's get those now! my $bar = new XMLTV::ProgressBar( 'getting list of channels', 1 ) if not $opt_quiet; my ($secret, %channels) = get_channels(); $bar->update() if not $opt_quiet; $bar->finish() if not $opt_quiet; my @requests; ## read our configuration file now my $line = 1; foreach(@config_lines) { $line ++; next unless defined; if (/^channel:?\s+(\S+)/) { warn("\nConfigured channel $1 not available anymore. \nPlease reconfigure tv_grab_ch_search.\n"), next unless(defined($channels{$1})); push @requests, $1; } else { warn "$config_file:$line: bad line\n"; } } ## if we're requested to do so, write out a new config file ... if ($mode eq 'configure') { open(CONFIG, ">$config_file") or die("cannot write to $config_file, due to: $!"); ## now let's annoy the user, sorry, I meant ask .. my @chs = sort keys %channels; my @names = map { $channels{$_} } @chs; my @qs = map { "add channel $_?" } @names; my @want = ask_many_boolean(1, @qs); foreach (@chs) { my $w = shift @want; my $chname = shift @names; warn("cannot read input, stopping to ask questions ..."), last if not defined $w; print CONFIG '#' if not $w; #- comment line out if user answer 'no' # shall we store the display name in the config file? # leave it in, since it probably makes it a lot easier for the # user to choose which channel to comment/uncommet - when manually # viing the config file -- are there people who do that? print CONFIG "channel $_ #$chname\n"; } close CONFIG or warn "unable to nicely close the config file: $!"; say("Finished configuration."); exit(); } ## well, we don't have to write a config file, so, probably it's some xml stuff :) ## if not, let's go dying ... die unless($mode eq 'grab' or $mode eq 'list-channels'); my %writer_args; if (defined $opt_output) { my $handle = new IO::File(">$opt_output"); die "cannot write to output file, $opt_output: $!" unless (defined $handle); $writer_args{'OUTPUT'} = $handle; } $writer_args{'encoding'} = 'utf-8'; if( defined( $opt_days )) { $writer_args{offset} = $opt_offset; $writer_args{days} = $opt_days; $writer_args{cutoff} = "000000"; } ## create our writer object my $writer = new XMLTV::Writer(%writer_args); $writer->start($head); if ($mode eq 'list-channels') { foreach (keys %channels) { my %channel = ('id' => channel_id($_), 'display-name' => [[$channels{$_}, $lang]]); $writer->write_channel(\%channel); } $writer->end(); exit(); } ## there's only one thing, why we might exist: write out tvdata! die unless ($mode eq 'grab'); die "No channels specified, run me with --configure flag\n" unless(scalar(@requests)); ## write out <channel> tags my $paramstr ="&secret=".$secret; foreach(@requests) { my $id = channel_id($_); my %channel = ('id' => $id, 'display-name' => [[$channels{$_}, $lang]]); $writer->write_channel(\%channel); $paramstr = $paramstr."&channels[]=".$_; } ## the page doesn't specify the year when the programmes begin or end, thus ## we need to guess, store current year and month globally as needed for every ## programme ... my $cur_year = DateTime->now()->year(); my $cur_month = DateTime->now()->month(); my $url=$head->{q(source-data-url)}; my $req = HTTP::Request->new(POST => $url); $req->content_type('application/x-www-form-urlencoded'); $req->content(substr ( $paramstr, 1)); # store the selected channels $ua->request($req); ## write out <programme> tags grab_channels(); ## hey, looks like we've finished ... $writer->end(); ## channel_id($s) :: turn site channel id into an xmltv id sub channel_id($) { my $s = shift; $s =~ s|^tv_||; return "$s.search.ch" } sub parse_page { my ($tb, $start_parse_date) = @_; foreach my $tv_channel ( $tb->look_down('class' => 'sl-card tv-index-channel') ) { my $channel_id = substr($tv_channel->attr('id'), 3); # tv-sf1 -> sf1 if ( defined($channel_id) ) { foreach my $tv_show ( $tv_channel ->look_down('class', qr/(^| )tv-tooltip( |$)/) ) { my %show; $show{channel} = channel_id($channel_id); my $tmp = $tv_show->look_down('_tag', 'a'); next unless defined($tmp); my %params = URI::URL->new($tmp->attr('href'))->query_form(); my $start_date = $params{'start'}; my $end_date = $params{'end'}; next unless defined($start_date); my $show_start = DateTime->new ( year => substr($start_date, 0, 4) ,month => substr($start_date, 5, 2) ,day => substr($start_date, 8, 2) ,hour => substr($start_date, 11, 2) ,minute => substr($start_date, 14, 2) ,second => substr($start_date, 17, 2) ,time_zone => 'Europe/Zurich'); $show{start} = $show_start->strftime( "%Y%m%d%H%M%S %z" ); # skip shows starting before the start date to prevent duplicates next if $show_start < $start_parse_date; $show{stop} = DateTime->new ( year => substr($end_date, 0, 4) ,month => substr($end_date, 5, 2) ,day => substr($end_date, 8, 2) ,hour => substr($end_date, 11, 2) ,minute => substr($end_date, 14, 2) ,second => substr($end_date, 17, 2) ,time_zone => 'Europe/Zurich' )->strftime( "%Y%m%d%H%M%S %z" ); my $title_tag = $tv_show->look_down('_tag' => 'h2'); $title_tag->objectify_text(); my $title = $title_tag->look_down('_tag', '~text')->attr('text'); $show{'title'} = [[$title, $lang]]; my $sub_title = $tv_show->look_down('_tag' => 'h3'); $show{'sub-title'} = [[$sub_title->as_text(), $lang]] if($sub_title); # Note: The layout is using dl lists for displaying this data # and only the dt tag is marked with meaningful classes. That's # why $variable->right()-as_text() is employed here to get the # content of the unmarked dd tag. # Beschreibung foreach my $description ($tv_show->look_down('class' => 'tv-detail-description')) { $show{desc} = [[ $description->right()->as_text(), $lang ]] } # Produktionsjahr foreach my $year ($tv_show->look_down('class' => 'tv-detail-year tv-detail-short')) { $show{date} = $year->right()->as_text(); } # Kategorie foreach my $category ($tv_show->look_down('class' => 'tv-detail-catname tv-detail-short')) { my $s = $category->right()->as_text(); my @categories = split(m/\s*[\/]\s*/, $s); foreach (@categories) { push @{$show{category}}, [$_, $lang ] if ($_) } } # Produktionsinfos foreach my $category ($tv_show->look_down('class' => 'tv-detail-production tv-detail-short')) { my $s = $category->right()->as_text(); $s=~ s/\(.*//; my @categories = split(m/\s*[\/,]\s*/, $s); foreach my $category (@categories) { if ($category) { my $is_defined = 0; foreach ( @{$show{category}} ) { if ("${$_}[0]" eq "$category" ) { $is_defined = 1; last; } } push @{$show{category}}, [$category, $lang ] if ($is_defined == 0); } } } # Produktionsland foreach my $country ($tv_show->look_down('class' => 'tv-detail-country tv-detail-short')) { my @countries = split(m/\s*[\/,]\s*/, $country->right()->as_text()); foreach (@countries) { push @{$show{country}}, [$_, $lang ]; } } # Cast foreach my $cast ($tv_show->look_down('class' => 'tv-detail-cast')) { my $s = $cast->right()->as_text(); $s=~ s/\(.*//; my @actors = split(m/\s*,\s*/, $s); $show{credits}{actor} = \@actors; } # Regisseur foreach my $directors ($tv_show->look_down('class' => 'tv-detail-director tv-detail-short')) { my @directors = split(m/\s*,\s*/, $directors->right()->as_text()); $show{credits}{director} = \@directors; } # Drehbuch foreach my $writers ($tv_show->look_down('class' => 'tv-detail-writer tv-detail-short')) { my @writers = split(m/\s*,\s*/, $writers->right()->as_text()); $show{credits}{writer} = \@writers; } # Wiederholung foreach my $previously_shown ($tv_show->look_down('class' => 'tv-detail-repetition')) { $show{'previously-shown'} = {} } # Episode foreach my $episode ($tv_show->look_down('class' => 'tv-detail-episode tv-detail-short')) { $show{'episode-num'} = [[ $episode->right()->as_text(), 'onscreen' ]] } # Untertitel fr Gehrlose foreach my $deaf ($tv_show->look_down('_tag' => 'img', 'title' => encode("utf-8", "Untertitel fr Gehrlose"))) { $show{subtitles} = [{ type => 'teletext' }]; } # Zweikanalton foreach my $bilingual ($tv_show->look_down('_tag' => 'img', 'title' => 'Zweikanalton')) { $show{audio}{stereo} = 'bilingual'; } # 16:9 foreach my $aspect ($tv_show->look_down('_tag' => 'img', 'title' => '16:9')) { $show{video}{aspect} = '16:9'; } $writer->write_programme(\%show); } } } } sub grab_channels { my $grabDate = $grab_start; my $url = $head->{q(source-info-url)}; $bar = new XMLTV::ProgressBar('grabbing channels ', (6*$opt_days)) if not $opt_quiet; grab_channel_loop: for (my $count = 0; $count < 6; $count++) { my $tb = HTML::TreeBuilder->new(); my $loop_date = $grabDate->year() . '-' . substr("0".$grabDate->month(),-2) . '-' . substr("0".$grabDate->day(),-2); my $req = HTTP::Request->new(GET => "$url?time=$loop_date+" . 4*$count .".00"); $req->header('Accept' => 'text/html'); $tb->ignore_unknown(0); # otherwise, html5 tags like section are stripped out $tb->parse(($ua->request($req))->content) or die "cannot parse content of http://search.ch/tv/?time=$loop_date+" . 4*$count .".00"; $tb->eof; parse_page($tb, $grabDate->clone()->truncate("to" => "hour")->set_hour(4*$count)); $tb->delete(); update $bar if not $opt_quiet; } $grabDate = $grabDate->add ( days => 1 ); if( DateTime->compare ( $grab_stop, $grabDate ) > 0) { goto grab_channel_loop; } $bar->finish() unless($opt_quiet); } ## get channel listing sub get_channels() { my %channels; my $url=$head->{q(source-data-url)}; my $tb=new HTML::TreeBuilder(); # For some reason, we need to fetch this page twice. Probably a bug in search.ch. # If you open https://search.ch/tv/channels as first page in private mode in Firefox, the first click on the Save button doesn't work either. $ua->get($url); $tb->parse($ua->get($url)->content) or die "cannot parse content of $url"; $tb->eof; my $secret = ($tb->look_down('name', 'secret'))->attr('value'); ## getting the channels directly selectable foreach($tb->look_down('_tag' => 'label')) { my $id = ($_->look_down('_tag' => 'input'))->id; # tv-channel-sf1 next unless(substr($id, 0, 10) eq "tv-channel"); my $channel_name = $_->as_text(); $channels{uri_escape(substr($id, 11))} = $channel_name; } $tb->delete; return ($secret, %channels); } ## get_page($url) :: try to download $url via http://, look for closing </body> tag or die sub get_page($) { my $url = shift; my $retry = 0; local $SIG{__DIE__} = sub { die "\n$url: $_[0]" }; while($retry < 2) { my $got = eval { get_nice($url . ($retry ? "&retry=$retry" : "")); }; $retry ++; next if($@); # unable to download, doesn't look too good for us. return $got; } die "cannot grab webpage $url (tried $retry times). giving up. sorry"; } �������������������������������xmltv-1.4.0/grab/combiner/��������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0015373�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/combiner/test.conf�����������������������������������������������������������������0000664�0000000�0000000�00000000216�15000742332�0017220�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������grabber=../blib/script/tv_grab_ch_search;channel 13thstreet grabber=../blib/script/tv_grab_uk_freeview;format=number&nregion=64336&nchannel=1 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/combiner/tv_grab_combiner����������������������������������������������������������0000775�0000000�0000000�00000015251�15000742332�0020627�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w =pod =head1 NAME tv_grab_combiner - Grab listings by combining data from several grabbers. =head1 SYNOPSIS tv_grab_combiner --help tv_grab_combiner --configure [--config-file FILE] tv_grab_combiner [--config-file FILE] [--days N] [--offset N] [--output FILE] [--quiet] =head1 DESCRIPTION Output TV and listings in XMLTV format by combining data from several other grabbers. First you must run B<tv_grab_combiner --configure> to choose which grabbers you want to grab data with and how these grabbers should be configured. Then you can run B<tv_grab_combiner> with the --days and --offset options to grab data. Omitting these options will use the default values for these parameters for each grabber. Since these defaults differs between grabbers, you might end up with data for different periods of time for different channels. =head1 OPTIONS B<--configure> Prompt for which grabbers to use, how these grabbers shall be configured and write the configuration file. B<--config-file FILE> Set the name of the configuration file, the default is B<~/.xmltv/tv_grab_combiner.conf>. This is the file written by B<--configure> and read when grabbing. B<--output FILE> When grabbing, write output to FILE rather than standard output. B<--days N> When grabbing, grab N days rather than 5. B<--offset N> Start grabbing at today + N days. N may be negative. B<--quiet> Suppress the progress-bar normally shown on standard error. B<--version> Show the version of the grabber. B<--help> Print a help message and exit. =head1 ERROR HANDLING If any of the called grabbers exit with an error, tv_grab_combiner will exit with a status code of 1 to indicate that the data is incomplete. If any grabber produces output that is not well-formed xml, the output from that grabber will be ignored and tv_grab_combiner will exit with a status code of 1. =head1 ENVIRONMENT VARIABLES The environment variable HOME can be set to change where configuration files are stored. All configuration is stored in $HOME/.xmltv/. =head1 AUTHOR Mattias Holmlund, mattias -at- holmlund -dot- se. =head1 BUGS =cut use strict; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use XMLTV::Capabilities qw/baseline manualconfig/; use XMLTV::Description "Combine data from several other grabbers"; use XMLTV::Usage << "END"; To configure: $0 --configure [--config-file FILE] To grab listings: $0 [--config-file FILE] [--output FILE] [--quiet] END use XMLTV::Ask; use XMLTV::Config_file; use XML::LibXML; use Getopt::Long; use File::Temp qw/tempfile/; my $opt = { help => 0, configure => 0, 'config-file' => XMLTV::Config_file::filename(undef, 'tv_grab_combiner', 1 ), days => undef, offset => undef, quiet => 0, output => undef, }; GetOptions( $opt, qw/help configure config-file=s days=s offset=s quiet output=s/ ) or usage(); if( $opt->{configure} ) { configure(); exit; } if( grab_data() > 0 ) { exit 1; } else { exit 0; } sub grab_data { my @lines = XMLTV::Config_file::read_lines( $opt->{'config-file'} ); my $parser = XML::LibXML->new(); my $result; my $errors=0; my $grabber; foreach my $line (@lines) { next if not defined $line; my( $key, $value ) = split( '=', $line, 2 ); die "Unknown key $key in $opt->{'config-file'}" unless $key eq 'grabber'; my( $grabber, $config ) = split( /;/, $value, 2 ); my( $exitcode, $data ) = run_grabber( $grabber, $config ); if( $exitcode ) { print STDERR "$grabber exited with an error-code.\n"; $errors++; } my $t; eval { $t = $parser->parse_string( $data ); }; if( not defined $t ) { print STDERR "$grabber returned invalid data. Ignoring.\n"; $errors++; next; } if( defined $result ) { concatenate( $result, $t ); } else { $result = $t; } } if( defined $result ) { if( defined( $opt->{output} ) ) { $result->toFile( $opt->{output}, 1 ); } else { $result->toFH( *STDOUT, 1 ); } } return $errors; } sub run_grabber { my( $grabber, $config ) = @_; my( $fh, $filename ) = tempfile(UNLINK => 1); write_config( $config, $fh ); close($fh); my $options = ""; $options .= " --quiet" if $opt->{quiet}; $options .= " --days $opt->{days}" if defined $opt->{days}; $options .= " --offset $opt->{offset}" if defined $opt->{offset}; print STDERR "Running $grabber\n" unless $opt->{quiet}; my $result = qx/$grabber $options --config-file $filename/; return ($? >> 8, $result ); } sub configure_grabber { my( $grabber ) = @_; print "Configuring $grabber\n"; my( $fh, $filename ) = tempfile(UNLINK => 1); system( $grabber, '--configure', '--config-file', $filename ); return read_config( $filename ); } # Read a config-file from disk and encode it into a one-line string. sub read_config { my( $filename ) = @_; my $result = slurp( $filename ) or die "Failed to read from $filename"; $result =~ s/&/&a/g; $result =~ s/\n/&n/g; $result =~ s/#/&c/g; return $result; } sub write_config { my( $config, $fh ) = @_; $config =~ s/&c/#/g; $config =~ s/&n/\n/g; $config =~ s/&a/&/g; print $fh $config; } sub configure { XMLTV::Config_file::check_no_overwrite( $opt->{'config-file'} ); print "Looking for grabbers...\n"; my @result = qx/tv_find_grabbers baseline manualconfig/; my %grabbers; foreach (reverse @result) { chomp; chomp; my($g, $t) = split( /\|/, $_ ); $grabbers{$t}=$g unless $g=~/tv_grab_combiner/; } open( CONF, "> " . $opt->{'config-file'}); while( 1 ) { my $t = ask_choice( "Select a grabber:", "Done", "Done", sort( keys %grabbers) ); last if $t eq "Done"; my $g = $grabbers{$t}; my $config = configure_grabber( $g ); print CONF "grabber=$g;$config\n"; } close( CONF ); } # Takes two XML::LibXML representations of XMLTV files as parameters # and merges the second tree into the first. sub concatenate { my( $t1, $t2 ) = @_; $t1->setEncoding('UTF-8'); my $root = $t1->findnodes( '/tv' )->[0]; my $last_chan = $root->findnodes( 'channel[last()]' )->[0] or die "Failed to find any channel entries"; foreach my $chan ( $t2->findnodes( '//channel' ) ) { $root->insertAfter( $chan, $last_chan ); $last_chan = $chan; } my $last_prog; $last_prog = $root->findnodes( 'programme[last()]' )->[0] or $last_prog = $last_chan; foreach my $prog ( $t2->findnodes( '//programme' ) ) { $root->insertAfter( $prog, $last_prog ); $last_prog = $prog; } } sub slurp { local( $/, @ARGV ) = ( wantarray ? $/ : undef, @_ ); return <ARGV>; } �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/��������������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0014173�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/fi/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0014571�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/fi/common.pm��������������������������������������������������������������������0000664�0000000�0000000�00000021336�15000742332�0016424�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- mode: perl; coding: utf-8 -*- ########################################### # # tv_grab_fi: common code # ############################################################################### # # Setup # # INSERT FROM HERE ############################################################ package fi::common; use strict; use warnings; use base qw(Exporter); our @EXPORT = qw(message debug fetchRaw fetchTree fetchJSON cloneUserAgentHeaders restoreUserAgentHeaders timeToEpoch fullTimeToEpoch); our @EXPORT_OK = qw(setQuiet setDebug setTimeZone); our %EXPORT_TAGS = ( main => [qw(message debug setQuiet setDebug setTimeZone)], ); # Perl core modules use Carp; use Encode qw(decode); use Time::Local qw(timelocal); # Other modules use HTML::TreeBuilder; use JSON qw(); use XMLTV::Get_nice; # # Work around <meta>-in-body bug in HTML::TreeBuilder, See # # https://rt.cpan.org/Public/Bug/Display.html?id=76051 # # Example: # # <html> # <head> # </head> # <body> # <div> # <div> # <meta itemprop="test" content="test"> # <div> # </div> # </div> # </div> # </body> # </html> # # is incorrectly parsed as ($tree->dump() output): # # html # head # meta # body # div # div # div <--- incorrect level for innermost <div> # # Enable <meta> as valid body element $HTML::Tagset::isBodyElement{meta}++; $HTML::Tagset::isHeadOrBodyElement{meta}++; # Normal message, disabled with --quiet { my $quiet = 0; sub message(@) { print STDERR "@_\n" unless $quiet } sub setQuiet($) { ($quiet) = @_ } } # Debug message, enabled with --debug { my $debug = 0; sub debug($@) { my $level = shift; print STDERR "@_\n" unless $debug < $level; } sub setDebug($) { if (($debug) = @_) { # Debug messages may contain Unicode binmode(STDERR, ":encoding(utf-8)"); debug(1, "Debug level set to $debug."); } } } # Fetch URL as UTF-8 encoded string sub fetchRaw($;$$) { my($url, $encoding, $nofail) = @_; debug(2, "Fetching URL '$url'"); my $content; my $retries = 5; # this seems to be enough? RETRY: while (1) { eval { local $SIG{ALRM} = sub { die "Timeout" }; # Default TCP timeouts are too long. If we don't get a response # within 20 seconds, then that's usually an indication that # something is really wrong on the server side. alarm(20); $content = get_nice($url); alarm(0); }; unless ($@) { # Everything is OK # NOTE: utf-8 means "strict UTF-8 standard encoding" $content = decode($encoding || "utf-8", $content); last RETRY; } elsif (($@ =~ /error: 500 Timeout/) && $retries--) { # Let's try this one more time carp "fetchRaw(): timeout. Retrying..."; } elsif ($nofail) { # Caller requested not to fail $content = ""; last RETRY; } else { # Fail on everything else croak "fetchRaw(): $@"; } } debug(5, $content); return($content); } # Fetch URL as parsed HTML::TreeBuilder sub fetchTree($;$$$) { my($url, $encoding, $nofail, $unknown) = @_; my $content = fetchRaw($url, $encoding, $nofail); my $tree = HTML::TreeBuilder->new(); $tree->ignore_unknown(!$unknown); local $SIG{__WARN__} = sub { carp("fetchTree(): $_[0]") }; $tree->parse($content) or croak("fetchTree() parse failure for '$url'"); $tree->eof; return($tree); } # Fetch URL as parsed JSON object and return contents of the given key sub fetchJSON($$) { my($url, $key) = @_; # Fetch raw JSON text my $text = fetchRaw($url); if ($text) { my $decoded = JSON->new->decode($text); if (ref($decoded) eq "HASH") { # debug(5, JSON->new->pretty->encode($decoded)); return $decoded->{$key}; } } croak("fetchJSON() couldn't fetch from '$url'"); } # get_nice() user agent default headers handling sub cloneUserAgentHeaders() { # fetch current HTTP::Headers object my $headers = $XMLTV::Get_nice::ua->default_headers(); # clone it and use the clone in the user agent my $clone = $headers->clone(); $XMLTV::Get_nice::ua->default_headers($clone); return($headers, $clone); } sub restoreUserAgentHeaders($) { my($headers) = @_; $XMLTV::Get_nice::ua->default_headers($headers); } # # Time zone handling # # After setting up the day list we switch to a fixed time zone in order to # interpret the program start times from finnish sources. In this case we of # course use # # Europe/Helsinki # # which can mean # # EET = GMT+02:00 (East European Time) # EEST = GMT+03:00 (East European Summer Time) # # depending on the day of the year. By using a fixed time zone this grabber # will always be able to correctly calculate the program start time in UTC, # no matter what the time zone of the local system is. # # Test program: # ---------------------- CUT HERE --------------------------------------------- # use 5.008009; # use strict; # use warnings; # use Time::Local; # use POSIX qw(strftime); # # # DST test days for Europe 2010 # my @testdays = ( # # hour, minute, mday, month # [ 2, 00, 1, 1], # [ 2, 59, 28, 3], # [ 3, 00, 28, 3], # [ 3, 01, 28, 3], # [ 3, 00, 1, 7], # [ 3, 59, 31, 10], # [ 4, 00, 31, 10], # [ 4, 01, 31, 10], # [ 2, 00, 1, 12], # ); # # print strftime("System time zone is: %Z\n", localtime(time())); # if (@ARGV) { # $ENV{TZ} = "Europe/Helsinki"; # } # print strftime("Script time zone is: %Z\n", localtime(time())); # # foreach my $date (@testdays) { # my $time = timelocal(0, @{$date}[1, 0, 2], $date->[3] - 1, 2010); # print # "$time: ", strftime("%d-%b-%Y %T %z", localtime($time)), # " -> ", strftime("%d-%b-%Y %T +0000", gmtime($time)), "\n"; # } # ---------------------- CUT HERE --------------------------------------------- # # Test runs: # # 1) system on Europe/Helsinki time zone [REFERENCE] # # $ perl test.pl # System time zone is: EET # Script time zone is: EET # 1262304000: 01-Jan-2010 02:00:00 +0200 -> 01-Jan-2010 00:00:00 +0000 # 1269737940: 28-Mar-2010 02:59:00 +0200 -> 28-Mar-2010 00:59:00 +0000 # 1269738000: 28-Mar-2010 04:00:00 +0300 -> 28-Mar-2010 01:00:00 +0000 # 1269738060: 28-Mar-2010 04:01:00 +0300 -> 28-Mar-2010 01:01:00 +0000 # 1277942400: 01-Jul-2010 03:00:00 +0300 -> 01-Jul-2010 00:00:00 +0000 # 1288486740: 31-Oct-2010 03:59:00 +0300 -> 31-Oct-2010 00:59:00 +0000 # 1288490400: 31-Oct-2010 04:00:00 +0200 -> 31-Oct-2010 02:00:00 +0000 # 1288490460: 31-Oct-2010 04:01:00 +0200 -> 31-Oct-2010 02:01:00 +0000 # 1291161600: 01-Dec-2010 02:00:00 +0200 -> 01-Dec-2010 00:00:00 +0000 # # 2) system on America/New_York time zone # # $ TZ="America/New_York" perl test.pl # System time zone is: EST # Script time zone is: EST # 1262329200: 01-Jan-2010 02:00:00 -0500 -> 01-Jan-2010 07:00:00 +0000 # 1269759540: 28-Mar-2010 02:59:00 -0400 -> 28-Mar-2010 06:59:00 +0000 # 1269759600: 28-Mar-2010 03:00:00 -0400 -> 28-Mar-2010 07:00:00 +0000 # 1269759660: 28-Mar-2010 03:01:00 -0400 -> 28-Mar-2010 07:01:00 +0000 # 1277967600: 01-Jul-2010 03:00:00 -0400 -> 01-Jul-2010 07:00:00 +0000 # 1288511940: 31-Oct-2010 03:59:00 -0400 -> 31-Oct-2010 07:59:00 +0000 # 1288512000: 31-Oct-2010 04:00:00 -0400 -> 31-Oct-2010 08:00:00 +0000 # 1288512060: 31-Oct-2010 04:01:00 -0400 -> 31-Oct-2010 08:01:00 +0000 # 1291186800: 01-Dec-2010 02:00:00 -0500 -> 01-Dec-2010 07:00:00 +0000 # # 3) system on America/New_York time zone, script on Europe/Helsinki time zone # [compare to output from (1)] # # $ TZ="America/New_York" perl test.pl switch # System time zone is: EST # Script time zone is: EET # 1262304000: 01-Jan-2010 02:00:00 +0200 -> 01-Jan-2010 00:00:00 +0000 # 1269737940: 28-Mar-2010 02:59:00 +0200 -> 28-Mar-2010 00:59:00 +0000 # 1269738000: 28-Mar-2010 04:00:00 +0300 -> 28-Mar-2010 01:00:00 +0000 # 1269738060: 28-Mar-2010 04:01:00 +0300 -> 28-Mar-2010 01:01:00 +0000 # 1277942400: 01-Jul-2010 03:00:00 +0300 -> 01-Jul-2010 00:00:00 +0000 # 1288486740: 31-Oct-2010 03:59:00 +0300 -> 31-Oct-2010 00:59:00 +0000 # 1288490400: 31-Oct-2010 04:00:00 +0200 -> 31-Oct-2010 02:00:00 +0000 # 1288490460: 31-Oct-2010 04:01:00 +0200 -> 31-Oct-2010 02:01:00 +0000 # 1291161600: 01-Dec-2010 02:00:00 +0200 -> 01-Dec-2010 00:00:00 +0000 # # Setup fixed time zone for program start time interpretation sub setTimeZone() { $ENV{TZ} = "Europe/Helsinki"; } # Take a fi::day (day/month/year) and the program start time (hour/minute) # and convert it to seconds since Epoch in the current time zone sub timeToEpoch($$$) { my($date, $hour, $minute) = @_; return(timelocal(0, $minute, $hour, $date->day(), $date->month() - 1, $date->year())); } # Same thing but without fi::day object sub fullTimeToEpoch($$$$$) { my($year, $month, $day, $hour, $minute) = @_; return(timelocal(0, $minute, $hour, $day, $month - 1, $year)); } # That's all folks 1; ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/fi/day.pm�����������������������������������������������������������������������0000664�0000000�0000000�00000003066�15000742332�0015711�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- mode: perl; coding: utf-8 -*- ########################################### # # tv_grab_fi: day class # ############################################################################### # # Setup # # INSERT FROM HERE ############################################################ package fi::day; use strict; use warnings; use Carp; use Date::Manip qw(DateCalc ParseDate UnixDate); # Overload stringify operation use overload '""' => "ymd"; # Constructor (private) sub _new { my($class, $day, $month, $year) = @_; my $self = { day => $day, month => $month, year => $year, ymd => sprintf("%04d%02d%02d", $year, $month, $day), ymdd => sprintf("%04d-%02d-%02d", $year, $month, $day), dmy => sprintf("%02d.%02d.%04d", $day, $month, $year), }; return(bless($self, $class)); } # instance methods sub day { $_[0]->{day} }; sub dmy { $_[0]->{dmy} }; sub month { $_[0]->{month} }; sub year { $_[0]->{year} }; sub ymd { $_[0]->{ymd} }; sub ymdd { $_[0]->{ymdd} }; # class methods sub generate { my($class, $offset, $days) = @_; # Start one day before offset my $date = DateCalc(ParseDate("today"), ($offset - 1) . " days") or croak("can't calculate start day"); # End one day after offset + days my @dates; for (0..$days + 1) { my($year, $month, $day) = split(':', UnixDate($date, "%Y:%m:%d")); push(@dates, $class->_new(int($day), int($month), int($year))); $date = DateCalc($date, "+1 day") or croak("can't calculate next day"); } return(\@dates); } # That's all folks 1; ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/fi/programme.pm�����������������������������������������������������������������0000664�0000000�0000000�00000031673�15000742332�0017132�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- mode: perl; coding: utf-8 -*- ########################################### # # tv_grab_fi: programme class # ############################################################################### # # Setup # # INSERT FROM HERE ############################################################ package fi::programme; use strict; use warnings; use Carp; use POSIX qw(strftime); use URI::Escape qw(uri_unescape); # Import from internal modules fi::common->import(); sub _trim { return unless defined($_[0]); $_[0] =~ s/^\s+//; $_[0] =~ s/\s+$//; } # Constructor sub new { my($class, $channel, $language, $title, $start, $stop) = @_; _trim($title); croak "${class}::new called without valid title or start" unless defined($channel) && defined($title) && (length($title) > 0) && defined($start); my $self = { channel => $channel, language => $language, title => $title, start => $start, stop => $stop, }; return(bless($self, $class)); } # instance methods sub category { my($self, $category) = @_; _trim($category); $self->{category} = $category if defined($category) && length($category); } sub description { my($self, $description) = @_; _trim($description); $self->{description} = $description if defined($description) && length($description); } sub episode { my($self, $episode, $language) = @_; _trim($episode); if (defined($episode) && length($episode)) { $episode =~ s/\.$//; push(@{ $self->{episode} }, [$episode, $language]); } } sub episode_number { my($self, $episode_number) = @_; # only accept valid, positive integers if (defined($episode_number)) { $episode_number = int($episode_number); if ($episode_number > 0) { $self->{episode_number} = $episode_number; } } } sub episode_total { my($self, $episode_total) = @_; # only accept valid, positive integers if (defined($episode_total)) { $episode_total = int($episode_total); if ($episode_total > 0) { $self->{episode_total} = $episode_total; } } } sub season { my($self, $season) = @_; # only accept valid, positive integers if (defined($season)) { $season = int($season); if ($season > 0) { $self->{season} = $season; } } } sub start { my($self, $start) = @_; $self->{start} = $start if defined($start) && length($start); $start = $self->{start}; croak "${self}::start: object without valid start time" unless defined($start); return($start); } sub stop { my($self, $stop) = @_; $self->{stop} = $stop if defined($stop) && length($stop); $stop = $self->{stop}; croak "${self}::stop: object without valid stop time" unless defined($stop); return($stop); } # read-only sub language { $_[0]->{language} } sub title { $_[0]->{title} } # Convert seconds since Epoch to XMLTV time stamp # # NOTE: We have to generate the time stamp using local time plus time zone as # some XMLTV users, e.g. mythtv in the default configuration, ignore the # XMLTV time zone value. # sub _epoch_to_xmltv_time($) { my($time) = @_; # Unfortunately strftime()'s %z is not portable... # # return(strftime("%Y%m%d%H%M%S %z", localtime($time)); # # ...so we have to roll our own: # my @time = localtime($time); # is_dst return(strftime("%Y%m%d%H%M%S +0", @time) . ($time[8] ? "3": "2") . "00"); } # Configuration data my %option; my %series_description; my %series_title; my @title_map; my $title_strip_parental; # Common regular expressions # ($left, $special, $right) = ($description =~ $match_description) my $match_description = qr/^\s*([^.!?]+[.!?])([.!?]+\s+)?\s*(.*)/; sub dump { my($self, $writer) = @_; my $language = $self->{language}; my $title = $self->{title}; my $category = $self->{category}; my $description = $self->{description}; my $episode = $self->{episode_number}; my $season = $self->{season}; my $subtitle = $self->{episode}; my $total = $self->{episode_total}; # # Programme post-processing # # Parental level removal (catch also the duplicates) $title =~ s/(?:\s+\(\s*(?:S|T|K?7|K?9|K?12|K?16|K?18)\s*\))+\s*$// if $title_strip_parental; # # Title mapping # foreach my $map (@title_map) { if ($map->($title)) { debug(3, "XMLTV title '$self->{title}' mapped to '$title'"); last; } } # # Check 1: object already contains episode # my($left, $special, $right); if (defined($subtitle)) { # nothing to be done } # # Check 2: title contains episode name # # If title contains a colon (:), check to see if the string on the left-hand # side of the colon has been defined as a series in the configuration file. # If it has, assume that the string on the left-hand side of the colon is # the name of the series and the string on the right-hand side is the name # of the episode. # # Example: # # config: series title Prisma # title: Prisma: Totuus tappajadinosauruksista # # This will generate a program with # # title: Prisma # sub-title: Totuus tappajadinosauruksista # elsif ((($left, $right) = ($title =~ /([^:]+):\s*(.*)/)) && (exists $series_title{$left})) { debug(3, "XMLTV series title '$left' episode '$right'"); ($title, $subtitle) = ($left, $right); } # # Check 3: description contains episode name # # Check if the program has a description. If so, also check if the title # of the program has been defined as a series in the configuration. If it # has, assume that the first sentence (i.e. the text before the first # period, question mark or exclamation mark) marks the name of the episode. # # Example: # # config: series description Batman # description: Pingviinin paluu. Amerikkalainen animaatiosarja.... # # This will generate a program with # # title: Batman # sub-title: Pingviinin paluu # description: Amerikkalainen animaatiosarja.... # # Special cases # # text: Pingviinin paluu?. Amerikkalainen animaatiosarja.... # sub-title: Pingviinin paluu? # description: Amerikkalainen animaatiosarja.... # # text: Pingviinin paluu... Amerikkalainen animaatiosarja.... # sub-title: Pingviinin paluu... # description: Amerikkalainen animaatiosarja.... # # text: Pingviinin paluu?!? Amerikkalainen animaatiosarja.... # sub-title: Pingviinin paluu?!? # description: Amerikkalainen animaatiosarja.... # elsif ((defined($description)) && (exists $series_description{$title}) && (($left, $special, $right) = ($description =~ $match_description))) { my($desc_subtitle, $desc_total); # Check for "Kausi <season>, osa <episode>. <maybe sub-title>...." if (my($desc_season, $desc_episode, $remainder) = ($description =~ m/^Kausi\s+(\d+),\s+osa\s+(\d+)\.\s*(.*)$/)) { $season = $desc_season; $episode = $desc_episode; # Repeat the above match on remaining description ($left, $special, $right) = ($remainder =~ $match_description); # Take a guess if we have a episode title in description or not my $words; $words++ while $left =~ /\S+/g; if ($words > 5) { # More than 5 words probably means no episode title undef $left; undef $special; $right = $remainder; } # Check for "Kausi <season>[.,] (Jakso )?<episode>/<# of episodes>. <sub-title>...." } elsif (($desc_season, $desc_episode, $desc_total, $remainder) = ($description =~ m!^Kausi\s+(\d+)[.,]\s+(?:Jakso\s+)?(\d+)(?:/(\d+))?\.\s*(.*)$!)) { $season = $desc_season; $episode = $desc_episode; $total = $desc_total if $desc_total; # Repeat the above match on remaining description ($left, $special, $right) = ($remainder =~ $match_description); # Check for "K<season>, J<episode: <sub-title>...." } elsif (($desc_season, $desc_episode, $remainder) = ($description =~ m/^K(\d+),\s+J(\d+):\s+(.*)$/)) { $season = $desc_season; $episode = $desc_episode; # Repeat the above match on remaining description ($left, $special, $right) = ($remainder =~ $match_description); # Check for "<sub-title>. Kausi <season>, (jakso )?<episode>/<# of episodes>...." } elsif (($desc_subtitle, $desc_season, $desc_episode, $desc_total, $remainder) = ($description =~ m!^(.+)\s+Kausi\s+(\d+),\s+(?:jakso\s+)?(\d+)(?:/(\d+))?\.\s*(.*)$!)) { $left = $desc_subtitle; $season = $desc_season; $episode = $desc_episode; $total = $desc_total if $desc_total; # Remainder is already the final episode description $right = $remainder; undef $special; # Check for "<episode>/<# of episodes>. <sub-title>...." } elsif (($desc_episode, $desc_total, $remainder) = ($description =~ m!^(\d+)/(\d+)\.\s+(.*)$!)) { # default to season 1 $season = 1 unless defined($season); $episode = $desc_episode; $total = $desc_total; # Repeat the above match on remaining description ($left, $special, $right) = ($remainder =~ $match_description); } if (defined($left)) { unless (defined($special)) { # We only remove period from episode title, preserve others $left =~ s/\.$//; } elsif (($left !~ /\.$/) && ($special =~ /^\.\s/)) { # Ignore extraneous period after sentence } else { # Preserve others, e.g. ellipsis $special =~ s/\s+$//; $left .= $special; } debug(3, "XMLTV series title '$title' episode '$left'"); } ($subtitle, $description) = ($left, $right); } # XMLTV programme desciptor (mandatory parts) my %xmltv = ( channel => $self->{channel}, start => _epoch_to_xmltv_time($self->{start}), stop => _epoch_to_xmltv_time($self->{stop}), title => [[$title, $language]], ); debug(3, "XMLTV programme '$xmltv{channel}' '$xmltv{start} -> $xmltv{stop}' '$title'"); # XMLTV programme descriptor (optional parts) if (defined($subtitle)) { $subtitle = [[$subtitle, $language]] unless ref($subtitle); $xmltv{'sub-title'} = $subtitle; debug(3, "XMLTV programme episode ($_->[1]): $_->[0]") foreach (@{ $xmltv{'sub-title'} }); } if (defined($category) && length($category)) { $xmltv{category} = [[$category, $language]]; debug(4, "XMLTV programme category: $category"); } if (defined($description) && length($description)) { $xmltv{desc} = [[$description, $language]]; debug(4, "XMLTV programme description: $description"); } if (defined($season) && defined($episode)) { if (defined($total)) { $xmltv{'episode-num'} = [[ ($season - 1) . '.' . ($episode - 1) . '/' . $total . '.', 'xmltv_ns' ]]; debug(4, "XMLTV programme season/episode: $season/$episode of $total"); } else { $xmltv{'episode-num'} = [[ ($season - 1) . '.' . ($episode - 1) . '.', 'xmltv_ns' ]]; debug(4, "XMLTV programme season/episode: $season/$episode"); } } $writer->write_programme(\%xmltv); } # class methods # Parse config line sub parseConfigLine { my($class, $line) = @_; # Extract words my($command, $keyword, $param) = split(' ', $line, 3); # apply URI unescaping if string contains '%XX' if ($param =~ /%[0-9A-Fa-f]{2}/) { $param = uri_unescape($param); } if ($command eq "option") { # option <source> key=value my($key, $value) = split('=', $param, 2); if ($keyword && $key && $value) { $option{$keyword}->{$key} = $value; } else { # Corrupted option return; } } elsif ($command eq "series") { if ($keyword eq "description") { $series_description{$param}++; } elsif ($keyword eq "title") { $series_title{$param}++; } else { # Unknown series configuration return; } } elsif ($command eq "title") { if (($keyword eq "map") && # Accept "title" and 'title' for each parameter - 2nd may be empty (my(undef, $from, undef, $to) = ($param =~ /^([\'\"])([^\1]+)\1\s+([\'\"])([^\3]*)\3/))) { debug(3, "title mapping from '$from' to '$to'"); $from = qr/^\Q$from\E/; push(@title_map, sub { $_[0] =~ s/$from/$to/ }); } elsif (($keyword eq "strip") && ($param =~ /parental\s+level/)) { debug(3, "stripping parental level from titles"); $title_strip_parental++; } else { # Unknown title configuration return; } } else { # Unknown command return; } return(1); } # Fix overlapping programmes sub fixOverlaps { my($class, $list) = @_; # No need to cleanup empty/one-entry lists return unless defined($list) && (@{ $list } >= 2); my $current = $list->[0]; foreach my $next (@{ $list }[1..$#{ $list }]) { # Does next programme start before current one ends? if ($current->{stop} > $next->{start}) { debug(3, "Fixing overlapping programme '$current->{title}' $current->{stop} -> $next->{start}."); $current->{stop} = $next->{start}; } # Next programme $current = $next; } } # Get option value for source sub getOption($$) { my($source, $key) = @_; # Avoid creating empty entries for non-existing options return $option{$source}->{$key} if exists $option{$source}->{$key}; } # That's all folks 1; ���������������������������������������������������������������������xmltv-1.4.0/grab/fi/fi/programmeStartOnly.pm��������������������������������������������������������0000664�0000000�0000000�00000007501�15000742332�0021003�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- mode: perl; coding: utf-8 -*- ########################################### # # tv_grab_fi: generate programme list using start times only # ############################################################################### # # Setup # # INSERT FROM HERE ############################################################ package fi::programmeStartOnly; use strict; use warnings; use base qw(Exporter); our @EXPORT = qw(startProgrammeList appendProgramme convertProgrammeList); # Import from internal modules fi::common->import(); sub startProgrammeList($$) { my($id, $language) = @_; return({ id => $id, language => $language, programmes => [] }); } sub appendProgramme($$$$) { my($self, $hour, $minute, $title) = @_; # NOTE: start time in minutes from midnight -> must be converted to epoch my $object = fi::programme->new($self->{id}, $self->{language}, $title, $hour * 60 + $minute); push(@{ $self->{programmes} }, $object); return($object); } sub convertProgrammeList($$$$) { my($self, $yesterday, $today, $tomorrow) = @_; my $programmes = $self->{programmes}; my $id = $self->{id}; # No data found -> return empty list to indicate failure return([]) unless @{ $programmes }; # Check for day crossings my @dates = ($today, $tomorrow); if (@{ $programmes } > 1) { my @day_crossings; my $last = @{ $programmes } - 1; for (my $index = 0; $index < $last; $index++) { push(@day_crossings, $index) if ($programmes->[$index]->start() > $programmes->[$index + 1]->start()); } # more than one day crossing? if (@day_crossings > 1) { # Did caller specify yesterday? if (defined $yesterday) { unshift(@dates, $yesterday); } else { # No, assume the entry after first crossing is broken -> drop it splice(@{ $programmes }, $day_crossings[0] + 1, 1); } } } my @objects; my $date = shift(@dates); my $current = shift(@{ $programmes }); my $current_start = $current->start(); my $current_epoch = timeToEpoch($date, int($current_start / 60), $current_start % 60); foreach my $next (@{ $programmes }) { # Start of next program might be on the next day my $next_start = $next->start(); if ($current_start > $next_start) { # # Sanity check: try to detect fake day changes caused by broken data # # Incorrect date change example: # # 07:00 Voittovisa # 07:50 Ostoskanava # 07:20 F1 Ennakkolähetys <-- INCORRECT DAY CHANGE # 07:50 Dino, pikku dinosaurus # 08:15 Superpahisten liiga # # -> 07:50 (= 470) - 07:20 (= 440) = 30 minutes < 2 hours # # Correct date change example # # 22:35 Irene Huss: Tulitanssi # 00:30 Formula 1: Extra # # -> 22:35 (= 1355) - 00:30 (= 30) = 1325 minutes > 2 hours # # I grabbed the 2 hour limit out of thin air... # if ($current_start - $next_start > 2 * 60) { $date = shift(@dates); # Sanity check unless ($date) { message("WARNING: corrupted data for $id on $today: two date changes detected. Ignoring data!"); return([]); } } else { message("WARNING: corrupted data for $id on $today: fake date change detected. Ignoring."); } } my $next_epoch = timeToEpoch($date, int($next_start / 60), $next_start % 60); my $title = $current->title(); debug(3, "Programme $id ($current_epoch -> $next_epoch) $title"); # overwrite start & stop times with epoch: see appendProgramme() $current->start($current_epoch); $current->stop($next_epoch); push(@objects, $current); # Move to next program $current = $next; $current_start = $next_start; $current_epoch = $next_epoch; } return(\@objects); } # That's all folks 1; �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/fi/source/����������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0016071�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/fi/source/iltapulu.pm�����������������������������������������������������������0000664�0000000�0000000�00000011472�15000742332�0020273�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- mode: perl; coding: utf-8 -*- ########################################### # # tv_grab_fi: source specific grabber code for https://www.iltapulu.fi # ############################################################################### # # Setup # # INSERT FROM HERE ############################################################ package fi::source::iltapulu; use strict; use warnings; # # NOTE: this data source was earlier known as http://tv.hs.fi # NOTE: this data source was earlier known as http://tv.tvnyt.fi # BEGIN { our $ENABLED = 1; } use Carp; # Import from internal modules fi::common->import(); fi::programmeStartOnly->import(); # Category mapping our %categories = ( e => "elokuvat", f => "fakta", kf => "kotimainen fiktio", l => "lapsi", nan => undef, # ??? e.g. "Astral TV" u => "uutiset", ur => "urheilu", us => "ulkomaiset sarjat", vm => "viihde", # "ja musiiki"??? ); # Description sub description { 'iltapulu.fi' } # Grab channel list sub channels { my %channels; # Fetch & parse HTML my $root = fetchTree("https://www.iltapulu.fi/kaikki-kanavat", undef, undef, 1); if ($root) { # # Channel list can be found in sections # # <div id="content"> # <div id="programtable" class="programtable-running"> # <section id="channel-1" ...> # <a href="/kanava/yle-tv1"> # <h2 class="channel-logo"> # <img src="/static/img/kanava/yle_tv1.png" alt="YLE TV1 tv-ohjelmat 26.12.2020"> # </h2> # </a> # ... # </section> # ... # </div> # </div> # if (my $table = $root->look_down("id" => "programtable")) { if (my @sections = $table->look_down("_tag" => "section", "id" => qr/^channel-\d+$/)) { foreach my $section (@sections) { if (my $header = $section->look_down("class" => "channel-logo")) { if (my $image = $header->find("img")) { my $name = $image->attr("alt"); $name =~ s/\s+tv-ohjelmat.*$//; if (defined($name) && length($name)) { my($channel_id) = $section->attr("id") =~ /(\d+)$/; $channel_id .= ".iltapulu.fi"; debug(3, "channel '$name' ($channel_id)"); $channels{$channel_id} = "fi $name"; } } } } } } # Done with the HTML tree $root->delete(); } debug(2, "Source iltapulu.fi parsed " . scalar(keys %channels) . " channels"); return(\%channels); } # Grab one day sub grab { my($self, $id, $yesterday, $today, $tomorrow, $offset) = @_; # Get channel number from XMLTV id return unless my($channel) = ($id =~ /^([-\w]+)\.iltapulu\.fi$/); # Fetch & parse HTML my $root = fetchTree("https://www.iltapulu.fi/" . $today->ymdd(), undef, undef, 1); if ($root) { my $opaque = startProgrammeList($id, "fi"); # # Programme data is contained inside a li class="g-<category>" # # <div id="content"> # <div id="programtable" class="programtable-running"> # <section id="channel-1" ...> # <a href="/kanava/yle-tv1"> # <h2 class="channel-logo"> # <img src="/static/img/kanava/yle_tv1.png" alt="YLE TV1 tv-ohjelmat 26.12.2020"> # </h2> # </a> # <ul> # <li class="running g-e"> # <time datetime="2020-12-26T15:20:00+02:00">15.20</time> # <b class="pl"> # <a href="/joulumaa" class="op" ... title="... description ..."> # Joulumaa # </a> # ... # </b> # ... # </li> # ... # </ul> # <ul> # if (my $table = $root->look_down("id" => "programtable")) { if (my $section = $table->look_down("_tag" => "section", "id" => qr/^channel-${channel}/)) { if (my @entries = $section->look_down("_tag" => "li")) { foreach my $entry (@entries) { my $start = $entry->look_down("_tag" => "time"); my $link = $entry->look_down("class" => "op"); if ($start && $link) { if (my($hour, $minute) = $start->as_text() =~ /^(\d{2})[:.](\d{2})$/) { my $title = $link->as_text(); if (length($title)) { my $desc = $link->attr("title"); my($category) = ($entry->attr("class") =~ /g-(\w+)$/); $category = $categories{$category} if $category; debug(3, "List entry ${id} ($hour:$minute) $title"); debug(4, $desc) if $desc; debug(4, $category) if defined $category; my $object = appendProgramme($opaque, $hour, $minute, $title); $object->description($desc); $object->category($category); } } } } } } } # Done with the HTML tree $root->delete(); # Convert list to program objects # # First entry always starts on $yesteday # Last entry always ends on $tomorrow. return(convertProgrammeList($opaque, $yesterday, $today, $tomorrow)); } return; } # That's all folks 1; ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/fi/source/star.pm���������������������������������������������������������������0000664�0000000�0000000�00000013126�15000742332�0017403�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- mode: perl; coding: utf-8 -*- ########################################### # # tv_grab_fi: source specific grabber code for https://www.starchannel.fi # ############################################################################### # # Setup # # INSERT FROM HERE ############################################################ package fi::source::star; use strict; use warnings; # # NOTE: this data source was earlier known as https://www.foxtv.fi # BEGIN { our $ENABLED = 1; } # Import from internal modules fi::common->import(); fi::programmeStartOnly->import(); # Cleanup filter regexes our $cleanup_match = qr!\s*(?:(?:\d+\.\s+)?(?:Kausi|Jakso|Osa)\.?(?:\s+(:?\d+/)?\d+\.\s+)?){1,2}!i; # Description sub description { 'star.fi' } # Grab channel list - only one channel available, no need to fetch anything... sub channels { { 'star.fi' => 'fi STAR Channel' } } # Grab one day sub grab { my($self, $id, $yesterday, $today, $tomorrow, $offset) = @_; # Get channel number from XMLTV id return unless ($id eq "star.fi"); # Fetch & parse HTML (do not ignore HTML5 <section>) # Anything beyond 14 days results in 404 error -> ignore errors my $root = fetchTree("https://www.starchannel.fi/ohjelmaopas/star/$today", undef, 1, 1); if ($root) { # # Each page contains the programmes from previous day to next day from requested day. # Sometimes page doesn't have programmes older than ~4 hours # All program info is contained within a section with class "row day" # Each row day section starts at 06.00 and ends 05.59 next day # # <div id="scheduleContainer"> # <section class="row day" data-magellan-destination="day20160514" ...> # <ul class="... scheduleGrid"> # <li ...> # ... # <h5>15:00</h5> # ... # <h3>Family Guy</h3> # ... # <h4>Maaseudun taikaa, Kausi 12 | Jakso 21</h4> # <p>Kauden Suomen tv-ensiesitys. ...</p> # ... # </li> # ... # </ul> # </section> # ... # </div> # my $opaque = startProgrammeList($id, "fi"); if (my $container = $root->look_down("_tag" => "div", "id" => "scheduleContainer")) { my @programmes_today = $root->look_down("_tag" => "li", "class" => qr/acilia-schedule-event/, "data-datetime-date" => $today->ymdd()); my $first_tomorrow = $root->look_down("_tag" => "li", "class" => qr/acilia-schedule-event/, "data-datetime-date" => $tomorrow->ymdd()); foreach my $programme (@programmes_today) { my $start = $programme->find("h5"); my $title = $programme->find("h3"); if ($start && $title) { if (my($hour, $minute) = $start->as_text() =~ /^(\d{2})[:.](\d{2})$/) { my $desc = $programme->find("p"); my $extra = $programme->find("h4"); $title = $title->as_text(); my($episode_name, $season, $episode_number) = $extra->as_text() =~ /^(.*)?,\s+Kausi\s+(\d+)\s+\S\s+Jakso\s+(\d+)\s*$/ if $extra; # Cleanup some of the most common inconsistencies.... $episode_name =~ s/^$cleanup_match// if defined $episode_name; # If cleanup leaves only numbers to episode name cleanup them # Example if episode name is Jakso 8, cleanup removes Jakso but don't space and number $episode_name =~ s/^(\s+\d{1,2})$// if defined $episode_name; if ($desc) { ($desc = $desc->as_text()) =~ s/^$cleanup_match//; # Title can be first in description too $desc =~ s/^$title\.\s+?//; # Episode title can be first in description too $desc =~ s/^$episode_name(?:\.\s+)?// if defined $episode_name; # Description can be empty undef $desc if $desc eq ''; } # Episode name can be the same as the title undef $episode_name if defined($episode_name) && (($episode_name eq '') || ($episode_name eq $title)); debug(3, "List entry fox ($hour:$minute) $title"); debug(4, $episode_name) if defined $episode_name; debug(4, $desc) if defined $desc; debug(4, sprintf("s%02de%02d", $season, $episode_number)) if (defined($season) && defined($episode_number)); my $object = appendProgramme($opaque, $hour, $minute, $title); $object->description($desc); $object->episode($episode_name, "fi"); $object->season($season); $object->episode_number($episode_number); } } } # Get stop time for last entry in the table: first start if ($first_tomorrow) { my $start = $first_tomorrow->find("h5"); if (my($hour, $minute) = $start->as_text() =~ /^(\d{2})[:.](\d{2})$/) { appendProgramme($opaque, $hour, $minute, "DUMMY"); } } } # Done with the HTML tree $root->delete(); # Convert list to program objects # # First entry always starts on $today -> don't use $yesterday # Last entry always ends on $tomorrow. # # Unfortunately we don't have a stop time for the last entry. We fix this # (see above) by adding the start time of the first entry from tomorrow # as a DUMMY program. return(convertProgrammeList($opaque, undef, $today, $tomorrow)); } return; } # That's all folks 1; ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/fi/source/telkku.pm�������������������������������������������������������������0000664�0000000�0000000�00000013732�15000742332�0017734�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- mode: perl; coding: utf-8 -*- ########################################### # # tv_grab_fi: source specific grabber code for https://www.telkku.com # ############################################################################### # # Setup # # INSERT FROM HERE ############################################################ package fi::source::telkku; use strict; use warnings; use Date::Manip qw(UnixDate); BEGIN { our $ENABLED = 1; } # Import from internal modules fi::common->import(); # Description sub description { 'telkku.com' } our %categories = ( SPORTS => "urheilu", MOVIE => "elokuvat", ); # # Unfortunately the embedded JSON data generated into the HTML page by # the server is (temporarily?) broken and unreliable. The web application # is not affected by this, because it always updates its state via XHR # calls to the JSON API endpoints. # sub _getJSON($) { my($api_path) = @_; # Fetch JSON object from API endpoint and return contents of "response" property return fetchJSON("https://www.telkku.com/api/channel-groups/$api_path", "response"); } # cache for group name to API ID mapping our %group2id; # Grab channel list sub channels { # Fetch & extract JSON sub-part my $data = _getJSON(""); # # channel-groups response has the following structure # # [ # { # id => "default_builtin_channelgroup1" # slug => "peruskanavat", # channels => [ # { # id => "yle-tv1", # name => "Yle TV1", # ... # }, # ... # ], # ... # }, # ... # ] # if (ref($data) eq "ARRAY") { my %channels; my %duplicates; foreach my $item (@{$data}) { if ((ref($item) eq "HASH") && (exists $item->{id}) && (exists $item->{slug}) && (exists $item->{channels}) && (ref($item->{channels}) eq "ARRAY")) { my($api_id, $group, $channels) = @{$item}{qw(id slug channels)}; if (defined($api_id) && length($api_id) && defined($group) && length($group) && (ref($channels) eq "ARRAY")) { debug(2, "Source telkku.com found group '$group' ($api_id) with " . scalar(@{$channels}) . " channels"); # initialize group name to API ID map $group2id{$group} = $api_id; foreach my $channel (@{$channels}) { if (ref($channel) eq "HASH") { my $id = $channel->{id}; my $name = $channel->{name}; if (defined($id) && length($id) && (not exists $duplicates{$id}) && length($name)) { debug(3, "channel '$name' ($id)"); $channels{"${id}.${group}.telkku.com"} = "fi $name"; # Same ID can appear in multiple groups - avoid duplicates $duplicates{$id}++; } } } } } } debug(2, "Source telkku.com parsed " . scalar(keys %channels) . " channels"); return(\%channels); } return; } sub _group2id($) { my($group) = @_; # Make sure group to ID map is initialized channels() unless %group2id; return $group2id{$group}; } # Grab one day sub grab { my($self, $id, $yesterday, $today, $tomorrow, $offset) = @_; # Get channel number from XMLTV id return unless my($channel, $group) = ($id =~ /^([\w-]+)\.([\w-]+)\.telkku\.com$/); # Map group name to API ID return unless my $api_id = _group2id($group); # # API parameters: # # - date is $today # - range is 24 hours (start 00:00:00.000 - end 00:00:00.000) # - max. 1000 entries per channel # - detailed information # # Response will include programmes from $yesterday that end $today, to # $tomorrow where a programme of $today ends. # my $data = _getJSON("$api_id/offering?endTime=00:00:00.000&limit=1000&startTime=00:00:00.000&view=PublicationDetails&tvDate=" . $today->ymdd()); # # Programme data has the following structure # # publicationsByChannel => [ # { # channel => { # id => "yle-tv1", # ... # }, # publications => [ # { # startTime => "2016-08-18T06:25:00.000+03:00", # endTime => "2016-08-18T06:55:00.000+03:00", # title => "Helil kyläs", # description => "Osa 9/10. Asiaohjelma, mikä ...", # programFormat => "MOVIE", # ... # }, # ... # ] # }, # ... # ] # if ((ref($data) eq "HASH") && (ref($data->{publicationsByChannel}) eq "ARRAY")) { my @objects; foreach my $item (@{ $data->{publicationsByChannel} }) { if ((ref($item) eq "HASH") && (ref($item->{channel}) eq "HASH") && (ref($item->{publications}) eq "ARRAY") && ($item->{channel}->{id} eq $channel)) { foreach my $programme (@{$item->{publications}}) { my($start, $end, $title, $desc) = @{$programme}{qw(startTime endTime title description)}; #debug(5, JSON->new->pretty->encode($programme)); if ($start && $end && $title && $desc) { $start = UnixDate($start, "%s"); $end = UnixDate($end, "%s"); # NOTE: entries with same start and end time are invalid if ($start && $end && ($start != $end)) { my $category = $categories{$programme->{programFormat}}; debug(3, "List entry $channel.$group ($start -> $end) $title"); debug(4, $desc); debug(4, $category) if defined $category; # Create program object my $object = fi::programme->new($id, "fi", $title, $start, $end); $object->category($category); $object->description($desc); push(@objects, $object); } } } } } # Fix overlapping programmes fi::programme->fixOverlaps(\@objects); return(\@objects); } return; } # That's all folks 1; ��������������������������������������xmltv-1.4.0/grab/fi/fi/source/telsu.pm��������������������������������������������������������������0000664�0000000�0000000�00000010224�15000742332�0017562�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- mode: perl; coding: utf-8 -*- ########################################### # # tv_grab_fi: source specific grabber code for https://www.telsu.fi # ############################################################################### # # Setup # # INSERT FROM HERE ############################################################ package fi::source::telsu; use strict; use warnings; BEGIN { # Currently www.telsu.fi is rejecting HTTP requests generated by XMLTV our $ENABLED = 0; } # Import from internal modules fi::common->import(); # Description sub description { 'telsu.fi' } # Grab channel list sub channels { my %channels; # Fetch & parse HTML my $root = fetchTree("https://www.telsu.fi/tanaan/kaikki"); if ($root) { # # Channel list can be found from <div class="ch">: # # <div id="prg"> # <div class="ch" rel="yle1"> # <a href="/perjantai/yle1" title="Yle TV1"> # <div>...</div> # </a> # ... # </div> # ... # </div> # if (my $container = $root->look_down("id" => "prg")) { if (my @channels = $container->look_down("class" => "ch")) { foreach my $channel (@channels) { if (my $link = $channel->find("a")) { my $id = $channel->attr("rel"); my $name = $link->attr("title"); if (defined($id) && length($id) && defined($name) && length($name)) { debug(3, "channel '$name' ($id)"); $channels{"${id}.telsu.fi"} = "fi $name"; } } } } } # Done with the HTML tree $root->delete(); } else { return; } debug(2, "Source telsu.fi parsed " . scalar(keys %channels) . " channels"); return(\%channels); } # Grab one day sub grab { my($self, $id, $yesterday, $today, $tomorrow, $offset) = @_; # Get channel number from XMLTV id return unless my($channel) = ($id =~ /^([^.]+)\.telsu\.fi$/); # Fetch & parse HTML my $root = fetchTree("https://www.telsu.fi/" . $today->ymd() . "/$channel"); if ($root) { my @objects; # # Each programme can be found in a separate <div class="dets stat"> node # # <div class="dets stat" rel="..."> # <div class="c"> # <div class="h"> # <h1> # <b>Uutisikkuna</b> # <em class="k0" title="Ohjelma on sallittu kaikenikäisille.">S</em> # </h1> # <h2> # <i>ma 24.07.2017</i> 04:00 - 06:50 <img src="..."> # <div class="rate" ...>...</div> # </h2> # </div> # <div class="t"> # <div>Uutisikkuna</div> # </div> # ... # </div> # </div> # if (my @programmes = $root->look_down("class" => "dets stat")) { my @offsets = ($yesterday, $today, $tomorrow); my $current = ''; # never matches -> $yesterday will be removed first foreach my $programme (@programmes) { my $title = $programme->find("b"); my $time = $programme->find("h2"); my $desc = $programme->look_down("class" => "t"); if ($title && $time && $desc) { if (my($new, $start_h, $start_m, $end_h, $end_m) = $time->as_text() =~ /^(.+)\s(\d{2})[:.](\d{2})\s-\s(\d{2})[:.](\d{2})/) { $title = $title->as_text(); if (length($title)) { $desc = $desc->as_text(); # Detect day change if ($new ne $current) { $current = $new; shift(@offsets); } my $start = timeToEpoch($offsets[0], $start_h, $start_m); my $end = timeToEpoch($offsets[0], $end_h, $end_m); # Detect end time on next day if ($end < $start) { # Are there enough day offsets left to handle a day change? # No -> more programmes than we asked for, exit loop last if @offsets < 2; $end = timeToEpoch($offsets[1], $end_h, $end_m); } debug(3, "List entry ${id} ($start -> $end) $title"); debug(4, $desc) if $desc; # Create program object my $object = fi::programme->new($id, "fi", $title, $start, $end); $object->description($desc); push(@objects, $object); } } } } } # Done with the HTML tree $root->delete(); # Fix overlapping programmes fi::programme->fixOverlaps(\@objects); return(\@objects); } return; } # That's all folks 1; ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/fi/source/yle.pm����������������������������������������������������������������0000664�0000000�0000000�00000013225�15000742332�0017223�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- mode: perl; coding: utf-8 -*- ########################################### # # tv_grab_fi: source specific grabber code for https://www.yle.fi # ############################################################################### # # Setup # # INSERT FROM HERE ############################################################ package fi::source::yle; use strict; use warnings; use Carp; use Date::Manip qw(UnixDate); use JSON qw(); BEGIN { our $ENABLED = 1; } # Import from internal modules fi::common->import(); # Description sub description { 'yle.fi' } our %languages = ( "fi" => [ "areena", "opas" ], "sv" => [ "arenan", "guide" ], ); sub _getJSON($$$) { my($slug, $language, $date) = @_; # Options "app_id" & "app_key" are mandatory my $app_id = fi::programme::getOption(description(), "app_id"); my $app_key = fi::programme::getOption(description(), "app_key"); croak("You must set yle.fi options 'app_id' & 'app_key' in the configuration") unless $app_id && $app_key; # Fetch JSON object from API endpoint and return contents of "data" property return fetchJSON("https://areena.api.yle.fi/v1/ui/schedules/${slug}/${date}.json?v=10&language=${language}&app_id=${app_id}&app_key=${app_key}", "data"); } sub _set_ua_headers() { my($headers, $clone) = cloneUserAgentHeaders(); # since a DDoS attack on yle.fi on 22-Oct-2022 this header is required $clone->header('Accept-Language', 'en'); # Return old headers to restore them at the end return $headers; } # Grab channel list sub channels { my %channels; # set up user agent default headers my $headers = _set_ua_headers(); # yle.fi offers program guides in multiple languages foreach my $code (sort keys %languages) { # Fetch & parse HTML (do not ignore HTML5 <time>) my $root = fetchTree("https://$languages{$code}[0].yle.fi/tv/$languages{$code}[1]", undef, undef, 1); if ($root) { # # Channel list can be found from Next.js JSON data # if (my $script = $root->look_down("_tag" => "script", "id" => "__NEXT_DATA__", "type" => "application/json")) { my($json) = $script->content_list(); my $decoded = JSON->new->decode($json); if ((ref($decoded) eq "HASH") && (ref($decoded->{props}) eq "HASH") && (ref($decoded->{props}->{pageProps}) eq "HASH") && (ref($decoded->{props}->{pageProps}->{view}) eq "HASH") && (ref($decoded->{props}->{pageProps}->{view}->{tabs}) eq "ARRAY")) { foreach my $tab (@{ $decoded->{props}->{pageProps}->{view}->{tabs} }) { if ((ref($tab) eq "HASH") && (ref($tab->{content}) eq "ARRAY")) { my($content) = @{ $tab->{content} }; if ((ref($content) eq "HASH") && (ref($content->{source}) eq "HASH")) { my $name = $tab->{title}; my $uri = $content->{source}->{uri}; if ($name && length($name) && $uri) { my($slug) = $uri =~ m,/ui/schedules/([^/]+)/[\d-]+\.json,; if ($slug) { debug(3, "channel '$name' ($slug)"); $channels{"${slug}.${code}.yle.fi"} = "$code $name"; } } } } } } } # Done with the HTML tree $root->delete(); } else { restoreUserAgentHeaders($headers); return; } } debug(2, "Source yle.fi parsed " . scalar(keys %channels) . " channels"); restoreUserAgentHeaders($headers); return(\%channels); } # Grab one day sub grab { my($self, $id, $yesterday, $today, $tomorrow, $offset) = @_; # Get channel number from XMLTV id return unless my($channel, $code) = ($id =~ /^([^.]+)\.([^.]+)\.yle\.fi$/); # Fetch & parse HTML (do not ignore HTML5 <time>) my $data = _getJSON($channel, $code, $today->ymdd()); # # Programme data has the following structure # # [ # { # type => "card", # presentation => "scheduleCard", # labels => [ # { # type => "broadcastStartDate", # raw => "2023-07-09T07:00:00+03:00", # ... # }, # { # type => "broadcastEndDate", # raw => "2023-07-09T07:55:26+03:00", # ... # }, # ... # ], # title => "Suuri keramiikkakisa", # description => "Kausi 4, 2/10. Tiiliä ja laastia. ...", # ... # }, # ... # ], # if ((ref($data) eq "ARRAY")) { my @objects; foreach my $item (@{ $data }) { if ((ref($item) eq "HASH") && ($item->{type} eq "card") && (ref($item->{labels}) eq "ARRAY")) { my($title, $desc) = @{$item}{qw(title description)}; my($category, $start, $end); foreach my $label (@{ $item->{labels} }) { if (ref($label) eq "HASH") { my($type, $raw) = @{$label}{qw(type raw)}; if ($type && $raw) { if ( $type eq "broadcastStartDate") { $start = UnixDate($raw, "%s"); } elsif ($type eq "broadcastEndDate") { $end = UnixDate($raw, "%s"); } elsif ($type eq "highlight") { $category = "elokuvat" if $raw eq "movie"; } } } } # NOTE: entries with same start and end time are invalid if ($start && $end && ($start != $end) && $title && $desc) { debug(3, "List entry $channel ($start -> $end) $title"); debug(4, $desc); debug(4, $category) if defined $category; # Create program object my $object = fi::programme->new($id, $code, $title, $start, $end); $object->category($category); $object->description($desc); push(@objects, $object); } } } # Fix overlapping programmes fi::programme->fixOverlaps(\@objects); return(\@objects); } return; } # That's all folks 1; ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/get_latest_version.sh�����������������������������������������������������������0000775�0000000�0000000�00000001260�15000742332�0020431�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#/bin/sh set -e # GitHub does not offer tarballs of directories :-( files=( merge.PL tv_grab_fi.pl fi/common.pm fi/day.pm fi/programme.pm fi/programmeStartOnly.pm fi/source/iltapulu.pm fi/source/star.pm fi/source/telkku.pm fi/source/yle.pm ) # Fetch files from GitHub repository echo "Fetching latest tv_grab_fi code..." rm -rf fi for _f in ${files[@]}; do echo "Fetching ${_f}..." mkdir -p $(dirname fi/${_f}) wget --quiet --clobber -O fi/${_f} \ "https://raw.githubusercontent.com/XMLTV/xmltv/master/grab/fi/${_f}" done # Generate tv_grab_fi rm -f tv_grab_fi perl fi/merge.PL tv_grab_fi rm -rf fi ls -l tv_grab_fi* echo "DONE." ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/merge.PL������������������������������������������������������������������������0000775�0000000�0000000�00000004774�15000742332�0015546�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w # # Merger to generate tv_grab_fi # use 5.008; use strict; use warnings; use File::Basename; # output file name my($outfile) = @ARGV or die "no output file specified!"; # working directory my $dir = dirname($0); # output file open(my $ofh, ">", $outfile) or die "can't open output file: $!"; # source modules my @sources = ( sort(<$dir/fi/*.pm>), sort(<$dir/fi/source/*.pm>)); print "Found modules: ", map({ basename($_) . " " } @sources), "\n"; # open main script open(my $ifh, "<", "$dir/tv_grab_fi.pl") or die "can't open main script file: $!"; # query version information from git my %versions = map { # returns empty string if not a git directory chomp(my $v = qx(git 2>/dev/null log -n1 --date="format:%Y/%m/%d %H:%M:%S" --pretty="%h %ad" HEAD -- $_)); (basename($_), $v); } @sources, "$dir/tv_grab_fi.pl"; # Merge while (<$ifh>) { # insert marker for source modules if (/^\# INSERT: SOURCES/) { print $ofh <<END_OF_MERGE_TEXT; # # This is the merged version of the script. # # !!! DO NOT EDIT - YOUR CHANGES WILL BE LOST !!! # # Any changes should be done to the original modules instead. # ############################################################################### END_OF_MERGE_TEXT foreach my $source (@sources) { open(my $sfh, "<", $source) or die "can't open source module '$source': $!"; print "Inserting module '", basename($source), "'\n"; while (<$sfh>) { next if 1../^\# INSERT FROM HERE /; next if /^__END__/..0; # right side always false -> cut to the end print $ofh $_; # Don't insert the code if source module has been disabled print($ofh <<END_OF_DISABLED_TEXT), last } # THIS DATA SOURCE HAS BEEN DISABLED! 1; END_OF_DISABLED_TEXT if /^\s*our\s+\$ENABLED\s*=\s*0;/; } close($sfh); print $ofh "\n###############################################################################\n"; } # delete marker for code } elsif (/^\# CUT CODE START/../^\# CUT CODE END/) { # insert version string } elsif (/^use XMLTV::Version /) { my $version = 'generated from\n\t' . join('\n\t', map { sprintf("%-25s %s", $_, $versions{$_}) } sort keys %versions); s/VERSION;$/"$version";/; print $ofh $_; # normal line } else { print $ofh $_; } } # check for write errors close($ofh) or die "error while writing to output file: $!"; # set executable flag chmod(0755, $outfile); # That's all folks... print "Merge done.\n"; exit 0; ����xmltv-1.4.0/grab/fi/test.conf�����������������������������������������������������������������������0000664�0000000�0000000�00000023534�15000742332�0016030�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- coding: utf-8 -*- # # How to regenerate: # # $ tv_grab_fi.pl --list-channels | \ # perl -MFile::Slurp -MHTML::Entities -e '$a=read_file(\*STDIN); \ # my @m = ($a =~ m,id="([^"]+)">\s+<display-name lang="..">([^<]+)</,mg); \ # while (my($i, $n) = splice(@m, 0, 2)) { \ # $n = decode_entities($n); \ # print "##channel $i $n\n"; \ # }' | \ # sort >test.txt # # NOTE: ##channel are those channels that should not be unmasked during testing # ##channel 100.iltapulu.fi Discovery Science ##channel 101.iltapulu.fi Food Network ##channel 102.iltapulu.fi Travel Channel ##channel 103.iltapulu.fi Investigation Discovery ##channel 104.iltapulu.fi MTV Urheilu 3 ##channel 105.iltapulu.fi Nat Geo Wild ##channel 106.iltapulu.fi Eveo #channel 10.iltapulu.fi MTV Ava #channel 11.iltapulu.fi Yle Teema & Fem ##channel 12.iltapulu.fi MTV Juniori #channel 13.iltapulu.fi Liv ##channel 17.iltapulu.fi MTV Max channel 1.iltapulu.fi Yle TV1 ##channel 201.iltapulu.fi BBC Nordic ##channel 202.iltapulu.fi Cartoon Network ##channel 203.iltapulu.fi CNN ##channel 229.iltapulu.fi TV5 Monde ##channel 23.iltapulu.fi TV Finland ##channel 26.iltapulu.fi MTV Urheilu 2 ##channel 28.iltapulu.fi MTV Aitio channel 2.iltapulu.fi Yle TV2 ##channel 32.iltapulu.fi MTV Viihde ##channel 34.iltapulu.fi MTV Urheilu 1 channel 3.iltapulu.fi MTV3 ##channel 41.iltapulu.fi SF-kanalen ##channel 42.iltapulu.fi V Film Premiere ##channel 43.iltapulu.fi V Film Action ##channel 46.iltapulu.fi V Film Family ##channel 49.iltapulu.fi V Sport 1 #channel 4.iltapulu.fi Nelonen ##channel 51.iltapulu.fi V Sport Golf ##channel 52.iltapulu.fi V Sport Vinter ##channel 53.iltapulu.fi Viasat Explore Nordic ##channel 54.iltapulu.fi Viasat History ##channel 55.iltapulu.fi Viasat Nature/Crime ##channel 56.iltapulu.fi Disney Channel ##channel 58.iltapulu.fi Discovery Channel ##channel 59.iltapulu.fi Eurosport #channel 5.iltapulu.fi TV5 ##channel 60.iltapulu.fi Eurosport 2 ##channel 61.iltapulu.fi MTV Finland #channel 62.iltapulu.fi Kutonen ##channel 69.iltapulu.fi TV7 #channel 6.iltapulu.fi MTV Sub ##channel 70.iltapulu.fi V Sport+ Suomi #channel 73.iltapulu.fi Hero ##channel 74.iltapulu.fi Frii ##channel 76.iltapulu.fi V Film Hits ##channel 77.iltapulu.fi V Sport 2 Suomi ##channel 78.iltapulu.fi V Sport 1 Suomi ##channel 79.iltapulu.fi V Sport Premium #channel 7.iltapulu.fi Jim ##channel 80.iltapulu.fi V Sport Football ##channel 81.iltapulu.fi TLC ##channel 82.iltapulu.fi National Geographic ##channel 87.iltapulu.fi Viaplay Urheilu ##channel 89.iltapulu.fi Yle Areena #channel 8.iltapulu.fi STAR Channel ##channel 90.iltapulu.fi Veikkaus TV ##channel 91.iltapulu.fi Ruutu ##channel 92.iltapulu.fi Animal Planet ##channel 93.iltapulu.fi V Sport Ultra HD ##channel 94.iltapulu.fi V Sport Motor ##channel al-jazeera.uutiset.telkku.com Al Jazeera ##channel animal-planet.dokumentit.telkku.com Animal Planet #channel ava.peruskanavat.telkku.com MTV Ava ##channel barnkanalen.lapset.telkku.com Barnkanalen ##channel bbc-nordic.muut.telkku.com BBC Nordic ##channel bbc-world-news.uutiset.telkku.com BBC World News ##channel bloomberg-tv.uutiset.telkku.com Bloomberg TV ##channel cartoonito.lapset.telkku.com Cartoonito ##channel cartoon-network.lapset.telkku.com Cartoon Network ##channel cmore-first.elokuvat.telkku.com MTV Aitio ##channel cmore-series.elokuvat.telkku.com MTV Viihde ##channel cnbc.uutiset.telkku.com CNBC ##channel cnn.uutiset.telkku.com CNN ##channel deutsche-welle.uutiset.telkku.com Deutsche Welle ##channel discovery-channel.dokumentit.telkku.com Discovery Channel ##channel discovery-science.dokumentit.telkku.com Discovery Science ##channel disney-channel.lapset.telkku.com Disney Channel ##channel euronews.uutiset.telkku.com EuroNews ##channel eurosport-2.urheilu.telkku.com Eurosport 2 ##channel eurosport.urheilu.telkku.com Eurosport ##channel eveo.peruskanavat.telkku.com Eveo ##channel extreme-sports.urheilu.telkku.com Extreme Sports ##channel fashion-tv.lifestyle.telkku.com Fashion TV ##channel food-network.lifestyle.telkku.com Food Network #channel fox.peruskanavat.telkku.com Star ##channel frii.peruskanavat.telkku.com Frii ##channel h2.dokumentit.telkku.com H2 #channel hero.peruskanavat.telkku.com Hero ##channel himlen-tv7.muut.telkku.com Himlen TV7 ##channel history-channel.dokumentit.telkku.com The History Channel ##channel iconcerts.musiikki.telkku.com iConcerts ##channel investigation-discovery.dokumentit.telkku.com Investigation Discovery #channel jim.peruskanavat.telkku.com Jim ##channel kanal5.ruotsi.telkku.com Kanal5 #channel kutonen.peruskanavat.telkku.com Kutonen #channel liv.peruskanavat.telkku.com Liv channel mtv3.peruskanavat.telkku.com MTV3 ##channel mtv-80s.musiikki.telkku.com MTV 80's ##channel mtv-dance.musiikki.telkku.com Club MTV ##channel mtv-finland.musiikki.telkku.com MTV Finland ##channel mtv-juniori.lapset.telkku.com MTV juniori ##channel mtv-liiga-1.urheilu.telkku.com MTV Liiga 1 ##channel mtv-liiga-2.urheilu.telkku.com MTV Liiga 2 ##channel mtv-liiga-3.urheilu.telkku.com MTV Liiga 3 ##channel mtv-liiga-4.urheilu.telkku.com MTV Liiga 4 ##channel mtv-liiga-5.urheilu.telkku.com MTV Liiga 5 ##channel mtv-liiga-6.urheilu.telkku.com MTV Liiga 6 ##channel mtv-liiga-7.urheilu.telkku.com MTV Liiga 7 ##channel mtv-liiga-uhd.urheilu.telkku.com MTV Liiga UHD ##channel mtv-live-hd.musiikki.telkku.com MTV Live HD ##channel mtv-max.urheilu.telkku.com MTV Max ##channel mtv-sport-1.urheilu.telkku.com MTV Urheilu 1 ##channel mtv-sport-2.urheilu.telkku.com MTV Urheilu 2 ##channel mtv-urheilu-3.urheilu.telkku.com MTV Urheilu 3 ##channel nat-geo-wild-scandinavia.v-sport-series-film.telkku.com Nat Geo Wild Scandinavia ##channel national-geographic.peruskanavat.telkku.com National Geographic #channel nelonen.peruskanavat.telkku.com Nelonen ##channel nick-jr.lapset.telkku.com Nick Jr. ##channel rtl.muut.telkku.com RTL ##channel sf-kanalen.mtv-katsomo.telkku.com SF-kanalen ##channel sky-news.uutiset.telkku.com Sky News channel star.fi STAR Channel #channel sub.peruskanavat.telkku.com MTV Sub ##channel svt-1.ruotsi.telkku.com SVT 1 ##channel svt24.ruotsi.telkku.com SVT24 ##channel svt-2.ruotsi.telkku.com SVT 2 ##channel tlc-finland.peruskanavat.telkku.com TLC ##channel travel-channel.dokumentit.telkku.com Travel Channel ##channel tv3.ruotsi.telkku.com TV3 ##channel tv4.ruotsi.telkku.com TV4 ##channel tv5-monde.muut.telkku.com TV5 Monde #channel tv5.peruskanavat.telkku.com TV5 ##channel tv6.ruotsi.telkku.com TV6 ##channel tv7.muut.telkku.com TV7 ##channel tv-finland.fi.yle.fi TV Finland ##channel tv-finland.muut.telkku.com TV Finland ##channel tv-finland.sv.yle.fi TV Finland ##channel viasat-explore.v-sport-series-film.telkku.com Viasat Explore ##channel viasat-film-action.elokuvat.telkku.com V film ACTION ##channel viasat-film.elokuvat.telkku.com V film PREMIERE ##channel viasat-film-family.elokuvat.telkku.com V film FAMILY #channel viasat-film-hits.elokuvat.telkku.com V film HITS ##channel viasat-fotboll-hd.urheilu.telkku.com V sport FOOTBALL ##channel viasat-golf.urheilu.telkku.com V sport GOLF ##channel viasat-history.v-sport-series-film.telkku.com Viasat History ##channel viasat-hockey.urheilu.telkku.com V sport vinter ##channel viasat-jaakiekko-hd.urheilu.telkku.com V sport 1 Suomi ##channel viasat-jalkapallo-hd.urheilu.telkku.com V sport 2 Suomi ##channel viasat-nature-crime.v-sport-series-film.telkku.com Viasat Nature/Crime ##channel viasat-sport-premium-hd.urheilu.telkku.com V sport PREMIUM ##channel viasat-sport.urheilu.telkku.com V sport ##channel viasat-ultra-hd.v-sport-series-film.telkku.com V sport ULTRA HD ##channel viasat-urheilu-hd.urheilu.telkku.com V sport + Suomi ##channel viron-etv.muut.telkku.com Viron ETV ##channel vsport-live-1.urheilu.telkku.com V Sport Live 1 ##channel Vsport-live-2.urheilu.telkku.com V Sport Live 2 ##channel Vsport-live-3.urheilu.telkku.com V Sport Live 3 ##channel Vsport-live-4.urheilu.telkku.com V Sport Live 4 ##channel Vsport-live-5.urheilu.telkku.com V Sport Live 5 ##channel yle-areena.fi.yle.fi Yle Areena ##channel yle-areena.sv.yle.fi Yle Arenan #channel yle-teema-fem.fi.yle.fi Yle Teema Fem #channel yle-teema-fem.peruskanavat.telkku.com Yle Teema #channel yle-teema-fem.sv.yle.fi Yle Teema Fem ##channel yle-tv1.fi.yle.fi Yle TV1 ##channel yle-tv1.peruskanavat.telkku.com Yle TV1 ##channel yle-tv1.sv.yle.fi Yle TV1 ##channel yle-tv2.fi.yle.fi Yle TV2 ##channel yle-tv2.peruskanavat.telkku.com Yle TV2 ##channel yle-tv2.sv.yle.fi Yle TV2 # Source options option yle.fi app_id=areena-web-items option yle.fi app_key=<SECRET> # Title name mappings title map "70’s show" "70's show" title map "70s show" "70's show" # Strip unnecessary "movie"-type prefixes from title # NOTE: '#' is comment character -> apply URI escaping to line title map "%23Subleffa: " "" title map "Elokuva: " "" title map "Kino: " "" title map "Kino Klassikko: " "" # Strip parental level from titles title strip parental level # Series definitions series description 70's show series description Bomb Girls series description Castle series description Casualty series description Doc Martin series description Frasier series description Frendit series description Game of Thrones series description Goldbergit series description The Handmaid's Tale - Orjattaresi series description Hulluna sinuun series description Kauniit ja rohkeat series description Kummeli series description Last Man on Earth series description Leila leipoo Ranskassa series description Moderni perhe series description Olipa kerran series description Pikku naisia series description Poliisit series description Pulmuset series description Rillit huurussa series description Ruotsin miljonääriäidit series description Salatut elämät series description Simpsonit series description Sohvaperunat series description South Park series description Tannbach, vartioitu kylä series description Toisenlaiset frendit series description Tyhjätaskut series description Valaistunut series description Viikingit series description Weeds series title Prisma ��������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/test.sh�������������������������������������������������������������������������0000775�0000000�0000000�00000011206�15000742332�0015511�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/bin/sh # # Based on the tests exsecuted on <http://www.crustynet.org.uk/~xmltv-tester> # # Check log file for errors check_log() { local log=$1 if [ -s "$log" ]; then ( \ echo "Test with log '$log' failed:"; \ echo; \ cat $log; \ echo; \ ) 1>&2 test_failure=1 fi } check_perl_warnings() { local log=$1 if [ -s "$log" ]; then perl <$log >"${log}_warnings" -ne 'print if / at \S+ line \d+\.$/' if [ -s "${log}_warnings" ]; then ( \ echo "Test with log '$log' caused Perl warnings:"; \ echo; \ cat "${log}_warnings"; \ echo; \ ) 1>&2 test_failure=1 fi fi } validate_xml() { local xml=$1 ${xmltv_script}/tv_validate_file $xml if [ $? -ne 0 ]; then test_failure=1 fi } # Configuration set -e build_dir=$(pwd) script_dir=${build_dir}/grab/fi script_file=${script_dir}/tv_grab_fi.pl test_dir=${build_dir}/test-fi xmltv_lib=${build_dir}/blib/lib xmltv_script=${build_dir}/blib/script export PERL5LIB=${xmltv_lib} # Command line options for arg in $*; do case $arg in debug) debug="$debug --debug" ;; merge) merge_script=1 ;; norandomize) debug="$debug --no-randomize" ;; reuse) preserve_directory=1 ;; *) echo 1>&2 "unknown option '$arg" exit 1 ;; esac done # Setup if [ -n "$merge_script" ]; then script_file=${script_dir}/tv_grab_fi ${script_dir}/merge.PL ${script_file} fi if [ -z "$preserve_directory" ]; then echo "Deleting results from last run." rm -rf ${test_dir} fi mkdir -p ${test_dir} script_file="${script_file} ${debug} --test-mode" cd ${test_dir} set -x +e # # Tests # # Original test run with 2 days and using test.conf from repository # perl -I ${xmltv_lib} ${script_file} --ahdmegkeja > /dev/null 2>&1 perl -I ${xmltv_lib} ${script_file} --version > /dev/null 2>&1 perl -I ${xmltv_lib} ${script_file} --description > /dev/null 2>&1 perl -I ${xmltv_lib} ${script_file} --list-channels --cache t_fi_cache > t_fi_channels.xml --quiet 2>t_fi_channels.log if [ $? -ne 0 ]; then check_perl_warnings t_fi_channels.log tail -1 t_fi_channels.log test_failure=1 fi perl -I ${xmltv_lib} ${script_file} --config-file ${script_dir}/test.conf --offset 1 --days 2 --cache t_fi_cache > t_fi_1_2.xml --quiet 2>t_fi_1.log check_perl_warnings t_fi_1.log validate_xml t_fi_1_2.xml ${xmltv_script}/tv_cat t_fi_1_2.xml --output /dev/null 2>t_fi_6.log check_log t_fi_6.log ${xmltv_script}/tv_sort --duplicate-error t_fi_1_2.xml --output t_fi_1_2.sorted.xml 2>t_fi_1_2.sort.log check_log t_fi_1_2.sort.log perl -I ${xmltv_lib} ${script_file} --config-file ${script_dir}/test.conf --offset 1 --days 1 --cache t_fi_cache --output t_fi_1_1.xml 2>t_fi_2.log check_perl_warnings t_fi_2.log perl -I ${xmltv_lib} ${script_file} --config-file ${script_dir}/test.conf --offset 2 --days 1 --cache t_fi_cache > t_fi_2_1.xml 2>t_fi_3.log check_perl_warnings t_fi_3.log perl -I ${xmltv_lib} ${script_file} --config-file ${script_dir}/test.conf --offset 1 --days 2 --cache t_fi_cache --quiet --output t_fi_4.xml 2>t_fi_4.log check_perl_warnings t_fi_4.log ${xmltv_script}/tv_cat t_fi_1_1.xml t_fi_2_1.xml --output t_fi_1_2-2.xml 2>t_fi_5.log check_log t_fi_5.log ${xmltv_script}/tv_sort --duplicate-error t_fi_1_2-2.xml --output t_fi_1_2-2.sorted.xml 2>t_fi_7.log check_log t_fi_7.log diff t_fi_1_2.sorted.xml t_fi_1_2-2.sorted.xml > t_fi__1_2.diff check_log t_fi__1_2.diff # # Modified test run with 7 days and modified test.conf # perl -pe 's/^#channel\s+/channel /' <${script_dir}/test.conf >${test_dir}/test.conf perl -I ${xmltv_lib} ${script_file} --config-file ${test_dir}/test.conf --offset 1 --days 7 --cache t_fi_cache >t_fi_full_7.xml --quiet 2>t_fi_full.log check_perl_warnings t_fi_full.log validate_xml t_fi_full_7.xml rm -f t_fi_single.log for d in $(seq 1 7); do perl -I ${xmltv_lib} ${script_file} --config-file ${test_dir}/test.conf --offset $d --days 1 --cache t_fi_cache >t_fi_single_$d.xml --quiet 2>>t_fi_single.log done check_perl_warnings t_fi_single.log ${xmltv_script}/tv_cat t_fi_full_7.xml --output /dev/null 2>t_fi_output.log ${xmltv_script}/tv_sort --duplicate-error t_fi_full_7.xml --output t_fi_full_7.sorted.xml 2>>t_fi_output.log check_log t_fi_output.log ${xmltv_script}/tv_cat t_fi_single_*.xml --output t_fi_full_7-2.xml 2>t_fi_output-2.log ${xmltv_script}/tv_sort --duplicate-error t_fi_full_7-2.xml --output t_fi_full_7-2.sorted.xml 2>>t_fi_output-2.log check_log t_fi_output-2.log diff t_fi_full_7.sorted.xml t_fi_full_7-2.sorted.xml >t_fi__7.diff check_log t_fi__7.diff # # All tests done # set +x if [ -n "$test_failure" ]; then echo "TEST FAILED!" exit 1 else echo "All tests OK." exit 0 fi ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi/tv_grab_fi.pl�������������������������������������������������������������������0000775�0000000�0000000�00000036475�15000742332�0016654�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w # -*- mode: perl; coding: utf-8 -*- ########################################### # # Setup # ############################################################################### # - we process Unicode texts # - calling POSIX::tzset() is no longer for Perl >= 5.8.9 use 5.008009; use strict; use warnings; use XMLTV; use constant VERSION => "$XMLTV::VERSION"; ############################################################################### # INSERT: SOURCES ############################################################################### package main; # Perl core modules use Getopt::Long; use List::Util qw(shuffle); use Pod::Usage; # CUT CODE START ############################################################################### # Load internal modules use FindBin qw($Bin); BEGIN { foreach my $source (<$Bin/fi/*.pm>, <$Bin/fi/source/*.pm>) { require "$source"; } } ############################################################################### # CUT CODE END # Generate source module list my @sources; BEGIN { @sources = map { s/::$//; $_ } map { "fi::source::" . $_ } sort grep { ${ $::{'fi::'}->{'source::'}->{$_}->{ENABLED} } } keys %{ $::{'fi::'}->{'source::'} }; die "$0: couldn't find any source modules?" unless @sources; } # Import from internal modules fi::common->import(':main'); # Basic XMLTV modules use XMLTV::Version VERSION; use XMLTV::Capabilities qw(baseline manualconfig cache); use XMLTV::Description 'Finland (' . join(', ', map { $_->description() } @sources ) . ')'; # NOTE: We will only reach the rest of the code only when the script is called # without --version, --capabilities or --description # Reminder of XMLTV modules use XMLTV::Get_nice; use XMLTV::Memoize; ############################################################################### # # Main program # ############################################################################### # Forward declarations sub doConfigure(); sub doListChannels(); sub doGrab(); # Command line option default values my %Option = ( days => 14, quiet => 0, debug => 0, offset => 0, ); # Enable caching. This will remove "--cache [file]" from @ARGV XMLTV::Memoize::check_argv('XMLTV::Get_nice::get_nice_aux'); # Process command line options if (GetOptions(\%Option, "configure", "config-file=s", "days=i", "debug|d+", "gui:s", "help|h|?", "list-channels", "no-randomize", "offset=i", "output=s", "quiet", "test-mode")) { pod2usage(-exitstatus => 0, -verbose => 2) if $Option{help}; setDebug($Option{debug}); setQuiet($Option{quiet}); if ($Option{configure}) { # Configure mode doConfigure(); } elsif ($Option{'list-channels'}) { # List channels mode doListChannels(); } else { # Grab mode (default) doGrab(); } } else { pod2usage(2); } # That's all folks exit 0; ############################################################################### # # Utility functions for the different modes # ############################################################################### sub _getConfigFile() { require XMLTV::Config_file; return(XMLTV::Config_file::filename($Option{'config-file'}, "tv_grab_fi", $Option{quiet})); } { my $ofh; sub _createXMLTVWriter() { # Output file handling $ofh = \*STDOUT; if (defined $Option{output}) { open($ofh, ">", $Option{output}) or die "$0: cannot open file '$Option{output}' for writing: $!"; } # Create XMLTV writer for UTF-8 encoded text binmode($ofh, ":utf8"); my $writer = XMLTV::Writer->new( encoding => 'UTF-8', OUTPUT => \*STDOUT, ); #### HACK CODE #### $writer->start({ "generator-info-name" => "XMLTV", "generator-info-url" => "http://xmltv.org/", "source-info-url" => "multiple", # TBA "source-data-url" => "multiple", # TBA }); #### HACK CODE #### return($writer); } sub _closeXMLTVWriter($) { my($writer) = @_; $writer->end(); # close output file if ($Option{output}) { close($ofh) or die "$0: write error on file '$Option{output}': $!"; } message("DONE"); } } sub _addChannel($$$$) { my($writer, $id, $name, $language) = @_; $writer->write_channel({ id => $id, 'display-name' => [[$name, $language]], }); } { my $bar; sub _createProgressBar($$) { my($label, $count) = @_; return if $Option{quiet}; require XMLTV::Ask; require XMLTV::ProgressBar; XMLTV::Ask::init($Option{gui}); $bar = XMLTV::ProgressBar->new({ name => $label, count => $count, }); } sub _updateProgressBar() { $bar->update() if defined $bar } sub _destroyProgressBar() { $bar->finish() if defined $bar } } sub _getChannels($$) { my($callback, $opaque) = @_; # Get channels from all sources _createProgressBar("getting list of channels", @sources); foreach my $source (@sources) { debug(1, "requesting channel list from source '" . $source->description ."'"); if (my $list = $source->channels()) { die "test failure: source '" . $source->description . "' didn't find any channels!\n" if ($Option{'test-mode'} && (keys %{$list} == 0)); while (my($id, $value) = each %{ $list }) { my($language, $name) = split(" ", $value, 2); $callback->($opaque, $id, $name, $language); } } _updateProgressBar(); } _destroyProgressBar(); } ############################################################################### # # Configure Mode # ############################################################################### sub doConfigure() { # Get configuration file name my $file = _getConfigFile(); XMLTV::Config_file::check_no_overwrite($file); # Open configuration file. Assume UTF-8 encoding open(my $fh, ">:utf8", $file) or die "$0: can't open configuration file '$file': $!"; print $fh "# -*- coding: utf-8 -*-\n"; # Get channels my %channels; _getChannels(sub { # We only need name and ID my(undef, $id, $name) = @_; $channels{$id} = $name; }, undef); # Query user my @sorted = sort keys %channels; my @answers = XMLTV::Ask::ask_many_boolean(1, map { "add channel $channels{$_} ($_)?" } @sorted); # Generate configuration file contents from answers foreach my $id (@sorted) { warn("\nunexpected end of input reached\n"), last unless @answers; # Write selection to configuration file my $answer = shift(@answers); print $fh ($answer ? "" : "#"), "channel $id $channels{$id}\n"; } # Check for write errors close($fh) or die "$0: can't write to configuration file '$file': $!"; message("DONE"); } ############################################################################### # # List Channels Mode # ############################################################################### sub doListChannels() { # Create XMLTV writer my $writer = _createXMLTVWriter(); # Get channels _getChannels(sub { my($writer, $id, $name, $language) = @_; _addChannel($writer, $id, $name, $language); }, $writer); # Done writing _closeXMLTVWriter($writer); } ############################################################################### # # Grab Mode # ############################################################################### sub doGrab() { # Sanity check die "$0: --offset must be a non-negative integer" unless $Option{offset} >= 0; die "$0: --days must be an integer larger than 0" unless $Option{days} > 0; # Get configuation my %channels; { # Get configuration file name my $file = _getConfigFile(); # Open configuration file. Assume UTF-8 encoding open(my $fh, "<:utf8", $file) or die "$0: can't open configuration file '$file': $!"; # Process configuration information while (<$fh>) { # Comment removal, white space trimming and compressing s/\#.*//; s/^\s+//; s/\s+$//; next unless length; # skip empty lines s/\s+/ /; # Channel definition if (my($id, $name) = /^channel (\S+) (.+)/) { debug(1, "duplicate channel definion in line $.:$id ($name)") if exists $channels{$id}; $channels{$id} = $name; # Programme definition } elsif (fi::programme->parseConfigLine($_)) { # Nothing to be done here } else { warn("bad configuration line in file '$file', line $.: $_\n"); } } close($fh); } # Generate list of days my $dates = fi::day->generate($Option{offset}, $Option{days}); # Set up time zone setTimeZone(); # Create XMLTV writer my $writer = _createXMLTVWriter(); # Generate task list with one task per channel and day my @tasklist; foreach my $id (sort keys %channels) { for (my $i = 1; $i < $#{ $dates }; $i++) { push(@tasklist, [$id, @{ $dates }[$i - 1..$i + 1], $Option{offset} + $i - 1]); } } # Randomize the task list in order to create a random access pattern # NOTE: if you use only one source, then this is basically a no-op if (not $Option{'no-randomize'}) { debug(1, "Randomizing task list"); @tasklist = shuffle(@tasklist); } # For each entry in the task list my %seen; my @programmes; _createProgressBar("getting listings", @tasklist); foreach my $task (@tasklist) { my($id, $yesterday, $today, $tomorrow, $offset) = @{$task}; debug(1, "XMLTV channel ID '$id' fetching day $today"); foreach my $source (@sources) { if (my $programmes = $source->grab($id, $yesterday, $today, $tomorrow, $offset)) { if (@{ $programmes }) { # Add channel ID & name (once) _addChannel($writer, $id, $channels{$id}, $programmes->[0]->language()) unless $seen{$id}++; # Add programmes to list push(@programmes, @{ $programmes }); } elsif ($Option{'test-mode'}) { die "test failure: source '" . $source->description . "' didn't retrieve any programmes for '$id'!\n"; } } } _updateProgressBar(); } _destroyProgressBar(); # Dump programs message("writing XMLTV programme data"); $_->dump($writer) foreach (@programmes); # Done writing _closeXMLTVWriter($writer); } ############################################################################### # # Man page # ############################################################################### __END__ =pod =head1 NAME tv_grab_fi - Grab TV listings for Finland =head1 SYNOPSIS tv_grab_fi [--cache E<lt>FILEE<gt>] [--config-file E<lt>FILEE<gt>] [--days E<lt>NE<gt>] [--gui [E<lt>OPTIONE<gt>]] [--no-randomize] [--offset E<lt>NE<gt>] [--output E<lt>FILEE<gt>] [--quiet] tv_grab_fi --capabilities tv_grab_fi --configure [--cache E<lt>FILEE<gt>] [--config-file E<lt>FILEE<gt>] [--gui [E<lt>OPTIONE<gt>]] [--quiet] tv_grab_fi --description tv_grab_fi --help|-h|-? tv_grab_fi --list-channels [--cache E<lt>FILEE<gt>] [--gui [E<lt>OPTIONE<gt>]] [--quiet] tv_grab_fi --version =head1 DESCRIPTION Grab TV listings for several channels available in Finland. The data comes from various sources, e.g. www.telkku.com. The grabber relies on parsing HTML, so it might stop working when the web page layout is changed. You need to run C<tv_grab_fi --configure> first to create the channel configuration for your setup. Subsequently runs of C<tv_grab_fi> will grab the latest data, process them and produce XML data on the standard output. =head1 COMMANDS =over 8 =item B<NONE> Grab mode. =item B<--capabilities> Show the capabilities this grabber supports. See also L<http://wiki.xmltv.org/index.php/XmltvCapabilities>. =item B<--configure> Generate the configuration file by asking the users which channels to grab. =item B<--description> Print the description for this grabber. =item B<--help|-h|-?> Show this help page. =item B<--list-channels> Fetch all available channels from the various sources and write them to the standard output. =item B<--version> Show the version of this grabber. =back =head1 GENERIC OPTIONS =over 8 =item B<--cache F<FILE>> File name to cache the fetched HTML data in. This speeds up subsequent runs using the same data. =item B<--gui [OPTION]> Enable the graphical user interface. If you don't specify B<OPTION> then XMLTV will automatically choose the best available GUI. Allowed values are: =over 4 =item B<Term> Terminal output with a progress bar =item B<TermNoProgressBar> Terminal output without progress bar =item B<Tk> Tk-based GUI =back =item B<--quiet> Suppress any progress messages to the standard output. =back =head1 CONFIGURE MODE OPTIONS =over 8 =item B<--config-file F<FILE>> File name to write the configuration to. Default is F<$HOME/.xmltv/tv_grab_fi.conf>. =back =head1 GRAB MODE OPTIONS =over 8 =item B<--config-file F<FILE>> File name to read the configuration from. Default is F<$HOME/.xmltv/tv_grab_fi.conf>. =item B<--days C<N>> Grab C<N> days of TV data. Default is 14 days. =item B<--no-randomize> Grab TV data in deterministic order, i.e. first fetch channel 1, days 1 to N, then channel 2, and so on. Default is to use a random access pattern. If you only grab TV data from one source then the randomizing is a no-op. =item B<--offset C<N>> Grab TV data starting at C<N> days in the future. Default is 0, i.e. today. =item B<--output F<FILE>> Write the XML data to F<FILE> instead of the standard output. =back =head1 CONFIGURATION FILE SYNTAX The configuration file is line oriented, each line can contain one command. Empty lines and everything after the C<#> comment character is ignored. Supported commands are: =over 8 =item B<channel ID NAME> Grab information for this channel. C<ID> depends on the source, C<NAME> is ignored and forwarded as is to the XMLTV output file. This information can be automatically generated using the grabber in the configuration mode. =item B<series description NAME> If a programme title matches C<NAME> then the first sentence of the description, i.e. everything up to the first period (C<.>), question mark (C<?>) or exclamation mark (C<!>), is removed from the description and is used as the name of the episode. =item B<series title NAME> If a programme title contains a colon (C<:>) then the grabber checks if the left-hand side of the colon matches C<NAME>. If it does then the left-hand side is used as programme title and the right-hand side as the name of the episode. =item B<title map "FROM" 'TO'> If the programme title starts with the string C<FROM> then replace this part with the string C<TO>. The strings must be enclosed in single quotes (C<'>) or double quotes (C<">). The title mapping occurs before the C<series> command processing. =item B<title strip parental level> At the beginning of 2012 some programme descriptions started to include parental levels at the end of the title, e.g. C<(S)>. With this command all parental levels will be removed from the titles automatically. This removal occurs before the title mapping. =back =head1 SEE ALSO L<xmltv>. =head1 AUTHORS =head2 Current =over =item Stefan Becker C<chemobejk at gmail dot com> =item Ville Ahonen C<ville dot ahonen at iki dot fi> =back =head2 Retired =over =item Matti Airas =back =head1 BUGS The channels are identified by channel number rather than the RFC2838 form recommended by the XMLTV DTD. =cut ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi_sv/�����������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0014703�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi_sv/test.conf��������������������������������������������������������������������0000664�0000000�0000000�00000000563�15000742332�0016535�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������channel!tvfinland.yle.fi channel!tv5.yle.fi channel=tv2.yle.fi channel!sub.yle.fi channel!liv.yle.fi channel=tv1.yle.fi channel!frii.yle.fi channel!mtv3.yle.fi channel!natgeo.yle.fi channel!hero.yle.fi channel!star.yle.fi channel!jim.yle.fi channel!tlc.yle.fi channel!kutonen.yle.fi channel!ava.yle.fi channel!areena.yle.fi channel!nelonen.yle.fi channel=teemafem.yle.fi ���������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fi_sv/tv_grab_fi_sv����������������������������������������������������������������0000664�0000000�0000000�00000064724�15000742332�0017455�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl =pod =head1 NAME tv_grab_fi_sv - Grab TV listings for Finland in Swedish. =head1 SYNOPSIS tv_grab_fi_sv --help tv_grab_fi_sv --version tv_grab_fi_sv --capabilities tv_grab_fi_sv --description tv_grab_fi_sv [--config-file FILE] [--days N] [--offset N] [--output FILE] [--quiet] [--debug] tv_grab_fi_sv --configure [--config-file FILE] tv_grab_fi_sv --configure-api [--stage NAME] [--config-file FILE] [--output FILE] tv_grab_fi_sv --list-channels [--config-file FILE] [--output FILE] [--quiet] [--debug] =head1 DESCRIPTION Retrieves and displays TV listings for the Finnish YLE channels plus some of the most popular commercial channels. The data comes from www.yle.fi and the Swedish listings are retrieved rather than the Finnish. Just like tv_grab_fi, this grabber relies on parsing HTML so it could very well stop working at any time. You have been warned. =head1 OPTIONS B<--help> Print a help message and exit. B<--version> Show the versions of the XMLTV libraries, the grabber and of key modules used for processing listings. B<--capabilities> Show which capabilities the grabber supports. For more information, see L<http://xmltv.org/wiki/xmltvcapabilities.html> B<--description> Show a brief description of the grabber. B<--config-file FILE> Specify the name of the configuration file to use. If not specified, a default of B<~/.xmltv/tv_grab_fi_sv.conf> is used. This is the file written by B<--configure> and read when grabbing. B<--output FILE> When grabbing, write output to FILE rather than to standard output. B<--days N> When grabbing, grab N days of data instead of all available. Supported values are 1-14. Default: 14 B<--offset N> Start grabbing at today + N days. Supported values are 0-13. Default: 0 =head1 SEE ALSO L<xmltv(5)>. =head1 AUTHOR Per Lundberg, (perlun at gmail dot com). Inspired/based on other grabbers, like tv_grab_uk_rt, tv_grab_se_swedb and tv_grab_fi. =cut use strict; use warnings; use utf8; use DateTime; use Encode; use HTML::TreeBuilder; use IO::Scalar; use XMLTV; use XMLTV::Ask qw/say/; use XMLTV::Configure::Writer; use XMLTV::Get_nice 0.005070; use XMLTV::Memoize; use XMLTV::Options qw/ParseOptions/; sub t; # Constants. # my $DATA_SITE_ROOT = 'https://areena.yle.fi/'; # Finnish my $DATA_SITE_ROOT = 'https://arenan.yle.fi/'; # Swedish my $GRABBER_NAME = 'tv_grab_fi_sv'; my $GRABBER_VERSION = "$XMLTV::VERSION"; my $XML_ENCODING = 'utf-8'; my $LANGUAGE_CODE = 'sv'; # This is not the timezone for the machine on which the grabber is # being run, but rather the timezone in which all the grabbed data is # being specified. my $TIMEZONE = 'Europe/Helsinki'; # Attributes of the root element in output. my $xmltv_attributes = { 'source-info-url' => 'http://www.yle.fi/', 'source-data-url' => "$DATA_SITE_ROOT/", 'generator-info-name' => "XMLTV/$XMLTV::VERSION, $GRABBER_NAME $GRABBER_VERSION", 'generator-info-url' => 'http://www.xmltv.org', }; XMLTV::Memoize::check_argv('XMLTV::Get_nice::get_nice_aux'); # The list of channels available from the Yle Program Guide. Their # names are deliberately specified in a manner which would be natural # for people watching e.g. TV channels from Sweden (so that "TV1" # would in their mindset not necessarily refer to Yle's TV1 channel - # thus, the reason behind the "Yle" prefixing here). # # The key in this hash is the name of the channel as given on the Yle # program guide web page. my $channels = { 'tv1.yle.fi' => { 'id' => 'tv1.yle.fi', 'display-name' => [[ 'YLE TV1', $LANGUAGE_CODE ]] }, 'tv2.yle.fi' => { 'id' => 'tv2.yle.fi', 'display-name' => [[ 'YLE TV2', $LANGUAGE_CODE ]] }, 'teemafem.yle.fi' => { 'id' => 'teemafem.yle.fi', 'display-name' => [[ 'YLE TEEMA/FEM', $LANGUAGE_CODE ]] }, 'arenan.yle.fi' => { 'id' => 'arenan.yle.fi', 'display-name' => [[ 'ARENAN', $LANGUAGE_CODE ]] }, 'mtv3.yle.fi' => { 'id' => 'mtv3.yle.fi', 'display-name' => [[ 'MTV3', $LANGUAGE_CODE ]] }, 'nelonen.yle.fi' => { 'id' => 'nelonen.yle.fi', 'display-name' => [[ 'NELONEN', $LANGUAGE_CODE ]] }, 'sub.yle.fi' => { 'id' => 'sub.yle.fi', 'display-name' => [[ 'SUB', $LANGUAGE_CODE ]] }, 'tv5.yle.fi' => { 'id' => 'tv5.yle.fi', 'display-name' => [[ 'TV5', $LANGUAGE_CODE ]] }, 'liv.yle.fi' => { 'id' => 'liv.yle.fi', 'display-name' => [[ 'LIV', $LANGUAGE_CODE ]] }, 'jim.yle.fi' => { 'id' => 'jim.yle.fi', 'display-name' => [[ 'JIM', $LANGUAGE_CODE ]] }, 'kutonen.yle.fi' => { 'id' => 'kutonen.yle.fi', 'display-name' => [[ 'KUTONEN', $LANGUAGE_CODE ]] }, 'tlc.yle.fi' => { 'id' => 'tlc.yle.fi', 'display-name' => [[ 'TLC', $LANGUAGE_CODE ]] }, 'star.yle.fi' => { 'id' => 'star.yle.fi', 'display-name' => [[ 'STAR', $LANGUAGE_CODE ]] }, 'ava.yle.fi' => { 'id' => 'ava.yle.fi', 'display-name' => [[ 'AVA', $LANGUAGE_CODE ]] }, 'hero.yle.fi' => { 'id' => 'hero.yle.fi', 'display-name' => [[ 'HERO', $LANGUAGE_CODE ]] }, 'frii.yle.fi' => { 'id' => 'frii.yle.fi', 'display-name' => [[ 'FRII', $LANGUAGE_CODE ]] }, 'natgeo.yle.fi' => { 'id' => 'natgeo.yle.fi', 'display-name' => [[ 'NATIONAL GEOGRAPHIC', $LANGUAGE_CODE ]] }, 'tvfinland.yle.fi' => { 'id' => 'tvfinland.yle.fi', 'display-name' => [[ 'TV FINLAND', $LANGUAGE_CODE ]] }, }; # Map between channel names (as presented by the YLE data) and channel # IDs, as created by us. my $channel_id_map = { 'yle-tv1' => 'tv1.yle.fi', 'yle-tv2' => 'tv2.yle.fi', 'yle-teema-fem' => 'teemafem.yle.fi', 'yle-arenan' => 'arenan.yle.fi', 'mtv3' => 'mtv3.yle.fi', 'nelonen' => 'nelonen.yle.fi', 'sub' => 'sub.yle.fi', 'tv5' => 'tv5.yle.fi', 'liv' => 'liv.yle.fi', 'jim' => 'jim.yle.fi', 'kutonen' => 'kutonen.yle.fi', 'tlc' => 'tlc.yle.fi', 'star-channel' => 'star.yle.fi', 'ava' => 'ava.yle.fi', 'hero' => 'hero.yle.fi', 'frii' => 'frii.yle.fi', 'national-geographic' => 'natgeo.yle.fi', 'tv-finland' => 'tvfinland.yle.fi', }; my @ARGUMENTS = @ARGV; # Parse the standard XMLTV grabber options. my ($opt, $conf) = ParseOptions( { grabber_name => "tv_grab_fi_sv", capabilities => [qw/baseline manualconfig apiconfig/], stage_sub => \&config_stage, listchannels_sub => \&list_channels, version => $GRABBER_VERSION, description => "Finland (Swedish)", defaults => { days => 14, offset => 0, quiet => 0, debug => 0 }, }); t("Command line arguments: " . join(' ', @ARGUMENTS)); # When we get here, we know that we are invoked in such a way that the # channel data should be grabbed. # Configure the output and write the XMLTV data - header, channels, # listings, and footer my $writer; setup_xmltv_writer(); write_xmltv_header(); write_channel_list(@{ $conf->{channel} }); write_listings_data(@{ $conf->{channel} }); write_xmltv_footer(); # For the moment, we always claim that we've exited successfully... exit 0; sub t { my $message = shift; print STDERR $message . "\n" if $opt->{debug}; } sub config_stage { my($stage, $conf) = shift; die "Unknown stage $stage" if $stage ne "start"; # This grabber doesn't need any configuration (except for # possibly channel, selection), so this subroutine doesn't need # to do very much at all. my $result; my $writer = new XMLTV::Configure::Writer(OUTPUT => \$result, encoding => $XML_ENCODING); $writer->start({ grabber => 'tv_grab_fi_sv' }); $writer->end('select-channels'); return $result; } # Returns a string containing an xml-document with <channel>-elements # for all available channels. sub list_channels { my ($conf, $opt) = shift; my $result = ''; my $fh = new IO::Scalar \$result; my $oldfh = select($fh); # Create an XMLTV::Writer object. The important part here is that # the output should go to $fh (in other words, to the $result # string), NOT to stdout... my %writer_args = ( encoding => $XML_ENCODING, OUTPUT => $fh ); my $writer = new XMLTV::Writer(%writer_args); $writer->start($xmltv_attributes); # Loop over all channels and write them to this XMLTV::Writer. foreach my $channel_id (keys %{ $channels }) { my $channel = $channels->{$channel_id}; $writer->write_channel($channel); } $writer->end; select($oldfh); $fh->close(); return $result; } # Determine options for XMLTV::Writer, and instantiate it. sub setup_xmltv_writer { # output options my %g_args = (); if (defined $opt->{output}) { t("\nOpening XML output file '$opt->{output}'\n"); my $fh = new IO::File ">$opt->{output}"; die "Error: Cannot write to '$opt->{output}', exiting" if (!$fh); %g_args = (OUTPUT => $fh); } # Determine how many days of listings are required and # range-check, applying default values if necessary. If --days or # --offset is specified we must ensure that the values for days, # offset and cutoff are passed to XMLTV::Writer. my %d_args = (); if (defined $opt->{days} || defined $opt->{offset}) { if (defined $opt->{days}) { if ($opt->{days} < 1 || $opt->{days} > 14) { if (!$opt->{quiet}) { say("Specified --days option is not possible (1-14). " . "Retrieving all available listings."); } $opt->{days} = 14 } } else { # No --days parameter were given. Use the default. $opt->{days} = 14; } if (defined $opt->{offset}) { if ($opt->{offset} < 0 || $opt->{offset} > 13) { if (!$opt->{quiet}) { say("Specified --offset option is not possible (0-13). " . "Retrieving all available listings."); } $opt->{offset} = 0; } } else { $opt->{offset} = 0; } $d_args{days} = $opt->{days}; $d_args{offset} = $opt->{offset}; $d_args{cutoff} = "000000"; } t("Setting up XMLTV::Writer using \"" . $XML_ENCODING . "\" for output"); $writer = new XMLTV::Writer(%g_args, %d_args, encoding => $XML_ENCODING); } # Writes the XMLTV header. sub write_xmltv_header { t("Writing XMLTV header"); $writer->start($xmltv_attributes); } # Writes the channel list for all configured channels sub write_channel_list { my (@channels) = @_; t("Started writing <channel> elements"); foreach my $channel_id (sort @channels) { my $channel = $channels->{$channel_id}; $writer->write_channel($channel); } t("Finished writing <channel> elements"); } # Download listings data for all the configured channels sub write_listings_data { my (@channels) = @_; my $programmes = {}; my $previous_programmes = {}; say(scalar @channels ." configured channels") if !$opt->{quiet}; $XMLTV::Get_nice::ua->default_header('Accept-Language' => "$LANGUAGE_CODE"); my $dt_today = DateTime->today( time_zone => $TIMEZONE ); say(" Today: " . $dt_today->strftime( '%Y-%m-%dT%H:%M:%S %z')) if $opt->{debug}; # Get start & stop times for the grab my $dt_grab_start = $dt_today->clone->add( days => $opt->{offset} ); say("Grab start: " . $dt_grab_start->strftime( '%Y-%m-%dT%H:%M:%S %z')) if $opt->{debug}; my $dt_grab_stop = $dt_grab_start->clone->add( days => $opt->{days} ); say(" Grab stop: " . $dt_grab_stop->strftime( '%Y-%m-%dT%H:%M:%S %z')) if $opt->{debug}; # schedules run from 06:00-06:00 so to pass tv_validate_file we need # to get the day before also DAY: for (my $i = $opt->{offset} - 1; $i < $opt->{offset} + $opt->{days}; $i++) { # Create URL for the schedules for this channel/month/day combination. # e.g. https://areena.yle.fi/tv/guide?t=2017-09-08 my $date = $dt_today->clone->add( days => $i ); my $url = sprintf('%stv/guide?t=%s', $DATA_SITE_ROOT, $date->strftime( '%Y-%m-%d' )); say("Downloading $url") if $opt->{debug}; # The yle website does not parse correctly via HTML::TreeBuilder unless # we accept it as-is, hence $t->implicit_tags(0) my $htb_opts = { 'implicit_tags' => '0', 'ignore_unknown' => '0', }; my $tree = get_nice_tree($url, undef, undef, $htb_opts); my @t_channels = $tree->look_down('_tag' => 'li', 'class' => 'guide-channels__channel'); say ' Found '.scalar @t_channels.' channels' if $opt->{debug}; next DAY if scalar @t_channels == 0; # Can't use foreach because of clumpidx processing below CHANNEL: for (my $j = 0; $j < scalar @t_channels; $j++) { my $t_channel = $t_channels[$j]; my $chan_id_raw = $t_channel->look_down('_tag' => 'div')->attr('aria-label'); my $chan_id = lc (join ('-', split (' ', $chan_id_raw)) ); # Check if this program belongs to one of the # configured channels. If it doesn't, ignore it. say " Found $chan_id" if $opt->{debug}; my $c_channel_id = $channel_id_map->{$chan_id}; if (!$c_channel_id) { say " UNKNOWN CHANNEL ID $chan_id, skipping" if $opt->{debug}; next CHANNEL; } # skip if channel not requested by user next CHANNEL if !(grep { $_ eq $c_channel_id } @channels); say " Processing $c_channel_id" if $opt->{debug}; my $t_schedule = $t_channel->look_down('_tag' => 'ul', 'class' => 'schedule-list'); my @t_progs = $t_schedule->look_down('_tag' => 'li', 'class' => qr/schedule-card/); say " Found " . scalar @t_progs . " programmes" if $opt->{debug}; PROGRAMME: foreach my $t_prog (@t_progs) { my ($t_prog_label, $t_prog_desc, $t_prog_link, $t_prog_title, $t_prog_film); my ($p_start, $e_start, $p_end, $p_dtstart, $p_dtend); my ($p_title, $p_subtitle, $p_desc, $p_url, $p_category, $p_rating); my ($p_season, $p_episode_num, $p_episode_total, $p_part_num, $p_number); $t_prog_label = $t_prog->look_down('_tag' => 'span', 'itemprop' => 'name'); next PROGRAMME unless $t_prog_label; $p_title = $t_prog_label->as_text(); say " Processing title: " . $p_title if $opt->{debug}; # Extract film category from title if ($p_title =~ s/^(Film|Ny film|Elokuva):\s*//) { $p_category = $1; } # Extract rating details if ($p_title =~ s/\s*\((S|T|7|12|16|18)\)$//) { $p_rating = $1; } # Extract season number from title if present if ($p_title =~ s/\s(\d+)\.\skausi//i) { $p_season = $1; } $t_prog_desc = $t_prog->look_down('_tag' => 'span', 'itemprop' => 'description'); # Extract possible sub-title/season/episode numbering from description # Try to handle both Finnish and Swedish versions if ($t_prog_desc) { $p_desc = $t_prog_desc->as_text(); # Extract rating details from desc if ($p_desc =~ s/^\((S|T|7|12|16|18)\)\.\s*//) { $p_rating = $1; } # Remove/update new season text before processing for ($p_desc) { $_ =~ s/^Sarja alkaa\.\s*//i; $_ =~ s/^Sarja alkaa uusintana\.\s*//i; $_ =~ s/^Sarja alkaa alusta uusintana\.\s*//i; $_ =~ s/^(\d+)\. kausi alkaa uusintana. Osa (\d+)\./Kausi $1. Osa $2./i; $_ =~ s/^Kausi (\d+) \w+\. (\d+)\/(\d+)\./Kausi $1. Osa $2\/$3./i; $_ =~ s/^(\d+)\. tuotantokausi, (\d+)\/(\d+)/Kausi $1, $2\/$3/i; $_ =~ s/^Sarja alkaa, (?:osa)? (\d+)\/(\d+)/Kausi 1, $1\/$2/i; $_ =~ s/^Uusi kausi\!\s*//i; $_ =~ s/^Uusi sarja\!\s*//i; } # Extract likely sub-title if ($p_desc =~ s/^(.*)\. K?ausi (\d+)/Kausi $2/) { $p_subtitle = $1; } if ($p_desc =~ s/^(?:Kausi|Säsong)\s*(\d+)[.,]\s*(?:Jakso|Avsnitt|Osa|Del)?\s*(\d+)\s*\/?\s*(\d+)?\s*\.//i) { $p_season = $1; $p_episode_num = $2; $p_episode_total = $3; } elsif ($p_desc =~ s/^(?:Kausi|Säsong)\s*(\d+)[.,]\s*(\d+)\s*\/\s*(\d+)\s*(\w)/$4/i) { $p_season = $1; $p_episode_num = $2; $p_episode_total = $3; } elsif ($p_desc =~ s/^(?:Kausi|Säsong)\s*(\d+)[.,]\s*(?:Jakso|Avsnitt)\s*(\d+)[.,]\s*(?:Osa|Del)\s*(\d+)\s*\.//i) { $p_season = $1; $p_episode_num = $2; $p_part_num = $3; } elsif ($p_desc =~ s/^(?:Jakso|Avsnitt|Osa|Del)\s*(\d+)\s*\/?\s*(\d+)?\s*\.//i) { $p_episode_num = $1; $p_episode_total = $2; } elsif ($p_desc =~ s/\. (?:Jakso|Avsnitt|Osa|Del)\s*(\d+)\s*\/?\s*(\d+)?\s*\.?$//i) { $p_episode_num = $1; $p_episode_total = $2; } elsif ($p_desc =~ s/^(?:Jakso|Avsnitt|Osa|Del)\s*(\d+):\s*//i) { $p_episode_num = $1; } # Remove/update new season text after processing for ($p_desc) { $_ =~ s/^\s*Uusi \d+\. kausi\!//i; $_ =~ s/^\s*Uusi kausi\!\s*//i; $_ =~ s/^\s*Uusi sarja\!\s*//i; } $p_desc = trim( tidy( $p_desc ) ); } # FIXME # extract cast information from description # extract movie year # extract original movie title # Create correctly-indexed programme season/episode numbering # # series number is zero-indexed if (defined $p_season && $p_season > 0) { $p_season--; } else { $p_season = ''; } # episode number is zero-indexed if (defined $p_episode_num && $p_episode_num > 0) { $p_episode_num--; } else { $p_episode_num = ''; } # episode total is one-indexed and should always be greater than the # max episode number (which is zero-indexed) if (defined $p_episode_total && $p_episode_total > 0 && $p_episode_total > $p_episode_num ) { $p_episode_total = "/" . $p_episode_total; } else { $p_episode_total = ''; } # part number is zero-indexed if (defined $p_part_num && $p_part_num > 0) { $p_part_num--; } else { $p_part_num = ''; } $p_number = "" . $p_season . "." . $p_episode_num . $p_episode_total . "." . $p_part_num; # If programme is outside of requested timeframe then drop it $p_start = $t_prog->look_down('_tag' => 'time', 'itemprop' => 'startDate')->attr('datetime'); # say(" StartDate: " . $p_start) if $opt->{debug}; my $dt_prog_start = dt_from_itempropdate($p_start); if ($dt_prog_start < $dt_grab_start || $dt_prog_start >= $dt_grab_stop) { # say(" ** Programme starts outside of grabbing window, skipping...\n") if $opt->{debug}; next PROGRAMME; } $p_end = $t_prog->look_down('_tag' => 'time', 'itemprop' => 'endDate')->attr('datetime'); my $dt_prog_stop = dt_from_itempropdate($p_end); $p_dtstart = xmltv_isotime( $p_start ); $p_dtend = xmltv_isotime( $p_end ); # If the previous programme on this channel overlaps with the start time of this programme, # trust our start time and update the stop time of the previous programme if (exists $previous_programmes->{$c_channel_id} and $previous_programmes->{$c_channel_id}->{'stop'} gt $p_dtstart) { say(" ** Overlap detected, updating previous programme's stop time") if !$opt->{quiet}; $previous_programmes->{$c_channel_id}->{'stop'} = $p_dtstart; } $t_prog_link = $t_prog->look_down('_tag' => 'a', 'class' => 'schedule-card__link'); $p_url = $DATA_SITE_ROOT . $t_prog_link->attr('href') if $t_prog_link; $p_url =~ s/(?<!:)\/\//\// if $p_url; # Create the data structure for the program. my $programme = { 'channel' => $c_channel_id, 'title' => [[ encode('utf-8', $p_title), $LANGUAGE_CODE ]], 'start' => $p_dtstart, 'stop' => $p_dtend, }; $programme->{'desc'} = [[ encode('utf-8', $p_desc ), $LANGUAGE_CODE ]] if (defined $p_desc && $p_desc ne ''); $programme->{'sub-title'} = [[ encode('utf-8', $p_subtitle), $LANGUAGE_CODE ]] if (defined $p_subtitle && $p_subtitle ne ''); $programme->{'category'} = [[ encode('utf-8', $p_category) ]] if (defined $p_category && $p_category ne ''); $programme->{'url'} = [ $p_url ] if (defined $p_url && $p_url ne ''); $programme->{'rating'} = [ [ $p_rating, 'KAVI' ] ] if (defined $p_rating && $p_rating ne ''); $programme->{'episode-num'} = [ [ $p_number, "xmltv_ns" ] ] if defined $p_number && $p_number ne '..'; # store the programme avoiding duplicates # also check for duplicate start times and set clumpidx if ( defined $programmes->{$c_channel_id}->{$p_dtstart} ) { # duplicate prog or contemporary? my $dup = 0; foreach my $p ( @{ $programmes->{$c_channel_id}->{$p_dtstart} } ) { $dup = 1 if ( $p->{'title'}[0][0] eq $programme->{'title'}[0][0] ); # duplicate } next PROGRAMME if $dup; # ignore duplicates if (!$dup) { # contemporary programme so set clumpidx my $numclumps = scalar @{ $programmes->{$c_channel_id}->{$p_dtstart} } + 1; # set (or adjust) clumpidx of existing programmes my $i = 0; foreach my $p ( @{ $programmes->{$c_channel_id}->{$p_dtstart} } ) { $p->{'clumpidx'} = "$i/$numclumps"; $i++; } # set clumpidx for new programme $programme->{'clumpidx'} = "$i/$numclumps"; } } # store the programme push @{ $programmes->{$c_channel_id}->{$p_dtstart} }, $programme; # remember previous programme on this channel to check start/stop overlap $previous_programmes->{$c_channel_id} = $programme; } } } # All data has been gathered. We can now write the programmes hash to output. foreach ( sort keys %{$programmes} ) { my $ch_progs = $programmes->{$_}; foreach ( sort keys %{$ch_progs} ) { my $dt_progs = $ch_progs->{$_}; foreach (@{ $dt_progs }) { $writer->write_programme($_); } } } } # Writes the XMLTV footer. sub write_xmltv_footer { t("\nWriting XMLTV footer\n"); $writer->end; } # Remove bad chars from an element sub tidy( $ ) { return $_[0] if !defined $_[0]; $_[0] =~ s/(\s)\xA0/$1/og; # replace 'space- ' with 'space' $_[0] =~ s/\xA0/ /og; # replace any remaining   with space $_[0] =~ s/\xAD//og; # delete soft hyphens return $_[0]; } # Trim function to remove whitespace from the start and end of the # string. sub trim ($) { my $string = shift; $string =~ s/^\s+//; $string =~ s/\s+$//; return $string; } # Converts a DateTime + time of the form "2014-04-12T09:00:00+03:00" to something suitable # for XMLTV, i.e. 20140412090000 +0300 sub xmltv_isotime ($) { my $time = shift; # let's not overthink this... just use a regexp! $time =~ s/[:-]//g; $time =~ /^(\d{8})T(\d{6}).*(\+\d{4})$/; return $1.$2.' '.$3; } # Convert a string of the form "2017-09-09T06:00:00+03:00" to a DateTime sub dt_from_itempropdate { my $date = shift; my ($y, $m, $d, $h, $i, $s, $t, $th, $tm) = $date =~ /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})([\+-])(\d{2}):(\d{2})$/; return DateTime->new( year => $y, month => $m, day => $d, hour => $h, minute => $i, second => $s, time_zone => "$t$th$tm", ); } ��������������������������������������������xmltv-1.4.0/grab/fr/��������������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0014204�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fr/test.conf�����������������������������������������������������������������������0000664�0000000�0000000�00000001345�15000742332�0016035�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������cachedir=/tmp/.xmltv/cache bouquet=grandes-chaines-et-tnt channel!6ter.telestar.fr channel!arte.telestar.fr channel!bfmtv.telestar.fr channel!c8.telestar.fr channel=canalplus.telestar.fr channel!cherie25.telestar.fr channel!cnews.telestar.fr channel!cstar.telestar.fr channel=france2.telestar.fr channel!france3.telestar.fr channel!france4.telestar.fr channel!france5.telestar.fr channel!franceo.telestar.fr channel!gulli.telestar.fr channel!lequipe.telestar.fr channel!m6.telestar.fr channel!nrj12.telestar.fr channel!publicsenatlcpan.telestar.fr channel!rmcdecouverte.telestar.fr channel!rmcstory.telestar.fr channel=tf1.telestar.fr channel!tf1seriesfilms.telestar.fr channel!tfx.telestar.fr channel!tmc.telestar.fr channel!w9.telestar.fr �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/fr/tv_grab_fr����������������������������������������������������������������������0000775�0000000�0000000�00000104272�15000742332�0016253�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl use warnings; use strict; use XMLTV::Ask; use XMLTV::Capabilities qw/baseline manualconfig cache/; use XMLTV::Configure::Writer; use XMLTV::DST; use XMLTV::Get_nice qw(get_nice_tree); $XMLTV::Get_nice::ua->parse_head(0); $XMLTV::Get_nice::FailOnError = 0; use XMLTV::Memoize; XMLTV::Memoize::check_argv('XMLTV::Get_nice::get_nice_aux'); use XMLTV::Options qw/ParseOptions/; use XMLTV::ProgressBar; use DateTime; use DateTime::Duration; use Encode qw/decode encode/; use File::Path; use Getopt::Long; use HTML::Entities; use HTML::TreeBuilder; use HTTP::Cache::Transparent; use IO::Scalar; ############################################################################ # Main declarations # ############################################################################ # Grabber details my $GRABBER_NAME = 'tv_grab_fr'; my $GRABBER_VERSION = "$XMLTV::VERSION"; my $ROOT_URL = "https://www.telestar.fr"; my $GRID_FOR_CHANNEL = "$ROOT_URL/programme-tv/"; my $GRID_FOR_BOUQUET = "$ROOT_URL/programme-tv/bouquets/"; my $GRID_BY_CHANNEL_PER_DAY = "$ROOT_URL/programme-tv/grille-chaine/"; my $ENCODING = "utf-8"; my $LANG = "fr"; my $MAX_RETRY = 5; my %tv_attributes = ( 'source-info-name' => 'Tele Star', 'source-info-url' => 'telestar.tv', 'source-data-url' => "$GRID_FOR_CHANNEL", 'generator-info-name' => "XMLTV/$XMLTV::VERSION, $GRABBER_NAME", ); my ( $opt, $conf ) = ParseOptions( { grabber_name => "$GRABBER_NAME", version => "$GRABBER_VERSION", description => "France (Tele Star)", capabilities => [qw/baseline manualconfig cache apiconfig/], defaults => { days => 14, offset => 0, quiet => 0, debug => 0, slow => 0 }, extra_options => [qw/slow/], stage_sub => \&config_stage, listchannels_sub => \&list_channels, } ); ############################################################################ # At this point, the script takes over from ParseOptions # ############################################################################ validate_options(); validate_config(); initialise_cache(); print_version_info(); my $channels = get_configured_channels(1); my $writer = setup_xmltv_writer(); write_xmltv_header($writer); write_channel_list($writer, $channels); write_listings_data($writer, $channels); write_xmltv_footer($writer); ############################################################################ # Subroutines # ############################################################################ sub config_stage { my ( $stage, $conf ) = @_; my $result; my $writer = new XMLTV::Configure::Writer( OUTPUT => \$result, encoding => $ENCODING ); $writer->start( { grabber => "$GRABBER_NAME" } ); if ($stage eq 'start') { $writer->write_string( { id => 'cachedir', title => [ [ "Directory to store $GRABBER_NAME cache", 'en' ] ], description => [ [ $GRABBER_NAME . ' uses a cache to store files that have been '. 'downloaded. Please specify path to cache directory. ', 'en' ] ], default => get_default_cachedir(), } ); $writer->end('bouquet'); } elsif ($stage eq 'bouquet') { $writer->start_selectone( { id => 'bouquet', title => [ [ 'Please select your TV service (bouquet)', 'en' ] ], description => [ [ "When choosing which channels to download listings for, $GRABBER_NAME " . "will only show the channels on your selected TV service.", 'en' ] ], } ); my $bouquets = get_bouquets(); foreach my $b ( sort keys %$bouquets ) { my $name = $b; my $id = $bouquets->{$b}; $writer->write_option( { value => $id, text => [ [ $name, 'fr' ] ], } ); } $writer->end_selectone(); # The select-channels stage must be the last stage called $writer->end('select-channels'); } else { die "Unknown stage $stage"; } return $result; } sub list_channels { my ( $conf, $opt ) = @_; # Do not filter channels with --list-channels my $filtered = $opt->{'list-channels'} ? 0 : 1; my $channels = get_available_channels($opt, $conf, $filtered); my $result = ""; my $fh = new IO::Scalar \$result; my $oldfh = select( $fh ); my %g_args = (OUTPUT => $fh); my $writer = new XMLTV::Writer(%g_args, encoding => $ENCODING); $writer->start(\%tv_attributes); foreach my $c_id (sort keys %{$channels}) { $writer->write_channel($channels->{$c_id}); } $writer->end; select( $oldfh ); $fh->close(); return $result; } sub get_bouquets { my %bouquets; debug_print("get_bouquets(): searching for available bouquets"); my $url = $GRID_FOR_CHANNEL . "bouquets"; my $t = get_nice_tree($url, undef, undef, undef); debug_print("get_bouquets(): url = '$url'"); if (not defined $t) { print STDERR "Unable to retrieve bouquets page\n"; return; } foreach my $b_tree ( $t->look_down( "_tag", "div", "class", "bouquet" ) ) { my $b_title = $b_tree->look_down("_tag", "h2")->as_text(); debug_print(" Found bouquet name: $b_title"); my $b_url = $b_tree->look_down("_tag", "a", "class", "red-link")->attr('href'); debug_print(" Found bouquet URL $b_url"); my ($b_id) = $b_url =~ /^\/programme-tv\/bouquets\/(.+)/; debug_print(" Found bouquet ID $b_id"); $bouquets{$b_title} = $b_id; } $t->delete(); undef $t; return \%bouquets; } sub get_available_channels { my ($opt, $conf, $filtered) = @_; my $bouquet_id; if ($filtered) { $bouquet_id = $conf->{'bouquet'}[0]; if (not defined $bouquet_id) { debug_print("get_available_channels(): no bouquet specified, please re-configure grabber"); return; } debug_print("get_available_channels(): filtering out unconfigured channels"); debug_print("get_available_channels(): searching for channels on bouquet ID: $bouquet_id"); } else { debug_print("get_available_channels(): searching all available channels"); } my $url = $GRID_FOR_CHANNEL . "bouquets"; my $t = get_nice_tree($url, undef, undef, undef); if (not defined $t) { print STDERR "Error: Unable to retrieve bouquets page\n"; return; } my %available_channels; BOUQUET: foreach my $b_tree ( $t->look_down( "_tag", "div", "class", "bouquet" ) ) { my $b_title = $b_tree->look_down("_tag", "h2")->as_text(); my $b_url = $b_tree->look_down("_tag", "a", "class", "red-link")->attr('href'); my ($b_id) = $b_url =~ /^\/programme-tv\/bouquets\/(.+)/; next BOUQUET unless (!$filtered || ($b_id eq $bouquet_id)); debug_print("get_available_channels(): found requested bouquet ID: $b_id"); CHANNEL: my @b_chans = $b_tree->look_down( "_tag", "a" ); debug_print(" Found " . scalar @b_chans . " channels"); foreach my $b_chan (@b_chans) { my $c_url = $b_chan->attr('href'); if ( $c_url =~ /^\/programme-tv\/grille-chaine\/(.+)/ ) { my $c_name = $b_chan->as_text(); my $c_id = $1; debug_print(" available channel: $c_name ($c_id)"); my %ch = ( 'id' => $c_id . ".telestar.fr", 'display-name' => [[ $c_name, 'fr' ]], ); $available_channels{$c_id} = \%ch; } } } $t->delete(); undef $t; return \%available_channels; } sub setup_xmltv_writer { my %g_args = (); if (defined $opt->{output}) { debug_print("\nOpening XMLTV output file '$opt->{output}'\n"); my $fh = new IO::File ">$opt->{output}"; die "Error: Cannot write to '$opt->{output}', exiting" if (! $fh); %g_args = (OUTPUT => $fh); } return new XMLTV::Writer(%g_args, encoding => $ENCODING); } sub write_xmltv_header { my $writer = shift; debug_print("Writing XMLTV header"); $writer->start(\%tv_attributes); } sub write_channel_list { my ($writer, $channels) = @_; debug_print("write_channel_list: writing <channel> elements"); foreach my $c_id (sort keys %{$channels}) { my $c_name = encode_and_trim( $channels->{$c_id}{'display-name'}[0][0]); my %ch = ( 'id' => $c_id . ".telestar.fr", 'display-name' => [[ $c_name, 'fr' ]], ); $writer->write_channel( \%ch ); } } sub get_configured_channels { my $filtered = shift; my $available_channels = get_available_channels($opt, $conf, $filtered); my %seen_ids; foreach (keys %{$available_channels}) { $seen_ids{$_} = 0; } debug_print("get_configured_channels(): checking configured channels"); foreach my $c_id (@{$conf->{'channel'}}) { ($c_id) = $c_id =~ /^([\w%]+)\.telestar\.fr$/; if (! exists $seen_ids{$c_id}) { debug_print("** UNAVAILABLE channel: '$c_id'"); } else { my $c_name = $available_channels->{$c_id}{'display-name'}[0][0]; debug_print(" configured channel: $c_name ($c_id)"); $seen_ids{$c_id} = 1; } } # remove any channels not flagged my %available_configured; foreach my $c_id (keys %{$available_channels}) { if ($seen_ids{$c_id}) { $available_configured{$c_id} = $available_channels->{$c_id}; } } if ($opt->{'debug'}) { my $wanted = scalar @{$conf->{'channel'}}; my $actual = scalar keys %available_configured; debug_print("get_configured_channels(): $actual/$wanted configured channels supported by grabber"); } return \%available_configured; } sub write_listings_data { my ($writer, $channels) = @_; my $dates = get_dates_to_grab(); my $urls = generate_urls_to_grab($dates); my $bar; if (not $opt->{quiet} and not $opt->{debug}) { $bar = new XMLTV::ProgressBar('Getting listings...', scalar keys %$urls); } # Store individual programmes in a list and write each channel in full later. # key = upstream channel ID, value = listref of programme elements my %programmes; debug_print("\nProcessing list of URLs to grab...\n"); foreach my $ymd (sort keys %$urls) { my $url = $urls->{$ymd}; my $progs_on_channel = get_daily_data_for_requested_channels($url, $ymd, $channels); foreach my $c (keys %$progs_on_channel) { push @{ $programmes{$c} }, @{ $progs_on_channel->{$c} }; } if (not $opt->{quiet} and not $opt->{debug}) { update $bar; } } # use Data::Dumper; print STDERR Dumper(\%programmes); # exit; # No stop times are given in the listings (only inaccurate durations), so # we can use the start time of a following programme as the stop time of # the previous programme. (May fail if channel does not have listings # for full 24hrs). foreach my $c (keys %programmes) { debug_print(" Analysing/updating schedule gaps between programmes on channel ID '$c'"); $programmes{$c} = update_programme_stop_times($programmes{$c}); } # Write out all available programme elements for each channel foreach my $c_id (sort keys %programmes) { debug_print(" Writing listings for channel '$c_id'"); foreach my $p (@{$programmes{$c_id}}) { $writer->write_programme($p); } } } sub get_dates_to_grab { my @dates_to_grab = (); # First date to grab listings for my $grab_start_date = get_date_today_with_offset($opt->{offset}); push @dates_to_grab, $grab_start_date; # Remaining dates to grab listings for for (my $offset = 1; $offset < $opt->{days}; $offset++) { push @dates_to_grab, $grab_start_date + DateTime::Duration->new( days => $offset ); } debug_print("Will grab listings for following dates:"); if ($opt->{debug}) { foreach (@dates_to_grab) { print STDERR " " . $_->strftime("%a, %d %b %Y") . "\n"; } } return \@dates_to_grab; } sub generate_urls_to_grab { my ($dates_to_grab) = @_; my $bouquet = $conf->{'bouquet'}[0]; my %urls; debug_print("Creating list of URLs to grab based on configured bouquet..."); foreach my $d (@$dates_to_grab) { my $ymd = $d->strftime("%Y%m%d"); my $dmy = $d->strftime("%d-%m-%Y"); my $url = $GRID_FOR_BOUQUET . $bouquet . "/journee/(date)/" . $dmy . "/(ajax)/1"; $urls{$ymd} = $url; debug_print( " Adding URL: $url" ); } return \%urls; } sub get_daily_data_for_requested_channels { my ($url, $ymd, $channels) = @_; debug_print("get_daily_data_for_requested_channels(): url=$url"); # Get the page's tree my $t = get_nice_tree($url, undef, undef, undef); if (not defined $t) { debug_print("Error: Could not get data for URL: $url"); return; } # Locate the listings grid my $grid = $t->look_down('_tag', 'div', 'class', 'grid-content'); # Locate the channel container in the grid and list of available channels my $c_cont = $grid->look_down('_tag', 'div', 'id', 'channels'); my @available_channels = $c_cont->look_down('_tag', 'div', 'class', 'channel'); # Locate the programme container in the grid my $p_cont = $grid->look_down('_tag', 'div', 'id', 'programs'); my @programmes = $p_cont->look_down('_tag', 'div', 'class', 'channel'); my %progs_on_channel; foreach my $i (0 .. (scalar @available_channels -1)) { my $c = $available_channels[$i]; my $c_url = $c->look_down('_tag', 'a')->attr('href'); my ($c_id) = $c_url =~ /^\/programme-tv\/grille-chaine\/(.+)/; if (exists($channels->{$c_id})) { $progs_on_channel{$c_id} = process_channel_row($c_id, $programmes[$i], $ymd); } } $t->delete(); undef $t; return \%progs_on_channel; } sub process_channel_row { my ($c_id, $row, $ymd) = @_; debug_print("process_channel_row: processing listings for: $c_id ($ymd)"); my @programmes = (); PROGRAMME: foreach my $programme ($row->look_down('_tag', 'div', 'class', qr/program /) ) { # skip empty program cells if ($programme->attr('class') =~ /no-program/) { debug_print(" skipping 'no-program' entry\n"); next PROGRAMME; } # extract the programme data my $p = process_program($c_id, $programme, $ymd); push @programmes, $p if defined $p; debug_print("\n"); } return \@programmes; } sub process_program { my ($c_id, $programme, $ymd) = @_; my $title_text; my $prog_page; my $title = $programme->look_down('_tag', 'p', 'class', 'title'); if ($title) { if ($title->as_text() =~ /\w+/) { $title_text = trim($title->as_text()); debug_print("process_program: found programme title '" . $title_text . "'"); my $link = $title->look_down('_tag', 'a', 'class', 'lien-fiche'); if ($link and $link->attr('href') =~ /programme-tv/) { $prog_page = $ROOT_URL . $link->attr('href'); debug_print(" Programme subpage found '" . $prog_page . "'"); } } else { debug_print(" No programme title text found, skipping programme"); return undef; # REQUIRED } } else { debug_print(" No programme title tag found, skipping programme"); return undef; # REQUIRED } my $start_time; my $duration_mins; my $start = $programme->look_down('_tag', 'p', 'class', 'time'); if ($start) { if ($start->as_text() =~ /(\d\d)h(\d\d)/) { my ($hh, $mm) = ($1, $2); $start_time = $ymd.$hh.$mm."00"; debug_print(" Found programme start '" . $hh."h".$mm . "'"); } else { debug_print(" Start time not parsed, skipping programme'"); return undef; # REQUIRED } # Programme durations are given, but rarely agree with the difference # between this programme's start time and the next $duration_mins = $start->look_down('_tag', 'span'); if ($duration_mins) { if ($duration_mins->as_text() =~ /\((\d+) min\)/) { $duration_mins = $1; debug_print(" Found programme duration '" . $duration_mins ." mins'"); } else { debug_print(" No programme duration found"); } } } else { debug_print(" No start time found, skipping programme'"); return undef; # REQUIRED } debug_print(" Creating programme hash for '" . $title_text . " / " . $start_time); my %prog = (channel => $c_id.".telestar.fr", title => [ [ encode_and_trim($title_text), $LANG ] ], start => utc_offset($start_time, "+0100"), ); # Store some temp data for later processing. A leading underscore in # a key name means the data is not written by XMLTV::Writer if ($duration_mins and $duration_mins > 0) { $prog{'_duration_mins'} = $duration_mins; } if ($prog_page) { $prog{'_prog_page'} = $prog_page; } my $episodenumber = $programme->look_down('_tag', 'p', 'class', 'title-episode'); if ($episodenumber) { if ($episodenumber->as_text() =~ /Saison (\d+) Episode (\d+)/) { my ($season_num, $episode_num) = ($1, $2); # Season/episode number is zero-indexed. (Totals are one-indexed.) # Sometimes, a series or episode number of 0 is seen, so we ignore it if ($season_num == 0) { $season_num = ""; } else { $season_num--; } if ($episode_num == 0) { $episode_num = ""; } else { $episode_num--; } $episodenumber = $episodenumber->as_text(); $prog{'episode-num'} = [ [ $season_num . "." . $episode_num . ".", "xmltv_ns" ] ]; debug_print(" Found programme episode numbering '" . $episodenumber . "'"); } # Likely the programme's sub-title if not an episode number elsif ($episodenumber->as_text() =~ /\w+/) { $episodenumber = $episodenumber->as_text(); $prog{'sub-title'} = [ [ encode_and_trim( $episodenumber ), $LANG ] ]; debug_print(" Found programme sub-title '" . $episodenumber . "'"); } } else { debug_print(" No episode numbering found"); } my $category = $programme->look_down('_tag', 'p', 'class', 'category'); if ($category and $category->as_text() =~ /\w+/) { $category = trim($category->as_text()); $prog{category} = [ [ encode_and_trim($category), $LANG ] ]; debug_print(" Found programme genre '" . $category . "'"); } else { debug_print(" No category found"); } my $synopsis = $programme->look_down('_tag', 'p', 'class', 'synopsis'); if ($synopsis and $synopsis->as_text() =~ /\w+/) { $synopsis = trim($synopsis->as_text()); $prog{desc} = [ [ encode_and_trim($synopsis), $LANG ] ]; debug_print(" Found programme short synopsis '" . $synopsis . "'"); } else { debug_print(" No synopsis found"); } my $rating = $programme->look_down('_tag', 'span', 'class', 'pastille csa'); if ($rating and trim($rating->as_text()) =~ /^(-(?:10|12|16|18))$/) { $rating = $1;; $prog{rating} = [ [ $rating, "CSA" ] ]; debug_print(" Found programme rating '" . $rating . "'"); } else { debug_print(" No rating found"); } my $thumbnail = $programme->look_down('_tag', 'img', 'class', 'thumbnail'); if ($thumbnail) { my $url = $thumbnail->attr('src'); push @{$prog{icon}}, {src => $url}; debug_print(" Found programme icon: '" . $url . "'"); } if ($opt->{'slow'} && $prog_page) { process_programme_page(\%prog); } return \%prog; } sub process_programme_page { my $prog = shift; my $prog_page = $prog->{'_prog_page'}; debug_print("process_programme_page(): $prog_page"); # Get the page's tree my $t = get_nice_tree($prog_page, undef, undef, undef); if (not defined $t) { debug_print(" *** Error: Could not get tree for '" . $prog_page . "' ***"); return $prog; } # constrain searching to main content pane my $c = $t->look_down('_tag', 'div', 'class', qr/content left/); if (not defined $c) { debug_print(" *** Error: Could not get programme info for '" . $prog_page . "' ***"); return $prog; } my $prog_info = $c->look_down('_tag', 'ul', 'class', 'list-fiche'); if ($prog_info) { my @info_fields = $prog_info->look_down('_tag', 'li'); if (@info_fields) { # each info field comprises 2 <span> tags giving a key and a value foreach my $info_field (@info_fields) { if ($info_field->as_text() =~ /^Titre : (.+)$/) { my $episode_name = trim($1); $prog->{'sub-title'} = [ [ encode_and_trim( $episode_name ), $LANG ] ]; debug_print(" Found programme sub-title: " . $episode_name); } elsif ($info_field->as_text() =~ /de production : (\d{4})$/) { my $date_created = trim($1); $prog->{'date'} = $date_created; debug_print(" Found production year: " . $date_created); } elsif ($info_field->as_text() =~ /^Genre : (.+)$/) { my $genre = trim($1); my $subgenre; ($genre, $subgenre) = split(/,|\s-\s/, $genre); if (defined $genre && $genre =~ /\w+/) { $genre = trim($genre); debug_print(" Found programme genre: " . $genre); if (defined $subgenre && $subgenre =~ /\w+/) { $subgenre = trim($subgenre); debug_print(" Found programme sub-genre: " . $subgenre); $prog->{category} = [ [ encode_and_trim( $subgenre ), $LANG ], [ encode_and_trim( $genre ), $LANG ] ]; } else { $prog->{category} = [ [ encode_and_trim( $genre ), $LANG ] ]; } } } } } } # [2022-08-27] this no longer works - title-block element can now contain other things, and synopsis is now moved outside of title-block element # my $title_block = $c->look_down('_tag', 'div', 'class', qr/title-block/); # if ($title_block) { # # Remove <div> containing 'Synopsis' text # my $parent = $title_block->parent(); # $title_block->delete(); # # Process remaining text # my $synopsis = trim($parent->as_text()); # $prog->{desc} = [ [ encode_and_trim( $synopsis ), $LANG ] ]; # debug_print(" Found programme long synopsis: " . $synopsis); # } # my $synopsis_h = $c->look_down('_tag', 'h2', sub { $_[0]->as_text =~ m/\bsynopsis\b/i } ); if ($synopsis_h) { # Remove <h2> containing 'Synopsis' text my $container = $synopsis_h->look_up('_tag', 'div', 'class', 'section-fiche-program'); if ($container) { $synopsis_h->delete(); # Process remaining text my $synopsis = trim($container->as_text()); $prog->{desc} = [ [ encode_and_trim( $synopsis ), $LANG ] ] unless $synopsis =~ m/^\s*$/; debug_print(" Found programme long synopsis: " . $synopsis); } } # Casting information on the default programme information page is # typically limited to series. my $casting = $c->look_down('_tag', 'div', 'id', 'block-casting', 'class', 'block-casting'); if ($casting) { my @casting_titles; @casting_titles = $casting->look_down('_tag', 'h3', 'class', 'title'); # some page styles (fiche-emission) may use h4 instead unless (@casting_titles) { @casting_titles = $casting->look_down('_tag', 'h4', 'class', 'title'); } foreach my $ct (@casting_titles) { if ($ct->as_text() =~ /R.alisateur/) { my $parent = $ct->parent(); my @directors = $parent->look_down('_tag', 'span', 'class', 'name'); foreach my $director (@directors) { $director = trim($director->as_text()); push @{$prog->{credits}{director}}, encode_and_trim( $director ); debug_print(" Found programme director: " . $director); } } elsif ($ct->as_text() =~ /Sc.nario/) { my $parent = $ct->parent(); my @writers = $parent->look_down('_tag', 'span', 'class', 'name'); foreach my $writer (@writers) { $writer = trim($writer->as_text()); push @{$prog->{credits}{writer}}, encode_and_trim( $writer ); debug_print(" Found programme writer: " . $writer); } } elsif ($ct->as_text() =~ /Acteurs et actrices/) { my $parent = $ct->parent(); my @actors = $parent->look_down('_tag', 'span', 'class', 'name'); foreach my $actor (@actors) { $actor = trim($actor->as_text()); push @{$prog->{credits}{actor}}, encode_and_trim( $actor ); debug_print(" Found programme actor: " . $actor); } } } } $c->delete(); undef $c; $t->delete(); undef $t; return $prog; } sub update_programme_stop_times { my $programmes = shift; # Stop at penultimate programme foreach my $i (0 .. (scalar @{$programmes} -2)) { my $p0 = $programmes->[$i]; my $p0_stop = get_datetime_from_start_duration($p0); my $p0_title = decode($ENCODING, $p0->{title}[0][0]); my $p1 = $programmes->[$i+1]; my $p1_start = get_datetime_from_xmltv_time($p1->{start}); my $p1_title = decode($ENCODING, $p1->{title}[0][0]); if ($p1_start == $p0_stop) { # "This is good..." debug_print(" No gap detected between '$p0_title' and '$p1_title'"); $p0->{stop} = $p1->{start}; } elsif ($p1_start < $p0_stop) { # Trust the published start time if ($opt->{debug}) { my $dur = $p0_stop - $p1_start; my $gap = $dur->minutes; debug_print(" Calculated stop time for '$p0_title' is $gap minutes later than start time of '$p1_title'"); } $p0->{stop} = $p1->{start}; } elsif ($p1_start > $p0_stop) { my $dur = $p1_start - $p0_stop; my $gap = $dur->minutes; if ($gap <= 10) { # For small gaps less than 10 minutes, use the next # programme's start time debug_print(" There is a small gap of $gap minutes between '$p0_title' and '$p1_title'"); $p0->{stop} = $p1->{start}; } else { # Otherwise, use the current programme's duration debug_print(" There is a large gap of $gap minutes between '$p0_title' and '$p1_title'"); $p0->{stop} = get_xmltv_time_from_datetime($p0_stop); } } } # Handle final programme separately: add duration to start time my $p_last = $programmes->[-1]; my $p_last_stop = get_datetime_from_start_duration($p_last); $p_last->{stop} = get_xmltv_time_from_datetime($p_last_stop); # Return updates listref of programmes return $programmes; } sub get_datetime_from_start_duration { my $prog = shift; my $dt_start = get_datetime_from_xmltv_time($prog->{'start'}); my $dt_duration = DateTime::Duration->new( minutes => $prog->{'_duration_mins'}); return $dt_start + $dt_duration; } sub get_datetime_from_xmltv_time { my $date_string = shift; my ($y, $m, $d, $hh, $mm, $ss) = $date_string =~ /^(\d{4})(\d\d)(\d\d)(\d\d)(\d\d)(\d\d)/; my $dt = DateTime->new( year => $y, month => $m, day => $d, hour => $hh, minute => $mm, second => $ss, time_zone => 'Europe/Paris', ); return $dt; } sub get_xmltv_time_from_datetime { my $dt = shift; return utc_offset($dt->strftime("%Y%m%d%H%M%S"), "+0100"); } sub get_date_today { return DateTime->now( time_zone => 'Europe/Paris' ); } sub get_date_today_with_offset { my $offset = DateTime::Duration->new( days => shift ); return get_date_today() + $offset; } sub write_xmltv_footer { my $writer = shift; debug_print("\nWriting XMLTV footer\n"); $writer->end; } sub validate_options { if ($opt->{quiet} && $opt->{debug}) { die "Error: You cannot specify --quiet with --debug, exiting"; } if ($opt->{offset} < 0 or $opt->{offset} > 13) { print STDERR "Invalid value for --offset. Please adjust to a value in range 0-13\n"; exit 1; } if ($opt->{days} < 1 or $opt->{days} > 14) { print STDERR "Invalid value for --days. Please adjust to a value in range 1-14\n"; exit 1; } my $max_days_after_offset = 14 - $opt->{offset}; if ($opt->{days} > $max_days_after_offset) { print STDERR "Cannot retrieve more than $max_days_after_offset days of listings\n" . "Please adjust --days and/or --offset.\n"; exit 1; } } sub validate_config { my @required_keys = ("cachedir", "bouquet", "channel"); foreach my $key (@required_keys) { if (! defined $conf->{$key}) { print STDERR "No configured $key found in config file ($opt->{'config-file'})\n"; print STDERR "Please reconfigure the grabber ($GRABBER_NAME --configure)\n"; exit 1; } } } sub initialise_cache { init_cachedir( $conf->{cachedir}->[0] ); HTTP::Cache::Transparent::init( { 'BasePath' => $conf->{cachedir}->[0], 'MaxAge' => 24, 'NoUpdate' => 60*60*3, 'Verbose' => $opt->{debug}, } ); } sub init_cachedir { my $path = shift; if (! -d $path) { mkpath($path) or die "Failed to create cache-directory $path: $@"; } debug_print("init_cachedir: cache directory created at $path"); } sub get_default_dir { my $winhome = $ENV{HOMEDRIVE} . $ENV{HOMEPATH} if (defined $ENV{HOMEDRIVE} and defined $ENV{HOMEPATH}); my $home = $ENV{HOME} || $winhome || "."; debug_print("get_default_dir: home directory found at $home"); return $home; } sub get_default_cachedir { my $cachedir = get_default_dir() . "/.xmltv/cache"; debug_print("get_default_cachedir: default cache directory set to $cachedir"); return $cachedir; } sub print_version_info { debug_print("Program/library version information:\n"); debug_print("XMLTV library version: $XMLTV::VERSION"); debug_print(" grabber version: $GRABBER_VERSION"); debug_print(" libwww-perl version: $LWP::VERSION\n"); } sub encode_and_trim { my $s = shift; $s = trim($s); $s = encode( $ENCODING, $s ); return $s; } sub trim { for (my $s = shift) { s/^\s*//; s/\s*$//; return $s; } } sub debug_print { if ($opt->{debug}) { my ($msg) = shift; print STDERR encode_and_trim( $msg ) . "\n"; } } __END__ =pod =encoding utf8 =head1 NAME tv_grab_fr - Grab TV listings for France (Télé Star). =head1 SYNOPSIS To configure: tv_grab_fr --configure [--config-file FILE] [--gui OPTION] To list available channels: tv_grab_fr --list-channels To grab listings: tv_grab_fr [--config-file FILE] [--output FILE] [--days N] [--offset N] [--slow] [--quiet | --debug] To show capabilities: tv_grab_fr --capabilities To show version: tv_grab_fr --version To display help: tv_grab_fr --help =head1 DESCRIPTION Output TV listings for many channels available in France (Orange, Free, cable/ADSL/satellite, Canal+ Sat). The data comes from Télé Star (telestar.fr). The default is to grab 14 days. B<--configure> Choose which bouquet/channels to grab listings data for. B<--list-channels> List available channels. B<--config-file FILE> Use FILE as config file instead of the default config file. This allows for different config files for different applications. B<--gui OPTION> Use this option to enable a graphical interface to be used. OPTION may be 'Tk', or left blank for the best available choice. Additional allowed values of OPTION are 'Term' for normal terminal output (default) and 'TermNoProgressBar' to disable the use of Term::ProgressBar. B<--output FILE> Write to FILE rather than standard output. B<--days N> Grab N days (default: 14) starting from today. B<--offset N> Start grabbing N days from today, rather than starting today. B<--slow> Download additional information (e.g. longer description, cast details) for each programme, where available. This option significantly slows down the grabber and is disabled by default. B<--quiet> Suppress the progress messages normally written to standard error. B<--debug> Provide additional debugging messages during processing. B<--capabilities> Show which capabilities the grabber supports. For more information, see L<http://wiki.xmltv.org/index.php/XmltvCapabilities>. B<--version> Show the version of the grabber. B<--help> Print a help message and exit. =head1 SEE ALSO L<xmltv(5)> =head1 AUTHOR The current tv_grab_fr script was rewritten by Nick Morrott, knowledgejunkie at gmail dot com, to support the new telestar.fr site. =cut ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/huro/������������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0014552�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/huro/catmap.hu���������������������������������������������������������������������0000664�0000000�0000000�00000002673�15000742332�0016365�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Hungarian %catmap translations by Balazs Molnar <mbdev@freemail.hu> ################################################################################ Action:Akci:(akci|kaland|katasztrfa|krimi) Adult:Felntteknek:(felntt|horror|thriller|sz?ex|erotikus|porn[o]) Animals:Termszet:(termszet|llat|nvny) Art_Music:Zene:(zen[e]|muzsik[ua]|koncert) Business:zleti:(zleti|piac) Children:Gyermek:(gyermek|mesefilm|rajzfilm|bbfilm|animcis) Comedy:Vgjtk:(vgjtk|komdia|szrakoztat) Crime_Mystery:Kirmi:(kirmi) Documentary:Dokumentalista:(dokument|dok\. (film|sor\.)) Drama:Drma:(drma) Educational:Oktats:(oktats|iskola|lecke|ismerett(\.|erjeszt)|ism\. sor|portrf) Food:tel-Ital:(\btel|\bital|gasztron|recept) Game:Jtk:(\bjtk\b|kvz) Health_Medical:Orvosi:(orvosi) History:Trtnelmi:(trtnelmi|dokumentum) HowTo:Hogyan:(hogyan|miknt) Horror:Horror:(horror|thriller) Misc:Vegyes:(vegyes) News:Hrek:(hr|kzszolglati) Reality:Valsg:(valsg) Romance:Romantika:(romantik) Science_Nature:Tudomnyos:(tudomnyos|akadmi) SciFi_Fantasy:Sci-Fi:(sci.?fi|fantasztikus) Shopping:Vsrls:(vsrls|shop) Soaps:Szappanopera:(szappan|brazil|venezuela|venez\.) Spiritual:Spiritulis:(spiritu) Sports:Sport:(sport|labda|tenisz|verseny|s.?lesikls|korcsolya|biatlon|jgkorong) Talk:Talk-Show:(talk|show|magazin|riport) Travel:Utazs:(utazs|pihens) War:Hbor:(hbor) Western:Western:(western|vad.?nyugat|wild.?west) Unknown:Ismeretlen:(ismeretlen) ���������������������������������������������������������������������xmltv-1.4.0/grab/huro/jobmap������������������������������������������������������������������������0000664�0000000�0000000�00000006207�15000742332�0015752�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Hungarian %jobmap translations by # Gbor Ziegler <ziegler@alpha.tmit.bme.hu> # # FIXME seems we've got to extend the DTD # r:writer rendez:director operatr: # no field for 'cinematographer/cameraman' producer:producer executive producer:producer vg: # no field for 'editor' in DTD zene: # no field for "music" in DTD dszlettervez: # no field for "set-decorator" in DTD jelmeztervez: # no field for "costume-designer" ltvnytervez: # "visual-effects designer" msorvezet:presenter # no field for "host" in DTD forgatknyvr:writer # script-writer zeneszerz: # composer trs-producer:producer # co-producer trsproducer:producer # co-producer ltvny: # no field for "scene" in DTD vide: # no field for "video" in DTD interj: # no field for "interview" in DTD dramaturg: # no field for "casting"(?) in DTD gyrtsvezet:producer # not really equals to producer tlet: # no field for "idea" in DTD kzremkd:guest # no field for "participant" in DTD hang: # no field for "sound" in DTD hang effektek: # no field for "sound effects" in DTD make up: # no field for "make up" in DTD koreogrfus: # no field for "coreographer" in DTD karigazgat: # no field for "chorus director" in DTD szerkeszt:director # programme-editor szerkeszt-riporter:director # prog.-editor-riporter hangmrnk: # no field for "sound-engineer" in DTD vizulis effektek: # visual effects" in DTD dszlet: # interior-set adaptci: # adaptation narrtor:commentator # narrator jelmez: # costume alkottrs: # creative partner szerepl:actor # player/participant szerepl(k):actor # player/participant szakmai tancsad: # prof. consultant/specialist alaptlet szerz: # author for original idea zensz: # musician ruha: # costumes dalszveg szerz: # lyrics author zenei szerkeszt: # musical editor maszk: # mask produkcis vezet: # leader of production works animci: # animation krus: # chorus trsrendez:director #co-director, or sth. like that szereposzts: # Romanian %jobmap translations by # Lucian Muresan <lucianm@users.sourceforge.net> # regizor:director director artistic:director scriitor:writer scenarist:writer adaptarea:adapter actor:actor distribu?ia:actor personaj:actor role invitat special:guest productor:producer productor executiv:producer co-productor:producer monteur:editor operator: scenograf: compozitor:composer imagine: costume: director de imagine: muzica:composer sunet: decor: personaj: machiaj: idee: adaptare: mti: masti: # costly masks / make up, etc. efecte vizuale: efecte speciale: prezentator:presenter comentator:commentator narator: # narrator prezentator tv:presenter co-producer:producer participa: # with participation of particip: # with participation of director artistic:director distribuia:actor # Czech %jobmap translations # reie:director scnr:writer kamera: # camera scnograf: # set designer kostmy: # costumes hudba:composer produkce:producer efekty: # special effects strih: # editor herec:actor # Slovak %jobmap translations # ria:director scenr:writer kamera: # camera hudba:composer # music produkcia:producer prava: # editing herec:actor # Further %jobmap's seen - not yet translated by # any of the contributors # mvszeti vezet: �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/huro/test.conf���������������������������������������������������������������������0000664�0000000�0000000�00000004272�15000742332�0016405�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������country hu # Hungary channel 001 m1 channel 002 m2 channel 003 TV2 #channel 004 CNN #channel 005 RTL Klub #channel 006 DUNA Televzi #channel 007 ARD #channel 008 HBO #channel 009 Spektrum #channel 010 hrTV #channel 011 VIVA #channel 012 Sailing Channel #channel 013 Budapest TV #channel 014 National Geographic #channel 015 Magyar ATV #channel 016 Minimax #channel 017 Soundtrack Channel #channel 019 Spice Platinum #channel 020 TV Deko #channel 021 VIASAT3 #channel 022 VH1 Classic #channel 023 TV5 #channel 024 Sexview #channel 025 3sat #channel 026 ORF1 #channel 027 ORF2 #channel 028 RTL II #channel 029 VOX #channel 030 Discovery Civilisation #channel 031 BBC Prime #channel 032 film+ #channel 033 Video Italia #channel 034 Trace TV #channel 035 Jetix #channel 036 ESPN Classic #channel 037 Eurosport 2 #channel 041 Nickelodeon #channel 042 AXN #channel 044 Sport2 #channel 045 Irisz TV #channel 046 TV Paprika #channel 047 Cinemax #channel 048 Echo TV #channel 049 MTV2 #channel 050 Wine TV #channel 051 TCM #channel 052 Performance #channel 053 Hustler TV #channel 054 XXXtreme #channel 055 XXL #channel 056 AB Moteurs #channel 057 Chasse et Peche #channel 058 Motors #channel 059 HBO 2 #channel 060 Cinemax 2 #channel 063 Fashion TV #channel 064 MCM Top #channel 065 Viasat Explorer #channel 066 Viasat History #channel 067 A+ #channel 068 CNBC #channel 069 Baby TV #channel 071 Euronews #channel 072 Aljazeera #channel 073 RTR Planeta #channel 074 Pro TV International #channel 075 ARTE #channel 076 RAI UNO #channel 077 Boomerang #channel 078 RTP I. #channel 079 Cool #channel 080 Mezzo #channel 082 Animal Planet #channel 083 Discovery #channel 084 Travel Channel #channel 085 Discovery Science #channel 086 Discovery Travel & Living #channel 087 Hallmark #channel 089 Romantica #channel 090 Sport1 #channel 091 Cartoon Network / TCM #channel 092 Reality TV #channel 093 Europa Europa #channel 094 Eurosport #channel 095 Filmmzeum #channel 096 fix.tv #channel 097 Club #channel 098 Extreme Sports #channel 099 VH1 #channel 100 MTV Europe #channel 114 Fnix TV #channel 115 PAX Tv #channel 116 MTV Base #channel 117 MTV Hits #channel 118 RTL #channel 119 SAT.1 #channel 120 Pro7 #channel 122 Hlzat TV #channel 124 TVE ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/huro/tv_grab_huro.PL���������������������������������������������������������������0000664�0000000�0000000�00000001351�15000742332�0017470�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Generate tv_grab_huro from tv_grab_huro.in. This is to # set the path to the files in /usr/local/share/xmltv or wherever. # # The second argument is the share directory for the final # installation. use IO::File; my $out = shift @ARGV; die "no output file given" if not defined $out; my $share_dir = shift @ARGV; die "no final share/ location given" if not defined $share_dir; my $in = 'grab/huro/tv_grab_huro.in'; my $in_fh = new IO::File "< $in" or die "cannot read $in: $!"; my $out_fh = new IO::File "> $out" or die "cannot write to $out: $!"; my $seen = 0; # example line: # my $SHARE_DIR = undef; while (<$in_fh>) { print $out_fh $_; } close $out_fh or die "cannot close $out: $!"; close $in_fh or die "cannot close $in: $!"; ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/huro/tv_grab_huro.in���������������������������������������������������������������0000775�0000000�0000000�00000256715�15000742332�0017606�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w # vi:noet:ts=4 #------------------------------------------------------------------------------- # documentation #------------------------------------------------------------------------------- =pod =head1 NAME tv_grab_huro - Grab TV listings for Hungary. =head1 SYNOPSIS tv_grab_huro --help tv_grab_huro [--config-file FILE] --configure [--gui GUITYPE] tv_grab_huro [--config-file FILE] [--output FILE] [--days N] [--offset N] [--slow] [--get-full-description] [--max-desc-length LENGTH] [--icons | (--local-icons DIRECTORY [--no-fetch-icons])] [--gui GUITYPE] [--quiet] tv_grab_huro --list-channels [--icons | (--local-icons DIRECTORY [--no-fetch-icons])] tv_grab_huro --capabilities tv_grab_huro --version =head1 DESCRIPTION Output TV listings for several channels available in Hungary. The grabber relies on parsing HTML so it might stop working at any time. First run B<tv_grab_huro --configure> to choose, which channels you want to download. Then running B<tv_grab_huro> with no arguments will output listings in XML format to standard output. B<--configure> Prompt for which channels, and write the configuration file. B<--config-file FILE> Set the name of the configuration file, the default is B<~/.xmltv/tv_grab_huro.conf>. This is the file written by B<--configure> and read when grabbing. B<--gui GUITYPE> Use this option to enable a graphical interface to be used. OPTION may be 'Tk', or left blank for the best available choice. Additional allowed values of OPTION are 'Term' for normal terminal output (default) and 'TermNoProgressBar' to disable the use of Term::ProgressBar. B<--output FILE> Write to FILE rather than standard output. B<--days N> Grab N days. The default is eight. B<--offset N> Start N days in the future. The default is to start from today. B<--quiet> Suppress the progress messages normally written to standard error. B<--slow> Enables long strategy run: port.hu publishes only some (vital) information on the actual listing pages, the rest is shown in a separate popup window. If you'd like to parse the data from these popups as well, supply this flag. But consider that the grab process takes much longer when doing so, since many more web pages have to be retrieved. B<--get-full-description> This is quite like B<--slow> but doesn't always download data from popup window. Instead this is only requested if description in overview is truncated. B<--list-channels> Write output giving <channel> elements for every channel available (ignoring the config file), but no programmes. B<--capabilities> Show which capabilities the grabber supports. For more information, see L<http://wiki.xmltv.org/index.php/XmltvCapabilities> B<--version> Show the version of the grabber. B<--icons> and B<--local-icons DIRECTORY> get the URL for channel-logos together with the channel-list. With B<--icons> specified the logos(images) will be not fetched just their URL (http://...) will be written in the output XML. If called with B<--local-icons>, the generated URL (file://...) will point to the the local directory DIRECTORY and all channel logos will be grabbed and saved under this place. Use B<--no-fetch-icons> option to disable the icon fetching. B<--max-desc-length LENGTH> can be used to maximize the length of the grabbed program long description. This can be useful if you have a viewer program (using this xmltv output), which can not be display userfriendly the description if it is more then LENGTH character. B<--help> Print a help message and exit. =head1 SEE ALSO L<xmltv(5)>. =head1 AUTHOR Attila Szekeres and Zsolt Varga. Based on tv_grab_fi by Matti Airas. Heavily patched and earlier maintained by Stefan siegl <stesie@brokenpipe.de>, reworked and now maintained by Balazs Molnar <mbdev@freemail.hu>. =head1 BUGS The data source does not include full channels information and the channels are identified by short names rather than the RFC2838 form recommended by the XMLTV DTD. This grabber no longer supports Romanian listings as of 2024. See L<https://github.com/XMLTV/xmltv/issues/227> =cut #------------------------------------------------------------------------------- # initializations #------------------------------------------------------------------------------- use utf8; use strict; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use XMLTV::Capabilities qw/baseline manualconfig cache/; use XMLTV::Description 'Hungary'; use XMLTV::Supplement qw/GetSupplement/; use Getopt::Long; use Date::Manip; use Cwd; use HTML::TreeBuilder; use HTML::Entities; # parse entities use IO::File; use File::Basename; use JSON; use Encode; use Time::Piece (); use Time::Seconds; use XMLTV::Memoize; use XMLTV::ProgressBar; use XMLTV::Ask; use XMLTV::DST; use XMLTV::Get_nice; use XMLTV::Mode; use XMLTV::Config_file; use XMLTV::Date; use XMLTV::Gunzip; # Todo: perhaps we should internationalize messages and docs? use XMLTV::Usage <<"END" $0: get Hungarian television listings in XMLTV format To configure: $0 --configure [--config-file FILE] [--gui GUITYPE] To grab listings: $0 [--config-file FILE] [--output FILE] [--days N] [--offset N] [--slow] [--get-full-description] [--max-desc-length LENGTH] [--icons | (--local-icons DIRECTORY [--no-fetch-icons])] [--gui GUITYPE] [--quiet] To list channels: $0 --list-channels [--icons | (--local-icons DIRECTORY [--no-fetch-icons])] To show capabilities: $0 --capabilities To show version: $0 --version END ; # ${Log::TraceMessages::On} = 1; # to switch TRACE in remove the comment from prev. line # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } } my ($opt_days, $opt_offset, $opt_help, $opt_output, $opt_configure, $opt_config_file, $opt_gui, $opt_quiet, $opt_list_channels, $opt_loc, $opt_slow, $opt_full_desc, $opt_local_icons, $opt_icons, $opt_no_fetch_icons, $opt_max_desc_length, $opt_worker_times, $opt_now); our $FETCHOFFSET = 0; our ($DAYSPERPAGE, $TZ, $COUNTRY, $CONFIG_FILE, $WNAME, $WSTIME); our (%CATMAP, %JOBMAP, %CHANNELS, %WTIMES); #our %COUNTRIES = (Hungary => [ 'hu', '+0100' ], # Czech => [ 'cz', '+0100' ], # Romania => [ 'ro', '+0200' ], # Slovakia => [ 'sk', '+0100' ]); our %COUNTRIES = (Hungary => [ 'hu', '+0100' ]); our %WORDS = ( cz => { episode => "Epizoda", minute => "minut", links => "Linky" }, hu => { episode => "rész", minute => "perc", links => "linkek" }, ro => { episode => "episodul", # patch #84 minute => "minute", links => "Linkuri" }, sk => { episode => "Epizóda", minute => "minút", links => "Linky" } ) ; our $DEFAULT_ENCODING = 'ISO-8859-2'; our $rating_baseurl = 'http://media.port-network.com/page_elements/'; our %AGE_LIMITS = ( # Todo: insert cz & sk translations 'ageLimitList-1' => [{'hu' => 'korhatárra tekintet nélkül megtekinthető', 'ro' => 'Audienţă generală'}, 'nmhh_akk/mobil_35x35/0_age_icon_mobil.png'], 'ageLimitList-5' => [{'hu' => '16 éven aluliak számára nem ajánlott', 'ro' => 'Acest program este interzis minorilor sub 16 ani'}, 'nmhh_akk/mobil_35x35/16_age_icon_mobil.png'], 'ageLimitList-3' => [{'hu' => '12 éven aluliak számára a megtekintése nagykorú felügyelete mellett ajánlott', 'ro' => 'Acest program este interzis minorilor sub 12 ani'}, 'nmhh_akk/mobil_35x35/12_age_icon_mobil.png'], 'ageLimitList-4' => [{'hu' => '14 éven alul nem ajánlott', 'ro' => 'Acest program este interzis minorilor sub 14 ani'}, 'm_14_age_mini_pix.png'], 'ageLimitList-6' => [{'hu' => '18 éven aluliak számára nem ajánlott', 'ro' => 'Acest program este interzis minorilor sub 18 ani'}, 'nmhh_akk/mobil_35x35/18_age_icon_mobil.png'], 'ageLimitList-8' => [{'hu' => '7 éven aluliak számára nem ajánlott', 'ro' => 'Acest program este interzis minorilor sub 7 ani'}, 'm_7_age_mini_pix.png'], 'ageLimitList-10' => [{'hu' => '6 éven aluliak számára nem ajánlott', 'ro' => 'Acest program este interzis minorilor sub 6 ani'}, 'nmhh_akk/mobil_35x35/6_age_icon_mobil.png'], 'ageLimitList-2' => [{'hu' => 'szülői engedéllyel', 'ro' => 'Recomandat acordul părinţilor'}, 'm_parental_guidance_mini_pix_hu.png'], 'ageLimitList-7' => [{'hu' => '15 éven aluliak számára nem ajánlott', 'ro' => 'Acest program este interzis minorilor sub 15 ani'}, 'm_15_age_mini_pix.png']); our %PROGRAM_CATEGORIES = ( # Todo: insert cz & sk translations 'tvEventType-0' => {'hu' => 'egyéb', 'ro' => 'nedefinit'}, 'tvEventType-11' => {'hu' => 'vallási műsor', 'ro' => 'emisiune religioasă'}, 'tvEventType-4' => {'hu' => 'gyermek műsor', 'ro' => 'copii'}, 'tvEventType-10' => {'hu' => 'dokumentumfilm', 'ro' => 'documentar'}, 'tvEventType-12' => {'hu' => 'filmsorozat', 'ro' => 'serial'}, 'tvEventType-13' => {'hu' => 'szabadidős műsor', 'ro' => 'family'}, 'tvEventType-14' => {'hu' => 'zenei műsor', 'ro' => 'muzica'}, 'tvEventType-15' => {'hu' => 'hírműsor', 'ro' => 'ştiri'}, 'tvEventType-1' => {'hu' => 'sportműsor', 'ro' => 'sport'}, 'tvEventType-3' => {'hu' => 'hír-, politikai műsor', 'ro' => 'tv show'}, 'tvEventType-7' => {'hu' => 'művészeti műsor', 'ro' => 'tv show'}, 'tvEventType-8' => {'hu' => 'ismeretterjesztő műsor', 'ro' => 'stiinta'}, 'tvEventType-9' => {'hu' => 'szappanopera', 'ro' => 'telenovelă'}, 'tvEventType-18' => {'hu' => 'gasztronómiai műsor', 'ro' => 'gastro'}, 'tvEventType-20' => {'hu' => 'életstílus', 'ro' => 'life style'}, 'tvEventType-2' => {'hu' => 'film', 'ro' => 'film'}, 'tvEventType-5' => {'hu' => 'szórakoztató műsor', 'ro' => 'reality show'}, 'tvEventType-6' => {'hu' => 'szolgáltató műsor', 'ro' => 'tv show'}, 'tvEventType-16' => {'hu' => 'divat', 'ro' => 'modă'}, 'tvEventType-17' => {'hu' => 'felnőtt', 'ro' => 'pentru adulţi'}, 'tvEventType-19' => {'hu' => 'reality', 'ro' => 'reality-show'}); sub domain(); sub xid( $ ); sub xhead(); sub process_table( $$$$ ); sub process_json( $$$$ ); sub parse_short_desc ( $ ); sub get_channels( ;$ ); sub get_channels_json( ;$ ); sub get_infourl_data( $$ ); sub get_infourl_data_json( $$ ); sub add_person ( $$$ ); sub extract_episode( $ ); sub grab_icon( $ ); sub get_channel_urls( $ ); sub worker( $ ); sub showworkers(); sub get_all_text ( $ ); # Get options, including undocumented --cache option. XMLTV::Memoize::check_argv('XMLTV::Get_nice::get_nice_aux'); #------------------------------------------------------------------------------- # domain #------------------------------------------------------------------------------- # desc : construct the www host's hostname's domain part # arguments : none # returns : "port.hu", "port.ro" based on country id #------------------------------------------------------------------------------- sub domain() { "port.$COUNTRY" }; #------------------------------------------------------------------------------- # xid #------------------------------------------------------------------------------- # desc : turn a site channel id into an XMLTV id # arguments : 1- port site channel id: 005 (f.e) # returns : port.hu, port.ro based on country id #------------------------------------------------------------------------------- sub xid( $ ) { my $id = shift; return "$id." . domain(); } #------------------------------------------------------------------------------- # xhead #------------------------------------------------------------------------------- # desc : provide the head of the XML output # arguments : none # returns : hash, containing the info's XML tags #------------------------------------------------------------------------------- sub xhead() { my $d = &domain; return { 'source-info-url' => "https://www.$d/", 'source-data-url' => "https://www.$d/tv/", 'generator-info-name' => 'XMLTV', 'generator-info-url' => 'http://xmltv.org/', }; } # function to parse all of text data of a HTML element and his childs: #------------------------------------------------------------------------------- # get_all_text #------------------------------------------------------------------------------- # desc : parse all of text data of a HTML element and in his childs # arguments : 1- HTML:Element object from here will be started the downhill # returns : @arrays of founded text elements #------------------------------------------------------------------------------- sub get_all_text( $ ) { my @tmplines; my $e = $_[0]; if (ref $e) { foreach my $c ($e->content_list) { push @tmplines, get_all_text($c); } } else { push @tmplines, $e; } return @tmplines; } #------------------------------------------------------------------------------ # process_table #------------------------------------------------------------------------------ # desc : fetch a URL and process it # arguments : 1- Date::Manip object, basedate/startdate of grabbing (e.g. 20060205) # 2- xmltv id of channel # 3- site id of channel # 4- dayindex of the requested page on port.hu|ro # returns : list of the programme hashes to write #------------------------------------------------------------------------------ sub process_table( $$$$ ) { my ($basedate, $ch_xmltv_id, $ch_port_id, $baseday) = @_; $basedate = UnixDate(DateCalc(parse_date($basedate),"- 1 day"), '%Q'); my $days_to_request = $DAYSPERPAGE+1; # We have to request at minimum of four days $days_to_request = 4 if ($days_to_request<4); my $basedateday = UnixDate(parse_date($basedate), '%e'); my $urlfmt = "http://www." . domain() . "/pls/tv/".($COUNTRY eq 'hu' || $COUNTRY eq 'ro' ? 'old' : '')."tv.channel?i_ch=$ch_port_id" . "&i_date=%d&i_xday=".$days_to_request."&i_where=1"; # bug #501 my $url = "$urlfmt"; my $currday = $baseday + $FETCHOFFSET - 1; my ($tree, $body); local $SIG{__WARN__} = sub { warn "$url: $_[0]"; }; # make (maximum) two loop to fetch program data: # # if the grabber runs in eraly hours (e.g. 01:00, 02:00) port.hu returns # the yesterdays's program as today's program... so we have to check it, # example.hu: <span class="ctxt">Péntek (február 27.)</span> # example.ro: <span class="ctxt">Duminic\u0103 (26 februarie)</span> # if this failes, we construct the previous (or next?) day's url foreach (1, 2) { $url = sprintf($urlfmt, $basedate); t "fetching url: $url"; worker("base-downloading"); $XMLTV::Get_nice::FailOnError = 1; my $data=get_nice($url); # strip links to bet-at-home.com # bug #447 $data =~ s|\x0A||g; # strip new line $data =~ s|<span class=\"spons_link.*?</span>||g; $data =~ s|<script type=\"text\/javascript\">.*?</script>||g; # strip links to divido.hu and provideo.ro $data =~ s|<a onclick=\"loggin.*?</a>||g; $tree = HTML::TreeBuilder->new_from_content($data) or die "could not fetch/parse $url (progamtable)\n"; worker("base-parsing"); $body = $tree->look_down("_tag"=>"body"); my @daysonpage = () ; foreach ($body->look_down("_tag"=>"p", "class" => "date_box")) { $_ = $_->as_text(); if ((($COUNTRY eq 'hu') && (m/^\s*\S+\s+\(\s*\S+\s+(\d+)\.\s*\)\s*$/)) || (($COUNTRY eq 'ro') && (m/^\s*\S+\s+\(\s*(\d+)\s+\S+\s*\)\s*$/)) || (($COUNTRY eq 'sk' || $COUNTRY eq 'cz') && (m/^\s*\S+\s+\(\s*(\d+)\.\s+\S+\s*\)\s*$/))) { t "added founded date of the month on the grabbed page: $1"; push @daysonpage, $1; } } if (@daysonpage) { # check date ... is the first founded date on the page the requested? last if ($basedateday == $daysonpage[0]); $body = undef; $tree->delete(); t "requested from $basedate, but port.$COUNTRY returned programs from wrong day: $daysonpage[0]"; if (UnixDate(DateCalc($basedate, "- 1 days"), '%e') == $daysonpage[0]) { # port.hu returned the programms from yesterday $FETCHOFFSET += 1 ; } elsif (UnixDate(DateCalc($basedate, "+ 1 days"), '%e') == $daysonpage[0]) { # port.hu returned the programms from tommorrow $FETCHOFFSET -= -1 ; } else { t "fetched HTML page do not contain 0, +1 or -1 day of the reuested one"; last; } t "global fetch offset was set to: $FETCHOFFSET"; } else { warn "no date data found on the fetched HTML page, trying to continue"; last; } } if (! defined($body)) { warn "Could not found the requested day's data on the grabbed HTML page, " . "some programs on $ch_xmltv_id channel will be not fetched."; return; } # the page consists of two major tables, where one holds the data # until 'bout 20 o'clock, the other, i.e. lower, one the evening program # the programs are in <tr> statements in more tables, this tables are # intermited with other tables, which contain images # - we need only the rows, which hold program data; because tables are # structured into other tables, we need only the most inner row # -> if it contains program data, and # -> has no child-tables in it # - there are tables, which will be the lower region delimiter # this is most inner and contains the vonal.gif, we will add this row # only as "lower region" string my @rows; my $empty_first_column; foreach ($body->look_down("_tag"=>"tr")) { if ($_->look_down("_tag"=>"img", "src" => "/tv/kep/vonal.gif")) { # ther is the "vonal.gif" in the table? t "+ most inner row containing vonal.gif found"; push @rows, "lower region"; } # look for line.gif as separator as well, per zolih@hotmail.com elsif ($_->look_down("_tag"=>"img", "src" => qr/\/line.gif$/)) { # ther is the "vonal.gif" in the table? t "+ most inner row containing vonal.gif found"; # in some cases the lower part of the first column is empty (See ticket #2890433) # We're trying to identify this case if (! scalar $_->look_down("_tag"=>"table")->look_down("_tag"=>"td", class=>"time_container")) { t "first column is empty"; $empty_first_column = 1; } push @rows, "lower region"; } # have childrens? if yes, skip this row next if ($_->look_down("_tag"=>"table")); # actual show is duplicated sometimes, and one of them is hidden next if ($_->parent()->attr("class") && $_->parent()->attr("class") eq "actual_showtime_hide"); if ($_->as_text() =~ /[012]?[0-9]:[0-5][0-9]/ || ($_->attr("class") && $_->attr("class") =~ /^event_/)) { t "+ most inner row containing programs found:"; t $_->as_text(); push @rows, $_; } } # walk through the rows to create programs # if you grab channel programs for 3 days in one fetch, you will have # the order of the rows is, how thay appear in the output, so the # 1-day, 2-day, 3-day, 1-night, 2-night-part1, 2-night-part2, 3-night my @programs; my $region = "upper"; my $startdate = $basedate; my $lasttime = 0; my $lasttimeHHMM = ""; # we need all the rows, because this is a program record: foreach my $row (@rows) { # check whether if we are first in lower tables ... -> reset date if (! ref($row) && $row eq "lower region") { if ($region eq "upper" && @programs) { t "upper/lower delimiter found, setting date back to startdate ($basedate)"; $startdate = $basedate; $lasttime = 12 * 60; # bottom row start after 18 o'clock, this gives us some safety, if the current show indicator is around $currday = $baseday + $FETCHOFFSET - 1; $region = "lower"; # in some cases the lower part of the first column is empty (See ticket #2890433) # In this case we have to skip the first day if ($empty_first_column) { t "first column is empty, bumped to next day"; $currday++; $startdate = UnixDate(DateCalc(parse_date($startdate), "+ 1 day"), '%Q'); } } next; } my (@urls, %program); foreach my $col ($row->look_down("_tag"=>"td")) { # the column can hold following type of data: # begin time | title | long desc | url | category my $begin_time; # this matches the currently running programme only: if ($col->attr("colspan") && ($begin_time = $col->look_down("_tag"=>"p", "class"=>"begin_time" ))) { $_ = $begin_time->as_text(); } elsif ($begin_time = $col->look_down("_tag"=>"td", "class"=>"time_container")) { $_ = $begin_time->look_down("_tag"=>"div")->as_text(); } else { $_ = $col->as_text(); $_ =~ tr/\xA0/ /; $_ =~ s/^\s+//; $_ =~ s/\s+$//; } s/^\s+//;s/\s+$//; # port.hu makes sometimes empty td elements next if (! length); t "col contents as text:" . d $_; if (m/^([012]?[0-9]):([0-5][0-9])$/) { s/^Kb[.]//; # means 'approx' in Magyar s/^24:/00:/; my $time = $1 * 60 + $2; if ($time < $lasttime) { t "bumped to the the next day"; $currday++; $startdate = UnixDate(DateCalc(parse_date($startdate), "+ 1 day"), '%Q'); die if not defined $startdate; } $lasttime = $time; # Fix the time format to be suitable for sorting $program{time} = length($_) == 4 ? "0".$_ : $_ ; $program{startdate} = $startdate ; $lasttimeHHMM = $program{time}; } else { my @span; if (@span = $col->look_down("class"=>"btxt")) { $program{title} = $span[0]->as_text(); } elsif (@span = $col->look_down("class"=>"lbbtxt")) { $program{title} = $span[0]->as_text(); } else { warn "cannot found title: $startdate" ; } # add one space after the title, if there is none # ??? $program{title} = ' ' if $program{title} eq ''; # bug #408 my @tmp = get_all_text($col); $_ = join(' ', @tmp); s/ +/ /g; s/[^\w]*putbox\(\"[0-9][0-9]\"\)[\s\n\r]*//g; s/Megvásárolható (DVD[ ]?-n|VHS[ ]?-en)//g; # strip leading   (and other spaces) s/^[ \t\xA0]*//g; # turn left over   into normal ones # (also take care of tabs and multiple spaces while here) s/[ \t\xA0]+/ /g; $program{desc} = $_ if length($_); $program{day} = $currday; foreach my $a ($col->look_down("_tag"=>"a")) { push @urls, $a->attr(q(href)); } # support ticket #202 # get the rating if available # <img alt="(15)" title="nevhodné do 15ti let" class="age_limit_icon" src="http://media.port-network.com/page_elements/15_age_mini_pix.png"> if (my $img = $span[0]->parent()->look_down('_tag' => 'img', 'class' => 'age_limit_icon')) { t "age limit icon found"; my $rating = $img->attr('alt'); $rating =~ s/[\(\)]//g; # strip the brackets my $rating_icon = $img->attr('src'); t "age rating = $rating"; $program{rating} = [[ $rating, '', [{'src' => $rating_icon }] ]]; } } } # foreach $col # New type of row: contains only the time of the event # but not the actual program data. So in this case we try to # jump to the next row if (! $row->attr("class") || (not $row->attr("class") =~ /[\s]*event_/ )) { t "found line without event, the time is: $lasttimeHHMM"; next; } if (! $program{time} && $lasttimeHHMM) { # Fix the time format to be suitable for sorting $program{time} = $lasttimeHHMM; $program{startdate} = $startdate ; } # add all parsed info, as program{time, title, desc, category, date} $program{infourl} = \@urls if (@urls); parse_short_desc(\%program); t "pushing ".$program{title}; push @programs, \%program if (defined $program{time}) && (defined $program{title}); } # foreach $row $tree->delete; # get rid of HTML::TreeBuilder's in memory representation if (not @programs) { warn "no programs found, skipping\n"; return (); } # make a sort on programs, short compare function: cmp startdate, time # stringwise (this gives the same rsult as comapre datewise) # Date_Cmp(UnixDate($left->{time},'%H:%M'),UnixDate($right->{time},'%H:%M'); sub bytime { ($a->{startdate}.$a->{time}) cmp ($b->{startdate}.$b->{time}); } @programs = sort bytime @programs; t "programs in sorted order:"; t "program:" . d $_ foreach (@programs); my (@r, $prev); # assume lang == country my $lang = $COUNTRY; foreach my $program (@programs) { my $prog; $prog->{channel}=$ch_xmltv_id; $prog->{title}=[ [ $program->{title}, $lang ] ]; my $start=parse_local_date("$program->{startdate} $program->{time}", $TZ); my ($start_base, $start_tz) = @{date_to_local($start, $TZ)}; $prog->{start}=UnixDate($start_base, '%q') . " $start_tz"; $prog->{desc} = [[ $program->{desc}, $lang ]] if defined $program->{desc}; $prog->{category} = $program->{category} if (defined $program->{category}); $prog->{date} = $program->{date} if defined $program->{date}; $prog->{qw(episode-num)} = $program->{qw(episode-num)} if defined $program->{qw(episode-num)}; $prog->{length} = $program->{length} if defined $program->{length}; $prog->{rating} = $program->{rating} if defined $program->{rating}; # support #202 # Setting stop date for the previous programm # Last program in the grabbed list has no stop attribute, sorry. # Port.hu uses a virtual program as the last programme # anyway if ((defined($prev)) && $prev->{start} ne $prog->{start}) { $prev->{stop} = $prog->{start}; } # We skip those programs, that are out of the requested time frame if ($program->{day} > $opt_days+$opt_offset || $program->{day}<1+$opt_offset) { $prev = undef; next; } worker("slow-parsing"); foreach my $infourl (@{$program->{infourl}}) { # always read data from linked page (in --slow mode) # in --get-full-description mode read if description ends in '...' if ( ($opt_slow) || ($opt_full_desc && (defined $prog->{desc}) && ($prog->{desc})->[0]->[0] =~ m/\.\.\.$/) ) { get_infourl_data($prog, $infourl); } } worker("base-parsing"); push @r, $prog; if ((defined($prev)) && $prev->{start} eq $prog->{start}) { # starttime of previous and current programme is equal, # therefore use clumpidx to express relation my $clumps_num = 2; if (defined($r[-2]->{q(clumpidx)})) { # previous programme already has a clumpidx arg assigned. ($clumps_num) = $r[-2]->{q(clumpidx)} =~ m|^\d+/(\d+)$|; } # okay, assign new clumpidx values ... for (0 .. ($clumps_num-1)) { $r[-$clumps_num+$_]->{q(clumpidx)} = "$_/$clumps_num"; } } $prev = $prog; } return @r; } #------------------------------------------------------------------------------ # process_json #------------------------------------------------------------------------------ # desc : fetch a URL and process it # arguments : 1- Date::Manip object, basedate/startdate of grabbing (e.g. 20060205) # 2- xmltv id of channel # 3- site id of channel # 4- dayindex of the requested page on port.hu|ro # returns : list of the programme hashes to write #------------------------------------------------------------------------------ sub process_json( $$$$ ) { my ($basedate, $ch_xmltv_id, $ch_port_id, $baseday) = @_; # $basedate = UnixDate(DateCalc(parse_date($basedate),"- 1 day"), '%Q'); $basedate = UnixDate(parse_date($basedate), '%Q'); my $days_to_request = $DAYSPERPAGE; my $basedateday = UnixDate(parse_date($basedate), '%e'); my $to_date = Time::Piece->strptime( $basedate, '%Y%m%d'); $to_date += ONE_DAY * $days_to_request; $ch_port_id =~ s/^0+//; my $d = domain(); my $urlfmt = "https://" . $d . (($COUNTRY eq 'hu') ? "/tvapi?channel_id=tvchannel-" : "/pls/w/tv_api.event_list?i_channel_id=").$ch_port_id. "&i_datetime_from=%s&i_datetime_to=".$to_date->strftime('%Y-%m-%d'); my $url = "$urlfmt"; local $SIG{__WARN__} = sub { warn "$url: $_[0]"; }; my $json_data; my $lang = $COUNTRY; # make (maximum) two loop to fetch program data: # # if the grabber runs in early hours (e.g. 01:00, 02:00) port.hu returns # the yesterdays's program as today's program... so we have to check it, # example.hu: <span class="ctxt">Péntek (február 27.)</span> # example.ro: <span class="ctxt">Duminic\u0103 (26 februarie)</span> # if this fails, we construct the previous (or next?) day's url my @daysonpage = (); foreach (1, 2) { $url = sprintf($urlfmt, UnixDate($basedate, '%Y-%m-%d')); t "fetching url: $url"; worker("base-downloading"); $XMLTV::Get_nice::FailOnError = 1; my $data=get_nice($url); $data =~ s/<\/?span[^>]{0,}>\s?//g; # remove html elements $json_data = (($DEFAULT_ENCODING !~ /utf\-?8/i) && ($COUNTRY eq 'hu')) ? JSON->new->utf8(0)->decode(encode($DEFAULT_ENCODING,decode('utf-8', $data))) : JSON->new->utf8(0)->decode($data) or die "could not fetch/parse $url (json structure)\n"; worker("base-parsing"); foreach my $act_secs (sort(keys(%{$json_data}))) { my $wday = UnixDate(($COUNTRY eq 'hu') ? $json_data->{$act_secs}->{'date_from'} : $json_data->{$act_secs}->{'datetime_from'}, '%d'); t "added founded date of the month on the grabbed page: $wday"; push @daysonpage, $wday; } if (@daysonpage) { # check date ... is the first founded date on the page the requested? last if ($basedateday == $daysonpage[0]); t "requested from $basedate, but port.$COUNTRY returned programs from wrong day: $daysonpage[0]"; if (UnixDate(DateCalc($basedate, "- 1 days"), '%e') == $daysonpage[0]) { # port.hu returned the programms from yesterday $FETCHOFFSET += 1 ; } elsif (UnixDate(DateCalc($basedate, "+ 1 days"), '%e') == $daysonpage[0]) { # port.hu returned the programms from tommorrow $FETCHOFFSET -= -1 ; } else { t "fetched HTML page do not contain 0, +1 or -1 day of the reuested one"; last; } t "global fetch offset was set to: $FETCHOFFSET"; } else { warn "no date data found on the fetched HTML page, trying to continue"; last; } } if (! defined($json_data)) { warn "Could not found the requested day's data on the grabbed JSON structure, " . "some programs on $ch_xmltv_id channel will be not fetched."; return; } my @programs; # JSON structure # <date_in_secs_since_1970> => # { # 'date_from' => 'YYYY-MM-DDTHH:MM:SS+HH:MM' # 'date_to' => 'YYYY-MM-DDTHH:MM:SS+HH:MM' # 'channels' => [{ # 'date' => 'YYYY-MM-DDTHH:MM:SS+HH:MM' (date of query) # 'date_from' => 'YYYY-MM-DDTHH:MM:SS+HH:MM' # 'date_until' => 'YYYY-MM-DDTHH:MM:SS+HH:MM' (equals with date_to) # 'domain' => 'port.hu' # 'id' => 'tvchannel-N' # 'name' => 'Channel Name' # ... # 'programs' => [{ # 'id' => 'event-t-NNNNNNNN' # 'start_ts' => <date_in_secs_since_1970> # 'start_datetime' => 'YYYY-MM-DDTHH:MM:SS+HH:MM' # 'end_datetime' => 'YYYY-MM-DDTHH:MM:SS+HH:MM' # 'start_time' => 'HH:MM' # 'end_time' => 'HH:MM' # 'title' => 'Show title' # 'episode_title' => 'Ep title' or undef # 'short_description' => 'Short desc.' or undef # 'description' => 'Desc.' or undef # 'film_url' => '/adatlap/film/tv/...' # 'restriction' => { 'category' => 'tvEventType-N', 'age_limit' => 'ageLimitList-N' ] # 'attributes_text' => '(ism.)' / '(élő)' / '(DS)' etc. or '' # 'italics' => 'Feliratozva ...' or undef # ... # # }] # }] # } foreach my $act_secs (sort(keys(%{$json_data}))) { my $all_prog_data = $json_data->{$act_secs}->{'channels'}[0]->{'programs'}; foreach my $prog_data (@{$all_prog_data}) { my %program; $program{startdate} = UnixDate($prog_data->{'start_datetime'}, '%Q'); my $currday = $program{startdate}; $currday =~ s/^[0-9]{6}([0-9]{2})$/$1/; $program{day} = $currday - $daysonpage[0] + 1; # We skip those programs, that are out of the requested time frame if ($program{day} != 1) { next; } $program{time} = $prog_data->{'start_time'}; $program{time} =~ s/^([012]?[0-9]):([0-5][0-9])$/$1$2/; $program{enddate} = UnixDate($prog_data->{'end_datetime'}, '%Q') if (defined($prog_data->{'end_datetime'})); t "--- missing end_time (".$prog_data->{'title'}.") ".d $prog_data if (!defined($prog_data->{'end_time'})); $program{endtime} = $prog_data->{'end_time'}; $program{endtime} =~ s/^([012]?[0-9]):([0-5][0-9])$/$1$2/ if (defined($program{endtime})); $program{title} = (defined($prog_data->{'title'}) && $prog_data->{'title'} ne "") ? $prog_data->{'title'} : ' '; if ($prog_data->{'short_description'}) { $program{desc} = $prog_data->{'short_description'}; } elsif ($prog_data->{'description'}) { $program{desc} = $prog_data->{'description'}; } # if ($prog_data->{'attributes_text'}) { # $program{desc} = ($program{desc}) ? $program{desc}.', '.$prog_data->{'attributes_text'} : $prog_data->{'attributes_text'}; # } if ($prog_data->{'episode_title'}) { if (($program{desc}) && ($program{desc} =~ /$WORDS{$COUNTRY}->{episode}/)) { $program{desc} =~ s/($WORDS{$COUNTRY}->{episode})/$1, $prog_data->{'episode_title'}/; } else { $program{desc} = ($program{desc}) ? $program{desc}.', '.$prog_data->{'episode_title'} : $prog_data->{'episode_title'}; } } if ($program{desc}) { $program{desc} =~ s/ +/ /g; $program{desc} =~ s/Megvásárolható (DVD[ ]?-n|VHS[ ]?-en)//g; } if ($prog_data->{'film_url'}) { my @url = ((($COUNTRY eq 'hu') ? 'https://'.domain() : '').$prog_data->{'film_url'}); $program{infourl} = \@url; } # support ticket #202 # get the rating if available my $actcat = ""; if ($prog_data->{'restriction'}) { $prog_data->{'restriction'}->{'age_limit'} =~ s/(\d+)/ageLimitList-$1/ if (($COUNTRY eq 'ro') && ($prog_data->{'restriction'}->{'age_limit'} !~ /ageLimitList/)); $prog_data->{'restriction'}->{'category'} =~ s/(\d+)/tvEventType-$1/ if (($COUNTRY eq 'ro') && ($prog_data->{'restriction'}->{'category'} !~ /tvEventType/)); if (($prog_data->{'restriction'}->{'age_limit'}) # 'ageLimitList-N' && ($AGE_LIMITS{$prog_data->{'restriction'}->{'age_limit'}})) { my $rating = ($DEFAULT_ENCODING !~ /utf\-?8/i) ? encode($DEFAULT_ENCODING, $AGE_LIMITS{$prog_data->{'restriction'}->{'age_limit'}}[0]->{$lang}) : $AGE_LIMITS{$prog_data->{'restriction'}->{'age_limit'}}[0]->{$lang}; $program{rating} = [[ $rating, '', [{'src' => $rating_baseurl.$AGE_LIMITS{$prog_data->{'restriction'}->{'age_limit'}}[1] }] ]]; } if (($prog_data->{'restriction'}->{'category'}) # 'tvEventType-N' && ($PROGRAM_CATEGORIES{$prog_data->{'restriction'}->{'category'}})) { $actcat = ($DEFAULT_ENCODING !~ /utf\-?8/i) ? encode($DEFAULT_ENCODING, $PROGRAM_CATEGORIES{$prog_data->{'restriction'}->{'category'}}->{$lang}) : $PROGRAM_CATEGORIES{$prog_data->{'restriction'}->{'category'}}->{$lang}; $program{desc} = ($program{desc}) ? $program{desc}.', '.$actcat : $actcat; # insert actual category - see parse_short_desc } } # add all parsed info, as program{time, title, desc, category, date} parse_short_desc(\%program); if ((defined($program{desc})) && ($program{desc} =~ $actcat)) { # remove actual category $program{desc} =~ s/[,\ ]{0,2}$actcat$//; delete($program{desc}) if (!length($program{desc})); } worker("slow-parsing"); foreach my $infourl (@{$program{infourl}}) { # always read data from linked page (in --slow mode) # in --get-full-description mode read if description ends in '...' if ( ($opt_slow) || ( $opt_full_desc && ( ((defined $program{desc}) && ($program{desc} =~ m/\.\.\.$/) ) || ((!$program{desc}) && (!$program{category}) && ($actcat eq $PROGRAM_CATEGORIES{'tvEventType-2'}->{$lang})) ) ) ) { # program without desc & category + actcat = 'film'? get_infourl_data_json(\%program, $infourl); # parse_short_desc(\%program); } } t "pushing ".$program{title}; push @programs, \%program if (defined $program{time}) && (defined $program{title}); } # foreach $prog_data } # foreach $json_data if (not @programs) { warn "no programs found, skipping\n"; return (); } # make a sort on programs, short compare function: cmp startdate, time # stringwise (this gives the same rsult as comapre datewise) # Date_Cmp(UnixDate($left->{time},'%H:%M'),UnixDate($right->{time},'%H:%M'); # sub bytime { # ($a->{startdate}.$a->{time}) cmp ($b->{startdate}.$b->{time}); # } @programs = sort {$a->{startdate}.$a->{time} cmp $b->{startdate}.$b->{time}} @programs; t "programs in sorted order:"; t "program:" . d $_ foreach (@programs); my (@r, $prev); # assume lang == country # my $lang = $COUNTRY; foreach my $program (@programs) { my $prog; $prog->{channel}=$ch_xmltv_id; $prog->{title}=[ [ $program->{title}, $lang ] ]; my $start=parse_local_date("$program->{startdate} $program->{time}", $TZ); my ($start_base, $start_tz) = @{date_to_local($start, $TZ)}; $prog->{start}=UnixDate($start_base, '%q') . " $start_tz"; if (defined($program->{enddate}) && defined($program->{endtime})) { my $stop=parse_local_date("$program->{enddate} $program->{endtime}", $TZ); my ($stop_base, $stop_tz) = @{date_to_local($stop, $TZ)}; $prog->{stop}=UnixDate($stop_base, '%q') . " $stop_tz"; } else { t "--- missing enddate + endtime - ch: ".$prog->{channel}.", title: ".$program->{title}; } $prog->{desc} = [[ $program->{desc}, $lang ]] if defined $program->{desc}; $prog->{category} = $program->{category} if (defined $program->{category}); $prog->{date} = $program->{date} if defined $program->{date}; $prog->{qw(episode-num)} = $program->{qw(episode-num)} if defined $program->{qw(episode-num)}; $prog->{length} = $program->{length} if defined $program->{length}; $prog->{rating} = $program->{rating} if defined $program->{rating}; # support #202 # Setting stop date for the previous program # Last program in the grabbed list has no stop attribute, sorry. # Port.hu uses a virtual program as the last program # anyway if ((defined($prev)) && ((!defined($prev->{stop})) || $prev->{stop} ne $prog->{start})) { if ($prev->{start} ne $prog->{start}) { $prev->{stop} = $prog->{start}; } else { t "--- remove previous program: ".d @r; pop(@r); } } worker("base-parsing"); push @r, $prog; # if ((defined($prev)) && $prev->{start} eq $prog->{start}) { # starttime of previous and current programme is equal, # therefore use clumpidx to express relation # my $clumps_num = 2; # if (defined($r[-2]->{q(clumpidx)})) { # previous programme already has a clumpidx arg assigned. # ($clumps_num) = $r[-2]->{q(clumpidx)} =~ m|^\d+/(\d+)$|; # } # okay, assign new clumpidx values ... # for (0 .. ($clumps_num-1)) { # $r[-$clumps_num+$_]->{q(clumpidx)} = "$_/$clumps_num"; # } # } $prev = $prog; } return @r; } #------------------------------------------------------------------------------- # parse_short_desc #------------------------------------------------------------------------------- # desc : parse the short description of a program, founded on the program # listing page (this is mostly 1-2 lines ~ 120 characters), but # sometimes contains categrory, date, length # arguments : 1- reference to a program HASH, there is the grabbed description in it # and there should be attached the other newly found informations, # such as: # ( category => [ [Animals, en], [Természet, hu], [..], ... ] # date => 2001 ) # returns : none #------------------------------------------------------------------------------- sub parse_short_desc ($) { my $prog = shift; my (%result, $desc, $cont, $episode, $minutes, $year, @categories); if ((defined $prog->{desc}) && length($prog->{desc})) { $desc = $prog->{desc}; } else { return } # 1: if there is () in the desc grab from there # 2: if no () found, try in the first 120 character # # examples: # Hegylako - A hollo (amerikai-francia-kanadai kalandfilmsorozat, 1998) # Lisa. Animációs sorozat. # Slayers - A kis boszorkány. (12). Japan animacios sorozat. # # sometimes only the proposed minimal age of watching person is # presented in parentheses eg: (12), so parse this only if it is # longer as for example 6 (4 is not enough, because (ism.) is no category...) if (! (($cont) = $desc =~ m/[^\(]*\(([^\)]{6,})\)/)) { $cont = substr($desc, 0, (length($desc) < 120 ? length($desc) : 120)); } t "parse_short_desc: text: '$cont'"; $WORDS{$COUNTRY}->{episode}="zdontmatchz" unless exists $WORDS{$COUNTRY}->{episode}; t "episode (country: $COUNTRY): ".(defined($WORDS{$COUNTRY}->{episode}) ? $WORDS{$COUNTRY}->{episode} : "undef"); # port.hu episode style with season (# patch #80) if ($cont =~ /\s*([IVX]+\.?\s?\/\s?[0-9]+)\. $WORDS{$COUNTRY}->{episode}/) { $episode = $1; } # port.hu episode style without season elsif ($cont =~ /\s*([0-9\/]+)\. $WORDS{$COUNTRY}->{episode}/) { $episode = $1; } # port.ro episode style with season (# patch #74) elsif ($cont =~ /$WORDS{$COUNTRY}->{episode} \s*([0-9]+, [A-Za-z]+ [0-9])/) { $episode = $1; } # port.cz/.sk episode style with season elsif ($cont =~ /$WORDS{$COUNTRY}->{episode} \s*([0-9]+, [IVX]+)\./) { $episode = $1; } # port.cz/.sk episode style for two episodes back to back in one slot elsif ($cont =~ /$WORDS{$COUNTRY}->{episode} \s*(\d+, \d+)/) { $episode = $1; } # port.cz/.sk/.ro episode style without season elsif ($cont =~ /$WORDS{$COUNTRY}->{episode} \s*([0-9\/]+)/) { $episode = $1; } if ($cont =~ /\s*(\d+)'/) { $minutes = $1; } if ($cont =~ /\(.*?((?:19|20)[0-9]{2})/) { $year = $1; } # bug #448 elsif ($cont =~ /$WORDS{$COUNTRY}->{episode},\s((?:19|20)[0-9]{2}),\s/) { $year = $1; } # ex.: '... II / 4. rész, 2016, ...' t "found episode: '$episode'" if defined $episode; t "found minutes: '$minutes'" if defined $minutes; t "found year: '$year'" if defined $year; # Sort the category keys so they appear in consistent order foreach (sort keys %CATMAP) { next unless defined $CATMAP{$_}; # bug #443 if ($cont =~ /$CATMAP{$_}[0]/i) { push @categories, [$_, "en"]; push @categories, [$CATMAP{$_}[1], $COUNTRY]; t "found category: '$_'"; } } $prog->{q(category)} = \@categories if @categories; $prog->{q(length)} = $minutes * 60 if defined $minutes; $prog->{q(date)} = $year if defined $year ; $prog->{q(episode-num)} = extract_episode( $episode ) if defined $episode ; } #------------------------------------------------------------------------------- # get_nice_gzip #------------------------------------------------------------------------------- # desc : get url with get_nice, check for gzip encoding, return # decoded if necessary # arguments : url to fetch from # returns : html as scalar #------------------------------------------------------------------------------- sub get_nice_gzip( $ ) { my $data = get_nice($_[0]); # Use heuristics to check for valid html if ($data =~ /<html|<head|<body|<table|<div/i) { return $data; } t "content is gzipped, deflating"; return gunzip($data); } #------------------------------------------------------------------------------- # get_channels #------------------------------------------------------------------------------- # desc : get channel listing for a country # arguments : none # returns : sets global CHANNELS hash to the grabbed channels: # ( '$channel_id' => # ( 'display-name' => [ [ $channel_name, $COUNTRY ] ], # 'id' => "$channel_id.$d", # 'icon' => [ { src => $iconurl } ] ) #------------------------------------------------------------------------------- sub get_channels( ;$ ) { my $mode = shift; my $d = domain(); my $bar = new XMLTV::ProgressBar('getting list of channels', 1) if not $opt_quiet; my $url="https://www.$d/pls/tv/".($COUNTRY eq 'hu' || $COUNTRY eq 'ro' ? 'old' : '')."tv.prog"; # bug #501 worker("base-downloading"); t "fetching $url..."; $XMLTV::Get_nice::FailOnError = 1; my $data = get_nice_gzip($url); my $tree = HTML::TreeBuilder->new_from_content($data) or die "could not fetch/parse $url (channel listing)"; worker("base-parsing"); my @menus = $tree->find_by_tag_name("_tag"=>"select"); foreach my $elem (@menus) { my $cname = $elem->attr('name'); $cname = '' if (!$cname); if ($cname eq "i_ch") { my @ocanals = $elem->find_by_tag_name("_tag"=>"option"); @ocanals = sort @ocanals; foreach my $opt (@ocanals) { my %channel; if (not $opt->attr('value') eq "") { my $channel_id = $opt->attr('value'); my $channel_name = $opt->as_text; if (length $channel_id eq 1) { $channel_id = "00" . $channel_id } if (length $channel_id eq 2) { $channel_id = "0" . $channel_id } # Assume country code and lang. code the same. %channel = ( 'display-name' => [ [ $channel_name, $COUNTRY ] ], 'id' => "$channel_id.$d" ) ; if (!defined $mode || $mode ne 'grab') { # no point doing this for 'grab' # fetch and get icon url worker("base-downloading"); if (my $iconurl = grab_icon( $channel_id )) { $channel{'icon'} = [ { src => $iconurl } ]; } worker("base-parsing"); } $CHANNELS{$channel_id} = \%channel; } } } } die "no CHANNELS could be found" if not %CHANNELS; update $bar if not $opt_quiet; $bar->finish() if not $opt_quiet; t "CHANNELS:" . d \%CHANNELS; } #------------------------------------------------------------------------------- # get_channels_json #------------------------------------------------------------------------------- # desc : get channel listing for a country # arguments : none # returns : sets global CHANNELS hash to the grabbed channels: # ( '$channel_id' => # ( 'display-name' => [ [ $channel_name, $COUNTRY ] ], # 'id' => "$channel_id.$d", # 'icon' => [ { src => $iconurl } ] ) #------------------------------------------------------------------------------- sub get_channels_json( ;$ ) { my $mode = shift; my $d = domain(); my $bar = new XMLTV::ProgressBar('getting list of channels', 1) if not $opt_quiet; my $url="https://www.$d/".(($COUNTRY eq 'hu') ? "tvapi/init" : "pls/w/tv_api.init?i_page_id=1"); worker("base-downloading"); t "fetching $url..."; $XMLTV::Get_nice::FailOnError = 1; binmode(STDOUT,":encoding(UTF-8)"); my $data = get_nice($url); my $json_data = (($DEFAULT_ENCODING !~ /utf\-?8/i) && ($COUNTRY eq 'hu')) ? JSON->new->utf8(0)->decode(encode($DEFAULT_ENCODING,decode('utf-8', $data))) : JSON->new->utf8(0)->decode($data) or die "could not fetch/parse $url (channel listing)"; worker("base-parsing"); foreach my $ch (@{$json_data->{'channels'}}) { my $channel_id = $ch->{'id'}; $channel_id =~ s/^[^0-9]+//; # 'tvchannel-N' -> 'N' $channel_id = sprintf("%03d", $channel_id); # 'N' -> '00N' my @urls = (($COUNTRY eq 'hu') ? 'https://'.domain() : '').$ch->{'link'}; my %channel = ( 'display-name' => [ [ $ch->{'name'}, $COUNTRY ] ], 'id' => "$channel_id.$d", 'url' => \@urls ); if ($ch->{'logo'}) { $channel{'icon'} = [ { src => $ch->{'logo'} } ]; } $CHANNELS{$channel_id} = \%channel; } die "no CHANNELS could be found" if not %CHANNELS; update $bar if not $opt_quiet; $bar->finish() if not $opt_quiet; t "CHANNELS:" . d \%CHANNELS; } #------------------------------------------------------------------------------- # add_person #------------------------------------------------------------------------------- # desc : check and maybe add the person to the credits # arguments : 1- found hungarian/roumanian jobname on the HTML page # 2- name of the person # 3- reference to the global creadits hash # returns : none #------------------------------------------------------------------------------- sub add_person ( $$$ ) { my ($job, $person, $rcredits) = @_; $person =~ s/\s+/ /g; return unless length($person); # suppress if job is not known, or if not mapped to DTD if (defined $JOBMAP{$job} && length($JOBMAP{$job})) { push @{$$rcredits{$JOBMAP{$job}}}, $person; t "credits: added: '$job -> $person'"; } else { t "credits: NOT added: '$job -> $person'"; } } #------------------------------------------------------------------------------- # get_infourl_data #------------------------------------------------------------------------------- # desc : merge data from linked info page into programme hash # arguments : 1- reference to the program, whom detailed descr should be grabbed # 2- url to fetch # returns : none #------------------------------------------------------------------------------- sub get_infourl_data( $$ ) { my $prog = shift; my $d = domain(); my $url = shift; # add port.hu/port.ro base url only if url is not contains the "://" uri separator if (! ($url =~ "://")) { $url = "https://www.$d" . $url; } # no info, so don't add it to anywhere # -> calendar.event_popup if ($url =~ "calendar\.event_popup") { t "SKIP fetching of slow url: $url"; return; } # do not grab: # -> pictures: ... pls/me/picture.popup?i_area_id # -> dvd rent links page: ... pls/w/logging.page_log?i_page_id=20... # -> sample movie ... video.link_popup?i_object_id=18822 # -> dvd sales page: www.divido.hu... # -> bet on a sport event -> sprotingbet # -> general advert links: adverticum if ($url =~ "(picture.popup|logging.page_log|video.link_popup|www\.divido\.hu|sportingbet|adverticum\.net)") { # add this url to the program push @{$prog->{q(url)}}, $url; t "SKIP fetching of slow url: $url"; return; } t "fetching slow url" . d $url; worker("slow-downloading"); t "fetching $url..."; $XMLTV::Get_nice::FailOnError = 0; my $data; if (! defined($data = get_nice($url))) { worker("slow-parsing"); warn "Could not get URL: $url, the detailed description for the program [" . $prog->{channel} . ", " . $prog->{title}[0][0] . ", " . $prog->{start} . "] will be not available. Error message: " . error_msg($url) . "." ; return; } my $tree = HTML::TreeBuilder->new_from_content($data) or die "could not fetch/parse $url (infopage)"; worker("slow-parsing"); my (@lines, $line, $anchor, $left, $right, $parent, $elem, $joined); # SUBTITLE # anchor point: # the title will be tagged aw follows: # <span class="blackbigtitle"> title </span> # these siblings buld the subtitle, until a table follows... e.g.: # <br><span class="txt"> ... subtitle line 1 ...<br> # <span class="bsubtitle"> ...subtitle line 2 ... </span></center><p> # <table .... # we have to search the title tag and collect the text until # table found t "subtitle parsing ..."; ($anchor) = $tree->look_down(_tag=>"span", class=>"blackbigtitle"); ($anchor) = $tree->look_down(_tag=>"h1", class=>"blackbigtitle") if !defined $anchor; # 2014-04-09 it seems to now be in <h1> if ($anchor) { $elem = $anchor; my ($engtitle, @tmp); while (($elem = $elem->right()) && ((ref $elem) && ($elem->tag() ne "table"))) { # if a whole line is surrounded with parentheses, on port.hu # this is the program's english title, add this as title and # also as subtitle: because some viewer shows only infos of # selected language (so the english title will be not # visible otherwise just in sub-title) @tmp = get_all_text($elem); push @lines, @tmp; $line = join(' ', @tmp); if (($engtitle) = $line =~ m/^\s*\(([^\)]+)\)\s*$/) { push @{$prog->{q(title)}}, [$engtitle, 'en']; t "engtitle added: $engtitle"; } } $joined = join(", ", @lines); $joined =~ s/\xA0//; # remove the to_text()'s results of   $joined =~ s/^\s+//; # remove blanks $joined =~ s/\s+$//; # remove blanks $joined =~ s/,$//; # remove trailing comma t "anchor and right sibling found, joinedlines parsed :'$joined'"; $prog->{q(sub-title)} = [[$joined, $COUNTRY]] if length($joined); } # ICON # anchor point: # the programme image will be tagged as follows: # <div class="random-media-wrapper"><img class="object_picture" src="http://media.port-network.com/picture/instance_2/92418_2.jpg" width="250" height="221" style="float:" alt="Închisoarea îngerilor - Tim Robbins" border="0" /> t "programme icon parsing ..."; $anchor = $tree->look_down(_tag=>"div", class=>"random-media-wrapper"); $anchor = $anchor->look_down(_tag=>"img", class=>"object_picture") if $anchor; if ($anchor) { my %icon; $icon{'src'} = $anchor->attr('src') if $anchor->attr('src'); $icon{'width'} = $anchor->attr('width') if $anchor->attr('width'); $icon{'height'} = $anchor->attr('height') if $anchor->attr('height'); $prog->{q(icon)} = [ \%icon ] if $anchor->attr('src'); } # LINKS: # try to grab IMDB, All Movie, official web site of the program # anchor point: # (the Links are listed between dots, but it is not allways the 5th) # (dots block, because not all blocks presented allways, so this is not) # (suggested to use) the Links are listed after the text Linkek(hu) # or Linkuri(ro), some line # after come a 'dots' (which is in a TABLE element, so: # find a span element with Linkek(hu) or Linkuri(ro) contents, get all # A element until TABLE not reached t "links parsing ..."; $anchor = undef; my @spans = $tree->look_down(_tag => "span"); t "spans found: " . $#spans; foreach (@spans) { if ($_->as_text() =~ /$WORDS{$COUNTRY}->{links}/) { t "anchor point found"; $anchor = $_; last; } } my @links; if ($anchor) { $elem = $anchor; while (($elem = $elem->right()) && ((ref $elem) && ($elem->tag() ne "table"))) { foreach ($elem->find_by_tag_name("_tag"=>"a")) { # is this not begins with 'https?://' add prefix push @links, ($_->attr(q(href)) =~ /^https?:\/\// ? "" : "http://www.$d") . $_->attr(q(href)); t "link url added: " . $_->attr(q(href)) ; } } } push @links, $url; if (defined $prog->{q(url)}) { @{$prog->{q(url)}} = ( @links, @{$prog->{q(url)}} ); } else { push @{$prog->{q(url)}}, @links; } # LONG DESCRIPTION: # new format uses the <div class="separator"><!-- ++++++++++ --></div> block # to separate contents # anchor point: # long desc is in the 3. block; this is right sibling of the 3rd separator # the actual content is inbetween the <span class="txt">....</span> elements t "long desc parsing ..."; my @separators = $tree->look_down(_tag=>"div", class=>"separator"); return if ($#separators < 2); @lines = (); if (($anchor) = $separators[2]->right()) { $joined = $anchor->tag(); if ($anchor->tag() eq "span" && $anchor->attr('class') eq "txt") { push @lines, get_all_text($anchor); $joined = join(" ", @lines); $joined =~ s/\xA0//; # remove the to_text()'s results of   $joined =~ s/^\s+//; # remove blanks t "found description: $joined"; if (length($joined)) { delete($prog->{q(desc)}); # strip the desc at the specified command line option (if spec) if (defined ($opt_max_desc_length) && ($opt_max_desc_length < length($joined))) { t "long desc was stripped, at: $opt_max_desc_length."; $joined = substr($joined, 0, $opt_max_desc_length - 3) . "..."; } $prog->{q(desc)} = [[ $joined, $COUNTRY ]] } } } # SERIES NUMBER, CATEGORY, YEAR # anchor point: 2nd separator # all text data is in/under the parent TD element of the 2nd separator # We collect all text data, and parse it from known datas. return if ($#separators < 1); ($anchor) = $separators[1]; if ($anchor->parent()->tag() ne "td" || !defined $anchor->parent->attr('width') || $anchor->parent->attr('width') ne "98%") { # bug #445 t "credits section not found"; return; } # get the rating if available # <img alt="(AP)" title="Recomandat acordul parintilor" class="age_limit_icon" src="http://media.port-network.com/page_elements/parental_guidance_mini_pix_ro.png"> if (my $img = $anchor->parent()->look_down('_tag' => 'img', 'class' => 'age_limit_icon')) { my $rating = $img->attr('alt'); $rating =~ s/[\(\)]//g; # strip the brackets my $rating_icon = $img->attr('src'); $prog->{q(rating)} = [[ $rating, '', [{'src' => $rating_icon }] ]]; } # unfortunately the star-rating (vote_box) if fetched with an AJAX call # e.g. http://www.port.sk/arrow/pls/fi/vote.print_vote_box?i_object_id=139692&i_area_id=6&i_reload_container=id%3D%22vote_box%22&i_is_separator=0 # # we could use TreeBuilder->store_cdata(true) to store the cdata under the root node, but I think it's easier to just regexp the html # /*<![CDATA[*/ # ajaxRequest( # {'url':'vote.print_vote_box?', # 'parameters':'i_object_id=139692&i_area_id=6&i_reload_container=id%3D%22vote_box%22&i_is_separator=0', # 'method':'GET', # ... # if ( $data =~ m/ajaxRequest\(.*?{'url':'(vote.print_vote_box\?)',.*?'parameters':'(.*?)',.*?'method':'GET'/s ) { my $ajaxurl = $1 . $2; # ajax uri (my $baseurl) = $url =~ m#(.*/)#; # greedy match up to last / char $ajaxurl = $baseurl . $ajaxurl; # prepend the uri t "fetching ajax url" . d $ajaxurl; # store current values of get_nice my $get_fail = $XMLTV::Get_nice::FailOnError; my $get_delay = $XMLTV::Get_nice::Delay; $XMLTV::Get_nice::FailOnError = 0; # don't abort if not found $XMLTV::Get_nice::Delay = 0; # no delay my $ajaxdata; if (defined($ajaxdata = get_nice($ajaxurl))) { worker("ajax-downloading"); my $ajaxtree = HTML::TreeBuilder->new_from_content($ajaxdata) or die "could not fetch/parse $ajaxurl (infopage)"; worker("ajax-parsing"); # get the "star-rating" # The rating includes the number of votes. Testing for statistical significance calculates that, at # a confidence level of .95, a population of 100 will give 70% confidence in the score being accurate # (and is largely independent of population size). # Therefore only output the rating where the population is > 100. # if (my $anchor = $ajaxtree->look_down(_tag=>"div", class=>qr/starholder/)) { my $starsval; if (my $stars = $anchor->look_down(_tag=>"span", class=>"ctxt")) { $starsval = $stars->as_text(); $starsval =~ s/,/./; $starsval += 0; # convert to float } if (my $votes = $anchor->look_down(_tag=>"div", class=>"votenum")) { my $votesval = $votes->as_text(); if ($votesval) { (my $num_votes) = $votesval =~ /(\d*)/; # if number of votes is >100 then output the star-rating if ($num_votes ne '' && int($num_votes) > 100) { $prog->{q(star-rating)} = [[ sprintf("%.0f / 10", $starsval), "uservotes" ]]; } } } } } # restore the previous values $XMLTV::Get_nice::FailOnError = $get_fail; $XMLTV::Get_nice::Delay = $get_delay; } # collect all text lines, we # achive this to jump to the parent first, and walk all the childs until # the anchor is reached @lines = (); foreach $elem ($anchor->parent()->content_list()) { last if ((ref $elem) && ($elem == $anchor)); push @lines, get_all_text($elem); } # 0:{we are in credits secton}, 1:{duration,year section} my $section = 0; my $job = "foobar"; my $part = ""; my (%credits, $episode, $minutes, $year); my $person = ""; foreach $line (@lines) { $line =~ s/\xA0//; # remove to_text()'s results of &npsp t "processing line: '" . d $line . "'"; foreach $part (split /, */, $line) { $part =~ s/^\s+//; # remove heading blanks $part =~ s/\s+$//; # remove ending blanks $part =~ s/^,*$//; next unless length $part; t "processing part: '$part'"; # we are in credits block if a known hungarian "job:" found $section = 1 if (($section == 0) && (($_) = $part =~ m/\b(.+):/) && (defined($JOBMAP{$_}))); if ($section == 0) { # duration, year, category # possibilitys # 1: amerikai filmdráma sorozat, 90 perc, 2000, 2. rész # 12 éven aluliak számára .... # added 2004-04-07 : # (ro) Coreea de Sud, 2009, serial de aventuri, episodul 5 $_ = $part; SWITCH: { if ((m/\s*([0-9\/]+)\. $WORDS{$COUNTRY}->{episode}/) && (! defined $episode)) { $episode = $1; last SWITCH;} if ((m/$WORDS{$COUNTRY}->{episode} \s*([0-9\/]+)/) && (! defined $episode)) { $episode = $1; last SWITCH;} if ((m/\s*(\d+) $WORDS{$COUNTRY}->{minute}/) && (! defined $minutes)) { $minutes = $1; last SWITCH;} if ((m/\s*([12][0-9]{3})/) && (! defined $year)) { $year = $1; last SWITCH;} { ; } # default -> category, was processed over } t "found episode: '$episode'" if defined $episode; t "found minutes: '$minutes'" if defined $minutes; t "found year: '$year'" if defined $year; } # section 0 if ($section == 1) { # # is there a "hu-job:" string in the part? if yes, we should # push the last readed person, and clear the person string. # if a job is defined (hu-job) but not supported in the DTD # we will add # the person(s) as: # <actor> some_job: Foo Bar, Dummy Name, ...<actor> # note: \b(.+): do not match to " író: ", because í is not # part of \b # bug #451 [line deleted] if (($_) = $part =~ /^\s*(\S+):/) { # does the $line include a ':' # remove the "jobname:" string $part =~ s/^\s*(\S+):\s*//; t "assuming string is a jobname"; # this means, we should add our until now collected # person to the credits, and begin to collect new # actors... add_person($job, $person, \%credits); t "is this a known job?: '$_'"; # e.g.: hu-job if (defined $JOBMAP{lc($_)} && length($JOBMAP{lc($_)})) { # newly readed part has a en-job (this is defined in DTD, so # this will be the next used job for XML generation t "job known in DTD as: $JOBMAP{lc($_)}"; $job = lc($_); $person = $part; } #en-job else { # this job is not known in DTD, so only en-job, no hu-job; # add as descriped above, set job to foobar to add as actor $job = "foobar"; $person = "$_: $part "; } #hu-job next; } #: in the part # we are here, if: # -> $part holds ':' but it is no hu-job (no en-job) # -> it have no : if ($part =~ /^\(.*\)$/) { t "found () expression, addint it to person string"; # if it has the from '(...)' the found HTML was: # actor: Arnold Schweizenegger (as the Terminator) # add this to persons and do not push, it. $person .= " $part"; } else { # this is a new name, check how looks person, if it ends # with ":" do not add this to credits, only append, because in the # previuos iteration only hu-job was found. if ($person =~ /:\s*$/) { $person .= " $part"; } else { add_person($job, $person, \%credits); $person = $part; } } } #section 1 } #loop over parts } #loop over $lines # add the last processed data to credits... add_person($job, $person, \%credits) if length($person); t "CREDITS: " . d \%credits; $prog->{q(credits)} = \%credits; #$prog->{q(category)} = [[ $category, $COUNTRY ]] #if defined $category and length $category; $prog->{q(length)} = $minutes * 60 if defined $minutes; $prog->{q(date)} = $year if defined $year ; $prog->{q(episode-num)} = extract_episode( $episode ) if defined $episode ; $tree->delete; } #------------------------------------------------------------------------------- # get_infourl_data_json #------------------------------------------------------------------------------- # desc : merge data from linked info page into programme hash # arguments : 1- reference to the program, whom detailed descr should be grabbed # 2- url to fetch # returns : none #------------------------------------------------------------------------------- sub get_infourl_data_json( $$ ) { my $prog = shift; my $d = domain(); my $url = shift; # add port.hu/port.ro base url only if url is not contains the "://" uri separator if (! ($url =~ "://")) { $url = "https://www.$d" . $url; } # no info, so don't add it to anywhere # -> calendar.event_popup if ($url =~ "calendar\.event_popup") { t "SKIP fetching of slow url: $url"; return; } # do not grab: # -> pictures: ... pls/me/picture.popup?i_area_id # -> dvd rent links page: ... pls/w/logging.page_log?i_page_id=20... # -> sample movie ... video.link_popup?i_object_id=18822 # -> dvd sales page: www.divido.hu... # -> bet on a sport event -> sprotingbet # -> general advert links: adverticum if ($url =~ "(picture.popup|logging.page_log|video.link_popup|www\.divido\.hu|sportingbet|adverticum\.net)") { # add this url to the program push @{$prog->{q(url)}}, $url; t "SKIP fetching of slow url: $url"; return; } t "fetching slow url" . d $url; worker("slow-downloading"); t "fetching $url..."; $XMLTV::Get_nice::FailOnError = 0; my $data; if (! defined($data = get_nice($url))) { worker("slow-parsing"); warn "Could not get URL: $url, the detailed description for the program [" . $prog->{channel} . ", " . $prog->{title} . ", " . $prog->{start} . "] will be not available. Error message: " . error_msg($url) . "." ; return; } else { if ($data =~ /<!\-\-\ title\ \-\->/) { my $orig_title = substr($data, index($data, '<!-- title -->')+14, index($data, '<!-- /title -->', index($data, '<!-- title -->'))-(index($data, '<!-- title -->')+14)); $orig_title =~ s/<[^>]+>//g; $orig_title =~ s/^\s+|\s+$//g; # trim spaces $orig_title =~ s/^[^\n]+\n//g; # remove translated title $orig_title =~ s/^[^\/]+\/(.*)\/$/($1)/g; $orig_title = encode($DEFAULT_ENCODING, decode('utf-8', $orig_title)) if ($DEFAULT_ENCODING !~ /utf\-?8/i); $prog->{q(desc)} = (defined($prog->{q(desc)}) && $prog->{q(desc)} ne "") ? $prog->{q(desc)}.' '.$orig_title : $orig_title; } if ($data =~ /<!\-\-\ summary\ \-\->/) { my $sum = substr($data, index($data, '<!-- summary -->')+16, index($data, '<!-- /summary -->', index($data, '<!-- summary -->'))-(index($data, '<!-- summary -->')+16)); $sum =~ s/<[^>]+>//g; $sum =~ s/^\s+|\s+$//g; # trim spaces $sum =~ s/magyarul\ besz..l..\,\ //g; $sum = encode($DEFAULT_ENCODING, decode('utf-8', $sum)) if ($DEFAULT_ENCODING !~ /utf\-?8/i); $prog->{q(desc)} = (defined($prog->{q(desc)}) && $prog->{q(desc)} ne "") ? $prog->{q(desc)}.' '.$sum : $sum; } if ($data =~ /<div\ class="description">/) { $data = substr($data, index($data, '<div class="description">')+25, index($data, '</div>', index($data, '<div class="description">'))-(index($data, '<div class="description">')+25)); $data =~ s/<\/?article>//ig; $data =~ s/<br\/?>/ /ig; $data =~ s/<strong>[^\<]+<\/strong>//ig; # <strong>Feliratozva a teletext ...</strong> $data =~ s/<b>.*//ig; # <b>Forgalmazó:</> ... / <b>Bemutató dátuma:</b> ... $data =~ s/^\s+|\s+$//g; # trim spaces $data = encode($DEFAULT_ENCODING, decode('utf-8', $data)) if ($DEFAULT_ENCODING !~ /utf\-?8/i); $prog->{q(desc)} = (defined($prog->{q(desc)}) && $prog->{q(desc)} ne "") ? $prog->{q(desc)}.', '.$data : $data; } worker("slow-parsing"); } } #------------------------------------------------------------------------------- # extract_episode #------------------------------------------------------------------------------- # desc : parse text containing the episode details # arguments : 1- episode data # returns : xmltv episode-num definition #------------------------------------------------------------------------------- sub extract_episode( $ ) { my $episode = shift; my ($episode_num, $season); if(defined($episode)) { if($episode =~ m#(\d+)/(\d+)#) { # episode-num spec with the total number specified. # swap numbers for port.hu, they have total/num my ($num, $total) = ($1, $2); ($num, $total) = ($2, $1) if ($num > $total); # however XMLTV counts from 0 on ... $episode_num = [[ sprintf('. %d/%d .', $num - 1, $total), "xmltv_ns" ], [ $episode, "onscreen" ]]; } elsif($episode =~ m#([IVX]+)\./(\d+)#) { # patch #80 # port.hu style episode numbering: <serie_in_roman>./<episode_in_arabic>. e.g. V./3 # episode-num spec with the total number specified. # decode season from roman numeral $season = arabic ($1); # however XMLTV counts from 0 on ... $episode_num = [[ sprintf('%d . %d .', $season - 1, $2 - 1), "xmltv_ns" ], [ $episode, "onscreen" ]]; } elsif($episode =~ m#(\d+), ([IVX]+)#) { # episode-num spec with the total number specified. # decode season from roman numeral $season = arabic ($2); # however XMLTV counts from 0 on ... $episode_num = [[ sprintf('%d . %d .', $season - 1, $1 - 1), "xmltv_ns" ], [ $episode, "onscreen" ]]; } elsif($episode =~ m#(\d+)#) { # episode-num spec with just the episode number # however XMLTV counts from 0 on ... $episode_num = [[ sprintf('. %d .', $1 - 1), "xmltv_ns" ], [ $episode, "onscreen" ]]; } else { $episode_num = [[ $episode, "onscreen" ]]; } } return $episode_num; } #------------------------------------------------------------------------------- # grab_icon #------------------------------------------------------------------------------- # desc : fetch (if needed and specified) channel icons, returns pointing URL # arguments : 1- channel id (eg 003) # returns : url pointing to tha program's logo (icon) http:|file:... #------------------------------------------------------------------------------- sub grab_icon( $ ) { # if icon not requested return unless ($opt_icons || $opt_local_icons); my $channelid = shift; my $fetchurl = "https://www." . domain() . "/tv/kep_ado/al_".(int(${channelid}) % 10000).".gif"; my ($file, $iconurl); # that $fetchurl no longer works for RO, so... #test if url is valid $XMLTV::Get_nice::FailOnError = 0; my $image = get_nice($fetchurl); if (!defined $image) { # image url not valid, so we must get it from the programmes page. Ideally we would do that during the main grab but this is a Q&D fix # and I don't want to change too much of this code my $url = "https://www." . domain() . "/pls/w/".($COUNTRY eq 'hu' || $COUNTRY eq 'ro' ? 'old' : '')."tv.channel?i_ch=".$channelid."&i_date=".UnixDate('today','%Y-%m-%d')."&i_where=1"; # bug #501 my $data=get_nice($url); my $tree = HTML::TreeBuilder->new_from_content($data) or die "could not fetch/parse $url (grab_icon)\n"; worker("base-parsing"); my $body = $tree->look_down("_tag"=>"body"); my $container = $body->look_down("_tag" => "div", "class" => qr/main-container-100/); if ($container) { if (my $imgdiv = $container->look_down("_tag" => "div", "style" => qr/float\s*:\s*left/, sub { my $imgtag = $_[0]->look_down('_tag' => 'img'); return 0 unless $imgtag; return $imgtag->attr('src') =~ m/https:\/\/media/; } )) { $fetchurl = $imgdiv->look_down('_tag' => 'img')->attr('src'); } } } $XMLTV::Get_nice::FailOnError = 1; return $fetchurl if ($opt_icons && ! $opt_local_icons); # create directory mkdir $opt_local_icons unless (-d $opt_local_icons); # remove multiple /; make absoluth path $_ = "${opt_local_icons}/${channelid}.gif"; s!//!/!g; $file = Cwd::abs_path( $_ ); $iconurl = "file://${file}"; return $iconurl if ($opt_local_icons && $opt_no_fetch_icons); if (! -d $opt_local_icons) { warn "directory not exists, and cannot create: $opt_local_icons; " . "icon will be not grabbed"; return $fetchurl; } if (open(FILE,">$file")) { t "fetching $fetchurl..."; $XMLTV::Get_nice::FailOnError = 0; #if (my $image = get_nice($fetchurl)) { # now grabbed above if (!$image) { $image = get_nice($fetchurl); } if ($image) { t "icon for $channelid grabbed successfully"; print FILE $image; close FILE; # success return $iconurl; } else { warn "Could not download channel-logo for channel $channelid, using remote URL instead. " . "Error message: " . error_msg($fetchurl) . "."; close FILE; unlink $file; return $fetchurl; } } else { warn "cannot create icon file ($file) for channel $channelid, using remote URL instead"; close FILE; unlink $file; return $fetchurl; } return; } #------------------------------------------------------------------------------- # get_channel_urls #------------------------------------------------------------------------------- # desc : grab a channel page fetch (if needed and specified) channel icons, returns pointing URL # arguments : 1- channel id (eg 003) (grab a webpage parse data form there) # OR # 2- reference to a HTML tree's (root) object (searching in it) # returns : array of urls pointing to tha channel's pages/emails #------------------------------------------------------------------------------- sub get_channel_urls( $ ) { my $ch_did = shift; my @result = (); my $chdata; # two sprintf parameters: first: channel_id,, second how many days grabbed my $churlfmt = "https://www." . domain() . "/pls/tv/".($COUNTRY eq 'hu' || $COUNTRY eq 'ro' ? 'old' : '')."tv.channel?i_ch=%d&" . "i_days=1&i_xday=%d&i_where=1"; # bug #501 # url to grab now (4 days - this is the minimum) my $churl = sprintf($churlfmt, $ch_did, 4); # url to add as the information source (4 days - this is the minimum) my $portchurl = sprintf($churlfmt, $ch_did, 4); t "fetching page for channel urls: $churl\n"; worker("base-downloading"); t "fetching $churl..."; $XMLTV::Get_nice::FailOnError = 0; if (! defined($chdata = get_nice($churl))) { worker("base-parsing"); warn "Could not get URL: $churl, the information urls for the channel $ch_did will be not available. " . "Error message: " . error_msg($churl) . "."; push @result, $portchurl; return @result; } my $tree = HTML::TreeBuilder->new_from_content($chdata) or die "could not fetch/parse $churl (channel infopage)"; worker("base-parsing"); my ($anchor, $elem); # we have to way to find the channel URLs: # -> find the channel image (this is in the same TABLE element as the # requested A elements, and if this is not found: # -> try to find a HR element (only one is presented on the page), this # Nth left sibling is the searched TABLE. if (($anchor) = $tree->look_down( _tag => "b", sub { lc($_[0]->as_text()) =~ /web:/ } ) ) { if ($anchor = ($anchor->look_up("_tag"=>"p")->look_down("_tag"=>"a"))) { push @result, $anchor->attr(q(href)); } else { t "channel url not found"; } } else { t "channel url not found"; } # add PORT url, too, this should be the last (and open 3 days if clicked) push @result, $portchurl if defined $portchurl; return @result; } #------------------------------------------------------------------------------- # load_configs #------------------------------------------------------------------------------- # desc : load the tv_grab_huro.conf, jobmap, catmap.$COUNTRY files, and # sets the globals: %CATMAP, %JOBMAP # arguments : none # returns : array of port channel ids: ( 001, 005 ) #------------------------------------------------------------------------------- sub load_configs() { my @config_lines = XMLTV::Config_file::read_lines($CONFIG_FILE); my $line_num = 0; my (@portids, $where, @fields); foreach (@config_lines) { ++ $line_num; next if not defined; $where = "$CONFIG_FILE:$line_num"; if (/^country:?\s+(\w\w)$/) { if ($1 ne 'hu') { die "$where: Country '$1' no longer supported in grabber!\n"; } warn "$where: already seen country\n" if defined $COUNTRY; $COUNTRY = $1; } elsif (/^channel:?\s+(\S+)\s+([^\#]+)/) { my $ch_did = $1; my $ch_name = $2; $ch_name =~ s/\s*$//; push @portids, $ch_did; # FIXME do not store display-name in the config file - it is # ignored here. } else { warn "$CONFIG_FILE:$.: bad line\n"; } } for ($COUNTRY) { if (not defined) { $_ = 'hu'; warn "country not seen in $CONFIG_FILE, assuming '$_'\n"; } } # Lame reverse lookup on %COUNTRIES. foreach (values %COUNTRIES) { if ($_->[0] eq $COUNTRY) { $TZ = $_->[1]; last; } } die "$where: unknown country $COUNTRY\n" if not defined $TZ; # jobmap file # (this is a file, where we store translations of job names from # Hungarian or Romanian language to English. However we leave some # translations blank, namely these that have no field in the credits # structure) # # Read the file with channel mappings. my $jobmap_file = "jobmap"; my $jobmap_str = GetSupplement( 'tv_grab_huro', $jobmap_file ); $line_num = 0; foreach (split( /\n/, $jobmap_str )) { ++ $line_num; tr/\r//d; s/#.*//; next if m/^\s*$/; s/^\s+|\s+$//g; # trim spaces $where = "$jobmap_file:$line_num"; @fields = split m/:/; die "$where: wrong number of fields" if @fields > 2; my ($huro_job, $credits_id) = @fields; $JOBMAP{$huro_job} = defined($credits_id) ? $credits_id : ""; } # read the file with category mappings. # cat_en:cat_hu:regexp my $catmap_file = "catmap.$COUNTRY"; my $catmap_str=""; $catmap_str = GetSupplement( 'tv_grab_huro', $catmap_file ); $line_num = 0; foreach (split( /\n/, $catmap_str )) { ++ $line_num; tr/\r//d; s/#.*//; next if m/^\s*$/; s/^\s+|\s+$//g; # trim spaces $where = "$catmap_file:$line_num"; @fields = split m/:/; die "$where: wrong number of fields" if @fields > 3; my ($cat_en, $cat_hu, $cat_reg) = @fields; $CATMAP{$cat_en} = defined($cat_reg) ? [$cat_reg, $cat_hu] : [$cat_hu, $cat_hu]; } return @portids; } #------------------------------------------------------------------------------- # worker #------------------------------------------------------------------------------- # desc : measure how many seconds will be executed some port of this program # arguments : 1- name of the worker part of this program, currently: # xml-writing, base-downloading, slow-downloading # base-parsing, slow-parsing # returns : none #------------------------------------------------------------------------------- sub worker( $ ) { my $now = time(); my $newworker = shift; if (! defined $WNAME) { $WNAME = $newworker; $WTIMES{$WNAME} = 0; $WSTIME = $now; return; } $WTIMES{$WNAME} += $now - $WSTIME; $WSTIME = $now; $WNAME = $newworker; } #------------------------------------------------------------------------------- # showworkers #------------------------------------------------------------------------------- # desc : prints $WTIMES to the stdout # arguments : none # returns : none #------------------------------------------------------------------------------- sub showworkers() { return if $opt_quiet; return if not $opt_worker_times; my $total = 0; $total += $_ foreach values %WTIMES; $total = 1 unless $total ; # division by zero printf STDERR ("%-20s: %3d:%02dm %3d%%\n", $_, $WTIMES{$_} / 60, $WTIMES{$_} % 60, 100 * $WTIMES{$_} / $total) foreach keys %WTIMES; printf STDERR ("%-20s: %3d:%02dm\n", "total", $total / 60, $total % 60); } #------------------------------------------------------------------------------- # arabic #------------------------------------------------------------------------------- # desc : This example uses Robin Houston's entry in the Perl # : Institute's Roman Numeral Challenge to convert Roman numerals # : to Arabic. http://www.perl.org/wits.html # : (we could just use Roman, but don't want to add the dependency) # arguments : roman number # returns : integer #------------------------------------------------------------------------------- sub arabic( $ ) { my ($n, $d); ($n, $d, $_) = (1, 2, @_); $_ = uc $_ if !/[^a-z]/; for my $v(split//, 'IVXLCDM') { s/\+.*$v/)/; s/$v([^$v+-])/-$n$1/g; s/$v/+$n/g; $n *= $d ^= 7 } /[^-+\d]/ ? () : eval } #------------------------------------------------------------------------------- # M A I N #------------------------------------------------------------------------------- # Whether zero-length programmes should be included in the output. my $WRITE_ZERO_LENGTH = 0; # Get options, including undocumented --cache option. XMLTV::Memoize::check_argv('XMLTV::Get_nice::get_nice_aux'); $opt_slow = 0; $opt_full_desc = 0; $opt_days = 8; # default $opt_offset = 0; # default $opt_quiet = 0; # default GetOptions( 'days=i' => \$opt_days, 'offset=i' => \$opt_offset, 'help' => \$opt_help, 'configure' => \$opt_configure, 'gui:s' => \$opt_gui, 'config-file=s' => \$opt_config_file, 'output=s' => \$opt_output, 'quiet' => \$opt_quiet, 'slow' => \$opt_slow, 'list-channels' => \$opt_list_channels, 'icons' => \$opt_icons, 'local-icons=s' => \$opt_local_icons, 'no-fetch-icons'=> \$opt_no_fetch_icons, 'now=s' => \$opt_now, 'worker-times' => \$opt_worker_times, 'get-full-description' => \$opt_full_desc, 'max-desc-length=i' => \$opt_max_desc_length ) or usage(0); die 'number of days must not be negative' if (defined $opt_days && $opt_days < 0); usage(1) if $opt_help; my $mode = XMLTV::Mode::mode('grab', # default $opt_configure => 'configure', $opt_list_channels => 'list-channels'); XMLTV::Ask::init($opt_gui); # File that stores which channels to download. $CONFIG_FILE = XMLTV::Config_file::filename($opt_config_file, 'tv_grab_huro', $opt_quiet, 'tv_grab_hu'); #------------------------------------------------------------------------------- # only configuration #------------------------------------------------------------------------------- if ($mode eq 'configure') { worker("base-parsing"); XMLTV::Config_file::check_no_overwrite($CONFIG_FILE); open(CONF, ">$CONFIG_FILE") or die "cannot write to $CONFIG_FILE: $!"; my $default_cn = 'Hungary'; my $cn = ask_choice('Grab listings for which country?', $default_cn, sort keys %COUNTRIES); $COUNTRY = $COUNTRIES{$cn}[0]; print CONF "country $COUNTRY\t# $cn\n"; # Ask about each channel. (($COUNTRY) && (($COUNTRY eq 'hu') || ($COUNTRY eq 'ro'))) ? get_channels_json() : get_channels; # sets %CHANNELS my @portids = sort keys %CHANNELS; my @names = map { $CHANNELS{$_}->{qw(display-name)}->[0][0] } @portids; my @qs = map { "add channel $_?" } @names; my @want = ask_many_boolean(1, @qs); foreach (@portids) { my $w = shift @want; warn("cannot read input, stopping channel questions"), last if not defined $w; # No need to print to user - XMLTV::Ask is verbose enough. # Print a config line, but comment it out if channel not wanted. print CONF '#' if not $w; my $name = shift @names; print CONF "channel $_ $name\n"; # TODO don't store display-name in config file. } close CONF or warn "cannot close $CONFIG_FILE: $!"; say("Finished configuration."); worker("base-parsing"); showworkers(); exit(); } # Options to be used for XMLTV::Writer. my %w_args; $w_args{encoding} = $DEFAULT_ENCODING; if (defined $opt_output) { my $fh = new IO::File(">$opt_output"); die "cannot write to $opt_output: $!" if not defined $fh; $w_args{OUTPUT} = $fh; } #------------------------------------------------------------------------------- # only channel listing #------------------------------------------------------------------------------- if ($mode eq 'list-channels') { # Write channels mode. worker("base-parsing"); $COUNTRY='hu'; worker("xml-writing"); my $writer = new XMLTV::Writer(%w_args); $writer->start(xhead()); worker("base-parsing"); (($COUNTRY) && (($COUNTRY eq 'hu') || ($COUNTRY eq 'ro'))) ? get_channels_json() : get_channels(); # sets %CHANNELS # sort channels based on their portid my @portids = sort keys %CHANNELS; worker("xml-writing"); $writer->write_channel($CHANNELS{$_}) foreach @portids; $writer->end(); worker("base-parsing"); showworkers(); exit(); } #------------------------------------------------------------------------------- # only grabbing #------------------------------------------------------------------------------- if ($mode eq 'grab') { worker("base-parsing"); my $ch_did; my $bar; my @portids = load_configs(); # sets %CHANNELS (($COUNTRY) && (($COUNTRY eq 'hu') || ($COUNTRY eq 'ro'))) ? get_channels_json($mode) : get_channels($mode); worker("xml-writing"); my $writer = new XMLTV::Writer(%w_args); worker("base-parsing"); # we have to fetch @portids icons, and @portids pages for channel URL # (e.g.: www.hbo.hu) $bar = new XMLTV::ProgressBar('getting channel details ', 2 * @portids) if not $opt_quiet; worker("xml-writing"); $writer->start(xhead()); worker("base-parsing"); # Write channel elements foreach $ch_did (@portids) { if (! $CHANNELS{$ch_did}) { warn "\nWARNING: Channel with port-id $ch_did no more exists on the site, skipping it's channel description grabbing!"; next; } my %channel = %{$CHANNELS{$ch_did}}; worker("base-downloading"); # fetch and get icon url if (my $iconurl = grab_icon( $ch_did )) { $channel{'icon'} = [ { src => $iconurl } ]; } update $bar if not $opt_quiet; worker("base-parsing"); if (($COUNTRY) && ($COUNTRY ne 'hu') && ($COUNTRY ne 'ro') && (my @churls = get_channel_urls( $ch_did ))) { $channel{'url'} = \@churls; } update $bar if not $opt_quiet; worker("xml-writing"); $writer->write_channel(\%channel); worker("base-parsing"); } $bar->finish() if not $opt_quiet; if (!defined($COUNTRY) || (($COUNTRY ne 'hu') && ($COUNTRY ne 'ro'))) { # old, HTML based pages # The grabber's source allows requests of more than one day per page. This can # be done by specifying the i_xday argument with the GET request. # # To not load their server too much (requesting e.g. 14 channels in one shot # should 'cause quite some traffic to the SQL server) I think we shouldn't # query for more then 5 channels per page. With the default of requesting data # for 8 days this leads to 2 requests per channel and grab ... $DAYSPERPAGE = int($opt_days / 5) + (($opt_days % 5) ? 1 : 0); $DAYSPERPAGE = int($opt_days / $DAYSPERPAGE); # We have to request at minimum of four days $DAYSPERPAGE = 4 if ($DAYSPERPAGE<4); } else { # JSON $DAYSPERPAGE = 1; } t "requesting $DAYSPERPAGE days per scraped webpage ..."; # port.hu|ro provide the today's program based on the localtime on # Hungary. So in other lands e.g. Australia (thx Zsolt Bayer) (TZ: EST/AEST) if # there is f.e. friday 22:38 here in Hungary it is saturday 04:38 # so Zsolt will get the programs not for the requested day (the XML will be # correct, just the wrong day is in) # # we cannot use Date::Manip's Date_ConvTZ, because it does not detects # correctly f.e. the Australia/Melbourne zone. (because it uses `date +%Z` # to get the zone, and date will output EST and not AEST :-(). # [we could not use f.e. `date +%z`, becuase what happen on windows?] # # that means: we will here not set the global FETCHOFFSET to fetch # the "today's" program from everywhere on the world, but we will grab # at first 3 pages (0, -1, +1) to find the correct offset. my $now = parse_date("now"); # developer's options --now: what time is it? (measured in local time) $now = parse_date( $opt_now ) if ($opt_now); t "now=$now"; my $startat = DateCalc($now, "$opt_offset days"); my $startatdate = UnixDate($startat, '%Q'); t "start grabbing from (offset added, localtime): $startatdate"; # make list: which date is which day on the website, we will make grabbing # requests based on the @days array my @days; for (my $i = 1 + $opt_offset; $i < 1 + $opt_offset + $opt_days; $i += $DAYSPERPAGE) { push @days, [ $startatdate, $i ]; # calculate the next date: bump a YYYYMMDD date by $DAYSPERPAGE day $startatdate = UnixDate(DateCalc(parse_date($startatdate), "+ $DAYSPERPAGE days"), '%Q'); die "Could not calculate next grabbing date $days[$#days][0] (+$DAYSPERPAGE days)" if not defined $startatdate; } # This progress bar is for both downloading and parsing. Maybe # they could be separate stages. $bar = new XMLTV::ProgressBar('getting program listings', @days * @portids) if not $opt_quiet; foreach my $date_n_day (@days) { my ($idate, $iday) = @$date_n_day; my $some_success = 0; foreach $ch_did (@portids) { if (! $CHANNELS{$ch_did}) { warn "\nWARNING: Channel with port-id $ch_did no more exists on the site, skipping it's program grabbing!"; next; } my @ps = (($COUNTRY) && (($COUNTRY eq 'hu') || ($COUNTRY eq 'ro'))) ? process_json($idate, xid($ch_did), $ch_did, $iday) : process_table($idate, xid($ch_did), $ch_did, $iday); $some_success = 1 if @ps; worker("xml-writing"); $writer->write_programme($_) foreach @ps; worker("base-parsing"); update $bar if not $opt_quiet; } if (@portids and not $some_success) { warn "failed to get any listings for day $iday, stopping\n"; last; } } $bar->finish() if not $opt_quiet; worker("xml-writing"); $writer->end(); worker("base-parsing"); showworkers(); exit(0); } die; ���������������������������������������������������xmltv-1.4.0/grab/il/��������������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0014201�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/il/test.conf�����������������������������������������������������������������������0000664�0000000�0000000�00000003034�15000742332�0016027�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������channel!199.tv-guide.walla.co.il channel!201.tv-guide.walla.co.il channel!202.tv-guide.walla.co.il channel!204.tv-guide.walla.co.il channel!208.tv-guide.walla.co.il channel!216.tv-guide.walla.co.il channel!217.tv-guide.walla.co.il channel!222.tv-guide.walla.co.il channel!223.tv-guide.walla.co.il channel!225.tv-guide.walla.co.il channel!231.tv-guide.walla.co.il channel!238.tv-guide.walla.co.il channel!255.tv-guide.walla.co.il channel!275.tv-guide.walla.co.il channel!277.tv-guide.walla.co.il channel!289.tv-guide.walla.co.il channel=292.tv-guide.walla.co.il channel=293.tv-guide.walla.co.il channel=294.tv-guide.walla.co.il channel!295.tv-guide.walla.co.il channel!297.tv-guide.walla.co.il channel!301.tv-guide.walla.co.il channel!3045.tv-guide.walla.co.il channel!3046.tv-guide.walla.co.il channel!314.tv-guide.walla.co.il channel!315.tv-guide.walla.co.il channel!316.tv-guide.walla.co.il channel!334.tv-guide.walla.co.il channel!349.tv-guide.walla.co.il channel!353.tv-guide.walla.co.il channel!3547.tv-guide.walla.co.il channel!3551.tv-guide.walla.co.il channel!3555.tv-guide.walla.co.il channel!3588.tv-guide.walla.co.il channel!3594.tv-guide.walla.co.il channel!3595.tv-guide.walla.co.il channel!3599.tv-guide.walla.co.il channel!373.tv-guide.walla.co.il channel!3762.tv-guide.walla.co.il channel!3764.tv-guide.walla.co.il channel!3770.tv-guide.walla.co.il channel!3819.tv-guide.walla.co.il channel!3821.tv-guide.walla.co.il channel!3825.tv-guide.walla.co.il channel!559.tv-guide.walla.co.il channel!77.tv-guide.walla.co.il channel!94.tv-guide.walla.co.il ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/il/tv_grab_il����������������������������������������������������������������������0000775�0000000�0000000�00000022623�15000742332�0016244�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl =pod =head1 NAME tv_grab_il - Grab TV listings for Israel. =head1 SYNOPSIS tv_grab_il --help tv_grab_il --version tv_grab_il --capabilities tv_grab_il --description tv_grab_il [--config-file FILE] [--days N] [--offset N] [--slow] [--output FILE] [--quiet] tv_grab_il --configure [--config-file FILE] tv_grab_il --configure-api [--stage NAME] [--config-file FILE] [--output FILE] tv_grab_il --list-channels [--config-file FILE] [--output FILE] [--quiet] =head1 DESCRIPTION Output TV listings in XMLTV format for many channels available in Israel. The data comes from tv-guide.walla.co.il. 5 days of listings (including the current day) are available. First you must run B<tv_grab_il --configure> to choose which channels you want to receive. Then running B<tv_grab_il> with no arguments will get a listings in XML format for the channels you chose for available days including today. =head1 OPTIONS B<--configure> Prompt for which channels to download and write the configuration file. B<--config-file FILE> Set the name of the configuration file, the default is B<~/.xmltv/tv_grab_il.conf>. This is the file written by B<--configure> and read when grabbing. B<--output FILE> When grabbing, write output to FILE rather than standard output. B<--days N> When grabbing, grab N days rather than all available days. B<--offset N> Start grabbing at today + N days. B<--quiet> Suppress the progress-bar normally shown on standard error. B<--list-channels> Write output giving <channel> elements for every channel available (ignoring the config file), but no programmes. B<--capabilities> Show which capabilities the grabber supports. For more information, see L<http://wiki.xmltv.org/index.php/XmltvCapabilities> B<--version> Show the version of the grabber. B<--help> Print a help message and exit. =head1 ERROR HANDLING If the grabber fails to download data for some channel on a specific day, it will print an error message to STDERR and then continue with the other channels and days. The grabber will exit with a status code of 1 to indicate that the data is incomplete. =head1 ENVIRONMENT VARIABLES The environment variable HOME can be set to change where configuration files are stored. All configuration is stored in $HOME/.xmltv/. On Windows, it might be necessary to set HOME to a path without spaces in it. =head1 SUPPORTED CHANNELS For information on supported channels, see https://tv-guide.walla.co.il/ =head1 AUTHOR The original author was lightpriest. This documentation and parts of the code are based on various other grabbers from the XMLTV project. =head1 SEE ALSO L<xmltv(5)>. =cut use warnings; use strict; use DateTime; use DateTime::Duration; use Encode; use JSON; use POSIX qw(strftime); use XMLTV; use XMLTV::Options qw/ParseOptions/; use XMLTV::ProgressBar; use XMLTV::Configure::Writer; use XMLTV::Get_nice qw(get_nice_tree); my ($opt, $conf) = ParseOptions({ grabber_name => "tv_grab_il", version => "$XMLTV::VERSION", capabilities => [qw/baseline manualconfig apiconfig/], stage_sub => \&config_stage, listchannels_sub => \&write_channels, description => "Israel (tv-guide.walla.co.il)", }); sub config_stage { my ($stage, $conf) = @_; die "Unknown stage $stage" unless $stage eq "start"; my $result; my $writer = new XMLTV::Configure::Writer(OUTPUT => \$result, encoding => 'utf-8'); $writer->start({'generator-info-name' => 'tv_grab_il'}); $writer->end('select-channels'); return $result; } sub fetch_channels { my ($opt, $conf) = @_; my $channels = {}; my $bar = new XMLTV::ProgressBar({ name => "Fetching channels", count => 1, }) unless ($opt->{quiet} || $opt->{debug}); # Get the page containing the list of channels my $tree = XMLTV::Get_nice::get_nice_tree('https://tv-guide.walla.co.il', undef); my @channels = $tree->look_down("_tag", "a", "class", "tv-guide-channels-logos-title", "href", qr{^/channel/\d+$}, ); $bar->update() && $bar->finish && undef $bar if defined $bar; $bar = new XMLTV::ProgressBar({ name => "Parsing result", count => scalar @channels, }) unless ($opt->{quiet} || $opt->{debug}); # Browse through the downloaded list of channels and map them to a hash XMLTV::Writer would understand foreach my $channel (@channels) { if ($channel->as_text()) { my ($id) = $channel->attr('href') =~ m{^/channel/(\d+)$}; $channels->{"$id.tv-guide.walla.co.il"} = { id => "$id.tv-guide.walla.co.il", 'display-name' => [[ encode( 'utf-8', $channel->as_text()) ]], url => [ $channel->attr('href') ] }; my $icon = $channel->look_down('_tag', 'img'); if ($icon) { $icon = $icon->attr('src'); $channels->{"$id.tv-guide.walla.co.il"}->{icon} = [ {src => ($icon || '')} ]; } } $bar->update() if defined $bar; } $bar->finish() && undef $bar if defined $bar; # Notifying the user :) $bar = new XMLTV::ProgressBar({ name => "Reformatting", count => 1, }) unless ($opt->{quiet} || $opt->{debug}); $bar->update() && $bar->finish() if defined $bar; return $channels; } sub write_channels { my $channels = fetch_channels($opt, $conf); # Let XMLTV::Writer format the results as a valid xmltv file my $result; my $writer = new XMLTV::Writer(OUTPUT => \$result, encoding => 'utf-8'); $writer->start({'generator-info-name' => 'tv_grab_il'}); $writer->write_channels($channels); $writer->end(); return $result; } # Fetch the channels again to see what's available my $channels = fetch_channels($opt, $conf); # Configure initial elements for XMLTV::Writer # # Create a new hash for the channels so that channels without programmes # won't appear in the final XML my $encoding = 'UTF-8'; my $credits = {'generator-info-name' => 'tv_grab_il'}; my $w_channels = {}; my $programmes = []; # Progress Bar :) my $bar = new XMLTV::ProgressBar({ name => "Fetching channels listings", count => (scalar @{$conf->{channel}}) * $opt->{days} }) unless ($opt->{quiet} || $opt->{debug}); # Fetch listings per channel foreach my $channel_id (@{$conf->{channel}}) { # Check each channel still exists in walla's channels page if ($channels->{$channel_id}) { my ($walla_id) = ($channel_id =~ /^(\d+)\..*$/); # Now grab listings for each channel on each day, according to the options in $opt # N.B.: only 5 days available, including today # Each channel's 5-day listings are presented on a single page # my $url = "https://tv-guide.walla.co.il/channel/$walla_id"; my $tree = XMLTV::Get_nice::get_nice_tree($url, undef); use Data::Dumper; if ($tree) { for (my $i=$opt->{offset}; $i < ($opt->{offset} + $opt->{days}); $i++) { my $channel_line = $tree->look_down('_tag', 'li', 'class', qr/channel-line/, 'data-id', "$i"); my @shows = $channel_line->look_down('_tag', 'li', 'style', qr/flex/, sub { !defined($_[0]->attr('class')) }); if (@shows) { SHOW: foreach my $show (@shows) { my $prog_data_obj = $show->attr("data-obj"); my $json = decode_json $prog_data_obj; my $title = $json->{'name'}; next SHOW unless $title; my $desc = $json->{'description'}; my ($start_hr, $start_min) = $json->{'start_time'} =~ m/(\d\d):(\d\d)/; my ($end_hr, $end_min) = $json->{'end_time'} =~ m/(\d\d):(\d\d)/; my $show_duration = DateTime::Duration->new( minutes => $json->{'duration'} ); my $current_day = DateTime->today()->add(days => $i)->set_time_zone('Asia/Jerusalem'); my $start_time = $current_day->clone->set(hour => $start_hr, minute => $start_min, second => 0); my $end_time = $start_time + $show_duration; my $prog = { start => $start_time->strftime("%Y%m%d%H%M%S %z"), title => [[ encode('utf-8', $title) ]], channel => $channel_id, }; $prog->{'stop'} = $end_time->strftime("%Y%m%d%H%M%S %z") if defined $end_time; $prog->{'desc'} = [[ encode('utf-8', $desc) ]] if $desc ne ''; push @{$programmes}, $prog; } $w_channels->{$channel_id} = $channels->{$channel_id} unless $w_channels->{$channel_id}; } } } $bar->update if defined $bar; } } $bar->finish() && undef $bar if defined $bar; my %w_args; if (($opt->{offset} != 0) || ($opt->{days} != -999)) { $w_args{offset} = $opt->{offset}; $w_args{days} = ($opt->{days} == -999) ? 100 : $opt->{days}; $w_args{cutoff} = '000000'; } my $data = []; $data->[0] = $encoding; $data->[1] = $credits; $data->[2] = $w_channels; $data->[3] = $programmes; XMLTV::write_data($data, %w_args); �������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/is/��������������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0014210�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/is/category_map��������������������������������������������������������������������0000664�0000000�0000000�00000000222�15000742332�0016601�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������1==children 2==series 3==news 4==educational 5==sports 6==misc 7==movie 8==culture 9==music 11==entertainment 13==news magazine 19==arts festival ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/is/test.conf�����������������������������������������������������������������������0000664�0000000�0000000�00000000104�15000742332�0016031�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������channel RUV Ríkissjónvarpið channel plus.RUV Ríkissjónvarpið+ ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/grab/is/tv_grab_is����������������������������������������������������������������������0000664�0000000�0000000�00000067303�15000742332�0016263�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������#!/usr/bin/perl -w eval 'exec /usr/bin/perl -w -S $0 ${1+"$@"}' if 0; # not running under some shell =pod =encoding utf8 =head1 NAME tv_grab_is - Grab TV listings for Iceland. =head1 SYNOPSIS tv_grab_is --help tv_grab_is [--config-file FILE] --configure [--gui OPTION] tv_grab_is [--config-file FILE] [--output FILE] [--days N] [--offset N] [--quiet] tv_grab_is --capabilities tv_grab_is --version =head1 DESCRIPTION Output TV listings for several channels available in Iceland. First run B<tv_grab_is --configure> to choose, which channels you want to download. Then running B<tv_grab_is> with no arguments will output listings in XML format to standard output. B<--configure> Prompt for which channels, and write the configuration file. B<--config-file FILE> Set the name of the configuration file, the default is B<~/.xmltv/tv_grab_is.conf>. This is the file written by B<--configure> and read when grabbing. B<--gui OPTION> Use this option to enable a graphical interface to be used. OPTION may be 'Tk', or left blank for the best available choice. Additional allowed values of OPTION are 'Term' for normal terminal output (default) and 'TermNoProgressBar' to disable the use of Term::ProgressBar. B<--output FILE> Write to FILE rather than standard output. B<--days N> Grab N days. The default is as many as the source carries. B<--offset N> Start N days in the future. The default is to start from today. B<--quiet> Suppress the progress messages normally written to standard error. B<--capabilities> Show which capabilities the grabber supports. For more information, see L<http://wiki.xmltv.org/index.php/XmltvCapabilities> B<--version> Show the version of the grabber. B<--help> Print a help message and exit. =head1 SEE ALSO L<xmltv(5)>. =head1 AUTHOR Yngvi Þór Sigurjónsson (yngvi@teymi.is). Heavily based on tv_grab_dk by Jesper Skov (jskov@zoftcorp.dk). tv_grab_dk originally based on tv_grab_nl by Guido Diepen and Ed Avis (ed@membled.com) and tv_grab_fi by Matti Airas. Version 1.1, Eggert Thorlacius (eggert@thorlacius.com). Started out by replacing a couple of channels with XML feeds, but ended up by removing the sjonvarp.is code completely. Left in the "xxx.sjonvarp.is" XMLTV IDs for backwards compatibility. =head1 BUGS =cut use strict; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use XMLTV::Capabilities qw/baseline manualconfig cache/; use XMLTV::Description 'Iceland'; use Getopt::Long; use HTML::TreeBuilder; use HTML::Entities; # parse entities use IO::File; use URI; use utf8; # source code is encoded in utf8 use Date::Manip; use XML::LibXSLT; use XML::DOM; use XMLTV::Memoize; use XMLTV::ProgressBar; use XMLTV::Ask; use XMLTV::Mode; use XMLTV::Config_file; use XMLTV::DST; use XMLTV::Get_nice; use XMLTV::Supplement qw/GetSupplement/; use XMLTV::Date; use XMLTV::Usage <<END $0: get Icelandic television listings in XMLTV format To configure: $0 --configure [--config-file FILE] To grab listings: $0 [--config-file FILE] [--output FILE] [--days N] [--offset N] [--quiet] To show capabilities: $0 --capabilities To show version: $0 --version END ; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } if (int(Date::Manip::DateManipVersion) >= 6) { Date::Manip::Date_Init("SetDate=now,UTC"); } else { Date::Manip::Date_Init("TZ=UTC"); } } # default language my $LANG = 'is'; sub basechid($); sub ispluschannel($); sub process_xml_channel( $$$$$$ ); sub process_ruv_is( $$$$$$ ); sub process_skjarinn_is( $$$$$$ ); sub process_stod2_and_friends( $$$$$$ ); sub get_categories_map(); # Get options XMLTV::Memoize::check_argv('XMLTV::Get_nice::get_nice_aux'); my ($opt_days, $opt_offset, $opt_help, $opt_output, $opt_configure, $opt_config_file, $opt_gui, $opt_quiet, $opt_list_channels); $opt_days = 7; # default $opt_offset = 0; # default GetOptions('days=i' => \$opt_days, 'offset=i' => \$opt_offset, 'help' => \$opt_help, 'configure' => \$opt_configure, 'config-file=s' => \$opt_config_file, 'gui:s' => \$opt_gui, 'output=s' => \$opt_output, 'quiet' => \$opt_quiet, 'list-channels' => \$opt_list_channels, ) or usage(0); die 'number of days must not be negative' if (defined $opt_days && $opt_days < 0); usage(1) if $opt_help; XMLTV::Ask::init($opt_gui); my $mode = XMLTV::Mode::mode('grab', # default $opt_configure => 'configure', $opt_list_channels => 'list-channels', ); # File that stores which channels to download. my $config_file = XMLTV::Config_file::filename($opt_config_file, 'tv_grab_is', $opt_quiet); if ($mode eq 'configure') { XMLTV::Config_file::check_no_overwrite($config_file); open(CONF, ">$config_file") or die "cannot write to $config_file: $!"; # find list of available channels my $bar = new XMLTV::ProgressBar('getting list of channels', 1) if not $opt_quiet; my %channels = get_channels(); die 'no channels could be found' if (scalar(keys(%channels)) == 0); update $bar if not $opt_quiet; $bar->finish() if not $opt_quiet; my @chs = sort keys %channels; my @names = map { $channels{$_} } @chs; my @qs = map { "add channel $_?" } @names; my @want = ask_many_boolean(1, @qs); foreach (@chs) { my $w = shift @want; warn("cannot read input, stopping channel questions"), last if not defined $w; # No need to print to user - XMLTV::Ask is verbose enough. # Print a config line, but comment it out if channel not wanted. print CONF '#' if not $w; my $name = shift @names; print CONF "channel $_ $name\n"; # TODO don't store display-name in config file. } close CONF or warn "cannot close $config_file: $!"; say("Finished configuration."); exit(); } # Not configuring, we will need to write some output. die if $mode ne 'grab' and $mode ne 'list-channels'; # If we are grabbing, check we can read the config file before doing # anything else. # my @config_lines; if ($mode eq 'grab') { @config_lines = XMLTV::Config_file::read_lines($config_file); } my %w_args; if (defined $opt_output) { my $fh = new IO::File(">$opt_output"); die "cannot write to $opt_output: $!" if not defined $fh; $w_args{OUTPUT} = $fh; } $w_args{encoding} = 'ISO-8859-1'; $w_args{UNSAFE} = 1; # Needed by process_xml_channel my $writer = new XMLTV::Writer(%w_args); # TODO: standardize these things between grabbers. $writer->start ({ 'generator-info-name' => 'XMLTV', 'generator-info-url' => 'http://xmltv.org/', }); if ($mode eq 'list-channels') { my $bar = new XMLTV::ProgressBar('getting list of channels', 1) if not $opt_quiet; my %channels = get_channels(); die 'no channels could be found' if (scalar(keys(%channels)) == 0); update $bar if not $opt_quiet; foreach my $ch_did (sort(keys %channels)) { my $ch_name = $channels{$ch_did}; my $ch_xid = "$ch_did.sjonvarp.is"; $writer->write_channel({ id => $ch_xid, 'display-name' => [ [ $ch_name ] ], 'icon' => [{'src' => get_icon($ch_did)}] }); } $bar->finish() if not $opt_quiet; $writer->end(); exit(); } # Not configuring or writing channels, must be grabbing listings. die if $mode ne 'grab'; my (%channels, @channels, $ch_did, $ch_name); my $line_num = 1; foreach (@config_lines) { ++ $line_num; next if not defined; # FIXME channel data should be read from the site, and then the # config file only gives the XMLTV ids that are interesting. # # Using internal list of channels now to get correctly encoded # channel names. Config file format remains unchanged for compatibility # if (/^channel:?\s+(\S+)\s+([^\#]+)/) { $ch_did = $1; push @channels, $ch_did; } else { warn "$config_file:$.: bad line\n"; } } %channels = get_channels(); ###################################################################### # begin main program # Fetch map of category number to names my %categories; get_categories_map(); my $now = parse_date('now'); die if not defined $now; # Mapping from cannel ID to URL my %stod2AndFriends = ( "ST2" => "http://www.stod2.is/XML--dagskrar-feed/XML-Stod-2-dagskra-vikunnar", "ST2SPORT" => "http://www.stod2.is/XML--dagskrar-feed/XML-Stod-2-Sport-dagskra-vikunnar", "ST2SPORT2" => "http://www.stod2.is/XML--dagskrar-feed/XML-Stod-2-Sport-2-dagskra-vikunnar", "ST2SPORT3" => "http://www.stod2.is/XML--dagskrar-feed/XML-Stod-2-Sport-3-dagskra-vikunnar", "ST2SPORT4" => "http://www.stod2.is/XML--dagskrar-feed/XML-Stod-2-Sport-4-dagskra-vikunnar", "ST2SPORT5" => "http://www.stod2.is/XML--dagskrar-feed/XML-Stod-2-Sport-5-dagskra-vikunnar", "ST2SPORT6" => "http://www.stod2.is/XML--dagskrar-feed/XML-Stod-2-Sport-6-dagskra-vikunnar", "ST2EXTRA" => "http://www.stod2.is/XML--dagskrar-feed/XML-Stod-2-Extra-dagskra-vikunnar", "ST2BIO" => "http://www.stod2.is/XML--dagskrar-feed/XML-Stod-2-Bio-dagskra-vikunnar", ); # Declare array of { date, num_days, channel_id, channel_xid, function } # and populate it with all pages we need to get. For sites that only # support getting a single day at a time, we create $opt_days entries # per channel, but for the others, we create a single entry my @to_get; my $startday = UnixDate(DateCalc(UnixDate($now, '%Y-%m-%d'), "+ $opt_offset days"), '%Y-%m-%d'); die if not defined $startday; foreach $ch_did (@channels) { $ch_name = $channels{$ch_did}; my $ch_xid = "$ch_did.sjonvarp.is"; $writer->write_channel({ id => $ch_xid, 'display-name' => [ [ $ch_name ] ], 'icon' => [{'src' => get_icon($ch_did)}] }); my $timeoffset = ispluschannel($ch_did) ? 1 : 0; if (basechid ($ch_did) eq 'RUV') { push @to_get, [ $startday, $opt_days, $ch_did, $ch_xid, $timeoffset, \&process_ruv_is ]; } elsif (basechid ($ch_did) eq 'S1') { push @to_get, [ $startday, $opt_days, $ch_did, $ch_xid, $timeoffset, \&process_skjarinn_is ]; } elsif (defined ($stod2AndFriends{basechid ($ch_did)} ) ) { push @to_get, [ $startday, $opt_days, $ch_did, $ch_xid, $timeoffset, \&process_stod2_and_friends ]; } } my %warned_ch_name; # suppress duplicate warnings my $bar = new XMLTV::ProgressBar('fetching data', scalar @to_get) if not $opt_quiet; my @to_get_detailed; my $num_detailed = 0; foreach (@to_get) { my ($date, $num_days, $ch_tvgids_id, $ch_xmltv_id, $timeoffset, $func) = @$_; &{$func}($writer, $date, $num_days, $ch_tvgids_id, $timeoffset, $ch_xmltv_id); update $bar if not $opt_quiet; } $bar->finish() if not $opt_quiet; $writer->end(); ###################################################################### # subroutine definitions # Attepmts to parse director from a string. If successful, director # is removed from original string and returned sub get_director($) { my $director =""; if($_[0] =~ s/Leikstjóri:\s*([^.]*)\.// ) { $director = $2; } return $director; } # Attepmts to parse actors from a string. If successful, actor list # is removed from original string and returned as an array sub get_actors($) { my $actors; if($_[0] =~ s/\s*(Aðalhlutverk:|[mM]eðal leikenda eru|Aðalhlutverk leika|[Íí] aðalhutverkum eru|[Ll]eikendur eru)\s*([^.]*)\.// ) { my @a = split(/, | og /, $2); s/[.]$// foreach @a; push @$actors, @a; } return $actors; } # Cuts "plus." of xmlhannelID if appropriate sub basechid($) { my ( $id ) = @_; if ($id =~ /^plus./) { return substr($id, 5); } return $id; } sub ispluschannel($) { return substr($_[0], 0, 5) eq "plus." } # Takes XML and XSL transform that converts it to a list of <programme> # elements, and writes the elements to the XMLTV writer (after massaging them # a bit) sub process_xml_channel( $$$$$$ ) { my ($writer, $infromdate, $num_days, $timeoffset, $xsl_transform, $xml) = @_; my $fromdate = ParseDate($infromdate); my $enddate = ParseDate(UnixDate(DateCalc($fromdate, "+ " . ($num_days) . " days"), '%Y-%m-%d')); # Step one: Use XSLT to munge XML to XMLTV format... my $parser = XML::LibXML->new(); die unless defined $parser; my $xslt = XML::LibXSLT->new(); die unless defined $xslt; my $stylesheet = $xslt->parse_stylesheet($parser->parse_string($xsl_transform)); # suppress warning message caused by using Perl var rather than a physical document() my $results; { local $SIG{__WARN__} = sub { warn @_ unless (defined $_[0] && $_[0] =~ /^I\/O warning : failed to load external entity "unknown-/); }; $results = $stylesheet->transform($parser->parse_string($xml)); } # Step 2: Loop through all programmes and all their children. # Do some processing on the children and then dump the programme # into the writer. my @programmeElem = $results->getDocumentElement->getElementsByTagName('programme'); foreach my $programme (@programmeElem) { my $myDate = ParseDate($programme->getAttribute('start')); if ((Date_Cmp($myDate, $fromdate) >= 0) && (Date_Cmp($myDate, $enddate) < 0)) { $myDate = DateCalc($myDate, "+ " . $timeoffset . " hours"); # Add one hour if this is a plus channel $myDate = UnixDate($myDate,'%q') . " +0000"; # Convert date to "20081231235900 +0000" $programme->setAttribute('start', $myDate); foreach my $node ($programme->childNodes) { if ($node->nodeName eq 'title' && $node->hasChildNodes()) { my $titleStr = $node->firstChild->data; $titleStr =~ s/ - N.TT$//; # Strip garbage that no one cares about $titleStr =~ s/ - Loka..ttur$//; my $episode_num="$1:$2" if ($titleStr =~ s/\s*\((\d+):(\d+)\)//); if ($episode_num) { my $episodeNode = XML::LibXML::Element->new('episode-num'); $episodeNode->appendTextNode($episode_num); $programme->appendChild($episodeNode); } $node->firstChild->setData($titleStr); } elsif ($node->nodeName eq 'sub-title' && ( !$node->hasChildNodes() || ($node->firstChild->data =~ /^[\n ]*$/))) { $programme->removeChild($node); } elsif ($node->nodeName eq 'desc') { if ($node->hasChildNodes() && ($node->firstChild->data !~ /^[\n ]*$/)) { my $credits = $programme->find("credits")->get_node(0); if( !defined( $credits ) ) { $credits = XML::LibXML::Element->new( 'credits' ); $programme->insertAfter( $credits, $node ); } # If this is the description, extract director and actors my $director = get_director($node->firstChild->data); if (defined $director && $director ne "") { my $dirNode = XML::LibXML::Element->new('director'); $dirNode->appendTextNode($director); $credits->appendChild($dirNode); } my $actors = get_actors($node->firstChild->data); foreach my $actor (@$actors) { my $actNode = XML::LibXML::Element->new('actor'); $actNode->appendTextNode($actor); $credits->appendChild($actNode); } } else { $programme->removeChild($node); } } elsif ($node->nodeName eq 'category') { # FIXME where does the empty category element come from? # [bilbo] sometimes (for no determinable reason) the document lookup is failing - maybe because it's # a Perl var and not a real doc, for some reason...? It fails to load the document() (clue = no "I/O warning" in this case) # Run it again and it's fine. Or not! #if (!$node->hasChildNodes()) { # $programme->removeChild($node); #} # That's just too flaky so let's go "old school" for now (otherwise nightly validator fails) # print STDERR "was: ".$node."\n"; ## Now done as a Supplement as of v1.30 ## my %categories = ( '1'=>'children', '2'=>'series', '3'=>'news', '4'=>'educational', '5'=>'sports', '6'=>'misc', '7'=>'movie', '8'=>'culture', '9'=>'music', '11'=>'entertainment', '13'=>'news magazine' ); # output the category name from the Supplement, else the category numeric value my $category = $categories{$node->firstChild->data} || $node->firstChild->data; $node->removeChildNodes(); $node->appendText($category); # print STDERR "now: ".$node."\n"; } } $writer->raw($programme->toString . "\n"); } } } sub process_ruv_is ( $$$$$$ ){ my ($writer, $fromdate, $num_days, $tv2chan, $timeoffset, $ch_xmltv_id) = @_; my $xsl_transform = <<"EOF"; <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0" xmlns:data="urn:some.urn" exclude-result-prefixes="data"> <xsl:output method="xml" version="1.0" encoding="utf-8" indent="yes" /> <data:category> <entry key="1">children</entry> <entry key="2">series</entry> <entry key="3">news</entry> <entry key="4">educational</entry> <entry key="5">sports</entry> <entry key="6">misc</entry> <entry key="7">movie</entry> <entry key="8">culture</entry> <entry key="9">music</entry> <entry key="11">entertainment</entry> <entry key="13">news magazine</entry> </data:category> <xsl:template match="/"> <!-- XML needs a toplevel node, so we wrap the programmes in a channel_wrapper element. The code will then remove it --> <channel_wrapper> <xsl:apply-templates/> </channel_wrapper> </xsl:template> <xsl:template match="event"> <programme channel="$ch_xmltv_id" start=""> <xsl:attribute name="start"> <xsl:value-of select='translate(\@start-time,"- :","")'/> </xsl:attribute> <title> <xsl:apply-templates select="title"/> EOF # [bilbo] following code removed in v1.30, ... # # # # # # # # # # <!-- category is no good: --> # # ..and replaced with: $xsl_transform .= <<"EOF"; EOF $xsl_transform .= <<"EOF"; : EOF # Get the XML. Note that due to some edge cases, we must get data for the day before # and after the ones we're looking for and let process_xml_cannel remove most of the # data. If we don't, tv_validate_grabber will fail my $dayBefore = UnixDate(DateCalc($fromdate, "- 1 days"), '%Y-%m-%d'); my $dayAfter = UnixDate(DateCalc($fromdate, "+ " . ($num_days + 1) . " days"), '%Y-%m-%d'); my $url = "http://muninn.ruv.is/files/xml/sjonvarpid/$dayBefore/$dayAfter/"; my $xml = get_nice( $url ); die( "Failed to fetch $url" ) unless defined( $xml ); process_xml_channel($writer, $fromdate, $num_days, $timeoffset, $xsl_transform, $xml); } sub process_skjarinn_is ( $$$$$$ ){ my ($writer, $fromdate, $num_days, $tv2chan, $timeoffset, $ch_xmltv_id) = @_; # NOTE: For Skjar 1, we ignore $fromdate and $num_days and always load the next five days. my $xsl_transform = <<"EOF"; <xsl:apply-templates select="title"/> EOF my $url = "http://skjareinn.is/einn/dagskrarupplysingar/?channel_id=7&weeks=2&output_format=xml"; my $xml = get_nice( $url ); die( "Failed to fetch $url" ) unless defined( $xml ); process_xml_channel($writer, $fromdate, $num_days, $timeoffset, $xsl_transform, $xml); } sub process_stod2_and_friends ( $$$$$$ ){ my ($writer, $fromdate, $num_days, $tv2chan, $timeoffset, $ch_xmltv_id) = @_; # NOTE: For Stod 2 et al, we ignore $fromdate and $num_days and always load the next week. # [honir] I don't know what this piece of code was intending to do: # # Specifically # event[not(reference_number/\@value=following::event/reference_number/\@value)] # ignores nodes where the same 'reference_number' occurs on any following node. But the 'reference_number' is not unique to a showing, i.e. a repeat # showing of the programme will have the same 'reference_number'; however this code will ignore all matching 'event' except the last! # I.e. we only grab the last repeated showing of a programme. This can't be what was intended! # my $xsl_transform = <<"EOF"; <xsl:apply-templates select="title"/> : EOF my $url = $stod2AndFriends{basechid($tv2chan)}; my $xml = get_nice( $url ); die( "Failed to fetch $url" ) unless defined( $xml ); process_xml_channel($writer, $fromdate, $num_days, $timeoffset, $xsl_transform, $xml); } # get channel listing sub get_channels { # I'm too lazy to figure out how to append plus channels programatically return ( "RUV" => "Ríkissjónvarpið", # "S1" => "Skjár 1", # "ST2" => "Stöð 2", # "ST2SPORT" => "Stöð 2 Sport", # "ST2SPORT2" => "Stöð 2 Sport 2", # "ST2EXTRA" => "Stöð 2 Extra", # "ST2BIO" => "Stöð 2 bíó", "plus.RUV" => "Ríkissjónvarpið+", # "plus.S1" => "Skjár 1+", # "plus.ST2" => "Stöð 2+", # "plus.ST2SPORT" => "Stöð 2 Sport+", # "plus.ST2SPORT2" => "Stöð 2 Sport 2+", # "plus.ST2EXTRA" => "Stöð 2 Extra+", # "plus.ST2BIO" => "Stöð 2 bíó+", ); } # Icon URL for a given channel. sub get_icon { my ($ch_did) = @_; $ch_did = basechid($ch_did); my %logos = ( "RUV" => "ruv", "S1" => "s1", "ST2" => "stod2", "ST2SPORT" => "2sport", "ST2SPORT2" => "2sport2", "ST2EXTRA" => "2extra", "ST2BIO" => "2bio", ); if (defined($logos{$ch_did})) { return "http://www.sjonvarp.is/images/logos/".$logos{$ch_did}.".gif"; } return ""; } # Fetch the map of category numbers from the Supplement server sub get_categories_map () { my $supplement = GetSupplement('tv_grab_is', 'category_map'); my @lines = split /[\r\n]+/, $supplement; foreach my $line (@lines) { chomp $line; chop($line) if ($line =~ m/\r$/); trim($line); next if $line =~ /^#/ || $line eq ''; my ($cat_num, $cat_name, $trash) = $line =~ /^(.*)==(.*?)([\s\t]*#.*)?$/; $categories{$cat_num} = $cat_name; } #use Data::Dumper; print Dumper (\%categorymap); } sub trim { # Remove leading & trailing spaces $_[0] =~ s/^\s+|\s+$//g; } xmltv-1.4.0/grab/it/000077500000000000000000000000001500074233200142115ustar00rootroot00000000000000xmltv-1.4.0/grab/it/channel_ids000066400000000000000000000546421500074233200164160ustar00rootroot00000000000000[skylife] www.raitre.rai.it;Rai Tre www.raiuno.rai.it;Rai Uno www.raidue.rai.it;Rai Due www.canale5.com;Canale 5 www.italia1.com;Italia 1 www.rete4.com;Rete 4 www.la7.it;La7 www.canaljimmy.it;Jimmy fox.skytv.it;Fox tve.skytv.it;TVE canal24horas.skytv.it;Canal 24 Horas live.skytv.it;Live! skyuno.skytv.it;Sky Uno skyunopiuuno.skytv.it;Sky Uno + 1 foxcrime.skytv.it;Fox Crime foxcrimetraccecriminali.skytv.it;Fox Crime Tracce Criminali foxcomedyhd.skylife.it;Fox Comedy HD paramountcomedy.skytv.it;Comedy Central moviepromo.skytv.it;Movie Promo sailingchannel.skytv.it;Sailing Channel test7.skytv.it;Test 7 sportspromo.skytv.it;Sports Promo inarrivo.skytv.it;In Arrivo adventureone.skytv.it;Adventure One mtchannel.skytv.it;MT Channel planete.skytv.it;Planete mosaicoaudio.skytv.it;Mosaico Audio gxt.skytv.it;GXT novehd.sky.it;NOVE HD gxtpiuuno.skytv.it;GXT +1 animalplanet.skytv.it;Animal Planet jetix.skytv.it;Jetix jetix1.skytv.it;Jetix +1 eentertaiment.skytv.it;E! Entertainment skyonair.skytv.it;Sky On Air extra.raisat.it;Raisat Extra premium.raisat.it;Raisat Premium skycinema1.skytv.it;Sky Cinema 1 skycinema2.skytv.it;SKY Cinema 2 skycinema3.skytv.it;SKY Cinema 3 skycinemahits.skytv.it;Sky Cinema Hits skycinemaautore.skytv.it;SKY Cinema Aut skycinemamax.skytv.it;Sky Cinema Max skycinema169.skytv.it;SKY Cinema 16:9 skysport169.skytv.it;SKY Sport 16:9 www.studiouniversal.it;Studio Universal cinemaworld.raisat.it;Raisat Cinema W. skycinemaclassics.skytv.it;Sky Cinema Classics skycinemaoscarhd.skytv.it;Sky Cinema Oscar HD foxlife.skytv.it;Fox Life www.happychannel.it;Happy Channel dueltv.skytv.it;Duel TV skytg24meteo.skytv.it;Sky TG 24 Meteo www.ngcitalia.it;National Geo www.thehistorychannel.co.uk;History Channel gamberorosso.raisat.it;Raisat Gambero Rosso www.marcopolo.tv;Marcopolo www.alice.tv;Alice www.leonardo.tv;Leonardo www.discovery-italia.com;Discovery Channel science.discovery-italia.com;Discovery Science civilisation.discovery-italia.com;Discovery Civilization rt.discovery-italia.com;Real Time a1.skytv.it;A1 piu1.thehistorychannel.co.uk;History Ch +1 piu1.ngcitalia.it;National Geo +1 piu1.cartoonnetwork.it;Cartoon network +1 www.cartoonnetwork.it;Cartoon Network cartoonnetworkhd.guidatv.sky.it;Cartoon Network HD hits.mtv.it;MTV Hits brandnew.mtv.it;MTV Brand New boomerang.cartoonnetwork.it;Boomerang www.disneychannel.it;Disney Channel ragazzi.raisat.it;Raisat Ragazzi www.italiateen.it;Italian Teen TV www.videoitalia.it;Video Italia www.deejay.it;Deejay TV www.nove.tv;Deejay TVNOVE www.matchmusic.it;Match Music www.rocktv.it;Rock TV nickelodeon.skytv.it;Nickelodeon www.bbcprime.com;BBC Prime skysport1.skytv.it;Sky Sport 1 skysport2.skytv.it;Sky Sport 2 skysport3.skytv.it;Sky Sport 3 sportitalia24.skytv.it;SportItalia 24 skysportextra.skytv.it;Sky Sport Extra primafila01.skytv.it;Primafila 1 primafila02.skytv.it;Primafila 2 primafila03.skytv.it;Primafila 3 primafila04.skytv.it;Primafila 4 primafila05.skytv.it;Primafila 5 primafila06.skytv.it;Primafila 6 primafila07.skytv.it;Primafila 7 primafila08.skytv.it;Primafila 8 primafila09.skytv.it;Primafila 9 primafila10.skytv.it;Primafila 10 primafila11.skytv.it;Primafila 11 primafila12.skytv.it;Primafila 12 primafila13.skytv.it;Primafila 13 primafila14.skytv.it;Primafila 14 primafila15.skytv.it;Primafila 15 primafila16.skytv.it;Primafila 16 primafila17.skytv.it;Primafila 17 primafila18.skytv.it;Primafila 18 primafila19.skytv.it;Primafila 19 primafila20.skytv.it;Primafila 20 primafila21.skytv.it;Prima Fila 21 primafila22.skytv.it;Prima Fila 22 primafila23.skytv.it;Prima Fila 23 primafila24.skytv.it;Prima Fila 24 primafila25.skytv.it;Prima Fila 25 primafila26.skytv.it;Primafila 26 primafila27.skytv.it;Primafila 27 primafila28.skytv.it;Primafila 28 primafila29.skytv.it;Prima Fila 29 primafila30.skytv.it;Prima Fila 30 primafila31.skytv.it;Prima Fila 31 primafila32.skytv.it;Prima Fila 32 primafila33.skytv.it;Prima Fila 33 primafila34.skytv.it;Prima Fila 34 primafila35.skytv.it;Prima Fila 35 primafila36.skytv.it;Prima Fila 36 primafila37.skytv.it;Prima fila 37 primafila38.skytv.it;Prima Fila 38 primafila39.skytv.it;Prima Fila 39 primafila40.skytv.it;Prima fila 40 primafilaHD.skytv.it;Prima Fila HD calciosky01.skytv.it;Sky Calcio 1 calciosky02.skytv.it;Sky Calcio 2 calciosky03.skytv.it;Sky Calcio 3 calciosky04.skytv.it;Sky Calcio 4 calciosky05.skytv.it;Sky Calcio 5 calciosky06.skytv.it;Sky Calcio 6 calciosky07.skytv.it;Sky Calcio 7 calciosky08.skytv.it;Sky Calcio 8 calciosky09.skytv.it;Sky Calcio 9 calciosky10.skytv.it;Sky Calcio 10 calciosky11.skytv.it;Sky Calcio 11 calciosky12.skytv.it;Sky Calcio 12 calciosky13.skytv.it;Sky Calcio 13 calciosky14.skytv.it;Sky Calcio 14 calciosky15.skytv.it;Sky Calcio 15 calciosky16.skytv.it;Sky Calcio 16 skysport24.skytv.it;Sky Sport24 foxfx.skytv.it;FX canale132.skytv.it;Canale 132 www.sportitalia.com;Sportitalia cacciaepesca.skytv.it;Caccia e Pesca www.eurosport.com;Eurosport hd.eurosport.com;Eurosport HD eurosport2.skytv.it;Eurosport 2 eurosportnews.skytv.it;Eurosport News www.espnclassicsport.com;ESPN Classic www.nuvolari.tv;Nuvolari www.snai.it;Snai Sat www.milanchannel.it;Milan Channel torinochannel.guidatv.sky.it;Torino Channel milantvhd.guidatv.sky.it;Milan TV HD interchannel.inter.it;Inter Channel interchannelhd.inter.it;Inter TV HD #www.asromacalcio.it;Roma Channel www.asromacalcio.it;Roma TV direttagol.skytv.it;SKY Diretta GOL skytg24.skytv.it;Sky TG24 www.tv5.org;TV5 Europe tv5monde.skytv.it;TV5 Monde www.cfn.it;CFN (Italian) www.bloomberg.it;Bloomberg www.reteconomy.it;Reteconomy www.rainews24.rai.it;Rai News edition.cnn.com;CNN Intl www.skynews.it;Sky News www.foxnews.com;Fox News www.cnbceurope.com;CNBC www.bbcworld.com;BBC World www.trmtv.it;TRM h24 classica.skytv.it;Classica www.hallmarkchannel.com;Hallmark Channel cultcni.skytv.it;Cult 109.skytv.it;SKY Canale 109 www.cfn.tv;Class-Cnbc skyvivo.skytv.it;SKY Vivo www.axn.it;AXN piuuno.axn.it;AXN + 1 cinema.raisat.it;Raisat Cinema skycinemahd.skytv.it;SKY Cinema HD skysporthd2.skytv.it;SKY Sport HD2 skysporthd_2.skytv.it;SKY Sport HD 2 skysporthd1.skytv.it;SKY Sport HD1 nationalgeohd.skytv.it;National Geo HD www.reteallmusic.it;ALL Music fantasy.skytv.it;Fantasy nexthd.skytv.it;Next HD piu.cartoonnetwork.it;Cartoon Network +1 smash.raisat.it;Raisat Smash playhousedisney.skytv.it;Play House Disney piu1.disneychannel.it;Disney Channel +1 toondisney.skytv.it;Toon Disney deejay50.skytv.it;DEEJAY 50 SONGS hititalia.skytv.it;HIT ITALIA italianvintage.skytv.it;ITALIAN VINTAGE yesterjay80.skytv.it;YESTERJAY '80 yesterjay90.skytv.it;YESTERJAY '90 capital70.skytv.it;CAPITAL '70 vintage60.skytv.it;VINTAGE '60 soulsista.skytv.it;SOULSISTA newrock.skytv.it;NEW ROCK heartnsong.skytv.it;HEART N' SONG m20.skytv.it;M2O OUT OF MIND ondalatina.skytv.it;ONDA LATINA capitallivetime.skytv.it;CAPITAL LIVETIME capitalcrock.skytv.it;CAPITAL C. ROCK rockshock.skytv.it;ROCK SHOCK bside.skytv.it;B-SIDE capitalgmaster.skytv.it;CAPITAL G. MASTER jazzgold.skytv.it;JAZZ GOLD soultrain.skytv.it;SOUL TRAIN extrabeat.skytv.it;EXTRA BEAT sinfonia.skytv.it;SINFONIA opera.skytv.it;OPERA stardust.skytv.it;STARDUST babymix.skytv.it;BABY MIX disckjoker.skytv.it;DISC JOKER skysporthd.skytv.it;SKY Sport HD juventuschannel.skytv.it;Juventus Channel juventustv.sky.it;Juventus TV skycinemamania.skytv.it;Sky Cinema Mania skyshow.skytv.it;SKY Show jimjam.skytv.it;Jim Jam skycalcioinfo.skytv.it;SKY Calcio Info skyprimafila.skytv.it;Sky PrimaFila natgeoadventure.skytv.it;Nat Geo Adventure travel.discovery-italia.com;DISCOVERY TRAVEL & LIVING www.moto.tv;Moto Tv english.aljazeera.net;Al Jazeera International www.france24.com;France 24 www.cultoon.tv;Cultoon www.cooltoon.tv;Cooltoon musicbox.skytv.it;Music Box foxcrimepiu1.skytv.it;Fox Crime +1 foxcrimepiu2.skytv.it;Fox Crime +2 jazz.skylife.it;JAZZ natgeowild.skylife.it;Nat Geo Wild natgeowildpiu1.skylife.it;Nat Geo Wild +1 nategomusic.skylife.it;Nat Geo Music skysupercalcio.skylife.it;Sky Supercalcio yachtandsail.skylife.it;Yacht & Sail mtvgold.skylife.it;MTV Gold mtvpulse.skylife.it;MTV Pulse vh1.skylife.it;VH1 foxpiuuno.skylife.it;Fox +1 foxlifepiuuno.skylife.it;Fox Life +1 jazzefusion.skylife.it;Jazz&Fusion www.mgmchannel.it;The MGM Channel www.nasn.com;NASN news.bbc.co.uk;BBC World News current.it;Current TV dovetv.skytv.it;Dove skyradio.skytv.it;Sky Radio skymusic.skytv.it;Sky Music la3.skytv.it;La3 manga.skytv.it;MAN-GA skycinemapiu1.skytv.it;Sky Cinema + 1 skycinemapiu24.skytv.it;Sky Cinema + 24 skycinemafamily.skytv.it;Sky Cinema Family skycinemaitalia.skytv.it;Sky Cinema Italia skymaxpiu1.skytv.it;Sky Cinema Max + 1 discoverypiu1.skytv.it;Discovery +1 espnamerica.skytv.it;ESPN America foxanimationhd.skylife.it;Fox Animation HD deakids.skytv.it;DeAKids boomerangpiu1.skytv.it;Boomerang +1 smashgirls.skytv.it;SmashGirls toondisneypiu1.skytv.it;Toon Disney + 1 disneyinenglish.skytv.it;Disney in English deakidspiu1.skytv.it;DeAKids +1 voce.skytv.it;Voce hiphoptv.skytv.it;Hip Hop TV mediasetplus.skytv.it;Mediaset Plus lei.skytv.it;Lei skycinemahdpiu24.skytv.it;SKY Cinema HD + 24 www.euronews.net;Euronews supertennis.skytv.it;SuperTennis mgm.guidatv.sky.it;MGM gamberorosso.guidatv.sky.it;Gambero Rosso discoverytravelliving.guidatv.sky.it;Discovery Travel & Living skyolimpia1.guidatv.sky.it;SKY Olimpia 1 sportitalia2.guidatv.sky.it;Sportitalia 2 pescaecaccia.guidatv.sky.it;Pesca e Caccia disneyxd.guidatv.sky.it;Disney XD disneyxd1.guidatv.sky.it;Disney XD +1 nickjr.guidatv.sky.it;Nick JR nickjr1.guidatv.sky.it;Nick JR +1 nickelodeon1.guidatv.sky.it;Nickelodeon +1 playhouse.guidatv.sky.it;Playhouse + playhousedisney.guidatv.sky.it;Playhouse Disney primafilahd1.guidatv.sky.it;Prima Fila HD1 primafilahd2.guidatv.sky.it;Prima Fila HD2 k2.guidatv.sky.it;K2 babytv.guidatv.sky.it;Baby TV mydeejay.guidatv.sky.it;My Deejay foxretro.guidatv.sky.it;FOX Retro ladychannel.guidatv.sky.it;Lady Channel cielo.guidatv.sky.it;Cielo ondalatina.guidatv.sky.it;Onda Latina comedycentral1.guidatv.sky.it;Comedy +1 bbcentertainment.guidatv.sky.it;BBC Entertainment deasuper.skytv.it;DeaSuper! primocanale.skytv.it;Primocanale videolina.skytv.it;Videolina skysport3d.skytv.it;Sky Sport 3D tgnorba24.skytv.it;TG Norba24 italia7.skytv.it;Italia 7 mtvpiu.skytv.it;MTV + weddingtv.guidatv.sky.it;Wedding Tv tg24primopiano.guidatv.sky.it;TG24 Primo Piano tg24eventi.guidatv.sky.it;TG24 Eventi tg24rassegne.guidatv.sky.it;Sky TG24 Rassegne telelombardiasat.guidatv.sky.it;Telelombardia Sat foxlife2.guidatv.sky.it;Fox Life +2 foxcrime2.guidatv.sky.it;Fox Crime+2 realtime1.guidatv.sky.it;Real Time+1 axnscifi.guidatv.sky.it;AXN Sci-Fi easybaby.guidatv.sky.it;Easy Baby arturo.guidatv.sky.it;Arturo babeltv.guidatv.sky.it;Babel TV www.mtv.it;MTV inevidenza.guidatv.sky.it;In evidenza antenna3nordest.guidatv.sky.it;Antenna 3 Nord Est telesur.guidatv.sky.it;TeleSUR mtvclassic.guidatv.sky.it;MTV Classic mtvrocks.guidatv.sky.it;MTV Rocks mtvdance.guidatv.sky.it;MTV Dance www.boingtv.it;Boing canale5dtt.guidatv.sky.it;Canale 5 (DTT) canale51.guidatv.sky.it;Canale 5 + 1 classhorsetv.guidatv.sky.it;Class Horse TV disneyjunior.mediasetpremium.mediaset.it;Disney Junior disneyjunior.guidatv.sky.it;Disney Junior + iris.mediaset.it;Iris divauniversal.guidatv.sky.it;Diva Universal foxbusiness.guidatv.sky.it;Fox Business frisbee.guidatv.sky.it;Frisbee skycinemacomedy.guidatv.sky.it;Sky Cinema Comedy skycinemapassion.guidatv.sky.it;Sky Cinema Passion nickjunior.guidatv.sky.it;Nick Junior nickjunior1.guidatv.sky.it;Nick Junior + 1 teennick.guidatv.sky.it;TeenNick italia1dtt.guidatv.sky.it;Italia 1 (DTT) italia11.guidatv.sky.it;Italia 1 + 1 jimjam1.guidatv.sky.it;Jim Jam + 1 rete4dtt.guidatv.sky.it;Rete 4 (DTT) rete41.guidatv.sky.it;Rete 4 + 1 youme.guidatv.sky.it;You & Me mediasetextra.guidatv.sky.it;Mediaset Extra mtvmusic.guidatv.sky.it;MTV Music pokeritalia24.guidatv.sky.it;PokerItalia24 qvc.guidatv.sky.it;QVC la5.guidatv.sky.it;La5 www.la7d.it;La7D rai1dtt.guidatv.sky.it;Rai 1 (DTT) rai2dtt.guidatv.sky.it;Rai 2 (DTT) rai3dtt.guidatv.sky.it;Rai 3 (DTT) rai4dtt.guidatv.sky.it;Rai 4 DTT rai4.raisat.it;Rai 4 rai5.rai.it;Rai 5 raigulp.rai.it;Rai Gulp raimovie.rai.it;Rai Movie raipremium.guidatv.sky.it;Rai Premium raisport.rai.it;Rai Sport raisport1.rai.it;Rai Sport 1 raisport2.rai.it;Rai Sport 2 raistoria.rai.it;Rai Storia yoyo.raisat.it;Rai Yoyo tv8.guidatv.sky.it;TV8 natgeoadventurehd.guidatv.sky.it;Nat Geo Adventure HD worldfashionchannel.guidatv.sky.it;World Fashion Channel fashiontv.guidatv.sky.it;Fashion Tv fashionone.guidatv.sky.it;Fashion One sky3dch209.guidatv.sky.it;Sky 3D - Ch 209 romauno.guidatv.sky.it;Roma Uno smtvsanmarino.guidatv.sky.it;SMtv San Marino nhkworldtv.guidatv.sky.it;NHK World TV russiatoday.guidatv.sky.it;Russia Today rtdochd.guidatv.sky.it;RT Doc HD protvinternational.guidatv.sky.it;ProTV International disneyjunior1.guidatv.sky.it;Disney Junior + 1 disneychannel2.guidatv.sky.it;Disney Channel +2 disneyxd2.guidatv.sky.it;Disney XD +2 deajunior.guidatv.sky.it;DeA Junior aljazeerachildren.guidatv.sky.it;Al Jazeera Children baraem.guidatv.sky.it;Baraem mtvlivehd.guidatv.sky.it;MTV Live HD rtl1025tv.guidatv.sky.it;RTL 102.5 TV virginradiotv.guidatv.sky.it;Virgin Radio TV horrorchannel.guidatv.sky.it;Horror Channel pawpatrol.guidatv.sky.it;Paw Patrol extremesportchannelhd.guidatv.sky.it;Extreme Sport Channel HD sky3dch150.guidatv.sky.it;Sky 3D - Ch 150 iliketv.guidatv.sky.it;ILIKETV sky3dch321.guidatv.sky.it;Sky 3D - Ch 321 doctorslife.guidatv.sky.it;Doctor's Life supertennishd.guidatv.sky.it;SuperTennis HD super.guidatv.sky.it;Super! skycinema1hd.guidatv.sky.it;Sky Cinema Uno HD skycinemapiu1hd.guidatv.sky.it;Sky Cinema + 1 HD skycinema24hd.guidatv.sky.it;Sky Cinema + 24 HD skycinemahitshd.guidatv.sky.it;Sky Cinema Hits HD skycinemafamilyhd.guidatv.sky.it;Sky Cinema Family HD skycinemafamily1hd.guidatv.sky.it;Sky Cinema Family +1 HD skycinemapassionhd.guidatv.sky.it;Sky Cinema Passion HD skycinemacomedyhd.guidatv.sky.it;Sky Cinema Comedy HD skycinemamaxhd.guidatv.sky.it;Sky Cinema Max HD skycinemamax1hd.guidatv.sky.it;Sky Cinema Max +1 HD skycinemaculthd.guidatv.sky.it;Sky Cinema Cult HD skycinemaclassicshd.guidatv.sky.it;Sky Cinema Classics HD skyartehd400.guidatv.sky.it;Sky Arte HD-400 discoverychannelhd.guidatv.sky.it;Discovery Channel HD discoverysciencehd.guidatv.sky.it;Discovery Science HD discoverytravellivinghd.guidatv.sky.it;Discovery Travel & Living HD historychannelhd.guidatv.sky.it;History Channel HD natgeowildhd.guidatv.sky.it;Nat Geo Wild HD gamberorossohd.guidatv.sky.it;Gambero Rosso HD deasaperehd.guidatv.sky.it;DeA Sapere HD skysport24hd.guidatv.sky.it;Sky Sport24 HD skysport1hd.guidatv.sky.it;Sky Sport 1 HD skysport2hd.guidatv.sky.it;Sky Sport 2 HD skysport3hd.guidatv.sky.it;Sky Sport 3 HD skysupercalciohd.guidatv.sky.it;Sky Supercalcio HD foxsportshd.guidatv.sky.it;Fox Sports HD skysportf1hd.guidatv.sky.it;Sky Sport F1 HD foxsportsplushd.guidatv.sky.it;Fox Sports Plus HD eurosport2hd.guidatv.sky.it;Eurosport 2 HD bikechannel.guidatv.sky.it;Bike Channel bikechannelhd.guidatv.sky.it;Bike Channel HD juventustv.guidatv.sky.it;JUVENTUS tv laziostylechannel.guidatv.sky.it;Lazio Style Channel skycalcio1hd.guidatv.sky.it;Sky Calcio 1 HD skycalcio2hd.guidatv.sky.it;Sky Calcio 2 HD skycalcio3hd.guidatv.sky.it;Sky Calcio 3 HD skycalcio4hd.guidatv.sky.it;Sky Calcio 4 HD skycalcio5hd.guidatv.sky.it;Sky Calcio 5 HD skycalcio6hd.guidatv.sky.it;Sky Calcio 6 HD skycalcio7hd.guidatv.sky.it;Sky Calcio 7 HD skycalcio8hd.guidatv.sky.it;Sky Calcio 8 HD skytg24hd.guidatv.sky.it;Sky TG24 HD france24english.guidatv.sky.it;France 24 English disneychannelhd.guidatv.sky.it;Disney Channel HD disneyxdhd.guidatv.sky.it;Disney XD HD planetkids.guidatv.sky.it;Planet Kids skyunohd.guidatv.sky.it;Sky Uno HD skyuno1hd.guidatv.sky.it;Sky Uno +1 HD skycinemapiratideicaraibihd.guidatv.sky.it;Sky Cinema Pirati dei Caraibi HD foxhd.guidatv.sky.it;Fox HD fox2.guidatv.sky.it;Fox +2 foxlifehd.guidatv.sky.it;Fox Life HD foxcrimehd.guidatv.sky.it;Fox Crime HD axnhd.guidatv.sky.it;AXN HD realtimehd.guidatv.sky.it;Real Time HD lei1.guidatv.sky.it;LEI +1 skyartehd.guidatv.sky.it;Sky Arte HD automototv.guidatv.sky.it;Automoto TV tv2000.guidatv.sky.it;TV2000 dmax.guidatv.sky.it;DMAX dmaxhd.guidatv.sky.it;DMAX HD italia2.mediaset.it;Mediaset Italia Due DTT natgeopeoplehd.guidatv.sky.it;Nat Geo People HD focustv.guidatv.sky.it;Focus Tv lasposa.guidatv.sky.it;La Sposa skysportmotogphd.guidatv.sky.it;Sky Sport MotoGP HD skysportmixhd.guidatv.sky.it;Sky Sport Mix HD skysportplushd.guidatv.sky.it;Sky Sport Plus HD foxsports2hd.guidatv.sky.it;Fox Sports 2 HD uniresat.guidatv.sky.it;UnireSat i24news.guidatv.sky.it;I24news mtvhd.guidatv.sky.it;MTV HD radioitaliatv.guidatv.sky.it;Radio Italia Tv radionorbatv.guidatv.sky.it;Radionorba TV skyatlantichd.guidatv.sky.it;Sky Atlantic HD skyatlantichd1.guidatv.sky.it;Sky Atlantic +1 HD blazehd.guidatv.sky.it;Blaze HD cicrimeinvestigationhd.guidatv.sky.it;CI Crime+ Investigation HD cielohd.guidatv.sky.it;Cielo HD dmax1.guidatv.sky.it;DMAX +1 classicahd.guidatv.sky.it;Classica HD laeffe.guidatv.sky.it;Laeffe giallo.guidatv.sky.it;Giallo primafila03hd.guidatv.sky.it;Primafila 3 HD primafila05hd.guidatv.sky.it;Primafila 5 HD primafila07hd.guidatv.sky.it;Primafila 7 HD primafila09hd.guidatv.sky.it;Primafila 9 HD primafila11hd.guidatv.sky.it;Primafila 11 HD primafila13hd.guidatv.sky.it;Primafila 13 HD la7dtt.guidatv.sky.it;La7 DTT mtvdtt.guidatv.sky.it;MTV DTT deejaytvdtt.guidatv.sky.it;Deejay TV DTT cielodtt.guidatv.sky.it;Cielo DTT tv2000dtt.guidatv.sky.it;TV2000 DTT realtimedtt.guidatv.sky.it;Real Time DTT qvcdtt.guidatv.sky.it;QVC DTT rtl1025tvdtt.guidatv.sky.it;RTL 102.5 TV DTT topcrime.guidatv.sky.it;Top Crime k2kidstvdtt.guidatv.sky.it;K2 Kids Tv DTT frisbeedtt.guidatv.sky.it;Frisbee DTT cartoonito.guidatv.sky.it;Cartoonito superdtt.guidatv.sky.it;Super! DTT rainewsdtt.guidatv.sky.it;Rai News DTT raisportdtt.guidatv.sky.it;Rai Sport DTT raisport1dtt.guidatv.sky.it;Rai Sport 1 DTT raisport2dtt.guidatv.sky.it;Rai Sport 2 DTT supertennisdtt.guidatv.sky.it;SuperTennis DTT mtvmusicdtt.guidatv.sky.it;MTV Music DTT france24dtt.guidatv.sky.it;France 24 DTT tv8dtt.guidatv.sky.it;TV8 DTT novedtt.guidatv.sky.it;NOVE DTT giallodtt.guidatv.sky.it;GIALLO DTT skytv24dtt2.guidatv.sky.it;Sky Tg24 DTT2 dmaxdtt.guidatv.sky.it;DMAX DTT focusdtt.guidatv.sky.it;Focus DTT [boingtv] www.boingtv.it;boingtv [sitcom1] www.sitcom1.it;sitcom1 [iris] iris.mediaset.it;iris [raisat] cinema.raisat.it;CINEMA extra.raisat.it;EXTRA gamberorosso.raisat.it;GAMBERO ROSSO premium.raisat.it;PREMIUM smash.raisat.it;SMASH yoyo.raisat.it;YOYO rai4.raisat.it;RAI4 [mediasetpremium] direttacalcio1.mediasetpremium.mediaset.it;Premium Calcio 1 direttacalcio2.mediasetpremium.mediaset.it;Premium Calcio 2 direttacalcio3.mediasetpremium.mediaset.it;Premium Calcio 3 direttacalcio4.mediasetpremium.mediaset.it;Premium Calcio 4 direttacalcio5.mediasetpremium.mediaset.it;Premium Calcio 5 direttacalcio6.mediasetpremium.mediaset.it;Premium Calcio 6 disney.mediasetpremium.mediaset.it;Disney disney1.mediasetpremium.mediaset.it;Disney +1 joi.mediasetpremium.mediaset.it;Joi joi1.mediasetpremium.mediaset.it;Joi +1 mya.mediasetpremium.mediaset.it;Mya mya1.mediasetpremium.mediaset.it;Mya +1 premiumcalcio24.mediasetpremium.mediaset.it;Premium Calcio steel.mediasetpremium.mediaset.it;Premium Action steel1.mediasetpremium.mediaset.it;Steel +1 hiro.mediasetpremium.mediaset.it;Hiro playhouse.mediasetpremium.mediaset.it;Playhouse cartoonnetwork.mediasetpremium.mediaset.it;Cartoon Network premiumcinema.mediasetpremium.mediaset.it;Premium Cinema studiouniversal.mediasetpremium.mediaset.it;Studio Universal premiumcinemaemotion.mediasetpremium.mediaset.it;Premium Cinema Emotion premiumcinemaenergy.mediasetpremium.mediaset.it;Premium Cinema Energy premiumextra.mediasetpremium.mediaset.it;Premium Extra premiumextra2.mediasetpremium.mediaset.it;Premium Extra 2 bbcknowledge.mediasetpremium.mediaset.it;BBC Knowledge discoveryworld.mediasetpremium.mediaset.it;Discovery World disneyjunior.mediasetpremium.mediaset.it;Disney Junior premiumcinemacomedy.mediasetpremium.mediaset.it;Premium Cinema Comedy premiumcrime.mediasetpremium.mediaset.it;Premium Crime [raiit] cinema.raisat.it;RaiSat Cinema extra.raisat.it;Extra gamberorosso.raisat.it;Gambero Rosso premium.raisat.it;Premium radio1.rai.it;Radio 1 radio2.rai.it;Radio 2 radio3.rai.it;Radio3 rai4.raisat.it;Rai 4 raisport1.rai.it;RaiSport 1 raisport2.rai.it;RaiSport 2 raiscuola.rai.it;Rai Scuola raimovie.rai.it;Rai Movie euronews.rai.it;EuroNews www.raidue.rai.it;Rai 2 raieducational.rai.it;Rai Educational raigulp.rai.it;Rai Gulp raisportpi.rai.it;Rai Sport pi raistoria.rai.it;Rai Storia www.raitre.rai.it;Rai 3 www.raiuno.rai.it;Rai 1 www.rainews24.rai.it;RaiNews24 smashgirls.skytv.it;Smash Girls yoyo.raisat.it;Yoyo rai5.rai.it;Rai5 [dahlia] sport.dahliatv.it;sport sport2.dahliatv.it;sport2 extra.dahliatv.it;extra xtreme.dahliatv.it;xtreme eros.dahliatv.it;eros explorer.dahliatv.it;explorer palermo.dahliatv.it;palermo calcio1.dahliatv.it;calcio1 calcio2.dahliatv.it;calcio2 calcio3.dahliatv.it;calcio3 calcio4.dahliatv.it;calcio4 calcio5.dahliatv.it;calcio5 adult1.dahliatv.it;adult1 adult2.dahliatv.it;adult2 adult3.dahliatv.it;adult3 adult4.dahliatv.it;adult4 [la7] www.la7.it;La7 www.la7d.it;La7D [mediaset] www.canale5.com;Canale 5 www.italia1.com;Italia 1 www.rete4.com;Rete 4 la5.mediaset.it;La5 mediasetextra.mediaset.it;Mediaset Extra iris.mediaset.it;Iris italia2.mediaset.it;Italia 2 www.boingtv.it;Boing cartoonito.mediaset.it;Cartoonito [mediaset_guidatv] www.italia1.com;Italia 1 www.canale5.com;Canale 5 www.rete4.com;Retequattro italia2.mediaset.it;Mediaset Italia 2 mediasetextra.mediaset.it;Mediaset Extra la5.mediaset.it;La5 discoveryworld.mediaset.it;Discovery World tgcom24.mediaset.it;TgCom24 topcrime.guidatv.sky.it;Top Crime premiumcrime.mediaset.it;Premium Crime premiumcrimehd.mediaset.it;Premium Crime HD steel.mediaset.it;Premium Action steelhd.mediaset.it;Premium Action HD direttacalcio.mediaset.it;Premium Calcio premiumcinema.mediaset.it;Premium Cinema premiumcinemahd.mediaset.it;Premium Cinema HD premiumcinemacomedy.mediaset.it;Premium Cinema Comedy premiumcinemaemotion.mediaset.it;Premium Cinema Emotion premiumcinemaenergy.mediaset.it;Premium Cinema Energy premiumcinemaenergyhd.mediaset.it;Premium Cinema Energy HD premiumstories.mediaset.it;Premium Stories premiumsport2.mediaset.it;Premium Sport 2 www.studiouniversal.it;Studio Universal bbcknowledge.mediaset.it;BBC Knowledge www.eurosport.com;Eurosport eurosport2.skytv.it;Eurosport 2 foxsports.guidatv.sky.it;Fox Sports foxsportsplus.guidatv.sky.it;Fox Sports Plus iris.mediaset.it;Iris www.boingtv.it;Boing joi.mediaset.it;Joi mya.mediaset.it;Mya disney.mediaset.it;Disney disneyjunior.mediaset.it;Disney Junior www.cartoonnetwork.it;Cartoon Network cartoonito.guidatv.sky.it;Cartoonito xmltv-1.4.0/grab/it/test.conf000066400000000000000000000001461500074233200160400ustar00rootroot00000000000000channel www.raiuno.rai.it # Rai 1 channel www.raidue.rai.it # Rai 2 channel www.raitre.rai.it # Rai 3 xmltv-1.4.0/grab/it/tv_grab_it.PL000066400000000000000000000250151500074233200165710ustar00rootroot00000000000000# Generate tv_grab_it from tv_grab_it.in. This is to set the path to # the files in /usr/local/share/xmltv or wherever. # # The second argument is the share directory for the final # installation. # 15/07/2005 # we switch the pod in the file according to the locale. # maybe we should ask the user if he wants to... # code below is based on lib/XMLTV.pm.pl use strict; use File::Copy qw/copy/; #here are the pods... my $ENG_POD=<t find the data with the first backend it will try the second one, and so on. You can specify your order of preference using the --backend option. Currently configured backends are (in default order): =over =item B - grabs data from www.mediaset.it =item B - grabs data from www.skylife.it =item B - grabs data from www.rai.it =item B - grabs data from www.mediaset.it/guidatv =item B - grabs data from www.mediasetpremium.it =item B - grabs data from www.iris.it =item B - grabs data from www.boingtv.it =item B - grabs data from www.la7.it =back First run B to choose which channels you want to download. Then running B with no arguments will output listings in XML format to standard output. B<--configure> Prompt for which channels, and writes the configuration file. B<--config-file FILE> Set the name of the configuration file, the default is B<~/.xmltv/tv_grab_it.conf>. This is the file written by B<--configure> and read when grabbing. B<--gui OPTION> Use this option to enable a graphical interface to be used. OPTION may be 'Tk', or left blank for the best available choice. Additional allowed values of OPTION are 'Term' for normal terminal output (default) and 'TermNoProgressBar' to disable the use of XMLTV::ProgressBar. B<--output FILE> write to FILE rather than standard output. B<--days N> Grab N days. The default is 7. B<--offset N> Start N days in the future. The default is to start from today. B<--quiet> Suppress the progress messages normally written to standard error. B<--slow> Downloads more details (descriptions, actors...). This means downloading a new file for each programme, so itE<39>s off by default to save time. B<--cache-slow> If you use the --cache option to speed up thing when you grab data several times a week, using this option you will cache only the --slow data, so you shouldnE<39>t miss changes in schedules. B<--verbose> Prints out verbose information useful for debugging. B<--errors-in-xml> Outputs warnings as programmes in the xml file, so that you can see errors in your favorite frontend in addition to the default STDERR. B<--backend> Set the backend (or backends) to use. See the examples. B<--version> Show the version of the grabber. B<--help> Print a help message and exit. =head1 CAVEATS If you use --quiet you should also use --errors-in-xml or you wonE<39>t be warned about errors. Note also that, as opposed to previous versions, this grabber doesnE<39>t die if it cannot find any data, but returns an empty (or optionally containing just warnings) xml file instead. The backendsE<39> data quality differs a lot. For example, mytv was very basic, yet complete and uses the least amount of bandwidth. Skytv has a lot of channels, but unless you use it with the --slow option the data is not very good (and in this case i would be VERY slow). wfactory is a good overall site if you donE<39>t need the whole sky package. =head1 EXAMPLES =over =item tv_grab_it --backend skylife --configure configures tv_grab_it using only the backend skylife =item tv_grab_it --backend skylife,wfactory --days 1 grabs one day of data overriding the default order (could also be --backend skylife --backend wfactory) =item tv_grab_it --cache --slow --days 3 grabs the full data for the next three days using the default backend order and using a disk cache. =back =head1 RECOMMENDED USAGE =over =item tv_grab_it --cache --slow --cache-slow --errors-in-xml =back =head1 SEE ALSO L. =head1 AUTHOR This grabber is currently unmantained. =cut END my $ITA_POD=< basato sullE<39>analisi del sorgente HTML dei siti, quindi potrebbe smettere di funzionare in qualsiasi momento. I dati vengono presi da piuE<39> fonti per poter ridurre i periodi di blackout dovuti a cambiamenti nei siti, ma anche per aumentare il numero di canali disponibili. Se il grabber non riesce ad ottenere dati dalla prima fonte passeraE<39> alla seconda, e cosE<39> via. Puoi specificare il tuo ordine usando lE<39>opzione --backend. Le fonti configurate al momento sono: (in ordine di utilizzo): =over =item B - prende i dati da www.skylife.it =item B - prende i dati da www.boingtv.it =item B - prende i dati da www.sitcom1.it =item B - prende i dati da www.risat.it =item B - prende i dati da www.rai.it =item B - prende i dati da www.iris.it =item B - prende i dati da www.mediasetpremium.it =back Per prima cosa esegui B per scegliere quali canali vuoi scaricare. In seguito lE<39>esecuzione di B senza opzioni manderE<39> sullo standard output i dati in formato XML. B<--configure> Chiede quali canali scaricare e scrive il file di configurazione. B<--config-file FILE> Imposta il nome del file di configurazione, di default eE<39> B<~/.xmltv/tv_grab_it.conf>. Questo file viene scritto usando B<--configure> e letto durante il grabbing. B<--gui OPTION> Usa questa opzione per abilitare una interfaccia grafica. OPTION puE<39> essere 'Tk', oppure lasciato in bianco per la scelta migliore. Altri valori possibili per OPTION sono: 'Term' per un terminale normale (default) e 'TermNoProgressBar' per disabilitare lE<39>uso di XMLTV::ProgressBar. B<--output FILE> scrive in questo file invece che sullo standard output. B<--days N> prende dati per N giorni. Di default eE<39> 7. B<--offset N> parte da N giorni in poi. Normalmente parte da oggi. B<--quiet> non usa i messaggi di avanzamento che normalmente vengono scritti su standard error. B<--slow> scarica piuE<39> dettagli (trame, attori...). Questo vuol dire scaricare un nuovo file per ogni programma, quindi di default eE<39> disabilitato per risparmiare tempo. B<--cache-slow> Se usi lE<39>opzione --cache per accelerare le cose se usi il programma piuE<39> volte alla settimana, con questa opzione verranno messe in cache solo le informazioni --slow, cosiE<39> non dovresti perdere cambiamenti nei palinsesti. B<--verbose> scrive piuE<39> informazioni su quello che sta facendo il programma, utile per il debugging. B<--errors-in-xml> scrive gli errori sotto forma di programmi nel file XML, cosiE<39> possono essere visti nel tuo frontend preferito oltre che in STDERR. B<--backend> imposta la sorgente da utilizzare. Vedi gli esempi. =head1 ATTENZIONE Se usi --quiet dovresti usare anche --errors-in-xml, o non avrai nessun avvertimento per eventuali errori. A differenza delle versioni precedenti, inoltre, se il grabber non riesce a scaricare nessun dato ritorna un file XML vuoto (o opzionalmente con solo i warning). La qualitaE<39> dei dati delle varie sorgenti cambia molto. Per esempio, mytv era molto semplice, ma completa e usava la minor banda possibile. Skytv ha molti canali, ma i dati non sono molto buoni a meno che non si usi lE<39>opzione --slow, (ed in quel caso serve MOLTO tempo). wfactory eE<39> tutto sommato un buon sito se non hai bisogno di tutto il pacchetto sky. =head1 ESEMPI =over =item tv_grab_it --backend skylife --configure configura tv_grab_it usando solo la sorgente skylife =item tv_grab_it --backend skylife,wfactory --days 1 prende solo un giorno di dati e utilizza un ordine diverso da quello di default (si sarebbe potuto scrivere anche cosiE<39>: --backend skylife --backend wfactory) =item tv_grab_it --cache --slow --days 3 prende tutti i dati per i prossimi tre giorni ed usa una cache su disco (sempre raccomandabile). =back =head1 UTILIZZO CONSIGLIATO =over =item tv_grab_it --cache --slow --cache-slow --errors-in-xml =back =head1 VEDERE ANCHE L. =head1 AUTORE Questo grabber non è più sviluppato. =cut END # and here it's the code itself # warn "Setting tv_grab_it pod..."; my $LANG = 'eng'; my $LANG_STR; if ($^O eq 'MSWin32') { eval { require Win32::Locale; }; if ($@) { warn "Win32::Locale not installed\n"; } else { $LANG_STR = Win32::Locale::get_language(); } } else { $LANG_STR = $ENV{LANG}; } #warn ' $ENV{LANG} is '.$LANG_STR."\n"; $LANG = 'ita' if (defined $LANG_STR and $LANG_STR=~/^it[-_]/i); #warn ' LANG is '.$LANG."\n"; my $out = 'grab/it/tv_grab_it.in2'; my $in = 'grab/it/tv_grab_it.in'; #warn "lang e' $LANG deflang e' $LANG_STR\n"; open(IN_FH, $in) or die "cannot read $in: $!"; open(OUT_FH, ">$out") or die "cannot write to $out: $!"; while () { if (/^my \$POD_GOES_HERE;$/) { if ($LANG eq 'ita') { print OUT_FH $ITA_POD; } else { print OUT_FH $ENG_POD; } } elsif (/^my \$DEF_LANG = 'eng';\E$/) { print OUT_FH 'my $DEF_LANG = \''.$LANG.'\';'; } else { print OUT_FH $_; } } close OUT_FH or die "cannot close $out: $!"; close IN_FH or die "cannot close $in: $!"; # stuff for setting share dir die "usage: $_ output_file share_dir" if @ARGV != 2; require './lib/set_share_dir.pl'; #warn "faccio $ARGV[0] $ARGV[1]\n"; #set_share_dir('grab/it/tv_grab_it.in2', $ARGV[0], $ARGV[1]); copy( 'grab/it/tv_grab_it.in2', $ARGV[0] ); xmltv-1.4.0/grab/it/tv_grab_it.in000077500000000000000000003534161500074233200167000ustar00rootroot00000000000000#!/usr/bin/perl ################################################# # Changelog: # 13/07/2005 # - removed backend lastampa, now using # wfactory.net instead (basically the same) # - removed backend mytv, site doesn't provide # data anymore. Didn't remove the code since # it might come in handy in the future. # 25/08/2005 # - updated after changes in skytv.it site # - first test with simple double language messages and docs # 14/06/2006 # - minor update for changes in skytv.it site (now skylife.it) # 16/08/2006 # - fixes to skytv # - skytv now handles categories when using --slow # 11/01/2007 # - added backend boingtv # - new option --cache-slow # 13/02/2007 # - added backend skylife (soon to replace skytv) # 27/05/2007 # - fixed boingtv.it after site change (thanks Paolo Asioli) # 02/07/2007 # - fixed skylife after site change (thanks Marco Coli) # 20/09/2007 # fixes for warnings when in quiet mode # 06/11/2007 # added backend mtv.it, as skylife strangely doesn't carry it, and wfactory is a stuttering site. # 08/12/2007 # skylife.it has moved to guidatv.sky.it # code cleanup # 15/01/2008 # major optimizations in skylife.it! (thanks Massimo Savazzi) # 30/06/2008 # better handling of season /episodes after site changes # now using also the dtd tags episode-num # 24/09/2008 # aggiunti mediasetpremium e raisat # 25/09/2008 # aggiunto iris # 04/02/2009 # update per sky per sito cambiato completamente # tolto wfactory, il sito non va piu' # sistemato rai4 # nuovi canali per mediasetpremium # 02/03/2009 # piccoli fix per skylife # aggiunto backend rai.it (che include rai4, rai gulp e altri canali 'inediti') # 10/11/2009 # sistemato boingtv (grazie r.ghetta) # 22/02/2010 # sistemato iris (grazie gpancot) # 23/02/2010 # sistemato rai.it (grazie gianni kurgo) # 14/09/2010 # aggiunto nuovo backend dahlia # aggiunto nuovo backend k2 (grazie a r.ghetta!) # fix per skylife di r.ghetta # 18/10/2010 # aggiunto backend la7 e riattivato mtv.it (grazie gpancot) # aggiunto backend mediaset # 25/10/2010 # patch da mennucc per possibili errori di parsing di data # patch da wyrdmeister per aggiungere la7d e un fix per raiit # 30/10/2010 # rimosso k2 (grazie rghetta) # fix per la7 (grazie rghetta) # rimosso searchch # 28/12/2010 # aggiunta opzione --mythweb-categories per utilizzare le categorie usate da mythweb invece di quelle del sito # 25/02/2011 # aggiunto patch per mediaset da charon66 # tolto backend dahlia # 24/07/2011 # nuovi canali # 23/09/2013 # nuovi canali e bugfixes # 23/08/2015 # disabled mtvit backend - site doesn't provide data anymore. ################################################# # TODO # - add more informative errors in xml ################################################# #pod below is handled at install time my $POD_GOES_HERE; #default language for warnings set at install time my $DEF_LANG = 'eng'; ###################################################################### # initializations use warnings; use strict; use XMLTV; use XMLTV::Version "$XMLTV::VERSION"; use XMLTV::Capabilities qw/baseline manualconfig cache/; use XMLTV::Description 'Italy'; use XMLTV::Supplement qw/GetSupplement/; use HTML::Entities; use HTML::Parser; use URI::Escape; use Getopt::Long; use Date::Manip; use Memoize; use XMLTV::Memoize; use XMLTV::Ask; use XMLTV::Config_file; use XMLTV::ProgressBar; use XMLTV::DST; use XMLTV::Get_nice; use XMLTV::Mode; #i hate to do this but it seems that skylife is blocking user agents not containing 'mozilla' #we still advertise ourselves as xmltv so they can block us if they really want to $XMLTV::Get_nice::ua->agent("Mozilla/5.0 xmltv/$XMLTV::VERSION"); use XMLTV::Usage < { domain => 'guidatv.sky.it', base_chan => 'http://guidatv.sky.it/app/guidatv/contenuti/data/grid/', base_icon => 'http://guidatv.sky.it/app/guidatv/images/epgimages/channels/grid/', base_data => 'http://guidatv.sky.it/app/guidatv/contenuti/data/grid/', base_slow => 'http://guidatv.sky.it/guidatv/programma/', rturl => "http://guidatv.sky.it/", needs_login => 0, needs_cookies => 0, fetch_data_sub => \&skylife_fetch_data, channel_list_sub => \&skylife_get_channels_list, }, 'raisat' => { domain => 'raisat.it', base_chan => 'http://www.raisat.it/canaliListForXML.jsp', #base_data => 'http://www.raisat.it/generaxmlpalinsesto.jsp', base_data => 'http://212.162.68.116/generaxmlpalinsesto.jsp', rturl => "http://www.raisat.it/", needs_login => 0, needs_cookies => 0, fetch_data_sub => \&raisat_fetch_data, channel_list_sub => \&raisat_get_channels_list, }, 'boingtv' => { domain => 'boingtv.it', base_chan => 'http://www.boingtv.it/xml/palinsesto.xml', base_data => 'http://www.boingtv.it/xml/palinsesto.xml', rturl => "http://www.boingtv.it/xml/palinsesto.xml", needs_login => 0, needs_cookies => 0, fetch_data_sub => \&boingtv_fetch_data, channel_list_sub => \&boingtv_get_channels_list, }, 'sitcom1' => { domain => 'sitcom1.it', base_chan => 'http://www.sitcom1.it/guidatv.asp', base_data => 'http://www.sitcom1.it/guidatv.asp', rturl => "http://www.sitcom1.it/guidatv.asp", needs_login => 0, needs_cookies => 0, fetch_data_sub => \&sitcom1_fetch_data, channel_list_sub => \&sitcom1_get_channels_list, }, 'iris' => { domain => 'iris.mediaset.it', base_chan => 'http://iris.mediaset.it/palinsesto/palinsesto1.shtml', base_data => 'http://iris.mediaset.it/palinsesto/', rturl => "http://iris.mediaset.it/palinsesto/palinsesto1.shtml", needs_login => 0, needs_cookies => 0, fetch_data_sub => \&iris_fetch_data, channel_list_sub => \&iris_get_channels_list, }, # Disabled 2015-08-23 by knowledgejunkie: site no longer provides listings # 'mtvit' => # { domain => 'www.mtv.it', # base_chan => 'http://www.mtv.it/', # base_data => 'http://www.mtv.it/', # rturl => "http://www.mtv.it/tv/guida-tv/", # needs_login => 0, # needs_cookies => 0, # fetch_data_sub => \&mtvit_fetch_data, # channel_list_sub => \&mtvit_get_channels_list, # }, 'mediasetpremium' => { domain => 'mediasetpremium.mediaset.it', base_chan => 'http://www.mediasetpremium.mediaset.it/export/palinsesto.xml', base_data => 'http://www.mediasetpremium.mediaset.it/export/palinsesto', rturl => "http://www.mediasetpremium.mediaset.it/", needs_login => 0, needs_cookies => 0, fetch_data_sub => \&mediasetpremium_fetch_data, channel_list_sub => \&mediasetpremium_get_channels_list, }, 'raiit' => { domain => 'rai.it', base_chan => 'http://www.rai.it/dl/portale/GuidaProgrammi.html', base_data => 'http://www.rai.it/dl/portale/html/palinsesti/guidatv/static/', rturl => "http://www.rai.it/", needs_login => 0, needs_cookies => 0, fetch_data_sub => \&raiit_fetch_data, channel_list_sub => \&raiit_get_channels_list, }, 'dahlia' => { domain => 'dahliatv.it', base_chan => 'http://www.dahliatv.it/guidatv', base_data => 'http://www.dahliatv.it/html/portlet/ext/epg/epg.jsp', rturl => "http://www.dahliatv.it/", needs_login => 0, needs_cookies => 0, fetch_data_sub => \&dahlia_fetch_data, channel_list_sub => \&dahlia_get_channels_list, }, 'la7' => { domain => 'la7.it', base_chan => 'http://www.la7.it/guidatv/index.html', base_data => 'http://www.la7.it/guidatv/index_', rturl => "http://www.la7.it/guidatv/index", needs_login => 0, needs_cookies => 0, fetch_data_sub => \&la7_fetch_data, channel_list_sub => \&la7_get_channels_list, }, 'mediaset' => { domain => 'mediaset.it', base_chan => 'http://www.tv.mediaset.it/dati/palinsesto/palinsesto-mondotv.xml', base_data => 'http://www.tv.mediaset.it/dati/palinsesto/palinsesto-mondotv.xml', rturl => "http://www.tv.mediaset.it/dati/palinsesto/palinsesto-mondotv.xml", needs_login => 0, needs_cookies => 0, fetch_data_sub => \&mediaset_fetch_data, channel_list_sub => \&mediaset_get_channels_list, }, 'mediaset_guidatv' => { domain => 'mediaset_guidatv.it', base_chan => 'http://www.mediaset.it/guidatv/palinsesto.xml', base_data => 'http://www.mediaset.it/guidatv/palinsesto.xml', rturl => "http://www.mediaset.it/guidatv/palinsesto.xml", needs_login => 0, needs_cookies => 0, fetch_data_sub => \&mediaset_guidatv_fetch_data, channel_list_sub => \&mediaset_guidatv_get_channels_list, }, ); ###################################################################### # Get options, including undocumented --cache option. XMLTV::Memoize::check_argv('XMLTV::Get_nice::get_nice_aux') # cache on disk or memoize('XMLTV::Get_nice::get_nice_aux') # cache in memory or die "cannot memoize 'XMLTV::Get_nice::get_nice_aux': $!"; my ($opt_days, $opt_offset, $opt_help, $opt_output, $opt_slow, $opt_verbose, $opt_configure, $opt_config_file, $opt_gui, $opt_quiet, $opt_errors_in_xml, @opt_backends, $opt_list_channels, $opt_cache_slow, $opt_mythweb_categories, ); # server only holds 7 days, so if there is an offset days must be # opt_days-offset or less. $opt_offset = 0; # default $opt_quiet = 0; # default $opt_slow = 0; # default $opt_verbose = 0; # default GetOptions('days=i' => \$opt_days, 'offset=i' => \$opt_offset, 'help' => \$opt_help, 'configure' => \$opt_configure, 'config-file=s' => \$opt_config_file, 'gui:s' => \$opt_gui, 'output=s' => \$opt_output, 'quiet' => \$opt_quiet, 'slow' => \$opt_slow, 'verbose' => \$opt_verbose, 'errors-in-xml' => \$opt_errors_in_xml, 'backend=s' => \@opt_backends, 'list-channels' => \$opt_list_channels, 'cache-slow' => \$opt_cache_slow, 'mythweb-categories' => \$opt_mythweb_categories, ) or usage(0); die ($DEF_LANG eq 'eng' ? "number of days (--days) must not be negative. You gave: $opt_days\n" : "il numero di giorni (--days) non puo' essere negativo. Hai specificato: $opt_days\n") if (defined $opt_days && $opt_days < 0); die ($DEF_LANG eq 'eng' ? "offset days (--offset) must not be negative. You gave: $opt_offset\n" : "l'intervallo di partenza (--offset) non puo' essere negativo. Hai specificato: $opt_offset\n") if ($opt_offset < 0); usage(1) if $opt_help; if ($opt_quiet) { $opt_verbose = 0; } $opt_days = $opt_days || $MAX_DAYS; $opt_slow = 1 if ($opt_cache_slow); my $mode = XMLTV::Mode::mode('grab', $opt_list_channels => 'list-channels', $opt_configure => 'configure'); # parse the --backend option @opt_backends = split(/,/,join(',',@opt_backends)); #we allow both multiple --backend and --backend=name1,name2 my @backends = (); foreach (@opt_backends) { if (defined $backend_info{$_}) { push @backends, $_; } else { warn ($DEF_LANG eq 'eng' ? "Unknown backend $_!\nProbably you need to update! go to xmltv.org for latest release.\nFor latest version get http://snapshot.xmltv.org or http://alpha-exe.xmltv.org if you are on windows." : "Fonte sconosciuta $_!Probabilmente devi aggiornare il programma! Vai su xmltv.org per l'ultima release.\nPer la versione piu' aggiornata vai su http://snapshot.xmltv.org o http://alpha-exe.xmltv.org per windows.\n" ); } } unless (@backends) { @backends = @default_backends; if (@opt_backends) { #we specified backends but we didn't like them, warn the user warn ($DEF_LANG eq 'eng' ? "No good backend specified, falling back on defaults\n" : "Nessuna fonte corretta specificata, uso i default\n" ); } } XMLTV::Ask::init($opt_gui); # reads the file channel_ids, which contains the tables to convert # between backends' ids and XMLTV ids of channels. # to support multiple backends i add a ini-style [section] header # there are two fields: xmltv_id and site_id. my $str = GetSupplement( "tv_grab_it", "channel_ids" ); my $CHANNEL_NAMES_FILE = "channel_ids"; my (%xmltv_chanid, %seen); my $line_num = 0; my $backend; foreach (split( /\n/, $str )) { ++ $line_num; tr/\r//d; s/#.*//; next if m/^\s*$/; my $where = "$CHANNEL_NAMES_FILE:$line_num"; if (/^\[(.*)\]$/) { if (defined $backend_info{$1}) { #esiste la configurazione $backend = $1; } else { warn ($DEF_LANG eq 'eng' ? "Unknown backend $1 in $where\n" : "Fonte sconosciuta $1 in $where\n"); $backend = undef; } } elsif ($backend) { my @fields = split /;/; die ($DEF_LANG eq 'eng' ? "$where: wrong number of fields" : "$where: numero di campi errato") if @fields != 2;#3; my ($xmltv_id, $site_id) = @fields; warn ($DEF_LANG eq 'eng' ? "$where: backend id $site_id for site '$backend' seen already\n" : "$where: fonte con id $site_id per il sito '$backend' gia' visto!\n" ) if defined $backend_info{$backend}{site_ids}{$xmltv_id}; $backend_info{$backend}{site_ids}{$xmltv_id}{site_id} = $site_id; #$backend_info{$backend}{site_ids}{$xmltv_id}{satellite} = $sat; warn ($DEF_LANG eq 'eng' ? "$where: XMLTV_id $xmltv_id for site '$backend' seen already\n" : "$where: XMLTV_id $xmltv_id per il sito '$backend' gia' visto!\n" ) if $seen{$backend.$xmltv_id}++; } } # File that stores which channels to download. Not needed for # list-channels mode. # my $config_file; unless ($mode eq 'list-channels') { $config_file = XMLTV::Config_file::filename($opt_config_file, 'tv_grab_it', $opt_quiet); } XMLTV::Config_file::check_no_overwrite($config_file) if $mode eq 'configure'; # Arguments for XMLTV::Writer. my %w_args; if (defined $opt_output) { die($DEF_LANG eq 'eng' ? "cannot give --output with --configure" : "non e' possibile specificare --output con --configure") if $mode eq 'configure'; my $fh = new IO::File(">$opt_output"); die ($DEF_LANG eq 'eng' ? "cannot write to $opt_output: $!" : "impossibile scrivere su $opt_output") if not defined $fh; $w_args{OUTPUT} = $fh; } $w_args{encoding} = 'ISO-8859-1'; $line_num = 0; my $foundchannels; my $bar = new XMLTV::ProgressBar(($DEF_LANG eq 'eng' ? 'getting list of channels' : 'prendo la lista dei canali'), scalar @backends) if not $opt_quiet; # find list of available channels foreach $backend (@backends) { %{$backend_info{$backend}{channels}} = &{$backend_info{$backend}{channel_list_sub}}($backend_info{$backend}{base_chan}); $foundchannels+=scalar(keys(%{$backend_info{$backend}{channels}})); if (not $opt_quiet) { update $bar; } } $bar->finish() if (not $opt_quiet); die ($DEF_LANG eq 'eng' ? "no channels could be found" : "nessun canale trovato") unless ($foundchannels); warn ($DEF_LANG eq 'eng' ? "VERBOSE: $foundchannels channels found.\n" : "VERBOSE: $foundchannels canali trovati.\n") if ($opt_verbose); ###################################################################### # write configuration if ($mode eq 'configure') { open(CONF, ">$config_file") or die ($DEF_LANG eq 'eng' ? "cannot write to $config_file: $!" : "impossibile scrivere su $config_file: $!"); my %channels; foreach $backend (@backends) { #faccio un hash con tutti gli id foreach (keys %{$backend_info{$backend}{channels}}) { $channels{$_} = xmltv_chanid($backend, $_); } #not used yet if ($backend_info{$backend}{needs_login}) { say "To get listings on '$backend' you will need a login on the site.\n"; my $username_wanted = ask_boolean('Do you have a login?', 0); if ($username_wanted) { $backend_info{$backend}{username} = ask("Username:"); print CONF "username: $backend:$backend_info{$backend}{username}\n"; } } } #double reverse to get rid of duplicates %channels = reverse %channels; %channels = reverse %channels; # Ask about each channel. my @names = sort keys %channels; my @qs = map { ($DEF_LANG eq 'eng' ? "add channel $_?" : "aggiungo il canale $_?") } @names; my @want = ask_many_boolean(1, @qs); foreach (@names) { die if $_ =~ tr/\r\n//; my $w = shift @want; warn("cannot read input, stopping channel questions"), last if not defined $w; # No need to print to user - XMLTV::Ask is verbose enough. # Print a config line, but comment it out if channel not wanted. print CONF '#' if not $w; print CONF "channel ".$channels{$_}." # $_\n"; } close CONF or warn ($DEF_LANG eq 'eng' ? "cannot close $config_file: $!" : "impossibile chiudere $config_file: $!"); say(($DEF_LANG eq 'eng' ? "Finished configuration." : "Configurazione terminata.")); exit(); } # Not configuring, must be writing some XML. my $w = new XMLTV::Writer(%w_args); my $source_info_str = join ",", map {'http://'.$backend_info{$_}{domain}} @backends; my $source_data_str = join ",", map {$backend_info{$_}{rturl}} @backends; $w->start({ 'source-info-url' => $source_info_str , 'source-data-url' => $source_data_str, 'generator-info-name' => 'XMLTV', 'generator-info-url' => 'http://xmltv.org/', }); my %display_names; my %list_display_names; foreach my $back (@backends) { foreach (keys %{$backend_info{$back}{site_ids}}) { $display_names{$_} = [$backend_info{$back}{site_ids}{$_}{site_id}, $back]; #per controllare altri attributi tipo l'icona devo sapere da che backend viene il canale $list_display_names{$_} = $backend_info{$back}{site_ids}{$_}{site_id}; } } if ($mode eq 'list-channels') { # Write all known channels then finish. foreach my $xmltv_id (sort keys %list_display_names) { next if not defined $display_names{$xmltv_id}; my @display_name= [ [ $display_names{$xmltv_id}->[0] ] ]; my $backend = $display_names{$xmltv_id}->[1]; my @chaninfo; if (defined $backend_info{$backend}{site_ids}{$xmltv_id}{channum}) { #abbiamo il numero di canale, lo mettiamo come display name secondario @chaninfo = ('display-name' => [ [ $display_names{$xmltv_id}->[0] ], [ $backend_info{$backend}{site_ids}{$xmltv_id}{channum}]]); } else { @chaninfo = ('display-name' => [ [ $display_names{$xmltv_id}->[0] ] ]); } #aggiungo l'icona se ce l'ho if (defined $backend_info{$backend}{site_ids}{$xmltv_id}{icon}) { push @chaninfo , (icon => [{src => $backend_info{$backend}{site_ids}{$xmltv_id}{icon}}]); } $w->write_channel({ id => $xmltv_id, @chaninfo }); } $w->end; exit; } ###################################################################### # read configuration my @channels; $line_num = 0; foreach (XMLTV::Config_file::read_lines($config_file)) { ++ $line_num; next if not defined; if (/^channel:?\s*(.*\S+)\s*$/) { push @channels, $1; } elsif (/^username:?\s+(\S+):(\S+)/){ if (defined $backend_info{$1}) { #esiste la configurazione $backend_info{$1}{username} = $2; } else { warn ($DEF_LANG eq 'eng' ? "Found username for unknown backend $1 in $config_file\n" : "Trovato un nome utente per una fonte sconosciuta $1 in $config_file\n"); } } else { warn ($DEF_LANG eq 'eng' ? "$config_file:$line_num: bad line\n" : "$config_file:$line_num: linea errata\n"); } } ###################################################################### # sort out problem in offset options if ($opt_offset >= $MAX_DAYS) { warn ($DEF_LANG eq 'eng' ? "Day offset too big. No program information will be fetched.\n" : "Intervallo specificato troppo grande. Nessun dato verra' scaricato.\n"); $opt_offset = 0; $opt_days = 0; } my $days2get; if (($opt_days+$opt_offset) > $MAX_DAYS) { $days2get=$MAX_DAYS-$opt_offset; warn ($DEF_LANG eq 'eng' ? "The server only has info for ".($MAX_DAYS-1)." days from today.\n" : "Il server ha informazioni solo per ".($MAX_DAYS-1)." giorni da oggi.\n"); if ($days2get > 1) { warn ($DEF_LANG eq 'eng' ? "You'll get listings for only $days2get days.\n" : "Scarico programmi solo per $days2get giorni.\n"); } else { warn ($DEF_LANG eq 'eng' ? "You'll get listings for only 1 day.\n" : "Scarico programmi solo per un giorno.\n"); } } else { $days2get=$opt_days; } t "will get $days2get days from $opt_offset onwards"; ###################################################################### # grabbing listings foreach my $xmltv_id (@channels) { next if not defined $display_names{$xmltv_id}; my @display_name= [ [ $display_names{$xmltv_id}->[0] ] ]; my $backend = $display_names{$xmltv_id}->[1]; my @chaninfo; if (defined $backend_info{$backend}{site_ids}{$xmltv_id}{channum}) { #abbiamo il numero di canale, lo mettiamo come display name secondario @chaninfo = ('display-name' => [ [ $display_names{$xmltv_id}->[0] ], [ $backend_info{$backend}{site_ids}{$xmltv_id}{channum}]]); } else { @chaninfo = ('display-name' => [ [ $display_names{$xmltv_id}->[0] ] ]); } #aggiungo l'icona se ce l'ho if (defined $backend_info{$backend}{site_ids}{$xmltv_id}{icon}) { push @chaninfo , (icon => [{src => $backend_info{$backend}{site_ids}{$xmltv_id}{icon}}]); } $w->write_channel({ id => $xmltv_id, @chaninfo }); } #make a list of channels and days to grab my @to_get; foreach my $day ($opt_offset .. ($days2get + $opt_offset - 1)) { foreach my $channel (@channels) { push @to_get, [$channel, $day]; } } $bar = new XMLTV::ProgressBar(($DEF_LANG eq 'eng' ? 'getting listings' : 'scarico programmi'), scalar @to_get) if not $opt_quiet; ## If we aren't getting any days of program data then clear out the list ## that was created to fetch \. #if ($days2get == 0) {@to_get = ();} foreach (@to_get) { my $day = $_->[1]; my $channel = $_->[0]; #this is where i would handle cookies and logins if needed warn ($DEF_LANG eq 'eng' ? "VERBOSE: Grabbing channel $channel, day $day\n" : "VERBOSE: Prendo dati per il canale $channel, giorno $day\n") if ($opt_verbose); my $error; foreach $backend (@backends) { warn ($DEF_LANG eq 'eng' ? "VERBOSE: Trying with $backend\n" : "VERBOSE: Provo con $backend\n") if ($opt_verbose); my @dati; $error = 0; ($error, @dati) = &{$backend_info{$backend}{fetch_data_sub}}($channel, $day); #TODO different kinds of errors? if ($error) { warn ($DEF_LANG eq 'eng' ? "VERBOSE: Error fetching channel $channel day $day with backend $backend\n" : "VERBOSE: Errore nello scaricare i dati per $channel, giorno $day con $backend\n") if ($opt_verbose); } else { $w->write_programme($_) foreach @dati; last; } } #nessuno ci e' riuscito if ($error) { #this is an easier way to know about errors if all of our scripts are automated if ($opt_errors_in_xml) { $w->write_programme( { title => [[($DEF_LANG eq 'eng' ? 'ERROR FETCHING DATA' : 'ERRORE DI SCARICAMENTO DATI'), $LANG]], start => xmltv_date('00:01', $day), stop => xmltv_date('23:59', $day), channel => $channel, desc => [[($DEF_LANG eq 'eng' ? "XMLTV couldn't grab data for $channel, day $day. Sorry about that." : "XMLTV non e' riuscito a scaricare i dati per $channel, giorno $day. Spiacente."), $LANG]], } ); } else { warn ($DEF_LANG eq 'eng' ? "I couldn't fetch data for channel $channel, day $day from any backend!!\n" : "Non sono riuscito a scaricare i dati per $channel, giorno $day da nessuna fonte!!\n") if (not $opt_quiet); } } update $bar if not $opt_quiet; } $w->end; $bar->finish() if not $opt_quiet; ##################### # general functions # ##################### #################################################### # xmltv_chanid # to handle channels that are not yet in the channel_ids file sub xmltv_chanid { my ($backend, $channel_id) = @_; my %chan_ids; #reverse id hash foreach my $xmltv_id (keys %{$backend_info{$backend}{site_ids}}) { my $site_id = $backend_info{$backend}{site_ids}{$xmltv_id}{site_id}; $chan_ids{$site_id} = $xmltv_id; next if (not defined $site_id); } if (defined $chan_ids{$channel_id}) { return $chan_ids{$channel_id}; } else { warn ($DEF_LANG eq 'eng' ? "***Channel |$channel_id| for '$backend' is not in channel_ids, should be updated.\n" : "***Il canale |$channel_id| su '$backend' non e' in channel_ids, andrebbe aggiornato.\n" ) unless $opt_quiet; my $oldid=$channel_id; $channel_id=~ s/\W//gs; #make up an id my $id = lc($channel_id).".".$backend_info{$backend}{domain}; #warn ("-->update: $id;$oldid\n"); ##update backend info #$backend_info{$backend}{site_ids}{$id}{site_id} = $channel_id; return $id; } } ########################################################## # tidy # decodes entities and removes some illegal chars sub tidy($) { for (my $tmp=shift) { s/[\000-\037]//gm; # remove control characters s/[\222]/\'/gm; # messed up char s/[\224]/\"/gm; # end quote s/[\205]/\.\.\./gm; # ... must be something messed up in my regexps? s/[\223]/\"/gm; #start quote s/[\221]/\'/gm; s/\\\'/\'/gm; #s///gm;# s/è//g;# s//\'/g;# s/è//g;# s/à//g;# s/ì//g;# s//\.\.\./g; #mah... if (s/[\200-\237]//g) { if ($opt_verbose){ warn ($DEF_LANG eq 'eng' ? "VERBOSE: removing illegal char: |\\".ord($&)."|\n" : "VERBOSE: tolgo carattere illegale: |\\".ord($&)."|\n"); } } # Remove leading white space s/^\s*//; # Remove trailing white space s/\s*$//; # FIXME handle a with a grave accent encoded as utf-8 (fallout from LWP::Simple?) s/\xc3\xa0/\xe0/g; return decode_entities($_); } } #################################################### # xmltv_date # this returns a date formatted like 20021229121300 CET # first argument is time (like '14:20') # second is date offset from today sub xmltv_date { my ($time, $offset) = @_; $time =~/([0-9]+?):([0-9]+).*/ or die ($DEF_LANG eq 'eng' ? "bad time $time" : "strano orario $time"); my $hour=$1; my $min=$2; my $data = &DateCalc("today","+ ".$offset." days"); die ($DEF_LANG eq 'eng' ? 'date calculation failed' : 'errore di calcolo data') if not defined $data; return utc_offset(UnixDate($data, '%Y%m%d').$hour.$min.'00', '+0100'); } ######################## # boingtv.it functions # ######################## ######################################################### # boingtv_get_channels_list # since this site only has one channel this is a fake sub sub boingtv_get_channels_list { my %chan_hash = ( 'boingtv' ,'www.boingtv.it'); return %chan_hash; } #################################################### # boingtv_fetch_data # 2 parameters: xmltv_id of channel # day offset # returns an error or an array of data sub boingtv_fetch_data { my ($xmltv_id, $offset) = @_; my $content; my $site_id = $backend_info{boingtv}{site_ids}{$xmltv_id}{site_id}; if (not defined $site_id) { warn ($DEF_LANG eq 'eng' ? "VERBOSE: \tThis site doesn't know about $xmltv_id!\n" : "VERBOSE: \tQuesto sito non sa niente di $xmltv_id!\n" ) if ($opt_verbose); return (1, ()); } # build url to grab # very strange site: only has data till next sunday. if the offset it's too big we return an empty array # but we don't return an error my $day_of_week = UnixDate("today", '%w'); #1 (Monday) to 7 (Sunday) if ($day_of_week + $offset > 7) { return (0, ()); } my $date_grab = &DateCalc("today","+ ".$offset." days"); die ($DEF_LANG eq 'eng' ? 'date calculation failed' : 'errore di calcolo di data') if not defined $date_grab; $date_grab = UnixDate($date_grab, '%Y%m%d'); my $cachestring = "?pippo=".UnixDate("today","%Y%m%d%H") if ($offset == 0); my $url = $backend_info{boingtv}{base_data}.$cachestring; warn ($DEF_LANG eq 'eng' ? "VERBOSE: fetching $url\n" : "VERBOSE: scarico $url\n") if ($opt_verbose); eval { $content=get_nice($url) }; if ($@) { #get_nice has died warn ($DEF_LANG eq 'eng' ? "VERBOSE: Error fetching $url channel $xmltv_id day $offset backend boingtv\n" : "VERBOSE: Errore nello scaricare $url, canale $xmltv_id, giorno $offset, fonte boingtv\n") if ($opt_verbose); # Indicate to the caller that we had problems return (1, ()); } my @programmes = (); warn "VERBOSE: parsing...\n" if ($opt_verbose); my @lines = split /\n/, $content; my $title = ''; my $time_start = ''; my $description = ''; #split the lines foreach my $line (@lines) { next unless $line=~/EVENT/; $line=~/timestamp="(.*?)".*name="(.*?)".*description="(.*?)"/; my %programme = (); eval { ($title, $time_start, $description) = ($2, utc_offset($1.'00', '+0100'), $3) ; } or do { warn 'skipping programme, error: ' . $@ ; next ; }; # Three mandatory fields: title, start, channel. if (not defined $title) { warn 'no title found, skipping programme'; next; } $programme{title}=[[tidy($title), $LANG] ]; if (not defined $time_start) { warn "no start time for title $title, skipping programme"; next; } #dobbiamo buttare via quello che non ci interessa next unless ($time_start=~/^$date_grab/); $programme{desc}=[[tidy($description), $LANG] ] if ($description ne ''); $programme{start}=$time_start;#xmltv_date($time_start, $offset + $past_midnight); $programme{channel}=$xmltv_id; #put info in array push @programmes, {%programme}; } if (scalar @programmes) { return (0, @programmes); } else { # there is a number of reasons why we could get an empty array. # so we return an error return (1, @programmes); } } ######################## # mtv.it functions # ######################## ######################################################### # mtvit_get_channels_list # since this site only has one channel this is a fake sub sub mtvit_get_channels_list { my %chan_hash = ( 'MTV' ,'www.mtv.it'); return %chan_hash; } #################################################### # mtvit_fetch_data # 2 parameters: xmltv_id of channel # day offset # returns an error or an array of data sub mtvit_fetch_data { my ($xmltv_id, $offset) = @_; my $content; my $site_id = $backend_info{mtvit}{site_ids}{$xmltv_id}{site_id}; if (not defined $site_id) { warn ($DEF_LANG eq 'eng' ? "VERBOSE: \tThis site doesn't know about $xmltv_id!\n" : "VERBOSE: \tQuesto sito non sa niente di $xmltv_id!\n" ) if ($opt_verbose); return (1, ()); } # build url to grab # http://tv.mtv.it/guidatv.php?tvguidedate=2014-11-05 my $grabdate = UnixDate(&DateCalc("today","+ ".$offset." days"), '%Y:%m:%d'); my ($anno, $mese, $giorno) = split /:/, $grabdate; #my $url = $backend_info{mtvit}{base_data}.'?canaleSel=MTV&giorno_guid='.$giorno.'%2F'.$mese.'%2F'.$anno; #my $url = $backend_info{mtvit}{base_data}.'guidatv.php?tvguidedate='.$anno.'-'.$mese.'-'.$giorno; # http://tv.mtv.it/guidatv.php?tvguidedate=2015-06-19 my $url = $backend_info{mtvit}{base_data}.'tv/guida-tv/'.$anno.'-'.$mese.'-'.$giorno; # http://www.mtv.it/tv/guida-tv/2015-06-21 warn ($DEF_LANG eq 'eng' ? "VERBOSE: fetching $url\n" : "VERBOSE: scarico $url\n") if ($opt_verbose); eval { $content=get_nice($url) }; if ($@) { #get_nice has died warn ($DEF_LANG eq 'eng' ? "VERBOSE: Error fetching $url channel $xmltv_id day $offset backend mtvit\n" : "VERBOSE: Errore nello scaricare $url, canale $xmltv_id, giorno $offset, fonte mtvit\n") if ($opt_verbose); # Indicate to the caller that we had problems return (1, ()); } #$content=~/colonna centrale(.*)colonna destra/s; $content=$1; #$content=~/
    (.*)?
    /s; $content=$1; #$content=~/
# my @shows = $tree->look_down('_tag' => 'table', 'border' => '0', 'cellpadding' => '0', 'style' => qr/background:\s*black;border-collapse:\s*collapse;/); if (@shows) { my $count = 0; foreach my $show (@shows) { # $show->dump; $count++; # are we processing yesterday's schedule? (see above) if ($i == ($opt->{offset} -1)) { my $showstart = $show->look_down('_tag' => 'span', 'class' => 'tvchannel'); my ($h, $i, $a) = $showstart->as_text =~ /(\d*):(\d*)\s*(am|pm)/; # 2014-04-02 see note below if (!defined $a) { $showstart = $show->look_down('_tag' => 'span', 'class' => 'season'); ($h, $i, $a) = $showstart->as_text =~ /(\d*):(\d*)\s*(am|pm)/; } if ($a eq 'am' && ($h < 6 || $h == 12)) { next if $count == 1; # we don't want first programme in file if it overlaps 6am boundary # continue processing of pre-6am programme } else { next; } $showstart = $h = $i = $a = undef; } my %prog = (); my $showtime; # see if we have a details page # look_down('_tag' => 'a', 'href' => qr/javascript:popup/); # 2014-12-03 The new website seems a bit flakey with these details pages, often returning a 500 Server Error # Here's an option to disable the details pages ( --nodetailspage ) # 2014-12-24 site changed # my $webdetails = $show->look_down('_tag' => 'a', 'href' => qr/\/engage\//); # if (!$opt->{nodetailspage}) { my $webdetails = $show->look_down('_tag' => 'a', 'href' => qr/\/detail\//); my $href = $webdetails->attr('href'); # my ($id) = $href =~ /javascript:popup\((\d*)\);/; # $url = $ROOT_URL . 'detail.asp?id=' . $id; # my ($url) = $href =~ /javascript:popupshow\('(.*)'\);/; my ($url) = $href; #debug "Fetching: $url"; # Fetch the page # my $showdetail = XMLTV::Get_nice::get_nice_tree($url); my $showdetail = fetch_url($url); # $showdetail->dump; if ($showdetail) { # Details page contains Director names and a better list of Actors # Get the cast and extract them into a new tree my @lis = $showdetail->look_down('_tag' => 'div', 'class' => 'cast-entry'); LOOP: foreach my $person (@lis) { # # 30/6/16 #
# Margaret Sellinger # # # Lesley-Anne Down # # # (IMDB) # #
my ($name, $role); if ( my ($_name) = $person->look_down('_tag' => 'span', 'itemprop' => 'name') ) { $name = $_name->as_text; } if ( my ($_role) = $person->look_down('_tag' => 'span', 'class' => 'role') ) { $role = $_role->as_text; } # drop the "Executive Director" & "Executive Producer" - any others we should drop? next LOOP if ( $role =~ /^(Executive Director|Executive Producer)/ ); # map the website role to an xmltv role my %xmltvroles = ( 'Director'=>'director', 'Producer'=>'producer', 'Series Producer'=>'producer', 'Writer'=>'writer', 'Co-Director'=>'director', 'Presenter'=>'presenter', 'Commentator'=>'commentator', 'Guest'=>'guest' ); my $credit; if (exists $xmltvroles{$role}) { $credit = $xmltvroles{$role}; } else { $credit = 'actor'; } if ($credit eq 'actor' && defined $role) { push @{$prog{'credits'}{$credit}}, [ encode('utf-8', $name), encode('utf-8', $role) ]; } else { push @{$prog{'credits'}{$credit}}, encode('utf-8', $name); } } undef @lis; # Get the "Left Panel" which contains the programme times and attributes my $lhs = $showdetail->look_down('_tag' => 'div', 'class' => qr/divLHS-section-2/); # Get the programme's "attributes" e.g. "Certificate" if ($lhs) { my @attrs = $lhs->look_down('_tag' => 'span', 'class' => 'LHS-attribute'); if (@attrs) { foreach my $attr (@attrs) { # $attr->dump; if ( my $showattr = $attr->as_text() ) { if ( $showattr =~ /^Certificate\s:\s(.*)\s*$/ ) { $prog{'rating'} = [[ $1, 'BBFC' ]] if $1; } } } } } # start time, and stop time (actually an optional DTD element) # 10:00am-11:50am (1 hour 50 minutes) Wed 20 Mar # (use the Date provided to avoid issues with the site running from 06:00-06:00) # # Note site displays stop time wrong on GMT/BST changeover, e.g.: # 12:45am-1:10am (25 minutes) Sun 31 Mar # this should be 12:45am-2:10am (BST) # this makes $showtime->set barf on "invalid local time for date in timezone" # # 1/Jan/17 times are now in the left panel. 'datetime' is used for user comments! # my $showtimes = $showdetail->look_down('_tag' => 'span', 'class' => 'datetime'); # # 25/Mar/2023 now displays the correct time on GMT/BST changeover...but the duration is wrong! # Sun 26 Mar 12:40am-2:40am (2 hours) # this should be '1 hour' # if ($lhs) { # Unfortunately the div with the date doesn't have any safe identifier. There are several ways we could remove the # cruft from the container but the following, although clunky, is probably the safest my ($dt, $h, $i, $a, $h2, $i2, $a2) = $lhs->as_text =~ /((?:Mon|Tue|Wed|Thu|Fri|Sat|Sun|Christmas\s(?:Eve|Day)|Boxing\sDay|New\sYears\s(?:Eve|Day))[\s<].*?)(\d*):(\d*)(am|pm)(?:-(\d*):(\d*)(am|pm))?/; # print STDERR $dt."\n"; if ($dt && $dt !~ /\D\D\D\s\d\d?\s\D\D\D/) { my @thedt = localtime(time); # ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) my ($yr1, $yr2) = ($thedt[5]+1900, $thedt[5]+1900); if ($thedt[4] == 11) { $yr2++; } if ($thedt[4] == 0) { $yr1--; } SWITCH: { $dt =~ 'Christmas\s+Eve' && do { $dt = '24 Dec '.$yr1; last SWITCH; }; $dt =~ 'Christmas\s+Day' && do { $dt = '25 Dec '.$yr1; last SWITCH; }; $dt =~ 'Boxing\s+Day' && do { $dt = '26 Dec '.$yr1; last SWITCH; }; $dt =~ 'New\s+Years\s+Eve' && do { $dt = '31 Dec '.$yr1; last SWITCH; }; $dt =~ 'New\s+Years\s+Day' && do { $dt = '1 Jan ' .$yr2; last SWITCH; }; undef $dt; } } if ($dt) { #v1.3: $showtime = DateTime::Format::DateParse->parse_datetime( $dt, 'Europe/London' ); # workaround for bug in Date::Time::str2time() which generates wrong dates for future months when no year is given # https://rt.cpan.org/Public/Bug/Display.html?id=92611 $dt .= ' '.( $theday->year() ) if $dt !~ /(19|20)\d\d/; $showtime = DateTime->from_epoch( epoch=>str2time( $dt, 'GMT' ) )->set_time_zone('Europe/London'); } else { $showtime = $theday->clone; } $h += 12 if $a eq 'pm' && $h < 12; # e.g. 12:30pm means 12:30 ! $h -= 12 if $a eq 'am' && $h == 12; # e.g. 12:30am means 00:30 ! $h2 += 12 if $a2 eq 'pm' && $h2 < 12; $h2 -= 12 if $a2 eq 'am' && $h2 == 12; $showtime->set(hour => $h, minute => $i, second => 0); $prog{'start'} = $showtime->strftime("%Y%m%d%H%M%S %z"); my $showtime_ = $showtime->clone; if (defined $h2 && $h2 >= 0) { $showtime->add (days => 1) if $h2 < $h; # see note above re errors with GMT/BST transition eval { # try $showtime->set(hour => $h2, minute => $i2, second => 0); $prog{'stop'} = $showtime->strftime("%Y%m%d%H%M%S %z"); } or do { # catch # let's see if we can get a duration my ($durh, $durm) = $lhs->as_text =~ /\((?:(\d*)\shours?)?\s?(?:(\d*)\sminutes?)?\)/; if (defined $durh || defined $durm) { $durh = 0 if !defined $durh; $durm = 0 if !defined $durm; $showtime_->set_time_zone('UTC')->add( hours => $durh, minutes => $durm )->set_time_zone('Europe/London'); $prog{'stop'} = $showtime_->strftime("%Y%m%d%H%M%S %z"); } else { # no output prog 'stop' time } } } else { # no output prog 'stop' time } # if the first programme starts before 06:00 we should ignore it as it will create a duplicate programme if ($h < 6) { # skip this prog as we have already retrieved it with 'yesterday's schedule next if ($count == 1); } } # programme image my $c = $showdetail->look_down('_tag' => 'div', 'id' => 'divCentre'); if ($c) { my $img = $c->look_down('_tag' => 'div', 'id' => 'headerImage'); if ($img) { my $showattr = $img->attr('style'); (my $showimage) = $showattr =~ /background-image:\s*url\((.*?\.jpg)\)/; if ($showimage) { $prog{'image'} = [[ $showimage, { 'system'=>'tvguide', 'type'=>'backdrop' } ]]; } } } } # end showdetail $showdetail->delete() if $showdetail; } # end nodetailspage # channel $prog{'channel'} = $xmlchannel_id; # title (mandatory) # Baywatch my $showtitle = $show->look_down('_tag' => 'span', 'class' => 'programmeheading'); $prog{'title'} = [[ encode('utf-8', $showtitle->as_text), 'en' ]]; $showtitle->detach; # Note: is used by StartTime then SubTitle then Category then Subtitles/B&W/etc # start (mandatory) # 3:00pm # (don't add it even we already have it from the detail page but we still need to delete it from the tree) # @ 2014-04-02 the site has changed to # 6:00 am # but this just doesn't sound right to me (i.e. I think it might change again), so let's try both ways # # @2016-08-05 looks like this is permanent # my $showstart = $show->look_down('_tag' => 'span', 'class' => 'tvchannel'); my $showstart = $show->look_down('_tag' => 'span', 'class' => 'season'); if (!$prog{'start'}) { my ($h, $i, $a) = $showstart->as_text =~ /(\d*):(\d*)\s*(am|pm)/; if (!defined $a) { $showstart = $show->look_down('_tag' => 'span', 'class' => 'season'); ($h, $i, $a) = $showstart->as_text =~ /(\d*):(\d*)\s*(am|pm)/; } $h += 12 if $a eq 'pm' && $h < 12; # e.g. 12:30pm means 12:30 ! $h -= 12 if $a eq 'am' && $h == 12; # e.g. 12:30am means 00:30 ! $showtime = $theday->clone; # @ 2023-03-25 The assumption that the site works from 06:00 - 06:00 breaks when the first programme overlaps 06:00 # so it seems the rule is the 'first' programme ends *after* 6:00 am # Saturday, March 25, 2023 ITVBe # 1:00am Teleshopping # 7:00am Sam Faiers: The Mummy Diaries # # Saturday, March 25, 2023 BBC Two HD # 3:45am This Is BBC Two # 6:35am Hey Duggee # Only way I can think of to handle this is to tag the first programme on a day # In addition, this programme should be ignored to avoid duplicates # if ($h < 6) { # skip this prog as we have already retrieved it with 'yesterday's schedule next if ($count == 1); $showtime->add(days => 1) if ($h < 6); # site runs from 06:00-06:00 so anything <06:00 is for tomorrow } $showtime->set(hour => $h, minute => $i, second => 0); $prog{'start'} = $showtime->strftime("%Y%m%d%H%M%S %z"); # no prog 'stop' time available } $showstart->detach; # category # Category General Movie/Drama my $showcategory = $show->look_down('_tag' => 'span', 'class' => 'tvchannel', sub { $_[0]->as_text =~ /^Category\s*$/ } ); if ($showcategory) { $showcategory = $showcategory->right; my @showcategory = split(/\//, $showcategory->as_text); my @showcategories = (); foreach my $category (@showcategory) { # category translation? if (defined(&map_category)) { $category = map_category($category); } if ($category =~ /\|/) { foreach my $cat (split(/\|/, $category)) { push @showcategories, $cat unless grep(/$cat/, @showcategories); } } elsif ($category ne '') { push @showcategories, $category unless grep(/$category/, @showcategories); } } foreach my $category (@showcategories) { push @{$prog{'category'}}, [ encode('utf-8', $category), 'en' ]; } $showcategory->left->detach; } # desc # Dissolving bikinis cause a stir on the beach my $showdesc = $show->look_down('_tag' => 'span', 'class' => 'programmetext'); if ($showdesc) { $showdesc = $showdesc->as_text; $showdesc .= '.' if ( (length $showdesc) && ((substr $showdesc,-1,1) ne '.') ); # append a fullstop if (length $showdesc) { $prog{'desc'} = [[ encode('utf-8', $showdesc), 'en' ]]; } } # year # strip this off the title e.g. "A Useful Life (2010)" my ($showyear) = $prog{'title'}->[0][0] =~ /.*\((\d\d\d\d)\)$/; if ($showyear) { $prog{'date'} = $showyear; # assume anything with a year is a film - add Films category group push @{$prog{'category'}}, [ 'Films', 'en' ]; } # flags # (Subtitles) (Black & White) my $showflags = $show->look_down('_tag' => 'span', 'class' => 'tvchannel', sub { $_[0]->as_text =~ /Subtitles/ } ); if ($showflags) { push @{$prog{'subtitles'}}, {'type' => 'teletext'}; $showflags->detach; } $showflags = $show->look_down('_tag' => 'span', 'class' => 'tvchannel', sub { $_[0]->as_text =~ /Audio Described/ } ); if ($showflags) { # push @{$prog{'subtitles'}}, {'type' => 'deaf-signed'}; <-- Audio Described is not deaf-signed $showflags->detach; } $showflags = $show->look_down('_tag' => 'span', 'class' => 'tvchannel', sub { $_[0]->as_text =~ /Repeat/ } ); if ($showflags) { # push @{$prog{'previously-shown'}}, {}; $prog{'previously-shown'} = {}; $showflags->detach; } my $showvideo = $show->look_down('_tag' => 'span', 'class' => 'tvchannel', sub { $_[0]->as_text =~ /Black & White/ } ); if ($showvideo) { $prog{'video'}->{'colour'} = '0'; $showvideo->detach; } #if ($showflags && $showflags->as_text =~ '\[REP\]') { # push @{$prog{'previously-shown'}}, {}; #} $showflags = $show->look_down('_tag' => 'span', 'class' => 'tvchannel', sub { $_[0]->as_text =~ /Interactive/ } ); if ($showflags) { # no flag in DTD for this $showflags->detach; } # episode number # Season 2 Episode 3 of 22 my @showepisode = $show->look_down('_tag' => 'span', 'class' => 'season'); my $showepisode; foreach my $el (@showepisode) { $showepisode .= $el->as_text; } if ($showepisode) { my ($showsxx, $showexx, $showeof) = ( $showepisode =~ /^(?:(?:Series|Season) (\d+)(?:[., :]+)?)?(?:Episode (\d+)(?: of (\d+))?)?/ ); # scan the description for any "Part x of x." info my ($showpxx, $showpof) = ('', ''); ($showpxx, $showpof) = ( $showdesc =~ /Part (one|two|three|four|five|six|seven|eight|nine|\d+)(?: of (one|two|three|four|five|six|seven|eight|nine|\d+))?/ ) if ($showdesc); my $showepnum = make_ns_epnum($showsxx, $showexx, $showeof, $showpxx, $showpof); if ($showepnum && $showepnum ne '...') { $prog{'episode-num'} = [[ $showepnum, 'xmltv_ns' ]]; } #debug "--$showepnum-- ".$showepisode->as_text; } # episode title # The Fabulous Buchannon Boys my $showeptitle = $show->look_down('_tag' => 'span', 'class' => 'tvchannel'); if ($showeptitle) { if ($showeptitle->as_text =~ /\(?Premiere\)?/) { $prog{'premiere'} = []; } else { $prog{'sub-title'} = [[ encode('utf-8', $showeptitle->as_text), 'en' ]]; } $showeptitle->detach; } # director # never seen one but let's assume they're in the description if (!$prog{'credits'}->{'director'}) { if ($showdesc) { my ($directors) = ( $showdesc =~ /(?:Directed by|Director) ([^\.]*)\.?/ ); if ($directors) { $directors =~ s/ (with|and) /,/ig; $directors =~ s/ (singer|actor|actress) //ig; # strip these words $directors =~ s/,,/,/g; # delete empties $directors = encode('utf-8', $directors); # encode names into utf-8 #debug $directors; my @directors = split(/,/, $directors); s{^\s+|\s+$}{}g foreach @directors; # strip leading & trailing spaces $prog{'credits'}->{'director'} = \@directors if (scalar @directors > 0); } } } # actors # these are buried in the description :-( if (!$prog{'credits'}->{'actor'}) { if ($showdesc) { my ($actors) = ( $showdesc =~ /(?:starring)([^\.]*)\.?/i ); if ($actors) { $actors =~ s/ (also|starring|with|and) /,/ig; # may be used to separate names $actors =~ s/ (singer|actor|actress) //ig; # strip these words $actors =~ s/,,/,/g; # delete empties $actors = encode('utf-8', $actors); # encode names into utf-8 #debug $actors; my @actors = split(/,/, $actors); s{^\s+|\s+$}{}g foreach @actors; # strip leading & trailing spaces $prog{'credits'}->{'actor'} = \@actors if (scalar @actors > 0); } } } # rating # Rating
3.9 my $showrating = $show->look_down('_tag' => 'span', 'class' => 'programmetext', sub { $_[0]->as_trimmed_text =~ /^Rating$/ } ); if ($showrating) { $showrating = $showrating->right; $showrating = $showrating->right if ($showrating->tag eq 'br'); if ($showrating->tag eq 'span' && $showrating->attr('class') eq 'programmeheading') { if ($showrating->as_text) { $prog{'star-rating'} = [ $showrating->as_text . '/10' ]; } } } # programme url # my $showurl = $show->look_down('_tag' => 'a', 'title' => 'Click to rate and review'); if ($showurl) { $prog{'url'} = [ encode( 'utf-8', $showurl->attr('href') ) ]; } # programme image #
my $showattr = $show->attr('style'); (my $showimage) = $showattr =~ /background-image:\s*url\((.*?\.jpg)\)/; if ($showimage) { $prog{'image'} = [] if not defined $prog{'image'} or not @{$prog{'image'}}; push @{$prog{'image'}}, [ $showimage, { 'system'=>'tvguide', 'type'=>'backdrop' } ]; } # debug Dumper \%prog; push(@{$programmes}, \%prog); $alt_success = 1; } # if this is an alternative id then remember it if ($alt_success) { if ($alt_channel_id != $channel_id) { $channels_alt_found->{$channel_id} = $alt_channel_id; debug('Found working alternative ID '.$alt_channel_id.' for '.$channel_id); } } } else { # no schedule found debug "No schedule found for channel ID: $alt_channel_id"; # append alternative channel numbers to list if (scalar @alts == 1) { push @alts, get_alt_channel_ids($channel_id, $channelname); debug "Found alternative IDs: @alts[1..$#alts]" if (scalar @alts > 1); } # issue warning only when all alternatives have been tried warning 'No schedule found' if ($alt_channel_id == $alts[$#alts]); } undef @shows; # Add to the channels hash $channels->{$channel_id} = { 'id'=> $xmlchannel_id , 'display-name' => [[ encode('utf-8', $channelname), 'en' ]] }; $tree->delete(); } else { # tree conversion failed warning 'Could not parse the page'; } last if $alt_success; } $bar->update if defined $bar; debug('-' x 30); } } } # ############################################################################# # # THE VEG ###################################################################### # ------------------------------------------------------------------------------------------------------------------------------------- # sub make_ns_epnum { # Convert an episode number to its xmltv_ns compatible - i.e. reset the base to zero # Input = series number, episode number, total episodes, part number, total parts, # e.g. "1, 3, 6, 2, 4" >> "0.2/6.1/4", "3, 4" >> "2.3." # my ($s, $e, $e_of, $p, $p_of) = @_; # debug Dumper(@_); # "Part x of x" may contain integers or words (e.g. "Part 1 of 2", or "Part one") $p = text_to_num($p) if defined $p; $p_of = text_to_num($p_of) if defined $p_of; # re-base the series/episode/part numbers $s-- if (defined $s && $s > 0); $e-- if (defined $e && $e > 0); $p-- if (defined $p && $p && $p=~/^\d+$/ && $p > 0); # make the xmltv_ns compliant episode-num my $episode_ns = ''; $episode_ns .= $s if defined $s; $episode_ns .= '.'; $episode_ns .= $e if defined $e; $episode_ns .= '/'.$e_of if defined $e_of; $episode_ns .= '.'; $episode_ns .= $p if $p; $episode_ns .= '/'.$p_of if $p_of; #debug "--$episode_ns--"; return $episode_ns; } sub text_to_num { # Convert a word number to int e.g. 'one' >> '1' # my ($text) = @_; if ($text !~ /^[+-]?\d+$/) { # standard test for an int my %nums = (one => 1, two => 2, three => 3, four => 4, five => 5, six => 6, seven => 7, eight => 8, nine => 9); return $nums{$text} if exists $nums{$text}; } return $text } sub map_channel_id { # Map the fetched channel_id to a different value (e.g. our PVR needs specific channel ids) # mapped channels should be stored in a file called tv_grab_uk_tvguide.map.conf # containing lines of the form: map==fromchan==tochan e.g. 'map==109==BBC4' # my ($channel_id) = @_; my $mapchannels = \%mapchannelhash; if (%mapchannelhash && exists $mapchannels->{$channel_id}) { return $mapchannels->{$channel_id} ; } return $channel_id; } sub map_category { # Map the fetched category to a different value (e.g. our PVR needs specific genres) # mapped categories should be stored in a file called tv_grab_uk_guardian.map.conf # containing lines of the form: cat==fromcategory==tocategory e.g. 'cat==General Movie==Film' # my ($category) = @_; my $mapcategories = \%mapcategoryhash; if (%mapcategoryhash && exists $mapcategories->{$category}) { return $mapcategories->{$category} ; } return $category; } sub loadmapconf { # Load the conf file containing mapped channels and categories information # # This file contains 2 record types: # lines starting with "map" are used to 'translate' the incoming channel id to those required by your PVR # e.g. map==dave==DAVE will output "DAVE" in your XML file instead of "dave" # lines starting with "cat" are used to translate categories (genres) in the incoming data to those required by your PVR # e.g. cat==Science Fiction==Sci-fi will output "Sci-Fi" in your XML file instead of "Science Fiction" # my $mapchannels = \%mapchannelhash; my $mapcategories = \%mapcategoryhash; # my $supplementdir = $ENV{XMLTV_SUPPLEMENT} || GetSupplementDir(); # get default file from supplement.xmltv.org if local file not exist if (-f File::Spec->catfile( $supplementdir, $GRABBER_NAME, $GRABBER_NAME . '.map.conf' ) ) { SetSupplementRoot($supplementdir); } my $lines = GetSupplementLines($GRABBER_NAME, $GRABBER_NAME . '.map.conf'); foreach my $line (@$lines) { my ($type, $mapfrom, $mapto, $trash) = $line =~ /^(.*)==(.*)==(.*?)([\s\t]*#.*)?$/; SWITCH: { lc($type) eq 'map' && do { $mapchannels->{$mapfrom} = $mapto; last SWITCH; }; lc($type) eq 'cat' && do { $mapcategories->{$mapfrom} = $mapto; last SWITCH; }; warning "Unknown type in map file: \n $line"; } } # debug Dumper ($mapchannels, $mapcategories); } sub fetch_all_channel_ids { # Fetch all channel IDs with method 1, used for channel list creation and alternative ID searches # my $channels = {}; my $tree = fetch_url('https://www.tvguide.co.uk/mychannels.asp?gw=1242', 'post', [ thisDay => '', thisTime => '', gridSpan => '', emailaddress => '', regionid => 1, systemid => 5, xn => 'Show me the channels' ]); my @c = $tree->look_down('_tag' => qr/table|tr/, 'class' => qr/^tr[XC]/); my $j = 0 if $opt->{test}; # --test is an undocumented (private) option foreach (@c) { my ($ch, $id, $l, $t); if ($_->id =~ /^trX?\d+/) { ($id) = $_->id =~ /^trX?(\d+)/; ($ch) = $ROOT_URL.'channellistings.asp?ch='.$id; ($l) = $_->as_HTML =~ /background-image:url\(([^)]+)\)/; ($t) = $_->as_text; } $channels->{$id} = {id => $id . (!$opt->{'list-channels'}?" # ".encode('utf-8', $t):(!$opt->{legacychannels}?'.tvguide.co.uk':'')), 'display-name' => [[ encode('utf-8', $t), 'en' ]], icon => [{ 'src'=>$l }], url => [ $ch ], } if $id; debug $id if $opt->{test}; last if $opt->{test} and (++$j >= $opt->{test}); # limit during testing } return $channels; } sub get_alt_channel_ids ($$) { # Find alternate IDs for the same exact channel name # my ($channel_id, $channel_name) = @_; if (defined $channels_alt->{$channel_id}) { return @{$channels_alt->{$channel_id}}; } $channels_cache = fetch_all_channel_ids() if (scalar keys %$channels_cache == 0); my @alts; foreach my $id (keys %$channels_cache) { next if $id == $channel_id; push @alts, $id if ($channels_cache->{$id}->{'display-name'}[0][0] eq encode('utf-8', $channel_name) ); } $channels_alt->{$channel_id} = \@alts; return @{$channels_alt->{$channel_id}}; } sub fetch_channels { # ParseOptions() handles --configure and --list-channels internally without returning, # so we do not have global $opt available during --configure # unless we copy vars to main package. (Note package variable must be declared with 'our') # ($main::conf, $main::opt) = @_; my $channels = {}; # Get the list of available channels #------------------------------------------------------------------------------------------- # preferred method (uses a html select list of channels) # if ((scalar keys %$channels == 0) && (!defined $opt->{method} || $opt->{method} == 0)) { my $bar = new XMLTV::ProgressBar({ name => "Fetching channels", count => 1 }) unless ($opt->{quiet} || $opt->{debug}); # Fetch channels via a dummy call to BBC1 listings # my $channel_list = $ROOT_URL.'channellistings.asp?ch=74&cTime='; my @channels; my $tree = XMLTV::Get_nice::get_nice_tree($channel_list); # $tree->dump; my $_channels = $tree->look_down('_tag' => 'select', 'name' => 'ch'); if (defined $_channels) { @channels = $_channels->look_down('_tag' => 'option'); # debug $_channels->as_HTML; # foreach my $xchannel (@channels) { debug $xchannel->as_HTML; } } $bar->update() if defined $bar; $bar->finish() && undef $bar if defined $bar; if (scalar @channels > 0) { $bar = new XMLTV::ProgressBar({ name => "Parsing result", count => scalar @channels }) unless ($opt->{quiet} || $opt->{debug}); # Browse through the downloaded list of channels and map them to a hash XMLTV::Writer would understand foreach my $channel (@channels) { if ($channel->as_text) { my ($id) = $channel->attr('value'); my ($url) = 'channellistings.asp?ch=' . $channel->attr('value'); my ($name) = $channel->as_text; $channels->{$id} = { id => $id . (!$opt->{'list-channels'}?" # ".encode('utf-8', $name):(!$opt->{legacychannels}?'.tvguide.co.uk':'')), 'display-name' => [[ encode('utf-8', $name), 'en' ]], url => [ $ROOT_URL.$url ] }; } $bar->update() if defined $bar; } $bar->finish() && undef $bar if defined $bar; } warning "No channels found in TVGuide" if (scalar keys %$channels == 0); } #------------------------------------------------------------------------------------------- # workaround for broken TVGuide website [2022-01-26] # # alternative Method 1 (credit: mkbloke ) # # if ((scalar keys %$channels == 0) && (!defined $opt->{method} || $opt->{method} == 1)) { # alternative method 1 # fetches channels from the website's "mychannels" page # (preferred method found 899, this method finds 939 ) # (as at 2023-01-25 it finds 1207 channels, but many are duplicates with no data) warning "Trying alternative method 1"; my $bar = new XMLTV::ProgressBar({ name => "Fetching channels", count => 1 }) unless ($opt->{quiet} || $opt->{debug}); $channels = fetch_all_channel_ids(); $bar->update() if defined $bar; $bar->finish() && undef $bar if defined $bar; warning "Found ".(scalar keys %$channels)." channels"; } #------------------------------------------------------------------------------------------- # alternative Method 2 # # if ((scalar keys %$channels == 0) && (!defined $opt->{method} || $opt->{method} == 2)) { # alternative method 2 # fetches channels from the website's mobile-friendly page # (preferred method found 899, this method finds 387 ) warning "Trying alternative method 2"; my $bar = new XMLTV::ProgressBar({ name => "Fetching channels", count => 12 }) unless ($opt->{quiet} || $opt->{debug}); foreach (qw/ 7 3 12 5 25 8 22 19 10 29 18 23 /) { my $url = 'https://www.tvguide.co.uk/mobile/?systemid='.$_; #debug "Fetching: $url"; my $tree = XMLTV::Get_nice::get_nice_tree($url); my @c = $tree->look_down('_tag' => 'div', 'class' => 'div-channel-progs'); foreach (@c) { my ($ch, $id, $l, $t); $ch = $_->look_down('_tag' => 'a')->attr('href'); ($id) = $ch =~ m/\?ch=(\d+)$/; my $_dl = $_->look_down('_tag' => 'div', 'class' => 'div-channel-logo'); if ($_dl) { # don't chain the look_downs (for better robustness) my $_il = $_dl->look_down('_tag' => 'img', 'class' => 'img-channel-logo'); if ($_il) { $l = $_il->attr('src'); $t = $_il->attr('alt'); $t =~ s/ TV Listings$//; } } $ch =~ s/mobile\/channellisting\.asp/channellistings\.asp/; $channels->{$id} = {id => $id . (!$opt->{'list-channels'}?" # ".encode('utf-8', $t):(!$opt->{legacychannels}?'.tvguide.co.uk':'')), 'display-name' => [[ encode('utf-8', $t), 'en' ]], icon => [{ 'src'=>$l }], url => [ $ch ], } if $id; } $bar->update() if defined $bar; } $bar->finish() && undef $bar if defined $bar; warning "Found ".(scalar keys %$channels)." channels \n"; } #------------------------------------------------------------------------------------------- # does user want to process the list of channels to: # 1) make a list of channels without programme schedule data on tvg website # 2) remove these channels from their configuration list if (defined $opt->{makeignorelist}) { makeignorelist($channels); } if (defined $opt->{useignorelist}) { useignorelist($channels); warning "Remaining ".(scalar keys %$channels)." channels \n"; } #debug Dumper $channels; #------------------------------------------------------------------------------------------- # Format & write out the config file # Notifying the user :) #$bar = new XMLTV::ProgressBar({ # name => "Reformatting", # count => 1 #}) unless ($opt->{quiet} || $opt->{debug}); if (scalar keys %$channels == 0) { warning "No channels found in TVGuide \n"; exit 1; } # Let XMLTV::Writer format the results as a valid xmltv file my $result; my $writer = new XMLTV::Writer(OUTPUT => \$result, encoding => 'utf-8'); $writer->start({'generator-info-name' => $generator_info_name}); # # this writes the channels sorted by 'id' but the TVG id has no relation to the actual channel number, # and makes it harder to select the channels to fetch ### $writer->write_channels($channels); # so let's write them by name foreach (sort { $channels->{$a}->{'display-name'}[0][0] cmp $channels->{$b}->{'display-name'}[0][0] } keys %$channels) { $writer->write_channel($channels->{$_}); } # $writer->end(); #$bar->update() if defined $bar; $bar->finish() && undef $bar if defined $bar; return $result; } sub makeignorelist ($) { my $channels = shift; if (defined $opt->{makeignorelist}) { #-------------------------------------------------------------------------------------------# # this fetch pulls in a lot of 'duplicate' channels (16x Sky Sports Golf !) which, when you look at them # are devoid of listings. # let's see if we can root those out at this stage and not present them to the poor wee user (who doesn't know # which one to pick during '--configure' ) # get the listings page for this channel and see if it has data # this reduces to 1x Sky Sports Golf :-) # # reduced total to 494 channels (from 1207 in method 1) # # doing this for each channel will, of course, slow down the channel fetch # - so we only do it when asked with --makeignorelist # #-------------------------------------------------------------------------------------------# warning "Checking channels for listings data"; warning "Creating the check list will take about an hour.\n You don't need to do this every time - unless your check list \n is very out-of-date you can simply use your existing check list. \n (Ctrl+C to abort)"; my $bar = new XMLTV::ProgressBar({ name => "Checking channels", count => (scalar keys %$channels) }) unless ($opt->{quiet} || $opt->{debug}); my $skipchannels = []; my $i = 0 if $opt->{test}; my @channels = keys %$channels; for my $channel (@channels) { last if $opt->{test} and (++$i >= $opt->{test}); # limit during testing (my $id) = $channels->{$channel}->{id} =~ /^(\d+)/; next if !$id; #debug "Checking: $id"; # are there any listings for tomorrow, for this channel? my $url = $ROOT_URL.'channellistings.asp' . '?ch=' . $id . '&cTime=' . uri_escape( DateTime->today->add(days => 1)->set_time_zone('Europe/London')->strftime('%m/%d/%Y 00:00:00') ); #debug "Fetching: $url"; my $tree = fetch_url($url); if ($tree) { # (see fetch_listings for details) my @shows = $tree->look_down('_tag' => 'table', 'border' => '0', 'cellpadding' => '0', 'style' => qr/background:\s*black;border-collapse:\s*collapse;/); if (scalar @shows == 0) { # empty listings schedule push @$skipchannels, $id; debug "Channel ".$channels->{$channel}->{id}." - SKIPPED"; } } $bar->update() if defined $bar; } $bar->finish() && undef $bar if defined $bar; # write the ignore list my $fn = ($opt->{makeignorelist} || 'uktvguideignorelist'); debug "Writing $fn"; open my $fh, '>', $fn or die "Can't open file $!"; print $fh '# LIST OF IGNORED CHANNELS'."\n"; print $fh '# These channels contain no data, so it is safe to '."\n"; print $fh '# remove them from the configuration file.'."\n"; foreach (@$skipchannels) { print $fh 'channel='.$_."\n"; } close $fh; } } sub useignorelist ($) { my $channels = shift; if (defined $opt->{useignorelist}) { warning "Removing ignored channels"; # read the ignore list my $fn = ($opt->{useignorelist} || 'uktvguideignorelist'); debug "Reading $fn"; open my $fh, '<', $fn or die "Can't open file $!"; while (my $r = <$fh>) { chomp $r; next if $r =~ /^\s*#/; (my $id) = $r =~ /^channel=(\d+)$/; next if !$id; # remove from channellist delete $channels->{$id} if ($channels->{$id}); } close $fh; } } sub config_stage { my( $stage, $conf ) = @_; die "Unknown stage $stage" if $stage ne "start"; my $result; my $writer = new XMLTV::Configure::Writer( OUTPUT => \$result, encoding => 'utf-8' ); $writer->start( { grabber => $GRABBER_NAME } ); $writer->write_string({ id => 'cachedir', title => [ [ 'Directory to store the cache in', 'en' ] ], description => [ [ $GRABBER_NAME.' uses a cache with files that it has already '. 'downloaded. Please specify where the cache shall be stored. ', 'en' ] ], default => get_default_cachedir(), }); $writer->end( 'select-channels' ); return $result; } sub config_check { if (not defined( $conf->{cachedir} )) { print STDERR "No cachedir defined in configfile " . $opt->{'config-file'} . "\n" . "Please run the grabber with --configure.\n"; exit 1; } if (not defined( $conf->{'channel'} )) { print STDERR "No channels selected in configfile " . $opt->{'config-file'} . "\n" . "Please run the grabber with --configure.\n"; exit 1; } } sub fetch_url ($;$$) { # fetch a url with up to 5 retries my ($url, $method, $varhash) = @_; $XMLTV::Get_nice::FailOnError = 0; my $content; my $maxretry = 5; my $retry = 0; if (defined $method && lc($method) eq 'post') { my $ua = initialise_ua(); while ( (not defined($content)) || (length($content) == 0) ) { my $r = $ua->post($url, $varhash); $content = $r->content; if ( $r->is_error || (length($content) == 0) ) { print STDERR "HTTP error: ".$r->status_line."\n"; $retry++; return undef if $retry > $maxretry; print STDERR "Retrying URL: $url (attempt $retry of $maxretry) \n"; } } } else { while ( (not defined($content = XMLTV::Get_nice::get_nice($url))) || (length($content) == 0) ) { my $r = $XMLTV::Get_nice::Response; print STDERR "HTTP error: ".$r->status_line."\n"; $retry++; return undef if $retry > $maxretry; print STDERR "Retrying URL: $url (attempt $retry of $maxretry) \n"; } } $content = decode('UTF-8', $content); my $t = HTML::TreeBuilder->new(); $t->parse($content) or die "cannot parse content of $url\n"; $t->eof; return $t; } sub get_default_dir { my $winhome = $ENV{HOMEDRIVE} . $ENV{HOMEPATH} if defined( $ENV{HOMEDRIVE} ) and defined( $ENV{HOMEPATH} ); my $home = $ENV{HOME} || $winhome || "."; return $home; } sub get_default_cachedir { return get_default_dir() . "/.xmltv/cache"; } sub init_cachedir { my( $path ) = @_; if( not -d $path ) { mkpath( $path ) or die "Failed to create cache-directory $path: $@"; } } sub debug ( $$ ) { my( $message, $nonewline ) = @_; print STDERR $message if $opt->{debug}; print STDERR "\n" if $opt->{debug} && (!defined $nonewline || $nonewline != 1); } sub warning ( $ ) { my( $message ) = @_; print STDERR $message . "\n"; $warnings++; } sub initialise_ua { my $cookies = HTTP::Cookies->new; #my $ua = LWP::UserAgent->new(keep_alive => 1); my $ua = LWP::UserAgent->new; # Cookies $ua->cookie_jar($cookies); # Define user agent type $ua->agent('Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US)'); # Define timouts $ua->timeout(240); # Use proxy if set in http_proxy etc. $ua->env_proxy; return $ua; } # ############################################################################# __END__ =pod =head1 NAME B - Grab TV listings for UK from the TVGuide website. =head1 SYNOPSIS tv_grab_uk_tvguide --usage tv_grab_uk_tvguide --version tv_grab_uk_tvguide --configure [--config-file FILE] [--gui OPTION] [--method N] [--makeignorelist FILE] [--useignorelist FILE] [--debug] tv_grab_uk_tvguide [--config-file FILE] [--output FILE] [--days N] [--offset N] [--nodetailspage] [--legacychannels] [--quiet] [--debug] tv_grab_uk_tvguide --list-channels [--output FILE] [--method N] [--debug] =head1 DESCRIPTION Output TV listings in XMLTV format for many channels available in UK. The data come from tvguide.co.uk First you must run B to choose which channels you want to receive. Then running B with no arguments will get programme listings in XML format for the channels you chose, for available days including today. =head1 OPTIONS B<--configure> Prompt for which channels to fetch the schedule for, and write the configuration file. B<--config-file FILE> Set the name of the configuration file, the default is B<~/.xmltv/tv_grab_uk_tvguide.conf>. This is the file written by --configure and read when grabbing. B<--gui OPTION> Use this option to enable a graphical interface to be used. OPTION may be 'Tk'. Additional allowed values of OPTION are 'Term' for normal terminal output (default) and 'TermNoProgressBar' to disable the use of Term::ProgressBar. (May not work on Windows OS.) B<--days N> Grab N days. The default is 5 days. B<--offset N> Start N days in the future. The default is to start from today. B<--nodetailspage> Only fetch summary information for each programme. (See discussion below). B<--legacychannels> Channel ids were made compliant with the XMLTV specification in December 2020. Use --legacychannels to output channel ids in the previous format (i.e. number only). B<--output FILE> Write to FILE rather than standard output. B<--method N> This program has three methods for fetching the list of channels available. The preferred method can be overridden with an option of either --method 1 or --method 2 to use one of the two alternative methods. If no --method parameter is supplied then the various methods will be tried in sequence until a channels list is obtained. A parameter of --method 0 will run the preferred method only. Normally you should omit this parameter. B<--makeignorelist FILE> The source channel list contains many channels which have no programme listings available. By using a local file of channel ids these can be filtered out during the configure process. This option creates a list of channel ids which can be excluded from your config file makeignorelist can take over an hour to run but you only need run it infrequently (not every time you run --configure). B<--useignorelist FILE> Specify the file containing a list of channel ids to be removed during the configure stage. This may be the file just created with --makeignorelist (e.g. --configure --makeignorelist myfile --useignorelist myfile ) B<--quiet> Suppress the progress messages normally written to the console. B<--debug> Provide more information on progress to standard error to help in debugging. B<--list-channels> Output a list (in xmltv format) of all channels that can be fetched. B<--version> Show the version of the grabber. B<--usage> Print a help message and exit. =head1 INSTALLATION The file F has two purposes. Firstly you can map the channel ids used by the site into something more meaningful to your PVR. E.g. map==74==BBC 1 will change "74" to "BBC 1" in the output XML. Note: the lines are of the form "map=={channel id}=={my name}". The second purpose is to likewise translate genre names. So if your PVR doesn"t have a category for "Science Fiction" but uses "Sci-fi" instead, then you can specify cat==Science Fiction==Sci-fi and the output XML will have "Sci-fi". IMPORTANT: the downloaded "tv_grab_uk_tvguide.map.conf" contains example lines to illustrate the format - you should edit this file to suit your own purposes! =head1 ERROR HANDLING If the grabber fails to download data for some channel on a specific day, it will print an errormessage to STDERR and then continue with the other channels and days. The grabber will exit with a status code of 1 to indicate that the data is incomplete. =head1 ENVIRONMENT VARIABLES The environment variable HOME can be set to change where configuration files are stored. All configuration is stored in $HOME/.xmltv/. On Windows, it might be necessary to set HOME to a path without spaces in it. =head1 SUPPORTED CHANNELS For information on supported channels, see http://tvguide.co.uk/ =head1 XMLTV VALIDATION B may report an error similar to: "Line 5 Invalid channel-id BBC 1" This is a because ValidateFile.pm insists the channel-id adheres to RFC2838 despite the xmltv.dtd only saying "preferably" not "SHOULD". (Having channel ids of the form "bbc1.bbc.co.uk" will be rejected by many PVRs since they require the data to match their own list.) It may also report: "tv_sort failed on the concatenated data. Probably due to overlapping data between days." Both these errors can be ignored. =head1 USING --nodetailspage This option may be useful if you have problems accessing the tvguide website: it will considerably speed up your grabbing, but at the expense of data richness. The details page has a better list of actors, as well as director's names, film classifications, and background images. More significantly, the details page includes programme duration and programme end time. If you don't include the details page then your output programmes will not have stop times. Although stop times are an optional XMLTV element, many downstream programs expect them and break without them. Fortunately, these can be added by piping your xml file through the tv_sort filter: this will add stop times to all programmes except for the last programme on every channel. For example: tv_grab_uk_tvguide --days 3 --nodetailspage | tv_sort --by-channel --output myprogrammes.xml B As at January 2022 it seems the actor/director names and film classification is no longer present in the website, although it may be there for some channels(?). Therefore you may find the --nodetailspage option useful to significantly reduce your run time. B, the details page does include background image(s) of the programme, which is not present in your xml file when using --nodetailspage =head1 BE KIND If using the details page from the website then your grabbing might benefit from a more targeted strategy, rather than blithely getting all days for all channels. Since the published schedule rarely changes, a strategy of grabbing the next 3 days plus the 1 newest day would give you any new programmes as well as any last minute changes. Simply fetch "--offset 0 --days 3" and concatenate it with a separate fetch of "--offset 7 --days 1". For example: =over 6 ( tv_grab_uk_tvguide --days 3 --offset 0 --output temp.xml ) && ( tv_grab_uk_tvguide --days 1 --offset 7 | tv_cat --output myprogrammes.xml temp.xml - ) && ( rm -f temp.xml ) =back A similar strategy could be employed when using --nodetailspage if you have a lot of channels: =over 6 ( tv_grab_uk_tvguide --days 3 --offset 0 --nodetailspage | tv_sort --by-channel >temp.xml ) && ( tv_grab_uk_tvguide --days 1 --offset 7 --nodetailspage | tv_sort --by-channel | tv_cat --output myprogrammes.xml temp.xml - ) && ( rm -f temp.xml ) =back This avoids overloading the TVGuide website, and significantly reduces your runtime with minimal impact on your viewing schedule. TVGuide provide this data for free, so let's not abuse their generosity. =head1 DISCLAIMER The TVGuide website's license for these data does not allow non-personal use. Certainly, any commercial use of listings data obtained by using this grabber will breach copyright law, but if you are just using the data for your own personal use then you are probably fine. By using this grabber you aver you are using the listings data for your own personal use only and you absolve the author(s) from any liability under copyright law or otherwise. =head1 AUTHOR Geoff Westcott. This documentation and parts of the code based on various other tv_grabbers from the XMLTV-project. =head1 SEE ALSO L. =cut To Do ===== 1. Improve the progress bar update frequency 2. Add actor 'character' attribute DONE 30/6/16 3. Currently only does Actor, Director, Producer, Writer - does anyone actually use any of the others present in the DTD? xmltv-1.4.0/grab/uk_tvguide/tv_grab_uk_tvguide.map.conf000066400000000000000000000023301500074233200232360ustar00rootroot00000000000000# # Sample ~.map.conf file # # This file has two purposes: # 1) Map the channel ids used by the website into something more meaningful to your PVR, e.g. # map==74==BBC 1 # will change '74' to 'BBC 1' in the output XML. # # These lines are of the form " map=={channel id}=={my name} ". # # Note: to pass tv_validate the "my name" portion must be a RFC2838 format name, e.g. # map==74==bbc1.tvguide.co.uk # # 2) The second purpose is to likewise translate genre names. So if your PVR doesn't have # a category for 'Science Fiction' but uses 'Sci-fi' instead, then you can specify # cat==Science Fiction==Sci-fi # and the output XML will have 'Sci-fi'. # # These lines are of the form " cat=={incoming genre}=={translated genre} ".. # # # Some example lines follow - delete these and replace with your own requirements. # # # Translate the programme genres # cat==Environment==Nature cat==General Movie==Film cat==Advertisement==Shopping cat==Science Fiction==Sci-fi # # Convert incoming channel ids to names which are acceptable to tv_validate_grabber # (but may be rejected by your pvr!) # map==74==bbc1.tvguide.co.uk map==89==bbc2.tvguide.co.uk map==172==itv1.tvguide.co.uk map==145==film4.tvguide.co.uk xmltv-1.4.0/grab/zz_sdjson/000077500000000000000000000000001500074233200156205ustar00rootroot00000000000000xmltv-1.4.0/grab/zz_sdjson/tv_grab_zz_sdjson000066400000000000000000001110471500074233200212760ustar00rootroot00000000000000#!/usr/bin/perl -w =head1 NAME tv_grab_zz_sdjson - Grab TV listings from Schedules Direct SD-JSON service. =head1 SYNOPSIS tv_grab_zz_sdjson --help tv_grab_zz_sdjson --info tv_grab_zz_sdjson --version tv_grab_zz_sdjson --capabilities tv_grab_zz_sdjson --description tv_grab_zz_sdjson [--config-file FILE] [--days N] [--offset N] [--output FILE] [--quiet] [--debug] tv_grab_zz_sdjson --configure [--config-file FILE] =head1 DESCRIPTION This is an XMLTV grabber for the Schedules Direct (http://www.schedulesdirect.org) JSON API. =head1 CONFIGURATION Run tv_grab_zz_sdjson with the --configure option to create a config file. MythTV does not use the default XMLTV config file path. If using MythTV you should also specify the config file such as: tv_grab_zz_sdjson --configure --config-file ~/.mythtv/source_name.xmltv Doing the XMLTV config from within the MythTV GUI seems very flaky so you are probably better off configuring from the command line. =head1 AUTHOR Kevin Groeneveld (kgroeneveld at gmail dot com) =cut use strict; use XMLTV; use XMLTV::Options qw(ParseOptions); use XMLTV::Configure::Writer; use XMLTV::Ask; use Cwd; use Storable; use LWP::UserAgent; use JSON; use Digest::SHA qw(sha1_hex); use DateTime; use Scalar::Util qw(looks_like_number); use Try::Tiny; use Data::Dumper; my $grabber_name = 'tv_grab_zz_sdjson'; my $grabber_version = "$XMLTV::VERSION"; # The XMLTV::Writer docs only indicate you need to set 'encoding'. However, # this value does not get passed to the underlying XML::Writer object. Unless # 'ENCODING' is also specified XML::Writer does not actually encode the data! my %w_args = ( 'encoding' => 'utf-8', 'ENCODING' => 'utf-8', 'UNSAFE' => 1, ); my %tv_attributes = ( 'source-info-name' => 'Schedules Direct', 'source-info-url' => 'http://www.schedulesdirect.org', 'generator-info-name' => "$grabber_name $grabber_version", ); my @channel_id_formats = ( [ 'default', 'I%s.json.schedulesdirect.org', 'Default Format' ], [ 'zap2it', 'I%s.labs.zap2it.com', 'tv_grab_na_dd Format' ], [ 'mythtv', '%s', 'MythTV Internal DD Grabber Format' ], ); my @previously_shown_formats = ( [ 'date', '%Y%m%d', 'Date Only' ], [ 'datetime', '%Y%m%d%H%M%S %z', 'Date And Time' ], ); my $cache_schema = 1; my $sd_json_baseurl = 'https://json.schedulesdirect.org'; my $sd_json_api = '/20141201/'; my $sd_json_token; my $sd_json_status; my $sd_json_request_max = 5000; my $ua = LWP::UserAgent->new(agent => "$grabber_name $grabber_version"); $ua->default_header('accept-encoding' => scalar HTTP::Message::decodable()); $ua->requests_redirectable(['GET', 'HEAD', 'POST', 'PUT', 'DELETE']); my $debug; my $quiet; # In general we rely on ParseOptions to parse the command line options. However # ParseOptions does not pass the options to stage_sub so we check for some # options on our own. for my $opt (@ARGV) { $debug = 1 if($opt =~ /--debug/i); $quiet = 1 if($opt =~ /--quiet/i); } $quiet = 0 if $debug; $ua->show_progress(1) unless $quiet; my ($opt, $conf) = ParseOptions({ grabber_name => $grabber_name, version => $grabber_version, description => 'Schedules Direct JSON API', capabilities => [qw/baseline manualconfig preferredmethod/], stage_sub => \&config_stage, listchannels_sub => \&list_channels, preferredmethod => 'allatonce', defaults => { days => -1 }, }); sub get_conf_format { my ($config, $options, $text) = @_; my $result; if($conf->{$config}->[0]) { for my $format (@{$options}) { if($format->[0] eq $conf->{$config}->[0]) { $result = $format->[1]; last; } } } if(!$result) { print STDERR "Valid $text not specified in config, using default.\n" unless $quiet; $result = $options->[0]->[1]; } return $result; } my $channel_id_format = get_conf_format('channel-id-format', \@channel_id_formats, 'channel ID format'); my $previously_shown_format = get_conf_format('previously-shown-format', \@previously_shown_formats, 'previously shown format'); # default days to largish value if($opt->{'days'} < 0) { $opt->{'days'} = 100; } sub get_start_stop_time { # calculate start and stop time from offset and days options my $dt_start = DateTime->today(time_zone => 'local'); $dt_start->add(days => $opt->{'offset'}); my $dt_stop = $dt_start->clone(); $dt_stop->add(days => $opt->{'days'}); # source data has times in UTC $dt_start->set_time_zone('UTC'); $dt_stop->set_time_zone('UTC'); # convert DateTime to seconds from epoch which will allow for a LOT faster # comparisons than comparing DateTime objects return ($dt_start->epoch(), $dt_stop->epoch()); } my ($time_start, $time_stop) = get_start_stop_time(); my $cache_file = $conf->{'cache'}->[0]; sub get_default_cache_file { my $winhome; if(defined $ENV{HOMEDRIVE} && defined $ENV{HOMEPATH}) { $winhome = $ENV{HOMEDRIVE} . $ENV{HOMEPATH}; } my $home = $ENV{HOME} || $winhome || getcwd(); return "$home/.xmltv/$grabber_name.cache"; } # days to add to day of month to get days since Jan 1st my @days_norm = ( -1, 30, 58, 89, 119, 150, 180, 211, 242, 272, 303, 333 ); my @days_leap = ( -1, 30, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334 ); sub is_leap_year { return (!($_[0] % 4) && (($_[0] % 100) || !($_[0] % 400))); } sub parse_airtime { use integer; my ($year, $month, $day, $hour, $min, $sec) = ($_[0] =~ /(\d+)-(\d+)-(\d+)T(\d+):(\d+):(\d+)Z/); # determine number of days since Jan 1st of requested year $month -= 1; $day += is_leap_year($year) ? $days_leap[$month] : $days_norm[$month]; # add number of days (minus leap days) for years since 1970 $day += ($year - 1970) * 365; # add leap days from previous years since year 0 (we already included leap # day for this year), subtract number of leap days between 0 and 1970 (477) $year -= 1; $day += $year / 4 - $year / 100 + $year / 400 - 477; return ($day * 86400 + $hour * 3600 + $min * 60 + $sec); } sub format_airtime { my ($sec, $min, $hour, $day, $month, $year) = gmtime($_[0]); return sprintf('%04d%02d%02d%02d%02d%02d +0000', $year + 1900, $month + 1, $day, $hour, $min, $sec); } my $dt_zone_local = DateTime::TimeZone->new(name => 'local'); # SD-JSON only specifies a date for originalAirDate. Older versions of # mythtv need full date and time even though xmltv only requires date. # We assume local time as mythtv expects and set the time to noon to # minimize the chance of an error causing the day to be off by one. sub parse_original_airdate { my ($year, $month, $day) = ($_[0] =~ /(\d+)-(\d+)-(\d+)/); no warnings 'once'; local $Params::Validate::NO_VALIDATION = 1; return DateTime->new( year => $year, month => $month, day => $day, hour => 12, time_zone => $dt_zone_local, ); } sub retry { my ($action) = @_; my $retry = 3; my $result; for(;;) { try { $result = $action->(); } catch { if(--$retry) { print STDERR $_, "Retry in 10 seconds...\n" unless $quiet; sleep 10; } else { die $_, "Retry count exceeded."; } }; return $result if $result; } } sub sd_json_request { my ($method, $path, $content) = @_; my $url; if($path =~ /^\//) { $url = $sd_json_baseurl . $path; } else { $url = $sd_json_baseurl . $sd_json_api . $path; } my @params; push(@params, content_type => 'application/json'); push(@params, token => $sd_json_token) unless $path eq 'token'; push(@params, content => encode_json($content)) if defined $content; my $response = $ua->$method($url, @params); if($response->is_success()) { return decode_json($response->decoded_content()); } else { my $msg = $response->decoded_content(); if($response->header('content-type') =~ m{application/json}i) { my $error = decode_json($msg); # for lineups request don't consider 4102/NO_LINEUPS an error if($path eq 'lineups' && $error->{'code'} == 4102) { return undef; } $msg = "Server (ID=$error->{'serverID'} Time=$error->{'datetime'}) returned an error:\n" ."$error->{'message'} ($error->{'code'}/$error->{'response'})"; } print STDERR Dumper($response) if $debug; die $msg, "\n"; } } sub sd_json_get_token { my ($username, $password) = @_; retry sub { my $response = sd_json_request('post', 'token', { username => $username, password => $password }); if(ref $response ne 'HASH' || !exists $response->{'token'}) { die "Invalid token response.\n"; } return $response->{'token'}; }; } sub sd_json_get_status { retry sub { my $status = sd_json_request('get', 'status'); if(ref $status ne 'HASH' || ref $status->{'systemStatus'} ne 'ARRAY' || ref $status->{'systemStatus'}->[0] ne 'HASH' || ref $status->{'account'} ne 'HASH' || ref $status->{'lineups'} ne 'ARRAY') { die "Invalid status response.\n" } return $status; } } sub sd_json_get_available { my ($type) = @_; my $result = sd_json_request('get', 'available'); if($type) { for my $entry (@{$result}) { if($entry->{'type'} eq $type) { return $entry; } } } return $result; } sub sd_json_get_lineups { return sd_json_request('get', 'lineups'); } sub sd_json_get_headends { my ($country, $postalcode) = @_; return sd_json_request('get', "headends?country=$country&postalcode=$postalcode"); } sub sd_json_get_transmitters { my ($country) = @_; return sd_json_request('get', "transmitters/$country"); } sub sd_json_add_lineup { my ($lineup) = @_; return sd_json_request('put', "lineups/$lineup"); } sub sd_json_delete_lineup { my ($lineup) = @_; return sd_json_request('delete', "lineups/$lineup"); } sub sd_json_get_lineup { my ($lineup) = @_; retry sub { my $lineup = sd_json_request('get', $lineup); if(ref $lineup ne 'HASH') { die "Invalid lineup response.\n" } return $lineup; } } sub sd_json_get_schedules_md5 { my ($channels) = @_; my @stations; for my $channel (@{$channels}) { push(@stations, { stationID => $channel }); } return sd_json_request('post', 'schedules/md5', \@stations); } sub sd_json_get_schedules { my ($schedules) = @_; return sd_json_request('post', 'schedules', $schedules); } sub sd_json_get_programs { my ($programs) = @_; return sd_json_request('post', 'programs', $programs); } sub sd_json_init { my ($conf) = @_; if(!defined $sd_json_status) { $sd_json_token = sd_json_get_token($conf->{'username'}->[0], sha1_hex($conf->{'password'}->[0])); $sd_json_status = sd_json_get_status(); my $status = $sd_json_status->{'systemStatus'}->[0]->{'status'}; if($status !~ /online/i) { die "Schedules Direct system status: $status\n"; } } } sub sd_json_get_image_url { my ($url) = @_; if($url =~ /^http/) { return $url; } else { return $sd_json_baseurl . $sd_json_api . 'image/' . $url; } } sub get_lineup_description { my ($lineup) = @_; my $location = $lineup->{'location'} // 'unknown'; my $transport = $lineup->{'transport'} // 'unknown'; my $name = $lineup->{'name'} // 'unknown'; my $id = $lineup->{'lineup'} // 'unknown'; if($lineup->{'isDeleted'}) { return "$id | $name"; } elsif($transport eq 'QAM') { return "$id | $transport"; } else { return "$id | $name | $location | $transport"; } } my %transmitter_countries; sub ask_search_by_transmitter { my ($country) = @_; if(!%transmitter_countries) { my $available = sd_json_get_available('DVB-T'); for ($available->{'description'} =~ /[A-Z]{3}/g) { $transmitter_countries{$_} = undef; } } if(exists $transmitter_countries{$country}) { my @options; push(@options, 'transmitter'); push(@options, 'postal' ); if(ask_choice('Search by Transmitter or Postal Code:', $options[0], @options) eq $options[0]) { return 1; } } return 0; } sub config_stage { my ($stage, $conf) = @_; if($stage ne 'start' && $stage ne 'login') { sd_json_init($conf); } my $result; my $w = new XMLTV::Configure::Writer(OUTPUT => \$result, %w_args); $w->start(\%tv_attributes); if($stage eq 'start') { $w->write_string({ id => 'cache', description => [ [ 'Cache file for lineups, schedules and programs.', 'en' ] ], title => [ [ 'Cache file', 'en' ] ], default => get_default_cache_file(), }); $w->start_selectone({ id => 'channel-id-format', description => [ [ 'If you are migrating from a different grabber selecting an alternate channel ID format can make the migration easier.', 'en' ] ], title => [ [ 'Select channel ID format', 'en' ] ], }); for my $format (@channel_id_formats) { $w->write_option({ value => $format->[0], text => [ [ $format->[2].' (eg: '.sprintf($format->[1], 12345).')', 'en' ] ], }); } $w->end_selectone(); $w->start_selectone({ id => 'previously-shown-format', description => [ [ 'As the JSON data only includes the previously shown date normally the XML output should only have the date. However some programs such as older versions of MythTV also need a time.', 'en' ] ], title => [ [ 'Select previously shown format', 'en' ] ], }); for my $format (@previously_shown_formats) { $w->write_option({ value => $format->[0], text => [ [ $format->[2], 'en' ] ], }); } $w->end_selectone(); $w->end('login'); } elsif($stage eq 'login') { $w->write_string({ id => 'username', description => [ [ 'Schedules Direct username.', 'en' ] ], title => [ [ 'Username', 'en' ] ], }); $w->write_secretstring({ id => 'password', description => [ [ 'Schedules Direct password.', 'en' ] ], title => [ [ 'Password', 'en' ] ], }); $w->end('account-lineups'); } elsif($stage eq 'account-lineups') { # This stage doesn't work with configapi and I am not sure if there is # currently any good way to make it work... my $edit; do { my $max = $sd_json_status->{'account'}->{'maxLineups'}; my $lineups = sd_json_get_lineups(); $lineups = $lineups->{'lineups'}; my $count = 0; say("This step configures the lineups enabled for your Schedules " ."Direct account. It impacts all other configurations and " ."programs using the JSON API with your account. A maximum of " ."$max lineups can by added to your account. In a later step " ."you will choose which lineups or channels to actually use " ."for this configuration.\n" ."Current lineups enabled for your Schedules Direct account:" ); say('#. Lineup ID | Name | Location | Transport'); for my $lineup (@{$lineups}) { $count++; my $desc = get_lineup_description($lineup); say("$count. $desc"); } if(!$count) { say('(none)'); } my @options; push(@options, 'continue') if $count; push(@options, 'add' ) if($count < $max); push(@options, 'delete') if $count; $edit = ask_choice('Edit account lineups:', $options[0], @options); try { if($edit eq 'add') { my $country = uc(ask('Lineup ID or Country (ISO-3166-1 alpha 3 such as USA or CAN):')); if(length($country) > 3) { sd_json_add_lineup("$country"); } else { my $count = 0; my @lineups; if(ask_search_by_transmitter($country)) { my $transmitters = sd_json_get_transmitters($country); say('#. Lineup ID | Transmitter'); for my $transmitter (sort(keys %{$transmitters})) { $count++; my $lineup = $transmitters->{$transmitter}; push(@lineups, $lineup); say("$count. $lineup | $transmitter"); } } else { my $postalcode = ask(($country eq 'USA') ? 'Zip Code:' : 'Postal Code:'); my $headends = sd_json_get_headends($country, $postalcode); say('#. Lineup ID | Name | Location | Transport'); for my $headend (@{$headends}) { for my $lineup (@{$headend->{'lineups'}}) { $count++; my $id = $lineup->{'lineup'}; push(@lineups, $id); say("$count. $id | $lineup->{'name'} | $headend->{'location'} | $headend->{'transport'}"); } } } my $add = ask_choice('Add lineup (0 = none):', 0, (0 .. $count)); if($add) { sd_json_add_lineup($lineups[$add - 1]); } } } elsif($edit eq 'delete') { my $delete = ask_choice('Delete lineup (0 = none):', 0, (0 .. $count)); if($delete) { sd_json_delete_lineup($lineups->[$delete - 1]->{'lineup'}); } } } catch { say($_); }; } while($edit ne 'continue'); $w->end('select-mode'); } elsif($stage eq 'select-mode') { $w->start_selectone({ id => 'mode', description => [ [ 'Choose whether you want to include complete lineups or individual channels for this configuration.', 'en' ] ], title => [ [ 'Select mode', 'en' ] ], }); $w->write_option({ value => 'lineup', text => [ [ 'lineups', 'en' ] ], }); $w->write_option({ value => 'channels', text => [ [ 'channels', 'en' ] ], }); $w->end_selectone(); $w->end('select-lineups'); } elsif($stage eq 'select-lineups') { my $lineups = sd_json_get_lineups(); $lineups = $lineups->{'lineups'}; my $desc; if($conf->{'mode'}->[0] eq 'lineup') { $desc = 'Choose lineups to use for this configuration.'; } else { $desc = 'Choose lineups from which you want to select channels for this configuration.'; } $w->start_selectmany({ id => $conf->{'mode'}->[0], description => [ [ $desc, 'en' ] ], title => [ [ 'Select linups', 'en' ] ], }); for my $lineup (@{$lineups}) { my $id = $lineup->{'lineup'}; $w->write_option({ value => $id, text => [ [ $id, 'en' ] ], }); } $w->end_selectmany(); $w->end('select-channels'); } else { die "Unknown stage $stage"; } return $result; } my $cache; my $cache_lineups; my $cache_schedules; my $cache_programs; my %channel_index; my %channel_map; sub cache_load { sub get_hash { my $hash = $cache->{$_[0]}; return (ref $hash eq 'HASH') ? $hash : {}; } # make sure the cache file is readable and writable if(open(my $fh, '+>>', $cache_file)) { close($fh); } else { die "Cannot open $cache_file for read/write.\n"; } # attempt to retreive cached data try { $cache = retrieve($cache_file); if(ref $cache ne 'HASH') { die "Invalid cache file.\n"; } if($cache->{'schema'} == $cache_schema) { $cache_lineups = get_hash('lineups'); $cache_schedules = get_hash('schedules'); $cache_programs = get_hash('programs'); } else { die "Ignoring cache file with old schema.\n"; } } catch { print STDERR unless $quiet; $cache_lineups = {}; $cache_schedules = {}; $cache_programs = {}; }; $cache = { schema => $cache_schema, lineups => $cache_lineups, schedules => $cache_schedules, programs => $cache_programs }; } sub cache_update_lineups { print STDERR "Updating lineups...\n" unless $quiet; my $now = DateTime->now()->epoch(); my %lineups_enabled; my @lineups_update; # check for out of date lineups for my $lineup (@{$sd_json_status->{'lineups'}}) { if(ref $lineup ne 'HASH') { print STDERR "Invalid lineup in account status.\n" unless $quiet; next; } my $id = $lineup->{'lineup'}; if(!$id || ref $id) { print STDERR "Invalid lineup in account status.\n" unless $quiet; next; } $lineups_enabled{$id} = 1; my $metadata = $cache_lineups->{$id}->{'metadata'}; if(ref $metadata ne 'HASH') { print STDERR "lineup $id: new\n" if $debug; push(@lineups_update, $lineup); } elsif($metadata->{'modified'} ne $lineup->{'modified'}) { print STDERR "lineup $id: old\n" if $debug; push(@lineups_update, $lineup); } else { print STDERR "lineup $id: current\n" if $debug; $cache_lineups->{$id}->{'accessed'} = $now; } } # check that configured lineups are actually enabled for the account my $lineup_error; for my $lineup (@{$conf->{'lineup'}}, @{$conf->{'channels'}}) { if(!$lineups_enabled{$lineup}) { $lineup_error = 1; print STDERR "Lineup $lineup in the current configuration is not enabled on your account.\n"; } } if($lineup_error) { die "Please reconfigure the grabber or your account settings.\n" } # update lineups for my $lineup (@lineups_update) { my $id = $lineup->{'lineup'}; my $uri = $lineup->{'uri'}; if(!$uri || ref $uri) { print STDERR "Invalid lineup URI in account status.\n" unless $quiet; next; } my $update = sd_json_get_lineup($uri); $cache_lineups->{$id} = $update; $cache_lineups->{$id}->{'accessed'} = $now; } } sub cache_update_schedules { my ($channels) = @_; print STDERR "Updating schedules...\n" unless $quiet; my $now = DateTime->now()->epoch(); my $schedules_md5 = sd_json_get_schedules_md5($channels); my @channels_update; while(my ($channel, $schedule) = each %{$schedules_md5}) { if(ref $schedule ne 'HASH') { print STDERR "Invalid schedule for channel $channel\n" unless $quiet; next; } my @dates; while(my ($date, $latest) = each %{$schedule}) { my $metadata = $cache_schedules->{$channel}->{$date}->{'metadata'}; if(!defined $metadata) { print STDERR "channel $channel $date: new\n" if $debug; push(@dates, $date); } elsif($metadata->{'md5'} ne $latest->{'md5'}) { print STDERR "channel $channel $date: old\n" if $debug; push(@dates, $date); } else { print STDERR "channel $channel $date: current\n" if $debug; } } if(@dates) { push(@channels_update, { stationID => $channel, date => \@dates }); } } # update schedules while(my @block = splice(@channels_update, 0, $sd_json_request_max)) { my $schedules = sd_json_get_schedules(\@block); for my $schedule (@{$schedules}) { my $channel = $schedule->{'stationID'}; my $date = $schedule->{'metadata'}->{'startDate'}; $cache_schedules->{$channel}->{$date} = $schedule; } } print STDERR "Updating programs...\n" unless $quiet; my %programs_update_hash; # create list of programs to update for my $channel (@{$channels}) { for my $schedule (values %{$cache_schedules->{$channel}}) { for my $program (@{$schedule->{'programs'}}) { my $airtime = parse_airtime($program->{'airDateTime'}); my $dur = int($program->{'duration'}); if(($airtime + $dur) > $time_start && $airtime < $time_stop) { my $id = $program->{'programID'}; my $cached = $cache_programs->{$id}; if(!defined $cached) { print STDERR "program $id: new\n" if $debug; $programs_update_hash{$id} = 1; } elsif($cached->{'md5'} ne $program->{'md5'}) { print STDERR "program $id: old\n" if $debug; $programs_update_hash{$id} = 1; } else { print STDERR "program $id: current\n" if $debug; $cache_programs->{$id}->{'accessed'} = $now; } } } } } # update programs my @programs_update = keys %programs_update_hash; while(my @block = splice(@programs_update, 0, $sd_json_request_max)) { my $programs = sd_json_get_programs(\@block); for my $id (@block) { $cache_programs->{$id} = shift @{$programs}; $cache_programs->{$id}->{'accessed'} = $now; } } } sub cache_drop_old { my $limit = DateTime->now()->subtract(days => 10)->epoch(); print STDERR "Removing old cache entries...\n" unless $quiet; while(my ($key, $hash) = each %{$cache}) { if($key eq 'lineups' || $key eq 'programs') { # remove old lineups and programs while(my ($key, $value) = each %{$hash}) { if(ref $value ne 'HASH' || !exists $value->{'accessed'} || $value->{'accessed'} < $limit) { print STDERR "$key: drop\n" if $debug; delete $hash->{$key}; } } } elsif($key eq 'schedules') { # remove old schedules my $today = DateTime->today()->strftime('%Y-%m-%d'); while(my ($channel, $schedules) = each %{$hash}) { if(ref $schedules ne 'HASH') { print STDERR "$channel: drop\n" if $debug; delete $cache_schedules->{$channel}; next; } while(my ($date, $schedule) = each %{$schedules}) { if($date lt $today) { print STDERR "$channel $date: drop\n" if $debug; delete $schedules->{$date}; } } if(scalar keys %{$schedules} == 0) { print STDERR "$channel: drop\n" if $debug; delete $cache_schedules->{$channel}; } } } elsif($key ne 'schema') { # remove unknown keys delete $cache->{$key}; } } } sub cache_save { store($cache, $cache_file); } sub cache_index_channels { print STDERR "Indexing channels...\n" unless $quiet; # create index for my $id (@{$conf->{'lineup'}}, @{$conf->{'channels'}}) { my $lineup = $cache_lineups->{$id}; if(ref $lineup ne 'HASH' || ref $lineup->{'stations'} ne 'ARRAY') { print STDERR "Invalid stations array for lineup $id\n" unless $quiet; next; } for my $channel (@{$lineup->{'stations'}}) { if(ref $channel ne 'HASH') { print STDERR "Invalid channel in lineup $id\n" unless $quiet; next; } $channel_index{$channel->{'stationID'}} = $channel; } my $qam = $lineup->{'qamMappings'}; my $map; if($qam) { $map = $lineup->{'map'}->{$qam->[0]}; } else { $map = $lineup->{'map'}; } for my $channel (@{$map}) { $channel_map{$channel->{'stationID'}} = $channel; } } } sub get_channel_list { my ($conf) = @_; my %hash; if($conf->{'mode'}->[0] eq 'lineup') { for my $lineup (@{$conf->{'lineup'}}) { if(ref $cache_lineups->{$lineup}->{'stations'} ne 'ARRAY') { print STDERR "Invalid stations array for lineup $lineup\n" unless $quiet; next; } for my $channel (@{$cache_lineups->{$lineup}->{'stations'}}) { if(ref $channel ne 'HASH' || !$channel->{'stationID'}) { print STDERR "Invalid channel in lineup $lineup\n" unless $quiet; next; } $hash{$channel->{'stationID'}} = 1; } } } else { for my $channel (@{$conf->{'channel'}}) { if(exists $channel_index{$channel}) { $hash{$channel} = 1; } else { print STDERR "Channel ID $channel in the current configuration is not found in any enabled lineup.\n" unless $quiet; } } } my @list = sort(keys %hash); return \@list; } sub get_channel_number { my ($map) = @_; if($map->{'virtualChannel'}) { return $map->{'virtualChannel'}; } elsif($map->{'atscMajor'}) { return "$map->{'atscMajor'}_$map->{'atscMinor'}"; } elsif($map->{'channel'}) { return $map->{'channel'}; } elsif($map->{'frequencyHz'}) { return $map->{'frequencyHz'}; } return undef; } sub get_icon { my ($url, $width, $height) = @_; my %result; if($url) { $result{'src'} = sd_json_get_image_url($url); if($width && $height) { $result{'width'} = $width; $result{'height'} = $height; } return [ \%result ]; } else { return undef; } } sub write_channel { my ($w, $channel, $map) = @_; my %ch; # mythtv seems to assume that the first three display-name elements are # name, callsign and channel number. We follow that scheme here. $ch{'id'} = sprintf($channel_id_format, $channel->{'stationID'}); $ch{'display-name'} = [ [ $channel->{'name'} || 'unknown name' ], [ $channel->{'callsign'} || 'unknown callsign' ], [ get_channel_number($map) || 'unknown number' ] ]; my $logo = $channel->{'logo'}; my $icon = get_icon($logo->{'URL'}, $logo->{'width'}, $logo->{'height'}); $ch{'icon'} = $icon if $icon; $w->write_channel(\%ch); } # this is used by the last stage of --configure sub list_channels { my ($conf, $opt) = @_; # use raw channel id in configuration files $channel_id_format = '%s'; my $result; my $w = new XMLTV::Writer(OUTPUT => \$result, %w_args); $w->start(\%tv_attributes); for my $id (@{$conf->{'channels'}}) { my $lineup = sd_json_get_lineup("lineups/$id"); for my $channel (@{$lineup->{'stations'}}) { write_channel($w, $channel); } } $w->end(); return $result; } sub get_program_title { my ($details) = @_; my $title = $details->{'titles'}->[0]->{'title120'}; if($title) { return [ [ $title ] ]; } else { return [ [ 'unknown' ] ]; } } sub get_program_subtitle { my ($details) = @_; my $subtitle = $details->{'episodeTitle150'}; if($subtitle) { return [ [ $subtitle ] ]; } else { return undef; } } sub get_program_description { my ($details) = @_; my $descriptions = $details->{'descriptions'}; if(exists $descriptions->{'description1000'}) { return [ [ $descriptions->{'description1000'}->[0]->{'description'} ] ]; } elsif(exists $descriptions->{'description100'}) { return [ [ $descriptions->{'description100'}->[0]->{'description'} ] ]; } else { return undef; } } sub get_program_credits { my ($details) = @_; my %credits; for my $credit (@{$details->{'cast'}}, @{$details->{'crew'}}) { my $role = $credit->{'role'}; my $name = $credit->{'name'}; my $key; if($role =~ /director/i) { $key = 'director'; } elsif($role =~ /(actor|voice)/i) { $key = 'actor'; if($credit->{'characterName'}) { $name = [ $name, $credit->{'characterName'} ]; } } elsif($role =~ /writer/i) { $key = 'writer'; } elsif($role =~ /producer/i) { $key = 'producer'; } elsif($role =~ /(host|anchor)/i) { $key = 'presenter'; } elsif($role =~ /(guest|contestant)/i) { $key = 'guest'; } else { # print STDERR "$role\n"; } if($key) { if(exists $credits{$key}) { push(@{$credits{$key}}, $name); } else { $credits{$key} = [ $name ]; } } } if(scalar keys %credits) { return \%credits; } else { return undef; } } sub get_program_date { my ($details) = @_; my $year = $details->{'movie'}->{'year'}; if($year) { return $year; } return undef; } sub get_program_category { my ($channel, $details) = @_; my %seen; my @result; sub add { my ($result, $category, $seen) = @_; if($category && !exists $seen->{$category}) { $seen->{$category} = 1; push(@{$result}, [ $category ]); } } for my $genre (@{$details->{'genres'}}) { add(\@result, $genre, \%seen); } add(\@result, $details->{'showType'}, \%seen); # mythtv specifically looks for movie|series|sports|tvshow my $entity_type = $details->{'entityType'}; if($entity_type =~ /movie/i) { add(\@result, 'movie', \%seen); } elsif($entity_type =~ /episode/i) { add(\@result, 'series', \%seen); } elsif($entity_type =~ /sports/i) { add(\@result, 'sports', \%seen); } elsif($channel->{'isRadioStation'}) { add(\@result, 'radio', \%seen); } else { add(\@result, 'tvshow', \%seen); } if(scalar @result) { return \@result; } else { return undef; } } sub get_program_length { my ($details) = @_; my $duration = $details->{'duration'} || $details->{'movie'}->{'duration'}; if($duration) { return $duration; } else { return undef; } } sub get_program_icon { my ($details) = @_; my $episode_image = $details->{'episodeImage'}; return get_icon($episode_image->{'uri'}, $episode_image->{'width'}, $episode_image->{'height'}); } sub get_program_url { my ($details) = @_; my $url = $details->{'officialURL'}; if($url) { return [ $url ]; } return undef; } sub _get_program_episode { my ($number, $total) = @_; my $result = ''; if(looks_like_number($number) && int($number)) { $result = sprintf('%d', $number - 1); if(looks_like_number($total) && int($total)) { $result .= sprintf('/%d', $total); } } return $result; } sub get_program_episode { my ($program, $details) = @_; my $season = ''; my $episode = ''; my $part = ''; my @result; metadata: for my $metadata (@{$details->{'metadata'}}) { keys %{$metadata}; while(my ($key, $value) = each %{$metadata}) { # prefer Gracenote metadata but use first available as fallback my $is_gracenote = $key eq 'Gracenote'; if ($is_gracenote || !(length($season) || length($episode))) { $season = _get_program_episode($value->{'season'}, $value->{'totalSeason'}); $episode = _get_program_episode($value->{'episode'}, $value->{'totalEpisodes'}); } last metadata if $is_gracenote; } } my $multipart = $program->{'multipart'}; if($multipart) { $part = _get_program_episode($multipart->{'partNumber'}, $multipart->{'totalParts'}); } if(length($season) || length($episode) || length($part)) { push(@result, [ sprintf('%s.%s.%s', $season, $episode, $part), 'xmltv_ns' ]); } push(@result, [ $program->{'programID'}, 'dd_progid' ]); return \@result; } sub get_program_video { my ($program) = @_; my %video; for my $item (@{$program->{'videoProperties'}}) { if($item =~ /hdtv/i) { $video{'quality'} = 'HDTV'; } } if(scalar keys %video) { return \%video; } else { return undef; } } sub get_program_audio { my ($program) = @_; my %audio; for my $item (@{$program->{'audioProperties'}}) { if($item =~ /mono/i) { $audio{'stereo'} = 'mono'; } elsif($item =~ /stereo/i) { $audio{'stereo'} = 'stereo'; } elsif($item =~ /DD/i) { $audio{'stereo'} = 'dolby digital'; } } if(scalar keys %audio) { return \%audio; } return undef; } # The xmltv docs state this field is "When and where the programme was last shown". # Programs that are marked as new by Schedules Direct can not have a XMLTV previously_shown. sub get_program_previously_shown { my ($program, $details) = @_; my %previously_shown; return undef if(get_program_new($program)); my $date = $details->{'originalAirDate'}; if($date) { my $dt = parse_original_airdate($date); $previously_shown{'start'} = $dt->strftime($previously_shown_format); } if(scalar keys %previously_shown) { return \%previously_shown; } return undef; } sub get_program_premiere { my ($program) = @_; my $premiere = $program->{'isPremiereOrFinale'}; if(defined $premiere && $premiere =~ /premiere/i) { return [ $premiere ]; } return undef; } sub get_program_new { my ($program) = @_; my $new = $program->{'new'}; if(defined $new) { return 1; } return undef; } sub get_program_subtitles { my ($program) = @_; if(grep('^cc$', @{$program->{'audioProperties'}})) { return [ { 'type' => 'teletext' } ]; } return undef; } sub get_program_rating { my ($program, $details) = @_; # first check 'contentRating' then 'ratings' my $ratings = $details->{'contentRating'}; if(!defined $ratings || ref $ratings ne 'ARRAY') { $ratings = $program->{'ratings'}; if(!defined $ratings || ref $ratings ne 'ARRAY') { return undef; } } my @result; for my $rating (@{$ratings}) { my $code = $rating->{'code'}; my $body = $rating->{'body'}; if($code) { push(@result, [ $code, $body ]); } } if(scalar @result) { return \@result; } return undef; } sub get_program_star_rating { my ($details) = @_; my $rating = $details->{'movie'}->{'qualityRating'}->[0]; if($rating) { return [ [ "$rating->{'rating'}/$rating->{'maxRating'}", $rating->{'ratingsBody'} ] ]; } else { return undef; } } sub write_programme { my ($w, $channel, $program, $details) = @_; my $airtime = parse_airtime($program->{'airDateTime'}); my $dur = int($program->{'duration'}); if(($airtime + $dur) > $time_start && $airtime < $time_stop) { my $start = format_airtime($airtime); my $stop = format_airtime($airtime + $dur); $w->write_programme({ 'channel' => sprintf($channel_id_format, $channel->{'stationID'}), 'start' => $start, 'stop' => $stop, 'title' => get_program_title($details), 'sub-title' => get_program_subtitle($details), 'desc' => get_program_description($details), 'credits' => get_program_credits($details), 'date' => get_program_date($details), 'category' => get_program_category($channel, $details), # 'keyword' => undef, # 'language' => undef, # 'orig-language' => undef, 'length' => get_program_length($details), 'icon' => get_program_icon($details), 'url' => get_program_url($details), # 'country' => undef, 'episode-num' => get_program_episode($program, $details), 'video' => get_program_video($program), 'audio' => get_program_audio($program), 'previously-shown' => get_program_previously_shown($program, $details), 'premiere' => get_program_premiere($program), # 'last-chance' => undef, # 'new' => undef, 'subtitles' => get_program_subtitles($program), 'rating' => get_program_rating($program, $details), 'star-rating' => get_program_star_rating($details), # 'review' => undef, }); } } sub grab_listings { my ($conf) = @_; my $channels; print STDERR "Initializing...\n" unless $quiet; cache_load(); sd_json_init($conf); cache_update_lineups(); cache_index_channels(); $channels = get_channel_list($conf); if(!@{$channels}) { die "No lineups or channels configured.\n"; } cache_update_schedules($channels); cache_drop_old(); cache_save(); print STDERR "Writing output...\n" unless $quiet; my $w = new XMLTV::Writer(%w_args); $w->start(\%tv_attributes); # write channels for my $channel (@{$channels}) { write_channel($w, $channel_index{$channel}, $channel_map{$channel}); } # write programs for my $channel (@{$channels}) { my $schedules = $cache_schedules->{$channel}; for my $day (sort(keys %{$schedules})) { for my $program (@{$schedules->{$day}->{'programs'}}) { write_programme($w, $channel_index{$channel}, $program, $cache_programs->{$program->{'programID'}}); } } } $w->end(); print STDERR "Done\n" unless $quiet; } grab_listings($conf); xmltv-1.4.0/grab/zz_sdjson_sqlite/000077500000000000000000000000001500074233200172015ustar00rootroot00000000000000xmltv-1.4.0/grab/zz_sdjson_sqlite/fixups.txt000066400000000000000000000052031500074233200212600ustar00rootroot00000000000000 FIXUPS Some applications are known to not be compliant with the XMLTV specifications, or have other peculiarities in the way they interact with the grabber output. Users of those applications should request that the developers correct their implementation or add the required additional functionality they desire and then migrate to those fixed versions, but there are cases where that cannot be accomplished quickly. To address this issue, this grabber will recognize a request for fixups in an environment variable. Fixups are intended to be a temporary measure. It is imperative that users work with the developers of their application to release an updated version, and that one updates to that release. While it is a goal that a fixup be supported for at least a year after the application has been identified as being non-complaint with the XMLTV definition or that the application does not provide the desired functionality, if code changes or refactor in this grabber impact the ability to support a fixup, the fixup may be removed sooner. NOTE: Requests to add fixups should include a patch or pull request. The environmental variable is TV_GRAB_TARGET_APPLICATION_FIXUPS and the requested fixups are separated by a colon. Example usage: TV_GRAB_TARGET_APPLICATION_FIXUPS=NO_XMLTV_NS_TOTAL_SEASONS:NO_PREVIOUSLY_SHOWN_ZONE_OFFSET tv_grab_zz_sdjson_sqlite Currently implemented fixups: NO_XMLTV_NS_TOTAL_SEASONS Do not add in the total seasons value to season value in the xmltv_ns episode numbering. Known apps: MythTV master before 585f509 (fixes/0.28 before e26a33c) NO_PREVIOUSLY_SHOWN_ZONE_OFFSET Do not add in the zulu offset for previously shown. Known apps: MythTV versions before ff5ab27 (legacy unsupported versions) NO_STATION_LOGOS Do not add the station logos/icons to the generated result. This is (mostly?) useful when an individual has carefully curated a set of logos and the application will replace them with the logos provided by the xmltv provided values without further user interaction. Known apps: MythTV (feature request posted) NO_MULTIPLE_STATION_LOGOS Only return the first station logo/icon. This is mostly useful when the app chooses the last, rather than the first, logo when presented with more than one logo. Known apps: MythTV before 96e307a (legacy unsupported versions) NO_ACTOR_GUEST_ATTRIBUTE Do not emit the guest attribute for actors even for actors identified as guest stars/voices. Known apps: None (but there are likely some apps which need the fixup) xmltv-1.4.0/grab/zz_sdjson_sqlite/tv_grab_zz_sdjson_sqlite000066400000000000000000012457111500074233200242470ustar00rootroot00000000000000#!/usr/bin/perl -w # # tv_grab_zz_sdjson_sqlite # # Copyright (c) 2016, 2017 Gary Buhrmaster # # This code is distributed under the GNU General Public License v2 (GPLv2) # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # version 2 as published by the Free Software Foundation. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. # # # For extended help information run # tv_grab_zz_sdjson_sqlite --info # # # NOTE - Automated XMLTV testing will report failure since Schedules Direct # requires an account for downloading of data. The automated testing # likely needs a way (a new capability?) that indicates that the grabber # cannot be tested. In addition, for many real world lineups, we again # fall into the different interpretations of the terms "station" and # "channel". Unlike how XMLTV uses the term, we consider a "station" # as a programming entity which has a schedule of programs. A "channel" # is a technical means of delivering a particular "station". XMLTV # uses channel when they mean station. For many lineups (for example, # a cable/satellite provider, or OTA repeaters) the exact same "station" # is on multiple "channels", which results in "duplicate" messages from # the automated testing tool which presumes that the channels need not # be duplicated. In an ideal world, that might be true, but as the # channel (in XMLTV terms) is also overloaded with the display name # which is used for automated discovery and updates in PVRs, we report # each "channel" seperately, even when the "station" is the same. # # # Version history: # # 2025/01/16 - 1.138 - add deaf-signed subtitles element # 2024/08/02 - 1.137 - fix error handling in DB_open # 2024/01/27 - 1.136 - extend detail in schedules direct user-agent # 2024/01/27 - 1.135 - move from legacy cvs to modern versioning # 2024/01/25 - 1.134 - parameterize retries # 2024/01/25 - 1.133 - provide for extended video quality values # 2024/01/24 - 1.132 - default 3rdparty metadata in configure to enabled # 2024/01/24 - 1.131 - simplify 3rdparty metadata tests # 2024/01/23 - 1.130 - add tvmaze metadata # 2024/01/23 - 1.129 - provide fallback for program episode value # 2023/09/10 - 1.128 - protect against no database in manage-lineups # 2023/09/04 - 1.127 - various minor message text updates # 2023/09/04 - 1.126 - always download current account lineups # 2023/09/04 - 1.125 - improve retry logic for station schedules hashes # 2023/09/02 - 1.124 - protect against missing or malformed json string # 2023/09/01 - 1.123 - eliminate redundent json utf8 invokations # 2023/09/01 - 1.122 - update cast and crew mappings # 2023/08/31 - 1.121 - protect against missing or malformed json string # 2023/08/18 - 1.120 - adjust retry numbers # 2023/08/14 - 1.119 - add routeto option for Schedules Direct debugging # 2023/08/12 - 1.118 - limit potential messages for retry cases # 2023/08/11 - 1.117 - add in delay for schedule hash retry # 2023/08/11 - 1.116 - support schedule hash download retry # 2023/08/11 - 1.115 - remove last updated datetime support # 2023/08/11 - 1.114 - add in atsc 3.0 system type for get-lineup # 2023/07/19 - 1.113 - rename downloadQueued to downloadRetry # 2023/07/18 - 1.112 - remove legacy commented out code # 2023/07/17 - 1.111 - adjust _resetSession and _resetError usage # 2022/02/28 - 1.110 - improve token revalidation logic # 2022/02/24 - 1.109 - update pod for resturl option # 2022/02/24 - 1.108 - improve rating agency data validation # 2021/05/14 - 1.107 - allow specification of SD REST endpoint # 2021/05/14 - 1.106 - clean up some perl critic warnings # 2021/04/08 - 1.105 - update rating agency mappings # 2021/03/30 - 1.104 - add guest and self attributes to actors # 2021/03/29 - 1.103 - update cast and crew mappings # 2020/09/15 - 1.102 - update for cherry-pick typo correction # 2020/06/21 - 1.101 - rename scaledownload to scale-download # 2020/06/20 - 1.100 - add support for --scaledownload # 2020/06/12 - 1.99 - include programID in metadata # 2020/05/18 - 1.98 - support ordering of station logos # 2020/05/17 - 1.97 - explicitly specify stable sort # 2020/05/09 - 1.96 - improve passwordhash option handling # 2020/05/08 - 1.95 - improve metadata names based on feedback # 2020/05/08 - 1.94 - extend metadata with schedules direct values # 2020/05/08 - 1.93 - refactor obtainStationsSchedules # 2020/05/05 - 1.92 - error checking and handling improvements # 2020/05/05 - 1.91 - increase potential grabber concurrency phase 3 # 2020/04/27 - 1.90 - increase potential grabber concurrency phase 2 # 2020/04/26 - 1.89 - increase potential grabber concurrency phase 1 # 2020/04/23 - 1.88 - reorganize database open/validation # 2020/04/13 - 1.87 - additional validation of returned data # 2020/04/10 - 1.86 - refactor obtainStationsSchedulesHash # 2020/04/07 - 1.85 - partially revert location removal # 2020/04/07 - 1.84 - fix for manage-lineups channel selection # 2020/04/06 - 1.83 - fix for manage-lineups with no database # 2020/04/05 - 1.82 - change lineup to lineupID for obtainLineups # 2020/04/04 - 1.81 - refactor/rename obtainHeadends # 2020/04/02 - 1.80 - remove location from lineup displays # 2020/04/01 - 1.79 - do not validate postal code via regex # 2020/03/30 - 1.78 - robustify token reuse validation # 2020/03/28 - 1.77 - use obtainLineups where appropriate # 2020/03/28 - 1.76 - supplement lineup data with status data # 2020/03/27 - 1.75 - remove legacy (20131021) api name # 2020/03/23 - 1.74 - refactor obtainLineups to return lineup array # 2020/03/22 - 1.73 - allow Schedules Direct endpoint redirects # 2020/03/22 - 1.72 - reuse existing token when possible # 2020/03/18 - 1.71 - handle obtainAvailable undef # 2020/03/18 - 1.70 - minor whitespace cleanup # 2020/03/17 - 1.69 - stable output order # 2020/03/17 - 1.68 - return all icons for channels unless fixup # 2019/11/08 - 1.67 - handle no-download for list-lineups # 2018/12/21 - 1.66 - default 3rdparty metadata in configure to disabled # 2018/12/17 - 1.65 - clean up whitespace and duplicate lines # 2018/12/16 - 1.64 - add support for gracenote rating body advisories # 2018/12/15 - 1.63 - remove (no longer existing) schedule ratings # 2018/12/14 - 1.62 - 3rdparty metadata emission via configure # 2018/12/13 - 1.61 - add tvdb metadata # 2018/11/10 - 1.60 - include subscription info in additional paths # 2018/11/03 - 1.59 - additional protection against bad data # 2018/10/30 - 1.58 - initial protections against bad data # 2018/09/15 - 1.57 - support lineup selection by transmitter # 2018/09/15 - 1.56 - enhance obtainAvailble to use uri # 2018/09/14 - 1.55 - support explicit lineup name for --manage-lineup add # 2018/09/13 - 1.54 - revise previously-shown (new is new and no supplemental) # 2018/09/05 - 1.53 - multi-lineup plumbing - configure # 2018/09/05 - 1.52 - add initial Schedules Direct "IPTV" transport # 2018/09/05 - 1.51 - multi-lineup plumbing - getLineup (v2) # 2018/09/05 - 1.50 - provide proper bind data types # 2018/09/04 - 1.49 - support short-name in getLineup for other types # 2018/09/03 - 1.48 - use available channum # 2018/09/02 - 1.47 - remove dead code # 2018/09/02 - 1.46 - fix grammer # 2018/08/30 - 1.45 - multi-lineup plumbing - mainline code # 2018/08/30 - 1.44 - multi-lineup plumbing - getLineup # 2018/08/30 - 1.43 - multi-lineup plumbing - SD_isLineupFetchRequired # 2018/08/30 - 1.42 - multi-lineup plumbing - lineupValidate # 2018/08/30 - 1.41 - multi-lineup plumbing - channelWriter # 2018/08/29 - 1.40 - multi-lineup plumbing - SD_cleanLineups # 2018/08/29 - 1.39 - multi-lineup plumbing - canonical lineup details # 2018/08/25 - 1.38 - remove dead code # 2018/08/25 - 1.37 - minor whitespace adjustments # 2018/06/17 - 1.36 - handle deleted lineups not having description # 2018/02/03 - 1.35 - remove lang from channel display-name # 2018/02/03 - 1.34 - remove use warning nonfatal experimental in package # 2017/07/21 - 1.33 - dtd compliance. Only actors can have roles # 2017/06/19 - 1.32 - derive category from showtype # 2017/04/20 - 1.31 - provide fixup support # 2017/04/18 - 1.30 - fix typo (in version history) # 2017/04/06 - 1.29 - add supplemental SHow data to EPisodes # 2017/04/02 - 1.28 - misc. code cleanup # 2017/03/26 - 1.27 - misc. code cleanup # 2017/03/21 - 1.26 - fix sth typos # 2017/03/21 - 1.25 - fix trailing whitespace # 2016/09/10 - 1.24 - change (improve) cast mapping # 2016/09/10 - 1.23 - remove use warning nonfatal experimental decl # 2016/08/25 - 1.22 - no warning messages for malformed SD data if quiet # 2016/08/25 - 1.21 - additional error checking of SD data # 2016/08/24 - 1.20 - correct sql error reporting # 2016/08/03 - 1.19 - reflect multinational capability (and fix docs) # 2016/08/03 - 1.18 - rename grabber based on xmltv agreed convention # 2016/07/30 - 1.17 - don't report radio stations as tvshow category # 2016/07/30 - 1.16 - eliminate XML:Writer validation for performance # 2016/07/17 - 1.15 - use Digest::SHA rather than Digest::SHA1 # 2016/06/07 - 1.14 - support multipart episodes # 2016/06/07 - 1.13 - improved season/episode value checks # 2016/05/28 - 1.12 - add support for episodeImage # 2016/05/26 - 1.11 - use program duration as length # 2016/05/26 - 1.10 - hack for tv_find_grabbers source parsing of desc # 2016/05/25 - 1.9 - Support total seasons, and more robust validation # 2016/05/24 - 1.8 - retry limit updates and get-lineup improvements # 2016/05/21 - 1.7 - protect against bad json returned by server # 2016/05/21 - 1.6 - correct (mis)use of global variable in package # 2016/05/20 - 1.5 - minor output formatting improvements for xmltv_ns # 2016/05/19 - 1.4 - correct totalEpisodes output # 2016/05/19 - 1.3 - add support for totalEpisodes metadata # 2016/04/28 - 1.2 - update version number in history and output # 2016/04/23 - 1.1 - Minor update for improved(?) category ordering # 2016/04/01 - 1.0 - First release # eval 'exec /usr/bin/perl -w -S $0 ${1+"$@"}' if 0; # not running under some shell require 5.016; use feature ':5.16'; use strict; use warnings FATAL => 'all'; use warnings NONFATAL => qw(exec recursion internal malloc newline deprecated portable); no warnings 'once'; use utf8; STDERR->autoflush(1); # Autoflush STDERR use XMLTV; use XMLTV::Options qw/ParseOptions/; use XMLTV::Configure::Writer; use XMLTV::Configure qw/LoadConfig SaveConfig/; use XMLTV::Ask; use Getopt::Long; use XML::Writer; use Encode qw/decode encode/; use JSON; use Digest::SHA qw(sha1 sha1_hex sha1_base64); use File::Basename; use File::Which; use File::HomeDir; use File::Path qw(make_path); use DateTime; use DateTime::TimeZone; use DateTime::Format::ISO8601; use DateTime::Format::SQLite; use POSIX qw(strftime); use List::MoreUtils qw(natatime); use List::Util qw/min max/; use DBI; use DBI qw(:sql_types); use DBD::SQLite; use Scalar::Util qw/looks_like_number/; use Data::Dumper; use sort 'stable'; my $RFC2838_COMPLIANT = 1; # RFC2838 compliant station ids, which makes XMLTV # validate even though the docs say "SHOULD" not "MUST" my $SCRIPT_URL = 'https://github.com/garybuhrmaster/tv_grab_zz_sdjson_sqlite'; my $SCRIPT_NAME = basename("$0"); my $SCRIPT_NAME_DIR = dirname("$0"); my $SCRIPT_VERSION = '1.138'; my $SCRIPT_DB_VERSION = 2; # Used for script/db updates (see DB_open) my $SD_DESC = 'Schedules Direct'; my $SD_SITEURL = 'https://www.schedulesdirect.org'; my $SD_COMMENT = 'Note: This data has been downloaded from Schedules Direct, ' . 'and use of the data is restricted by the subscriber agreement ' . 'to non-commercial use with open source projects. Refer to ' . 'the Schedules Direct subscriber agreement for more information'; my $SD_SCHEDULE_HASH_CHUNK = 250; # Request stations schedules hash in chunk sizes my $SD_SCHEDULE_CHUNK = 1000; # Request stations schedules in chunk sizes my $SD_PROGRAM_CHUNK = 4000; # Request program data in chunk sizes my $SD_SCHEDULE_HASH_CHUNK_MAX = 5000; # Schedules Direct max request size my $SD_SCHEDULE_CHUNK_MAX = 5000; # Schedules Direct max request size my $SD_PROGRAM_CHUNK_MAX = 5000; # Schedules Direct max request size my $SD_SCHEDULE_HASH_RETRIES = 3; # Schedules Direct schedule hash request retries my $SD_SCHEDULE_RETRIES = 3; # Schedules Direct schedule request retries my $SD_PROGRAM_RETRIES = 3; # Schedules Direct program request retries my $JSON = JSON->new()->shrink(1)->utf8(1); my $SD = SchedulesDirect->new(UserAgent => "$SCRIPT_NAME ($^O) $SCRIPT_NAME/$SCRIPT_VERSION xmltv/$XMLTV::VERSION perl/" . sprintf("%vd", $^V) . " "); my $DBH; # DataBase Handle my $nowDateTime = DateTime->now( time_zone => 'UTC' ); my $nowDateTimeSQLite = DateTime::Format::SQLite->format_datetime($nowDateTime); my $GRABBER_FIXUPS = {}; # Grabber fixups for broken applications foreach my $fixup(split(':', $ENV{'TV_GRAB_TARGET_APPLICATION_FIXUPS'} || '')) { $GRABBER_FIXUPS->{$fixup} = undef; } my $quiet = 0; my $debug = 0; my $download = 1; my $passwordHash; my $resturl; my $routeto; my $opt; my $conf; # # We attempt to pick off the --passwordhash option due to # the XMLTV ParseOptions not allowing extra_options to be # processed in the configure stage. # Getopt::Long::Configure("pass_through"); GetOptions('passwordhash=s' => \$passwordHash); Getopt::Long::Configure("no_pass_through"); # # We attempt to pick off the --resturl option due to # the XMLTV ParseOptions not allowing extra_options to be # processed in the configure stage. # Getopt::Long::Configure("pass_through"); GetOptions('resturl=s' => \$resturl); Getopt::Long::Configure("no_pass_through"); if (defined($resturl)) { $SD->RESTUrl($resturl); } # # We attempt to pick off the --routeto option due to # the XMLTV ParseOptions not allowing extra_options to be # processed in the configure stage. # Getopt::Long::Configure("pass_through"); GetOptions('routeto=s' => \$routeto); Getopt::Long::Configure("no_pass_through"); if (defined($routeto)) { $SD->RouteTo($routeto); } ( $opt, $conf ) = ParseOptions ( { grabber_name => "$SCRIPT_NAME", capabilities => [qw/baseline manualconfig preferredmethod lineups apiconfig/], stage_sub => \&configureGrabber, listchannels_sub => \&listChannels, list_lineups_sub => \&listLineups, get_lineup_sub => \&getLineup, load_old_config_sub => \&loadOldConfig, preferredmethod => 'allatonce', version => "$SCRIPT_VERSION", description => 'Multinational (Schedules Direct JSON web services with SQLite DB)', extra_options => [qw/manage-lineups force-download download-only no-download passwordhash=s scale-download=f resturl=s routeto=s/], defaults => { days => 30 }, } ); $debug = $opt->{'debug'}; $quiet = $opt->{'quiet'}; $SD->Debug(1) if ($debug); # # Special case for managing lineups # # This should (possibly) be done at the Schedules Direct # site itself (as it is done now), or a seperate program, # but as of now, this is it. # if ($opt->{'manage-lineups'}) { manageLineups(); exit(0); } # # Verify we have what we need to proceed and # perform a few checks for things we do not # support # configValidate($conf, $opt); if ($opt->{'offset'} < 0) { # Note: While it is (in theory) possible to # support an offset of -1, it requires a # bit of hoop jumping to get that data from # Schedules Direct, and it is not really # considered to be worth it for the edge # cases that might exist. The data may be # in the database in some cases. print (STDERR "Offset value may not be less than 0\n"); exit(1); } if ($opt->{'days'} < 0) { print (STDERR "Day value may not be less than 0\n"); exit(1); } if (defined($opt->{'scale-download'}) && looks_like_number($opt->{'scale-download'})) { $SD_SCHEDULE_HASH_CHUNK = min($SD_SCHEDULE_HASH_CHUNK_MAX, max(1, int($SD_SCHEDULE_HASH_CHUNK * $opt->{'scale-download'}))); $SD_SCHEDULE_CHUNK = min($SD_SCHEDULE_CHUNK_MAX, max(1, int($SD_SCHEDULE_CHUNK * $opt->{'scale-download'}))); $SD_PROGRAM_CHUNK = min($SD_PROGRAM_CHUNK_MAX, max(1, int($SD_PROGRAM_CHUNK * $opt->{'scale-download'}))); } if (!defined(eval {require JSON::XS})) { print (STDERR "WARNING: Perl module JSON::XS not installed. JSON encode/decode performance will be poor.\n") if (!$quiet); } $download = 0 if ($opt->{'no-download'}); # # Various sql and statement handles for accessing our database # my $sql; my $sql0; my $sql1; my $sql2; my $sql3; my $sql4; my $sth; my $sth0; my $sth1; my $sth2; my $sth3; my $sth4; my $param; my $param0; my $param1; my $param2; my $param3; my $param4; # # Open database # print (STDERR "Opening the local database\n") if (!$quiet); DB_open($conf->{'database'}->[0]); # # Provide the ability to force a (mostly) complete download # for all data by deleting most of the data in the database, # making this (in effect) a "first download". # if ($opt->{'force-download'}) { print (STDERR " clearing existing database to force full download\n") if (!$quiet); DB_clean(); } # # If we are not downloading data, we need to verify # that the lineup is in the database now. # if (!$download) { lineupValidate($conf->{'lineup'}); my $token = DB_settingsGet('token'); $SD->Token($token) if (defined($token)); goto skipDownload; } # # Login and perform the usual checks # print (STDERR "Obtaining authentication token for Schedules Direct\n") if (!$quiet); SD_login(); my $expiry = $SD->accountExpiry; if (!defined($expiry)) { print (STDERR "Unable to obtain the account expiration date: " . $SD->ErrorString . "\n"); exit(1); } my $expiryDateTime = DateTime::Format::ISO8601->parse_datetime($expiry); print (STDERR " Schedules Direct account expires on " . $expiryDateTime . "\n") if (!$quiet); # # Start the download process # print (STDERR "Downloading data from Schedules Direct\n") if (!$quiet); # # Always make sure we have a current lineup list # print (STDERR " downloading account lineups from Schedules Direct\n") if (!$quiet); SD_downloadLineups(); # # Validate that the configured lineup exists in our database # lineupValidate($conf->{'lineup'}); # # Get our current Schedules Direct maps (channels and # stations) for our lineup and feed to our DB if needed # for my $lineup(@{$conf->{'lineup'}}) { if (SD_isLineupFetchRequired([$lineup])) { print (STDERR " downloading channel and station maps for lineup $lineup \n") if (!$quiet); SD_downloadLineupMaps($lineup); } else { print (STDERR " not downloading channel and station maps for lineup $lineup (data current)\n") if (!$quiet); } } # # Obtain the current schedule hash values for our lineup stations # for (my $retry = 0; $retry < ($SD_SCHEDULE_HASH_RETRIES + 1); $retry++) { my $downloadRetry = 0; $sql = 'select distinct stations.station from stations as stations where stations.station in (select distinct channels.station from channels as channels where channels.lineup in ( ' . join(', ', ('?') x scalar(@{$conf->{'lineup'}})) . ' ) and channels.selected = 1)'; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $param = 1; for (my $i=0; $i < scalar(@{$conf->{'lineup'}}); $i++) { $sth->bind_param( $param, @{$conf->{'lineup'}}[$i], SQL_VARCHAR); $param++; } $sth->execute(); if ($sth->err()) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, undef, SQL_VARCHAR ); my $stationsSchedulesHashList = $sth->fetchall_arrayref([0]); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); undef $sth; print (STDERR " downloading station schedule hashes for " . scalar(@{$stationsSchedulesHashList}) . " stations" . (($retry == 0) ? "" : " (retry $retry)") . "\n") if (!$quiet); sleep(min(30, (10 * $retry))); my $stationsSchedulesHashIter; $stationsSchedulesHashIter = natatime $SD_SCHEDULE_HASH_CHUNK, @{$stationsSchedulesHashList}; while(my @chunk = $stationsSchedulesHashIter->()) { print (STDERR " downloading schedule hashes for " . scalar(@chunk) . " stations in this chunk\n") if ((!$quiet) && ((scalar(@chunk) != scalar(@{$stationsSchedulesHashList})))); my $stationsSchedulesHashRequest = []; foreach (@chunk) { my $s = {}; $s->{'stationID'} = $_->[0]; push(@{$stationsSchedulesHashRequest}, $s); } my $r = $SD->obtainStationsSchedulesHash(@{$stationsSchedulesHashRequest}); if (!defined($r)) { if ($downloadRetry < 10) { print (STDERR "Unexpected error when obtaining station schedules hashes: " . $SD->ErrorString() . " (will retry)\n"); } $downloadRetry++; } if (ref($r) ne 'ARRAY') { if ($downloadRetry < 10) { print (STDERR "Unexpected return data type " . ref($r) . " when obtaining station schedules hashes. (will retry)\n"); } $downloadRetry++; } $sql = "replace into stations_schedules_hash (station, day, hash, details) values ( ?, ?, ?, ?)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } foreach my $e(@{$r}) { if (ref($e) ne 'HASH') { if ($downloadRetry < 10) { print (STDERR "Unexpected return data type " . ref($e) . " while iterating station schedules hashes (will retry)\n"); } $downloadRetry++; next; } my $code = $e->{'code'} || 0; if ($code != 0) { if ($code == 7100) # request queued { if ($downloadRetry < 10) { print (STDERR "Request for schedule queued when obtaining station schedules hashes: (will retry)\n") if (!$quiet); } $downloadRetry++; } next; } if ((!defined($e->{'stationID'})) || (!defined($e->{'date'})) || ((substr($e->{'date'}, 0, 10)) !~ /^\d{4}-\d{2}-\d{2}$/) || (!defined($e->{'MD5'}))) { if ($downloadRetry < 10) { print (STDERR "Station, date, or hash not provided while iterating station schedules hashes (will retry)\n"); } $downloadRetry++; next; } my $station = $e->{'stationID'}; my $date = substr($e->{'date'}, 0, 10); my $hash = $e->{'MD5'}; my $details = $JSON->encode($e); $sth->bind_param( 1, $station, SQL_VARCHAR ); $sth->bind_param( 2, $date, SQL_DATE ); $sth->bind_param( 3, $hash, SQL_VARCHAR ); $sth->bind_param( 4, $details, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } } $DBH->commit(); undef $sth; } if (!$downloadRetry) { # # Indicate we have downloaded the data # $sql = 'update lineups set downloaded = ? where lineup in ( ' . join(', ', ('?') x scalar(@{$conf->{'lineup'}})) . ' )'; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $param = 1; $sth->bind_param( $param, $nowDateTimeSQLite, SQL_DATETIME ); $param++; for (my $i=0; $i < scalar(@{$conf->{'lineup'}}); $i++) { $sth->bind_param( $param, @{$conf->{'lineup'}}[$i], SQL_VARCHAR); $param++; } $sth->execute(); if ($sth->err()) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); undef $sth; # # And we are done here # last; } } # # Obtain the station schedules for days for which we do # not have current information based on hash values and # feed to our DB # for (my $retry = 0; $retry < ($SD_SCHEDULE_RETRIES + 1); $retry++) { my $downloadRetry = 0; my $startDateTime = DateTime->now(time_zone => 'UTC')->add(days => $opt->{'offset'}); my $endDateTime = DateTime->now(time_zone => 'UTC')->add(days => $opt->{'offset'})->add(days => $opt->{'days'}); # # Note that we only update schedules (if needed) for days which will # produce reports for. This may, in some cases, reduce the overheads # # Note also that it is important to check for the day to be >= today # in order to skip retrieving schedules where the hash is obsolete. # Since Schedules Direct does not update past station_schedules, but # we keep them around for a bit, our past schedule hash can be invalid, # but we do not want to force a request for such schedules, which would # likely fail since Schedules Direct does not make available data # which older than (about) 24 hours ago. # $sql = "select distinct stations_schedules_hash.station, stations_schedules_hash.day from stations_schedules_hash as stations_schedules_hash " . " left outer join schedules_hash as schedules_hash on stations_schedules_hash.station = schedules_hash.station " . " AND stations_schedules_hash.day = schedules_hash.day " . " where (stations_schedules_hash.station in (select distinct channels.station " . " from channels as channels where channels.lineup in ( " . join(', ', ('?') x scalar(@{$conf->{'lineup'}})) . " ) " . " and channels.selected = 1)) " . " AND (schedules_hash.station is NULL OR schedules_hash.hash != stations_schedules_hash.hash) " . " AND stations_schedules_hash.day >= ? AND stations_schedules_hash.day < ? " . " ORDER by stations_schedules_hash.station, stations_schedules_hash.day"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $param = 1; for (my $i=0; $i < scalar(@{$conf->{'lineup'}}); $i++) { $sth->bind_param( $param, @{$conf->{'lineup'}}[$i], SQL_VARCHAR); $param++; } $sth->bind_param( $param, DateTime::Format::SQLite->format_date($startDateTime), SQL_DATE ); $param++; $sth->bind_param( $param, DateTime::Format::SQLite->format_date($endDateTime), SQL_DATE ); $param++; $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, undef, SQL_VARCHAR ); $sth->bind_col( 2, undef, SQL_DATE ); my $stationsSchedulesList = $sth->fetchall_arrayref(); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); undef $sth; if (scalar(@{$stationsSchedulesList}) == 0) { if ($retry == 0) { print (STDERR " not downloading daily schedules (data current)\n") if (!$quiet); } last; } print (STDERR " downloading " . scalar(@{$stationsSchedulesList}) . " new, updated, or missing daily schedules" . (($retry == 0) ? "" : " (retry $retry)") . "\n") if (!$quiet); sleep(min(30, (10 * $retry))); my $schedulesIter; $schedulesIter = natatime $SD_SCHEDULE_CHUNK, @{$stationsSchedulesList}; while(my @chunk = $schedulesIter->()) { print (STDERR " downloading " . scalar(@chunk) . " new, updated, or missing daily schedules in this chunk\n") if ((!$quiet) && ((scalar(@chunk) != scalar(@{$stationsSchedulesList})))); my $stationsSchedulesRequest = []; foreach (@chunk) { my $s = {}; $s->{'stationID'} = $_->[0]; $s->{'date'} = [$_->[1]]; push (@{$stationsSchedulesRequest}, ($s)); } my $r = $SD->obtainStationsSchedules(@{$stationsSchedulesRequest}); if (!defined($r)) { # For some reason, sometimes Schedules Direct returns malformed response (I believe due to # their optimization for the program array returns, which can result in partial data). # We will force a retry under those conditions. if ($downloadRetry < 10) { print (STDERR "Unexpected error when obtaining station schedules: " . $SD->ErrorString() . " (will retry)\n") if (!$quiet); } $downloadRetry++; next; } if (ref($r) ne 'ARRAY') { # For some reason, sometimes Schedules Direct returns malformed response (I believe due to # their optimization for the program array returns, which can result in partial data). # We will force a retry under those conditions. if ($downloadRetry < 10) { print (STDERR "Unexpected error when obtaining station schedules: " . $SD->ErrorString() . " (will retry)\n") if (!$quiet); } $downloadRetry++; next; } $sql1 = "delete from schedules where station = ? and day = ?"; $sql2 = "replace into schedules (station, day, starttime, duration, program, program_hash, details) values (?, ?, ?, ?, ?, ?, ?)"; $sql3 = "replace into schedules_hash (station, day, hash) values (?, ?, ?)"; $sth1 = $DBH->prepare_cached($sql1); if (!defined($sth1)) { print (STDERR "Unexpected error when preparing statement ($sql1): " . $DBH->errstr . "\n"); exit(1); } $sth2 = $DBH->prepare_cached($sql2); if (!defined($sth2)) { print (STDERR "Unexpected error when preparing statement ($sql2): " . $DBH->errstr . "\n"); exit(1); } $sth3 = $DBH->prepare_cached($sql3); if (!defined($sth3)) { print (STDERR "Unexpected error when preparing statement ($sql3): " . $DBH->errstr . "\n"); exit(1); } foreach my $sched(@{$r}) { my $hash = $sched->{'MD5'}; my $dayDateTime; $dayDateTime = DateTime::Format::ISO8601->parse_datetime($sched->{'date'}) if (defined($sched->{'date'})); my $sID = $sched->{'stationID'}; my $code = $sched->{'code'} || 0; if ($code != 0) { if ($code == 7100) # request queued { if ($downloadRetry < 10) { print (STDERR "Request for schedule queued when obtaining station schedules: (will retry)\n") if (!$quiet); } $downloadRetry++; } next; } my $programs = $sched->{'programs'}; if ((!defined($hash)) || (!defined($dayDateTime)) || (!defined($programs))) { next; } $sth1->bind_param( 1, $sID, SQL_VARCHAR ); $sth1->bind_param( 2, DateTime::Format::SQLite->format_date($dayDateTime), SQL_DATE ); $sth1->execute(); if ($sth1->err) { print (STDERR "Unexpected error when executing statement ($sql1): " . $sth1->errstr . "\n"); $DBH->rollback(); exit(1); } foreach my $program(@{$programs}) { my $pID = $program->{'programID'}; my $airDateTime = $program->{'airDateTime'}; my $duration = $program->{'duration'}; my $phash = $program->{'md5'}; my $details = $JSON->encode($program); if ((!defined($duration)) || (!defined($phash)) || (!defined($pID)) || (!defined($airDateTime))) { print (STDERR "Unexpected parsing error in program (data malformed) in schedule for $sID on " . $sched->{'date'} . ", skipping\n") if (!$quiet); next; } my $starttime = DateTime::Format::ISO8601->parse_datetime($airDateTime); $sth2->bind_param( 1, $sID, SQL_VARCHAR ); $sth2->bind_param( 2, DateTime::Format::SQLite->format_date($dayDateTime), SQL_DATE ); $sth2->bind_param( 3, DateTime::Format::SQLite->format_datetime($starttime), SQL_DATETIME ); $sth2->bind_param( 4, $duration, SQL_INTEGER ); $sth2->bind_param( 5, $pID, SQL_VARCHAR ); $sth2->bind_param( 6, $phash, SQL_VARCHAR ); $sth2->bind_param( 7, $details, SQL_VARCHAR ); $sth2->execute(); if ($sth2->err) { print (STDERR "Unexpected error when executing statement ($sql2): " . $sth2->errstr . "\n"); $DBH->rollback(); exit(1); } } $sth3->bind_param( 1, $sID, SQL_VARCHAR ); $sth3->bind_param( 2, DateTime::Format::SQLite->format_date($dayDateTime), SQL_DATE ); $sth3->bind_param( 3, $hash, SQL_VARCHAR ); $sth3->execute(); if ($sth3->err) { print (STDERR "Unexpected error when executing statement ($sql3): " . $sth3->errstr . "\n"); $DBH->rollback(); exit(1); } } $DBH->commit(); undef $sth1; undef $sth2; undef $sth3; } # We are done unless one (or more) entities indicate that the server queued the request or needs to retry last if (!$downloadRetry); } # # Obtain the program information for programs for which # we do not have current information based on hash values # and feed to our DB # for (my $retry = 0; $retry < ($SD_PROGRAM_RETRIES + 1); $retry++) { my $downloadRetry = 0; my $startDateTime = DateTime->now(time_zone => 'UTC')->add(days => $opt->{'offset'}); my $endDateTime = DateTime->now(time_zone => 'UTC')->add(days => $opt->{'offset'})->add(days => $opt->{'days'}); # # Note that we only update programs (if needed) for days which will # produce reports for. This may, in some cases, reduce the overheads # # Note also that it is important to check for the day to be >= today # in order to skip retrieving programs where the program hash is # obsolete. Since Schedules Direct does not update past schedules, # but we keep then around for a bit, our past program hash can # be invalid, but we do not want to request such programs (since # the program hash will be updated). # $sql = "select distinct schedules.program from schedules as schedules " . " left outer join programs as programs on schedules.program = programs.program " . " where (schedules.station in (select distinct stations.station " . " from stations as stations where stations.station " . " in (select distinct channels.station from channels channels " . " where channels.lineup in ( " . join(', ', ('?') x scalar(@{$conf->{'lineup'}})) . " ) and channels.selected = 1)) " . " AND (programs.program is null OR schedules.program_hash != programs.hash)) " . " AND schedules.day >= ? AND schedules.day < ?" ; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $param = 1; for (my $i=0; $i < scalar(@{$conf->{'lineup'}}); $i++) { $sth->bind_param( $param, @{$conf->{'lineup'}}[$i], SQL_VARCHAR); $param++; } $sth->bind_param( $param, DateTime::Format::SQLite->format_date($startDateTime), SQL_DATE ); $param++; $sth->bind_param( $param, DateTime::Format::SQLite->format_date($endDateTime), SQL_DATE ); $param++; $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, undef, SQL_VARCHAR ); my $programsList = $sth->fetchall_arrayref([0]); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); undef $sth; if (scalar(@{$programsList}) == 0) { if ($retry == 0) { print (STDERR " not downloading programs (data current)\n") if (!$quiet); } last; } print (STDERR " downloading " . scalar(@{$programsList}) . " new, updated, or missing programs" . (($retry == 0) ? "" : " (retry $retry)") . "\n") if (!$quiet); sleep(min(30, (10 * $retry))); my $programsIter; $programsIter = natatime $SD_PROGRAM_CHUNK, @{$programsList}; while(my @chunk = $programsIter->()) { print (STDERR " downloading " . scalar(@chunk) . " new, updated, or missing programs in this chunk\n") if ((!$quiet) && ((scalar(@chunk) != scalar(@{$programsList})))); my $pl = []; foreach (@chunk) { push (@{$pl}, $_->[0]); } my $r = $SD->obtainPrograms(@{$pl}); if (!defined($r)) { # For some reason, sometimes Schedules Direct returns malformed response (I believe due to # their optimization for the program array returns, which can result in partial data). # We will force a retry under those conditions. if ($downloadRetry < 10) { print (STDERR "Unexpected error when obtaining programs: " . $SD->ErrorString() . " (will retry)\n") if (!$quiet); } $downloadRetry++; next; } if (ref($r) ne 'ARRAY') { # For some reason, sometimes Schedules Direct return malformed response (I believe due to # their optiomization for the program array returns, which can result in partial data). # We will force a retry under those conditions. if ($downloadRetry < 10) { print (STDERR "Unexpected return data type " . ref($r) . " when obtaining program array (will retry)\n") if (!$quiet); } $downloadRetry++; next; } $sql1 = "replace into programs (program, hash, details, program_supplemental, downloaded) values (?, ?, ?, ?, ?)"; $sth1 = $DBH->prepare_cached($sql1); if (!defined($sth1)) { print (STDERR "Unexpected error when preparing statement ($sql1): " . $DBH->errstr . "\n"); exit(1); } foreach my $program(@{$r}) { my $pID = $program->{'programID'}; next if (!defined($pID)); my $hash = $program->{'md5'} || 0; my $code = $program->{'code'} || 0; if ($code != 0) { if ($code == 6001) # request queued { if ($downloadRetry < 10) { print (STDERR "Request for program queued when obtaining program data: (will retry)\n") if (!$quiet); } $downloadRetry++; } next; } my $details = $JSON->encode($program); my $supplemental; if (substr($pID, 0, 2) eq 'EP') { $supplemental = 'SH' . substr($pID, 2, 8) . '0000'; } $sth1->bind_param( 1, $pID, SQL_VARCHAR ); $sth1->bind_param( 2, $hash, SQL_VARCHAR ); $sth1->bind_param( 3, $details, SQL_VARCHAR ); $sth1->bind_param( 4, $supplemental, SQL_VARCHAR ); $sth1->bind_param( 5, $nowDateTimeSQLite, SQL_DATETIME ); $sth1->execute(); if ($sth1->err) { print (STDERR "Unexpected error when executing statement ($sql1): " . $sth1->errstr . "\n"); $DBH->rollback(); exit(1); } } $DBH->commit(); undef $sth1; } # We are done unless one (or more) entities indicate that the server queued the request or needs to retry last if (!$downloadRetry); } # # Obtain the program supplemental information for programs # for which we do not have current information # for (my $retry = 0; $retry < ($SD_PROGRAM_RETRIES + 1); $retry++) { my $startDateTime = DateTime->now(time_zone => 'UTC')->add(days => $opt->{'offset'}); my $endDateTime = DateTime->now(time_zone => 'UTC')->add(days => $opt->{'offset'})->add(days => $opt->{'days'}); my $downloadRetry = 0; # Select all necessary supplemental program entities, and # randomly select others with an age bias (older more likely) $sql = "select distinct p1.program_supplemental from programs as p1 " . " left join programs as supplemental on supplemental.program = p1.program_supplemental " . " where p1.program_supplemental is not null " . " and ( (supplemental.program is null) " . " or ((((julianday('now') - julianday(supplemental.downloaded)) / 30.0) + " . " (0.5 - random() / cast(-9223372036854775808 as real) / 2.0)) > 1.40 )) " . " and p1.program in ( select schedules.program from schedules as schedules " . " where (schedules.station in (select distinct stations.station from stations as stations " . " where stations.station in (select distinct channels.station from channels channels " . " where channels.lineup in ( " . join(', ', ('?') x scalar(@{$conf->{'lineup'}})) . " ) " . " and channels.selected = 1)) " . " and schedules.day >= ? and schedules.day < ? ) ) "; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $param = 1; for (my $i=0; $i < scalar(@{$conf->{'lineup'}}); $i++) { $sth->bind_param( $param, @{$conf->{'lineup'}}[$i], SQL_VARCHAR); $param++; } $sth->bind_param( $param, DateTime::Format::SQLite->format_date($startDateTime), SQL_DATE ); $param++; $sth->bind_param( $param, DateTime::Format::SQLite->format_date($endDateTime), SQL_DATE ); $param++; $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col(1, undef, SQL_VARCHAR ); my $programsList = $sth->fetchall_arrayref([0]); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); undef $sth; if (scalar(@{$programsList}) == 0) { if ($retry == 0) { print (STDERR " not downloading programs for supplemental data (data current)\n") if (!$quiet); } last; } print (STDERR " downloading " . scalar(@{$programsList}) . " new, updated, or missing programs for supplemental data" . (($retry == 0) ? "" : " (retry $retry)") . "\n") if (!$quiet); sleep(min(30, (10 * $retry))); my $programsIter; $programsIter = natatime $SD_PROGRAM_CHUNK, @{$programsList}; while(my @chunk = $programsIter->()) { print (STDERR " downloading " . scalar(@chunk) . " new, updated, or missing programs for supplemental data in this chunk\n") if ((!$quiet) && ((scalar(@chunk) != scalar(@{$programsList})))); my $pl = []; foreach (@chunk) { push (@{$pl}, $_->[0]); } my $r = $SD->obtainPrograms(@{$pl}); if (!defined($r)) { # For some reason, sometimes Schedules Direct returns malformed response (I believe due to # their optimization for the program array returns, which can result in partial data). # We will force a retry under those conditions. if ($downloadRetry < 10) { print (STDERR "Unexpected error when obtaining programs: " . $SD->ErrorString() . " (will retry)\n") if (!$quiet); } $downloadRetry++; next; } if (ref($r) ne 'ARRAY') { # For some reason, sometimes Schedules Direct return malformed response (I believe due to # their optiomization for the program array returns, which can result in partial data). # We will force a retry under those conditions. if ($downloadRetry < 10) { print (STDERR "Unexpected return data type " . ref($r) . " when obtaining program array (will retry)\n") if (!$quiet); } $downloadRetry++; next; } $sql1 = "replace into programs (program, hash, details, program_supplemental, downloaded) values (?, ?, ?, ?, ?)"; $sth1 = $DBH->prepare_cached($sql1); if (!defined($sth1)) { print (STDERR "Unexpected error when preparing statement ($sql1): " . $DBH->errstr . "\n"); exit(1); } foreach my $program(@{$r}) { my $pID = $program->{'programID'}; next if (!defined($pID)); my $hash = $program->{'md5'} || 0; my $code = $program->{'code'} || 0; if ($code != 0) { if ($code == 6001) # request queued { if ($downloadRetry < 10) { print (STDERR "Request for program queued when obtaining program data: (will retry)\n") if (!$quiet); } $downloadRetry++; } next; } my $details = $JSON->encode($program); my $supplemental; if (substr($pID, 0, 2) eq 'EP') { $supplemental = 'SH' . substr($pID, 2, 8) . '0000'; } $sth1->bind_param( 1, $pID, SQL_VARCHAR ); $sth1->bind_param( 2, $hash, SQL_VARCHAR ); $sth1->bind_param( 3, $details, SQL_VARCHAR ); $sth1->bind_param( 4, $supplemental, SQL_VARCHAR ); $sth1->bind_param( 5, $nowDateTimeSQLite, SQL_DATETIME ); $sth1->execute(); if ($sth1->err) { print (STDERR "Unexpected error when executing statement ($sql1): " . $sth1->errstr . "\n"); $DBH->rollback(); exit(1); } } $DBH->commit(); undef $sth1; } # We are done unless one (or more) entities indicate that the server queued the request or needs to retry last if (!$downloadRetry); } # # Process data and report # skipDownload: # # If we were requested to only download data, # we are now complete # goto finalize if ($opt->{'download-only'}); # # Start at the start # print (STDERR "Processing data and creating XMLTV output\n") if (!$quiet); # # Create some mappings for processing programs # # (Known) Schedules Direct cast roles and XMLTV map # (this map likely needs review and correction) my $castMap = { 'Actor' => { 'role' => 'actor' }, 'Self' => { 'role' => 'actor', 'self' => 1 }, 'Voice' => { 'role' => 'actor' }, 'Guest Star' => { 'role' => 'actor', 'guest' => 1 }, 'Guest Voice' => { 'role' => 'actor', 'guest' => 1 }, 'Athlete' => { 'role' => 'guest' }, 'Correspondent' => { 'role' => 'guest' }, 'Contestant' => { 'role' => 'guest' }, 'Guest' => { 'role' => 'guest' }, 'Musicial Guest' => { 'role' => 'guest' }, 'Music Performer' => { 'role' => 'guest' }, 'Anchor' => { 'role' => 'presenter' }, 'Host' => { 'role' => 'presenter' }, 'Narrator' => { 'role' => 'presenter' }, 'Judge' => { 'role' => 'presenter' } }; # (Known) Schedules Direct crew roles and XMLTV map # for those XMLTV roles we will use (there is no # XMLTV role for make up artist, for example) # (this map likely needs review and correction) my $crewMap = { 'Adaptation' => { 'role' => 'writer' }, 'Co-Screenwriter' => { 'role' => 'writer' }, 'Co-Writer' => { 'role' => 'writer' }, 'Creator' => { 'role' => 'writer' }, 'Film Consultant' => { 'role' => 'writer' }, 'Screen Story' => { 'role' => 'writer' }, 'Screenwriter' => { 'role' => 'writer' }, 'Script' => { 'role' => 'writer' }, 'Script Editor' => { 'role' => 'writer' }, 'Story Supervisor' => { 'role' => 'writer' }, 'Story' => { 'role' => 'writer' }, 'Writer' => { 'role' => 'writer' }, 'Writer (Adaptation)' => { 'role' => 'writer' }, 'Writer (Additional Dialogue)' => { 'role' => 'writer' }, 'Writer (Autobiography)' => { 'role' => 'writer' }, 'Writer (Book)' => { 'role' => 'writer' }, 'Writer (Characters)' => { 'role' => 'writer' }, 'Writer (Comic Book)' => { 'role' => 'writer' }, 'Writer (Continuity)' => { 'role' => 'writer' }, 'Writer (Dialogue)' => { 'role' => 'writer' }, 'Writer (Earlier Screenplay)' => { 'role' => 'writer' }, 'Writer (Idea)' => { 'role' => 'writer' }, 'Writer (Miniseries)' => { 'role' => 'writer' }, 'Writer (Narration)' => { 'role' => 'writer' }, 'Writer (Novel)' => { 'role' => 'writer' }, 'Writer (Opera)' => { 'role' => 'writer' }, 'Writer (Original Film)' => { 'role' => 'writer' }, 'Writer (Original Screenplay)' => { 'role' => 'writer' }, 'Writer (Play)' => { 'role' => 'writer' }, 'Writer (Poem)' => { 'role' => 'writer' }, 'Writer (Scenario)' => { 'role' => 'writer' }, 'Writer (Screen Story)' => { 'role' => 'writer' }, 'Writer (Screenplay)' => { 'role' => 'writer' }, 'Writer (Screenplay and Dialogue)' => { 'role' => 'writer' }, 'Writer (Screenplay and Novel)' => { 'role' => 'writer' }, 'Writer (Script)' => { 'role' => 'writer' }, 'Writer (Short Story)' => { 'role' => 'writer' }, 'Writer (Stage Musical)' => { 'role' => 'writer' }, 'Writer (Story)' => { 'role' => 'writer' }, 'Writer (Story and Screenplay)' => { 'role' => 'writer' }, 'Writer (Teleplay)' => { 'role' => 'writer' }, 'Writer (Television Series)' => { 'role' => 'writer' }, 'Writer (Treatment)' => { 'role' => 'writer' }, 'Action Director' => { 'role' => 'director' }, 'Animation Director' => { 'role' => 'director' }, 'Art Direction' => { 'role' => 'director' }, 'Art Director' => { 'role' => 'director' }, 'Artistic Director' => { 'role' => 'director' }, 'Assistant Art Director' => { 'role' => 'director' }, 'Assistant Director' => { 'role' => 'director' }, 'Associate Art Direction' => { 'role' => 'director' }, 'Associate Director' => { 'role' => 'director' }, 'Casting Director' => { 'role' => 'director' }, 'Cinematographer' => { 'role' => 'director' }, 'Co-Art Director' => { 'role' => 'director' }, 'Co-Director' => { 'role' => 'director' }, 'Creative Director' => { 'role' => 'director' }, 'Dance Director' => { 'role' => 'director' }, 'Director' => { 'role' => 'director' }, 'Director of Cinematography' => { 'role' => 'director' }, 'Director of Photography' => { 'role' => 'director' }, 'First Assistant Director' => { 'role' => 'director' }, 'Key Second Asst. Director' => { 'role' => 'director' }, 'Managing Technical Director' => { 'role' => 'director' }, 'Musical Director' => { 'role' => 'director' }, 'Music Director' => { 'role' => 'director' }, 'Recording Director' => { 'role' => 'director' }, 'Second Assistant Director' => { 'role' => 'director' }, 'Second Second Assistant Director' => { 'role' => 'director' }, 'Second Unit Director' => { 'role' => 'director' }, 'Senior Art Director' => { 'role' => 'director' }, 'Set Director' => { 'role' => 'director' }, 'Stunt Action Director' => { 'role' => 'director' }, 'Supervising Art Direction' => { 'role' => 'director' }, 'Third Assistant Director' => { 'role' => 'director' }, 'Trainee Assistant Director' => { 'role' => 'director' }, 'Unit Director' => { 'role' => 'director' }, 'Voice Director' => { 'role' => 'director' }, 'Wardrobe Director' => { 'role' => 'director' }, 'Additional Editor' => { 'role' => 'editor' }, 'Assistant Dialogue Editor' => { 'role' => 'editor' }, 'Assistant Editor' => { 'role' => 'editor' }, 'Assistant Sound Editor' => { 'role' => 'editor' }, 'Associate Film Editor' => { 'role' => 'editor' }, 'Background Sound Editor' => { 'role' => 'editor' }, 'Co-Editor' => { 'role' => 'editor' }, 'Dialogue Editor' => { 'role' => 'editor' }, 'Editing' => { 'role' => 'editor' }, 'Editor' => { 'role' => 'editor' }, 'Film Editing' => { 'role' => 'editor' }, 'Film Editor' => { 'role' => 'editor' }, 'Foley Editor' => { 'role' => 'editor' }, 'Music Editor' => { 'role' => 'editor' }, 'Sound Editor' => { 'role' => 'editor' }, 'Sound Effects Editor' => { 'role' => 'editor' }, 'Supervising ADR Editor' => { 'role' => 'editor' }, 'Supervising Editor' => { 'role' => 'editor' }, 'Supervising Foley Editor' => { 'role' => 'editor' }, 'Supervising Sound Editor' => { 'role' => 'editor' }, 'Animation Producer' => { 'role' => 'producer' }, 'Assistant Producer' => { 'role' => 'producer' }, 'Associate Executive Producer' => { 'role' => 'producer' }, 'Associate Producer' => { 'role' => 'producer' }, 'Chief Producer' => { 'role' => 'producer' }, 'Co-Associate Producer' => { 'role' => 'producer' }, 'Co-Executive Producer' => { 'role' => 'producer' }, 'Consulting Producer' => { 'role' => 'producer' }, 'Coordinating Producer' => { 'role' => 'producer' }, 'Co-Producer' => { 'role' => 'producer' }, 'Executive Co-Producer' => { 'role' => 'producer' }, 'Executive in Charge of Production' => { 'role' => 'producer' }, 'Executive Music Producer' => { 'role' => 'producer' }, 'Executive Producer' => { 'role' => 'producer' }, 'Line Producer' => { 'role' => 'producer' }, 'Location Producer' => { 'role' => 'producer' }, 'Makeup Effects Producer' => { 'role' => 'producer' }, 'Music Producer' => { 'role' => 'producer' }, 'Producer' => { 'role' => 'producer' }, 'Score Producer' => { 'role' => 'producer' }, 'Senior Producer' => { 'role' => 'producer' }, 'Special Effects Makeup Producer' => { 'role' => 'producer' }, 'Supervising Producer' => { 'role' => 'producer' }, 'Visual Effects Producer' => { 'role' => 'producer' }, 'Additional Music' => { 'role' => 'composer' }, 'Composer' => { 'role' => 'composer' }, 'Lyricist' => { 'role' => 'composer' }, 'Lyrics' => { 'role' => 'composer' }, 'Music' => { 'role' => 'composer' }, 'Music Arranger' => { 'role' => 'composer' }, 'Music Score' => { 'role' => 'composer' }, 'Music Supervisor' => { 'role' => 'composer' }, 'Music Theme' => { 'role' => 'composer' }, 'Non-Original Music' => { 'role' => 'composer' }, 'Original Music' => { 'role' => 'composer' }, 'Original Music and Songs' => { 'role' => 'composer' }, 'Original Score' => { 'role' => 'composer' }, 'Original Song' => { 'role' => 'composer' }, 'Original Songs' => { 'role' => 'composer' }, 'Original Theme' => { 'role' => 'composer' }, 'Songs' => { 'role' => 'composer' }, # # The following crew roles at Schedules Direct # have no clear XMLTV equivalent (suggestions # encouraged). We list them here so it will # be easier to identify new roles in the future. # 'Animation Supervisor' => { 'role' => 'undefined' }, 'Animator' => { 'role' => 'undefined' }, 'Art Department' => { 'role' => 'undefined' }, 'Assistant Makeup Artist' => { 'role' => 'undefined' }, 'Assistant Production Manager' => { 'role' => 'undefined' }, 'Associate Costume Designer' => { 'role' => 'undefined' }, 'Associate Set Decorator' => { 'role' => 'undefined' }, 'Athlete' => { 'role' => 'undefined' }, 'Background Music' => { 'role' => 'undefined' }, 'Boom Operator' => { 'role' => 'undefined' }, 'Cameraman' => { 'role' => 'undefined' }, 'Camera Operator' => { 'role' => 'undefined' }, 'Casting' => { 'role' => 'undefined' }, 'Characters' => { 'role' => 'undefined' }, 'Chief Hair Stylist' => { 'role' => 'undefined' }, 'Chief Makeup Artist' => { 'role' => 'undefined' }, 'Choreographer' => { 'role' => 'undefined' }, 'Cinematography' => { 'role' => 'undefined' }, 'Conductor' => { 'role' => 'undefined' }, 'Construction Coordinator' => { 'role' => 'undefined' }, 'Co-Production Designer' => { 'role' => 'undefined' }, 'Costume Design' => { 'role' => 'undefined' }, 'Costume Designer' => { 'role' => 'undefined' }, 'Costume Supervisor' => { 'role' => 'undefined' }, 'Creative Consultant' => { 'role' => 'undefined' }, 'Design' => { 'role' => 'undefined' }, 'Dialogue' => { 'role' => 'undefined' }, 'Executive Assistant' => { 'role' => 'undefined' }, 'Foley Artist' => { 'role' => 'undefined' }, 'Graphic Artist' => { 'role' => 'undefined' }, 'Graphic Design' => { 'role' => 'undefined' }, 'Graphic Designer' => { 'role' => 'undefined' }, 'Graphics' => { 'role' => 'undefined' }, 'Hair Designer' => { 'role' => 'undefined' }, 'Hair Stylist Supervisor' => { 'role' => 'undefined' }, 'Hair Stylist' => { 'role' => 'undefined' }, 'Key Grip' => { 'role' => 'undefined' }, 'Key Hair Stylist' => { 'role' => 'undefined' }, 'Key Makeup Artist' => { 'role' => 'undefined' }, 'Lighting' => { 'role' => 'undefined' }, 'Location Manager' => { 'role' => 'undefined' }, 'Makeup' => { 'role' => 'undefined' }, 'Makeup Artist' => { 'role' => 'undefined' }, 'Makeup Assistant' => { 'role' => 'undefined' }, 'Makeup Department Head' => { 'role' => 'undefined' }, 'Makeup Designer' => { 'role' => 'undefined' }, 'Makeup Supervisor' => { 'role' => 'undefined' }, 'Martial Arts Advisor' => { 'role' => 'undefined' }, 'Martial Arts Choreographer' => { 'role' => 'undefined' }, 'Mixing' => { 'role' => 'undefined' }, 'Music Mixer' => { 'role' => 'undefined' }, 'Music Recordist' => { 'role' => 'undefined' }, 'Original Concept' => { 'role' => 'undefined' }, 'Photographer' => { 'role' => 'undefined' }, 'Post-Production' => { 'role' => 'undefined' }, 'Post-Production Manager' => { 'role' => 'undefined' }, 'Post-Production Supervisor' => { 'role' => 'undefined' }, 'Production Assistant' => { 'role' => 'undefined' }, 'Production Coordinator' => { 'role' => 'undefined' }, 'Production Designer' => { 'role' => 'undefined' }, 'Production Design' => { 'role' => 'undefined' }, 'Production Executive' => { 'role' => 'undefined' }, 'Production Manager' => { 'role' => 'undefined' }, 'Production Sound Editor' => { 'role' => 'undefined' }, 'Production Sound Mixer' => { 'role' => 'undefined' }, 'Production Supervisor' => { 'role' => 'undefined' }, 'Property Master' => { 'role' => 'undefined' }, 'Recording Engineer' => { 'role' => 'undefined' }, 'Recording Supervisor' => { 'role' => 'undefined' }, 'Researcher' => { 'role' => 'undefined' }, 'Scene Designer' => { 'role' => 'undefined' }, 'Scenic Artist' => { 'role' => 'undefined' }, 'Score Mixer' => { 'role' => 'undefined' }, 'Screen Story Writer' => { 'role' => 'undefined' }, 'Script Supervisor' => { 'role' => 'undefined' }, 'Set Construction' => { 'role' => 'undefined' }, 'Set Decoration' => { 'role' => 'undefined' }, 'Set Designer' => { 'role' => 'undefined' }, 'Set Dresser' => { 'role' => 'undefined' }, 'Singer' => { 'role' => 'undefined' }, 'Sound' => { 'role' => 'undefined' }, 'Sound Assistant' => { 'role' => 'undefined' }, 'Sound Consultant' => { 'role' => 'undefined' }, 'Sound Designer' => { 'role' => 'undefined' }, 'Sound Effects' => { 'role' => 'undefined' }, 'Sound Engineer' => { 'role' => 'undefined' }, 'Sound Mixer' => { 'role' => 'undefined' }, 'Sound Recordist' => { 'role' => 'undefined' }, 'Sound Re-Recording Mixer' => { 'role' => 'undefined' }, 'Sound Supervisor' => { 'role' => 'undefined' }, 'Sound Technician' => { 'role' => 'undefined' }, 'Special Effects' => { 'role' => 'undefined' }, 'Special Effects Coordinator' => { 'role' => 'undefined' }, 'Special Effects Supervisor' => { 'role' => 'undefined' }, 'Special Makeup Effects Artist' => { 'role' => 'undefined' }, 'Special Makeup Effects' => { 'role' => 'undefined' }, 'Special Photographic Effects' => { 'role' => 'undefined' }, 'Storyboard' => { 'role' => 'undefined' }, 'Storyboard Artist' => { 'role' => 'undefined' }, 'Storyboard Supervisor' => { 'role' => 'undefined' }, 'Stunt Choreographer' => { 'role' => 'undefined' }, 'Stunt Coordinator' => { 'role' => 'undefined' }, 'Stunt Supervisor' => { 'role' => 'undefined' }, 'Stunts' => { 'role' => 'undefined' }, 'Supervising Animator' => { 'role' => 'undefined' }, 'Unit Manager' => { 'role' => 'undefined' }, 'Unit Production' => { 'role' => 'undefined' }, 'Unit Production Manager' => { 'role' => 'undefined' }, 'Visual Effects' => { 'role' => 'undefined' }, 'Visual Effects Coordinator' => { 'role' => 'undefined' }, 'Visual Effects Designer' => { 'role' => 'undefined' }, 'Visual Effects Supervisor' => { 'role' => 'undefined' }, 'Voice-Over' => { 'role' => 'undefined' }, 'Wardrobe' => { 'role' => 'undefined' } }; my $w = XML::Writer->new( 'ENCODING' => 'UTF-8', 'DATA_MODE' => 1, 'DATA_INDENT' => 1, 'UNSAFE' => (!$debug) ); $w->xmlDecl('UTF-8'); $w->comment($SD_COMMENT); $w->doctype( 'tv', undef, 'xmltv.dtd' ); $w->startTag('tv', 'generator-info-name' => $SCRIPT_NAME, 'generator-info-url' => $SCRIPT_URL, 'source-info-name' => $SD_DESC, 'source-info-url' => $SD_SITEURL ); my $channelsWritten = channelWriter($conf->{'lineup'}, $w); print (STDERR " $channelsWritten channels processed\n") if (!$quiet); # # Select out schedules/programs # # This select has (the only) sqlite specific SQL in it # to deal with datetime processing. Perl performance # for operating on datetime is poor (it is arguably # reasonable given the complexity of datatime operations) # so we let sqlite do the work for us. It is not # desirable, but when you get back 40-50% of the cpu # it is a necessary compromise # $sql = "select schedules.station, schedules.starttime, schedules.duration, schedules.program, schedules.details, programs.details, strftime('%Y%m%d%H%M%S', schedules.starttime), strftime('%Y%m%d%H%M%S', datetime(schedules.starttime, '+' || schedules.duration || ' seconds')), stations.details, supplemental.details from schedules as schedules left join programs as programs on programs.program = schedules.program left join stations as stations on stations.station = schedules.station left join programs as supplemental on programs.program_supplemental = supplemental.program where schedules.station in (select distinct stations.station from stations as stations where stations.station in ( select distinct channels.station from channels as channels where channels.lineup in ( " . join(', ', ('?') x scalar(@{$conf->{'lineup'}})) . " ) and channels.selected = 1)) AND schedules.day >= ? and schedules.day < ? order by schedules.station, schedules.starttime"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } # # Determine our start and end days # my $startDay = DateTime->now(time_zone => 'UTC')->add(days => $opt->{'offset'}); my $endDay = DateTime->now(time_zone => 'UTC')->add(days => $opt->{'offset'})->add(days => $opt->{'days'}); $param = 1; for (my $i=0; $i < scalar(@{$conf->{'lineup'}}); $i++) { $sth->bind_param( $param, @{$conf->{'lineup'}}[$i], SQL_VARCHAR); $param++; } $sth->bind_param( $param, DateTime::Format::SQLite->format_date($startDay), SQL_DATE ); $param++; $sth->bind_param( $param, DateTime::Format::SQLite->format_date($endDay), SQL_DATE ); $param++; $sth->execute(); if ($sth->err()) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, undef, SQL_VARCHAR ); $sth->bind_col( 2, undef, SQL_DATETIME ); $sth->bind_col( 3, undef, SQL_INTEGER ); $sth->bind_col( 4, undef, SQL_VARCHAR ); $sth->bind_col( 5, undef, SQL_VARCHAR ); $sth->bind_col( 6, undef, SQL_VARCHAR ); $sth->bind_col( 7, undef, SQL_VARCHAR ); $sth->bind_col( 8, undef, SQL_VARCHAR ); $sth->bind_col( 9, undef, SQL_VARCHAR ); $sth->bind_col(10, undef, SQL_VARCHAR ); my $programsWritten = 0; while (my $r = $sth->fetchrow_arrayref()) { my $sID = $r->[0]; # Note that we should legitmately parse the datetime here, but # the performance absolutely sucks, so we let sqlite do this # my $startTime = DateTime::Format::SQLite->parse_datetime($r->[1]); # my $endTime = $startTime->clone()->add(seconds => $r->[2]); my $pID = $r->[3]; my $scheduleDetails = eval { $JSON->decode($r->[4]) } || {}; my $programDetails = eval { $JSON->decode($r->[5]) } || {}; my $stationDetails = eval { $JSON->decode($r->[8]) } || {};; my $supplementalDetails = eval { $JSON->decode($r->[9]) } || {};; $w->startTag('programme', 'channel' => generateRFC2838($sID), 'start' => "$r->[6] +0000", 'stop' => "$r->[7] +0000"); # Mandatory title array should (must?) contain title120, but may # contain others? if (defined($programDetails->{'titles'})) { foreach my $title(@{$programDetails->{'titles'}}) { if (defined($title->{'title120'})) { $w->dataElement('title', $title->{'title120'}); } } } elsif (defined($supplementalDetails->{'titles'})) { foreach my $title(@{$supplementalDetails->{'titles'}}) { if (defined($title->{'title120'})) { $w->dataElement('title', $title->{'title120'}); } } } if (defined($programDetails->{'episodeTitle150'})) { $w->dataElement('sub-title', $programDetails->{'episodeTitle150'}); } elsif (defined($supplementalDetails->{'episodeTitle150'})) { $w->dataElement('sub-title', $supplementalDetails->{'episodeTitle150'}); } # Choose the "best" (i.e. longer) description if available if (defined($programDetails->{'descriptions'}->{'description1000'})) { foreach my $d(@{$programDetails->{'descriptions'}->{'description1000'}}) { my $lang = $d->{'descriptionLanguage'}; my $desc = $d->{'description'}; next if ((!defined($lang) || (!defined($desc)))); $w->dataElement('desc', $desc, 'lang' => $lang); } } elsif (defined($programDetails->{'descriptions'}->{'description100'})) { foreach my $d(@{$programDetails->{'descriptions'}->{'description100'}}) { my $lang = $d->{'descriptionLanguage'}; my $desc = $d->{'description'}; next if ((!defined($lang) || (!defined($desc)))); $w->dataElement('desc', $desc, 'lang' => $lang); } } elsif (defined($supplementalDetails->{'descriptions'}->{'description1000'})) { foreach my $d(@{$supplementalDetails->{'descriptions'}->{'description1000'}}) { my $lang = $d->{'descriptionLanguage'}; my $desc = $d->{'description'}; next if ((!defined($lang) || (!defined($desc)))); $w->dataElement('desc', $desc, 'lang' => $lang); } } elsif (defined($supplementalDetails->{'descriptions'}->{'description100'})) { foreach my $d(@{$supplementalDetails->{'descriptions'}->{'description100'}}) { my $lang = $d->{'descriptionLanguage'}; my $desc = $d->{'description'}; next if ((!defined($lang) || (!defined($desc)))); $w->dataElement('desc', $desc, 'lang' => $lang); } } # XMLTV roles for this program my $roles = { 'director' => {}, 'actor' => {}, 'writer' => {}, 'adapter' => {}, 'producer' => {}, 'composer' => {}, 'editor' => {}, 'presenter' => {}, 'commentator' => {}, 'guest' => {} }; # XMLTV dtd requires us to collect the various cast and # crew items in order to process in the dtd order. In # addition, the dtd specifies that the order of the # roles is meaningful (first billing should come first). # All too often the supplemental (SHow) has more # detailed cast/crew data that the program (EPisode) # itself, and in particular, the character. if (defined($programDetails->{'cast'})) { foreach my $cast(@{$programDetails->{'cast'}}) { my $castAttributes = $castMap->{$cast->{'role'}}; next if (!defined($castAttributes)); my $role = $castAttributes->{'role'}; next if (!defined($role)); addRole($roles, $role, $cast->{'name'}, $cast->{'billingOrder'}, $cast->{'characterName'}, $castAttributes); } } if (defined($programDetails->{'crew'})) { foreach my $crew(@{$programDetails->{'crew'}}) { my $crewAttributes = $crewMap->{$crew->{'role'}}; next if (!defined($crewAttributes)); my $role = $crewAttributes->{'role'}; next if (!defined($role)); addRole($roles, $role, $crew->{'name'}, $crew->{'billingOrder'}, undef, $crewAttributes); } } if (defined($supplementalDetails->{'cast'})) { foreach my $cast(@{$supplementalDetails->{'cast'}}) { my $castAttributes = $castMap->{$cast->{'role'}}; next if (!defined($castAttributes)); my $role = $castAttributes->{'role'}; next if (!defined($role)); addRole($roles, $role, $cast->{'name'}, '100' . $cast->{'billingOrder'}, $cast->{'characterName'}, $castAttributes); } } if (defined($supplementalDetails->{'crew'})) { foreach my $crew(@{$supplementalDetails->{'crew'}}) { my $crewAttributes = $crewMap->{$crew->{'role'}}; next if (!defined($crewAttributes)); my $role = $crewAttributes->{'role'}; next if (!defined($role)); addRole($roles, $role, $crew->{'name'}, '100' . $crew->{'billingOrder'}, undef, $crewAttributes); } } $w->startTag('credits'); foreach my $role('director', 'actor', 'writer', 'adapter', 'producer', 'composer', 'editor', 'presenter', 'commentator', 'guest') { foreach my $person(sort {$roles->{$role}->{$a}->{'order'} <=> $roles->{$role}->{$b}->{'order'}} (keys %{$roles->{$role}})) { my $attributes = $roles->{$role}->{$person}->{'attributes'}; # Only actors have characters and guest/self attributes if ($role eq 'actor') { my $extendedAttributes = {}; if (($attributes->{'guest'}) && (!exists($GRABBER_FIXUPS->{'NO_ACTOR_GUEST_ATTRIBUTE'}))) { $extendedAttributes->{'guest'} = 'yes'; } if (0 == scalar(@{$roles->{$role}->{$person}->{'character'}})) { if ($attributes->{'self'}) { $extendedAttributes->{'role'} = 'Self'; } $w->dataElement($role, $person, %{$extendedAttributes}); } else { foreach my $character(@{$roles->{$role}->{$person}->{'character'}}) { $w->dataElement($role, $person, 'role' => $character, %{$extendedAttributes}); } } } else { $w->dataElement($role, $person); } } } $w->endTag('credits'); # Only movies (likely) have a date if (defined($programDetails->{'movie'}->{'year'})) { $w->dataElement('date', $programDetails->{'movie'}->{'year'}); } elsif (defined($supplementalDetails->{'movie'}->{'year'})) { $w->dataElement('date', $supplementalDetails->{'movie'}->{'year'}); } if (defined($conf->{'mythtv-categories'}->[0]) && ($conf->{'mythtv-categories'}->[0] eq 'enabled')) { # For MythTV, we need to specify the first category # in the xmltv file as one of movie, series, sports, # or tvshow. We can derive that from the entityType. # If the station is a radio station, we do not add # tvshow, but add radio (because the first category # is not processed in the usual way). my $radioStation = 0; if (defined($stationDetails->{'isRadioStation'})) { $radioStation = $stationDetails->{'isRadioStation'}; } if (defined($programDetails->{'entityType'})) { my $entityType = $programDetails->{'entityType'}; if ($entityType eq 'Movie') { $w->dataElement('category', 'movie'); } elsif ($entityType eq 'Sports') { $w->dataElement('category', 'sports'); } elsif ($entityType eq 'Episode') { $w->dataElement('category', 'series'); } else # Should be Show { my $showType = ''; if (defined($programDetails->{'showType'})) { $showType = $programDetails->{'showType'}; } elsif (defined($supplementalDetails->{'showType'})) { $showType = $supplementalDetails->{'showType'}; } if (($showType eq 'Feature Film') || ($showType eq 'Short Film') || ($showType eq 'TV Movie')) { $w->dataElement('category', 'movie'); } elsif (($showType eq 'Sports event') || ($showType eq 'Sports non-event')) { $w->dataElement('category', 'sports'); } elsif (($showType eq 'Series') || ($showType eq 'Miniseries')) { $w->dataElement('category', 'series'); } else { if ($radioStation) { $w->dataElement('category', 'radio'); } else { $w->dataElement('category', 'tvshow'); } } } } else # entityType is supposed to be manditory, but.... { if ($radioStation) { $w->dataElement('category', 'radio'); } else { $w->dataElement('category', 'tvshow'); } } } # XMLTV categories are somewhat arbitrary. We collect the # genres, showType, and entityType as categories. There is # no order implication in the XMLTV dtd for categories, # but at least one well known app cares about the order, # so we try to be accomodating, and priorize program # over supplemental data. my $categories = {}; my $categorynum = 0; if (defined($programDetails->{'genres'})) { foreach my $genre(@{$programDetails->{'genres'}}) { $categories->{$genre} = $categorynum++ if (!defined($categories->{$genre})); } } if (defined($supplementalDetails->{'genres'})) { foreach my $genre(@{$supplementalDetails->{'genres'}}) { $categories->{$genre} = $categorynum++ if (!defined($categories->{$genre})); } } if (defined($programDetails->{'showType'})) { $categories->{$programDetails->{'showType'}} = $categorynum++ if (!defined($categories->{$programDetails->{'showType'}})); } if (defined($supplementalDetails->{'showType'})) { $categories->{$supplementalDetails->{'showType'}} = $categorynum++ if (!defined($categories->{$supplementalDetails->{'showType'}})); } if (defined($programDetails->{'entityType'})) { $categories->{$programDetails->{'entityType'}} = $categorynum++ if (!defined($categories->{$programDetails->{'entityType'}})); } if (defined($supplementalDetails->{'entityType'})) { $categories->{$supplementalDetails->{'entityType'}} = $categorynum++ if (!defined($categories->{$supplementalDetails->{'entityType'}})); } foreach my $category (sort {$categories->{$a} <=> $categories->{$b}} (keys %{$categories})) { $w->dataElement('category', $category); } # MythTV does not currently have a concept of keywords, # so this is output is likely meaningless. Perhaps a # future enhancement (a new "programkeywords" table?), # or keywords should be added as categories? Some of # the keywords might make usable categories. There # is no order implication for keywords in the XMLTV dtd. my $keywords = {}; if (defined($programDetails->{'keyWords'})) { foreach my $keyCat(keys %{$programDetails->{'keyWords'}}) { foreach my $kw(@{$programDetails->{'keyWords'}->{$keyCat}}) { $keywords->{$kw} = 1 } } } if (defined($supplementalDetails->{'keyWords'})) { foreach my $keyCat(keys %{$supplementalDetails->{'keyWords'}}) { foreach my $kw(@{$supplementalDetails->{'keyWords'}->{$keyCat}}) { $keywords->{$kw} = 1 } } } foreach my $keyword (sort keys %{$keywords}) { $w->dataElement('keyword', $keyword); } if (defined($programDetails->{'duration'})) { $w->dataElement('length', $programDetails->{'duration'}, 'units' => 'seconds'); } elsif (defined($supplementalDetails->{'duration'})) { $w->dataElement('length', $supplementalDetails->{'duration'}, 'units' => 'seconds'); } if (defined($programDetails->{'episodeImage'}) && defined($programDetails->{'episodeImage'}->{'uri'})) { my $url = $SD->uriResolve($programDetails->{'episodeImage'}->{'uri'}, '/image'); if (defined($programDetails->{'episodeImage'}->{'width'}) && defined($programDetails->{'episodeImage'}->{'height'})) { $w->emptyTag('icon', 'src' => $url, 'width' => $programDetails->{'episodeImage'}->{'width'}, 'height' => $programDetails->{'episodeImage'}->{'height'}); } else { $w->emptyTag('icon', 'src' => $url); } } elsif (defined($supplementalDetails->{'episodeImage'}) && defined($supplementalDetails->{'episodeImage'}->{'uri'})) { my $url = $SD->uriResolve($supplementalDetails->{'episodeImage'}->{'uri'}, '/image'); if (defined($supplementalDetails->{'episodeImage'}->{'width'}) && defined($supplementalDetails->{'episodeImage'}->{'height'})) { $w->emptyTag('icon', 'src' => $url, 'width' => $supplementalDetails->{'episodeImage'}->{'width'}, 'height' => $supplementalDetails->{'episodeImage'}->{'height'}); } else { $w->emptyTag('icon', 'src' => $url); } } if (defined($programDetails->{'officialURL'})) { $w->dataElement('url', $programDetails->{'officialURL'}); } elsif (defined($supplementalDetails->{'officialURL'})) { $w->dataElement('url', $supplementalDetails->{'officialURL'}); } my $prodid = $pID; if (length($prodid) == 14) { $prodid = substr($prodid, 0, 10) . '.' . substr($prodid, 10, 4); $w->dataElement('episode-num', $prodid, 'system' => 'dd_progid' ); } # Season/Episode numbering is "special" as SHows and # EPisodes use slightly different interpretations of # the exact same terms. my $season = ''; my $episode = ''; my $part = ''; my $programEpisode; my $programTotalEpisodes; my $programSeason; my $showSeason; my $showEpisode; my $showTotalEpisodes; my $showTotalSeasons; my $showingPart; my $showingTotalParts; my $TVDBseriesID; my $TVDBepisodeID; my $TVmazeSeason; my $TVmazeEpisode; my $TVmazeURL; if (defined($supplementalDetails->{'metadata'})) { foreach my $meta(@{$supplementalDetails->{'metadata'}}) { if (defined($meta->{'Gracenote'})) { if ((defined($supplementalDetails->{'programID'})) && ('SH' eq substr($supplementalDetails->{'programID'}, 0, 2))) { $showSeason = $meta->{'Gracenote'}->{'season'}; $showEpisode = $meta->{'Gracenote'}->{'episode'}; $showTotalSeasons = $meta->{'Gracenote'}->{'totalSeasons'}; $showTotalEpisodes = $meta->{'Gracenote'}->{'totalEpisodes'}; } } if (defined($meta->{'TheTVDB'})) { if ((defined($meta->{'TheTVDB'}->{'episodeID'})) && (looks_like_number($meta->{'TheTVDB'}->{'episodeID'})) && ($meta->{'TheTVDB'}->{'episodeID'} > 0)) { $TVDBepisodeID = 0 + $meta->{'TheTVDB'}->{'episodeID'}; } if ((defined($meta->{'TheTVDB'}->{'seriesID'})) && (looks_like_number($meta->{'TheTVDB'}->{'seriesID'})) && ($meta->{'TheTVDB'}->{'seriesID'} > 0)) { $TVDBseriesID = 0 + $meta->{'TheTVDB'}->{'seriesID'}; } } if (defined($meta->{'TVmaze'})) { if ((defined($meta->{'TVmaze'}->{'episode'})) && (looks_like_number($meta->{'TVmaze'}->{'episode'})) && ($meta->{'TVmaze'}->{'episode'} > 0)) { $TVmazeEpisode = 0 + $meta->{'TVmaze'}->{'episode'}; } if ((defined($meta->{'TVmaze'}->{'season'})) && (looks_like_number($meta->{'TVmaze'}->{'season'})) && ($meta->{'TVmaze'}->{'season'} > 0)) { $TVmazeSeason = 0 + $meta->{'TVmaze'}->{'season'}; } if ((defined($meta->{'TVmaze'}->{'url'})) && ($meta->{'TVmaze'}->{'url'} ne '')) { $TVmazeURL = $meta->{'TVmaze'}->{'url'}; } } } } if (defined($programDetails->{'metadata'})) { foreach my $meta(@{$programDetails->{'metadata'}}) { if (defined($meta->{'Gracenote'})) { if (substr($pID, 0, 2) eq 'SH') { $showSeason = $meta->{'Gracenote'}->{'season'}; $showEpisode = $meta->{'Gracenote'}->{'episode'}; $showTotalSeasons = $meta->{'Gracenote'}->{'totalSeasons'}; $showTotalEpisodes = $meta->{'Gracenote'}->{'totalEpisodes'}; } else { $programSeason = $meta->{'Gracenote'}->{'season'}; $programEpisode = $meta->{'Gracenote'}->{'episode'}; $programTotalEpisodes = $meta->{'Gracenote'}->{'totalEpisodes'}; } } if (defined($meta->{'TheTVDB'})) { if ((defined($meta->{'TheTVDB'}->{'episodeID'})) && (looks_like_number($meta->{'TheTVDB'}->{'episodeID'})) && ($meta->{'TheTVDB'}->{'episodeID'} > 0)) { $TVDBepisodeID = 0 + $meta->{'TheTVDB'}->{'episodeID'}; } if ((defined($meta->{'TheTVDB'}->{'seriesID'})) && (looks_like_number($meta->{'TheTVDB'}->{'seriesID'})) && ($meta->{'TheTVDB'}->{'seriesID'} > 0)) { $TVDBseriesID = 0 + $meta->{'TheTVDB'}->{'seriesID'}; } } if (defined($meta->{'TVmaze'})) { if ((defined($meta->{'TVmaze'}->{'episode'})) && (looks_like_number($meta->{'TVmaze'}->{'episode'})) && ($meta->{'TVmaze'}->{'episode'} > 0)) { $TVmazeEpisode = 0 + $meta->{'TVmaze'}->{'episode'}; } if ((defined($meta->{'TVmaze'}->{'season'})) && (looks_like_number($meta->{'TVmaze'}->{'season'})) && ($meta->{'TVmaze'}->{'season'} > 0)) { $TVmazeSeason = 0 + $meta->{'TVmaze'}->{'season'}; } if ((defined($meta->{'TVmaze'}->{'url'})) && ($meta->{'TVmaze'}->{'url'} ne '')) { $TVmazeURL = $meta->{'TVmaze'}->{'url'}; } } } } if (defined($scheduleDetails->{'multipart'})) { $showingPart = $scheduleDetails->{'multipart'}->{'partNumber'}; $showingTotalParts = $scheduleDetails->{'multipart'}->{'totalParts'}; } $programSeason = $showSeason if (!defined($programSeason)); $programEpisode = $showEpisode if (!defined($programEpisode)); $showTotalSeasons = undef if (exists($GRABBER_FIXUPS->{'NO_XMLTV_NS_TOTAL_SEASONS'})); $season = generateXMLTV_NS($programSeason, $showTotalSeasons); $episode = generateXMLTV_NS($programEpisode, $programTotalEpisodes); $part = generateXMLTV_NS($showingPart, $showingTotalParts); if (($season ne '') || ($episode ne '') || ($part ne '')) { $w->dataElement('episode-num', " $season . $episode . $part ", 'system' => 'xmltv_ns'); } # # Potentionally need to know if this is a new showing # in the extra metadata section, and again for the # previously shown determination # my $newShowing = 0; $newShowing = $scheduleDetails->{'new'} if (defined($scheduleDetails->{'new'})); # # Emit 3rdparty metadata if not disabled # if ((!defined($conf->{'3rdparty-metadata'}->[0])) || ($conf->{'3rdparty-metadata'}->[0] ne 'disabled')) { if (defined($TVDBepisodeID)) { $w->dataElement('episode-num', "episode/" . $TVDBepisodeID, 'system' => 'thetvdb.com'); } if (defined($TVDBseriesID)) { $w->dataElement('episode-num', "series/" . $TVDBseriesID, 'system' => 'thetvdb.com'); } if (defined($TVmazeEpisode)) { $w->dataElement('episode-num', "episode/" . $TVmazeEpisode, 'system' => 'tvmaze.com'); } if (defined($TVmazeSeason)) { $w->dataElement('episode-num', "series/" . $TVmazeSeason, 'system' => 'tvmaze.com'); } if (defined($TVmazeURL)) { $w->dataElement('episode-num', "url/" . $TVmazeURL, 'system' => 'tvmaze.com'); } $w->dataElement('episode-num', "programID/$pID", 'system' => 'schedulesdirect.org' ); if (defined($programSeason)) { $w->dataElement('episode-num', "series/" . $programSeason, 'system' => 'schedulesdirect.org'); } if (defined($programEpisode)) { $w->dataElement('episode-num', "episode/" . $programEpisode, 'system' => 'schedulesdirect.org'); } if (defined($programDetails->{'resourceID'})) { $w->dataElement('episode-num', "resourceID/$programDetails->{'resourceID'}", 'system' => 'schedulesdirect.org'); } elsif (defined($supplementalDetails->{'resourceID'})) { $w->dataElement('episode-num', "resourceID/$supplementalDetails->{'resourceID'}", 'system' => 'schedulesdirect.org'); } if ($newShowing) { $w->dataElement('episode-num', "newEpisode/true", 'system' => 'schedulesdirect.org'); } if (defined($programDetails->{'originalAirDate'})) { my $originalAirDate = $programDetails->{'originalAirDate'}; my $d = substr($originalAirDate, 0, 4) . substr($originalAirDate, 5, 2) . substr($originalAirDate, 8, 2) . ' +0000'; $w->dataElement('episode-num', "originalAirDate/$d", 'system' => 'schedulesdirect.org'); } if (defined($programDetails->{'eventDetails'}) && (ref($programDetails->{'eventDetails'}) eq 'HASH')) { if (defined($programDetails->{'eventDetails'}->{'venue100'})) { $w->dataElement('episode-num', "eventVenue/$programDetails->{'eventDetails'}->{'venue100'}", 'system' => 'schedulesdirect.org'); } if (defined($programDetails->{'eventDetails'}->{'gameDate'})) { my $gameDate = $programDetails->{'eventDetails'}->{'gameDate'}; my $d = substr($gameDate, 0, 4) . substr($gameDate, 5, 2) . substr($gameDate, 8, 2) . ' +0000'; $w->dataElement('episode-num', "eventDate/$d", 'system' => 'schedulesdirect.org'); } if (defined($programDetails->{'eventDetails'}->{'teams'}) && (ref($programDetails->{'eventDetails'}->{'teams'}) eq 'ARRAY')) { foreach my $t(@{$programDetails->{'eventDetails'}->{'teams'}}) { if (ref($t) eq 'HASH') { if (defined($t->{'name'})) { if ((defined($t->{'isHome'})) && ($t->{'isHome'})) { $w->dataElement('episode-num', "eventHomeTeam/$t->{'name'}", 'system' => 'schedulesdirect.org'); } else { $w->dataElement('episode-num', "eventTeam/$t->{'name'}", 'system' => 'schedulesdirect.org'); } } } } } if (defined($programDetails->{'eventDetails'}->{'season'}) && (ref($programDetails->{'eventDetails'}->{'season'}) eq 'HASH')) { if (defined($programDetails->{'eventDetails'}->{'season'}->{'season'})) { $w->dataElement('episode-num', "eventSeason/$programDetails->{'eventDetails'}->{'season'}->{'season'}", 'system' => 'schedulesdirect.org'); } if (defined($programDetails->{'eventDetails'}->{'season'}->{'type'})) { $w->dataElement('episode-num', "eventSeasonType/$programDetails->{'eventDetails'}->{'season'}->{'type'}", 'system' => 'schedulesdirect.org'); } } } } if (defined($scheduleDetails->{'videoProperties'})) { my $videoHDTV = 0; my $videoUHDTV = 0; my $videoHDR = 0; foreach my $videoProperty(@{$scheduleDetails->{'videoProperties'}}) { $videoHDTV = 1 if ($videoProperty eq 'hdtv'); $videoUHDTV = 1 if ($videoProperty eq 'uhdtv'); $videoHDR = 1 if ($videoProperty eq 'hdr'); } if ($videoHDTV || $videoUHDTV || $videoHDR) { $w->startTag('video'); if ((defined($conf->{'extended-video-quality'}->[0])) && ($conf->{'extended-video-quality'}->[0] eq 'enabled')) { if ($videoHDR) # HDR is UHDTV, but not all UHDTV is HDR { $w->dataElement('quality', 'HDR'); } elsif ($videoUHDTV) { $w->dataElement('quality', 'UHDTV'); } else { $w->dataElement('quality', 'HDTV'); } } else { $w->dataElement('quality', 'HDTV'); } $w->endTag('video'); } } # XMLTV only supports one audio quality report, so we try # to determine the best available to report. We also need # to collect the closed caption information for future # reporting. my $audioHasCC = 0; # Need to carry forward my $audioHasSubtitle = 0; my $audioHasDVS = 0; my $audioHasSigned = 0; if (defined($scheduleDetails->{'audioProperties'})) { # Ugly because dtd only allows one type, and source data # may have many (in any order) my $audioHasDolbySurround = 0; my $audioHasDolby = 0; my $audioHasStereo = 0; foreach my $audioProperty(@{$scheduleDetails->{'audioProperties'}}) { $audioHasDolbySurround = 1 if ($audioProperty eq 'DD 5.1'); $audioHasDolby = 1 if ($audioProperty eq 'Dolby'); $audioHasStereo = 1 if ($audioProperty eq 'stereo'); $audioHasCC = 1 if ($audioProperty eq 'cc'); $audioHasSubtitle = 1 if ($audioProperty eq 'subtitled'); $audioHasDVS = 1 if ($audioProperty eq 'dvs'); $audioHasSigned = 1 if ($audioProperty eq 'signed'); } if ($audioHasDolbySurround || $audioHasDolby || $audioHasStereo) { $w->startTag('audio'); if ($audioHasDolbySurround) { $w->dataElement('stereo', 'dolby digital'); } elsif ($audioHasDolby) { $w->dataElement('stereo', 'dolby'); } elsif ($audioHasStereo) { $w->dataElement('stereo', 'stereo'); } $w->endTag('audio'); } } # If the schedule has marked this as a new showing, do not add in # any previously-shown indication. Don't use supplemental data for # originalAirDate since generic data is not relevant for this showing. # Date transformation occurs because XMLTV uses their standardized # dates, while Schedules Direct uses YYYY-MM-DD if (!$newShowing) { if (defined($programDetails->{'originalAirDate'})) { my $originalAirDate = $programDetails->{'originalAirDate'}; my $offset = ' +0000'; $offset = '' if (exists($GRABBER_FIXUPS->{'NO_PREVIOUSLY_SHOWN_ZONE_OFFSET'})); my $start = substr($originalAirDate, 0, 4) . substr($originalAirDate, 5, 2) . substr($originalAirDate, 8, 2) . $offset; $w->emptyTag('previously-shown', start => $start); } else { $w->emptyTag('previously-shown'); } } # XMLTV premiere/last-chance is sort of arbitrarily # defined, so we decide on our own mapping (while # season finale may not be a last-chance, since # in the US every season finale may be a series # finale (no renewal before its time) we just treat # it as the last-chance). if (defined($scheduleDetails->{'premiere'})) { my $premiere = $scheduleDetails->{'premiere'}; $w->emptyTag('premiere') if ($premiere); } elsif (defined($scheduleDetails->{'isPremiereOrFinale'})) { my $premiereType = $scheduleDetails->{'isPremiereOrFinale'}; if (($premiereType eq 'Series Premiere') || ($premiereType eq 'Season Premiere')) { $w->dataElement('premiere', $premiereType); } if (($premiereType eq 'Series Finale') || ($premiereType eq 'Season Finale')) { $w->dataElement('last-chance', $premiereType); } } # Carried forward from audio eval to match DTD $w->emptyTag('subtitles', 'type' => 'teletext') if ($audioHasCC); $w->emptyTag('subtitles', 'type' => 'onscreen') if ($audioHasSubtitle); $w->emptyTag('subtitles', 'type' => 'deaf-signed') if ($audioHasSigned); # XMLTV ratings (and the system) are arbitrary values, and # have no implied priority or order. However, for # Schedules Direct data there maybe different ratings for # a body (one in the show/series, and one in the program, # so we store only the more specific if multiple exist. # We remap the rating agency to the MythTV standard, as # it is as good of a standard as anything else, and makes # importing the data much easier. A value add for # Schedules Direct is that they also provide contentAdvisory # markings at the top level. Gracenote (the upstream) is # adding advisories to the individual ratings body in the # contentRating, which is usually correct, as any advisory # mostly only apply to the agency like VCHIP ratings such # as TV-14, with an advisory of Dialog (VCHIP is one of # the few ratings with such advisory modifiers). That # said, various advisories have been seen in other ratings # systems occasionally, and some movies appear to have # advisories marked even without a VCHIP rating applied. # Also note that an advisory is rating specific. A dialog # that causes an advisory for a TV-PG rating might very # well not result in an advisory for a TV-14 rating where # the more mature audiences can handle the specific dialog. # This is a work in progress, and currently there are # anomalies in the data. As part of the transition from # where we were to where we will eventually be, we will at # this time just collect all possible advisories and add # them to the eventual advisory ratings. This will likely # change as Gracenote (and Schedules Direct) modify their # approach and get their data more consistent. my $ratings = {}; my $advisories = {}; # Supplemental can have generic show ratings if (defined($supplementalDetails->{'contentRating'})) { foreach my $rating(@{$supplementalDetails->{'contentRating'}}) { my $body = $rating->{'body'}; my $code = $rating->{'code'}; ($body, $code) = mapRatingAgency($body, $code); if (defined($body) && defined($code) && ($code ne '')) { $ratings->{$body} = $code; } if ((defined($rating->{'contentAdvisory'})) && (ref($rating->{'contentAdvisory'}) eq 'ARRAY')) { foreach my $advisory(@{$rating->{'contentAdvisory'}}) { $advisories->{$advisory} = 1; } } } } if (defined($supplementalDetails->{'contentAdvisory'})) { foreach my $advisory(@{$supplementalDetails->{'contentAdvisory'}}) { $advisories->{$advisory} = 1; } } # Programs can have rating for this specific program if (defined($programDetails->{'contentRating'})) { foreach my $rating(@{$programDetails->{'contentRating'}}) { my $body = $rating->{'body'}; my $code = $rating->{'code'}; ($body, $code) = mapRatingAgency($body, $code); if (defined($body) && defined($code) && ($code ne '')) { $ratings->{$body} = $code; } if ((defined($rating->{'contentAdvisory'})) && (ref($rating->{'contentAdvisory'}) eq 'ARRAY')) { foreach my $advisory(@{$rating->{'contentAdvisory'}}) { $advisories->{$advisory} = 1; } } } } if (defined($programDetails->{'contentAdvisory'})) { foreach my $advisory(@{$programDetails->{'contentAdvisory'}}) { $advisories->{$advisory} = 1; } } # Write out the the collected ratings and advisories foreach my $rating(sort keys %{$ratings}) { $w->startTag('rating', 'system' => $rating); $w->dataElement('value', $ratings->{$rating}); $w->endTag('rating'); } foreach my $advisory(sort keys %{$advisories}) { $w->startTag('rating', 'system' => 'advisory'); $w->dataElement('value', $advisory); $w->endTag('rating') } # XMLTV star-rating starts from zero (so if rating agency is 1-4, # we adjust the reported values to be from 0-3. my $starRatings = {}; if (defined($supplementalDetails->{'movie'}->{'qualityRating'})) { foreach my $quality(@{$supplementalDetails->{'movie'}->{'qualityRating'}}) { my $body = $quality->{'ratingsBody'}; my $min = $quality->{'minRating'}; my $max = $quality->{'maxRating'}; my $incr = $quality->{'increment'}; my $rating = $quality->{'rating'}; if (defined($body) && defined($min) && defined($max) && defined($incr) && defined($rating) && looks_like_number($min) && looks_like_number($max) && looks_like_number($incr) && looks_like_number($rating)) { $min = 0 + $min; $max = 0 + $max; $incr = 0 + $incr; $rating = 0 + $rating; $rating = $min if ($rating < $min); $rating = $max if ($rating > $max); my $adjustedRating = ($rating - $min); my $adjustedMax = ($max - $min); $starRatings->{$body} = "$adjustedRating/$adjustedMax"; } } } if (defined($programDetails->{'movie'}->{'qualityRating'})) { foreach my $quality(@{$programDetails->{'movie'}->{'qualityRating'}}) { my $body = $quality->{'ratingsBody'}; my $min = $quality->{'minRating'}; my $max = $quality->{'maxRating'}; my $incr = $quality->{'increment'}; my $rating = $quality->{'rating'}; if (defined($body) && defined($min) && defined($max) && defined($incr) && defined($rating) && looks_like_number($min) && looks_like_number($max) && looks_like_number($incr) && looks_like_number($rating)) { $min = 0 + $min; $max = 0 + $max; $incr = 0 + $incr; $rating = 0 + $rating; $rating = $min if ($rating < $min); $rating = $max if ($rating > $max); my $adjustedRating = ($rating - $min); my $adjustedMax = ($max - $min); $starRatings->{$body} = "$adjustedRating/$adjustedMax"; } } } foreach my $body(sort keys %{$starRatings}) { $w->startTag('star-rating', 'system' => $body); $w->dataElement('value', $starRatings->{$body}); $w->endTag('star-rating'); } $w->endTag('programme'); $programsWritten++; } $DBH->commit(); undef $sth; print (STDERR " $programsWritten program schedules processed\n") if (!$quiet); $w->endTag('tv'); $w->end(); # # Our work here is done # finalize: print (STDERR "Pruning the local database\n") if (!$quiet); DB_prune(); exit(0); # # configureGrabber # # Perform the configure function for XMLTV # # NOTE: While this grabber is (technically) apiconfig # compliant, one must run (outside of --configure) # this script with the --manage-lineups option to # create the local database with the username and # password hash, and to add/delete lineups from the # Schedules Direct account. # # NOTE: We do not utilze the "select-channels" functionality # in XMLTV, because it addresses the (actual) selection # of "stations", and not "channels". A "station" is a # programming entity which has a schedule of programs. # A "channel" is a technical means of delivering a # particular "station". Typically, in the real world, # many "channels" deliver the same "station". # # Input: # stage - the "stage" for configure # conf - the (current) conf hash # Output: # result - the xml configure string # sub configureGrabber { my ($stage, $conf, undef) = @_; my $result; my $writer = XMLTV::Configure::Writer->new( OUTPUT => \$result, grabber => $SCRIPT_NAME, encoding => 'iso-8859-1' ); $writer->start ( { grabber => $SCRIPT_NAME } ); if ($stage eq 'start') { $writer->write_string ( { id => 'database', title => [ [ 'Database for Schedules Direct EPG', 'en' ] ], description => [ [ "$SCRIPT_NAME uses a local database for downloaded EPG data. Please specify the database name created via $SCRIPT_NAME --manage-lineups", 'en' ] ], default => File::HomeDir->my_home . "/.xmltv/SchedulesDirect.DB", } ); $writer->end('select-lineup'); } elsif ($stage eq 'select-lineup') { DB_open($conf->{'database'}->[0]); SD_login(); # Login SD_downloadLineups(); # Update our SD lineups in the DB my $sql = "select lineup, name, transport, location, details from lineups order by lineup"; my $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->execute(); if ($sth->err()) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, undef, SQL_VARCHAR ); $sth->bind_col( 2, undef, SQL_VARCHAR ); $sth->bind_col( 3, undef, SQL_VARCHAR ); $sth->bind_col( 4, undef, SQL_VARCHAR ); $sth->bind_col( 5, undef, SQL_VARCHAR ); my $lu = $sth->fetchall_arrayref(); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); undef $sth; if (scalar(@{$lu}) == 0) { print (STDERR "No lineups are defined in your Schedules Direct account\n"); print (STDERR "To manage your lineups, please re-run $SCRIPT_NAME --manage-lineups\n"); print (STDERR "and re-run $SCRIPT_NAME --configure to complete the configuration\n"); exit(1); } $writer->start_selectmany ( { id => 'lineup', title => [ [ 'Schedules Direct Lineup', 'en' ] ], description => [ [ 'Select the lineup(s) associated with this configuration', 'en' ] ], } ); for my $l (@{$lu}) { my $id = $l->[0]; my $lineupDesc = lineupDesc($l->[1], $l->[2], $l->[3]); $writer->write_option ( { value => $id, text => [ [ "$id - $lineupDesc", 'en' ] ] } ); } $writer->end_selectmany(); $writer->end('3rdparty-metadata'); } elsif ($stage eq '3rdparty-metadata') { $writer->start_selectone ( { id => '3rdparty-metadata', title => [ [ '3rd party metadata references', 'en' ], ], description => [ [ 'Specify whether to include 3rd party metadata references in the generated XMLTV', 'en' ] ], } ); $writer->write_option ( { value => 'enabled', text => [ [ 'Yes - Enable 3rd party metadata references', 'en'] ] } ); $writer->write_option ( { value => 'disabled', text => [ [ 'No - Disable 3rd party metadata references', 'en'] ] } ); $writer->end_selectone(); $writer->end('extended-video-quality'); } elsif ($stage eq 'extended-video-quality') { $writer->start_selectone ( { id => 'extended-video-quality', title => [ [ 'Extended XMLTV video quality values (HDR, UHDTV)', 'en' ], ], description => [ [ 'Specify whether to specify the extended video quality values (HDR, UHDTV) in the generated XMLTV', 'en' ] ], } ); $writer->write_option ( { value => 'disabled', text => [ [ 'No - Disable extended quality values', 'en'] ] } ); $writer->write_option ( { value => 'enabled', text => [ [ 'Yes - Enable extended quality values', 'en'] ] } ); $writer->end_selectone(); $writer->end('mythtv'); } elsif ($stage eq 'mythtv') { $writer->start_selectone ( { id => 'mythtv-categories', title => [ [ 'MythTV category processing', 'en' ], ], description => [ [ 'Specify whether the XMLTV categories should be MythTV ordered', 'en' ] ], } ); $writer->write_option ( { value => 'enabled', text => [ [ 'Yes - Enable MythTV Category order', 'en'] ] } ); $writer->write_option ( { value => 'disabled', text => [ [ 'No - Do not enable MythTV Category order', 'en'] ] } ); $writer->end_selectone(); $writer->end('station-logo-order'); } elsif ($stage eq 'station-logo-order') { $writer->start_selectone ( { id => 'station-logo-order', title => [ [ 'Station logo ordering', 'en' ], ], description => [ [ 'Specify the order of station logos', 'en' ] ], } ); $writer->write_option ( { value => '', text => [ [ 'None specified (order as received)', 'en' ] ] } ); $writer->write_option ( { value => 'Gracenote/dark', text => [ [ 'Gracenote/dark ordered first (Gracenote logo for dark backgrounds)', 'en' ] ] } ); $writer->write_option ( { value => 'Gracenote/light', text => [ [ 'Gracenote/light ordered first (Gracenote logo for light backgrounds)', 'en' ] ] } ); $writer->write_option ( { value => 'Gracenote/gray', text => [ [ 'Gracenote/gray ordered first (Gracenote logo with grayscale for light backgrounds)', 'en' ] ] } ); $writer->write_option ( { value => 'Gracenote/white', text => [ [ 'Gracenote/white ordered first (Gracenote logo with all white for dark backgrounds)', 'en' ] ] } ); $writer->end_selectone(); $writer->end('select-channels'); } else { die "Unknown stage $stage"; } return $result; } # # listChannels # # Perform the list-channels function per the XMLTV standard # # Input: # conf - the conf hash # opt - the opt hash # Output: # result - the xml configure string # sub listChannels { ($conf, $opt, undef) = @_; configValidate($conf, $opt); $debug = $opt->{'debug'}; $quiet = $opt->{'quiet'}; $SD->Debug(1) if ($debug); $download = 0 if ($opt->{'no-download'}); print (STDERR "Opening the local database\n") if (!$quiet); DB_open($conf->{'database'}->[0]); if ($opt->{'force-download'}) { print (STDERR " clearing existing database to force full download\n") if (!$quiet); DB_clean(); } # # If we are downloading, allow for optimization # if ($download) { print (STDERR "Obtaining authentication token for Schedules Direct\n") if (!$quiet); SD_login(); my $expiry = $SD->accountExpiry; if (!defined($expiry)) { print (STDERR "Unable to obtain the account expiration date: " . $SD->ErrorString . "\n"); exit(1); } my $expiryDateTime = DateTime::Format::ISO8601->parse_datetime($expiry); print (STDERR " Schedules Direct account expires on " . $expiryDateTime . "\n") if (!$quiet); # # Start the download process # print (STDERR "Downloading data from Schedules Direct\n") if (!$quiet); # # Always make sure we have a current lineup list # print (STDERR " downloading account lineups from Schedules Direct\n") if (!$quiet); SD_downloadLineups(); # # Validate that the configured lineup exists in our database # lineupValidate($conf->{'lineup'}); # # Get our current Schedules Direct maps (channels and # stations) for our lineup and feed to our DB if needed # for my $lineup(@{$conf->{'lineup'}}) { if (SD_isLineupFetchRequired([$lineup])) { print (STDERR " downloading channel and station maps for lineup $lineup \n") if (!$quiet); SD_downloadLineupMaps($lineup); } else { print (STDERR " not downloading channel and station maps for lineup $lineup (data current)\n") if (!$quiet); } } } else { lineupValidate($conf->{'lineup'}); my $token = DB_settingsGet('token'); $SD->Token($token) if (defined($token)); } print (STDERR "Processing data and creating XMLTV output\n") if (!$quiet); my $w = XML::Writer->new( 'ENCODING' => 'UTF-8', 'DATA_MODE' => 1, 'DATA_INDENT' => 1, 'OUTPUT' => 'self' ); $w->xmlDecl('UTF-8'); $w->comment($SD_COMMENT); $w->doctype( 'tv', undef, 'xmltv.dtd' ); $w->startTag('tv', 'generator-info-name' => $SCRIPT_NAME, 'generator-info-url' => $SCRIPT_URL, 'source-info-name' => $SD_DESC, 'source-info-url' => $SD_SITEURL ); my $channelsWritten = channelWriter($conf->{'lineup'}, $w); $w->endTag('tv'); $w->end(); print (STDERR " $channelsWritten channels processed\n") if (!$quiet); return(encode('UTF-8', $w->to_string)); } # # channelWriter # # Convenience routine to write the XMLTV channels. # Output is written to the xmltv writer # # Input: # lineup(s) - the lineup(s) to use # writer - the xmltv writer # Output: # written - number of channels written # sub channelWriter { my ($lineups, $writer, undef) = @_; my $sql; my $sth; my $param; my $channelsWritten = 0; # # Select our lineup channels/stations # $sql = 'select distinct channels.station, channels.channum, channels.details, stations.details from channels as channels left join stations as stations on stations.station = channels.station where channels.lineup in ( ' . join(', ', ('?') x scalar(@{$lineups})) . ' ) and channels.selected = 1 order by channels.station'; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $param = 1; for (my $i=0; $i < scalar(@{$lineups}); $i++) { $sth->bind_param( $param, @{$lineups}[$i], SQL_VARCHAR); $param++; } $sth->execute(); if ($sth->err()) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, undef, SQL_VARCHAR ); $sth->bind_col( 2, undef, SQL_VARCHAR ); $sth->bind_col( 3, undef, SQL_VARCHAR ); $sth->bind_col( 4, undef, SQL_VARCHAR ); my $channels = $sth->fetchall_arrayref(); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); undef $sth; # Process each channel in our lineup foreach my $r(@{$channels}) { my $sID = $r->[0]; my $channum = $r->[1]; my $c = eval { $JSON->decode($r->[2]) } || {}; my $s = eval { $JSON->decode($r->[3]) } || {};; $writer->startTag('channel', 'id' => generateRFC2838($sID) ) ; my $name = ''; $name = $s->{'name'} if (defined($s->{'name'})); my $callsign = ''; $callsign = $s->{'callsign'} if (defined($s->{'callsign'})); $name = $callsign if ($name eq ''); $name = $channum if ($name eq ''); $callsign = $channum if ($callsign eq ''); $writer->dataElement('display-name', $name) if ($name ne ''); $writer->dataElement('display-name', $callsign) if ($callsign ne ''); $writer->dataElement('display-name', $channum) if ($channum ne ''); # We return all stationLogo's unless asked to not return any, # or to return only the first. if ((defined($s->{'stationLogo'})) && (ref($s->{'stationLogo'}) eq 'ARRAY') && (!exists($GRABBER_FIXUPS->{'NO_STATION_LOGOS'}))) { for my $sl (sort { logoPriority($b) <=> logoPriority($a) } @{$s->{'stationLogo'}}) { next if (ref($sl) ne 'HASH'); if(defined($sl->{'URL'})) { if (defined($sl->{'width'}) && defined($sl->{'height'})) { $writer->emptyTag('icon', 'src' => $sl->{'URL'}, 'width' => $sl->{'width'}, 'height' => $sl->{'height'}); } else { $writer->emptyTag('icon', 'src' => $sl->{'URL'}); } last if (exists($GRABBER_FIXUPS->{'NO_MULTIPLE_STATION_LOGOS'})); } } } $writer->endTag('channel'); $channelsWritten++; } return ($channelsWritten); } # # listLineups # # Perform the list-lineups function per XMLTV # # Input: # opt - the opt hash # Output: # result - the xml configure string # sub listLineups { ($opt, undef) = @_; $conf = LoadConfig($opt->{'config-file'}); my $sql; my $sth; my $param; configValidate($conf, $opt); $debug = $opt->{'debug'}; $quiet = $opt->{'quiet'}; $SD->Debug(1) if ($debug); $download = 0 if ($opt->{'no-download'}); print (STDERR "Opening the local database\n") if (!$quiet); DB_open($conf->{'database'}->[0]); if ($opt->{'force-download'}) { print (STDERR " clearing existing database to force full download\n") if (!$quiet); DB_clean(); } if ($download) { print (STDERR "Obtaining authentication token for Schedules Direct\n") if (!$quiet); SD_login(); my $expiry = $SD->accountExpiry; if (!defined($expiry)) { print (STDERR "Unable to obtain the account expiration date: " . $SD->ErrorString . "\n"); exit(1); } my $expiryDateTime = DateTime::Format::ISO8601->parse_datetime($expiry); print (STDERR " Schedules Direct account expires on " . $expiryDateTime . "\n") if (!$quiet); # # Start the download process # print (STDERR "Downloading data from Schedules Direct\n") if (!$quiet); # # Always make sure we have a current lineup list # print (STDERR " downloading account lineups from Schedules Direct\n") if (!$quiet); SD_downloadLineups(); } else { my $token = DB_settingsGet('token'); $SD->Token($token) if (defined($token)); } $sql = 'select lineup, name, transport, location, details from lineups order by lineup'; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->execute(); if ($sth->err()) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, undef, SQL_VARCHAR ); $sth->bind_col( 2, undef, SQL_VARCHAR ); $sth->bind_col( 3, undef, SQL_VARCHAR ); $sth->bind_col( 4, undef, SQL_VARCHAR ); $sth->bind_col( 5, undef, SQL_VARCHAR ); my $lu = $sth->fetchall_arrayref(); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); undef $sth; print (STDERR "Processing data and creating XMLTV output\n") if (!$quiet); my $lineupsWritten = 0; my $w = XML::Writer->new( 'ENCODING' => 'UTF-8', 'DATA_MODE' => 1, 'DATA_INDENT' => 1, OUTPUT => 'self' ); $w->xmlDecl('UTF-8'); $w->comment('Note: list-lineups and get-lineup is still unofficial in XMLTV, and the format and content of this xml is liable to change.'); $w->comment($SD_COMMENT); $w->startTag('xmltv-lineups', 'modified' => strftime("%FT%T %z", localtime), 'generator-info-name' => $SCRIPT_NAME, 'generator-info-url' => $SCRIPT_URL, 'source-info-name' => $SD_DESC, 'source-info-url' => $SD_SITEURL ); for my $l (@{$lu}) { my $id = $l->[0]; my $lineupDesc = lineupDesc($l->[1], $l->[2], $l->[3]); $w->startTag('xmltv-lineup', 'id' => $id ); my $type = mapTransport($l->[2]); $w->dataElement('type', $type); $w->dataElement('display-name', $lineupDesc); $w->endTag('xmltv-lineup'); $lineupsWritten++; } $w->endTag('xmltv-lineups'); $w->end(); print (STDERR " $lineupsWritten lineups processed\n") if (!$quiet); return(encode('UTF-8', $w->to_string)); } # # getLineup # # Perform the get-lineup function per XMLTV # # Input: # conf - the conf has # opt - the opt hash # Output: # result - the xml configure string # sub getLineup { ($conf, $opt, undef) = @_; my $sql; my $sth; my $param; configValidate($conf, $opt); $debug = $opt->{'debug'}; $quiet = $opt->{'quiet'}; $SD->Debug(1) if ($debug); $download = 0 if ($opt->{'no-download'}); print (STDERR "Opening the local database\n") if (!$quiet); DB_open($conf->{'database'}->[0]); if ($opt->{'force-download'}) { print (STDERR " clearing existing database to force full download\n") if (!$quiet); DB_clean(); } # # If we are downloading, allow for optimization # if ($download) { print (STDERR "Obtaining authentication token for Schedules Direct\n") if (!$quiet); SD_login(); my $expiry = $SD->accountExpiry; if (!defined($expiry)) { print (STDERR "Unable to obtain the account expiration date: " . $SD->ErrorString . "\n"); exit(1); } my $expiryDateTime = DateTime::Format::ISO8601->parse_datetime($expiry); print (STDERR " Schedules Direct account expires on " . $expiryDateTime . "\n") if (!$quiet); # # Start the download process # print (STDERR "Downloading data from Schedules Direct\n") if (!$quiet); # # Always make sure we have a current lineup list # print (STDERR " downloading account lineups from Schedules Direct\n") if (!$quiet); SD_downloadLineups(); # # Validate that the configured lineup exists in our database # lineupValidate($conf->{'lineup'}); # # Get our current Schedules Direct maps (channels and # stations) for our lineup and feed to our DB if needed # for my $lineup(@{$conf->{'lineup'}}) { if (SD_isLineupFetchRequired([$lineup])) { print (STDERR " downloading channel and station maps for lineup $lineup \n") if (!$quiet); SD_downloadLineupMaps($lineup); } else { print (STDERR " not downloading channel and station maps for lineup $lineup (data current)\n") if (!$quiet); } } } else { lineupValidate($conf->{'lineup'}); my $token = DB_settingsGet('token'); $SD->Token($token) if (defined($token)); } # # Collect our lineup(s) information. # $sql = 'select lineup, name, transport, location, details from lineups where lineup in ( ' . join(', ', ('?') x scalar(@{$conf->{'lineup'}})) . ' ) order by lineup'; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $param = 1; for (my $i=0; $i < scalar(@{$conf->{'lineup'}}); $i++) { $sth->bind_param( $param, @{$conf->{'lineup'}}[$i], SQL_VARCHAR); $param++; } $sth->execute(); if ($sth->err()) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, undef, SQL_VARCHAR ); $sth->bind_col( 2, undef, SQL_VARCHAR ); $sth->bind_col( 3, undef, SQL_VARCHAR ); $sth->bind_col( 4, undef, SQL_VARCHAR ); $sth->bind_col( 5, undef, SQL_VARCHAR ); my $lu = $sth->fetchall_arrayref(); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); undef $sth; print (STDERR "Processing data and creating XMLTV output\n") if (!$quiet); my $channelsWritten = 0; my $w = XML::Writer->new( 'ENCODING' => 'UTF-8', 'DATA_MODE' => 1, 'DATA_INDENT' => 1, OUTPUT => 'self' ); $w->xmlDecl('UTF-8'); $w->comment('Note: list-lineups and get-lineup is still unofficial in XMLTV, and the format and content of this xml is liable to change.'); $w->comment($SD_COMMENT); $w->startTag('xmltv-lineups', 'modified' => strftime("%FT%T %z", localtime), 'generator-info-name' => $SCRIPT_NAME, 'generator-info-url' => $SCRIPT_URL, 'source-info-name' => $SD_DESC, 'source-info-url' => $SD_SITEURL ); for my $l (@{$lu}) { my $id = $l->[0]; my $lineupDesc = lineupDesc($l->[1], $l->[2], $l->[3]); $w->startTag('xmltv-lineup', 'id' => $id ); my $type = mapTransport($l->[2]); $w->dataElement('type', $type); $w->dataElement('display-name', $lineupDesc); # # Process each channel/station in the lineup # $sql = 'select distinct channels.station, channels.channum, channels.details, stations.details, lineups.transport from channels as channels left join stations as stations on stations.station = channels.station left join lineups as lineups on lineups.lineup = channels.lineup where channels.lineup = ? and channels.selected = 1 order by channels.station'; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $id, SQL_VARCHAR ); $sth->execute(); if ($sth->err()) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, undef, SQL_VARCHAR ); $sth->bind_col( 2, undef, SQL_VARCHAR ); $sth->bind_col( 3, undef, SQL_VARCHAR ); $sth->bind_col( 4, undef, SQL_VARCHAR ); $sth->bind_col( 5, undef, SQL_VARCHAR ); my $channels = $sth->fetchall_arrayref(); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); undef $sth; foreach my $r(@{$channels}) { my $sID = $r->[0]; my $channum = $r->[1]; my $c = eval { $JSON->decode($r->[2]) } || {}; my $s = eval { $JSON->decode($r->[3]) } || {}; my $SDtype = $r->[4] || 'Unknown'; my $type = mapTransport($r->[4]); $w->startTag('lineup-entry'); if (defined($channum) && ($channum ne '')) { $w->dataElement('preset', $channum); } $w->startTag('station', 'rfc2838' => generateRFC2838($sID) ); my $name = $s->{'name'}; my $shortname = $s->{'callsign'}; $name = $shortname if (!defined($name) || ($name eq '')); $w->dataElement('name', $name) if (defined($name) && ($name ne '')); $w->dataElement('short-name', $shortname) if (defined($shortname) && ($shortname ne '')); # We return all stationLogo's unless asked to not return any, # or to return only the first. if ((defined($s->{'stationLogo'})) && (ref($s->{'stationLogo'}) eq 'ARRAY') && (!exists($GRABBER_FIXUPS->{'NO_STATION_LOGOS'}))) { for my $sl (sort { logoPriority($b) <=> logoPriority($a) } @{$s->{'stationLogo'}}) { next if (ref($sl) ne 'HASH'); if(defined($sl->{'URL'})) { if (defined($sl->{'width'}) && defined($sl->{'height'})) { $w->emptyTag('logo', 'url' => $sl->{'URL'}, 'width' => $sl->{'width'}, 'height' => $sl->{'height'}); } else { $w->emptyTag('logo', 'url' => $sl->{'URL'}); } last if (exists($GRABBER_FIXUPS->{'NO_MULTIPLE_STATION_LOGOS'})); } } } $w->endTag('station'); if (($SDtype eq 'Cable') || ($SDtype eq 'Satellite') || ($SDtype eq 'IPTV')) { $w->startTag('stb-channel'); if (defined($c->{'channel'}) && looks_like_number($c->{'channel'})) { my $preset = $c->{'channel'}; $preset = 0 + $preset; $w->dataElement('stb-preset', $preset); } $w->endTag('stb-channel'); } elsif ($SDtype eq 'Antenna') { my $atscMajor = $c->{'atscMajor'}; my $atscMinor = $c->{'atscMinor'}; my $atscType = $c->{'atscType'}; my $ATSC = (defined($atscMajor) && defined($atscMinor) && looks_like_number($atscMajor) && looks_like_number($atscMinor)); my $ATSC3 = ($ATSC && defined($atscType) && ($atscType eq '3.0')); if ($ATSC) { $atscMajor = 0 + $atscMajor; $atscMinor = 0 + $atscMinor; $w->startTag('atsc-channel'); if ($ATSC3) { $w->dataElement('system', 'US-ATSC-3.0'); } else { $w->dataElement('system', 'US-ATSC'); } } else { $w->startTag('analog-channel'); $w->dataElement('system', 'NTSC-M'); } if (defined($channum) && ($channum ne '')) { $w->dataElement('number', $channum); } my $fccChan = $c->{'uhfVhf'}; if (defined($fccChan)) { $w->dataElement('frequency', mapUSATSCChannelToFrequency($fccChan)); } if ($ATSC) { # This will be wrong some of the time, but until # we get better data, it is what it is (and it # turns out it is correct a lot of the time) $w->dataElement('program', $atscMinor); } if (defined($s->{'callsign'})) { $w->dataElement('fcc-callsign', $s->{'callsign'}); } # Needed for xsd compliance, even though it was supposed to be optional for US analog $w->emptyTag('cni','tt-8-30-1' => '') if (!$ATSC); if ($ATSC) { $w->endTag('atsc-channel'); } else { $w->endTag('analog-channel'); } } elsif (($SDtype eq 'DVB-T') || ($SDtype eq 'DVB-S') | ($SDtype eq 'DVB-C')) { $w->startTag('dvb-channel'); my $freq = $c->{'frequencyHz'}; if (defined($freq) && looks_like_number($freq)) { $freq = 0 + $freq; $w->dataElement('frequency', $freq); } my $networkID = $c->{'networkID'}; if (defined($networkID) && looks_like_number($networkID)) { $networkID = 0 + $networkID; $w->dataElement('original-network-id', $networkID); } my $transportID = $c->{'transportID'}; if (defined($transportID) && looks_like_number($transportID)) { $transportID = 0 + $transportID; $w->dataElement('transport-id', $transportID); } my $serviceID = $c->{'serviceID'}; if (defined($serviceID) && looks_like_number($serviceID)) { $serviceID = 0 + $serviceID; $w->dataElement('service-id', $serviceID); } my $lcn = $c->{'logicalChannelNumber'}; if (defined($lcn) && looks_like_number($lcn)) { $lcn = 0 + $lcn; $w->dataElement('lcn', $lcn); } my $provider = $c->{'providerCallsign'}; if (defined($provider)) { $w->dataElement('provider-name', $provider); } $w->endTag('dvb-channel'); } $channelsWritten++; $w->endTag('lineup-entry'); } $DBH->commit(); undef $sth; $w->endTag('xmltv-lineup'); } $w->endTag('xmltv-lineups'); $w->end(); print (STDERR " $channelsWritten channels processed\n") if (!$quiet); return(encode('UTF-8', $w->to_string)); } # # loadOldConfig # # Perform the (internal) load old config function per XMLTV # # Note: This sub exists only to allow the grabber to # manage lineups without a configuration file # # Input: # opt - the opt hash # Output: # result - the xml configure string # sub loadOldConfig { return {}; } # # SD_login # # Convenience function for login and checks # for success. All errors are fatal. # # Input: # # Output: # # sub SD_login { my $username = DB_settingsGet('username'); my $passwordhash = DB_settingsGet('passwordhash'); my $pswdhash = $passwordHash || $passwordhash; my $token; $token = DB_settingsGet('token') if (!defined($passwordHash)); if (!defined($username)) { print (STDERR "Your database is not configured to access the Schedules Direct\n"); print (STDERR "service (the username is not available in the settings table).\n"); print (STDERR "Please re-run $SCRIPT_NAME --manage-lineups to\n"); print (STDERR "initialize the database\n"); exit(1); } if (!defined($pswdhash)) { print (STDERR "Your database is not configured to access the Schedules Direct\n"); print (STDERR "service automatically without manually entering the passwordhash.\n"); print (STDERR "Either invoke the grabber specifying the --passwordhash option,\n"); print (STDERR "or re-run $SCRIPT_NAME --manage-lineups to initialize\n"); print (STDERR "and update the database to store the hash in the database.\n"); exit(1); } if (!defined($token = $SD->obtainToken($username, undef, $pswdhash, $token))) { print (STDERR "Unable to authenticate to Schedules Direct: " . $SD->ErrorString() . "\n"); exit(1); } if ((defined($token)) && (defined($passwordhash)) && (!defined($passwordHash))) { DB_settingsSet('token', $token); $DBH->commit(); } if (!defined($SD->obtainStatus())) { print (STDERR "Unable to obtain Schedules Direct server status: " . $SD->ErrorString() . "\n"); exit(1); } my $online = $SD->isOnline; if (!defined($online)) { print (STDERR "Unable to obtain Schedules Direct server online status: " . $SD->ErrorString() . "\n"); exit(1); } if (!$online) { print (STDERR "The Schedules Direct service is not currently online, Try again later.\n"); exit(1); } my $expiry = $SD->accountExpiry; if (!defined($expiry)) { print (STDERR "Unable to obtain Schedules Direct account expiration: " . $SD->ErrorString() . "\n"); exit(1); } my $expiryDateTime = DateTime::Format::ISO8601->parse_datetime($expiry); if ($nowDateTime > $expiryDateTime) { print (STDERR "Schedules Direct account expired on " . $expiryDateTime . "\n"); exit(1); } return; } # # SD_isLineupFetchRequired # # We can avoid downloading lineup and map information # if we have updated our maps more recently than the # account lineup information in the account status # indicates (small, but occasionally useful, optimization). # # Input: # lineup(s) - the lineup(s) to check # Output: # result - true (fetch required) or false # sub SD_isLineupFetchRequired { my ($lineups, undef) = @_; my $sql; my $sth; my $accountLineups; my $accountLineupModifiedDateTime; my $fetchRequired = 0; $accountLineups = $SD->obtainLineups(); if (!defined($accountLineups)) { print (STDERR "Unable to obtain Schedules Direct account lineups: " . $SD->ErrorString() . "\n"); exit(1); } # Since each lineup has a (potential) different modified date, we will do it the long way for my $lineup(@{$lineups}) { undef $accountLineupModifiedDateTime; for my $l (@{$accountLineups}) { next if (ref($l) ne 'HASH'); if (defined($l->{'lineupID'}) && ($l->{'lineupID'} eq $lineup)) { $accountLineupModifiedDateTime = DateTime::Format::ISO8601->parse_datetime($l->{'modified'}) if (defined($l->{'modified'})); last; } } $accountLineupModifiedDateTime = $nowDateTime->clone() if (!defined($accountLineupModifiedDateTime)); $sql = 'select 1 from lineups l1 where (l1.lineup = ? and l1.modified <= ?) ' . 'union select 1 where not exists (select 1 from lineups l2 where l2.lineup = ?)'; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->bind_param( 2, DateTime::Format::SQLite->format_datetime($accountLineupModifiedDateTime), SQL_DATETIME ); $sth->bind_param( 3, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err()) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $fetchRequired |= ($sth->fetchrow_array() || 0); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->finish(); $DBH->commit(); undef $sth; } return ($fetchRequired); } # # SD_downloadLineups # # Convenience routine to download lineups and # place into our database. Errors are fatal. # # Input: # # Output: # # sub SD_downloadLineups { my $sql; my $sth; my $sql1; my $sth1; my $sql2; my $sth2; my $lu; my $lineups; my $param; my @accountLineups = (); # # Obtain our lineups # $lineups = $SD->obtainLineups(); if (!defined($lineups)) { print (STDERR "Unable to obtain Schedules Direct account lineups: " . $SD->ErrorString() . "\n"); exit(1); } # # insert or ignore, and then update in order to initialize # downloaded and modified as 1970-01-01 00:00:00 if new, # and maintain the dates if existing. # $sql = "insert or ignore into lineups (lineup, name, location, transport, details) values (?, ?, ?, ?, ?)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sql1 = "update lineups set name = ?, location = ?, transport = ?, details = ? where lineup = ?"; $sth1 = $DBH->prepare_cached($sql1); if (!defined($sth1)) { print (STDERR "Unexpected error when preparing statement ($sql1): " . $DBH->errstr . "\n"); exit(1); } for my $l (@{$lineups}) { next if ((ref($l) ne 'HASH') || (!defined($l->{'lineupID'}))); my $id = $l->{'lineupID'}; push(@accountLineups, $id); my $name = $l->{'name'} || ''; my $transport = $l->{'transport'} || ''; my $location = $l->{'location'} || ''; my $details = $JSON->canonical->encode($l); $sth->bind_param( 1, $id, SQL_VARCHAR ); $sth->bind_param( 2, $name, SQL_VARCHAR ); $sth->bind_param( 3, $location, SQL_VARCHAR ); $sth->bind_param( 4, $transport, SQL_VARCHAR ); $sth->bind_param( 5, $details, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth1->bind_param( 1, $name, SQL_VARCHAR ); $sth1->bind_param( 2, $location, SQL_VARCHAR ); $sth1->bind_param( 3, $transport, SQL_VARCHAR ); $sth1->bind_param( 4, $details, SQL_VARCHAR ); $sth1->bind_param( 5, $id, SQL_VARCHAR ); $sth1->execute(); if ($sth1->err) { print (STDERR "Unexpected error when executing statement ($sql1): " . $sth1->errstr . "\n"); $DBH->rollback(); exit(1); } } # # Remove any lineups from our database not in our schedules direct account # $sql2 = 'delete from lineups where lineup not in ( ' . join(', ', ('?') x scalar(@accountLineups)) . ' )'; $sth2 = $DBH->prepare_cached($sql2); if (!defined($sth2)) { print (STDERR "Unexpected error when preparing statement ($sql2): " . $DBH->errstr . "\n"); exit(1); } $param = 1; for (my $i=0; $i < scalar(@accountLineups); $i++) { $sth2->bind_param( $param, $accountLineups[$i], SQL_VARCHAR); $param++; } $sth2->execute(); if ($sth2->err) { print (STDERR "Unexpected error when executing statement ($sql2): " . $sth2->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); return; } # # SD_downloadLineupMaps # # Convenience routine to download maps for a lineup # and place into our database. Errors are fatal. # # Input: # lineup - Lineup to update # Output: # - database updated # sub SD_downloadLineupMaps { my ($lineup, undef) = @_; my $maps = $SD->obtainLineupMaps($lineup); if (!defined($maps)) { print (STDERR "Unable to obtainLineupMap for lineup $lineup: " . $SD->ErrorString() . "\n"); exit(1); } if (!defined($maps->{'map'})) { print (STDERR "Lineup map for lineup $lineup does not contain a channel entity\n"); exit(1); } if (!defined($maps->{'stations'})) { print (STDERR "Lineup map for lineup $lineup does not contain a station entity\n"); exit(1); } my $sql; my $sth; my $lineupChannelsSelected = 1; my $lineupTransport = ''; $sql = "select new_channels_selected, transport from lineups where lineup = ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected database error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, \$lineupChannelsSelected, SQL_INTEGER ); $sth->bind_col( 2, \$lineupTransport, SQL_VARCHAR ); $sth->fetchrow_arrayref(); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->finish(); $DBH->commit(); undef $sth; $sql = "create temp table if not exists channels_backup as select * from channels where 1<>1"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->execute(); if ($sth->err) { print (STDERR "Unexpected database error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sql = "delete from channels_backup"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->execute(); if ($sth->err) { print (STDERR "Unexpected database error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sql = "insert into channels_backup select * from channels where lineup = ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected database error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sql = "delete from channels where lineup = ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected database error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sql = "replace into channels (lineup, station, selected, channum, details) values (?, ?, ?, ?, ?)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } foreach my $c (@{$maps->{'map'}}) { my $station = $c->{'stationID'}; $station = '' if (!defined($station)); my $details = $JSON->canonical->encode($c); my $channum = ''; if (($lineupTransport eq 'Cable') || ($lineupTransport eq 'Satellite') || ($lineupTransport eq 'DVB-C') || ($lineupTransport eq 'DVB-T') || ($lineupTransport eq 'DVB-S') || ($lineupTransport eq 'IPTV')) { $channum = $c->{'channel'} if (defined($c->{'channel'})); $channum = 0 + $channum if (looks_like_number($channum)); } elsif ($lineupTransport eq 'Antenna') { my $atscMajor = $c->{'atscMajor'}; my $atscMinor = $c->{'atscMinor'}; my $uhfVhf = $c->{'uhfVhf'}; if (defined($atscMajor) && defined($atscMinor) && looks_like_number($atscMajor) && looks_like_number($atscMinor)) { $atscMajor = 0 + $atscMajor; $atscMinor = 0 + $atscMinor; $channum = "$atscMajor.$atscMinor"; } elsif (defined($uhfVhf) && looks_like_number($uhfVhf)) { $channum = 0 + $uhfVhf; } } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->bind_param( 2, $station, SQL_VARCHAR ); $sth->bind_param( 3, $lineupChannelsSelected, SQL_INTEGER ); $sth->bind_param( 4, $channum, SQL_VARCHAR ); $sth->bind_param( 5, $details, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } } # Preserve previous selected values (if they exist) by copying them across # The match must be by lineup, station, and channum. So if the station # changes, the selection resets (it may be a new station on that channel, # or it could be a change in feed (east coast to west coast), either are # likely for Cable/Satellite, but it is impossible to know the details, # so we have to consider it a new/revised channel. Simliarly, if the # channum changes, we have to consider this a different channel, even # if the station is the same (another channel on the STB, or sometimes # a (new) HD version of a channel, or a repeater channel). In other # words, the preservation works (reasonably well) only when the channel # really stays the same, but it is vulnerable to a certain class of # well known changes in real world lineups. $sql = "update channels set selected = (select selected from channels_backup where channels.lineup = channels_backup.lineup and channels.channum = channels_backup.channum and channels.station = channels_backup.station ) where lineup = ? and exists(select 1 from channels_backup where channels.lineup = channels_backup.lineup and channels.channum = channels_backup.channum and channels.station = channels_backup.station)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected database error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sql = "replace into stations (station, details) values (?, ?)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } foreach my $s (@{$maps->{'stations'}}) { my $station = $s->{'stationID'}; next if (!defined($station)); my $details = $JSON->canonical->encode($s); $sth->bind_param( 1, $station, SQL_VARCHAR ); $sth->bind_param( 2, $details, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } } $sql = "update lineups set modified = ? where lineup = ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $nowDateTimeSQLite, SQL_DATETIME ); $sth->bind_param( 2, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); return; } # # lineupValidate # # Convenience routine to validate that the specified # lineup(s) are still in our Schedules Direct lineup # # If the lineup is not valid we write a message and exit # # Input: # lineup(s) - Lineup(s) to validate # Output: # # sub lineupValidate { my ($lineups, undef) = @_; my $fatal = 0; my $sql; my $sth; foreach my $lineup (@{$lineups}) { $sql = 'select lineup, name, transport, location, details from lineups where lineup = ?'; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err()) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, undef, SQL_VARCHAR ); $sth->bind_col( 2, undef, SQL_VARCHAR ); $sth->bind_col( 3, undef, SQL_VARCHAR ); $sth->bind_col( 4, undef, SQL_VARCHAR ); $sth->bind_col( 5, undef, SQL_VARCHAR ); my $llu = $sth->fetchrow_arrayref(); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->finish(); $DBH->commit(); undef $sth; if (!defined($llu)) { print (STDERR "Lineup $lineup is no longer configured in your account at Schedules Direct.\n"); print (STDERR "Please run $SCRIPT_NAME --manage-lineups to manage your Schedules Direct lineups,\n"); print (STDERR "and/or $SCRIPT_NAME --configure to change the configured lineups.\n"); $fatal = 1; } else { my $lineupDeleted = eval { $JSON->decode($llu->[4])->{'isDeleted'} } || 0; if (defined($lineupDeleted) && $lineupDeleted) { print (STDERR "Lineup $lineup has been deleted at Schedules Direct.\n"); print (STDERR "Please run $SCRIPT_NAME --manage-lineups to manage your Schedules Direct lineups,\n"); print (STDERR "and/or $SCRIPT_NAME --configure to change the configured lineups.\n"); $fatal = 1; } } } if ($fatal) { exit(1); } } # # configValidate # # Convenience routine to validate that the configuration # file contains some basic information (database file # and lineup). # # If the configuration does not contain the basic info # we write a message and exit # # Input: # conf - The $conf array # opt - The $opt array # Output: # # sub configValidate { my ($conf, $opt, undef) = @_; if (!defined($conf->{'database'}->[0])) { print (STDERR "Database not defined in config file $opt->{'config-file'}.\n"); print (STDERR "Please run '$SCRIPT_NAME --configure'\n"); exit(1); } if (!defined($conf->{'lineup'}->[0])) { print (STDERR "Lineup not defined in config file $opt->{'config-file'}.\n"); print (STDERR "Please run '$SCRIPT_NAME --configure'\n"); exit(1); } } # # askChoice # # Convenience routine to ask for a selection and # return the value # # Input: # prompt - Prompt # default - (or undef which means the first) # options - array of arrays (inner array is [value, text]) # Output: # value - selected value (or undef for ctrl-D) # sub askChoice { my ($prompt, $default, @options) = @_; my @optionsvalue; my @optionstext; foreach my $option ( @options ) { push @optionsvalue, @{$option}[0]; push @optionstext, @{$option}[1]; } if (!defined($default)) { $default = $optionstext[0]; } my $selection = ask_choice($prompt, $default, @optionstext); return if (!defined($selection)); for ( my $i=0; $i # sub DB_open { my ($dbname, undef) = @_; my $version; my $rc; # # Quick exit if we already have the database open # return if (defined($DBH)); if (!defined($dbname)) { print (STDERR "The Schedules Direct EPG database location is not specified\n"); print (STDERR "Please re-run $SCRIPT_NAME --manage-lineups and/or $SCRIPT_NAME --configure\n"); exit(1); } # # Insure base directory exists # if (! -d dirname("$dbname")) { eval { local $SIG{'__DIE__'}; # ignore user-defined die handlers make_path(dirname("$dbname")); }; if ($@) { print (STDERR "Unable to create parent directory for $dbname: $@"); exit(1); } } $DBH = DBI->connect("DBI:SQLite:dbname=$dbname", "", "", { RaiseError => 0, PrintError => 0, AutoCommit => 0, sqlite_use_immediate_transaction => 0 }); if (!defined($DBH)) { print (STDERR "Unable to open database file $dbname: " . DBI->errstr . "\n"); exit(1); } # # Set extended timeout # $DBH->sqlite_busy_timeout(30000); # # Validate DB version support by first checking # if the database seems to be initialized. # $rc = $DBH->do("select value from settings where tag = 'version'"); if ((!defined($rc)) || ($rc < 0)) { $version = 0; } else { $version = DB_settingsGet('version'); $version = 0 if (!defined($version)); } if ($version =~ /^\d+$/) { $version = 0 + $version; } else { print (STDERR "Database version ($version) is not a valid version number\n"); exit(1); } if ($version > $SCRIPT_DB_VERSION) { print (STDERR "Database version $version is not supported (newer than grabber supported version $SCRIPT_DB_VERSION)\n"); exit(1); } elsif ($version < $SCRIPT_DB_VERSION) { if (0 == $version) ## Initial database creation { $version = 1; print (STDERR "Initializing database $dbname\n") if (!$quiet); # # SQLite specific optimizations that need to # be applied at initial database creation. # $DBH->{'AutoCommit'} = 1; $DBH->do('PRAGMA page_size=4096'); $DBH->do('PRAGMA journal_mode=WAL'); $DBH->do('PRAGMA auto_vacuum=2'); $DBH->do('vacuum'); $DBH->{'AutoCommit'} = 0; $rc = $DBH->do("create table if not exists settings (" . "tag varchar(256) not null primary key, " . "value varchar(256))"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create settings table in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create table lineups ( " . "lineup varchar(128) not null primary key, " . "name varchar(128) not null, " . "location varchar(128) not null, " . "transport varchar(64) not null, " . "downloaded datetime not null default '1970-01-01 00:00:00', " . "modified datetime not null default '1970-01-01 00:00:00', " . "new_channels_selected integer not null default 1, " . "details blob not null )"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create lineups table in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create table programs ( " . "program varchar(128) not null primary key, " . "hash varchar(64) not null, " . "details blob not null )"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create programs table in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create table stations ( " . "station varchar(128) not null primary key, " . "details blob not null )"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create stations table in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create table stations_schedules_hash ( " . "station varchar(128) not null, " . "day date not null, " . "hash varchar(64) not null, " . "details blob not null, " . "primary key(station, day) )"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create stations_schedules_hash table in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create index stations_schedules_hash_index_hash on stations_schedules_hash (hash)"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create stations schedules hash index in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create table channels ( " . "lineup varchar(128) not null, " . "station varchar(128) not null, " . "channum varchar(128) not null default '', " . "selected integer not null default 1, " . "details blob not null )"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create channels table in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create index channels_index_lineup_station on channels (lineup, station)"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create channel index in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create table schedules_hash ( " . "station varchar(128) not null, " . "day date not null, " . "hash varchar(64) not null, " . "primary key (station, day) )"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create schedules_hash table in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create index schedules_hash_index_hash on schedules_hash (hash)"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create schedules hash index in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create table schedules ( " . "station varchar(128) not null, " . "day date not null, " . "starttime datetime not null, " . "duration integer not null, " . "program varchar(128) not null, " . "program_hash varchar(64) not null, " . "details blob not null, " . "primary key (station, day, starttime, duration) )"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create schedules table in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create index schedules_index_station_starttime on schedules (station, starttime)"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create schedules index in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create index schedules_index_program on schedules (program)"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create schedules program index in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } DB_settingsSet('version', $version); $DBH->commit(); } if (1 == $version) { $version = 2; print (STDERR "Upgrading database to version $version\n") if (!$quiet); $rc = $DBH->do("alter table programs add column program_supplemental varchar(128)"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to add column program_supplemental to programs table in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("alter table programs add column downloaded datetime not null default '1970-01-01 00:00:00'"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to add column downloaded to programs table in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create index programs_index_program_supplemental on programs(program_supplemental)"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create programs_index_program_supplemental in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $rc = $DBH->do("create index programs_index_downloaded on programs(downloaded)"); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to create programs_index_downloaded in database $dbname: " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } # Update existing programs $sql = "update programs set downloaded = ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_param( 1, $nowDateTimeSQLite, SQL_DATETIME ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sql = "update programs set program_supplemental = 'SH' || substr(program,3,8) || '0000' where program like 'EP%'"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } DB_settingsSet('version', $version); $DBH->commit(); } ## if (2 == $version ## Example upgrade (version 2 to 3) ## { ## $version = 3; ## print (STDERR "Updating database to version $version\n") if (!$quiet); ## ## Alter table, create index, ? ## DB_settingsSet('version', $version); ## $DBH->commit(); ## } } $DBH->commit(); return; } # # DB_settingsGet # # Convenience routine to get a setting from the database # # Input: # tag - the tag # Output: # value - of the tag (or undef) # sub DB_settingsGet { my ($tag, undef) = @_; my $value; my $sql; my $sth; $sql = "select value from settings where tag = ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $tag, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, \$value, SQL_VARCHAR ); $sth->fetchrow_arrayref(); if ($sth->err) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->finish(); $DBH->commit(); undef $sth; return ($value); } # # DB_settingsSet # # Convenience routine to set a setting in the database # # Input: # tag - the tag # value - the value to set # Output: # # sub DB_settingsSet { my ($tag, $value, undef) = @_; my $sql; my $sth; $sql = "replace into settings (tag, value) values (?, ?)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $tag, SQL_VARCHAR ); $sth->bind_param( 2, $value, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); undef $sth; return; } # # DB_settingsDelete # # Convenience routine to delete a setting from the database # # Input: # tag - the tag # Output: # # sub DB_settingsDelete { my ($tag, undef) = @_; my $value; my $sql; my $sth; $sql = "delete from settings where tag = ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $tag, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->finish(); $DBH->commit(); undef $sth; return; } # # DB_prune # # Convenience routine to prune the database of old # or obsolete content. # # Input: # # Output: # # sub DB_prune { return if (!defined($DBH)); my $sql; my $sth; my $rc; my $expireBeforeDateTime = DateTime->now(time_zone => 'UTC')->subtract(days => 1); my $expireAfterDateTime = DateTime->now(time_zone => 'UTC')->add(days => 30); # Update any lineups where the downloaded datetime is in the future (bad rtc?) $sql = "update lineups set downloaded = '1970-01-01 00:00:00' where downloaded > ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $nowDateTimeSQLite, SQL_DATETIME ); $sth->execute(); if ($sth->err) { print (STDERR "Unable to update lineups with downloaded dates in the future in database: " . $sth->errstr . "\n"); } # Update any lineups where the modified datetime is in the future (bad rtc?) $sql = "update lineups set modified = '1970-01-01 00:00:00' where modified > ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $nowDateTimeSQLite, SQL_DATETIME ); $sth->execute(); if ($sth->err) { print (STDERR "Unable to update linesups with modified dates in the future in database: " . $sth->errstr . "\n"); } # Delete channels no longer in any of our lineups $sql = "delete from channels where lineup not in (select distinct lineups.lineup from lineups as lineups)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->execute(); if ($sth->err) { print (STDERR "Unable to prune channels no longer in our lineups in database: " . $sth->errstr . "\n"); } # Delete stations no longer in any of our channels $sql = "delete from stations where station not in (select distinct channels.station from channels as channels)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->execute(); if ($sth->err) { print (STDERR "Unable to prune stations no longer in our channels in database: " . $sth->errstr . "\n"); } # Delete schedules no longer referenced by our stations $sql = "delete from schedules where station not in (select distinct stations.station from stations as stations)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->execute(); if ($sth->err) { print (STDERR "Unable to prune schedules no longer associated with stations in database: " . $sth->errstr . "\n"); } # Delete schedules which have "expired" $sql = "delete from schedules where day < ? OR day > ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, DateTime::Format::SQLite->format_date($expireBeforeDateTime), SQL_DATE ); $sth->bind_param( 2, DateTime::Format::SQLite->format_date($expireAfterDateTime), SQL_DATE ); $sth->execute(); if ($sth->err) { print (STDERR "Unable to prune schedules for past and far future dates in database: " . $sth->errstr . "\n"); } # Delete schedules_hash no longer referenced by our stations $sql = "delete from schedules_hash where station not in (select distinct stations.station from stations as stations)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->execute(); if ($sth->err) { print (STDERR "Unable to prune schedules_hash not associated with a station in database: " . $sth->errstr . "\n"); } # Delete schedules_hash which have "expired" $sql = "delete from schedules_hash where day < ? OR day > ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, DateTime::Format::SQLite->format_date($expireBeforeDateTime), SQL_DATE ); $sth->bind_param( 2, DateTime::Format::SQLite->format_date($expireAfterDateTime), SQL_DATE ); $sth->execute(); if ($sth->err) { print (STDERR "Unable to prune schedules_hash for past and far future dates in database: " . $sth->errstr . "\n"); } # Delete schedules_hash which have no matching schedules $sql = "delete from schedules_hash where not exists (select * from schedules as schedules where schedules.station = schedules_hash.station and schedules.day = schedules_hash.day)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->execute(); if ($sth->err) { print (STDERR "Unable to prune schedules_hash which have no matching schedule in database: " . $sth->errstr . "\n"); } # Delete schedules for which there is no schedules_hash $sql = "delete from schedules where not exists (select * from schedules_hash as schedules_hash where schedules.station = schedules_hash.station and schedules.day = schedules_hash.day)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->execute(); if ($sth->err) { print (STDERR "Unable to prune schedules for unmatched schedule hashes in database: " . $sth->errstr . "\n"); } # Delete stations_schedules_hash no longer referenced by our stations $sql = "delete from stations_schedules_hash where station not in (select distinct channels.station from channels as channels)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->execute(); if ($sth->err) { print (STDERR "Unable to prune stations_schedules_hash in database: " . $sth->errstr . "\n"); } # Delete stations_schedules_hash which have "expired" $sql = "delete from stations_schedules_hash where day < ? OR day > ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, DateTime::Format::SQLite->format_date($expireBeforeDateTime), SQL_DATE ); $sth->bind_param( 2, DateTime::Format::SQLite->format_date($expireAfterDateTime), SQL_DATE ); $sth->execute(); if ($sth->err) { print (STDERR "Unable to prune stations_schedules_hash for past and far future dates in database: " . $sth->errstr . "\n"); } # Delete programs no longer referenced by a schedule and are not supplemental $sql = "delete from programs where program not in (select distinct schedules.program from schedules as schedules) and program not in (select distinct p2.program_supplemental from programs as p2 where p2.program_supplemental is not null)"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->execute(); if ($sth->err) { print (STDERR "Unable to prune programs no longer referenced in database: " . $sth->errstr . "\n"); } # Update programs which have a downloaded data in the future (bad rtc?) # (this should force a refresh of any supplemental programs downloaded with bad dates) $sql = "update programs set downloaded = '1970-01-01 00:00:00' where downloaded > ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $nowDateTimeSQLite, SQL_DATETIME ); $sth->execute(); if ($sth->err) { print (STDERR "Unable to update programs with downloaded dates in the future in database: " . $sth->errstr . "\n"); } $DBH->commit(); undef $sth; # # Because the database may not have the needed configuration # for incremental vacuum, we issue the command, but do not # check the results of the execution (it works, or not) # $DBH->{'AutoCommit'} = 1; $DBH->do('PRAGMA incremental_vacuum'); $DBH->{'AutoCommit'} = 0; # # vacuum can be a resource intensive activity, so we do # not perform it by default. Incremental vacuum will # handle the low hanging fruit, and users can choose # to perform a full vacuum as desired # #$DBH->{'AutoCommit'} = 1; #$sql = "vacuum"; #$rc = $DBH->do($sql); #if ((!defined($rc)) || ($rc < 0)) # { # print (STDERR "Unable to prune programs in database: " . $DBH->errstr . "\n"); # } #$DBH->{'AutoCommit'} = 0; # # In many (real world) runs, substantive data has been # added/deleted/updated, so inform sqlite to consider # updating the database statistics for future query # planner activity. # $DBH->{'AutoCommit'} = 1; $DBH->do('PRAGMA optimize'); $DBH->{'AutoCommit'} = 0; return; } # # DB_clean # # Convenience routine to clean the database of all data # (commonly used to force a complete download) # # Input: # # Output: # # sub DB_clean { return if (!defined($DBH)); my $sql; my $rc; # # We do not delete the lineups, channels, or stations in order to try to # preserve any channel (de)selection that may have occurred. By setting # the downloaded and modified dates to long ago, we will refresh those. # # $sql = "delete from lineups"; # $rc = $DBH->do($sql); # if ((!defined($rc)) || ($rc < 0)) # { # print (STDERR "Unable to delete lineups in database: " . $DBH->errstr . "\n"); # exit(1); # } # $sql = "delete from channels"; # $rc = $DBH->do($sql); # if ((!defined($rc)) || ($rc < 0)) # { # print (STDERR "Unable to delete channels in database: " . $DBH->errstr . "\n"); # exit(1); # } # $sql = "delete from stations"; # $rc = $DBH->do($sql); # if ((!defined($rc)) || ($rc < 0)) # { # print (STDERR "Unable to delete stations in database: " . $DBH->errstr . "\n"); # exit(1); # } # $sql = "update lineups set downloaded = '1970-01-01 00:00:00', modified = '1970-01-01 00:00:00'"; $rc = $DBH->do($sql); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to update lineups in database: " . $DBH->errstr . "\n"); exit(1); } $sql = "delete from stations_schedules_hash"; $rc = $DBH->do($sql); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to delete stations_schedules_hash in database: " . $DBH->errstr . "\n"); exit(1); } $sql = "delete from schedules_hash"; $rc = $DBH->do($sql); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to delete schedules_hash in database: " . $DBH->errstr . "\n"); exit(1); } $sql = "delete from schedules"; $rc = $DBH->do($sql); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to delete schedules in database: " . $DBH->errstr . "\n"); exit(1); } $sql = "delete from programs"; $rc = $DBH->do($sql); if ((!defined($rc)) || ($rc < 0)) { print (STDERR "Unable to delete programs in database: " . $DBH->errstr . "\n"); exit(1); } $DBH->commit(); return; } # # manageLineups # # NOTE: This should not be in this grabber, but there # is no obvious alternative place to provide it.... # # The username/passwordhash is obtained from the # database if it exists (and can be opened) but # the lineup can be managed without a database # # Input: # # Output: # # sub manageLineups { my $username; my $passwordhash; my $pswdhash; my $token; if ((defined($conf->{'database'}->[0])) && (-f $conf->{'database'}->[0])) { DB_open($conf->{'database'}->[0]); $username = DB_settingsGet('username'); $passwordhash = DB_settingsGet('passwordhash'); $pswdhash = $passwordHash || $passwordhash; $token = DB_settingsGet('token') if (!defined($passwordHash)); } # Try obtained username/password, but allow re-entry my $auth_prompted = 0; while(1) { if (!defined($username)) { $username = ask("Enter your username at Schedules Direct:"); $pswdhash = undef; $auth_prompted = 1; } if (!defined($pswdhash)) { my $password = ask_password("Enter your password for $username at Schedules Direct:"); $pswdhash = sha1_hex($password); $auth_prompted = 1; } last if (defined($token = $SD->obtainToken($username, undef, $pswdhash, $token))); print (STDERR "Unable to authenticate to Schedules Direct: " . $SD->ErrorString() . "\n"); $username = undef; $passwordhash = undef; $pswdhash = undef; $token = undef; $auth_prompted = 1; } if ((defined($DBH)) && (!$auth_prompted) && (defined($token)) && (defined($passwordhash)) && (!defined($passwordHash))) { DB_settingsSet('token', $token); $DBH->commit(); } if (!defined($SD->obtainStatus())) { print (STDERR "Unable to obtain the service status at Schedules Direct: " . $SD->ErrorString() . "\n"); exit(1); } if (!$SD->isOnline) { print (STDERR "The Schedules Direct service is not currently online, Try again later.\n"); exit(1); } my $prompt = ''; my $choice = ''; my @choices = (); while ($choice ne 'Exit') { my $lineups = $SD->obtainLineups(); if (!defined($lineups)) { print (STDERR "Fatal error obtaining lineups: " . $SD->ErrorString() . "\n"); print (STDERR "Please re-run $SCRIPT_NAME --manage-lineups and/or $SCRIPT_NAME --configure\n"); exit(1); } $prompt .= "\n"; $prompt .= "Your Schedules Direct account has the following lineups configured:\n"; $prompt .= "Lineup ID Description\n"; $prompt .= "======================================================================\n"; for my $l (@{$lineups}) { next if ((ref($l) ne 'HASH') || (!defined($l->{'lineupID'}))); my $lineupDesc = ''; if (defined($l->{'isDeleted'}) && $l->{'isDeleted'}) { $lineupDesc = "DELETED LINEUP"; } else { $lineupDesc = lineupDesc($l->{'name'}, $l->{'transport'}, $l->{'location'}); } $prompt .= sprintf("%-20s %s\n", $l->{'lineupID'}, $lineupDesc); } $prompt .= "Specify a Schedules Direct account lineup management action"; @choices = ( [ 'Exit', 'Exit lineup management'] , [ 'Add', 'Add an additional lineup to your account' ], [ 'Delete', 'Delete an existing lineup from your account' ], [ 'Display Password Hash', 'Display your password hash'], [ 'Delete Password Hash', 'Delete any password hash stored in the database'], [ 'Initialize Database' , 'Initialize/update the local database'], [ 'Channel Selection', 'Manage database lineup channel selection'], ); $choice = askChoice($prompt, undef, @choices); $choice = 'Exit' if (!defined($choice)); $prompt = "\n"; if ($choice eq 'Add') { my $guided = ask_boolean("\nDo you want to use guided lineup addition?",1); next if (!defined($guided)); if (!$guided) { my $lineup_to_add = ask("\nEnter the Schedules Direct lineup to add: "); next if (!defined($lineup_to_add)); $lineup_to_add =~ s/^\s+|\s+$//g; if ($lineup_to_add eq '') { $prompt .= "No lineup entered to add\n"; next; } if ($SD->addLineup($lineup_to_add)) { $prompt .= "Lineup $lineup_to_add added\n"; } else { $prompt .= "Lineup addition of $lineup_to_add failed: " . $SD->ErrorString() . "\n"; } next; } # Obtain the list of countries (by region) my $available = $SD->obtainAvailable('COUNTRIES'); if ((!defined($available)) || (ref($available) ne 'HASH') || (scalar($available) == 0)) { $prompt .= "Regions are not available\n"; next; } @choices = (); foreach my $reg (sort(keys(%{$available}))) { push (@choices, ["$reg", "$reg"]); } my $region = askChoice("\nSelect the region for the new lineup (ctrl-D to skip)", undef, @choices); next if (!defined($region)); my @choices = (); my $clist = $available->{$region}; if (!defined($clist)) { $prompt .= "Region $region is ill-formed\n"; next; } if (scalar(@{$clist}) == 0) { $prompt .= "Region $region has no countries defined\n"; next; } for (my $i = 0; $i < scalar(@{$clist}); $i++) { next if ((!defined(@{$clist}[$i]->{'shortName'})) || (!defined(@{$clist}[$i]->{'fullName'}))); push (@choices, [$i, "@{$clist}[$i]->{'shortName'} - @{$clist}[$i]->{'fullName'}"]); } if (scalar(@choices) == 0) { $prompt .= "Region $region countries are improperly defined, no valid entries exist\n"; next; } my $cindex = askChoice("\nSelect the country code for the new lineup (ctrl-D to skip)", undef, @choices); next if (!defined($cindex)); # check if we can offer transmitter selection my $transmitters = $SD->obtainAvailable('DVB-T', '/' . @{$clist}[$cindex]->{'shortName'}); if (defined($transmitters) && (ref($transmitters) eq 'HASH') && (scalar($transmitters) != 0)) { my $selectXMTR = ask_boolean("\nDo you want to select by transmitter?",0); next if (!defined($selectXMTR)); if ($selectXMTR) { my @choices = (); my $aprompt = ''; foreach my $location(sort(keys(%{$transmitters}))) { my $lineup = $transmitters->{$location}; if (scalar(@choices) < 10) { push (@choices, [ "$lineup", sprintf (" %-20s %s", $lineup, "$location") ]); } else { push (@choices, [ "$lineup", sprintf ("%-20s %s", $lineup, "$location") ]); } } $aprompt = "\n"; $aprompt .= "Select one of the following lineups to add to your Schedules Direct account (ctrl-D to skip)\n"; $aprompt .= " Lineup ID Description\n"; $aprompt .= " ======================================================================\n"; my $lineup_to_add = askChoice($aprompt, undef, @choices); next if (!defined($lineup_to_add)); if ($SD->addLineup($lineup_to_add)) { $prompt .= "Lineup $lineup_to_add added\n"; } else { $prompt .= "Lineup addition of $lineup_to_add failed: " . $SD->ErrorString() . "\n"; } next; } } my $country_code = @{$clist}[$cindex]->{'shortName'}; my $postal_code_regex = @{$clist}[$cindex]->{'postalCode'}; $postal_code_regex =~ s/^\/(.*)\/[a-z]*$/\^$1\$/; # Adjust for perl my $postal_code_example = @{$clist}[$cindex]->{'postalCodeExample'}; my $postal_code_required = 1; $postal_code_required = !(@{$clist}[$cindex]->{'onePostalCode'}) if (defined(@{$clist}[$cindex]->{'onePostalCode'})); my $postal_code = ''; if ($postal_code_required) { my $pprompt = ''; while ((defined($postal_code) && ($postal_code eq ''))) { $pprompt .= "\nSpecify the postal code for the new lineup (ex: $postal_code_example) (ctrl-D to skip)"; $postal_code = ask($pprompt); $pprompt = ''; if (defined($postal_code)) { $postal_code =~ s/^\s+|\s+$//g; # Check regex (removed due bad regex's in /available) #if ("$postal_code" !~ m/$postal_code_regex/) # { # $pprompt .= "The specified postal code is not valid\n"; # $postal_code = ''; # } } } next if (!defined($postal_code)); } else { $postal_code = $postal_code_example; } my $availablelineups = $SD->obtainLineupsAvailable($country_code, $postal_code); if (!defined($availablelineups)) { print (STDERR "Fatal error obtaining available lineups: " . $SD->ErrorString() . "\n"); exit(1); } if ((ref($availablelineups) ne 'ARRAY') || (scalar(@{$availablelineups})) == 0) { $prompt .= "Unable to add lineup, Schedules Direct has no lineups in $country_code/$postal_code\n"; } else { my @choices = (); my $aprompt = ''; for my $l (@{$availablelineups}) { next if ((ref($l) ne 'HASH') || (!defined($l->{'lineupID'})) || ($l->{'lineupID'} eq '')); my $lineup = $l->{'lineupID'}; my $lineupDesc = lineupDesc($l->{'name'}, $l->{'transport'}, $l->{'location'}); if (scalar(@choices) < 10) { push (@choices, [ $lineup, sprintf (" %-20s %s", $lineup, $lineupDesc) ]); } else { push (@choices, [ $lineup, sprintf ("%-20s %s", $lineup, $lineupDesc) ]); } } $aprompt = "\n"; $aprompt .= "Select one of the following lineups to add to your Schedules Direct account (ctrl-D to skip)\n"; $aprompt .= " Lineup ID Description\n"; $aprompt .= " ======================================================================\n"; my $lineup_to_add = askChoice($aprompt, undef, @choices); next if (!defined($lineup_to_add)); if ($SD->addLineup($lineup_to_add)) { $prompt .= "Lineup $lineup_to_add added\n"; } else { $prompt .= "Lineup addition of $lineup_to_add failed: " . $SD->ErrorString() . "\n"; } } } elsif ($choice eq 'Delete') { if (scalar(@{$lineups}) == 0) { $prompt .= "No lineups available to delete\n"; next; } my @choices = (); for my $l (@{$lineups}) { my $lineupDesc = ''; if (defined($l->{'isDeleted'}) && $l->{'isDeleted'}) { $lineupDesc = "DELETED LINEUP"; } else { $lineupDesc = lineupDesc($l->{'name'}, $l->{'transport'}, $l->{'location'}); } push (@choices, [ $l->{'lineupID'}, sprintf("%-20s %s", $l->{'lineupID'}, $lineupDesc) ]);; } my $lineup_to_delete = askChoice("\nLineup to delete (ctrl-D to skip)", undef, @choices); next if (!defined($lineup_to_delete)); if ($SD->deleteLineup($lineup_to_delete)) { $prompt .= "Lineup $lineup_to_delete deleted\n"; } else { $prompt .= "Lineup deletion of $lineup_to_delete failed: " . $SD->ErrorString() . "\n"; } } elsif ($choice eq 'Display Password Hash') { $prompt .= "Your password hash is: $pswdhash\n"; } elsif ($choice eq 'Delete Password Hash') { if (!defined($DBH)) { $prompt .= "No database available, unable to delete any stored password hash\n"; } else { DB_settingsDelete('passwordhash'); DB_settingsDelete('token'); $prompt .= "Password hash deleted from the database\n"; } } elsif ($choice eq 'Initialize Database') { if (!defined($DBH)) { my $db = $conf->{'database'}->[0] || File::HomeDir->my_home . "/.xmltv/SchedulesDirect.DB"; my $newdb = ask("\nEnter your database[$db]:"); $db = $newdb if ($newdb ne ''); DB_open($db); $prompt .= "Database initialized.\n"; } DB_settingsSet('username', $username); my $storehash = ask_boolean( "\n" . "*WARNING* While your password is stored as a sha1 hash,\n" . "(i.e. the actual password is not stored in the database)\n" . "the sha1 hash can be used to update your schedules direct\n" . "lineup information, and since the sha1 hash is unsalted,\n" . "a poor password can easily be brute forced (or more likely\n" . "found in an existing online rainbow table). Ensure that\n" . "your database is appropriately protected. Note that it is\n" . "STRONGLY recommended that your Schedules Direct password\n" . "be a long random sequence of characters that is not shared\n" . "with any other service. If you choose not to store the\n" . "passwordhash in the database, you will need to specify it\n" . "at every invokation of the grabber.\n\n" . "Confirm that you want to store the passwordhash in the database", 1); $storehash = 0 if (!defined($storehash)); if ($storehash) { DB_settingsSet('passwordhash', $pswdhash); $prompt .= "Schedules Direct username/passwordhash stored in database\n"; } else { DB_settingsDelete('passwordhash'); DB_settingsDelete('token'); $prompt .= "Schedules Direct username stored in database\n"; } $DBH->commit(); } elsif ($choice eq 'Channel Selection') { if (!defined($DBH)) { $prompt .= "Database has not been initialized (or cannot be opened)\n"; next; } if (scalar(@{$lineups}) == 0) { $prompt .= "No lineups available to manage channels\n"; next; } my $choice = ''; my @choices = (); my $sql; my $sth; my $lineup; my $prompt = ''; @choices = (); for my $l (@{$lineups}) { my $lineupDesc = lineupDesc($l->{'name'}, $l->{'transport'}, $l->{'location'}); push (@choices, [ $l->{'lineupID'}, sprintf("%-20s %s", $l->{'lineupID'}, $lineupDesc) ]);; } $lineup = askChoice("\nLineup to manage channels (ctrl-D to skip)", undef, @choices); next if (!defined($lineup)); SD_downloadLineupMaps($lineup); while ($choice ne 'Exit') { $prompt .= "\nSelect lineup channel action for lineup $lineup:"; @choices = ( [ 'Exit', 'Exit lineup channel management'] , [ 'MarkFuture', 'Set future new or updated lineup channels as selected' ], [ 'ClearFuture', 'Set future new or updated lineup channels as unselected'], [ 'MarkExisting', 'Set all existing lineup channels as selected'], [ 'ClearExisting', 'Set all existing lineup channels as unselected'], [ 'Select', 'Choose which channels are selected'], ); $choice = askChoice($prompt, undef, @choices); $choice = 'Exit' if (!defined($choice)); $prompt = "\n"; # Changing selected values needs to force downloads # (it may not always be necessary, but it is the # only way to make sure) $sql = "update lineups set downloaded = '1970-01-01 00:00:00', modified = '1970-01-01' where lineup = ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected database error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); if ($choice eq 'MarkFuture') { $sql = "update lineups set new_channels_selected = 1 where lineup = ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected database error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); $prompt .= "Future channels set as selected\n"; } elsif ($choice eq 'ClearFuture') { $sql = "update lineups set new_channels_selected = 0 where lineup = ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected database error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); $prompt .= "Future channels set as not selected\n"; } elsif ($choice eq 'MarkExisting') { $sql = "update channels set selected = 1 where lineup = ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected database error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); $prompt .= "Existing channels set as selected\n"; } elsif ($choice eq 'ClearExisting') { $sql = "update channels set selected = 0 where lineup = ?"; $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected database error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); $prompt .= "Existing channels set as not selected\n"; } elsif ($choice eq 'Select') { # # two by two, hands of blue # my $sql = "select channels.rowid, channels.station, channels.channum, channels.selected, channels.details, stations.details from channels as channels left join stations as stations on stations.station = channels.station where channels.lineup = ? order by channels.station"; my $sth = $DBH->prepare_cached($sql); if (!defined($sth)) { print (STDERR "Unexpected error when preparing statement ($sql): " . $DBH->errstr . "\n"); exit(1); } my $sqlupd = "update channels set selected = ? where rowid = ?"; my $sthupd = $DBH->prepare_cached($sqlupd); if (!defined($sthupd)) { print (STDERR "Unexpected error when preparing statement ($sqlupd): " . $DBH->errstr . "\n"); exit(1); } $sth->bind_param( 1, $lineup, SQL_VARCHAR ); $sth->execute(); if ($sth->err) { print (STDERR "Unexpected database error when executing statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $sth->bind_col( 1, undef, SQL_INTEGER ); $sth->bind_col( 2, undef, SQL_VARCHAR ); $sth->bind_col( 3, undef, SQL_VARCHAR ); $sth->bind_col( 4, undef, SQL_INTEGER ); $sth->bind_col( 5, undef, SQL_VARCHAR ); $sth->bind_col( 6, undef, SQL_VARCHAR ); my $channelsSelect = $sth->fetchall_arrayref(); if ($sth->err()) { print (STDERR "Unexpected error when executing fetch after execute of statement ($sql): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); undef $sth; foreach my $channelSelect(@{$channelsSelect}) { my $rowid = $channelSelect->[0]; my $station = $channelSelect->[1]; my $channum = $channelSelect->[2]; my $selected = $channelSelect->[3]; my $cdetails = $channelSelect->[4]; my $sdetails = $channelSelect->[5]; my $c = eval { $JSON->decode($cdetails) } || {}; my $s = eval { $JSON->decode($sdetails) } || {}; my $name; $name = $s->{'name'} if (defined($s->{'name'})); my $callsign; $callsign = $s->{'callsign'} if (defined($s->{'callsign'})); my $i = ''; $i .= "$channum " if ($channum ne ''); $i .= "$name " if (defined($name)); $i .= "$callsign " if (defined($callsign)); if ($i eq '') { $i = 'Unknown'; } my $ans = ask_boolean($i, $selected); $ans = $selected if (!defined($ans)); $sthupd->bind_param( 1, $ans, SQL_INTEGER ); $sthupd->bind_param( 2, $rowid, SQL_VARCHAR ); $sthupd->execute(); if ($sthupd->err) { print (STDERR "Unexpected database error when executing statement ($sqlupd): " . $sth->errstr . "\n"); $DBH->rollback(); exit(1); } $DBH->commit(); } } } } } return; } # # generateRFC2838 # # Per the XMLTV definition, the station must be # in RFC2838 format, even though there are no # (realistic) tables that provide any consistent # or reliable mappings (for at least NA stations). # So, we meet the definition by making up a # compliant name. # # Input: # name - the station name # Output: # RFC2838 - rfc2838 station name # sub generateRFC2838 { my ($station, undef) = @_; if ($RFC2838_COMPLIANT) { return (sprintf("I%s.json.schedulesdirect.org", $station)); } else { return ($station); } } # # generateXMLTV_NS # # Per the XMLTV definition, the xmltv_ns string has # two parts, the number (zero origin) and the total # separated by a '/' if the total exists. This routine # provides the formatting conversion. # # Input: # number - the base number # total - the total # Output: # xmltv_ns - the string representing the number/total # sub generateXMLTV_NS { my ($number, $total, undef) = @_; return '' if (!defined($number)); if ($number =~ /^\d+$/) { $number = $number - 1; if ($number >= 0) { return "$number" if (!defined($total)); if ($total =~ /^\d+$/) { $total = $total + 0; if ($total > 0) { return "$number / $total"; } } return "$number"; } } return ''; } # # addRole # # Add a role to the roles hash, eliminating duplicates # and treating an empty character as incomplete if # other (better?) character entries are provided later. # Note that (unfortunately), we have no way of knowing # which "character" is better if we get more than one # (and the same actor could be playing multiple roles), # so we just return them all. # # Input: # roles - existing $roles array # role - role to add # person - person to add to role # order - billing order # character - character being played # attributes - attributes # Output: # none - $roles array updated # sub addRole { my ($roles, $role, $person, $order, $character, $attributes, undef) = @_; return if (!defined($role)); return if (!defined($person)); return if (!defined($order)); return if ($order !~ /^\d+$/); my $ra = $roles->{$role}; return if (!defined($ra)); $character = '' if (!defined($character)); $attributes = {} if (!defined($attributes)); if (!defined($ra->{$person})) # Add person if we do not have them { $ra->{$person}->{'order'} = 0 + $order; $ra->{$person}->{'character'} = []; $ra->{$person}->{'attributes'} = $attributes; } # Merge attributes (no known case to merge, but plan ahead) $ra->{$person}->{'attributes'} = {%{$ra->{$person}->{'attributes'}}, %{$attributes}}; # Add characters to the list the actor is playing (if more than one) if ($character ne '') { foreach my $c(@{$ra->{$person}->{'character'}}) # Do not duplicate characters { return if ($c eq $character); } push(@{$ra->{$person}->{'character'}}, $character); } return; } # # mapTransport # # The XMLTV definition specifies the allowed transport # types. Schedules Direct has slightly different # transport types. Map the Schedules Direct type to # an XMLTV type. # # Input: # SDtype - Schedules Direct transport type # Ouput: # XMLTVtype - XMLTV transport type # sub mapTransport { my ($transport, undef) = @_; return 'Unknown' if (!defined($transport)); state $transportTypeMap = # Map for Schedules Direct transport to XMLTV type { 'DVB-C' => 'DTV', # DVB-C 'DVB-T' => 'DTV', # DVB-T 'DVB-S' => 'DTV', # DVB-S (should be STB?) 'Cable' => 'STB', # Cable (most use a STB?) 'Antenna' => 'DTV', # Antenna (US ATSC and/or analog) 'Satellite' => 'STB', # Satellite (most use a STB?) 'IPTV' => 'STB' # Schedules Direct IPTV is STB-like }; if (defined($transportTypeMap->{$transport})) { return($transportTypeMap->{$transport}); } return 'Unknown'; } # # mapRatingAgency # # Map the Schedules Direct rating agency to the expected # (short) name for MythTV. # # Input: # body - rating Body # rating - rating # Output: # body - rating Body (abbrev) # rating - rating (adjusted for VCHIP) sub mapRatingAgency { my ($body, $rating, undef) = @_; my $mappedBody = $body; my $mappedRating = $rating; # Maps partially derived from wikipedia and the wiki page located at # http://www.filmo.gs/wiki/Identifying-Film-Classification-Symbols, # based on the Schedules Direct rating agency names from sample data. # There are likely many missing country agencies. Updates welcome. state $bodyMap = { 'Australian Classification Board' => 'CB', 'British Board of Film Classification' => 'BBFC', 'USA Parental Rating' => 'VCHIP', 'Motion Picture Association' => 'MPAA', 'Motion Picture Association of America' => 'MPAA', 'Freiwillige Selbstkontrolle der Filmwirtschaft' => 'FSK', 'Film & Publication Board' => 'FPB', 'Manitoba Film Classification Board' => 'MFCB', 'B.C. Film Classification Office' => 'BCFCO', 'Saskatchewan Film and Video Classification Board' => 'SFVCB', 'Medietilsynet' => 'NMA', 'Departamento de Justiça, Classificação, Títulos e Qualificação' => 'ClassInd', 'Alberta\'s Film Classification Board' => 'AFR', 'Régie du cinéma' => 'RCQ', 'The Régie du cinéma' => 'RCQ', 'Ontario Film Review Board' => 'OFRB', 'Maritime Film Classification Board' => 'MFC', 'Canadian Parental Rating' => 'CHVRS', 'Conseil Supérieur de l\'Audiovisuel' => 'CSA', 'Dirección General de Radio, Televisión y Cinematografía' => 'RTC', 'Instituto de Cinematografía y de las Artes Visuales' => 'ICAA', 'Mediakasvatus- ja kuvaohjelmayksikkö' => 'MEKU', 'UK Content Provider' => 'UK', 'Centre national du cinéma et de l\'image animée' => 'CNC', 'Irish Film Classification Office' => 'IFCO', # Guess 'Statens filmgranskningsbyrå' => 'VET', # Guess 'Nemzeti Média- és Hirközlési Hatóság' => 'NMHH', # Guess 'Nederlands Instituut voor de Classificatie van Audiovisuele Media' => 'NICAM', # Guess 'Office of Film and Literature Classification' => 'OFLC', # Guess 'Board of Film Censors' => 'BFC', # Guess 'Korea Media Rating Board' => 'KMRB' # Guess }; if (defined($bodyMap->{$body})) { $mappedBody = $bodyMap->{$body}; } # # Special hack for the VCHIP rating, as currently the # Schedules Direct rating does not include the '-' # if (defined($mappedBody) && ($mappedBody eq 'VCHIP')) { # Currently, the USA Parental Rating does not include the '-'? if (defined($mappedRating) && (length($mappedRating) > 2) && (substr($mappedRating,2,1) ne '-')) { $mappedRating = (substr($mappedRating,0,2) . '-' . substr($mappedRating, 2)); } } return ($mappedBody, $mappedRating); } # # mapUSATSCChannelToFrequency # # Map the US FCC channel number to a transmission # frequency # # Input: # channel - the FCC channel # Output: # frequency - frequency in HZ # sub mapUSATSCChannelToFrequency { my ($channel, undef) = @_; $channel =~ s/^\s+|\s+$//g; # Remove any leading/trailing spaces if ($channel =~ m/^\d+$/) { $channel = 0 + $channel; } my $frequency; state $USATSCFrequenciesMap = # US ATSC frequencies { 2 => 57000000, 3 => 63000000, 4 => 69000000, 5 => 79000000, 6 => 85000000, 7 => 177000000, 8 => 183000000, 9 => 189000000, 10 => 195000000, 11 => 201000000, 12 => 207000000, 13 => 213000000, 14 => 473000000, 15 => 479000000, 16 => 485000000, 17 => 491000000, 18 => 497000000, 19 => 503000000, 20 => 509000000, 21 => 515000000, 22 => 521000000, 23 => 527000000, 24 => 533000000, 25 => 539000000, 26 => 545000000, 27 => 551000000, 28 => 557000000, 29 => 563000000, 30 => 569000000, 31 => 575000000, 32 => 581000000, 33 => 587000000, 34 => 593000000, 35 => 599000000, 36 => 605000000, 37 => 611000000, 38 => 617000000, 39 => 623000000, 40 => 629000000, 41 => 635000000, 42 => 641000000, 43 => 647000000, 44 => 653000000, 45 => 659000000, 46 => 665000000, 47 => 671000000, 48 => 677000000, 49 => 683000000, 50 => 689000000, 51 => 695000000 }; $frequency = $USATSCFrequenciesMap->{$channel} || '0'; return $frequency; } # # logoPriority # # Return the station logo priority based on # the configuration station-logo-order. # # Input: # stationLogo - the station logo hash # Output: # priority - the logo priority # sub logoPriority { my ($stationLogo, undef) = @_; my $source; my $category; # # Internal one-time priority mapping initialization # local *logoPriorityInit = sub { my $priority = {}; my $pnum = 9999; if (defined($conf->{'station-logo-order'}->[0])) { foreach my $o(split(',', $conf->{'station-logo-order'}->[0])) { $o =~ s/^\s+|\s+$//g; next if ($o eq ''); $priority->{$o} = $pnum--; } } return $priority; }; state $logoPrio = logoPriorityInit($conf); return 0 if (ref($stationLogo) ne 'HASH'); $source = $stationLogo->{'source'} || ''; $source =~ s/^\s+|\s+$//g; $category = $stationLogo->{'category'} || ''; $category =~ s/^\s+|\s+$//g; return $logoPrio->{"$source/$category"} || 0; } # # lineupDesc # # return a consistent description for a lineup # # Input: # name - the lineup name/short descr # transport - the lineup transport # location - the lineup location # Output: # lineupDesc - standard description # sub lineupDesc { my ($name, $transport, $location, undef) = @_; my $lineupDesc = ''; $name = '' if (!defined($name)); $transport = '' if (!defined($transport)); $location = '' if (!defined($location)); $name =~ s/^\s+|\s+$//g; $transport =~ s/^\s+|\s+$//g; $location =~ s/^\s+|\s+$//g; $name = '[UPSTREAM BUG: Open ticket with Schedules Direct regarding missing name for this lineup]' if ($name eq ''); $lineupDesc = $name; $lineupDesc = $lineupDesc . ' (' if (($transport ne '') || ($location ne '')); $lineupDesc = $lineupDesc . $transport if ($transport ne ''); $lineupDesc = $lineupDesc . ' ' if (($transport ne '') && ($location ne '')); $lineupDesc = $lineupDesc . $location if ($location ne ''); $lineupDesc = $lineupDesc . ')' if (($transport ne '') || ($location ne '')); return($lineupDesc); } # # A little info # =pod =head1 NAME tv_grab_zz_sdjson_sqlite - Grab TV and radio program listings from Schedules Direct (subscription required). =head1 SYNOPSIS tv_grab_zz_sdjson_sqlite --help tv_grab_zz_sdjson_sqlite --info tv_grab_zz_sdjson_sqlite --version tv_grab_zz_sdjson_sqlite --capabilities tv_grab_zz_sdjson_sqlite --description tv_grab_zz_sdjson_sqlite --manage-lineups [--config-file FILE] [--quiet] [--debug] [--passwordhash HASH] tv_grab_zz_sdjson_sqlite [--days N] [--offset N] [--config-file FILE] [--output FILE] [--quiet] [--debug] [--passwordhash HASH] [--resturl URL] [--routeto ROUTETO] tv_grab_zz_sdjson_sqlite --configure [--config-file FILE] [--quiet] [--debug] [--passwordhash HASH] [--resturl URL] [--routeto ROUTETO] tv_grab_zz_sdjson_sqlite --list-channels [--config-file FILE] [--output FILE] [--quiet] [--debug] [--passwordhash HASH] [--resturl URL] [--routeto ROUTETO] tv_grab_zz_sdjson_sqlite --list-lineups [--config-file FILE] [--output FILE] [--quiet] [--debug] [--passwordhash HASH] [--resturl URL] [--routeto ROUTETO] tv_grab_zz_sdjson_sqlite --get-lineup [--config-file FILE] [--output FILE] [--quiet] [--debug] [--passwordhash HASH] [--resturl URL] [--routeto ROUTETO] =head1 DESCRIPTION Output TV listings in XMLTV format for many locations available in North America (US/CA) and other selected countries internationally. The data comes from L and an account must be created on the Schedules Direct site in order to grab data. Refer to the Schedules Direct site for signup requirements and supported locations. This grabber uses a shared local database which allows for downloading only new/changed/updated information, and in the case of mixed OTA, Cable, and/or Satellite providers can substantially reduce the download times (as some data such as schedules and program details are commonly shared between sources in the same location). First, you must run B to manage the lineups available to your grabber configuration at the Schedules Direct service. Second, you must run B to choose which lineup this configuration will grab (this grabber will share the downloaded information for multiple lineups, and can substantially reduce the royal overheads in those cases). =head1 OPTIONS B<--manage-lineups> Perform Schedules Direct lineup management functions (adding/deleting lineups from your account, and creating the local EPG database). Managing lineups can be performed without a configuration file (it will prompt for the needed information) but if it exists, it will be used to obtain initial credentials. If you change your password at Schedules Direct, you will need to update the database (or display the new password hash) using --manage-lineups. B<--configure> Prompt for which lineup to download and write the configuration file. Note that one must run --manage-lineups first to create and initialize the database and configure lineups. B<--config-file FILE> Set the name of the configuration file, the default is B<~/.xmltv/tv_grab_zz_sdjson_sqlite.conf>. This is the file written by B<--configure> and read when grabbing. B<--output FILE> When grabbing, write output to FILE rather than standard output. B<--download-only> Perform a download of the data only (no output). B<--no-download> Do not download data, but use the existing contents of the local database. Since the code optimizes the data downloaded, this is nominally useful only in offline situations. B<--force-download> Deletes most existing local database data and forces a download of the data. If there is a suspicion that the data is corrupt (and not being automatically corrected), forcing a new download might be necessary. B<--days N> When grabbing, grab N days rather than all available days. B<--offset N> Start grabbing at today/now + N days. B<--quiet> Suppress various informational messages shown on standard error. B<--debug> Provide more information on progress to stderr to help in debugging. This can get very verbose, but too much data is better that not enough if errors need to be squashed. Note that the debug data may contain information you might prefer to be confidential such as your password hash, so treat the output appropriately. B<--passwordhash HASH> Provide the password hash on the command line. This is necessary if the hash is not stored in the database. B<--scale-download N> Scale the download chunks from the default sizes. A value of .5 would reduce the sizes of the chunks requested by half. The resulting number is bound between 1 and the max value. B<--resturl URL> Provide the Schedules Direct service endpoint URL. This is primarily useful for testing when directed by Schedules Direct staff. B<--routeto ROUTETO> Provide the Schedules Direct service endpoint RouteTo header. This is primarily useful for testing when directed by Schedules Direct staff. B<--list-channels> Write output giving elements for every channel available in the current configuration. B<--list-lineups> Write output giving list of available viewing regions. Note that list-lineups is not fully standardized, so the output is subject to change. B<--get-lineup> Write output giving elements for every channel available in the current lineup. Note that get-lineup is not fully standardized, so the output is subject to change. B<--capabilities> Show which capabilities the grabber supports. For more information, see L B<--version> Show the version of the grabber. B<--help> Print a help message and exit. B<--info> Print a help page and exit. =head1 INSTALLATION 1. First you must signup for an account at Schedules Direct. This is a paid service providing EPG data for North America and other selected countries. See L for signup requirements, and the countries served. 2. Second you need to configure the lineups that you will have access to using your account with this grabber. Run B to add your lineups and to initialize the database. 3. Third, you will need to configure this specific instance of the grabber to select the lineup to use. Run B. 4. (Optionally) run B to download and "fill" the local database copies of your data. In future runs, only updated information will be downloaded, and the local database will be pruned to delete old/obsolete information. =head1 USAGE All the normal XMLTV capabilities are included. Note that Schedules Direct only has data for a maximum of about 21 days, (although may be less for some channels) but the accuracy of the data at the end of the period tends to be poor. =head1 ERROR HANDLING If the grabber encounters a fatal error, it will write a message to STDERR and exit(1). Some errors are retriable, and the code performs retries. =head1 ENVIRONMENT VARIABLES The environment variable HOME can be set to change where configuration files are stored. All configuration is stored in $HOME/.xmltv/. On Windows, it might be necessary to set HOME to a path without spaces in it. The environment variable TV_GRAB_TARGET_APPLICATION_FIXUPS can be set to indicate that the grabber should apply fixups for applications that are not fully XMLTV compliant, or that are currently missing some specific functionality. The fixups can be combined by separating them with colons. Available fixups are NO_XMLTV_NS_TOTAL_SEASONS (do not include the total seasons in the generated xmltv_ns episode numbering), NO_PREVIOUSLY_SHOWN_ZONE_OFFSET (do not include the zone offset in previously-shown), and NO_STATION_LOGOS (do not include station logos in the output). The fixups are intended to be temporary until the application(s) can be updated. =head1 SUPPORTED CHANNELS Schedules Direct lineups should support all the channels from your provider or OTA antenna. If there are missing channels, or incorrect guide data, you should contact Schedules Direct to request updates. =head1 XMLTV VALIDATION B may report an error similar to: "Line 123 Duplicate channel-tag for 'I12345.json.schedulesdirect.org'" This is because at least some providers (typically Cable/Satellite, but sometimes OTA repeaters that you may have in your lineup) actually have the exact same station available on multiple channels. XMLTV does not like seeing the same station reported twice, even though the full display-name info does show that the channel number is different. This error can (should/must?) be ignored. =head1 XMLTV STATIONS vs CHANNELS XMLTV (despite a couple of proposals to update the specifications) has a legacy confusion regarding the differences between a "station", which is a supplier of content (programs) and schedules, and a "channel" which is method of delivery/transport. XMLTV uses the term where they likely should be using the term , because they deal with programming, not transport. Regardless, such a transition would be understandably be a challenge, and the lineup proposals to extend the capability to provide a mechanism to support "channels" has not progressed in years. This also results in a failing of the configuration capability which treats the selecting of content as being station based, which is not always the same thing as a (for example, for Cable providers, a "station" may be transmitted on many "channels" (perhaps in different resolutions), but an individual may only be authorized to receive some of the "channels"). One may want the "station" schedules and programs, but not to see the "channel" returned because they cannot tune it. =head1 CHANNEL SELECTION Due to the XMLTV interpretations of , this grabber implements its own "channel" (transport) selection mechanism (which parallels that on the Schedules Direct site). It is implemented within the --manage-lineups capability. The grabber defaults will result in all channels and stations associated with the lineup being written. In some cases it may be desired by some to limit the channels to a small subset of all available channels (the most common being a Cable or Satellite service which has billions and billions of channels, but you are subscribed to a significantly reduced programming tier, and your application does not have the ability to restrict the display/access to that large number of channels). There is just enough flexibility to allow one to confuse oneself some of the time. Note that while an effort is made to maintain the existing selection value when the lineup mapping (channels and stations) are updated, new or changed station assignments per channel will result in the lineup defaults being assigned to the new or updated channel. The lineup channel selection default can also be set for an existing lineup. Due to the potential of future surprises or confusion, if one can avoid using the channel selection capability one is likely better off. =head1 FAQs No FAQs yet.... =head1 DISCLAIMER The Schedules Direct service requires a subscription, and only allows for usage for personal use with approved open source projects. Refer to the Schedules Direct site for their requirements and how to sign up. =head1 AUTHOR Gary Buhrmaster. As with most tv_grabbers, documentation, ideas, and parts of the code may have been leveraged from other existing grabbers from the XMLTV-project. We stand on the shoulders of those that came before us. =head1 COPYRIGHT Copyright (c) 2016, 2017, 2018, 2022 Gary Buhrmaster This code is distributed under the GNU General Public License v2 (GPLv2) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License version 2 as published by the Free Software Foundation. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. =head1 SEE ALSO L. =cut ######################################################################## package SchedulesDirect v20141201.0.0; # # Public methods # # Debug - set/return debug value # RaiseError - set/return croak value # PrintError - set/return carp value # Error - return error value # ErrorString - return error string value # RESTUrl - set/return the SD RESTUrl # Username - set/return username to use # Password - set/return passwordhash to use # PasswordHash - set/return passwordhash to use # Token - set/return SD Token to use # obtainToken - obtain and return SD token # obtainStatus - obtain and return SD status # isOnline - return true if SD systems online # accountExpiry - return account expiration datetime # obtainDataLastUpdated - return data last updated datetime # addLineup - add lineup to account # deleteLineup - delete lineup from account # obtainLineups - return lineups in account # obtainLineupMaps - return maps for lineup # obtainLineupsAvailable - return lineups available in country/postal # obtainStationsSchedules - return stations schedules # obtainStationsSchedulesHash - return stations schedules hash # obtainPrograms - return program data for programs # obtainAvailable - return available counties/satellites # deleteMessage - delete message # uriResolve - convert uri to absolute # require 5.016; use feature ':5.16'; use strict; use warnings FATAL => 'all'; use warnings NONFATAL => qw(exec recursion internal malloc newline deprecated portable); no warnings 'once'; use Carp; use Digest::SHA qw(sha1 sha1_hex sha1_base64); use URI; use URI::Escape; use Compress::Zlib; use HTTP::Request; use HTTP::Message; use JSON; use LWP::UserAgent::Determined; use LWP::Simple; use LWP::Protocol::https; use LWP::ConnCache; use Time::HiRes qw( time ); use Data::Dumper; sub new { my $proto = shift; my $class = ref($proto) || $proto; my $self = {@_}; $self->{'Username'} = undef unless $self->{'Username'}; $self->{'PasswordHash'} = undef unless $self->{'PasswordHash'}; $self->{'PasswordHash'} = sha1_hex($self->{'Password'}) if defined($self->{'Password'}); delete $self->{'Password'}; $self->{'UserAgent'} = 'tv_grab_zz_sdjson_sqlite' unless $self->{'UserAgent'}; $self->{'Debug'} = 0 unless $self->{'Debug'}; $self->{'RESTUrl'} = 'https://json.schedulesdirect.org/20141201' unless $self->{'RESTUrl'}; $self->{'RouteTo'} = undef unless $self->{'RouteTo'}; $self->{'RaiseError'} = 0 unless $self->{'RaiseError'}; # Not (yet) implemented $self->{'PrintError'} = 0 unless $self->{'PrintError'}; # Not (yet) implemented $self->{'_Token'} = undef; $self->{'_TokenAcquired'} = 0; # Refresh token every 20 hours $self->{'_TokenValidated'} = 0; if (defined($self->{'Token'})) { my ($token, $acquired) = split(' ', $self->{'Token'}, 2); if (defined($token) && ($token =~ /^[0-9A-Fa-f]+$/) && defined($acquired) && ($acquired =~ /^-?\d+\.?\d*$/)) { $self->{'_Token'} = $token; $self->{'_TokenAcquired'} = $acquired; } } delete $self->{'Token'}; $self->{'_Error'} = 0; $self->{'_ErrorString'} = ''; $self->{'_Status'} = undef; $self->{'_StatusAcquired'} = 0; # Refresh status every 15 minutes? $self->{'_JSON'} = JSON->new()->shrink(1)->utf8(1); $self->{'ConnCache'} = 10 unless $self->{'ConnCache'}; $self->{'_LWP'} = LWP::UserAgent::Determined->new(agent => $self->{'UserAgent'}, conn_cache => LWP::ConnCache->new(total_capacity => $self->{'ConnCache'})); $self->{'_LWP'}->timing('1,2,5,10,20,20,20,20,20,20'); $self->{'_LWP'}->requests_redirectable(['GET', 'HEAD', 'POST', 'PUT', 'DELETE']); $self->{'_LWP'}->default_header('Accept-Encoding' => scalar HTTP::Message::decodable(), 'Accept' => 'application/json', 'Content-Type' => 'application/json', 'Pragma' => 'no-cache', 'Cache-Control' => 'no-cache'); if (defined($self->{'RouteTo'})) { $self->{'_LWP'}->default_headers->header('RouteTo' => $self->{'RouteTo'}); } bless($self, $class); return $self; } END { } sub DESTROY { my $self = shift; return; } # # Convenience method since many times you only # need to know if Schedules Direct is 'online'. # sub isOnline { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . "\n") if ($self->{'Debug'}); my $return; $self->_resetError; $self->obtainStatus; if ($self->{'_Error'}) { $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (defined($self->{'_Status'}->{'systemStatus'}->[0]->{'status'})) { my $status = $self->{'_Status'}->{'systemStatus'}->[0]->{'status'}; if ($status eq 'Online') { $return = 1; } else { $return = 0; } print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("Unable to obtain the Schedules Direct system status"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # Convenience method for when the account expires # sub accountExpiry { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . "\n") if ($self->{'Debug'}); my $return; $self->_resetError; $self->obtainStatus; if ($self->{'_Error'}) { $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (defined($self->{'_Status'}->{'account'}->{'expires'})) { $return = $self->{'_Status'}->{'account'}->{'expires'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("Unable to obtain the Schedules Direct account expiration date"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # Convenience method to obtain when the data was last updated # sub obtainDataLastUpdated { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . "\n") if ($self->{'Debug'}); my $return; $self->_resetError; $self->obtainStatus; if ($self->{'_Error'}) { $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (defined($self->{'_Status'}->{'lastDataUpdate'})) { $return = $self->{'_Status'}->{'lastDataUpdate'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $self->_setErrorString("Unable to obtain the Schedules Direct data last updated"); $return = undef; $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # Return error # sub Error { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . "\n") if ($self->{'Debug'}); my $return; $return = $self->{'_Error'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # Return error string # sub ErrorString { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . "\n") if ($self->{'Debug'}); my $return; $return = $self->{'_ErrorString'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # set/return debug status # sub Debug { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); $self->_resetError; if (@_) { $self->{'Debug'} = shift } my $return = $self->{'Debug'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # set/return RaiseError (croak) status # sub RaiseError { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; $self->_resetError; if (@_) { $self->{'RaiseError'} = shift } $return = $self->{'RaiseError'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # set/return PrintError (carp) status # sub PrintError { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; $self->_resetError; if (@_) { $self->{'PrintError'} = shift } $return = $self->{'PrintError'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # set/return username # sub Username { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; $self->_resetError; if (@_) { $self->{'Username'} = shift; $self->_resetSession; } $return = $self->{'Username'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # set/return password (return hash) # sub Password { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; $self->_resetError; if (@_) { my $p = shift; $self->{'PasswordHash'} = sha1_hex($p); $self->_resetSession; } $return = $self->{'PasswordHash'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # set/return password hash # sub PasswordHash { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; $self->_resetError; if (@_) { $self->{'PasswordHash'} = shift; $self->_resetSession; } $return = $self->{'PasswordHash'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # set/return RESTUrl # sub RESTUrl { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; $self->_resetError; if (@_) { $self->{'RESTUrl'} = shift; $self->_resetSession; } $return = $self->{'RESTUrl'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # set/return RouteTo # sub RouteTo { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; $self->_resetError; if (@_) { $self->{'RouteTo'} = shift; $self->{'_LWP'}->default_headers->remove_header('RouteTo'); if (defined($self->{'RouteTo'})) { $self->{'_LWP'}->default_headers->header('RouteTo' => $self->{'RouteTo'}); } $self->_resetSession; } $return = $self->{'RouteTo'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # set/return the (extended) SDToken # sub Token { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; $self->_resetError; if (@_) { $self->_resetSession; my $t = shift; if (defined($t)) { my ($token, $acquired) = split(' ', $t, 2); if (defined($token) && ($token =~ /^[0-9A-Fa-f]+$/) && defined($acquired) && ($acquired =~ /^-?\d+\.?\d*$/)) { $self->{'_Token'} = $token; $self->{'_TokenAcquired'} = $acquired; $self->{'_TokenValidated'} = 0; } } } if (defined($self->{'_Token'})) { $return = "$self->{'_Token'} $self->{'_TokenAcquired'}"; } else { $return = undef; } print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # Resolve a possible relative uri to absolute URL # sub uriResolve { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $uri = shift; my $path = shift || ''; my $return; $self->_resetError; $return = URI->new_abs( $uri, "$self->{'RESTUrl'}" . $path . "/" )->as_string(); print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # Delete a message # sub deleteMessage { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; my $messageID = shift; $self->_resetError; if (!defined($messageID)) { $return = 0; $self->_setErrorString("messageID is not specified to delete"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!$self->isOnline) { if ($self->{'_Error'}) { $return = 0; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = 0; $self->_setErrorString("Schedules Direct web services is not online"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $request = HTTP::Request->new(DELETE => "$self->{'RESTUrl'}/messages/$messageID"); $request->header(Token => "$self->{'_Token'}"); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $responseCode = $response->code(); my $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if ($responseCode != 200) { $return = 0; $self->_setErrorString("HTTP response code was not successful ($responseCode)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $return = 0; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = 0; $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->decode($responseContent) }; if (!defined($r)) { $return = 0; $self->_setErrorString("HTTP response content was not parseable ($responseContent)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $code = $r->{'code'}; my $msg = $r->{'message'} || ''; if (!defined($code)) { $return = 0; $self->_setErrorString("Delete response was not valid"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($code != 0) { $return = 0; $self->_setError($code); $self->_setErrorString("Delete request failed, code: $code, message: $msg"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = 1; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # Add a lineup to the account # sub addLineup { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; my $lineup = shift; $self->_resetError; if (!defined($lineup)) { $return = 0; $self->_setErrorString("Lineup is not specified to add"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!$self->isOnline) { if ($self->{'_Error'}) { $return = 0; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = 0; $self->_setErrorString("Schedules Direct web services is not online"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # After add (success or failure) make sure we get a new Status $self->{'_Status'} = undef; my $request = HTTP::Request->new(PUT => "$self->{'RESTUrl'}/lineups/$lineup"); $request->header(Token => "$self->{'_Token'}"); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $responseCode = $response->code(); my $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if (($responseCode == 403) || ($responseCode == 400)) { if (defined($responseContent)) { my $r = eval { $self->{'_JSON'}->decode($responseContent) }; if (defined($r)) { $self->_setError($r->{'code'}) if (defined($r->{'code'})); my $msg = $r->{'message'} || "(no message text returned for code)"; $self->_setErrorString("$msg"); $return = 0; $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = 0; $self->_setErrorString("HTTP response content was not parseable: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = 0; $self->_setErrorString("HTTP response content could not be decoded for response code $responseCode"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseCode != 200) { $return = 0; $self->_setErrorString("HTTP response code was not successful ($responseCode)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $return = 9; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = 0; $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->decode($responseContent) }; if (!defined($r)) { $return = 0; $self->_setErrorString("HTTP response content was not parseable ($responseContent)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $code = $r->{'code'}; my $msg = $r->{'message'} || ''; if (!defined($code)) { $return = 0; $self->_setErrorString("Add lineup response was not valid (code not returned)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($code != 0) { $return = 0; $self->_setError($code); $self->_setErrorString("Add lineup request failed with code: $code, message: $msg"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = 1; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # Delete a lineup from the account # sub deleteLineup { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; my $lineup = shift; $self->_resetError; if (!defined($lineup)) { $return = 0; $self->_setErrorString("Lineup is not specified to delete"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!$self->isOnline) { if ($self->{'_Error'}) { $return = 0; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = 0; $self->_setErrorString("Schedules Direct web services is not online"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # After delete (success or failure) make sure we get a new Status $self->{'_Status'} = undef; my $request = HTTP::Request->new(DELETE => "$self->{'RESTUrl'}/lineups/$lineup"); $request->header(Token => "$self->{'_Token'}"); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $responseCode = $response->code(); my $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if ($responseCode != 200) { $return = 0; $self->_setErrorString("HTTP response code was not successful ($responseCode)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $return = 0; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = 0; $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->decode($responseContent) }; if (!defined($r)) { $return = 0; $self->_setErrorString("HTTP response content was not parseable ($responseContent)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $code = $r->{'code'}; my $msg = $r->{'message'} || ''; if (!defined($code)) { $return = 0; $self->_setErrorString("Delete response was not valid"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($code != 0) { $return = 0; $self->_setError($code); $self->_setErrorString("Delete request failed, code: $code, message: $msg"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = 1; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # Obtain the lineups in the account # sub obtainLineups { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . "\n") if ($self->{'Debug'}); my $return; $self->_resetError; if (!$self->isOnline) { if ($self->{'_Error'}) { $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("Schedules Direct web services is not online"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $request = HTTP::Request->new(GET => "$self->{'RESTUrl'}/lineups"); $request->header('Token' => "$self->{'_Token'}"); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $responseCode = $response->code(); my $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if ($responseCode == 400) { # (bug?) Rather than returning an empty array, SD returns 400 error # We will convert this to an empty array (no lineups) if (defined($responseContent)) { my $r = eval { $self->{'_JSON'}->decode($responseContent) }; if (defined($r) && defined($r->{'code'}) && ($r->{'code'} == 4102)) { $return = []; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("HTTP response content was not parseable: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseCode != 200) { $return = undef; $self->_setErrorString("HTTP response code was not successful: $responseCode"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $return = undef; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = undef; $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->decode($responseContent) }; if (!defined($r)) { $return = undef; $self->_setErrorString("HTTP response content was not parseable: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (ref($r) ne 'HASH') { $return = undef; $self->_setErrorString("HTTP response content was not a hash: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($r->{'lineups'})) { $return = undef; $self->_setErrorString("HTTP response content was not a hash containing a lineup entity: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (ref($r->{'lineups'}) ne 'ARRAY') { $return = undef; $self->_setErrorString("HTTP response content was not a hash containing the lineup array: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = $r->{'lineups'}; for my $e(@{$return}) { next if (!defined($e->{'lineup'})); $e->{'lineupID'} = delete $e->{'lineup'}; } if ((ref($self->{'_Status'}) eq 'HASH') && (defined($self->{'_Status'}->{'lineups'})) && (ref($self->{'_Status'}->{'lineups'}) eq 'ARRAY')) { for my $e(@{$return}) { next if (!defined($e->{'lineupID'})); for my $se(@{$self->{'_Status'}->{'lineups'}}) { if ((ref($se) eq 'HASH') && (defined($se->{'lineup'})) && ($se->{'lineup'} eq $e->{'lineupID'}) && (defined($se->{'modified'}))) { $e->{'modified'} = $se->{'modified'}; last; } } } } print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # ObtainLineupMaps # sub obtainLineupMaps { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; my $lineup = shift; $self->_resetError; if (!defined($lineup)) { $return = undef; $self->_setErrorString("Schedules Direct lineup not specified"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!$self->isOnline) { if ($self->{'_Error'}) { $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("Schedules Direct web services is not online"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $request = HTTP::Request->new(GET => "$self->{'RESTUrl'}/lineups/$lineup"); $request->header('Token' => "$self->{'_Token'}", 'verboseMap' => 'true'); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $responseCode = $response->code(); my $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if ($responseCode != 200) { $return = undef; $self->_setErrorString("HTTP response code was not successful ($responseCode)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $return = undef; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = undef; $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->decode($responseContent) }; if (!defined($r)) { $return = undef; $self->_setErrorString("HTTP response content was not parseable ($responseContent)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = $r; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # Return list of lineups available in country/postal code # sub obtainLineupsAvailable { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; my ($country, $postalcode, undef) = @_; $self->_resetError; if (!defined($country) || ($country eq '')) { $return = undef; $self->_setErrorString("Country code not provided for lineup list"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($postalcode) || ($postalcode eq '')) { $return = undef; $self->_setErrorString("Postal code code not provided for lineup list"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!$self->isOnline) { if ($self->{'_Error'}) { $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("Schedules Direct web services is not online"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $country = uri_escape($country); $postalcode = uri_escape($postalcode); my $request = HTTP::Request->new(GET => "$self->{'RESTUrl'}/headends?country=$country\&postalcode=$postalcode"); $request->header(Token => "$self->{'_Token'}"); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $responseCode = $response->code(); my $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if ($responseCode == 400) { # (bug?) Rather than returning an empty array, SD returns error # we will convert this to an empty array my $r = eval { $self->{'_JSON'}->decode($responseContent) }; if (defined($r)) { my $code = $r->{'code'}; my $msg = $r->{'message'} || ''; if (defined($code)) { $self->_setError($code); if ($code == 2102) { $return = []; $self->_setErrorString("No lineups in specified country/postalcode"); print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("Error obtaining lineups ($code): $msg"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } } $return = undef; $self->_setErrorString("HTTP response code was not successful ($responseCode)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseCode != 200) { $return = undef; $self->_setErrorString("HTTP response code was not successful ($responseCode)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $return = undef; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = undef; $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->decode($responseContent) }; if (!defined($r)) { $return = undef; $self->_setErrorString("HTTP response content was not parseable ($responseContent)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (ref($r) ne 'ARRAY') { $return = undef; $self->_setErrorString("HTTP response content was not an array ($responseContent)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = []; for my $e(@{$r}) { next if ((ref($e) ne 'HASH') || (!defined($e->{'lineups'}))); my $lineups = $e->{'lineups'}; next if (ref($lineups) ne 'ARRAY'); for my $lu(@{$lineups}) { my $el = {}; next if ((ref($lu) ne 'HASH') || (!defined($lu->{'lineup'}))); $el->{'lineupID'} = $lu->{'lineup'}; $el->{'transport'} = $e->{'transport'} if (defined($e->{'transport'})); $el->{'location'} = $e->{'location'} if (defined($e->{'location'})); $el->{'name'} = $lu->{'name'} if (defined($lu->{'name'})); $el->{'uri'} = $lu->{'uri'} if (defined($lu->{'uri'})); push(@{$return}, $el); } } print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # obtainPrograms # sub obtainPrograms { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; my (@programs) = @_; $self->_resetError; if (!$self->isOnline) { if ($self->{'_Error'}) { $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("Schedules Direct web services is not online"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (scalar(@programs) == 0) { $return = []; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $request = HTTP::Request->new(POST => "$self->{'RESTUrl'}/programs"); $request->header('Token' => "$self->{'_Token'}"); $request->content($self->{'_JSON'}->encode(\@programs)); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $responseCode = $response->code(); my $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if ($responseCode != 200) { $return = undef; $self->_setErrorString("HTTP response code was not successful ($responseCode)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $return = undef; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = undef; $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->convert_blessed->decode($responseContent) }; if (!defined($r)) { $return = undef; $self->_setErrorString("HTTP response content was not parseable: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = $r; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # obtainStationsSchedules # sub obtainStationsSchedules { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; my (@schedulesRequest) = @_; $self->_resetError; if (!$self->isOnline) { if ($self->{'_Error'}) { $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("Schedules Direct web services is not online"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (scalar(@schedulesRequest) == 0) { $return = []; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $request = HTTP::Request->new(POST => "$self->{'RESTUrl'}/schedules"); $request->content($self->{'_JSON'}->encode(\@schedulesRequest)); $request->header('Token' => "$self->{'_Token'}"); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $responseCode = $response->code(); my $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if ($responseCode != 200) { $return = undef; $self->_setErrorString("HTTP response code was not successful ($responseCode)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $return = undef; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = undef; $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->convert_blessed->decode($responseContent) }; if (!defined($r)) { $return = undef; $self->_setErrorString("HTTP response content was not parseable: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (ref($r) ne 'ARRAY') { $return = undef; $self->_setErrorString("HTTP response content was malformed (not an array)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } foreach my $e(@{$r}) { if (ref($e) ne 'HASH') { $return = undef; $self->_setErrorString("HTTP response content was malformed (not an array of hashes)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $hash = $e->{'metadata'}->{'md5'}; my $startDate = $e->{'metadata'}->{'startDate'}; my $modified = $e->{'metadata'}->{'modified'}; $e->{'MD5'} = $hash if (defined($hash)); $e->{'date'} = $startDate if (defined($startDate)); $e->{'modified'} = $modified if (defined($modified)); if ((defined($hash)) && (defined($startDate)) && (defined($modified))) { $e->{'code'} = 0 if (!defined($e->{'code'})); $e->{'message'} = 'OK' if (!defined($e->{'message'})); $e->{'response'} = 'OK' if (!defined($e->{'response'})); } delete $e->{'metadata'}; } $return = $r; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # obtainStationsSchedulesHash # sub obtainStationsSchedulesHash { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; my (@stationsRequest) = @_; $self->_resetError; if (!$self->isOnline) { if ($self->{'_Error'}) { $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("Schedules Direct web services is not online"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $request = HTTP::Request->new(POST => "$self->{'RESTUrl'}/schedules/md5"); $request->content($self->{'_JSON'}->encode(\@stationsRequest)); $request->header('Token' => "$self->{'_Token'}"); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $responseCode = $response->code(); my $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if ($responseCode != 200) { $return = undef; $self->_setErrorString("HTTP response code was not successful ($responseCode)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $return = undef; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = undef; $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->convert_blessed->decode($responseContent) }; if (!defined($r)) { $return = undef; $self->_setErrorString("HTTP response content was not parseable: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (ref($r) ne 'HASH') { $return = undef; $self->_setErrorString("HTTP response content was not a hash: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = []; for my $station(keys %{$r}) { if (ref($r->{$station}) eq 'HASH') { for my $date(keys %{$r->{$station}}) { my $e = {}; $e->{'stationID'} = $station; $e->{'date'} = $date; $e->{'code'} = $r->{$station}->{$date}->{'code'} if (defined($r->{$station}->{$date}->{'code'})); $e->{'MD5'} = $r->{$station}->{$date}->{'md5'} if (defined($r->{$station}->{$date}->{'md5'})); $e->{'message'} = $r->{$station}->{$date}->{'message'} if (defined($r->{$station}->{$date}->{'message'})); $e->{'lastModified'} = $r->{$station}->{$date}->{'lastModified'} if (defined($r->{$station}->{$date}->{'lastModified'})); $e->{'response'} = "OK" if ($r->{$station}->{$date}->{'code'} eq 0); push(@{$return}, $e); } } } print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # obtainAvailable # sub obtainAvailable { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; my $type = shift; my $path = shift; $type = '' if (!defined($type)); $path = '' if (!defined($path)); $self->_resetError; if (!$self->isOnline) { if ($self->{'_Error'}) { $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("Schedules Direct web services is not online"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (($path ne '') && ($type eq '')) { $return = undef; $self->_setErrorString("obtainAvailable request is not valid (type=$type, path=$path)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $request = HTTP::Request->new(GET => "$self->{'RESTUrl'}/available"); $request->header('Token' => "$self->{'_Token'}"); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $responseCode = $response->code(); my $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if ($responseCode != 200) { $return = undef; $self->_setErrorString("HTTP response code was not successful ($responseCode)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $return = undef; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = undef; $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->convert_blessed->decode($responseContent) }; if (!defined($r)) { $return = undef; $self->_setErrorString("HTTP response content was not parseable: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($type eq '') { $return = $r; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (ref($r) ne 'ARRAY') { $return = undef; $self->_setErrorString("HTTP response content was malformed (not an array)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } foreach my $e(@{$r}) { if (ref($e) ne 'HASH') { $return = undef; $self->_setErrorString("HTTP response content was malformed (not an array of hashes)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (defined($e->{'type'}) && ($e->{'type'} eq "$type")) { if (defined($e->{'uri'})) { my $uri = $e->{'uri'}; $uri =~ s/\/\{.*?\}$//; # Bad adjustment? $request = HTTP::Request->new(GET => $self->uriResolve($uri) . "$path"); $request->header('Token' => "$self->{'_Token'}"); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); $responseCode = $response->code(); $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if ($responseCode != 200) { $return = undef; $self->_setErrorString("HTTP response code was not successful ($responseCode)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $return = undef; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = undef; $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $r = eval { $self->{'_JSON'}->convert_blessed->decode($responseContent) }; if (!defined($r)) { $return = undef; $self->_setErrorString("HTTP response content was not parseable: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = $r; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } else { $return = undef; $self->_setErrorString("HTTP response content was malformed (uri not specified in available response)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } } } $return = undef; $self->_setErrorString("HTTP response did not match type=$type"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # obtainStatus # sub obtainStatus { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . "\n") if ($self->{'Debug'}); my $return; $self->_resetError; $self->obtainToken; if ($self->{'_Error'}) { $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $now = time(); #Reuse existing status if in current session and last status update < 15 min ago if (defined($self->{'_Status'}) && ($self->{'_StatusAcquired'} > ($now - 900))) { print (STDERR "DEBUG: (re)using current status\n") if ($self->{'Debug'}); $return = $self->{'_Status'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $self->{'_Status'} = undef; my $request = HTTP::Request->new(GET => "$self->{'RESTUrl'}/status"); $request->header('Token' => "$self->{'_Token'}"); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $responseCode = $response->code(); my $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if ($responseCode != 200) { $self->_setErrorString("HTTP response code was not successful ($responseCode)"); $self->_CroakOrCarp; $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->decode($responseContent) }; if (!defined($r)) { $self->_setErrorString("HTTP response content was not parseable: $responseContent"); $self->_CroakOrCarp; $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $code = $r->{'code'}; my $message = $r->{'message'} || '' ; if (!defined($code) || !defined($message)) { $self->_setErrorString("Schedules Direct status request response was not valid: $responseContent"); $self->_CroakOrCarp; $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (($code != 0)) { $self->_setError($code); $self->_setErrorString("Schedules Direct status request response message: $message ($code)"); $self->_CroakOrCarp; $return = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $self->{'_Status'} = $r; $self->{'_StatusAcquired'} = $now; $return = $r; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } # # obtainToken # sub obtainToken { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; my ($username, $password, $passwordHash, $existingtoken, undef) = @_; $self->_resetError; my $now = time(); $self->Username($username) if defined($username); $self->Password($password) if defined($password); $self->PasswordHash($passwordHash) if defined($passwordHash); $self->Token($existingtoken) if defined($existingtoken); # Reuse existing token if available, acquired < 20 hours ago, and validated or we can validate if (defined($self->{'_Token'}) && ($self->{'_TokenAcquired'} > ($now - 72000))) { if ($self->{'_TokenValidated'}) { print (STDERR "DEBUG: (re)using current token\n") if ($self->{'Debug'}); $return = $self->Token; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } else { # Try a status request to validate provided token print (STDERR "DEBUG: attempting to validate token\n") if ($self->{'Debug'}); $self->{'_TokenValidated'} = 1; # Avoid infinite recursion by presuming success $self->obtainStatus; if ($self->{'_Error'}) { $self->{'_TokenValidated'} = 0; print (STDERR "DEBUG: unable to validate token, will attempt to obtain a new token\n") if ($self->{'Debug'}); } else { print (STDERR "DEBUG: using validated token\n") if ($self->{'Debug'}); $return = $self->Token; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } } } $self->_resetSession; $self->_resetError; if (!defined($self->{'Username'})) { $return = undef; $self->_setErrorString("Username not provided for obtaining Schedules Direct token"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($self->{'PasswordHash'})) { $return = undef; $self->_setErrorString("Password not provided for obtaining Schedules Direct token"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $request = HTTP::Request->new(POST => "$self->{'RESTUrl'}/token"); my %json_data = ("username" => $self->{'Username'}, "password" => $self->{'PasswordHash'}); $request->content($self->{'_JSON'}->encode(\%json_data)); print (STDERR "DEBUG: HTTP request:\n" . Data::Dumper->new([$request])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $response = $self->{'_LWP'}->request($request); print (STDERR "DEBUG: HTTP response:\n" . Data::Dumper->new([$response])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $responseCode = $response->code(); my $responseContent = $response->decoded_content(); print (STDERR "DEBUG: HTTP decoded response content:\n" . Data::Dumper->new([$responseContent])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); if ($responseCode == 400) { if (!defined($responseContent)) { $return = undef; $self->_setErrorString("HTTP response content could not be decoded for response code 400"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = undef; $self->_setErrorString("HTTP response content was empty for response code 400."); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->decode($responseContent) }; if (!defined($r)) { $return = undef; $self->_setErrorString("HTTP response content was not valid JSON for response code 400: $responseContent)"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $code = $r->{'code'}; my $message = $r->{'message'}; if (!defined($code) || !defined($message)) { $return = undef; $self->_setErrorString("Schedules Direct authorization token response was not valid 400: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (($code != 0) || ("$message" ne "OK")) { $return = undef; $self->_setErrorString("Schedules Direct authorization token request code: $code, message: $message"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $return = undef; $self->_setErrorString("HTTP response code and content inconsistent for code 400: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseCode != 200) { $return = undef; $self->_setErrorString("HTTP response code was $responseCode"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($responseContent)) { $return = undef; $self->_setErrorString("HTTP response content could not be decoded"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if ($responseContent eq '') { $return = undef; $self->_setErrorString("HTTP response content was empty"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $r = eval { $self->{'_JSON'}->decode($responseContent) }; if (!defined($r)) { $return = undef; $self->_setErrorString("HTTP response content was not parseable: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } my $code = $r->{'code'}; my $message = $r->{'message'}; my $token = $r->{'token'}; if (!defined($code) || !defined($message)) { $return = undef; $self->_setErrorString("Schedules Direct authorization token response was not valid: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (($code != 0) || ("$message" ne "OK")) { $return = undef; $self->_setError($code); $self->_setErrorString("Schedules Direct authorization token response code: $code, message: $message"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } if (!defined($token)) { $return = undef; $self->_setErrorString("Schedules Direct authorization token was not returned: $responseContent"); $self->_CroakOrCarp; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } $self->{'_Token'} = $token; $self->{'_TokenAcquired'} = $now; $self->{'_TokenValidated'} = 1; $return = $self->Token; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } sub _resetError { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . "\n") if ($self->{'Debug'}); $self->{'_Error'} = 0; $self->{'_ErrorString'} = ''; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . "\n") if ($self->{'Debug'}); return; } sub _setError { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; if (@_) { $self->{'_Error'} = shift || (-1) } $return = $self->{'_Error'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } sub _setErrorString { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . " with args: \n" . Data::Dumper->new(\@_)->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); my $return; $self->{'_Error'} = (-1) if (!defined($self->{'_Error'}) || $self->{'_Error'} == 0); if (@_) { $self->{'_ErrorString'} = shift || '' } $return = $self->{'_ErrorString'}; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . " with: \n" . Data::Dumper->new([$return])->Pad('DEBUG: ')->Useqq(1)->Dump) if ($self->{'Debug'}); return $return; } sub _resetSession { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . "\n") if ($self->{'Debug'}); $self->{'_Token'} = undef; $self->{'_TokenAcquired'} = 0; $self->{'_TokenValidated'} = 0; $self->{'_Status'} = undef; print (STDERR "DEBUG: Returning from " . (caller(0))[3] . "\n") if ($self->{'Debug'}); return; } sub _CroakOrCarp { my $self = shift; print (STDERR "DEBUG: Entering " . (caller(0))[3] . "\n") if ($self->{'Debug'}); if ($self->{'_Error'}) { if ($self->{'RaiseError'}) { Carp::croak($self->{'_ErrorString'}); } if ($self->{'PrintError'}) { Carp::carp($self->{'_ErrorString'}); } } print (STDERR "DEBUG: Returning from " . (caller(0))[3] . "\n") if ($self->{'Debug'}); return; } 1; # vim: set expandtab tabstop=2 shiftwidth=2 softtabstop=2 autoindent smartindent filetype=perl syntax=perl: xmltv-1.4.0/lib/000077500000000000000000000000001500074233200134305ustar00rootroot00000000000000xmltv-1.4.0/lib/Ask.pm000066400000000000000000000027701500074233200145120ustar00rootroot00000000000000# A few routines for asking the user questions. Used in --configure # and also by Makefile.PL, so this file should not depend on any # nonstandard libraries. package XMLTV::Ask; use strict; use XMLTV::GUI; use XMLTV::ProgressBar; use vars qw(@ISA @EXPORT); use Exporter; @ISA = qw(Exporter); @EXPORT = qw(ask ask_password ask_choice ask_boolean ask_many_boolean say ); # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } my $real_class = 'XMLTV::Ask::Term'; sub AUTOLOAD { use vars qw($AUTOLOAD); (my $method_name = $AUTOLOAD) =~ s/.*::(.*?)/$1/; (my $real_class_path = $real_class.".pm") =~ s/::/\//g; require $real_class_path; import $real_class_path; $real_class->$method_name(@_); } # Must be called before we use this module if we want to use a gui. sub init( $ ) { my $opt_gui = shift; # Ask the XMLTV::GUI module for the graphics type we will use my $gui_type = XMLTV::GUI::get_gui_type($opt_gui); if ($gui_type =~ /^term/) { $real_class = 'XMLTV::Ask::Term'; } elsif ($gui_type eq 'tk') { $real_class = 'XMLTV::Ask::Tk'; } else { die "Unknown gui type: '$gui_type'."; } # Initialise the ProgressBar module XMLTV::ProgressBar::init($opt_gui); } 1; xmltv-1.4.0/lib/Ask/000077500000000000000000000000001500074233200141465ustar00rootroot00000000000000xmltv-1.4.0/lib/Ask/Term.pm000066400000000000000000000134041500074233200154150ustar00rootroot00000000000000# A few routines for asking the user questions. Used in --configure # and also by Makefile.PL, so this file should not depend on any # nonstandard libraries. # # package XMLTV::Ask::Term; use strict; use Carp qw(croak carp); use Term::ReadKey; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } sub ask( $$ ); sub ask_password( $$ ); sub ask_choice( $$$@ ); sub ask_boolean( $$$ ); sub ask_many_boolean( $$@ ); sub say( $$ ); # Ask a question with a free text answer. # Parameters: # current module # question text # Returns the text entered by the user. sub ask( $$ ) { shift; my $prompt = shift; chomp $prompt; $prompt .= ' ' if $prompt !~ /\s$/; print STDERR $prompt; my $r = ; for ($r) { return undef if not defined; s/^\s+//; s/\s+$//; return $_; } } # Ask a question with a password answer. # Parameters: # current module # question text # Returns the text entered by the user. sub ask_password( $$ ) { shift; my $prompt = shift; chomp $prompt; $prompt .= ' ' if $prompt !~ /\s$/; print STDERR $prompt; Term::ReadKey::ReadMode('noecho'); chomp( my $r = ); Term::ReadKey::ReadMode('restore'); print STDERR "\n"; return $r; } # Ask a question where the answer is one of a set of alternatives. # # Parameters: # current module # question text # default choice # Remaining arguments are the choices available. # # Returns one of the choices, or undef if input could not be read. # sub ask_choice( $$$@ ) { shift; my $question=shift(@_); die if not defined $question; chomp $question; my $default=shift(@_); die if not defined $default; my @options=@_; die if not @options; t "asking question $question, default $default"; croak "default $default not in options" if not grep { $_ eq $default } @options; my $options_size = length("@options"); t "size of options: $options_size"; my $all_digits = not ((my $tmp = join('', @options)) =~ tr/0-9//c); t "all digits? $all_digits"; if ($options_size < 20 or $all_digits) { # Simple style, one line question. my $str = "$question [".join(',',@options)." (default=$default)] "; while ( 1 ) { my $res=ask(undef, $str); return undef if not defined $res; return $default if $res eq ''; # Single character shortcut for yes/no questions return 'yes' if $res =~ /^y$/i; return 'no' if $res =~ /^n$/i; # Check for exact match, then for substring matching. foreach (@options) { return $_ if $_ eq $res; } my @poss; foreach (@options) { push @poss, $_ if /\Q$res\E/i; } if (@poss == 1) { # Unambiguous substring match. return $poss[0]; } print STDERR "invalid response, please choose one of ".join(',', @options)."\n\n"; } } else { # Long list of options, present as numbered multiple choice. print STDERR "$question\n"; my $optnum = 0; my (%num_to_choice, %choice_to_num); foreach (@options) { print STDERR "$optnum: $_\n"; $num_to_choice{$optnum} = $_; $choice_to_num{$_} = $optnum; ++ $optnum; } $optnum--; my $r=undef; my $default_num = $choice_to_num{$default}; die if not defined $default_num; until (defined $r) { $r = ask_choice(undef, 'Select one:', $default_num, 0 .. $optnum); return undef if not defined $r; for ($num_to_choice{$r}) { return $_ if defined } print STDERR "invalid response, please choose one of " .0 .. $optnum."\n\n"; undef $r; } } } # Ask a yes/no question. # # Parameters: # current module # question text # default (true or false) # # Returns true or false, or undef if input could not be read. # sub ask_boolean( $$$ ) { shift; my ($text, $default) = @_; my $r = ask_choice(undef, $text, ($default ? 'yes' : 'no'), 'yes', 'no'); return undef if not defined $r; return 1 if $r eq 'yes'; return 0 if $r eq 'no'; die; } # Ask yes/no questions with option 'default to all'. # # Parameters: # current module # default (true or false), # question texts (one per question). # # Returns: lots of booleans, one for each question. If input cannot # be read, then a partial list is returned. # sub ask_many_boolean( $$@ ) { shift; my $default = shift; # Catch a common mistake - passing the answer string as default # instead of a Boolean. # carp "default is $default, should be 0 or 1" if $default ne '0' and $default ne '1'; my @r; while (@_) { my $q = shift @_; my $r = ask_choice(undef, $q, ($default ? 'yes' : 'no'), 'yes', 'no', 'all', 'none'); last if not defined $r; if ($r eq 'yes') { push @r, 1; } elsif ($r eq 'no') { push @r, 0; } elsif ($r eq 'all' or $r eq 'none') { my $bool = ($r eq 'all'); push @r, $bool; foreach (@_) { print STDERR "$_ ", ($bool ? 'yes' : 'no'), "\n"; push @r, $bool; } last; } else { die } } return @r; } # Give some information to the user # Parameters: # current module # text to show to the user sub say( $$ ) { shift; my $question = shift; print STDERR "$question\n"; } 1; xmltv-1.4.0/lib/Ask/Tk.pm000066400000000000000000000170531500074233200150700ustar00rootroot00000000000000# A few GUI routines for asking the user questions using the Tk library. package XMLTV::Ask::Tk; use strict; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } use Tk; my $main_window; my $top_frame; my $middle_frame; my $bottom_frame; my $mid_bottom_frame; # Ask a question with a free text answer. # Parameters: # current module # question text # what character to show instead of the one typed # Returns the text entered by the user. sub ask( $$$ ) { shift; my $question = shift; my $show = shift; my $textbox; $main_window = MainWindow->new; $main_window->title("Question"); $main_window->minsize(qw(400 250)); $main_window->geometry('+250+150'); $top_frame = $main_window->Frame()->pack; $middle_frame = $main_window->Frame()->pack; $bottom_frame = $main_window->Frame()->pack(-side => 'bottom'); $top_frame->Label(-height => 2)->pack; $top_frame->Label(-text => $question)->pack; my $ans; $bottom_frame->Button(-text => "OK", -command => sub {$ans = $textbox->get(); $main_window->destroy;}, -width => 10 )->pack(-padx => 2, -pady => 4); if (defined $show) { $textbox = $middle_frame->Entry(-show => $show)->pack(); } else { $textbox = $middle_frame->Entry()->pack(); } MainLoop(); return $ans; } # Ask a question with a password answer. # Parameters: # current module # question text # Returns the text entered by the user. sub ask_password( $$ ) { ask($_[0], $_[1], "*") } # Ask a question where the answer is one of a set of alternatives. # # Parameters: # current module # question text # default choice # Remaining arguments are the choices available. # # Returns one of the choices, or undef if input could not be read. # sub ask_choice( $$$@ ) { shift; my $question = shift; die if not defined $question; my $default = shift; die if not defined $default; my @options = @_; die if not @options; t "asking question $question, default $default"; warn "default $default not in options" if not grep { $_ eq $default } @options; return _ask_choices( $question, $default, 0, @options ); } # Ask a yes/no question. # # Parameters: # current module # question text # default (true or false) # # Returns true or false, or undef if input could not be read. # sub ask_boolean( $$$ ) { shift; my ($text, $default) = @_; t "asking question $text, default $default"; $main_window = MainWindow->new; $main_window->title('Question'); $main_window->minsize(qw(400 250)); $main_window->geometry('+250+150'); $top_frame = $main_window->Frame()->pack; $middle_frame = $main_window->Frame()->pack; $bottom_frame = $main_window->Frame()->pack(-side => 'bottom'); $top_frame->Label(-height => 2)->pack; $top_frame->Label(-text => $text)->pack; my $ans = 0; $bottom_frame->Button(-text => "Yes", -command => sub { $ans = 1; $main_window->destroy; }, -width => 10, )->pack(-side => 'left', -padx => 2, -pady => 4); $bottom_frame->Button(-text => "No", -command => sub { $ans = 0; $main_window->destroy; }, -width => 10 )->pack(-side => 'left', -padx => 2, -pady => 4); MainLoop(); return $ans; } # Ask yes/no questions with option 'default to all'. # # Parameters: # current module # default (true or false), # question texts (one per question). # # Returns: lots of booleans, one for each question. If input cannot # be read, then a partial list is returned. # sub ask_many_boolean( $$@ ) { shift; my $default=shift; my @options = @_; return _ask_choices('', $default, 1, @options); } # A helper routine used to create the listbox for both # ask_choice and ask_many_boolean sub _ask_choices( $$$@ ) { my $question=shift; my $default=shift; my $allowedMany=shift; my @options = @_; return if not @options; my $select_all_button; my $select_none_button; my $listbox; my $i; $main_window = MainWindow->new; $main_window->title('Question'); $main_window->minsize(qw( 400 250 )); $main_window->geometry('+250+150'); $top_frame = $main_window->Frame()->pack; $middle_frame = $main_window->Frame()->pack(-fill => 'both'); $top_frame->Label(-height => 2)->pack; $top_frame->Label(-text => $question)->pack; $listbox = $middle_frame->ScrlListbox(); $listbox->insert(0, @options); if ($allowedMany) { $listbox->configure( -selectmode => 'multiple' ); if ($default) { $listbox->selectionSet( 0, 'end' ); } $mid_bottom_frame = $main_window->Frame()->pack(); $select_all_button = $mid_bottom_frame->Button (-text => 'Select All', -command => sub { $listbox->selectionSet(0, 1000) }, -width => 10, )->pack(-side => 'left'); $select_none_button = $mid_bottom_frame->Button (-text => 'Select None', -command => sub { $listbox->selectionClear(0, 1000) }, -width => 10, )-> pack(-side => 'right'); } else { $listbox->configure(-selectmode => 'single'); $listbox->selectionSet(_index_array($default, @options)); } $listbox->pack(-fill => 'x', -padx => '5', -pady => '2'); $bottom_frame = $main_window->Frame()->pack(-side => 'bottom'); my @cursel; $bottom_frame->Button(-text => 'OK', -command => sub { @cursel = $listbox->curselection; $main_window->destroy; }, -width => 10, )->pack(-padx => 2, -pady => 4); MainLoop(); if ($allowedMany) { my @choices; my @choice_numbers = @cursel; $i=0; foreach (@options) { push @choices, 0; foreach( @choice_numbers ) { if ($options[$_] eq $options[$i]) { $choices[$i] = 1; } } $i++; } return @choices; } else { my $ans = $options[$cursel[0]]; return $ans; } } # Give some information to the user # Parameters: # current module # text to show to the user sub say( $$ ) { shift; my $question = shift; $main_window = MainWindow->new; $main_window->title("Information"); $main_window->minsize(qw(400 250)); $main_window->geometry('+250+150'); $top_frame = $main_window->Frame()->pack; $middle_frame = $main_window->Frame()->pack; $bottom_frame = $main_window->Frame()->pack(-side => 'bottom'); $top_frame->Label(-height => 2)->pack; $top_frame->Label(-text => $question)->pack; $bottom_frame->Button(-text => "OK", -command => sub { $main_window->destroy; }, -width => 10, )->pack(-padx => 2, -pady => 4); MainLoop(); } # A hekper routine that returns the index in an array # of the supplied argument # Parameters: # the item to find # the array to find it in # Returns the index of the item in the array, or -1 if not found sub _index_array($@) { my $s=shift; my @array = @_; for (my $i = 0; $i < $#array; $i++) { return $i if $array[$i] eq $s; } return -1; } 1; xmltv-1.4.0/lib/Augment.pm000066400000000000000000003640031500074233200153740ustar00rootroot00000000000000=pod =head1 NAME XMLTV::Augment - Augment XMLTV listings files with automatic and user-defined rules. =head1 DESCRIPTION Augment an XMLTV xml file by applying corrections ("fixups") to programmes matching defined criteria ("rules"). Two types of rules are actioned: (i) automatic, (ii) user-defined. Automatic rules use pre-programmed input and output to modify the input programmes. E.g. removing a "title" where it is repeated in a "sub-title" (e.g. "Horizon" / "Horizon: Star Wars"), or trying to identify and extract series/episode numbers from the programme"s title, sub-title or description. User-defined rules use the content of a "rules" file which allows programmes matching certain user-defined criteria to be corrected/enhanced with the user data supplied (e.g. adding/changing categories for all episodes of "Horizon", or fixing misspellings in titles, etc.) By setting appropriate options in the "config" file, the "rules" file can be automatically downloaded using XMLTV::Supplement. =head1 EXPORTED FUNCTIONS =over =item B> Set the assumed encoding of the rules file. =item B> Store each channel found in the input programmes file for later processing by "stats". =item B> Augment a programme using (i) pre-determined rules and (ii) user-defined rules. Which rules are processed is determined by the options set in the "config" file. =item B> Print the lists of actions taken and suggestions for further fixups. =item B> Do any final processing before exit (e.g. close the log file if necessary). =back =head1 INSTANTIATION new XMLTV::Augment( { ...parameters...} ); Possible parameters: rule => filename of file containing fixup rules (if omitted then no user-defined rules will be actioned) (overrides auto-fetch Supplement if that is defined; see sample options file) config => filename of config file to read (if omitted then no config file will be used) encoding => assumed encoding of the rules file (default = UTF-8) stats => whether to print the audit stats in the log (values = 0,1) (default = 1) log => filename of output log debug => debug level (values 0-10) (default = no debug) note debug level > 3 is not likely to be of much use unless you are developing code =head1 TYPICAL USAGE 1) Create the XMLTV::Augment object 2) Pass each channel to inputChannel() 3) Pass each programme to augmentProgramme() 4) Tidy up using printInfo() & end() #instantiate the object my $augment = new XMLTV::Augment( "rule" => "myrules.txt", "config" => "myconfig.txt", "log" => "augment.log", ); die "failed to create XMLTV::Augment object" if !$augment; for each channel... { # store the channel details $augment->inputChannel( $ch ); } for each programme... { # augmentProgramme will now do any requested processing of the input xml $prog = $augment->augmentProgramme( $prog ); } # log the stats $augment->printInfo(); # close the log file if necessary $augment->end(); Note: you are responsible for reading/writing to the XMLTV .xml file; the package will not do that for you. =head1 RULES =over =cut # TODO # ==== # # Handle multiple 'title' in the $prog # # Routine to validate 'rules' file and report errors / inconsistencies # # Modify rule #3 processing - currently we have to check every type 3 fixup for every programme! # # ? Add a rule to prioritise the categories. Use case: some grabbers generate multiple categories for a programme, # but some downstream apps (e.g. MythTV) can only handle 1 category. This means the "best" category may not be # visible to Myth users (i.e. they just see the first one in the list which may not be the most appropriate one). # package XMLTV::Augment; use XMLTV::Date 0.005066 qw( time_xmltv_to_epoch ); use Encode; our $VERSION = 0.006001; use base 'Exporter'; our @EXPORT = qw(setEncoding inputChannel augmentProgramme printInfo end); # use XMLTV::Supplement qw/GetSupplement SetSupplementRoot/; my $debug = 0; my $logh; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub { print STDERR @_ . "\n"; }; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } # Constructor. # Takes an array of params+value. # # 'rule' => file containing fixup rules # 'config' => config file to read # 'encoding' => assumed encoding of the rules file # 'stats' => whether to print the 'audit' stats in the log # 'log' => filename of output log # 'debug' => debug level # sub new { my ($class) = shift; my $self={ @_ }; # remaining args become attributes # check we have required arguments #for ('rule', 'config') { # die "invalid usage - no $_" if !defined($self->{$_}); #} # Encoding of the rules file $self->{'encoding'} = 'UTF-8' if !defined($self->{'encoding'}); # Does user want stats printed in the log file? $self->{'stats'} = 1 if !defined($self->{'stats'}); bless($self, $class); # Turn on debug. Note: debug is a 'level between 1-10. $debug = ( $self->{'debug'} || 0 ); if (exists $self->{'log'} && defined $self->{'log'} && $self->{'log'} ne '') { open_log( $self->{'log'} ); } else { $self->{'stats'} = 0; } # Load the requested options from the config file $self->{'options_all'} = 1; $self->{'options'} = {}; $self->load_config($self->{'config'}) if defined($self->{'config'}) && $self->{'config'} ne ''; $self->{'options_all'} = 0 if defined $self->{'options'}->{'enable_all_options'} && $self->{'options'}->{'enable_all_options'} == 0; $self->{'language_code'} = $self->{'options'}{'language_code'}; # e.g. 'en' or undef l("\n".'Data shown in brackets after each processing entry refers to the rule type'."\n".' and line number in the rules file, e.g. "(#3.103)" means rule type 3 on line 103 was applied.'."\n"); # Hash to store the loaded rules $self->{'rules'} = {}; # Read in the 'rules' file. Barf on error. if ( $self->load_rules( $self->{'rule'} ) > 0 ){ return undef; } # Hash to store the augmentation results $self->{'audit'} = {}; return $self; } # Do any final processing before we exit. # sub end () { close_log(); } # Set the assumed encoding of the rules file. # sub setEncoding () { my ($self, $encoding) = @_; $self->{'encoding'} = ($encoding ne '') ? $encoding : 'UTF-8'; } # Store each channel found in the input programmes file # for later processing by 'stats'. # sub inputChannel () { my ($self, $channel) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $value = 'input_channels'; my $key = $channel->{'id'}; my $ch_name = ( defined $channel->{'display-name'} ? $channel->{'display-name'}[0][0] : '' ); $self->{'audit'}{$value}{$key}{'display_name'} = $ch_name; } # Augment a programme using (i) pre-determined rules and (ii) user-defined rules. # Which rules are processed is determined by the options set in the 'config' file. # sub augmentProgramme () { my ($self, $prog) = @_; _d(3,'Prog in:',dd(3,$prog)); l("Processing title~~~episode : {" . $prog->{'title'}[0][0] . '~~~' . (defined $prog->{'sub-title'} ? $prog->{'sub-title'}[0][0] : '') . "}" ); # Remove "New $title" if seen in episode field (rule A1) $self->remove_duplicated_new_title_in_ep($prog); # Remove a duplicated programme title/ep if seen in episode field (rule A2) $self->remove_duplicated_title_and_ep_in_ep($prog); # Remove a duplicated programme title if seen in episode field (rule A3) $self->remove_duplicated_title_in_ep($prog); # Check description for possible premiere/repeat hints (rule A4) $self->update_premiere_repeat_flags_from_desc($prog); # Look for series/episode/part numbering in programme title/subtitle/description (rule A5) $self->check_potential_numbering_in_text($prog); # Title and episode processing. (user rules) # We process titles if the user has # not explicitly disabled title processing during configuration # and we have supplement data to process programmes against. $self->process_user_rules($prog); # Look again for series/episode/part numbering in programme title/subtitle/description (rule A5) # This is to allow any new series/episode/part numbering added via a user rule to be extracted. $self->check_potential_numbering_in_text($prog); # Tidy text after title processing $self->tidy_title_text($prog); # Tidy <sub-title> (episode) text after title processing $self->tidy_episode_text($prog); # Tidy $desc text after title processing $self->tidy_desc_text($prog); # Add missing language codes to <title>, <sub-title> and <desc> elements $self->add_missing_language_codes($prog); l("\t Post-processing title/episode: {" . $prog->{'title'}[0][0] . '~~~' . (defined $prog->{'sub-title'} ? $prog->{'sub-title'}[0][0] : '') . "}" ); # Store title debug info for later analysis # (printed out in the log for manual inspection - # allows you to check for new rule requirements) $self->store_title_debug_info($prog); # Store genre debug info for later analysis $self->store_genre_debug_info($prog); _d(3,'Prog out:',dd(3,$prog)); return $prog; } # Tidy <title> sub tidy_title_text () { my ($self, $prog) = @_; if (defined $prog->{'title'}) { for (my $i=0; $i < scalar @{$prog->{'title'}}; $i++) { # replace repeated spaces $prog->{'title'}[$i][0] =~ s/\s+/ /g; # remove trailing character if any of .,:;-| and not ellipsis # bug #503 : don't remove trailing period if it could be an abbreviation, e.g. 'M.I.A.' $prog->{'title'}[$i][0] =~ s/[|\.,:;-]$// if $prog->{'title'}[$i][0] !~ m/\.{3}$/ && $prog->{'title'}[$i][0] !~ m/\..\.$/; } } } # Tidy <sub-title> # Remove <sub-title> if empty/whitespace sub tidy_episode_text () { my ($self, $prog) = @_; if (defined $prog->{'sub-title'}) { for (my $i=0; $i < scalar @{$prog->{'sub-title'}}; $i++) { # replace repeated spaces $prog->{'sub-title'}[$i][0] =~ s/\s+/ /g; # remove trailing character if any of .,:;-| and not ellipsis # bug #503 : don't remove trailing period if it could be an abbreviation, e.g. 'In the U.S.' $prog->{'sub-title'}[$i][0] =~ s/[|\.,:;-]$//g if $prog->{'sub-title'}[$i][0] !~ m/\.{3}$/ && $prog->{'sub-title'}[$i][0] !~ m/\..\.$/; } } # delete sub-title if now empty # TODO: needs modifying to properly handle multiple sub-titles if (defined $prog->{'sub-title'}) { if ( $prog->{'sub-title'}[0][0] =~ m/^\s*$/ ) { splice(@{$prog->{'sub-title'}},0,1); if (scalar @{$prog->{'sub-title'}} == 0) { delete $prog->{'sub-title'}; } } } } # Tidy <desc> description text # Remove <desc> if empty/whitespace sub tidy_desc_text () { my ($self, $prog) = @_; if (defined $prog->{'desc'}) { for (my $i=0; $i < scalar @{$prog->{'desc'}}; $i++) { # replace repeated spaces $prog->{'desc'}[$i][0] =~ s/\s+/ /g; # remove trailing character if any of ,:;-| $prog->{'desc'}[$i][0] =~ s/[|,:;-]$//g; } } # delete desc if now empty # TODO: needs modifying to properly handle multiple descriptions if (defined $prog->{'desc'}) { if ( $prog->{'desc'}[0][0] =~ m/^\s*$/ ) { splice(@{$prog->{'desc'}},0,1); if (scalar @{$prog->{'desc'}} == 0) { delete $prog->{'desc'}; } } } } # Add missing language codes to <title>, <sub-title> and <desc> elements sub add_missing_language_codes () { _d(3,self()); my ($self, $prog) = @_; my @elems = ('title', 'sub-title', 'desc'); foreach my $elem (@elems) { if (defined $prog->{$elem}) { dd(3,$prog->{$elem}); for (my $i=0; $i < scalar @{$prog->{$elem}}; $i++) { # add language code if missing (leave existing codes alone) # my $v = $prog->{$elem}[$i][0]; # $prog->{$elem}[$i] = [ $v, $self->{'language_code'} ]; push @{$prog->{$elem}[$i]}, $self->{'language_code'} if (scalar @{$prog->{$elem}[$i]} == 1); } } } } =item B<remove_duplicated_new_title_in_ep> Rule #A1 Remove "New $title :" from <sub-title> If sub-title starts with "New" + <title> + separator, then it will be removed from the sub-title "separator" can be any of .,:;- in : "Antiques Roadshow / New Antiques Roadshow: Doncaster" out: "Antiques Roadshow / Doncaster" =cut # Rule A1 # # Remove "New $title" from episode field # # Listings may contain "New $title" duplicated at the start of the episode field # sub remove_duplicated_new_title_in_ep () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 'A1'; if (defined $prog->{'sub-title'}) { my $tmp_title = $prog->{'title'}[0][0]; my $tmp_episode = $prog->{'sub-title'}[0][0]; my $key = $tmp_title . "|" . $tmp_episode; # Remove the "New $title" text from episode field if we find it if ( $tmp_episode =~ m/^New \Q$tmp_title\E\s*[\.,:;-]\s*(.+)$/i ) { $prog->{'sub-title'}[0][0] = $1; l(sprintf("\t Removing 'New \$title' text from beginning of episode field (#%s)", $ruletype)); $self->add_to_audit ($me, $key, { '_title'=>$tmp_title, '_episode'=>$tmp_episode }) } } } =item B<remove_duplicated_title_and_ep_in_ep> Rule #A2 Remove duplicated programme title *and* episode from <sub-title> If sub-title starts with <title> + separator + <episode> + separator + <episode>, then it will be removed from the sub-title "separator" can be any of .,:;- in : "Antiques Roadshow / Antiques Roadshow: Doncaster: Doncaster" out: "Antiques Roadshow / Doncaster" =cut # Rule A2 # # Remove duplicated programme title *and* episode from episode field # # Listings may contain the programme title *and* episode duplicated in the episode field: # i) at the start separated from the episode by colon - "$title: $episode: $episode" # sub remove_duplicated_title_and_ep_in_ep () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 'A2'; if (defined $prog->{'sub-title'}) { my $tmp_title = $prog->{'title'}[0][0]; my $tmp_episode = $prog->{'sub-title'}[0][0]; my $key = $tmp_title . "|" . $tmp_episode; # Remove the duplicated title/ep from episode field if we find it # Use a backreference to match the second occurence of the episode text if ( $tmp_episode =~ m/^\Q$tmp_title\E\s*[\.,:;-]\s*(.+)\s*[\.,:;-]\s*\1$/i ) { $prog->{'sub-title'}[0][0] = $1; l(sprintf("\t Removing duplicated title/ep text from episode field (#%s)", $ruletype)); $self->add_to_audit ($me, $key, { '_title'=>$tmp_title, '_episode'=>$tmp_episode }) } } } =item B<remove_duplicated_title_in_ep> Rule #A3 Remove duplicated programme title from <sub-title> i) If sub-title starts with <title> + separator, then it will be removed from the sub-title ii) If sub-title ends with separator + <title>, then it will be removed from the sub-title iii) If sub-title starts with <title>(...), then the sub-title will be set to the text in brackets iv) If sub-title equals <title>, then the sub-title will be removed "separator" can be any of .,:;- in : "Antiques Roadshow / Antiques Roadshow: Doncaster" out: "Antiques Roadshow / Doncaster" in : "Antiques Roadshow / Antiques Roadshow (Doncaster)" out: "Antiques Roadshow / Doncaster" in : "Antiques Roadshow / Antiques Roadshow" out: "Antiques Roadshow / " =cut # Rule A3 # # Remove duplicated programme title from episode field # # Listings may contain the programme title duplicated in the episode field, either: # i) at the start followed by the 'real' episode in parentheses (rare), # ii) at the start separated from the episode by a colon/hyphen, # iii) at the end separated from the episode by a colon/hyphen, # iv) no episode at all (e.g. "Teleshopping" / "Teleshopping") # sub remove_duplicated_title_in_ep () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 'A3'; if (defined $prog->{'sub-title'}) { my $tmp_title = $prog->{'title'}[0][0]; my $tmp_episode = $prog->{'sub-title'}[0][0]; my $key = $tmp_title . "|" . $tmp_episode; # Remove the duplicated title from episode field if we find it if ($tmp_episode =~ m/^\Q$tmp_title\E\s*[\.,:;-]\s*(.+)?$/i || $tmp_episode =~ m/^\Q$tmp_title\E\s+\((.+)\)$/i || $tmp_episode =~ m/^\Q$tmp_title\E\s*$/i ) { $prog->{'sub-title'}[0][0] = defined $1 ? $1 : ''; l(sprintf("\t Removing title text from beginning of episode field (#%s)", $ruletype)); $self->add_to_audit ($me, $key, { '_title'=>$tmp_title, '_episode'=>$tmp_episode }) } # Look for title appearing at end of episode field elsif ($tmp_episode =~ m/^(.+?)\s*[\.,:;-]\s*\Q$tmp_title\E$/i ) { $prog->{'sub-title'}[0][0] = $1; l(sprintf("\t Removing title text from end of episode field (#%s)", $ruletype)); $self->add_to_audit ($me, $key, { '_title'=>$tmp_title, '_episode'=>$tmp_episode }) } } } =item B<update_premiere_repeat_flags_from_desc> Rule #A4 Set the <premiere> element and remove any <previously-shown> element if <desc> starts with "Premiere." or "New series". Remove the "Premiere." text. Set the <previously-shown> element and remove any <premiere> element if <desc> starts with "Another chance" or "Rerun" or "Repeat" =cut # Rule A4 # # Update the premiere/repeat flags based on contents of programme desc # Do this before check_potential_numbering_in_text() # sub update_premiere_repeat_flags_from_desc () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 'A4'; my $tmp_title = $prog->{'title'}[0][0]; my $tmp_episode = (defined $prog->{'sub-title'} ? $prog->{'sub-title'}[0][0] : ''); #(always remove the "Premiere." text even if <premiere> is already set) #if (!defined $prog->{'premiere'}) { if (defined $prog->{'desc'}) { my $key = $prog->{'title'}[0][0]; # Check if desc start with "Premiere.". Remove if found and set flag if ($prog->{'desc'}[0][0] =~ s/^Premiere\.\s*//i ) { l("\t Setting premiere flag based on description (Premiere. )"); $prog->{'premiere'} = []; delete $prog->{'previously-shown'}; $self->add_to_audit ($me, $key, { '_title'=>$tmp_title, '_episode'=>$tmp_episode }) } # Check if desc starts with "New series..." elsif ($prog->{'desc'}[0][0] =~ m/^New series/i ) { l("\t Setting premiere flag based on description (New series...)"); $prog->{'premiere'} = []; delete $prog->{'previously-shown'}; $self->add_to_audit ($me, $key, { '_title'=>$tmp_title, '_episode'=>$tmp_episode }) } } #} if (!defined $prog->{'previously-shown'}) { if (defined $prog->{'desc'}) { my $key = $prog->{'title'}[0][0]; # Flag showings described as repeats if ($prog->{'desc'}[0][0] =~ m/^(Another chance|Rerun|Repeat)/i ) { l("\t Setting repeat flag based on description (Another chance...)"); $prog->{'previously-shown'} = {}; delete $prog->{'premiere'}; $self->add_to_audit ($me, $key, { '_title'=>$tmp_title, '_episode'=>$tmp_episode }) } } } } =item B<check_potential_numbering_in_text> Rule #A5 Check for potential series, episode and part numbering in the title, episode and description fields. =cut # Rule A5 # # Check for potential episode numbering in the title # or episode or description fields # sub check_potential_numbering_in_text () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); _d(5,'Prog, before potential numbering:',dd(5,$prog)); # extract the existing episode-num my $xmltv_ns = ''; my $episode_num = $self->extract_ns_epnum($prog, \$xmltv_ns); # make a work copy of $prog my $_prog = {'_title' => (defined $prog->{'title'} ? $prog->{'title'}[0][0] : undef), '_episode' => (defined $prog->{'sub-title'} ? $prog->{'sub-title'}[0][0] : undef), '_desc' => (defined $prog->{'desc'} ? $prog->{'desc'}[0][0] : undef), '_series_num' => $episode_num->{'season'}, '_series_total' => $episode_num->{'season_total'}, '_episode_num' => $episode_num->{'episode'}, '_episode_total' => $episode_num->{'episode_total'}, '_part_num' => $episode_num->{'part'}, '_part_total' => $episode_num->{'part_total'}, }; _d(4,'_Prog, before numbering:',dd(4,$_prog)); my $t_title = $_prog->{'_title'}.' / '.($_prog->{'_episode'} || 'undef'); $self->extract_numbering_from_episode($_prog); $self->extract_numbering_from_title($_prog); $self->extract_numbering_from_desc($_prog); $self->make_episode_from_part_numbers($_prog); _d(4,'_Prog, after numbering:',dd(4,$_prog)); # Writer will barf if the title is empty if (!defined $_prog->{'_title'} || $_prog->{'_title'} eq '') { _d(0,"Prog title is now empty! Was \{$t_title\} Now {",$_prog->{'_title'},' / ',($_prog->{'_episode'} || 'undef').'}'); $_prog->{'_title'} = '(no title)'; } # update the title and sub-title and description in the programme $prog->{'title'}[0][0] = $_prog->{'_title'} if defined $_prog->{'_title'}; $prog->{'sub-title'}[0][0] = $_prog->{'_episode'} if defined $_prog->{'_episode'}; $prog->{'desc'}[0][0] = $_prog->{'_desc'} if defined $_prog->{'_desc'}; # update the episode-num $episode_num->{'season'} = $_prog->{'_series_num'}; $episode_num->{'season_total'} = $_prog->{'_series_total'}; $episode_num->{'episode'} = $_prog->{'_episode_num'}; $episode_num->{'episode_total'} = $_prog->{'_episode_total'}; $episode_num->{'part'} = $_prog->{'_part_num'}; $episode_num->{'part_total'} = $_prog->{'_part_total'}; # remake the episode-num my $xmltv_ns_new = $self->make_ns_epnum($prog, $episode_num); if ($xmltv_ns_new ne $xmltv_ns) { $key = $_prog->{'_title'}; $self->add_to_audit ($me, $key, $_prog); } _d(5,'Prog, after potential numbering:',dd(5,$prog)); } =item B<extract_numbering_from_title> Rule #A5.1 Extract series/episode numbering found in <title>. =cut # Rule A5.1 # # Check for potential season numbering in <title> # sub extract_numbering_from_title () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); if (defined $prog->{'_title'}) { _d(3,'Checking <title> for numbering'); $self->extract_numbering($prog, 'title'); } } =item B<extract_numbering_from_episode> Rule #A5.2 Extract series/episode numbering found in <sub-title>. =cut # Rule A5.2 # # Extract series/episode numbering found in <sub-title>. Series # and episode numbering are parsed out of the text and eventually made # available in the <episode-num> element. # sub extract_numbering_from_episode () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); if (defined $prog->{'_episode'}) { _d(3,'Checking <sub-title> for numbering'); $self->extract_numbering($prog, 'episode'); } } =item B<extract_numbering_from_desc> Rule #A5.3 Extract series/episode numbering found in <desc>. =cut # Rule A5.3 # # Check for potential season/episode numbering in description. Only # use numbering found in the desc if we have not already found it # elsewhere (i.e. prefer data provided in the subtitle field of the # raw data). # sub extract_numbering_from_desc () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); if (defined $prog->{'_desc'}) { _d(3,'Checking <desc> for numbering'); $self->extract_numbering($prog, 'desc'); } } # used by: extract_numbering_from_title(), extract_numbering_from_episode(), extract_numbering_from_desc() # 2 params: 1) working program hash 2) type: 'title', 'episode' or 'desc' sub extract_numbering () { my ($self, $prog, $field) = @_; my %elems = ( 'title' => '_title', 'episode' => '_episode', 'desc' => '_desc', 'description' => '_desc' ); my $elem = $elems{$field}; my ($s, $stot, $e, $etot, $p, $ptot); my $int; # TODO: set $lang_words according to 'language_code' option # also the 'series', 'season' and 'episode' text in the regexs my $lang_words = qr/one|two|three|four|five|six|seven|eight|nine/i; # By default, we do not update existing numbering if also extracted from # series/episode/part fields $self->{'options'}{'update_existing_numbering'} = 0 unless exists $self->{'options'}{'update_existing_numbering'}; _d(4,"\t extract_numbering: $field : in : ","<$prog->{$elem}>"); # Theoretically it's possible to do this in one regex but it gets too unwieldy when we start catering # for "series" at the front as well as the back, and it's not easy to maintain # so we'll parse out the values in separate passes # First, remove any part numbering from the *end* of the element # Should match --v # Dead Man's Eleven # Dead Man's Eleven: 1 # Dead Man's Eleven (Part 1) # Dead Man's Eleven - (Part 1) # Dead Man's Eleven - (Part 1/2) # Dead Man's Eleven (Pt 1) # Dead Man's Eleven - (Pt. 1) # Dead Man's Eleven - (Pt. 1/2) # Dead Man's Eleven - Part 1 # Dead Man's Eleven: Part 1 # Dead Man's Eleven; Pt 1 # Dead Man's Eleven, Pt. 1 # Dead Man's Eleven Part 1 # Dead Man's Eleven Pt 1 # Dead Man's Eleven Pt 1/2 # Dead Man's Eleven Pt. 1 # Dead Man's Eleven - Part One # Dead Man's Eleven: Part One # Dead Man's Eleven; Pt One # Dead Man's Eleven, Pt. One # Dead Man's Eleven Part One # Dead Man's Eleven Pt One # Dead Man's Eleven Pt. One # Dead Man's Eleven (Part One) # Dead Man's Eleven - (Part One) # Dead Man's Eleven (Pt One) # Dead Man's Eleven - (Pt. One) # Part One # Pt Two # Pt. Three # Part One of Two # Pt Two / Three # Pt. Three of Four # Part 1 # Part 1/3 # Pt 2 # Pt 2/3 # Pt. 3 # # Should not match --v # Burnley v Preston North End: 2006/07 if ( $prog->{$elem} =~ s{ (?: [\s,:;-]* \(? (?: part|pt\.? ) \s* (?! (?:19|20)\d\d ) (\d+ | ${lang_words}) # $1 \s* (?: (?: /|of ) \s* (\d+ | ${lang_words}) # $2 )? \)? | : \s* (\d+) # $3 ) [\s.,:;-]* $ } { }ix ) { _d(4,"\t matched 'part' regex"); $int = word_to_digit($3 || $1); # $3 is in use case "Dead Man's Eleven: 1" $p = $int if defined $int and $int > 0; $int = word_to_digit($2) if defined $2; $ptot = $int if defined $2 && defined $int and $int > 0; } # Next, extract and strip "series x/y" # # Should match --v # Series 1 # Series one : # Series 2/4 # Series 12. Abc # Series 1. # Wheeler Dealers - (Series 1) # Wheeler Dealers, - (Series 1.) # Wheeler Dealers (Series 1, Episode 4) # Wheeler Dealers Series 1, Episode 4. # Series 8. Abc # Series 8/10. Abc # Wheeler Dealers - (Series 1) # Wheeler Dealers (Season 1) # Wheeler Dealers Series 1 # Wheeler Dealers Series 1, 3 # # Does not match --v # Series 4/7. Part one. Abc # Series 6, Episode 4/7. Part one. Abc if ( $prog->{$elem} =~ s{ (?: [\s.,:;-]* \(? ) (?: series|season ) \s* ( \d+ | ${lang_words} ) # $1 (?: [\s/]* ( \d+ ) # $2 )? [.,]? \)? [\s\.,:;-]* } { }ix ) { _d(4,"\t matched 'series' regex"); $int = word_to_digit($1); $s = $int if defined $int and $int > 0; $stot = $2 if defined $2; } # Extract and strip the "episode" # # i) check for "Episode x/x" format covering following formats: # # Should match --v # Episode one : # Episode 2/4 # Episode 12. Abc # Episode 1. # Wheeler Dealers - (Episode 1) # Wheeler Dealers, - (Episode 1.) # Wheeler Dealers (Series 1, Episode 4) # Wheeler Dealers Series 1, Episode 4. # # Should not match --v # Series 8. Abc # Series 8/10. Abc # 1/6 - Abc # 1/6, series 1 - Abc # 1, series 1 - Abc # 1/6, series one - Abc # 1/6. Abc # 1/6; series one # 1, series one - Abc # # Does not match --v # Episode 4/7. Part one. Abc # Series 6, Episode 4/7. Part one. Abc if ( $prog->{$elem} =~ s{ (?: [\s.,:;-]* \(? ) (?: episode ) \s* (\d+ | ${lang_words}) # $1 (?: [\s/]* (\d+) # $2 )? [.,]? \)? [\s.,:;-]* } { }ix ) { _d(4,"\t matched 'episode' regex"); $int = word_to_digit($1); $e = $int if defined $int and $int > 0; $etot = $2 if defined $2; } # Extract and strip the episode "x/y" if number at start of data # # Note: beware of false positives with e.g. "10, Rillington Place" or "1984". # Those entries below tagged "<-- cannot match" are not matched to avoid # false positives c.f. "10, Rillington Place") # # I'm not convinced we should be matching things like "1." - can we be sure # this is an ep number? # # Should match --v # 1/6 - Abc # 1/6, series 1 - Abc # 1/6, series one - Abc # 1/6. Abc # 1/6; series one # 4/25 Wirral v Alicante, Spain # 1/6 - Abc # 1/6, Abc # 1/6. Abc # 1/6; # (1/6) # [1/6] # 1. # 1, # 2/25 Female Problems # # Should not match --v # 1, series 1 - Abc <-- cannot match # 1, series one - Abc <-- cannot match # 1, Abc <-- cannot match # 3rd Rock # 3 rd Rock # 10, Rillington Place # 10 Rillington Place # 1984 # 1984. <-- false positive # Episode 1 # Episode one # Episode 2/4 # {Premier League Years~~~1999/00} elsif ( $prog->{$elem} =~ # note we insist on the "/" unless the title is just "number." or "number," # this is to avoid false matching on "1984" but even here we will falsely match "1984." # s{ ^ [([]* (?! (?:19|20)\d\d ) (\d+) # $1 \s* (?: / \s* | [.,] $ ) (\d*) # $2 [.,]? [)\]]* \s? (?: [\s.,:;-]+ \s* | \s* $ ) } {}ix ) { _d(4,"\t matched 'leading episode' regex"); $int = word_to_digit($1); $e = $int if defined $int and $int > 0; $etot = $2 if defined $2; } # Extract and strip the episode "x/y" if number at end of data # # Should match --v # 1/6 # 1/6. # (1/6) # [1/6] # 1 / 6 # ( 1/6 ) # In the Mix. 2 / 6 # In the Mix. ( 2/6 ). # # Should not match --v # £20,000. # 20,000 # the 24. # go 24. # 24 # 1984 # 2015/16 # 2015/2016 (could theoretically be ok but statistically unlikely) # 2015/3000 (could be ok but the likes of Eastenders don't have a 'total' so again this is unlikely) elsif ( $prog->{$elem} =~ s{ (?: ^ | [\s(\[]+ ) (?! (?:19|20)\d\d ) (\d+) # $1 \s* \/ \s* (\d+) # $2 [\s.,]? [)\]]* [\s.]* $ } {}ix ) { _d(4,"\t matched 'trailing episode' regex"); $int = word_to_digit($1); $e = $int if defined $int and $int > 0; $etot = $2 if defined $2; } # Extract and strip the series/episode "sXeY" at end of data # # Should match --v # e6 # [E6] # s1e6 # (s1e6) # In the Mix. S1E6 # In the Mix. ( S1E6 ). elsif ( $prog->{$elem} =~ s{ (?: ^ | [\s([]+ ) s? (\d+)? # $1 \s* e \s* (\d+) # $2 [\s.,]? [)\]]* [\s.]* $ } {}ix ) { _d(4,"\t matched 'trailing series/episode' regex"); $s = $1 if defined $1 and $1 > 0; $e = $2 if defined $2 and $2 > 0; } # tidy any leading/trailing spaces we've left behind trim($prog->{$elem}); # _d(4,"\t extract_numbering: $field : out : ","<$prog->{$elem}>"); my @vals = ( [ 'series', '_series_num', $s ], [ 'series total', '_series_total', $stot ], [ 'episode', '_episode_num', $e ], [ 'episode total', '_episode_total', $etot ], [ 'part', '_part_num', $p ], [ 'part total', '_part_total', $ptot ] ); foreach (@vals) { my ($text, $key, $val) = @$_; if (defined $val && $val ne '' && $val > 0) { # do we already have a number? if (defined $prog->{$key} && $prog->{$key} != $val) { if ($self->{'options'}{'update_existing_numbering'}) { l(sprintf("\t %s number (%s) already defined. Updating with new %s number (%s) in %s.", ucfirst($text), $prog->{$key}, $text, $val, $field)); $prog->{$key} = $val; } else { l(sprintf("\t %s number (%s) already defined. Ignoring different %s number (%s) in %s.", ucfirst($text), $prog->{$key}, $text, $val, $field)); } } else { l(sprintf("\t %s number found: %s %s (from %s)", ucfirst($text), $text, $val, $field)); $prog->{$key} = $val; } } } # Check that a programme's given series/episode/part number is not greater than # the total number of series/episodes/parts. # # Rather than discard the given number, we discard the total instead, which # is more likely to be incorrect based on observation. # my @comps = ( [ 'series', '_series_num', '_series_total', ], [ 'episode', '_episode_num', '_episode_total', ], [ 'part', '_part_num', '_part_total', ], ); foreach (@comps) { my ($key, $key_num, $key_total) = @$_; if (defined $prog->{$key_num} && defined $prog->{$key_total}) { if ($prog->{$key_num} > $prog->{$key_total}) { l(sprintf("\t Bad %s total found: %s %s of %s, discarding total (from %s)", $key, $key, $prog->{$key_num}, $prog->{$key_total}, $field)); $prog->{$key_total} = 0; } } } } =item B<make_episode_from_part_numbers> Rule #A5.4 If no <sub-title> then make one from "part" numbers. in : "Panorama / " desc = "Part 1/2..." out: "Panorama / Part 1 of 2" =cut # Rule A5.4 # # if no episode title then make one from part numbers # sub make_episode_from_part_numbers () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); if (!defined $prog->{'_episode'} || $prog->{'_episode'} =~ m/^\s*$/ ) { if (defined $prog->{'_part_num'} ) { _d(4,"\t creating 'episode' from part number(s)"); # no episode title so make one $prog->{'_episode'} = "Part $prog->{'_part_num'}" . ($prog->{'_part_total'} ? ' of '.$prog->{'_part_total'} : ''); l(sprintf("\t Created episode from part number(s): %s", $prog->{'_episode'})); $self->add_to_audit ($me, $prog->{'_title'}, $prog); } } } =item B<process_user_rules> Rule #user Process programme against user-defined fixups The individual rules each have their own option to run or not; consider this like an on/off switch for all of them. I.e. if this option is off then no user rules will be run (irrespective of any other option flags). =cut # Rule user # # Process programme against user-defined title fixups # sub process_user_rules () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); _d(5,'Prog, before title fixups:',dd(5,$prog)); # extract the existing episode-num my $xmltv_ns = ''; my $episode_num = $self->extract_ns_epnum($prog, \$xmltv_ns); # make a work copy of $prog # (this is mainly for ease of use with the _uk_rt code on which this class is based) # my $_prog = {'_title' => (defined $prog->{'title'} ? $prog->{'title'}[0][0] : undef), '_episode' => (defined $prog->{'sub-title'} ? $prog->{'sub-title'}[0][0] : undef), '_desc' => (defined $prog->{'desc'} ? $prog->{'desc'}[0][0] : undef), '_genres' => (defined $prog->{'category'} ? $prog->{'category'} : undef), '_channel' => (defined $prog->{'channel'} ? $prog->{'channel'} : undef), '_series_num' => $episode_num->{'season'}, '_series_total' => $episode_num->{'season_total'}, '_episode_num' => $episode_num->{'episode'}, '_episode_total' => $episode_num->{'episode_total'}, '_part_num' => $episode_num->{'part'}, '_part_total' => $episode_num->{'part_total'}, '_has_numbering' => (defined $episode_num ? 1 : 0), }; _d(4,'_Prog, before title fixups:',dd(4,$_prog)); # TODO : the user rules are not processed in numerical order - there's no clues # in uk_rt grabber (on whch this class is based) as to why the following # order was chosen or even if it matters (since most of the rules are # not cumulative) # Remove non-title text found in programme title (type = 1) $self->process_non_title_info($_prog); # Track when titles/subtitles have been updated - # allows skip certain rules if the programme has already been processed by another rule. # (NOTE: this means the rules are not cumulative) $_prog->{'_titles_processed'} = 0; $_prog->{'_subtitles_processed'} = 0; # Next, process titles to make them consistent # One-off demoted title replacements (type = 11) $self->process_demoted_titles($_prog) if (! $_prog->{'_titles_processed'}); # One-off title and episode replacements (type = 10) $self->process_replacement_titles_desc($_prog) if (! $_prog->{'_titles_processed'}); # One-off title and episode replacements (type = 8) $self->process_replacement_titles_episodes($_prog) if (! $_prog->{'_titles_processed'}); # Look for $title:$episode in source title (type = 2) $self->process_mixed_title_subtitle($_prog) if (! $_prog->{'_titles_processed'}); # Look for $episode:$title in source title (type = 3) $self->process_mixed_subtitle_title($_prog) if (! $_prog->{'_titles_processed'}); # Look for reversed title and subtitle information (type = 4) $self->process_reversed_title_subtitle($_prog) if (! $_prog->{'_titles_processed'}); # Look for inconsistent programme titles (type = 5) # # This fixup is applied to all titles (processed or not) to handle # titles split out in fixups of types 2-4 above $self->process_replacement_titles($_prog); # Remove programme numbering for a 'corrected' title # (optionally limited to a specified channel identifier) $self->process_remove_numbering_from_programmes($_prog); # (type=16) # Next, process subtitles to make them consistent # Remove text from programme subtitles (type = 13) $self->process_subtitle_remove_text($_prog) if (! $_prog->{'_subtitles_processed'}); # Look for inconsistent programme subtitles (type = 7) $self->process_replacement_episodes($_prog) if (! $_prog->{'_subtitles_processed'}); # Replace subtitle based on description (type = 9) $self->process_replacement_ep_from_desc($_prog) if (! $_prog->{'_subtitles_processed'}); # Insert/update a programme's category based on 'corrected' title $self->process_replacement_genres($_prog); # (type=6) $self->process_replacement_film_genres($_prog); # (type=12) # Replace specified categories with another $self->process_translate_genres($_prog); # (type=14) # Add specified categories to all progs on a channel $self->process_add_genres_to_channel($_prog); # (type=15) _d(4,'_Prog, after title fixups:',dd(4,$_prog)); # update the title and sub-title and description in the programme $prog->{'title'}[0][0] = $_prog->{'_title'} if defined $_prog->{'_title'}; $prog->{'sub-title'}[0][0] = $_prog->{'_episode'} if defined $_prog->{'_episode'}; $prog->{'desc'}[0][0] = $_prog->{'_desc'} if defined $_prog->{'_desc'}; $prog->{'category'} = $_prog->{'_genres'} if defined $_prog->{'_genres'}; # update the episode-num $episode_num->{'season'} = $_prog->{'_series_num'}; $episode_num->{'season_total'} = $_prog->{'_series_total'}; $episode_num->{'episode'} = $_prog->{'_episode_num'}; $episode_num->{'episode_total'} = $_prog->{'_episode_total'}; $episode_num->{'part'} = $_prog->{'_part_num'}; $episode_num->{'part_total'} = $_prog->{'_part_total'}; # remake the episode-num $xmltv_ns = $self->make_ns_epnum($prog, $episode_num); _d(5,'Prog, after title fixups:',dd(5,$prog)); } =item B<process_non_title_info> Rule #1 Remove specified non-title text from <title>. If title starts with text + separator, then it will be removed from the title "separator" can be any of :;- rule: 1|Python Night in : "Python Night: Monty Python - Live at the Hollywood Bowl / " out: "Monty Python - Live at the Hollywood Bowl / " =cut # Rule 1 # # Remove non-title text found in programme title. # # Listings may contain channel teasers (e.g. "Python Night", "Arnie Season") in the programme title # # Data type 1 # The text in the second field is non-title text that is to be removed from # any programme titles found containing this text at the beginning of the # <title> element, separated from the actual title with a colon. # sub process_non_title_info () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 1; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} && $prog->{'_title'} =~ m/[:;-]/ ) { my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ( $prog->{'_title'} =~ s/^\Q$key\E\s*[:;-]\s*//i ) { l(sprintf("\t Removed '%s' from title. New title '%s' (#%s.%s)", $key, $prog->{'_title'}, $ruletype, $line)); $self->add_to_audit ($me, $key, $prog); last LOOP; } } } } =item B<process_demoted_titles> Rule #11 Promote demoted title from <sub-title> to <title>. If title matches, and sub-title starts with text then remove matching text from sub-title and move it into the title. Any text after 'separator' in the sub-title is preserved. 'separator' can be any of .,:;- rule: 11|Blackadder~Blackadder II in : "Blackadder / Blackadder II: Potato" out: "Blackadder II / Potato" =cut # Rule 11 # # Promote demoted title from subtitle field to title field, replacing whatever # text is in the title field at the time. If the demoted title if followed by # a colon and the subtitle text, that is preserved in the subtitle field. # # A title can be demoted to the subtitle field if the programme's "brand" # is present in the title field, as can happen with data output from Atlas. # # Data type 11 # The text in the second field contains a programme 'brand' and a new title to # be extracted from subtitle field and promoted to programme title, replacing # the brand title. # sub process_demoted_titles () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 11; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} && defined $prog->{'_episode'} ) { my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_title'} eq $key) { if ( $prog->{'_episode'} =~ s/^\Q$value\E(?:\s*[.,:;-]\s*)?//i ) { $prog->{'_title'} = $value; l(sprintf("\t Promoted title '%s' from subtitle for brand '%s'. New subtitle '%s' (#%s.%s)", $value, $key, $prog->{'_episode'}, $ruletype, $line)); $self->add_to_audit ($me, $key, $prog); $prog->{'_titles_processed'} = 1; $prog->{'_subtitles_processed'} = 1; last LOOP; } } } } } =item B<process_replacement_titles_desc> Rule #10 Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. If title & sub-title match supplied data, then replace <title> and <sub-title> with new data supplied. rule: 10|Which Doctor~~Gunsmoke~Which Doctor~Festus and Doc go fishing, but are captured by a family that is feuding with the Haggens. in : "Which Doctor / " desc> = " Festus and Doc go fishing, but are captured by a family that is feuding with the Haggens. ..." out: "Gunsmoke / Which Doctor" =cut # Rule 10 # # Allow arbitrary replacement of one title/episode pair with another, based # on a given description. # # Intended to be used where previous title/episode replacement routines # do not allow a specific enough correction to the listings data (i.e. for # one-off changes). # # *** THIS MUST BE USED WITH CARE! *** # # Data type 10 # The text in the second field contains an old programme title, an old episode # value, a new programme title, a new episode value and the episode description. # The old and new titles and description MUST be given, the episode fields can # be left empty but the field itself must be present. # sub process_replacement_titles_desc () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 10; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} ) { my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_title'} eq $key) { # $value comprises 'old episode', 'new title', 'new episode' and 'old description' separated by tilde my ($old_episode, $new_title, $new_episode, $old_desc) = split /~/, $value; # ensure we have an episode (to simplify the following code) $prog->{'_episode'} = '' if !defined $prog->{'_episode'}; # if the sub-title contains episode numbering then preserve it in the new episode title # extract any episode numbering (x/y) (c.f. extract_numbering() ) my ($epnum, $epnum_text) = ('', ''); if ($prog->{'_episode'} =~ m/^([\(\[]*\d+(?:[\s\/]*\d+)?[\.,]?[\)\]]*[\s\.,:;-]*(?:(?:series|season)[\d\s\.,:;-]*)?)(.*)$/) { $epnum = $1; $prog->{'_episode'} =~ s/\Q$epnum\E//; $epnum_text = ' (preserved existing numbering)'; } # check the other parts of the match triplet # # the original uk_rt grabber used an exact match # - a 'startswith' match would be better # - a 'fuzzy' (e.g. word count) match would be even better! # if ( $prog->{'_episode'} eq $old_episode && defined $prog->{'_desc'} && $prog->{'_desc'} =~ m/^\Q$old_desc\E/i ) { # update the title & episode my $old_title = $prog->{'_title'}; $prog->{'_title'} = $new_title; $prog->{'_episode'} = $epnum . ' ' . $new_episode; l(sprintf("\t Replaced old title/ep '%s / %s' with '%s / %s' using desc%s (#%s.%s)", $old_title, $old_episode, $prog->{'_title'}, $prog->{'_episode'}, $epnum_text, $ruletype, $line)); $self->add_to_audit ($me, $key, $prog); $prog->{'_titles_processed'} = 1; last LOOP; } } } } } =item B<process_replacement_titles_episodes> Rule #8 Replace specified <title> / <sub-title> with title/episode pair supplied. If title & sub-title match supplied data, then replace <title> and <sub-title> with new data supplied. rule: 8|Top Gear USA Special~Detroit~Top Gear~USA Special in : "Top Gear USA Special / Detroit" out: "Top Gear / USA Special" rule: 8|Top Gear USA Special~~Top Gear~USA Special in : "Top Gear USA Special / " out: "Top Gear / USA Special" or in : "Top Gear USA Special / 1/6." out: "Top Gear / 1/6. USA Special" =cut # Rule 8 # # Allow arbitrary replacement of one title/episode pair with another. # Intended to be used where previous title/episode replacement routines # do not allow the desired correction (i.e. for one-off changes). # # *** THIS MUST BE USED WITH CARE! *** # # Data type 8 # The text in the second field contains an old programme title, an old episode # value, a new programme title and a new episode value. The old and new titles # MUST be given, the episode fields can be left empty but the field itself # must be present. # sub process_replacement_titles_episodes () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 8; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} ) { my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_title'} eq $key) { # $value comprises 'old episode', 'new title' and 'new episode' separated by tilde my ($old_episode, $new_title, $new_episode) = split /~/, $value; # ensure we have an episode (to simplify the following code) $prog->{'_episode'} = '' if !defined $prog->{'_episode'}; # if the sub-title contains episode numbering then preserve it in the new episode title # extract any episode numbering (x/y) (c.f. extract_numbering() ) my ($epnum, $epnum_text) = ('', ''); if ($prog->{'_episode'} =~ m/^([\(\[]*\d+(?:[\s\/]*\d+)?[\.,]?[\)\]]*[\s\.,:;-]*(?:(?:series|season)[\d\s\.,:;-]*)?)(.*)$/) { $epnum = $1; $prog->{'_episode'} =~ s/\Q$epnum\E//; $epnum_text = ' (preserved existing numbering)'; } # check the other part of the match pair if ( $prog->{'_episode'} eq $old_episode ) { # update the title & episode my $old_title = $prog->{'_title'}; $prog->{'_title'} = $new_title; $prog->{'_episode'} = $epnum . ' ' . $new_episode; l(sprintf("\t Replaced old title/ep '%s / %s' with '%s / %s'%s (#%s.%s)", $old_title, $old_episode, $prog->{'_title'}, $prog->{'_episode'}, $epnum_text, $ruletype, $line)); $self->add_to_audit ($me, $key, $prog); $prog->{'_titles_processed'} = 1; last LOOP; } } } } } =item B<process_mixed_title_subtitle> Rule #2 Extract sub-title from <title>. If title starts with text + separator, then the text after it will be moved into the sub-title "separator" can be any of :;- rule: 2|Blackadder II in : "Blackadder II: Potato / " out: "Blackadder II / Potato" =cut # Rule 2 # # Some programme titles contain both the title and episode data, # separated by a colon ($title:$episode), semicolon ($title; $episode) # or a hyphen ($title - $episode). # # Here we reassign the episode to the $episode element, leaving only the # programme's title in the $title element # # Data type 2 # The text in the second field is the desired title of a programme when the # raw listings data contains both the programme's title _and_ episode in # the title ($title:$episode). We reassign the episode information to the # <episode> element, leaving only the programme title in the <title> element. # sub process_mixed_title_subtitle () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 2; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} && $prog->{'_title'} =~ m/[:;-]/ ) { my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_title'} =~ m/^(\Q$key\E)\s*[:;-]\s*(.*)$/i) { # store the captured text my $new_title = $1; my $new_episode = $2; # if no sub-title... if (! defined $prog->{'_episode'}) { l(sprintf("\t Moved '%s' to sub-title, new title is '%s' (#%s.%s)", $new_episode, $new_title, $ruletype, $line)); $prog->{'_title'} = $new_title; $prog->{'_episode'} = $new_episode; } # sub-title already equals the captured text elsif ($prog->{'_episode'} eq $new_episode) { l(sprintf("\t Sub-title '%s' seen in title already exists, new title is '%s' (#%s.%s)", $new_episode, $new_title, $ruletype, $line)); $prog->{'_title'} = $new_title; } # already have a sub-title (and which contains episode numbering), # merge the captured text after any episode numbering (x/y) (c.f. extract_numbering() ) elsif ($prog->{'_episode'} =~ m/^([\(\[]*\d+(?:[\s\/]*\d+)?[\.,]?[\)\]]*[\s\.,:;-]*(?:(?:series|season)[\d\s\.,:;-]*)?)(.*)$/) { l(sprintf("\t Merged sub-title '%s' seen in title after existing episode numbering '%s' (#%s.%s)", $new_episode, $prog->{'_episode'}, $ruletype, $line)); $prog->{'_title'} = $new_title; $prog->{'_episode'} = $1 . $new_episode . ': ' . $2; } # already have a sub-title, so prepend the captured text else { l(sprintf("\t Joined sub-title '%s' seen in title with existing episode info '%s' (#%s.%s)", $new_episode, $prog->{'_episode'}, $ruletype, $line)); $prog->{'_title'} = $new_title; $prog->{'_episode'} = $new_episode . ": " . $prog->{'_episode'}; } $self->add_to_audit ($me, $key, $prog); $prog->{'_titles_processed'} = 1; last LOOP; } } } } =item B<process_mixed_subtitle_title> Rule #3 Extract sub-title from <title>. If title ends with separator + text, then the text before it will be moved into the sub-title "separator" can be any of :;- rule: 3|Storyville in : "Kings of Pastry :Storyville / " out: "Storyville / Kings of Pastry" =cut # Rule 3 # # Some programme titles contain both the episode and title data, # separated by a colon ($episode:$title), semicolon ($episode; $title) or a # hyphen ($episode - $title). # # Here we reassign the episode to the $episode element, leaving only the # programme's title in the $title element # # Data type 3 # The text in the second field is the desired title of a programme when the # raw listings data contains both the programme's episode _and_ title in # the title ($episode:$title). We reassign the episode information to the # <episode> element, leaving only the programme title in the <title> element. # sub process_mixed_subtitle_title () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 3; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} && $prog->{'_title'} =~ m/[:;-]/ ) { # can't use the index for this one since we don't know what the incoming title begins with # (i.e. the rule doesn't specify it) LOOP: foreach my $k (keys %{ $self->{'rules'}->{$ruletype} }) { foreach (@{ $self->{'rules'}->{$ruletype}->{$k} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_title'} =~ m/^(.*?)\s*[:;-]\s*(\Q$key\E)$/i) { # store the captured text my $new_title = $2; my $new_episode = $1; # if no sub-title... if (! defined $prog->{'_episode'}) { l(sprintf("\t Moved '%s' to sub-title, new title is '%s' (#%s.%s)", $new_episode, $new_title, $ruletype, $line)); $prog->{'_title'} = $new_title; $prog->{'_episode'} = $new_episode; } # sub-title already equals the captured text elsif ($prog->{'_episode'} eq $new_episode) { l(sprintf("\t Sub-title '%s' seen in title already exists, new title is '%s' (#%s.%s)", $new_episode, $new_title, $ruletype, $line)); $prog->{'_title'} = $new_title; } # already have a sub-title (and which contains episode numbering), # merge the captured text after any episode numbering (x/y) (c.f. extract_numbering() ) elsif ($prog->{'_episode'} =~ m/^([\(\[]*\d+(?:[\s\/]*\d+)?[\.,]?[\)\]]*[\s\.,:;-]*(?:(?:series|season)[\d\s\.,:;-]*)?)(.*)$/) { l(sprintf("\t Merged sub-title '%s' seen in title after existing episode numbering '%s' (#%s.%s)", $new_episode, $prog->{'_episode'}, $ruletype, $line)); $prog->{'_title'} = $new_title; $prog->{'_episode'} = $1 . $new_episode . ': ' . $2; } # already have a sub-title, so prepend the captured text else { l(sprintf("\t Joined sub-title '%s' seen in title with existing episode info '%s' (#%s.%s)", $new_episode, $prog->{'_episode'}, $ruletype, $line)); $prog->{'_title'} = $new_title; $prog->{'_episode'} = $new_episode . ": " . $prog->{'_episode'}; } $self->add_to_audit ($me, $key, $prog); $prog->{'_titles_processed'} = 1; last LOOP; } } } } } =item B<process_reversed_title_subtitle> Rule #4 Reverse <title> and <sub-title> If sub-title matches the rule's text, then swap the title and sub-title rule: 4|Storyville in : "Kings of Pastry / Storyville" out: "Storyville / Kings of Pastry" =cut # Rule 4 # # Listings for some programmes may have reversed title and sub-title information # ($title = 'real' episode and $episode = 'real' title. Here we everse the given # title and sub-title when found. # # Data type 4 # The text in the second field is the desired title of a programme which is # listed in the raw listings data as the programme's episode (i.e. the title # and episode details have been reversed). We therefore reverse the # assignment to ensure the <title> and <episode> elements contain the correct # information. # sub process_reversed_title_subtitle () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 4; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_episode'} ) { my $idx = lc(substr $prog->{'_episode'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_episode'} eq $key) { $prog->{'_episode'} = $prog->{'_title'}; $prog->{'_title'} = $key; l(sprintf("\t Reversed title-subtitle for '%s / %s'. New title is '%s' (#%s.%s)", $prog->{'_episode'}, $prog->{'_title'}, $prog->{'_title'}, $ruletype, $line)); $self->add_to_audit ($me, $key, $prog); $prog->{'_titles_processed'} = 1; last LOOP; } } } } =item B<process_replacement_titles> Rule #5 Replace <title> with supplied text. If title matches the rule's text, then use the replacement text supplied rule: 5|A Time Team Special~Time Team in : "A Time Team Special / Doncaster" out: "Time Team / Doncaster" This is the one which you will probably use most. It can be used to fix most incorrect titles - e.g. spelling mistakes; punctuation; character case; etc. =cut # Rule 5 # # Process inconsistent titles, replacing any flagged bad titles with good titles. # # Data type 5 # The text in the second field contains two programme titles, separated by a # tilde (~). The first title is the inconsistent programme title to search # for during processing, and the second title is a consistent title to # as a replacement in the listings output. Programme titles can be # inconsistent across channels (e.g. Law and Order vs Law & Order) or use # inconsistent grammar (xxxx's vs xxxxs'), so we provide a consistent # title, obtained from the programme itself, its website or other media, # to use instead. # sub process_replacement_titles () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 5; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} ) { my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_title'} eq $key) { $prog->{'_title'} = $value; l(sprintf("\t Replaced title '%s' with '%s' (#%s.%s)", $key, $prog->{'_title'}, $ruletype, $line)); $self->add_to_audit ($me, $key, $prog); $prog->{'_titles_processed'} = 1; last LOOP; } } } } =item B<process_subtitle_remove_text> Rule #13 Remove specified text from <sub-title> for a given <title>. If sub-title starts with text + separator, or ends with separator + text, then it will be removed from the sub-title. "separator" can be any of .,:;- and is optional. rule: 13|Time Team~A Time Team Special in : "Time Team / Doncaster : A Time Team Special " out: "Time Team / Doncaster" =cut # Rule 13 # # Process text to remove from subtitles. # # Data type 13 # The text in the second field contains a programme title and arbitrary text to # be removed from the start/end of the programme's subtitle, separated by a # tilde (~). If the text to be removed precedes or follows a colon/hyphen, the # colon/hyphen is removed also. # sub process_subtitle_remove_text () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 13; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} && defined $prog->{'_episode'} ) { my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_title'} eq $key) { if ( $prog->{'_episode'} =~ s/^\Q$value\E\s*[.,:;-]?\s*//i || $prog->{'_episode'} =~ s/\s*[.,:;-]?\s*\Q$value\E$//i ) { $prog->{'_episode'} = ucfirst($prog->{'_episode'}); l(sprintf("\t Removed text '%s' from subtitle. New subtitle is '%s' (#%s.%s)", $value, $prog->{'_episode'}, $ruletype, $line)); $self->add_to_audit ($me, $key, $prog); $prog->{'_subtitles_processed'} = 1; last LOOP; } } } } } =item B<process_replacement_episodes> Rule #7 Replace <sub-title> with supplied text. If sub-title matches the rule's text, then use the replacement text supplied rule: 7|Time Team~Time Team Special: Doncaster~Doncaster in : "Time Team / Time Team Special: Doncaster" out: "Time Team / Doncaster" =cut # Rule 7 # # Process inconsistent episodes, replacing any flagged bad episodes with good episodes. # # Data type 7 # The text in the second field contains a programme title, an old episode # value and a new episode value, all separated by tildes (~). # sub process_replacement_episodes () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 7; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} && defined $prog->{'_episode'} ) { my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_title'} eq $key) { # $value comprises 'old episode' & 'new episode' separated by tilde my ($old_episode, $new_episode) = split /~/, $value; if ( $prog->{'_episode'} eq $old_episode ) { $prog->{'_episode'} = $new_episode; l(sprintf("\t Replaced episode '%s' with '%s' (#%s.%s)", $old_episode, $prog->{'_episode'}, $ruletype, $line)); $self->add_to_audit ($me, $key, $prog); $prog->{'_subtitles_processed'} = 1; last LOOP; } } } } } =item B<process_replacement_ep_from_desc> Rule #9 Replace <sub-title> with supplied text when the <desc> matches that given. If sub-title matches the rule's text, then use the replacement text supplied rule: 9|Heroes of Comedy~The Goons~The series celebrating great British comics pays tribute to the Goons. in : "Heroes of Comedy / " out: "Heroes of Comedy / The Goons" or in : "Heroes of Comedy / Spike Milligan" out: "Heroes of Comedy / The Goons" =cut # Rule 9 # # Replace an inconsistent or missing episode subtitle based a given description. # The description should therefore be unique for each episode of the programme. # # Data type 9 # The text in the second field contains a programme title, a new episode # value to update, and a description to match against. # sub process_replacement_ep_from_desc () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 9; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} && defined $prog->{'_desc'} ) { my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_title'} eq $key) { # $value comprises 'new episode' & 'description' separated by tilde my ($new_episode, $old_desc) = split /~/, $value; # the original uk_rt grabber used an exact match # - a 'startswith' match would be better # - a 'fuzzy' (e.g. word count) match would be even better! # ##if ( $prog->{'_desc'} eq $old_desc ) { if ( $prog->{'_desc'} =~ m/^\Q$old_desc\E/ ) { my $old = $prog->{'_episode'} || ''; $prog->{'_episode'} = $new_episode; l(sprintf("\t Replaced episode '%s' with '%s' (#%s.%s)", $old, $prog->{'_episode'}, $ruletype, $line)); $self->add_to_audit ($me, $key, $prog); $prog->{'_subtitles_processed'} = 1; last LOOP; } } } } } =item B<process_replacement_genres> Rule #6 Replace <category> with supplied text. If title matches the rule's text, then use the replacement category(-ies) supplied (note ALL existing categories are replaced) rule: 6|Antiques Roadshow~Entertainment~Arts~Shopping in : "Antiques Roadshow / " category "Reality" out: "Antiques Roadshow / " category "Entertainment" + "Arts" + "Shopping" You can specify a wildcard with the title by using %% which represents any number of characters. So for example "News%%" will match "News", "News and Weather", "Newsnight", etc. But be careful; "%%News%%" will also match "John Craven's Newsround", "Eurosport News", "Election Newsroom Live", "Have I Got News For You", "Scuzz Meets Jason Newsted", etc. =cut # Rule 6 # # Process programmes that may not be categorised, or are categorised with # various categories in the source data. # See rule type 12 for films. # # Data type 6 # The text in the second field contains a programme title and a programme # category (genre), separated by a tilde (~). Categories can be assigned # to uncategorised programmes. # sub process_replacement_genres () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 6; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} ) { my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: # append any rules which start with a wildcard # (todo: this doesn't work for "x%%...") foreach ( @{ $self->{'rules'}->{$ruletype}->{$idx} } , @{ $self->{'rules'}->{$ruletype}->{'%%'} } ) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); my $qr_key = $self->replace_wild($key); _d(5,"\t $line, $qr_key, $value"); if ($prog->{'_title'} =~ $qr_key ) { #_d(4,dd(4,$prog->{'_genres'})); my $old = ''; if (defined $prog->{'_genres'}) { foreach my $genre (@{ $prog->{'_genres'} }) { $old .= $genre->[0] . ','; } chop $old; } my $new = $value; $new =~ s/~/, /g; $prog->{'_genres'} = undef; # the original uk_rt grabber only allowed one genre, but let's enhance that # and allow multiple genres separated by tilde my @values = split /~/, $value; my $i=0; foreach (@values) { $prog->{'_genres'}[$i++] = [ $_, $self->{'language_code'} ]; } l(sprintf("\t Replaced genre(s) '%s' with '%s' (#%s.%s)", $old, $new, $ruletype, $line)); # if using a wildcard, we could mod many progs with this one rule so let's report all of them if ($key =~ m/%%/) { $self->add_to_audit ($me, $key.'~'.$prog->{'_title'}, $prog); } else { $self->add_to_audit ($me, $key, $prog); } $prog->{'_subtitles_processed'} = 1; last LOOP; } } } } =item B<process_replacement_film_genres> Rule #12 Replace "Film"/"Films" <category> with supplied text. If title matches the rule's text and the prog has category "Film" or "Films", then use the replacement category(-ies) supplied (note ALL categories are replaced, not just "Film") rule: 12|The Hobbit Special~Entertainment~Interview in : "The Hobbit Special / " category "Film" + "Drama" out: "The Hobbit Special / " category "Entertainment" + "Interview" =cut # Rule 12 # # Process programmes incorrectly categorised as films with replacement categories (genres) # See rule type 6 for non-films. # # Data type 12 # The text in the second field contains a film title and a programme # category (genre), separated by a tilde (~). Some film-related programmes are # incorrectly flagged as films and should to be re-assigned to a more suitable # genre. # sub process_replacement_film_genres () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 12; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} && defined $prog->{'_genres'} ) { my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_title'} eq $key) { #_d(4,dd(4,$prog->{'_genres'})); my $isfilm = 0; my $old = ''; foreach my $genre (@{ $prog->{'_genres'} }) { $old .= $genre->[0] . ','; # is it a film? if ( $genre->[0] =~ m/films?/i ) { $isfilm = 1; } } chop $old; if (!$isfilm) { last LOOP; } my $new = $value; $new =~ s/~/, /g; $prog->{'_genres'} = undef; # the original uk_rt grabber only allowed one genre, but let's enhance that # and allow multiple genres separated by tilde my @values = split /~/, $value; my $i=0; foreach (@values) { $prog->{'_genres'}[$i++] = [ $_, $self->{'language_code'} ]; } l(sprintf("\t Replaced genre(s) '%s' with '%s' (#%s.%s)", $old, $new, $ruletype, $line)); $self->add_to_audit ($me, $key, $prog); $prog->{'_subtitles_processed'} = 1; last LOOP; } } } } =item B<process_translate_genres> Rule #14 Replace <category> with supplied value(s). If category matches one found in the prog, then replace it with the category(-ies) supplied (note any other categories are left alone) rule: 14|Soccer~Football in : "Leeds v Arsenal" category "Soccer" out: "Leeds v Arsenal" category "Football" rule: 14|Adventure/War~Action Adventure~War in : "Leeds v Arsenal" category "Adventure/War" out: "Leeds v Arsenal" category "Action Adventure" + "War" =cut # Rule 14 # # Replace any occurrence of one genre with another. The replacement may be a single or multiple genres. # # Data type 14 # The content contains a category (genre) value followed by replacement # category(-ies) separated by a tilde (~). # Use case: useful if your PVR doesn't understand some of the category # values in the incoming data; you can translate them to another value. # sub process_translate_genres () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 14; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} && defined $prog->{'_genres'} ) { #_d(4,dd(4,$prog->{'_genres'})); # To ensure the replacements are NOT iterative, we'll store the new values separate for now $prog->{'_newgenres'} = undef; my $storenewgenres = 0; foreach my $genre (@{ $prog->{'_genres'} }) { my $haschanged = 0; my $old = $genre->[0]; my $idx = lc(substr $genre->[0], 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($genre->[0] eq $key) { my $new = $value; $new =~ s/~/, /g; my @values = split /~/, $value; foreach (@values) { push @{$prog->{'_newgenres'}}, [ $_, $self->{'language_code'} ]; } l(sprintf("\t Replaced genre(s) '%s' with '%s' (#%s.%s)", $old, $new, $ruletype, $line)); $self->add_to_audit ($me, $key, $prog); $haschanged = 1; $storenewgenres = 1; last LOOP; } } if (!$haschanged) { push @{$prog->{'_newgenres'}}, $genre; } } if ($storenewgenres) { # store the new categories $prog->{'_genres'} = $prog->{'_newgenres'}; } } } =item B<process_add_genres_to_channel> Rule #15 Add a category to all programmes on a specified channel. If channel matches this prog, the add the supplied category(-ies) to the programme (note any other categories are left alone) rule: 15|travelchannel.co.uk~Travel in : "World's Greatest Motorcycle Rides" category "Motoring" out: "World's Greatest Motorcycle Rides" category "Motoring" + "Travel" rule: 15|cnbc.com~News~Business in : "Investing in India" category "" out: "Investing in India" category "News" + "Business" You should be very careful with this one as it will add the category you specify to EVERY programme broadcast on that channel. This may not be what you always want (e.g. Teleshopping isn't really "music" even if it is on MTV!) =cut # Rule 15 # # Add a genre to all programmes on a specified channel.. The addition may be a single or multiple genres. # # Data type 15 # The content contains a channel value followed by # category(-ies) separated by a tilde (~). # Use case: can add a category if data from your supplier is always missing; e.g. add "News" to a news channel, or "Music" to a music vid channel. # sub process_add_genres_to_channel () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 15; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} && defined $prog->{'_channel'} ) { #_d(4,dd(4,$prog->{'_channel'})); my $idx = lc(substr $prog->{'_channel'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_channel'} eq $key) { #_d(4,dd(4,$prog->{'_genres'})); my %h_old = (); my $old = ''; if (defined $prog->{'_genres'}) { foreach my $genre (@{ $prog->{'_genres'} }) { $old .= $genre->[0] . ','; $h_old{$genre->[0]} = 1; } chop $old; } my $new = $value; $new =~ s/~/, /g; # allow multiple genres separated by tilde my @values = split /~/, $value; my $i = defined $prog->{'_genres'} ? scalar( @{ $prog->{'_genres'} } ) : 0; my $msgdone = 0; foreach (@values) { if ( ! exists( $h_old{$_} ) ) { $prog->{'_genres'}[$i++] = [ $_, $self->{'language_code'} ]; l(sprintf("\t Added genre(s) '%s' to '%s' (#%s.%s)", $new, $old, $ruletype, $line)) if !$msgdone; $msgdone = 1; #(we could mod many progs with this one rule so let's report all of them) #$self->add_to_audit ($me, $key, $prog); $self->add_to_audit ($me, $key.'~'.$prog->{'_title'}, $prog); } } last LOOP; } } } } =item B<process_remove_numbering_from_programmes> Rule #16 Remove episode numbering from a given programme title (on an optionally-specified channel). If title matches the one in the prog, all programme numbering for the programme is removed, on any channel. An optional channel identifier can be provided to restrict the removal of programme numbering to the given channel. rule: 16|Bedtime Story in : "CBeebies Bedtime Story" episode-num ".700." out: "CBeebies Bedtime Story" episode-num "" rule: 16|CBeebies Bedtime Story~cbeebies.bbc.co.uk in : "CBeebies Bedtime Story" episode-num ".700." out: "CBeebies Bedtime Story" episode-num "" Remember to specify the optional channel limiter if you have good programme numbering for a given programme title on some channels but not others. =cut # Rule 16 # # Remove episode numbering from a given programme title (on an optionally-specified channel). # # Data type 16 # The content contains a title value, followed by an optional channel (separated by a tilde (~)). # Use case: can remove programme numbering from a specific title if it is regularly wrong or inconsistent over time. # sub process_remove_numbering_from_programmes () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'options_all'} && ! $self->{'options'}{$me} ) { return 0; } _d(3,self()); my $ruletype = 16; if (!defined $self->{'rules'}->{$ruletype}) { return 0; } if ( defined $prog->{'_title'} && $prog->{'_has_numbering'} ) { #_d(4,dd(4,$prog->{'_title'})); my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $line, $key, $value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); _d(4,"\t $line, $key, $value"); if ($prog->{'_title'} eq $key) { #_d(4,dd(4,$prog->{'_channel'})); my @num_keys = ( '_series_num', '_series_total', '_episode_num', '_episode_total', '_part_num', '_part_total', ); # if an optional channel is not specified in the rule definition, # $value will be the empty string if ($value ne '') { if ($prog->{'_channel'} eq $value) { $prog->{$_} = '' foreach @num_keys; delete $prog->{'_has_numbering'}; l(sprintf("\t Removed all programme numbering for title '%s' on channel '%s' (#%s.%s)", $key, $value, $ruletype, $line)); } } else { $prog->{$_} = '' foreach @num_keys; delete $prog->{'_has_numbering'}; l(sprintf("\t Removed all programme numbering for title '%s' (#%s.%s)", $key, $ruletype, $line)); } last LOOP; } } } } # Store a variety of title debugging information for later analysis # and debug output # # (Note no changes are made to the incoming records) # sub store_title_debug_info () { my ($self, $prog) = @_; my $me = self(); _d(3,self()); if ($self->{'stats'}) { my $tmp_prog = {}; $tmp_prog->{'_title'} = defined $prog->{'title'} ? $prog->{'title'}[0][0] : ''; $tmp_prog->{'_episode'} = defined $prog->{'sub-title'} ? $prog->{'sub-title'}[0][0] : ''; $tmp_prog->{'_genres'} = defined $prog->{'category'} ? $prog->{'category'} : ''; $tmp_prog->{'_channel'} = defined $prog->{'channel'} ? $prog->{'channel'} : ''; _d(4,dd(4,$tmp_prog)); $self->check_numbering_in_text ($tmp_prog); $self->check_title_in_subtitle ($tmp_prog); $self->check_titles_with_colons ($tmp_prog); $self->check_titles_with_hyphens ($tmp_prog); $self->check_subtitles_with_hyphens ($tmp_prog); $self->check_uc_titles_post ($tmp_prog); $self->check_new_titles ($tmp_prog); $self->check_titles_with_years ($tmp_prog); $self->check_titles_with_bbfc_ratings ($tmp_prog); $self->check_titles_with_mpaa_ratings ($tmp_prog); $self->check_flagged_title_eps ($tmp_prog); $self->check_dotdotdot_titles ($tmp_prog); $self->make_frequency_distribution ($tmp_prog); $self->count_progs_by_channel ($tmp_prog); } } # Monitor for case/punctuation-insensitive title variations # Build a hash of variations for a title. Will be processed after # we have read all the progs in the input file, to create a list # of progs which may need a new title fixup. # sub make_frequency_distribution () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); # remove some punctuation, etc from title my $title_nopunc = lc $prog->{'_title'}; $title_nopunc =~ s/^the\s+//; $title_nopunc =~ s/(\s+and\s+|\s+&\s+)/ /g; $title_nopunc =~ s/\s+No 1'?s$//g; $title_nopunc =~ s/\s+Number Ones$//g; $title_nopunc =~ s/' //g; $title_nopunc =~ s/'s/s/g; $title_nopunc =~ s/\W//g; my $value = 'case_insens_titles'; my $key = $title_nopunc; # count number of each variant by genre and channel name if ( (!defined $prog->{'_genres'}) || (ref $prog->{'_genres'} ne 'ARRAY') ) { $prog->{'_genres'} = [ [ '(no genre)' ] ]; } foreach (@{ $prog->{'_genres'} }) { my $genre = $_->[0]; $self->{'audit'}{$value}{$key}{ $prog->{'_title'} }{$genre}{ $prog->{'_channel'} }++; } $self->{'audit'}{$value}{$key}{ $prog->{'_title'} }{'count'}++; } # Count the programmes seen for each channel sub count_progs_by_channel () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $value = 'input_channels'; my $key = $prog->{'_channel'}; # frequency count of progs for each channel $self->{'audit'}{$value}{$key}{'count'}++; } # Process the titles previously stored by make_frequency_distribution() # Look for possible title variants: i.e. where 2 incoming progs have # different title but may be the same # (e.g. one of them is misspelt / capitalisation / etc.) # sub check_title_variants { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $value = 'case_insens_titles'; if (!defined $self->{'audit'}{$value}) { return 0; } # temp hash to avoid too many mods to following code my $case_insens_titles = $self->{'audit'}{$value}; # iterate over each 'unique' title (i.e. the title without punctuation etc) foreach my $title_nopunc (sort keys %{$case_insens_titles}) { if (scalar keys %{$case_insens_titles->{$title_nopunc}} > 1) { my %variants; # iterate over each actual title seen in listings foreach my $title (sort keys %{$case_insens_titles->{$title_nopunc}}) { # need to remove 'count' key before genre processing later my $title_cnt = delete $case_insens_titles->{$title_nopunc}{$title}{'count'}; # hash lists of title variants keyed on frequency push @{$variants{$title_cnt}}, $title; my $line = "$title ("; # iterate over each title's genres foreach my $genre (sort keys %{$case_insens_titles->{$title_nopunc}{$title}}) { # iterate over each title's channel availability by genre foreach my $chan (sort keys %{$case_insens_titles->{$title_nopunc}{$title}{$genre}}) { $line .= $genre . "/" . $chan . " [" . $case_insens_titles->{$title_nopunc}{$title}{$genre}{$chan} . " times], "; } } $line =~ s/,\s*$//; # remove last comma $line .= ")"; $self->add_to_audit ('possible_title_variants', $title, { '_title' => $line }); } # now find list of titles with highest freq and check if it contains # a single entry to use in suggested fixups my @title_freqs = sort {$b <=> $a} keys %variants; my $highest_freq = $title_freqs[0]; if (scalar @{$variants{$highest_freq}} == 1) { # extract title with highest frequency and remove key from $case_insens_titles{$unique_title} my $best_title = shift @{$variants{$highest_freq}}; delete $case_insens_titles->{$title_nopunc}{$best_title}; # now iterate over remaining variations of title and generate fixups foreach (keys %{$case_insens_titles->{$title_nopunc}}) { my $fixup = "5|" . $_ . "~" . $best_title; push @{ $self->{'audit'}{'possible_title_variants_fixups'} }, $fixup; } } } } } # Check to see if prog contains possible series,episode or part numbering # (c.f. rule A5 check_potential_numbering_in_text() ) # sub check_numbering_in_text () { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); # audit the quality of extraction rule A5 my $key = $prog->{'_title'}; # check if $title still contains "season" text if ($prog->{'_title'} =~ m/(season|series|episode)/i ) { l("\t Check title text for possible series/episode text: " . $prog->{'_title'}); $self->add_to_audit ('title_text_to_remove', $key, $prog); } # check for potential series numbering left unprocessed if ($prog->{'_episode'} =~ m/(season|series)/i ) { l("\t Possible series numbering still seen: " . $prog->{'_episode'}); $self->add_to_audit ('possible_series_nums', $key, $prog); } # check for potential part numbering left unprocessed (i.e. the regex missed it) # TODO: don't run this test if we created episode with make_episode_from_part_numbers() if ($prog->{'_episode'} =~ m/\b(Part|Pt(\.)?)(\d+|\s+\w+)/i ) { l("\t Possible part numbering still seen: " . $prog->{'_episode'}); $self->add_to_audit ('possible_part_nums', $key, $prog); } # check for potential episode numbering left unprocessed elsif ($prog->{'_episode'} =~ m/(^\d{1,2}\D|\D\d{1,2}\.?$)/ || $prog->{'_episode'} =~ m/episode/i ) { l("\t Possible episode numbering still seen: " . $prog->{'_episode'}); $self->add_to_audit ('possible_episode_nums', $key, $prog); } } # Check for title text still present in episode details # sub check_title_in_subtitle { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $key = $prog->{'_title'}; if (defined $prog->{'_episode'}) { if ($prog->{'_episode'} =~ m/^\Q$prog->{'_title'}\E/) { l("\t Possible title in subtitle: " . $prog->{'_episode'}); $self->add_to_audit ('title_in_subtitle_notfixed', $key, $prog); } } } # Check to see if title contains a colon # - this may indicate a 'title:sub-title' which should be extracted # sub check_titles_with_colons { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $key = $prog->{'_title'}; # match a colon or semi-colon if ($prog->{'_title'} =~ m/^(.*?)\s*[:;]\s*(.*)$/ ) { my ($pre, $post) = ($1, $2); l("\t Title contains colon: " . $prog->{'_title'}); $self->add_to_audit ('colon_in_title', $key, $prog); # the uk_rt code only generated a fixup hint when there was >1 prog with the same # value before or after the colon. Doeesn't say why: presumably to reduce false positives. # I'm not sure this works as expected though, e.g. it doesn't account for repeats of the same prog # if ( defined $self->{'audit'}{'colon_in_title_pre'}{$pre} ) { $self->{'audit'}{'colon_in_title_pre'}{$pre}++; # only print each fixup once! if ($self->{'audit'}{'colon_in_title_pre'}{$pre} == 2) { my $fixup = "2|" . $pre; push @{ $self->{'audit'}{'colon_in_title_fixups'} }, $fixup; } } else { $self->{'audit'}{'colon_in_title_pre'}{$pre} = 1; } if ( defined $self->{'audit'}{'colon_in_title_post'}{$post} ) { $self->{'audit'}{'colon_in_title_post'}{$post}++; # only print each fixup once! if ($self->{'audit'}{'colon_in_title_post'}{$post} == 2) { my $fixup = "3|" . $post; push @{ $self->{'audit'}{'colon_in_title_fixups'} }, $fixup; } } else { $self->{'audit'}{'colon_in_title_post'}{$post} = 1; } } } # Check to see if title contains hyphen # - this may indicate a sub-title or ep numbering # sub check_titles_with_hyphens { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $key = $prog->{'_title'}; # match a hyphen not preceded by a colon if ($prog->{'_title'} =~ m/^(.*?)(?:\s?[^:]-\s|\s[^:]-\s?)(.*)$/ ) { my $fixup = "5|" . $prog->{'_title'} . '~' . "$1: $2"; l("\t Possible hyphenated title: " . $prog->{'_title'}); $self->add_to_audit ('possible_hyphenated_title', $key, $prog); push @{ $self->{'audit'}{'possible_hyphenated_title_fixups'} }, $fixup; } } # Check for episode details that contain a colon or hyphen - # - this may indicate a title in the episode field which needs # to be moved into the title field # sub check_subtitles_with_hyphens { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $key = $prog->{'_title'}; if (defined $prog->{'_episode'}) { # match a colon, or a hyphen if not a hyphenated word if ($prog->{'_episode'} =~ m/(:|\s-\s|-\s|\s-)/ ) { # (note '\s-\s' is superfluous!) l("\t Possible hyphenated subtitle: " . $prog->{'_episode'}); $self->add_to_audit ('colon_in_subtitle', $key, $prog); } } } # Check if title is all upper case # sub check_uc_titles_post { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $key = $prog->{'_title'}; # title is all uppercase? if ($prog->{'_title'} eq uc($prog->{'_title'}) && $prog->{'_title'} !~ m/^\d+$/) { $self->add_to_audit ('uppercase_title', $key, $prog); } } # Look for various text in the prog title # sub check_new_titles { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $key = $prog->{'_title'}; # match a title containing various text if ( $prog->{'_title'} =~ m/Special\b/i ) { l("\t Title contains 'Special': " . $prog->{'_title'}); $self->add_to_audit ('titles_word_special', $key, $prog); } if ( $prog->{'_title'} =~ m/^(All New|New)\b/i || $prog->{'_title'} =~ m/(Premiere|Final|Finale|Anniversary)\b/i ) { my $match = $1; l("\t Title contains 'New/Premiere/Finale/etc.': " . $prog->{'_title'}); $self->add_to_audit ('titles_word_various1', $match.':::'.$key, $prog); } if ( $prog->{'_title'} =~ m/\b(Day|Night|Week)\b/i ) { my $match = $1; l("\t Title contains 'Day/Night/Week': " . $prog->{'_title'}); $self->add_to_audit ('titles_word_various2', $match.':::'.$key, $prog); } if ( $prog->{'_title'} =~ m/\b(Christmas|New\s+Year['s]?)\b/i ) { my $match = $1; l("\t Title contains 'Christmas/New Year': " . $prog->{'_title'}); $self->add_to_audit ('titles_word_various3', $match.':::'.$key, $prog); } if ( $prog->{'_title'} =~ m/\b(Best of|Highlights|Results|Top)\b/i ) { my $match = $1; l("\t Title contains 'Results/Best of/Highlights/Top': " . $prog->{'_title'}); $self->add_to_audit ('titles_word_various4', $match.':::'.$key, $prog); } } # Look for titles which include a possible year (e.g. for films) # sub check_titles_with_years { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $key = $prog->{'_title'}; # match a title containing what could be a year # if ( $prog->{'_title'} =~ m/\b(19|20)\d{2}\b/ ) { l("\t Title contains year: " . $prog->{'_title'}); $self->add_to_audit ('titles_with_years', $key, $prog); } } # Look for titles which may contain BBFC film rating # sub check_titles_with_bbfc_ratings { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $key = $prog->{'_title'}; # match a title containing a BBFC rating # if ( $prog->{'_title'} =~ m/\((E|U|PG|12|12A|15|18|R18)\)/ ) { l("\t Title contains possible BBFC rating: " . $prog->{'_title'}); $self->add_to_audit ('titles_with_bbfc', $key, $prog); } } # Look for titles which may contain MPAA film rating # sub check_titles_with_mpaa_ratings { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $key = $prog->{'_title'}; # match a title containing a MPAA rating # if ( $prog->{'_title'} =~ m/\((G|PG|PG-?13|R|NC-?17)\)/ ) { l("\t Title contains possible MPAA rating: " . $prog->{'_title'}); $self->add_to_audit ('titles_with_mpaa', $key, $prog); } } # I'm not sure what this is trying to check - I think it's trying to # suggest if we already have a fixup (code 8) for this title then # maybe we need another one for this 'new' prog? Bit nebulous I think! # sub check_flagged_title_eps { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $key = $prog->{'_title'}; my $ruletype = 8; my $idx = lc(substr $prog->{'_title'}, 0, 2); LOOP: foreach (@{ $self->{'rules'}->{$ruletype}->{$idx} }) { my ( $_line, $_key, $_value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); if (lc $_key eq lc $prog->{'_title'}) { l("\t Title matches a rule $ruletype fixup: " . $prog->{'_title'}); $self->add_to_audit ('flagged_title_eps', $key, $prog); last LOOP; } } } # Here's another one which is somewhat obscure. # If title contains ellipsis and we already have a fixup (code 8 or 10) # containing ellipsis for this title in the *corrected* title then # maybe we need another one for the 'new' prog? # e.g. # 8|All I Want For Christmas Is Katy Perry!~~All I Want For Christmas Is...~Katy Perry! # 8|All I Want For Christmas Is Mariah Carey!~~All I Want For Christmas Is...~Mariah Carey! # then if incoming = '.*All I Want For Christmas Is.*' then print it # sub check_dotdotdot_titles { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $key = $prog->{'_title'}; # create a hash of rule types 8 and 10 which contain ellipsis in the 'new title' if (!defined $self->{'rules'}->{'ellipsis'} ) { foreach my $ruletype (qw/8 10/) { foreach my $k (keys %{ $self->{'rules'}->{$ruletype} }) { foreach (@{ $self->{'rules'}->{$ruletype}->{$k} }) { my ( $_line, $_key, $_value ) = ( $_->{'line'}, $_->{'key'}, $_->{'value'} ); # $value comprises # type 8 : 'old episode', 'new title', 'new episode' # type 10 : 'old episode', 'new title', 'new episode' and 'old description' # separated by tilde my ($old_episode, $new_title, $new_episode, $old_desc) = split /~/, $_value; # store titles that are being corrected with an existing "some title..." fixup # store the title without a leading "The" or "A" or the trailing "..." if ($new_title =~ m/^(?:The\s+|A\s+)?(.*)\.\.\.$/) { $self->{'rules'}{'ellipsis'}{$1} = $new_title; } } } } #_d(4,'Ellipsis hash',dd(4,$self->{'rules'}->{'ellipsis'})); } # if title does not contain ellipsis see if we already have a fixup for this title # which *does* contain an ellipsis if ( $prog->{'_title'} !~ m/\.{3}$/ ) { LOOP: foreach (keys %{ $self->{'rules'}{ellipsis} }) { my ($_k, $_v) = ($_, $self->{'rules'}->{ellipsis}->{$_} ); if ( $prog->{'_title'} =~ m/\b\Q$_k\E\b/i) { l("\t Title may need to be fixed based on fixup '$_v' : " . $prog->{'_title'}); $prog->{'_msg'} = "based on fixup '$_v'"; $self->add_to_audit ('dotdotdot_titles', $key, $prog); last LOOP; } } } } # Store details of uncategorised programmes, programmes having different # genres throughout the listings, and films having a duration of less than # 75 minutes for further analysis sub store_genre_debug_info () { my ($self, $prog) = @_; my $me = self(); _d(3,self()); if ($self->{'stats'}) { my $tmp_title = $prog->{'title'}[0][0]; my $tmp_episode = (defined $prog->{'sub-title'} ? $prog->{'sub-title'}[0][0] : ''); my $key = $tmp_title . "|" . $tmp_episode; # store genres for this prog as well as all genres seen across all programmes if (defined $prog->{'category'}) { my $all_cats; foreach (@{ $prog->{'category'} }) { my $genre = $_->[0]; $all_cats .= $genre .'~'; $self->{'audit'}{'all_genres'}{$genre}++; $self->{'audit'}{'cats_per_prog'}{$tmp_title}{$genre}++; } if ($all_cats) { chop $all_cats; $self->{'audit'}{'allcats_per_prog'}{$tmp_title}{$all_cats}++; } } # explode the genres my $genres = ''; if (defined $prog->{'category'}) { foreach (@{ $prog->{'category'} }) { $genres .= $_->[0] . ','; } chop $genres; } # Check for "Film" < 75 minutes long if ( $genres =~ m/Films?/i ) { if (defined $prog->{'stop'}) { my $start = time_xmltv_to_epoch($prog->{'start'}); my $stop = time_xmltv_to_epoch($prog->{'stop'}); if (($stop - $start) < 60 * 75) { #'_duration_mins'} < 75)) $me = 'short_films'; $self->add_to_audit ($me, $key, { '_title'=>$tmp_title, '_episode'=>$tmp_episode }); } } } # Check for progs without any genre elsif ( $genres eq '' ) { if ($prog->{'title'} !~ m/^(To Be Announced|TBA|Close)\.?$/i ) { $me = 'uncategorised_progs'; $self->add_to_audit ($me, $key, { '_title'=>$tmp_title, '_episode'=>$tmp_episode }); } } } } # Process the genres previously stored by store_genre_debug_info() # to print all categories seen across all programmes # sub check_categories { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $value = 'all_genres'; if (!defined $self->{'audit'}{$value}) { return 0; } # sort the genre counts descending and format as a list ready for printing my @keys = sort { $self->{'audit'}{$value}->{$b} <=> $self->{'audit'}{$value}->{$a} } keys( %{$self->{'audit'}{$value}} ); foreach my $key (@keys) { push @{$self->{'audit'}{'all_genres_sorted'}}, $key.' 'x(25-length $key).$self->{'audit'}{$value}->{$key}." times"; } } # Process the genres previously stored by store_genre_debug_info() # for each programme, to list progs with differing categories - # i.e. the same prog occuring more than once but with different cats. # Note this definition differs from that in uk_rt which only allowed # one cat per prog, whereas here we allow multiple cats per prog # (but it should be backwards compatible). # sub check_cats_per_prog { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $value = 'allcats_per_prog'; if (!defined $self->{'audit'}{$value}) { return 0; } # temp hash to avoid too many mods to following code my $cats_per_prog = $self->{'audit'}{$value}; # iterate over each title foreach my $title (sort keys %{$cats_per_prog}) { if (scalar keys %{$cats_per_prog->{$title}} > 1) { my ($best_cat, $best_cat_cnt); my $line = "$title is categorised as: "; foreach my $cat ( sort { $cats_per_prog->{$title}{$b} <=> $cats_per_prog->{$title}{$a} || $a cmp $b } ( keys %{$cats_per_prog->{$title}} ) ) { $line .= "\n\t $cat [" . $cats_per_prog->{$title}{$cat} . " times]"; if (!defined $best_cat) { $best_cat = $cat; $best_cat_cnt = $cats_per_prog->{$title}{$cat}; } else { my $fixup = "6|" . $title . "~" . $best_cat; push @{ $self->{'audit'}{'categories_per_prog_fixups'} }, $fixup; } } $self->add_to_audit ('categories_per_prog', $title, { '_title' => $line }); } } } # Process the hash of channels information to print # 1) <channel> which have no programmes in the file # 2) Channel names referenced in one or more progs, but missing <channel> element # sub print_empty_listings { my ($self, $prog) = @_; my $me = self(); if ( ! $self->{'stats'} ) { return 0; } _d(4,self()); my $value = 'input_channels'; if (!defined $self->{'audit'}{$value}) { return 0; } foreach my $key (keys %{ $self->{'audit'}->{$value} }) { if ( !defined $self->{'audit'}->{$value}->{$key}->{'count'} ) { $self->add_to_audit ('empty_listings', $key, { '_title' => $key }); } if ( !defined $self->{'audit'}->{$value}->{$key}->{'display_name'} ) { $self->add_to_audit ('listings_no_channel', $key, { '_title' => $key }); } } } # Some of the stats analysis works on the whole file of programmes # rather than just an individual prog. # For these we store the data as each prog is presented, and then # analyse them at time of printing the stats (i.e. after all the records # have been received). # # (Note no changes are made to the incoming records) # sub process_title_debug_info () { my ($self, $prog) = @_; my $me = self(); _d(3,self()); if ($self->{'stats'}) { $self->check_title_variants (); $self->check_categories (); $self->check_cats_per_prog (); $self->print_empty_listings (); } } # Add to our stats analysis hash data sub add_to_audit () { my ($self, $value, $key, $prog) = @_; # little bit of validation _d(0,'Missing $value in add_to_audit') if !defined $value || $value eq ''; _d(0,'Missing $key in add_to_audit'. " ($value)") if !defined $key || $key eq ''; _d(0,'Missing $prog->{_title} in add_to_audit') if !defined $prog->{'_title'} || $prog->{'_title'} eq ''; $self->{'audit'}{$value}{$key} = { 'title' => $prog->{'_title'}, 'episode' => $prog->{'_episode'}, 'msg' => $prog->{'_msg'}, }; } # Print the lists of actions taken and suggestions for further fixups # sub printInfo () { my ($self, $prog) = @_; my $me = self(); _d(3,self()); # Some of the stats analysis works on the whole file of programmes # rather than just an individual prog - we must do that analysis now $self->process_title_debug_info (); if ($self->{'stats'}) { my ($k,$v); l("\n".'++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++'); # Actions taken @audits = ( [ 'remove_duplicated_new_title_in_ep' , 'list of programmes where \'New \$title\' was removed from sub-title field (#A1)' ], [ 'remove_duplicated_title_and_ep_in_ep' , 'list of programmes where title/ep was removed from sub-title field (#A2)' ], [ 'remove_duplicated_title_in_ep' , 'list of programmes where title was removed from sub-title field (#A3)' ], [ 'update_premiere_repeat_flags_from_desc' , 'list of programmes where <premiere> or <previously-shown> was set from description content (#A4)' ], [ 'check_potential_numbering_in_text' , 'list of programmes where <episode-num was changed (#A5)' ], [ 'make_episode_from_part_numbers' , 'list of programmes where <sub-title> was created from \'part\' numbers (#A5.4)' ], [ 'process_non_title_info' , ': Remove specified non-title text from <title> (#1)' ], [ 'process_demoted_titles' , ': Promote demoted title from <sub-title> to <title> (#11)' ], [ 'process_replacement_titles_desc' , ': Replace specified <title> / <sub-title> with title/episode pair supplied using <desc> (#10)' ], [ 'process_replacement_titles_episodes' , ': Replace specified <title> / <sub-title> with title/episode pair supplied (#8)' ], [ 'process_mixed_title_subtitle' , ': Extract sub-title from <title> (#2)' ], [ 'process_mixed_subtitle_title' , ': Extract sub-title from <title> (#3)' ], [ 'process_reversed_title_subtitle' , ': Reverse <title> and <sub-title> (#4)' ], [ 'process_replacement_titles' , ': Replace <title> with supplied text (#5)' ], [ 'process_subtitle_remove_text' , ': Remove specified text from <sub-title> (#13)' ], [ 'process_replacement_episodes' , ': Replace <sub-title> with supplied text (#7)' ], [ 'process_replacement_ep_from_desc' , ': Replace <sub-title> with supplied text using <desc> (#9)' ], [ 'process_replacement_genres' , ': Replace <category> with supplied value(s) (#6)' ], [ 'process_replacement_film_genres' , ': Replace \'Film\'/\'Films\' <category> with supplied value(s) (#12)' ], [ 'process_translate_genres', ': Replace <category> with supplied value(s) (#14)' ], [ 'process_add_genres_to_channel', ': Add category to all programmes on <channel> (#15)' ], ); foreach (@audits) { ($k,$v) = @{$_}; $self->print_audit( $k, $v ); } ##l('++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++'); # Progs for possible modification @audits = ( [ 'title_in_subtitle_notfixed' , 'list of programmes where title is still present in sub-title field' ], [ 'colon_in_subtitle' , 'list of programmes where sub-title contains colon/hyphen' ], [ 'possible_series_nums' , 'list of possible series numbering seen in listings' ], [ 'possible_episode_nums' , 'list of possible episode numbering seen in listings' ], [ 'possible_part_nums', 'list of possible part numbering seen in listings' ], [ 'title_text_to_remove', 'list of titles containing \'Season\'' ], [ 'colon_in_title', 'list of titles containing colons', '"title:episode"' ], [ 'possible_hyphenated_title', 'list of titles containing hyphens', 'hyphenated titles' ], [ 'uppercase_title', 'list of uppercase titles' ], [ 'titles_word_special', 'list of titles containing \'Special\'' ], [ 'titles_word_various1', 'list of titles containing \'New/Premiere/Finale/etc.\'' ], [ 'titles_word_various2', 'list of titles containing \'Day/Night/Week\'' ], [ 'titles_word_various3', 'list of titles containing \'Christmas/New Year\'' ], [ 'titles_word_various4', 'list of titles containing \'Results/Best of/Highlights/Top\'' ], [ 'titles_with_years', 'list of titles including possible years' ], [ 'titles_with_bbfc', 'list of film titles including possible BBFC ratings' ], [ 'titles_with_mpaa', 'list of film titles including possible MPAA ratings' ], [ 'flagged_title_eps', 'list of titles that may need fixing individually' ], [ 'dotdotdot_titles', 'list of potential \'...\' titles that may need fixing individually' ], [ 'possible_title_variants', 'possible title variations' ], [ 'categories_per_prog', 'list of programmes with multiple categories' ], [ 'uncategorised_progs' , 'list of programmes with no category' ], [ 'short_films' , 'films < 75 minutes long' ], [ 'empty_listings', 'list of channels providing no listings' ], [ 'listings_no_channel', 'list of channels with no channel details' ], [ 'all_genres_sorted', 'all categories' ], #[ ], ); foreach (@audits) { ($k,$v) = @{$_}; $self->print_audit( $k, $v ); } } l("\n".'++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++'."\n\n"); } # Print the info stored in an 'audit' hash sub print_audit () { my ($self, $k, $t) = @_; _d(4,$k); if ($self->{'audit'}{$k} && ( (ref $self->{'audit'}{$k} eq 'HASH' && scalar keys %{$self->{'audit'}{$k}} > 0) || (ref $self->{'audit'}{$k} eq 'ARRAY' && scalar $self->{'audit'}{$k} > 0) )) { l("\nStart of $t"); if (ref $self->{'audit'}{$k} eq 'ARRAY') { foreach my $v (@{$self->{'audit'}{$k}}) { l("\t $v"); } } else { foreach my $v (sort keys %{$self->{'audit'}{$k}} ) { l("\t $self->{'audit'}{$k}{$v}->{'title'}" . (defined($self->{'audit'}{$k}{$v}->{'episode'}) ? " / $self->{'audit'}{$k}{$v}->{'episode'}" : '') . (defined $self->{'audit'}{$k}{$v}->{'msg'} ? ' -- '.$self->{'audit'}{$k}{$v}->{'msg'} : '') ); } } # see if there's a fixup hash for this audit my $k2 = $k . '_fixups'; if ($self->{'audit'}{$k2} && scalar @{ $self->{'audit'}{$k2} } > 0) { l("\nPossible fixups "); foreach my $v2 (sort @{ $self->{'audit'}->{$k2} } ) { l("$v2"); } l(""); } l("End of $t"); } } =back =cut # Load the configuration file sub load_config () { my ($self, $fn) = @_; if ( -e $fn ) { my $fhok = open my $fh, '<', $fn or v("Cannot open config file $fn"); if ($fhok) { my $c = 0; while (my $line = <$fh>) { $c++; chomp $line; chop($line) if ($line =~ m/\r$/); trim($line); next if $line =~ /^#/ || $line eq ''; my ($key, $value, $trash) = $line =~ /^(.*?)\s*=\s*(.*?)([\s\t]*#.*)?$/; $self->{'options'}{$key} = $value; } close $fh; } } else { v("File not found $fn"); return 1; } _d(8,'Loaded options:',dd(8,$self->{'options'})); } # Load the augmentation rules sub load_rules () { my ($self, $fn) = @_; # if filename is undefined then look to see if user wants us to fetch a grabber's Supplement file if ((!defined $fn) && $self->{'options'}{'use_supplement'}) { # Retrieve prog_titles_to_process via XMLTV::Supplement require XMLTV::Supplement; XMLTV::Supplement->import(GetSupplement); my $rules_file = GetSupplement($self->{'options'}{'supplement_grabber_name'}, $self->{'options'}{'supplement_grabber_file'}); if (!defined $rules_file) { v('Cannot fetch rules file: '.$self->{'options'}{'supplement_grabber_name'}.'/'.$self->{'options'}{'supplement_grabber_file'}); return 1; } my @rules = split /[\n\r]+/, $rules_file; my $c = 0; foreach my $line (@rules) { $c++; chomp $line; chop($line) if ($line =~ m/\r$/); trim($line); if ( $line =~ /\$id:\s(.*?)(\sExp)+.*?\$/i ) { l("Using Supplement: $1 \n"); } next if $line =~ /^#/ || $line eq ''; $self->load_rule($c, $line); } } elsif ( defined $fn ) { if ( -e $fn ) { my $fhok = open my $fh, '<', $fn or v("Cannot open rules file $fn"); if ($fhok) { my $c = 0; while (my $line = <$fh>) { $c++; chomp $line; chop($line) if ($line =~ m/\r$/); trim($line); next if $line =~ /^#/ || $line eq ''; $self->load_rule($c, $line); } close $fh; } } else { v("File not found $fn"); return 1; } } else { v("No rules file"); return 1; } _d(9,'Loaded rules:',dd(9,$self->{'rules'})); return 0; } # Load an augmentation rule into a hash of rules sub load_rule () { my ($self, $linenum, $rule) = @_; # Decode the rule data using the specified encoding (defaults to UTF-8) $rule = decode($self->{'encoding'}, $rule); # Each rule consists of rule 'type' followed by the rule itself, separated by | char my @f = split /\|/, $rule; if (scalar @f != 2) { v("Wrong number of fields on line $linenum \n"); return 1; } my ($ruletype, $ruletext) = @f; # Do some basic validation if (!defined $ruletype || $ruletype eq '' || $ruletype !~ m/\d+/) { v("Invalid rule type on line $linenum \n"); return 1; } if (!defined $ruletext || $ruletext eq '') { v("Invalid rule text on line $linenum \n"); return 1; } # Text to try and match against the programme title is stored in a hash of arrays # to shortcut the list of possible matches to those beginning with the same # first two characters as the title. It would seem to be quicker to use a regex # to match some amount of text up to colon character in the programme title, # and then use a hash lookup against the matched text. However, there may be # colons in the rule text, so this approach cannot be used. # Each rule contains a number of elements (the exact number depends on the rule type) separated by ~ chars. # The first element will be a key to identify which data records will be processed for this rule type @f = split /~/, $ruletext; my ($k, $v) = ($ruletext . '~') =~ /^(.*?)~(.*)$/; chop $v; # (don't do any further validation on the rules; to do so would mean parsing each and every rule in the file even when # only a few (if any) of them will be met for any given augmentation rule - i.e. very slow and generally pointless - # we'll validate them later at time-of-use) # TODO: make a separate validation function to do this # Text-based rules are stored segregated by the first 2 chars of the key (to make subsequent list searching faster) my $idx = lc(substr ($k, 0, 2)); # Store the rule my $data = { 'line' => $linenum, 'key' => $k, 'value' => $v }; push @{ $self->{'rules'}->{$ruletype}->{$idx} }, $data; } # Replace our wildcards ("%%") in the rule's key # sub replace_wild () { my ($self, $key) = @_; if ($key =~ m/%%/) { $key = quotemeta($key); $key =~ s/\\%\\%/%%/g; $key =~ s/%%/\.\*\?/g; return qr/^$key$/; } else { return qr/^\Q$key\E$/; } } # Create an xmltv_ns compatible episode number. # Automatically resets the base to zero on series/episode/part numbers # (input should NOT be rebased - e.g. pass the actual episode enuember) # Input = programme hash and hash of new data to be inserted # sub make_ns_epnum () { my ($self, $prog, $_prog) = @_; my $s = $_prog->{'season'} if defined $_prog->{'season'} && $_prog->{'season'} ne ''; my $s_tot = $_prog->{'season_total'} if defined $_prog->{'season_total'} && $_prog->{'season_total'} ne ''; my $e = $_prog->{'episode'} if defined $_prog->{'episode'} && $_prog->{'episode'} ne '' && $_prog->{'episode'} ne 0; my $e_tot = $_prog->{'episode_total'} if defined $_prog->{'episode_total'} && $_prog->{'episode_total'} ne ''; my $p = $_prog->{'part'} if defined $_prog->{'part'} && $_prog->{'part'} ne ''; my $p_tot = $_prog->{'part_total'} if defined $_prog->{'part_total'} && $_prog->{'part_total'} ne ''; # sanity check undef($s) if defined $s && $s eq '0'; undef($e) if defined $e && $e eq '0'; undef($p) if defined $p && $p eq '0'; undef($p_tot) if defined $p_tot && $p_tot eq '0'; # re-base the series/episode/part numbers $s-- if (defined $s && $s ne ''); $e-- if (defined $e && $e ne ''); $p-- if (defined $p && $p ne ''); # make the xmltv_ns compliant episode-num my $episode_ns = ''; $episode_ns .= $s if (defined $s && $s ne ''); $episode_ns .= '/'.$s_tot if (defined $s_tot && $s_tot ne ''); $episode_ns .= '.'; $episode_ns .= $e if (defined $e && $e ne ''); $episode_ns .= '/'.$e_tot if (defined $e_tot && $e_tot ne ''); $episode_ns .= '.'; $episode_ns .= $p if (defined $p && $p ne ''); $episode_ns .= '/'.$p_tot if (defined $p_tot && $p_tot ne ''); _d(3,'Make <episode-num>:',$episode_ns); # delete existing 'xmltv_ns' details if no series/ep/part # details are available if ($episode_ns eq '..') { if (defined $prog->{'episode-num'}) { @{$prog->{'episode-num'}} = map { $prog->{'episode-num'}[$_][1] eq 'xmltv_ns' ? () : $prog->{'episode-num'}[$_] } 0 .. $#{$prog->{'episode-num'}}; } return ''; } # otherwise, find the 'xmltv_ns' details in the prog my $xmltv_ns_old; if (defined $prog->{'episode-num'}) { foreach (@{$prog->{'episode-num'}}) { if ($_->[1] eq 'xmltv_ns') { # found it; insert our element $xmltv_ns_old = $_->[0]; $_->[0] = $episode_ns; last; } } } # no 'xmltv_ns' attribute found; create a suitable element if (!defined $xmltv_ns_old) { push @{$prog->{'episode-num'}}, [ $episode_ns, 'xmltv_ns' ]; } return $episode_ns; } # Parse an xmltv_ns <episode_num> element into its # component parts # # Second param should be passed by reference and returns the # text value of the <episode-num> (with spaces removed). # sub extract_ns_epnum () { my ($self, $prog, $xmltv_ns) = @_; if (defined $prog->{'episode-num'}) { # find the 'xmltv_ns' details ##my $xmltv_ns; foreach (@{$prog->{'episode-num'}}) { if ($_->[1] eq 'xmltv_ns') { $$xmltv_ns = $_->[0]; last; } } if (defined $$xmltv_ns) { # simplify the regex by stripping spaces $$xmltv_ns =~ s/\s//g; # extract the fields from the element # rebase appropriately if ( $$xmltv_ns =~ /^(\d+)?(?:\/(\d+))?(?:(?:\.(\d+)?(?:\/(\d+))?)(?:\.(\d+)?(?:\/(\d+))?)?)?$/ ) { my %episode_num; $episode_num{'season'} = $1 +1 if defined $1; $episode_num{'season_total'} = $2 if defined $2; $episode_num{'episode'} = $3 +1 if defined $3; $episode_num{'episode_total'} = $4 if defined $4; $episode_num{'part'} = $5 +1 if defined $5; $episode_num{'part_total'} = $6 if defined $6; _d(5,'Decoded <episode-num>:',dd(5,\%episode_num)); return \%episode_num; } } } _d(5,'No <episode-num> found'); return undef; } ############################################### ############ GENERAL SUBROUTINES ############## ############################################### # Return the digit equivalent of its word, i.e. "one" -> "1", # or return the word if it appears to consist of only digits sub word_to_digit ($;$) { my $word = shift; return undef if ! defined $word; return $word if $word =~ m/^\d+$/; my $lang = shift; $lang = 'EN' if !defined $lang; my %nums; if ($lang eq 'EN') { # handle 1-9 in roman numberals %nums = ( one => 1, two => 2, three => 3, four => 4, five => 5, six => 6, seven => 7, eight => 8, nine => 9, ten => 10, eleven => 11, twelve => 12, thirteen => 13, fourteen => 14, fifteen => 15, sixteen => 16, seventeen => 17, eighteen => 18, nineteen => 19, twenty => 20, i => 1, ii => 2, iii => 3, iv => 4, v => 5, vi => 6, vii => 7, viii => 8, ix => 9 ); } for (lc $word) { return $nums{$_} if exists $nums{$_}; } return undef; } # Remove leading & trailing spaces sub trim { # Remove leading & trailing spaces $_[0] =~ s/^\s+|\s+$//g; } ############################################### ############# DEBUG SUBROUTINES ############### ############################################### # open log file sub open_log (;$) { my $fn = shift; my $mode = ($debug ? '>>' : '>'); # open append while debugging - avoids issue with tail on truncated files open(my $fh, $mode, $fn) or die "cannot open $fn: $!"; $logh = $fh; print $logh "\n" . ($debug ? '-'x80 ."\n\n\n" : ''); } # close log file sub close_log () { if ($logh) { close($logh) or warn "close failed on log file: $!"; } } # write to log file sub l ($) { my ($msg) = @_; print $logh $msg . "\n" if $logh; } # print a message sub v ($) { my ($msg) = @_; print STDERR $msg . "\n"; l($msg . "\n"); } # write a debug message sub _d ($@) { my ($level, @msg) = @_; return if $debug < $level; foreach (@msg) { print STDERR $_ . " "; } print STDERR "\n"; } # dump a variable (for use with _d) sub dd ($$) { # don't call Dumper if we aren't going to be output! my $level = $_[0]; return if $debug < $level; my $s = $_[1]; require Data::Dumper; my $d = Data::Dumper::Dumper($s); $d =~ s/^\$VAR1 =\s*/ /; $d =~ s/;$//; chomp $d; return "\n".$d; } # get the caller's subroutine name sub self () { my $self = (caller(1))[3]; $self =~ s/(.*::)//; # drop the package return $self; } ############################################### 1; # keep eval happy ;-) __END__ =pod =head1 AUTHOR Geoff Westcott, honir.at.gmail.dot.com, Dec. 2014. This code is based on the "fixup" method/code defined in tv_grab_uk_rt grabber and credit is given to the author Nick Morrott. =cut �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Capabilities.pm���������������������������������������������������������������������0000664�0000000�0000000�00000001034�15000742332�0016355�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Add a --capabilities argument to your program, eg # # use XMLTV::Version qw/baseline manualconfig/; # package XMLTV::Capabilities; my $opt = '--capabilities'; sub import( $$ ) { die "usage: use $_[0] qw/<capabilities>/" if @_ < 2; my $seen = 0; foreach (@ARGV) { # This doesn't handle abbreviations in the GNU style. last if $_ eq '--'; if ($_ eq $opt) { $seen++ && warn "seen '$opt' twice\n"; } } return if not $seen; eval { print join "\n", @_[1..$#_]; print "\n"; }; exit(); } 1; ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Clumps.pm���������������������������������������������������������������������������0000664�0000000�0000000�00000032504�15000742332�0015235�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Routines for handling the 'clump index' associated with some # programmes. This is a way of working around missing information in # some listings sources by saying that two or more programmes share a # timeslot, they appear in a particular order, but we don't know the # exact time when one stops and the next begins. # # For example if the listings source gives at 11:00 'News; Weather' # then we know that News has start time 11:00 and clumpidx 0/2, while # Weather has start time 11:00 and clumpidx 1/2. We know that Weather # follows News, and they are both in the 11:00 timeslot, but not more # than that. # # This clumpidx stuff does its job, but it's ugly to deal with - as # demonstrated by the existence of this library. I plan to replace it # soonish. # # The purpose of this module is to let you alter or delete programmes # which are part of a clump without having to worry about updating the # others. The module exports routines for building a symmetric # 'relation' relating pairs of scalars; you should use that to relate # programmes which share a clump. Then after modifying a programme # which has a clumpidx set, call fix_clumps() passing in the relation, # and it will modify the other programmes in the clump. # # Again, this all works but a better mechanism is needed. package XMLTV::Clumps; use XMLTV::Date; use Date::Manip; # no Date_Init(), that can be done by the app use Tie::RefHash; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } # Won't Memoize, you can do that yourself. use base 'Exporter'; our @EXPORT_OK = qw(new_relation related relate unrelate nuke_from_rel relatives clump_relation fix_clumps); sub new_relation(); sub related( $$$ ); sub relate( $$$ ); sub unrelate( $$$ ); sub nuke_from_rel( $$ ); sub relatives( $$ ); sub clump_relation( $ ); sub fix_clumps( $$$ ); sub check_same_channel( $ ); # private # Routines to handle a symmmetric 'relation'. This is used to keep # track of which programmes are sharing a clump so that fix_clumps() # can sort them out if needed. # # FIXME make this OO. # sub new_relation() { die 'usage: new_relation()' if @_; my %h; tie %h, 'Tie::RefHash'; return \%h; } sub related( $$$ ) { die 'usage: related(relation, a, b)' if @_ != 3; my ($rel, $a, $b) = @_; my $list = $rel->{$a}; return 0 if not defined $list; foreach (@$list) { return 1 if "$_" eq "$b"; } return 0; } sub relate( $$$ ) { die 'usage: related(relation, a, b)' if @_ != 3; my ($rel, $a, $b) = @_; unless (related($rel, $a, $b)) { check_same_channel([$a, $b]); push @{$rel->{$a}}, $b; push @{$rel->{$b}}, $a; } } sub unrelate( $$$ ) { die 'usage: related(relation, a, b)' if @_ != 3; my ($rel, $a, $b) = @_; die unless related($rel, $a, $b) and related($rel, $b, $a); @{$rel->{$a}} = grep { "$_" ne "$b" } @{$rel->{$a}}; @{$rel->{$b}} = grep { "$_" ne "$a" } @{$rel->{$b}}; } sub nuke_from_rel( $$ ) { die 'usage: nuke_from_rel(relation, a)' if @_ != 2; my ($rel, $a) = @_; die unless ref($rel) eq 'HASH'; foreach (@{relatives($rel, $a)}) { die unless related($rel, $a, $_); unrelate($rel, $a, $_); } # Tidy up by removing from hash die if defined $rel->{$a} and @{$rel->{$a}}; delete $rel->{$a}; } sub relatives( $$ ) { die 'usage: relatives(relation, a)' if @_ != 2; my ($rel, $a) = @_; die unless ref($rel) eq 'HASH'; if ($rel->{$a}) { return [ @{$rel->{$a}} ]; # make a copy } else { return []; } } # Private. Wrappers for Date::Manip and XMLTV::Date; sub pd( $ ) { for ($_[0]) { return undef if not defined; return parse_date($_); } } # Make a relation grouping together programmes sharing a clump. # # Parameter: reference to list of programmes # # Returns: a relation saying which programmes share clumps. # sub clump_relation( $ ) { my $progs = shift; my $related = new_relation(); my %todo; foreach (@$progs) { my $clumpidx = $_->{clumpidx}; next if not defined $clumpidx or $clumpidx eq '0/1'; push @{$todo{$_->{channel}}->{pd($_->{start})}}, $_; } t 'updating $related from todo list'; foreach my $ch (keys %todo) { our %times; local *times = $todo{$ch}; my $times = $todo{$ch}; foreach my $t (keys %times) { t "todo list for channel $ch, time $t"; my @l = @{$times{$t}}; t 'list of programmes: ' . d(\@l); foreach my $ai (0 .. $#l) { foreach my $bi ($ai+1 .. $#l) { my $a = $l[$ai]; my $b = $l[$bi]; t "$a and $b related"; die if "$a" eq "$b"; warn "$a, $b over-related" if related($related, $a, $b); relate($related, $a, $b); } } } } return $related; } # fix_clumps() # # When a programme sharing a clump has been modified or replaced, # patch things up so that other things in the clump are consistent. # # Parameters: # original programme # (ref to) list of new programmes resulting from it # clump relation # # Modifies the programme and others in its clump as necessary. # sub fix_clumps( $$$ ) { die 'usage: fix_clumps(old programme, listref of replacements, clump relation)' if @_ != 3; my ($orig, $new, $rel) = @_; # Optimize common case. return if not defined $orig->{clumpidx} or $orig->{clumpidx} eq '0/1'; die if ref($rel) ne 'HASH'; die if ref($new) ne 'ARRAY'; our @new; local *new = $new; # local $Log::TraceMessages::On = 1; t 'fix_clumps() ENTRY'; t 'original programme: ' . d $orig; t 'new programmes: ' . d \@new; t 'clump relation: ' . d $rel; sub by_start { Date_Cmp(pd($a->{start}), pd($b->{start})) } sub by_clumpidx { $a->{clumpidx} =~ m!^(\d+)/(\d+)$! or die; my ($ac, $n) = ($1, $2); $b->{clumpidx} =~ m!^(\d+)/$n$! or die; my $bc = $1; if ($ac == $bc) { t 'do not sort: ' . d($a) . ' and ' . d($b); warn "$a->{clumpidx} and $b->{clumpidx} do not sort"; } $ac <=> $bc; } sub by_date { by_start($a, $b) || by_clumpidx($a, $b) || warn "programmes do not sort"; } my @relatives = @{relatives($rel, $orig)}; if (not @relatives) { # local $Log::TraceMessages::On = 1; t 'programme without relatives: ' . d $orig; warn "programme has clumpidx of $orig->{clumpidx}, but cannot find others in same clump\n"; return; } check_same_channel(\@relatives); @relatives = sort by_date @relatives; t 'relatives of orig (sorted): ' . d \@relatives; check_same_channel(\@new); # could relax this later t 'orig turned into: ' . d \@new; t 'how many programmes has $prog been split into?'; if (@new == 0) { t 'deleted programme entirely!'; nuke_from_rel($rel, $orig); if (@relatives == 0) { die; } elsif (@relatives == 1) { delete $relatives[0]->{clumpidx}; } elsif (@relatives >= 2) { # Just decrement the index of all following programmes. my $orig_clumpidx = $orig->{clumpidx}; $orig_clumpidx =~ /^(\d+)/ or die; $orig_clumpidx = $1; foreach (@relatives) { my $rel_clumpidx = $_->{clumpidx}; $rel_clumpidx =~ /^(\d+)/ or die; $rel_clumpidx = $1; -- $rel_clumpidx if $rel_clumpidx > $orig_clumpidx; $_->{clumpidx} = "$rel_clumpidx/" . scalar @relatives; } } else { die } } elsif (@new >= 1) { # local $Log::TraceMessages::On = 1; t 'split into one or more programmes'; @new = sort by_date @$new; nuke_from_rel($rel, $orig); if (@relatives) { # Find where the original programme slotted into the clump # and insert the new programmes there. # my @old_all = sort by_date ($orig, @relatives); check_same_channel(\@old_all); t 'old clump sorted by date (incl. orig): ' . d \@old_all; @new = sort by_date @new; t 'new shows sorted by date: ' . d \@new; # Fix the start and end times of the other shows in the # clump. The shows in @new may give different (narrower) # times to the one show they came from, so that we have # more information about the start and end times of the # other shows in the clump. Eg 09:30 0/2 '09:30 AAA, # 10:00 BBB' sharing a clump with 09:30 1/2 'CCC'. When # the first programme gets split into two, we know that # the start time for C must be 10:00 at the earliest. # Clear? # # Keep around both parsed and unparsed versions of the # same date, to keep timezone information. This needs to # be handled better. # my $start_new_unp = $new->[0]->{start}; my $start_new = pd($start_new_unp); t "new shows start at $start_new"; # The known stop time for @new is the last date # mentioned. Eg if the last show ends at 10:00 we know # @new as a whole ends at 10:00. But if the last show has # no stop time but starts at 09:30 then we know @new as a # whole ends at *at the earliest* 09:30. # my $stop_new; foreach (reverse @new) { foreach (pd($_->{start}), pd($_->{stop})) { next if not defined; if (not defined $stop_new or Date_Cmp($_, $stop_new) > 0) { $stop_new = $_; } } } t "lub of new shows is $stop_new"; # However if other shows shared a clump, they do not start # at the stop time of @new! They overlap with it. The # shows coming later in the clump will have the same start # time as the last show of @new. # # For example, two shows in a clump from 10:00 to 11:00. # The first is split into something at 10:00 and something # at 10:30. The second part of the original clump will # now 'start' at 10:30 and overlap with the last of the # new shows. # my $start_last_new_unp = $new[-1]->{start}; my $start_last_new = pd($start_last_new_unp); t 'last of the new programmes starts at: ' . d $start_last_new; # Add the programmes coming before @new to the output. # These should have stop times before @new's start. # my @new_all; t 'add shows coming before replaced one'; while (@old_all) { my $old = shift @old_all; last if $old eq $orig; t "adding 'before' show: " . d $old; die if not defined $old->{start}; die if not defined $start_new; die unless Date_Cmp(pd($old->{start}), $start_new) <= 0; my $old_stop = pd($old->{stop}); t 'has stop time: ' . d $old_stop; # if (defined $old_stop) { # die if not defined $stop_new; # die "stop time $old_stop of old programme is earlier than lub of new shows $stop_new" # if Date_Cmp($old_stop, $stop_new) < 0; # die "stop time $old_stop of old programme is earlier than start of new shows $start_new" # if Date_Cmp($old_stop, $start_new) < 0; # } $old->{stop} = $start_new_unp; t "set stop time to $old->{stop}"; push @new_all, $old; } # Slot in the new programmes. t 'got to orig show, slot in new programmes'; push @new_all, @new; t 'so far, list of new programmes: ' . d \@new_all; # Now the shows at the end, after the programme which was # split. # t 'do shows coming after the orig one'; while (@old_all) { my $old = shift @old_all; t "doing 'after' show: " . d $old; my $old_start = pd($old->{start}); die if not defined $old_start; t "current start time: $old_start"; die if not defined $start_new; die if not defined $stop_new; die unless Date_Cmp($start_new, $old_start) <= 0; die unless Date_Cmp($old_start, $stop_new) <= 0; # These shows overlapped with the old programme. So # now they will overlap with the last of the shows it # was split into. # $old->{start} = $start_last_new_unp; t "set start time to $old->{start}"; t 'adding programme to list: ' . d $old; push @new_all, $old; } t 'new list of programmes from original clump: ' . d \@new_all; check_same_channel(\@new_all); t 'now regenerate the clumpidxes'; while (@new_all) { my $first = shift @new_all; t 'taking first programme from list: ' . d $first; t 'building clump for this programme'; my @clump = ($first); my $start = pd($first->{start}); die if not defined $start; while (@new_all) { my $next = shift @new_all; die if not defined $next->{start}; if (not Date_Cmp(pd($next->{start}), $start)) { push @clump, $next; } else { unshift @new_all, $next; last; } } t 'clump is: ' . d \@clump; my $clump_size = scalar @clump; t "$clump_size shows in clump"; for (my $i = 0; $i < $clump_size; $i++) { my $c = $clump[$i]; if ($clump_size == 1) { t 'deleting clumpidx from programme'; delete $c->{clumpidx}; } else { $c->{clumpidx} = "$i/$clump_size"; t "set clumpidx for programme to $c->{clumpidx}"; } } t 're-relating programmes in this clump (if more than one)'; foreach my $a (@clump) { foreach my $b (@clump) { next if $a == $b; relate($rel, $a, $b); } } } t 'finished regenerating clumpidxes'; } } } # Private. sub check_same_channel( $ ) { my $progs = shift; my $ch; foreach my $prog (@$progs) { for ($prog->{channel}) { if (not defined) { t 'no channel! ' . d $prog; die 'programme has no channel'; } if (not defined $ch) { $ch = $_; } elsif ($ch eq $_) { # Okay. } else { t 'same clump, different channels: ' . d($progs->[0]) . ' and ' . d($prog); die "programmes in same clump have different channels: $_, $ch"; } } } } 1; ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Configure.pm������������������������������������������������������������������������0000664�0000000�0000000�00000026037�15000742332�0015717�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������package XMLTV::Configure; # use version number for feature detection: # 0.005065 : can use 'constant' in write_string() # 0.005065 : comments in config file not restricted to starting in first column # 0.005066 : make writes to the config-file atomic our $VERSION = 0.005066; BEGIN { use Exporter (); our (@ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS); @ISA = qw(Exporter); @EXPORT = qw( ); %EXPORT_TAGS = ( ); # eg: TAG => [ qw!name1 name2! ], @EXPORT_OK = qw/LoadConfig SaveConfig Configure SelectChannelsStage/; } our @EXPORT_OK; use XMLTV::Ask; use XMLTV::Config_file; use XML::LibXML; =head1 NAME XMLTV::Configure - Configuration file handling for XMLTV grabbers =head1 DESCRIPTION Utility library that helps grabbers read from configuration files and implement a configuration method that can be run from the command-line. =head1 EXPORTED FUNCTIONS All these functions are exported on demand. =over 4 =cut =item LoadConfig Takes the name of the configuration file to load as a parameter. Returns a hashref with configuration fieldnames as keys. Note that the values of the hash are references to an array of values. Example: { username => [ 'mattias' ], password => [ 'xxx' ], channel => [ 'svt1.svt.se', 'kanal5.se' ], no_channel => ['svt2.svt.se' ], } Note that unselected options from a selectmany are collected in an entry named after the key with a prefix of 'no_'. See the channel and no_channel entry in the example. They are the result of a selectmany with id=channel. The configuration file must be in the format described in the file "ConfigurationFiles.txt". If the file does not exist or if the format is wrong, LoadConfig returns undef. =cut sub LoadConfig { my( $config_file ) = @_; my $data = {}; open IN, "< $config_file" or return undef; foreach my $line (<IN>) { $line =~ tr/\n\r//d; next if $line =~ /^\s*$/; next if $line =~ /^\s*#/; # Only accept lines with key=value or key!value. # No white-space is allowed before # the equal-sign. White-space after the equal-sign is considered # part of the value, except for white-space at the end of the line # which is ignored. my( $key, $sign, $value ) = ($line=~ /^(\S+?)([=!])(.*?)\s*(#.*)?$/ ); return undef unless defined $key; if( $sign eq '=' ) { push @{$data->{$key}}, $value; } else { push @{$data->{"no_$key"}}, $value; } } close IN; return $data; } =item SaveConfig Write a configuration hash in the format returned by LoadConfig to a file that can be loaded with LoadConfig. Takes two parameters, a reference to a configuration hash and a filename. Note that a grabber should normally never have to call SaveConfig. This is done by the Configure-method. =cut sub SaveConfig { my( $conf, $config_file ) = @_; # Test if configuration file is writeable if (-f $config_file && !(-w $config_file)) { die "Cannot write to $config_file"; } # Create temporary configuration file. open OUT, "> $config_file.TMP" or die "Failed to open $config_file.TMP for writing."; foreach my $key (keys %{$conf}) { next if $key eq "channel"; next if $key eq "lineup"; foreach my $value (@{$conf->{$key}}) { print OUT "$key=$value\n"; } } if (exists $conf->{lineup}) { print OUT "lineup=$conf->{lineup}[0]\n"; } elsif( exists( $conf->{channel} ) ) { foreach my $value (@{$conf->{channel}}) { print OUT "$key=$value\n"; } } close OUT; # Store temporary configuration file rename "$config_file.TMP", $config_file or die "Failed to write to $config_file"; } =item Configure Generates a configuration file for the grabber. Takes three parameters: stagesub, listsub and the name of the configuration file. stagesub shall be a coderef that takes a stage-name or undef and a configuration hashref as a parameter and returns an xml-string that describes the configuration necessary for that stage. The xml-string shall follow the xmltv-configuration.dtd. listsub shall be a coderef that takes a configuration hash as returned by LoadConfig as the first parameter and an option hash as returned by ParseOptions as the second parameter and returns an xml-string containing a list of all the channels that the grabber can deliver data for using the supplied configuration. Note that the listsub shall not use any channel-configuration from the hashref. =cut sub Configure { my( $stagesub, $listsub, $conffile, $opt ) = @_; # How can we read the language from the environment? my $lang = 'en'; my $nextstage = 'start'; # Test if configuration file is writeable if (-f $conffile && !(-w $conffile)) { die "Cannot write to $conffile"; } # Create temporary configuration file. open OUT, "> $conffile.TMP" or die "Failed to write to $conffile.TMP"; close OUT; do { my $stage = &$stagesub( $nextstage, LoadConfig( "$conffile.TMP" ) ); $nextstage = configure_stage( $stage, $conffile, $lang ); } while ($nextstage ne "select-channels" ); # No more nextstage. Let the user select channels. Do not present # channel selection if the configuration is using lineups where # channels are determined automatically my $conf = LoadConfig( "$conffile.TMP" ); if (! exists $conf->{lineup}) { my $channels = &$listsub( $conf, $opt ); select_channels( $channels, $conffile, $lang ); } # Store temporary configuration file rename "$conffile.TMP", $conffile or die "Failed to write to $conffile"; } sub configure_stage { my( $stage, $conffile, $lang ) = @_; my $nextstage = undef; open OUT, ">> $conffile.TMP" or die "Failed to open $conffile.TMP for writing"; my $xml = XML::LibXML->new; my $doc = $xml->parse_string($stage); binmode(STDERR, ":utf8") if ($doc->encoding eq "utf-8"); my $ns = $doc->find( "//xmltvconfiguration/*" ); foreach my $p ($ns->get_nodelist) { my $tag = $p->nodeName; if( $tag eq "nextstage" ) { $nextstage = $p->findvalue( '@stage' ); last; } my $id = $p->findvalue( '@id' ); my $title = getvalue( $p, 'title', $lang ); my $description = getvalue( $p, 'description', $lang ); my $default = $p->findvalue( '@default' ); my $constant = $p->findvalue( '@constant' ); my $value; my $q = $default ne '' ? "$title: [$default]" : "$title:"; say( "$description" ) if $constant eq ''; if( $tag eq 'string' ) { $value = $constant if $constant ne ''; $value = ask( "$q" ) if $constant eq ''; $value = $default if $value eq ""; print OUT "$id=$value\n"; } elsif( $tag eq 'secretstring' ) { $value = ask_password( "$q" ); $value = $default if $value eq ""; print OUT "$id=$value\n"; } # This must be a selectone or selectmany my( @optionvalues, @optiontexts ); my $ns2 = $p->find( "option" ); foreach my $p2 ($ns2->get_nodelist) { push @optionvalues, $p2->findvalue( '@value' ); push @optiontexts, getvalue( $p2, 'text', $lang ); } if( $tag eq "selectone" ) { my $selected = ask_choice( "$title:", $optiontexts[0], @optiontexts ); for( my $i=0; $i<scalar( @optiontexts ); $i++ ) { if( $optiontexts[$i] eq $selected ) { $value=$optionvalues[$i]; } } print OUT "$id=$value\n"; } elsif( $tag eq "selectmany" ) { my @answers = ask_many_boolean( 0, @optiontexts ); for( my $i=0; $i < scalar( @answers ); $i++ ) { if( $answers[$i] ) { print OUT "$id=$optionvalues[$i]\n"; } else { print OUT "$id!$optionvalues[$i]\n"; } } } } close OUT; return $nextstage; } sub select_channels { my( $channels, $conffile, $lang ) = @_; open OUT, ">> $conffile.TMP" or die "Failed to open $conffile.TMP for writing"; my $xml = XML::LibXML->new; my $doc; $doc = $xml->parse_string($channels); my $ns = $doc->find( "//channel" ); my @channelname; my @channelid; foreach my $p ($ns->get_nodelist) { push @channelid, $p->findvalue( '@id' ); push @channelname, getvalue($p, "display-name", $lang ); } # We need to internationalize this string. say( "Select the channels that you want to receive data for." ); my @answers = ask_many_boolean( 0, @channelname ); for( my $i=0; $i < scalar( @answers ); $i++ ) { if( $answers[$i] ) { print OUT "channel=$channelid[$i]\n"; } else { print OUT "channel!$channelid[$i]\n"; } } close OUT; } sub SelectChannelsStage { my( $channels, $grabber_name ) = @_; my $xml = XML::LibXML->new; my $doc; $doc = $xml->parse_string($channels); my $encoding = $doc->encoding; my $ns = $doc->find( "//channel" ); my $result; my $writer = new XMLTV::Configure::Writer( OUTPUT => \$result, encoding => $encoding ); $writer->start( { grabber => $grabber_name } ); $writer->start_selectmany( { id => 'channel', title => [ [ 'Channels', 'en' ] ], description => [ [ "Select the channels that you want to receive data for.", 'en' ] ], } ); foreach my $p ($ns->get_nodelist) { # FIXME: Preserve all languages for the display-name $writer->write_option( { value=>$p->findvalue( '@id' ), text=> => [ [ getvalue($p, "display-name", 'en' ), 'en'] ], } ); } $writer->end_selectmany(); $writer->end( 'end' ); return $result; } sub getvalue { my( $p, $field, $lang ) = @_; # Try the correct language first my $value = $p->findvalue( $field . "[\@lang='$lang']"); # Use English if there is no value for the correct language. $value = $p->findvalue( $field . "[\@lang='en']") unless length( $value ) > 0; # Take the first available value as a last resort. $value = $p->findvalue( $field . "[1]") unless length( $value ) > 0; $value =~ s/^\s+//; $value =~ s/\s+$//; $value =~ tr/\n\r / /s; return $value; } =back =head1 COPYRIGHT Copyright (C) 2005 Mattias Holmlund. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. =cut ### Setup indentation in Emacs ## Local Variables: ## perl-indent-level: 4 ## perl-continued-statement-offset: 4 ## perl-continued-brace-offset: 0 ## perl-brace-offset: -4 ## perl-brace-imaginary-offset: 0 ## perl-label-offset: -2 ## cperl-indent-level: 4 ## cperl-brace-offset: 0 ## cperl-continued-brace-offset: 0 ## cperl-label-offset: -2 ## cperl-extra-newline-before-brace: t ## cperl-merge-trailing-else: nil ## cperl-continued-statement-offset: 2 ## indent-tabs-mode: t ## End: �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Configure/��������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0015351�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Configure/Writer.pm�����������������������������������������������������������������0000664�0000000�0000000�00000021610�15000742332�0017163�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������package XMLTV::Configure::Writer; use strict; use warnings; # use version number for feature detection: # 0.005065 : can use 'constant' in write_string() our $VERSION = 0.005065; BEGIN { use Exporter (); our (@ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS); @ISA = qw(Exporter); @EXPORT = qw( ); %EXPORT_TAGS = ( ); # eg: TAG => [ qw!name1 name2! ], @EXPORT_OK = qw//; } our @EXPORT_OK; use XML::Writer 0.600; use base 'XML::Writer'; use Carp; =pod =encoding utf8 =head1 NAME XMLTV::Configure::Writer - Configuration file writer for XMLTV grabbers =head1 DESCRIPTION Utility class that helps grabbers write configuration descriptions. =head1 SYNOPSIS use XMLTV::Configure::Writer; my $result; my $writer = new XMLTV::Writer::Configure( OUTPUT => \$result, encoding => 'iso-8859-1' ); $writer->start( { grabber => 'tv_grab_xxx' } ); $writer->write_string( { id => 'username', title => [ [ 'Username', 'en' ], [ 'Användarnamn', 'sv' ] ], description => [ [ 'The username for logging in to DataDirect.', 'en' ], [ 'Användarnamn hos DataDirect', 'sv' ] ], } ); $writer->start_selectone( { id => 'lineup', title => [ [ 'Lineup', 'en' ], [ 'Programpaket', 'sv' ] ], description => [ [ 'The lineup of channels for your region.', 'en' ], [ 'Programpaket för din region', 'sv' ] ], } ); $writer->write_option( { value=>'eastcoast', text=> => [ [ 'East Coast', 'en' ], [ 'Östkusten', 'sv' ] ] } ); $writer->write_option( { value=>'westcoast', text=> => [ [ 'West Coast', 'en' ], [ 'Västkusten', 'sv' ] ] } ); $writer->end_selectone(); $writer->end(); print $result; =head1 EXPORTED FUNCTIONS None. =cut sub new { my $proto = shift; my $class = ref($proto) || $proto; my %args = @_; croak 'OUTPUT requires a filehandle, not a filename or anything else' if exists $args{OUTPUT} and not ref $args{OUTPUT}; my $encoding = delete $args{encoding}; my $self = $class->SUPER::new(DATA_MODE => 1, DATA_INDENT => 2, %args); bless($self, $class); if (defined $encoding) { $self->xmlDecl($encoding); } else { # XML::Writer puts in 'encoding="UTF-8"' even if you don't ask # for it. # warn "assuming default UTF-8 encoding for output\n"; $self->xmlDecl(); } # { # What is a correct doctype??? # local $^W = 0; $self->doctype('tv', undef, 'xmltv.dtd'); # } $self->{xmltv_state} = 'new'; return $self; } =head1 METHODS =over =item start() Write the start of the <xmltvconfiguration> element. Parameter is a hashref which gives the attributes of this element. =cut sub start { my $self = shift; die 'usage: XMLTV::Writer->start(hashref of attrs)' if @_ != 1; my $attrs = shift; $self->{xmltv_state} eq 'new' or croak 'cannot call start() more than once on XMLTV::Writer'; $self->startTag( 'xmltvconfiguration', %{$attrs} ); $self->{xmltv_state}='root'; } =item write_string() Write a <string> element. Parameter is a hashref with the data for the element: $writer->write_string( { id => 'username', title => [ [ 'Username', 'en' ], [ 'Användarnamn', 'sv' ] ], description => [ [ 'The username for logging in to DataDirect.', 'en' ], [ 'Användarnamn hos DataDirect', 'sv' ] ], default => "", } ); To add a constant use 'constant' key: If constant value is empty then revert to 'ask' procedure. $writer->write_string( { id => 'version', title => [ [ 'Version number', 'en' ] ], description => [ [ 'Automatically added version number - no user input', 'en' ] ], constant => '123', } ); =back =cut sub write_string { my ($self, $ch) = @_; $self->write_string_tag( 'string', $ch ); } sub write_secretstring { my ($self, $ch) = @_; $self->write_string_tag( 'secretstring', $ch ); } sub write_string_tag { my ($self, $tag, $ch) = @_; croak 'undef parameter hash passed' if not defined $ch; croak "expected a hashref, got: $ch" if ref $ch ne 'HASH'; for ($self->{xmltv_state}) { if ($_ eq 'new') { croak 'must call start() on XMLTV::Configure::Writer first'; } elsif ($_ eq 'root') { # Okay. } elsif ($_ eq 'selectone') { croak 'cannot write string inside selectone'; } elsif ($_ eq 'selectmany') { croak 'cannot write string inside selectmany'; } elsif ($_ eq 'end') { croak 'cannot write string after end()'; } else { die } } my %ch = %$ch; # make a copy my $id = delete $ch{id}; die "missing 'id' in string" if not defined $id; my %h = ( id => $id ); my $default = delete $ch{default}; $h{default} = $default if defined $default; my $constant = delete $ch{constant}; $h{constant} = $constant if defined $constant; $self->startTag( $tag, %h ); my $titles = delete $ch{title}; die "missing 'title' in string" if not defined $titles; $self->write_lang_tag( 'title', $titles ); my $descriptions = delete $ch{description}; die "missing 'description' in string" if not defined $descriptions; $self->write_lang_tag( 'description', $descriptions ); $self->endTag( $tag ) } sub start_selectone { my ($self, $ch) = @_; $self->start_select( 'selectone', $ch ); } sub start_selectmany { my ($self, $ch) = @_; $self->start_select( 'selectmany', $ch ); } sub end_selectone { my $self = shift; $self->end_select( 'selectone' ); } sub end_selectmany { my $self = shift; $self->end_select( 'selectmany' ); } sub start_select { my ($self, $tag, $ch) = @_; croak 'undef parameter hash passed' if not defined $ch; croak "expected a hashref, got: $ch" if ref $ch ne 'HASH'; for ($self->{xmltv_state}) { if ($_ eq 'new') { croak 'must call start() on XMLTV::Configure::Writer first'; } elsif ($_ eq 'root') { # Okay. } elsif ($_ eq 'selectone') { croak "cannot write $tag inside selectone"; } elsif ($_ eq 'selectmany') { croak "cannot write $tag inside selectmany"; } elsif ($_ eq 'end') { croak "cannot write $tag after end()"; } else { die } } my %ch = %$ch; # make a copy my $id = delete $ch{id}; die "missing 'id' in $tag" if not defined $id; my %h = ( id => $id ); my $default = delete $ch{default}; $h{default} = $default if defined $default; $self->startTag( $tag, %h ); my $titles = delete $ch{title}; die "missing 'title' in string" if not defined $titles; $self->write_lang_tag( 'title', $titles ); my $descriptions = delete $ch{description}; die "missing 'description' in string" if not defined $descriptions; $self->write_lang_tag( 'description', $descriptions ); $self->{xmltv_state} = $tag; } sub end_select { my( $self, $tag ) = @_; if( $self->{xmltv_state} ne $tag ) { croak "cannot write end-tag for $tag without a matching start-tag"; } $self->endTag( $tag ); $self->{xmltv_state} = 'root'; } sub write_option { my $self = shift; my( $data ) = @_; for ($self->{xmltv_state}) { if ($_ eq 'new') { croak 'must call start() on XMLTV::Configure::Writer first'; } elsif ($_ eq 'root') { croak "cannot write option outside of selectone or selectmany"; } elsif ($_ eq 'selectone') { # Okay } elsif ($_ eq 'selectmany') { # Okay } elsif ($_ eq 'end') { croak "cannot write option after end()"; } else { die } } my $value = delete $data->{value}; croak "Missing value for option-tag" unless defined $value; $self->startTag( 'option', value => $value ); $self->write_lang_tag( 'text', $data->{text} ); $self->endTag( 'option' ); } sub write_lang_tag { my $self = shift; my( $tag, $aref ) = @_; foreach my $texts (@{$aref}) { my $text =$texts->[0]; my $lang = $texts->[1]; $self->startTag( $tag, lang => $lang ); $self->characters( $text ); $self->endTag( $tag ); } } sub end { my $self = shift; my( $nextstage ) = @_; if( not defined $nextstage ) { croak "must supply a nextstage parameter to end()"; } for ($self->{xmltv_state}) { if ($_ eq 'new') { croak 'must call start() first'; } elsif ($_ eq 'end') { croak 'cannot call end twice'; } } $self->emptyTag( 'nextstage', ( stage => $nextstage ) ); $self->endTag('xmltvconfiguration'); $self->SUPER::end(@_); } 1; ### Setup indentation in Emacs ## Local Variables: ## perl-indent-level: 4 ## perl-continued-statement-offset: 4 ## perl-continued-brace-offset: 0 ## perl-brace-offset: -4 ## perl-brace-imaginary-offset: 0 ## perl-label-offset: -2 ## cperl-indent-level: 4 ## cperl-brace-offset: 0 ## cperl-continued-brace-offset: 0 ## cperl-label-offset: -2 ## cperl-extra-newline-before-brace: t ## cperl-merge-trailing-else: nil ## cperl-continued-statement-offset: 2 ## indent-tabs-mode: t ## End: ������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Data/�������������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0014301�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Data/Recursive/���������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0016250�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Data/Recursive/Encode.pm������������������������������������������������������������0000664�0000000�0000000�00000012005�15000742332�0020001�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# A wrapper around Data::Recursive::Encode from Tokuhiro Matsuno # http://search.cpan.org/~tokuhirom/Data-Recursive-Encode-0.04/lib/Data/Recursive/Encode.pm # package XMLTV::Data::Recursive::Encode; ##################################### #package Data::Recursive::Encode; ##use 5.008001; # in e-mails the author has said he can't support versions <5.8.1 but he can't see why it won't work in earlier versions use strict; use warnings FATAL => 'all'; our $VERSION = '0.04'; use Encode (); use Carp (); use Scalar::Util qw(blessed refaddr); sub _apply { my $code = shift; my $seen = shift; my @retval; for my $arg (@_) { if(my $ref = ref $arg){ my $refaddr = refaddr($arg); my $proto; if(defined($proto = $seen->{$refaddr})){ # noop } elsif($ref eq 'ARRAY'){ $proto = $seen->{$refaddr} = []; @{$proto} = _apply($code, $seen, @{$arg}); } elsif($ref eq 'HASH'){ $proto = $seen->{$refaddr} = {}; %{$proto} = _apply($code, $seen, %{$arg}); } elsif($ref eq 'REF' or $ref eq 'SCALAR'){ $proto = $seen->{$refaddr} = \do{ my $scalar }; ${$proto} = _apply($code, $seen, ${$arg}); } else{ # CODE, GLOB, IO, LVALUE etc. $proto = $seen->{$refaddr} = $arg; } push @retval, $proto; } else{ push @retval, defined($arg) ? $code->($arg) : $arg; } } return wantarray ? @retval : $retval[0]; } sub decode { my ($class, $encoding, $stuff, $check) = @_; $encoding = Encode::find_encoding($encoding) || Carp::croak("$class: unknown encoding '$encoding'"); $check ||= 0; _apply(sub { $encoding->decode($_[0], $check) }, {}, $stuff); } sub encode { my ($class, $encoding, $stuff, $check) = @_; $encoding = Encode::find_encoding($encoding) || Carp::croak("$class: unknown encoding '$encoding'"); $check ||= 0; _apply(sub { $encoding->encode($_[0], $check) }, {}, $stuff); } sub decode_utf8 { my ($class, $stuff, $check) = @_; _apply(sub { Encode::decode_utf8($_[0], $check) }, {}, $stuff); } sub encode_utf8 { my ($class, $stuff) = @_; _apply(sub { Encode::encode_utf8($_[0]) }, {}, $stuff); } sub from_to { my ($class, $stuff, $from_enc, $to_enc, $check) = @_; @_ >= 4 or Carp::croak("Usage: $class->from_to(OCTET, FROM_ENC, TO_ENC[, CHECK])"); $from_enc = Encode::find_encoding($from_enc) || Carp::croak("$class: unknown encoding '$from_enc'"); $to_enc = Encode::find_encoding($to_enc) || Carp::croak("$class: unknown encoding '$to_enc'"); _apply(sub { Encode::from_to($_[0], $from_enc, $to_enc, $check) }, {}, $stuff); return $stuff; } 1; __END__ =encoding utf8 =head1 NAME XMLTV::Data::Recursive::Encode - Encode/Decode Values In A Structure =head1 SYNOPSIS use XMLTV::Data::Recursive::Encode; XMLTV::Data::Recursive::Encode->decode('euc-jp', $data); XMLTV::Data::Recursive::Encode->encode('euc-jp', $data); XMLTV::Data::Recursive::Encode->decode_utf8($data); XMLTV::Data::Recursive::Encode->encode_utf8($data); XMLTV::Data::Recursive::Encode->from_to($data, $from_enc, $to_enc[, $check]); =head1 DESCRIPTION XMLTV::Data::Recursive::Encode visits each node of a structure, and returns a new structure with each node's encoding (or similar action). If you ever wished to do a bulk encode/decode of the contents of a structure, then this module may help you. =head1 METHODS =over 4 =item decode my $ret = XMLTV::Data::Recursive::Encode->decode($encoding, $data, [CHECK]); Returns a structure containing nodes which are decoded from the specified encoding. =item encode my $ret = XMLTV::Data::Recursive::Encode->encode($encoding, $data, [CHECK]); Returns a structure containing nodes which are encoded to the specified encoding. =item decode_utf8 my $ret = XMLTV::Data::Recursive::Encode->decode_utf8($data, [CHECK]); Returns a structure containing nodes which have been processed through decode_utf8. =item encode_utf8 my $ret = XMLTV::Data::Recursive::Encode->encode_utf8($data); Returns a structure containing nodes which have been processed through encode_utf8. =item from_to my $ret = XMLTV::Data::Recursive::Encode->from_to($data, FROM_ENC, TO_ENC[, CHECK]); Returns a structure containing nodes which have been processed through from_to. =back =head1 AUTHOR Tokuhiro Matsuno E<lt>tokuhirom AAJKLFJEF GMAIL COME<gt> gfx =head1 SEE ALSO This module is inspired from L<Data::Visitor::Encode>, but this module depended to too much modules. I want to use this module in pure-perl, but L<Data::Visitor::Encode> depend to XS modules. L<Unicode::RecursiveDowngrade> does not supports perl5's Unicode way correctly. =head1 LICENSE Copyright (C) 2010 Tokuhiro Matsuno All rights reserved. This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. =cut ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Date.pm�����������������������������������������������������������������������������0000664�0000000�0000000�00000012240�15000742332�0014642�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������=pod =head1 NAME XMLTV::Date - Date parsing routines for the xmltv package =head1 SEE ALSO L<Date::Manip> =cut package XMLTV::Date; # use version number for feature detection: # 0.005066 : added time_xxx subs our $VERSION = 0.005066; use warnings; use strict; use Carp qw(croak); use base 'Exporter'; our @EXPORT = qw(parse_date time_xmltv_to_iso time_iso_to_xmltv time_xmltv_to_epoch time_iso_to_epoch); use Date::Manip; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } # These are populated when needed with the current time but then # reused later. # my $now; my $this_year; =pod =head1 C<parse_date()> Wrapper for C<Date::Manip::ParseDate()> that does two things: firstly, if the year is not specified it chooses between last year, this year and next year depending on which date would be closest to now. (If only one of those dates is valid, for example because day-of-week is specified, then the valid one is chosen; if the time can only be parsed without adding an explicit year then that is chosen.) Secondly, an exception is thrown if the date cannot be parsed. Argument is a single string. =cut sub parse_date( $ ) { my $raw = shift; my $parsed; # Assume any string of 4 digits means the year. if ($raw =~ /\d{4}/) { $parsed = ParseDate($raw); croak "cannot parse date '$raw'" if not $parsed; return $parsed; } # Year not specified, see which fits best. if (not defined $now) { $now = ParseDate('now'); die if not $now; $this_year = UnixDate($now, '%Y'); die if $this_year !~ /^\d{4}$/; } my @poss; foreach (map { ParseDate("$raw $_") } ($this_year - 1 .. $this_year + 1)) { push @poss, $_ if $_; } if (not @poss) { # Well, tacking on a year didn't work, perhaps we'll have to # just parse the string as supplied. # $parsed = ParseDate($raw); croak "cannot parse date '$raw'" if not $parsed; return $parsed; } my $best_distance; my $best; foreach (@poss) { die if not defined; my $delta = DateCalc($now, $_); my $seconds = Delta_Format($delta, 0, '%st'); die "cannot get seconds for delta '$delta'" if not length $seconds; $seconds = abs($seconds); if (not defined $best or $seconds < $best_distance) { $best = $_; $best_distance = $seconds; } } die if not defined $best; return $best; } =pod =head1 C<time_xmltv_to_iso()> Converts a XMLTV time e.g. "20140412090000 +0300" to ISO format i.e. "2014-04-12T09:00:00.000+03:00" Argument is string time to convert. =cut sub time_xmltv_to_iso ( $ ) { $_[0] =~ m/^(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})\s([\+-])(\d{2})(\d{2})$/; return "$1-$2-$3T$4:$5:$6.000$7$8:$9"; } =pod =head1 C<time_iso_to_xmltv()> Converts an ISO time e.g. "2014-04-12T09:00:00.000+03:00" to XMLTV format, i.e. "20140412090000 +0300" Argument is string time to convert. =cut sub time_iso_to_xmltv ( $ ) { my $time = shift; $time =~ s/[:-]//g; $time =~ /^(\d{8})T(\d{6}).*(\+\d{4})$/; return $1.$2.' '.$3; } =pod =head1 C<time_xmltv_to_epoch()> Converts a XMLTV time e.g. "20140412090000 +0300" to seconds since the epoch (uses POSIX::mktime rather than Date::Manip to avoid issues with the latter) Alternatively you could use DateTime::Format::XMLTV on CPAN Argument is string time to convert. Optional 2nd argument: set to 1 ignore the tz offset in the calculation =cut sub time_xmltv_to_epoch ( $;$ ) { my $time = shift; my $ignoreoffset = shift; # set to 1 to ignore tz offset (i.e. 'local' epoch; else will get utc) my ($y, $m, $d, $h, $i, $s, $t, $th, $tm) = $time =~ m/^(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})\s([\+-])(\d{2})(\d{2})$/; $y -= 1900; $m -= 1; # re-base for mktime() use POSIX qw(mktime); my $epoch = POSIX::mktime($s, $i, $h, $d, $m, $y); if (!defined $ignoreoffset || !$ignoreoffset) { # note this isn't exact since it doesn't account for leap seconds, etc my $offset = ($th * 3600) + ($tm * 60); $epoch += $offset if $t eq '-'; $epoch -= $offset if $t eq '+'; } return $epoch; } =pod =head1 C<time_iso_to_epoch()> Converts an iso time (e.g. "2014-04-12T09:00:00.000+03:00") to epoch time (uses POSIX::mktime rather than Date::Manip to avoid issues with the latter) Alternatively you could use DateTime::Format::XMLTV on CPAN Argument is string time to convert. Optional 2nd argument: set to 1 ignore the tz offset in the calculation =cut sub time_iso_to_epoch ( $;$ ) { my $time = shift; my $ignoreoffset = shift; # set to 1 to ignore tz offset (i.e. 'local' epoch; else will get utc) my ($y, $m, $d, $h, $i, $s, $ms, $t, $th, $tm) = $time =~ m/^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})\.(\d{3})([\+-])(\d{2}):(\d{2})$/; $y -= 1900; $m -= 1; # re-base for mktime() use POSIX qw(mktime); my $epoch = POSIX::mktime($s, $i, $h, $d, $m, $y); if (!defined $ignoreoffset || !$ignoreoffset) { # note this isn't exact since it doesn't account for leap seconds, etc my $offset = ($th * 3600) + ($tm * 60); $epoch += $offset if $t eq '-'; $epoch -= $offset if $t eq '+'; } return $epoch; } 1; ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Description.pm����������������������������������������������������������������������0000664�0000000�0000000�00000000763�15000742332�0016257�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Add a --description argument to your program, eg # # use XMLTV::Description "Sweden (tv.swedb.se)"; # package XMLTV::Description; my $opt = '--description'; sub import( $$ ) { die "usage: use $_[0] \"Sweden (tv.swedb.se)" if @_ != 2; my $seen = 0; foreach (@ARGV) { # This doesn't handle abbreviations in the GNU style. last if $_ eq '--'; if ($_ eq $opt) { $seen++ && warn "seen '$opt' twice\n"; } } return if not $seen; print $_[1] . "\n"; exit(); } 1; �������������xmltv-1.4.0/lib/GUI.pm������������������������������������������������������������������������������0000664�0000000�0000000�00000004661�15000742332�0014421�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������=pod =head1 NAME XMLTV::GUI - Handles the choice of UI for XMLTV =head1 SYNOPSIS use XMLTV::GUI; my $gui_type = get_gui_type($opt_gui); where $opt_gui is the commandline option given for --gui and $gui_type is one of 'term+progressbar', 'term' and 'tk'. Determines the type of output the user has requested for XMLTV to communicate through. =head1 AUTHOR Andy Balaam, axis3x3@users.sourceforge.net. Distributed as part of the xmltv package. =head1 SEE ALSO L<XMLTV> =cut package XMLTV::GUI; use strict; use vars qw(@ISA @EXPORT_OK); use Exporter; @ISA = qw(Exporter); @EXPORT_OK = qw(get_gui_type); # Use Log::TraceMessages if installed, and choose graphical or not. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } sub get_gui_type( $ ) { my $opt_gui = shift; # If the user passed in a --gui option, work on that basis, otherwise use # the environment variable if (defined $opt_gui) { return _get_specified_gui_type($opt_gui); } else { return _get_specified_gui_type($ENV{XMLTV_GUI}); } } sub _get_specified_gui_type( $ ) { my $spec_gui = shift; # Return the best match to the specified gui if it is available # If we haven't got windows, or we were asked for terminal, we do # terminal stylee. if ( !_check_for_windowing_env() or !defined($spec_gui) or $spec_gui =~ /^term/i) { # Check whether we at least have the terminal progress bar if (defined($spec_gui) && $spec_gui =~ /^termnoprogressbar$/i or !eval{ require Term::ProgressBar }) { return 'term'; } else { return 'term+progressbar'; } # Now try Tk first } elsif ( $spec_gui eq '' or $spec_gui =~ /^tk$/i or $spec_gui eq '1' ) { if ( _check_for_tk() ) { return 'tk'; } else { warn "The Tk gui library is unavailable. Reverting to terminal"; return 'term'; } # And finally give up and go to terminal } else { warn "Unknown gui type requested: '$spec_gui'. Reverting to terminal"; return 'term'; } } sub _check_for_windowing_env() { return defined($ENV{DISPLAY}) || $^O eq 'MSWin32'; } sub _check_for_tk() { return eval{ require Tk && require Tk::ProgressBar }; } 1; �������������������������������������������������������������������������������xmltv-1.4.0/lib/Gunzip.pm���������������������������������������������������������������������������0000664�0000000�0000000�00000012705�15000742332�0015247�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������=pod =head1 NAME XMLTV::Gunzip - Wrapper to Compress::Zlib or gzip(1) =head1 SYNOPSIS use XMLTV::Gunzip; my $decompressed = gunzip($gzdata); my $fh = gunzip_open('file.gz') or die; while (<$fh>) { print } Compress::Zlib will be used if installed, otherwise an external gzip will be spawned. gunzip() returns the decompressed data and throws an exception if things go wrong; gunzip_open() returns a filehandle, or undef. =head1 AUTHOR Ed Avis, ed@membled.com. Distributed as part of the xmltv package. =head1 SEE ALSO L<Compress::Zlib>, L<gzip(1)>, L<XMLTV>. =cut use warnings; use strict; package XMLTV::Gunzip; use base 'Exporter'; our @EXPORT; @EXPORT = qw(gunzip gunzip_open); use File::Temp; # Implementations of gunzip(). # sub zlib_gunzip( $ ) { for (Compress::Zlib::memGunzip(shift)) { die 'memGunzip() failed' if not defined; return $_; } } sub external_gunzip( $ ) { my ($fh, $fname) = File::Temp::tempfile(); print $fh shift or die "cannot write to $fname: $!"; close $fh or die "cannot close $fname: $!"; open(GZIP, "gzip -d <$fname |") or die "cannot run gzip: $!"; local $/ = undef; my $r = <GZIP>; close GZIP or die "cannot close pipe from gzip: $!"; unlink $fname or die "cannot unlink $fname: $!"; return $r; } my $gunzip_f; sub gunzip( $ ) { return $gunzip_f->(shift) } # Implementations of gunzip_open(). # sub perlio_gunzip_open( $ ) { my $fname = shift; # Use PerlIO::gzip. local *FH; open FH, '<:gzip', $fname or die "cannot open $fname via PerlIO::gzip: $!"; return *FH; } sub zlib_gunzip_open( $ ) { my $fname = shift; # Use the XMLTV::Zlib_handle package defined later in this file. local *FH; tie *FH, 'XMLTV::Zlib_handle', $fname, 'r' or die "cannot open $fname using XMLTV::Zlib_handle: $!"; return *FH; } sub external_gunzip_open( $ ) { my $fname = shift; local *FH; if (not open(FH, "gzip -d <$fname |")) { warn "cannot run gzip: $!"; return undef; } return *FH; } my $gunzip_open_f; sub gunzip_open( $ ) { return $gunzip_open_f->(shift) } # Switch between implementations depending on whether Compress::Zlib # is available. # BEGIN { eval { require Compress::Zlib }; my $have_zlib = not $@; eval { require PerlIO::gzip }; my $have_perlio = not $@; if (not $have_zlib and not $have_perlio) { $gunzip_f = \&external_gunzip; $gunzip_open_f = \&external_gunzip_open; } elsif (not $have_zlib and $have_perlio) { # Could gunzip by writing to a file and reading that with # PerlIO, but won't bother yet. # $gunzip_f = \&external_gunzip; $gunzip_open_f = \&perlio_gunzip_open; } elsif ($have_zlib and not $have_perlio) { $gunzip_f = \&zlib_gunzip; $gunzip_open_f = \&zlib_gunzip_open; } elsif ($have_zlib and $have_perlio) { $gunzip_f = \&zlib_gunzip; $gunzip_open_f = \&perlio_gunzip_open; } else { die } } #### # This is a filehandle wrapper around Compress::Zlib, but supporting # only read at the moment. # package XMLTV::Zlib_handle; require Tie::Handle; use base 'Tie::Handle'; use Carp; sub TIEHANDLE { croak 'usage: package->TIEHANDLE(file, mode)' if @_ != 3; my ($pkg, $file, $mode) = @_; croak "only mode 'r' is supported" if $mode ne 'r'; # This object is a reference to a Compress::Zlib handle. I did # try to inherit directly from Compress::Zlib, but got weird # errors of '(in cleanup) gzclose is not a valid Zlib macro'. # my $fh = Compress::Zlib::gzopen($file, $mode); if (not $fh) { warn "could not gzopen $file"; return undef; } return bless(\$fh, $pkg); } # Assuming that WRITE() is like print(), not like syswrite(). sub WRITE { my ($self, $scalar, $length, $offset) = @_; return 1 if not $length; my $r = $$self->gzwrite(substr($scalar, $offset, $length)); if ($r == 0) { warn "gzwrite() failed"; return 0; } elsif (0 < $r and $r < $length) { warn "gzwrite() wrote only $r of $length bytes"; return 0; } elsif ($r == $length) { return 1; } else { die } } # PRINT(), PRINTF() inherited from Tie::Handle sub READ { my ($self, $scalar, $length, $offset) = @_; local $_; my $n = $$self->gzread($_, $length); if ($n == -1) { warn 'gzread() failed'; return undef; } elsif ($n == 0) { # EOF. return 0; } elsif (0 < $n and $n <= $length) { die if $n != length; substr($scalar, $offset, $n) = $_; return $n; } else { die } } sub READLINE { my $self = shift; # When gzreadline() uses $/, this can be removed. die '$/ not supported' if $/ ne "\n"; local $_; my $r = $$self->gzreadline($_); if ($r == -1) { warn 'gzreadline() failed'; return undef; } elsif ($r == 0) { # EOF. die if length; return undef; } else { # Number of bytes read. die if $r != length; return $_; } } # GETC inherited from Tie::Handle # This seems to segfault in my perl installation. sub CLOSE { my $self = shift; gzclose $$self; # no meaningful return value? return 1; } sub OPEN { # Compress::Zlib doesn't support reopening. my $self = shift; die 'not yet implemented'; } sub BINMODE {} sub EOF { my $self = shift; return $$self->gzeof(); } sub TELL { # Could track position manually. But Compress::Zlib should do it. die 'not implemented'; } sub SEEK { # Argh, fairly impossible. Could simulate, but probably better to # throw. # die 'not implemented'; } sub DESTROY { &CLOSE } 1; �����������������������������������������������������������xmltv-1.4.0/lib/IMDB.pm�����������������������������������������������������������������������������0000664�0000000�0000000�00000336234�15000742332�0014514�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# The IMDB file contains two packages: # 1. XMLTV::IMDB::Cruncher package which parses and manages IMDB "lists" files # from ftp.imdb.com # 2. XMLTV::IMDB package that uses data files from the Cruncher package to # update/add details to XMLTV programme nodes. # # FUTURE - multiple hits on the same 'title only' could try and look for # character names matching from description to imdb.com character # names. # # FUTURE - multiple hits on 'title only' should probably pick latest # tv series over any older ones. May make for better guesses. # # BUG - we identify 'presenters' by the word "Host" appearing in the character # description. For some movies, character names include the word Host. # ex. Animal, The (2001) has a character named "Badger Milk Host". # # BUG - if there is a matching title with > 1 entry (say made for tv-movie and # at tv-mini series) made in the same year (or even "close" years) it is # possible for us to pick the wrong one we should pick the one with the # closest year, not just the first closest match based on the result ordering # for instance Ghost Busters was made in 1984, and into a tv series in # 1986. if we have a list of GhostBusters 1983, we should pick the 1984 movie # and not 1986 tv series...maybe :) but currently we'll pick the first # returned close enough match instead of trying the closest date match of # the approx hits. # use strict; package XMLTV::IMDB; use Search::Dict; use open ':encoding(iso-8859-1)'; # try to enforce file encoding (does this work in Perl <5.8.1? ) # # HISTORY # .6 = what was here for the longest time # .7 = fixed file size est calculations # = moviedb.info now includes _file_size_uncompressed values for each downloaded file # .8 = updated file size est calculations # = moviedb.dat directors and actors list no longer include repeated names (which mostly # occured in episodic tv programs (reported by Alexy Khrabrov) # .9 = added keywords data # .10 = added plot data # .11 = revised method for database creation to reduce memory use # bug: remove duplicated genres # bug: if TV-version and movie in same year then one (random) was lost # bug: multiple films with same title in same year then one was lost # bug: movies with (aka...) in title not handled properly # bug: incorrect data generated for a tv series (only the last episode found is stored) # bug: genres and cast are rolled-up from all episodes to the series record (misleading) # bug: multiple matches can sometimes extract the first one it comes across as a 'hit' # (potentially wrong - it should not augment incoming prog when multiple matches) # dbbuild: --filesort to sort interim data on disc rather than in memory # dbbuild: --nosystemsort to use File::Sort rather than operating system shell's 'sort' command # dbbuild: --movies-only to exclude tv-series (etc.) from database build # # our $VERSION = '0.11'; # version number of database sub new { my ($type) = shift; my $self={ @_ }; # remaining args become attributes for ('imdbDir', 'verbose') { die "invalid usage - no $_" if ( !defined($self->{$_})); } #$self->{verbose}=2; $self->{replaceDates}=0 if ( !defined($self->{replaceDates})); $self->{replaceTitles}=0 if ( !defined($self->{replaceTitles})); $self->{replaceCategories}=0 if ( !defined($self->{replaceCategories})); $self->{replaceKeywords}=0 if ( !defined($self->{replaceKeywords})); $self->{replaceURLs}=0 if ( !defined($self->{replaceURLs})); $self->{replaceDirectors}=1 if ( !defined($self->{replaceDirectors})); $self->{replaceActors}=0 if ( !defined($self->{replaceActors})); $self->{replacePresentors}=1 if ( !defined($self->{replacePresentors})); $self->{replaceCommentators}=1 if ( !defined($self->{replaceCommentators})); $self->{replaceStarRatings}=0 if ( !defined($self->{replaceStarRatings})); $self->{replacePlot}=0 if ( !defined($self->{replacePlot})); $self->{updateDates}=1 if ( !defined($self->{updateDates})); $self->{updateTitles}=1 if ( !defined($self->{updateTitles})); $self->{updateCategories}=1 if ( !defined($self->{updateCategories})); $self->{updateCategoriesWithGenres}=1 if ( !defined($self->{updateCategoriesWithGenres})); $self->{updateKeywords}=0 if ( !defined($self->{updateKeywords})); # default is to NOT add keywords $self->{updateURLs}=1 if ( !defined($self->{updateURLs})); $self->{updateDirectors}=1 if ( !defined($self->{updateDirectors})); $self->{updateActors}=1 if ( !defined($self->{updateActors})); $self->{updatePresentors}=1 if ( !defined($self->{updatePresentors})); $self->{updateCommentators}=1 if ( !defined($self->{updateCommentators})); $self->{updateStarRatings}=1 if ( !defined($self->{updateStarRatings})); $self->{updatePlot}=0 if ( !defined($self->{updatePlot})); # default is to NOT add plot $self->{numActors}=3 if ( !defined($self->{numActors})); # default is to add top 3 actors $self->{moviedbIndex}="$self->{imdbDir}/moviedb.idx"; $self->{moviedbData}="$self->{imdbDir}/moviedb.dat"; $self->{moviedbInfo}="$self->{imdbDir}/moviedb.info"; $self->{moviedbOffline}="$self->{imdbDir}/moviedb.offline"; # default is not to cache lookups $self->{cacheLookups}=0 if ( !defined($self->{cacheLookups}) ); $self->{cacheLookupSize}=0 if ( !defined($self->{cacheLookupSize}) ); $self->{cachedLookups}->{tv_series}->{_cacheSize_}=0; bless($self, $type); $self->{categories}={'movie' =>'Movie', 'tv_movie' =>'TV Movie', # made for tv 'video_movie' =>'Video Movie', # went straight to video or was made for it 'tv_series' =>'TV Series', 'tv_mini_series' =>'TV Mini Series'}; $self->{stats}->{programCount}=0; for my $cat (keys %{$self->{categories}}) { $self->{stats}->{perfect}->{$cat}=0; $self->{stats}->{close}->{$cat}=0; } $self->{stats}->{perfectMatches}=0; $self->{stats}->{closeMatches}=0; $self->{stats}->{startTime}=time(); return($self); } sub loadDBInfo($) { my $file=shift; my $info; open(INFO, "< $file") || return("imdbDir index file \"$file\":$!\n"); while(<INFO>) { chomp(); if ( s/^([^:]+)://o ) { $info->{$1}=$_; } } close(INFO); return($info); } sub checkIndexesOkay($) { my $self=shift; if ( ! -d "$self->{imdbDir}" ) { return("imdbDir \"$self->{imdbDir}\" does not exist\n"); } if ( -f "$self->{moviedbOffline}" ) { return("imdbDir index offline: check $self->{moviedbOffline} for details"); } for my $file ($self->{moviedbIndex}, $self->{moviedbData}, $self->{moviedbInfo}) { if ( ! -f "$file" ) { return("imdbDir index file \"$file\" does not exist\n"); } } $VERSION=~m/^(\d+)\.(\d+)$/o || die "package corrupt, VERSION string invalid ($VERSION)"; my ($major, $minor)=($1, $2); my $info=loadDBInfo($self->{moviedbInfo}); return($info) if ( ref $info eq 'SCALAR' ); if ( !defined($info->{db_version}) ) { return("imdbDir index db missing version information, rerun --prepStage all\n"); } if ( $info->{db_version}=~m/^(\d+)\.(\d+)$/o ) { if ( $1 != $major || $2 < $minor ) { return("imdbDir index db requires updating, rerun --prepStage all\n"); } if ( $1 == 0 && $2 == 1 ) { return("imdbDir index db requires update, rerun --prepStage 5 (bug:actresses never appear)\n"); } if ( $1 == 0 && $2 == 2 ) { # 0.2 -> 0.3 upgrade requires prepStage 5 to be re-run return("imdbDir index db requires minor reindexing, rerun --prepStage 3 and 5\n"); } if ( $1 == 0 && $2 == 3 ) { # 0.2 -> 0.3 upgrade requires prepStage 5 to be re-run return("imdbDir index db requires major reindexing, rerun --prepStage 2 and new prepStages 5,6,7,8 and 9\n"); } if ( $1 == 0 && $2 == 4 ) { # 0.2 -> 0.3 upgrade requires prepStage 5 to be re-run return("imdbDir index db corrupt (got version 0.4), rerun --prepStage all\n"); } # okay return(undef); } else { return("imdbDir index version of '$info->{db_version}' is invalid, rerun --prepStage all\n". "if problem persists, submit bug report to xmltv-devel\@lists.sf.net\n"); } } sub basicVerificationOfIndexes($) { my $self=shift; # check that the imdbdir is valid and up and running my $title="Army of Darkness"; my $year=1992; $self->openMovieIndex() || return("basic verification of indexes failed\n". "database index isn't readable"); my $verbose = $self->{verbose}; $self->{verbose} = 0; my $res=$self->getMovieMatches($title, $year); $self->{verbose} = $verbose; undef $verbose; if ( !defined($res) ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "no match for basic verification of movie \"$title, $year\"\n"); } if ( !defined($res->{exactMatch}) ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "no exact match for movie \"$title, $year\"\n"); } if ( scalar(@{$res->{exactMatch}})!= 1) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "got more than one exact match for movie \"$title, $year\"\n"); } my @exact=@{$res->{exactMatch}}; if ( $exact[0]->{title} ne $title ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "title associated with key \"$title, $year\" is bad\n"); } if ( $exact[0]->{year} ne "$year" ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "year associated with key \"$title, $year\" is bad\n"); } my $id=$exact[0]->{id}; $res=$self->getMovieIdDetails($id); if ( !defined($res) ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "no movie details for movie \"$title, $year\" (id=$id)\n"); } if ( !defined($res->{directors}) ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "movie details didn't provide any director for movie \"$title, $year\" (id=$id)\n"); } if ( !$res->{directors}[0]=~m/Raimi/o ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "movie details didn't show Raimi as the main director for movie \"$title, $year\" (id=$id)\n"); } if ( !defined($res->{actors}) ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "movie details didn't provide any cast movie \"$title, $year\" (id=$id)\n"); } if ( !$res->{actors}[0]=~m/Campbell/o ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "movie details didn't show Bruce Campbell as the main actor in movie \"$title, $year\" (id=$id)\n"); } my $matches=0; for (@{$res->{genres}}) { if ( $_ eq "Action" || $_ eq "Comedy" || $_ eq "Fantasy" || $_ eq "Horror" || $_ eq "Romance" ) { $matches++; } } if ( $matches == 0 ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "movie details didn't show genres correctly for movie \"$title, $year\" (id=$id)\n"); } if ( !defined($res->{ratingDist}) || !defined($res->{ratingVotes}) || !defined($res->{ratingRank}) ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "movie details didn't show imdbratings for movie \"$title, $year\" (id=$id)\n"); } $self->closeMovieIndex(); # all okay return(undef); } sub sanityCheckDatabase($) { my $self=shift; my $errline; $errline=$self->checkIndexesOkay(); return($errline) if ( defined($errline) ); $errline=$self->basicVerificationOfIndexes(); return($errline) if ( defined($errline) ); # all okay return(undef); } sub error($$) { print STDERR "tv_imdb: $_[1]\n"; } sub status($$) { if ( $_[0]->{verbose} ) { print STDERR "tv_imdb: $_[1]\n"; } } sub debug($$) { my $self=shift; my $mess=shift; if ( $self->{verbose} > 1 ) { print STDERR "tv_imdb: $mess\n"; } } sub openMovieIndex($) { my $self=shift; if ( !open($self->{INDEX_FD}, "< $self->{moviedbIndex}") ) { return(undef); } if ( !open($self->{DBASE_FD}, "< $self->{moviedbData}") ) { close($self->{INDEX_FD}); return(undef); } return(1); } sub closeMovieIndex($) { my $self=shift; close($self->{INDEX_FD}); delete($self->{INDEX_FD}); close($self->{DBASE_FD}); delete($self->{DBASE_FD}); return(1); } # moviedbIndex is a TSV file with the format: # searchtitle title year progtype lineno # sub getMovieMatches($$$) { my $self=shift; my $title=shift; my $year=shift; # Articles are put at the end of a title ( in all languages ) #$match=~s/^(The|A|Une|Las|Les|Los|L\'|Le|La|El|Das|De|Het|Een)\s+(.*)$/$2, $1/og; # append year to title my $match="$title"; if ( defined($year) && $title!~m/\s+\((19|20)\d\d\)/o ) { $match.=" ($year)"; } # to encode s/([^a-zA-Z0-9_.-])/uc sprintf("%%%02x",ord($1))/oeg # to decode s/%(?:([0-9a-fA-F]{2})|u([0-9a-fA-F]{4}))/defined($1)? chr hex($1) : utf8_chr(hex($2))/oge; # url encode $match=lc($match); $match=~s/([^a-zA-Z0-9_.-])/uc sprintf("%%%02x",ord($1))/oeg; $self->debug("looking for \"$match\" in $self->{moviedbIndex}"); if ( !$self->{INDEX_FD} ) { die "internal error: index not open"; } my $FD=$self->{INDEX_FD}; Search::Dict::look(*{$FD}, $match, 0, 0); my $results; while (<$FD>) { last if ( !m/^$match/ ); chomp(); my @arr=split('\t', $_); if ( scalar(@arr) != 5 ) { warn "$self->{moviedbIndex} corrupt (correct key:$_)"; next; } if ( $arr[0] eq $match ) { # return title and id #$arr[1]=~s/(.*),\s*(The|A|Une|Las|Les|Los|L\'|Le|La|El|Das|De|Het|Een)$/$2 $1/og; #$arr[0]=~s/%(?:([0-9a-fA-F]{2})|u([0-9a-fA-F]{4}))/defined($1)? chr hex($1) : utf8_chr(hex($2))/oge; #$self->debug("exact:$arr[1] ($arr[2]) qualifier=$arr[3] id=$arr[4]"); my $title=$arr[1]; if ( $title=~s/\s+\((\d\d\d\d|\?\?\?\?)\)$//o ) { } elsif ( $title=~s/\s+\((\d\d\d\d|\?\?\?\?)\/[IVXL]+\)$//o ) { } else { die "unable to decode year from title key \"$title\", report to xmltv-devel\@lists.sf.net"; } $title=~s/(.*),\s*(The|A|Une|Las|Les|Los|L\'|Le|La|El|Das|De|Het|Een)$/$2 $1/og; $self->debug("exact:$title ($arr[2]) qualifier=$arr[3] id=$arr[4]"); push(@{$results->{exactMatch}}, {'key'=> $arr[1], 'title'=>$title, 'year'=>$arr[2], 'qualifier'=>$arr[3], 'id'=>$arr[4]}); } else { # decode #s/%(?:([0-9a-fA-F]{2})|u([0-9a-fA-F]{4}))/defined($1)? chr hex($1) : utf8_chr(hex($2))/oge; # return title #$arr[1]=~s/(.*),\s*(The|A|Une|Las|Les|Los|L\'|Le|La|El|Das|De|Het|Een)$/$2 $1/og; #$arr[0]=~s/%(?:([0-9a-fA-F]{2})|u([0-9a-fA-F]{4}))/defined($1)? chr hex($1) : utf8_chr(hex($2))/oge; #$self->debug("close:$arr[1] ($arr[2]) qualifier=$arr[3] id=$arr[4]"); my $title=$arr[1]; if ( $title=~m/^\"/o && $title=~m/\"\s*\(/o ) { #" $title=~s/^\"//o; #" $title=~s/\"(\s*\()/$1/o; #" } if ( $title=~s/\s+\((\d\d\d\d|\?\?\?\?)\)$//o ) { } elsif ( $title=~s/\s+\((\d\d\d\d|\?\?\?\?)\/[IVXL]+\)$//o ) { } else { die "unable to decode year from title key \"$title\", report to xmltv-devel\@lists.sf.net"; } $title=~s/(.*),\s*(The|A|Une|Las|Les|Los|L\'|Le|La|El|Das|De|Het|Een)$/$2 $1/og; $self->debug("close:$title ($arr[2]) qualifier=$arr[3] id=$arr[4]"); push(@{$results->{closeMatch}}, {'key'=> $arr[1], 'title'=>$title, 'year'=>$arr[2], 'qualifier'=>$arr[3], 'id'=>$arr[4]}); } } #print "MovieMatches on ($match) = ".Dumper($results)."\n"; return($results); } sub getMovieExactMatch($$$) { my $self=shift; my $title=shift; my $year=shift; my $res=$self->getMovieMatches($title, $year); return(undef, 0) if ( !defined($res) ); if ( !defined($res->{exactMatch}) ) { return(undef, 0); } if ( scalar(@{$res->{exactMatch}}) != 1 ) { return(undef, scalar(@{$res->{exactMatch}})); } return($res->{exactMatch}[0], 1); } sub getMovieCloseMatches($$) { my $self=shift; my $title=shift; my $res=$self->getMovieMatches($title, undef) || return(undef); if ( defined($res->{exactMatch})) { die "corrupt imdb database - hit on \"$title\""; } return(undef) if ( !defined($res->{closeMatch}) ); my @arr=@{$res->{closeMatch}}; #print "CLOSE DUMP=".Dumper(@arr)."\n"; return(@arr); } # moviedbData file is a TSV file with the format: # lineno:directors actors genres ratingDist ratingVotes ratingRank keywords plot # sub getMovieIdDetails($$) { my $self=shift; my $id=shift; if ( !$self->{DBASE_FD} ) { die "internal error: index not open"; } my $results; my $FD=$self->{DBASE_FD}; Search::Dict::look(*{$FD}, "$id:", 0, 0); while (<$FD>) { last if ( !m/^$id:/ ); chomp(); if ( s/^$id:// ) { my ($directors, $actors, $genres, $ratingDist, $ratingVotes, $ratingRank, $keywords, $plot)=split('\t', $_); if ( $directors ne "<>" ) { for my $name (split('\|', $directors)) { # remove (I) etc from imdb.com names (kept in place for reference) $name=~s/\s\([IVXL]+\)$//o; # switch name around to be surname last $name=~s/^([^,]+),\s*(.*)$/$2 $1/o; push(@{$results->{directors}}, $name); } } if ( $actors ne "<>" ) { for my $name (split('\|', $actors)) { # remove (I) etc from imdb.com names (kept in place for reference) my $HostNarrator; if ( $name=~s/\s?\[([^\]]+)\]$//o ) { $HostNarrator=$1; } $name=~s/\s\([IVXL]+\)$//o; # switch name around to be surname last $name=~s/^([^,]+),\s*(.*)$/$2 $1/o; if ( $HostNarrator ) { if ( $HostNarrator=~s/,*Host//o ) { push(@{$results->{presenter}}, $name); } if ( $HostNarrator=~s/,*Narrator//o ) { push(@{$results->{commentator}}, $name); } } else { push(@{$results->{actors}}, $name); } } } if ( $genres ne "<>" ) { push(@{$results->{genres}}, split('\|', $genres)); } if ( $keywords ne "<>" ) { push(@{$results->{keywords}}, split(',', $keywords)); } $results->{ratingDist}=$ratingDist if ( $ratingDist ne "<>" ); $results->{ratingVotes}=$ratingVotes if ( $ratingVotes ne "<>" ); $results->{ratingRank}=$ratingRank if ( $ratingRank ne "<>" ); $results->{plot}=$plot if ( $plot ne "<>" ); } else { warn "lookup of movie (id=$id) resulted in garbage ($_)"; } } if ( !defined($results) ) { # some movies we don't have any details for $results->{noDetails}=1; } #print "MovieDetails($id) = ".Dumper($results)."\n"; return($results); } # # FUTURE - close hit could be just missing or extra # punctuation: # "Run Silent, Run Deep" for imdb's "Run Silent Run Deep" # "Cherry, Harry and Raquel" for imdb's "Cherry, Harry and Raquel!" # "Cat Women of the Moon" for imdb's "Cat-Women of the Moon" # "Baywatch Hawaiian Wedding" for imdb's "Baywatch: Hawaiian Wedding" :) # # FIXED - "Victoria and Albert" appears for imdb's "Victoria & Albert" (and -> &) # FIXED - "Columbo Cries Wolf" appears instead of "Columbo:Columbo Cries Wolf" # FIXED - Place the article last, for multiple languages. For instance # Los amantes del crculo polar -> amantes del crculo polar, Los # FIXED - common international vowel changes. For instance # "Anna Karnin" (->e) # sub alternativeTitles($) { my $title=shift; my @titles; push(@titles, $title); # try the & -> and conversion if ( $title=~m/\&/o ) { my $t=$title; while ( $t=~s/(\s)\&(\s)/$1and$2/o ) { push(@titles, $t); } } # try the and -> & conversion if ( $title=~m/\sand\s/io ) { my $t=$title; while ( $t=~s/(\s)and(\s)/$1\&$2/io ) { push(@titles, $t); } } # try the "Columbo: Columbo cries Wolf" -> "Columbo cries Wolf" conversion my $max=scalar(@titles); for (my $i=0; $i<$max ; $i++) { my $t=$titles[$i]; if ( $t=~m/^[^:]+:.+$/io ) { while ( $t=~s/^[^:]+:\s*(.+)\s*$/$1/io ) { push(@titles, $t); } } } # Place the articles last $max=scalar(@titles); for (my $i=0; $i<$max ; $i++) { my $t=$titles[$i]; if ( $t=~m/^(The|A|Une|Les|Los|Las|L\'|Le|La|El|Das|De|Het|Een)\s+(.*)$/io ) { $t=~s/^(The|A|Une|Les|Los|Las|L\'|Le|La|El|Das|De|Het|Een)\s+(.*)$/$2, $1/iog; push(@titles, $t); } if ( $t=~m/^(.+),\s*(The|A|Une|Les|Los|Las|L\'|Le|La|El|Das|De|Het|Een)$/io ) { $t=~s/^(.+),\s*(The|A|Une|Les|Los|Las|L\'|Le|La|El|Das|De|Het|Een)$/$2 $1/iog; push(@titles, $t); } } # convert all the special language characters $max=scalar(@titles); for (my $i=0; $i<$max ; $i++) { my $t=$titles[$i]; if ( $t=~m/[]/io ) { $t=~s/[]/a/gio; $t=~s/[]/e/gio; $t=~s/[]/i/gio; $t=~s/[]/o/gio; $t=~s/[]/u/gio; $t=~s/[]/ae/gio; $t=~s/[]/c/gio; $t=~s/[]/n/gio; $t=~s/[]/ss/gio; $t=~s/[]/y/gio; $t=~s/[]//gio; push(@titles, $t); } } # optional later possible titles include removing the '.' from titles # ie "Project V.I.P.E.R." matching imdb "Project VIPER" $max=scalar(@titles); for (my $i=0; $i<$max ; $i++) { my $t=$titles[$i]; if ( $t=~s/\.//go ) { push(@titles,$t); } } return(\@titles); } sub findMovieInfo($$$$) { my ($self, $title, $year, $exact)=@_; my @titles=@{alternativeTitles($title)}; if ( $exact == 1 ) { # try an exact match first :) for my $mytitle ( @titles ) { my ($info,$matchcount) = $self->getMovieExactMatch($mytitle, $year); if ($matchcount > 1) { # if multiple records exactly match title+year then we don't know which one is correct $self->status("multiple hits on movie \"$mytitle".($mytitle=~m/\s+\((19|20)\d\d\)/?'':" ($year)")."\""); return(undef, $matchcount); } if ( defined($info) ) { if ( $info->{qualifier} eq "movie" ) { $self->status("perfect hit on movie \"$info->{key}\""); $info->{matchLevel}="perfect"; return($info); } elsif ( $info->{qualifier} eq "tv_movie" ) { $self->status("perfect hit on made-for-tv-movie \"$info->{key}\""); $info->{matchLevel}="perfect"; return($info); } elsif ( $info->{qualifier} eq "video_movie" ) { $self->status("perfect hit on made-for-video-movie \"$info->{key}\""); $info->{matchLevel}="perfect"; return($info); } elsif ( $info->{qualifier} eq "video_game" ) { next; } elsif ( $info->{qualifier} eq "tv_series" ) { } elsif ( $info->{qualifier} eq "tv_mini_series" ) { } else { $self->error("$self->{moviedbIndex} responded with wierd entry for \"$info->{key}\""); $self->error("weird trailing qualifier \"$info->{qualifier}\""); $self->error("submit bug report to xmltv-devel\@lists.sf.net"); } } $self->debug("no exact title/year hit on \"$mytitle".($mytitle=~m/\s+\((19|20)\d\d\)/?'':" ($year)")."\""); } return(undef); } elsif ( $exact == 2 ) { # looking for first exact match on the title, don't have a year to compare for my $mytitle ( @titles ) { # try close hit if only one :) my $cnt=0; my @closeMatches=$self->getMovieCloseMatches("$mytitle"); # we traverse the hits twice, first looking for success, # then again to produce warnings about missed close matches for my $info (@closeMatches) { next if ( !defined($info) ); $cnt++; # within one year with exact match good enough if ( lc($mytitle) eq lc($info->{title}) ) { if ( $info->{qualifier} eq "movie" ) { $self->status("close enough hit on movie \"$info->{key}\" (since no 'date' field present)"); $info->{matchLevel}="close"; return($info); } elsif ( $info->{qualifier} eq "tv_movie" ) { $self->status("close enough hit on made-for-tv-movie \"$info->{key}\" (since no 'date' field present)"); $info->{matchLevel}="close"; return($info); } elsif ( $info->{qualifier} eq "video_movie" ) { $self->status("close enough hit on made-for-video-movie \"$info->{key}\" (since no 'date' field present)"); $info->{matchLevel}="close"; return($info); } elsif ( $info->{qualifier} eq "video_game" ) { next; } elsif ( $info->{qualifier} eq "tv_series" ) { } elsif ( $info->{qualifier} eq "tv_mini_series" ) { } else { $self->error("$self->{moviedbIndex} responded with wierd entry for \"$info->{key}\""); $self->error("weird trailing qualifier \"$info->{qualifier}\""); $self->error("submit bug report to xmltv-devel\@lists.sf.net"); } } } } # nothing worked return(undef); } # otherwise we're looking for a title match with a close year for my $mytitle ( @titles ) { # try close hit if only one :) my $cnt=0; my @closeMatches=$self->getMovieCloseMatches("$mytitle"); # we traverse the hits twice, first looking for success, # then again to produce warnings about missed close matches for my $info (@closeMatches) { next if ( !defined($info) ); $cnt++; # within one year with exact match good enough if ( lc($mytitle) eq lc($info->{title}) ) { my $yearsOff=abs(int($info->{year})-$year); $info->{matchLevel}="close"; if ( $yearsOff <= 2 ) { my $showYear=int($info->{year}); if ( $info->{qualifier} eq "movie" ) { $self->status("close enough hit on movie \"$info->{key}\" (off by $yearsOff years)"); return($info); } elsif ( $info->{qualifier} eq "tv_movie" ) { $self->status("close enough hit on made-for-tv-movie \"$info->{key}\" (off by $yearsOff years)"); return($info); } elsif ( $info->{qualifier} eq "video_movie" ) { $self->status("close enough hit on made-for-video-movie \"$info->{key}\" (off by $yearsOff years)"); return($info); } elsif ( $info->{qualifier} eq "video_game" ) { $self->status("ignoring close hit on video-game \"$info->{key}\""); next; } elsif ( $info->{qualifier} eq "tv_series" ) { $self->status("ignoring close hit on tv series \"$info->{key}\""); #$self->status("close enough hit on tv series \"$info->{key}\" (off by $yearsOff years)"); } elsif ( $info->{qualifier} eq "tv_mini_series" ) { $self->status("ignoring close hit on tv mini-series \"$info->{key}\""); #$self->status("close enough hit on tv mini-series \"$info->{key}\" (off by $yearsOff years)"); } else { $self->error("$self->{moviedbIndex} responded with wierd entry for \"$info->{key}\""); $self->error("weird trailing qualifier \"$info->{qualifier}\""); $self->error("submit bug report to xmltv-devel\@lists.sf.net"); } } } } # if we found at least something, but nothing matched # produce warnings about missed, but close matches for my $info (@closeMatches) { next if ( !defined($info) ); # within one year with exact match good enough if ( lc($mytitle) eq lc($info->{title}) ) { my $yearsOff=abs(int($info->{year})-$year); if ( $yearsOff <= 2 ) { #die "internal error: key \"$info->{key}\" failed to be processed properly"; } elsif ( $yearsOff <= 5 ) { # report these as status $self->status("ignoring close, but not good enough hit on \"$info->{key}\" (off by $yearsOff years)"); } else { # report these as debug messages $self->debug("ignoring close hit on \"$info->{key}\" (off by $yearsOff years)"); } } else { $self->debug("ignoring close hit on \"$info->{key}\" (title did not match)"); } } } #$self->status("failed to lookup \"$title ($year)\""); return(undef); } sub findTVSeriesInfo($$) { my ($self, $title)=@_; if ( $self->{cacheLookups} ) { my $id=$self->{cachedLookups}->{tv_series}->{$title}; if ( defined($id) ) { #print STDERR "REF= (".ref($id).")\n"; if ( $id ne '' ) { return($id); } return(undef); } } my @titles=@{alternativeTitles($title)}; # try an exact match first :) my $idInfo; for my $mytitle ( @titles ) { # try close hit if only one :) my $cnt=0; my @closeMatches=$self->getMovieCloseMatches("$mytitle"); for my $info (@closeMatches) { next if ( !defined($info) ); $cnt++; if ( lc($mytitle) eq lc($info->{title}) ) { $info->{matchLevel}="perfect"; if ( $info->{qualifier} eq "movie" ) { #$self->status("ignoring close hit on movie \"$info->{key}\""); } elsif ( $info->{qualifier} eq "tv_movie" ) { #$self->status("ignoring close hit on tv movie \"$info->{key}\""); } elsif ( $info->{qualifier} eq "video_movie" ) { #$self->status("ignoring close hit on made-for-video-movie \"$info->{key}\""); } elsif ( $info->{qualifier} eq "video_game" ) { #$self->status("ignoring close hit on made-for-video-movie \"$info->{key}\""); next; } elsif ( $info->{qualifier} eq "tv_series" ) { $idInfo=$info; $self->status("perfect hit on tv series \"$info->{key}\""); last; } elsif ( $info->{qualifier} eq "tv_mini_series" ) { $idInfo=$info; $self->status("perfect hit on tv mini-series \"$info->{key}\""); last; } else { $self->error("$self->{moviedbIndex} responded with wierd entry for \"$info->{key}\""); $self->error("weird trailing qualifier \"$info->{qualifier}\""); $self->error("submit bug report to xmltv-devel\@lists.sf.net"); } } } last if ( defined($idInfo) ); } if ( $self->{cacheLookups} ) { # flush cache after this lookup if its gotten too big if ( $self->{cachedLookups}->{tv_series}->{_cacheSize_} > $self->{cacheLookupSize} ) { delete($self->{cachedLookups}->{tv_series}); $self->{cachedLookups}->{tv_series}->{_cacheSize_}=0; } if ( defined($idInfo) ) { $self->{cachedLookups}->{tv_series}->{$title}=$idInfo; } else { $self->{cachedLookups}->{tv_series}->{$title}=""; } $self->{cachedLookups}->{tv_series}->{_cacheSize_}++; } if ( defined($idInfo) ) { return($idInfo); } else { #$self->status("failed to lookup tv series \"$title\""); return(undef); } } # # todo - add country of origin # todo - video (colour/aspect etc) details # todo - audio (stereo) details # todo - ratings ? - use certificates.list # todo - add description - plot summaries ? - which one do we choose ? # todo - writer # todo - producer # todo - running time (duration) # todo - identify 'Host' and 'Narrator's and put them in as # credits:presenter and credits:commentator resp. # todo - check program length - probably a warning if longer ? # can we update length (separate from runnning time in the output ?) # todo - icon - url from www.imdb.com of programme image ? # this could be done by scraping for the hyper linked poster # <a name="poster"><img src="http://ia.imdb.com/media/imdb/01/I/60/69/80m.jpg" height="139" width="99" border="0"></a> # and grabbin' out the img entry. (BTW ..../npa.jpg seems to line up with no poster available) # # sub applyFound($$$) { my ($self, $prog, $idInfo)=@_; my $title=$prog->{title}->[0]->[0]; if ( $self->{updateDates} ) { my $date; # don't add dates only fix them for tv_series if ( $idInfo->{qualifier} eq "movie" || $idInfo->{qualifier} eq "video_movie" || $idInfo->{qualifier} eq "tv_movie" ) { #$self->debug("adding 'date' field (\"$idInfo->{year}\") on \"$title\""); $date=int($idInfo->{year}); } else { #$self->debug("not adding 'date' field to $idInfo->{qualifier} \"$title\""); $date=undef; } if ( $self->{replaceDates} ) { if ( defined($prog->{date}) && defined($date) ) { $self->debug("replacing 'date' field"); delete($prog->{date}); $prog->{date}=$date; } } else { # only set date if not already defined if ( !defined($prog->{date}) && defined($date) ) { $prog->{date}=$date; } } } if ( $self->{updateTitles} ) { if ( $idInfo->{title} ne $title ) { if ( $self->{replaceTitles} ) { $self->debug("replacing (all) 'title' from \"$title\" to \"$idInfo->{title}\""); delete($prog->{title}); } my @list; push(@list, [$idInfo->{title}, undef]); if ( defined($prog->{title}) ) { my $name=$idInfo->{title}; my $found=0; for my $v (@{$prog->{title}}) { if ( lc($v->[0]) eq lc($name) ) { $found=1; } else { push(@list, $v); } } } $prog->{title}=\@list; } } if ( $self->{updateURLs} ) { if ( $self->{replaceURLs} ) { if ( defined($prog->{url}) ) { $self->debug("replacing (all) 'url'"); delete($prog->{url}); } } # add url to programme on www.imdb.com my $url=$idInfo->{key}; # {key} will include " marks if a tv series - remove these from search url [#148] $url = $1.$2 if ( $url=~m/^"(.*?)"(.*)$/ ); # encode the title $url=~s/([^a-zA-Z0-9_.-])/uc sprintf("%%%02x",ord($1))/oeg; $url="https://www.imdb.com/find?q=".$url."&s=tt&exact=true"; if ( defined($prog->{url}) ) { my @rep; push(@rep, $url); for (@{$prog->{url}}) { # skip urls for imdb.com that we're probably safe to replace if ( !m;^http://us.imdb.com/M/title-exact;o && !m;^https://www.imdb.com/find;o ) { push(@rep, $_); } } $prog->{url}=\@rep; } else { push(@{$prog->{url}}, $url); } } # squirrel away movie qualifier so its first on the list of replacements my @categories; push(@categories, [$self->{categories}->{$idInfo->{qualifier}}, 'en']); if ( !defined($self->{categories}->{$idInfo->{qualifier}}) ) { die "how did we get here with an invalid qualifier '$idInfo->{qualifier}'"; } my $details=$self->getMovieIdDetails($idInfo->{id}); if ( $details->{noDetails} ) { # we don't have any details on this movie } else { # add directors list if ( $self->{updateDirectors} && defined($details->{directors}) ) { # only update directors if we have exactly one or if # its a movie of some kind, add more than one. if ( scalar(@{$details->{directors}}) == 1 || $idInfo->{qualifier} eq "movie" || $idInfo->{qualifier} eq "video_movie" || $idInfo->{qualifier} eq "tv_movie" ) { if ( $self->{replaceDirectors} ) { if ( defined($prog->{credits}->{director}) ) { $self->debug("replacing director(s)"); delete($prog->{credits}->{director}); } } my @list; # add top 3 billing directors list form www.imdb.com for my $name (splice(@{$details->{directors}},0,3)) { push(@list, $name); } # preserve all existing directors listed if we did't already have them. if ( defined($prog->{credits}->{director}) ) { for my $name (@{$prog->{credits}->{director}}) { my $found=0; for(@list) { if ( lc eq lc($name) ) { $found=1; } } if ( !$found ) { push(@list, $name); } } } $prog->{credits}->{director}=\@list; } else { $self->debug("not adding 'director' field to $idInfo->{qualifier} \"$title\""); } } if ( $self->{updateActors} && defined($details->{actors}) ) { if ( $self->{replaceActors} ) { if ( defined($prog->{credits}->{actor}) ) { $self->debug("replacing actor(s) on $idInfo->{qualifier} \"$idInfo->{key}\""); delete($prog->{credits}->{actor}); } } my @list; # add top billing actors (default = 3) from www.imdb.com for my $name (splice(@{$details->{actors}},0,$self->{numActors})) { push(@list, $name); } # preserve all existing actors listed if we did't already have them. if ( defined($prog->{credits}->{actor}) ) { for my $name (@{$prog->{credits}->{actor}}) { my $found=0; for(@list) { if ( lc eq lc($name) ) { $found=1; } } if ( !$found ) { push(@list, $name); } } } $prog->{credits}->{actor}=\@list; } if ( $self->{updatePresentors} && defined($details->{presenter}) ) { if ( $self->{replacePresentors} ) { if ( defined($prog->{credits}->{presenter}) ) { $self->debug("replacing presentor"); delete($prog->{credits}->{presenter}); } } $prog->{credits}->{presenter}=$details->{presenter}; } if ( $self->{updateCommentators} && defined($details->{commentator}) ) { if ( $self->{replaceCommentators} ) { if ( defined($prog->{credits}->{commentator}) ) { $self->debug("replacing commentator"); delete($prog->{credits}->{commentator}); } } $prog->{credits}->{commentator}=$details->{commentator}; } # push genres as categories if ( $self->{updateCategoriesWithGenres} ) { if ( defined($details->{genres}) ) { for (@{$details->{genres}}) { push(@categories, [$_, 'en']); } } } if ( $self->{updateStarRatings} && defined($details->{ratingRank}) ) { if ( $self->{replaceStarRatings} ) { if ( defined($prog->{'star-rating'}) ) { $self->debug("replacing 'star-rating'"); delete($prog->{'star-rating'}); } unshift( @{$prog->{'star-rating'}}, [ $details->{ratingRank} . "/10", 'IMDB User Rating' ] ); } else { # add IMDB User Rating in front of all other star-ratings unshift( @{$prog->{'star-rating'}}, [ $details->{ratingRank} . "/10", 'IMDB User Rating' ] ); } } if ( $self->{updateKeywords} ) { my @keywords; if ( defined($details->{keywords}) ) { for (@{$details->{keywords}}) { push(@keywords, [$_, 'en']); } } if ( $self->{replaceKeywords} ) { if ( defined($prog->{keywords}) ) { $self->debug("replacing (all) 'keywords'"); delete($prog->{keywords}); } } if ( defined($prog->{keyword}) ) { for my $value (@{$prog->{keyword}}) { my $found=0; for my $k (@keywords) { if ( lc($k->[0]) eq lc($value->[0]) ) { $found=1; } } if ( !$found ) { push(@keywords, $value); } } } $prog->{keyword}=\@keywords; } if ( $self->{updatePlot} ) { # plot is held as a <desc> entity # if 'replacePlot' then delete all existing <desc> entities and add new # else add this plot as an additional <desc> entity # if ( $self->{replacePlot} ) { if ( defined($prog->{desc}) ) { $self->debug("replacing (all) 'desc'"); delete($prog->{desc}); } } if ( defined($details->{plot}) ) { # check it's not already there my $found = 0; for my $_desc ( @{$prog->{desc}} ) { $found = 1 if ( @{$_desc}[0] eq $details->{plot} ); } push @{$prog->{desc}}, [ $details->{plot}, 'en' ] if !$found; } } } if ( $self->{updateCategories} ) { if ( $self->{replaceCategories} ) { if ( defined($prog->{category}) ) { $self->debug("replacing (all) 'category'"); delete($prog->{category}); } } if ( defined($prog->{category}) ) { for my $value (@{$prog->{category}}) { my $found=0; #print "checking category $value->[0] with $mycategory\n"; for my $c (@categories) { if ( lc($c->[0]) eq lc($value->[0]) ) { $found=1; } } if ( !$found ) { push(@categories, $value); } } } $prog->{category}=\@categories; } return($prog); } sub augmentProgram($$$) { my ($self, $prog, $movies_only)=@_; $self->{stats}->{programCount}++; # assume first title in first language is the one we want. my $title=$prog->{title}->[0]->[0]; if ( defined($prog->{date}) && $prog->{date}=~m/^\d\d\d\d$/o ) { # for programs with dates we try: # - exact matches on movies # - exact matches on tv series # - close matches on movies my ($id, $matchcount) = $self->findMovieInfo($title, $prog->{date}, 1); # exact match if (defined $matchcount && $matchcount > 1) { $self->status("failed to find a sole match for movie \"$title".($title=~m/\s+\((19|20)\d\d\)/?'':" ($prog->{date})")."\""); return(undef); } if ( !defined($id) ) { $id=$self->findTVSeriesInfo($title); if ( !defined($id) ) { ($id, $matchcount) = $self->findMovieInfo($title, $prog->{date}, 0); # close match } } if ( defined($id) ) { $self->{stats}->{$id->{matchLevel}."Matches"}++; $self->{stats}->{$id->{matchLevel}}->{$id->{qualifier}}++; return($self->applyFound($prog, $id)); } $self->status("failed to find a match for movie \"$title".($title=~m/\s+\((19|20)\d\d\)/?'':" ($prog->{date})")."\""); return(undef); # fall through and try again as a tv series } if ( !$movies_only ) { my $id=$self->findTVSeriesInfo($title); if ( defined($id) ) { $self->{stats}->{$id->{matchLevel}."Matches"}++; $self->{stats}->{$id->{matchLevel}}->{$id->{qualifier}}++; return($self->applyFound($prog, $id)); } if ( 0 ) { # this has hard to support 'close' results, unless we know # for certain we're looking for a movie (ie duration etc) # this is a bad idea. my ($id, $matchcount) = $self->findMovieInfo($title, undef, 2); # any title match if ( defined($id) ) { $self->{stats}->{$id->{matchLevel}."Matches"}++; $self->{stats}->{$id->{matchLevel}}->{$id->{qualifier}}++; return($self->applyFound($prog, $id)); } } $self->status("failed to find a match for show \"$title\""); } return(undef); } # # todo - add in stats on other things added (urls ?, actors, directors,categories) # separate out from what was added or updated # sub getStatsLines($) { my $self=shift; my $totalChannelsParsed=shift; my $endTime=time(); my %stats=%{$self->{stats}}; my $ret=sprintf("Checked %d programs, on %d channels\n", $stats{programCount}, $totalChannelsParsed); for my $cat (sort keys %{$self->{categories}}) { $ret.=sprintf(" found %d %s titles", $stats{perfect}->{$cat}+$stats{close}->{$cat}, $self->{categories}->{$cat}); if ( $stats{close}->{$cat} != 0 ) { if ( $stats{close}->{$cat} == 1 ) { $ret.=sprintf(" (%d was not perfect)", $stats{close}->{$cat}); } else { $ret.=sprintf(" (%d were not perfect)", $stats{close}->{$cat}); } } $ret.="\n"; } $ret.=sprintf(" augmented %.2f%% of the programs, parsing %.2f programs/sec\n", ($stats{programCount}!=0)?(($stats{perfectMatches}+$stats{closeMatches})*100)/$stats{programCount}:0, ($endTime!=$stats{startTime} && $stats{programCount} != 0)? $stats{programCount}/($endTime-$stats{startTime}):0); return($ret); } 1; # ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ # ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ package XMLTV::IMDB::Crunch; use LWP; use XMLTV::Gunzip; use IO::File; # is system sort available? use constant HAS_SYSTEMSORT => ($^O=~'linux|cygwin|MSWin32'); # is File::Sort available? use constant HAS_FILESORT => defined eval { require File::Sort }; use open ':encoding(iso-8859-1)'; # try to enforce file encoding (does this work in Perl <5.8.1? ) # Use Term::ProgressBar if installed. use constant Have_bar => eval { require Term::ProgressBar; $Term::ProgressBar::VERSION >= 2; }; my $VERSION = '0.11'; # version number of database my %titlehash = (); # # This package parses and manages to index imdb plain text files from # ftp.imdb.com/interfaces. (see http://www.imdb.com/interfaces for # details) # # I might, given time build a download manager that: # - downloads the latest plain text files # - understands how to download each week's diffs and apply them # Currently, the 'downloadMissingFiles' flag in the hash of attributes # passed triggers a simple-minded downloader. # # I may also roll this project into a xmltv-free imdb-specific # perl interface that just supports callbacks and understands more of # the imdb file formats. # # [honir] 2020-12-27 An undocumented option --sample n will fetch only n records from each IMDb data file # Note the output will not be valid (since the n records will not cross-reference from the different files) # it's simply a way to avoid having to process all 4.5 million titles when you are debugging! sub new { my ($type) = shift; my $self={ @_ }; # remaining args become attributes for ($self->{downloadMissingFiles}) { $_=0 if not defined; # default } for ('imdbDir', 'verbose') { die "invalid usage - no $_" if ( !defined($self->{$_})); } $self->{stageLast} = 9; # set the final stage in the build - i.e. the one which builds the final database $self->{stages} = { 1=>'movies', 2=>'directors', 3=>'actors', 4=>'actresses', 5=>'genres', 6=>'ratings', 7=>'keywords', 8=>'plot' }; $self->{optionalStages} = { 'keywords' => 7, 'plot' => 8 }; # list of optional stages - no need to download files for these $self->{moviedbIndex}="$self->{imdbDir}/moviedb.idx"; $self->{moviedbData}="$self->{imdbDir}/moviedb.dat"; $self->{moviedbInfo}="$self->{imdbDir}/moviedb.info"; $self->{moviedbOffline}="$self->{imdbDir}/moviedb.offline"; # only leave progress bar on if its available if ( !Have_bar ) { $self->{showProgressBar}=0; } bless($self, $type); if ( $self->{filesort} && !( HAS_FILESORT || HAS_SYSTEMSORT ) ) { $self->error("filesort requested but not available"); return(undef); } $self->{usefilesort} = ( (HAS_FILESORT || HAS_SYSTEMSORT) && $self->{filesort} ); # --filesort => 1 --nofilesort => 0 $self->{usesystemsort} = ( HAS_SYSTEMSORT && $self->{filesort} && $self->{systemsort}); # use linux sort in preference to File::Sort as it is sooo much faster on big files if ( $self->{stageToRun} ne $self->{stageLast} ) { # unless this is the last stage, check we have the necessary files return(undef) if ( $self->checkFiles() != 0 ); } return($self); } sub checkFiles () { my ($self)=@_; if ( ! -d "$self->{imdbDir}" ) { if ( $self->{downloadMissingFiles} ) { warn "creating directory $self->{imdbDir}\n"; mkdir $self->{imdbDir}, 0777 or die "cannot mkdir $self->{imdbDir}: $!"; } else { die "$self->{imdbDir}:does not exist"; } } my $listsDir = "$self->{imdbDir}/lists"; if ( ! -d $listsDir ) { mkdir $listsDir, 0777 or die "cannot mkdir $listsDir: $!"; } CHECK_FILES: my %missingListFiles; # maps 'movies' to filename ...movies.gz FILES_CHECK: while ( my( $key, $value ) = each %{ $self->{stages} } ) { # don't check *all* files - only the ones we are crunching next FILES_CHECK if ( lc($self->{stageToRun}) ne 'all' && $key != int($self->{stageToRun}) ); my $file=$value; my $filename="$listsDir/$file.list"; my $filenameGz="$filename.gz"; my $filenameExists = -f $filename; my $filenameSize = -s $filename; my $filenameGzExists = -f $filenameGz; my $filenameGzSize = -s $filenameGz; if ( $filenameExists and not $filenameSize ) { warn "removing zero-length $filename\n"; unlink $filename or die "cannot unlink $filename: $!"; $filenameExists = 0; } if ( $filenameGzExists and not $filenameGzSize ) { warn "removing zero-length $filenameGz\n"; unlink $filenameGz or die "cannot unlink $filenameGz: $!"; $filenameGzExists = 0; } if ( not $filenameExists and not $filenameGzExists ) { # Just report one of the filenames, keep the message simple. warn "$filenameGz does not exist\n"; if ( $self->{optionalStages}{$file} && lc($self->{stageToRun}) eq 'all' ) { warn "$file will not be added to database\n"; } else { $missingListFiles{$file}=$filenameGz; } } elsif ( not $filenameExists and $filenameGzExists ) { $self->{imdbListFiles}->{$file}=$filenameGz; } elsif ( $filenameExists and not $filenameGzExists ) { $self->{imdbListFiles}->{$file}=$filename; } elsif ( $filenameExists and $filenameGzExists ) { die "both $filename and $filenameGz exist, remove one of them\n"; } else { die } } if ( $self->{downloadMissingFiles} ) { my $baseUrl = 'ftp://ftp.fu-berlin.de/pub/misc/movies/database/frozendata'; foreach ( sort keys %missingListFiles ) { my $url = "$baseUrl/$_.list.gz"; my $filename = delete $missingListFiles{$_}; my $partial = "$filename.partial"; if (-e $partial) { if (not -s $partial) { print STDERR "removing empty $partial\n"; unlink $partial or die "cannot unlink $partial: $!"; } else { die <<END $partial already exists, remove it or try renaming to $filename and resuming the download of <$url> by hand. END ; } } print STDERR <<END Trying to download <$url>. With a slow network link this could fail; it might be better to download the file by hand and save it as $filename. END ; # For downloading we use LWP # my $ua = LWP::UserAgent->new(); $ua->env_proxy(); $ua->show_progress(1); my $req = HTTP::Request->new(GET => $url); $req->authorization_basic('anonymous', 'tv_imdb'); my $resp = $ua->request($req, $filename); my $got_size = -s $filename; if (defined $resp and $resp->is_success ) { die if not $got_size; print STDERR "<$url>\n\t-> $filename, success\n\n"; } else { my $msg = "failed to download $url to $filename"; $msg .= ", http response code: ".$resp->status_line if defined $resp; warn $msg; if ($got_size) { warn "renaming $filename -> $partial\n"; rename $filename, $partial or die "cannot rename $filename to $partial: $!"; warn "You might try continuing the download of <$url> manually.\n"; } exit(1); } } $self->{downloadMissingFiles} = 0; goto CHECK_FILES; } if ( %missingListFiles ) { print STDERR "tv_imdb: requires you to download the above files from ftp.fu-berlin.de \n"; #print STDERR " see http://www.imdb.com/interfaces for details\n"; print STDERR " or try the --download option\n"; #return(undef); return 1; } return 0; } sub sortfile ($$$) { my ($self, $stage, $file)=@_; # file already written : sort it using (1) system sort command, or (2) File::Sort package my $f=$file; my $st = time; my $res; if ($self->{usesystemsort}) { # use shell sort if we can (much faster on big files) $self->status("using system sort on stage $stage"); # which OS are we on? if ($^O=~'linux|cygwin') { # TODO: untested on cygwin if ($stage == 1) { $res = system( "sort", "-t", "\t", qw(-k 1 -o), "$f.sorted", "$f" ); } else { $res = system( "sort", qw(-t : -k 1n -o), "$f.sorted", "$f" ); } if ($? == -1) { $self->error("failed to execute: $! \n"); } elsif ( $? & 127 || $? & 128 ) { $self->error("system call died with signal %d \n"); } else { $res = $? >> 8; } $res = 1 if $res == 0; # successful call returns 0 in $? } elsif ($^O=~'MSWin32') { # TODO: untested on Windows $res = system( "sort", "/O ", "$f.sorted", "$f"); $res = 1 if $res == 0; # successful call returns 0 in $? } } else { $self->status("using filesort on stage $stage (this might take up to 1 hour)"); if ($stage == 1) { $res = File::Sort::sort_file({ t =>"\t", k=>'1', y=>200000, I=>"$f", o=>"$f.sorted" }); } else { $res = File::Sort::sort_file({ t =>':', k=>'1n', y=>200000, I=>"$f", o=>"$f.sorted" }); } } $self->status("sorting took ".(int(((time - $st)/60)*10)/10)." minutes") if (time - $st > 60); if (!$res) { die "Filesort failed on $f"; } else { unlink($f); rename "$f.sorted", $f or die "Cannot rename file: $!"; } return($res); } sub redirect($$) { my ($self, $file)=@_; if ( defined($file) ) { if ( !open($self->{logfd}, "> $file") ) { print STDERR "$file:$!\n"; return(0); } $self->{errorCountInLog}=0; } else { close($self->{logfd}); $self->{logfd}=undef; } return(1); } sub error($$) { my $self=shift; if ( defined($self->{logfd}) ) { print {$self->{logfd}} $_[0]."\n"; $self->{errorCountInLog}++; } else { print STDERR $_[0]."\n"; } } sub status($$) { my $self=shift; if ( $self->{verbose} ) { print STDERR $_[0]."\n"; } } sub withThousands ($) { my ($val) = @_; $val =~ s/(\d{1,3}?)(?=(\d{3})+$)/$1,/g; return $val; } sub openMaybeGunzip($) { for ( shift ) { return gunzip_open($_) if m/\.gz$/; return new IO::File("< $_"); } } sub closeMaybeGunzip($$) { if ( $_[0]=~m/\.gz$/o ) { # Would close($fh) but that causes segfaults on my system. # Investigating, but in the meantime just leave it open. # #return gunzip_close($_[1]); } # Apparently this can also segfault (wtf?). #return close($_[1]); } sub beginProgressBar($$$) { my ($self, $what, $countEstimate)=@_; print STDERR $what.' '.$countEstimate; if ($self->{showProgressBar}) { $self->{progress} = Term::ProgressBar->new({name => "$what", count => $countEstimate*1.01, ETA => 'linear'}); $self->{progress}->minor(0) if ($self->{showProgressBar}); $self->{progress}->max_update_rate(1) if ($self->{showProgressBar}); $self->{count_estimate} = $countEstimate; $self->{next_update} = 0; } } sub updateProgressBar($$$) { my ($self, $what, $count)=@_; if ( $self->{showProgressBar} ) { # re-adjust target so progress bar doesn't seem too wonky if ( $count > $self->{count_estimate} ) { $self->{count_estimate} = $self->{progress}->target($count*1.05); $self->{next_update} = $self->{progress}->update($count); } elsif ( $count > $self->{next_update} ) { $self->{next_update} = $self->{progress}->update($count); } } } sub endProgressBar($$$) { my ($self, $what, $count)=@_; if ( $self->{showProgressBar} ) { $self->{progress}->update($self->{count_estimate}); } } sub makeTitleKey($$) { # make a unique key for each prog title. Also determine the prog type. # some edge cases we need to handle: # 1] multiple titles with same year, e.g. # '83 (2017/I) # '83 (2017/II) # # 2] multiple films with same year but different type, e.g. # Journey to the Center of the Earth (2008) # cinema release # Journey to the Center of the Earth (2008) (TV) # TV movie # Journey to the Center of the Earth (2008) (V) # straight to video # # 3] tv series and film with same year, e.g. # "Ashes to Ashes" (2008) # tv series # Ashes to Ashes (2008) # movie # # 4] titles without a year, e.g. # California Cornflakes (????) # Zed (????/II) # # 5] titles including alternatiove title, e.g. # Family Prayers (aka Karim & Suha) (2010) # my ($self, $progtitle)=@_; # tidy the film title, and extract the prog type # my $dbkey = $progtitle; my $progtype; # drop episode information - ex: "Supernatural" (2005) {A Very Supernatural Christmas (#3.8)} my $isepisode = $dbkey=~s/\s*\{[^\}]+\}//go; # remove 'aka' details from prog-title $dbkey =~ s/\s*\((?:aka|as) ([^\)]+)\)//o; # todo - this would make things easier # change double-quotes around title to be (made-for-tv) suffix instead if ( $dbkey=~m/^\"/o && #" $dbkey=~m/\"\s*\(/o ) { #" $dbkey.=" (tv_series)"; $progtype=4; } # how rude, some entries have (TV) appearing more than once. $dbkey=~s/\(TV\)\s*\(TV\)$/(TV)/o; my $qualifier; if ( $dbkey=~m/\s+\(TV\)$/ ) { # don't strip from title - it's considered part of the title: so we need it for matching against other source files $qualifier="tv_movie"; $progtype=2; } elsif ( $dbkey=~m/\s+\(V\)$/ ) { # ditto $qualifier="video_movie"; $progtype=3; } elsif ( $dbkey=~m/\s+\(VG\)$/ ) { # ditto $qualifier="video_game"; $progtype=5; } elsif ( $dbkey=~s/\s+\(mini\) \(tv_series\)$// ) { # but strip the rest $qualifier="tv_mini_series"; $progtype=4; } elsif ( $dbkey=~s/\s+\(tv_series\)$// ) { $qualifier="tv_series"; $progtype=4; } elsif ( $dbkey=~s/\s+\(mini\)$//o ) { $qualifier="tv_mini_series"; $progtype=4; } else { $qualifier="movie"; $progtype=1; } # make a key from the title # my $year; my $yearcount; my $title = $dbkey; if ( $title=~m/^\"/o && $title=~m/\"\s*\(/o ) { # remove " marks around title $title=~s/^\"//o; #" $title=~s/\"(\s*\()/$1/o; #" } # strip the above progtypes from the hashkey $title=~s/\s*\((TV|V|VG)\)$//; # extract the year from the title if ( $title=~s/\s+\((\d\d\d\d)\)$//o || $title=~s/\s+\((\d\d\d\d)\/([IVXL]+)\)$//o ) { $year=$1; } elsif ( $title=~s/\s+\((\?\?\?\?)\)$//o || $title=~s/\s+\((\?\?\?\?)\/([IVXL]+)\)$//o ) { $year="0000"; } else { $self->error("movie list format failed to decode year from title '$title'"); $year="0000"; } $title=~s/(.*),\s*(The|A|Une|Las|Les|Los|L\'|Le|La|El|Das|De|Het|Een)$/$2 $1/og; # move definite article to front of title $title=~s/\t/ /g; # remove tab chars (there shouldn't be any but it will corrupt our data output if we find one) my $hashkey=lc("$title ($year)"); # use calculated year to avoid things like "72 Hours (????/I)" $hashkey=~s/([^a-zA-Z0-9_.-])/uc sprintf("%%%02x",ord($1))/oeg; #print STDERR "input:$dbkey\n\tdbkey:$hashkey\n\ttitle=$title\n\tyear=$year\n\tcounter=$yearcount\n\tqualifier=$qualifier\n"; return ( $hashkey, $dbkey, $year, $yearcount, $qualifier, $progtype, $isepisode ); } sub readMovies($$$$$) { # build %movieshash from movies.list source file my ($self, $which, $countEstimate, $file, $stage)=@_; my $startTime=time(); my $header; my $whatAreWeParsing; my $lineCount=0; if ( $which eq "Movies" ) { $header="MOVIES LIST"; $whatAreWeParsing=1; } $self->beginProgressBar('parsing '.$which, $countEstimate); #----------------------------------------------------------- # find the start of the actual data my $fh = openMaybeGunzip($file) || return(-2); while(<$fh>) { chomp(); $lineCount++; if ( m/^$header/ ) { if ( !($_=<$fh>) || !m/^===========/o ) { $self->error("missing ======= after $header at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } if ( !($_=<$fh>) || !m/^\s*$/o ) { $self->error("missing empty line after ======= at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } last; } elsif ( $lineCount > 1000 ) { # didn't find the header within the first 1000 lines in the file! (wrong file? file corrupt? data changed?) $self->error("$file: stopping at line $lineCount, didn't see \"$header\" line"); closeMaybeGunzip($file, $fh); return(-1); } } #----------------------------------------------------------- # read the movies data, and create the db IDX file (as a temporary file called stage1.data) # input data are "film-name year" separated by one or more tabs # Army of Darkness (1992) 1992 my $count=0; my $countout=0; while(<$fh>) { chomp(); $lineCount++; my $line=$_; next if ( length($line) == 0 ); last if ( $self->{sample} != 0 && $self->{sample} < $count ); # undocumented option (used in debugging) #$self->status("read line $lineCount:$line"); # end of data is line consisting of only '-' last if ( $line =~ m/^\-\-\-\-\-\-\-+/o ); my $tabstop = index($line, "\t"); # there is always at least one tabstop in the incoming data if ( $tabstop != -1 ) { my ($mtitle, $myear) = $line =~ m/^(.*?)\t+(.*)$/; next if ($mtitle =~ m/\s*\{\{SUSPENDED\}\}/o); # returned count is number of titles found $count++; # compute the data we need for the IDX file # key title year title id # my ($hashkey, $title, $year, $yearcount, $qualifier, $progtype, $isepisode) = $self->makeTitleKey($mtitle); # we don't want "video games" if ($qualifier eq "video_game") { next; } # we don't keep episode information TODO: enhancement: change tv_imdb to do episodes? if ($isepisode == 1) { next; } next if ($self->{moviesonly} && ($progtype != 1 && $progtype != 2) ); # user requested movies_only # store the movies data if ($self->{usefilesort}) { # if sorting on disc then write the extracted movies data to an interim file print {$self->{fhdata}} $hashkey."\t".$title."\t".$year."\t".$qualifier."\n"; } else { # store the title in a hash of $key=>{$title} if ( defined($self->{movieshash}{$hashkey}) ) { # check for duplicates # # there's a lot (c. 9,000!) instances of duplicate titles in the movies.list file # so only report where titles are different if ( defined $self->{movieshash}{$hashkey}{$title} && $self->{movieshash}{$hashkey}{$title} ne $year."\t".$qualifier ) { # {."\t".$progtype} $self->error("duplicate moviedb key computed $hashkey - this programme will be ignored $mtitle"); #$self->error(" ".$self->{movieshash}{$hashkey}{$title}); next; } } # the output IDX and DAT files must be sorted by dbkey (because of the way the searching is done) # so we need to store all the incoming 4 million records and then sort them # $self->{movieshash}{$hashkey}{$title} = $year."\t".$qualifier; # we don't currently use the progtype flag so don't print it {."\t".$progtype} } # return number of titles kept $countout++; $self->updateProgressBar('', $lineCount); } else { $self->error("$file:$lineCount: unrecognized format (missing tab)"); $self->updateProgressBar('', $lineCount); } } $self->endProgressBar(); $self->status(sprintf("parsing $which found ".withThousands($countout)." titles in ". withThousands($lineCount)." lines in %d seconds",time()-$startTime)); closeMaybeGunzip($file, $fh); #----------------------------------------------------------- return($count, $countout); } sub readCastOrDirectors($$$$$) { my ($self, $which, $countEstimate, $file, $stage)=@_; my $startTime=time(); my $header; my $whatAreWeParsing; my $lineCount=0; if ( $which eq "Actors" ) { $header="THE ACTORS LIST"; $whatAreWeParsing=1; } elsif ( $which eq "Actresses" ) { $header="THE ACTRESSES LIST"; $whatAreWeParsing=2; } elsif ( $which eq "Directors" ) { $header="THE DIRECTORS LIST"; $whatAreWeParsing=3; } else { die "why are we here ?"; } $self->beginProgressBar('parsing '.$which, $countEstimate); # # note: not all movies end up with a cast, but we include these movies anyway. # #----------------------------------------------------------- # find the start of the actual data my $fh = openMaybeGunzip($file) || return(-2); while(<$fh>) { chomp(); $lineCount++; if ( m/^$header/ ) { if ( !($_=<$fh>) || !m/^===========/o ) { $self->error("missing ======= after $header at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } if ( !($_=<$fh>) || !m/^\s*$/o ) { $self->error("missing empty line after ======= at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } if ( !($_=<$fh>) || !m/^Name\s+Titles\s*$/o ) { $self->error("missing name/titles line after ======= at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } if ( !($_=<$fh>) || !m/^[\s\-]+$/o ) { $self->error("missing name/titles suffix line after ======= at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } last; } elsif ( $lineCount > 1000 ) { $self->error("$file: stopping at line $lineCount, didn't see \"$header\" line"); closeMaybeGunzip($file, $fh); return(-1); } } #----------------------------------------------------------- # read the cast or directors data, and create the stagex.data file # input data are "person-name film-title" separated by one or more tabs # Raimi,Sam Army of Darkness (1992) # person name appears only once for multiple film entries my $count=0; my $countnames=0; my $cur_name; while(<$fh>) { chomp(); $lineCount++; my $line=$_; next if ( length($line) == 0 ); last if ( $self->{sample} != 0 && $self->{sample} < $count ); # undocumented option (used in debugging) #$self->status("read line $lineCount:$line"); # end is line consisting of only '-' last if ( $line =~ m/^\-\-\-\-\-\-\-+/o ); my $tabstop = index($line, "\t"); # there is always at least one tabstop in the incoming data if ( $tabstop != -1 ) { my ($mname, $mtitle) = $line =~ m/^(.*?)\t+(.*)$/; # get person-name (everything up to the first tab) next if ($mtitle=~m/\s*\{\{SUSPENDED\}\}/o); # skip enties that have {} in them since they're tv episodes next if ($mtitle=~m/\s*\{[^\}]+\}$/ ); # skip "video games" next if ($mtitle=~m/\s+\(VG\)(\s|$)/ ); # note may not be end of line e.g. "Ahad, Alex (I) Skullgirls (2012) (VG) (creative director)" # returned count is number of directors found $count++; $mname =~ s/^\s+|\s+$//g; # trim # person name appears only on the first record in a group for this person if ($mname ne '') { $countnames++; $cur_name = $mname; } # Directors' processing # A. Guggenheim, Sonia After Maiko (2015) (as Sonia Guggenheim) # Journey (2015/III) (as Sonia Guggenheim) # A. Solla, Ricardo "7 vidas" (1999) {(#2.37)} # "7 vidas" (1999) {Atahualpa Yupanqui (#6.20)} # # Actors' processing # -Gradowska, Kasia Lewandowska Who are the WWP Women? (2015) (V) [Herself] <1> # 'Rovel' Torres, Crystal "The Tonight Show Starring Jimmy Fallon" (2014) {Ice T/Andrew Rannells/Lupe Fiasco (#2.105)} [Herself - Musical Support] # 's Gravemade, Nienke A Short Tour & Farewell (2015) # Tweeduizendseks (2010) (TV) [Yolanda van der Graaf] # Bennett, Mollie "Before the Snap" (2011) (voice) [Narrator] # 'Twinkie' Bird, Tracy "Casting Qs" (2010) {An Interview with Tracy 'Twinkie' Byrd (#2.14)} (as Twinkie Byrd) [Herself] # Abbott, Tasha (I) "Electives" (2018) [Julie] <41> # my $billing; my $hostnarrator; if ( $whatAreWeParsing < 3 ) { # actors or actresses # extract/strip the billing $billing="9999"; if ( $mtitle =~ s/\s*<(\d+)>//o ) { # e.g. <41> $billing = sprintf("%04d", int($1)); } # extract/strip the role/character if ( $mtitle =~ s/\s*\[(.*?)\]//o ) { # e.g. [Julie] or [Narrator] if ( $1 =~ m/(Host|Narrator)/ ) { # also picks up "Hostess", "Co-Host" $hostnarrator = $1; } } } #------------------------------------------------------- # tidy the title # remove the episode if a series if ( $mtitle =~ s/\s*\{[^\}]+\}//o ) { #redundant # $attr=$1; next; # skip tv episodes (we only output main titles so don't store episode data against the main title) } # remove 'aka' details from prog-title if ( $mtitle =~ s/\s*\((?:aka|as) ([^\)]+)\)//o ) { # $attr=$1; } # remove prog type (e.g. "(V)" or "(TV)" ) # no: don't strip from title - it's considered part of the title: so we need it for matching against movies.list ##if ( $mtitle =~ s/\s(\((TV|V|VG)\))//o ) { # $attrs=$1; ##} # junk everything after " (" (e.g. " (collaborating director)" ) if ( $mtitle =~ s/ (\(.*)$//o ) { # $attrs=$1; } $mtitle =~ s/^\s+|\s+$//g; # trim #------------------------------------------------------- # $mtitle should now contain the programme's title my $title = $mtitle; # find the IDX id from the hash of titles ($title=>$lineno) created in stage 1 my $idxid = $self->{titleshash}{$title}; if (!$idxid ) { ## no, don't print errors where we can't match the incoming title - there are 100s of these in the incoming data ## often where the year on the actor record is 1 year out ## people will get worried if we report over 1000 errors and there's nothing we can sensibly do about them ##$self->error("$file:$lineCount: cannot find $title in titles list"); ### if we reinstate this test then we'd need to allow for 'moviesonly' option (i.e. a lot of titles will have been deliberately excluded) next; } #------------------------------------------------------- # the output ".data" files must be sorted by id so they can be merged in stage final # so we need to store all the incoming records and then sort them # my $mperson = ''; $mperson = "$billing:" if ( defined($billing) ); $mperson .= $cur_name; $mperson .= " [$hostnarrator]" if ( defined($hostnarrator) ); # this is wrong: incoming data are "lastname, firstname" so this creates "Huwyler, Fabio [Host]" if ($self->{usefilesort}) { # write the extracted imdb data to a temporary file, preceeded by the IDX id for each record my $k = sprintf("%07d", $idxid); print {$self->{fhdata}} $k.':'.$mperson."\n"; } else { my $h = "stage${stage}hash"; if (defined( $self->{$h}{$idxid} )) { $self->{$h}{$idxid} .= "|".$mperson; } else { $self->{$h}{$idxid} = $mperson; } } $self->updateProgressBar('', $lineCount); } else { $self->error("$file:$lineCount: unrecognized format (missing tab)"); $self->updateProgressBar('', $lineCount); } } $self->endProgressBar(); $self->status(sprintf("parsing $which found ".withThousands($countnames)." names, ". withThousands($count)." titles in ".withThousands($lineCount)." lines in %d seconds",time()-$startTime)); closeMaybeGunzip($file, $fh); #----------------------------------------------------------- return($count); } sub readGenres($$$$$) { my ($self, $which, $countEstimate, $file, $stage)=@_; my $startTime=time(); my $header; my $whatAreWeParsing; my $lineCount=0; if ( $which eq "Genres" ) { $header="8: THE GENRES LIST"; $whatAreWeParsing=1; } $self->beginProgressBar('parsing '.$which, $countEstimate); #----------------------------------------------------------- # find the start of the actual data my $fh = openMaybeGunzip($file) || return(-2); while(<$fh>) { chomp(); $lineCount++; if ( m/^$header/ ) { if ( !($_=<$fh>) || !m/^===========/o ) { $self->error("missing ======= after $header at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } if ( !($_=<$fh>) || !m/^\s*$/o ) { $self->error("missing empty line after ======= at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } last; } elsif ( $lineCount > 1000 ) { $self->error("$file: stopping at line $lineCount, didn't see \"$header\" line"); closeMaybeGunzip($file, $fh); return(-1); } } #----------------------------------------------------------- # read the genres data, and create the stagex.data file # input data are "film-title genre" separated by one or more tabs # multiple genres are searated by | # Army of Darkness (1992) Horror # King Jeff (2009) Comedy|Short my $count=0; while(<$fh>) { chomp(); $lineCount++; my $line=$_; next if ( length($line) == 0 ); last if ( $self->{sample} != 0 && $self->{sample} < $lineCount ); # undocumented option (used in debugging) #$self->status("read line $lineCount:$line"); # end is line consisting of only '-' last if ( $line=~m/^\-\-\-\-\-\-\-+/o ); my $tabstop = index($line, "\t"); # there is always at least one tabstop in the incoming data if ( $tabstop != -1 ) { my ($mtitle, $mgenres) = $line =~ m/^(.*?)\t+(.*)$/; # get film-title (everything up to the first tab) next if ($mtitle=~m/\s*\{\{SUSPENDED\}\}/o); # skip enties that have {} in them since they're tv episodes next if ($mtitle=~m/\s*\{[^\}]+\}/ ); # skip "video games" next if ($mtitle=~m/\s+\(VG\)$/ ); # returned count is number of titles found $count++; if ( $whatAreWeParsing == 1 ) { # genres # genres sometimes contains tabs $mgenres=~s/^\t+//og; } #------------------------------------------------------- # tidy the title # remove the episode if a series if ( $mtitle =~ s/\s*\{[^\}]+\}//o ) { #redundant # $attr=$1; } # remove 'aka' details from prog-title if ( $mtitle =~ s/\s*\((?:aka|as) ([^\)]+)\)//o ) { # $attr=$1; } $mtitle =~ s/^\s+|\s+$//g; # trim #------------------------------------------------------- # $mtitle should now contain the programme's title my $title = $mtitle; # find the IDX id from the hash of titles ($title=>$lineno) created in stage 1 my $idxid = $self->{titleshash}{$title}; if (!$idxid ) { ## no, don't print errors where we can't match the incoming title - there are 100s of these in the incoming data ## often where the year on the actor record is 1 year out ##$self->error("$file:$lineCount: cannot find $title in titles list"); next; } #------------------------------------------------------- # the output ".data" files must be sorted by id so they can be merged in stage final # so we need to store all the incoming records and then sort them # if ($self->{usefilesort}) { # write the extracted imdb data to a temporary file, preceeded by the IDX id for each record my $k = sprintf("%07d", $idxid); print {$self->{fhdata}} $k.':'.$mgenres."\n"; } else { my $h = "stage${stage}hash"; if (defined( $self->{$h}{$idxid} )) { $self->{$h}{$idxid} .= "|".$mgenres; } else { $self->{$h}{$idxid} = $mgenres; } } $self->updateProgressBar('', $lineCount); } else { $self->error("$file:$lineCount: unrecognized format (missing tab)"); $self->updateProgressBar('', $lineCount); } } $self->endProgressBar(); $self->status(sprintf("parsing $which found ".withThousands($count)." titles in ". withThousands($lineCount)." lines in %d seconds",time()-$startTime)); closeMaybeGunzip($file, $fh); #----------------------------------------------------------- return($count); } sub readRatings($$$$$) { my ($self, $which, $countEstimate, $file, $stage)=@_; my $startTime=time(); my $header; my $whatAreWeParsing; my $lineCount=0; if ( $which eq "Ratings" ) { $header="MOVIE RATINGS REPORT"; $whatAreWeParsing=1; } $self->beginProgressBar('parsing '.$which, $countEstimate); #----------------------------------------------------------- # find the start of the actual data my $fh = openMaybeGunzip($file) || return(-2); while(<$fh>) { chomp(); $lineCount++; if ( m/^$header/ ) { # there is no ====== in ratings data! if ( !($_=<$fh>) || !m/^\s*$/o ) { $self->error("missing empty line after $header at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } if ( !($_=<$fh>) || !m/^New Distribution Votes Rank Title/o ) { $self->error("missing \"New Distribution Votes Rank Title\" at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } last; } elsif ( $lineCount > 1000 ) { $self->error("$file: stopping at line $lineCount, didn't see \"$header\" line"); closeMaybeGunzip($file, $fh); return(-1); } } #----------------------------------------------------------- # read the ratings data, and create the stagex.data file # input data are "flag-new disribution votes rank film-title" separated by one or more spaces # 0000002211 000001 9.9 Army of Darkness (1992) # 0000000133 225568 8.9 12 Angry Men (1957) my $count=0; while(<$fh>) { chomp(); $lineCount++; my $line=$_; next if ( length($line) == 0 ); last if ( $self->{sample} != 0 && $self->{sample} < $lineCount ); # undocumented option (used in debugging) #$self->status("read line $lineCount:$line"); # skip empty lines (only really appear right before last line ending with ---- next if ( $line=~m/^\s*$/o ); # end is line consisting of only '-' last if ( $line=~m/^\-\-\-\-\-\-\-+/o ); my $tabstop = index($line, " "); # there is always at least one space in the incoming data if ( $tabstop != -1 ) { my ($mdistrib, $mvotes, $mrank, $mtitle) = $line =~ m/^\s+([\.|\*|\d]+)\s+(\d+)\s+(\d+\.\d+)\s+(.*)$/; next if ($mtitle=~m/\s*\{\{SUSPENDED\}\}/o); next if ($mtitle=~m/\s*\{[^\}]+\}/ ); # skip tv episodes next if ($mtitle=~m/\s+\(VG\)$/ ); # we don't want "video games" # returned count is number of titles found $count++; if ( $whatAreWeParsing == 1 ) { # ratings # null } #------------------------------------------------------- # tidy the title # remove the episode if a series if ( $mtitle =~ s/\s*\{[^\}]+\}//o ) { #redundant # $attr=$1; } # remove 'aka' details from prog-title if ( $mtitle =~ s/\s*\((?:aka|as) ([^\)]+)\)//o ) { # $attr=$1; } $mtitle =~ s/^\s+|\s+$//g; # trim #------------------------------------------------------- # $mtitle should now contain the programme's title my $title = $mtitle; # find the IDX id from the hash of titles ($title=>$lineno) created in stage 1 my $idxid = $self->{titleshash}{$title}; if (!$idxid ) { ## no, don't print errors where we can't match the incoming title - there are 100s of these in the incoming data ## often where the year on the actor record is 1 year out ##$self->error("$file:$lineCount: cannot find $title in titles list"); next; } #------------------------------------------------------- # the output ".data" files must be sorted by id so they can be merged in stage final # so we need to store all the incoming records and then sort them # if ($self->{usefilesort}) { # write the extracted imdb data to a temporary file, preceeded by the IDX id for each record my $k = sprintf("%07d", $idxid); print {$self->{fhdata}} $k.':'."$mdistrib;$mvotes;$mrank"."\n"; } else { my $h = "stage${stage}hash"; if (defined( $self->{$h}{$idxid} )) { # we shouldn't get duplicates $self->error("$file: duplicate film found at line $lineCount - this rating will be ignored $mtitle"); } else { $self->{$h}{$idxid} = "$mdistrib;$mvotes;$mrank"; } } $self->updateProgressBar('', $lineCount); } else { $self->error("$file:$lineCount: unrecognized format (missing tab)"); $self->updateProgressBar('', $lineCount); } } $self->endProgressBar(); $self->status(sprintf("parsing $which found ".withThousands($count)." titles in ". withThousands($lineCount)." lines in %d seconds",time()-$startTime)); closeMaybeGunzip($file, $fh); #----------------------------------------------------------- return($count); } sub readKeywords($$$$$) { my ($self, $which, $countEstimate, $file, $stage)=@_; my $startTime=time(); my $header; my $whatAreWeParsing; my $lineCount=0; if ( $which eq "Keywords" ) { $header="8: THE KEYWORDS LIST"; $whatAreWeParsing=1; } $self->beginProgressBar('parsing '.$which, $countEstimate); #----------------------------------------------------------- # find the start of the actual data my $fh = openMaybeGunzip($file) || return(-2); while(<$fh>) { chomp(); $lineCount++; if ( m/^$header/ ) { if ( !($_=<$fh>) || !m/^===========/o ) { $self->error("missing ======= after $header at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } if ( !($_=<$fh>) || !m/^\s*$/o ) { $self->error("missing empty line after ======= at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } last; } elsif ( $lineCount > 150000 ) { # line 101935 as at 2020-12-23 $self->error("$file: stopping at line $lineCount, didn't see \"$header\" line"); closeMaybeGunzip($file, $fh); return(-1); } } #----------------------------------------------------------- # read the keywords data, and create the stagex.data file # input data are "film-title keyword" separated by one or more tabs # multiple keywords are searated by | # Army of Darkness (1992) Horror # King Jeff (2009) Comedy|Short my $count=0; while(<$fh>) { chomp(); $lineCount++; my $line=$_; next if ( length($line) == 0 ); last if ( $self->{sample} != 0 && $self->{sample} < $lineCount ); # undocumented option (used in debugging) #$self->status("read line $lineCount:$line"); # end is line consisting of only '-' last if ( $line=~m/^\-\-\-\-\-\-\-+/o ); my $tabstop = index($line, "\t"); # there is always at least one tabstop in the incoming data if ( $tabstop != -1 ) { my ($mtitle, $mkeywords) = $line =~ m/^(.*?)\t+(.*)$/; # get film-title (everything up to the first tab) next if ($mtitle=~m/\s*\{\{SUSPENDED\}\}/o); next if ($mtitle=~m/\s*\{[^\}]+\}/ ); # skip tv episodes next if ($mtitle=~m/\s+\(VG\)$/ ); # we don't want "video games" # returned count is number of titles found $count++; if ( $whatAreWeParsing == 1 ) { # genres # ignore anything which is an episode (e.g. "{Doctor Who (#10.22)}" ) next if $mtitle =~ m/^.*\s+(\{.*\})$/; } #------------------------------------------------------- # tidy the title # remove the episode if a series # [honir] this is wrong - this puts all the keywords as though they are in the entire series! if ( $mtitle =~ s/\s*\{[^\}]+\}//o ) { #redundant # $attr=$1; } # remove 'aka' details from prog-title if ( $mtitle =~ s/\s*\((?:aka|as) ([^\)]+)\)//o ) { # $attr=$1; } $mtitle =~ s/^\s+|\s+$//g; # trim #------------------------------------------------------- # $mtitle should now contain the programme's title my $title = $mtitle; # find the IDX id from the hash of titles ($title=>$lineno) created in stage 1 my $idxid = $self->{titleshash}{$title}; if (!$idxid ) { ## no, don't print errors where we can't match the incoming title - there are 100s of these in the incoming data ## often where the year on the actor record is 1 year out ##$self->error("$file:$lineCount: cannot find $title in titles list"); next; } #------------------------------------------------------- # the output ".data" files must be sorted by id so they can be merged in stage final # so we need to store all the incoming records and then sort them # if ($self->{usefilesort}) { # write the extracted imdb data to a temporary file, preceeded by the IDX id for each record my $k = sprintf("%07d", $idxid); print {$self->{fhdata}} $k.':'.$mkeywords."\n"; } else { my $h = "stage${stage}hash"; if (defined( $self->{$h}{$idxid} )) { $self->{$h}{$idxid} .= "|".$mkeywords; } else { $self->{$h}{$idxid} = $mkeywords; } } $self->updateProgressBar('', $lineCount); } else { $self->error("$file:$lineCount: unrecognized format (missing tab)"); $self->updateProgressBar('', $lineCount); } } $self->endProgressBar(); $self->status(sprintf("parsing $which found ".withThousands($count)." titles in ". withThousands($lineCount)." lines in %d seconds",time()-$startTime)); closeMaybeGunzip($file, $fh); #----------------------------------------------------------- return($count); } sub readPlots($$$$$) { my ($self, $which, $countEstimate, $file, $stage)=@_; my $startTime=time(); my $header; my $whatAreWeParsing; my $lineCount=0; if ( $which eq "Plot" ) { $header="PLOT SUMMARIES LIST"; $whatAreWeParsing=1; } $self->beginProgressBar('parsing '.$which, $countEstimate); #----------------------------------------------------------- # find the start of the actual data my $fh = openMaybeGunzip($file) || return(-2); while(<$fh>) { chomp(); $lineCount++; if ( m/^$header/ ) { if ( !($_=<$fh>) || !m/^===========/o ) { $self->error("missing ======= after $header at line $lineCount"); closeMaybeGunzip($file, $fh); return(-1); } # no blank line in plot data! ##if ( !($_=<$fh>) || !m/^\s*$/o ) { ## $self->error("missing empty line after ======= at line $lineCount"); ## closeMaybeGunzip($file, $fh); ## return(-1); ##} last; } elsif ( $lineCount > 1000 ) { $self->error("$file: stopping at line $lineCount, didn't see \"$header\" line"); closeMaybeGunzip($file, $fh); return(-1); } } #----------------------------------------------------------- # read the plot data, and create the stagex.data file # input data are "flag-new disribution votes rank film-title" separated by one or more spaces # there can be multiple entries for each film # ------------------------------------------------------------------------------- # MV: Army of Darkness (1992) # # PL: Ash is transported with his car to 1,300 A.D., where he is captured by Lord # PL: Arthur and turned slave with Duke Henry the Red and a couple of his men. # [...] # PL: battle between Ash's 20th Century tactics and the minions of darkness. # # BY: David Thiel <d-thiel@uiuc.edu> # # PL: Ash finds himself stranded in the year 1300 AD with his car, his shotgun, # PL: and his chainsaw. Soon he is discovered and thought to be a spy for a rival # [...] # PL: forces at play in the land. Ash accidentally releases the Army of Darkness # PL: when retrieving the book, and a fight to the finish ensues. # # BY: Ed Sutton <esutton@mindspring.com> my $count=0; while(<$fh>) { chomp(); $lineCount++; my $line=$_; next if ( length($line) == 0 ); last if ( $self->{sample} != 0 && $self->{sample} < $lineCount ); # undocumented option (used in debugging) #$self->status("read line $lineCount:$line"); # skip empty lines next if ( $line=~m/^\s*$/o ); next if ( $line=~m/\s*\{[^\}]+\}/ ); # skip tv episodes next if ( $line=~m/\s+\(VG\)$/ ); # skip "video games" # process a data block - starts with "MV:" # my ($mtitle, $mepisode) = ($line =~ m/^MV:\s(.*?)\s?(\{.*\})?$/); if ( defined($mtitle) ) { my $mplot = ''; # ignore anything which is an episode (e.g. "{Doctor Who (#10.22)}" ) if ( !defined $mepisode || $mepisode eq '' ) { LOOP: while (1) { if ( $line = <$fh> ) { $lineCount++; chomp($line); next if ($line =~ m/^\s*$/); if ( $line =~ m/PL:\s(.*)$/ ) { # plot summary is a number of lines starting "PL:" $mplot .= ($mplot ne ''?' ':'') . $1; } last LOOP if ( $line =~ m/BY:\s(.*)$/ ); # the author line "BY:" signals the end of the plot summary } else { last LOOP; } } # ensure there's no tab chars in the plot or else the db stage will barf $mplot =~ s/\t//og; # returned count is number of unique titles found $count++; } #------------------------------------------------------- # tidy the title # remove the episode if a series if ( $mtitle =~ s/\s*\{[^\}]+\}//o ) { #redundant # $attr=$1; } # remove 'aka' details from prog-title if ( $mtitle =~ s/\s*\((?:aka|as) ([^\)]+)\)//o ) { # $attr=$1; } $mtitle =~ s/^\s+|\s+$//g; # trim #------------------------------------------------------- # $mtitle should now contain the programme's title my $title = $mtitle; # find the IDX id from the hash of titles ($title=>$lineno) created in stage 1 my $idxid = $self->{titleshash}{$title}; if (!$idxid ) { ## no, don't print errors where we can't match the incoming title - there are 100s of these in the incoming data ## often where the year on the actor record is 1 year out ##$self->error("$file:$lineCount: cannot find $title in titles list"); next; } #------------------------------------------------------- # the output ".data" files must be sorted by id so they can be merged in stage final # so we need to store all the incoming records and then sort them # if ($self->{usefilesort}) { # write the extracted imdb data to a temporary file, preceeded by the IDX id for each record my $k = sprintf("%07d", $idxid); print {$self->{fhdata}} $k.':'.$mplot."\n"; } else { my $h = "stage${stage}hash"; if (defined( $self->{$h}{$idxid} )) { # we shouldn't get duplicates $self->error("$file: duplicate film found at line $lineCount - this plot will be ignored $mtitle"); } else { $self->{$h}{$idxid} = $mplot; } } $self->updateProgressBar('', $lineCount); } else { # skip lines up to the next "MV:" (this means we only get the first plot summary for each film) if ($line !~ m/^(---|PL:|BY:)/ ) { $self->error("$file:$lineCount: unrecognized format \"$line\""); } $self->updateProgressBar('', $lineCount); } } $self->endProgressBar(); $self->status(sprintf("parsing $which found ".withThousands($count)." in ". withThousands($lineCount)." lines in %d seconds",time()-$startTime)); closeMaybeGunzip($file, $fh); #----------------------------------------------------------- return($count); } sub stageComplete($) { my ($self, $stage)=@_; if ( -f "$self->{imdbDir}/stage$stage.data" ) { return(1); } return(0); } sub dbinfoLoad($) { my $self=shift; my $ret=XMLTV::IMDB::loadDBInfo($self->{moviedbInfo}); if ( ref $ret eq 'SCALAR' ) { return($ret); } $self->{dbinfo}=$ret; return(undef); } sub dbinfoAdd($$$) { my ($self, $key, $value)=@_; $self->{dbinfo}->{$key}=$value; } sub dbinfoGet($$$) { my ($self, $key, $defaultValue)=@_; if ( defined($self->{dbinfo}->{$key}) ) { return($self->{dbinfo}->{$key}); } return($defaultValue); } sub dbinfoSave($) { my $self=shift; open(INFO, "> $self->{moviedbInfo}") || return(1); for (sort keys %{$self->{dbinfo}}) { print INFO "".$_.":".$self->{dbinfo}->{$_}."\n"; } close(INFO); return(0); } sub dbinfoGetFileSize($$) { my ($self, $key)=@_; if ( !defined($self->{imdbListFiles}->{$key}) ) { die ("invalid call"); } my $fileSize=int(-s "$self->{imdbListFiles}->{$key}"); # if compressed, then attempt to run gzip -l if ( $self->{imdbListFiles}->{$key}=~m/.gz$/) { if ( open(my $fd, "gzip -l ".$self->{imdbListFiles}->{$key}."|") ) { # if parse fails, then defalt to wild ass guess of compression of 65% $fileSize=int(($fileSize*100)/(100-65)); while(<$fd>) { if ( m/^\s*\d+\s+(\d+)/ ) { $fileSize=$1; } } close($fd); } else { # wild ass guess of compression of 65% $fileSize=int(($fileSize*100)/(100-65)); } } return($fileSize); } sub dbinfoCalcEstimate($$$) { my ($self, $key, $estimateSizePerEntry)=@_; my $fileSize=$self->dbinfoGetFileSize($key); my $countEstimate=int($fileSize/$estimateSizePerEntry); $self->dbinfoAdd($key."_list_file", $self->{imdbListFiles}->{$key}); $self->dbinfoAdd($key."_list_file_size", int(-s "$self->{imdbListFiles}->{$key}")); $self->dbinfoAdd($key."_list_file_size_uncompressed", $fileSize); $self->dbinfoAdd($key."_list_count_estimate", $countEstimate); return($countEstimate); } sub dbinfoCalcBytesPerEntry($$$) { my ($self, $key, $calcActualForThisNumber)=@_; my $fileSize=$self->dbinfoGetFileSize($key); return(int($fileSize/$calcActualForThisNumber)); } sub gettitleshash($$) { # load the titles list (stage1.data) into memory my ($self, $countEstimate)=@_; my $startTime=time(); my $lineCount=0; undef $self->{titleshash}; $self->beginProgressBar('loading titles list', $countEstimate); open(IN, "< $self->{imdbDir}/stage1.data") || die "$self->{imdbDir}/stage1.data:$!"; my $count=0; my $maxidxid=0; while(<IN>) { chomp(); my $line=$_; next if ( length($line) == 0 ); #$self->status("read line $lineCount:$line"); $lineCount++; # check the database version number if ($lineCount == 1) { if ( m/^0000000:version ([\d\.]*)$/ ) { if ($1 ne $VERSION) { $self->error("incorrect database version"); return(1); } else { next; } } else { $self->error("missing database version at line $lineCount"); return(1); } } if (index($line, ":") != -1 ) { $count++; # extract the title-idx-id and the film-title # 0000002:army%20of%20darkness%20%281992%29 Army of Darkness (1992) 1992 movie 0000002 # my ($midxid, $mhashkey, $mtitle) = $line =~ m/^(\d*):(.*?)\t+(.*?)\t/; if ($midxid && $mtitle) { $self->{titleshash}{$mtitle} = int($midxid); # build the hash $maxidxid = $midxid if ( $midxid > $maxidxid ); } $self->updateProgressBar('', $lineCount); } else { $self->error("$lineCount: unrecognized format (missing tab)"); $self->updateProgressBar('', $lineCount); } } $self->endProgressBar(); $self->status(sprintf("found ".withThousands($count)." titles in ". withThousands($lineCount-1)." lines in %d seconds",time()-$startTime)); # drop 1 for the "version" line close(IN); #----------------------------------------------------------- return($count, $maxidxid); } sub dedupe($$$) { # basic deduping of data entries my ($self, $data, $sep)=@_; my @outarr; my @arr = split( ($sep eq '|' ? '\|' : $sep) , $$data); my %out; foreach my $v (@arr) { my ($a, $b) = $v =~ m/^(\d*):?(.*)\s*$/; if (!defined $out{$b}) { push @outarr, $v; $out{$b} = $v; } } $$data = join($sep, @outarr); return; } sub stripbilling($$$) { # strip the billing from the names # also strip the "(I)" etc suffix from names my ($self, $data, $sep)=@_; my @outarr; my @arr = split( ($sep eq '|' ? '\|' : $sep) , $$data); foreach my $v (@arr) { my ($a, $b) = $v =~ m/^(\d*):?(.*)\s*$/; $b=~s/\s\([IVXL]+\)\[/\[/o; $b=~s/\s\([IVXL]+\)$//o; push @outarr, $b; } $$data = join($sep, @outarr); return; } sub sortnames($$$) { # basic sorting of names my ($self, $data, $sep)=@_; my @arr = split( ($sep eq '|' ? '\|' : $sep) , $$data); $$data = join($sep, sort(@arr) ); return; } sub stripprogtype($$) { # strip the (TV) or (V) or (VG) suffix from title my ($self, $data)=@_; my ($midx, $mtitle, $mrest) = $$data =~ m/^(.*?)\t(.*?)\t(.*)$/; $mtitle =~ s/\s(\((TV|V|VG)\))//; $$data = $midx ."\t". $mtitle ."\t". $mrest; return; } sub readfilesbyidxid($$$$) { # read lines from the data files 2..8 looking for matches on a passed idxid # (don't use this for stage1 data - use a call to readdatafile to simply get the next record my ($self, $fhs, $fdat, $idxid)=@_; while (my ($stage, $fh) = each ( %$fhs )) { $fdat->{$stage} = { k=>0, v=>'' } if !defined $fdat->{$stage}{k}; if ($fdat->{$stage}{k} < $idxid) { #print STDERR "fetching from $stage ".$fdat->{$stage}{k}." < $idxid \n"; my ($fstage, $fidxid, $fdata) = $self->readdatafile( $fhs->{$stage}, $stage, $idxid, -1); if ($self->{usefilesort}) { # if we are using filesort then there will be multiple records with the same idxid # we need to fetch all of these and combine them my $_fidxid = $fidxid; while ( $_fidxid == $fidxid && $_fidxid != 9999999 ) { # read next record (my $_fstage, $_fidxid, my $_fdata) = $self->readdatafile( $fhs->{$stage}, $stage, $idxid, $_fidxid ); if ($_fidxid == $fidxid) { $fdata .= '|' . $_fdata; } } # need to dedupe our merged data ($fstage, $fidxid, $fdata) = $self->tidydatafile( $fstage, $fidxid, $fdata ); } # store the file record $fdat->{$stage} = { k=>$fidxid, v=>$fdata }; } } # here's a fudge: we need to merge the actors (stage 3) and actresses (stage 4) together my @pnames; push ( @pnames, $fdat->{3}{v} ) if ( $fdat->{3}{k} == $idxid ); push ( @pnames, $fdat->{4}{v} ) if ( $fdat->{4}{k} == $idxid ); if (scalar @pnames) { # join the two data values, sort, strip... my $pnames = join('|', @pnames); $self->sortnames(\$pnames, '|'); # sorts by "billing:name" $self->stripbilling(\$pnames, '|'); # strip "billing:" and "(I)" on name ### ...and then store in one of the actors/actresses value while nulling the other if ( $fdat->{3}{k} == $idxid ) { $fdat->{3}{v} = $pnames; $fdat->{4}{v} = ':::' if ( $fdat->{4}{k} == $idxid ); } elsif ( $fdat->{4}{k} == $idxid ) { $fdat->{4}{v} = $pnames; $fdat->{3}{v} = ':::' if ( $fdat->{3}{k} == $idxid ); } } # end fudge return; } sub readdatafile($$$$$) { my ($self, $fh, $stage, $idxid, $lidxid)=@_; # read a line from a file my $line; # if we have a parked record then use that one if ( defined $self->{datafile}{$stage} ) { $line = $self->{datafile}{$stage}; undef $self->{datafile}{$stage}; } else { if ( eof($fh) ) { return ($stage, 9999999, ''); } defined( $line = readline $fh ) or die "readline failed on file for stage $stage : $!"; } # extract the idxid from the start of each line # 0000002:army%20of%20darkness%20%281992%29 Army of Darkness (1992) 1992 movie 0000002 my ($midxid, $mdata) = $line =~ m/^(\d*):(.*)$/; if ($midxid) { # there should not be any records in datafile n which are not in datafile 1 if ( $midxid < $idxid ) { $self->error("unexpected record in stage $stage data file at $midxid (expected $idxid)"); } else { # processing on the data for each interim file ($stage, $midxid, $mdata) = $self->tidydatafile( $stage, $midxid, $mdata ); } # if the incoming idxid has changed then park the record if ( $lidxid != -1 && $midxid != $lidxid ) { $self->{datafile}{$stage} = $line; } } return ($stage, $midxid, $mdata); } sub tidydatafile($$$$) { my ($self, $stage, $midxid, $mdata)=@_; # tidy/reformat the data from a stagex.data file if ($midxid) { # processing on the data for each interim file # movies #1 : strip the (TV) (V) markers from the movie title # directors #2 : (i) dedupe (ii) sort into name order (not correct but there's no sequencing in the imdb data) # actors/actresses #3,#4 : (i) dedeupe (ii) sort into billing order (iii) strip billing id Note: need to merge actors and actresses # genres #5 : (i) dedupe # ratings #6 : (i) split elements and separate by tabs # keywords #7 : (i) dedupe, (ii) replace separator with comma # plots #8 : # if ($stage == 1) { $self->stripprogtype(\$mdata); } elsif ($stage == 2) { $self->dedupe(\$mdata, '|'); $self->stripbilling(\$mdata, '|'); $self->sortnames(\$mdata, '|'); # sorts by "lastname, firstname" } elsif ($stage == 3 || $stage == 4) { $self->dedupe(\$mdata, '|'); # defer sorting and strip billing deferred until after we have joined actors + actresses ## $self->sortnames(\$mdata, '|'); # sorts by "billing:name" ## $self->stripbilling(\$mdata, '|'); } elsif ($stage == 5) { $self->dedupe(\$mdata, '|'); } elsif ($stage == 6) { $mdata =~ s/;/\t/g; # replace ";" separator with tabs } elsif ($stage == 7) { $self->dedupe(\$mdata, '|'); $mdata =~ s/\|/,/g; } elsif ($stage == 8) { # noop } } return ($stage, $midxid, $mdata); } sub invokeStage($$) { my ($self, $stage)=@_; my $startTime=time(); #---------------------------------------------------------------------------- if ( $stage == 1 ) { $self->status("parsing Movies list for stage $stage ..."); my $countEstimate=$self->dbinfoCalcEstimate("movies", 45); # if we are using --filesort then write output file direct (and not use a hash) if ($self->{usefilesort}) { open($self->{fhdata}, ">", "$self->{imdbDir}/stage$stage.data.tmp") || die "$self->{imdbDir}/stage$stage.data.tmp:$!"; } my ($num, $numout) = $self->readMovies("Movies", $countEstimate, "$self->{imdbListFiles}->{movies}", $stage); if ($self->{usefilesort}) { close($self->{fhdata}); } if ( $num < 0 ) { if ( $num == -2 ) { $self->error("you need to download $self->{imdbListFiles}->{movies} from the ftp site, or use the --download option"); } return(1); } elsif ( abs($num - $countEstimate) > $countEstimate*.10 ) { my $better=$self->dbinfoCalcBytesPerEntry("movies", $num); ##not accurate: $self->status("ARG estimate of $countEstimate for movies needs updating, found $num ($better bytes/entry)"); } $self->dbinfoAdd("db_stat_movie_count", "$numout"); #use Data::Dumper;print STDERR Dumper($self->{movieshash}); #use Data::Dumper;my $_h="stage${stage}hash";print STDERR Dumper( $self->{$_h} ); #----------------------------------------------------------- # sort the title keys and write the stage1.data file # # if we are using --filesort then write output file direct (and not use a hash) if ($self->{usefilesort}) { $self->beginProgressBar("writing stage $stage data", $self->dbinfoGet("db_stat_movie_count", 0) ); # movies are in an interim file (stage1.data.tmp). # We need to (1) sort the file, # (2) translate to stage1.data (adding the idxid) # (3) store in %titleshash my $res; # (1) sort the file in situ $res = $self->sortfile($stage, "$self->{imdbDir}/stage$stage.data.tmp"); # if (!$res) { do something? } # (2) & (3) read the sorted file and create out stage1.data while building titleshash hash undef $self->{titleshash}; open(IN, "< $self->{imdbDir}/stage$stage.data.tmp") || die "$self->{imdbDir}/stage$stage.data.tmp:$!"; open(OUT, "> $self->{imdbDir}/stage$stage.data") || die "$self->{imdbDir}/stage$stage.data:$!"; print OUT '0000000:version '.$VERSION."\n"; my $count=0; while(<IN>) { my $line=$_; $count++; my $idxid=sprintf("%07d", $count); my ($k, $k2, $v2) = $line =~ m/^(.*?)\t(.*?)\t(.*?)$/; # the following equates to # print OUT $idxid.":".$dbkey."\t".$title."\t".$year."\t".$qualifier."\t".$lineno."\n"; print OUT $idxid.':'.$k."\t".$k2."\t".$v2."\t".$idxid."\n"; # and create a shared hash of $title=>$lineno (i.e. IDX 'id') $self->{titleshash}{$k2} = $count; # store the idx id for this title $self->updateProgressBar('', $count); } $self->endProgressBar(); $self->{maxid} = $count; # remember the largest values of title id (for loop stop) close(OUT); close(IN); unlink "$self->{imdbDir}/stage$stage.data.tmp"; } else { # movies data are in a hash (%movieshash) to we need to write that to disc (stage1.data) $self->beginProgressBar("writing stage $stage data", $num); open(OUT, "> $self->{imdbDir}/stage$stage.data") || die "$self->{imdbDir}/stage$stage.data:$!"; print OUT '0000000:version '.$VERSION."\n"; my $count=0; foreach my $k (sort keys( %{$self->{movieshash}} )) { while ( my ($k2, $v2) = each %{$self->{movieshash}{$k}} ) { # movieshash is a hash of hashes $count++; my $idxid=sprintf("%07d", $count); # the following equates to # print OUT $idxid.":".$dbkey."\t".$title."\t".$year."\t".$qualifier."\t".$lineno."\n"; print OUT $idxid.':'.$k."\t".$k2."\t".$v2."\t".$idxid."\n"; # and create a shared hash of $title=>$lineno (i.e. IDX 'id') $self->{titleshash}{$k2} = $count; # store the int version of the id for this title # (note multiple titles may have the same hashkey) } delete( $self->{movieshash}{$k} ); $self->updateProgressBar('', $count); } $self->endProgressBar(); $self->{maxid} = $count; # remember the largest values of title id (for loop stop) close(OUT); } #use Data::Dumper;print STDERR Dumper( $self->{titleshash} );die; } #---------------------------------------------------------------------------- elsif ( $stage >= 2 && $stage < $self->{stageLast} ) { # these stages need the hash of film-title=>idxid # if we have come from stage 1 (i.e. "prep-stage=all" then we will have that from stage=1 # otherwise we will need to build *.e.g "prep-stage=2" # if (!defined( $self->{titleshash} ) ) { my $countEstimate = $self->dbinfoGet("db_stat_movie_count", 0); my ($titlecount, $maxid) = $self->gettitleshash($countEstimate); if ($titlecount == -1) { $self->error('could not make title list - quitting'); return(1); } $self->{maxid} = $maxid; # remember the largest values of title id (for loop stop) #use Data::Dumper;print STDERR Dumper( $self->{titleshash} ); } # nb: {stages} = { 1=>'movies', 2=>'directors', 3=>'actors', 4=>'actresses', 5=>'genres', 6=>'ratings', 7=>'keywords', 8=>'plot' }; my $stagename = $self->{stages}{$stage}; my $stagenametext = ucfirst $self->{stages}{$stage}; $self->status("parsing $stagenametext list for stage $stage ..."); # skip optional stages if ( ( !defined $self->{imdbListFiles}->{$stagename} ) && ( defined $self->{optionalStages}->{$stagename} ) ) { return(0); } # approx average record length for each incoming data file (used to guesstimate number of records in file) my %countestimates = ( 1=>'45', 2=> '40', 3=> '55', 4=> '55', 5=> '35', 6=> '65', 7=> '20', 8=> '50' ); my $countEstimate = $self->dbinfoCalcEstimate($stagename, $countestimates{$stage}); my %stagefunctions = ( 1=>\&readMovies, 2=>\&readCastOrDirectors, 3=>\&readCastOrDirectors, 4=>\&readCastOrDirectors, 5=>\&readGenres, 6=>\&readRatings, 7=>\&readKeywords, 8=>\&readPlots ); # if we are using --filesort then write output file direct (and not use a hash) if ($self->{usefilesort}) { open($self->{fhdata}, ">", "$self->{imdbDir}/stage$stage.data") || die "$self->{imdbDir}/stage$stage.data:$!"; print {$self->{fhdata}} '0000000:version '.$VERSION."\n"; } my $num=$stagefunctions{$stage}->($self, $stagenametext, $countEstimate, "$self->{imdbListFiles}->{$stagename}", $stage); if ($self->{usefilesort}) { close($self->{fhdata}); } if ( $num < 0 ) { if ( $num == -2 ) { $self->error("you need to download $self->{imdbListFiles}->{$stagename} from the ftp site, or use the --download option"); } return(1); } elsif ( $num > 0 && abs($num - $countEstimate) > $countEstimate*.10 ) { my $better=$self->dbinfoCalcBytesPerEntry($stagename, $num); $self->status("ARG estimate of $countEstimate for $stagename needs updating, found $num ($better bytes/entry)"); } $self->dbinfoAdd("db_stat_${stagename}_count", "$num"); #----------------------------------------------------------- # print the title keys in IDX id order : write the stagex.data file # if ($self->{usefilesort}) { # file already written : just needs sorting (in situ) my $f="$self->{imdbDir}/stage$stage.data"; my $res = $self->sortfile($stage, $f); # todo: check the reply? } else { #use Data::Dumper;my $_h="stage${stage}hash";print STDERR Dumper( $self->{$_h} ); # write the stage.data file from the memory hash $self->beginProgressBar("writing stage $stage data", $num); open(OUT, "> $self->{imdbDir}/stage$stage.data") || die "$self->{imdbDir}/stage$stage.data:$!"; print OUT '0000000:version '.$VERSION."\n"; # don't sort the hash keys - that will just cost memory. Just pull them out in numerical order. my $h = "stage${stage}hash"; # # read the stage data hash in idxid order for (my $i = 0; $i <= $self->{maxid}; $i++){ # write the extracted imdb data to a temporary file, preceeded by the IDX id for each record my $k = sprintf("%07d", $i); if ( $self->{$h}{$i} ) { my $v = $self->{$h}{$i}; delete ( $self->{$h}{$i} ); # print OUT $k.':'.$v."\n"; } $self->updateProgressBar('', $i); } $self->endProgressBar(); close(OUT); #use Data::Dumper;print STDERR "leftovers: $stage ".Dumper( $self->{$h} )."\n"; delete ( $self->{$h} ); } #use Data::Dumper;print STDERR Dumper( $self->{titleshash} ); } #---------------------------------------------------------------------------- elsif ( $stage == $self->{stageLast} ) { # delete existing IDX; trim stage1.data to IDX; merge stage 2-8.data into DAT # free up some memory undef $self->{titleshash}; my $tab=sprintf("\t"); $self->status("indexing all previous stage's data for stage ".$self->{stageLast}."..."); #---------------------------------------------------------------------- # read all the parsed data files created in stages 1-8 and merges them # read one record at a time from each file! my $countEstimate=$self->dbinfoGet("db_stat_movie_count", 0); $self->beginProgressBar('writing database', $countEstimate); open(IDX, "> $self->{moviedbIndex}") || die "$self->{moviedbIndex}:$!"; open(DAT, "> $self->{moviedbData}") || die "$self->{moviedbData}:$!"; my $i; my %fh; for $i (1..($self->{stageLast}-1)) { # skip optional files if they don't exist if ( ($i == 7 && !( -f "$self->{imdbDir}/stage7.data" )) || ($i == 8 && !( -f "$self->{imdbDir}/stage8.data" )) ) { next; } # open($fh{$i}, "< $self->{imdbDir}/stage$i.data") || die "$self->{imdbDir}/stage$i.data:$!"; } # check the file version numbers while (my ($k, $v) = each (%fh)) { $_ = readline $v; if ( m/^0000000:version ([\d\.]*)$/ ) { if ($1 ne $VERSION) { $self->error("incorrect database version in stage $k file"); return(1); } else { next; } } else { $self->error("missing database version in stage $k file"); return(1); } } #---------------------------------------------------------------------- my %fdat; my $count=0; my $go=1; while ($go) { last if ( eof($fh{1}) ); # I suppose we ought to check if there any recs remaining in the other files (todo) # read a movie record my ($fstage, $fidxid, $fdata) = $self->readdatafile($fh{1}, 1, -1, -1); $fdat{$fstage} = { k=>$fidxid, v=>$fdata }; if ($fidxid) { $count++; # get matching records from other data files $self->readfilesbyidxid(\%fh, \%fdat, $fidxid); # merge data from other records my $mdata = $fidxid.':'; for $i (2..($self->{stageLast}-1)) { # we can join actors and actresses - only 1 of them will have data now next if ( $fdat{$i}{k} == $fidxid && $fdat{$i}{v} eq ':::' ); # only output either actors or actresses but not both (otherwise we'll get an extra marker in the output next if ($i == 3) && ( $fdat{3}{k} != $fidxid ); next if ($i == 4) && ( $fdat{4}{k} != $fidxid ) && ( $fdat{3}{k} == $fidxid ); # don't output marker if we've just done it for actors # drop through if actresses (#4) and no actors (#3) for this film if ( $fdat{$i}{k} == $fidxid ) { $mdata .= $fdat{$i}{v}; } else { # don't data for this stage ($i) so just print the 'empty' marker $mdata .= '<>'; if ($i == 6) { $mdata .= "\t".'<>'."\t".'<>'; } # fudge to add extra spacers in ratings data } $mdata .= "\t" unless $i == ($self->{stageLast}-1); } #print STDERR "mdata ".$mdata."\n"; # write the DAT record print DAT $mdata ."\n"; # write the IDX record print IDX $fdata ."\n"; } $self->updateProgressBar('', $count); } $self->endProgressBar(); $self->status(sprintf("wrote ".withThousands($count)." titles in %d seconds",time()-$startTime)); close(IDX); close(IN); while (my ($k, $v) = each (%fh)) { close($v); } # --------------------------------------------------------------------------------------- $self->dbinfoAdd("db_version", $XMLTV::IMDB::VERSION); if ( $self->dbinfoSave() ) { $self->error("$self->{moviedbInfo}:$!"); return(1); } $self->status("running quick sanity check on database indexes..."); my $imdb=new XMLTV::IMDB('imdbDir' => $self->{imdbDir}, 'verbose' => $self->{verbose}); if ( -e "$self->{moviedbOffline}" ) { unlink("$self->{moviedbOffline}"); } if ( my $errline=$imdb->sanityCheckDatabase() ) { open(OFF, "> $self->{moviedbOffline}") || die "$self->{moviedbOffline}:$!"; print OFF $errline."\n"; print OFF "one of the prep stages' must have produced corrupt data\n"; print OFF "report the following details to xmltv-devel\@lists.sf.net\n"; my $info=XMLTV::IMDB::loadDBInfo($self->{moviedbInfo}); if ( ref $info eq 'SCALAR' ) { print OFF "\tdbinfo file corrupt\n"; print OFF "\t$info"; } else { for my $key (sort keys %{$info}) { print OFF "\t$key:$info->{$key}\n"; } } print OFF "database taken offline\n"; close(OFF); open(OFF, "< $self->{moviedbOffline}") || die "$self->{moviedbOffline}:$!"; while(<OFF>) { chop(); $self->error($_); } close(OFF); return(1); } $self->status("sanity intact :)"); } else { $self->error("tv_imdb: invalid stage $stage: only 1-".$self->{stageLast}." are valid"); return(1); } $self->dbinfoAdd("seconds_to_complete_prep_stage_$stage", (time()-$startTime)); if ( $self->dbinfoSave() ) { $self->error("$self->{moviedbInfo}:$!"); return(1); } return(0); } sub crunchStage($$) { my ($self, $stage)=@_; if ( $stage == $self->{stageLast} ) { # check all the pre-requisite stages have been run for (my $st=1 ; $st < $self->{stageLast}; $st++ ) { if ( !$self->stageComplete($st) ) { #$self->error("prep stages must be run in sequence.."); $self->error("prepStage $st either has never been run or failed"); if ( grep { $_ == $st } values %{$self->{optionalStages}} ) { $self->error("data for this stage will NOT be added"); ####### todo: unless flag present } else { $self->error("rerun tv_imdb with --prepStage=$st"); return(1); } } } } if ( -f "$self->{moviedbInfo}" && $stage != 1 ) { my $ret=$self->dbinfoLoad(); if ( $ret ) { $self->error($ret); return(1); } } # open stage logfile and run the requested stage $self->redirect("$self->{imdbDir}/stage$stage.log") || return(1); my $ret=$self->invokeStage($stage); $self->redirect(undef); if ( $ret == 0 ) { if ( $self->{errorCountInLog} == 0 ) { $self->status("prep stage $stage succeeded with no errors"); } else { $self->status("prep stage $stage succeeded with $self->{errorCountInLog} errors in $self->{imdbDir}/stage$stage.log"); if ( $stage == $self->{stageLast} && $self->{errorCountInLog} > 30 && $self->{errorCountInLog} < 80 ) { $self->status("this stage commonly produces around 60 (or so) warnings because of imdb"); $self->status("list file inconsistancies, they can usually be safely ignored"); } } } else { if ( $self->{errorCountInLog} == 0 ) { $self->status("prep stage $stage failed (with no logged errors)"); } else { $self->status("prep stage $stage failed with $self->{errorCountInLog} errors in $self->{imdbDir}/stage$stage.log"); } } return($ret); } 1; ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Options.pm��������������������������������������������������������������������������0000664�0000000�0000000�00000036100�15000742332�0015421�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������package XMLTV::Options; use strict; use warnings; # use version number for feature detection: # 0.005065 : added --info / --man option (prints POD and then exits) our $VERSION = 0.005065; BEGIN { use Exporter (); our (@ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS); @ISA = qw(Exporter); @EXPORT = qw( ); %EXPORT_TAGS = ( ); # eg: TAG => [ qw!name1 name2! ], @EXPORT_OK = qw/ParseOptions/; } our @EXPORT_OK; =head1 NAME XMLTV::Options - Command-line parsing for XMLTV grabbers =head1 DESCRIPTION Utility library that implements command-line parsing and handles a lot of functionality that is common to all XMLTV grabbers. =head1 EXPORTED FUNCTIONS All these functions are exported on demand. =cut use XMLTV; use XMLTV::Configure qw/LoadConfig Configure SelectChannelsStage/; use Getopt::Long; use Carp qw/croak/; my %cap_options = ( all => [qw/ help|h version capabilities description info man /], baseline => [qw/ days=i offset=i quiet output=s debug config-file=s /], manualconfig => [qw/configure/], apiconfig => [qw/ configure-api stage=s list-channels /], tkconfig => [qw/gui=s/], # The cache option is normally handled by XMLTV::Memoize # but in case it is not used, we handle it here as well. cache => [qw/ cache:s /], share => [qw/ share=s /], preferredmethod => [qw/ preferredmethod /], lineups => [qw/ get-lineup list-lineups /], ); my %cap_defaults = ( all => { capabilities => 0, help => 0, version => 0, info => 0, man => 0, }, baseline => { quiet => 0, days => 5, offset => 0, output => undef, debug => 0, gui => undef, }, manualconfig => { configure => 0, }, apiconfig => { 'configure-api' => 0, stage => 'start', 'list-channels' => 0, }, tkconfig => { }, cache => { cache => undef, }, share => { share => undef, }, preferredmethod => { preferredmethod => 0, }, lineups => { 'get-lineup' => 0, 'list-lineups' => 0, } ); =head1 USAGE =over =item B<ParseOptions> ParseOptions shall be called by a grabber to parse the command-line options supplied by the user. It takes a single hashref as a parameter. The entries in the hash configure the behaviour of ParseOptions. my( $opt, $conf ) = ParseOptions( { grabber_name => 'tv_grab_test', version => '1.2', description => 'Sweden (tv.swedb.se)', capabilities => [qw/baseline manualconfig apiconfig lineups/], stage_sub => \&config_stage, listchannels_sub => \&list_channels, list_lineups_sub => \&list_lineups, get_lineup_sub => \&get_lineup, } ); ParseOptions returns two hashrefs: =over =item * A hashref with the values for all command-line options in the format returned by Getopt::Long (See "Storing options in a hash" in L<Getopt::Long>). This includes both options that the grabber must handle as well as options that ParseOptions handles for the grabber. =item * A hashref to the data loaded from the configuration file. See L<XMLTV::Configure> for the format of $conf. =back ParseOptions handles the following options automatically without returning: =over =item --help =item --info (or --man) =item --capabilities =item --version =item --description =item --preferredmethod Handled automatically if the preferredmethod capability has been set and the preferredmethod option has been specified in the call to ParseOptions. =back ParseOptions also takes care of the following options without returning, by calling the stage_sub, listchannels_sub, list_lineups_sub and get_lineup_sub callbacks supplied by the grabber: =over =item --configure =item --configure-api =item --stage =item --list-channels =item --list-lineups =item --get-lineup =back ParseOptions will thus only return to the grabber when the grabber shall actually grab data. If the --output option is specified, STDOUT will be redirected to the specified file. The grabber must check the following options on its own: =over =item --days =item --offset =item --quiet =item --debug =back and any other options that are grabber specific. This can be done by reading $opt->{days} etc. =item B<Changing the behaviour of ParseOptions> The behaviour of ParseOptions can be influenced by passing named arguments in the hashref. The following arguments are supported: =over =item grabber_name Required. The name of the grabber (e.g. tv_grab_se_swedb). This is used when printing the synopsis. =item description Required. The description for the grabber. This is returned in response to the --description option and shall say which region the grabber returns data for. Examples: "Sweden", or "Sweden (tv.swedb.se)" if there are several grabbers for a region or country). =item version Required. The version number of the grabber to be displayed. Supported version string formats include "x", "x.y", and "x.y.z". =item capabilities Required. The capabilities that the grabber shall support. Only capabilities that XMLTV::Options knows how to handle can be specified. Example: capabilities => [qw/baseline manualconfig apiconfig/], Note that XMLTV::Options guarantees that the grabber supports the manualconfig and apiconfig capabilities. The capabilities share and cache can be specified if the grabber supports them. XMLTV::Options will then automatically accept the command-line parameters --share and --cache respectively. =item stage_sub Required. A coderef that takes a stage-name and a configuration hashref as a parameter and returns an xml-string that describes the configuration necessary for that stage. The xml-string shall follow the xmltv-configuration.dtd. =item listchannels_sub Required. A coderef that takes a configuration hash as returned by XMLTV::Configure::LoadConfig as the first parameter and an option hash as returned by ParseOptions as the second parameter, and returns an xml-string containing a list of all the channels that the grabber can deliver data for using the supplied configuration. Note that the listsub shall not use any channel-configuration from the hashref. =item load_old_config_sub Optional. Default undef. A coderef that takes a filename as a parameter and returns a configuration hash in the same format as returned by XMLTV::Configure::LoadConfig. load_old_config_sub is called if XMLTV::Configure::LoadConfig fails to parse the configuration file. This allows the grabber to load configuration files created with an older version of the grabber. =item list_lineups_sub Optional. A coderef that takes an option hash as returned by ParseOptions as a parameter, and returns an xml-string containing a list of all the channel lineups for which the grabber can deliver data. The xml-string shall follow the xmltv-lineups.xsd schema. =item get_lineup_sub Optional. A coderef that returns an xml-string describing the configured lineup. The xml-string shall follow the xmltv-lineups.xsd schema. =item preferredmethod Optional. A value to return when the grabber is called with the --preferredmethod parameter. Example: my( $opt, $conf ) = ParseOptions( { grabber_name => 'tv_grab_test', version => '1.2', description => 'Sweden (tv.swedb.se)', capabilities => [qw/baseline manualconfig apiconfig preferredmethod/], stage_sub => \&config_stage, listchannels_sub => \&list_channels, preferredmethod => 'allatonce', list_lineups_sub => \&list_lineups, get_lineup_sub => \&get_lineup, } ); =item defaults Optional. Default {}. A hashref that contains default values for the command-line options. It shall be in the same format as returned by Getopt::Long (See "Storing options in a hash" in L<Getopt::Long>). =item extra_options Optional. Default []. An arrayref containing option definitions in the format accepted by Getopt::Long. This can be used to support grabber-specific options. The use of grabber-specific options is discouraged. =back =back =cut sub ParseOptions { my( $p ) = @_; my @optdef=(); my $opt={}; my $capabilities = {}; foreach my $cap (keys %{cap_options}) { $capabilities->{$cap} = 0; } if( not defined( $p->{version} ) ) { croak "No version specified in call to ParseOptions"; } if( not defined( $p->{description} ) ) { croak "No description specified in call to ParseOptions"; } push( @optdef, @{$cap_options{all}} ); hash_push( $opt, $cap_defaults{all} ); $opt->{'config-file'} = XMLTV::Config_file::filename( undef, $p->{grabber_name}, 1 ); foreach my $cap (@{$p->{capabilities}}) { if (not exists $cap_options{$cap}) { my @known = sort keys %cap_options; croak "Unknown capability $cap (known: @known)"; } push( @optdef, @{$cap_options{$cap}} ); hash_push( $opt, $cap_defaults{$cap} ); $capabilities->{$cap} = 1; } if( $capabilities->{preferredmethod} and not exists($p->{preferredmethod}) ) { croak "You must specify which preferredmethod to use"; } if( exists($p->{preferredmethod}) and not $capabilities->{preferredmethod} ) { croak "You must include the capability preferredmethod to specify " . "which preferredmethod to use."; } push( @optdef, @{$p->{extra_options}} ) if( defined( $p->{extra_options} ) ); hash_push( $opt, $p->{defaults} ) if( defined( $p->{defaults} ) ); my $res = GetOptions( $opt, @optdef ); if( (not $res) || $opt->{help} || scalar( @ARGV ) > 0 ) { PrintUsage( $p ); exit 1; } elsif( $opt->{capabilities} ) { print join( "\n", @{$p->{capabilities}} ) . "\n"; exit 0; } elsif( $opt->{preferredmethod} ) { print $p->{preferredmethod} . "\n"; exit 0; } elsif( $opt->{version} ) { eval { require XMLTV; print "XMLTV module version $XMLTV::VERSION\n"; } or print "could not load XMLTV module, xmltv is not properly installed\n"; if( $p->{version} =~ m/^(?:\d+)(?:\.\d+){0,2}(?:_\d*)?$/) { print "This is $p->{grabber_name} version $p->{version}\n"; } elsif( $p->{version} =~ m!\$Id: [^,]+,v (\S+) ([0-9/: -]+)! ) { print "This is $p->{grabber_name} version $1\n"; } else { croak "Invalid version $p->{version}"; } exit 0; } elsif( $opt->{description} ) { print $p->{description} . "\n"; exit 0; } elsif( $opt->{info} || $opt->{man} ) { # pod2usage(-verbose => 2) doesn't work under Windows xmltv.exe (since it needs perldoc) if ($^O eq 'MSWin32') { require Pod::Text; Pod::Text->new (sentence => 0, margin => 2, width => 78)->parse_from_file($0); } else { require Pod::Usage; import Pod::Usage; pod2usage(-verbose => 2); } exit 0; } XMLTV::Ask::init($opt->{gui}); if( defined( $opt->{output} ) ) { # Redirect STDOUT to the file. if( not open( STDOUT, "> $opt->{output}" ) ) { print STDERR "Cannot write to $opt->{output}.\n"; exit 1; } # Redirect default output to STDOUT select( STDOUT ); } if( $opt->{configure} ) { Configure( $p->{stage_sub}, $p->{listchannels_sub}, $opt->{"config-file"}, $opt ); exit 0; } # no config needed to list lineups supported by grabber if( $opt->{"list-lineups"} ) { print &{$p->{list_lineups_sub}}($opt); exit 0; } my $conf = LoadConfig( $opt->{'config-file'} ); if( not defined( $conf ) and defined( $p->{load_old_config_sub} ) ) { $conf = &{$p->{load_old_config_sub}}( $opt->{'config-file'} ); } if( $opt->{"configure-api"} ) { if( (not defined $conf) and ( $opt->{stage} ne 'start' ) ) { print STDERR "You need to start configuration with the 'start' stage.\n"; exit 1; } if( $opt->{stage} eq 'select-channels' ) { my $chanxml = &{$p->{listchannels_sub}}($conf, $opt); print SelectChannelsStage( $chanxml, $p->{grabber_name} ); } else { print &{$p->{stage_sub}}( $opt->{stage}, LoadConfig( $opt->{"config-file"} ) ); } exit 0; } if( $opt->{"list-channels"} ) { if( not defined( $conf ) ) { print STDERR "You need to configure the grabber before you can list " . "the channels.\n"; exit 1; } print &{$p->{listchannels_sub}}($conf,$opt); exit 0; } if( $opt->{"get-lineup"} ) { if( not defined( $conf ) ) { print STDERR "You need to configure the grabber before you can output " . "your chosen lineup.\n"; exit 1; } print &{$p->{get_lineup_sub}}($conf,$opt); exit 0; } if( not defined( $conf ) ) { print STDERR "You need to configure the grabber by running it with --configure \n"; exit 1; } return ($opt, $conf); } sub PrintUsage { my( $p ) = @_; my $gn = $p->{grabber_name}; my $en = " " x length( $gn ); print qq/ $gn --help $gn --info $gn --version $gn --capabilities $gn --description /; if( supports( "baseline", $p ) ) { print qq/ $gn [--config-file FILE] $en [--days N] [--offset N] $en [--output FILE] [--quiet] [--debug] /; } if( supports( "manualconfig", $p ) ) { print qq/ $gn --configure [--config-file FILE] /; } if( supports( "apiconfig", $p ) ) { print qq/ $gn --configure-api [--stage NAME] $en [--config-file FILE] $en [--output FILE] $gn --list-channels [--config-file FILE] $en [--output FILE] [--quiet] [--debug] /; } if( supports( "lineups", $p ) ) { print qq/ $gn --list-lineups [--output FILE] $gn --get-lineup [--config-file FILE] [--output FILE] /; } } sub supports { my( $cap, $p ) = @_; foreach my $sc (@{$p->{capabilities}}) { return 1 if( $sc eq $cap ); } return 0; } sub hash_push { my( $h, $n ) = @_; foreach my $key (keys( %{$n} )) { $h->{$key} = $n->{$key}; } } 1; =head1 COPYRIGHT Copyright (C) 2005,2006 Mattias Holmlund. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. =cut ### Setup indentation in Emacs ## Local Variables: ## perl-indent-level: 4 ## perl-continued-statement-offset: 4 ## perl-continued-brace-offset: 0 ## perl-brace-offset: -4 ## perl-brace-imaginary-offset: 0 ## perl-label-offset: -2 ## cperl-indent-level: 4 ## cperl-brace-offset: 0 ## cperl-continued-brace-offset: 0 ## cperl-label-offset: -2 ## cperl-extra-newline-before-brace: t ## cperl-merge-trailing-else: nil ## cperl-continued-statement-offset: 2 ## indent-tabs-mode: t ## End: ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/PreferredMethod.pm������������������������������������������������������������������0000664�0000000�0000000�00000001717�15000742332�0017053�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������=head1 NAME XMLTV::PreferredMethod - Adds a preferredmethod argument to XMLTV grabbers =head1 DESCRIPTION Add a --preferredmethod argument to your program, eg use XMLTV::PreferredMethod 'allatonce'; If a --preferredmethod parameter is supplied on the command-line, it will be caught already by the "use" statement, the string supplied in the use-line will be printed to STDOUT and the program will exit. Don't forget to announce the preferredmethod capability as well. =head1 SEE ALSO L<XMLTV::Options>, L<XMLTV::Capabilities>. =cut package XMLTV::PreferredMethod; my $opt = '--preferredmethod'; sub import( $$ ) { my( $class, $method ) = @_; die "usage: use $class 'method'" if scalar(@_) != 2; my $seen = 0; foreach (@ARGV) { # This doesn't handle abbreviations in the GNU style. last if $_ eq '--'; if ($_ eq $opt) { $seen++ && warn "seen '$opt' twice\n"; } } return if not $seen; print $method . "\n"; exit(); } 1; �������������������������������������������������xmltv-1.4.0/lib/ProgressBar.pm����������������������������������������������������������������������0000664�0000000�0000000�00000003653�15000742332�0016226�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# A few routines for asking the user questions. Used in --configure # and also by Makefile.PL, so this file should not depend on any # nonstandard libraries. # package XMLTV::ProgressBar; use strict; use XMLTV::GUI; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } my $real_class = 'XMLTV::ProgressBar::None'; my $bar; # Must be called before we use this module if we want to use a gui. sub init( $ ) { my $opt_gui = shift; # Ask the XMLTV::GUI module for the graphics type we will use my $gui_type = XMLTV::GUI::get_gui_type($opt_gui); if ($gui_type eq 'term') { $real_class = 'XMLTV::ProgressBar::None'; } elsif ($gui_type eq 'term+progressbar') { $real_class = 'XMLTV::ProgressBar::Term'; } elsif ($gui_type eq 'tk') { $real_class = 'XMLTV::ProgressBar::Tk'; } else { die "Unknown gui type: '$gui_type'."; } } # Create and return a new progress bar. # Parameters: # text to display # maximum value # Or the syntax for Term::ProgressBar may be used, but much of it will be # ignored in some of the implementations. sub new { my $class = shift; ((my $real_class_path = $real_class.".pm") =~ s/::/\//g); require $real_class_path; import $real_class_path; $bar = $real_class->new(@_); my $self = {}; return bless $self, $class; } # Alter the value displayed in this progress bar # Parameters: # the value to change this bar to display (optional) # If no value is given, the value will be incremented by 1. sub update { my $self = shift; return $bar->update( @_ ); } # Close the progress bar. sub finish { # Only does anything for the GUI ones. if ($real_class eq 'XMLTV::ProgressBar::Tk') { return $bar->finish(); } } 1; �������������������������������������������������������������������������������������xmltv-1.4.0/lib/ProgressBar/������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0015661�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/ProgressBar/None.pm�����������������������������������������������������������������0000664�0000000�0000000�00000000521�15000742332�0017114�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# The lack of a progress bar package XMLTV::ProgressBar::None; use strict; sub new { my $class = shift; my $self = {}; my $args = shift; unless (ref($args)) { $args = { 'name' => $args }; } print STDERR $args->{"name"} . "\n"; return bless $self, $class; } sub AUTOLOAD { # Do nothing } 1; �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/ProgressBar/Term.pm�����������������������������������������������������������������0000664�0000000�0000000�00000000764�15000742332�0017135�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# A wrapper around Term::ProgressBar package XMLTV::ProgressBar::Term; use strict; use Exporter; our @EXPORT = qw(close_bar); sub new { my $class = shift; $ENV{LINES}=24 unless exists $ENV{LINES}; $ENV{COLUMNS}=80 unless exists $ENV{COLUMNS}; return Term::ProgressBar->new(@_); } sub close_bar() { # Do nothing } sub AUTOLOAD { use vars qw($AUTOLOAD); print STDERR "dog $AUTOLOAD\n"; #return Term::ProgressBar->$AUTOLOAD(@_); } 1; ������������xmltv-1.4.0/lib/ProgressBar/Tk.pm�������������������������������������������������������������������0000664�0000000�0000000�00000003545�15000742332�0016604�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# A wrapper around Tk::ProgressBar package XMLTV::ProgressBar::Tk; use strict; use Tk; use Tk::ProgressBar; my $main_window; my $tk_progressbar; my $pid; my $unused; sub new { my $class = shift; my $self = {}; bless $self, $class; $self->_init(@_); } sub _init { my $self = shift; # Term::ProgressBar V1 Compatibility if (@_==2) { return $self->_init({count => $_[1], name => $_[0], term_width => 50, bar_width => 50, major_char => '#', minor_char => '', lbrack => '', rbrack => '', term => 0, }) } my %params = %{$_[0]}; my $main_window = MainWindow->new; $main_window->title("Please Wait"); $main_window->minsize(qw(400 250)); $main_window->geometry('+250+150'); my $top_frame = $main_window->Frame()->pack; my $middle_frame = $main_window->Frame()->pack( -fill => "x" ); my $bottom_frame = $main_window->Frame()->pack(-side => 'bottom'); $top_frame->Label(-height => 2)->pack; $top_frame->Label(-text => $params{name})->pack; my $tk_progressbar = $middle_frame->ProgressBar( -width => 20, -height => 300, -from => 0, -to => $params{count}, -variable => \$unused )->pack( -fill=>"x", -pady => 24, -padx => 8 ); $self->{main_window} = $main_window; $self->{tk_progressbar} = $tk_progressbar; $main_window->update(); return $self; } sub update { my $self = shift; my $set_to_value = shift; if (not $set_to_value) { $set_to_value = $self->{tk_progressbar}->value + 1; } $self->{tk_progressbar}->value( $set_to_value ); $self->{main_window}->update(); } sub finish { my $self = shift; $self->{main_window}->destroy(); } 1; �����������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Summarize.pm������������������������������������������������������������������������0000664�0000000�0000000�00000014015�15000742332�0015743�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������package XMLTV::Summarize; use strict; use base 'Exporter'; our @EXPORT = (); our @EXPORT_OK = qw(summarize); use Date::Manip; use XMLTV; use XMLTV::TZ qw(gettz ParseDate_PreservingTZ); BEGIN { if (int(Date::Manip::DateManipVersion) >= 6) { Date::Manip::Date_Init("SetDate=now,UTC"); } else { Date::Manip::Date_Init("TZ=UTC"); } } =pod =head1 NAME XMLTV::Summarize - Perl extension to summarize XMLTV data =head1 SYNOPSIS # First get some data from the XMLTV module, eg: use XMLTV; my $data = XMLTV::parsefile('tv_sorted.xml'); my ($encoding, $credits, $ch, $progs) = @$data; # Now turn the sorted programmes into a printable summary. use XMLTV::Summarize qw(summarize); foreach (summarize($ch, $progs)) { if (not ref) { print "\nDay: $_\n\n"; } else { my ($start, $stop, $title, $sub_title, $channel) = @$_; print "programme starts at $start, "; print "stops at $stop, " if defined $stop; print "has title $title "; print "and episode title $sub_title" if defined $sub_title; print ", on channel $channel.\n"; } } =head1 DESCRIPTION This module processes programme and channel data from the XMLTV module to help produce a human-readable summary or TV guide. It takes care of choosing the correct language (based on the LANG environment variable) and of looking up the name of channels from their id. There is one public routine, C<summarize()>. This takes (references to) a channels hash and a programmes list, the same format as those returned by the XMLTV module. It returns a list of 'summary' elements where each element is a list of five items: start time, stop time, title, 'sub-title', and channel name. The stop time and sub-title may be undef. The times are formatted as hh:mm, with a timezone appended when the timezone changes in the middle of listings. For the titles and channel name, the shortest string that is in an acceptable language is chosen. The list of acceptable languages normally contains just one element, taken from LANG, but you can set it manually as @XMLTV::Summarize::PREF_LANGS if wished. =head1 AUTHOR Ed Avis, ed@membled.com =head1 SEE ALSO L<XMLTV(1)>. =cut # List of preferred languages. Hopefully the environment variable # $LANG will be set. # # After loading this module you are free to change @PREF_LANGS. It is # just a list of language codes. # our @PREF_LANGS; my $el = $ENV{LANG}; if (defined $el and $el =~ /\S/) { $el =~ s/\..+$//; # remove character set @PREF_LANGS = ($el); } else { @PREF_LANGS = ('en'); } # Private. sub shorter( $$ ) { length($_[0]) <=> length($_[1]) } # Generate summary information of programmes, suitable for generating # a terse printed listings guide. # # Parameters: # channels hash # programmes list # (both these from XMLTV::parsefiles() or whatever) # # It works best if the programmes are sorted by date. # sub summarize( $$ ) { my ($ch, $progs) = @_; my @r; my $ch_name = find_channel_names($ch); my ($curr_date, $curr_tz); foreach (@$progs) { my ($start, $start_tz, $start_hhmm); $start = ParseDate_PreservingTZ($_->{start}); $start_tz = gettz($_->{start}) || 'UTC'; $start_hhmm = UnixDate($start, '%R'); my ($stop, $stop_tz, $stop_hhmm); if (defined $_->{stop}) { $stop = ParseDate_PreservingTZ($_->{stop}); $stop_tz = gettz($_->{stop}) || 'UTC'; $stop_hhmm = UnixDate($stop, '%R'); } my $date = UnixDate($start, '%m-%d (%A)'); if (not defined $curr_date or $curr_date ne $date) { $curr_date = $date; push @r, $date; } my $title = XMLTV::best_name(\@PREF_LANGS, $_->{title}, \&shorter)->[0]; my $sub_title; if (defined $_->{'sub-title'}) { $sub_title = XMLTV::best_name(\@PREF_LANGS, $_->{'sub-title'}, \&shorter)->[0]; } my $desc; if (defined $_->{'desc'}) { # No comparator, just get the first one in the preferred language (this is probably the best/shortest in most cases) $desc = XMLTV::best_name(\@PREF_LANGS, $_->{'desc'})->[0]; $desc =~ tr/\t\n/ /; # remove tabs and newlines } if (not defined $curr_tz) { # Assume that the first item in a listing doesn't need an # explicit timezone. # $curr_tz = $start_tz; } if ((not defined $curr_tz) or ($curr_tz ne $start_tz) or (defined $stop_tz and $start_tz ne $stop_tz)) { # The timezone has changed somehow - make it explicit. $start_hhmm .= " $start_tz"; $stop_hhmm .= " $stop_tz" if defined $stop_hhmm; undef $curr_tz; } unless (defined $stop_tz and $start_tz ne $stop_tz) { # The programme probably starts and stops in the same TZ - # we can assume that this is the one to use from now on. # $curr_tz = $start_tz; } # Look up pretty name of channel. my $channel = $ch_name->{$_->{channel}}; if (not defined $channel) { # No <channel> with this id. That's okay, since the XMLTV # format doesn't mandate it... yet. We choose the XMLTV # id instead. # $channel = $_->{channel}; } push @r, [ $start_hhmm, $stop_hhmm, $title, $sub_title, $channel, $desc ]; } return @r; } # find_channel_names() # # Parameter: refhash of channels data from parsefiles() # Returns: ref to hash mapping channel id to printable channel name # sub find_channel_names( $ ) { my $h = shift; my %r; foreach my $id (keys %$h) { my @names = @{$h->{$id}->{'display-name'}}; die "channels hash has no name for $id" if not @names; my $best = XMLTV::best_name(\@PREF_LANGS, \@names, \&shorter)->[0]; die "couldn't get name for channel $id" if not defined $best; # There's no need to warn about more than one channel having # the same human-readable name: that's deliberate (eg regional # variants of the same channel may all have the same number). # Maybe it could be checked when the channel id is actually # looked up to get the name, that the name hasn't been used # for a different channel id. But we won't even do that for # now. # $r{$id} = $best; } return \%r; } 1; �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Supplement.pm.PL��������������������������������������������������������������������0000664�0000000�0000000�00000001053�15000742332�0016433�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Generate Supplement.pm from Supplement.pm.in and set the share directory # correctly. use strict; use IO::File; my( $out, $share ) = @ARGV; die "no output file given" if not defined $out; die "no share-dir given" if not defined $share; my $in = 'lib/Supplement.pm.in'; my $in_fh = new IO::File "< $in" or die "cannot read $in: $!"; my $out_fh = new IO::File "> $out" or die "cannot write to $out: $!"; my $seen = 0; while (<$in_fh>) { print $out_fh $_; } close $out_fh or die "cannot close $out: $!"; close $in_fh or die "cannot close $in: $!"; �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Supplement.pm.in��������������������������������������������������������������������0000664�0000000�0000000�00000024673�15000742332�0016543�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������package XMLTV::Supplement; use strict; # use version number for feature detection: # 0.005065 : addition of GetSupplementLines, GetSupplementRoot, GetSupplementDir our $VERSION = 0.005065; BEGIN { use Exporter (); our (@ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS); @ISA = qw(Exporter); @EXPORT = qw( ); %EXPORT_TAGS = ( ); # eg: TAG => [ qw!name1 name2! ], @EXPORT_OK = qw/GetSupplement GetSupplementLines SetSupplementRoot GetSupplementRoot GetSupplementDir/; } our @EXPORT_OK; use File::Slurp qw/read_file/; use File::Spec; use File::Path; use LWP::UserAgent; use HTTP::Status qw/RC_NOT_MODIFIED RC_OK/; use XMLTV; # We only need VERSION... =head1 NAME XMLTV::Supplement - Supplementary file loader for XMLTV grabbers =head1 DESCRIPTION Utility library that loads supplementary files for xmltv-grabbers and other programs in the xmltv-distribution. Supplementary files can be loaded either via http or from a local file, depending on the configuration of the module. The default is to load the files from http://supplement.xmltv.org. This can be changed by setting the environment variable XMLTV_SUPPLEMENT to the new root-directory or root-url for supplementary files. =head1 EXPORTED FUNCTIONS All these functions are exported on demand. =over 4 =cut sub d { print STDERR "XMLTV::Supplement: $_[0]\n" if $ENV{XMLTV_SUPPLEMENT_VERBOSE}; } my $cachedir; sub create_cachedir { my $base; if ($^O eq 'MSWin32') { require Win32; Win32->import( qw(CSIDL_LOCAL_APPDATA) ); $base = Win32::GetFolderPath( CSIDL_LOCAL_APPDATA() ); $base =~ s/ /\ /g; if( not -d $base ) { $base =~ s/(.*?)\\Local Settings(.*)/$1$2/; } elsif( not -d $base ) { die "Unable to find suitable cache-directory: $base"; exit 1; } $cachedir = File::Spec->catfile( $base, "xmltv", "supplement" ); } else { $cachedir = File::Spec->catfile( $ENV{HOME}, ".xmltv", "supplement" ); } d( "Using cachedir '$cachedir'" ); create_dir( $cachedir ); } sub create_dir { my( $dir ) = @_; eval { mkpath($dir) }; if ($@) { print STDERR "Failed to create $dir: $@"; exit 1; } } my $supplement_root; sub set_supplement_root { if( defined( $ENV{XMLTV_SUPPLEMENT} ) ) { $supplement_root = $ENV{XMLTV_SUPPLEMENT}; } else { $supplement_root = "http://supplement.xmltv.org/"; } } my $ua; sub init_ua { $ua = LWP::UserAgent->new( agent => "XMLTV::Supplement/" . $XMLTV::VERSION ); $ua->env_proxy(); } =item GetSupplement Load a supplement file and return it as a string. Takes two parameters: directory and filename. my $content = GetSupplement( 'tv_grab_uk_rt', 'channel_ids' ); GetSupplement will always return a string with the content. If it fails to get the content, it prints an error-message and aborts the program. =cut sub GetSupplement { my( $directory, $name ) = @_; set_supplement_root() if not defined $supplement_root; if( $supplement_root =~ m%^http(s){0,1}://% ) { return GetSupplementUrl( $directory, $name ); } else { return GetSupplementFile( $directory, $name ); } } =item GetSupplementLines Load a supplement file and return it as an array. Any comments or blank lines will be removed. Takes two parameters: directory and filename. my $content = GetSupplementLines( 'tv_grab_uk_rt', 'channel_ids' ); GetSupplementLines will always return an array with the content. If it fails to get the content, it prints an error-message and aborts the program. =cut sub GetSupplementLines { my $supplement_string = GetSupplement( @_ ); my @supplement_array; my @supplement_lines = split( /\n+/, $supplement_string ); foreach ( @supplement_lines ) { tr/\r//d; # strip CRs s/#.*//; # strip comment next if m/^\s*$/; # ignore blank lines s/^\s+|\s+$//g; # remove leading & trailing spaces push @supplement_array, $_; } return \@supplement_array; } =item GetSupplementFile Get the supplement file from the local machine =cut sub GetSupplementFile { my( $directory, $name ) = @_; my $filename; if( defined( $directory ) ) { $filename = File::Spec->catfile( $supplement_root, $directory, $name ); } else { $filename = File::Spec->catfile( $supplement_root, $name ); } if ($^O=~'MSWin32') { # replace illegal characters in cache file names my ($f1,$f2) = $filename =~ m/^([a-zA-Z]:[\/\\])?(.*)$/; $f2 =~ s/://g; $filename = $f1.$f2; } my $result; d( "Reading $filename" ); eval { $result = read_file( $filename ) }; if( not defined( $result ) ) { print STDERR "XMLTV::Supplement: Failed to read from $filename.\n"; exit 1; } return $result; } =item GetSupplementUrl Get the supplement file from a URL =cut sub GetSupplementUrl { my( $directory, $name ) = @_; create_cachedir() if not defined $cachedir; init_ua() if not defined $ua; my $dir; if( defined( $directory ) ) { $dir = File::Spec->catfile( $cachedir, $directory ); create_dir( $dir ); } else { $dir = $cachedir; } # Remove trailing slash $supplement_root =~ s%/$%%; my $url; if( defined( $directory ) ) { $url = "$supplement_root/$directory/$name"; } else { $url = "$supplement_root/$name"; } d( "Going to fetch $url" ); $name =~ s/[:\?]/_/g; my $meta = read_meta( $directory, $name ); my $cached = read_cache( $directory, $name ); my %p; if( defined( $meta->{Url} ) and ($meta->{Url} eq $url ) ) { # The right url is stored in the cache. if( defined( $cached ) and defined( $meta->{'LastUpdated'} ) and 1*60*60 > (time - $meta->{'LastUpdated'} ) ) { d("LastUpdated ok. Using cache."); return $cached; } if( defined( $cached ) ) { $p{'If-Modified-Since'} = $meta->{'Last-Modified'} if defined $meta->{'Last-Modified'}; $p{'If-None-Match'} = $meta->{ETag} if defined $meta->{ETag}; } } my $resp = $ua->get( $url, %p ); if( $resp->code == RC_NOT_MODIFIED ) { write_meta( $directory, $url, $name, $resp, $meta ); d("Not Modified. Using cache."); return $cached; } elsif( $resp->is_success ) { write_meta( $directory, $url, $name, $resp, $meta ); write_cache( $directory, $name, $resp ); d("Cache miss."); return $resp->content; } elsif( defined( $cached ) ) { print STDERR "XMLTV::Supplement: Failed to fetch $url: " . $resp->status_line . ". Using cached info.\n"; return $cached; } else { print STDERR "XMLTV::Supplement: Failed to fetch $url: " . $resp->status_line . ".\n"; exit 1; } } =item GetSupplementDir Get the base directory containing supplementary files. e.g. $ENV{HOME}/.xmltv/supplement =cut sub GetSupplementDir { create_cachedir() if not defined $cachedir; return $cachedir; } =item SetSupplementRoot Set the root directory for loading supplementary files. SetSupplementRoot( '/usr/share/xmltv' ); SetSupplementRoot( 'http://my.server.org/xmltv' ); =cut sub SetSupplementRoot { my( $root ) = @_; $supplement_root = $root; } =item GetSupplementRoot Get the root directory for loading supplementary files. =cut sub GetSupplementRoot { set_supplement_root() if not defined $supplement_root; return $supplement_root; } sub write_meta { my( $directory, $url, $file, $resp, $meta ) = @_; my $metafile = cache_filename( $directory, "$file.meta" ); open OUT, "> $metafile" or die "Failed to write to $metafile"; print OUT "LastUpdated " . time() . "\n"; print OUT "Url $url\n"; if( defined $resp->header( 'Last-Modified' ) ) { print OUT "Last-Modified " . $resp->header( 'Last-Modified' ) . "\n"; } elsif( defined $meta->{'Last-Modified'} ) { print OUT "Last-Modified " . $meta->{ 'Last-Modified' } . "\n"; } print OUT "ETag " . $resp->header( 'ETag' ) . "\n" if defined $resp->header( 'ETag' ); close( OUT ); } sub read_meta { my( $directory, $file ) = @_; my $metafile = cache_filename( $directory, "$file.meta" ); return {} if not -f( $metafile ); my $str = read_file( $metafile ); my @lines = split( "\n", $str ); my $result = {}; foreach my $line (@lines) { my($key, $value ) = ($line =~ /(.*?) (.*)/); $result->{$key} = $value; } return $result; } sub read_cache { my( $directory, $file ) = @_; my $filename = cache_filename( $directory, $file ); my $result; eval { $result = read_file( $filename ) }; return $result; } sub write_cache { my( $directory, $file, $resp ) = @_; my $filename = cache_filename( $directory, $file ); open OUT, "> $filename" or die "Failed to write to $filename"; binmode OUT; print OUT $resp->content; close( OUT ); } sub cache_filename { my( $directory, $file ) = @_; if( defined( $directory ) ) { return File::Spec->catfile( $cachedir, $directory, $file ); } else { return File::Spec->catfile( $cachedir, $file ); } } =back =head1 CACHING The module stores all downloaded files in a cache. The cache is stored on disk in ~/.xmltv/supplement on Unix and in CSIDL_LOCAL_APPDATA//xmltv/supplement on Windows. If a file has been downloaded less than 1 hour ago, the file from the cache is used without contacting the server. Otherwise, if the file has been downloaded more than 1 hour ago, then the module checks with the server to see if an updated file is available and downloads it if necessary. If the server does not respond or returns an error-message, a warning is printed to STDERR and the file from the cache is used. =head1 ENVIRONMENT VARIABLES The XMLTV_SUPPLEMENT environment variable can be used to tell the module where the supplementary files are found. XMLTV_SUPPLEMENT=/usr/share/xmltv XMLTV_SUPPLEMENT=http://supplementary.xmltv.se The XMLTV_SUPPLEMENT_VERBOSE environment variable can be used to get more debug output from XMLTV::Supplement. XMLTV_SUPPLEMENT_VERBOSE=1 =head1 COPYRIGHT Copyright (C) 2007 Mattias Holmlund. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. =cut 1; ���������������������������������������������������������������������xmltv-1.4.0/lib/TMDB.pm�����������������������������������������������������������������������������0000664�0000000�0000000�00000165060�15000742332�0014524�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Author: Geoff Westcott, 2021 # (based on IMDB.pm Author: Jerry Veldhuis) # use strict; use warnings; package XMLTV::TMDB; # # CHANGE LOG # 0.1 = development version (no public release) # 0.2 = change API calls to use 'append_to_response' to reduce number of calls # 0.3 = added content-ids to output xml # 0.4 = added <image> and <url> elements to cast & crew # added <image> to programme (in place of <url>) # min version of XMLTV.pm is > 1.0.0 # our $VERSION = '0.4'; use LWP::Protocol::https; use HTTP::Response; use Data::Dumper qw(Dumper); #--------------------------------------------------------------- use XMLTV 1.0.1; # min version of xmltv.pm required use XMLTV::TMDB::API; # version 0.04 of WWW::TMDB::API is broken for movie searching (TMDB changed the API with a breaking change) # a custom version was created for use here # - redeclare the search() method with correct path "movie" instead of "movies" # - also add the year as a search param # - also casts() is now called 'credits' # - also images() : add the image_language (c.f. https://www.themoviedb.org/talk/583238a6c3a3685ba1032b0e ) # - add new method configuration() to fetch API config data #--------------------------------------------------------------- sub error($$) { print STDERR "tv_tmdb: $_[1]\n"; } sub status($$) { if ( $_[0]->{verbose} ) { print STDERR "tv_tmdb: $_[1]\n"; } } sub debug($$) { my $self=shift; my $mess=shift; if ( $self->{verbose} > 1 ) { print STDERR "tv_tmdb: $mess\n"; } } sub debugmore($$$) { my $self=shift; my $level=shift; my $mess=shift; my $dump=shift; if ( $self->{verbose} >= $level ) { print STDERR "tv_tmdb: " . (split(/::/,(caller(1))[3]))[-1] . ' : ' . $mess . " = \n" . Dumper($dump); } } sub unique (@) # de-dupe two (or more) arrays TODO: make this case insensitive { # From CPAN List::MoreUtils, version 0.22 my %h; map { $h{$_}++ == 0 ? $_ : () } @_; } sub uniquemulti (@) # de-dupe two (or more) array of arrays on first value TODO: make this case insensitive { my %h; map { $h{$_->[0]}++ == 0 ? $_ : () } @_; } #--------------------------------------------------------------------- # constructor # sub new { my ($type) = shift; my $self={ @_ }; # remaining args become attributes $self->{wwwUrl} = 'https://www.themoviedb.org/'; for ('apikey', 'verbose') { die "invalid usage - no $_" if ( !defined($self->{$_})); } $self->{replaceDates}=0 if ( !defined($self->{replaceDates})); $self->{replaceTitles}=0 if ( !defined($self->{replaceTitles})); $self->{replaceCategories}=0 if ( !defined($self->{replaceCategories})); $self->{replaceKeywords}=0 if ( !defined($self->{replaceKeywords})); $self->{replaceURLs}=0 if ( !defined($self->{replaceURLs})); $self->{replaceDirectors}=1 if ( !defined($self->{replaceDirectors})); $self->{replaceActors}=0 if ( !defined($self->{replaceActors})); $self->{replacePresentors}=1 if ( !defined($self->{replacePresentors})); $self->{replaceCommentators}=1 if ( !defined($self->{replaceCommentators})); $self->{replaceGuests}=1 if ( !defined($self->{replaceGuests})); $self->{replaceStarRatings}=0 if ( !defined($self->{replaceStarRatings})); $self->{replaceRatings}=0 if ( !defined($self->{replaceRatings})); $self->{replacePlot}=0 if ( !defined($self->{replacePlot})); $self->{replaceReviews}=0 if ( !defined($self->{replaceReviews})); $self->{updateDates}=1 if ( !defined($self->{updateDates})); $self->{updateTitles}=1 if ( !defined($self->{updateTitles})); $self->{updateCategories}=1 if ( !defined($self->{updateCategories})); $self->{updateCategoriesWithGenres}=1 if ( !defined($self->{updateCategoriesWithGenres})); $self->{updateKeywords}=0 if ( !defined($self->{updateKeywords})); # default is to NOT add keywords $self->{updateURLs}=1 if ( !defined($self->{updateURLs})); $self->{updateDirectors}=1 if ( !defined($self->{updateDirectors})); $self->{updateActors}=1 if ( !defined($self->{updateActors})); $self->{updatePresentors}=1 if ( !defined($self->{updatePresentors})); $self->{updateCommentators}=1 if ( !defined($self->{updateCommentators})); $self->{updateGuests}=1 if ( !defined($self->{updateGuests})); $self->{updateStarRatings}=1 if ( !defined($self->{updateStarRatings})); $self->{updateRatings}=1 if ( !defined($self->{updateRatings})); # add programme's classification (MPAA/BBFC etc) $self->{updatePlot}=0 if ( !defined($self->{updatePlot})); # default is to NOT add plot $self->{updateReviews}=1 if ( !defined($self->{updateReviews})); # default is to add reviews $self->{updateRuntime}=1 if ( !defined($self->{updateRuntime})); # add programme's runtime $self->{updateActorRole}=1 if ( !defined($self->{updateActorRole})); # add roles to cast in output $self->{updateImage}=1 if ( !defined($self->{updateImage})); # add programme's poster image $self->{updateCastImage}=1 if ( !defined($self->{updateCastImage})); # add image url to actors and directors (needs updateActors/updateDirectors) $self->{updateCastUrl}=1 if ( !defined($self->{updateCastUrl})); # add url to actors and directors webpage (needs updateActors/updateDirectors) $self->{updateContentId}=1 if ( !defined($self->{updateContentId})); # add programme's id $self->{numActors}=3 if ( !defined($self->{numActors})); # default is to add top 3 actors $self->{numReviews}=1 if ( !defined($self->{numReviews})); # default is to add top 1 review $self->{removeYearFromTitles}=1 if ( !defined($self->{removeYearFromTitles})); # strip trailing "(2021)" from title $self->{getYearFromTitles}=1 if ( !defined($self->{getYearFromTitles})); # if no 'date' incoming then see if title ends with a "(year)" $self->{moviesonly}=0 if ( !defined($self->{moviesonly})); # default to augment both movies and tv $self->{minVotes}=50 if ( !defined($self->{minVotes})); # default to needing 50 votes before 'star-rating' value is accepted # default is not to cache lookups $self->{cacheLookups}=0 if ( !defined($self->{cacheLookups}) ); $self->{cacheLookupSize}=0 if ( !defined($self->{cacheLookupSize}) ); $self->{cachedLookups}->{tv_series}->{_cacheSize_}=0; bless($self, $type); # stats counters $self->{categories}={'movie' =>'Movie', # 'tv_movie' =>'TV Movie', # made for tv # 'video_movie' =>'Video Movie', # went straight to video or was made for it 'tv_series' =>'TV Series', # 'tv_mini_series' =>'TV Mini Series' }; # note there is no 'qualifier' in TMDB data - we only have either 'movie' or 'tv' $self->{stats}->{programCount}=0; for my $cat (keys %{$self->{categories}}) { $self->{stats}->{perfect}->{$cat}=0; $self->{stats}->{close}->{$cat}=0; } $self->{stats}->{perfectMatches}=0; $self->{stats}->{closeMatches}=0; $self->{stats}->{startTime}=time(); #print STDERR Dumper($self); die(); return($self); } #-----------------------------------------------------------------------------------------# #-----------------------------------------------------------------------------------------# # The methods below follow their equivalents in IMDB.pm to simplify future maintenance. # # So we keep the names the same even though they may not be strictly accurate. # # Some methods aren't applicable to the TMDB lookup but we keep them anyway. # #-----------------------------------------------------------------------------------------# #-----------------------------------------------------------------------------------------# # not applicable in this package # sub checkIndexesOkay($) { my $self=shift; # nothing to do here return(undef); } # check that the tmdb api is up and running by fetching a known-to-exist programme # sub basicVerificationOfIndexes($) { my $self=shift; # check that the tmdb api is up and running by fetching a known-to-exist programme my $title="Army of Darkness"; my $year=1992; $self->openMovieIndex() || return("basic verification of api failed\n". "api is not accessible"); # tempo hide the verbose setting while we fetch the test movie my $verbose = $self->{verbose}; $self->{verbose} = 0; my $res = $self->getMovieMatches($title, $year); $self->{verbose} = $verbose; undef $verbose; # there is less to do here than with IMDB.pm as we don't need to check on the database build # so a simple check that the API is accessible (i.e. api_key is valid) and returns the movie # we expect will be sufficient if ( !defined($res) ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "no match for basic verification of movie \"$title, $year\"\n"); } if ( !defined($res->{exactMatch}) ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "no exact match for movie \"$title, $year\"\n"); } if ( scalar(@{$res->{exactMatch}})!= 1) { # we expect only 1 matching hit for the test movie $self->closeMovieIndex(); return("basic verification of indexes failed\n". "got more than one exact match for movie \"$title, $year\"\n"); } if ( @{$res->{exactMatch}}[0]->{year} ne "$year" ) { $self->closeMovieIndex(); return("basic verification of indexes failed\n". "year associated with key \"$title, $year\" is bad\n"); } $self->closeMovieIndex(); # all okay return(undef); } # check the api is accessible # sub sanityCheckDatabase($) { my $self=shift; my $errline; $errline=$self->checkIndexesOkay(); return($errline) if ( defined($errline) ); $errline=$self->basicVerificationOfIndexes(); return($errline) if ( defined($errline) ); # all okay return(undef); } # instantiate a TMDB::API object # sub openMovieIndex($) { my $self=shift; my $tmdb_client = XMLTV::TMDB::API->new( 'api_key' => $self->{apikey}, 'ua' => LWP::UserAgent->new( 'agent' => "XMLTV-TMDB-API/$VERSION"), 'soft' => 1 ); # force https $tmdb_client->{url} =~ s/^http:/https:/; $self->{tmdb_client} = $tmdb_client; return(1); } # destroy the TMDB::API object # sub closeMovieIndex($) { my $self=shift; undef $self->{tmdb_client}; return(1); } # strip some punctuation from the title # be conservative with this: we're really only looking at periods, # so that "Dr. Who" and "Dr Who" are treated as equal # # TODO: we need to match incoming accented chars with unaccented input # e.g. tmdb "Amélie" does not match (line 441) if xml is "Amelie" (no accent) # sub tidy($$) { my $self=shift; my $title=shift; # actions: # 1. strip periods, commas, apostrophes, quotes, colons, semicolons, hyphen # 2. strip articles: the, le, la, das # 3. strip year (e.g. " (2005)" # 4-5. tidy spaces # 6. lowercase $title =~ s/[\.,'":;\-]//g; $title =~ s/\b(the|le|la|das)\b//ig; # TODO: add others? $title =~ s/\s+\((19|20)\d\d\)\s*$//; $title =~ s/(^\s+)|(\s+$)//; $title =~ s/\s+/ /g; return lc($title); } # check the fetch reply from the API # sub checkHttpError($$) { my $self=shift; my $uaresult=shift; my $res = @{$uaresult}[0]; my $uaresponse = @{$uaresult}[1]; $self->debugmore(5, "ua response", $uaresponse); if ( $uaresponse->{code} == 401 ) { $self->error("TMDB said 'Unauthorised' : is your apikey correct? : cannot continue"); exit(1); # TODO: more graceful exit! } elsif ( $uaresponse->{code} >= 500 ) { $self->error("TMDB said '$uaresponse->{msg}' : cannot continue"); exit(1); # TODO: more graceful exit! } elsif ( $uaresponse->{code} == 429 ) { # HTTP_TOO_MANY_REQUESTS $self->error("TMDB said '$uaresponse->{msg}' : cannot continue"); exit(1); # TODO: more graceful exit! } elsif ( $uaresponse->{code} >= 400 ) { $self->status("TMDB said '$uaresponse->{msg}' : will try to continue"); } return $res; } # get matches (either movie or tv) from TMDB using title + year (optional) # # TODO: handle paged results (but not seen any yet) # sub getSearchMatches($$$) { my $self=shift; my $title=shift; my $year=shift; my $type=shift; # strip year from the title, if present ($title,my $junk,my $titleyear) = $title =~ m/^(.*?)(\s+\(((19|20)\d\d)\))?$/; # $year = '' if !defined $year; # note: don't override tha param year or else 'close matches' won't work $self->debug("looking for $type \"$title\" ".($year ne ''?"+ $year ":'')."") if $type eq 'movie'; $self->debug("looking for $type \"$title\"") if $type eq 'tv'; # lookup TMDB my $matches; $matches = $self->checkHttpError( $self->{tmdb_client}->$type->search( query => $title, year => $year ) ) if $type eq 'movie'; $matches = $self->checkHttpError( $self->{tmdb_client}->$type->search( query => $title ) ) if $type eq 'tv'; # API.pm returns a Perl data structure from the JSON reply # #-------------------------------------------------------------------------------------------- # MOVIE # { # 'total_pages' => 1, # 'total_results' => 1, # 'page' => 1, # 'results' => [ # { # 'original_language' => 'en', # 'video' => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ), # 'release_date' => '1992-10-31', # 'vote_count' => 2247, # 'genre_ids' => [ # 14, # 27, # 35 # ], # 'original_title' => 'Army of Darkness', # 'vote_average' => '7.3', # 'overview' => 'Ash is transported back to medieval days, where he is captured by the dreaded Lord Arthur. Aided by the deadly chainsaw that has become his only friend, Ash is sent on a perilous mission to recover the Book of the Dead, a powerful tome that gives its owner the power to summon an army of ghouls.', # 'popularity' => '18.055', # 'poster_path' => '/mOsWtjRGABrPrqqtm0U6WQp4GVw.jpg', # 'title' => 'Army of Darkness', # 'backdrop_path' => '/5Tfj9mbCq8KzJZ1cnPEWPhqecI7.jpg', # 'id' => 766, # 'adult' => $VAR1->{'results'}[0]{'video'} # } # ] # }; #-------------------------------------------------------------------------------------------- # TV SERIES # { # "total_pages": 1, # "total_results": 14 # "page": 1, # "results": [ # { # "backdrop_path": "/sRfl6vyzGWutgG0cmXmbChC4iN6.jpg", # "first_air_date": "2005-03-26", # "genre_ids": [ # 10759, # 18, # 10765 # ], # "id": 57243, # "name": "Doctor Who", # "origin_country": [ # "GB" # ], # "original_language": "en", # "original_name": "Doctor Who", # "overview": "The Doctor is a Time Lord: a 900 year old alien with 2 hearts, part of a gifted civilization who mastered time travel. The Doctor saves planets for a living—more of a hobby actually, and the Doctor's very, very good at it.", # "popularity": 169.084, # "poster_path": "/sz4zF5z9zyFh8Z6g5IQPNq91cI7.jpg", # "vote_average": 7.3, # "vote_count": 2225 # }, #-------------------------------------------------------------------------------------------- my $results; foreach my $match (@{ $matches->{results} }) { my $airdate; $airdate = $match->{release_date} if $type eq 'movie'; $airdate = $match->{first_air_date} if $type eq 'tv'; $airdate = '' if !defined $airdate; (my $yr) = $airdate =~ m/^((19|20)\d\d)/; $yr = '' if !defined $yr || $yr eq '1900'; # TMDB says: "If you see a '1900' referenced on a movie, that means that no release date has been added." my ($progtitle,$progtitle_alt); $progtitle = $match->{title} if $type eq 'movie'; $progtitle = $match->{name} if $type eq 'tv'; $progtitle_alt = $match->{original_title} if $type eq 'movie'; $progtitle_alt = $match->{original_name} if $type eq 'tv'; # year must match for an "exact" match # # TODO: title match will fail if incoming title is missing accented character # e.g. if incoming title is "Amelie" - tmdb is "Amélie" and therefore won't match # an ugly fudge would be to tempo convert TMDB accented chars to basic character for comparison purposes # (c.f. the code block "convert all the special language characters" in alternativeTitles() # if ( ( $self->tidy($progtitle) eq $self->tidy($title) || $self->tidy($progtitle_alt) eq $self->tidy($title) ) && ( $yr eq $year && $year ne '' ) && $type eq 'movie' ) { # return all tv matches as 'close' not 'exact' $self->debug("exact: \"$progtitle\" $yr [$match->{id}]"); push(@{$results->{exactMatch}}, {'key' => $title, 'title' => $progtitle, 'year' => $yr, 'qualifier'=> ($type eq 'tv' ? 'tv_series' : $type), 'type' => $type, 'id' => $match->{id}, 'lang' => $match->{original_language} # might this be different to the language of the current title? }); } else { $self->debug("close: \"$progtitle\" $yr [$match->{id}]"); push(@{$results->{closeMatch}}, {'key' => $title, 'title' => $progtitle, 'year' => $yr, 'qualifier'=> ($type eq 'tv' ? 'tv_series' : $type), 'type' => $type, 'id' => $match->{id}, 'lang' => $match->{original_language} }); } } $self->debugmore(4,'search results',$results); return($results); } # get movie matches from TMDB using title + year (optional) # sub getMovieMatches($$$) { my $self=shift; my $title=shift; my $year=shift; return $self->getSearchMatches($title, $year, 'movie'); } # get tv matches from TMDB using title + year (optional) # sub getTvMatches($$$) { my $self=shift; my $title=shift; my $year=shift; return $self->getSearchMatches($title, $year, 'tv'); } # get movie matches from TMDB - look for a single hit - match on title + year # sub getMovieExactMatch($$$) { my $self=shift; my $title=shift; my $year=shift; $self->debug("looking for exact match"); my $res=$self->getMovieMatches($title, $year); $self->debugmore(3,'matches',$res); return(undef, 0) if ( !defined($res) ); if ( !defined($res->{exactMatch}) ) { return( undef, 0 ); } if ( scalar(@{$res->{exactMatch}}) < 1 ) { return( undef, scalar(@{$res->{exactMatch}}) ); } $self->debugmore(2,'match',$res->{exactMatch}); return( $res->{exactMatch}, scalar(@{$res->{exactMatch}}) ); } # get movie matches from TMDB - match on title only (no year) # sub getMovieCloseMatches($$) { my $self=shift; my $title=shift; $self->debug("looking for close match"); my $res=$self->getMovieMatches($title, undef); $self->debugmore(4,'matches',$res); if ( defined($res->{exactMatch})) { $self->status("unexpected exact match on movie \"$title\""); } return( undef, 0 ) if ( !defined($res->{closeMatch}) ); $self->debugmore(3,'match',$res->{closeMatch}); return( $res->{closeMatch}, scalar(@{$res->{closeMatch}}) ); } # get Tv matches from TMDB - match on title only (no year) # sub getTvCloseMatches($$) { my $self=shift; my $title=shift; $self->debug("looking for close match"); my $res=$self->getTvMatches($title, undef); $self->debugmore(4,'matches',$res); if ( defined($res->{exactMatch})) { $self->status("unexpected exact match on tv \"$title\""); } return( undef, 0 ) if ( !defined($res->{closeMatch}) ); $self->debugmore(3,'match',$res->{closeMatch}); return( $res->{closeMatch}, scalar(@{$res->{closeMatch}}) ); } # BUG - we identify 'presenters' by the word "Host" appearing in the character # description. For some movies, character names include the word Host. # ex. Animal, The (2001) has a character named "Badger Milk Host". # sub getMovieOrTvIdDetails($$$) { my $self=shift; my $id=shift; my $type=shift; # get the API configuration from TMDB if we don't already have it # $self->{tmdb_conf} = $self->checkHttpError( $self->{tmdb_client}->config->configuration() ); # # set base url for actor/director images my $profile_base = $self->{tmdb_conf}->{images}->{base_url} . $self->{tmdb_conf}->{images}->{profile_sizes}[1]; # arbitrarily pick the second one (expecting w185 = 185x278 ) # # set base url for movie poster images my $poster_base = $self->{tmdb_conf}->{images}->{base_url} . $self->{tmdb_conf}->{images}->{poster_sizes}[4]; # arbitrarily pick the fifth one (expecting w500 = 500x750 ) my $backdrop_base = $self->{tmdb_conf}->{images}->{base_url} . $self->{tmdb_conf}->{images}->{backdrop_sizes}[1]; # arbitrarily pick the second one (expecting w780 = 780x439 ) # smaller versions: my $poster_base_s = $self->{tmdb_conf}->{images}->{base_url} . $self->{tmdb_conf}->{images}->{poster_sizes}[0]; # arbitrarily pick the first one (expecting w92 = 92x138 ) my $backdrop_base_s = $self->{tmdb_conf}->{images}->{base_url} . $self->{tmdb_conf}->{images}->{backdrop_sizes}[0]; # arbitrarily pick the first one (expecting w300 = 300x169 ) # get the movie details from TMDB # my $tmdb_info; # v0.1 method-> $tmdb_info = $self->checkHttpError( $self->{tmdb_client}->$type->info( ID => $id ) ); # note different path for release_dates vs content_ratings $tmdb_info = $self->checkHttpError( $self->{tmdb_client}->$type->info( ID => $id, append_to_response => 'keywords,credits,release_dates,reviews' ) ) if ( $type eq 'movie' ); $tmdb_info = $self->checkHttpError( $self->{tmdb_client}->$type->info( ID => $id, append_to_response => 'keywords,credits,content_ratings,reviews' ) ) if ( $type eq 'tv' ); my $results; # response to be returned to caller # let's replicate what IMDB.pm retrieved : # ($directors, $actors, $genres, $ratingDist, $ratingVotes, $ratingRank, $keywords, $plot); also $presenter, $commentator $results->{ratingVotes} = $tmdb_info->{vote_count} if defined $tmdb_info->{vote_count}; $results->{ratingRank} = $tmdb_info->{vote_average} if defined $tmdb_info->{vote_average} && $tmdb_info->{vote_average} > 0; $results->{plot} = $tmdb_info->{overview} if defined $tmdb_info->{overview}; foreach my $genre (@{ $tmdb_info->{genres} }) { push(@{$results->{genres}}, $genre->{name}); } # get the keywords # v0.1 method-> my $tmdb_keywords = $self->checkHttpError( $self->{tmdb_client}->$type->keywords( ID => $id ) ); # $tmdb_keywords->{keywords} # my $tmdb_keywords = $tmdb_info->{keywords}; # if ( defined $tmdb_keywords->{keywords} ) { foreach my $keyword (@{ $tmdb_keywords->{keywords} }) { push(@{$results->{keywords}}, $keyword->{name}); } } # get the credits (cast and crew) # v0.1 method-> my $tmdb_credits = $self->checkHttpError( $self->{tmdb_client}->$type->credits( ID => $id ) ); # $tmdb_credits->{cast} & $tmdb_credits->{crew} # my $tmdb_credits = $tmdb_info->{credits}; # # TMDB seems to return the list pre-sorted by 'order', which is nice # if ( defined $tmdb_credits->{cast} ) { foreach my $cast (@{ $tmdb_credits->{cast} }) { # if ( defined $cast->{character} && $cast->{character}=~m/Host|Presenter/ ) { push(@{$results->{presenter}}, $cast->{name}); } elsif ( defined $cast->{character} && $cast->{character}=~m/Narrator|Commentator/ ) { push(@{$results->{commentator}}, $cast->{name}); } elsif ( defined $cast->{character} && $cast->{character}=~m/Guest/ ) { push(@{$results->{guest}}, $cast->{name}); } else { push(@{$results->{actors}}, $cast->{name}); # use this to keep consistency with TMDB } # # # note in TMDB an actor can appear twice with different 'character'. See Chris Shields in Christmas Joy (2018) : character1 = 'Chris Andrews', character2='Dr. Walsh'. We should concatenate these if outputting 'character' attribute. # my $found=0; if ( defined $cast->{character} && $cast->{character} ne '' && defined $results->{actorsplus} ) { # find the matching array entry my @list = @{ $results->{actorsplus} }; for (my $i=0; $i < scalar @list; $i++) { my $h = $list[$i]; if ( $h->{id} eq $cast->{id} ) { # already in list, so concatenate 'character's @{$results->{actorsplus}}[$i]->{'character'} .= '/'. $cast->{character}; $found=1; last; } } } # this is a new entry - just add it to actors' list if (!$found) { push(@{$results->{actorsplus}}, { 'name' =>$cast->{name}, 'id' =>$cast->{id}, 'character' =>$cast->{character}, 'order' =>$cast->{order}, 'imageurl' =>(defined $cast->{profile_path} ? $profile_base.$cast->{profile_path} : '') } ); # use this to add TMDB unique data } } } if ( defined $tmdb_credits->{crew} ) { foreach my $crew (@{ $tmdb_credits->{crew} }) { # if ( $crew->{job}=~m/^Director$/ ) { # this may be too strict but we must avoid "Director of Photography" etc push(@{$results->{directors}}, $crew->{name}); # use this to keep consistency with TMDB # push(@{$results->{directorsplus}}, {'name' =>$crew->{name}, 'id' =>$crew->{id}, 'job' =>$crew->{job}, 'order' =>$crew->{order}, 'imageurl' =>(defined $crew->{profile_path} ? $profile_base.$crew->{profile_path} : '') } ); # use this to add TMDB unique data } } } # get the classification if ( $type eq 'movie' ) { # v0.1 method-> my $tmdb_classifications = $self->checkHttpError( $self->{tmdb_client}->$type->release_dates( ID => $id ) ); # $tmdb_classifications->{results} # my $tmdb_classifications = $tmdb_info->{release_dates}; # if ( defined $tmdb_classifications->{results} ) { foreach my $classification (@{ $tmdb_classifications->{results} }) { # just get US,CA,GB # TODO: select based on user's country code (somehow) my %lookup = map { $_ => undef } qw( US CA GB ); if ( exists $lookup{ $classification->{iso_3166_1} } ) { my $rating; my $i=0; foreach my $release (@{ $classification->{release_dates} }) { $i++; # # TMDB has various 'types' of classification (premiere, theatrical, tv, etc) # which one to use? # Let's go for type=1 "Premiere", or if missing then just grab the first in the list if ( $release->{type} == 1 || $i == 1 ) { $rating = $release->{certification} if $release->{certification} ne ''; } } if ( $rating ) { # sometimes the field is empty! push(@{$results->{classifications}}, {'system' =>$classification->{iso_3166_1}, 'rating' =>$rating } ); } } } } } else { # v0.1 method-> my $tmdb_classifications = $self->checkHttpError( $self->{tmdb_client}->$type->content_ratings( ID => $id ) ); # my $tmdb_classifications = $tmdb_info->{content_ratings}; # if ( defined $tmdb_classifications->{results} ) { foreach my $classification (@{ $tmdb_classifications->{results} }) { # just get US,CA,GB # TODO: select based on user's country code (somehow) my %lookup = map { $_ => undef } qw( US CA GB ); if ( exists $lookup{ $classification->{iso_3166_1} } ) { if ( $classification->{rating} ne '' ) { push(@{$results->{classifications}}, {'system' =>$classification->{iso_3166_1}, 'rating' =>$classification->{rating} } ); } } } } } # get the reviews # v0.1 method-> my $tmdb_keywords = $self->checkHttpError( $self->{tmdb_client}->$type->reviews( ID => $id ) ); # my $tmdb_reviews = $tmdb_info->{reviews}; # if ( defined $tmdb_reviews->{results} ) { foreach my $review (@{ $tmdb_reviews->{results} }) { push(@{$results->{reviews}}, {'author' =>$review->{author}, 'content' =>$review->{content}, 'date' =>$review->{created_at}, 'url' =>$review->{url} } ); } } # new stuff not in IMDB.pm # $results->{runtime} = $tmdb_info->{runtime} if defined $tmdb_info->{runtime} && $tmdb_info->{runtime} > 0; $results->{tmdb_id} = $tmdb_info->{id} if defined $tmdb_info->{id}; $results->{imdb_id} = $tmdb_info->{imdb_id} if defined $tmdb_info->{imdb_id}; $results->{posterurl} = $poster_base.$tmdb_info->{poster_path} if defined $tmdb_info->{poster_path}; $results->{backdropurl} = $backdrop_base.$tmdb_info->{backdrop_path} if defined $tmdb_info->{backdrop_path}; $results->{posterurl_sm} = $poster_base_s.$tmdb_info->{poster_path} if defined $tmdb_info->{poster_path}; $results->{backdropurl_sm} = $backdrop_base_s.$tmdb_info->{backdrop_path} if defined $tmdb_info->{backdrop_path}; if ( !defined($results) ) { # some movies we don't have any details for $results->{noDetails}=1; } return($results); } # make some possible alternative titles (spelling, punctuation, etc) # (these probably aren't necessary for TMDB which seems to cater for these alternatives automagically) # sub alternativeTitles($) { my $title=shift; my @titles; push(@titles, $title); # try the & -> and conversion if ( $title=~m/\&/o ) { my $t=$title; while ( $t=~s/(\s)\&(\s)/$1and$2/o ) { push(@titles, $t); } } # try the and -> & conversion if ( $title=~m/\sand\s/io ) { my $t=$title; while ( $t=~s/(\s)and(\s)/$1\&$2/io ) { push(@titles, $t); } } # try the "Columbo: Columbo cries Wolf" -> "Columbo cries Wolf" conversion my $max=scalar(@titles); for (my $i=0; $i<$max ; $i++) { my $t=$titles[$i]; if ( $t=~m/^[^:]+:.+$/io ) { while ( $t=~s/^[^:]+:\s*(.+)\s*$/$1/io ) { push(@titles, $t); } } } # deprecated - not required for TMDB # # Place the articles last # $max=scalar(@titles); # for (my $i=0; $i<$max ; $i++) { # my $t=$titles[$i]; # if ( $t=~m/^(The|A|Une|Les|Los|Las|L\'|Le|La|El|Das|De|Het|Een)\s+(.*)$/io ) { # $t=~s/^(The|A|Une|Les|Los|Las|L\'|Le|La|El|Das|De|Het|Een)\s+(.*)$/$2, $1/iog; # push(@titles, $t); # } # if ( $t=~m/^(.+),\s*(The|A|Une|Les|Los|Las|L\'|Le|La|El|Das|De|Het|Een)$/io ) { # $t=~s/^(.+),\s*(The|A|Une|Les|Los|Las|L\'|Le|La|El|Das|De|Het|Een)$/$2 $1/iog; # push(@titles, $t); # } # } # deprecated - not required for TMDB # # convert all the special language characters # $max=scalar(@titles); # for (my $i=0; $i<$max ; $i++) { # my $t=$titles[$i]; # if ( $t=~m/[ÀÁÂÃÄÅàáâãäåÈÉÊËèéêëÌÍÎÏìíîïÒÓÔÕÖØòóôõöøÙÚÛÜùúûüÆæÇçÑñßÝýÿ]/io ) { # $t=~s/[ÀÁÂÃÄÅàáâãäå]/a/gio; # $t=~s/[ÈÉÊËèéêë]/e/gio; # $t=~s/[ÌÍÎÏìíîï]/i/gio; # $t=~s/[ÒÓÔÕÖØòóôõöø]/o/gio; # $t=~s/[ÙÚÛÜùúûü]/u/gio; # $t=~s/[Ææ]/ae/gio; # $t=~s/[Çç]/c/gio; # $t=~s/[Ññ]/n/gio; # $t=~s/[ß]/ss/gio; # $t=~s/[Ýýÿ]/y/gio; # $t=~s/[¿]//gio; # push(@titles, $t); # } # } # later possible titles include removing the '.' from titles # ie "Project V.I.P.E.R." matching imdb "Project VIPER" $max=scalar(@titles); for (my $i=0; $i<$max ; $i++) { my $t=$titles[$i]; if ( $t=~s/\.//go ) { push(@titles,$t); } } return(\@titles); } # find matching movie records on TMDB # $exact = 1 : exact matches on movies - needs title & year # $exact = 2 : exact matches on movies - title only # else : close matches on movies - needs title & year # # TODO: partial matching, e.g. "Cuckoos Nest" should match with "One Flew over the Cuckoos Nest" # sub findMovieInfo($$$$$) { my ($self, $title, $prog, $year, $exact)=@_; my @titles=@{alternativeTitles($title)}; $self->debugmore(3,'altvetitles',\@titles); if ( $exact == 1 ) { # looking for an exact match on title + year for my $mytitle ( @titles ) { # look-up TMDB my ($info, $matchcount) = $self->getMovieExactMatch($mytitle, $year); if ($matchcount > 1) { # if multiple records exactly match title+year then we don't know which one is correct # see if we can match Director (c.f. "Chaos" (2005) or "20,000 Leagues Under the Sea" (1997) ) if ( defined $prog->{credits}->{director} ) { my @infonew; my $found=0; DIRCHK: foreach my $infoeach (@$info) { my $tmdb_credits = $self->checkHttpError( $self->{tmdb_client}->movie->credits( ID => $infoeach->{id} ) ); # $tmdb_credits->{cast} & {crew} if ( defined $tmdb_credits->{crew} ) { foreach my $director ( @{ $prog->{credits}->{director} } ) { if ( scalar ( grep { lc($_->{job}) eq 'director' && $self->tidy($_->{name}) eq $self->tidy($director) } @{ $tmdb_credits->{crew} } ) >0 ) { # TODO: of course this will break if the same director directed BOTH films (but how likely is that?) $found = 1; push (@infonew, $infoeach); last DIRCHK; } } } } ( $info, $matchcount ) = ( \@infonew, 1 ) if $found; # reset the search result to just this one match } if ($matchcount > 1) { $self->status("multiple hits on movie \"$mytitle\" (".($year eq ''?'-':$year).")"); return(undef, $matchcount); } } if ($matchcount == 1) { if ( defined($info) ) { $info = @$info[0]; if ( $info->{qualifier} eq "movie" ) { $self->status("perfect hit on movie \"$info->{title}\" [$info->{id}]"); $info->{matchLevel}="perfect"; return($info); } # note there is no 'qualifier' in TMDB data - we only have either 'movie' or 'tv' } } $self->status("no exact title/year hit on \"$mytitle\" (".($year eq ''?'-':$year).")"); } return(undef); } elsif ( $exact == 2 ) { # looking for first exact match on the title, don't have a year to compare for my $mytitle ( @titles ) { my ($closeMatches, $matchcount) = $self->getMovieCloseMatches($mytitle); if ( $matchcount == 1 ) { # this seems unlikely! for my $info (@$closeMatches) { if ( $self->tidy($mytitle) eq $self->tidy($info->{title}) ) { if ( $info->{qualifier} eq "movie" ) { $self->status("close enough hit on movie \"$info->{title}\" [$info->{id}] (since no 'date' field present)"); $info->{matchLevel}="close"; return($info); } # note there is no 'qualifier' in TMDB data - we only have either 'movie' or 'tv' } } } } return(undef); } # otherwise we're looking for a title match with a close year (within 2 years) # for my $mytitle ( @titles ) { my ($closeMatches, $matchcount) = $self->getMovieCloseMatches($mytitle); if ( $matchcount> 0 ) { # we traverse the hits twice, first looking for success, # then again to produce warnings about missed close matches # for my $info (@$closeMatches) { # within one year with title match good enough if ( $self->tidy($mytitle) eq $self->tidy($info->{title}) && $info->{year} ne '' ) { my $yearsOff=abs(int($info->{year})-$year); $info->{matchLevel}="close"; if ( $yearsOff <= 2 ) { my $showYear=int($info->{year}); if ( $info->{qualifier} eq "movie" ) { $self->status("close enough hit on movie \"$info->{title}\" (off by $yearsOff years)"); return($info); } elsif ( $info->{qualifier} eq "tv_series" ) { #$self->status("close enough hit on tv series \"$info->{title}\" (off by $yearsOff years)"); #return($info); } else { $self->error("$self->{moviedbIndex} responded with wierd entry for \"$info->{title}\""); $self->error("weird trailing qualifier \"$info->{qualifier}\""); $self->error("submit bug report to xmltv-devel\@lists.sf.net"); } # note there is no 'qualifier' in TMDB data - we only have either 'movie' or 'tv' } } } # if we found at least something, but nothing matched # produce warnings about missed, but close matches # for my $info (@$closeMatches) { # title match? if ( $self->tidy($mytitle) eq $self->tidy($info->{title}) && $info->{year} ne '' ) { my $yearsOff=abs(int($info->{year})-$year); if ( $yearsOff <= 2 ) { #die "internal error: key \"$info->{title}\" failed to be processed properly"; } elsif ( $yearsOff <= 5 ) { # report these as status $self->status("ignoring close, but not good enough hit on \"$info->{title}\" (off by $yearsOff years)"); } else { # report these as debug messages $self->debug("ignoring close hit on \"$info->{title}\" (off by $yearsOff years)"); } } else { $self->debug("ignoring close hit on \"$info->{title}\" (title did not match)"); } } $self->status("no close title/year hit on \"$mytitle\" (".($year eq ''?'-':$year).")"); } } #$self->status("failed to lookup \"$title ($year)\""); return(undef); } # find matching tv records on TMDB # sub findTVSeriesInfo($$) { my ($self, $title)=@_; # if using cache... if ( $self->{cacheLookups} ) { my $idInfo=$self->{cachedLookups}->{tv_series}->{$title}; if ( defined($idInfo) ) { #print STDERR "REF= (".ref($idInfo).")\n"; if ( $idInfo ne '' ) { return($idInfo); } return(undef); } } my @titles=@{alternativeTitles($title)}; my $idInfo; for my $mytitle ( @titles ) { # looking for matches on title my ($closeMatches, $matchcount) = $self->getTvCloseMatches($mytitle); # if there are multiple matches with the same 'title' then we don't know which one is right # e.g. Doctor Who (1963), Doctor Who (2005) # # TODO: fix this somehow? we could pick up the airdate from the tv prog but that is unlikely to # match the 'series' record ("first_air_date") on TMDB, so that won't help # if (scalar ( grep { $self->tidy($_->{title}) eq $self->tidy($mytitle) } @$closeMatches ) > 1) { $self->status("multiple hits on tv series \"$mytitle\""); last; } # ok there's only 1 match # get the matching entry # for my $info (@$closeMatches) { if ( $self->tidy($mytitle) eq $self->tidy($info->{title}) ) { $info->{matchLevel}="perfect"; if ( $info->{qualifier} eq "tv_series" ) { $idInfo=$info; $self->status("perfect hit on tv series \"$info->{key}\" [$info->{id}]"); last; } # note there is no 'qualifier' in TMDB data - we only have either 'movie' or 'tv' } } last if ( defined($idInfo) ); } # if using cache... if ( $self->{cacheLookups} ) { # flush cache after this lookup if its gotten too big if ( $self->{cachedLookups}->{tv_series}->{_cacheSize_} > $self->{cacheLookupSize} ) { delete($self->{cachedLookups}->{tv_series}); $self->{cachedLookups}->{tv_series}->{_cacheSize_}=0; } if ( defined($idInfo) ) { $self->{cachedLookups}->{tv_series}->{$title}=$idInfo; } else { $self->{cachedLookups}->{tv_series}->{$title}=""; } $self->{cachedLookups}->{tv_series}->{_cacheSize_}++; } if ( defined($idInfo) ) { return($idInfo); } else { #$self->status("failed to lookup tv series \"$title\""); return(undef); } } # apply the changes to the source record # # todo - ratings : not available in TMDB sadly # we could add the following (but who cares?) # todo - writer # todo - producer # todo - adapter ( 'job' = 'Teleplay' ) # sub applyFound($$$) { my ($self, $prog, $idInfo)=@_; # get the movie/tv details from TMDB my $details = $self->getMovieOrTvIdDetails($idInfo->{id}, $idInfo->{type}); $self->debugmore(5,'tdmb_match',$idInfo); $self->debugmore(5,'details',$details); my $title=$prog->{title}->[0]->[0]; $self->debug("augmenting $idInfo->{qualifier} \"$title\""); if ( $self->{updateDates} ) { my $date; # don't add dates to tv_series (only replace them) if ( $idInfo->{qualifier} eq "movie" || $idInfo->{qualifier} eq "video_movie" || $idInfo->{qualifier} eq "tv_movie" ) { $self->debug("adding 'date' field (\"$idInfo->{year}\") on \"$title\""); $date=int($idInfo->{year}); } else { #$self->debug("not adding 'date' field to $idInfo->{qualifier} \"$title\""); $date=undef; } if ( $self->{replaceDates} ) { if ( defined($prog->{date}) && defined($date) ) { $self->debug("replacing 'date' field"); delete($prog->{date}); $prog->{date}=$date; } } else { # only set date if not already defined if ( !defined($prog->{date}) && defined($date) ) { $prog->{date}=$date; } } } if ( $self->{removeYearFromTitles} ) { if ( $title =~ m/\s+\((19|20)\d\d\)\s*$/ ) { $self->debug("removing year from all 'title'"); my @list; if ( defined($prog->{title}) ) { for my $v (@{$prog->{title}}) { my $otitle = $v->[0]; if ( $v->[0] =~ s/\s+\((19|20)\d\d\)\s*$// ) { $self->debug("removing year from 'title' \"$otitle\" to \"$v->[0]\""); } push(@list, $v); } } $prog->{title}=\@list; } } if ( $self->{updateTitles} ) { if ( $idInfo->{title} ne $title ) { if ( $self->{replaceTitles} ) { $self->debug("replacing (all) 'title' from \"$title\" to \"$idInfo->{title}\""); delete($prog->{title}); } my @list; push(@list, [$idInfo->{title}, $idInfo->{lang}]); if ( defined($prog->{title}) ) { my $name=$idInfo->{title}; my $found=0; for my $v (@{$prog->{title}}) { if ( $self->tidy($v->[0]) eq $self->tidy($name) ) { $found=1; } else { push(@list, $v); } } } $prog->{title}=\@list; } } if ( $self->{updateURLs} ) { if ( $self->{replaceURLs} ) { if ( defined($prog->{url}) ) { $self->debug("replacing (all) 'url'"); delete($prog->{url}); } } # add url pointing to programme on www.themoviedb.org my $url2; if ( defined($details->{tmdb_id}) ) { $url2= $self->{wwwUrl} . ( $idInfo->{qualifier} =~ /movie/ ? 'movie' : 'tv' ) . "/" . $details->{tmdb_id}; } $self->debug("adding 'url' $url2") if $url2; # add url pointing to programme on www.imdb.com my $url; # # see if TMDB has an IMDb "tt" identifier # if ( defined($details->{imdb_id}) && ( $details->{imdb_id} =~ m/^tt\d*$/ ) ) { $url="https://www.imdb.com/title/".$details->{imdb_id}."/"; } else # no tt id found - revert to old 'search' url { $url=$idInfo->{key}; # encode the title $url=~s/([^a-zA-Z0-9_.-])/uc sprintf("%%%02x",ord($1))/oeg; $url="https://www.imdb.com/find?q=".$url."&s=tt&exact=true"; # possible altve url using 'search' instead of 'find', but there's no option for 'exact' hits only # https://www.imdb.com/search/title/?title=titanic&release_date=1995-01-01,1999-12-31&view=simple # c.f. https://www.imdb.com/search/title/ } $self->debug("adding 'url' $url"); if ( defined($prog->{url}) ) { my @rep; push(@rep, [ $url2, 'TMDB' ]) if $url2; push(@rep, [ $url, 'IMDb' ]) if $url; for (@{$prog->{url}}) { # skip urls for imdb.com that we're probably safe to replace if ( !m;^http://us.imdb.com/M/title-exact;o && !m;^https://www.imdb.com/find;o ) { push(@rep, $_); } } $prog->{url}=\@rep; } else { push(@{$prog->{url}}, [ $url2, 'TMDB' ]) if $url2; push(@{$prog->{url}}, [ $url, 'IMDb' ]) if $url; } } # squirrel away movie qualifier so it's first on the list of replacements my @categories; push(@categories, [$self->{categories}->{$idInfo->{qualifier}}, 'en']); if ( !defined($self->{categories}->{$idInfo->{qualifier}}) ) { die "how did we get here with an invalid qualifier '$idInfo->{qualifier}'"; } # now done above (so we can get IMDb tt id ### my $details=$self->getMovieIdDetails($idInfo->{id}); if ( $details->{noDetails} ) { # we don't have any details on this movie } else { # ---- update directors list if ( $self->{updateDirectors} && defined($details->{directors}) ) { # only update directors if we have exactly one or if # it's a movie of some kind if ( scalar(@{$details->{directors}}) == 1 || $idInfo->{qualifier} eq "movie" || $idInfo->{qualifier} eq "video_movie" || $idInfo->{qualifier} eq "tv_movie" ) { if ( $self->{replaceDirectors} ) { if ( defined($prog->{credits}->{director}) ) { $self->debug("replacing director(s)"); delete($prog->{credits}->{director}); } } # add top 3 billing directors from TMDB data # preserve all existing directors from the prog + de-dupe the list my @list; if ( $self->{updateCastImage} || $self->{updateCastUrl} ) { # add director image foreach (@{ $details->{directorsplus} }) { my $subels = {}; # add actor image # TODO : remove existing image(s) / avoid duplicates $subels->{image} = [[ $_->{imageurl}, {'system'=>'TMDB','type'=>'person'} ]] if $self->{updateCastImage} && $_->{imageurl} ne ''; # add actor url $subels->{url} = [[ $self->{wwwUrl} . 'person/' . $_->{id}, 'TMDB' ]] if $self->{updateCastUrl}; push(@list, [ $_->{name}, $subels ] ); } # merge and dedupe the lists from incoming xml + tmdb. Give TMDB entries priority. @list = uniquemulti( splice(@list,0,3), map{ ref($_) eq 'ARRAY' && scalar($_) > 1 ? $_ : [ $_ ] } @{ $prog->{credits}->{director} } ); # 'map' because uniquemulti needs an array @list = map{ ( ref($_) eq 'ARRAY' && scalar(@$_) == 1 ) ? shift @$_ : $_ } @list; # flatten any single-index arrays } else { # simple merge and dedupe @list = unique( splice(@{$details->{directors}},0,3), @{ $prog->{credits}->{director} } ); } # $prog->{credits}->{director}=\@list; } else { $self->debug("not adding 'director'"); } } # ---- update actors list if ( $self->{updateActors} && defined($details->{actors}) ) { if ( $self->{replaceActors} ) { if ( defined($prog->{credits}->{actor}) ) { $self->debug("replacing actor(s)"); delete($prog->{credits}->{actor}); } } # add top billing actors (default = 3) from TMDB # preserve all existing actors from the prog + de-dupe the list # my @list; if ( $self->{updateActorRole} || $self->{updateCastImage} || $self->{updateCastUrl} ) { foreach (@{ $details->{actorsplus} }) { # add character attribute to actor name my $character = ( $self->{updateActorRole} ? $_->{character} : '' ); my $subels = {}; # add actor image # TODO : remove existing image(s) / avoid duplicates $subels->{image} = [[ $_->{imageurl}, {'system'=>'TMDB','type'=>'person'} ]] if $self->{updateCastImage} && $_->{imageurl} ne ''; # add actor url $subels->{url} = [[ $self->{wwwUrl} . 'person/' . $_->{id}, 'TMDB' ]] if $self->{updateCastUrl}; push(@list, [ $_->{name}, $character, '', $subels ] ); } # merge and dedupe the lists from incoming xml + tmdb. Give TMDB entries priority. # note: will ignore 'role' attribute - i.e. de-dupe on 'name' only @list = uniquemulti( splice(@list,0,$self->{numActors}), map{ ref($_) eq 'ARRAY' && scalar($_) > 1 ? $_ : [ $_ ] } @{ $prog->{credits}->{actor} } ); # 'map' because uniquemulti needs an array @list = map{ ( scalar(@$_) == 3 && @$_[2] eq '' ) ? [ @$_[0], @$_[1] ] : $_ } @list; # remove blank 'image' values @list = map{ ( scalar(@$_) == 2 && @$_[1] eq '' ) ? @$_[0] : $_ } @list; # remove blank 'character' values @list = map{ ( ref($_) eq 'ARRAY' && scalar(@$_) == 1 ) ? shift @$_ : $_ } @list; # flatten any single-index arrays (as per the xmltv data struct) } else { # simple merge and dedupe @list = unique( splice(@{$details->{actors}},0,$self->{numActors}), @{ $prog->{credits}->{actor} } ); } # $prog->{credits}->{actor}=\@list; } # ---- update presenters list if ( $self->{updatePresentors} && defined($details->{presenter}) ) { if ( $idInfo->{qualifier} eq "tv_series" ) { # only do this for TV (not movies as 'presenter' might be a valid character) if ( $self->{replacePresentors} ) { if ( defined($prog->{credits}->{presenter}) ) { $self->debug("replacing presentor"); delete($prog->{credits}->{presenter}); } } $prog->{credits}->{presenter}=$details->{presenter}; } } # ---- update commentators list if ( $self->{updateCommentators} && defined($details->{commentator}) ) { if ( $idInfo->{qualifier} eq "tv_series" ) { # only do this for TV (not movies as 'commentator' might be a valid character) if ( $self->{replaceCommentators} ) { if ( defined($prog->{credits}->{commentator}) ) { $self->debug("replacing commentator"); delete($prog->{credits}->{commentator}); } } $prog->{credits}->{commentator}=$details->{commentator}; } } # ---- update guests list if ( $self->{updateGuests} && defined($details->{guest}) ) { if ( $idInfo->{qualifier} eq "tv_series" ) { # only do this for TV (not movies as 'guest' might be a valid character) if ( $self->{replaceGuests} ) { if ( defined($prog->{credits}->{guest}) ) { $self->debug("replacing guest"); delete($prog->{credits}->{guest}); } } $prog->{credits}->{guest}=$details->{guest}; } } # ---- update categories (genres) list if ( $self->{updateCategoriesWithGenres} ) { # deprecated? if ( defined($details->{genres}) ) { for (@{$details->{genres}}) { push(@categories, [$_, 'en']); } } } # if ( $self->{updateCategories} ) { if ( $self->{replaceCategories} ) { if ( defined($prog->{category}) ) { $self->debug("replacing (all) 'category'"); delete($prog->{category}); } } if ( defined($prog->{category}) ) { # merge and dedupe @categories = uniquemulti( @categories, @{$prog->{category}} ); # } $prog->{category}=\@categories; } # ---- update ratings (film classifications) if ( $self->{updateRatings} ) { if ( $self->{replaceRatings} ) { if ( defined($prog->{rating}) ) { $self->debug("replacing (all) 'rating'"); delete($prog->{rating}); } } if ( defined($details->{classifications}) ) { # we need to sort the classifications to ensure a consistent order from TMDB @{$details->{classifications}} = sort { $a->{system} cmp $b->{system} } @{$details->{classifications}}; # for (@{$details->{classifications}}) { push(@{$prog->{rating}}, [ $_->{rating}, $_->{system} ] ); } } } # ---- update star ratings if ( $self->{updateStarRatings} && defined($details->{ratingRank}) ) { # ignore the TMDB rating if there are too few votes (to avoid skewed data). # what's 'too few'...good question! if ( $details->{ratingVotes} >= $self->{minVotes} ) { if ( $self->{replaceStarRatings} ) { if ( defined($prog->{'star-rating'}) ) { $self->debug("replacing 'star-rating'"); delete($prog->{'star-rating'}); } unshift( @{$prog->{'star-rating'}}, [ $details->{ratingRank} . "/10", 'TMDB User Rating' ] ); } else { # add TMDB User Rating in front of all other star-ratings unshift( @{$prog->{'star-rating'}}, [ $details->{ratingRank} . "/10", 'TMDB User Rating' ] ); } } } # ---- update keywords if ( $self->{updateKeywords} ) { my @keywords; if ( defined($details->{keywords}) ) { for (@{$details->{keywords}}) { push(@keywords, [$_, 'en']); } } if ( $self->{replaceKeywords} ) { if ( defined($prog->{keywords}) ) { $self->debug("replacing (all) 'keywords'"); delete($prog->{keywords}); } } if ( defined($prog->{keyword}) ) { # merge and dedupe @keywords = unique( @keywords, @{$prog->{keyword}} ); # } $prog->{keyword}=\@keywords; } # ---- update desc (plot) if ( $self->{updatePlot} ) { # plot is held as a <desc> entity # if 'replacePlot' then delete all existing <desc> entities and add new # else add this plot as an additional <desc> entity # if ( $self->{replacePlot} ) { if ( defined($prog->{desc}) ) { $self->debug("replacing (all) 'desc'"); delete($prog->{desc}); } } if ( defined($details->{plot}) ) { # check it's not already there my $found = 0; for my $_desc ( @{$prog->{desc}} ) { $found = 1 if ( @{$_desc}[0] eq $details->{plot} ); } push @{$prog->{desc}}, [ $details->{plot}, 'en' ] if !$found; } } # ---- update runtime if ( $self->{updateRuntime} ) { if ( defined($details->{runtime}) ) { $prog->{length} = $details->{runtime} * 60; # XMLTV.pm only accepts seconds } } # ---- update reference id if ( $self->{updateContentId} ) { # remove existing values @{$prog->{'episode-num'}} = grep ( @$_[1] !~ /tmdb_id|imdb_id/, @{$prog->{'episode-num'}} ); # add new values if ( defined($details->{tmdb_id}) ) { push(@{$prog->{'episode-num'}}, [ $details->{tmdb_id}, 'tmdb_id' ] ); } if ( defined($details->{imdb_id}) && ( $details->{imdb_id} =~ m/^tt\d*$/ ) ) { push(@{$prog->{'episode-num'}}, [ $details->{imdb_id}, 'imdb_id' ] ); } } # ---- update image if ( $self->{updateImage} ) { if ( defined($details->{posterurl}) ) { if ( $details->{posterurl} =~ m|/w500/| ) { push @{$prog->{image}}, [ $details->{posterurl}, { type => 'poster', orient => 'P', size => 3, system => 'TMDB' } ]; } else { push @{$prog->{image}}, [ $details->{posterurl} ]; } } if ( defined($details->{backdropurl}) ) { if ( $details->{backdropurl} =~ m|/w780/| ) { push @{$prog->{image}}, [ $details->{backdropurl}, { type => 'backdrop', orient => 'L', size => 3, system => 'TMDB' } ]; } else { push @{$prog->{image}}, [ $details->{backdropurl} ]; } } # smaller versions if ( defined($details->{posterurl_sm}) ) { if ( $details->{posterurl_sm} =~ m|/w92/| ) { push @{$prog->{image}}, [ $details->{posterurl_sm}, { type => 'poster', orient => 'P', size => 1, system => 'TMDB' } ]; } else { push @{$prog->{image}}, [ $details->{posterurl_sm} ]; } } if ( defined($details->{backdropurl_sm}) ) { if ( $details->{backdropurl_sm} =~ m|/w300/| ) { push @{$prog->{image}}, [ $details->{backdropurl_sm}, { type => 'backdrop', orient => 'L', size => 2, system => 'TMDB' } ]; } else { push @{$prog->{image}}, [ $details->{backdropurl_sm} ]; } } } # ---- update reviews if ( $self->{updateReviews} ) { if ( $self->{replaceReviews} ) { if ( defined($prog->{review}) ) { $self->debug("replacing (all) 'reviews'"); delete($prog->{review}); } } if ( defined($details->{reviews}) ) { my $i=0; for (@{$details->{reviews}}) { last if ++$i > $self->{numReviews}; push @{$prog->{review}}, [ $_->{content}, { reviewer=>$_->{author}, source=>'TMDB', type=>'text' } ]; push @{$prog->{review}}, [ $_->{url}, { reviewer=>$_->{author}, source=>'TMDB', type=>'url' } ] if $_->{url} ne ''; } } } } return($prog); } # main entry point # augment program data with TMDB data # # TODO: try to identify whether programme is movie or tv (e.g. via categories?) # and process accordingly # sub augmentProgram($$$) { my ($self, $prog, $movies_only)=@_; $self->{stats}->{programCount}++; # assume first title in first language is the one we want. my $title=$prog->{title}->[0]->[0]; # if no date but a date is in the title then extract it if ( $self->{getYearFromTitles} ) { if ( !defined($prog->{date}) || $prog->{date} eq '' ) { ( $prog->{date} ) = $title =~ m/\s+\(((19|20)\d\d)\)$/; } } # try to work out if we are a movie or a tv prog # 1. categories includes any of 'Movie','Films',"Film" = movie # (but note 'Film' (singular) as that could be a film review talking about films e.g. "Film 2018" TODO: fix me) # 2. categories includes any of 'tv*' = tv_series # 3. date is '(19|20)xx' = movie # my $progtype = '?'; $progtype = 'movie' if ( defined($prog->{date}) && $prog->{date}=~m/^\d\d\d\d$/ ) or ( scalar ( grep { lc(@$_[0]) =~ /movies|movie|films|film/ } @{ $prog->{category} } ) >0 ); $progtype = 'tv_series' if ( defined($prog->{date}) && $prog->{date}=~m/^\d\d\d\d-\d\d-\d\d$/ ) or ( scalar ( grep { lc($_ -> [0]) =~ /^tv.*/ } @{ $prog->{category} } ) >0 ); $self->status("input: \"$title\" [$progtype]"); if ( defined($prog->{date}) && $prog->{date}=~m/^\d\d\d\d$/ && $progtype ne 'tv_series' ) { # for programs with dates we try: # - exact matches on movies (using title + year) # - exact matches on tv series (using title) # - close matches on movies (using title) # my ($id, $matchcount) = $self->findMovieInfo($title, $prog, $prog->{date}, 1); # exact match if (defined $matchcount && $matchcount > 1) { $self->status("failed to find a sole match for movie \"$title".($title=~m/\s+\((19|20)\d\d\)/?'':" ($prog->{date})")."\""); return(undef); } if ( !defined($id) ) { if ( !$movies_only && $progtype ne 'movie' ) { $id = $self->findTVSeriesInfo($title); # match tv series } if ( !defined($id) ) { ($id, $matchcount) = $self->findMovieInfo($title, $prog, $prog->{date}, 0); # close match } } if ( defined($id) ) { $self->{stats}->{$id->{matchLevel}."Matches"}++; $self->{stats}->{$id->{matchLevel}}->{$id->{qualifier}}++; return($self->applyFound($prog, $id)); } $self->status("failed to find a match for movie \"$title".($title=~m/\s+\((19|20)\d\d\)/?'':" ($prog->{date})")."\""); return(undef); } if ( !$movies_only ) { # for programs without dates we try: # - exact matches on tv series (using title) # - close matches on movie # if ( $progtype ne 'movie' ) { my $id=$self->findTVSeriesInfo($title); if ( defined($id) ) { $self->{stats}->{$id->{matchLevel}."Matches"}++; $self->{stats}->{$id->{matchLevel}}->{$id->{qualifier}}++; return($self->applyFound($prog, $id)); } } if ( $progtype eq 'movie' ) { # this has hard to support 'close' results, unless we know # for certain we're looking for a movie (ie duration etc) my ($id, $matchcount) = $self->findMovieInfo($title, $prog, undef, 2); # any title match if ( defined($id) ) { $self->{stats}->{$id->{matchLevel}."Matches"}++; $self->{stats}->{$id->{matchLevel}}->{$id->{qualifier}}++; return($self->applyFound($prog, $id)); } } $self->status("failed to find a match for show \"$title\""); } return(undef); } # print some stats # # TODO - add in stats on other things added (urls ?, actors, directors,categories) # separate out from what was added or updated # sub getStatsLines($) { my $self=shift; my $totalChannelsParsed=shift; my $endTime=time(); my %stats=%{$self->{stats}}; my $ret=sprintf("Checked %d programs, on %d channels\n", $stats{programCount}, $totalChannelsParsed); for my $cat (sort keys %{$self->{categories}}) { $ret.=sprintf(" found %d %s titles", $stats{perfect}->{$cat}+$stats{close}->{$cat}, $self->{categories}->{$cat}); if ( $stats{close}->{$cat} != 0 ) { if ( $stats{close}->{$cat} == 1 ) { $ret.=sprintf(" (%d was not perfect)", $stats{close}->{$cat}); } else { $ret.=sprintf(" (%d were not perfect)", $stats{close}->{$cat}); } } $ret.="\n"; } $ret.=sprintf(" augmented %.2f%% of the programs, parsing %.2f programs/sec\n", ($stats{programCount}!=0)?(($stats{perfectMatches}+$stats{closeMatches})*100)/$stats{programCount}:0, ($endTime!=$stats{startTime} && $stats{programCount} != 0)? $stats{programCount}/($endTime-$stats{startTime}):0); return($ret); } 1; ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/TMDB/�������������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0014156�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/TMDB/API.pm�������������������������������������������������������������������������0000664�0000000�0000000�00000010164�15000742332�0015127�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Lightweight package to retrieve movie/tv programme data from The Movie Database (http://www.themoviedb.org/ ) # # This is a custom version of the CPAN package : # WWW::TMDB::API - TMDb API (http://api.themoviedb.org) client # Version 0.04 (2012) # Author Maria Celina Baratang, <maria at zambale.com> # https://metacpan.org/pod/WWW::TMDB::API # # Modified for XMLTV use to # - fix broken methods # - add methods for TV programmes, and Configuration # - 'version' changed to be 0.05 # # Modifications: Geoff Westcott, December 2021 # package XMLTV::TMDB::API; # Package changes for XMLTV # - add new namespace for package Tv.pm and Config.pm # - remove ID= url parameter (since it's already added to the URL path) # - add http response to return array # - add 'soft' param to constructor to return http errors instead of carp # use 5.006; use strict; use warnings; use Carp; our $VERSION = '0.05'; use utf8; use LWP::UserAgent; use HTTP::Request; use JSON; use URI; our @namespaces = qw( Person Movie Tv Config ); for (@namespaces) { my $package = __PACKAGE__ . "::$_"; my $name = "\L$_"; eval qq( use $package; sub $name { my \$self = shift; if ( \$self->{'_$name'} ) { return \$self->{'_$name'}; }else{ \$self->{'_$name'} = $package->new( api => \$self ); } }; package $package; sub api { return shift->{api}; }; sub new { my ( \$class, \%params ) = \@_; my \$self = bless \\\%params, \$class; \$self->{api} = \$params{api}; return \$self; }; 1; ); croak "Cannot create namespace $name: $@\n" if $@; } sub send_api { my ( $self, $command, $params_spec, $params ) = @_; $self->check_parameters( $params_spec, $params ); my $url = $self->url( $command, $params ); my $request = HTTP::Request->new( GET => $url ); $request->header( 'Accept' => 'application/json' ); my $json_response = $self->{ua}->request($request); if ( $json_response->is_success ) { return [ decode_json( $json_response->content() ), { 'code' => $json_response->code(), 'msg' => $json_response->status_line, 'url' => $url } ]; } elsif ( $json_response->is_error && $self->{soft} ) { return [ {}, { 'code' => $json_response->code(), 'msg' => $json_response->status_line, 'url' => $url } ]; } else { croak sprintf( "%s returned by %s", $json_response->status_line, $url ); } } # Checks items that will be sent to the API($input) # $params - an array that identifies valid parameters # example : # {'ID' => 1 }, 1- field is required, 0- field is optional sub check_parameters { my $self = shift; my ( $params, $input ) = @_; foreach my $k ( keys(%$params) ) { croak "Required parameter $k missing." if ( $params->{$k} == 1 and !defined $input->{$k} ); } foreach my $k ( keys(%$input) ) { croak "Unknown parameter - $k." if ( !defined $params->{$k} ); } } sub url { my $self = shift; my ( $command, $params ) = @_; my $url = new URI( $self->{url} ); $url->path_segments( $self->{ver}, @$command ); $params->{api_key} = $self->{api_key}; delete $params->{ID} if defined $params->{ID}; $url->query_form($params); return $url->as_string(); } sub new { my $class = shift; my (%params) = @_; croak "Required parameter api_key not provided." unless $params{api_key}; if ( !defined $params{ua} ) { $params{ua} = LWP::UserAgent->new( 'agent' => "Perl-WWW-TMDB-API/$VERSION", ); } else { croak "LWP::UserAgent expected." unless $params{ua}->isa('LWP::UserAgent'); } my $self = { api_key => $params{api_key}, ua => $params{ua}, ver => '3', url => 'http://api.themoviedb.org', soft => (defined $params{soft} ? $params{soft} : 0), }; bless $self, $class; return $self; } 1; # End of XMLTV::TMDB::API ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/TMDB/API/���������������������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0014567�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/TMDB/API/Config.pm������������������������������������������������������������������0000664�0000000�0000000�00000001507�15000742332�0016335�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Lightweight package to retrieve movie/tv programme data from The Movie Database (http://www.themoviedb.org/ ) # # This is a custom version of the CPAN package : # WWW::TMDB::API - TMDb API (http://api.themoviedb.org) client # Version 0.04 (2012) # Author Maria Celina Baratang, <maria at zambale.com> # https://metacpan.org/pod/WWW::TMDB::API # # Modified for XMLTV use to # - fix broken methods # - add methods for TV programmes, and Configuration # - 'version' changed to be 0.05 # # Modifications: Geoff Westcott, December 2021 # # Package changes for XMLTV # - new package for XMLTV - not in WWW::TMDB::API # package XMLTV::TMDB::API::Config; use strict; use warnings; our $VERSION = '0.05'; sub configuration { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'configuration' ] ); } 1; �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/TMDB/API/Movie.pm�������������������������������������������������������������������0000664�0000000�0000000�00000005617�15000742332�0016215�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Lightweight package to retrieve movie/tv programme data from The Movie Database (http://www.themoviedb.org/ ) # # This is a custom version of the CPAN package : # WWW::TMDB::API - TMDb API (http://api.themoviedb.org) client # Version 0.04 (2012) # Author Maria Celina Baratang, <maria at zambale.com> # https://metacpan.org/pod/WWW::TMDB::API # # Modified for XMLTV use to # - fix broken methods # - add methods for TV programmes, and Configuration # - 'version' changed to be 0.05 # # Modifications: Geoff Westcott, December 2021 # # Package changes for XMLTV # - search() - now 'movie'(not 'movies') # - search() - added 'year' attribute # - casts() - now called 'credits' # - images() - add 'include_image_language' # - latest() - endpoint is now 'movie/latest' # - releases()- now called release_dates() # - trailers()- now called videos() # - info() - add 'append_to_response' param # - reviews()- new endpoint package XMLTV::TMDB::API::Movie; use strict; use warnings; our $VERSION = '0.05'; sub info { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'movie', $params{ID} ], { ID => 1, language => 0, append_to_response =>0 }, \%params ); } sub search { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'search', 'movie' ], { query => 1, page => 0, language => 0, 'include_adult' => 0, year => 0 }, \%params ); } sub alternative_titles { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'movie', $params{ID}, 'alternative_titles' ], { ID => 1, country => 0 }, \%params ); } sub credits { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'movie', $params{ID}, 'credits' ], { ID => 1 }, \%params ); } sub images { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'movie', $params{ID}, 'images' ], { ID => 1, language => 0, include_image_language => 0 }, \%params ); } sub keywords { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'movie', $params{ID}, 'keywords' ], { ID => 1 }, \%params ); } sub release_dates { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'movie', $params{ID}, 'release_dates' ], { ID => 1 }, \%params ); } sub translations { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'movie', $params{ID}, 'translations' ], { ID => 1 }, \%params ); } sub videos { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'movie', $params{ID}, 'videos' ], { ID => 1, language => 1 }, \%params ); } sub reviews { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'movie', $params{ID}, 'reviews' ], { ID => 1, language => 1 }, \%params ); } sub latest { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'movie', 'latest' ] ); } 1; �����������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/TMDB/API/Person.pm������������������������������������������������������������������0000664�0000000�0000000�00000002525�15000742332�0016377�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Lightweight package to retrieve movie/tv programme data from The Movie Database (http://www.themoviedb.org/ ) # # This is a custom version of the CPAN package : # WWW::TMDB::API - TMDb API (http://api.themoviedb.org) client # Version 0.04 (2012) # Author Maria Celina Baratang, <maria at zambale.com> # https://metacpan.org/pod/WWW::TMDB::API # # Modified for XMLTV use to # - fix broken methods # - add methods for TV programmes, and Configuration # - 'version' changed to be 0.05 # # Modifications: Geoff Westcott, December 2021 # # Package changes for XMLTV # - info() - add 'append_to_response' param # package XMLTV::TMDB::API::Person; use strict; use warnings; our $VERSION = '0.05'; sub info { my $self = shift; my (%params) = @_; $self->api->send_api( [ 'person', $params{ID} ], { ID => 1, append_to_response =>0 }, \%params ); } sub credits { my $self = shift; my (%params) = @_; $self->api->send_api( [ 'person', $params{ID}, 'credits' ], { ID => 1, language => 0 }, \%params ); } sub images { my $self = shift; my (%params) = @_; $self->api->send_api( [ 'person', $params{ID}, 'images' ], { ID => 1 }, \%params ); } sub search { my $self = shift; my (%params) = @_; $self->api->send_api( [ 'search', 'person' ], { query => 1, page => 0 }, \%params ); } 1; ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/TMDB/API/Tv.pm����������������������������������������������������������������������0000664�0000000�0000000�00000004376�15000742332�0015530�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Lightweight package to retrieve movie/tv programme data from The Movie Database (http://www.themoviedb.org/ ) # # This is a custom version of the CPAN package : # WWW::TMDB::API - TMDb API (http://api.themoviedb.org) client # Version 0.04 (2012) # Author Maria Celina Baratang, <maria at zambale.com> # https://metacpan.org/pod/WWW::TMDB::API # # Modified for XMLTV use to # - fix broken methods # - add methods for TV programmes, and Configuration # - 'version' changed to be 0.05 # # Modifications: Geoff Westcott, December 2021 # # Package changes for XMLTV # - new package for XMLTV - not in WWW::TMDB::API # package XMLTV::TMDB::API::Tv; use strict; use warnings; our $VERSION = '0.05'; sub info { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'tv', $params{ID} ], { ID => 1, language => 0, append_to_response =>0 }, \%params ); } sub search { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'search', 'tv' ], { query => 1, page => 0, language => 0, 'include_adult' => 0 }, \%params ); } sub alternative_titles { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'tv', $params{ID}, 'alternative_titles' ], { ID => 1, country => 0 }, \%params ); } sub credits { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'tv', $params{ID}, 'credits' ], { ID => 1 }, \%params ); } sub images { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'tv', $params{ID}, 'images' ], { ID => 1, language => 0 }, \%params ); } sub keywords { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'tv', $params{ID}, 'keywords' ], { ID => 1 }, \%params ); } sub translations { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'tv', $params{ID}, 'translations' ], { ID => 1 }, \%params ); } sub content_ratings { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'tv', $params{ID}, 'content_ratings' ], { ID => 1 }, \%params ); } sub reviews { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'tv', $params{ID}, 'reviews' ], { ID => 1, language => 1 }, \%params ); } sub latest { my $self = shift; my (%params) = @_; $self->{api}->send_api( [ 'tv', 'latest' ] ); } 1; ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/TZ.pm�������������������������������������������������������������������������������0000664�0000000�0000000�00000011503�15000742332�0014323�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Miscellaneous timezone routines. The code in DST.pm builds on # these for handling summer time conventions. This should # probably be moved into Date::Manip somehow. package XMLTV::TZ; use Carp; use Date::Manip; # no Date_Init(), that can be done by the app use XMLTV::Date; # Won't Memoize, you can do that yourself. use base 'Exporter'; our @EXPORT_OK; @EXPORT_OK = qw(gettz ParseDate_PreservingTZ tz_to_num parse_local_date offset_to_gmt); # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } # gettz() # # Parameters: unparsed date string # Returns: timezone (a substring), or undef # # We just pick up anything that looks like a timezone. # sub gettz($) { croak 'usage: gettz(unparsed date string)' if @_ != 1; local $_ = shift; croak 'undef argument to gettz()' if not defined; /\s([A-Z]{1,4})$/ && return $1; /\s([+-]\d\d:?(\d\d)?)$/ && return $1; return undef; } # ParseDate_PreservingTZ() # # A wrapper for Date::Manip's ParseDate() that makes sure the date is # stored in the timezone it was given in. That's helpful when you # want to produce human-readable output and the user expects to see # the same timezone going out as went in. # sub ParseDate_PreservingTZ($) { croak 'usage: ParseDate_PreservingTZ(unparsed date string)' if @_ != 1; my $u = shift; my $p = ParseDate($u); die "cannot parse $u" if not $p; my $tz = gettz($u) || 'UTC'; my $ltz=Date_TimeZone(); # avoid bug in Date::Manip 6.05 $ltz=$tz if $ltz eq "1"; # if Date_TimeZone returns a bad value, use something ok # print STDERR "date $u parsed to $p (timezone read as $tz)\n"; $p = Date_ConvTZ($p, offset_to_gmt($ltz), offset_to_gmt($tz)); # print STDERR "...converted to $p\n"; return $p; } # Date::Manip version 6 has problems with +nnnn offsets # It seems to treat +0000 as equivalent to "Europe/London", meaning that during DST +0000 actually refers to GMT + 1 hour. # However, a timezone of etc/gmt+1 will always work. # Using this function on arguments to Date_ConvTZ should work around this bug. sub offset_to_gmt($) { my $tz = shift; return $tz unless $tz =~ /^([+-])0(\d)00/; if ($Date::Manip::VERSION >= 6) { if ($2 == 0) { $tz = "etc/gmt"; } else { $tz = "etc/gmt$1$2"; } } return $tz; } # tz_to_num() # # Turn a timezone string into a numeric form. For example turns 'CET' # into '+0100'. If the timezone is already numeric it's unchanged. # # Throws an exception if the timezone is not recognized. # sub tz_to_num( $ ) { my $tz = shift; # It should be possible to use numeric timezones and have them # come out unchanged. But due to a bug in Date::Manip, '+0100' is # treated as equivalent to 'UTC' by (WTF?) and we have to # special-case numeric timezones. # return $tz if $tz =~ /^[+-]?\d\d:?(?:\d\d)?$/; # To convert to a number we parse a date with this timezone and # then compare against the same date with UTC. # my $date_str = '2000-08-01 00:00:00'; # arbitrary my $base = parse_date("$date_str UTC"); t "parsed '$date_str UTC' as $base"; my $d = parse_date("$date_str $tz"); t "parsed '$date_str $tz' as $d"; my $err; my $delta = DateCalc($d, $base, \$err); die "error code from DateCalc: $err" if defined $err; # A timezone difference must be less than one day, and must be a # whole number of minutes. # my @df = Delta_Format($delta, 0, "%hv", "%Mv"); return sprintf('%s%02d%02d', ($df[0] < 0) ? '-' : '+', abs($df[0]), $df[1]); } # Date::Manip seems to have difficulty with changes of timezone: if # you parse some dates in a local timezone then do # Date_Init('TZ=UTC'), the existing dates are not changed, so # comparisons with later parsed dates (in UTC) will be wrong. Script # to reproduce the bug: # # #!/usr/bin/perl -w # use Date::Manip; # # First parse a date in the timezone +0100. # Date_Init('TZ=+0100'); # my $a = ParseDate('2000-01-01 00:00:00'); # # Now parse another one, in timezone +0000. # Date_Init('TZ=+0000'); # my $b = ParseDate('2000-01-01 00:00:00'); # # The two dates should differ by one hour. # print Date_Cmp($a, $b), "\n"; # # The script should print -1 but it prints 0. # # NB, use this function _before_ changing the default timezone to UTC, # if you want to parse some dates in the user's local timezone! # # Throws an exception on error. # sub parse_local_date( $ ) { my $d = shift; # local $Log::TraceMessages::On = 1; t 'parse_local_date() parsing: ' . d $d; my $pd = ParseDate($d); t 'ParseDate() returned: ' . d $pd; die "cannot parse date $d" if not $pd; my $r = Date_ConvTZ($pd, offset_to_gmt(Date_TimeZone()), 'UTC'); t 'converted into UTC: ' . d $r; return $r; } 1; ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/Usage.pm����������������������������������������������������������������������������0000664�0000000�0000000�00000002606�15000742332�0015036�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# A simple package to provide usage messages. Example # # use XMLTV::Usage <<END # usage: $0 [--help] [--whatever] FILES... # END # ; # # Then the usage() subroutine will print the message you gave to # stderr and exit with failure. An optional Boolean argument to # usage(), if true, will make it a 'help message', which is the same # except it prints to stdout and exits successfully. # # Alternatively, if your usage message is not known at compile time, # you can pass it as a string to usage(). In this case you need two # arguments: the 'is help' flag mentioned above, and the message. # # It's up to you to call the usage() subroutine, I've thought about # processing --help with a check_argv() routine in this module, but # some programs have different help messages depending on what other # options were given. package XMLTV::Usage; use base 'Exporter'; our @EXPORT = qw(usage); my $msg; sub import( @ ) { if (@_ == 1) { # No message specifed at import. } elsif (@_ == 2) { $msg = pop; } else { die "usage: use XMLTV::Usage [usage-message]"; } goto &Exporter::import; } sub usage( ;$$ ) { my $is_help = shift; my $got_msg = shift; my $m = (defined $got_msg) ? $got_msg : $msg; die "need to 'import' this module to set message" if not defined $m; if ($is_help) { print $m; exit(0); } else { print STDERR $m; exit(1); } } 1; ��������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/ValidateFile.pm���������������������������������������������������������������������0000664�0000000�0000000�00000035533�15000742332�0016330�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������package XMLTV::ValidateFile; use strict; BEGIN { use Exporter (); our (@ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS); @ISA = qw(Exporter); @EXPORT = qw( ); %EXPORT_TAGS = ( ); # eg: TAG => [ qw!name1 name2! ], @EXPORT_OK = qw/LoadDtd ValidateFile/; } our @EXPORT_OK; use XML::LibXML; use File::Slurp qw/read_file/; use XMLTV::Supplement qw/GetSupplement/; our $REQUIRE_CHANNEL_ID=1; my( $dtd, $parser ); =head1 NAME XMLTV::ValidateFile - Validates an XMLTV file =head1 DESCRIPTION Utility library that validates that a file is correct according to http://wiki.xmltv.org/index.php/XMLTVFormat. =head1 EXPORTED FUNCTIONS All these functions are exported on demand. =over 4 =cut =item LoadDtd Load the xmltv dtd. Takes a single parameter which is the name of the xmltv dtd file. LoadDtd must be called before ValidateFile can be called. =cut sub LoadDtd { my( $dtd_file ) = @_; my $dtd_str = read_file($dtd_file) or die "Failed to read $dtd_file"; $dtd = XML::LibXML::Dtd->parse_string($dtd_str); } =item ValidateFile Validate that a file is valid according to the XMLTV dtd and try to check that it contains valid information. ValidateFile takes a filename as parameter and optionally also a day and an offset and prints error messages to STDERR. ValidateFile returns a list of errors that it found with the file. Each error takes the form of a keyword: ValidateFile checks the following: =over =item notwell The file is not well-formed XML. =item notvalid The file does not follow the XMLTV DTD. =item invalidid An xmltvid does not look like a proper id, i.e. it does not match /^[-a-zA-Z0-9]+(\.[-a-zA-Z0-9]+)+$/. =item duplicateid More than one channel-entry found for a channelid. =item unknownid No channel-entry found for a channelid that is used in a programme-entry. =item noprogrammes No programme entries were found in the file. =item emptytitle A programme entry with an empty or missing title was found. =item emptydescription A programme entry with an empty desc-element was found. The desc-element shall be omitted if there is no description. =item badstart A programme entry with an invalid start-time was found. =item badstop A programme entry with an invalid stop-time was found. =item badepisode A programme entry with an invalid episode number was found. =item missingtimezone The start/stop time for a programme entry does not include a timezone. =item invalidtimezone The start/stop time for a programme entry contains an invalid timezone. =item badiso8859 The file is encoded in iso-8859 but contains characters that have no meaning in iso-8859 (or are control characters). If it's iso-8859-1 (aka Latin 1) it might be some characters in windows-1252 encoding. =item badutf8 The file is encoded in utf-8 but contains characters that look strange. 1) Mis-encoded single characters represented with [EF][BF][BD] bytes 2) Mis-encoded single characters represented with [C3][AF][C2][BF][C2][BD] bytes 3) Mis-encoded single characters in range [C2][80-9F] =item badentity The file contains one or more undefined XML entities. =back If no errors are found, an empty list is returned. =cut my %errors; my %timezoneerrors; sub ValidateFile { my( $file ) = @_; if( not defined( $parser ) ) { $parser = XML::LibXML->new(); $parser->line_numbers(1); } if( not defined( $dtd ) ) { my $dtd_str = GetSupplement( undef, 'xmltv.dtd'); $dtd = XML::LibXML::Dtd->parse_string( $dtd_str ); } %errors = (); my $doc; eval { $doc = $parser->parse_file( $file ); }; if ( $@ ) { w( "The file is not well-formed xml:\n$@ ", 'notwell'); return (keys %errors); } eval { $doc->validate( $dtd ) }; if ( $@ ) { w( "The file is not valid according to the xmltv dtd:\n $@", 'notvalid' ); return (keys %errors); } if( $doc->encoding() =~ m/^iso-8859-\d+$/i ) { verify_iso8859xx( $file, $doc->encoding() ); } elsif( $doc->encoding() =~ m/^utf-8$/i ) { verify_utf8( $file ); } verify_entities( $file ); my $w = sub { my( $p, $msg, $id ) = @_; w( "Line " . $p->line_number() . " $msg", $id ); }; my %channels; my $ns = $doc->find( "//channel" ); foreach my $ch ($ns->get_nodelist) { my $channelid = $ch->findvalue('@id'); my $display_name = $ch->findvalue('display-name/text()'); $w->( $ch, "Invalid channel-id '$channelid'", 'invalidid' ) if $channelid !~ /^[-a-zA-Z0-9]+(\.[-a-zA-Z0-9]+)+$/; $w->( $ch, "Duplicate channel-tag for '$channelid'", 'duplicateid' ) if defined( $channels{$channelid} ); $channels{$channelid} = 0; } $ns = $doc->find( "//programme" ); if ($ns->size() == 0) { w( "No programme entries found.", 'noprogrammes' ); return (keys %errors); } foreach my $p ($ns->get_nodelist) { my $channelid = $p->findvalue('@channel'); my $start = $p->findvalue('@start'); my $stop = $p->findvalue('@stop'); my $title = $p->findvalue('title/text()'); my $desc; $desc = $p->findvalue('desc/text()') if $p->findvalue( 'count(desc)' ); my $xmltv_episode = $p->findvalue('episode-num[@system="xmltv_ns"]' ); if ($REQUIRE_CHANNEL_ID and not exists( $channels{$channelid} )) { $w->( $p, "Channel '$channelid' does not have a <channel>-entry.", 'unknownid' ); $channels{$channelid} = 0; } $channels{$channelid}++; $w->( $p, "Empty title", 'emptytitle' ) if $title =~ /^\s*$/; $w->( $p, "Empty description", 'emptydescription' ) if defined($desc) and $desc =~ /^\s*$/; $w->( $p, "Invalid start-time '$start'", 'badstart' ) if not verify_time( $start ); $w->( $p, "Invalid stop-time '$stop'", 'badstop' ) if $stop ne "" and not verify_time( $stop ); if( $xmltv_episode =~ /\S/ ) { $w->($p, "Invalid episode-number '$xmltv_episode'", 'badepisode' ) if $xmltv_episode !~ /^\s*\d* (\s* \/ \s*\d+)? \s* \. \s*\d* (\s* \/ \s*\d+)? \s* \. \s*\d* (\s* \/ \s*\d+)? \s* $/x; } } foreach my $channel (keys %channels) { if ($channels{$channel} == 0) { w( "No programme entries found for $channel", 'channelnoprogramme' ); } } return (keys %errors); } sub verify_time { my( $timestamp ) = @_; # $tz is optional per the XMLTV DTD my( $date, $time, $tz ) = ($timestamp =~ /^(\d{8})(\d{4,6})(\s+([A-Z]+|[+-]\d{4}))?$/ ); return 0 unless defined $date; return 0 unless defined $time; return 1; } sub verify_iso8859xx { # code points not used in iso-8859 according to http://de.wikipedia.org/wiki/ISO_8859 my %unused_iso8859 = ( 'iso-8859-1' => undef, 'iso-8859-2' => undef, 'iso-8859-3' => '\xa5\xae\xbe\xc3\xd0\xe3\xf0', 'iso-8859-4' => undef, 'iso-8859-5' => undef, 'iso-8859-6' => '\xa1-\xa3\xa5-\xab\xae-\xba\xbc-\xbe\xc0\xdb-\xdf\xf3-xff', 'iso-8859-7' => '\xae\xd2\xff', 'iso-8859-8' => '\xa1\xbf-\xde\xfb-\xfc\xff', 'iso-8859-9' => undef, 'iso-8859-10' => undef, 'iso-8859-11' => '\xdb-\xde\xfc-\xff', 'iso-8859-12' => undef, 'iso-8859-13' => undef, 'iso-8859-14' => undef, 'iso-8859-15' => undef, ); # code points of unusual control characters used in iso-8859 according to http://de.wikipedia.org/wiki/ISO_8859 my %unusual_iso8859 = ( 'iso-8859-1' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-2' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-3' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-4' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-5' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-6' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-7' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-8' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-9' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-10' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-11' => '\x00-\x08\x0b-\x1f\x7f-\xa0', 'iso-8859-12' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-13' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-14' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', 'iso-8859-15' => '\x00-\x08\x0b-\x1f\x7f-\xa0\xad', ); my( $filename, $encoding ) = @_; $encoding = lc( $encoding ); my $file_str = read_file($filename); my $unusual = $unusual_iso8859{$encoding}; my $unused = $unused_iso8859{$encoding}; if( defined( $unusual ) ) { if( $file_str =~ m/[$unusual]+/ ) { my ($hintpre, $hint, $hintpost) = ( $file_str =~ m/(.{0,15})([$unusual]+)(.{0,15})/ ); w( "file contains unexpected control characters" . "\nlook here \"" . $hintpre . $hint . $hintpost . "\"" . sprintf( "\n%*s", 12+length( $hintpre ) , "^" ) , 'badiso8859' ); } } if( defined( $unused ) ) { if( $file_str =~ m/[$unused]+/ ) { my ($hintpre, $hint, $hintpost) = ( $file_str =~ m/(.{0,15})([$unused]+)(.{0,15})/ ); w( "file contains bytes without meaning in " . $encoding . "\nlook here \"" . $hintpre . $hint . $hintpost . "\"" . sprintf( "\n%*s", 12+length( $hintpre ) , "^" ) , 'badiso8859' ); } } return 1; } # inspired by utf8 fixups in _uk_rt sub verify_utf8 { my( $filename ) = @_; my $file_str = read_file($filename); # 1) Mis-encoded single characters represented with [EF][BF][BD] bytes if( $file_str =~ m/\xEF\xBF\xBD]/ ) { my ($hintpre, $hint, $hintpost) = ( $file_str =~ m/(.{0,15})(\xEF\xBF\xBD)(.{0,15})/ ); w( "file contains misencoded characters" . "\nlook here \"" . $hintpre . $hint . $hintpost . "\"" . sprintf( "\n%*s", 11+length( $hintpre ) , "^^^" ) , 'badutf8' ); } # 2) Mis-encoded single characters represented with [C3][AF][C2][BF][C2][BD] bytes if( $file_str =~ m/\xC3\xAF\xC2\xBF\xC2\xBD/ ) { my ($hintpre, $hint, $hintpost) = ( $file_str =~ m/(.{0,15})(\xC3\xAF\xC2\xBF\xC2\xBD)(.{0,15})/ ); w( "file contains misencoded characters" . "\nlook here \"" . $hintpre . $hint . $hintpost . "\"" . sprintf( "\n%*s", 11+length( $hintpre ) , "^^^^^^" ) , 'badutf8' ); } # 3) Mis-encoded single characters in range [C2][80-9F] if( $file_str =~ m/\xC2[\x80-\x9F]/ ) { my ($hintpre, $hint, $hintpost) = ( $file_str =~ m/(.{0,15})(\xC2[\x80-\x9F])(.{0,15})/ ); w( "file contains unexpected control characters, misencoded windows-1252?" . "\nlook here \"" . $hintpre . $hint . $hintpost . "\"" . sprintf( "\n%*s", 11+length( $hintpre ) , "^^" ) , 'badutf8' ); } # 4) The first two (C0 and C1) could only be used for overlong encoding of basic ASCII characters. if( $file_str =~ m/[\xC0-\xC1]/ ) { my ($hintpre, $hint, $hintpost) = ( $file_str =~ m/(.{0,15})([\xC0-\xC1])(.{0,15})/ ); w( "file contains bytes that should never appear in utf-8" . "\nlook here \"" . $hintpre . $hint . $hintpost . "\"" . sprintf( "\n%*s", 11+length( $hintpre ) , "^" ) , 'badutf8' ); } # 5) start bytes of sequences that could only encode numbers larger than the 0x10FFFF limit of Unicode. if( $file_str =~ m/[\xF5-\xFF]/ ) { my ($hintpre, $hint, $hintpost) = ( $file_str =~ m/(.{0,15})([\xF5-\xFF])(.{0,15})/ ); w( "file contains bytes that should never appear in utf-8" . "\nlook here \"" . $hintpre . $hint . $hintpost . "\"" . sprintf( "\n%*s", 11+length( $hintpre ) , "^" ) , 'badutf8' ); } # 6) first continuation byte missing after start of sequence if( $file_str =~ m/[\xC2-\xF4][\x00-\x7F\xC0-\xFF]/ ) { my ($hintpre, $hint, $hintpost) = ( $file_str =~ m/(.{0,15})([\xC2-\xF4][\x00-\x7F\xC0-\xFF])(.{0,15})/ ); w( "file contains an utf-8 sequence with missing continuation bytes" . "\nlook here \"" . $hintpre . $hint . $hintpost . "\"" . sprintf( "\n%*s", 11+length( $hintpre )+1 , "^" ) , 'badutf8' ); } # 7) second continuation byte missing after start of sequence if( $file_str =~ m/[\xE0-\xF4][\x80-\xBF][\x00-\x7F\xC0-\xFF]/ ) { my ($hintpre, $hint, $hintpost) = ( $file_str =~ m/(.{0,15})([\xE0-\xF4][\x80-\xBF][\x00-\x7F\xC0-\xFF])(.{0,15})/ ); w( "file contains an utf-8 sequence with missing continuation bytes" . "\nlook here \"" . $hintpre . $hint . $hintpost . "\"" . sprintf( "\n%*s", 11+length( $hintpre )+2 , "^" ) , 'badutf8' ); } # 8) third continuation byte missing after start of sequence if( $file_str =~ m/[\xF0-\xF4][\x80-\xBF][\x80-\xBF][\x00-\x7F\xC0-\xFF]/ ) { my ($hintpre, $hint, $hintpost) = ( $file_str =~ m/(.{0,15})([\xF0-\xF4][\x80-\xBF][\x80-\xBF][\x00-\x7F\xC0-\xFF])(.{0,15})/ ); w( "file contains an utf-8 sequence with missing continuation bytes" . "\nlook here \"" . $hintpre . $hint . $hintpost . "\"" . sprintf( "\n%*s", 11+length( $hintpre )+3 , "^" ) , 'badutf8' ); } return 1; } sub verify_entities { my( $filename ) = @_; my $file_str = read_file($filename); if( $file_str =~ m/&[^#].+?;/ ) { my ($entity) = ( $file_str =~ m/&([^#].+?);/ ); my %fiveentities = ('quot' => 1, 'amp' => 1, 'apos' => 1, 'lt' => 1, 'gt' => 1); if (!exists($fiveentities{$entity})) { w( "file contains undefined entity: $entity", 'badentity' ); } } return 1; } sub w { my( $msg, $id ) = @_; print "$msg\n"; $errors{$id}++ if defined $id; } 1; =back =head1 BUGS It is currently necessary to specify the path to the xmltv dtd-file. This should not be necessary. =head1 COPYRIGHT Copyright (C) 2006 Mattias Holmlund. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. =cut ### Setup indentation in Emacs ## Local Variables: ## perl-indent-level: 4 ## perl-continued-statement-offset: 4 ## perl-continued-brace-offset: 0 ## perl-brace-offset: -4 ## perl-brace-imaginary-offset: 0 ## perl-label-offset: -2 ## cperl-indent-level: 4 ## cperl-brace-offset: 0 ## cperl-continued-brace-offset: 0 ## cperl-label-offset: -2 ## cperl-extra-newline-before-brace: t ## cperl-merge-trailing-else: nil ## cperl-continued-statement-offset: 2 ## indent-tabs-mode: t ## End: ���������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/ValidateGrabber.pm������������������������������������������������������������������0000664�0000000�0000000�00000031710�15000742332�0017006�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������package XMLTV::ValidateGrabber; use strict; BEGIN { use Exporter (); our (@ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS); @ISA = qw(Exporter); @EXPORT = qw( ); %EXPORT_TAGS = ( ); # eg: TAG => [ qw!name1 name2! ], @EXPORT_OK = qw/ConfigureGrabber ValidateGrabber/; } our @EXPORT_OK; my $CMD_TIMEOUT = 600; =head1 NAME XMLTV::ValidateGrabber - Validates an XMLTV grabber =head1 DESCRIPTION Utility library that validates that a grabber properly implements the capabilities described at http://wiki.xmltv.org/index.php/XmltvCapabilities The ValidateGrabber call first asks the grabber which capabilities it claims to support and then validates that it actually does support these capabilities. =head1 EXPORTED FUNCTIONS All these functions are exported on demand. =over 4 =cut use XMLTV::ValidateFile qw/ValidateFile/; use File::Slurp qw/read_file/; use File::Spec::Functions qw/devnull/; use List::Util qw(min); my $runfh; sub w; sub run; sub run_capture; =item ConfigureGrabber ConfigureGrabber( "./tv_grab_new", "./tv_grab_new.conf" ) =cut sub ConfigureGrabber { my( $exe, $conf ) = @_; if ( run( "$exe --configure --config-file $conf" ) ) { w "Error returned from grabber during configure."; return 1; } return 1; } =item ValidateGrabber Run the validation for a grabber. ValidateGrabber( "tv_grab_new", "./tv_grab_new", "./tv_grab_new.conf", "/tmp/new_", "./blib/share", 0 ) ValidateGrabber takes the following parameters: =over =item * a short name for the grabber. This is only used when printing error messages. =item * the command to run the grabber. =item * the name of a configuration-file for the grabber. =item * a file-prefix that is added to all output-files. =item * a path to a directory with metadata for the grabber. This path is passed to the grabber via the --share option if the grabber supports the capability 'share'. undef if no --share parameter shall be used. =item * a boolean specifying if the --cache parameter shall be used for grabbers that support the 'cache' capability. =back ValidateGrabber returns a list of errors that it found with the grabber. Each error takes the form of a keyword: =over =item noparamcheck The grabber accepts any parameter without returning an error-code. =item noversion The grabber returns an error when run with --version. =item nodescription The grabber returns an error when run with --description. =item nocapabilities The grabber returns an error when run with --capabilities. =item nobaseline The grabber does not list 'baseline' as one of its supported capabilities. =item nomanualconfig The grabber does not list 'manualconfig' as one of its supported capabilities. =item noconfigurationfile The specified configuration-file does not exist. =item graberror The grabber returned with an error-code when asked to grab data. =item notquiet The grabber printed something to STDERR even though the --quiet option was used. =item outputdiffers The grabber produced different output when called with different combinations of --output and --quiet. =item caterror tv_cat returned an error-code when we asked it to process the output from the grabber. =item sorterror tv_sort found errors in the data generated by the grabber. Probably overlapping programmes. =item notadditive grabbing data for tomorrow first and then for the day after tomorrow and concatenating them does not yield the same result as grabbing the data for tomorrow and the day after tomorrow at once. =back Additionally, the list of errors will contain error keywords from XMLTV::ValidateFile if the xmltv-file generated by the grabber was not valid. If no errors are found, an empty list is returned. =cut sub ValidateGrabber { my( $shortname, $exe, $conf, $op, $sharedir, $usecache ) = @_; # if sharedir contains 'blib' we should prepend the relevant development paths! if( defined $sharedir && $sharedir =~ m|/blib/share/$| ) { my( $blib )=( $sharedir =~ m|^(.*/blib)/share/$| ); use Env qw(@PATH @PERL5LIB); unshift( @PATH, $blib . '/script' ); unshift( @PERL5LIB, $blib . '/lib' ); } my @errors; open( $runfh, ">${op}commands.log" ) or die "Failed to write to ${op}commands.log"; if (not run( "$exe --ahdmegkeja > " . devnull() . " 2>&1" )) { w "$shortname with --ahdmegkeja did not fail. The grabber seems to " . "accept any command-line parameter without returning an error."; push @errors, "noparamcheck"; } if (run( "$exe --version > " . devnull() . " 2>&1" )) { w "$shortname with --version failed: $?, $!"; push @errors, "noversion"; } if (run( "$exe --description > " . devnull() . " 2>&1" )) { w "$shortname with --description failed: $?, $!"; push @errors, "nodescription"; } my $cap = run_capture( "$exe --capabilities 2> " . devnull() ); if (not defined $cap) { w "$shortname with --capabilities failed: $?, $!"; push @errors, "nocapabilities"; } my @capabilities = split( /\s+/, $cap ); my %capability; foreach my $c (@capabilities) { $capability{$c} = 1; } if (not defined( $capability{baseline} )) { w "The grabber does not claim to support the 'baseline' capability."; push @errors, "nobaseline"; } if (not defined( $capability{manualconfig} )) { w "The grabber does not claim to support the 'manualconfig' capability."; push @errors, "nomanualconfig"; } my $extraop = ""; $extraop .= "--cache ${op}cache " if $capability{cache} and $usecache; $extraop .= "--share $sharedir " if $capability{share} and defined( $sharedir ); if (not -f $conf) { w "Configuration file $conf does not exist. Aborting."; close( $runfh ); push @errors, "noconfigurationfile"; goto bailout; } # Should we test for --list-channels? my $cmd = "$exe --config-file $conf --offset 1 --days 2 $extraop"; my $output = "${op}1_2"; if (run "$cmd > $output.xml --quiet 2>${op}1.log") { w "$shortname failed: See ${op}1.log"; push @errors, "graberror"; goto bailout; } else { if ( -s "${op}1.log" ) { w "$shortname with --quiet produced output to STDERR when it " . "shouldn't have. See ${op}1.log"; push @errors, "notquiet"; } else { unlink( "${op}1.log" ); } # Okay, it ran, and we have the result in $output.xml. Validate. my @xmlerr = ValidateFile( "$output.xml" ); if (scalar(@xmlerr) > 0) { w "Errors found in $output.xml"; close( $runfh ); push @errors, @xmlerr; goto bailout; } w "$output.xml validates ok"; # Run through tv_cat, which makes sure the data looks like XMLTV. # What kind of errors does this catch that ValidateFile misses? if (not cat_file( "$output.xml", devnull(), "${op}6.log" )) { w "$output.xml makes tv_cat choke, see ${op}6.log"; push @errors, "caterror"; goto bailout; } # Do tv_sort sanity checks. One day it would be better to put # this stuff in a Perl library. my $sort_errors = "$output.sort.log"; if (not sort_file( "$output.xml", "$output.sorted.xml", $sort_errors )) { w "tv_sort failed on $output.xml, probably because of strange " . "start or stop times. See $sort_errors"; push @errors, "sorterror"; } } # Run again to see that --output and --quiet works and to see that # --offset 1 --days 2 equals --offset 1 days 1 plus --offset 2 --days 1. my $output2 = "${op}1_1.xml"; my $cmd2 = "$exe --config-file $conf --offset 1 --days 1 $extraop" . " --output $output2 2>${op}2.log"; if (run $cmd2) { w "$shortname with --output failed: See ${op}2.log"; push @errors, "graberror"; } my $output3 = "${op}2_1.xml"; my $cmd3 = "$exe --config-file $conf --offset 2 --days 1 $extraop" . " > $output3 2>${op}3.log"; if (run $cmd3 ) { w "$shortname with --quiet failed: See ${op}3.log"; push @errors, "graberror"; } else { unlink( "${op}3.log" ); } my $output4 = "${op}4.xml"; my $cmd4 = "$cmd --quiet --output $output4 2>${op}4.log"; if (run $cmd4 ) { w "$shortname with --quiet and --output failed: See ${op}4.log"; push @errors, "graberror"; } else { if ( -s "${op}4.log" ) { w "$shortname with --quiet and --output produced output " . "to STDERR when it shouldn't have. See ${op}4.log"; push @errors, "notquiet"; } else { unlink( "${op}4.log" ); } } if (not cat_files( $output2, $output3, "${op}1_2-2.xml", "${op}5.log" )) { w "tv_cat failed to concatenate the data. See ${op}5.log"; push @errors, "caterror"; } if (not sort_file( "${op}1_2-2.xml", "${op}1_2-2.sorted.xml", "${op}7.log" )) { w "tv_sort failed on the concatenated data. Probably due " . "to overlapping data between days. See ${op}7.log"; push @errors, "notadditive"; } if( !compare_files( "$output.sorted.xml", "${op}1_2-2.sorted.xml", "${op}1_2.diff" ) ) { w "The data is not additive. See ${op}1_2.diff"; push @errors, "notadditive"; } bailout: close( $runfh ); $runfh = undef; # Remove duplicate entries. my $lasterror = "nosucherror"; my @ferrors; foreach my $err (@errors) { push( @ferrors, $err ) if $err ne $lasterror; $lasterror = $err; } if (scalar( @ferrors )) { w "$shortname did not validate ok. See ${op}commands.log for a " . "list of the commands that were used"; } else { w "$shortname validated ok."; } return @ferrors; } sub w { print "$_[0]\n"; } # Run an external command. Exit if the command is interrupted with ctrl-c. sub run { my( $cmd ) = @_; print $runfh "$cmd\n" if defined $runfh; my $killed = 0; # Set a timer and run the real command. eval { local $SIG{ALRM} = sub { # ignore SIGHUP here so the kill only affects children. local $SIG{HUP} = 'IGNORE'; kill 1,(-$$); $killed = 1; }; alarm $CMD_TIMEOUT; system ( $cmd ); alarm 0; }; $SIG{HUP} = 'DEFAULT'; if ($killed) { w "Timeout"; return 1; } if ($? == -1) { w "Failed to execute $cmd: $!"; return 1; } elsif ($? & 127) { w "Terminated by signal " . ($? & 127); exit 1; } return $? >> 8; } # Run an external command and return the output. Exit if the command is # interrupted with ctrl-c. sub run_capture { my( $cmd ) = @_; # print "Running $cmd\n"; my $killed = 0; my $result; # Set a timer and run the real command. eval { local $SIG{ALRM} = sub { # ignore SIGHUP here so the kill only affects children. local $SIG{HUP} = 'IGNORE'; kill 1,(-$$); $killed = 1; }; alarm $CMD_TIMEOUT; $result = qx/$cmd/; alarm 0; }; $SIG{HUP} = 'DEFAULT'; if ($killed) { w "Timeout"; return undef; } if ($? == -1) { w "Failed to execute $cmd: $!"; return undef; } elsif ($? & 127) { w "Terminated by signal " . ($? & 127); exit 1; } if ($? >> 8) { return undef; } else { return $result; } } # Compare two files. Return true if they have the same contents. sub compare_files { my( $file1, $file2, $output ) = @_; $output = devnull() unless defined $output; run("diff $file1 $file2 > $output"); return $? ? 0 : 1; } # Run an xmltv-file through tv_cat. Return true on success. sub cat_file { my( $file1, $outfile, $logfile ) = @_; my $ret = run( "tv_cat $file1 > $outfile 2>$logfile" ); return $ret ? 0 : 1; } # Concatenate two xmltv-files. Return true on success. sub cat_files { my( $file1, $file2, $outfile, $logfile ) = @_; my $ret = run( "tv_cat $file1 $file2 > $outfile 2>$logfile" ); return $ret ? 0 : 1; } # Sort an xmltv-file. Return true on success sub sort_file { my( $file1, $outfile, $logfile ) = @_; my $ret = run( "tv_sort --duplicate-error $file1 > $outfile 2>$logfile" ); return 0 if -s $logfile > 0; return $ret ? 0 : 1; } 1; =back =head1 COPYRIGHT Copyright (C) 2006 Mattias Holmlund. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. =cut ### Setup indentation in Emacs ## Local Variables: ## perl-indent-level: 4 ## perl-continued-statement-offset: 4 ## perl-continued-brace-offset: 0 ## perl-brace-offset: -4 ## perl-brace-imaginary-offset: 0 ## perl-label-offset: -2 ## cperl-indent-level: 4 ## cperl-brace-offset: 0 ## cperl-continued-brace-offset: 0 ## cperl-label-offset: -2 ## cperl-extra-newline-before-brace: t ## cperl-merge-trailing-else: nil ## cperl-continued-statement-offset: 2 ## indent-tabs-mode: t ## End: ��������������������������������������������������������xmltv-1.4.0/lib/Version.pm��������������������������������������������������������������������������0000664�0000000�0000000�00000002121�15000742332�0015407�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������=head1 NAME XMLTV::Version - Adds a --version argument to XMLTV grabbers =head1 DESCRIPTION Add a --version argument to your program, eg use XMLTV::Version '1.2'; If a --version parameter is supplied on the command-line, it will be caught already by the "use" statement, a message will be printed to STDOUT and the program will exit. It is best to put the use XMLTV::Version statement before other module imports, so that even if they fail --version will still work. =head1 SEE ALSO L<XMLTV::Options> =cut package XMLTV::Version; my $opt = '--version'; sub import( $$ ) { die "usage: use $_[0] <version-string>" if @_ != 2; my $seen = 0; foreach (@ARGV) { # This doesn't handle abbreviations in the GNU style. last if $_ eq '--'; if ($_ eq $opt) { $seen++ && warn "seen '$opt' twice\n"; } } return if not $seen; eval { require XMLTV; print "XMLTV module version $XMLTV::VERSION\n"; }; print "could not load XMLTV module, xmltv is not properly installed\n" if $@; for ($_[1]) { print "This program version $_\n"; } exit(); } 1; �����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/XMLTV.pm.PL�������������������������������������������������������������������������0000664�0000000�0000000�00000002124�15000742332�0015211�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# Generate XMLTV.pm from XMLTV.pm.PL. This is to generate part of # the pod documentation from the list of handlers. use strict; sub print_list( $$ ); my $out = shift @ARGV; die "no output file given" if not defined $out; my $in = './lib/XMLTV.pm.in'; require $in; open(IN_FH, $in) or die "cannot read $in: $!"; die if not @XMLTV::Channel_Handlers; die if not @XMLTV::Programme_Handlers; # Vaguely sane so far, let's create the output file and hope it works. open(OUT_FH, ">$out") or die "cannot write to $out: $!"; while (<IN_FH>) { if (/^\s*\@CHANNEL_HANDLERS\s*$/) { print_list(\*OUT_FH, \@XMLTV::Channel_Handlers); } elsif (/^\s*\@PROGRAMME_HANDLERS\s*$/) { print_list(\*OUT_FH, \@XMLTV::Programme_Handlers); } else { print OUT_FH $_; } } close OUT_FH or die "cannot close $out: $!"; close IN_FH or die "cannot close $in: $!"; sub print_list( $$ ) { local *FH = shift; my $l = shift; print FH "\n=over\n"; foreach (@$l) { my ($elem_name, $h_name, $mult) = @$_; print FH "\n=item $elem_name, I<$h_name>, B<$mult>\n"; } print FH "\n\n=back\n"; } ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/lib/XMLTV.pm.in�������������������������������������������������������������������������0000664�0000000�0000000�00000255327�15000742332�0015323�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# -*- perl -*- package XMLTV; use strict; use base 'Exporter'; our @EXPORT = (); our @EXPORT_OK = qw(read_data parse parsefile parsefiles write_data best_name list_channel_keys list_programme_keys); # For the time being the version of this library is tied to that of # the xmltv package as a whole. This number should be checked by the # mkdist tool. # our $VERSION = '1.4.0'; # Work around changing behaviour of XML::Twig. On some systems (like # mine) it always returns UTF-8 data unless KeepEncoding is specified. # However the encoding() method tells you the encoding of the original # document, not of the data you receive. To be sure of what you're # getting, it is easiest on such a system to not give KeepEncoding and # just use UTF-8. # # But on other systems (seemingly perl 5.8 and above), XML::Twig tries # to keep the original document's encoding in the strings returned. # You then have to call encoding() to find out what you're getting. # To make sure of this behaviour we set KeepEncoding to true on such a # system. # # Setting KeepEncoding true everywhere seems to do no harm, it's a # pity that we lose conversion to UTF-8 but at least it's the same # everywhere. So the library is distributed with this flag on. # my $KEEP_ENCODING = 1; # We need a way of telling parsefiles_callback() to optionally *not* die when presented with multiple encodings, # but without affecting any other packages which uses it (i.e. so a new sub param is out of the question) # - best I can think of for the minute is a global (yuk) my $DIE_ON_MULTIPLE_ENCODINGS = 1; my %warned_unknown_key; sub warn_unknown_keys( $$ ); =pod =head1 NAME XMLTV - Perl extension to read and write TV listings in XMLTV format =head1 SYNOPSIS use XMLTV; my $data = XMLTV::parsefile('tv.xml'); my ($encoding, $credits, $ch, $progs) = @$data; my $langs = [ 'en', 'fr' ]; print 'source of listings is: ', $credits->{'source-info-name'}, "\n" if defined $credits->{'source-info-name'}; foreach (values %$ch) { my ($text, $lang) = @{XMLTV::best_name($langs, $_->{'display-name'})}; print "channel $_->{id} has name $text\n"; print "...in language $lang\n" if defined $lang; } foreach (@$progs) { print "programme on channel $_->{channel} at time $_->{start}\n"; next if not defined $_->{desc}; foreach (@{$_->{desc}}) { my ($text, $lang) = @$_; print "has description $text\n"; print "...in language $lang\n" if defined $lang; } } The value of $data will be something a bit like: [ 'UTF-8', { 'source-info-name' => 'Ananova', 'generator-info-name' => 'XMLTV' }, { 'radio-4.bbc.co.uk' => { 'display-name' => [ [ 'en', 'BBC Radio 4' ], [ 'en', 'Radio 4' ], [ undef, '4' ] ], 'id' => 'radio-4.bbc.co.uk' }, ... }, [ { start => '200111121800', title => [ [ 'Simpsons', 'en' ] ], channel => 'radio-4.bbc.co.uk' }, ... ] ] =head1 DESCRIPTION This module provides an interface to read and write files in XMLTV format (a TV listings format defined by xmltv.dtd). In general element names in the XML correspond to hash keys in the Perl data structure. You can think of this module as a bit like B<XML::Simple>, but specialized to the XMLTV file format. The Perl data structure corresponding to an XMLTV file has four elements. The first gives the character encoding used for text data, typically UTF-8 or ISO-8859-1. (The encoding value could also be undef meaning 'unknown', when the library canE<39>t work out what it is.) The second element gives the attributes of the root <tv> element, which give information about the source of the TV listings. The third element is a list of channels, each list element being a hash corresponding to one <channel> element. The fourth element is similarly a list of programmes. More details about the data structure are given later. The easiest way to find out what it looks like is to load some small XMLTV files and use B<Data::Dumper> to print out the resulting structure. =head1 USAGE =cut use XML::Twig; use XML::Writer 0.600; use Date::Manip; use Carp; use Data::Dumper; # Use Lingua::Preferred if available, else kludge a replacement. sub my_which_lang { return $_[1]->[0] } BEGIN { eval { require Lingua::Preferred }; *which_lang = $@ ? \&my_which_lang : \&Lingua::Preferred::which_lang; } # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } # Attributes and subelements of channel. Each subelement additionally # needs a handler defined. Multiplicity is given for both, but for # attributes the only allowable values are '1' and '?'. # # Ordering of attributes is not really important, but we keep the same # order as they are given in the DTD so that output looks nice. # # The ordering of the subelements list gives the order in which these # elements must appear in the DTD. In fact, these lists just # duplicate information in the DTD and add details of what handlers # to call. # our @Channel_Attributes = ([ 'id', '1' ]); our @Channel_Handlers = ( [ 'display-name', 'with-lang', '+' ], [ 'icon', 'icon', '*' ], [ 'url', 'url', '*' ], ); # Same for <programme> elements. our @Programme_Attributes = ( [ 'start', '1' ], [ 'stop', '?' ], [ 'pdc-start', '?' ], [ 'vps-start', '?' ], [ 'showview', '?' ], [ 'videoplus', '?' ], [ 'channel', '1' ], [ 'clumpidx', '?' ], ); our @Programme_Handlers = ( [ 'title', 'with-lang', '+' ], [ 'sub-title', 'with-lang', '*' ], [ 'desc', 'with-lang/m', '*' ], [ 'credits', 'credits', '?' ], [ 'date', 'scalar', '?' ], [ 'category', 'with-lang', '*' ], [ 'keyword', 'with-lang', '*' ], [ 'language', 'with-lang', '?' ], [ 'orig-language', 'with-lang', '?' ], [ 'length', 'length', '?' ], [ 'icon', 'icon', '*' ], [ 'url', 'url', '*' ], [ 'country', 'with-lang', '*' ], [ 'episode-num', 'episode-num', '*' ], [ 'video', 'video', '?' ], [ 'audio', 'audio', '?' ], [ 'previously-shown', 'previously-shown', '?' ], [ 'premiere', 'with-lang/em', '?' ], [ 'last-chance', 'with-lang/em', '?' ], [ 'new', 'presence', '?' ], [ 'subtitles', 'subtitles', '*' ], [ 'rating', 'rating', '*' ], [ 'star-rating', 'star-rating', '*' ], [ 'review', 'review', '*' ], [ 'image', 'image', '*' ], ); # And a hash mapping names like 'with-lang' to pairs of subs. The # first for reading, the second for writing. Note that the writers # alter the passed-in data as a side effect! (If the writing sub is # called with an undef XML::Writer then it writes nothing but still # warns for (most) bad data checks - and still alters the data.) # our %Handlers = (); # Undocumented interface for adding extensions to the XMLTV format: # first add an entry to @XMLTV::Channel_Handlers or # @XMLTV::Programme_Handlers with your new element's name, 'type' and # multiplicity. The 'type' should be a string you invent yourself. # Then $XMLTV::Handlers{'type'} should be a pair of subroutines, a # reader and a writer. (Unless you want to use one of the existing # types such as 'with-lang' or 'scalar'.) # # Note that elements and attributes beginning 'x-' are skipped over # _automatically_, so you can't parse them with this method. A better # way to add extensions is needed - doing this not encouraged but is # sometimes necessary. # # read_data() is a deprecated name for parsefile(). sub read_data( $ ) { # FIXME remove altogether warn "XMLTV::read_data() deprecated, use XMLTV::parsefile() instead\n"; &parsefile; } # Private. sub sanity( $ ) { for (shift) { croak 'no <tv> element found' if not /<tv/; } } =over =item parse(document) Takes an XMLTV document (a string) and returns the Perl data structure. It is assumed that the document is valid XMLTV; if not the routine may die() with an error (although the current implementation just warns and continues for most small errors). The first element of the listref returned, the encoding, may vary according to the encoding of the input document, the versions of perl and C<XML::Parser> installed, the configuration of the XMLTV library and other factors including, but not limited to, the phase of the moon. With luck it should always be either the encoding of the input file or UTF-8. Attributes and elements in the XML file whose names begin with 'x-' are skipped silently. You can use these to include information which is not currently handled by the XMLTV format, or by this module. =cut sub parse( $ ) { my $str = shift; sanity($str); # FIXME commonize with parsefiles() my ($encoding, $credits); my %channels; my @programmes; parse_callback($str, sub { $encoding = shift }, sub { $credits = shift }, sub { for (shift) { $channels{$_->{id}} = $_ } }, sub { push @programmes, shift }); return [ $encoding, $credits, \%channels, \@programmes ]; } =pod =item parsefiles(filename...) Like C<parse()> but takes one or more filenames instead of a string document. The data returned is the merging of those file contents: the programmes will be concatenated in their original order, the channels just put together in arbitrary order (ordering of channels should not matter). It is necessary that each file have the same character encoding, if not, an exception is thrown. Ideally the credits information would also be the same between all the files, since there is no obvious way to merge it - but if the credits information differs from one file to the next, one file is picked arbitrarily to provide credits and a warning is printed. If two files give differing channel definitions for the same XMLTV channel id, then one is picked arbitrarily and a warning is printed. In the simple case, with just one file, you neednE<39>t worry about mismatching of encodings, credits or channels. The deprecated function C<parsefile()> is a wrapper allowing just one filename. =cut sub parsefiles( @ ) { die 'one or more filenames required' if not @_; my ($encoding, $credits); my %channels; my @programmes; parsefiles_callback(sub { $encoding = shift }, sub { $credits = shift }, sub { for (shift) { $channels{$_->{id}} = $_ } }, sub { push @programmes, shift }, @_); return [ $encoding, $credits, \%channels, \@programmes ]; } sub parsefile( $ ) { parsefiles(@_) } =pod =item parse_callback(document, encoding_callback, credits_callback, channel_callback, programme_callback) An alternative interface. Whereas C<parse()> reads the whole document and then returns a finished data structure, with this routine you specify a subroutine to be called as each <channel> element is read and another for each <programme> element. The first argument is the document to parse. The remaining arguments are code references, one for each part of the document. The callback for encoding will be called once with a string giving the encoding. In present releases of this module, it is also possible for the value to be undefined meaning 'unknown', but itE<39>s hoped that future releases will always be able to figure out the encoding used. The callback for credits will be called once with a hash reference. For channels and programmes, the appropriate function will be called zero or more times depending on how many channels / programmes are found in the file. The four subroutines will be called in order, that is, the encoding and credits will be done before the channel handler is called and all the channels will be dealt with before the first programme handler is called. If any of the code references is undef, nothing is called for that part of the file. For backwards compatibility, if the value for 'encoding callback' is not a code reference but a scalar reference, then the encoding found will be stored in that scalar. Similarly if the 'credits callback' is a scalar reference, the scalar it points to will be set to point to the hash of credits. This style of interface is deprecated: new code should just use four callbacks. For example: my $document = '<tv>...</tv>'; my $encoding; sub encoding_cb( $ ) { $encoding = shift } my $credits; sub credits_cb( $ ) { $credits = shift } # The callback for each channel populates this hash. my %channels; sub channel_cb( $ ) { my $c = shift; $channels{$c->{id}} = $c; } # The callback for each programme. We know that channels are # always read before programmes, so the %channels hash will be # fully populated. # sub programme_cb( $ ) { my $p = shift; print "got programme: $p->{title}->[0]->[0]\n"; my $c = $channels{$p->{channel}}; print 'channel name is: ', $c->{'display-name'}->[0]->[0], "\n"; } # Let's go. XMLTV::parse_callback($document, \&encoding_cb, \&credits_cb, \&channel_cb, \&programme_cb); =cut # Private. sub new_doc_callback( $$$$ ) { my ($enc_cb, $cred_cb, $ch_cb, $p_cb) = @_; t 'creating new XML::Twig'; t '\@Channel_Handlers=' . d \@Channel_Handlers; t '\@Programme_Handlers=' . d \@Programme_Handlers; new XML::Twig(StartTagHandlers => { '/tv' => sub { my ($t, $node) = @_; my $enc; if ($KEEP_ENCODING) { t 'KeepEncoding on, get original encoding'; $enc = $t->encoding(); } else { t 'assuming UTF-8 encoding'; $enc = 'UTF-8'; } if (defined $enc_cb) { for (ref $enc_cb) { if ($_ eq 'CODE') { $enc_cb->($enc); } elsif ($_ eq 'SCALAR') { $$enc_cb = $enc; } else { die "callback should be code ref or scalar ref, or undef"; } } } if (defined $cred_cb) { my $cred = get_attrs($node); for (ref $cred_cb) { if ($_ eq 'CODE') { $cred_cb->($cred); } elsif ($_ eq 'SCALAR') { $$cred_cb = $cred; } else { die "callback should be code ref or scalar ref, or undef"; } } } # Most of the above code can be removed in the # next release. # }, }, TwigHandlers => { '/tv/channel' => sub { my ($t, $node) = @_; die if not defined $node; my $c = node_to_channel($node); $t->purge(); if (not $c) { warn "skipping bad channel element\n"; } else { $ch_cb->($c); } }, '/tv/programme' => sub { my ($t, $node) = @_; die if not defined $node; my $p = node_to_programme($node); $t->purge(); if (not $p) { warn "skipping bad programme element\n"; } else { $p_cb->($p); } }, }, KeepEncoding => $KEEP_ENCODING, ); } sub parse_callback( $$$$$ ) { my ($str, $enc_cb, $cred_cb, $ch_cb, $p_cb) = @_; sanity($str); new_doc_callback($enc_cb, $cred_cb, $ch_cb, $p_cb)->parse($str); } =pod =item parsefiles_callback(encoding_callback, credits_callback, channel_callback, programme_callback, filenames...) As C<parse_callback()> but takes one or more filenames to open, merging their contents in the same manner as C<parsefiles()>. Note that the reading is still gradual - you get the channels and programmes one at a time, as they are read. Note that the same <channel> may be present in more than one file, so the channel callback will get called more than once. ItE<39>s your responsibility to weed out duplicate channel elements (since writing them out again requires that each have a unique id). For compatibility, there is an alias C<parsefile_callback()> which is the same but takes only a single filename, B<before> the callback arguments. This is deprecated. =cut sub parsefile_callback( $$$$$ ) { my ($f, $enc_cb, $cred_cb, $ch_cb, $p_cb) = @_; parsefiles_callback($enc_cb, $cred_cb, $ch_cb, $p_cb, $f); } sub parsefiles_callback( $$$$@ ) { my ($enc_cb, $cred_cb, $ch_cb, $p_cb, @files) = @_; die "one or more files required" if not @files; my $all_encoding; my $have_encoding = 0; my $all_credits; my %all_channels; my $do_next_file; # sub to parse file ( defined below) my $do_file_number; # current file in @files array my $my_enc_cb = sub( $ ) { my $e = shift; t 'encoding callback'; if ($have_encoding) { t 'seen encoding before, just check'; my ($da, $de) = (defined $all_encoding, defined $e); if (not $da and not $de) { warn "two files both have unspecified character encodings, hope they're the same\n"; } elsif (not $da and $de) { ##warn "encoding $e not being returned to caller\n"; $all_encoding = $e; } elsif ($da and not $de) { warn "input file with unspecified encoding, assuming same as others ($all_encoding)\n"; } elsif ($da and $de) { if (uc($all_encoding) ne uc($e)) { if ( defined $DIE_ON_MULTIPLE_ENCODINGS && !$DIE_ON_MULTIPLE_ENCODINGS ) { warn "this file's encoding $e differs from others' $all_encoding \n"; } else { die "this file's encoding $e differs from others' $all_encoding - aborting\n"; } } } else { die } t 'have encoding, call user'; $enc_cb->($e, $do_file_number) if $enc_cb; } else { t 'not seen encoding before, call user'; $enc_cb->($e, $do_file_number) if $enc_cb; $all_encoding = $e; $have_encoding = 1; } }; my $my_cred_cb = sub( $ ) { my $c = shift; $Data::Dumper::Sortkeys = 1; # ensure consistent order of dumped hash if (defined $all_credits) { if (Dumper($all_credits) ne Dumper($c)) { warn "different files have different credits, picking one arbitrarily\n"; # In fact, we pick the last file in the list since this is the # first to be opened. # } } else { $cred_cb->($c) if $cred_cb; $all_credits = $c; } }; my $my_ch_cb = sub( $ ) { my $c = shift; my $id = $c->{id}; $Data::Dumper::Sortkeys = 1; # ensure consistent order of dumped hash if (defined $all_channels{$id} and Dumper($all_channels{$id}) ne Dumper($c)) { warn "differing channels with id $id, picking one arbitrarily\n"; } else { $all_channels{$id} = $c; $ch_cb->($c, $do_file_number) if $ch_cb; } }; my $my_p_cb = sub( $ ) { my $doing_file = $do_file_number; $do_next_file->(); # if any $do_file_number = $doing_file; $p_cb->(@_, $do_file_number) if $p_cb; }; $do_next_file = sub() { while (@files) { # Last first. my $f = pop @files; $do_file_number = scalar @files; # In older versions of perl there were segmentation faults # when calling die() inside the parsing callbacks, so we # needed to override $SIG{__DIE__} here. Assume that # newer perls don't have this issue. # my $t = new_doc_callback($my_enc_cb, $my_cred_cb, $my_ch_cb, $my_p_cb); if ($f eq '-') { $t->parse(\*STDIN); } else { $t->parsefile($f); } } }; # Let's go. $do_next_file->(); } =pod =item write_data(data, options...) Takes a data structure and writes it as XML to standard output. Any extra arguments are passed on to XML::WriterE<39>s constructor, for example my $f = new IO::File '>out.xml'; die if not $f; write_data($data, OUTPUT => $f); The encoding used for the output is given by the first element of the data. Normally, there will be a warning for any Perl data which is not understood and cannot be written as XMLTV, such as strange keys in hashes. But as an exception, any hash key beginning with an underscore will be skipped over silently. You can store 'internal use only' data this way. If a programme or channel hash contains a key beginning with 'debug', this key and its value will be written out as a comment inside the <programme> or <channel> element. This lets you include small debugging messages in the XML output. =cut sub write_data( $;@ ) { my $data = shift; my $writer = new XMLTV::Writer(encoding => $data->[0], @_); $writer->start($data->[1]); $writer->write_channels($data->[2]); $writer->write_programme($_) foreach @{$data->[3]}; $writer->end(); } # Private. # # get_attrs() # # Given a node, return a hashref of its attributes. Skips over # the 'x-whatever' attributes. # sub get_attrs( $ ) { my $node = shift; die if not defined $node; my %r = %{$node->atts()}; foreach (keys %r) { if (/^x-/) { delete $r{$_}; } else { tidy(\$r{$_}); } } return \%r; } # Private. # # get_text() # # Given a node containing only text, return that text (with whitespace # either side stripped). If the node has no children (as in # <foo></foo> or <foo />), this is considered to be the empty string. # # Parameter: whether newlines are allowed (defaults to false) # sub get_text( $;$ ) { my $node = shift; my $allow_nl = shift; $allow_nl = 0 if not defined $allow_nl; my @children = get_subelements($node); if (@children == 0) { return ''; } elsif (@children == 1) { my $v = $children[0]->pcdata(); t 'got pcdata: ' . d $v; if (not defined $v) { my $name = get_name($node); warn "node $name expected to contain text has other stuff\n"; } else { # Just hope that the encoding we got uses \n... if (not $allow_nl and $v =~ tr/\n//d) { my $name = get_name($node); warn "removing newlines from content of node $name\n"; } tidy(\$v); } t 'returning: ' . d $v; return $v; } elsif (@children > 1) { my $name = get_name($node); warn "node $name expected to contain text has more than one child\n"; return undef; } else { die } } # Private. # # get_pcdata() # # Given a node containing mixed data, return the first occurring node # with character text (with whitespace either side stripped). # # Parameter: whether newlines are allowed (defaults to false) # sub get_pcdata( $;$ ) { my $node = shift; my $allow_nl = shift; $allow_nl = 0 if not defined $allow_nl; my @children = get_subelements($node); if (@children == 0) { return ''; } foreach (@children) { if ( $_->tag eq '#PCDATA' ) { my $v = $_->pcdata(); t 'got pcdata: ' . d $v; if (not defined $v) { my $name = get_name($node); warn "node $name expected to contain text has other stuff\n"; } else { # Just hope that the encoding we got uses \n... if (not $allow_nl and $v =~ tr/\n//d) { my $name = get_name($node); #warn "removing newlines from content of node $name\n"; } tidy(\$v); } t 'returning: ' . d $v; return $v; } } my $name = get_name($node); warn "node $name expected to contain text but none found\n"; return undef; } # Private. Clean up parsed text. Takes ref to scalar. sub tidy( $ ) { our $v; local *v = shift; die if not defined $v; if ($XML::Twig::VERSION < 3.01 || $KEEP_ENCODING) { # Old versions of XML::Twig had stupid behaviour with # entities - and so do the new ones if KeepEncoding is on. # for ($v) { s/>/>/g; s/</</g; s/'/\'/g; s/"/\"/g; s/&/&/g; # needs to be last } } else { t 'new XML::Twig, not KeepEncoding, entities already dealt with'; } for ($v) { s/^\s+//; s/\s+$//; # On Windows there seems to be an inconsistency between # XML::Twig and XML::Writer. The former returns text with # \r\n line endings to the application, but the latter adds \r # characters to text outputted. So reading some text and # writing it again accumulates an extra \r character. We fix # this by removing \r from the input here. # tr/\r//d; } } # Private. # # get_subelements() # # Return a list of all subelements of a node. Whitespace is # ignored; anything else that isn't a subelement is warned about. # Skips over elements with name 'x-whatever'. # sub get_subelements( $ ) { grep { (my $tmp = get_name($_)) !~ /^x-/ } $_[0]->children(); } # Private. # # get_name() # # Return the element name of a node. # sub get_name( $ ) { $_[0]->gi() } # Private. # # dump_node() # # Return some information about a node for debugging. # sub dump_node( $ ) { my $n = shift; # Doesn't seem to be easy way to get 'type' of node. my $r = 'name: ' . get_name($n) . "\n"; for (trunc($n->text())) { $r .= "value: $_\n" if defined and length; } return $r; } # Private. Truncate a string to a reasonable length and add '...' if # necessary. # sub trunc { local $_ = shift; return undef if not defined; if (length > 1000) { return substr($_, 0, 1000) . '...'; } return $_; } =pod =item best_name(languages, pairs [, comparator]) The XMLTV format contains many places where human-readable text is given an optional 'lang' attribute, to allow mixed languages. This is represented in Perl as a pair [ text, lang ], although the second element may be missing or undef if the language is unknown. When several alernatives for an element (such as <title>) can be given, the representation is a list of [ text, lang ] pairs. Given such a list, what is the best text to use? It depends on the userE<39>s preferred language. This function takes a list of acceptable languages and a list of [string, language] pairs, and finds the best one to use. This means first finding the appropriate language and then picking the 'best' string in that language. The best is normally defined as the first one found in a usable language, since the XMLTV format puts the most canonical versions first. But you can pass in your own comparison function, for example if you want to choose the shortest piece of text that is in an acceptable language. The acceptable languages should be a reference to a list of language codes looking like 'ru', or like 'de_DE'. The text pairs should be a reference to a list of pairs [ string, language ]. (As a special case if this list is empty or undef, that means no text is present, and the result is undef.) The third argument if present should be a cmp-style function that compares two strings of text and returns 1 if the first argument is better, -1 if the second better, 0 if theyE<39>re equally good. Returns: [s, l] pair, where s is the best of the strings to use and l is its language. This pair is 'live' - it is one of those from the list passed in. So you can use C<best_name()> to find the best pair from a list and then modify the content of that pair. (This routine depends on the C<Lingua::Preferred> module being installed; if that module is missing then the first available language is always chosen.) Example: my $langs = [ 'de', 'fr' ]; # German or French, please # Say we found the following under $p->{title} for a programme $p. my $pairs = [ [ 'La CitE des enfants perdus', 'fr' ], [ 'The City of Lost Children', 'en_US' ] ]; my $best = best_name($langs, $pairs); print "chose title $best->[0]\n"; =cut sub best_name( $$;$ ) { my ($wanted_langs, $pairs, $compare) = @_; t 'best_name() ENTRY'; t 'wanted langs: ' . d $wanted_langs; t '[text,lang] pairs: ' . d $pairs; t 'comparison fn: ' . d $compare; return undef if not defined $pairs; my @pairs = @$pairs; my @avail_langs; my (%seen_lang, $seen_undef); # Collect the list of available languages. foreach (map { $_->[1] } @pairs) { if (defined) { next if $seen_lang{$_}++; } else { next if $seen_undef++; } push @avail_langs, $_; } my $pref_lang = which_lang($wanted_langs, \@avail_langs); # Gather up [text, lang] pairs which have the desired language. my @candidates; foreach (@pairs) { my ($text, $lang) = @$_; next unless ((not defined $lang) or (defined $pref_lang and $lang eq $pref_lang)); push @candidates, $_; } return undef if not @candidates; # If a comparison function was passed in, use it to compare the # text strings from the candidate pairs. # @candidates = sort { $compare->($a->[0], $b->[0]) } @candidates if defined $compare; # Pick the first candidate. This will be the one ordered first by # the comparison function if given, otherwise the earliest in the # original list. # return $candidates[0]; } =item list_channel_keys(), list_programme_keys() Some users of this module may wish to enquire at runtime about which keys a programme or channel hash can contain. The data in the hash comes from the attributes and subelements of the corresponding element in the XML. The values of attributes are simply stored as strings, while subelements are processed with a handler which may return a complex data structure. These subroutines returns a hash mapping key to handler name and multiplicity. This lets you know what data types can be expected under each key. For keys which come from attributes rather than subelements, the handler is set to 'scalar', just as for subelements which give a simple string. See L<"DATA STRUCTURE"> for details on what the different handler names mean. It is not possible to find out which keys are mandatory and which optional, only a list of all those which might possibly be present. An example use of these routines is the L<tv_grep> program, which creates its allowed command line arguments from the names of programme subelements. =cut # Private. sub list_keys( $$ ) { my %r; # Attributes. foreach (@{shift()}) { my ($k, $mult) = @$_; $r{$k} = [ 'scalar', $mult ]; } # Subelements. foreach (@{shift()}) { my ($k, $h_name, $mult) = @$_; $r{$k} = [ $h_name, $mult ]; } return \%r; } # Public. sub list_channel_keys() { list_keys(\@Channel_Attributes, \@Channel_Handlers); } sub list_programme_keys() { list_keys(\@Programme_Attributes, \@Programme_Handlers); } =pod =item catfiles(w_args, filename...) Concatenate several listings files, writing the output to somewhere specified by C<w_args>. Programmes are catenated together, channels are merged, for credits we just take the first and warn if the others differ. The first argument is a hash reference giving information to pass to C<XMLTV::Writer>E<39>s constructor. But do not specify encoding, this will be taken from the input files. C<catfiles()> will abort if the input files have different encodings, unless the 'UTF8'=1 argument is passed in. =cut sub catfiles( $@ ) { my $w_args = shift; my $w; my $enc; # encoding of current file my @encs; # encoding of all files being catenated my %seen_ch; my %seen_progs; $DIE_ON_MULTIPLE_ENCODINGS = ( defined $w_args->{UTF8} ? 0 : 1 ); $Data::Dumper::Sortkeys = 1; # ensure consistent order of dumped hash XMLTV::parsefiles_callback (sub { $enc = shift; my $do_file = shift; $encs[$do_file] = (defined $enc ? $enc : 'unknown'); t "file $do_file = $enc" if defined $enc; $w = new XMLTV::Writer(%$w_args, encoding => ( defined $w_args->{UTF8} ? 'UTF-8' : $enc ) ) if !defined $w; }, sub { $w->start(shift) }, sub { my $c = shift; my $id = $c->{id}; if (not defined $seen_ch{$id}) { my $do_file = shift; if (defined $w_args->{UTF8}) { if (uc($encs[$do_file]) ne 'UTF-8' && $encs[$do_file] ne 'unknown') { # recode the incoming channel name t 'recoding channel from '.$encs[$do_file].' to UTF-8'; require XMLTV::Data::Recursive::Encode; $c = XMLTV::Data::Recursive::Encode->from_to($c, $encs[$do_file], 'UTF-8'); } } $w->write_channel($c); $seen_ch{$id} = $c; } elsif (Dumper($seen_ch{$id}) eq Dumper($c)) { # They're identical, okay. } else { warn "channel $id may differ between two files, " . "picking one arbitrarily\n"; } }, sub { my $p = shift; my $do_file = shift; if (defined $w_args->{UTF8}) { if (uc($encs[$do_file]) ne 'UTF-8' && $encs[$do_file] ne 'unknown') { # recode the incoming programme t 'recoding prog from '.$encs[$do_file].' to UTF-8'; require XMLTV::Data::Recursive::Encode; $p = XMLTV::Data::Recursive::Encode->from_to($p, $encs[$do_file], 'UTF-8'); } } if (! $seen_progs{ $p->{start} . "|" . $p->{title}[0][0] . "|" . $p->{channel} }++) { $w->write_programme($p); } else { # warn "duplicate programme detected, skipping\n" # . " " . $p->{start} . "|" . $p->{stop} . "|" . $p->{title}[0][0] . "|" . $p->{channel} . "\n"; } }, @_); $w->end(); } =pod =item cat(data, ...) Concatenate (and merge) listings data. Programmes are catenated together, channels are merged, for credits we just take the first and warn if the others differ (except that the 'date' of the result is the latest date of all the inputs). Whereas C<catfiles()> reads and writes files, this function takes already-parsed listings data and returns some more listings data. It is much more memory-hungry. =cut sub cat( @ ) { cat_aux(1, @_) } =pod =item cat_noprogrammes Like C<cat()> but ignores the programme data and just returns encoding, credits and channels. This is in case for scalability reasons you want to handle programmes individually, but still merge the smaller data. =cut sub cat_noprogrammes( @ ) { cat_aux(0, @_) } sub cat_aux( @ ) { my $all_encoding; my ($all_credits_nodate, $all_credits_date); my %all_channels; my @all_progs; my $do_progs = shift; $Data::Dumper::Sortkeys = 1; # ensure consistent order of dumped hash foreach (@_) { t 'doing arg: ' . d $_; my ($encoding, $credits, $channels, $progs) = @$_; if (not defined $all_encoding) { $all_encoding = $encoding; } elsif ($encoding ne $all_encoding) { die "different files have different encodings, cannot continue\n"; } # If the credits are different between files there's not a lot # we can do to merge them. Apart from 'date', that is. There # we can say that the date of the concatenated listings is the # newest date from all the sources. # my %credits_nodate = %$credits; # copy my $d = delete $credits_nodate{date}; if (defined $d) { # Need to 'require' rather than 'use' this because # XMLTV.pm is loaded during the build process and # XMLTV::Date isn't available then. Urgh. # require XMLTV::Date; my $dp = XMLTV::Date::parse_date($d); for ($all_credits_date) { if (not defined or Date_Cmp(XMLTV::Date::parse_date($_), $dp) < 0) { $_ = $d; } } } # Now in uniqueness checks ignore the date. if (not defined $all_credits_nodate) { $all_credits_nodate = \%credits_nodate; } elsif (Dumper(\%credits_nodate) ne Dumper($all_credits_nodate)) { warn "different files have different credits, taking from first file\n"; } foreach (keys %$channels) { if (not defined $all_channels{$_}) { $all_channels{$_} = $channels->{$_}; } elsif (Dumper($all_channels{$_}) ne Dumper($channels->{$_})) { warn "channel $_ differs between two files, taking first appearance\n"; } } push @all_progs, @$progs if $do_progs; } $all_encoding = 'UTF-8' if not defined $all_encoding; my %all_credits; %all_credits = %$all_credits_nodate if defined $all_credits_nodate; $all_credits{date} = $all_credits_date if defined $all_credits_date; if ($do_progs) { @all_progs = reverse @all_progs; my %seen_progs; my @dupe_indexes = reverse(grep { $seen_progs{ $all_progs[$_]->{start} . "|" . $all_progs[$_]->{stop} . "|" . $all_progs[$_]->{title}[0][0] . "|" . $all_progs[$_]->{channel} }++ } 0..$#all_progs); foreach my $item (@dupe_indexes) { # warn "duplicate programme detected, skipping\n" # . " " . $all_progs[$item]->{start} . "|" . $all_progs[$item]->{stop} . "|" . $all_progs[$item]->{title}[0][0] . "|" . $all_progs[$item]->{channel} . "\n"; splice (@all_progs,$item,1); } @all_progs = reverse @all_progs; return [ $all_encoding, \%all_credits, \%all_channels, \@all_progs ]; } else { return [ $all_encoding, \%all_credits, \%all_channels ]; } } # For each subelement of programme, we define a subroutine to read it # and one to write it. The reader takes an node for a single # subelement and returns its value as a Perl scalar (warning and # returning undef if error). The writer takes an XML::Writer, an # element name and a scalar value and writes a subelement for that # value. Note that the element name is passed in to the writer just # for symmetry, so that neither the writer or the reader have to know # what their element is called. # =pod =back =head1 DATA STRUCTURE For completeness, we describe more precisely how channels and programmes are represented in Perl. Each element of the channels list is a hashref corresponding to one <channel> element, and likewise for programmes. The possible keys of a channel (programme) hash are the names of attributes or subelements of <channel> (<programme>). The values for attributes are not processed in any way; an attribute C<fred="jim"> in the XML will become a hash element with key C<'fred'>, value C<'jim'>. But for subelements, there is further processing needed to turn the XML content of a subelement into Perl data. What is done depends on what type of data is stored under that subelement. Also, if a certain element can appear several times then the hash key for that element points to a list of values rather than just one. The conversion of a subelementE<39>s content to and from Perl data is done by a handler. The most common handler is I<with-lang>, used for human-readable text content plus an optional 'lang' attribute. There are other handlers for other data structures in the file format. Often two subelements will share the same handler, since they hold the same type of data. The handlers defined are as follows; note that many of them will silently strip leading and trailing whitespace in element content. Look at the DTD itself for an explanation of the whole file format. Unless specified otherwise, it is not allowed for an element expected to contain text to have empty content, nor for the text to contain newline characters. =over =item I<credits> Turns a list of credits (for director, actor, writer, etc.) into a hash mapping 'role' to a list of names. The names in each role are kept in the same order. An optional 'guest' attribute for actor with values 'yes' and 'no' is converted to Boolean. =cut $Handlers{credits}->[0] = sub( $ ) { my $node = shift; my @roles = qw(director actor writer adapter producer composer editor presenter commentator guest); my %known_role; ++$known_role{$_} foreach @roles; my %r; foreach (get_subelements($node)) { my $role = get_name($_); unless ($known_role{$role}++) { warn "unknown thing in credits: $role"; next; } my $text=get_pcdata($_); # get any image and url sub-elements my ( @image, @url ); foreach (get_subelements($_)) { t 'got element '.$_->tag; if ( $_->tag eq 'image' || $_->tag eq 'url') { # would prefer to use call_handlers_read() but that doesn't work on individual hashes (only the entire programme) # # Lookup the handler my $found_pos; my $name = $_->tag; my $handlers = \@Programme_Handlers; for (my $i = 0; $i < @$handlers; $i++) { if ($handlers->[$i]->[0] eq $name) { t 'found handler'; $found_pos = $i; last; } } # Call the handler. t 'calling handler'; my ($handler_name, $h_name, $multiplicity) = @{$handlers->[$found_pos]}; die if $handler_name ne $name; my $h = $Handlers{$h_name}; die "no handler $h_name" if not $h; my $result = $h->[0]->($_); # call reader sub t 'result: ' . d $result; warn("skipping bad $name\n"), next if not defined $result; push @image, $result if defined $result and $_->tag eq 'image'; push @url, $result if defined $result and $_->tag eq 'url';; } } # get attributes on role element (only actor has these at present) my @attr; my %attrs = %{get_attrs($_)}; my $character = $attrs{role} if exists $attrs{role}; my $guest = $attrs{guest} if exists $attrs{guest}; if ($role eq 'actor' && (defined $character || defined $guest)) { # attributes are (1) role, (2) guest push @attr, (defined $character ? $character : ''); push @attr, decode_boolean($guest) if defined $guest; # only add guest if present } # join the #pcdata, attributes, and sub-elements if (scalar @image || scalar @url || scalar @attr) { if (scalar @image || scalar @url) { my $h={}; $h->{image} = \@image if (scalar @image); $h->{url} = \@url if (scalar @url); if ($role eq 'actor') { $#attr=1; # ensure @attr is exactly 2 items push @{$r{$role}}, [ $text, @attr, $h ]; } else { push @{$r{$role}}, [ $text, $h ]; # only 'actor' has attributes } } else { push @{$r{$role}}, [ $text, @attr ]; } } else { push @{$r{$role}}, $text; } } return \%r; }; # $Handlers{'credits'}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; die if not defined $v; my %h = %$v; return if not %h; # don't write empty element t 'writing credits: ' . d \%h; # TODO some 'do nothing' setting in XML::Writer to replace this # convention of passing undef. # $w->startTag($e) if $w; foreach ( qw[director actor writer adapter producer composer editor presenter commentator guest] ) { next unless defined $h{$_}; my $role = $_; my @people = @{delete $h{$_}}; foreach my $person (@people) { die if not defined $person; if (ref($person) eq 'ARRAY') { # do we have more than just a person's name? my @attrs; my $h; if ( $role eq 'actor' ) { # attributes to 'actor' are (1) role, (2) guest my %attrs = ( 1=>'role', 2=>'guest' ); @attrs = map{ $attrs{$_}, ( $_ eq 2 ? encode_boolean(@{$person}[$_]) : @{$person}[$_] ) } grep { defined @{$person}[$_] && @{$person}[$_] ne '' } keys %attrs; $h = @{$person}[3] if defined @{$person}[3]; # is there an 'image' or 'url' list? } else { # other roles do not have any attributes $h = @{$person}[1] if defined @{$person}[1]; } $w->startTag($role, @attrs) if $w; # open the container $w->characters(@{$person}[0]) if $w; # write the #pcdata if ( defined $h && scalar keys %$h ) { # add the 'image' and 'url' elements # writer is opened with DATA_MODE => 1 but that prevents mixed content, so tempo turn it off my $w_mode = $w->getDataMode() if ($w); $w->setDataMode(0) if ($w); my $j=2; $j = $w->getDataIndent() if ($w); $j *= 4; # now we'll have to indent manually :-( Guess we're at level 4. my $i=0; foreach my $k ( qw[ image url ] ) { next unless defined $h->{$k}; my $v = $h->{$k}; my $h = $Handlers{$k}; die "no handler $k" if not $h; # write each sub-element foreach (@{$v}) { $w->characters("\n".(' ' x $j)) if ($w); my $ret = $h->[1]->($w, $k, $_) if ($w); # call writer sub $i++; } } $j = $j/4*3; $w->characters("\n".(' ' x $j)) if ($w and $i); # data_mode doesn't reset cleanly # reset the data_mode $w->setDataMode($w_mode) if ($w); } $w->endTag($role) if $w; } else { $w->dataElement($_, $person) if $w; } } } warn_unknown_keys($e, \%h); $w->endTag($e) if $w; }; =pod =item I<scalar> Reads and writes a simple string as the content of the XML element. =cut $Handlers{scalar}->[0] = sub( $ ) { my $node = shift; return get_text($node); }; $Handlers{scalar}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; t 'scalar'; $w->dataElement($e, $v) if $w; }; =pod =item I<url> Reads and writes the <url> element. For backward compatibility it returns a scalar if no 'system' attribute. Otherwise it returns an array of value + 'system' attribute =cut $Handlers{'url'}->[0] = sub( $ ) { my $node = shift; die if not defined $node; my %attrs = %{get_attrs($node)}; my $system = delete $attrs{system} if exists $attrs{system}; foreach (keys %attrs) { warn "unknown attribute in url: $_"; } my $url = get_text($node); return $url if not defined $system or $system eq ''; return [ $url, $system ]; }; $Handlers{'url'}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; t 'url'; if (not ref $v or ref $v ne 'ARRAY') { $w->dataElement($e, $v) if $w; return; } my ($url, $system) = @$v; # second in list is assumed to be the 'system' attribute $w->dataElement($e, $url, system => $system) if $w; }; =pod =item I<length> Converts the content of a <length> element into a number of seconds (so <length units="minutes">5</minutes> would be returned as 300). On writing out again tries to convert a number of seconds to a time in minutes or hours if that would look better. =cut $Handlers{length}->[0] = sub( $ ) { my $node = shift; die if not defined $node; my %attrs = %{get_attrs($node)}; my $d = get_text($node); if ($d =~ /^\s*$/) { warn "empty 'length' element"; return undef; } if ($d !~ tr/0-9// or $d =~ tr/0-9//c) { warn "bad content of 'length' element: $d"; return undef; } my $units = $attrs{units}; if (not defined $units) { warn "missing 'units' attr in 'length' element"; return undef; } # We want to return a length in seconds. if ($units eq 'seconds') { # Okay. } elsif ($units eq 'minutes') { $d *= 60; } elsif ($units eq 'hours') { $d *= 60 * 60; } else { warn "bad value of 'units': $units"; return undef; } return $d; }; $Handlers{length}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; t 'length'; my $units; if ($v % 3600 == 0) { $units = 'hours'; $v /= 3600; } elsif ($v % 60 == 0) { $units = 'minutes'; $v /= 60; } else { $units = 'seconds'; } $w->dataElement($e, $v, units => $units) if $w; }; =pod =item I<episode-num> The representation in Perl of XMLTVE<39>s odd episode numbers is as a pair of [ content, system ]. As specified by the DTD, if the system is not given in the file then 'onscreen' is assumed. Whitespace in the 'xmltv_ns' system is unimportant, so on reading it is normalized to a single space on either side of each dot. =cut $Handlers{'episode-num'}->[0] = sub( $ ) { my $node = shift; die if not defined $node; my %attrs = %{get_attrs($node)}; my $system = $attrs{system}; $system = 'onscreen' if not defined $system; my $content = get_text($node); if ($system eq 'xmltv_ns') { # Make it look nice. $content =~ s/\s+//g; $content =~ s/\./ . /g; } return [ $content, $system ]; }; $Handlers{'episode-num'}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; t 'episode number'; if (not ref $v or ref $v ne 'ARRAY') { warn "not writing episode-num whose content is not an array"; return; } my ($content, $system) = @$v; $system = 'onscreen' if not defined $system; $w->dataElement($e, $content, system => $system) if $w; }; =pod =item I<video> The <video> section is converted to a hash. The <present> subelement corresponds to the key 'present' of this hash, 'yes' and 'no' are converted to Booleans. The same applies to <colour>. The content of the <aspect> subelement is stored under the key 'aspect'. These keys can be missing in the hash just as the subelements can be missing in the XML. =cut $Handlers{video}->[0] = sub ( $ ) { my $node = shift; my %r; foreach (get_subelements($node)) { my $name = get_name($_); my $value = get_text($_); if ($name eq 'present') { warn "'present' seen twice" if defined $r{present}; $r{present} = decode_boolean($value); } elsif ($name eq 'colour') { warn "'colour' seen twice" if defined $r{colour}; $r{colour} = decode_boolean($value); } elsif ($name eq 'aspect') { warn "'aspect' seen twice" if defined $r{aspect}; $value =~ /^\d+:\d+$/ or warn "bad aspect ratio: $value"; $r{aspect} = $value; } elsif ($name eq 'quality') { warn "'quality' seen twice" if defined $r{quality}; $r{quality} = $value; } } return \%r; }; $Handlers{video}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; t "'video' element"; my %h = %$v; return if not %h; # don't write empty element $w->startTag($e) if $w; if (defined (my $val = delete $h{present})) { $w->dataElement('present', encode_boolean($val)) if $w; } if (defined (my $val = delete $h{colour})) { $w->dataElement('colour', encode_boolean($val)) if $w; } if (defined (my $val = delete $h{aspect})) { $w->dataElement('aspect', $val) if $w; } if (defined (my $val = delete $h{quality})) { $w->dataElement('quality', $val) if $w; } warn_unknown_keys("zz $e", \%h); $w->endTag($e) if $w; }; =pod =item I<audio> This is similar to I<video>. <present> is a Boolean value, while the content of <stereo> is stored unchanged. =cut $Handlers{audio}->[0] = sub( $ ) { my $node = shift; my %r; foreach (get_subelements($node)) { my $name = get_name($_); my $value = get_text($_); if ($name eq 'present') { warn "'present' seen twice" if defined $r{present}; $r{present} = decode_boolean($value); } elsif ($name eq 'stereo') { warn "'stereo' seen twice" if defined $r{stereo}; if ($value eq '') { warn "empty 'stereo' element not permitted, should be <stereo>stereo</stereo>"; $value = 'stereo'; } warn "bad value for 'stereo': '$value'" if ($value ne 'mono' and $value ne 'stereo' and $value ne 'bilingual' and $value ne 'surround' and $value ne 'dolby digital' and $value ne 'dolby'); $r{stereo} = $value; } } return \%r; }; $Handlers{audio}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; my %h = %$v; return if not %h; # don't write empty element $w->startTag($e) if $w; if (defined (my $val = delete $h{present})) { $w->dataElement('present', encode_boolean($val)) if $w; } if (defined (my $val = delete $h{stereo})) { $w->dataElement('stereo', $val) if $w; } warn_unknown_keys($e, \%h); $w->endTag($e) if $w; }; =pod =item I<previously-shown> The 'start' and 'channel' attributes are converted to keys in a hash. =cut $Handlers{'previously-shown'}->[0] = sub( $ ) { my $node = shift; die if not defined $node; my %attrs = %{get_attrs($node)}; my $r = {}; foreach (qw(start channel)) { my $v = delete $attrs{$_}; $r->{$_} = $v if defined $v; } foreach (keys %attrs) { warn "unknown attribute $_ in previously-shown"; } return $r; }; $Handlers{'previously-shown'}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; my @v = map{ $_, $v->{$_} } grep {defined $v->{$_}} qw( start channel ); $w->emptyTag($e, @v) if $w; }; =pod =item I<presence> The content of the element is ignored: it signfies something by its very presence. So the conversion from XML to Perl is a constant true value whenever the element is found; the conversion from Perl to XML is to write out the element if true, donE<39>t write anything if false. =cut $Handlers{presence}->[0] = sub( $ ) { my $node = shift; # The 'new' element is empty, it signifies newness by its very # presence. # return 1; }; $Handlers{presence}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; if (not $v) { # Not new, so don't create an element. } else { $w->emptyTag($e) if $w; } }; =pod =item I<subtitles> The 'type' attribute and the 'language' subelement (both optional) become keys in a hash. But see I<language> for what to pass as the value of that element. =cut $Handlers{subtitles}->[0] = sub( $ ) { my $node = shift; die if not defined $node; my %attrs = %{get_attrs($node)}; my %r; $r{type} = $attrs{type} if defined $attrs{type}; foreach (get_subelements($node)) { my $name = get_name($_); if ($name eq 'language') { warn "'language' seen twice" if defined $r{language}; $r{language} = read_with_lang($_, 0, 0); } else { warn "bad content of 'subtitles' element: $name"; } } return \%r; }; $Handlers{subtitles}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; t 'subtitles'; my ($type, $language) = ($v->{type}, $v->{language}); my %attrs; $attrs{type} = $type if defined $type; if (defined $language) { $w->startTag($e, %attrs) if $w; write_with_lang($w, 'language', $language, 0, 0); $w->endTag($e) if $w; } else { $w->emptyTag($e, %attrs) if $w; } }; =pod =item I<rating> The rating is represented as a tuple of [ rating, system, icons ]. The last element is itself a listref of structures returned by the I<icon> handler. =cut $Handlers{rating}->[0] = sub( $ ) { my $node = shift; die if not defined $node; my %attrs = %{get_attrs($node)}; my $system = delete $attrs{system} if exists $attrs{system}; foreach (keys %attrs) { warn "unknown attribute in rating: $_"; } my @children = get_subelements($node); # First child node is value. my $value_node = shift @children; if (not defined $value_node) { warn "missing 'value' element inside rating"; return undef; } if ((my $name = get_name($value_node)) ne 'value') { warn "expected 'value' node inside rating, got '$name'"; return undef; } my $rating = read_value($value_node); # Remaining children are icons. my @icons = map { read_icon($_) } @children; return [ $rating, $system, \@icons ]; }; $Handlers{rating}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; if (not ref $v or ref $v ne 'ARRAY') { warn "not writing rating whose content is not an array"; return; } my ($rating, $system, $icons) = @$v; if (defined $system) { $w->startTag($e, system => $system) if $w; } else { $w->startTag($e) if $w; } write_value($w, 'value', $rating) if $w; if ($w) { write_icon($w, 'icon', $_) foreach @$icons }; $w->endTag($e) if $w; }; =pod =item I<star-rating> In XML this is a string 'X/Y' plus a list of icons. In Perl represented as a pair [ rating, icons ] similar to I<rating>. Multiple star ratings are now supported. For backward compatibility, you may specify a single [rating,icon] or the preferred double array [[rating,system,icon],[rating2,system2,icon2]] (like 'ratings') =cut $Handlers{'star-rating'}->[0] = sub( $ ) { my $node = shift; my %attrs = %{get_attrs($node)}; my $system = delete $attrs{system} if exists $attrs{system}; my @children = get_subelements($node); # First child node is value. my $value_node = shift @children; if (not defined $value_node) { warn "missing 'value' element inside star-rating"; return undef; } if ((my $name = get_name($value_node)) ne 'value') { warn "expected 'value' node inside star-rating, got '$name'"; return undef; } my $rating = read_value($value_node); # Remaining children are icons. my @icons = map { read_icon($_) } @children; return [ $rating, $system, \@icons ]; }; $Handlers{'star-rating'}->[1] = sub ( $$$ ) { my ($w, $e, $v) = @_; # # 10/31/2007 star-rating can now have multiple values (and system=) # let's make it so old code still works! # if (not ref $v or ref $v ne 'ARRAY') { $v=[$v]; # warn "not writing star-rating whose content is not an array"; # return; } my ($rating, $system, $icons) = @$v; if (defined $system) { $w->startTag($e, system => $system) if $w; } else { $w->startTag($e) if $w; } write_value($w, 'value', $rating) if $w; if ($w) { write_icon($w, 'icon', $_) foreach @$icons }; $w->endTag($e) if $w; }; =pod =item I<review> In XML this is text with a number of optional attributes. In Perl represented by an array containing [0] #PCDATA [1] hash of attributes. =cut $Handlers{'review'}->[0] = sub( $ ) { my $node = shift; die if not defined $node; my %attrs = %{get_attrs($node)}; my $r = {}; foreach (qw(type source reviewer lang)) { my $v = delete $attrs{$_}; $r->{$_} = $v if defined $v; } foreach (keys %attrs) { warn "unknown attribute $_ in review"; } my $value = get_text($node, 1); if (not length $value) { warn 'empty string for review value'; return undef; } if (keys %$r) { return [ $value, $r ]; } else { return [ $value ]; } }; $Handlers{'review'}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; t 'review'; my $content=@{$v}[0]; my $attrs=@{$v}[1]; if (!defined $attrs->{type} || $attrs->{type} eq '') { croak "type attribute empty in review"; return undef; } if ($attrs->{type} ne 'text' && $attrs->{type} ne 'url') { croak "type attribute invalid in review"; return undef; } if (not length $content) { warn 'empty string for review value'; return undef; } my @v = map{ $_, $attrs->{$_} } grep {defined $attrs->{$_}} qw( type source reviewer lang ); if (scalar @v) { $w->dataElement($e, $content, @v) if $w; } else { $w->dataElement($e, $content) if $w; } }; =pod =item I<image> In XML this is text with a number of optional attributes. In Perl represented by an array containing [0] #PCDATA [1] hash of attributes. =cut $Handlers{'image'}->[0] = sub( $ ) { my $node = shift; die if not defined $node; my %attrs = %{get_attrs($node)}; my $r = {}; foreach (qw(type orient size system)) { my $v = delete $attrs{$_}; $r->{$_} = $v if defined $v; } foreach (keys %attrs) { warn "unknown attribute $_ in image"; } my $value = get_text($node, 1); if (not length $value) { warn 'empty string for image value'; return undef; } if (keys %$r) { return [ $value, $r ]; } else { return [ $value ]; } }; $Handlers{'image'}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; t 'image'; my $content=@{$v}[0]; my $attrs=@{$v}[1]; # don't write empty element if (not length $content) { warn 'empty string for image value'; return undef; } # validation if (defined $attrs and $attrs) { my %r = %$attrs; foreach (qw(type orient size system)) { my $v = delete $r{$_}; } foreach (keys %r) { warn "unknown attribute $_ in image"; } } if (defined $attrs->{type} and $attrs->{type} !~ m/^(poster|backdrop|still|person|character)$/ ) { croak "type attribute invalid in image"; return undef; } if (defined $attrs->{orient} and $attrs->{orient} !~ m/^[PL]$/ ) { croak "orient attribute invalid in image"; return undef; } if (defined $attrs->{size} and $attrs->{size} !~ m/^[123]$/ ) { croak "size attribute invalid in image"; return undef; } # writing my @v = map{ $_, $attrs->{$_} } grep {defined $attrs->{$_}} qw( type orient size system ); if (scalar @v) { $w->dataElement($e, $content, @v) if $w; } else { $w->dataElement($e, $content) if $w; } }; =pod =item I<icon> An icon in XMLTV files is like the <img> element in HTML. It is represented in Perl as a hashref with 'src' and optionally 'width' and 'height' keys. =cut sub write_icon( $$$ ) { my ($w, $e, $v) = @_; croak "no 'src' attribute for icon\n" if not defined $v->{src}; croak "bad width $v->{width} for icon\n" if defined $v->{width} and $v->{width} !~ /^\d+$/; croak "bad height $v->{height} for icon\n" if defined $v->{height} and $v->{height} !~ /^\d+$/; foreach (keys %$v) { warn "unrecognized key in icon: $_\n" if $_ ne 'src' and $_ ne 'width' and $_ ne 'height'; } my @v = map{ $_, $v->{$_} } grep {defined $v->{$_}} qw( src width height ); $w->emptyTag($e, @v); } sub read_icon( $ ) { my $node = shift; die if not defined $node; my %attrs = %{get_attrs($node)}; warn "missing 'src' attribute in icon" if not defined $attrs{src}; return \%attrs; } $Handlers{icon}->[0] = \&read_icon; $Handlers{icon}->[1] = sub( $$$ ) { my ($w, $e, $v) = @_; write_icon($w, $e, $v) if $w; }; # To keep things tidy some elements that can have icons store their # textual content inside a subelement called 'value'. These two # routines are a bit trivial but they're here for consistency. # sub read_value( $ ) { my $value_node = shift; my $v = get_text($value_node); if (not defined $v or $v eq '') { warn "no content of 'value' element"; return undef; } return $v; } sub write_value( $$$ ) { my ($w, $e, $v) = @_; $w->dataElement($e, $v) if $w; }; # Booleans in XMLTV files are 'yes' or 'no'. sub decode_boolean( $ ) { my $value = shift; if ($value eq 'no') { return 0; } elsif ($value eq 'yes') { return 1; } else { warn "bad boolean: $value"; return undef; } } sub encode_boolean( $ ) { my $v = shift; warn "expected a Perl boolean like 0 or 1, not '$v'\n" if $v and $v != 1; return $v ? 'yes' : 'no'; } =pod =item I<with-lang> In XML something like title can be either <title>Foo or Foo. In Perl these are stored as [ 'Foo' ] and [ 'Foo', 'en' ]. For the former [ 'Foo', undef ] would also be okay. This handler also has two modifiers which may be added to the name after '/'. I means that empty text is allowed, and will be returned as the empty tuple [], to mean that the element is present but has no text. When writing with I, undef will also be understood as present-but-empty. You cannot however specify a language if the text is empty. The modifier I means that the text is allowed to span multiple lines. So for example I is a handler for text with language, where the text may be empty and may contain newlines. Note that the I of earlier releases has been replaced by I. =cut sub read_with_lang( $$$ ) { my ($node, $allow_empty, $allow_nl) = @_; die if not defined $node; my %attrs = %{get_attrs($node)}; my $lang = $attrs{lang} if exists $attrs{lang}; my $value = get_text($node, $allow_nl); if (not length $value) { if (not $allow_empty) { warn 'empty string for with-lang value'; return undef; } warn 'empty string may not have language' if defined $lang; return []; } if (defined $lang) { return [ $value, $lang ]; } else { return [ $value ]; } } $Handlers{'with-lang'}->[0] = sub( $ ) { read_with_lang($_[0], 0, 0) }; $Handlers{'with-lang/'}->[0] = sub( $ ) { read_with_lang($_[0], 0, 0) }; $Handlers{'with-lang/e'}->[0] = sub( $ ) { read_with_lang($_[0], 1, 0) }; $Handlers{'with-lang/m'}->[0] = sub( $ ) { read_with_lang($_[0], 0, 1) }; $Handlers{'with-lang/em'}->[0] = sub( $ ) { read_with_lang($_[0], 1, 1) }; $Handlers{'with-lang/me'}->[0] = sub( $ ) { read_with_lang($_[0], 1, 1) }; sub write_with_lang( $$$$$ ) { my ($w, $e, $v, $allow_empty, $allow_nl) = @_; if (not ref $v or ref $v ne 'ARRAY') { warn "not writing with-lang whose content is not an array"; return; } if (not @$v) { if (not $allow_empty) { warn "not writing no content for $e"; return; } $v = [ '' ]; } my ($text, $lang) = @$v; t 'writing character data: ' . d $text; if (not defined $text) { warn "not writing undefined value for $e"; return; } # # strip whitespace silently. # we used to use a warn, but later on the code catches this and drops the record # my $old_text = $text; $text =~ s/^\s+//; $text =~ s/\s+$//; if (not length $text) { if (not $allow_empty) { warn "not writing empty content for $e"; return; } if (defined $lang) { warn "not writing empty content with language for $e"; return; } $w->emptyTag($e) if $w; return; } if (not $allow_nl and $text =~ tr/\n//) { warn "not writing text containing newlines for $e"; return; } if (defined $lang) { $w->dataElement($e, $text, lang => $lang) if $w; } else { $w->dataElement($e, $text) if $w; } } $Handlers{'with-lang'}->[1] = sub( $$$ ) { write_with_lang($_[0], $_[1], $_[2], 0, 0) }; $Handlers{'with-lang/'}->[1] = sub( $$$ ) { write_with_lang($_[0], $_[1], $_[2], 0, 0) }; $Handlers{'with-lang/e'}->[1] = sub( $$$ ) { write_with_lang($_[0], $_[1], $_[2], 1, 0) }; $Handlers{'with-lang/m'}->[1] = sub( $$$ ) { write_with_lang($_[0], $_[1], $_[2], 0, 1) }; $Handlers{'with-lang/em'}->[1] = sub( $$$ ) { write_with_lang($_[0], $_[1], $_[2], 1, 1) }; $Handlers{'with-lang/me'}->[1] = sub( $$$ ) { write_with_lang($_[0], $_[1], $_[2], 1, 1) }; # Sanity check. foreach (keys %Handlers) { my $v = $Handlers{$_}; if (@$v != 2 or ref($v->[0]) ne 'CODE' or ref($v->[1]) ne 'CODE') { die "bad handler pair for $_\n"; } } =pod =back Now, which handlers are used for which subelements (keys) of channels and programmes? And what is the multiplicity (should you expect a single value or a list of values)? The following tables map subelements of and of to the handlers used to read and write them. Many elements have their own handler with the same name, and most of the others use I. The third column specifies the multiplicity of the element: B<*> (any number) will give a list of values in Perl, B<+> (one or more) will give a nonempty list, B (maybe one) will give a scalar, and B<1> (exactly one) will give a scalar which is not undef. =head2 Handlers for @CHANNEL_HANDLERS =head2 Handlers for @PROGRAMME_HANDLERS At present, no parsing or validation on dates is done because dates may be partially specified in XMLTV. For example '2001' means that the year is known but not the month, day or time of day. Maybe in the future dates will be automatically converted to and from B objects. For now they just use the I handler. Similar remarks apply to URLs. =cut # Private. sub node_to_programme( $ ) { my $node = shift; die if not defined $node; my %programme; # Attributes of programme element. %programme = %{get_attrs($node)}; t 'attributes: ' . d \%programme; # Check the required attributes are there. As with most checking, # this isn't an alternative to using a validator but it does save # some headscratching during debugging. # foreach (qw(start channel)) { if (not defined $programme{$_}) { warn "programme missing '$_' attribute\n"; return undef; } } my @known_attrs = map { $_->[0] } @Programme_Attributes; my %ka; ++$ka{$_} foreach @known_attrs; foreach (keys %programme) { unless ($ka{$_}) { warn "deleting unknown attribute '$_'"; delete $programme{$_}; } } call_handlers_read($node, \@Programme_Handlers, \%programme); return \%programme; } # Private. sub node_to_channel( $ ) { my $node = shift; die if not defined $node; my %channel; t 'node_to_channel() ENTRY'; %channel = %{get_attrs($node)}; t 'attributes: ' . d \%channel; if (not defined $channel{id}) { warn "channel missing 'id' attribute\n"; } foreach (keys %channel) { unless (/^_/ or $_ eq 'id') { warn "deleting unknown attribute '$_'"; delete $channel{$_}; } } t '\@Channel_Handlers=' . d \@Channel_Handlers; call_handlers_read($node, \@Channel_Handlers, \%channel); return \%channel; } # Private. # # call_handlers_read() # # Read the subelements of a node according to a list giving a # handler subroutine for each subelement. # # Parameters: # node # Reference to list of handlers: tuples of # [element-name, handler-name, multiplicity] # Reference to hash for storing results # # Warns if errors, but attempts to contine. # sub call_handlers_read( $$$ ) { my ($node, $handlers, $r) = @_; t 'call_handlers_read() using handlers: ' . d $handlers; die unless ref($r) eq 'HASH'; our %r; local *r = $r; t 'going through each child of node'; # Current position in handlers. We expect to read the subelements # in the correct order as specified by the DTD. # my $handler_pos = 0; SUBELEMENT: foreach (get_subelements($node)) { t 'doing subelement'; my $name = get_name($_); t "tag name: $name"; # Search for a handler - from $handler_pos onwards. But # first, just warn if somebody is trying to use an element in # the wrong place (trying to go backwards in the list). # my $found_pos; foreach my $i (0 .. $handler_pos - 1) { if ($name eq $handlers->[$i]->[0]) { warn "element $name not expected here"; next SUBELEMENT; } } for (my $i = $handler_pos; $i < @$handlers; $i++) { if ($handlers->[$i]->[0] eq $name) { t 'found handler'; $found_pos = $i; last; } else { t "doesn't match name $handlers->[$i]->[0]"; my ($handler_name, $h, $multiplicity) = @{$handlers->[$i]}; die if not defined $handler_name; die if $handler_name eq ''; # Before we skip over this element, check that we got # the necessary values for it. # if ($multiplicity eq '?') { # Don't need to check whether this set. } elsif ($multiplicity eq '1') { if (not defined $r{$handler_name}) { warn "no element $handler_name found"; } } elsif ($multiplicity eq '*') { # It's okay if nothing was ever set. We don't # insist on putting in an empty list. # } elsif ($multiplicity eq '+') { if (not defined $r{$handler_name}) { warn "no element $handler_name found"; } elsif (not @{$r{$handler_name}}) { warn "strangely, empty list for $handler_name"; } } else { warn "bad value of $multiplicity: $!"; } } } if (not defined $found_pos) { warn "unknown element $name"; next; } # Next time we begin searching from this position. $handler_pos = $found_pos; # Call the handler. t 'calling handler'; my ($handler_name, $h_name, $multiplicity) = @{$handlers->[$found_pos]}; die if $handler_name ne $name; my $h = $Handlers{$h_name}; die "no handler $h_name" if not $h; my $result = $h->[0]->($_); # call reader sub t 'result: ' . d $result; warn("skipping bad $name\n"), next if not defined $result; # Now set the value. We can't do multiplicity checking yet # because there might be more elements of this type still to # come. # if ($multiplicity eq '?' or $multiplicity eq '1') { warn "seen $name twice" if defined $r{$name}; $r{$name} = $result; } elsif ($multiplicity eq '*' or $multiplicity eq '+') { push @{$r{$name}}, $result; } else { warn "bad multiplicity: $multiplicity"; } } } sub warn_unknown_keys( $$ ) { my $elem_name = shift; our %k; local *k = shift; foreach (keys %k) { /^_/ or $warned_unknown_key{$elem_name}->{$_}++ or warn "unknown key $_ in $elem_name hash\n"; } } package XMLTV::Writer; use base 'XML::Writer'; use Carp; use Date::Manip qw/UnixDate DateCalc/; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } BEGIN { if (int(Date::Manip::DateManipVersion) >= 6) { Date::Manip::Date_Init("SetDate=now,UTC"); } else { Date::Manip::Date_Init("TZ=UTC"); } } # Override dataElement() to refuse writing empty or whitespace # elements. # sub dataElement( $$$@ ) { my ($self, $elem, $content, @rest) = @_; if ($content !~ /\S/) { warn "not writing empty content for $elem"; return; } return $self->SUPER::dataElement($elem, $content, @rest); } =pod =head1 WRITING When reading a file you have the choice of using C to gulp the whole file and return a data structure, or using C to get the programmes one at a time, although channels and other data are still read all at once. There is a similar choice when writing data: the C routine prints a whole XMLTV document at once, but if you want to write an XMLTV document incrementally you can manually create an C object and call methods on it. Synopsis: use XMLTV; my $w = new XMLTV::Writer(); $w->comment("Hello from XML::Writer's comment() method"); $w->start({ 'generator-info-name' => 'Example code in pod' }); my %ch = (id => 'test-channel', 'display-name' => [ [ 'Test', 'en' ] ]); $w->write_channel(\%ch); my %prog = (channel => 'test-channel', start => '200203161500', title => [ [ 'News', 'en' ] ]); $w->write_programme(\%prog); $w->end(); XMLTV::Writer inherits from XML::Writer, and provides the following extra or overridden methods: =over =item new(), the constructor Creates an XMLTV::Writer object and starts writing an XMLTV file, printing the DOCTYPE line. Arguments are passed on to XML::WriterE<39>s constructor, except for the following: the 'encoding' key if present gives the XML character encoding. For example: my $w = new XMLTV::Writer(encoding => 'ISO-8859-1'); If encoding is not specified, XML::WriterE<39>s default is used (currently UTF-8). XMLTW::Writer can also filter out specific days from the data. This is useful if the datasource provides data for periods of time that does not match the days that the user has asked for. The filtering is controlled with the days, offset and cutoff arguments: my $w = new XMLTV::Writer( offset => 1, days => 2, cutoff => "050000" ); In this example, XMLTV::Writer will discard all entries that do not have starttimes larger than or equal to 05:00 tomorrow and less than 05:00 two days after tomorrow. The time offset is stripped off the starttime before the comparison is made. =cut sub new { my $proto = shift; my $class = ref($proto) || $proto; my %args = @_; croak 'OUTPUT requires a filehandle, not a filename or anything else' if exists $args{OUTPUT} and not ref $args{OUTPUT}; # force OUTPUT explicitly to standard output to avoid warnings about # undefined OUTPUT in XML::Writer where it tests against 'self' if (!exists $args{OUTPUT}) { $args{OUTPUT} = \*STDOUT; } my $encoding = delete $args{encoding}; my $days = delete $args{days}; my $offset = delete $args{offset}; my $cutoff = delete $args{cutoff}; my $self = $class->SUPER::new(DATA_MODE => 1, DATA_INDENT => 2, %args); bless($self, $class); if (defined $encoding) { $self->xmlDecl($encoding); } else { # XML::Writer puts in 'encoding="UTF-8"' even if you don't ask # for it. # warn "assuming default UTF-8 encoding for output\n"; $self->xmlDecl(); } # $Log::TraceMessages::On = 1; $self->{mintime} = "19700101000000"; $self->{maxtime} = "29991231235959"; if (defined( $days ) and defined( $offset ) and defined( $cutoff )) { $self->{mintime} = UnixDate( DateCalc( "today", "+" . $offset . " days" ), "%Y%m%d") . $cutoff; t "using mintime $self->{mintime}"; $self->{maxtime} = UnixDate( DateCalc("today", "+" . ($offset+$days) . " days"), "%Y%m%d" ) . $cutoff; t "using maxtime $self->{maxtime}"; } elsif (defined( $days ) or defined( $offset ) or defined($cutoff)) { croak 'You must specify days, offset and cutoff or none of them'; } { local $^W = 0; $self->doctype('tv', undef, 'xmltv.dtd'); } $self->{xmltv_writer_state} = 'new'; return $self; } =pod =item start() Write the start of the element. Parameter is a hashref which gives the attributes of this element. =cut sub start { my $self = shift; die 'usage: XMLTV::Writer->start(hashref of attrs)' if @_ != 1; my $attrs = shift; for ($self->{xmltv_writer_state}) { if ($_ eq 'new') { # Okay. } elsif ($_ eq 'channels' or $_ eq 'programmes') { croak 'cannot call start() more than once on XMLTV::Writer'; } elsif ($_ eq 'end') { croak 'cannot do anything with end()ed XMLTV::Writer'; } else { die } $_ = 'channels'; } $self->startTag('tv', order_attrs(%{$attrs})); } =pod =item write_channels() Write several channels at once. Parameter is a reference to a hash mapping channel id to channel details. They will be written sorted by id, which is reasonable since the order of channels in an XMLTV file isnE<39>t significant. =cut sub write_channels { my ($self, $channels) = @_; t('write_channels(' . d($self) . ', ' . d($channels) . ') ENTRY'); croak 'expected hashref of channels' if ref $channels ne 'HASH'; for ($self->{xmltv_writer_state}) { if ($_ eq 'new') { croak 'must call start() on XMLTV::Writer first'; } elsif ($_ eq 'channels') { # Okay. } elsif ($_ eq 'programmes') { croak 'cannot write channels after writing programmes'; } elsif ($_ eq 'end') { croak 'cannot do anything with end()ed XMLTV::Writer'; } else { die } } my @ids = sort keys %$channels; t 'sorted list of channel ids: ' . d \@ids; foreach (@ids) { t "writing channel with id $_"; my $ch = $channels->{$_}; $self->write_channel($ch); } t('write_channels() EXIT'); } =pod =item write_channel() Write a single channel. You can call this routine if you want, but most of the time C is a better interface. =cut sub write_channel { my ($self, $ch) = @_; croak 'undef channel hash passed' if not defined $ch; croak "expected a hashref, got: $ch" if ref $ch ne 'HASH'; for ($self->{xmltv_writer_state}) { if ($_ eq 'new') { croak 'must call start() on XMLTV::Writer first'; } elsif ($_ eq 'channels') { # Okay. } elsif ($_ eq 'programmes') { croak 'cannot write channels after writing programmes'; } elsif ($_ eq 'end') { croak 'cannot do anything with end()ed XMLTV::Writer'; } else { die } } my %ch = %$ch; # make a copy my $id = delete $ch{id}; die "no 'id' attribute in channel" if not defined $id; write_element_with_handlers($self, 'channel', { id => $id }, \@XMLTV::Channel_Handlers, \%ch); } =pod =item write_programme() Write details for a single programme as XML. =cut sub write_programme { my $self = shift; die 'usage: XMLTV::Writer->write_programme(programme hash)' if @_ != 1; my $ref = shift; croak 'write_programme() expects programme hashref' if ref $ref ne 'HASH'; t('write_programme(' . d($self) . ', ' . d($ref) . ') ENTRY'); for ($self->{xmltv_writer_state}) { if ($_ eq 'new') { croak 'must call start() on XMLTV::Writer first'; } elsif ($_ eq 'channels') { $_ = 'programmes'; } elsif ($_ eq 'programmes') { # Okay. } elsif ($_ eq 'end') { croak 'cannot do anything with end()ed XMLTV::Writer'; } else { die } } # We make a copy of the programme hash and delete elements from it # as they are dealt with; then we can easily spot any unhandled # elements at the end. # my %p = %$ref; # First deal with those hash keys that refer to metadata on when # the programme is broadcast. After taking those out of the hash, # we can use the handlers to output individual details. # my %attrs; die if not @XMLTV::Programme_Attributes; foreach (@XMLTV::Programme_Attributes) { my ($name, $mult) = @$_; t "looking for key $name"; my $val = delete $p{$name}; if ($mult eq '?') { # No need to check anything. } elsif ($mult eq '1') { if (not defined $val) { warn "programme hash missing $name key, skipping"; return; } } else { die "bad multiplicity for attribute: $mult" } $attrs{$name} = $val if defined $val; } # We use string comparisons without timeoffsets for comparing times. my( $start ) = split( /\s+/, $attrs{start} ); if( $start lt $self->{mintime} or $start ge $self->{maxtime} ) { t "skipping programme with start $attrs{start}"; return; } t "beginning 'programme' element"; write_element_with_handlers($self, 'programme', \%attrs, \@XMLTV::Programme_Handlers, \%p); } =pod =item end() Say youE<39>ve finished writing programmes. This ends the element and the file. =cut sub end { my $self = shift; for ($self->{xmltv_writer_state}) { if ($_ eq 'new') { croak 'must call start() on XMLTV::Writer first'; } elsif ($_ eq 'channels' or $_ eq 'programmes') { $_ = 'end'; } elsif ($_ eq 'end') { croak 'cannot do anything with end()ed XMLTV::Writer'; } else { die } } $self->endTag('tv'); $self->SUPER::end(@_); } # Private. # order_attrs() # # In XML the order of attributes is not significant. But to make # things look nice we try to output them in the same order as given in # the DTD. # # Takes a list of (key, value, key, value, ...) and returns one with # keys in a nice-looking order. # sub order_attrs { die "expected even number of elements, from a hash" if @_ % 2; my @a = ((map { $_->[0] } (@XMLTV::Channel_Attributes, @XMLTV::Programme_Attributes)), qw(date source-info-url source-info-name source-data-url generator-info-name generator-info-url)); my @r; my %in = @_; foreach (@a) { if (exists $in{$_}) { my $v = delete $in{$_}; push @r, $_, $v; } } foreach (sort keys %in) { warn "unknown attribute $_" unless /^_/; push @r, $_, $in{$_}; } return @r; } # Private. # # Writes the elements of a hash to an XMLTV::Writer using a list of # handlers. Deletes keys (modifying the hash passed in) as they are # written. # # Requires all mandatory keys be present in the hash - if you're not # sure then use check_multiplicity() first. # # Returns true if the element was successfully written, or if any # errors found don't look serious enough to cause bad XML. If the # XML::Writer object passed in is undef then nothing is written (since # the write handlers are coded like that.) # sub call_handlers_write( $$$ ) { my ($self, $handlers, $input) = @_; t 'writing input hash: ' . d $input; die if not defined $input; my $bad = 0; foreach (@$handlers) { my ($name, $h_name, $multiplicity) = @$_; my $h = $XMLTV::Handlers{$h_name}; die "no handler $h_name" if not $h; my $writer = $h->[1]; die if not defined $writer; t "doing handler for $name$multiplicity"; local $SIG{__WARN__} = sub { warn "$name element: $_[0]"; $bad = 1; }; my $val = delete $input->{$name}; t 'got value(s): ' . d $val; if ($multiplicity eq '1') { $writer->($self, $name, $val); } elsif ($multiplicity eq '?') { $writer->($self, $name, $val) if defined $val; } elsif ($multiplicity eq '*' or $multiplicity eq '+') { croak "value for key $name should be an array ref" if defined $val and ref $val ne 'ARRAY'; foreach (@{$val}) { t 'writing value: ' . d $_; $writer->($self, $name, $_); t 'finished writing multiple values'; } } else { warn "bad multiplicity specifier: $multiplicity"; } } t 'leftover keys: ' . d([ sort keys %$input ]); return not $bad; } # Private. # # Warns about missing keys that are supposed to be mandatory. Returns # true iff everything is okay. # sub check_multiplicity( $$ ) { my ($handlers, $input) = @_; foreach (@$handlers) { my ($name, $h_name, $multiplicity) = @$_; t "checking handler for $name: $h_name with multiplicity $multiplicity"; if ($multiplicity eq '1') { if (not defined $input->{$name}) { warn "hash missing value for $name"; return 0; } } elsif ($multiplicity eq '?') { # Okay if not present. } elsif ($multiplicity eq '*') { # Not present, or undef, is treated as empty list. } elsif ($multiplicity eq '+') { t 'one or more, checking for a listref with no undef values'; my $val = $input->{$name}; if (not defined $val) { warn "hash missing value for $name (expected list)"; return 0; } if (ref($val) ne 'ARRAY') { die "hash has bad contents for $name (expected list)"; return 0; } t 'all values: ' . d $val; my @new_val = grep { defined } @$val; t 'values that are defined: ' . d \@new_val; if (@new_val != @$val) { warn "hash had some undef elements in list for $name, removed"; @$val = @new_val; } if (not @$val) { warn "hash has empty list of $name properties (expected at least one)"; return 0; } } else { warn "bad multiplicity specifier: $multiplicity"; } } return 1; } # Private. # # Write a complete element with attributes, and subelements written # using call_handlers_write(). The advantage over doing it by hand is # that if some required keys are missing, nothing is written (rather # than an incomplete and invalid element). # sub write_element_with_handlers( $$$$$ ) { my ($w, $name, $attrs, $handlers, $hash) = @_; if (not check_multiplicity($handlers, $hash)) { warn "keys missing in $name hash, not writing"; return; } # Special 'debug' keys written as comments inside the element. my %debug_keys; foreach (grep /^debug/, keys %$hash) { $debug_keys{$_} = delete $hash->{$_}; } # Call all the handlers with no writer object and make sure # they're happy. # if (not call_handlers_write(undef, $handlers, { %$hash })) { warn "bad data inside $name element, not writing\n"; return; } $w->startTag($name, order_attrs(%$attrs)); foreach (sort keys %debug_keys) { my $val = $debug_keys{$_}; $w->comment((defined $val) ? "$_: $val" : $_); } call_handlers_write($w, $handlers, $hash); XMLTV::warn_unknown_keys($name, $hash); $w->endTag($name); } =pod =back =head1 AUTHOR Ed Avis, ed@membled.com =head1 SEE ALSO The file format is defined by the DTD xmltv.dtd, which is included in the xmltv package along with this module. It should be installed in your systemE<39>s standard place for SGML and XML DTDs. The xmltv package has a web page at which carries information about the file format and the various tools and apps which are distributed with this module. =cut 1; xmltv-1.4.0/lib/exe_opt.pl000077500000000000000000000005671500074233200154430ustar00rootroot00000000000000#!perl -w # # This is a simple script to generate options so PerlApp can make the EXE # # Robert Eden rmeden@yahoo.com use File::Spec; # # output constants # print ' -M XMLTV:: -M Date::Manip:: -M DateTime:: -M Params::Validate:: -M Date::Language:: -M Class::MethodMaker:: -M Encode::Byte:: -X JSON::PP58 -X Test::Builder::IO::Scalar -X Win32::Console -a exe_files.txt '; xmltv-1.4.0/lib/set_share_dir.pl000066400000000000000000000020551500074233200166020ustar00rootroot00000000000000# Fragment of Perl code included from some .PL files. Arguments # # Input filename to read from # Output filename to write to # Share directory to set use IO::File; use strict; sub set_share_dir( $$$ ) { my $in = shift; my $out = shift; die "no output file given" if not defined $out; my $share_dir = shift; die "no final share/ location given" if not defined $share_dir; my $out_fh = new IO::File "> $out" or die "cannot write to $out: $!"; my $in_fh = new IO::File "< $in" or die "cannot read $in: $!"; my $seen = 0; while (<$in_fh>) { # Perl s/^my \$SHARE_DIR =.*/my \$SHARE_DIR='$share_dir'; \# by $0/ && $seen++; # Python s/^SHARE_DIR\s*=\s*None$/SHARE_DIR='$share_dir' \# by $0/ && $seen++; print $out_fh $_; } if ($seen == 0) { die "did not see SHARE_DIR line in $in"; } elsif ($seen == 1) { # Okay. } elsif ($seen >= 2) { warn "more than one SHARE_DIR line in $in"; } else { die } close $out_fh or die "cannot close $out: $!"; close $in_fh or die "cannot close $in: $!"; } 1; xmltv-1.4.0/lib/xmltv.pl000077500000000000000000000074331500074233200151510ustar00rootroot00000000000000#!perl -w # # This is a quick XMLTV shell routing to use with the windows exe # # A single EXE is needed to allow sharing of modules and dlls of all the # programs. # # Now uses PAR::Packer to build the exe. It takes a very long time on first run, which can # appear to be a problem. # # There currently isn't a way for PAR::Packer to warn users about a first time run. # I've modified the boot.c file in Par::Packer to do that. It's not great as it also # displays when building, but it's good enough. Here's what the change is (for documenation purposes) # I'm trying to work with the PAR::Packer folks for a better fix. # # boot.c:188 # rc = my_mkdir(stmpdir, 0700); #// 2021-01-18 rmeden hack to print a message on first run # if ( rc == 0 ) fprintf(stderr,"Note: This will take a while on first run\n"); #// rmeden # if ( rc == -1 && errno != EEXIST) { # # # Robert Eden rmeden@yahoo.com # use File::Basename; use Carp; use XMLTV; use Date::Manip; use DateTime; use Params::Validate; use Date::Language; use Class::MethodMaker; use Class::MethodMaker::Engine; $Carp::MaxEvalLen=40; # limit confess output # # check for --quiet # my $opt_quiet=0; foreach (@ARGV) {$opt_quiet = 1 if /--quiet/i }; # # get/check time zone # unless (exists $ENV{TZ}) { my $now = time(); my $lhour = (localtime($now))[2]; my $ghour = ( gmtime($now))[2]; my $tz = ($lhour - $ghour); $tz -= 24 if $tz > 12; $tz += 24 if $tz < -12; $tz= sprintf("%+03d00",$tz); $ENV{TZ}= $tz; } #timezone print STDERR "Timezone is $ENV{TZ}\n" unless $opt_quiet; $cmd = shift || ""; # --version (and abbreviations thereof) if (index('--version', $cmd) == 0 and length $cmd >= 3) { print "xmltv $XMLTV::VERSION\n"; exit; } # # some programs use a "share" directory # if ($cmd eq 'tv_grab_na_dd', or $cmd eq 'tv_grab_na_icons', ) { unless (grep(/^--share/i,@ARGV)) # don't add our --share if one supplied { my $dir = dirname($0); # get full program path $dir =~ s!\\!/!g; # use / not \ $dir .= "/share/xmltv"; unless (-d $dir ) { die "directory $dir not found\n If not kept with the executable, specify with --share\n" } print STDERR "adding '--share=$dir'\n" unless $opt_quiet; push @ARGV,"--share",$dir; } } # # special hack, allow "exec" to execute an arbitrary script # This will be used to allow XMLTV.EXE modules to be used on beta code w/o an alpha exe # # Note, no extra modules are included in the EXE. There is no guarantee this will work # it is an unsupported hack. # # syntax XMLTV.EXE exec filename --options # if ($cmd eq 'exec') { my $exe=shift; $0=$exe; print "doing $exe\n"; print STDERR "STDERR doing $exe\n"; do "./$exe"; print STDERR $@ if length($@); print "STDOUT $@" if length($@); exit 1 if length($@); exit 0; } # # scan through attached files and execute program if found # #main thread! $files=PAR::read_file("exe_files.txt"); foreach my $exe (split(/ /,$files)) { next unless length($exe)>3; #ignore trash $_=$exe; s!^.+/!!g; $cmds{$_}=1; # build command list (just in case) next unless $cmd eq $_; $exe="script/$cmd"; # # execute our command # do $exe; print STDERR $@ if length($@); exit 1 if length($@); exit 0; } # # command not found, print error # if ($cmd eq "" ) { print STDERR "you must specify the program to run\n for example: $0 tv_grab_fi --configure\n"; } else { print STDERR "$cmd is not a valid command.\n"; } print STDERR "Valid commands are:\n"; @cmds=sort keys %cmds; $rows = int($#cmds / 3)+1; map {$_='' unless defined $_} @cmds[0..($rows*3+2)]; unshift @cmds,undef; foreach (1..$rows) { printf STDERR " %-20s %-20s %-20s\n",@cmds[$_,$rows+$_,2*$rows+$_]; } exit 1; xmltv-1.4.0/lib/xmltv32.pl000077700000000000000000000000001500074233200170062xmltv.plustar00rootroot00000000000000xmltv-1.4.0/t/000077500000000000000000000000001500074233200131255ustar00rootroot00000000000000xmltv-1.4.0/t/README000066400000000000000000000002501500074233200140020ustar00rootroot00000000000000parallel_test is sometimes used to check new versions of grabbers - which depend on copyrighted listings data, so a full test suite for grabbers cannot be distributed. xmltv-1.4.0/t/add_time_info000077500000000000000000000067621500074233200156470ustar00rootroot00000000000000#!/usr/bin/perl -w # # add_time_info # # Quick kludge for testing output from two different grabbers. # Sometimes one listings source will be more informative than another # about the timing of programmes. Whereas the old source gave two # programmes sharing a clump from 11:00 to 12:00, the new one tells # you that one runs from 11:00 to 11:35 and the second from 11:35 to # 12:00. So the new listings source gives more information, but when # diffing the results there will be a discrepancy and seeming 'error'. # The answer is to patch up the old results where they agree with, but # are less detailed than, the new. # # Usage: reads 'less detailed' listings from stdin and 'more detailed' # given as a filename argument, outputs fixed-up version of 'less # detailed' to stdout. use strict; use XMLTV; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; } } # Use 'old' to mean the listings read from stdin, 'new' for those # given as an argument. Just as a shorthand. # my $old_data = XMLTV::parsefile('-'); my $new_data = XMLTV::parsefile(shift @ARGV); #$Log::TraceMessages::On = 1; my %interested; foreach (@{$old_data->[3]}) { next unless defined $_->{clumpidx} and $_->{clumpidx} ne '0/1'; push @{$interested{$_->{channel}}->{$_->{start}}}, $_; } t '\%interested=' . d \%interested; my (%new, %new_channels); foreach (@{$new_data->[3]}) { my $ch = $_->{channel}; push @{$new{$ch}->{$_->{start}}}, $_; $new_channels{$ch} = 1; } my %warned_ch; foreach my $ch (keys %interested) { if (not $new_channels{$ch}) { warn "unable to process channel $ch since not included in more detailed output\n" unless $warned_ch{$ch}++; next; } my $s = $interested{$ch}; my $n = $new{$ch}; t "doing channel $ch"; t 'fixing up: ' . d $s; t 'based on: ' . d $n; START: foreach my $start (keys %$s) { my @to_replace = @{$s->{$start}}; die "funny clump size at $start on $ch" if @to_replace < 2; t 'clump to replace: ' . d \@to_replace; my $r = $n->{$start}; die "no programmes to replace with at $start on $ch" if not defined $r; die if ref $r ne 'ARRAY'; my @replacement = @$r; die "no programmes to replace with at $start on $ch" if not @replacement; t 'replacement: ' . d \@replacement; my $i = 0; REPLACE: die "too many programmes to replace with" if @replacement > @to_replace; foreach (@replacement) { my $old = $to_replace[$i]; t 'updating: ' . d $old; t '...based on: ' . d $_; foreach my $key (qw(start stop clumpidx)) { if (exists $_->{$key}) { $old->{$key} = $_->{$key}; } else { delete $old->{$key}; } } t 'new version: ' . d $old; ++ $i; t "so far replaced $i programmes"; } die if $i > @to_replace; if ($i == @to_replace) { t 'end of clump'; next START; } t 'still some to replace, move forward in time'; my $prev = $replacement[-1]; die if not $prev; my $follow_on_start = $prev->{stop}; die "can't find follow-on replacement: no stop time in prev ($prev->{start}, $prev->{channel})" if not defined $follow_on_start; t "looking for programme in new listings at $follow_on_start"; my $follow_on = $n->{$follow_on_start}; die "can't find follow-on replacement: none at $follow_on_start on $ch" if not defined $follow_on; @replacement = @$follow_on; die if not @replacement; goto REPLACE; } } XMLTV::write_data($old_data); xmltv-1.4.0/t/data-tv_augment/000077500000000000000000000000001500074233200162055ustar00rootroot00000000000000xmltv-1.4.0/t/data-tv_augment/configs/000077500000000000000000000000001500074233200176355ustar00rootroot00000000000000xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_automatic_type_1.xml.conf000066400000000000000000000072661500074233200270160ustar00rootroot00000000000000# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from . remove_duplicated_new_title_in_ep = 1 # Rule #A2 - Remove duplicated programme title *and* episode from . remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from . remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in . # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 0 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_automatic_type_2.xml.conf��������������������������0000664�0000000�0000000�00000007266�15000742332�0027017�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 1 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 0 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_automatic_type_3.xml.conf��������������������������0000664�0000000�0000000�00000007266�15000742332�0027020�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 1 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 0 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_automatic_type_4.xml.conf��������������������������0000664�0000000�0000000�00000007266�15000742332�0027021�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 1 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 0 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_automatic_type_5-0a.xml.conf�����������������������0000664�0000000�0000000�00000007437�15000742332�0027320�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 1 # Update existing programme numbering if extracted from title/episode/part update_existing_numbering = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 1 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 1 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_automatic_type_5-0b.xml.conf�����������������������0000664�0000000�0000000�00000007437�15000742332�0027321�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 1 # Update existing programme numbering if extracted from title/episode/part update_existing_numbering = 1 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 1 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 1 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ���������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_automatic_type_5-1.xml.conf������������������������0000664�0000000�0000000�00000007266�15000742332�0027160�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 1 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 1 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 0 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_automatic_type_5-2.xml.conf������������������������0000664�0000000�0000000�00000007266�15000742332�0027161�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 1 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 1 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 0 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_automatic_type_5-3.xml.conf������������������������0000664�0000000�0000000�00000007266�15000742332�0027162�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 1 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 1 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 0 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_automatic_type_5-4.xml.conf������������������������0000664�0000000�0000000�00000007320�15000742332�0027152�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 1 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. # (requires #A5 enabled) make_episode_from_part_numbers = 1 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 0 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_1.xml.conf�������������������������������0000664�0000000�0000000�00000007266�15000742332�0026006�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 1 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_10.xml.conf������������������������������0000664�0000000�0000000�00000007266�15000742332�0026066�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 1 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_11.xml.conf������������������������������0000664�0000000�0000000�00000007266�15000742332�0026067�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 1 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_12.xml.conf������������������������������0000664�0000000�0000000�00000007266�15000742332�0026070�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 1 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_13.xml.conf������������������������������0000664�0000000�0000000�00000007266�15000742332�0026071�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 1 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_14.xml.conf������������������������������0000664�0000000�0000000�00000007266�15000742332�0026072�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 1 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_15.xml.conf������������������������������0000664�0000000�0000000�00000007266�15000742332�0026073�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 1 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_16.xml.conf������������������������������0000664�0000000�0000000�00000007441�15000742332�0026067�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 # Rule #16 - Remove episode numbering from a programme title process_remove_numbering_from_programmes = 1 �������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_2.xml.conf�������������������������������0000664�0000000�0000000�00000007266�15000742332�0026007�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 1 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_3.xml.conf�������������������������������0000664�0000000�0000000�00000007266�15000742332�0026010�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 1 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_4.xml.conf�������������������������������0000664�0000000�0000000�00000007266�15000742332�0026011�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 1 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_5.xml.conf�������������������������������0000664�0000000�0000000�00000007266�15000742332�0026012�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 1 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_6.xml.conf�������������������������������0000664�0000000�0000000�00000007266�15000742332�0026013�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 1 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_7.xml.conf�������������������������������0000664�0000000�0000000�00000007266�15000742332�0026014�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 1 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_8.xml.conf�������������������������������0000664�0000000�0000000�00000007266�15000742332�0026015�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 1 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 0 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/configs/tv_augment_user_type_9.xml.conf�������������������������������0000664�0000000�0000000�00000007266�15000742332�0026016�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample configuration file for tv_augment # # This file controls which augmentation rules are applied to the input XMLTV file # # # It also controls what reporting is printed in the program's output log file. # # Set an option to 1 to enable it, 0 to disable. # # If specified, then this language code will be written to e.g. <credit> elements language_code = en # Set the following values to have XMLTV::Augment automatically fetch a file # from the grabber's repository using XMLTV::Supplement use_supplement = 0 supplement_grabber_name = tv_grab_uk_rt supplement_grabber_file = prog_titles_to_process ############################################################################### # # RULES # ===== # # The option called 'enable_all_options' is a 'super-option' to quickly turn # on or off ALL automatic and user rules. If this is set then ALL individual # options are ignored. # enable_all_options = 0 # AUTOMATIC RULES # =============== # # The rules are pre-determined and use no user-defined data. # # Rule #A1 - Remove "New $title :" from <sub-title>. remove_duplicated_new_title_in_ep = 0 # Rule #A2 - Remove duplicated programme title *and* episode from <sub-title>. remove_duplicated_title_and_ep_in_ep = 0 # Rule #A3 - Remove duplicated programme title from <sub-title>. remove_duplicated_title_in_ep = 0 # Rule #A4 - update_premiere_repeat_flags_from_desc = 0 # Rule #A5 - Check for potential series, episode and part numbering in the title, episode and description fields. check_potential_numbering_in_text = 0 # Rule #A5.1 - Extract series/episode numbering found in <title>. # (requires #A5 enabled) # may generate false matches so use this only if your data needs it extract_numbering_from_title = 0 # Rule #A5.2 - Extract series/episode numbering found in <sub-title>. # (requires #A5 enabled)) # may generate false matches so use this only if your data needs it extract_numbering_from_episode = 0 # Rule #A5.3 - Extract series/episode numbering found in <desc>. # (requires #A5 enabled) extract_numbering_from_desc = 0 # Rule #A6 - If no <sub-title> then make one from "part" numbers. make_episode_from_part_numbers = 0 # USER RULES # ========== # # These rules use data contained within the 'fixup' rules file to control their action. # process_user_rules = 1 # Rule #1 - Remove specified non-title text from <title>. process_non_title_info = 0 # Rule #2 - Extract sub-title from <title>. process_mixed_title_subtitle = 0 # Rule #3 - Extract sub-title from <title>. process_mixed_subtitle_title = 0 # Rule #4 - Reverse <title> and <sub-title> process_reversed_title_subtitle = 0 # Rule #5 - Replace <title> with supplied text. process_replacement_titles = 0 # Rule #6 - Replace <category> with supplied text. process_replacement_genres = 0 # Rule #7 - Replace <sub-title> with supplied text. process_replacement_episodes = 0 # Rule #8 - Replace specified <title> / <sub-title> with title/episode pair supplied. process_replacement_titles_episodes = 0 # Rule #9 - Replace <sub-title> with supplied text when the <desc> matches that given. process_replacement_ep_from_desc = 1 # Rule #10 - Replace specified <title> / <sub-title> with title/episode pair supplied using <desc>. process_replacement_titles_desc = 0 # Rule #11 - Promote demoted title from <sub-title> to <title>. process_demoted_titles = 0 # Rule #12 - Replace "Film"/"Films" <category> with supplied text. process_replacement_film_genres = 0 # Rule #13 - Remove specified text from <sub-title> for a given <title>. process_subtitle_remove_text = 0 # Rule #14 - Replace specified categories with another value process_translate_genres = 0 # Rule #15 - Add specified categories to all programmes on a channel process_add_genres_to_channel = 0 ������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/rules/����������������������������������������������������������������0000775�0000000�0000000�00000000000�15000742332�0017337�5����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/rules/test_tv_augment.rules�������������������������������������������0000664�0000000�0000000�00000027440�15000742332�0023632�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������# # Sample "rules" file for use with tv_augment # ############################################################################### # # This file contains the rules used by XMLTV::Augment. # # The objective is to fix errors and inconsistencies in the incoming data from # a grabber, and to enhance the programme xml where certain data are missing. # # For example: # Some programme entries in the listings data may contain subtitle/episode # information in the title field, others may contain the programme title # and subtitle reversed, and yet more may contain 'episode' information that # should be in the episode-num field. # # Rules are divided into a number of 'types' according to what they do. # Individual types (rule sets) can be switched off in the augmentation config # file. # # Matching is usually (but not exclusively) done by comparing the incoming # <title> against the title specified in the rule below. # # A 'rule' definition consists of the rule 'type' separated rom the rule # content by a pipe character ('|'). The rule content has a variable (but fixed # for any given rule type) number of fields separated by tilde characters ('~') # # The action taken depends on the rule type: # # 1) non_title_info # The content is text that is to be removed from any programme titles where # this text occurs at the beginning of the <title> element followed by # any of : ; or , # e.g. # 1|Action Heroes Season # "Action Heroes Season: Rambo" --> "Rambo" # # 2) mixed_title_subtitle # The content is the desired title of a programme when the incoming title # contains both the programme's title *and* episode separated by : ; or - # ($title:$episode). We reassign the episode information to the <episode> # element, leaving only the programme title in the <title> element. # e.g. # 2|Blackadder II # "Blackadder II: Potato / " --> "Blackadder II / Potato" # # 3) mixed_subtitle_title # The content is the desired title of a programme when the incoming title # contains both the programme's episode *and* title separated by : ; or - # ($episode:$title). We reassign the episode information to the <episode> # element, leaving only the programme title in the <title> element. # e.g. # 3|Storyville # "Kings of Pastry: Storyville / " --> "Storyville / Kings of Pastry" # # 4) reversed_title_subtitle # The content is the desired title of a programme which is listed as the # programme's episode (i.e. the title and episode details have been # reversed). We therefore reverse the <title> and <episode> elements. # e.g. # 4|Storyville # "Kings of Pastry / Storyville" --> "Storyville / Kings of Pastry" # # 5) replacement_titles # The content contains two programme titles, separated by a # tilde (~). The first title is replaced by the second in the listings # output. # This is useful to fix inconsistent naming (e.g. "Law and Order" vs. # "Law & Order") or inconsistent grammar ("xxxx's" vs. "xxxxs'") # e.g. # 5|A Time Team Special~Time Team # "A Time Team Special / Doncaster" --> "Time Team / Doncaster" # # 6) replacement_genres # The content contains a programme title and a programme category(-ies) # (genres), separated by tilde (~). Categories can be assigned to # uncategorised programmes (which can be seen in the stats log). # (Note that *all* categories are replaced for the title.) # e.g. # 6|Antiques Roadshow~Entertainment~Arts~Shopping # "Antiques Roadshow / " category "Reality" --> # "Antiques Roadshow / " category "Entertainment" + "Arts" + "Shopping" # # 7) replacement_episodes # The content contains a programme title, an old episode value and a new # episode value, separated by tildes (~). # e.g. # 7|Time Team~Time Team Special: Doncaster~Doncaster # "Time Team / Time Team Special: Doncaster" --> "Time Team / Doncaster" # # 8) replacement_titles_episodes # The content contains an old programme title, an old episode value, a new # programme title and a new episode value. The old and new titles MUST be # given, the episode fields can be left empty but the field itself must be # present. # e.g. # 8|Top Gear USA Special~Detroit~Top Gear~USA Special # "Top Gear USA Special / Detroit" --> "Top Gear / USA Special" # # 8|Top Gear USA Special~~Top Gear~USA Special # "Top Gear USA Special / " --> "Top Gear / USA Special" # # 9) replacement_ep_from_desc # The content contains a programme title, a new episode value to update, # and a description (or at least the start of it) to match against. When # title matches incoming data and the incoming description startswith the # text given then the episode value will be replaced. # e.g. # 9|Heroes of Comedy~The Goons~The series celebrating great British # comics pays tribute to the Goons. # "Heroes of Comedy / " desc> = "The series celebrating great British # comics pays tribute to the Goons." # --> "Heroes of Comedy / The Goons" # Should be used with care; e.g. # "Heroes of Comedy / Spike Milligan" desc> = "The series celebrating # great British comics pays tribute to the Goons." # would *also* become # "Heroes of Comedy / The Goons" # this may not be what you want! # # 10) replacement_titles_desc # The content contains an old programme title, an old episode value, a new # programme title, a new episode value and the episode description (or at # least the start of it) to match against. # The old and new titles and description MUST be given, the episode fields # can be left empty but the field itself must be present. # This is useful to fix episodes where the series is unknown but can be # pre-determined from the programme's description. # e.g. # 10|Which Doctor~~Gunsmoke~Which Doctor~Festus and Doc go fishing, but # are captured by a family that is feuding with the Haggens. # "Which Doctor / " desc> = "Festus and Doc go fishing, but are captured # by a family that is feuding with the Haggens. [...]" # --> "Gunsmoke / Which Doctor" # # 11) demoted_titles # The content contains a programme 'brand' and a new title to be extracted # from subtitle field and promoted to programme title, replacing the brand # title. # In other words, if title matches, and sub-title starts with text then # remove the matching text from sub-title and move it into the title. # Any text after 'separator' (any of .,:;-) in the sub-title is preserved. # e.g. # 11|Blackadder~Blackadder II # "Blackadder / Blackadder II: Potato" --> "Blackadder II / Potato" # # 12) replacement_film_genres # The content contains a film title and a category (genre) or categories, # separated by a tilde (~). # If title matches the rule's text and the prog has category "Film" or # "Films", then use the replacement category(-ies) supplied. # Use case: some film-related programmes are incorrectly flagged as films # and should to be re-assigned to a more suitable category. # (Note ALL categories are replaced, not just "Film") # e.g. # 12|The Hobbit Special~Entertainment~Interview # "The Hobbit Special / " category "Film" + "Drama" --> # "The Hobbit Special / " category "Entertainment" + "Interview" # # 13) subtitle_remove_text # The content contains a programme title and arbitrary text to # be removed from the start/end of the programme's subtitle. If the text to # be removed precedes or follows a "separator" (any of .,:;-), the # separator is removed also. # e.g. # 13|Time Team~A Time Team Special # "Time Team / Doncaster : A Time Team Special" --> # "Time Team / Doncaster" # # 14) process_replacement_genres # The content contains a category (genre) value followed by replacement # category(-ies) separated by a tilde (~). # Use case: useful if your PVR doesn't understand some of the category # values in the incoming data; you can translate them to another value. # e.g. # 14|Adventure/War~Action Adventure~War # "The Guns of Navarone" category "Adventure/War" --> # "The Guns of Navarone" category "Action Adventure" + "War" # # 15) process_add_genres_to_channel # The content contains a channel id followed by replacement # category(-ies) separated by a tilde (~). # Use case: this can add a category if data from your supplier is always # missing; e.g. add "News" to a news channel, or "Music" to a music # vid channel. # e.g. # 15|travelchannel.co.uk~Travel # "World's Greatest Motorcycle Rides" category "Motoring" --> # "World's Greatest Motorcycle Rides" category "Motoring" + "Travel" # 15|cnbc.com~News~Business # "Investing in India" category "" --> # "Investing in India" category "News" + "Business" # You should be very careful with this one as it will add the category you # specify to EVERY programme broadcast on that channel. This may not be what # you always want (e.g. Teleshopping isn't really "music" even if it is on MTV!) # # 16) process_remove_numbering_from_programmes # The content contains a title value, followed by an optional channel # (separated by a tilde (~)). # Use case: can remove programme numbering from a specific title if it # is regularly wrong or inconsistent over time. # e.g. # 16|Bedtime Story # "CBeebies Bedtime Story" episode-num ".700." --> # "CBeebies Bedtime Story" episode-num "" # 16|CBeebies Bedtime Story~cbeebies.bbc.co.uk # "CBeebies Bedtime Story" episode-num ".700." --> # "CBeebies Bedtime Story" episode-num "" # Remember to specify the optional channel limiter if you have good # programme numbering for a given programme title on some channels but # not others. # # ############################################################################### # # Some sample rules follow; obviously you should delete these and replace with # your own! # 1|Action Heroes Season 1|Python Night 1|Western Season 2|Blackadder II 2|Comic Relief 2|Old Grey Whistle Test 3|Blackadder II 3|Comic Relief 3|Old Grey Whistle Test 3|Storyville 4|Storyville 4|Timewatch 5|A Time Team Special~Time Team 5|Cheaper By the Dozen~Cheaper by the Dozen 5|Later - with Jools Holland~Later... with Jools Holland 6|Antiques Roadshow~Entertainment~Arts~Shopping 6|Deal or No Deal~Game show 6|Later%%... with Jools Holland~Music 6|M*A*S*H%%~Sitcom 6|Men Behaving Badly~Sitcom 6|Mr. Robot~Drama 6|Springwatch 20%%~Nature 7|Midsomer Murders~Death in Chorus~7/8, series 9 - Death in Chorus 7|Time Team~Time Team Special: Doncaster~Doncaster 8|Blackadder~Blackadder Back and Forth~Blackadder: Back and Forth~ 8|Top Gear USA Special~Detroit~Top Gear~USA Special 8|Top Gear USA Special~~Top Gear~USA Special 9|Heroes of Comedy~The Goons~The series celebrating great British comics pays tribute to the Goons. 10|Which Doctor~~Gunsmoke~Which Doctor~Festus and Doc go fishing, but are captured by a family that is feuding with the Haggens. 10|Which Doctor~Which Doctor~Gunsmoke~Which Doctor~Festus and Doc go fishing, but are captured by a family that is feuding with the Haggens. 11|Blackadder~Blackadder II 11|Formula One~Live Formula One 11|Man on Earth~Man on Earth with Tony Robinson 12|Hell on Wheels~Drama 12|The Hobbit Special~Entertainment~Interview 13|Time Team~A Time Team Special 13|World's Busiest~World's Busiest 14|Adventure/War~Action Adventure~War 14|NFL~American Football 14|Soccer~Football 15|smashhits.net~Music 15|travelchannel.co.uk~Travel 16|Antiques Roadshow~bbc1.bbc.co.uk 16|Classic Antiques Roadshow # # (the sample rules shown here are in sorted order but that is not necessary # in your live file) ############################################################################### ��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_1.xml���������������������������������������0000664�0000000�0000000�00000006675�15000742332�0024445�0����������������������������������������������������������������������������������������������������ustar�00root����������������������������root����������������������������0000000�0000000������������������������������������������������������������������������������������������������������������������������������������������������������������������������<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE tv SYSTEM "xmltv.dtd"> <tv generator-info-name="tv_augment"> <channel id="bbc1.bbc.co.uk"> <display-name lang="en">BBC1</display-name> </channel> <programme start="201607011200 +0100" stop="201607011300 +0100" channel="bbc1.bbc.co.uk"> <title lang="en">Antiques Roadshow New Antiques Roadshow. Doncaster Blah blah blah blah blah. 2016 2 . 0/10 . 0/1 English PG 3/5 Antiques Roadshow New Antiques Roadshow, Norwich Blah blah blah blah blah. 2016 2 . 1/10 . 0/1 English PG 3/5 Antiques Roadshow New Antiques Roadshow: Harrogate Blah blah blah blah blah. 2016 2 . 2/10 . 0/1 English PG 3/5 Antiques Roadshow New Antiques Roadshow; Chester Blah blah blah blah blah. 2016 2 . 3/10 . 0/1 English PG 3/5 Antiques Roadshow New Antiques Roadshow - Edinburgh Blah blah blah blah blah. 2016 2 . 4/10 . 0/1 English PG 3/5 xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_1.xml-expected000066400000000000000000000065111500074233200262310ustar00rootroot00000000000000 BBC1 Antiques Roadshow Doncaster Blah blah blah blah blah. 2016 2 . 0/10 . 0/1 English PG 3/5 Antiques Roadshow Norwich Blah blah blah blah blah. 2016 2 . 1/10 . 0/1 English PG 3/5 Antiques Roadshow Harrogate Blah blah blah blah blah. 2016 2 . 2/10 . 0/1 English PG 3/5 Antiques Roadshow Chester Blah blah blah blah blah. 2016 2 . 3/10 . 0/1 English PG 3/5 Antiques Roadshow Edinburgh Blah blah blah blah blah. 2016 2 . 4/10 . 0/1 English PG 3/5 xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_2.xml000066400000000000000000000067361500074233200244440ustar00rootroot00000000000000 BBC1 Antiques Roadshow Antiques Roadshow. Doncaster: Doncaster Blah blah blah blah blah. 2016 2 . 0/10 . 0/1 English PG 3/5 Antiques Roadshow Antiques Roadshow, Norwich - Norwich Blah blah blah blah blah. 2016 2 . 1/10 . 0/1 English PG 3/5 Antiques Roadshow Antiques Roadshow: Harrogate: Harrogate Blah blah blah blah blah. 2016 2 . 2/10 . 0/1 English PG 3/5 Antiques Roadshow Antiques Roadshow; Chester; Chester Blah blah blah blah blah. 2016 2 . 3/10 . 0/1 English PG 3/5 Antiques Roadshow Antiques Roadshow - Edinburgh - Edinburgh Blah blah blah blah blah. 2016 2 . 4/10 . 0/1 English PG 3/5 xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_2.xml-expected000066400000000000000000000065111500074233200262320ustar00rootroot00000000000000 BBC1 Antiques Roadshow Doncaster Blah blah blah blah blah. 2016 2 . 0/10 . 0/1 English PG 3/5 Antiques Roadshow Norwich Blah blah blah blah blah. 2016 2 . 1/10 . 0/1 English PG 3/5 Antiques Roadshow Harrogate Blah blah blah blah blah. 2016 2 . 2/10 . 0/1 English PG 3/5 Antiques Roadshow Chester Blah blah blah blah blah. 2016 2 . 3/10 . 0/1 English PG 3/5 Antiques Roadshow Edinburgh Blah blah blah blah blah. 2016 2 . 4/10 . 0/1 English PG 3/5 xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_3.xml000066400000000000000000000066421500074233200244410ustar00rootroot00000000000000 BBC1 Antiques Roadshow Antiques Roadshow: Doncaster Blah blah blah blah blah. 2016 2 . 0/10 . 0/1 English PG 3/5 Antiques Roadshow Antiques Roadshow - Norwich Blah blah blah blah blah. 2016 2 . 1/10 . 0/1 English PG 3/5 Antiques Roadshow Antiques Roadshow (Harrogate) Blah blah blah blah blah. 2016 2 . 2/10 . 0/1 English PG 3/5 Antiques Roadshow Antiques Roadshow Blah blah blah blah blah. 2016 2 . 3/10 . 0/1 English PG 3/5 Antiques Roadshow Edinburgh - Antiques Roadshow Blah blah blah blah blah. 2016 2 . 4/10 . 0/1 English PG 3/5 xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_3.xml-expected000066400000000000000000000064341500074233200262370ustar00rootroot00000000000000 BBC1 Antiques Roadshow Doncaster Blah blah blah blah blah. 2016 2 . 0/10 . 0/1 English PG 3/5 Antiques Roadshow Norwich Blah blah blah blah blah. 2016 2 . 1/10 . 0/1 English PG 3/5 Antiques Roadshow Harrogate Blah blah blah blah blah. 2016 2 . 2/10 . 0/1 English PG 3/5 Antiques Roadshow Blah blah blah blah blah. 2016 2 . 3/10 . 0/1 English PG 3/5 Antiques Roadshow Edinburgh Blah blah blah blah blah. 2016 2 . 4/10 . 0/1 English PG 3/5 xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_4.xml000066400000000000000000000053101500074233200244310ustar00rootroot00000000000000 BBC1 Antiques Roadshow Doncaster Premiere. Blah blah blah blah blah. 2016 2 . 0/10 . 0/1 English PG 3/5 Antiques Roadshow Norwich Premiere. Blah blah blah blah blah. 2016 2 . 1/10 . 0/1 English PG 3/5 Antiques Roadshow Harrogate New series of blah blah blah blah blah. 2016 2 . 2/10 . 0/1 English PG 3/5 Antiques Roadshow Chester New series of blah blah blah blah blah. 2016 2 . 3/10 . 0/1 English PG 3/5 xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_4.xml-expected000066400000000000000000000053061500074233200262350ustar00rootroot00000000000000 BBC1 Antiques Roadshow Doncaster Blah blah blah blah blah. 2016 2 . 0/10 . 0/1 English PG 3/5 Antiques Roadshow Norwich Blah blah blah blah blah. 2016 2 . 1/10 . 0/1 English PG 3/5 Antiques Roadshow Harrogate New series of blah blah blah blah blah. 2016 2 . 2/10 . 0/1 English PG 3/5 Antiques Roadshow Chester New series of blah blah blah blah blah. 2016 2 . 3/10 . 0/1 English PG 3/5 xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_5-0a.xml000066400000000000000000000033711500074233200247350ustar00rootroot00000000000000 ITV1 Midsomer Murders Death in Chorus Barnaby and Troy blah blah blah blah blah. 2016 8 . 0 . English Midsomer Murders Death in Chorus Barnaby and Troy blah blah blah blah blah. 2016 8 . 0/7 . English Midsomer Murders Death in Chorus Barnaby and Troy blah blah blah blah blah. 2016 7 . 0 . English xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_5-0a.xml-expected000066400000000000000000000033641500074233200265360ustar00rootroot00000000000000 ITV1 Midsomer Murders Death in Chorus Barnaby and Troy blah blah blah blah blah. 2016 8.0/8. English Midsomer Murders Death in Chorus Barnaby and Troy blah blah blah blah blah. 2016 8.0/7. English Midsomer Murders Death in Chorus Barnaby and Troy blah blah blah blah blah. 2016 7.0/8. English xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_5-0b.xml000066400000000000000000000033711500074233200247360ustar00rootroot00000000000000 ITV1 Midsomer Murders Death in Chorus Barnaby and Troy blah blah blah blah blah. 2016 8 . 0 . English Midsomer Murders Death in Chorus Barnaby and Troy blah blah blah blah blah. 2016 8 . 0/7 . English Midsomer Murders Death in Chorus Barnaby and Troy blah blah blah blah blah. 2016 7 . 0 . English xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_5-0b.xml-expected000066400000000000000000000033641500074233200265370ustar00rootroot00000000000000 ITV1 Midsomer Murders Death in Chorus Barnaby and Troy blah blah blah blah blah. 2016 8.6/8. English Midsomer Murders Death in Chorus Barnaby and Troy blah blah blah blah blah. 2016 8.6/8. English Midsomer Murders Death in Chorus Barnaby and Troy blah blah blah blah blah. 2016 8.6/8. English xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_5-1.xml000066400000000000000000000321321500074233200245720ustar00rootroot00000000000000 ITV1 Midsomer Murders : Series 2/10 Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders : Series 2/10, Episode 3/4 Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders : Series 2/10, Episode 3/4, Part 1 Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders - (Series 2) Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders, - (Series 2.) Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (Series 2, Episode 3) Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Series 2, Episode 3. Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (Season 2) Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Series 2 Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders - Season 2, Episode 3 Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (Series two) Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders - Season two, Episode three Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (Series Two) Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders - Season Two, Episode Three Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Series 2/10 - Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Series 2/10, Episode 3/4 - Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English (Series 2) - Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English (Series 2.) Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English (Series 2, Episode 3) Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Series 2, Episode 3. Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English (Season 2) Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Series 2 Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Season 2, Episode 3 - Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English (Series two) Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Season two, Episode three - Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English (Series Two) Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Season Two, Episode Three - Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_5-1.xml-expected000066400000000000000000000340341500074233200263740ustar00rootroot00000000000000 ITV1 Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1/10.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1/10.2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1/10.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1/10.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1/10.2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_5-2.xml000066400000000000000000001352451500074233200246040ustar00rootroot00000000000000 ITV1 Midsomer Murders Dead Man's Eleven : Series 2/10 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven : Series 2/10, Episode 3/4 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven : Series 2/10, Episode 3/4, Part 1 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven - (Series 2) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven, - (Series 2.) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven (Series 2, Episode 3) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven Series 2, Episode 3. Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven (Season 2) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven Series 2 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven - Season 2, Episode 3 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven (Series two) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven - Season two, Episode three Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven (Series Two) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven - Season Two, Episode Three Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Series 2/10 - Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Series 2/10, Episode 3/4 - Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (Series 2) - Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (Series 2.) Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (Series 2, Episode 3) Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Series 2, Episode 3. Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (Season 2) Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Series 2 Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Season 2, Episode 3 - Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (Series two) Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Season two, Episode three - Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (Series Two) Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Season Two, Episode Three - Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4 - Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4; series two - Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4. Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4; series two - Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4 Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4. Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4; series two Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4; Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (3/4) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders [3/4] Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3. Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3, Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3, series 2 - Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3, series two - Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3, Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Premier League Years 1999/00 Football blah blah blah blah blah. 2016 English Midsomer Murders 3/4 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4. Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (3/4) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders [3/4] Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders s2e3 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders (S02E03) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3 / 4 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders ( 3/4 ) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven 3/4 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven 3/4. Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven (3/4) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven [3/4] Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven s2e3 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven (S02E03) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven 3 / 4 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven ( 3/4 ). Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven: 1 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven (Part 1) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven - (Part 1) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven - (Part 1/2) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven (Pt 1) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven - (Pt. 1) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven - (Pt. 1/2) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven - Part 1 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven: Part 1 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven; Pt 1 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven, Pt. 1 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven Part 1 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven Pt 1 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven Pt 1/2 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven Pt. 1 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven - Part One Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven: Part One Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven; Pt One Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven, Pt. One Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven Part One Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven Pt One Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven Pt. One Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven (Part One) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven - (Part One) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven (Pt One) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders 3/4, series 2 - Dead Man's Eleven - (Pt. One) Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Part One Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Pt Two Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Pt. Three Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Part One of Two Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Pt Two / Three Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Pt. Three of Four Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Part 1 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Part 1/3 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Pt 2 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Pt 2/3 Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Pt. 3 Barnaby and Troy blah blah blah blah blah. 2016 English Football Classics Burnley v Preston North End: 2006/07 Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_5-2.xml-expected000066400000000000000000001427421500074233200264030ustar00rootroot00000000000000 ITV1 Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1/10.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1/10.2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1/10.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1/10.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1/10.2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 1.2/4. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 .2. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 .2. English Midsomer Murders 3 Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders 3 Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.. English Midsomer Murders 3, Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 English Premier League Years 1999/00 Football blah blah blah blah blah. 2016 English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0/2 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0/2 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0/2 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4.0 English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 ..0 English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 ..1 English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 ..2 English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 ..0/2 English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 ..1/3 English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 ..2/4 English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 ..0 English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 ..0/3 English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 ..1 English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 ..1/3 English Midsomer Murders Barnaby and Troy blah blah blah blah blah. 2016 ..2 English Football Classics Burnley v Preston North End: 2006/07 Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_5-3.xml000066400000000000000000000143621500074233200246010ustar00rootroot00000000000000 ITV1 Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. Part 1. 2016 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. Part 1 of 2 2016 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. Part one of two 2016 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. S2E3 2016 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. (s2e3) 2016 English Midsomer Murders Dead Man's Eleven Part one of two. 2016 English Midsomer Murders Dead Man's Eleven Part 1. 2016 English Midsomer Murders Dead Man's Eleven Episode 3/4. 2016 English Midsomer Murders Dead Man's Eleven Episode 3. Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven Episode 3/4. Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven Series 2, episode 3/4. Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven Season two, episode three. Barnaby and Troy blah blah blah blah blah. 2016 English Midsomer Murders Dead Man's Eleven Season two, episode three. Barnaby and Troy blah blah blah blah blah. Part one. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_5-3.xml-expected000066400000000000000000000152371500074233200264020ustar00rootroot00000000000000 ITV1 Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 ..0 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 ..0/2 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 ..0/2 English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven 2016 ..0/2 English Midsomer Murders Dead Man's Eleven 2016 ..0 English Midsomer Murders Dead Man's Eleven 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 .2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2/4. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2. English Midsomer Murders Dead Man's Eleven Barnaby and Troy blah blah blah blah blah. 2016 1.2.0 English xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_5-4.xml000066400000000000000000000050761500074233200246040ustar00rootroot00000000000000 BBC1 Antiques Roadshow Part one of blah blah blah blah blah. 2016 2 . 0/10 . 0/2 English PG 3/5 Antiques Roadshow Part two of blah blah blah blah blah. 2016 2 . 1/10 . 1/2 English PG 3/5 Antiques Roadshow Part one of blah blah blah blah blah. 2016 2 . 2/10 . 0 English PG 3/5 Antiques Roadshow Part two of blah blah blah blah blah. 2016 2 . 3/10 . 1 English PG 3/5 xmltv-1.4.0/t/data-tv_augment/tv_augment_automatic_type_5-4.xml-expected000066400000000000000000000053501500074233200263760ustar00rootroot00000000000000 BBC1 Antiques Roadshow Part 1 of 2 Part one of blah blah blah blah blah. 2016 2.0/10.0/2 English PG 3/5 Antiques Roadshow Part 2 of 2 Part two of blah blah blah blah blah. 2016 2.1/10.1/2 English PG 3/5 Antiques Roadshow Part 1 Part one of blah blah blah blah blah. 2016 2.2/10.0 English PG 3/5 Antiques Roadshow Part 2 Part two of blah blah blah blah blah. 2016 2.3/10.1 English PG 3/5 xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_1.xml000066400000000000000000000031131500074233200234150ustar00rootroot00000000000000 BBC1 Action Heroes Season: Rambo Blah blah blah blah blah. 2016 English 15 Python Night - Monty Python - Live at the Hollywood Bowl Blah blah blah blah blah. 2016 English 15 Western Season; The Good, The Bad and The Ugly Blah blah blah blah blah. 2016 English PG xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_1.xml-expected000066400000000000000000000030261500074233200252170ustar00rootroot00000000000000 BBC1 Rambo Blah blah blah blah blah. 2016 English 15 Monty Python - Live at the Hollywood Bowl Blah blah blah blah blah. 2016 English 15 The Good, The Bad and The Ugly Blah blah blah blah blah. 2016 English PG xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_10.xml000066400000000000000000000021521500074233200234770ustar00rootroot00000000000000 BBC1 Which Doctor Festus and Doc go fishing, but are captured by a family that is feuding with the Haggens. 2016 English Which Doctor Which Doctor Festus and Doc go fishing, but are captured by a family that is feuding with the Haggens. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_10.xml-expected000066400000000000000000000022241500074233200252760ustar00rootroot00000000000000 BBC1 Gunsmoke Which Doctor Festus and Doc go fishing, but are captured by a family that is feuding with the Haggens. 2016 English Gunsmoke Which Doctor Festus and Doc go fishing, but are captured by a family that is feuding with the Haggens. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_11.xml000066400000000000000000000027631500074233200235100ustar00rootroot00000000000000 BBC1 Blackadder Blackadder II - Potato Blah blah blah blah blah. 2016 English Formula One Live Formula One: British Grand Prix Blah blah blah blah blah. 2016 English Man on Earth Man on Earth with Tony Robinson Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_11.xml-expected000066400000000000000000000026471500074233200253100ustar00rootroot00000000000000 BBC1 Blackadder II Potato Blah blah blah blah blah. 2016 English Live Formula One British Grand Prix Blah blah blah blah blah. 2016 English Man on Earth with Tony Robinson Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_12.xml000066400000000000000000000036531500074233200235100ustar00rootroot00000000000000 BBC1 Hell on Wheels Blah blah blah blah blah. 2016 Film English The Hobbit Special Blah blah blah blah blah. 2016 Drama Film English Hell on Wheels Blah blah blah blah blah. 2016 Films Action English The Hobbit Special Blah blah blah blah blah. 2016 Entertainment English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_12.xml-expected000066400000000000000000000036171500074233200253070ustar00rootroot00000000000000 BBC1 Hell on Wheels Blah blah blah blah blah. 2016 Drama English The Hobbit Special Blah blah blah blah blah. 2016 Entertainment Interview English Hell on Wheels Blah blah blah blah blah. 2016 Drama English The Hobbit Special Blah blah blah blah blah. 2016 Entertainment English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_13.xml000066400000000000000000000041131500074233200235010ustar00rootroot00000000000000 BBC1 Time Team A Time Team Special: Doncaster 2016 English Time Team Doncaster - A Time Team Special 2016 English Time Team A Time Team Special Doncaster 2016 English Time Team Doncaster A Time Team Special 2016 English World's Busiest World's Busiest Airport 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_13.xml-expected000066400000000000000000000037501500074233200253060ustar00rootroot00000000000000 BBC1 Time Team Doncaster 2016 English Time Team Doncaster 2016 English Time Team Doncaster 2016 English Time Team Doncaster 2016 English World's Busiest Airport 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_14.xml000066400000000000000000000031061500074233200235030ustar00rootroot00000000000000 BBC1 The Bridge on the River Kwai 2016 Film Adventure/War Drama English NFL Live Miami Dolphins vs Seattle Seahawks 2016 NFL Sport English MOTD Live Liverpool vs Manchester United 2016 Sport Soccer English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_14.xml-expected000066400000000000000000000032001500074233200252750ustar00rootroot00000000000000 BBC1 The Bridge on the River Kwai 2016 Film Action Adventure War Drama English NFL Live Miami Dolphins vs Seattle Seahawks 2016 American Football Sport English MOTD Live Liverpool vs Manchester United 2016 Sport Football English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_15.xml000066400000000000000000000057701500074233200235150ustar00rootroot00000000000000 Smash Hits TV Travel Channel The Travel Channel UK Top 40 Blah blah blah blah blah. 2016 Entertainment English UK Top 10 Blah blah blah blah blah. 2016 Entertainment Music English A Million Pound Place in the Sun Blah blah blah blah blah. 2016 Interests English A Place in the Sun Blah blah blah blah blah. 2016 Travel Interests English A Million Pound Place in the Sun Blah blah blah blah blah. 2016 Interests English A Million Pound Place in the Sun Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_15.xml-expected000066400000000000000000000061651500074233200253130ustar00rootroot00000000000000 Smash Hits TV Travel Channel The Travel Channel UK Top 40 Blah blah blah blah blah. 2016 Entertainment Music English UK Top 10 Blah blah blah blah blah. 2016 Entertainment Music English A Million Pound Place in the Sun Blah blah blah blah blah. 2016 Interests Travel English A Place in the Sun Blah blah blah blah blah. 2016 Travel Interests English A Million Pound Place in the Sun Blah blah blah blah blah. 2016 Interests English A Million Pound Place in the Sun Blah blah blah blah blah. 2016 Travel English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_16.xml000066400000000000000000000175361500074233200235210ustar00rootroot00000000000000 BBC1 BBC2 Antiques Roadshow Doncaster Blah blah blah blah blah. 2016 0 . 0/10 . 0/1 English PG 3/5 Antiques Roadshow Norwich Blah blah blah blah blah. 2016 0 . 1/10 . 0/1 English PG 3/5 Antiques Roadshow Harrogate Blah blah blah blah blah. 2016 0 . 2/10 . 0/1 English PG 3/5 Antiques Roadshow Chester Blah blah blah blah blah. 2016 0 . 3/10 . 0/1 English PG 3/5 Antiques Roadshow Edinburgh Blah blah blah blah blah. 2016 0 . 4/10 . 0/1 English PG 3/5 Classic Antiques Roadshow Doncaster Blah blah blah blah blah. 2016 1 . 0/10 . 0/1 English PG 3/5 Classic Antiques Roadshow Norwich Blah blah blah blah blah. 2016 1 . 1/10 . 0/1 English PG 3/5 Classic Antiques Roadshow Harrogate Blah blah blah blah blah. 2016 1 . 2/10 . 0/1 English PG 3/5 Classic Antiques Roadshow Chester Blah blah blah blah blah. 2016 1 . 3/10 . 0/1 English PG 3/5 Classic Antiques Roadshow Edinburgh Blah blah blah blah blah. 2016 1 . 4/10 . 0/1 English PG 3/5 Not the Antiques Roadshow Doncaster Blah blah blah blah blah. 2016 2 . 0/10 . 0/1 English PG 3/5 Definitely Not the Antiques Roadshow Doncaster Blah blah blah blah blah. 2016 3 . 0/10 . 0/1 English PG 3/5 xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_16.xml-expected000066400000000000000000000164221500074233200253110ustar00rootroot00000000000000 BBC1 BBC2 Antiques Roadshow Doncaster Blah blah blah blah blah. 2016 English PG 3/5 Antiques Roadshow Norwich Blah blah blah blah blah. 2016 English PG 3/5 Antiques Roadshow Harrogate Blah blah blah blah blah. 2016 English PG 3/5 Antiques Roadshow Chester Blah blah blah blah blah. 2016 English PG 3/5 Antiques Roadshow Edinburgh Blah blah blah blah blah. 2016 0.4/10.0/1 English PG 3/5 Classic Antiques Roadshow Doncaster Blah blah blah blah blah. 2016 English PG 3/5 Classic Antiques Roadshow Norwich Blah blah blah blah blah. 2016 English PG 3/5 Classic Antiques Roadshow Harrogate Blah blah blah blah blah. 2016 English PG 3/5 Classic Antiques Roadshow Chester Blah blah blah blah blah. 2016 English PG 3/5 Classic Antiques Roadshow Edinburgh Blah blah blah blah blah. 2016 English PG 3/5 Not the Antiques Roadshow Doncaster Blah blah blah blah blah. 2016 2.0/10.0/1 English PG 3/5 Definitely Not the Antiques Roadshow Doncaster Blah blah blah blah blah. 2016 3.0/10.0/1 English PG 3/5 xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_2.xml000066400000000000000000000031021500074233200234140ustar00rootroot00000000000000 BBC1 Blackadder II: Potato Blah blah blah blah blah. 2016 English 15 Comic Relief - 2016 The Total Blah blah blah blah blah. 2016 English 15 Old Grey Whistle Test; Pink Floyd Blah blah blah blah blah. 2016 English PG xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_2.xml-expected000066400000000000000000000032111500074233200252140ustar00rootroot00000000000000 BBC1 Blackadder II Potato Blah blah blah blah blah. 2016 English 15 Comic Relief 2016: The Total Blah blah blah blah blah. 2016 English 15 Old Grey Whistle Test Pink Floyd Blah blah blah blah blah. 2016 English PG xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_3.xml000066400000000000000000000040051500074233200234200ustar00rootroot00000000000000 BBC1 Potato: Blackadder II Blah blah blah blah blah. 2016 English 15 2016 - Comic Relief The Total Blah blah blah blah blah. 2016 English 15 Pink Floyd; Old Grey Whistle Test Blah blah blah blah blah. 2016 English PG Kings of Pastry :Storyville Blah blah blah blah blah. 2016 English PG xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_3.xml-expected000066400000000000000000000041601500074233200252210ustar00rootroot00000000000000 BBC1 Blackadder II Potato Blah blah blah blah blah. 2016 English 15 Comic Relief 2016: The Total Blah blah blah blah blah. 2016 English 15 Old Grey Whistle Test Pink Floyd Blah blah blah blah blah. 2016 English PG Storyville Kings of Pastry Blah blah blah blah blah. 2016 English PG xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_4.xml000066400000000000000000000020521500074233200234210ustar00rootroot00000000000000 BBC1 Kings of Pastry Storyville Blah blah blah blah blah. 2016 English Cleopatra: A Timewatch Guide Timewatch Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_4.xml-expected000066400000000000000000000020521500074233200252200ustar00rootroot00000000000000 BBC1 Storyville Kings of Pastry Blah blah blah blah blah. 2016 English Timewatch Cleopatra: A Timewatch Guide Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_5.xml000066400000000000000000000027171500074233200234320ustar00rootroot00000000000000 BBC1 A Time Team Special The Secrets of Westminster Abbey Blah blah blah blah blah. 2016 English Cheaper By the Dozen Blah blah blah blah blah. 2016 English PG Later - with Jools Holland Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_5.xml-expected000066400000000000000000000027061500074233200252270ustar00rootroot00000000000000 BBC1 Time Team The Secrets of Westminster Abbey Blah blah blah blah blah. 2016 English Cheaper by the Dozen Blah blah blah blah blah. 2016 English PG Later... with Jools Holland Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_6.xml000066400000000000000000000105471500074233200234330ustar00rootroot00000000000000 BBC1 Antiques Roadshow Blah blah blah blah blah. 2016 Interests English Deal or No Deal Blah blah blah blah blah. 2016 Entertainment English Men Behaving Badly Blah blah blah blah blah. 2016 Entertainment English Springwatch 2015 Blah blah blah blah blah. 2016 Entertainment English Springwatch 2016 Blah blah blah blah blah. 2016 Entertainment English M*A*S*H Blah blah blah blah blah. 2016 Entertainment English Mr. Robot Blah blah blah blah blah. 2016 Entertainment English Later... with Jools Holland Blah blah blah blah blah. 2016 Entertainment English Later Live... with Jools Holland Blah blah blah blah blah. 2016 Entertainment English M*A*S*H +1 ? Blah blah blah blah blah. 2016 Entertainment English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_6.xml-expected000066400000000000000000000106001500074233200252200ustar00rootroot00000000000000 BBC1 Antiques Roadshow Blah blah blah blah blah. 2016 Entertainment Arts Shopping English Deal or No Deal Blah blah blah blah blah. 2016 Game show English Men Behaving Badly Blah blah blah blah blah. 2016 Sitcom English Springwatch 2015 Blah blah blah blah blah. 2016 Nature English Springwatch 2016 Blah blah blah blah blah. 2016 Nature English M*A*S*H Blah blah blah blah blah. 2016 Sitcom English Mr. Robot Blah blah blah blah blah. 2016 Drama English Later... with Jools Holland Blah blah blah blah blah. 2016 Music English Later Live... with Jools Holland Blah blah blah blah blah. 2016 Music English M*A*S*H +1 ? Blah blah blah blah blah. 2016 Sitcom English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_7.xml000066400000000000000000000012041500074233200234220ustar00rootroot00000000000000 BBC1 Time Team Time Team Special: Doncaster Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_7.xml-expected000066400000000000000000000011611500074233200252230ustar00rootroot00000000000000 BBC1 Time Team Doncaster Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_8.xml000066400000000000000000000026451500074233200234350ustar00rootroot00000000000000 BBC1 Blackadder Blackadder Back and Forth Blah blah blah blah blah. 2016 English Top Gear USA Special Detroit Blah blah blah blah blah. 2016 English Top Gear USA Special Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_8.xml-expected000066400000000000000000000026231500074233200252300ustar00rootroot00000000000000 BBC1 Blackadder: Back and Forth Blah blah blah blah blah. 2016 English Top Gear USA Special Blah blah blah blah blah. 2016 English Top Gear USA Special Blah blah blah blah blah. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_9.xml000066400000000000000000000021161500074233200234270ustar00rootroot00000000000000 BBC1 Heroes of Comedy The series celebrating great British comics pays tribute to the Goons. 2016 English Heroes of Comedy Spike Milligan The series celebrating great British comics pays tribute to the Goons. 2016 English xmltv-1.4.0/t/data-tv_augment/tv_augment_user_type_9.xml-expected000066400000000000000000000021701500074233200252260ustar00rootroot00000000000000 BBC1 Heroes of Comedy The Goons The series celebrating great British comics pays tribute to the Goons. 2016 English Heroes of Comedy The Goons The series celebrating great British comics pays tribute to the Goons. 2016 English xmltv-1.4.0/t/data-tv_imdb/000077500000000000000000000000001500074233200154605ustar00rootroot00000000000000xmltv-1.4.0/t/data-tv_imdb/After-data-freeze.xml000066400000000000000000000003231500074233200214260ustar00rootroot00000000000000 Unarmed Man 2019 xmltv-1.4.0/t/data-tv_imdb/After-data-freeze.xml-expected000066400000000000000000000003241500074233200232260ustar00rootroot00000000000000 Unarmed Man 2019 xmltv-1.4.0/t/data-tv_imdb/Cast-actor-with-generation.xml000066400000000000000000000003211500074233200233000ustar00rootroot00000000000000 Murder101 2014 xmltv-1.4.0/t/data-tv_imdb/Cast-actor-with-generation.xml-expected000066400000000000000000000006271500074233200251100ustar00rootroot00000000000000 Murder101 Percy Daggs III 2014 Movie https://www.imdb.com/find?q=Murder101%20%282014%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Cast-actors-and-actresses.xml000066400000000000000000000003171500074233200231200ustar00rootroot00000000000000 Titanic 1997 xmltv-1.4.0/t/data-tv_imdb/Cast-actors-and-actresses.xml-expected000066400000000000000000000006671500074233200247270ustar00rootroot00000000000000 Titanic Leonardo DiCaprio Kate Winslet 1997 Movie https://www.imdb.com/find?q=Titanic%20%281997%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Cast-billing.xml000066400000000000000000000003141500074233200205100ustar00rootroot00000000000000 #Rip 2013 xmltv-1.4.0/t/data-tv_imdb/Cast-billing.xml-expected000066400000000000000000000007261500074233200223160ustar00rootroot00000000000000 #Rip Marilyn Ghigliotti Missi Pyle Naomi Grossman 2013 Movie https://www.imdb.com/find?q=%23Rip%20%282013%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Cast-duplicate.xml000066400000000000000000000003231500074233200210420ustar00rootroot00000000000000 #SketchPack 2015 xmltv-1.4.0/t/data-tv_imdb/Cast-duplicate.xml-expected000066400000000000000000000006421500074233200226450ustar00rootroot00000000000000 #SketchPack Lucy Scott-Smith 2015 TV Series https://www.imdb.com/find?q=%23SketchPack%20%282015%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Cast-host-or-narrator.xml000066400000000000000000000015531500074233200223170ustar00rootroot00000000000000 Bookclub 2015 LolliLove 2004 Breaking Genres 2015 The Jean Bowring Show 1957 New Now Next Awards 2008 3 Weeks in Yerevan 2016 xmltv-1.4.0/t/data-tv_imdb/Cast-host-or-narrator.xml-expected000066400000000000000000000041641500074233200241170ustar00rootroot00000000000000 Bookclub Fabio Huwyler 2015 TV Series https://www.imdb.com/find?q=Bookclub%20%282015%29&s=tt&exact=true LolliLove Peter Alton 2004 Movie https://www.imdb.com/find?q=LolliLove%20%282004%29&s=tt&exact=true Breaking Genres Amrit Singh 2015 TV Movie https://www.imdb.com/find?q=Breaking%20Genres%20%282015%29&s=tt&exact=true The Jean Bowring Show Jean Bowring 1957 TV Series https://www.imdb.com/find?q=The%20Jean%20Bowring%20Show%20%281957%29&s=tt&exact=true New Now Next Awards Gloria Bigelow 2008 TV Movie https://www.imdb.com/find?q=New%20Now%20Next%20Awards%20%282008%29&s=tt&exact=true 3 Weeks in Yerevan Mary Asatryan 2016 Movie https://www.imdb.com/find?q=3%20Weeks%20in%20Yerevan%20%282016%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Cast-name-with-suffix.xml000066400000000000000000000004111500074233200222610ustar00rootroot00000000000000 #Selfie cast: Elizabeth Kent should appear twice 2015 xmltv-1.4.0/t/data-tv_imdb/Cast-name-with-suffix.xml-expected000066400000000000000000000010271500074233200240640ustar00rootroot00000000000000 #Selfie cast: Elizabeth Kent should appear twice Karina Cornwell Elizabeth Kent Elizabeth Kent 2015 Movie https://www.imdb.com/find?q=%23Selfie%20%282015%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Cast-role.xml000066400000000000000000000005101500074233200200270ustar00rootroot00000000000000 #REV 2015 Titanic 1997 xmltv-1.4.0/t/data-tv_imdb/Cast-role.xml-expected000066400000000000000000000014301500074233200216300ustar00rootroot00000000000000 #REV Poroma Banerjee Sharon Zachariah 2015 Movie https://www.imdb.com/find?q=%23REV%20%282015%29&s=tt&exact=true Titanic Leonardo DiCaprio Kate Winslet 1997 Movie https://www.imdb.com/find?q=Titanic%20%281997%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Director-multiple-and-duplicate-directors.xml000066400000000000000000000005161500074233200263140ustar00rootroot00000000000000 #Illusion 2014 #iScream 2014 xmltv-1.4.0/t/data-tv_imdb/Director-multiple-and-duplicate-directors.xml-expected000066400000000000000000000014241500074233200301120ustar00rootroot00000000000000 #Illusion Teodora Berglund Alexandra Jousset 2014 Movie https://www.imdb.com/find?q=%23Illusion%20%282014%29&s=tt&exact=true #iScream Gibran Tanwir 2014 Movie https://www.imdb.com/find?q=%23iScream%20%282014%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Director-name-with-suffix.xml000066400000000000000000000003261500074233200231470ustar00rootroot00000000000000 Grease Monkeys 1979 xmltv-1.4.0/t/data-tv_imdb/Director-name-with-suffix.xml-expected000066400000000000000000000006441500074233200247510ustar00rootroot00000000000000 Grease Monkeys Mark Aaron 1979 Movie https://www.imdb.com/find?q=Grease%20Monkeys%20%281979%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Director-with-generation.xml000066400000000000000000000003201500074233200230520ustar00rootroot00000000000000 The Meek 2017 xmltv-1.4.0/t/data-tv_imdb/Director-with-generation.xml-expected000066400000000000000000000006401500074233200246560ustar00rootroot00000000000000 The Meek Harold Jackson III 2017 Movie https://www.imdb.com/find?q=The%20Meek%20%282017%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Genres-duplicate.xml000066400000000000000000000003171500074233200213760ustar00rootroot00000000000000 'C'-Man 1949 xmltv-1.4.0/t/data-tv_imdb/Genres-duplicate.xml-expected000066400000000000000000000007241500074233200231770ustar00rootroot00000000000000 'C'-Man 1949 Movie Crime Drama Film-Noir https://www.imdb.com/find?q=%27C%27-Man%20%281949%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Genres-multiple.xml000066400000000000000000000003271500074233200212600ustar00rootroot00000000000000 [Film #9 Title] 2015 xmltv-1.4.0/t/data-tv_imdb/Genres-multiple.xml-expected000066400000000000000000000007511500074233200230600ustar00rootroot00000000000000 [Film #9 Title] 2015 Movie Comedy Fantasy Short https://www.imdb.com/find?q=%5BFilm%20%239%20Title%5D%20%282015%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Genres-single.xml000066400000000000000000000003341500074233200207040ustar00rootroot00000000000000 (Mon) Jour de chance 2004 xmltv-1.4.0/t/data-tv_imdb/Genres-single.xml-expected000066400000000000000000000006361500074233200225100ustar00rootroot00000000000000 (Mon) Jour de chance 2004 Movie Short https://www.imdb.com/find?q=%28Mon%29%20Jour%20de%20chance%20%282004%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Movie-same-year-movie-and-series.xml000066400000000000000000000007561500074233200243170ustar00rootroot00000000000000 Journey to the Center of the Earth Multiple titles (movie,video,tv) with same title+year 2008 Ashes to Ashes Movie and tv-series with same title+year 2008 xmltv-1.4.0/t/data-tv_imdb/Movie-same-year-movie-and-series.xml-expected000066400000000000000000000007571500074233200261170ustar00rootroot00000000000000 Journey to the Center of the Earth Multiple titles (movie,video,tv) with same title+year 2008 Ashes to Ashes Movie and tv-series with same title+year 2008 xmltv-1.4.0/t/data-tv_imdb/Movie-startswith-hyphen.xml000066400000000000000000000003251500074233200227640ustar00rootroot00000000000000 -1: Minus One 2016 xmltv-1.4.0/t/data-tv_imdb/Movie-startswith-hyphen.xml-expected000066400000000000000000000005431500074233200245650ustar00rootroot00000000000000 -1: Minus One 2016 Movie https://www.imdb.com/find?q=-1%3A%20Minus%20One%20%282016%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Movie-two-in-same-year.xml000066400000000000000000000004501500074233200223540ustar00rootroot00000000000000 '83 tv_imdb cannot identify a sole hit - two films in same year with this title 2017 xmltv-1.4.0/t/data-tv_imdb/Movie-two-in-same-year.xml-expected000066400000000000000000000004511500074233200241540ustar00rootroot00000000000000 '83 tv_imdb cannot identify a sole hit - two films in same year with this title 2017 xmltv-1.4.0/t/data-tv_imdb/Movie-with-aka.xml000066400000000000000000000003261500074233200207650ustar00rootroot00000000000000 Family Prayers 2010 xmltv-1.4.0/t/data-tv_imdb/Movie-with-aka.xml-expected000066400000000000000000000005411500074233200225630ustar00rootroot00000000000000 Family Prayers 2010 Movie https://www.imdb.com/find?q=Family%20Prayers%20%282010%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Movie-with-unknown-year.xml000066400000000000000000000006411500074233200226660ustar00rootroot00000000000000 Zed Zed 2010 California Cornflakes xmltv-1.4.0/t/data-tv_imdb/Movie-with-unknown-year.xml-expected000066400000000000000000000006421500074233200244660ustar00rootroot00000000000000 Zed Zed 2010 California Cornflakes xmltv-1.4.0/t/data-tv_imdb/Movie1-case-insensitive.xml000066400000000000000000000003161500074233200226110ustar00rootroot00000000000000 MOVIE1 1990 xmltv-1.4.0/t/data-tv_imdb/Movie1-case-insensitive.xml-expected000066400000000000000000000013431500074233200244110ustar00rootroot00000000000000 Movie1 line 1 line 2 line 3 line 4 Joe Director Betty Actor Bruce Actor 1990 Movie Horror Horror Mystery https://www.imdb.com/find?q=Movie1%20%281990%29&s=tt&exact=true 1.0/10 xmltv-1.4.0/t/data-tv_imdb/Movie1-movies-only.xml000066400000000000000000000004661500074233200216270ustar00rootroot00000000000000 Movie1 1990 The Show1 xmltv-1.4.0/t/data-tv_imdb/Movie1-movies-only.xml-expected000066400000000000000000000015131500074233200234200ustar00rootroot00000000000000 Movie1 line 1 line 2 line 3 line 4 Joe Director Betty Actor Bruce Actor 1990 Movie Horror Horror Mystery https://www.imdb.com/find?q=Movie1%20%281990%29&s=tt&exact=true 1.0/10 The Show1 xmltv-1.4.0/t/data-tv_imdb/Movie1.xml000066400000000000000000000004541500074233200173450ustar00rootroot00000000000000 Movie1 1990 xmltv-1.4.0/t/data-tv_imdb/Movie1.xml-expected000066400000000000000000000013431500074233200211420ustar00rootroot00000000000000 Movie1 line 1 line 2 line 3 line 4 Joe Director Betty Actor Bruce Actor 1990 Movie Horror Horror Mystery https://www.imdb.com/find?q=Movie1%20%281990%29&s=tt&exact=true 1.0/10 xmltv-1.4.0/t/data-tv_imdb/Movie100-years.xml000066400000000000000000000034771500074233200206360ustar00rootroot00000000000000 Movie100 1915 Movie100 1914 Movie100 1913 Movie100 1912 Movie100 1916 Movie100 1917 Movie100 1918 Movie100 1943 Movie100 1953 Movie100 1989 Movie100 1993 xmltv-1.4.0/t/data-tv_imdb/Movie100-years.xml-expected000066400000000000000000000057011500074233200224250ustar00rootroot00000000000000 Movie100 In1915 Director 1915 Movie https://www.imdb.com/find?q=Movie100%20%281915%29&s=tt&exact=true Movie100 In1915 Director 1914 Movie https://www.imdb.com/find?q=Movie100%20%281915%29&s=tt&exact=true Movie100 In1915 Director 1913 Movie https://www.imdb.com/find?q=Movie100%20%281915%29&s=tt&exact=true Movie100 1912 Movie100 In1915 Director 1916 Movie https://www.imdb.com/find?q=Movie100%20%281915%29&s=tt&exact=true Movie100 In1915 Director 1917 Movie https://www.imdb.com/find?q=Movie100%20%281915%29&s=tt&exact=true Movie100 1918 Movie100 In1943 Director 1943 Movie https://www.imdb.com/find?q=Movie100%20%281943%29&s=tt&exact=true Movie100 In1953 Director 1953 Movie https://www.imdb.com/find?q=Movie100%20%281953%29&s=tt&exact=true Movie100 1989 Movie100 1993 Video Movie https://www.imdb.com/find?q=Movie100%20%281993%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Movie101-movie-and-tv.xml000066400000000000000000000013351500074233200220110ustar00rootroot00000000000000 Movie101 1992 Movie101 1993 Movie101 1988 Movie101 xmltv-1.4.0/t/data-tv_imdb/Movie101-movie-and-tv.xml-expected000066400000000000000000000021021500074233200236010ustar00rootroot00000000000000 Movie101 1992 Movie https://www.imdb.com/find?q=Movie101%20%281992%29&s=tt&exact=true Movie101 1993 Movie https://www.imdb.com/find?q=Movie101%20%281993%29&s=tt&exact=true Movie101 1988 TV Series https://www.imdb.com/find?q=Movie101%20%281988%29&s=tt&exact=true Movie101 TV Series https://www.imdb.com/find?q=Movie101%20%281988%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Movie21-accents.xml000066400000000000000000000035421500074233200210460ustar00rootroot00000000000000 Movie21 aeiouaecnssy 1991 Movie21 Àeiouaecnssy 1991 Movie21 aÈiouaecnssy 1991 Movie21 aeÌouaecnssy 1991 Movie21 aeiÒuaecnssy 1991 Movie21 aeioÙaecnssy 1991 Movie21 aeiouÆcnssy 1991 Movie21 aeiouaeÇnssy 1991 Movie21 aeiouaecÑssy 1991 Movie21 aeiouaecnßy 1991 Movie21 aeiouaecnssÝ 1991 Movie21 ÀÈÌÒÙæÇÑßÝ 1991 Movie21 ÀÈÌÒÙæÇÑßÝ¿ 1991 xmltv-1.4.0/t/data-tv_imdb/Movie21-accents.xml-expected000066400000000000000000000102231500074233200226370ustar00rootroot00000000000000 Movie21 aeiouaecnssy 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true Movie21 aeiouaecnssy Movie21 Àeiouaecnssy 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true Movie21 aeiouaecnssy Movie21 aÈiouaecnssy 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true Movie21 aeiouaecnssy Movie21 aeÌouaecnssy 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true Movie21 aeiouaecnssy Movie21 aeiÒuaecnssy 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true Movie21 aeiouaecnssy Movie21 aeioÙaecnssy 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true Movie21 aeiouaecnssy Movie21 aeiouÆcnssy 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true Movie21 aeiouaecnssy Movie21 aeiouaeÇnssy 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true Movie21 aeiouaecnssy Movie21 aeiouaecÑssy 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true Movie21 aeiouaecnssy Movie21 aeiouaecnßy 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true Movie21 aeiouaecnssy Movie21 aeiouaecnssÝ 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true Movie21 aeiouaecnssy Movie21 ÀÈÌÒÙæÇÑßÝ 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true Movie21 aeiouaecnssy Movie21 ÀÈÌÒÙæÇÑßÝ¿ 1991 Movie https://www.imdb.com/find?q=Movie21%20aeiouaecnssy%20%281991%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Movie22-dots.xml000066400000000000000000000007411500074233200203760ustar00rootroot00000000000000 Movie22 dots 1991 M.o.v.i.e.2.2. dots 1991 Movie22 d.o.t.s. 1991 xmltv-1.4.0/t/data-tv_imdb/Movie22-dots.xml-expected000066400000000000000000000016721500074233200222010ustar00rootroot00000000000000 Movie22 dots 1991 Movie https://www.imdb.com/find?q=Movie22%20dots%20%281991%29&s=tt&exact=true Movie22 dots M.o.v.i.e.2.2. dots 1991 Movie https://www.imdb.com/find?q=Movie22%20dots%20%281991%29&s=tt&exact=true Movie22 dots Movie22 d.o.t.s. 1991 Movie https://www.imdb.com/find?q=Movie22%20dots%20%281991%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Movie3-and-amp.xml000066400000000000000000000011471500074233200206620ustar00rootroot00000000000000 Movie3 and more 1991 Movie3 & more 1991 Movie4 & more 1991 Movie4 and more 1991 xmltv-1.4.0/t/data-tv_imdb/Movie3-and-amp.xml-expected000066400000000000000000000023441500074233200224610ustar00rootroot00000000000000 Movie3 and more 1991 Movie https://www.imdb.com/find?q=Movie3%20and%20more%20%281991%29&s=tt&exact=true Movie3 and more Movie3 & more 1991 Movie https://www.imdb.com/find?q=Movie3%20and%20more%20%281991%29&s=tt&exact=true Movie4 & more 1991 Movie https://www.imdb.com/find?q=Movie4%20%26%20more%20%281991%29&s=tt&exact=true Movie4 & more Movie4 and more 1991 Movie https://www.imdb.com/find?q=Movie4%20%26%20more%20%281991%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Movie5-ignore-punc.xml000066400000000000000000000007751500074233200216030ustar00rootroot00000000000000 Movie5 no punctuation 1991 Movie5 no .....punctuation 1991 Movie5:Movie5 no punctuation 1991 xmltv-1.4.0/t/data-tv_imdb/Movie5-ignore-punc.xml-expected000066400000000000000000000020111500074233200233630ustar00rootroot00000000000000 Movie5 no punctuation 1991 Movie https://www.imdb.com/find?q=Movie5%20no%20punctuation%20%281991%29&s=tt&exact=true Movie5 no punctuation Movie5 no .....punctuation 1991 Movie https://www.imdb.com/find?q=Movie5%20no%20punctuation%20%281991%29&s=tt&exact=true Movie5 no punctuation Movie5:Movie5 no punctuation 1991 Movie https://www.imdb.com/find?q=Movie5%20no%20punctuation%20%281991%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Movie5-with-punc.xml000066400000000000000000000003411500074233200212600ustar00rootroot00000000000000 Movie5's with punctuation 1992 xmltv-1.4.0/t/data-tv_imdb/Movie5-with-punc.xml-expected000066400000000000000000000005731500074233200230660ustar00rootroot00000000000000 Movie5's with punctuation 1992 Movie https://www.imdb.com/find?q=Movie5%27s%20with%20punctuation%20%281992%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Movie6-articles.xml000066400000000000000000000073151500074233200211610ustar00rootroot00000000000000 The Movie6 1991 The Movie7 1991 Movie7, The 1991 A Movie8 1991 Movie8, A 1991 Une Movie9 1991 Movie9, Une 1991 Les Movie10 1991 Movie10, Les 1991 Los Movie11 1991 Movie11, Los 1991 Las Movie12 1991 Movie12, Las 1991 L' Movie13 1991 Movie13, L' 1991 Le Movie14 1991 Movie14, Le 1991 La Movie15 1991 Movie15, La 1991 El Movie16 1991 Movie16, El 1991 Das Movie17 1991 Movie17, Das 1991 De Movie18 1991 Movie18, De 1991 Het Movie19 1991 Movie19, Het 1991 Een Movie20 1991 Movie20, Een 1991 xmltv-1.4.0/t/data-tv_imdb/Movie6-articles.xml-expected000066400000000000000000000176571500074233200227720ustar00rootroot00000000000000 The Movie6 1991 Movie https://www.imdb.com/find?q=The%20Movie6%20%281991%29&s=tt&exact=true The Movie7 1991 Movie https://www.imdb.com/find?q=The%20Movie7%20%281991%29&s=tt&exact=true The Movie7 Movie7, The 1991 Movie https://www.imdb.com/find?q=The%20Movie7%20%281991%29&s=tt&exact=true A Movie8 1991 Movie https://www.imdb.com/find?q=A%20Movie8%20%281991%29&s=tt&exact=true A Movie8 Movie8, A 1991 Movie https://www.imdb.com/find?q=A%20Movie8%20%281991%29&s=tt&exact=true Une Movie9 1991 Movie https://www.imdb.com/find?q=Une%20Movie9%20%281991%29&s=tt&exact=true Une Movie9 Movie9, Une 1991 Movie https://www.imdb.com/find?q=Une%20Movie9%20%281991%29&s=tt&exact=true Les Movie10 1991 Movie https://www.imdb.com/find?q=Les%20Movie10%20%281991%29&s=tt&exact=true Les Movie10 Movie10, Les 1991 Movie https://www.imdb.com/find?q=Les%20Movie10%20%281991%29&s=tt&exact=true Los Movie11 1991 Movie https://www.imdb.com/find?q=Los%20Movie11%20%281991%29&s=tt&exact=true Los Movie11 Movie11, Los 1991 Movie https://www.imdb.com/find?q=Los%20Movie11%20%281991%29&s=tt&exact=true Las Movie12 1991 Movie https://www.imdb.com/find?q=Las%20Movie12%20%281991%29&s=tt&exact=true Las Movie12 Movie12, Las 1991 Movie https://www.imdb.com/find?q=Las%20Movie12%20%281991%29&s=tt&exact=true L' Movie13 1991 Movie https://www.imdb.com/find?q=L%27%20Movie13%20%281991%29&s=tt&exact=true L' Movie13 Movie13, L' 1991 Movie https://www.imdb.com/find?q=L%27%20Movie13%20%281991%29&s=tt&exact=true Le Movie14 1991 Movie https://www.imdb.com/find?q=Le%20Movie14%20%281991%29&s=tt&exact=true Le Movie14 Movie14, Le 1991 Movie https://www.imdb.com/find?q=Le%20Movie14%20%281991%29&s=tt&exact=true La Movie15 1991 Movie https://www.imdb.com/find?q=La%20Movie15%20%281991%29&s=tt&exact=true La Movie15 Movie15, La 1991 Movie https://www.imdb.com/find?q=La%20Movie15%20%281991%29&s=tt&exact=true El Movie16 1991 Movie https://www.imdb.com/find?q=El%20Movie16%20%281991%29&s=tt&exact=true El Movie16 Movie16, El 1991 Movie https://www.imdb.com/find?q=El%20Movie16%20%281991%29&s=tt&exact=true Das Movie17 1991 Movie https://www.imdb.com/find?q=Das%20Movie17%20%281991%29&s=tt&exact=true Das Movie17 Movie17, Das 1991 Movie https://www.imdb.com/find?q=Das%20Movie17%20%281991%29&s=tt&exact=true De Movie18 1991 Movie https://www.imdb.com/find?q=De%20Movie18%20%281991%29&s=tt&exact=true De Movie18 Movie18, De 1991 Movie https://www.imdb.com/find?q=De%20Movie18%20%281991%29&s=tt&exact=true Het Movie19 1991 Movie https://www.imdb.com/find?q=Het%20Movie19%20%281991%29&s=tt&exact=true Het Movie19 Movie19, Het 1991 Movie https://www.imdb.com/find?q=Het%20Movie19%20%281991%29&s=tt&exact=true Een Movie20 1991 Movie https://www.imdb.com/find?q=Een%20Movie20%20%281991%29&s=tt&exact=true Een Movie20 Movie20, Een 1991 Movie https://www.imdb.com/find?q=Een%20Movie20%20%281991%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/Ratings.xml000066400000000000000000000003211500074233200176050ustar00rootroot00000000000000 #nitTWITS 2011 xmltv-1.4.0/t/data-tv_imdb/Ratings.xml-expected000066400000000000000000000006661500074233200214200ustar00rootroot00000000000000 #nitTWITS 2011 TV Series https://www.imdb.com/find?q=%23nitTWITS%20%282011%29&s=tt&exact=true 7.0/10 xmltv-1.4.0/t/data-tv_imdb/Show1-movies-only.xml000066400000000000000000000002731500074233200214640ustar00rootroot00000000000000 The Show1 xmltv-1.4.0/t/data-tv_imdb/Show1-movies-only.xml-expected000066400000000000000000000002741500074233200232640ustar00rootroot00000000000000 The Show1 xmltv-1.4.0/t/data-tv_imdb/Show1.xml000066400000000000000000000004721500074233200172060ustar00rootroot00000000000000 The Show1 The Show1 1990 xmltv-1.4.0/t/data-tv_imdb/Show1.xml-expected000066400000000000000000000011141500074233200207770ustar00rootroot00000000000000 The Show1 TV Series https://www.imdb.com/find?q=The%20Show1%20%282002%29&s=tt&exact=true The Show1 1990 TV Series https://www.imdb.com/find?q=The%20Show1%20%282002%29&s=tt&exact=true xmltv-1.4.0/t/data-tv_imdb/lists/000077500000000000000000000000001500074233200166165ustar00rootroot00000000000000xmltv-1.4.0/t/data-tv_imdb/lists/actors.list000066400000000000000000000017411500074233200210110ustar00rootroot00000000000000# # Actors list for testing # - file format follows imdb's format for actors.list.gz file # THE ACTORS LIST =============== Name Titles ---- ------ Campbell, Bruce (I) Army of Darkness (1992) [Ash] <1> Actor, Bruce (I) Movie1 (1990) [Ash] <1> Movie2 (1991) [Ash] <1> Dibnah, Fred A Tribute to Fred Dibnah (2004) (TV) (archive footage) [Himself] <2> Dig with Dibnah (2004) (TV) [Himself - Presenter] <1> Fred Dibnah: Steeplejack (1979) (TV) [Himself] <1> DiCaprio, Leonardo 'Catch Me If You Can': Behind the Camera (2003) (V) [Himself] <6> Titanic (1997) [Jack Dawson] <1> Huwyler, Fabio "Bookclub" (2015) [Himself - Host] Daggs III, Percy Murder101 (2014) [Carlyle] <9> Alton, Peter LolliLove (2004) (voice) [Narrator] <3> Singh, Amrit (I) 2016 Winter Film Awards (2016) (TV) [Presenter] A Social Conversation with Bernie (2016) (TV) [Himself - Host] Breaking Genres (2015) (TV) [Himself - Host] Singh, Amit (I) Corporate (2006) xmltv-1.4.0/t/data-tv_imdb/lists/actresses.list000066400000000000000000000026011500074233200215060ustar00rootroot00000000000000# # Actresses list for testing # - file format follows imdb's format for actresses.list.gz file # THE ACTRESSES LIST ================== Name Titles ---- ------ Actor, Betty (I) Movie1 (1990) [Betty] <1> Movie2 (1991) [Betty] <1> Banerjee, Poroma (II) #REV (2015) [Cinematographer] Zachariah, Sharon #REV (2015) [Interviewee] Ghigliotti, Marilyn #Rip (2013) (voice) [Lydia Walters] <1> Griffin, Martina #Rip (2013) [Juanita] <10> Grossman, Naomi (II) #Rip (2013) [Bella Tiavas] <3> Lee, Michelle (XXXVI) #Rip (2013) [Female News Anchor] <11> Leonards, Ammie #Rip (2013) [CourtNay] <6> Pyle, Missi #Rip (2013) [Lydia Walters] <2> Shea, Beth #Rip (2013) [Liz Tanner] <4> Cornwell, Karina (II) #Selfie (2015) (as Karina Cornell) [Robot Girl] Kent, Elizabeth (V) #Selfie (2015) [The Woman] Kent, Elizabeth (VI) #Selfie (2015) [The Woman] Winslet, Kate 11th Annual Screen Actors Guild Awards (2005) (TV) [Herself - Nominee & Presenter] Titanic (1997) [Rose Dewitt Bukater] <2> Reflections on Titanic (2012) [Herself] <3> Asatryan, Mary 3 Weeks in Yerevan (2016) [Radio Host #2] Bowring, Jean "The Jean Bowring Show" (1957) [Herself - Hostess] Bigelow, Gloria New Now Next Awards (2008) (TV) [Herself - Host] Haze, Roxxy "#BedTimeBitchin" (2014) [Herself - Host] Scott-Smith, Lucy "#SketchPack" (2015) Scott-Smith, Lucy "#SketchPack" (2015) [Various (2015)] xmltv-1.4.0/t/data-tv_imdb/lists/directors.list000066400000000000000000000014661500074233200215200ustar00rootroot00000000000000# # Directors list for testing # - file format follows imdb's format for directors.list.gz file # THE DIRECTORS LIST ================== Name Titles ---- ------ Raimi,Sam Army of Darkness (1992) Director,Joe Movie1 (1990) Movie2 (1991) Director,In1915 Movie100 (1915) Director,In1943 Movie100 (1943) Director,In1953 Movie100 (1953) Aaron, Mark (I) Grease Monkeys (1979) The Rivermen (1980) Berglund, Teodora (II) #Illusion (2014) #Illusion (2014) (co-director) Jousset, Alexandra #Illusion (2014) #Illusion (2014) (co-director) Tanwir, Gibran #iScream (2014) (segment "Beauty Boarding") #iScream (2014) (segment "Caller ID") #iScream (2014) (segment "Nightmare") #iScream (2014) (segment "The Anniversary") #iScream (2014) (segment "VooDoo") Jackson III, Harold The Meek (2017) xmltv-1.4.0/t/data-tv_imdb/lists/genres.list000066400000000000000000000007251500074233200210020ustar00rootroot00000000000000# # Genres list for testing # - file format follows imdb's format for genres.list.gz file # 8: THE GENRES LIST ================== Army of Darkness (1992) Horror Movie1 (1990) Horror Movie2 (1991) Mystery [Film #9 Title] (2015) Comedy [Film #9 Title] (2015) Fantasy [Film #9 Title] (2015) Short (Mon) Jour de chance (2004) Short 'C'-Man (1949) Crime 'C'-Man (1949) Drama 'C'-Man (1949) Film-Noir 'C'-Man (1949) Crime xmltv-1.4.0/t/data-tv_imdb/lists/keywords.list000066400000000000000000000003621500074233200213630ustar00rootroot00000000000000# # Keywords list for testing # - file format follows imdb's format for keywords.list.gz file # 8: THE KEYWORDS LIST ==================== Movie1 (1990) Horror Movie1 (1990) Mystery Movie2 (1991) Comedy Movie2 (1991) Mystery xmltv-1.4.0/t/data-tv_imdb/lists/movies.list000066400000000000000000000051731500074233200210230ustar00rootroot00000000000000# # Movies list for testing # - file format follows imdb's format for movies.list.gz file # MOVIES LIST =========== Army of Darkness (1992) 1992 Movie1 (1990) 1990 Movie2 (1991) 1991 Movie3 and more (1991) 1991 Movie4 & more (1991) 1991 Movie5 no punctuation (1991) 1991 Movie5's with punctuation (1992) 1992 The Movie6 (1991) 1991 The Movie7 (1991) 1991 A Movie8 (1991) 1991 Une Movie9 (1991) 1991 Les Movie10 (1991) 1991 Los Movie11 (1991) 1991 Las Movie12 (1991) 1991 L' Movie13 (1991) 1991 Le Movie14 (1991) 1991 La Movie15 (1991) 1991 El Movie16 (1991) 1991 Das Movie17 (1991) 1991 De Movie18 (1991) 1991 Het Movie19 (1991) 1991 Een Movie20 (1991) 1991 Movie21 aeiouaecnssy (1991) 1991 Movie22 dots (1991) 1991 "The Show1" (2002) 1991 "The Show1" (2002) {Episode title1 (#1.1)} 1991 "The Show1" (2002) {Episode title2 (#1.2)} 1991 "The Show1" (2002) {Episode title1 (#2.1)} 1991 "The Show1" (2002) {Episode title1 (#2.2)} 1991 "The Show1" (2002) {Episode title1 (#2.10)} 1991 Movie100 (1915) 1915 Movie100 (1943) 1943 Movie100 (1953) 1953 Movie100 (1984) (TV) 1984 Movie100 (1989) (VG) 1989 Movie100 (1993) (V) 1993 Movie101 (1992) 1992 Movie101 (1993) (V) 1993 "Movie101" (1988) 1988 "Movie101" (1988) {Episode1 Part 1 (#8.1)} 1992 '83 (2017/I) 2017 '83 (2017/II) 2017 Journey to the Center of the Earth (2008) 2008 Journey to the Center of the Earth (2008) (TV) 2008 Journey to the Center of the Earth (2008) (V) 2008 "Ashes to Ashes" (2008) 2008 Ashes to Ashes (2008) 2008 California Cornflakes (????) ???? Zed (????/II) ???? Family Prayers (aka Karim & Suha) (2010) 2010 "Grease Monkeys" (2003) 2003-???? "Grease Monkeys" (2003) {Almost Blue (#1.4)} 2003 Grease Monkeys (1979) 1979 #Illusion (2014) 2014 #iScream (2014) 2014 #REV (2015) 2015 #Rip (2013) 2013 #Selfie (2015) 2015 Titanic (1997) 1997 Titanic (2012) 2012 Fred Dibnah: Steeplejack (1979) (TV) 1979 "Bookclub" (2015) 2015-???? Murder101 (2014) 2014 LolliLove (2004) 2004 Breaking Genres (2015) (TV) 2015 Corporate (2006) 2006 3 Weeks in Yerevan (2016) 2016 "The Jean Bowring Show" (1957) 1957-1960 New Now Next Awards (2008) (TV) 2008 "#BedTimeBitchin" (2014) 2014-???? #ClivesClub: The Somers Solstice (2015) 2015 "#SketchPack" (2015) 2015-???? [Film #9 Title] (2015) 2015 (Mon) Jour de chance (2004) 2004 'C'-Man (1949) 1949 "#nitTWITS" (2011) 2011-???? The Meek (2015) 2015 The Meek (2017) 2017 -1: Minus One (2016) 2016 xmltv-1.4.0/t/data-tv_imdb/lists/plot.list000066400000000000000000000010241500074233200204660ustar00rootroot00000000000000# # Plots list for testing # - file format follows imdb's format for plot.list.gz file # PLOT SUMMARIES LIST =================== ------------------------------------------------------------------------------- MV: Movie1 (1990) PL: line 1 PL: line 2 PL: line 3 PL: line 4 BY: plot-author ------------------------------------------------------------------------------- MV: Movie2 (1991) PL: line 1 PL: line 2 PL: line 3 PL: line 4 BY: plot-author2 ------------------------------------------------------------------------------- xmltv-1.4.0/t/data-tv_imdb/lists/ratings.list000066400000000000000000000005431500074233200211640ustar00rootroot00000000000000# # Ratings list for testing # - file format follows imdb's format for ratings.list.gz file # # MOVIE RATINGS REPORT New Distribution Votes Rank Title 0000002211 000001 9.9 Army of Darkness (1992) 0000002211 000001 1.0 Movie1 (1990) 0000002211 000002 1.1 Movie2 (1991) 1.1..2...5 8 7.0 "#nitTWITS" (2011) xmltv-1.4.0/t/data-tv_tmdb/000077500000000000000000000000001500074233200154735ustar00rootroot00000000000000xmltv-1.4.0/t/data-tv_tmdb/Cast-actor-with-generation.xml000066400000000000000000000004041500074233200233150ustar00rootroot00000000000000 Murder101 credits include 'Percy Daggs III' 2014 xmltv-1.4.0/t/data-tv_tmdb/Cast-actor-with-generation.xml-expected000066400000000000000000000007661500074233200251270ustar00rootroot00000000000000 Murder101 credits include 'Percy Daggs III' Tom Sizemore Dante Basco Sheldon Robins Paige La Pierre Jasmine Waltz Percy Daggs III 2014 xmltv-1.4.0/t/data-tv_tmdb/Cast-actors-and-actresses.xml000066400000000000000000000004461500074233200231360ustar00rootroot00000000000000 Titanic merge actors and actresses (these were two separate files in tv_imdb) 1997 xmltv-1.4.0/t/data-tv_tmdb/Cast-actors-and-actresses.xml-expected000066400000000000000000000006551500074233200247370ustar00rootroot00000000000000 Titanic merge actors and actresses (these were two separate files in tv_imdb) Leonardo DiCaprio Kate Winslet Billy Zane 1997 xmltv-1.4.0/t/data-tv_tmdb/Cast-billing.xml000066400000000000000000000004041500074233200205230ustar00rootroot00000000000000 Gone with the Wind check cast billing order 1940 xmltv-1.4.0/t/data-tv_tmdb/Cast-billing.xml-expected000066400000000000000000000015351500074233200223300ustar00rootroot00000000000000 Gone with the Wind check cast billing order Victor Fleming Vivien Leigh Clark Gable Olivia de Havilland Leslie Howard Hattie McDaniel Thomas Mitchell Barbara O'Neil Evelyn Keyes Ann Rutherford George Reeves Fred Crane Oscar Polk Butterfly McQueen Victor Jory Everett Brown 1940 xmltv-1.4.0/t/data-tv_tmdb/Cast-duplicate.xml000066400000000000000000000006341500074233200210620ustar00rootroot00000000000000 Gone with the Wind cast names should not be duplicated Victor Fleming Vivien Leigh Barbara O'Neil 1940 xmltv-1.4.0/t/data-tv_tmdb/Cast-duplicate.xml-expected000066400000000000000000000007471500074233200226660ustar00rootroot00000000000000 Gone with the Wind cast names should not be duplicated Victor Fleming Vivien Leigh Clark Gable Olivia de Havilland Barbara O'Neil 1940 xmltv-1.4.0/t/data-tv_tmdb/Cast-host-or-narrator.xml000066400000000000000000000017071500074233200223330ustar00rootroot00000000000000 The Tonight Show with Jay Leno Output should show Jay Leno as "presenter" (converted from role="Self - Host") Talking Snooker Output should show Alistair McGowan as "presenter" (converted from role="Self - Presenter") Pushing Daisies Output should show Jim Dale as "commentator" (converted from "Narrator (voice)") Ted Output should show Patrick Stewart as role="Narrator (voice)" NOT "commentator" as this is a movie 2012 xmltv-1.4.0/t/data-tv_tmdb/Cast-host-or-narrator.xml-expected000066400000000000000000000042771500074233200241370ustar00rootroot00000000000000 The Tonight Show with Jay Leno Output should show Jay Leno as "presenter" (converted from role="Self - Host") Jay Leno Talking Snooker Output should show Alistair McGowan as "presenter" (converted from role="Self - Presenter") Alistair McGowan Pushing Daisies Output should show Jim Dale as "commentator" (converted from "Narrator (voice)") Lee Pace Anna Friel Chi McBride Kristin Chenoweth Swoosie Kurtz Ellen Greene Jim Dale Field Cate Jim Dale Ted Output should show Patrick Stewart as role="Narrator (voice)" NOT "commentator" as this is a movie Seth MacFarlane Mark Wahlberg Mila Kunis Seth MacFarlane Joel McHale Giovanni Ribisi Patrick Warburton Matt Walsh Jessica Barth Aedin Mincks Bill Smitrovich Patrick Stewart 2012 xmltv-1.4.0/t/data-tv_tmdb/Cast-image-url.xml000066400000000000000000000003761500074233200207750ustar00rootroot00000000000000 Titanic output actor "role,image,url" 1997 xmltv-1.4.0/t/data-tv_tmdb/Cast-image-url.xml-expected000066400000000000000000000023621500074233200225710ustar00rootroot00000000000000 Titanic output actor "role,image,url" James Cameron http://image.tmdb.org/t/p/w185/9NAZnTjBQ9WcXAQEzZpKy4vdQto.jpg https://www.themoviedb.org/person/2710 Leonardo DiCaprio http://image.tmdb.org/t/p/w185/wo2hJpn04vbtmh0B9utCFdsQhxM.jpg https://www.themoviedb.org/person/6193 Kate Winslet http://image.tmdb.org/t/p/w185/e3tdop3WhseRnn8KwMVLAV25Ybv.jpg https://www.themoviedb.org/person/204 Billy Zane http://image.tmdb.org/t/p/w185/9HIubetYWAVLlHNb9aObL0fc0sT.jpg https://www.themoviedb.org/person/1954 1997 xmltv-1.4.0/t/data-tv_tmdb/Cast-merge.xml000066400000000000000000000005701500074233200202060ustar00rootroot00000000000000 Titanic merge new credits with existing (+ duplicates removed) Mickey Mouse Kate Winslet 1997 xmltv-1.4.0/t/data-tv_tmdb/Cast-merge.xml-expected000066400000000000000000000007001500074233200220000ustar00rootroot00000000000000 Titanic merge new credits with existing (+ duplicates removed) Leonardo DiCaprio Kate Winslet Billy Zane Mickey Mouse 1997 xmltv-1.4.0/t/data-tv_tmdb/Cast-multiple-role.xml000066400000000000000000000004301500074233200216740ustar00rootroot00000000000000 The Matrix multiple roles combined for one actor (Keanu Reeves) 1999 xmltv-1.4.0/t/data-tv_tmdb/Cast-multiple-role.xml-expected000066400000000000000000000007451500074233200235040ustar00rootroot00000000000000 The Matrix multiple roles combined for one actor (Keanu Reeves) Keanu Reeves Laurence Fishburne Carrie-Anne Moss 1999 xmltv-1.4.0/t/data-tv_tmdb/Cast-role.xml000066400000000000000000000003641500074233200200510ustar00rootroot00000000000000 Titanic output actor "role" 1997 xmltv-1.4.0/t/data-tv_tmdb/Cast-role.xml-expected000066400000000000000000000015471500074233200216540ustar00rootroot00000000000000 Titanic output actor "role" James Cameron Leonardo DiCaprio Kate Winslet Billy Zane Kathy Bates Frances Fisher Gloria Stuart Bill Paxton Bernard Hill David Warner Victor Garber 1997 xmltv-1.4.0/t/data-tv_tmdb/Content-id.xml000066400000000000000000000005521500074233200202230ustar00rootroot00000000000000 Shaun of the Dead check episode_num 2004 999999 123 xmltv-1.4.0/t/data-tv_tmdb/Content-id.xml-expected000066400000000000000000000007471500074233200220300ustar00rootroot00000000000000 Shaun of the Dead check episode_num Edgar Wright 2004 123 747 tt0365748 xmltv-1.4.0/t/data-tv_tmdb/Director-multiple-and-duplicate-directors.xml000066400000000000000000000014351500074233200263300ustar00rootroot00000000000000 Fantastic Journey to Oz should have 3 directors 2017 Fantastic Journey to Oz should have 3 directors (no duplicate) Vladimir Toropchin 2017 Fantastic Voyage should have 1 director (no duplicate) Richard Fleischer 1966 xmltv-1.4.0/t/data-tv_tmdb/Director-multiple-and-duplicate-directors.xml-expected000066400000000000000000000020171500074233200301240ustar00rootroot00000000000000 Fantastic Journey to Oz should have 3 directors Vladimir Toropchin Fedor Dmitriev Darina Shmidt 2017 Fantastic Journey to Oz should have 3 directors (no duplicate) Vladimir Toropchin Fedor Dmitriev Darina Shmidt 2017 Fantastic Voyage should have 1 director (no duplicate) Richard Fleischer 1966 xmltv-1.4.0/t/data-tv_tmdb/Director-with-generation.xml000066400000000000000000000004021500074233200230660ustar00rootroot00000000000000 The Meek director is 'Harold Jackson III' 2017 xmltv-1.4.0/t/data-tv_tmdb/Director-with-generation.xml-expected000066400000000000000000000005161500074233200246730ustar00rootroot00000000000000 The Meek director is 'Harold Jackson III' Harold Jackson III 2017 xmltv-1.4.0/t/data-tv_tmdb/Genres-duplicate.xml000066400000000000000000000004521500074233200214110ustar00rootroot00000000000000 Scars of Dracula 'Horror' should only appear once 1970 Horror xmltv-1.4.0/t/data-tv_tmdb/Genres-duplicate.xml-expected000066400000000000000000000005361500074233200232130ustar00rootroot00000000000000 Scars of Dracula 'Horror' should only appear once 1970 Movie Horror xmltv-1.4.0/t/data-tv_tmdb/Genres-multiple.xml000066400000000000000000000004221500074233200212670ustar00rootroot00000000000000 The Flesh and Blood Show 'Movie' added to multiple genres 1972 xmltv-1.4.0/t/data-tv_tmdb/Genres-multiple.xml-expected000066400000000000000000000006221500074233200230700ustar00rootroot00000000000000 The Flesh and Blood Show 'Movie' added to multiple genres 1972 Movie Horror Thriller xmltv-1.4.0/t/data-tv_tmdb/Genres-single.xml000066400000000000000000000004071500074233200207200ustar00rootroot00000000000000 Scars of Dracula 'Movie' added to single genre 1970 xmltv-1.4.0/t/data-tv_tmdb/Genres-single.xml-expected000066400000000000000000000005331500074233200225170ustar00rootroot00000000000000 Scars of Dracula 'Movie' added to single genre 1970 Movie Horror xmltv-1.4.0/t/data-tv_tmdb/Movie-and-tv.xml000066400000000000000000000020001500074233200204530ustar00rootroot00000000000000 Wonder Woman regular movie (2009) 2009 Wonder Woman regular movie (1974) 1974 Wonder Woman this wrongly selects the 1974 movie (as off-by-one) - prog is assumed to be a movie (see next test case) 1975 Wonder Woman tv series (1975) 1975 TV Series Wonder Woman insufficient data to find on tmdb xmltv-1.4.0/t/data-tv_tmdb/Movie-and-tv.xml-expected000066400000000000000000000023351500074233200222650ustar00rootroot00000000000000 Wonder Woman regular movie (2009) Lauren Montgomery 2009 Wonder Woman regular movie (1974) Vincent McEveety 1974 Wonder Woman this wrongly selects the 1974 movie (as off-by-one) - prog is assumed to be a movie (see next test case) Vincent McEveety 1975 Wonder Woman tv series (1975) 1975 TV Series Wonder Woman insufficient data to find on tmdb xmltv-1.4.0/t/data-tv_tmdb/Movie-icon-and-url.xml000066400000000000000000000003751500074233200215670ustar00rootroot00000000000000 Shaun of the Dead check icon and url 2004 xmltv-1.4.0/t/data-tv_tmdb/Movie-icon-and-url.xml-expected000066400000000000000000000017231500074233200233640ustar00rootroot00000000000000 Shaun of the Dead check icon and url Edgar Wright 2004 https://www.themoviedb.org/movie/747 https://www.imdb.com/title/tt0365748/ http://image.tmdb.org/t/p/w500/dgXPhzNJH8HFTBjXPB177yNx6RI.jpg http://image.tmdb.org/t/p/w780/mrdHbaCp3ysDrzUHle5eQlY9Vzu.jpg http://image.tmdb.org/t/p/w92/dgXPhzNJH8HFTBjXPB177yNx6RI.jpg http://image.tmdb.org/t/p/w300/mrdHbaCp3ysDrzUHle5eQlY9Vzu.jpg xmltv-1.4.0/t/data-tv_tmdb/Movie-same-year-movie-and-series.xml000066400000000000000000000025151500074233200243250ustar00rootroot00000000000000 Journey to the Center of the Earth multiple titles (movie,video,tv) with same title+year - no match 2008 Journey to the Center of the Earth multiple titles (movie,video,tv) with same title - match on category, but year not used therefore no exact match (1999/1967) 1999 TV Series Journey to the Center of the Earth multiple titles (movie,video,tv) with same title+year - match using director name Eric Brevig 2008 Journey to the Center of the Earth multiple titles (movie,video,tv) with same title+year - match using director name T.J. Scott 2008 xmltv-1.4.0/t/data-tv_tmdb/Movie-same-year-movie-and-series.xml-expected000066400000000000000000000027231500074233200261250ustar00rootroot00000000000000 Journey to the Center of the Earth multiple titles (movie,video,tv) with same title+year - no match 2008 Journey to the Center of the Earth multiple titles (movie,video,tv) with same title - match on category, but year not used therefore no exact match (1999/1967) 1999 TV Series Journey to the Center of the Earth multiple titles (movie,video,tv) with same title+year - match using director name Eric Brevig Brendan Fraser Josh Hutcherson Anita Briem 2008 Journey to the Center of the Earth multiple titles (movie,video,tv) with same title+year - match using director name T.J. Scott Ricky Schroder Victoria Pratt Peter Fonda 2008 xmltv-1.4.0/t/data-tv_tmdb/Movie-two-in-same-year.xml000066400000000000000000000025101500074233200223660ustar00rootroot00000000000000 20,000 Leagues Under the Sea cannot identify a sole hit - two films in same year with this title 1997 Hell or High Water cannot identify a sole hit - two films in same year with this title 2016 Chaos cannot identify a sole hit - two films in same year with this title 2005 Chaos exact match - two films in same year with this title - using director name Tony Giglio 2005 Chaos exact match - two films in same year with this title - using director name David DeFalco 2005 xmltv-1.4.0/t/data-tv_tmdb/Movie-two-in-same-year.xml-expected000066400000000000000000000027561500074233200242010ustar00rootroot00000000000000 20,000 Leagues Under the Sea cannot identify a sole hit - two films in same year with this title 1997 Hell or High Water cannot identify a sole hit - two films in same year with this title 2016 Chaos cannot identify a sole hit - two films in same year with this title 2005 Chaos exact match - two films in same year with this title - using director name Tony Giglio Jason Statham Ryan Phillippe Wesley Snipes 2005 Chaos exact match - two films in same year with this title - using director name David DeFalco Kevin Gage Sage Stallone Kelly K.C. Quann 2005 xmltv-1.4.0/t/data-tv_tmdb/Movie-with-aka.xml000066400000000000000000000006531500074233200210030ustar00rootroot00000000000000 Angèle and Tony exact match 2011 Angèle et Tony exact match on alternative title 2011 xmltv-1.4.0/t/data-tv_tmdb/Movie-with-aka.xml-expected000066400000000000000000000011501500074233200225730ustar00rootroot00000000000000 Angèle and Tony exact match Alix Delaporte 2011 Angèle and Tony Angèle et Tony exact match on alternative title Alix Delaporte 2011 xmltv-1.4.0/t/data-tv_tmdb/Movie-with-unknown-year.xml000066400000000000000000000006311500074233200227000ustar00rootroot00000000000000 Zed no match as TMDB has no release-date Zed no match as TMDB has no release-date 2010 xmltv-1.4.0/t/data-tv_tmdb/Movie-with-unknown-year.xml-expected000066400000000000000000000006321500074233200245000ustar00rootroot00000000000000 Zed no match as TMDB has no release-date Zed no match as TMDB has no release-date 2010 xmltv-1.4.0/t/data-tv_tmdb/Movie-years.xml000066400000000000000000000016461500074233200204240ustar00rootroot00000000000000 The Matrix standard fetch but no urls 1999 The Matrix match - off by one year 2000 The Matrix match - off by one year 1998 The Matrix match - off by two years 1997 The Matrix no match - off by 3 years 1996 xmltv-1.4.0/t/data-tv_tmdb/Movie-years.xml-expected000066400000000000000000000025521500074233200222200ustar00rootroot00000000000000 The Matrix standard fetch but no urls Lilly Wachowski Lana Wachowski 1999 The Matrix match - off by one year Lilly Wachowski Lana Wachowski 2000 The Matrix match - off by one year Lilly Wachowski Lana Wachowski 1998 The Matrix match - off by two years Lilly Wachowski Lana Wachowski 1997 The Matrix no match - off by 3 years 1996 xmltv-1.4.0/t/data-tv_tmdb/Movie.xml000066400000000000000000000005041500074233200172730ustar00rootroot00000000000000 The Matrix standard fetch but no urls - check correct director, actors, plot, rating and keywords in output 1999 xmltv-1.4.0/t/data-tv_tmdb/Movie.xml-expected000066400000000000000000000021351500074233200210740ustar00rootroot00000000000000 The Matrix standard fetch but no urls - check correct director, actors, plot, rating and keywords in output Lilly Wachowski Lana Wachowski Keanu Reeves Laurence Fishburne Carrie-Anne Moss 1999 Movie Action Science Fiction 136 603 tt0133093 14A 15 8.2/10 xmltv-1.4.0/t/data-tv_tmdb/Movies-only.xml000066400000000000000000000006441500074233200204420ustar00rootroot00000000000000 The Matrix augmented with Director 1999 The Tonight Show with Jay Leno no augment - --movies-only xmltv-1.4.0/t/data-tv_tmdb/Movies-only.xml-expected000066400000000000000000000012351500074233200222360ustar00rootroot00000000000000 The Matrix augmented with Director Lilly Wachowski Lana Wachowski 1999 Movie Action Science Fiction The Tonight Show with Jay Leno no augment - --movies-only xmltv-1.4.0/t/data-tv_tmdb/Ratings.xml000066400000000000000000000014661500074233200176330ustar00rootroot00000000000000 Titanic add ratings - movie 1997 Father Ted: A Christmassy Ted add ratings - movie - not added as certification is blank, and too few votes for star-rating 1996 Father Ted add ratings - tv series The Matrix multiple censor classifications 1999 xmltv-1.4.0/t/data-tv_tmdb/Ratings.xml-expected000066400000000000000000000034551500074233200214320ustar00rootroot00000000000000 Titanic add ratings - movie James Cameron 1997 Movie 12 PG-13 7.9/10 Father Ted: A Christmassy Ted add ratings - movie - not added as certification is blank, and too few votes for star-rating Declan Lowney 1996 Movie Father Ted add ratings - tv series TV Series 15 8.1/10 The Matrix multiple censor classifications Lilly Wachowski Lana Wachowski 1999 Movie 14A 15 8.2/10 xmltv-1.4.0/t/data-tv_tmdb/Show-movies-only.xml000066400000000000000000000006701500074233200214170ustar00rootroot00000000000000 Father Ted no match (movies only) Father Ted no match (movies only) 1995 TV Series xmltv-1.4.0/t/data-tv_tmdb/Show-movies-only.xml-expected000066400000000000000000000006711500074233200232170ustar00rootroot00000000000000 Father Ted no match (movies only) Father Ted no match (movies only) 1995 TV Series xmltv-1.4.0/t/data-tv_tmdb/Show-two-in-same-year.xml000066400000000000000000000007141500074233200222330ustar00rootroot00000000000000 The IT Crowd no match - has 2 series with same name 2006 and 2007 The IT Crowd no match - has 2 series with same name 2006 and 2007 2006 xmltv-1.4.0/t/data-tv_tmdb/Show-two-in-same-year.xml-expected000066400000000000000000000007141500074233200240320ustar00rootroot00000000000000 The IT Crowd no match - has 2 series with same name 2006 and 2007 The IT Crowd no match - has 2 series with same name 2006 and 2007 2006 xmltv-1.4.0/t/data-tv_tmdb/Show.xml000066400000000000000000000011451500074233200171360ustar00rootroot00000000000000 Father Ted match on tv series Father Ted match on tv series 1995 TV Series Father Ted no match - assumed to be a film 1995 xmltv-1.4.0/t/data-tv_tmdb/Show.xml-expected000066400000000000000000000012161500074233200207340ustar00rootroot00000000000000 Father Ted match on tv series TV Series Father Ted match on tv series 1995 TV Series Father Ted no match - assumed to be a film 1995 xmltv-1.4.0/t/data-tv_tmdb/Title-and-amp.xml000066400000000000000000000006421500074233200206130ustar00rootroot00000000000000 The King and I correct title 1956 The King & I correct title is added 1956 xmltv-1.4.0/t/data-tv_tmdb/Title-and-amp.xml-expected000066400000000000000000000011271500074233200224110ustar00rootroot00000000000000 The King and I correct title Walter Lang 1956 The King and I The King & I correct title is added Walter Lang 1956 xmltv-1.4.0/t/data-tv_tmdb/Title-articles.xml000066400000000000000000000013621500074233200211040ustar00rootroot00000000000000 The Artist exact match 2011 Artist, The exact match - title is corrected 2011 Le Corbeau exact match 1943 Corbeau, Le exact match - added corrected title 1943 xmltv-1.4.0/t/data-tv_tmdb/Title-articles.xml-expected000066400000000000000000000020711500074233200227010ustar00rootroot00000000000000 The Artist exact match Michel Hazanavicius 2011 The Artist exact match - title is corrected Michel Hazanavicius 2011 Le Corbeau exact match Henri-Georges Clouzot 1943 Le Corbeau exact match - added corrected title Henri-Georges Clouzot 1943 xmltv-1.4.0/t/data-tv_tmdb/Title-case-insensitive.xml000066400000000000000000000004031500074233200225420ustar00rootroot00000000000000 THE MATRIX case insensitive match on title 1999 xmltv-1.4.0/t/data-tv_tmdb/Title-case-insensitive.xml-expected000066400000000000000000000006001500074233200243400ustar00rootroot00000000000000 The Matrix case insensitive match on title Lilly Wachowski Lana Wachowski 1999 xmltv-1.4.0/t/data-tv_tmdb/Title-dots.xml000066400000000000000000000012001500074233200202360ustar00rootroot00000000000000 R.O.T.O.R. dots should be preserved 1987 R.O.T.O.R dots should be preserved; one dot missing - should be added 1987 T.i.t.a.n.i.c dots are removed from corrected title 1997 xmltv-1.4.0/t/data-tv_tmdb/Title-dots.xml-expected000066400000000000000000000015421500074233200220460ustar00rootroot00000000000000 R.O.T.O.R. dots should be preserved Cullen Blaine 1987 R.O.T.O.R. dots should be preserved; one dot missing - should be added Cullen Blaine 1987 Titanic dots are removed from corrected title James Cameron 1997 xmltv-1.4.0/t/data-tv_tmdb/Title-ignore-punc.xml000066400000000000000000000011261500074233200215220ustar00rootroot00000000000000 The Matrix reduced fetch for reference 1999 The .....Matrix punctuation is removed 1999 The:The Matrix correct title is added 1999 xmltv-1.4.0/t/data-tv_tmdb/Title-ignore-punc.xml-expected000066400000000000000000000017321500074233200233240ustar00rootroot00000000000000 The Matrix reduced fetch for reference Lilly Wachowski Lana Wachowski 1999 The Matrix punctuation is removed Lilly Wachowski Lana Wachowski 1999 The Matrix The:The Matrix correct title is added Lilly Wachowski Lana Wachowski 1999 xmltv-1.4.0/t/data-tv_tmdb/Title-startswith-hyphen.xml000066400000000000000000000003501500074233200227770ustar00rootroot00000000000000 -1 matches "-1" 2015 xmltv-1.4.0/t/data-tv_tmdb/Title-startswith-hyphen.xml-expected000066400000000000000000000004561500074233200246050ustar00rootroot00000000000000 -1 matches "-1" Natassa Xydi 2015 xmltv-1.4.0/t/data-tv_tmdb/Title-with-accent.xml000066400000000000000000000011061500074233200215000ustar00rootroot00000000000000 Angèle and Tony title has e-grave 2011 Amélie title has e-acute 2001 Mädchen in Uniform title has a-umlaut 1931 xmltv-1.4.0/t/data-tv_tmdb/Title-with-accent.xml-expected000066400000000000000000000014401500074233200233000ustar00rootroot00000000000000 Angèle and Tony title has e-grave Alix Delaporte 2011 Amélie title has e-acute Jean-Pierre Jeunet 2001 Mädchen in Uniform title has a-umlaut Leontine Sagan 1931 xmltv-1.4.0/t/data-tv_tmdb/Title-with-not-az.xml000066400000000000000000000006101500074233200214520ustar00rootroot00000000000000 M*A*S*H exact match on tv series #RealMovie exact match on movie 2013 xmltv-1.4.0/t/data-tv_tmdb/Title-with-not-az.xml-expected000066400000000000000000000012141500074233200232520ustar00rootroot00000000000000 M*A*S*H exact match on tv series Alan Alda Mike Farrell Loretta Swit #RealMovie exact match on movie Eva Llorach Rocío León Pablo Maqueda 2013 xmltv-1.4.0/t/data-tv_tmdb/Title-with-punc.xml000066400000000000000000000023271500074233200212160ustar00rootroot00000000000000 Pete Kelly's Blues title contains valid punctuation 1955 Pete Kellys Blues title punctuation is added 1955 G.I. Joe: The Rise of Cobra exact match 2009 GI Joe: The Rise of Cobra exact match - punctuation is corrected 2009 G.I. Joe: Rise of Cobra exact match - punctuation is corrected, "The" is added 2009 GI Joe: Rise of Cobra exact match - punctuation is corrected, "The" is added 2009 xmltv-1.4.0/t/data-tv_tmdb/Title-with-punc.xml-expected000066400000000000000000000032611500074233200230130ustar00rootroot00000000000000 Pete Kelly's Blues title contains valid punctuation Jack Webb 1955 Pete Kelly's Blues title punctuation is added Jack Webb 1955 G.I. Joe: The Rise of Cobra exact match Stephen Sommers 2009 G.I. Joe: The Rise of Cobra exact match - punctuation is corrected Stephen Sommers 2009 G.I. Joe: The Rise of Cobra exact match - punctuation is corrected, "The" is added Stephen Sommers 2009 G.I. Joe: The Rise of Cobra exact match - punctuation is corrected, "The" is added Stephen Sommers 2009 xmltv-1.4.0/t/data-tv_tmdb/Title-year-from-title.xml000066400000000000000000000027311500074233200223170ustar00rootroot00000000000000 The Invisible Man (1933) exact match on movie - year removed from title Foxcatcher exact match 2014 Films Foxcatcher (2014) also exact match Films Foxcatcher (2014) matches using date from title (which is then stripped in output) Films Foxcatcher (2013) matches on a close hit Films Foxcatcher (2000) neither exact or close match (off by 14 years) Films xmltv-1.4.0/t/data-tv_tmdb/Title-year-from-title.xml-expected000066400000000000000000000035521500074233200241200ustar00rootroot00000000000000 The Invisible Man exact match on movie - year removed from title James Whale 1933 Foxcatcher exact match Bennett Miller 2014 Films Foxcatcher also exact match Bennett Miller 2014 Films Foxcatcher matches using date from title (which is then stripped in output) Bennett Miller 2014 Films Foxcatcher matches on a close hit Bennett Miller 2013 Films Foxcatcher (2000) neither exact or close match (off by 14 years) Films xmltv-1.4.0/t/data-tv_tmdb/Utf8.xml000066400000000000000000000007021500074233200170420ustar00rootroot00000000000000 Angèle and Tony title has e-grave 2011 Titanic has german director with umlaut - Lutz Büscher 1984 xmltv-1.4.0/t/data-tv_tmdb/Utf8.xml-expected000066400000000000000000000011201500074233200206340ustar00rootroot00000000000000 Angèle and Tony title has e-grave Alix Delaporte 2011 Titanic has german director with umlaut - Lutz Büscher Lutz Büscher 1984 xmltv-1.4.0/t/data-tv_tmdb/configs/000077500000000000000000000000001500074233200171235ustar00rootroot00000000000000xmltv-1.4.0/t/data-tv_tmdb/configs/Cast-actor-with-generation.conf000066400000000000000000000006441500074233200251000ustar00rootroot00000000000000# standard trimmed # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=0 updateActors=1 updatePresentors=1 updateCommentators=1 updateGuests=1 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=6 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Cast-actors-and-actresses.conf000066400000000000000000000006431500074233200247120ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=0 updateActors=1 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Cast-billing.conf000066400000000000000000000006531500074233200223060ustar00rootroot00000000000000# credits only; no roles # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=1 updatePresentors=1 updateCommentators=1 updateGuests=1 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=15 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=0 getYearFromTitles=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Cast-duplicate.conf000066400000000000000000000006521500074233200226370ustar00rootroot00000000000000# credits only; no roles # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=1 updatePresentors=1 updateCommentators=1 updateGuests=1 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=0 getYearFromTitles=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Cast-host-or-narrator.conf000066400000000000000000000006551500074233200241110ustar00rootroot00000000000000# credits only; with roles # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=1 updatePresentors=1 updateCommentators=1 updateGuests=1 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=11 updateActorRole=1 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=0 getYearFromTitles=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Cast-image-url.conf000066400000000000000000000006541500074233200225510ustar00rootroot00000000000000# credits only; with roles # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=1 updatePresentors=1 updateCommentators=1 updateGuests=1 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=1 updateCastImage=1 updateCastUrl=1 removeYearFromTitles=0 getYearFromTitles=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Cast-merge.conf000066400000000000000000000006431500074233200217640ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=0 updateActors=1 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Cast-multiple-role.conf000066400000000000000000000006541500074233200234610ustar00rootroot00000000000000# credits only; with roles # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=0 updateActors=1 updatePresentors=1 updateCommentators=1 updateGuests=1 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=1 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=0 getYearFromTitles=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Cast-role.conf000066400000000000000000000006551500074233200216310ustar00rootroot00000000000000# credits only; with roles # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=1 updatePresentors=1 updateCommentators=1 updateGuests=1 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=10 updateActorRole=1 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=0 getYearFromTitles=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Content-id.conf000066400000000000000000000006541500074233200220030ustar00rootroot00000000000000# content-id # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=1 updateImage=0 numActors=0 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=0 getYearFromTitles=0 movies-only=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Director-multiple-and-duplicate-directors.conf000066400000000000000000000006521500074233200301050ustar00rootroot00000000000000# credits only; no roles # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=1 updatePresentors=1 updateCommentators=1 updateGuests=1 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=0 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=0 getYearFromTitles=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Director-with-generation.conf000066400000000000000000000006531500074233200246530ustar00rootroot00000000000000# credits only; no actors # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=1 updatePresentors=1 updateCommentators=1 updateGuests=1 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=0 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=0 getYearFromTitles=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Genres-duplicate.conf000066400000000000000000000006621500074233200231710ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=1 updateCategoriesWithGenres=1 updateKeywords=0 updateURLs=0 updateDirectors=0 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Genres-multiple.conf000066400000000000000000000006621500074233200230520ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=1 updateCategoriesWithGenres=1 updateKeywords=0 updateURLs=0 updateDirectors=0 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Genres-single.conf000066400000000000000000000006621500074233200225000ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=1 updateCategoriesWithGenres=1 updateKeywords=0 updateURLs=0 updateDirectors=0 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Movie-and-tv.conf000066400000000000000000000006621500074233200222440ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Movie-icon-and-url.conf000066400000000000000000000006731500074233200233450ustar00rootroot00000000000000# icon and url; movies only # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=1 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=1 numActors=0 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=0 getYearFromTitles=0 movies-only=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Movie-same-year-movie-and-series.conf000066400000000000000000000006621500074233200261030ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=1 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Movie-two-in-same-year.conf000066400000000000000000000006621500074233200241510ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=1 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Movie-with-aka.conf000066400000000000000000000006751500074233200225640ustar00rootroot00000000000000# reduced output; movies only # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Movie-with-unknown-year.conf000066400000000000000000000006751500074233200244650ustar00rootroot00000000000000# reduced output; movies only # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Movie-years.conf000066400000000000000000000006751500074233200222020ustar00rootroot00000000000000# reduced output; movies only # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Movie.conf000066400000000000000000000006531500074233200210550ustar00rootroot00000000000000# standard except no urls # updateDates=1 updateTitles=1 updateCategories=1 updateCategoriesWithGenres=1 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=1 updatePresentors=1 updateCommentators=1 updateGuests=1 updateStarRatings=1 updateRatings=1 updatePlot=0 updateRuntime=1 updateContentId=1 updateImage=0 numActors=3 updateActorRole=1 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Movies-only.conf000066400000000000000000000006741500074233200222220ustar00rootroot00000000000000# reduced fetch; movies only # updateDates=1 updateTitles=1 updateCategories=1 updateCategoriesWithGenres=1 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=0 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Ratings.conf000066400000000000000000000006621500074233200214050ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=1 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=1 updateRatings=1 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Show-movies-only.conf000066400000000000000000000006621500074233200231750ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=1 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Show-two-in-same-year.conf000066400000000000000000000006621500074233200240120ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=1 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Show.conf000066400000000000000000000006621500074233200207160ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=1 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Title-and-amp.conf000066400000000000000000000006751500074233200223760ustar00rootroot00000000000000# reduced output; movies only # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Title-articles.conf000066400000000000000000000006751500074233200226670ustar00rootroot00000000000000# reduced output; movies only # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Title-case-insensitive.conf000066400000000000000000000006421500074233200243240ustar00rootroot00000000000000# reduced output # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=0 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Title-dots.conf000066400000000000000000000006751500074233200220320ustar00rootroot00000000000000# reduced output; movies only # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Title-ignore-punc.conf000066400000000000000000000006751500074233200233070ustar00rootroot00000000000000# reduced output; movies only # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Title-startswith-hyphen.conf000066400000000000000000000006751500074233200245660ustar00rootroot00000000000000# reduced output; movies only # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Title-with-accent.conf000066400000000000000000000006741500074233200232660ustar00rootroot00000000000000# reduced fetch; movies only # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=0 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Title-with-not-az.conf000066400000000000000000000006621500074233200232360ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=0 updateActors=1 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=0 xmltv-1.4.0/t/data-tv_tmdb/configs/Title-with-punc.conf000066400000000000000000000006751500074233200227770ustar00rootroot00000000000000# reduced output; movies only # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Title-year-from-title.conf000066400000000000000000000006621500074233200240750ustar00rootroot00000000000000# reduced output; # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=3 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data-tv_tmdb/configs/Utf8.conf000066400000000000000000000006741500074233200206270ustar00rootroot00000000000000# reduced fetch; movies only # updateDates=1 updateTitles=1 updateCategories=0 updateCategoriesWithGenres=0 updateKeywords=0 updateURLs=0 updateDirectors=1 updateActors=0 updatePresentors=0 updateCommentators=0 updateGuests=0 updateStarRatings=0 updateRatings=0 updatePlot=0 updateRuntime=0 updateContentId=0 updateImage=0 numActors=0 updateActorRole=0 updateCastImage=0 updateCastUrl=0 removeYearFromTitles=1 getYearFromTitles=1 movies-only=1 xmltv-1.4.0/t/data/000077500000000000000000000000001500074233200140365ustar00rootroot00000000000000xmltv-1.4.0/t/data/amp.xml000066400000000000000000000006721500074233200153420ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/attrs.xml000066400000000000000000000005411500074233200157150ustar00rootroot00000000000000 Attrib Rameau xmltv-1.4.0/t/data/clump.xml000066400000000000000000000006251500074233200157030ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/clump_extract.xml000066400000000000000000000006611500074233200174350ustar00rootroot00000000000000 Kilroy BBC News; Weather xmltv-1.4.0/t/data/clump_extract_1.xml000066400000000000000000000007071500074233200176560ustar00rootroot00000000000000 A news news xmltv-1.4.0/t/data/dups.xml000066400000000000000000000013051500074233200155320ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/empty.xml000066400000000000000000000001231500074233200157120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/intervals.xml000066400000000000000000000024241500074233200165710ustar00rootroot00000000000000 On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/length.xml000066400000000000000000000004471500074233200160460ustar00rootroot00000000000000 News Lots of news. 30 xmltv-1.4.0/t/data/overlap.xml000066400000000000000000000034111500074233200162270ustar00rootroot00000000000000 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle xmltv-1.4.0/t/data/simple.xml000066400000000000000000000014571500074233200160600ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/sort.xml000066400000000000000000000017501500074233200155520ustar00rootroot00000000000000 ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B xmltv-1.4.0/t/data/sort1.xml000066400000000000000000000011761500074233200156350ustar00rootroot00000000000000 A B C D E F xmltv-1.4.0/t/data/sort2.xml000066400000000000000000000005511500074233200156320ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/test.xml000066400000000000000000000036761500074233200155530ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 60 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/test_empty.xml000066400000000000000000000004541500074233200167600ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/test_livre.xml000066400000000000000000000003051500074233200167360ustar00rootroot00000000000000 xmltv-1.4.0/t/data/test_remove_some_overlapping.xml000066400000000000000000000030021500074233200225400ustar00rootroot00000000000000 Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 xmltv-1.4.0/t/data/test_sort_by_channel.xml000066400000000000000000000010131500074233200207630ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_cat_all_UTF8.expected000066400000000000000000000246051500074233200205060ustar00rootroot00000000000000 3SAT ARD Das Erste Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g A B D t Attrib Rameau Kilroy BBC News; Weather A news On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. News Lots of news. 30 C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B A B C D E F A B C blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg A programme with empty stuff that should not be written out again Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 A B C T Blah. xmltv-1.4.0/t/data/tv_cat_amp_xml.expected000066400000000000000000000006721500074233200205630ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_cat_amp_xml_amp_xml.expected000066400000000000000000000006721500074233200223000ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_cat_amp_xml_clump_xml.expected000066400000000000000000000013731500074233200226420ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_cat_amp_xml_dups_xml.expected000066400000000000000000000014221500074233200224700ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News A B D t xmltv-1.4.0/t/data/tv_cat_amp_xml_empty_xml.expected000066400000000000000000000006721500074233200226610ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_cat_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000013731500074233200270360ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_cat_attrs_xml.expected000066400000000000000000000005201500074233200211330ustar00rootroot00000000000000 Attrib Rameau xmltv-1.4.0/t/data/tv_cat_clump_extract_1_xml.expected000066400000000000000000000005131500074233200230720ustar00rootroot00000000000000 A news xmltv-1.4.0/t/data/tv_cat_clump_extract_xml.expected000066400000000000000000000006611500074233200226560ustar00rootroot00000000000000 Kilroy BBC News; Weather xmltv-1.4.0/t/data/tv_cat_clump_xml.expected000066400000000000000000000006251500074233200211240ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_cat_clump_xml_amp_xml.expected000066400000000000000000000013731500074233200226420ustar00rootroot00000000000000 First in clump b Second in clump g Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_cat_clump_xml_clump_xml.expected000066400000000000000000000006251500074233200232040ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_cat_clump_xml_dups_xml.expected000066400000000000000000000013551500074233200230400ustar00rootroot00000000000000 First in clump b Second in clump g A B D t xmltv-1.4.0/t/data/tv_cat_clump_xml_empty_xml.expected000066400000000000000000000006251500074233200232220ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_cat_dups_xml.expected000066400000000000000000000006541500074233200207610ustar00rootroot00000000000000 A B D t xmltv-1.4.0/t/data/tv_cat_dups_xml_amp_xml.expected000066400000000000000000000014221500074233200224700ustar00rootroot00000000000000 A B D t Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_cat_dups_xml_clump_xml.expected000066400000000000000000000013551500074233200230400ustar00rootroot00000000000000 A B D t First in clump b Second in clump g xmltv-1.4.0/t/data/tv_cat_dups_xml_dups_xml.expected000066400000000000000000000006541500074233200226740ustar00rootroot00000000000000 A B D t xmltv-1.4.0/t/data/tv_cat_dups_xml_empty_xml.expected000066400000000000000000000006541500074233200230570ustar00rootroot00000000000000 A B D t xmltv-1.4.0/t/data/tv_cat_empty_xml.expected000066400000000000000000000001231500074233200211330ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_cat_empty_xml_amp_xml.expected000066400000000000000000000006721500074233200226610ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_cat_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200232220ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_cat_empty_xml_dups_xml.expected000066400000000000000000000006541500074233200230570ustar00rootroot00000000000000 A B D t xmltv-1.4.0/t/data/tv_cat_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200232310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_cat_intervals_xml.expected000066400000000000000000000016421500074233200220130ustar00rootroot00000000000000 On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/tv_cat_length_xml.expected000066400000000000000000000004471500074233200212670ustar00rootroot00000000000000 News Lots of news. 30 xmltv-1.4.0/t/data/tv_cat_overlap_xml.expected000066400000000000000000000030411500074233200214470ustar00rootroot00000000000000 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle xmltv-1.4.0/t/data/tv_cat_simple_xml.expected000066400000000000000000000014261500074233200212750ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_cat_simple_xml_x_whatever_xml.expected000066400000000000000000000014261500074233200244110ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_cat_sort1_xml.expected000066400000000000000000000011761500074233200210560ustar00rootroot00000000000000 A B C D E F xmltv-1.4.0/t/data/tv_cat_sort2_xml.expected000066400000000000000000000005511500074233200210530ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_cat_sort_xml.expected000066400000000000000000000017501500074233200207730ustar00rootroot00000000000000 ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B xmltv-1.4.0/t/data/tv_cat_test_empty_xml.expected000066400000000000000000000003571500074233200222030ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_cat_test_livre_xml.expected000066400000000000000000000003061500074233200221600ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_cat_test_remove_some_overlapping_xml.expected000066400000000000000000000030021500074233200257610ustar00rootroot00000000000000 Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 xmltv-1.4.0/t/data/tv_cat_test_sort_by_channel_xml.expected000066400000000000000000000005771500074233200242220ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_cat_test_xml.expected000066400000000000000000000036401500074233200207630ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_cat_test_xml_test_xml.expected000066400000000000000000000036401500074233200227020ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_cat_whitespace_xml.expected000066400000000000000000000003111500074233200221300ustar00rootroot00000000000000 T Blah. xmltv-1.4.0/t/data/tv_cat_x_whatever_xml.expected000066400000000000000000000014261500074233200221600ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_extractinfo_en_all_UTF8.expected000066400000000000000000000305131500074233200227420ustar00rootroot00000000000000 3SAT ARD Das Erste Ampersand Land & & hello && there &amp; everyone < &< <amp; News news First in clump b Second in clump g A B A D B t t Attrib Rameau Kilroy BBC News; Weather news A news news news news On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. News Lots of news. news 30 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport news A B A B C D E F A B C blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg A programme with empty stuff that should not be written out again Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 A B C T Blah. T Blah. T Blah. T Blah. T Blah. The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_extractinfo_en_amp_xml.expected000066400000000000000000000007421500074233200230220ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News news xmltv-1.4.0/t/data/tv_extractinfo_en_amp_xml_amp_xml.expected000066400000000000000000000015601500074233200245360ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News news Ampersand Land & & hello && there &amp; everyone < &< <amp; News news xmltv-1.4.0/t/data/tv_extractinfo_en_amp_xml_clump_xml.expected000066400000000000000000000014431500074233200251010ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News news First in clump b Second in clump g xmltv-1.4.0/t/data/tv_extractinfo_en_amp_xml_dups_xml.expected000066400000000000000000000021231500074233200247300ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News news A B A D B t t xmltv-1.4.0/t/data/tv_extractinfo_en_amp_xml_empty_xml.expected000066400000000000000000000007421500074233200251200ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News news xmltv-1.4.0/t/data/tv_extractinfo_en_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000014431500074233200312750ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News news First in clump b Second in clump g xmltv-1.4.0/t/data/tv_extractinfo_en_attrs_xml.expected000066400000000000000000000005201500074233200233740ustar00rootroot00000000000000 Attrib Rameau xmltv-1.4.0/t/data/tv_extractinfo_en_clump_extract_1_xml.expected000066400000000000000000000010301500074233200253260ustar00rootroot00000000000000 A news news news news xmltv-1.4.0/t/data/tv_extractinfo_en_clump_extract_xml.expected000066400000000000000000000007311500074233200251150ustar00rootroot00000000000000 Kilroy BBC News; Weather news xmltv-1.4.0/t/data/tv_extractinfo_en_clump_xml.expected000066400000000000000000000006251500074233200233650ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_extractinfo_en_clump_xml_amp_xml.expected000066400000000000000000000014431500074233200251010ustar00rootroot00000000000000 First in clump b Second in clump g Ampersand Land & & hello && there &amp; everyone < &< <amp; News news xmltv-1.4.0/t/data/tv_extractinfo_en_clump_xml_clump_xml.expected000066400000000000000000000013261500074233200254440ustar00rootroot00000000000000 First in clump b Second in clump g First in clump b Second in clump g xmltv-1.4.0/t/data/tv_extractinfo_en_clump_xml_dups_xml.expected000066400000000000000000000020061500074233200252730ustar00rootroot00000000000000 First in clump b Second in clump g A B A D B t t xmltv-1.4.0/t/data/tv_extractinfo_en_clump_xml_empty_xml.expected000066400000000000000000000006251500074233200254630ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_extractinfo_en_dups_xml.expected000066400000000000000000000013051500074233200232140ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_extractinfo_en_dups_xml_amp_xml.expected000066400000000000000000000021231500074233200247300ustar00rootroot00000000000000 A B A D B t t Ampersand Land & & hello && there &amp; everyone < &< <amp; News news xmltv-1.4.0/t/data/tv_extractinfo_en_dups_xml_clump_xml.expected000066400000000000000000000020061500074233200252730ustar00rootroot00000000000000 A B A D B t t First in clump b Second in clump g xmltv-1.4.0/t/data/tv_extractinfo_en_dups_xml_dups_xml.expected000066400000000000000000000024661500074233200251400ustar00rootroot00000000000000 A B A D B t t A B A D B t t xmltv-1.4.0/t/data/tv_extractinfo_en_dups_xml_empty_xml.expected000066400000000000000000000013051500074233200253120ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_extractinfo_en_empty_xml.expected000066400000000000000000000001231500074233200233740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_extractinfo_en_empty_xml_amp_xml.expected000066400000000000000000000007421500074233200251200ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News news xmltv-1.4.0/t/data/tv_extractinfo_en_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200254630ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_extractinfo_en_empty_xml_dups_xml.expected000066400000000000000000000013051500074233200253120ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_extractinfo_en_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200254720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_extractinfo_en_intervals_xml.expected000066400000000000000000000016421500074233200242540ustar00rootroot00000000000000 On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/tv_extractinfo_en_length_xml.expected000066400000000000000000000005171500074233200235260ustar00rootroot00000000000000 News Lots of news. news 30 xmltv-1.4.0/t/data/tv_extractinfo_en_overlap_xml.expected000066400000000000000000000030411500074233200237100ustar00rootroot00000000000000 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle xmltv-1.4.0/t/data/tv_extractinfo_en_simple_xml.expected000066400000000000000000000014261500074233200235360ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_extractinfo_en_simple_xml_x_whatever_xml.expected000066400000000000000000000027301500074233200266510ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_extractinfo_en_sort1_xml.expected000066400000000000000000000011761500074233200233170ustar00rootroot00000000000000 A B C D E F xmltv-1.4.0/t/data/tv_extractinfo_en_sort2_xml.expected000066400000000000000000000005511500074233200233140ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_extractinfo_en_sort_xml.expected000066400000000000000000000020201500074233200232230ustar00rootroot00000000000000 ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport news A B xmltv-1.4.0/t/data/tv_extractinfo_en_test_empty_xml.expected000066400000000000000000000003571500074233200244440ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_extractinfo_en_test_livre_xml.expected000066400000000000000000000003061500074233200244210ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_extractinfo_en_test_remove_some_overlapping_xml.expected000066400000000000000000000030021500074233200302220ustar00rootroot00000000000000 Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 xmltv-1.4.0/t/data/tv_extractinfo_en_test_sort_by_channel_xml.expected000066400000000000000000000005771500074233200264630ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_extractinfo_en_test_xml.expected000066400000000000000000000036401500074233200232240ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_extractinfo_en_test_xml_test_xml.expected000066400000000000000000000067331500074233200251510ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_extractinfo_en_whitespace_xml.expected000066400000000000000000000012351500074233200243770ustar00rootroot00000000000000 T Blah. T Blah. T Blah. T Blah. T Blah. xmltv-1.4.0/t/data/tv_extractinfo_en_x_whatever_xml.expected000066400000000000000000000014261500074233200244210ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_a_all_UTF8.expected000066400000000000000000000301331500074233200211650ustar00rootroot00000000000000 3SAT ARD Das Erste Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g A B A D B t t Attrib Rameau Kilroy BBC News; Weather A news news On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. News Lots of news. 30 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B A B C D E F A B C blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg A programme with empty stuff that should not be written out again Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 A B C T Blah. T Blah. T Blah. T Blah. T Blah. The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_a_amp_xml.expected000066400000000000000000000006721500074233200212510ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_a_amp_xml_amp_xml.expected000066400000000000000000000014401500074233200227600ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_a_amp_xml_clump_xml.expected000066400000000000000000000013731500074233200233300ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_a_amp_xml_dups_xml.expected000066400000000000000000000020531500074233200231570ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News A B A D B t t xmltv-1.4.0/t/data/tv_grep_a_amp_xml_empty_xml.expected000066400000000000000000000006721500074233200233470ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_a_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000013731500074233200275240ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_a_attrs_xml.expected000066400000000000000000000005201500074233200216210ustar00rootroot00000000000000 Attrib Rameau xmltv-1.4.0/t/data/tv_grep_a_clump_extract_1_xml.expected000066400000000000000000000007101500074233200235570ustar00rootroot00000000000000 A news news xmltv-1.4.0/t/data/tv_grep_a_clump_extract_xml.expected000066400000000000000000000006611500074233200233440ustar00rootroot00000000000000 Kilroy BBC News; Weather xmltv-1.4.0/t/data/tv_grep_a_clump_xml.expected000066400000000000000000000006251500074233200216120ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_a_clump_xml_amp_xml.expected000066400000000000000000000013731500074233200233300ustar00rootroot00000000000000 First in clump b Second in clump g Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_a_clump_xml_clump_xml.expected000066400000000000000000000013261500074233200236710ustar00rootroot00000000000000 First in clump b Second in clump g First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_a_clump_xml_dups_xml.expected000066400000000000000000000020061500074233200235200ustar00rootroot00000000000000 First in clump b Second in clump g A B A D B t t xmltv-1.4.0/t/data/tv_grep_a_clump_xml_empty_xml.expected000066400000000000000000000006251500074233200237100ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_a_dups_xml.expected000066400000000000000000000013051500074233200214410ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_a_dups_xml_amp_xml.expected000066400000000000000000000020531500074233200231570ustar00rootroot00000000000000 A B A D B t t Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_a_dups_xml_clump_xml.expected000066400000000000000000000020061500074233200235200ustar00rootroot00000000000000 A B A D B t t First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_a_dups_xml_dups_xml.expected000066400000000000000000000024661500074233200233650ustar00rootroot00000000000000 A B A D B t t A B A D B t t xmltv-1.4.0/t/data/tv_grep_a_dups_xml_empty_xml.expected000066400000000000000000000013051500074233200235370ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_a_empty_xml.expected000066400000000000000000000001231500074233200216210ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_a_empty_xml_amp_xml.expected000066400000000000000000000006721500074233200233470ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_a_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200237100ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_a_empty_xml_dups_xml.expected000066400000000000000000000013051500074233200235370ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_a_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200237170ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_a_intervals_xml.expected000066400000000000000000000016421500074233200225010ustar00rootroot00000000000000 On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/tv_grep_a_length_xml.expected000066400000000000000000000004471500074233200217550ustar00rootroot00000000000000 News Lots of news. 30 xmltv-1.4.0/t/data/tv_grep_a_overlap_xml.expected000066400000000000000000000030411500074233200221350ustar00rootroot00000000000000 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle xmltv-1.4.0/t/data/tv_grep_a_simple_xml.expected000066400000000000000000000014261500074233200217630ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_a_simple_xml_x_whatever_xml.expected000066400000000000000000000027301500074233200250760ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_a_sort1_xml.expected000066400000000000000000000011761500074233200215440ustar00rootroot00000000000000 A B C D E F xmltv-1.4.0/t/data/tv_grep_a_sort2_xml.expected000066400000000000000000000005511500074233200215410ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_grep_a_sort_xml.expected000066400000000000000000000017501500074233200214610ustar00rootroot00000000000000 ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B xmltv-1.4.0/t/data/tv_grep_a_test_empty_xml.expected000066400000000000000000000003571500074233200226710ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_grep_a_test_livre_xml.expected000066400000000000000000000003061500074233200226460ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_a_test_remove_some_overlapping_xml.expected000066400000000000000000000030021500074233200264470ustar00rootroot00000000000000 Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 xmltv-1.4.0/t/data/tv_grep_a_test_sort_by_channel_xml.expected000066400000000000000000000005771500074233200247100ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_grep_a_test_xml.expected000066400000000000000000000036401500074233200214510ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_a_test_xml_test_xml.expected000066400000000000000000000067331500074233200233760ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_a_whitespace_xml.expected000066400000000000000000000012351500074233200226240ustar00rootroot00000000000000 T Blah. T Blah. T Blah. T Blah. T Blah. xmltv-1.4.0/t/data/tv_grep_a_x_whatever_xml.expected000066400000000000000000000014261500074233200226460ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_category_b_all_UTF8.expected000066400000000000000000000007121500074233200230630ustar00rootroot00000000000000 3SAT ARD Das Erste First in clump b xmltv-1.4.0/t/data/tv_grep_category_b_amp_xml.expected000066400000000000000000000001231500074233200231360ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200246530ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_amp_xml_clump_xml.expected000066400000000000000000000003451500074233200252240ustar00rootroot00000000000000 First in clump b xmltv-1.4.0/t/data/tv_grep_category_b_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200250510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200252340ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000003451500074233200314200ustar00rootroot00000000000000 First in clump b xmltv-1.4.0/t/data/tv_grep_category_b_attrs_xml.expected000066400000000000000000000001231500074233200235160ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_clump_extract_1_xml.expected000066400000000000000000000001231500074233200254530ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_clump_extract_xml.expected000066400000000000000000000001231500074233200252330ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_clump_xml.expected000066400000000000000000000003451500074233200235070ustar00rootroot00000000000000 First in clump b xmltv-1.4.0/t/data/tv_grep_category_b_clump_xml_amp_xml.expected000066400000000000000000000003451500074233200252240ustar00rootroot00000000000000 First in clump b xmltv-1.4.0/t/data/tv_grep_category_b_clump_xml_clump_xml.expected000066400000000000000000000006241500074233200255670ustar00rootroot00000000000000 First in clump b First in clump b xmltv-1.4.0/t/data/tv_grep_category_b_clump_xml_dups_xml.expected000066400000000000000000000003451500074233200254220ustar00rootroot00000000000000 First in clump b xmltv-1.4.0/t/data/tv_grep_category_b_clump_xml_empty_xml.expected000066400000000000000000000003451500074233200256050ustar00rootroot00000000000000 First in clump b xmltv-1.4.0/t/data/tv_grep_category_b_dups_xml.expected000066400000000000000000000001231500074233200233340ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200250510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_dups_xml_clump_xml.expected000066400000000000000000000003451500074233200254220ustar00rootroot00000000000000 First in clump b xmltv-1.4.0/t/data/tv_grep_category_b_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200252470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200254320ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_empty_xml.expected000066400000000000000000000001231500074233200235170ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200252340ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_empty_xml_clump_xml.expected000066400000000000000000000003451500074233200256050ustar00rootroot00000000000000 First in clump b xmltv-1.4.0/t/data/tv_grep_category_b_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200254320ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200256150ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_intervals_xml.expected000066400000000000000000000001231500074233200243700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_length_xml.expected000066400000000000000000000001231500074233200236420ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_overlap_xml.expected000066400000000000000000000001231500074233200240310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_simple_xml.expected000066400000000000000000000001231500074233200236520ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_simple_xml_x_whatever_xml.expected000066400000000000000000000001231500074233200267660ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_sort1_xml.expected000066400000000000000000000001231500074233200234310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_sort2_xml.expected000066400000000000000000000001231500074233200234320ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_sort_xml.expected000066400000000000000000000001231500074233200233500ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_test_empty_xml.expected000066400000000000000000000001231500074233200245560ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_test_livre_xml.expected000066400000000000000000000001301500074233200245370ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200303460ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200265710ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_test_xml.expected000066400000000000000000000005451500074233200233500ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_category_b_test_xml_test_xml.expected000066400000000000000000000005451500074233200252670ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_category_b_whitespace_xml.expected000066400000000000000000000001231500074233200245150ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_b_x_whatever_xml.expected000066400000000000000000000001231500074233200245350ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_all_UTF8.expected000066400000000000000000000004711500074233200254200ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_amp_xml.expected000066400000000000000000000001231500074233200254710ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200272060ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_amp_xml_clump_xml.expected000066400000000000000000000001231500074233200275510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200274040ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200275670ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200337450ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_attrs_xml.expected000066400000000000000000000001231500074233200260510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_clump_extract_1_xml.expected000066400000000000000000000001231500074233200300060ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_clump_extract_xml.expected000066400000000000000000000001231500074233200275660ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_clump_xml.expected000066400000000000000000000001231500074233200260340ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_clump_xml_amp_xml.expected000066400000000000000000000001231500074233200275510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_clump_xml_clump_xml.expected000066400000000000000000000001231500074233200301140ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_clump_xml_dups_xml.expected000066400000000000000000000001231500074233200277470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200301320ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_dups_xml.expected000066400000000000000000000001231500074233200256670ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200274040ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_dups_xml_clump_xml.expected000066400000000000000000000001231500074233200277470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200276020ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200277650ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_empty_xml.expected000066400000000000000000000001231500074233200260520ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200275670ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200301320ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200277650ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200301500ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_intervals_xml.expected000066400000000000000000000001231500074233200267230ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_length_xml.expected000066400000000000000000000001231500074233200261750ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_overlap_xml.expected000066400000000000000000000001231500074233200263640ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_simple_xml.expected000066400000000000000000000001231500074233200262050ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_simple_xml_x_whatever_xml.expected000066400000000000000000000001231500074233200313210ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_sort1_xml.expected000066400000000000000000000001231500074233200257640ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_sort2_xml.expected000066400000000000000000000001231500074233200257650ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_sort_xml.expected000066400000000000000000000001231500074233200257030ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_test_empty_xml.expected000066400000000000000000000001231500074233200271110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_test_livre_xml.expected000066400000000000000000000001301500074233200270720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200327010ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200311240ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_test_xml.expected000066400000000000000000000005451500074233200257030ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_test_xml_test_xml.expected000066400000000000000000000005451500074233200276220ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_whitespace_xml.expected000066400000000000000000000001231500074233200270500ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_e_and_title_f_x_whatever_xml.expected000066400000000000000000000001231500074233200270700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_all_UTF8.expected000066400000000000000000000117311500074233200253030ustar00rootroot00000000000000 3SAT ARD Das Erste Second in clump g BBC News; Weather On both 'before' and 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. Zero length, should not overlap with one starting at same time One hour long Zero length in the middle The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg A programme with empty stuff that should not be written out again The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_amp_xml.expected000066400000000000000000000001231500074233200253530ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200270700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_amp_xml_clump_xml.expected000066400000000000000000000003461500074233200274420ustar00rootroot00000000000000 Second in clump g xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200272660ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200274510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000003461500074233200336360ustar00rootroot00000000000000 Second in clump g xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_attrs_xml.expected000066400000000000000000000001231500074233200257330ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_clump_extract_1_xml.expected000066400000000000000000000001231500074233200276700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_clump_extract_xml.expected000066400000000000000000000003711500074233200274550ustar00rootroot00000000000000 BBC News; Weather xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_clump_xml.expected000066400000000000000000000003461500074233200257250ustar00rootroot00000000000000 Second in clump g xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_clump_xml_amp_xml.expected000066400000000000000000000003461500074233200274420ustar00rootroot00000000000000 Second in clump g xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_clump_xml_clump_xml.expected000066400000000000000000000006261500074233200300060ustar00rootroot00000000000000 Second in clump g Second in clump g xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_clump_xml_dups_xml.expected000066400000000000000000000003461500074233200276400ustar00rootroot00000000000000 Second in clump g xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_clump_xml_empty_xml.expected000066400000000000000000000003461500074233200300230ustar00rootroot00000000000000 Second in clump g xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_dups_xml.expected000066400000000000000000000001231500074233200255510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200272660ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_dups_xml_clump_xml.expected000066400000000000000000000003461500074233200276400ustar00rootroot00000000000000 Second in clump g xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200274640ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200276470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_empty_xml.expected000066400000000000000000000001231500074233200257340ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200274510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_empty_xml_clump_xml.expected000066400000000000000000000003461500074233200300230ustar00rootroot00000000000000 Second in clump g xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200276470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200300320ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_intervals_xml.expected000066400000000000000000000012401500074233200266060ustar00rootroot00000000000000 On both 'before' and 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_length_xml.expected000066400000000000000000000001231500074233200260570ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_overlap_xml.expected000066400000000000000000000007661500074233200262630ustar00rootroot00000000000000 Zero length, should not overlap with one starting at same time One hour long Zero length in the middle xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_simple_xml.expected000066400000000000000000000014261500074233200260760ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_simple_xml_x_whatever_xml.expected000066400000000000000000000027301500074233200312110ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_sort1_xml.expected000066400000000000000000000001231500074233200256460ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_sort2_xml.expected000066400000000000000000000001231500074233200256470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_sort_xml.expected000066400000000000000000000005651500074233200255770ustar00rootroot00000000000000 ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_test_empty_xml.expected000066400000000000000000000003571500074233200270040ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_test_livre_xml.expected000066400000000000000000000001301500074233200267540ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200325630ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200310060ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_test_xml.expected000066400000000000000000000036401500074233200255640ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_test_xml_test_xml.expected000066400000000000000000000067331500074233200275110ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_whitespace_xml.expected000066400000000000000000000001231500074233200267320ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_category_g_or_title_h_x_whatever_xml.expected000066400000000000000000000014261500074233200267610ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_all_UTF8.expected000066400000000000000000000016161500074233200254470ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_amp_xml.expected000066400000000000000000000001231500074233200255160ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200272330ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_amp_xml_clump_xml.expected000066400000000000000000000001231500074233200275760ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200274310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200276140ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200337720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_attrs_xml.expected000066400000000000000000000001231500074233200260760ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_clump_extract_1_xml.expected000066400000000000000000000001231500074233200300330ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_clump_extract_xml.expected000066400000000000000000000001231500074233200276130ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_clump_xml.expected000066400000000000000000000001231500074233200260610ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_clump_xml_amp_xml.expected000066400000000000000000000001231500074233200275760ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_clump_xml_clump_xml.expected000066400000000000000000000001231500074233200301410ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_clump_xml_dups_xml.expected000066400000000000000000000001231500074233200277740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200301570ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_dups_xml.expected000066400000000000000000000001231500074233200257140ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200274310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_dups_xml_clump_xml.expected000066400000000000000000000001231500074233200277740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200276270ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200300120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_empty_xml.expected000066400000000000000000000001231500074233200260770ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200276140ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200301570ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200300120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200301750ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_intervals_xml.expected000066400000000000000000000001231500074233200267500ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_length_xml.expected000066400000000000000000000001231500074233200262220ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_overlap_xml.expected000066400000000000000000000001231500074233200264110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_simple_xml.expected000066400000000000000000000007611500074233200262420ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_simple_xml_x_whatever_xml.expected000066400000000000000000000016161500074233200313560ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_sort1_xml.expected000066400000000000000000000001231500074233200260110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_sort2_xml.expected000066400000000000000000000001231500074233200260120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_sort_xml.expected000066400000000000000000000001231500074233200257300ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_test_empty_xml.expected000066400000000000000000000001231500074233200271360ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_test_livre_xml.expected000066400000000000000000000001301500074233200271170ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200327260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200311510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_test_xml.expected000066400000000000000000000001771500074233200257310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_test_xml_test_xml.expected000066400000000000000000000001771500074233200276500ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_whitespace_xml.expected000066400000000000000000000001231500074233200270750ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_channel4_com_x_whatever_xml.expected000066400000000000000000000007611500074233200271250ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_all_UTF8.expected000066400000000000000000000033451500074233200245610ustar00rootroot00000000000000 3SAT blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_amp_xml.expected000066400000000000000000000001231500074233200246270ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200263440ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_amp_xml_clump_xml.expected000066400000000000000000000001231500074233200267070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200265420ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200267250ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200331030ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_attrs_xml.expected000066400000000000000000000001231500074233200252070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_clump_extract_1_xml.expected000066400000000000000000000001231500074233200271440ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_clump_extract_xml.expected000066400000000000000000000001231500074233200267240ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_clump_xml.expected000066400000000000000000000001231500074233200251720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_clump_xml_amp_xml.expected000066400000000000000000000001231500074233200267070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_clump_xml_clump_xml.expected000066400000000000000000000001231500074233200272520ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_clump_xml_dups_xml.expected000066400000000000000000000001231500074233200271050ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200272700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_dups_xml.expected000066400000000000000000000001231500074233200250250ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200265420ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_dups_xml_clump_xml.expected000066400000000000000000000001231500074233200271050ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200267400ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200271230ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_empty_xml.expected000066400000000000000000000001231500074233200252100ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200267250ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200272700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200271230ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200273060ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_intervals_xml.expected000066400000000000000000000001231500074233200260610ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_length_xml.expected000066400000000000000000000001231500074233200253330ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_overlap_xml.expected000066400000000000000000000001231500074233200255220ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_simple_xml.expected000066400000000000000000000001231500074233200253430ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_simple_xml_x_whatever_xml.expected000066400000000000000000000001231500074233200304570ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_sort1_xml.expected000066400000000000000000000001231500074233200251220ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_sort2_xml.expected000066400000000000000000000001231500074233200251230ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_sort_xml.expected000066400000000000000000000001231500074233200250410ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_test_empty_xml.expected000066400000000000000000000001231500074233200262470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_test_livre_xml.expected000066400000000000000000000001301500074233200262300ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200320370ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200302620ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_test_xml.expected000066400000000000000000000034211500074233200250350ustar00rootroot00000000000000 3SAT blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_test_xml_test_xml.expected000066400000000000000000000065141500074233200267620ustar00rootroot00000000000000 3SAT blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_whitespace_xml.expected000066400000000000000000000001231500074233200262060ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_id_exp_sat_x_whatever_xml.expected000066400000000000000000000001231500074233200262260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_all_UTF8.expected000066400000000000000000000001231500074233200236540ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_amp_xml.expected000066400000000000000000000001231500074233200237330ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200254500ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_amp_xml_clump_xml.expected000066400000000000000000000001231500074233200260130ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200256460ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200260310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200322070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_attrs_xml.expected000066400000000000000000000001231500074233200243130ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_clump_extract_1_xml.expected000066400000000000000000000001231500074233200262500ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_clump_extract_xml.expected000066400000000000000000000001231500074233200260300ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_clump_xml.expected000066400000000000000000000001231500074233200242760ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_clump_xml_amp_xml.expected000066400000000000000000000001231500074233200260130ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_clump_xml_clump_xml.expected000066400000000000000000000001231500074233200263560ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_clump_xml_dups_xml.expected000066400000000000000000000001231500074233200262110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200263740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_dups_xml.expected000066400000000000000000000001231500074233200241310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200256460ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_dups_xml_clump_xml.expected000066400000000000000000000001231500074233200262110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200260440ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200262270ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_empty_xml.expected000066400000000000000000000001231500074233200243140ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200260310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200263740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200262270ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200264120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_intervals_xml.expected000066400000000000000000000001231500074233200251650ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_length_xml.expected000066400000000000000000000001231500074233200244370ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_overlap_xml.expected000066400000000000000000000001231500074233200246260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_simple_xml.expected000066400000000000000000000001231500074233200244470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_simple_xml_x_whatever_xml.expected000066400000000000000000000001231500074233200275630ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_sort1_xml.expected000066400000000000000000000001231500074233200242260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_sort2_xml.expected000066400000000000000000000001231500074233200242270ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_sort_xml.expected000066400000000000000000000001231500074233200241450ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_test_empty_xml.expected000066400000000000000000000001231500074233200253530ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_test_livre_xml.expected000066400000000000000000000001301500074233200253340ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200311430ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200273660ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_test_xml.expected000066400000000000000000000001771500074233200241460ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_test_xml_test_xml.expected000066400000000000000000000001771500074233200260650ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_whitespace_xml.expected000066400000000000000000000001231500074233200253120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_name_d_x_whatever_xml.expected000066400000000000000000000001231500074233200253320ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_all_UTF8.expected000066400000000000000000000033071500074233200257630ustar00rootroot00000000000000 3SAT ARD Das Erste Ampersand Land & & hello && there &amp; everyone < &< <amp; Attrib Rameau Kilroy BBC News; Weather The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_amp_xml.expected000066400000000000000000000004771500074233200260470ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_amp_xml_amp_xml.expected000066400000000000000000000010521500074233200275520ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_amp_xml_clump_xml.expected000066400000000000000000000004771500074233200301270ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_amp_xml_dups_xml.expected000066400000000000000000000004771500074233200277620ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_amp_xml_empty_xml.expected000066400000000000000000000004771500074233200301450ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000004771500074233200343230ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_attrs_xml.expected000066400000000000000000000005201500074233200264140ustar00rootroot00000000000000 Attrib Rameau xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_clump_extract_1_xml.expected000066400000000000000000000001231500074233200303500ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_clump_extract_xml.expected000066400000000000000000000006611500074233200301370ustar00rootroot00000000000000 Kilroy BBC News; Weather xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_clump_xml.expected000066400000000000000000000001231500074233200263760ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_clump_xml_amp_xml.expected000066400000000000000000000004771500074233200301270ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_clump_xml_clump_xml.expected000066400000000000000000000001231500074233200304560ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_clump_xml_dups_xml.expected000066400000000000000000000001231500074233200303110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200304740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_dups_xml.expected000066400000000000000000000001231500074233200262310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_dups_xml_amp_xml.expected000066400000000000000000000004771500074233200277620ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_dups_xml_clump_xml.expected000066400000000000000000000001231500074233200303110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200301440ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200303270ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_empty_xml.expected000066400000000000000000000001231500074233200264140ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_empty_xml_amp_xml.expected000066400000000000000000000004771500074233200301450ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200304740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200303270ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200305120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_intervals_xml.expected000066400000000000000000000001231500074233200272650ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_length_xml.expected000066400000000000000000000001231500074233200265370ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_overlap_xml.expected000066400000000000000000000001231500074233200267260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_simple_xml.expected000066400000000000000000000005711500074233200265560ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_simple_xml_x_whatever_xml.expected000066400000000000000000000012361500074233200316710ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_sort1_xml.expected000066400000000000000000000001231500074233200263260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_sort2_xml.expected000066400000000000000000000001231500074233200263270ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_sort_xml.expected000066400000000000000000000001231500074233200262450ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_test_empty_xml.expected000066400000000000000000000001231500074233200274530ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_test_livre_xml.expected000066400000000000000000000003061500074233200274410ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200332430ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200314660ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_test_xml.expected000066400000000000000000000005451500074233200262450ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_test_xml_test_xml.expected000066400000000000000000000005451500074233200301640ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_whitespace_xml.expected000066400000000000000000000001231500074233200274120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_x-whatever_xml.expected000066400000000000000000000005711500074233200273570ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_channel_b_x_whatever_xml.expected000066400000000000000000000005711500074233200274410ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_all_UTF8.expected000066400000000000000000000253151500074233200266460ustar00rootroot00000000000000 3SAT ARD Das Erste News First in clump b Second in clump g A B A D B t t A news news On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. News Lots of news. 30 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B A B C D E F A B C blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg A programme with empty stuff that should not be written out again Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 A B C T Blah. T Blah. T Blah. T Blah. T Blah. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_amp_xml.expected000066400000000000000000000003171500074233200267200ustar00rootroot00000000000000 News xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_amp_xml_amp_xml.expected000066400000000000000000000005121500074233200304320ustar00rootroot00000000000000 News News xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_amp_xml_clump_xml.expected000066400000000000000000000010201500074233200307700ustar00rootroot00000000000000 News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_amp_xml_dups_xml.expected000066400000000000000000000015001500074233200306260ustar00rootroot00000000000000 News A B A D B t t xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_amp_xml_empty_xml.expected000066400000000000000000000003171500074233200310160ustar00rootroot00000000000000 News tv_grep_channel_xyz_or_not_channel_b_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000010201500074233200351050ustar00rootroot00000000000000xmltv-1.4.0/t/data News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_attrs_xml.expected000066400000000000000000000001231500074233200272730ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_clump_extract_1_xml.expected000066400000000000000000000007101500074233200312320ustar00rootroot00000000000000 A news news xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_clump_extract_xml.expected000066400000000000000000000001231500074233200310100ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_clump_xml.expected000066400000000000000000000006251500074233200272650ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_clump_xml_amp_xml.expected000066400000000000000000000010201500074233200307700ustar00rootroot00000000000000 First in clump b Second in clump g News xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_clump_xml_clump_xml.expected000066400000000000000000000013261500074233200313440ustar00rootroot00000000000000 First in clump b Second in clump g First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_clump_xml_dups_xml.expected000066400000000000000000000020061500074233200311730ustar00rootroot00000000000000 First in clump b Second in clump g A B A D B t t xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_clump_xml_empty_xml.expected000066400000000000000000000006251500074233200313630ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_dups_xml.expected000066400000000000000000000013051500074233200271140ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_dups_xml_amp_xml.expected000066400000000000000000000015001500074233200306260ustar00rootroot00000000000000 A B A D B t t News xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_dups_xml_clump_xml.expected000066400000000000000000000020061500074233200311730ustar00rootroot00000000000000 A B A D B t t First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_dups_xml_dups_xml.expected000066400000000000000000000024661500074233200310400ustar00rootroot00000000000000 A B A D B t t A B A D B t t xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_dups_xml_empty_xml.expected000066400000000000000000000013051500074233200312120ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_empty_xml.expected000066400000000000000000000001231500074233200272740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_empty_xml_amp_xml.expected000066400000000000000000000003171500074233200310160ustar00rootroot00000000000000 News xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200313630ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_empty_xml_dups_xml.expected000066400000000000000000000013051500074233200312120ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200313720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_intervals_xml.expected000066400000000000000000000016421500074233200301540ustar00rootroot00000000000000 On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_length_xml.expected000066400000000000000000000004471500074233200274300ustar00rootroot00000000000000 News Lots of news. 30 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_overlap_xml.expected000066400000000000000000000030411500074233200276100ustar00rootroot00000000000000 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_simple_xml.expected000066400000000000000000000007611500074233200274370ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_simple_xml_x_whatever_xml.expected000066400000000000000000000016161500074233200325530ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_sort1_xml.expected000066400000000000000000000011761500074233200272170ustar00rootroot00000000000000 A B C D E F xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_sort2_xml.expected000066400000000000000000000005511500074233200272140ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_sort_xml.expected000066400000000000000000000017501500074233200271340ustar00rootroot00000000000000 ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_test_empty_xml.expected000066400000000000000000000003571500074233200303440ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_test_livre_xml.expected000066400000000000000000000001301500074233200303140ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_test_remove_some_overlapping_xml.expected000066400000000000000000000030021500074233200341220ustar00rootroot00000000000000 Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_test_sort_by_channel_xml.expected000066400000000000000000000005771500074233200323630ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_test_xml.expected000066400000000000000000000036401500074233200271240ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_test_xml_test_xml.expected000066400000000000000000000067331500074233200310510ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_whitespace_xml.expected000066400000000000000000000012351500074233200302770ustar00rootroot00000000000000 T Blah. T Blah. T Blah. T Blah. T Blah. xmltv-1.4.0/t/data/tv_grep_channel_xyz_or_not_channel_b_x_whatever_xml.expected000066400000000000000000000007611500074233200303220ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_all_UTF8.expected000066400000000000000000000077721500074233200245150ustar00rootroot00000000000000 3SAT ARD Das Erste First in clump b Second in clump g Attrib Rameau News Lots of news. 30 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg A programme with empty stuff that should not be written out again King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_amp_xml.expected000066400000000000000000000001231500074233200245530ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200262700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_amp_xml_clump_xml.expected000066400000000000000000000006251500074233200266420ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200264660ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200266510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200330360ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_attrs_xml.expected000066400000000000000000000005201500074233200251340ustar00rootroot00000000000000 Attrib Rameau xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_clump_extract_1_xml.expected000066400000000000000000000001231500074233200270700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_clump_extract_xml.expected000066400000000000000000000001231500074233200266500ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_clump_xml.expected000066400000000000000000000006251500074233200251250ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_clump_xml_amp_xml.expected000066400000000000000000000006251500074233200266420ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_clump_xml_clump_xml.expected000066400000000000000000000013261500074233200272040ustar00rootroot00000000000000 First in clump b Second in clump g First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_clump_xml_dups_xml.expected000066400000000000000000000006251500074233200270400ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_clump_xml_empty_xml.expected000066400000000000000000000006251500074233200272230ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_dups_xml.expected000066400000000000000000000001231500074233200247510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200264660ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_dups_xml_clump_xml.expected000066400000000000000000000006251500074233200270400ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200266640ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200270470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_empty_xml.expected000066400000000000000000000001231500074233200251340ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200266510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200272230ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200270470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200272320ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_intervals_xml.expected000066400000000000000000000001231500074233200260050ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_length_xml.expected000066400000000000000000000004471500074233200252700ustar00rootroot00000000000000 News Lots of news. 30 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_overlap_xml.expected000066400000000000000000000001231500074233200254460ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_simple_xml.expected000066400000000000000000000007611500074233200252770ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_simple_xml_x_whatever_xml.expected000066400000000000000000000016161500074233200304130ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_sort1_xml.expected000066400000000000000000000001231500074233200250460ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_sort2_xml.expected000066400000000000000000000001231500074233200250470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_sort_xml.expected000066400000000000000000000007651500074233200250010ustar00rootroot00000000000000 Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_test_empty_xml.expected000066400000000000000000000003571500074233200262040ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_test_livre_xml.expected000066400000000000000000000001301500074233200261540ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200317630ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200302060ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_test_xml.expected000066400000000000000000000036401500074233200247640ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_test_xml_test_xml.expected000066400000000000000000000067331500074233200267110ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_whitespace_xml.expected000066400000000000000000000001231500074233200261320ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_eval_scalar_keys_5_x_whatever_xml.expected000066400000000000000000000007611500074233200261620ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_all_UTF8.expected000066400000000000000000000021631500074233200251140ustar00rootroot00000000000000 3SAT ARD Das Erste King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_amp_xml.expected000066400000000000000000000001231500074233200251650ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200267020ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_amp_xml_clump_xml.expected000066400000000000000000000001231500074233200272450ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200271000ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200272630ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200334410ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_attrs_xml.expected000066400000000000000000000001231500074233200255450ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_clump_extract_1_xml.expected000066400000000000000000000001231500074233200275020ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_clump_extract_xml.expected000066400000000000000000000001231500074233200272620ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_clump_xml.expected000066400000000000000000000001231500074233200255300ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_clump_xml_amp_xml.expected000066400000000000000000000001231500074233200272450ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_clump_xml_clump_xml.expected000066400000000000000000000001231500074233200276100ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_clump_xml_dups_xml.expected000066400000000000000000000001231500074233200274430ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200276260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_dups_xml.expected000066400000000000000000000001231500074233200253630ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200271000ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_dups_xml_clump_xml.expected000066400000000000000000000001231500074233200274430ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200272760ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200274610ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_empty_xml.expected000066400000000000000000000001231500074233200255460ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200272630ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200276260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200274610ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200276440ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_intervals_xml.expected000066400000000000000000000001231500074233200264170ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_length_xml.expected000066400000000000000000000001231500074233200256710ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_overlap_xml.expected000066400000000000000000000001231500074233200260600ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_simple_xml.expected000066400000000000000000000007611500074233200257110ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_simple_xml_x_whatever_xml.expected000066400000000000000000000016161500074233200310250ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_sort1_xml.expected000066400000000000000000000001231500074233200254600ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_sort2_xml.expected000066400000000000000000000001231500074233200254610ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_sort_xml.expected000066400000000000000000000001231500074233200253770ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_test_empty_xml.expected000066400000000000000000000001231500074233200266050ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_test_livre_xml.expected000066400000000000000000000001301500074233200265660ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200323750ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200306200ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_test_xml.expected000066400000000000000000000005451500074233200253770ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_test_xml_test_xml.expected000066400000000000000000000005451500074233200273160ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_whitespace_xml.expected000066400000000000000000000001231500074233200265440ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_h_x_whatever_xml.expected000066400000000000000000000007611500074233200265740ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_all_UTF8.expected000066400000000000000000000004711500074233200251160ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_amp_xml.expected000066400000000000000000000001231500074233200251670ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200267040ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_amp_xml_clump_xml.expected000066400000000000000000000001231500074233200272470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200271020ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200272650ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200334430ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_attrs_xml.expected000066400000000000000000000001231500074233200255470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_clump_extract_1_xml.expected000066400000000000000000000001231500074233200275040ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_clump_extract_xml.expected000066400000000000000000000001231500074233200272640ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_clump_xml.expected000066400000000000000000000001231500074233200255320ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_clump_xml_amp_xml.expected000066400000000000000000000001231500074233200272470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_clump_xml_clump_xml.expected000066400000000000000000000001231500074233200276120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_clump_xml_dups_xml.expected000066400000000000000000000001231500074233200274450ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200276300ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_dups_xml.expected000066400000000000000000000001231500074233200253650ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200271020ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_dups_xml_clump_xml.expected000066400000000000000000000001231500074233200274450ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200273000ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200274630ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_empty_xml.expected000066400000000000000000000001231500074233200255500ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200272650ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200276300ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200274630ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200276460ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_intervals_xml.expected000066400000000000000000000001231500074233200264210ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_length_xml.expected000066400000000000000000000001231500074233200256730ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_overlap_xml.expected000066400000000000000000000001231500074233200260620ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_simple_xml.expected000066400000000000000000000001231500074233200257030ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_simple_xml_x_whatever_xml.expected000066400000000000000000000001231500074233200310170ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_sort1_xml.expected000066400000000000000000000001231500074233200254620ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_sort2_xml.expected000066400000000000000000000001231500074233200254630ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_sort_xml.expected000066400000000000000000000001231500074233200254010ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_test_empty_xml.expected000066400000000000000000000001231500074233200266070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_test_livre_xml.expected000066400000000000000000000001301500074233200265700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200323770ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200306220ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_test_xml.expected000066400000000000000000000005451500074233200254010ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_test_xml_test_xml.expected000066400000000000000000000005451500074233200273200ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_whitespace_xml.expected000066400000000000000000000001231500074233200265460ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_category_i_title_j_x_whatever_xml.expected000066400000000000000000000001231500074233200265660ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_all_UTF8.expected000066400000000000000000000004711500074233200240250ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_amp_xml.expected000066400000000000000000000001231500074233200240760ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200256130ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_amp_xml_clump_xml.expected000066400000000000000000000001231500074233200261560ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200260110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200261740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200323520ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_attrs_xml.expected000066400000000000000000000001231500074233200244560ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_clump_extract_1_xml.expected000066400000000000000000000001231500074233200264130ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_clump_extract_xml.expected000066400000000000000000000001231500074233200261730ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_clump_xml.expected000066400000000000000000000001231500074233200244410ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_clump_xml_amp_xml.expected000066400000000000000000000001231500074233200261560ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_clump_xml_clump_xml.expected000066400000000000000000000001231500074233200265210ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_clump_xml_dups_xml.expected000066400000000000000000000001231500074233200263540ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200265370ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_dups_xml.expected000066400000000000000000000001231500074233200242740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200260110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_dups_xml_clump_xml.expected000066400000000000000000000001231500074233200263540ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200262070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200263720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_empty_xml.expected000066400000000000000000000001231500074233200244570ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200261740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200265370ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200263720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200265550ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_intervals_xml.expected000066400000000000000000000001231500074233200253300ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_length_xml.expected000066400000000000000000000001231500074233200246020ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_overlap_xml.expected000066400000000000000000000001231500074233200247710ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_simple_xml.expected000066400000000000000000000001231500074233200246120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_simple_xml_x_whatever_xml.expected000066400000000000000000000001231500074233200277260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_sort1_xml.expected000066400000000000000000000001231500074233200243710ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_sort2_xml.expected000066400000000000000000000001231500074233200243720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_sort_xml.expected000066400000000000000000000001231500074233200243100ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_test_empty_xml.expected000066400000000000000000000001231500074233200255160ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_test_livre_xml.expected000066400000000000000000000001301500074233200254770ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200313060ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200275310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_test_xml.expected000066400000000000000000000005451500074233200243100ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_test_xml_test_xml.expected000066400000000000000000000005451500074233200262270ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_whitespace_xml.expected000066400000000000000000000001231500074233200254550ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_i_last_chance_c_x_whatever_xml.expected000066400000000000000000000001231500074233200254750ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_all_UTF8.expected000066400000000000000000000004711500074233200215400ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_new_amp_xml.expected000066400000000000000000000001231500074233200216110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200233260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_amp_xml_clump_xml.expected000066400000000000000000000001231500074233200236710ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200235240ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200237070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200300650ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_attrs_xml.expected000066400000000000000000000001231500074233200221710ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_clump_extract_1_xml.expected000066400000000000000000000001231500074233200241260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_clump_extract_xml.expected000066400000000000000000000001231500074233200237060ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_clump_xml.expected000066400000000000000000000001231500074233200221540ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_clump_xml_amp_xml.expected000066400000000000000000000001231500074233200236710ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_clump_xml_clump_xml.expected000066400000000000000000000001231500074233200242340ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_clump_xml_dups_xml.expected000066400000000000000000000001231500074233200240670ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200242520ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_dups_xml.expected000066400000000000000000000001231500074233200220070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200235240ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_dups_xml_clump_xml.expected000066400000000000000000000001231500074233200240670ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200237220ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200241050ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_empty_xml.expected000066400000000000000000000001231500074233200221720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200237070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200242520ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200241050ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200242700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_intervals_xml.expected000066400000000000000000000001231500074233200230430ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_length_xml.expected000066400000000000000000000001231500074233200223150ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_overlap_xml.expected000066400000000000000000000001231500074233200225040ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_simple_xml.expected000066400000000000000000000001231500074233200223250ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_simple_xml_x_whatever_xml.expected000066400000000000000000000001231500074233200254410ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_sort1_xml.expected000066400000000000000000000001231500074233200221040ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_sort2_xml.expected000066400000000000000000000001231500074233200221050ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_sort_xml.expected000066400000000000000000000001231500074233200220230ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_test_empty_xml.expected000066400000000000000000000001231500074233200232310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_test_livre_xml.expected000066400000000000000000000001301500074233200232120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200270210ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200252440ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_test_xml.expected000066400000000000000000000005451500074233200220230ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_new_test_xml_test_xml.expected000066400000000000000000000005451500074233200237420ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_new_whitespace_xml.expected000066400000000000000000000001231500074233200231700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_new_x_whatever_xml.expected000066400000000000000000000001231500074233200232100ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_all_UTF8.expected000066400000000000000000000264411500074233200263320ustar00rootroot00000000000000 3SAT ARD Das Erste Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g A B A D B t t Attrib Rameau Kilroy BBC News; Weather A news news On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. News Lots of news. 30 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B A B C D E F A B C blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg A programme with empty stuff that should not be written out again Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 A B C T Blah. T Blah. T Blah. T Blah. T Blah. The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_amp_xml.expected000066400000000000000000000006721500074233200264070ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_amp_xml_amp_xml.expected000066400000000000000000000014401500074233200301160ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_amp_xml_clump_xml.expected000066400000000000000000000013731500074233200304660ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_amp_xml_dups_xml.expected000066400000000000000000000020531500074233200303150ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News A B A D B t t xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_amp_xml_empty_xml.expected000066400000000000000000000006721500074233200305050ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News tv_grep_not_channel_id_channel4_com_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000013731500074233200346030ustar00rootroot00000000000000xmltv-1.4.0/t/data Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_attrs_xml.expected000066400000000000000000000005201500074233200267570ustar00rootroot00000000000000 Attrib Rameau xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_clump_extract_1_xml.expected000066400000000000000000000007101500074233200307150ustar00rootroot00000000000000 A news news xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_clump_extract_xml.expected000066400000000000000000000006611500074233200305020ustar00rootroot00000000000000 Kilroy BBC News; Weather xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_clump_xml.expected000066400000000000000000000006251500074233200267500ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_clump_xml_amp_xml.expected000066400000000000000000000013731500074233200304660ustar00rootroot00000000000000 First in clump b Second in clump g Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_clump_xml_clump_xml.expected000066400000000000000000000013261500074233200310270ustar00rootroot00000000000000 First in clump b Second in clump g First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_clump_xml_dups_xml.expected000066400000000000000000000020061500074233200306560ustar00rootroot00000000000000 First in clump b Second in clump g A B A D B t t xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_clump_xml_empty_xml.expected000066400000000000000000000006251500074233200310460ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_dups_xml.expected000066400000000000000000000013051500074233200265770ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_dups_xml_amp_xml.expected000066400000000000000000000020531500074233200303150ustar00rootroot00000000000000 A B A D B t t Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_dups_xml_clump_xml.expected000066400000000000000000000020061500074233200306560ustar00rootroot00000000000000 A B A D B t t First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_dups_xml_dups_xml.expected000066400000000000000000000024661500074233200305230ustar00rootroot00000000000000 A B A D B t t A B A D B t t xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_dups_xml_empty_xml.expected000066400000000000000000000013051500074233200306750ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_empty_xml.expected000066400000000000000000000001231500074233200267570ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_empty_xml_amp_xml.expected000066400000000000000000000006721500074233200305050ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200310460ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_empty_xml_dups_xml.expected000066400000000000000000000013051500074233200306750ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200310550ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_intervals_xml.expected000066400000000000000000000016421500074233200276370ustar00rootroot00000000000000 On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_length_xml.expected000066400000000000000000000004471500074233200271130ustar00rootroot00000000000000 News Lots of news. 30 xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_overlap_xml.expected000066400000000000000000000030411500074233200272730ustar00rootroot00000000000000 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_simple_xml.expected000066400000000000000000000005711500074233200271210ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_simple_xml_x_whatever_xml.expected000066400000000000000000000012361500074233200322340ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_sort1_xml.expected000066400000000000000000000011761500074233200267020ustar00rootroot00000000000000 A B C D E F xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_sort2_xml.expected000066400000000000000000000005511500074233200266770ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_sort_xml.expected000066400000000000000000000017501500074233200266170ustar00rootroot00000000000000 ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_test_empty_xml.expected000066400000000000000000000003571500074233200300270ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_test_livre_xml.expected000066400000000000000000000003061500074233200300040ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_test_remove_some_overlapping_xml.expected000066400000000000000000000030021500074233200336050ustar00rootroot00000000000000 Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_test_sort_by_channel_xml.expected000066400000000000000000000005771500074233200320460ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_test_xml.expected000066400000000000000000000036401500074233200266070ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_test_xml_test_xml.expected000066400000000000000000000067331500074233200305340ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_whitespace_xml.expected000066400000000000000000000012351500074233200277620ustar00rootroot00000000000000 T Blah. T Blah. T Blah. T Blah. T Blah. xmltv-1.4.0/t/data/tv_grep_not_channel_id_channel4_com_x_whatever_xml.expected000066400000000000000000000005711500074233200300040ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_all_UTF8.expected000066400000000000000000000301331500074233200245400ustar00rootroot00000000000000 3SAT ARD Das Erste Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g A B A D B t t Attrib Rameau Kilroy BBC News; Weather A news news On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. News Lots of news. 30 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B A B C D E F A B C blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg A programme with empty stuff that should not be written out again Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 A B C T Blah. T Blah. T Blah. T Blah. T Blah. The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_amp_xml.expected000066400000000000000000000006721500074233200246240ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_amp_xml_amp_xml.expected000066400000000000000000000014401500074233200263330ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_amp_xml_clump_xml.expected000066400000000000000000000013731500074233200267030ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_amp_xml_dups_xml.expected000066400000000000000000000020531500074233200265320ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News A B A D B t t xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_amp_xml_empty_xml.expected000066400000000000000000000006721500074233200267220ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000013731500074233200330770ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_attrs_xml.expected000066400000000000000000000005201500074233200251740ustar00rootroot00000000000000 Attrib Rameau xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_clump_extract_1_xml.expected000066400000000000000000000007101500074233200271320ustar00rootroot00000000000000 A news news xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_clump_extract_xml.expected000066400000000000000000000006611500074233200267170ustar00rootroot00000000000000 Kilroy BBC News; Weather xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_clump_xml.expected000066400000000000000000000006251500074233200251650ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_clump_xml_amp_xml.expected000066400000000000000000000013731500074233200267030ustar00rootroot00000000000000 First in clump b Second in clump g Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_clump_xml_clump_xml.expected000066400000000000000000000013261500074233200272440ustar00rootroot00000000000000 First in clump b Second in clump g First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_clump_xml_dups_xml.expected000066400000000000000000000020061500074233200270730ustar00rootroot00000000000000 First in clump b Second in clump g A B A D B t t xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_clump_xml_empty_xml.expected000066400000000000000000000006251500074233200272630ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_dups_xml.expected000066400000000000000000000013051500074233200250140ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_dups_xml_amp_xml.expected000066400000000000000000000020531500074233200265320ustar00rootroot00000000000000 A B A D B t t Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_dups_xml_clump_xml.expected000066400000000000000000000020061500074233200270730ustar00rootroot00000000000000 A B A D B t t First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_dups_xml_dups_xml.expected000066400000000000000000000024661500074233200267400ustar00rootroot00000000000000 A B A D B t t A B A D B t t xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_dups_xml_empty_xml.expected000066400000000000000000000013051500074233200271120ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_empty_xml.expected000066400000000000000000000001231500074233200251740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_empty_xml_amp_xml.expected000066400000000000000000000006721500074233200267220ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200272630ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_empty_xml_dups_xml.expected000066400000000000000000000013051500074233200271120ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200272720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_intervals_xml.expected000066400000000000000000000016421500074233200260540ustar00rootroot00000000000000 On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_length_xml.expected000066400000000000000000000004471500074233200253300ustar00rootroot00000000000000 News Lots of news. 30 xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_overlap_xml.expected000066400000000000000000000030411500074233200255100ustar00rootroot00000000000000 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_simple_xml.expected000066400000000000000000000014261500074233200253360ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_simple_xml_x_whatever_xml.expected000066400000000000000000000027301500074233200304510ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_sort1_xml.expected000066400000000000000000000011761500074233200251170ustar00rootroot00000000000000 A B C D E F xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_sort2_xml.expected000066400000000000000000000005511500074233200251140ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_sort_xml.expected000066400000000000000000000017501500074233200250340ustar00rootroot00000000000000 ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_test_empty_xml.expected000066400000000000000000000003571500074233200262440ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_test_livre_xml.expected000066400000000000000000000003061500074233200262210ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_test_remove_some_overlapping_xml.expected000066400000000000000000000030021500074233200320220ustar00rootroot00000000000000 Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_test_sort_by_channel_xml.expected000066400000000000000000000005771500074233200302630ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_test_xml.expected000066400000000000000000000036401500074233200250240ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_test_xml_test_xml.expected000066400000000000000000000067331500074233200267510ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_whitespace_xml.expected000066400000000000000000000012351500074233200261770ustar00rootroot00000000000000 T Blah. T Blah. T Blah. T Blah. T Blah. xmltv-1.4.0/t/data/tv_grep_not_channel_name_d_x_whatever_xml.expected000066400000000000000000000014261500074233200262210ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_all_UTF8.expected000066400000000000000000000105531500074233200245310ustar00rootroot00000000000000 3SAT ARD Das Erste News First in clump b Second in clump g t t Kilroy BBC News; Weather On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. News Lots of news. 30 A B A programme with empty stuff that should not be written out again Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 A B C xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_amp_xml.expected000066400000000000000000000003171500074233200246050ustar00rootroot00000000000000 News xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_amp_xml_amp_xml.expected000066400000000000000000000005121500074233200263170ustar00rootroot00000000000000 News News xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_amp_xml_clump_xml.expected000066400000000000000000000010201500074233200266550ustar00rootroot00000000000000 News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_amp_xml_dups_xml.expected000066400000000000000000000006011500074233200265140ustar00rootroot00000000000000 News t t xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_amp_xml_empty_xml.expected000066400000000000000000000003171500074233200267030ustar00rootroot00000000000000 News xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000010201500074233200330510ustar00rootroot00000000000000 News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_attrs_xml.expected000066400000000000000000000001231500074233200251600ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_clump_extract_1_xml.expected000066400000000000000000000001231500074233200271150ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_clump_extract_xml.expected000066400000000000000000000006611500074233200267040ustar00rootroot00000000000000 Kilroy BBC News; Weather xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_clump_xml.expected000066400000000000000000000006251500074233200251520ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_clump_xml_amp_xml.expected000066400000000000000000000010201500074233200266550ustar00rootroot00000000000000 First in clump b Second in clump g News xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_clump_xml_clump_xml.expected000066400000000000000000000013261500074233200272310ustar00rootroot00000000000000 First in clump b Second in clump g First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_clump_xml_dups_xml.expected000066400000000000000000000011071500074233200270610ustar00rootroot00000000000000 First in clump b Second in clump g t t xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_clump_xml_empty_xml.expected000066400000000000000000000006251500074233200272500ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_dups_xml.expected000066400000000000000000000004061500074233200250020ustar00rootroot00000000000000 t t xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_dups_xml_amp_xml.expected000066400000000000000000000006011500074233200265140ustar00rootroot00000000000000 t t News xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_dups_xml_clump_xml.expected000066400000000000000000000011071500074233200270610ustar00rootroot00000000000000 t t First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_dups_xml_dups_xml.expected000066400000000000000000000006701500074233200267200ustar00rootroot00000000000000 t t t t xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_dups_xml_empty_xml.expected000066400000000000000000000004061500074233200271000ustar00rootroot00000000000000 t t xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_empty_xml.expected000066400000000000000000000001231500074233200251610ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_empty_xml_amp_xml.expected000066400000000000000000000003171500074233200267030ustar00rootroot00000000000000 News xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200272500ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_empty_xml_dups_xml.expected000066400000000000000000000004061500074233200271000ustar00rootroot00000000000000 t t xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200272570ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_intervals_xml.expected000066400000000000000000000016421500074233200260410ustar00rootroot00000000000000 On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_length_xml.expected000066400000000000000000000004471500074233200253150ustar00rootroot00000000000000 News Lots of news. 30 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_overlap_xml.expected000066400000000000000000000001231500074233200254730ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_simple_xml.expected000066400000000000000000000001231500074233200253140ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_simple_xml_x_whatever_xml.expected000066400000000000000000000001231500074233200304300ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_sort1_xml.expected000066400000000000000000000001231500074233200250730ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_sort2_xml.expected000066400000000000000000000001231500074233200250740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_sort_xml.expected000066400000000000000000000004461500074233200250220ustar00rootroot00000000000000 A B xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_test_empty_xml.expected000066400000000000000000000003571500074233200262310ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_test_livre_xml.expected000066400000000000000000000001301500074233200262010ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_test_remove_some_overlapping_xml.expected000066400000000000000000000030021500074233200320070ustar00rootroot00000000000000 Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_test_sort_by_channel_xml.expected000066400000000000000000000005771500074233200302500ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_test_xml.expected000066400000000000000000000005451500074233200250120ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_test_xml_test_xml.expected000066400000000000000000000005451500074233200267310ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_whitespace_xml.expected000066400000000000000000000001231500074233200261570ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_2002_02_05_UTC_x_whatever_xml.expected000066400000000000000000000001231500074233200261770ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_all_UTF8.expected000066400000000000000000000045541500074233200245510ustar00rootroot00000000000000 3SAT ARD Das Erste On both 'before' and 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. A B Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_amp_xml.expected000066400000000000000000000001231500074233200246140ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200263310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_amp_xml_clump_xml.expected000066400000000000000000000001231500074233200266740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200265270ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200267120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200330700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_attrs_xml.expected000066400000000000000000000001231500074233200251740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_clump_extract_1_xml.expected000066400000000000000000000001231500074233200271310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_clump_extract_xml.expected000066400000000000000000000001231500074233200267110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_clump_xml.expected000066400000000000000000000001231500074233200251570ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_clump_xml_amp_xml.expected000066400000000000000000000001231500074233200266740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_clump_xml_clump_xml.expected000066400000000000000000000001231500074233200272370ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_clump_xml_dups_xml.expected000066400000000000000000000001231500074233200270720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200272550ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_dups_xml.expected000066400000000000000000000001231500074233200250120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200265270ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_dups_xml_clump_xml.expected000066400000000000000000000001231500074233200270720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200267250ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200271100ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_empty_xml.expected000066400000000000000000000001231500074233200251750ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200267120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200272550ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200271100ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200272730ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_intervals_xml.expected000066400000000000000000000010071500074233200260500ustar00rootroot00000000000000 On both 'before' and 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_length_xml.expected000066400000000000000000000001231500074233200253200ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_overlap_xml.expected000066400000000000000000000001231500074233200255070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_simple_xml.expected000066400000000000000000000001231500074233200253300ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_simple_xml_x_whatever_xml.expected000066400000000000000000000001231500074233200304440ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_sort1_xml.expected000066400000000000000000000001231500074233200251070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_sort2_xml.expected000066400000000000000000000001231500074233200251100ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_sort_xml.expected000066400000000000000000000004461500074233200250360ustar00rootroot00000000000000 A B xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_test_empty_xml.expected000066400000000000000000000001231500074233200262340ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_test_livre_xml.expected000066400000000000000000000001301500074233200262150ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_test_remove_some_overlapping_xml.expected000066400000000000000000000030021500074233200320230ustar00rootroot00000000000000 Container A Contained A 0 Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200302470ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_test_xml.expected000066400000000000000000000005451500074233200250260ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_test_xml_test_xml.expected000066400000000000000000000005451500074233200267450ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_whitespace_xml.expected000066400000000000000000000001231500074233200261730ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_after_200302161330_UTC_x_whatever_xml.expected000066400000000000000000000001231500074233200262130ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_all_UTF8.expected000066400000000000000000000245021500074233200247050ustar00rootroot00000000000000 3SAT ARD Das Erste Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g A B A D B t t Attrib Rameau Kilroy BBC News; Weather A news news On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. News Lots of news. 30 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B C D E F A B C blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg A programme with empty stuff that should not be written out again A B C T Blah. T Blah. T Blah. T Blah. T Blah. The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_amp_xml.expected000066400000000000000000000006721500074233200247660ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_amp_xml_amp_xml.expected000066400000000000000000000014401500074233200264750ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_amp_xml_clump_xml.expected000066400000000000000000000013731500074233200270450ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_amp_xml_dups_xml.expected000066400000000000000000000020531500074233200266740ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News A B A D B t t xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_amp_xml_empty_xml.expected000066400000000000000000000006721500074233200270640ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000013731500074233200332410ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_attrs_xml.expected000066400000000000000000000005201500074233200253360ustar00rootroot00000000000000 Attrib Rameau xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_clump_extract_1_xml.expected000066400000000000000000000007101500074233200272740ustar00rootroot00000000000000 A news news xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_clump_extract_xml.expected000066400000000000000000000006611500074233200270610ustar00rootroot00000000000000 Kilroy BBC News; Weather xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_clump_xml.expected000066400000000000000000000006251500074233200253270ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_clump_xml_amp_xml.expected000066400000000000000000000013731500074233200270450ustar00rootroot00000000000000 First in clump b Second in clump g Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_clump_xml_clump_xml.expected000066400000000000000000000013261500074233200274060ustar00rootroot00000000000000 First in clump b Second in clump g First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_clump_xml_dups_xml.expected000066400000000000000000000020061500074233200272350ustar00rootroot00000000000000 First in clump b Second in clump g A B A D B t t xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_clump_xml_empty_xml.expected000066400000000000000000000006251500074233200274250ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_dups_xml.expected000066400000000000000000000013051500074233200251560ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_dups_xml_amp_xml.expected000066400000000000000000000020531500074233200266740ustar00rootroot00000000000000 A B A D B t t Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_dups_xml_clump_xml.expected000066400000000000000000000020061500074233200272350ustar00rootroot00000000000000 A B A D B t t First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_dups_xml_dups_xml.expected000066400000000000000000000024661500074233200271020ustar00rootroot00000000000000 A B A D B t t A B A D B t t xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_dups_xml_empty_xml.expected000066400000000000000000000013051500074233200272540ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_empty_xml.expected000066400000000000000000000001231500074233200253360ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_empty_xml_amp_xml.expected000066400000000000000000000006721500074233200270640ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200274250ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_empty_xml_dups_xml.expected000066400000000000000000000013051500074233200272540ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200274340ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_intervals_xml.expected000066400000000000000000000014111500074233200262100ustar00rootroot00000000000000 On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_length_xml.expected000066400000000000000000000004471500074233200254720ustar00rootroot00000000000000 News Lots of news. 30 xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_overlap_xml.expected000066400000000000000000000030411500074233200256520ustar00rootroot00000000000000 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_simple_xml.expected000066400000000000000000000014261500074233200255000ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_simple_xml_x_whatever_xml.expected000066400000000000000000000027301500074233200306130ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_sort1_xml.expected000066400000000000000000000011761500074233200252610ustar00rootroot00000000000000 A B C D E F xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_sort2_xml.expected000066400000000000000000000005511500074233200252560ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_sort_xml.expected000066400000000000000000000014261500074233200251760ustar00rootroot00000000000000 ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_test_empty_xml.expected000066400000000000000000000003571500074233200264060ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_test_livre_xml.expected000066400000000000000000000003061500074233200263630ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200321650ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_test_sort_by_channel_xml.expected000066400000000000000000000005771500074233200304250ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_test_xml.expected000066400000000000000000000036401500074233200251660ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_test_xml_test_xml.expected000066400000000000000000000067331500074233200271130ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_whitespace_xml.expected000066400000000000000000000012351500074233200263410ustar00rootroot00000000000000 T Blah. T Blah. T Blah. T Blah. T Blah. xmltv-1.4.0/t/data/tv_grep_on_before_200302161330_UTC_x_whatever_xml.expected000066400000000000000000000014261500074233200263630ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_premiere_all_UTF8.expected000066400000000000000000000004711500074233200225570ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_premiere_amp_xml.expected000066400000000000000000000001231500074233200226300ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200243450ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_amp_xml_clump_xml.expected000066400000000000000000000001231500074233200247100ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200245430ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200247260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200311040ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_attrs_xml.expected000066400000000000000000000001231500074233200232100ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_clump_extract_1_xml.expected000066400000000000000000000001231500074233200251450ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_clump_extract_xml.expected000066400000000000000000000001231500074233200247250ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_clump_xml.expected000066400000000000000000000001231500074233200231730ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_clump_xml_amp_xml.expected000066400000000000000000000001231500074233200247100ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_clump_xml_clump_xml.expected000066400000000000000000000001231500074233200252530ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_clump_xml_dups_xml.expected000066400000000000000000000001231500074233200251060ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200252710ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_dups_xml.expected000066400000000000000000000001231500074233200230260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200245430ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_dups_xml_clump_xml.expected000066400000000000000000000001231500074233200251060ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200247410ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200251240ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_empty_xml.expected000066400000000000000000000001231500074233200232110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200247260ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200252710ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200251240ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200253070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_intervals_xml.expected000066400000000000000000000001231500074233200240620ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_length_xml.expected000066400000000000000000000001231500074233200233340ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_overlap_xml.expected000066400000000000000000000001231500074233200235230ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_simple_xml.expected000066400000000000000000000001231500074233200233440ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_simple_xml_x_whatever_xml.expected000066400000000000000000000001231500074233200264600ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_sort1_xml.expected000066400000000000000000000001231500074233200231230ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_sort2_xml.expected000066400000000000000000000001231500074233200231240ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_sort_xml.expected000066400000000000000000000001231500074233200230420ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_test_empty_xml.expected000066400000000000000000000001231500074233200242500ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_test_livre_xml.expected000066400000000000000000000001301500074233200242310ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200300400ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200262630ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_test_xml.expected000066400000000000000000000005451500074233200230420ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_premiere_test_xml_test_xml.expected000066400000000000000000000005451500074233200247610ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_premiere_whitespace_xml.expected000066400000000000000000000001231500074233200242070ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_premiere_x_whatever_xml.expected000066400000000000000000000001231500074233200242270ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_all_UTF8.expected000066400000000000000000000021631500074233200244060ustar00rootroot00000000000000 3SAT ARD Das Erste King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_previously_shown_amp_xml.expected000066400000000000000000000001231500074233200244570ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_amp_xml_amp_xml.expected000066400000000000000000000001231500074233200261740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_amp_xml_clump_xml.expected000066400000000000000000000001231500074233200265370ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_amp_xml_dups_xml.expected000066400000000000000000000001231500074233200263720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_amp_xml_empty_xml.expected000066400000000000000000000001231500074233200265550ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200327330ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_attrs_xml.expected000066400000000000000000000001231500074233200250370ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_clump_extract_1_xml.expected000066400000000000000000000001231500074233200267740ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_clump_extract_xml.expected000066400000000000000000000001231500074233200265540ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_clump_xml.expected000066400000000000000000000001231500074233200250220ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_clump_xml_amp_xml.expected000066400000000000000000000001231500074233200265370ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_clump_xml_clump_xml.expected000066400000000000000000000001231500074233200271020ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_clump_xml_dups_xml.expected000066400000000000000000000001231500074233200267350ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200271200ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_dups_xml.expected000066400000000000000000000001231500074233200246550ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_dups_xml_amp_xml.expected000066400000000000000000000001231500074233200263720ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_dups_xml_clump_xml.expected000066400000000000000000000001231500074233200267350ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_dups_xml_dups_xml.expected000066400000000000000000000001231500074233200265700ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_dups_xml_empty_xml.expected000066400000000000000000000001231500074233200267530ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_empty_xml.expected000066400000000000000000000001231500074233200250400ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_empty_xml_amp_xml.expected000066400000000000000000000001231500074233200265550ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200271200ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_empty_xml_dups_xml.expected000066400000000000000000000001231500074233200267530ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200271360ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_intervals_xml.expected000066400000000000000000000001231500074233200257110ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_length_xml.expected000066400000000000000000000001231500074233200251630ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_overlap_xml.expected000066400000000000000000000001231500074233200253520ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_simple_xml.expected000066400000000000000000000007611500074233200252030ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_previously_shown_simple_xml_x_whatever_xml.expected000066400000000000000000000016161500074233200303170ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_grep_previously_shown_sort1_xml.expected000066400000000000000000000001231500074233200247520ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_sort2_xml.expected000066400000000000000000000001231500074233200247530ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_sort_xml.expected000066400000000000000000000001231500074233200246710ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_test_empty_xml.expected000066400000000000000000000001231500074233200260770ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_test_livre_xml.expected000066400000000000000000000001301500074233200260600ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_test_remove_some_overlapping_xml.expected000066400000000000000000000001231500074233200316670ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_test_sort_by_channel_xml.expected000066400000000000000000000001231500074233200301120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_test_xml.expected000066400000000000000000000005451500074233200246710ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_previously_shown_test_xml_test_xml.expected000066400000000000000000000005451500074233200266100ustar00rootroot00000000000000 3SAT ARD Das Erste xmltv-1.4.0/t/data/tv_grep_previously_shown_whitespace_xml.expected000066400000000000000000000001231500074233200260360ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_grep_previously_shown_x_whatever_xml.expected000066400000000000000000000007611500074233200260660ustar00rootroot00000000000000 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_remove_some_overlapping_all_UTF8.expected000066400000000000000000000275611500074233200246710ustar00rootroot00000000000000 3SAT ARD Das Erste Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g A B A D B t t Attrib Rameau Kilroy BBC News; Weather A news news On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. News Lots of news. 30 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B A B C D E F A B C blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg A programme with empty stuff that should not be written out again Contained A 0 Contained A 1 Contained A 2 Contained B 0 Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 A B C T Blah. T Blah. T Blah. T Blah. T Blah. The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_remove_some_overlapping_amp_xml.expected000066400000000000000000000006721500074233200247420ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_remove_some_overlapping_amp_xml_amp_xml.expected000066400000000000000000000014401500074233200264510ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_remove_some_overlapping_amp_xml_clump_xml.expected000066400000000000000000000013731500074233200270210ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_remove_some_overlapping_amp_xml_dups_xml.expected000066400000000000000000000020531500074233200266500ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News A B A D B t t xmltv-1.4.0/t/data/tv_remove_some_overlapping_amp_xml_empty_xml.expected000066400000000000000000000006721500074233200270400ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_remove_some_overlapping_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000013731500074233200332150ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News First in clump b Second in clump g xmltv-1.4.0/t/data/tv_remove_some_overlapping_attrs_xml.expected000066400000000000000000000005201500074233200253120ustar00rootroot00000000000000 Attrib Rameau xmltv-1.4.0/t/data/tv_remove_some_overlapping_clump_extract_1_xml.expected000066400000000000000000000007101500074233200272500ustar00rootroot00000000000000 A news news xmltv-1.4.0/t/data/tv_remove_some_overlapping_clump_extract_xml.expected000066400000000000000000000006611500074233200270350ustar00rootroot00000000000000 Kilroy BBC News; Weather xmltv-1.4.0/t/data/tv_remove_some_overlapping_clump_xml.expected000066400000000000000000000006251500074233200253030ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_remove_some_overlapping_clump_xml_amp_xml.expected000066400000000000000000000013731500074233200270210ustar00rootroot00000000000000 First in clump b Second in clump g Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_remove_some_overlapping_clump_xml_clump_xml.expected000066400000000000000000000013261500074233200273620ustar00rootroot00000000000000 First in clump b Second in clump g First in clump b Second in clump g xmltv-1.4.0/t/data/tv_remove_some_overlapping_clump_xml_dups_xml.expected000066400000000000000000000020061500074233200272110ustar00rootroot00000000000000 First in clump b Second in clump g A B A D B t t xmltv-1.4.0/t/data/tv_remove_some_overlapping_clump_xml_empty_xml.expected000066400000000000000000000006251500074233200274010ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_remove_some_overlapping_dups_xml.expected000066400000000000000000000013051500074233200251320ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_remove_some_overlapping_dups_xml_amp_xml.expected000066400000000000000000000020531500074233200266500ustar00rootroot00000000000000 A B A D B t t Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_remove_some_overlapping_dups_xml_clump_xml.expected000066400000000000000000000020061500074233200272110ustar00rootroot00000000000000 A B A D B t t First in clump b Second in clump g xmltv-1.4.0/t/data/tv_remove_some_overlapping_dups_xml_dups_xml.expected000066400000000000000000000024661500074233200270560ustar00rootroot00000000000000 A B A D B t t A B A D B t t xmltv-1.4.0/t/data/tv_remove_some_overlapping_dups_xml_empty_xml.expected000066400000000000000000000013051500074233200272300ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_remove_some_overlapping_empty_xml.expected000066400000000000000000000001231500074233200253120ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_remove_some_overlapping_empty_xml_amp_xml.expected000066400000000000000000000006721500074233200270400ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_remove_some_overlapping_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200274010ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_remove_some_overlapping_empty_xml_dups_xml.expected000066400000000000000000000013051500074233200272300ustar00rootroot00000000000000 A B A D B t t xmltv-1.4.0/t/data/tv_remove_some_overlapping_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200274100ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_remove_some_overlapping_intervals_xml.expected000066400000000000000000000016421500074233200261720ustar00rootroot00000000000000 On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/tv_remove_some_overlapping_length_xml.expected000066400000000000000000000004471500074233200254460ustar00rootroot00000000000000 News Lots of news. 30 xmltv-1.4.0/t/data/tv_remove_some_overlapping_overlap_xml.expected000066400000000000000000000030411500074233200256260ustar00rootroot00000000000000 A B C D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle xmltv-1.4.0/t/data/tv_remove_some_overlapping_simple_xml.expected000066400000000000000000000014261500074233200254540ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_remove_some_overlapping_simple_xml_x_whatever_xml.expected000066400000000000000000000027301500074233200305670ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_remove_some_overlapping_sort1_xml.expected000066400000000000000000000011761500074233200252350ustar00rootroot00000000000000 A B C D E F xmltv-1.4.0/t/data/tv_remove_some_overlapping_sort2_xml.expected000066400000000000000000000005511500074233200252320ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_remove_some_overlapping_sort_xml.expected000066400000000000000000000017501500074233200251520ustar00rootroot00000000000000 ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B xmltv-1.4.0/t/data/tv_remove_some_overlapping_test_empty_xml.expected000066400000000000000000000003571500074233200263620ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_remove_some_overlapping_test_livre_xml.expected000066400000000000000000000003061500074233200263370ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_remove_some_overlapping_test_remove_some_overlapping_xml.expected000066400000000000000000000024301500074233200321440ustar00rootroot00000000000000 Contained A 0 Contained A 1 Contained A 2 Contained B 0 Contained B 1 Overlap 0 Overlap 1 Container C This is a description. Contained C 0 Contained C 1 xmltv-1.4.0/t/data/tv_remove_some_overlapping_test_sort_by_channel_xml.expected000066400000000000000000000005771500074233200304010ustar00rootroot00000000000000 A B C xmltv-1.4.0/t/data/tv_remove_some_overlapping_test_xml.expected000066400000000000000000000036401500074233200251420ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_remove_some_overlapping_test_xml_test_xml.expected000066400000000000000000000067331500074233200270670ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_remove_some_overlapping_whitespace_xml.expected000066400000000000000000000012351500074233200263150ustar00rootroot00000000000000 T Blah. T Blah. T Blah. T Blah. T Blah. xmltv-1.4.0/t/data/tv_remove_some_overlapping_x_whatever_xml.expected000066400000000000000000000014261500074233200263370ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_sort_all_UTF8.expected000066400000000000000000000257611500074233200207320ustar00rootroot00000000000000 3SAT ARD Das Erste A C B Zero length, should not overlap with one starting at same time D E F One hour long A Zero length in the middle B C blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg A news news T Blah. Ampersand Land & & hello && there &amp; everyone < &< <amp; The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. Attrib Rameau King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A C B D E E1 F B G H I J K D News Lots of news. 30 A programme with empty stuff that should not be written out again First in clump b Second in clump g t Kilroy BBC News; Weather News A C B On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. A B Contained A 0 Container A Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Contained C 0 Container C This is a description. Contained C 1 xmltv-1.4.0/t/data/tv_sort_amp_xml.expected000066400000000000000000000006721500074233200210030ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_sort_amp_xml_amp_xml.expected000066400000000000000000000006721500074233200225200ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_sort_amp_xml_clump_xml.expected000066400000000000000000000013731500074233200230620ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; First in clump b Second in clump g News xmltv-1.4.0/t/data/tv_sort_amp_xml_dups_xml.expected000066400000000000000000000014761500074233200227210ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; A B D t News xmltv-1.4.0/t/data/tv_sort_amp_xml_empty_xml.expected000066400000000000000000000006721500074233200231010ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_sort_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000013731500074233200272560ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; First in clump b Second in clump g News xmltv-1.4.0/t/data/tv_sort_attrs_xml.expected000066400000000000000000000005201500074233200213530ustar00rootroot00000000000000 Attrib Rameau xmltv-1.4.0/t/data/tv_sort_by_channel_all_UTF8.expected000066400000000000000000000257611500074233200231140ustar00rootroot00000000000000 3SAT ARD Das Erste A C B On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. D E F A B C A C B D E E1 F B D A C B G H A B Contained A 0 Container A Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Contained C 0 Container C This is a description. Contained C 1 blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg I J K News Lots of news. 30 News Zero length, should not overlap with one starting at same time One hour long Zero length in the middle Ampersand Land & & hello && there &amp; everyone < &< <amp; The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. Attrib Rameau A news news t ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport First in clump b Second in clump g A programme with empty stuff that should not be written out again T Blah. Kilroy BBC News; Weather xmltv-1.4.0/t/data/tv_sort_by_channel_amp_xml.expected000066400000000000000000000006721500074233200231650ustar00rootroot00000000000000 News Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_sort_by_channel_amp_xml_amp_xml.expected000066400000000000000000000006721500074233200247020ustar00rootroot00000000000000 News Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_sort_by_channel_amp_xml_clump_xml.expected000066400000000000000000000013731500074233200252440ustar00rootroot00000000000000 News Ampersand Land & & hello && there &amp; everyone < &< <amp; First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_by_channel_amp_xml_dups_xml.expected000066400000000000000000000014761500074233200251030ustar00rootroot00000000000000 A B D News Ampersand Land & & hello && there &amp; everyone < &< <amp; t xmltv-1.4.0/t/data/tv_sort_by_channel_amp_xml_empty_xml.expected000066400000000000000000000006721500074233200252630ustar00rootroot00000000000000 News Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_sort_by_channel_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000013731500074233200314400ustar00rootroot00000000000000 News Ampersand Land & & hello && there &amp; everyone < &< <amp; First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_by_channel_attrs_xml.expected000066400000000000000000000005201500074233200235350ustar00rootroot00000000000000 Attrib Rameau xmltv-1.4.0/t/data/tv_sort_by_channel_clump_extract_1_xml.expected000066400000000000000000000007101500074233200254730ustar00rootroot00000000000000 A news news xmltv-1.4.0/t/data/tv_sort_by_channel_clump_extract_xml.expected000066400000000000000000000006611500074233200252600ustar00rootroot00000000000000 Kilroy BBC News; Weather xmltv-1.4.0/t/data/tv_sort_by_channel_clump_xml.expected000066400000000000000000000006251500074233200235260ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_by_channel_clump_xml_amp_xml.expected000066400000000000000000000013731500074233200252440ustar00rootroot00000000000000 News Ampersand Land & & hello && there &amp; everyone < &< <amp; First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_by_channel_clump_xml_clump_xml.expected000066400000000000000000000006251500074233200256060ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_by_channel_clump_xml_dups_xml.expected000066400000000000000000000014311500074233200254350ustar00rootroot00000000000000 A B D t First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_by_channel_clump_xml_empty_xml.expected000066400000000000000000000006251500074233200256240ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_by_channel_dups_xml.expected000066400000000000000000000007301500074233200233560ustar00rootroot00000000000000 A B D t xmltv-1.4.0/t/data/tv_sort_by_channel_dups_xml_amp_xml.expected000066400000000000000000000014761500074233200251030ustar00rootroot00000000000000 A B D News Ampersand Land & & hello && there &amp; everyone < &< <amp; t xmltv-1.4.0/t/data/tv_sort_by_channel_dups_xml_clump_xml.expected000066400000000000000000000014311500074233200254350ustar00rootroot00000000000000 A B D t First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_by_channel_dups_xml_dups_xml.expected000066400000000000000000000007301500074233200252710ustar00rootroot00000000000000 A B D t xmltv-1.4.0/t/data/tv_sort_by_channel_dups_xml_empty_xml.expected000066400000000000000000000007301500074233200254540ustar00rootroot00000000000000 A B D t xmltv-1.4.0/t/data/tv_sort_by_channel_empty_xml.expected000066400000000000000000000001231500074233200235350ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_sort_by_channel_empty_xml_amp_xml.expected000066400000000000000000000006721500074233200252630ustar00rootroot00000000000000 News Ampersand Land & & hello && there &amp; everyone < &< <amp; xmltv-1.4.0/t/data/tv_sort_by_channel_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200256240ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_by_channel_empty_xml_dups_xml.expected000066400000000000000000000007301500074233200254540ustar00rootroot00000000000000 A B D t xmltv-1.4.0/t/data/tv_sort_by_channel_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200256330ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_sort_by_channel_intervals_xml.expected000066400000000000000000000016701500074233200244160ustar00rootroot00000000000000 On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/tv_sort_by_channel_length_xml.expected000066400000000000000000000004471500074233200236710ustar00rootroot00000000000000 News Lots of news. 30 xmltv-1.4.0/t/data/tv_sort_by_channel_overlap_xml.expected000066400000000000000000000031151500074233200240530ustar00rootroot00000000000000 A C B D E E1 F G H I J K Zero length, should not overlap with one starting at same time One hour long Zero length in the middle xmltv-1.4.0/t/data/tv_sort_by_channel_simple_xml.expected000066400000000000000000000014261500074233200236770ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_sort_by_channel_simple_xml_x_whatever_xml.expected000066400000000000000000000014261500074233200270130ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_sort_by_channel_sort1_xml.expected000066400000000000000000000012521500074233200234530ustar00rootroot00000000000000 D E F A B C xmltv-1.4.0/t/data/tv_sort_by_channel_sort2_xml.expected000066400000000000000000000006251500074233200234570ustar00rootroot00000000000000 A C B xmltv-1.4.0/t/data/tv_sort_by_channel_sort_xml.expected000066400000000000000000000020041500074233200233660ustar00rootroot00000000000000 A B ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport xmltv-1.4.0/t/data/tv_sort_by_channel_test_empty_xml.expected000066400000000000000000000003571500074233200246050ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_sort_by_channel_test_livre_xml.expected000066400000000000000000000003061500074233200245620ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_sort_by_channel_test_remove_some_overlapping_xml.expected000066400000000000000000000030021500074233200303630ustar00rootroot00000000000000 Contained A 0 Container A Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Contained C 0 Container C This is a description. Contained C 1 xmltv-1.4.0/t/data/tv_sort_by_channel_test_sort_by_channel.expected000066400000000000000000000006251500074233200257360ustar00rootroot00000000000000 C A B xmltv-1.4.0/t/data/tv_sort_by_channel_test_sort_by_channel_xml.expected000066400000000000000000000006251500074233200266160ustar00rootroot00000000000000 A C B xmltv-1.4.0/t/data/tv_sort_by_channel_test_xml.expected000066400000000000000000000036401500074233200233650ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_sort_by_channel_test_xml_test_xml.expected000066400000000000000000000036401500074233200253040ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_sort_by_channel_whitespace_xml.expected000066400000000000000000000003111500074233200245320ustar00rootroot00000000000000 T Blah. xmltv-1.4.0/t/data/tv_sort_by_channel_x_whatever_xml.expected000066400000000000000000000014261500074233200245620ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_sort_clump_extract_1_xml.expected000066400000000000000000000007101500074233200233110ustar00rootroot00000000000000 A news news xmltv-1.4.0/t/data/tv_sort_clump_extract_xml.expected000066400000000000000000000006611500074233200230760ustar00rootroot00000000000000 Kilroy BBC News; Weather xmltv-1.4.0/t/data/tv_sort_clump_xml.expected000066400000000000000000000006251500074233200213440ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_clump_xml_amp_xml.expected000066400000000000000000000013731500074233200230620ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; First in clump b Second in clump g News xmltv-1.4.0/t/data/tv_sort_clump_xml_clump_xml.expected000066400000000000000000000006251500074233200234240ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_clump_xml_dups_xml.expected000066400000000000000000000014311500074233200232530ustar00rootroot00000000000000 A B D First in clump b Second in clump g t xmltv-1.4.0/t/data/tv_sort_clump_xml_empty_xml.expected000066400000000000000000000006251500074233200234420ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_dups_xml.expected000066400000000000000000000007301500074233200211740ustar00rootroot00000000000000 A B D t xmltv-1.4.0/t/data/tv_sort_dups_xml_amp_xml.expected000066400000000000000000000014761500074233200227210ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; A B D t News xmltv-1.4.0/t/data/tv_sort_dups_xml_clump_xml.expected000066400000000000000000000014311500074233200232530ustar00rootroot00000000000000 A B D First in clump b Second in clump g t xmltv-1.4.0/t/data/tv_sort_dups_xml_dups_xml.expected000066400000000000000000000007301500074233200231070ustar00rootroot00000000000000 A B D t xmltv-1.4.0/t/data/tv_sort_dups_xml_empty_xml.expected000066400000000000000000000007301500074233200232720ustar00rootroot00000000000000 A B D t xmltv-1.4.0/t/data/tv_sort_empty_xml.expected000066400000000000000000000001231500074233200213530ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_sort_empty_xml_amp_xml.expected000066400000000000000000000006721500074233200231010ustar00rootroot00000000000000 Ampersand Land & & hello && there &amp; everyone < &< <amp; News xmltv-1.4.0/t/data/tv_sort_empty_xml_clump_xml.expected000066400000000000000000000006251500074233200234420ustar00rootroot00000000000000 First in clump b Second in clump g xmltv-1.4.0/t/data/tv_sort_empty_xml_dups_xml.expected000066400000000000000000000007301500074233200232720ustar00rootroot00000000000000 A B D t xmltv-1.4.0/t/data/tv_sort_empty_xml_empty_xml.expected000066400000000000000000000001231500074233200234510ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_sort_intervals_xml.expected000066400000000000000000000016701500074233200222340ustar00rootroot00000000000000 On 'before' but not 'after' 13:30. Straightforward 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. On 'before' but not 'after' 13:30. On both 'before' and 'after' 13:30. Straightforward 'after' but not 'before' 13:30. xmltv-1.4.0/t/data/tv_sort_length_xml.expected000066400000000000000000000004471500074233200215070ustar00rootroot00000000000000 News Lots of news. 30 xmltv-1.4.0/t/data/tv_sort_overlap_xml.expected000066400000000000000000000031151500074233200216710ustar00rootroot00000000000000 Zero length, should not overlap with one starting at same time One hour long Zero length in the middle A C B D E E1 F G H I J K xmltv-1.4.0/t/data/tv_sort_overlap_xml.expected_err000066400000000000000000000007701500074233200225450ustar00rootroot00000000000000overlapping programmes on channel 1: A at 20011228113100-|20011228113101 and C at 20011228113100-|20011228113101 overlapping programmes on channel 2: G at 20011228113101-|20011229000000 and H at 20011228113101-|20011229000000 overlapping programmes on channel 5: J at 20011228113101-|20011229000000 and K at 20011228113101-|20011229000000 overlapping programmes on channel a: One hour long at 20000101000000-|20000101010000 and Zero length in the middle at 20000101003000-|20000101003000 xmltv-1.4.0/t/data/tv_sort_simple_xml.expected000066400000000000000000000014261500074233200215150ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_sort_simple_xml_x_whatever_xml.expected000066400000000000000000000014261500074233200246310ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_sort_sort1_xml.expected000066400000000000000000000012521500074233200212710ustar00rootroot00000000000000 D E F A B C xmltv-1.4.0/t/data/tv_sort_sort2_xml.expected000066400000000000000000000006251500074233200212750ustar00rootroot00000000000000 A C B xmltv-1.4.0/t/data/tv_sort_sort_xml.expected000066400000000000000000000020041500074233200212040ustar00rootroot00000000000000 ITV Nightscreen Behind the scenes of ITV programmes, a guide to films being shown on the small screen, plus recipes and facts factual Motorsport Mundial High-speed footage, news and views from around the world, focusing on everything from touring car races to single-seater events sport A B xmltv-1.4.0/t/data/tv_sort_test_empty_xml.expected000066400000000000000000000003571500074233200224230ustar00rootroot00000000000000 A programme with empty stuff that should not be written out again xmltv-1.4.0/t/data/tv_sort_test_livre_xml.expected000066400000000000000000000003061500074233200224000ustar00rootroot00000000000000 xmltv-1.4.0/t/data/tv_sort_test_remove_some_overlapping_xml.expected000066400000000000000000000030021500074233200262010ustar00rootroot00000000000000 Contained A 0 Container A Contained A 1 Contained A 2 Contained B 0 Container B Contained B 1 Overlap 0 Overlap 1 Contained C 0 Container C This is a description. Contained C 1 xmltv-1.4.0/t/data/tv_sort_test_sort_by_channel_xml.expected000066400000000000000000000006251500074233200244340ustar00rootroot00000000000000 A C B xmltv-1.4.0/t/data/tv_sort_test_xml.expected000066400000000000000000000036401500074233200212030ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_sort_test_xml_test_xml.expected000066400000000000000000000036401500074233200231220ustar00rootroot00000000000000 3SAT ARD Das Erste blah blah Blah Blah Blah. blah a b c d e https://www.example.com/xxx.jpg https://www.themoviedb.org/person/204 19901011 Comedy 1 https://www.example.com/title/0365/ https://www.example.com/title/tt0365/ ES 2 . 9 . 0/1 English PG 3/3 More blah blah https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg https://www.example.com/xxxx.jpg xmltv-1.4.0/t/data/tv_sort_whitespace_xml.expected000066400000000000000000000003111500074233200223500ustar00rootroot00000000000000 T Blah. xmltv-1.4.0/t/data/tv_sort_x_whatever_xml.expected000066400000000000000000000014261500074233200224000ustar00rootroot00000000000000 The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/data/tv_to_latex_all_UTF8.expected000066400000000000000000000170301500074233200215500ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright \& } & bbc2.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 10-10 (Thursday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 +0100 & 09:55 +0100 & { \small \raggedright News } & TA \\ \end{tabular} \\ \section*{\sf 04-20 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 17:25 UTC & 18:00 UTC & { \small \raggedright First in clump } & foo \\ \smallskip 17:25 & 18:00 & { \small \raggedright Second in clump } & foo \\ \end{tabular} \\ \section*{\sf 12-28 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright B } & 1 \\ \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright D } & 1 \\ \smallskip 11:31 & 11:31 & { \small \raggedright B } & 1 \\ \end{tabular} \\ \section*{\sf 05-17 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 02:02 BST & & { \small \raggedright t } & c \\ \smallskip 02:02 & & { \small \raggedright t } & c \\ \end{tabular} \\ \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & 00:30 & { \small \raggedright Attrib Rameau } & bbc2.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 10-04 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 & 10:00 & { \small \raggedright Kilroy } & south-east.bbc1.bbc.co.uk \\ \smallskip 09:00 & 10:00 & { \small \raggedright BBC News; Weather } & south-east.bbc1.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 01-01 (Monday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 23:59 UTC & 23:59 UTC & { \small \raggedright A } & c \\ \smallskip 23:59 & 23:59 & { \small \raggedright news } & c \\ \smallskip 23:59 & 23:59 & { \small \raggedright news } & c \\ \end{tabular} \\ \section*{\sf 02-16 (Sunday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 13:00 & & { \small \raggedright On 'before' but not 'after' 13:30. } & 0 \\ \smallskip 13:00 & 14:00 & { \small \raggedright On both 'before' and 'after' 13:30. } & 0 \\ \smallskip 13:30 & 13:30 & { \small \raggedright On 'before' but not 'after' 13:30. } & 0 \\ \smallskip 13:00 & 13:10 & { \small \raggedright Straightforward 'before' but not 'after' 13:30. } & 0 \\ \smallskip 13:30 & 14:00 & { \small \raggedright On both 'before' and 'after' 13:30. } & 0 \\ \smallskip 13:40 & 14:00 & { \small \raggedright Straightforward 'after' but not 'before' 13:30. } & 0 \\ \end{tabular} \\ \section*{\sf 02-24 (Sunday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:00 +0100 & 00:30 +0100 & { \small \raggedright News } & CNN \\ \end{tabular} \\ \section*{\sf 12-28 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 11:31 UTC & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright B } & 1 \\ \smallskip 11:31 & & { \small \raggedright C } & 1 \\ \smallskip 11:31 & & { \small \raggedright D } & 1 \\ \smallskip 11:31 & & { \small \raggedright E } & 1 \\ \smallskip 11:31 & & { \small \raggedright F } & 1 \\ \smallskip 11:31 & 00:00 & { \small \raggedright G } & 2 \\ \smallskip 11:31 & 00:00 & { \small \raggedright H } & 2 \\ \smallskip 11:31 & 00:00 & { \small \raggedright I } & 4 \\ \smallskip 11:31 & 00:00 & { \small \raggedright J } & 5 \\ \smallskip 11:31 & 00:00 & { \small \raggedright K } & 5 \\ \end{tabular} \\ \section*{\sf 01-01 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:00 & 00:00 & { \small \raggedright Zero length, should not overlap with one starting at same time } & a \\ \smallskip 00:00 & 01:00 & { \small \raggedright One hour long } & a \\ \smallskip 00:30 & 00:30 & { \small \raggedright Zero length in the middle } & a \\ \end{tabular} \\ \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 BST & & { \small \raggedright The Phil Silvers Show } & bbc2.bbc.co.uk \\ \smallskip 09:55 & & { \small \raggedright King of the Hill // Meet the Propaniacs } & channel4.com \\ \end{tabular} \\ \section*{\sf 12-18 (Tuesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 04:05 UTC & & { \small \raggedright ITV Nightscreen } & carlton.com \\ \smallskip 04:05 & 04:30 & { \small \raggedright Motorsport Mundial } & channel5.co.uk \\ \end{tabular} \\ \section*{\sf 05-12 (Monday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 23:35 -0400 & & { \small \raggedright A } & 2 \\ \end{tabular} \\ \section*{\sf 05-13 (Tuesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:06 & 01:06 & { \small \raggedright B } & 2 \\ \end{tabular} \\ \section*{\sf 01-01 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:00 UTC & 02:00 UTC & { \small \raggedright A } & 1 \\ \smallskip 03:00 & & { \small \raggedright B } & 1 \\ \smallskip 03:00 & & { \small \raggedright C } & 1 \\ \smallskip 00:00 & & { \small \raggedright D } & 1 \\ \smallskip 00:00 & & { \small \raggedright E } & 1 \\ \smallskip 00:00 & 01:00 & { \small \raggedright F } & 1 \\ \end{tabular} \\ \section*{\sf 01-01 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:00 & & { \small \raggedright A } & 2 \\ \smallskip 00:00 & 01:00 & { \small \raggedright B } & 2 \\ \smallskip 00:00 & & { \small \raggedright C } & 2 \\ \end{tabular} \\ \section*{\sf 06-03 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 16:33 & & { \small \raggedright blah } & 3SAT \\ \end{tabular} \\ \section*{\sf 04-20 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 13:10 & & { \small \raggedright A programme with empty stuff that should not be written out again } & foo.com \\ \end{tabular} \\ \section*{\sf 10-25 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 07:00 & 07:45 & { \small \raggedright Container A } & 3 \\ \smallskip 07:00 & 07:05 & { \small \raggedright Contained A 0 } & 3 \\ \smallskip 07:05 & 07:15 & { \small \raggedright Contained A 1 } & 3 \\ \smallskip 07:15 & 07:45 & { \small \raggedright Contained A 2 } & 3 \\ \smallskip 08:00 & 08:05 & { \small \raggedright Contained B 0 } & 3 \\ \smallskip 08:00 & 08:15 & { \small \raggedright Container B } & 3 \\ \smallskip 08:05 & 08:15 & { \small \raggedright Contained B 1 } & 3 \\ \smallskip 09:00 & 10:00 & { \small \raggedright Overlap 0 } & 3 \\ \smallskip 09:00 & 10:00 & { \small \raggedright Overlap 1 } & 3 \\ \smallskip 10:00 & 10:15 & { \small \raggedright Container C } & 3 \\ \smallskip 10:00 & 10:05 & { \small \raggedright Contained C 0 } & 3 \\ \smallskip 10:05 & 10:15 & { \small \raggedright Contained C 1 } & 3 \\ \end{tabular} \\ \section*{\sf 02-16 (Sunday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:00 & & { \small \raggedright A } & 0 \\ \smallskip 00:30 & 00:30 & { \small \raggedright B } & 0 \\ \smallskip 00:00 & 00:10 & { \small \raggedright C } & 0 \\ \end{tabular} \\ \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 BST & & { \small \raggedright T } & foo.tv \\ \smallskip 00:05 & & { \small \raggedright T } & foo.tv \\ \smallskip 00:05 & & { \small \raggedright T } & foo.tv \\ \smallskip 00:05 & & { \small \raggedright T } & foo.tv \\ \smallskip 00:05 & & { \small \raggedright T } & foo.tv \\ \smallskip 00:05 & & { \small \raggedright The Phil Silvers Show } & bbc2.bbc.co.uk \\ \smallskip 09:55 & & { \small \raggedright King of the Hill // Meet the Propaniacs } & channel4.com \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_amp_xml.expected000066400000000000000000000007211500074233200216260ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright \& } & bbc2.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 10-10 (Thursday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 +0100 & 09:55 +0100 & { \small \raggedright News } & TA \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_amp_xml_amp_xml.expected000066400000000000000000000014411500074233200233430ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright \& } & bbc2.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 10-10 (Thursday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 +0100 & 09:55 +0100 & { \small \raggedright News } & TA \\ \end{tabular} \\ \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 BST & & { \small \raggedright \& } & bbc2.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 10-10 (Thursday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 +0100 & 09:55 +0100 & { \small \raggedright News } & TA \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_amp_xml_clump_xml.expected000066400000000000000000000013151500074233200237060ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright \& } & bbc2.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 10-10 (Thursday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 +0100 & 09:55 +0100 & { \small \raggedright News } & TA \\ \end{tabular} \\ \section*{\sf 04-20 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 17:25 UTC & 18:00 UTC & { \small \raggedright First in clump } & foo \\ \smallskip 17:25 & 18:00 & { \small \raggedright Second in clump } & foo \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_amp_xml_dups_xml.expected000066400000000000000000000020251500074233200235400ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright \& } & bbc2.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 10-10 (Thursday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 +0100 & 09:55 +0100 & { \small \raggedright News } & TA \\ \end{tabular} \\ \section*{\sf 12-28 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 11:31 UTC & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright B } & 1 \\ \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright D } & 1 \\ \smallskip 11:31 & 11:31 & { \small \raggedright B } & 1 \\ \end{tabular} \\ \section*{\sf 05-17 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 02:02 BST & & { \small \raggedright t } & c \\ \smallskip 02:02 & & { \small \raggedright t } & c \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_amp_xml_empty_xml.expected000066400000000000000000000007211500074233200237240ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright \& } & bbc2.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 10-10 (Thursday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 +0100 & 09:55 +0100 & { \small \raggedright News } & TA \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000013151500074233200301020ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright \& } & bbc2.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 10-10 (Thursday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 +0100 & 09:55 +0100 & { \small \raggedright News } & TA \\ \end{tabular} \\ \section*{\sf 04-20 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 17:25 UTC & 18:00 UTC & { \small \raggedright First in clump } & foo \\ \smallskip 17:25 & 18:00 & { \small \raggedright Second in clump } & foo \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_attrs_xml.expected000066400000000000000000000004701500074233200222070ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & 00:30 & { \small \raggedright Attrib Rameau } & bbc2.bbc.co.uk \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_clump_extract_1_xml.expected000066400000000000000000000006321500074233200241440ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 01-01 (Monday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 23:59 & 23:59 & { \small \raggedright A } & c \\ \smallskip 23:59 & 23:59 & { \small \raggedright news } & c \\ \smallskip 23:59 & 23:59 & { \small \raggedright news } & c \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_clump_extract_xml.expected000066400000000000000000000006351500074233200237270ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 10-04 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 & 10:00 & { \small \raggedright Kilroy } & south-east.bbc1.bbc.co.uk \\ \smallskip 09:00 & 10:00 & { \small \raggedright BBC News; Weather } & south-east.bbc1.bbc.co.uk \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_clump_xml.expected000066400000000000000000000005711500074233200221740ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 04-20 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 17:25 & 18:00 & { \small \raggedright First in clump } & foo \\ \smallskip 17:25 & 18:00 & { \small \raggedright Second in clump } & foo \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_clump_xml_amp_xml.expected000066400000000000000000000013111500074233200237020ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 04-20 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 17:25 & 18:00 & { \small \raggedright First in clump } & foo \\ \smallskip 17:25 & 18:00 & { \small \raggedright Second in clump } & foo \\ \end{tabular} \\ \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 BST & & { \small \raggedright \& } & bbc2.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 10-10 (Thursday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 +0100 & 09:55 +0100 & { \small \raggedright News } & TA \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_clump_xml_clump_xml.expected000066400000000000000000000010201500074233200242420ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 04-20 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 17:25 & 18:00 & { \small \raggedright First in clump } & foo \\ \smallskip 17:25 & 18:00 & { \small \raggedright Second in clump } & foo \\ \smallskip 17:25 & 18:00 & { \small \raggedright First in clump } & foo \\ \smallskip 17:25 & 18:00 & { \small \raggedright Second in clump } & foo \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_clump_xml_dups_xml.expected000066400000000000000000000016711500074233200241110ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 04-20 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 17:25 & 18:00 & { \small \raggedright First in clump } & foo \\ \smallskip 17:25 & 18:00 & { \small \raggedright Second in clump } & foo \\ \end{tabular} \\ \section*{\sf 12-28 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright B } & 1 \\ \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright D } & 1 \\ \smallskip 11:31 & 11:31 & { \small \raggedright B } & 1 \\ \end{tabular} \\ \section*{\sf 05-17 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 02:02 BST & & { \small \raggedright t } & c \\ \smallskip 02:02 & & { \small \raggedright t } & c \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_clump_xml_empty_xml.expected000066400000000000000000000005711500074233200242720ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 04-20 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 17:25 & 18:00 & { \small \raggedright First in clump } & foo \\ \smallskip 17:25 & 18:00 & { \small \raggedright Second in clump } & foo \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_dups_xml.expected000066400000000000000000000013051500074233200220230ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 12-28 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright B } & 1 \\ \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright D } & 1 \\ \smallskip 11:31 & 11:31 & { \small \raggedright B } & 1 \\ \end{tabular} \\ \section*{\sf 05-17 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 02:02 BST & & { \small \raggedright t } & c \\ \smallskip 02:02 & & { \small \raggedright t } & c \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_dups_xml_amp_xml.expected000066400000000000000000000020211500074233200235340ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 12-28 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright B } & 1 \\ \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright D } & 1 \\ \smallskip 11:31 & 11:31 & { \small \raggedright B } & 1 \\ \end{tabular} \\ \section*{\sf 05-17 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 02:02 BST & & { \small \raggedright t } & c \\ \smallskip 02:02 & & { \small \raggedright t } & c \\ \end{tabular} \\ \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright \& } & bbc2.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 10-10 (Thursday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 +0100 & 09:55 +0100 & { \small \raggedright News } & TA \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_dups_xml_clump_xml.expected000066400000000000000000000017011500074233200241030ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 12-28 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright B } & 1 \\ \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright D } & 1 \\ \smallskip 11:31 & 11:31 & { \small \raggedright B } & 1 \\ \end{tabular} \\ \section*{\sf 05-17 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 02:02 BST & & { \small \raggedright t } & c \\ \smallskip 02:02 & & { \small \raggedright t } & c \\ \end{tabular} \\ \section*{\sf 04-20 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 17:25 UTC & 18:00 UTC & { \small \raggedright First in clump } & foo \\ \smallskip 17:25 & 18:00 & { \small \raggedright Second in clump } & foo \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_dups_xml_dups_xml.expected000066400000000000000000000024111500074233200237350ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 12-28 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright B } & 1 \\ \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright D } & 1 \\ \smallskip 11:31 & 11:31 & { \small \raggedright B } & 1 \\ \end{tabular} \\ \section*{\sf 05-17 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 02:02 BST & & { \small \raggedright t } & c \\ \smallskip 02:02 & & { \small \raggedright t } & c \\ \end{tabular} \\ \section*{\sf 12-28 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 11:31 UTC & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright B } & 1 \\ \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright D } & 1 \\ \smallskip 11:31 & 11:31 & { \small \raggedright B } & 1 \\ \end{tabular} \\ \section*{\sf 05-17 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 02:02 BST & & { \small \raggedright t } & c \\ \smallskip 02:02 & & { \small \raggedright t } & c \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_dups_xml_empty_xml.expected000066400000000000000000000013051500074233200241210ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 12-28 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright B } & 1 \\ \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright D } & 1 \\ \smallskip 11:31 & 11:31 & { \small \raggedright B } & 1 \\ \end{tabular} \\ \section*{\sf 05-17 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 02:02 BST & & { \small \raggedright t } & c \\ \smallskip 02:02 & & { \small \raggedright t } & c \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_empty_xml.expected000066400000000000000000000002051500074233200222040ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_empty_xml_amp_xml.expected000066400000000000000000000007211500074233200237240ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright \& } & bbc2.bbc.co.uk \\ \end{tabular} \\ \section*{\sf 10-10 (Thursday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 09:00 +0100 & 09:55 +0100 & { \small \raggedright News } & TA \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_empty_xml_clump_xml.expected000066400000000000000000000005711500074233200242720ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 04-20 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 17:25 & 18:00 & { \small \raggedright First in clump } & foo \\ \smallskip 17:25 & 18:00 & { \small \raggedright Second in clump } & foo \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_empty_xml_dups_xml.expected000066400000000000000000000013051500074233200241210ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 12-28 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright B } & 1 \\ \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright D } & 1 \\ \smallskip 11:31 & 11:31 & { \small \raggedright B } & 1 \\ \end{tabular} \\ \section*{\sf 05-17 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 02:02 BST & & { \small \raggedright t } & c \\ \smallskip 02:02 & & { \small \raggedright t } & c \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_empty_xml_empty_xml.expected000066400000000000000000000002051500074233200243020ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_intervals_xml.expected000066400000000000000000000014451500074233200230640ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 02-16 (Sunday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 13:00 & & { \small \raggedright On 'before' but not 'after' 13:30. } & 0 \\ \smallskip 13:00 & 14:00 & { \small \raggedright On both 'before' and 'after' 13:30. } & 0 \\ \smallskip 13:30 & 13:30 & { \small \raggedright On 'before' but not 'after' 13:30. } & 0 \\ \smallskip 13:00 & 13:10 & { \small \raggedright Straightforward 'before' but not 'after' 13:30. } & 0 \\ \smallskip 13:30 & 14:00 & { \small \raggedright On both 'before' and 'after' 13:30. } & 0 \\ \smallskip 13:40 & 14:00 & { \small \raggedright Straightforward 'after' but not 'before' 13:30. } & 0 \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_length_xml.expected000066400000000000000000000004411500074233200223310ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 02-24 (Sunday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:00 & 00:30 & { \small \raggedright News } & CNN \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_overlap_xml.expected000066400000000000000000000023101500074233200225150ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 12-28 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 11:31 & & { \small \raggedright A } & 1 \\ \smallskip 11:31 & & { \small \raggedright B } & 1 \\ \smallskip 11:31 & & { \small \raggedright C } & 1 \\ \smallskip 11:31 & & { \small \raggedright D } & 1 \\ \smallskip 11:31 & & { \small \raggedright E } & 1 \\ \smallskip 11:31 & & { \small \raggedright F } & 1 \\ \smallskip 11:31 & 00:00 & { \small \raggedright G } & 2 \\ \smallskip 11:31 & 00:00 & { \small \raggedright H } & 2 \\ \smallskip 11:31 & 00:00 & { \small \raggedright I } & 4 \\ \smallskip 11:31 & 00:00 & { \small \raggedright J } & 5 \\ \smallskip 11:31 & 00:00 & { \small \raggedright K } & 5 \\ \end{tabular} \\ \section*{\sf 01-01 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:00 & 00:00 & { \small \raggedright Zero length, should not overlap with one starting at same time } & a \\ \smallskip 00:00 & 01:00 & { \small \raggedright One hour long } & a \\ \smallskip 00:30 & 00:30 & { \small \raggedright Zero length in the middle } & a \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_simple_xml.expected000066400000000000000000000006431500074233200223450ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright The Phil Silvers Show } & bbc2.bbc.co.uk \\ \smallskip 09:55 & & { \small \raggedright King of the Hill // Meet the Propaniacs } & channel4.com \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_simple_xml_x_whatever_xml.expected000066400000000000000000000011431500074233200254550ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright The Phil Silvers Show } & bbc2.bbc.co.uk \\ \smallskip 09:55 & & { \small \raggedright King of the Hill // Meet the Propaniacs } & channel4.com \\ \smallskip 00:05 & & { \small \raggedright The Phil Silvers Show } & bbc2.bbc.co.uk \\ \smallskip 09:55 & & { \small \raggedright King of the Hill // Meet the Propaniacs } & channel4.com \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_sort1_xml.expected000066400000000000000000000010661500074233200221240ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 01-01 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:00 & 02:00 & { \small \raggedright A } & 1 \\ \smallskip 03:00 & & { \small \raggedright B } & 1 \\ \smallskip 03:00 & & { \small \raggedright C } & 1 \\ \smallskip 00:00 & & { \small \raggedright D } & 1 \\ \smallskip 00:00 & & { \small \raggedright E } & 1 \\ \smallskip 00:00 & 01:00 & { \small \raggedright F } & 1 \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_sort2_xml.expected000066400000000000000000000006121500074233200221210ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 01-01 (Friday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:00 & & { \small \raggedright A } & 2 \\ \smallskip 00:00 & 01:00 & { \small \raggedright B } & 2 \\ \smallskip 00:00 & & { \small \raggedright C } & 2 \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_sort_xml.expected000066400000000000000000000012721500074233200220420ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 12-18 (Tuesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 04:05 & & { \small \raggedright ITV Nightscreen } & carlton.com \\ \smallskip 04:05 & 04:30 & { \small \raggedright Motorsport Mundial } & channel5.co.uk \\ \end{tabular} \\ \section*{\sf 05-12 (Monday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 23:35 -0400 & & { \small \raggedright A } & 2 \\ \end{tabular} \\ \section*{\sf 05-13 (Tuesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:06 & 01:06 & { \small \raggedright B } & 2 \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_test_empty_xml.expected000066400000000000000000000005371500074233200232530ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 04-20 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 13:10 & & { \small \raggedright A programme with empty stuff that should not be written out again } & foo.com \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_test_livre_xml.expected000066400000000000000000000004601500074233200232310ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 12-11 (Tuesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 18:00 & & { \small \raggedright } & south-east.bbc1.bbc.co.uk \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_test_remove_some_overlapping_xml.expected000066400000000000000000000020641500074233200270400ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 10-25 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 07:00 & 07:45 & { \small \raggedright Container A } & 3 \\ \smallskip 07:00 & 07:05 & { \small \raggedright Contained A 0 } & 3 \\ \smallskip 07:05 & 07:15 & { \small \raggedright Contained A 1 } & 3 \\ \smallskip 07:15 & 07:45 & { \small \raggedright Contained A 2 } & 3 \\ \smallskip 08:00 & 08:05 & { \small \raggedright Contained B 0 } & 3 \\ \smallskip 08:00 & 08:15 & { \small \raggedright Container B } & 3 \\ \smallskip 08:05 & 08:15 & { \small \raggedright Contained B 1 } & 3 \\ \smallskip 09:00 & 10:00 & { \small \raggedright Overlap 0 } & 3 \\ \smallskip 09:00 & 10:00 & { \small \raggedright Overlap 1 } & 3 \\ \smallskip 10:00 & 10:15 & { \small \raggedright Container C } & 3 \\ \smallskip 10:00 & 10:05 & { \small \raggedright Contained C 0 } & 3 \\ \smallskip 10:05 & 10:15 & { \small \raggedright Contained C 1 } & 3 \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_test_sort_by_channel_xml.expected000066400000000000000000000006171500074233200252650ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 02-16 (Sunday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:00 & & { \small \raggedright A } & 0 \\ \smallskip 00:30 & 00:30 & { \small \raggedright B } & 0 \\ \smallskip 00:00 & 00:10 & { \small \raggedright C } & 0 \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_test_xml.expected000066400000000000000000000005141500074233200220300ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 06-03 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 16:33 & & { \small \raggedright blah } & 3SAT \\ \end{tabular} \\ \end{flushleft} Generated by \textbf{my listings generator}. \end{document} xmltv-1.4.0/t/data/tv_to_latex_test_xml_test_xml.expected000066400000000000000000000006111500074233200237450ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 06-03 (Saturday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 16:33 & & { \small \raggedright blah } & 3SAT \\ \smallskip 16:33 & & { \small \raggedright blah } & 3SAT \\ \end{tabular} \\ \end{flushleft} Generated by \textbf{my listings generator}. \end{document} xmltv-1.4.0/t/data/tv_to_latex_whitespace_xml.expected000066400000000000000000000010171500074233200232040ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright T } & foo.tv \\ \smallskip 00:05 & & { \small \raggedright T } & foo.tv \\ \smallskip 00:05 & & { \small \raggedright T } & foo.tv \\ \smallskip 00:05 & & { \small \raggedright T } & foo.tv \\ \smallskip 00:05 & & { \small \raggedright T } & foo.tv \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_latex_x_whatever_xml.expected000066400000000000000000000006431500074233200232300ustar00rootroot00000000000000\documentclass[a4paper]{article} \usepackage[latin1]{inputenc} \begin{document} \sf \begin{flushleft} \section*{\sf 08-29 (Wednesday)} \begin{tabular}{r@{--}lp{0.7\textwidth}r} \smallskip 00:05 & & { \small \raggedright The Phil Silvers Show } & bbc2.bbc.co.uk \\ \smallskip 09:55 & & { \small \raggedright King of the Hill // Meet the Propaniacs } & channel4.com \\ \end{tabular} \\ \end{flushleft} \end{document} xmltv-1.4.0/t/data/tv_to_text_all_UTF8.expected000066400000000000000000000047741500074233200214320ustar00rootroot0000000000000008-29 (Wednesday) 00:05-- & bbc2.bbc.co.uk 10-10 (Thursday) 09:00 +0100--09:55 +0100 News TA 04-20 (Saturday) 17:25 UTC--18:00 UTC First in clump foo 17:25--18:00 Second in clump foo 12-28 (Friday) 11:31-- A 1 11:31-- B 1 11:31-- A 1 11:31-- D 1 11:31--11:31 B 1 05-17 (Friday) 02:02 BST-- t c 02:02-- t c 08-29 (Wednesday) 00:05--00:30 Attrib Rameau bbc2.bbc.co.uk 10-04 (Friday) 09:00--10:00 Kilroy south-east.bbc1.bbc.co.uk 09:00--10:00 BBC News; Weather south-east.bbc1.bbc.co.uk 01-01 (Monday) 23:59 UTC--23:59 UTC A c 23:59--23:59 news c 23:59--23:59 news c 02-16 (Sunday) 13:00-- On 'before' but not 'after' 13:30. 0 13:00--14:00 On both 'before' and 'after' 13:30. 0 13:30--13:30 On 'before' but not 'after' 13:30. 0 13:00--13:10 Straightforward 'before' but not 'after' 13:30. 0 13:30--14:00 On both 'before' and 'after' 13:30. 0 13:40--14:00 Straightforward 'after' but not 'before' 13:30. 0 02-24 (Sunday) 00:00 +0100--00:30 +0100 News CNN 12-28 (Friday) 11:31 UTC-- A 1 11:31-- B 1 11:31-- C 1 11:31-- D 1 11:31-- E 1 11:31-- F 1 11:31--00:00 G 2 11:31--00:00 H 2 11:31--00:00 I 4 11:31--00:00 J 5 11:31--00:00 K 5 01-01 (Saturday) 00:00--00:00 Zero length, should not overlap with one starting at same time a 00:00--01:00 One hour long a 00:30--00:30 Zero length in the middle a 08-29 (Wednesday) 00:05 BST-- The Phil Silvers Show bbc2.bbc.co.uk 09:55-- King of the Hill // Meet the Propaniacs channel4.com 12-18 (Tuesday) 04:05 UTC-- ITV Nightscreen carlton.com 04:05--04:30 Motorsport Mundial channel5.co.uk 05-12 (Monday) 23:35 -0400-- A 2 05-13 (Tuesday) 00:06--01:06 B 2 01-01 (Saturday) 00:00 UTC--02:00 UTC A 1 03:00-- B 1 03:00-- C 1 00:00-- D 1 00:00-- E 1 00:00--01:00 F 1 01-01 (Friday) 00:00-- A 2 00:00--01:00 B 2 00:00-- C 2 06-03 (Saturday) 16:33-- blah 3SAT 04-20 (Saturday) 13:10-- A programme with empty stuff that should not be written out again foo.com 10-25 (Saturday) 07:00--07:45 Container A 3 07:00--07:05 Contained A 0 3 07:05--07:15 Contained A 1 3 07:15--07:45 Contained A 2 3 08:00--08:05 Contained B 0 3 08:00--08:15 Container B 3 08:05--08:15 Contained B 1 3 09:00--10:00 Overlap 0 3 09:00--10:00 Overlap 1 3 10:00--10:15 Container C 3 10:00--10:05 Contained C 0 3 10:05--10:15 Contained C 1 3 02-16 (Sunday) 00:00-- A 0 00:30--00:30 B 0 00:00--00:10 C 0 08-29 (Wednesday) 00:05 BST-- T foo.tv 00:05-- T foo.tv 00:05-- T foo.tv 00:05-- T foo.tv 00:05-- T foo.tv 00:05-- The Phil Silvers Show bbc2.bbc.co.uk 09:55-- King of the Hill // Meet the Propaniacs channel4.com xmltv-1.4.0/t/data/tv_to_text_amp_xml.expected000066400000000000000000000001401500074233200214700ustar00rootroot0000000000000008-29 (Wednesday) 00:05-- & bbc2.bbc.co.uk 10-10 (Thursday) 09:00 +0100--09:55 +0100 News TA xmltv-1.4.0/t/data/tv_to_text_amp_xml_amp_xml.expected000066400000000000000000000003051500074233200232100ustar00rootroot0000000000000008-29 (Wednesday) 00:05-- & bbc2.bbc.co.uk 10-10 (Thursday) 09:00 +0100--09:55 +0100 News TA 08-29 (Wednesday) 00:05 BST-- & bbc2.bbc.co.uk 10-10 (Thursday) 09:00 +0100--09:55 +0100 News TA xmltv-1.4.0/t/data/tv_to_text_amp_xml_clump_xml.expected000066400000000000000000000002741500074233200235600ustar00rootroot0000000000000008-29 (Wednesday) 00:05-- & bbc2.bbc.co.uk 10-10 (Thursday) 09:00 +0100--09:55 +0100 News TA 04-20 (Saturday) 17:25 UTC--18:00 UTC First in clump foo 17:25--18:00 Second in clump foo xmltv-1.4.0/t/data/tv_to_text_amp_xml_dups_xml.expected000066400000000000000000000003431500074233200234100ustar00rootroot0000000000000008-29 (Wednesday) 00:05-- & bbc2.bbc.co.uk 10-10 (Thursday) 09:00 +0100--09:55 +0100 News TA 12-28 (Friday) 11:31 UTC-- A 1 11:31-- B 1 11:31-- A 1 11:31-- D 1 11:31--11:31 B 1 05-17 (Friday) 02:02 BST-- t c 02:02-- t c xmltv-1.4.0/t/data/tv_to_text_amp_xml_empty_xml.expected000066400000000000000000000001401500074233200235660ustar00rootroot0000000000000008-29 (Wednesday) 00:05-- & bbc2.bbc.co.uk 10-10 (Thursday) 09:00 +0100--09:55 +0100 News TA xmltv-1.4.0/t/data/tv_to_text_amp_xml_empty_xml_empty_xml_clump_xml.expected000066400000000000000000000002741500074233200277540ustar00rootroot0000000000000008-29 (Wednesday) 00:05-- & bbc2.bbc.co.uk 10-10 (Thursday) 09:00 +0100--09:55 +0100 News TA 04-20 (Saturday) 17:25 UTC--18:00 UTC First in clump foo 17:25--18:00 Second in clump foo xmltv-1.4.0/t/data/tv_to_text_attrs_xml.expected000066400000000000000000000000751500074233200220570ustar00rootroot0000000000000008-29 (Wednesday) 00:05--00:30 Attrib Rameau bbc2.bbc.co.uk xmltv-1.4.0/t/data/tv_to_text_clump_extract_1_xml.expected000066400000000000000000000001111500074233200240030ustar00rootroot0000000000000001-01 (Monday) 23:59--23:59 A c 23:59--23:59 news c 23:59--23:59 news c xmltv-1.4.0/t/data/tv_to_text_clump_extract_xml.expected000066400000000000000000000001671500074233200235760ustar00rootroot0000000000000010-04 (Friday) 09:00--10:00 Kilroy south-east.bbc1.bbc.co.uk 09:00--10:00 BBC News; Weather south-east.bbc1.bbc.co.uk xmltv-1.4.0/t/data/tv_to_text_clump_xml.expected000066400000000000000000000001231500074233200220340ustar00rootroot0000000000000004-20 (Saturday) 17:25--18:00 First in clump foo 17:25--18:00 Second in clump foo xmltv-1.4.0/t/data/tv_to_text_clump_xml_amp_xml.expected000066400000000000000000000002701500074233200235540ustar00rootroot0000000000000004-20 (Saturday) 17:25--18:00 First in clump foo 17:25--18:00 Second in clump foo 08-29 (Wednesday) 00:05 BST-- & bbc2.bbc.co.uk 10-10 (Thursday) 09:00 +0100--09:55 +0100 News TA xmltv-1.4.0/t/data/tv_to_text_clump_xml_clump_xml.expected000066400000000000000000000002241500074233200241160ustar00rootroot0000000000000004-20 (Saturday) 17:25--18:00 First in clump foo 17:25--18:00 Second in clump foo 17:25--18:00 First in clump foo 17:25--18:00 Second in clump foo xmltv-1.4.0/t/data/tv_to_text_clump_xml_dups_xml.expected000066400000000000000000000003221500074233200237500ustar00rootroot0000000000000004-20 (Saturday) 17:25--18:00 First in clump foo 17:25--18:00 Second in clump foo 12-28 (Friday) 11:31-- A 1 11:31-- B 1 11:31-- A 1 11:31-- D 1 11:31--11:31 B 1 05-17 (Friday) 02:02 BST-- t c 02:02-- t c xmltv-1.4.0/t/data/tv_to_text_clump_xml_empty_xml.expected000066400000000000000000000001231500074233200241320ustar00rootroot0000000000000004-20 (Saturday) 17:25--18:00 First in clump foo 17:25--18:00 Second in clump foo xmltv-1.4.0/t/data/tv_to_text_dups_xml.expected000066400000000000000000000001761500074233200216770ustar00rootroot0000000000000012-28 (Friday) 11:31-- A 1 11:31-- B 1 11:31-- A 1 11:31-- D 1 11:31--11:31 B 1 05-17 (Friday) 02:02 BST-- t c 02:02-- t c xmltv-1.4.0/t/data/tv_to_text_dups_xml_amp_xml.expected000066400000000000000000000003371500074233200234130ustar00rootroot0000000000000012-28 (Friday) 11:31-- A 1 11:31-- B 1 11:31-- A 1 11:31-- D 1 11:31--11:31 B 1 05-17 (Friday) 02:02 BST-- t c 02:02-- t c 08-29 (Wednesday) 00:05-- & bbc2.bbc.co.uk 10-10 (Thursday) 09:00 +0100--09:55 +0100 News TA xmltv-1.4.0/t/data/tv_to_text_dups_xml_clump_xml.expected000066400000000000000000000003321500074233200237510ustar00rootroot0000000000000012-28 (Friday) 11:31-- A 1 11:31-- B 1 11:31-- A 1 11:31-- D 1 11:31--11:31 B 1 05-17 (Friday) 02:02 BST-- t c 02:02-- t c 04-20 (Saturday) 17:25 UTC--18:00 UTC First in clump foo 17:25--18:00 Second in clump foo xmltv-1.4.0/t/data/tv_to_text_dups_xml_dups_xml.expected000066400000000000000000000004011500074233200236010ustar00rootroot0000000000000012-28 (Friday) 11:31-- A 1 11:31-- B 1 11:31-- A 1 11:31-- D 1 11:31--11:31 B 1 05-17 (Friday) 02:02 BST-- t c 02:02-- t c 12-28 (Friday) 11:31 UTC-- A 1 11:31-- B 1 11:31-- A 1 11:31-- D 1 11:31--11:31 B 1 05-17 (Friday) 02:02 BST-- t c 02:02-- t c xmltv-1.4.0/t/data/tv_to_text_dups_xml_empty_xml.expected000066400000000000000000000001761500074233200237750ustar00rootroot0000000000000012-28 (Friday) 11:31-- A 1 11:31-- B 1 11:31-- A 1 11:31-- D 1 11:31--11:31 B 1 05-17 (Friday) 02:02 BST-- t c 02:02-- t c xmltv-1.4.0/t/data/tv_to_text_empty_xml.expected000066400000000000000000000000001500074233200220440ustar00rootroot00000000000000xmltv-1.4.0/t/data/tv_to_text_empty_xml_amp_xml.expected000066400000000000000000000001401500074233200235660ustar00rootroot0000000000000008-29 (Wednesday) 00:05-- & bbc2.bbc.co.uk 10-10 (Thursday) 09:00 +0100--09:55 +0100 News TA xmltv-1.4.0/t/data/tv_to_text_empty_xml_clump_xml.expected000066400000000000000000000001231500074233200241320ustar00rootroot0000000000000004-20 (Saturday) 17:25--18:00 First in clump foo 17:25--18:00 Second in clump foo xmltv-1.4.0/t/data/tv_to_text_empty_xml_dups_xml.expected000066400000000000000000000001761500074233200237750ustar00rootroot0000000000000012-28 (Friday) 11:31-- A 1 11:31-- B 1 11:31-- A 1 11:31-- D 1 11:31--11:31 B 1 05-17 (Friday) 02:02 BST-- t c 02:02-- t c xmltv-1.4.0/t/data/tv_to_text_empty_xml_empty_xml.expected000066400000000000000000000000001500074233200241420ustar00rootroot00000000000000xmltv-1.4.0/t/data/tv_to_text_intervals_xml.expected000066400000000000000000000005231500074233200227270ustar00rootroot0000000000000002-16 (Sunday) 13:00-- On 'before' but not 'after' 13:30. 0 13:00--14:00 On both 'before' and 'after' 13:30. 0 13:30--13:30 On 'before' but not 'after' 13:30. 0 13:00--13:10 Straightforward 'before' but not 'after' 13:30. 0 13:30--14:00 On both 'before' and 'after' 13:30. 0 13:40--14:00 Straightforward 'after' but not 'before' 13:30. 0 xmltv-1.4.0/t/data/tv_to_text_length_xml.expected000066400000000000000000000000461500074233200222010ustar00rootroot0000000000000002-24 (Sunday) 00:00--00:30 News CNN xmltv-1.4.0/t/data/tv_to_text_overlap_xml.expected000066400000000000000000000005241500074233200223710ustar00rootroot0000000000000012-28 (Friday) 11:31-- A 1 11:31-- B 1 11:31-- C 1 11:31-- D 1 11:31-- E 1 11:31-- F 1 11:31--00:00 G 2 11:31--00:00 H 2 11:31--00:00 I 4 11:31--00:00 J 5 11:31--00:00 K 5 01-01 (Saturday) 00:00--00:00 Zero length, should not overlap with one starting at same time a 00:00--01:00 One hour long a 00:30--00:30 Zero length in the middle a xmltv-1.4.0/t/data/tv_to_text_simple_xml.expected000066400000000000000000000001751500074233200222140ustar00rootroot0000000000000008-29 (Wednesday) 00:05-- The Phil Silvers Show bbc2.bbc.co.uk 09:55-- King of the Hill // Meet the Propaniacs channel4.com xmltv-1.4.0/t/data/tv_to_text_simple_xml_x_whatever_xml.expected000066400000000000000000000003471500074233200253310ustar00rootroot0000000000000008-29 (Wednesday) 00:05-- The Phil Silvers Show bbc2.bbc.co.uk 09:55-- King of the Hill // Meet the Propaniacs channel4.com 00:05-- The Phil Silvers Show bbc2.bbc.co.uk 09:55-- King of the Hill // Meet the Propaniacs channel4.com xmltv-1.4.0/t/data/tv_to_text_sort1_xml.expected000066400000000000000000000001441500074233200217670ustar00rootroot0000000000000001-01 (Saturday) 00:00--02:00 A 1 03:00-- B 1 03:00-- C 1 00:00-- D 1 00:00-- E 1 00:00--01:00 F 1 xmltv-1.4.0/t/data/tv_to_text_sort2_xml.expected000066400000000000000000000000711500074233200217670ustar00rootroot0000000000000001-01 (Friday) 00:00-- A 2 00:00--01:00 B 2 00:00-- C 2 xmltv-1.4.0/t/data/tv_to_text_sort_xml.expected000066400000000000000000000002521500074233200217060ustar00rootroot0000000000000012-18 (Tuesday) 04:05-- ITV Nightscreen carlton.com 04:05--04:30 Motorsport Mundial channel5.co.uk 05-12 (Monday) 23:35 -0400-- A 2 05-13 (Tuesday) 00:06--01:06 B 2 xmltv-1.4.0/t/data/tv_to_text_test_empty_xml.expected000066400000000000000000000001441500074233200231140ustar00rootroot0000000000000004-20 (Saturday) 13:10-- A programme with empty stuff that should not be written out again foo.com xmltv-1.4.0/t/data/tv_to_text_test_livre_xml.expected000066400000000000000000000000651500074233200231010ustar00rootroot0000000000000012-11 (Tuesday) 18:00-- south-east.bbc1.bbc.co.uk xmltv-1.4.0/t/data/tv_to_text_test_remove_some_overlapping_xml.expected000066400000000000000000000005401500074233200267040ustar00rootroot0000000000000010-25 (Saturday) 07:00--07:45 Container A 3 07:00--07:05 Contained A 0 3 07:05--07:15 Contained A 1 3 07:15--07:45 Contained A 2 3 08:00--08:05 Contained B 0 3 08:00--08:15 Container B 3 08:05--08:15 Contained B 1 3 09:00--10:00 Overlap 0 3 09:00--10:00 Overlap 1 3 10:00--10:15 Container C 3 10:00--10:05 Contained C 0 3 10:05--10:15 Contained C 1 3 xmltv-1.4.0/t/data/tv_to_text_test_sort_by_channel_xml.expected000066400000000000000000000000761500074233200251330ustar00rootroot0000000000000002-16 (Sunday) 00:00-- A 0 00:30--00:30 B 0 00:00--00:10 C 0 xmltv-1.4.0/t/data/tv_to_text_test_xml.expected000066400000000000000000000001111500074233200216700ustar00rootroot0000000000000006-03 (Saturday) 16:33-- blah 3SAT Generated by my listings generator. xmltv-1.4.0/t/data/tv_to_text_test_xml_test_xml.expected000066400000000000000000000001331500074233200236130ustar00rootroot0000000000000006-03 (Saturday) 16:33-- blah 3SAT 16:33-- blah 3SAT Generated by my listings generator. xmltv-1.4.0/t/data/tv_to_text_whitespace_xml.expected000066400000000000000000000001501500074233200230500ustar00rootroot0000000000000008-29 (Wednesday) 00:05-- T foo.tv 00:05-- T foo.tv 00:05-- T foo.tv 00:05-- T foo.tv 00:05-- T foo.tv xmltv-1.4.0/t/data/tv_to_text_x_whatever_xml.expected000066400000000000000000000001751500074233200230770ustar00rootroot0000000000000008-29 (Wednesday) 00:05-- The Phil Silvers Show bbc2.bbc.co.uk 09:55-- King of the Hill // Meet the Propaniacs channel4.com xmltv-1.4.0/t/data/whitespace.xml000066400000000000000000000014701500074233200167160ustar00rootroot00000000000000 T Blah. T Blah. T Blah. T Blah. T Blah. xmltv-1.4.0/t/data/x-whatever.xml000066400000000000000000000017101500074233200166510ustar00rootroot00000000000000 No thank you The Phil Silvers Show Bilko claims he's had a close encounter with an alien in order to be given some compassionate leave so he can visit an old flame in New York. 11 King of the Hill Meet the Propaniacs Bobby tours with a comedy troupe who specialize in propane-related mirth. Mike Judge Lane Smith animation xmltv-1.4.0/t/parallel_test000077500000000000000000000120101500074233200157000ustar00rootroot00000000000000#!/usr/bin/perl -w # # parallel_test # # Quick and dirty test rig I use for checking that changes to # tv_grab_uk give the same results. Should be possible to # use it for testing any program where you want to make sure there are # no differences between the old and new versions. # # At present, the command to run is hardcoded in the script. You'll # need to make a copy of this script and edit it. # # Should not be installed as part of XMLTV, but can be included in # source tarball. # # -- Ed Avis, ed@membled.com, 2002-01-31 use strict; # Use Log::TraceMessages if installed. BEGIN { eval { require Log::TraceMessages }; if ($@) { *t = sub {}; *d = sub { '' }; } else { *t = \&Log::TraceMessages::t; *d = \&Log::TraceMessages::d; Log::TraceMessages::check_argv(); } } # Old command - directory it should run from, and command to run. my $a_dir = '/home/ed/work/apps/xmltv/old_version'; my @a_cmd = qw(perl -Iblib/lib blib/script/tv_grab_uk --share blib/share); # New command in its directory. my $b_dir = '/home/ed/work/apps/xmltv/cvs_working'; my @b_cmd = qw(perl -Iblib/lib blib/script/tv_grab_uk --share blib/share); # Directory to store test results. my $tmp = '/home/ed/vol/tmp'; foreach ($a_dir, $b_dir, $tmp) { if (not -d) { die "no such directory $_ - edit the script for your setup\n"; } } # Arguments to pass to each command for each test. A list of pairs # and each element of a pair is a list of arguments. # my @tests = map { [ $_, $_ ] } ( [qw(--days 1 --config-file grab/uk/test_configs/carlton)], [qw(--days 1 --config-file grab/uk/test_configs/bbc1)], [qw(--days 1 --config-file grab/uk/test_configs/radio4)], [qw(--config-file grab/uk/test_configs/carlton)], [qw(--days 1 --config-file grab/uk/test_configs/bbc1)], [qw(--config-file grab/uk/test_configs/tynetees)], [qw(--days 1 --config-file grab/uk/test_configs/radio)], [qw(--config-file grab/uk/test_configs/radio)], [qw(--config-file grab/uk/test_configs/satellite)], [qw(--config-file grab/uk/test_configs/all)], [qw(--config-file grab/uk/test_configs/gratis)], [qw(--config-file grab/uk/test_configs/gratis_radio)], [qw(--config-file grab/uk/test_configs/music_nickelodeon_e4)], ); # Arguments at the start of the command line for every test. my @constant_args; @constant_args = ('--cache', "$tmp/parallel_test.cache"); # Normally this script checks for identical output. But you may give # fixups to be applied to either version's output before comparison. # my (@a_fixups, @b_fixups); @a_fixups = @b_fixups = ('tv_sort'); # More advanced munging may depend on looking at one file and using it # to alter the other. These programs should act as filters, and the # filename of the 'other' file will be passed as an argument. # # Note that @fixup_a_given_b will be run in the directory of the old # version, and @fixup_b_given_a will run in the new directory. # my (@fixup_a_given_b, @fixup_b_given_a); use Getopt::Std; our ($opt_q, $opt_a, $opt_b); getopts('qab'); if ($opt_q) { warn "use -a to reuse results from old version, -b for new, -ab for both\n"; $opt_a = $opt_b = 1; } my $starting_test = $ARGV[0] || 0; my $num_failures = 0; $SIG{__WARN__} = sub { ++ $num_failures; warn @_ }; for (my $test_num = $starting_test; $test_num < @tests; $test_num++) { my ($a_args, $b_args) = @{$tests[$test_num]}; print STDERR "test $test_num: @$b_args\n"; chdir $a_dir or die; my $old_out = "$tmp/$test_num.old.out"; unless ($opt_a) { system("time @a_cmd @constant_args @$a_args >$old_out") && die "@a_cmd failed"; } foreach (@a_fixups) { t "in dir $a_dir, running fixup $_ <$old_out >$old_out.fix"; system("{ $_ ; } <$old_out >$old_out.fix") && die "$_ failed"; $old_out = "$old_out.fix"; } chdir $b_dir or die; my $new_out = "$tmp/$test_num.new.out"; unless ($opt_b) { system("time @b_cmd @constant_args @$b_args >$new_out") && die "@b_cmd failed"; } foreach (@b_fixups) { t "in dir $b_dir, running fixup $_ <$new_out >$new_out.fix"; system("{ $_ ; } <$new_out >$new_out.fix") && die "$_ failed"; $new_out = "$new_out.fix"; } if (@fixup_a_given_b and @fixup_b_given_a) { warn "fixing up old output given new, _then_ fixing up new given old\n"; } chdir $a_dir or die; foreach (@fixup_a_given_b) { t "in dir $a_dir, running fixup $_ $new_out <$old_out >$old_out.fix"; system("{ $_ $new_out ; } <$old_out >$old_out.fix") && die "$_ failed"; $old_out = "$old_out.fix"; } chdir $b_dir or die; foreach (@fixup_b_given_a) { t "in dir $b_dir, running fixup $_ $old_out <$new_out >$new_out.fix"; system("{ $_ $old_out ; } <$new_out >$new_out.fix") && die "$_ failed"; $new_out = "$new_out.fix"; } my $diff = "$tmp/$test_num.diff"; if (system("diff -u $old_out $new_out >$diff")) { print STDERR "diff found differences: \n"; open(DIFF, $diff) or die; while () { if ($. > 1000) { print "...\n"; last; } print; } exit 1; } } print STDERR "$num_failures errors\n"; exit($num_failures < 255 ? $num_failures : 255); xmltv-1.4.0/t/test_dst.t000077500000000000000000000006311500074233200151460ustar00rootroot00000000000000#!/usr/bin/perl use warnings; use strict; use XMLTV::DST; # These tests rely on the internal representation of dates, but what # the heck. # print "1..2\n"; my $r = parse_local_date('20040127021000', '+0100'); print 'not ' if $r ne '2004012701:10:00'; print "ok 1\n"; my ($d, $tz) = @{date_to_local('2004012701:10:00', '+0100')}; print 'not ' if $d ne '2004012702:10:00' or $tz ne '+0100'; print "ok 2\n"; xmltv-1.4.0/t/test_filters.t000077500000000000000000000343501500074233200160310ustar00rootroot00000000000000#!/usr/bin/perl -w # # Run lots of filter programs on lots of inputs and check the output # is as expected. Stderr is checked if there is an 'expected_err' # file but we do not allow for filters that return an error code. In # fact, they're not filters at all: we assume that each can take an # input filename and the --output option. # # -- Ed Avis, ed@membled.com, 2002-02-14 use strict; use Getopt::Long; use File::Copy; use XMLTV::Usage < \$tests_dir, 'cmds-dir=s' => \$cmds_dir, 'verbose' => \$verbose, 'full' => \$full) or usage(0); if (not $full) { warn "running small test suite, use $0 --full for the whole lot\n"; } # Commands to run. For each command and input file we have an # 'expected output' file to compare against. Also each command has an # 'idempotent' flag. If this is true then we check that (for example) # tv_cat | tv_cat has the same effect as tv_cat, for all input files. # # A list of pairs: the first element of the pair is a list of command # and arguments, the second is the idempotent flag. # my @cmds = ( [ [ 'tv_cat' ], 1 ], [ [ 'tv_extractinfo_en' ], 1 ], # We assume that most usages of tv_grep are idempotent on the sample # files given. But see BUGS section of manual page. [ [ 'tv_grep', '--channel-name', 'd' ], 1 ], [ [ 'tv_grep', '--not', '--channel-name', 'd' ], 1 ], [ [ 'tv_sort' ], 1 ], [ [ 'tv_sort', '--by-channel' ], 1 ], [ [ 'tv_to_latex' ], 0 ], [ [ 'tv_to_text', ], 0 ], [ [ 'tv_remove_some_overlapping' ], 1 ], [ [ 'tv_grep', '--on-after', '200302161330 UTC' ], 1 ], [ [ 'tv_grep', '--on-before', '200302161330 UTC' ], 1 ], ); if ($full) { push @cmds, ( [ [ 'tv_grep', '--channel', 'xyz', '--or', '--channel', 'b' ], 1 ], [ [ 'tv_grep', '--channel', 'xyz', '--or', '--not', '--channel', 'b' ], 1 ], [ [ 'tv_grep', '--previously-shown', '' ], 1 ], [ [ 'tv_grep', 'a' ], 1 ], [ [ 'tv_grep', '--category', 'b' ], 1 ], [ [ 'tv_grep', '-i', '--last-chance', 'c' ], 1 ], [ [ 'tv_grep', '--premiere', '' ], 1 ], [ [ 'tv_grep', '--new' ], 1 ], [ [ 'tv_grep', '--channel-id', 'channel4.com' ], 1 ], [ [ 'tv_grep', '--not', '--channel-id', 'channel4.com' ], 1 ], [ [ 'tv_grep', '--on-after', '2002-02-05 UTC' ], 1 ], [ [ 'tv_grep', '--eval', 'scalar keys %$_ > 5' ], 0 ], [ [ 'tv_grep', '--category', 'e', '--and', '--title', 'f' ], 1 ], [ [ 'tv_grep', '--category', 'g', '--or', '--title', 'h' ], 1 ], [ [ 'tv_grep', '-i', '--category', 'i', '--title', 'j' ], 1 ], [ [ 'tv_grep', '-i', '--category', 'i', '--title', 'h' ], 1 ], [ [ 'tv_grep', '--channel-id-exp', 'sat' ], 1 ], ); } if (@ARGV) { # Remaining arguments are regexps to match commands to run. my @new_cmds; my %seen; foreach my $arg (@ARGV) { foreach my $cmd (@cmds) { for (join(' ', @{$cmd->[0]})) { push @new_cmds, $cmd if /$arg/ and not $seen{$_}++; } } } die "no commands matched regexps: @ARGV" if not @new_cmds; @cmds = @new_cmds; print "running commands:\n", join("\n", map { join(' ', @{$_->[0]}) } @cmds), "\n"; } # Input files we could use to build test command lines. my @inputs = <$tests_dir/*.xml>; my @inputs_gz = <$tests_dir/*.xml.gz>; s/\.gz$// foreach @inputs_gz; @inputs = sort (@inputs, @inputs_gz); die "no test cases (*.xml, *.xml.gz) found in $tests_dir" if not @inputs; foreach (@inputs) { s!^\Q$tests_dir\E/!!o or die; } # We want to test multiple input files. But it would be way OTT to # test all permutations of all input files up to some length. Instead # we pick all single files and a handful of pairs. # my @tests; # The input file empty.xml is special: we particularly like to use it # in tests. Then there are another two files we refer to by name. # my $empty_input = 'empty.xml'; foreach ($empty_input, 'simple.xml', 'x-whatever.xml') { die "file $tests_dir/$_ not found" if not -f "$tests_dir/$_"; } # We need to track the encoding of each input file so we don't try to # mix them on the same command line (not allowed). # my %input_encoding; foreach (@inputs) { $input_encoding{$_} = ($_ eq 'test_livre.xml') ? 'ISO-8859-1' : 'UTF-8'; } my %all_encodings = reverse %input_encoding; # For historical reasons we like to have certain files at the front of # the list. Aargh, this is so horrible. # sub move_to_front( \@$ ) { our @l; local *l = shift; my $elem = shift; my @r; foreach (@l) { if ($_ eq $elem) { unshift @r, $_; } else { push @r, $_; } } @l = @r; } foreach ('dups.xml', 'clump.xml', 'amp.xml', $empty_input) { move_to_front @inputs, $_; } # Add a test to the list. Arguments are listref of filenames, and # optional name for this set of files. # sub add_test( $;$ ) { my ($files, $name) = @_; $name = join('_', @$files) if not defined $name; my $enc; foreach (@$files) { if (defined $enc and $enc ne $input_encoding{$_}) { die 'trying to add test with two different encodings'; } else { $enc = $input_encoding{$_}; } } push @tests, { inputs => $files, name => $name }; } # A quick and effective test for each command is to run it on all the # input files at once. But we have to segregate them by encoding. # my %used_enc_name; foreach my $enc (sort keys %all_encodings) { (my $enc_name = $enc) =~ tr/[A-Za-z0-9]//dc; die "cannot make name for encoding $enc" if $enc_name eq ''; die "two encodings go to same name $enc_name" if $used_enc_name{$enc_name}++; my @files = grep { $input_encoding{$_} eq $enc } @inputs; if (@files == 0) { # Shouldn't happen. die "strange, no files for $enc"; } elsif (@files == 1) { # No point adding this as it will be run as an individual # test. # } else { add_test(\@files, "all_$enc_name"); } } # One important test is two empty files in the middle of the list. add_test([ $inputs[1], $empty_input, $empty_input, $inputs[2] ]); # Another special case we want to run every time. add_test([ 'simple.xml', 'x-whatever.xml' ]); # Another - check that duplicate channels are removed. add_test([ 'test.xml', 'test.xml' ]); if ($full) { # Test some pairs of files, but not all possible pairs. my $pair_limit = 4; die "too few inputs" if $pair_limit > @inputs; foreach my $i (0 .. $pair_limit - 1) { foreach my $j (0 .. $pair_limit - 1) { add_test([ $inputs[$i], $inputs[$j] ]); } } # Then all the single files. add_test([ $_ ]) foreach @inputs; } else { # Check overlapping warning from tv_sort. This ends up giving the # input file to every command, not just tv_sort; oh well. # # Not needed in the case when $full is true because we test every # individual file then. # add_test([ 'overlap.xml' ]); } # Any other environment needed (relative to $tests_dir) $ENV{PERL5LIB} .= ":.."; my %seen; # Count total number of tests to run. my $num_tests = 0; foreach (@cmds) { $num_tests += scalar @tests; $num_tests += scalar @tests if $_->[1]; # idem. test } print "1..$num_tests\n"; my $test_num = 0; foreach my $pair (@cmds) { my ($cmd, $idem) = @$pair; foreach my $test (@tests) { my @test_inputs = @{$test->{inputs}}; ++ $test_num; my $test_name = join('_', @$cmd, $test->{name}); $test_name =~ tr/A-Za-z0-9/_/sc; die "two tests munge to $test_name" if $seen{$test_name}++; my @cmd = @$cmd; my $base = "$tests_dir/$test_name"; my $expected = "$base.expected"; my $out = "$base.out"; my $err = "$base.err"; # Gunzip automatically before testing, gzip back again # afterwards. Keys matter, values do not. # my (%to_gzip, %to_gunzip); foreach (@test_inputs, $expected) { my $gz = "$_.gz"; if (not -e and -e $gz) { $to_gunzip{$gz}++ && die "$gz seen twice"; $to_gzip{$_}++ && die "$_ seen twice"; } } system 'gzip', '-d', keys %to_gunzip if %to_gunzip; # To unlink when tests are done - this hash can change. # Again, only keys are important. (FIXME should encapsulate # as 'Set' datatype.) # my %to_unlink = ($out => undef, $err => undef); my $out_content; # contents of $out, to be filled in later # TODO File::Spec $cmd[0] = "$cmds_dir/$cmd[0]"; $cmd[0] =~ s!/!\\!g if $^O eq 'MSWin32'; if ($verbose) { print STDERR "test $test_num: @cmd @test_inputs\n"; } my @in = map { "$tests_dir/$_" } @test_inputs; my $okay = run(\@cmd, \@in, $out, $err); # assume: if $okay then -e $out. my $have_expected = -e $expected; if (not $okay) { print "not ok $test_num\n"; delete $to_unlink{$out}; delete $to_unlink{$err}; } elsif ($okay and not $have_expected) { # This should happen after adding a new test case, never # when just running the tests. # warn "creating $expected\n"; copy($out, $expected) or die "cannot copy $out to $expected: $!"; # Don't print any message - the test just 'did not run'. } elsif ($okay and $have_expected) { $out_content = read_file($out); my $expected_content = read_file($expected); if ($out_content ne $expected_content) { warn "failure for @cmd @in, see $base.*\n"; print "not ok $test_num\n"; $okay = 0; delete $to_unlink{$out}; delete $to_unlink{$err}; } else { # The output was correct: if there's also an 'expected # error' file check that. Otherwise we do not check # what was printed on stderr. # my $expected_err = "$base.expected_err"; if (-e $expected_err) { my $err_content = read_file($err); my $expected_content = read_file($expected_err); if ($err_content ne $expected_content) { warn "failure for stderr of @cmd @in, see $base.*\n"; print "not ok $test_num\n"; $okay = 0; delete $to_unlink{$out}; delete $to_unlink{$err}; } else { print "ok $test_num\n"; } } else { # Don't check stderr. print "ok $test_num\n"; } } } else { die } if ($idem) { ++ $test_num; if ($verbose) { print STDERR "test $test_num: "; print STDERR "check that @cmd is idempotent on this input\n"; } if ($okay) { die if not -e $out; # Run the command again, on its own output. my $twice_out = "$base.twice_out"; my $twice_err = "$base.twice_err"; $to_unlink{$twice_out} = $to_unlink{$twice_err} = undef; my $twice_okay = run(\@cmd, [ $out ], $twice_out, $twice_err); # assume: if $twice_okay then -e $twice_out. if (not $twice_okay) { print "not ok $test_num\n"; delete $to_unlink{$out}; delete $to_unlink{$twice_out}; delete $to_unlink{$twice_err}; } else { my $twice_out_content = read_file($twice_out); my $ok; if (not defined $out_content) { warn "cannot run idempotence test for @cmd\n"; $ok = 0; } elsif ($twice_out_content ne $out_content) { warn "failure for idempotence of @cmd, see $base.*\n"; $ok = 0; } else { $ok = 1 } if (not $ok) { print "not ok $test_num\n"; delete $to_unlink{$out}; delete $to_unlink{$twice_out}; delete $to_unlink{$twice_err}; } else { print "ok $test_num\n"; } } } else { warn "skipping idempotence test for @cmd on @test_inputs\n"; # Do not print 'ok' or 'not ok'. } } foreach (keys %to_unlink) { (not -e) or unlink or warn "cannot unlink $_: $!"; } system 'gzip', keys %to_gzip if %to_gzip; } } die "ran $test_num tests, expected to run $num_tests" if $test_num != $num_tests; # run() # # Run a Perl command redirecting input and output. This is not fully # general - it relies on the --output option working for redirecting # output. (Don't know why I decided this, but it does.) # # Parameters: # (ref to) list of command and arguments # (ref to) list of input filenames # output filename # error output filename # # This routine is specialized to Perl stuff running during the test # suite; it has the necessary -Iwhatever arguments. # # Dies if error opening or closing files, or if the command is killed # by a signal. Otherwise creates the output files, and returns # success or failure of the command. # sub run( $$$$ ) { my ($cmd, $in, $out, $err) = @_; die if not defined $cmd; my @cmd = (qw(perl -Iblib/arch -Iblib/lib), @$cmd, @$in, '--output', $out); # Redirect stderr to file $err. open(OLDERR, '>&STDERR') or die "cannot dup stderr: $!\n"; if (not open(STDERR, ">$err")) { print OLDERR "cannot write to $err: $!\n"; exit(1); } # Run the command. my $r = system(@cmd); # Restore old stderr. if (not close(STDERR)) { print OLDERR "cannot close $err: $!\n"; exit(1); } if (not open(STDERR, ">&OLDERR")) { print OLDERR "cannot dup stderr back again: $!\n"; exit(1); } # Check command return status. if ($r) { my ($status, $sig, $core) = ($? >> 8, $? & 127, $? & 128); if ($sig) { die "@cmd killed by signal $sig, aborting"; } warn "@cmd failed: $status, $sig, $core\n"; return 0; } return 1; } sub read_file( $ ) { my $f = shift; local $/ = undef; local *FH; open(FH, $f) or die "cannot open $f: $!"; my $content = ; close FH or die "cannot close $f: $!"; return $content; } xmltv-1.4.0/t/test_icon.t000077500000000000000000000011161500074233200153030ustar00rootroot00000000000000#!/usr/bin/perl use warnings; use strict; use XMLTV; # This checks only that the data can be written without crashing. print "1..1\n"; my %p = (title => [ [ 'Foo' ] ], start => '20000101000000 +0000', channel => '1.foo.com', rating => [ [ '18', 'BBFC', [ { src => 'img.png' } ] ] ], ); my $out = ($^O =~ /^win/i ? 'nul' : '/dev/null'); my $fh = new IO::File ">$out"; die "cannot write to $out\n" if not $fh; my $w = new XMLTV::Writer(OUTPUT => $fh, encoding => 'UTF-8'); $w->start({}); $w->write_programme(\%p); $w->end(); close $fh or warn "cannot close $out: $!"; print "ok 1\n"; xmltv-1.4.0/t/test_library.t000077500000000000000000000015571500074233200160300ustar00rootroot00000000000000#!/usr/bin/perl use warnings; use strict; use File::Temp qw(tempdir); use XMLTV; print "1..1\n"; my $tempdir = tempdir('XXXXXXXX', CLEANUP => 1); chdir $tempdir or die "cannot chdir to $tempdir: $!"; # Test for bug where write_programme would delete everything from the # hash passed in. # my $scratch = 'scratch'; my $fh = new IO::File ">$scratch"; die "cannot write to $scratch\n" if not $fh; my $w = new XMLTV::Writer(OUTPUT => $fh, encoding => 'UTF-8'); $w->start({}); my %prog = (start => '20000101000000', channel => 'c', title => [ [ 'Foo' ] ], ); my %prog_bak = %prog; $w->write_programme(\%prog); my $ok; if (keys %prog == keys %prog_bak) { foreach (keys %prog) { $ok = 0, last if $prog{$_} ne $prog_bak{$_}; } $ok = 1; } else { $ok = 0 }; print 'not ' if not $ok; print "ok 1\n"; $w->end(); close $fh or die "cannot close $scratch: $!"; xmltv-1.4.0/t/test_tv_augment.t000077500000000000000000000055531500074233200165350ustar00rootroot00000000000000#!/usr/bin/perl # # Run tv_augment against various input files and check the generated output # is as expected. # # This framework (borrowed from test_tv_imdb.t) tests each type of automatic # and user rule for tv_augment (lib/Augment.pm) by comparing the output # generated from input data against the expected output for each rule type. # # -- Nick Morrott, knowledgejunkie@gmail.com, 2016-07-07 use warnings; use strict; use Getopt::Long; use Cwd; use File::Temp qw(tempdir); use File::Copy; use XMLTV::Usage < \$tests_dir, 'cmds-dir=s' => \$cmds_dir, 'verbose' => \$verbose) or usage(0); usage(0) if @ARGV; my $tmpDir = tempdir(CLEANUP => 1); # my $tmpDir = tempdir(CLEANUP => 0); my @inputs = <$tests_dir/*.xml>; @inputs = sort (@inputs); die "no test cases (*.xml) found in $tests_dir" if not @inputs; my $numtests = scalar @inputs; print "1..$numtests\n"; my $n = 0; INPUT: foreach my $input (@inputs) { ++$n; use File::Basename; my $input_basename = File::Basename::basename($input); my $output="$tmpDir/".$input_basename."-output"; my $cmd="$cmds_dir/tv_augment --rule $tests_dir/rules/test_tv_augment.rules --config $tests_dir/configs/$input_basename.conf --input $input --output $output 2>&1"; # my $cmd="perl -I blib/lib $cmds_dir/tv_augment --rule $tests_dir/rules/test_tv_augment.rules --config $tests_dir/configs/$input_basename.conf --input $input --output $output --log $tmpDir/$input_basename.log --debug 5 >$tmpDir/$input_basename.debug 2>&1"; my $r = system($cmd); # Check command return status. if ($r) { my ($status, $sig, $core) = ($? >> 8, $? & 127, $? & 128); if ($sig) { die "$cmd killed by signal $sig, aborting"; } warn "$cmd failed: $status, $sig, $core\n"; print "not ok $n\n"; next INPUT; } open(FD, "$input-expected") || die "$input-expected:$!"; open(OD, "$output") || die "$output:$!"; my $line = 0; my $failed = 0; INPUT: while() { my $in=$_; $line++; # ignore single line XML comments in "expected" data next INPUT if ($in =~ m/\s* xmltv-1.4.0/xmltv_logo.ico000066400000000000000000000031761500074233200155570ustar00rootroot0000000000000000h(0`DDDDDDDDDDDDDDDDDDODDDDDDDDDDDDDDDDDDODDDDDDDDDDDDDDDDDDODDDDDDDDDDDDDDDDDDODDDDDDDDDDDDDDDDDDODDDDDDDDDDDDDDDDDDODDDDDDDDDDDDDDDDDDO       xmltv-1.4.0/xmltv_logo.png000066400000000000000000000131761500074233200155720ustar00rootroot00000000000000PNG  IHDR7 ,tEXtCreation TimeThu 27 Sep 2007 03:40:30 -0000stIME   pHYs  ~gAMA aIDATx puO%AqXeJq-#2 #*RwquGeB Q@AV # =_'o}｝)G~?Hsr4rhͱO4´/u9 8}>^B7YL~qa)~@Q7ifaUq} s/Y"==]ҥ:2sʇ w]KJJ2Q>BQOF9ƌ믿effM6ſDmڮ6`?Ԗ-[=)$|[̻tR ig϶* Oi9 \Rߧ*nRi5GRH<З cǎΜ9e۶mZvvj4}߲ٲe?y'}ȧ&d:d߂Dfߐ466Z P^}vÆ ӎ9b۩-aκV}iPqdd9s樦Axu~a3d?aMCYnMdS{l?0B%Kgb}q1o{呲n{=}}Vjm}@B%$~_{LO;x`6Zk%E7f/'uq5GTMa\ξt̙ \|tQK9>]wOM#25 {ǭC(ip}-))INqpCrihdg+#^+8ޡA3MR!Ѻ; ~g |8=&N AiΝm6ںu+RuuHFct#͛7k555ORQQO S|zvĞ={;#71Y2+~ꩧ BM Ms!ZZ^ B%.s m!VTTD99?Ν;4~ܹ3у222(55y_}4oLܚܾe }gTYUEgNrUa;VdågeQ^otԻwoG3xό3hժUt!n4ؼ'[kww|ŋ]Yϯ<nӗ {coϻw˯B|hN"zCW]uټ5oTw!eɜ9%;j29*i 2*=c7Ghk57B©'Ib>ZH& M`4ǝ|})ziΝSJJm${Y<ҙ3)TSOJ *+VWӶߦ?=lLɼYC>Ǐ c-[F_x^kVTTЪbwޙ~!0` g"Nk~-ki=;H,A[pm*++.mllZ6nGCXQWG? E+1XļB` E+1XļB` E+1XļB`qm^\u>pxkpôz:?6Ba=kӴ?VSU~!H4rQ\799Y wx{&3C[::*1oXz%xG̛ `\  EAMڸ5%X#m>Fmܛ+` \*YB`۝r Km¼ʏYvw'8+u ue5r fei,lUED-X[XFfTy$i߳<ϚʲDzu,7Բ,cnܥO#~+v|BՖż8 c'K5p GXLx; .hsV |ʚUjYV3P̌e^JDM,SD己TX8dY24nS8 >2mئOYfTpOX*(Me.f^Lj<k RAm֤Ve{},cy|TƱm^,3}`f3q= lÜ ڵy7 cFV:/a\'R aӼ}X*$d\+`n̛7_"l(qK(k|Bx/%y5F,ad17ɇ,f̵ڶ@,,gE̋ƓUc6ldV>+Z㥬Xp-johg|,'q 04x 3+YRzp= Xή-xfXe^5;=A7ؿu]bbQ3ÀLVz+ /Uw"g]1808@XNa\dp[X'E:jJj#ރ'VC5ְė~},j_"SYjVE&@\91C-0'(g5̃t ї2OͶX90Ї'FЈA\βc*0>p8f0 n|ӻ(~c|b\ ?Yumfa@d2iVQ!J5ԍ/_Yhb v)j/Aaj7ۑ0>e%|Uocά~wYVVSOcF60Yf 3p3@EMuVx)?vpvRΉpeAO\hlkwŨ{NBP1gsyȪyQc:!2E=8*;B`b}jƼȴ-RvH<2+?./vCѹsՉ}JI~\@饗RRRPGmހqgΜI6l:ҴO￟:DEEEq7pjޓ'ORaa!5FI>nX.fwqa\d\øٷ,YB7om">D0m&MqԲjK.2n!x\ׯg8A-M0Aw Ki˖-F4X,fw8nqUش~= ǓC>v7^'Ch{żh8p7O+elCrss6ƐXnԩSYoQ_lcSNw ֍?^ˀSi4eʔ-fw]-\uFVru^g/a#Nq'G5ʕ+n+m6Ok8ƙ6Ɔ{=WE#|Μ9jc[ˬLjW`/ :xctXBD