subliminal-0.7.4/0000755000175000017500000000000012271547017015003 5ustar antoineantoine00000000000000subliminal-0.7.4/README.rst0000644000175000017500000000316012271546332016471 0ustar antoineantoine00000000000000Subliminal ========== Subliminal is a python library to search and download subtitles. It comes with an easy to use CLI (command-line interface) suitable for direct use or cron jobs. .. image:: https://travis-ci.org/Diaoul/subliminal.png?branch=develop :target: https://travis-ci.org/Diaoul/subliminal .. image:: https://coveralls.io/repos/Diaoul/subliminal/badge.png?branch=develop :target: https://coveralls.io/r/Diaoul/subliminal?branch=develop Providers --------- Subliminal uses multiple providers to give users a vast choice and have a better chance to find the best matching subtitles. Providers are extensible through a dedicated entry point. * Addic7ed * BierDopje * OpenSubtitles * Podnapisi * TheSubDB * TvSubtitles Usage ----- CLI ^^^ Download english subtitles:: $ subliminal -l en -- The.Big.Bang.Theory.S05E18.HDTV.x264-LOL.mp4 1 subtitle downloaded Library ^^^^^^^ Download best subtitles in French and English for videos less than one week old in a video folder, skipping videos that already have subtitles whether they are embedded or not:: from babelfish import Language from datetime import timedelta import subliminal # configure the cache subliminal.cache_region.configure('dogpile.cache.dbm', arguments={'filename': '/path/to/cachefile.dbm'}) # scan for videos in the folder and their subtitles videos = subliminal.scan_videos(['/path/to/video/folder'], subtitles=True, embedded_subtitles=True, age=timedelta(weeks=1)) # download subliminal.download_best_subtitles(videos, {Language('eng'), Language('fra')}, age=timedelta(week=1)) License ------- MIT subliminal-0.7.4/subliminal.egg-info/0000755000175000017500000000000012271547017020634 5ustar antoineantoine00000000000000subliminal-0.7.4/subliminal.egg-info/requires.txt0000644000175000017500000000023512271547017023234 0ustar antoineantoine00000000000000beautifulsoup4>=4.3.2 guessit>=0.6.2,<0.7 requests>=2.0.1 enzyme>=0.4.0 html5lib>=0.99 dogpile.cache>=0.5.2 babelfish>=0.4.0,<0.5 charade>=1.0.3 pysrt>=0.5.0subliminal-0.7.4/subliminal.egg-info/entry_points.txt0000644000175000017500000000125512271547017024135 0ustar antoineantoine00000000000000[subliminal.providers] opensubtitles = subliminal.providers.opensubtitles:OpenSubtitlesProvider tvsubtitles = subliminal.providers.tvsubtitles:TVsubtitlesProvider podnapisi = subliminal.providers.podnapisi:PodnapisiProvider addic7ed = subliminal.providers.addic7ed:Addic7edProvider bierdopje = subliminal.providers.bierdopje:BierDopjeProvider thesubdb = subliminal.providers.thesubdb:TheSubDBProvider [babelfish.language_converters] podnapisi = subliminal.converters.podnapisi:PodnapisiConverter addic7ed = subliminal.converters.addic7ed:Addic7edConverter tvsubtitles = subliminal.converters.tvsubtitles:TVsubtitlesConverter [console_scripts] subliminal = subliminal.cli:subliminal subliminal-0.7.4/subliminal.egg-info/dependency_links.txt0000644000175000017500000000000112271547017024702 0ustar antoineantoine00000000000000 subliminal-0.7.4/subliminal.egg-info/PKG-INFO0000644000175000017500000001700412271547017021733 0ustar antoineantoine00000000000000Metadata-Version: 1.1 Name: subliminal Version: 0.7.4 Summary: Subtitles, faster than your thoughts Home-page: https://github.com/Diaoul/subliminal Author: Antoine Bertin Author-email: diaoulael@gmail.com License: MIT Description: Subliminal ========== Subliminal is a python library to search and download subtitles. It comes with an easy to use CLI (command-line interface) suitable for direct use or cron jobs. .. image:: https://travis-ci.org/Diaoul/subliminal.png?branch=develop :target: https://travis-ci.org/Diaoul/subliminal .. image:: https://coveralls.io/repos/Diaoul/subliminal/badge.png?branch=develop :target: https://coveralls.io/r/Diaoul/subliminal?branch=develop Providers --------- Subliminal uses multiple providers to give users a vast choice and have a better chance to find the best matching subtitles. Providers are extensible through a dedicated entry point. * Addic7ed * BierDopje * OpenSubtitles * Podnapisi * TheSubDB * TvSubtitles Usage ----- CLI ^^^ Download english subtitles:: $ subliminal -l en -- The.Big.Bang.Theory.S05E18.HDTV.x264-LOL.mp4 1 subtitle downloaded Library ^^^^^^^ Download best subtitles in French and English for videos less than one week old in a video folder, skipping videos that already have subtitles whether they are embedded or not:: from babelfish import Language from datetime import timedelta import subliminal # configure the cache subliminal.cache_region.configure('dogpile.cache.dbm', arguments={'filename': '/path/to/cachefile.dbm'}) # scan for videos in the folder and their subtitles videos = subliminal.scan_videos(['/path/to/video/folder'], subtitles=True, embedded_subtitles=True, age=timedelta(weeks=1)) # download subliminal.download_best_subtitles(videos, {Language('eng'), Language('fra')}, age=timedelta(week=1)) License ------- MIT Changelog ========= 0.7.4 ----- **release date:** 2014-01-27 * Fix requirements for guessit and babelfish 0.7.3 ----- **release date:** 2013-11-22 * Fix windows compatibility * Improve subtitle validation * Improve embedded subtitle languages detection * Improve unittests 0.7.2 ----- **release date:** 2013-11-10 * Fix TVSubtitles for ambiguous series * Add a CACHE_VERSION to force cache reloading on version change * Set CLI default cache expiration time to 30 days * Add podnapisi provider * Support script for languages e.g. Latn, Cyrl * Improve logging levels * Fix subtitle validation in some rare cases 0.7.1 ----- **release date:** 2013-11-06 * Improve CLI * Add login support for Addic7ed * Remove lxml dependency * Many fixes 0.7.0 ----- **release date:** 2013-10-29 **WARNING:** Complete rewrite of subliminal with backward incompatible changes * Use enzyme to parse metadata of videos * Use babelfish to handle languages * Use dogpile.cache for caching * Use charade to detect subtitle encoding * Use pysrt for subtitle validation * Use entry points for subtitle providers * New subtitle score computation * Hearing impaired subtitles support * Drop async support * Drop a few providers * And much more... 0.6.2 ----- **release date:** 2012-09-15 * Fix BierDopje * Fix Addic7ed * Fix SubsWiki * Fix missing enzyme import * Add Catalan and Galician languages to Addic7ed * Add possible services in help message of the CLI * Allow existing filenames to be passed without the ./ prefix 0.6.1 ----- **release date:** 2012-06-24 * Fix subtitle release name in BierDopje * Fix subtitles being downloaded multiple times * Add Chinese support to TvSubtitles * Fix encoding issues * Fix single download subtitles without the force option * Add Spanish (Latin America) exception to Addic7ed * Fix group_by_video when a list entry has None as subtitles * Add support for Galician language in Subtitulos * Add an integrity check after subtitles download for Addic7ed * Add error handling for if not strict in Language * Fix TheSubDB hash method to return None if the file is too small * Fix guessit.Language in Video.scan * Fix language detection of subtitles 0.6.0 ----- **release date:** 2012-06-16 **WARNING:** Backward incompatible changes * Fix --workers option in CLI * Use a dedicated module for languages * Use beautifulsoup4 * Improve return types * Add scan_filter option * Add --age option in CLI * Add TvSubtitles service * Add Addic7ed service 0.5.1 ----- **release date:** 2012-03-25 * Improve error handling of enzyme parsing 0.5 --- **release date:** 2012-03-25 **WARNING:** Backward incompatible changes * Use more unicode * New list_subtitles and download_subtitles methods * New Pool object for asynchronous work * Improve sort algorithm * Better error handling * Make sorting customizable * Remove class Subliminal * Remove permissions handling 0.4 --- **release date:** 2011-11-11 * Many fixes * Better error handling 0.3 --- **release date:** 2011-08-18 * Fix a bug when series is not guessed by guessit * Fix dependencies failure when installing package * Fix encoding issues with logging * Add a script to ease subtitles download * Add possibility to choose mode of created files * Add more checks before adjusting permissions 0.2 --- **release date:** 2011-07-11 * Fix plugin configuration * Fix some encoding issues * Remove extra logging 0.1 --- **release date:** not released yet * Initial release Keywords: subtitle subtitles video movie episode tv show Platform: UNKNOWN Classifier: Development Status :: 4 - Beta Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: MIT License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.3 Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: Topic :: Multimedia :: Video subliminal-0.7.4/subliminal.egg-info/SOURCES.txt0000644000175000017500000000167712271547017022533 0ustar antoineantoine00000000000000HISTORY.rst LICENSE MANIFEST.in README.rst requirements.txt setup.cfg setup.py subliminal/__init__.py subliminal/api.py subliminal/cache.py subliminal/cli.py subliminal/exceptions.py subliminal/score.py subliminal/subtitle.py subliminal/video.py subliminal.egg-info/PKG-INFO subliminal.egg-info/SOURCES.txt subliminal.egg-info/dependency_links.txt subliminal.egg-info/entry_points.txt subliminal.egg-info/requires.txt subliminal.egg-info/top_level.txt subliminal/converters/__init__.py subliminal/converters/addic7ed.py subliminal/converters/podnapisi.py subliminal/converters/tvsubtitles.py subliminal/providers/__init__.py subliminal/providers/addic7ed.py subliminal/providers/bierdopje.py subliminal/providers/opensubtitles.py subliminal/providers/podnapisi.py subliminal/providers/thesubdb.py subliminal/providers/tvsubtitles.py subliminal/tests/__init__.py subliminal/tests/common.py subliminal/tests/test_providers.py subliminal/tests/test_subliminal.pysubliminal-0.7.4/subliminal.egg-info/top_level.txt0000644000175000017500000000001312271547017023360 0ustar antoineantoine00000000000000subliminal subliminal-0.7.4/setup.py0000644000175000017500000000416512271546743016530 0ustar antoineantoine00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- from setuptools import setup, find_packages setup(name='subliminal', version='0.7.4', license='MIT', description='Subtitles, faster than your thoughts', long_description=open('README.rst').read() + '\n\n' + open('HISTORY.rst').read(), keywords='subtitle subtitles video movie episode tv show', url='https://github.com/Diaoul/subliminal', author='Antoine Bertin', author_email='diaoulael@gmail.com', packages=find_packages(), classifiers=['Development Status :: 4 - Beta', 'Intended Audience :: Developers', 'License :: OSI Approved :: MIT License', 'Operating System :: OS Independent', 'Programming Language :: Python', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.3', 'Topic :: Software Development :: Libraries :: Python Modules', 'Topic :: Multimedia :: Video'], entry_points={ 'console_scripts': ['subliminal = subliminal.cli:subliminal'], 'subliminal.providers': ['addic7ed = subliminal.providers.addic7ed:Addic7edProvider', 'bierdopje = subliminal.providers.bierdopje:BierDopjeProvider', 'opensubtitles = subliminal.providers.opensubtitles:OpenSubtitlesProvider', 'podnapisi = subliminal.providers.podnapisi:PodnapisiProvider', 'thesubdb = subliminal.providers.thesubdb:TheSubDBProvider', 'tvsubtitles = subliminal.providers.tvsubtitles:TVsubtitlesProvider'], 'babelfish.language_converters': ['addic7ed = subliminal.converters.addic7ed:Addic7edConverter', 'podnapisi = subliminal.converters.podnapisi:PodnapisiConverter', 'tvsubtitles = subliminal.converters.tvsubtitles:TVsubtitlesConverter'] }, install_requires=open('requirements.txt').readlines(), test_suite='subliminal.tests.suite') subliminal-0.7.4/MANIFEST.in0000644000175000017500000000005512233717102016531 0ustar antoineantoine00000000000000include LICENSE HISTORY.rst requirements.txt subliminal-0.7.4/requirements.txt0000644000175000017500000000023612271546501020265 0ustar antoineantoine00000000000000beautifulsoup4>=4.3.2 guessit>=0.6.2,<0.7 requests>=2.0.1 enzyme>=0.4.0 html5lib>=0.99 dogpile.cache>=0.5.2 babelfish>=0.4.0,<0.5 charade>=1.0.3 pysrt>=0.5.0 subliminal-0.7.4/setup.cfg0000664000175000017500000000026312271547017016627 0ustar antoineantoine00000000000000[build_sphinx] source-dir = docs/ build-dir = docs/_build all_files = 1 [upload_sphinx] upload-dir = docs/_build/html [egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 subliminal-0.7.4/HISTORY.rst0000644000175000017500000000660512271546727016714 0ustar antoineantoine00000000000000Changelog ========= 0.7.4 ----- **release date:** 2014-01-27 * Fix requirements for guessit and babelfish 0.7.3 ----- **release date:** 2013-11-22 * Fix windows compatibility * Improve subtitle validation * Improve embedded subtitle languages detection * Improve unittests 0.7.2 ----- **release date:** 2013-11-10 * Fix TVSubtitles for ambiguous series * Add a CACHE_VERSION to force cache reloading on version change * Set CLI default cache expiration time to 30 days * Add podnapisi provider * Support script for languages e.g. Latn, Cyrl * Improve logging levels * Fix subtitle validation in some rare cases 0.7.1 ----- **release date:** 2013-11-06 * Improve CLI * Add login support for Addic7ed * Remove lxml dependency * Many fixes 0.7.0 ----- **release date:** 2013-10-29 **WARNING:** Complete rewrite of subliminal with backward incompatible changes * Use enzyme to parse metadata of videos * Use babelfish to handle languages * Use dogpile.cache for caching * Use charade to detect subtitle encoding * Use pysrt for subtitle validation * Use entry points for subtitle providers * New subtitle score computation * Hearing impaired subtitles support * Drop async support * Drop a few providers * And much more... 0.6.2 ----- **release date:** 2012-09-15 * Fix BierDopje * Fix Addic7ed * Fix SubsWiki * Fix missing enzyme import * Add Catalan and Galician languages to Addic7ed * Add possible services in help message of the CLI * Allow existing filenames to be passed without the ./ prefix 0.6.1 ----- **release date:** 2012-06-24 * Fix subtitle release name in BierDopje * Fix subtitles being downloaded multiple times * Add Chinese support to TvSubtitles * Fix encoding issues * Fix single download subtitles without the force option * Add Spanish (Latin America) exception to Addic7ed * Fix group_by_video when a list entry has None as subtitles * Add support for Galician language in Subtitulos * Add an integrity check after subtitles download for Addic7ed * Add error handling for if not strict in Language * Fix TheSubDB hash method to return None if the file is too small * Fix guessit.Language in Video.scan * Fix language detection of subtitles 0.6.0 ----- **release date:** 2012-06-16 **WARNING:** Backward incompatible changes * Fix --workers option in CLI * Use a dedicated module for languages * Use beautifulsoup4 * Improve return types * Add scan_filter option * Add --age option in CLI * Add TvSubtitles service * Add Addic7ed service 0.5.1 ----- **release date:** 2012-03-25 * Improve error handling of enzyme parsing 0.5 --- **release date:** 2012-03-25 **WARNING:** Backward incompatible changes * Use more unicode * New list_subtitles and download_subtitles methods * New Pool object for asynchronous work * Improve sort algorithm * Better error handling * Make sorting customizable * Remove class Subliminal * Remove permissions handling 0.4 --- **release date:** 2011-11-11 * Many fixes * Better error handling 0.3 --- **release date:** 2011-08-18 * Fix a bug when series is not guessed by guessit * Fix dependencies failure when installing package * Fix encoding issues with logging * Add a script to ease subtitles download * Add possibility to choose mode of created files * Add more checks before adjusting permissions 0.2 --- **release date:** 2011-07-11 * Fix plugin configuration * Fix some encoding issues * Remove extra logging 0.1 --- **release date:** not released yet * Initial release subliminal-0.7.4/PKG-INFO0000644000175000017500000001700412271547017016102 0ustar antoineantoine00000000000000Metadata-Version: 1.1 Name: subliminal Version: 0.7.4 Summary: Subtitles, faster than your thoughts Home-page: https://github.com/Diaoul/subliminal Author: Antoine Bertin Author-email: diaoulael@gmail.com License: MIT Description: Subliminal ========== Subliminal is a python library to search and download subtitles. It comes with an easy to use CLI (command-line interface) suitable for direct use or cron jobs. .. image:: https://travis-ci.org/Diaoul/subliminal.png?branch=develop :target: https://travis-ci.org/Diaoul/subliminal .. image:: https://coveralls.io/repos/Diaoul/subliminal/badge.png?branch=develop :target: https://coveralls.io/r/Diaoul/subliminal?branch=develop Providers --------- Subliminal uses multiple providers to give users a vast choice and have a better chance to find the best matching subtitles. Providers are extensible through a dedicated entry point. * Addic7ed * BierDopje * OpenSubtitles * Podnapisi * TheSubDB * TvSubtitles Usage ----- CLI ^^^ Download english subtitles:: $ subliminal -l en -- The.Big.Bang.Theory.S05E18.HDTV.x264-LOL.mp4 1 subtitle downloaded Library ^^^^^^^ Download best subtitles in French and English for videos less than one week old in a video folder, skipping videos that already have subtitles whether they are embedded or not:: from babelfish import Language from datetime import timedelta import subliminal # configure the cache subliminal.cache_region.configure('dogpile.cache.dbm', arguments={'filename': '/path/to/cachefile.dbm'}) # scan for videos in the folder and their subtitles videos = subliminal.scan_videos(['/path/to/video/folder'], subtitles=True, embedded_subtitles=True, age=timedelta(weeks=1)) # download subliminal.download_best_subtitles(videos, {Language('eng'), Language('fra')}, age=timedelta(week=1)) License ------- MIT Changelog ========= 0.7.4 ----- **release date:** 2014-01-27 * Fix requirements for guessit and babelfish 0.7.3 ----- **release date:** 2013-11-22 * Fix windows compatibility * Improve subtitle validation * Improve embedded subtitle languages detection * Improve unittests 0.7.2 ----- **release date:** 2013-11-10 * Fix TVSubtitles for ambiguous series * Add a CACHE_VERSION to force cache reloading on version change * Set CLI default cache expiration time to 30 days * Add podnapisi provider * Support script for languages e.g. Latn, Cyrl * Improve logging levels * Fix subtitle validation in some rare cases 0.7.1 ----- **release date:** 2013-11-06 * Improve CLI * Add login support for Addic7ed * Remove lxml dependency * Many fixes 0.7.0 ----- **release date:** 2013-10-29 **WARNING:** Complete rewrite of subliminal with backward incompatible changes * Use enzyme to parse metadata of videos * Use babelfish to handle languages * Use dogpile.cache for caching * Use charade to detect subtitle encoding * Use pysrt for subtitle validation * Use entry points for subtitle providers * New subtitle score computation * Hearing impaired subtitles support * Drop async support * Drop a few providers * And much more... 0.6.2 ----- **release date:** 2012-09-15 * Fix BierDopje * Fix Addic7ed * Fix SubsWiki * Fix missing enzyme import * Add Catalan and Galician languages to Addic7ed * Add possible services in help message of the CLI * Allow existing filenames to be passed without the ./ prefix 0.6.1 ----- **release date:** 2012-06-24 * Fix subtitle release name in BierDopje * Fix subtitles being downloaded multiple times * Add Chinese support to TvSubtitles * Fix encoding issues * Fix single download subtitles without the force option * Add Spanish (Latin America) exception to Addic7ed * Fix group_by_video when a list entry has None as subtitles * Add support for Galician language in Subtitulos * Add an integrity check after subtitles download for Addic7ed * Add error handling for if not strict in Language * Fix TheSubDB hash method to return None if the file is too small * Fix guessit.Language in Video.scan * Fix language detection of subtitles 0.6.0 ----- **release date:** 2012-06-16 **WARNING:** Backward incompatible changes * Fix --workers option in CLI * Use a dedicated module for languages * Use beautifulsoup4 * Improve return types * Add scan_filter option * Add --age option in CLI * Add TvSubtitles service * Add Addic7ed service 0.5.1 ----- **release date:** 2012-03-25 * Improve error handling of enzyme parsing 0.5 --- **release date:** 2012-03-25 **WARNING:** Backward incompatible changes * Use more unicode * New list_subtitles and download_subtitles methods * New Pool object for asynchronous work * Improve sort algorithm * Better error handling * Make sorting customizable * Remove class Subliminal * Remove permissions handling 0.4 --- **release date:** 2011-11-11 * Many fixes * Better error handling 0.3 --- **release date:** 2011-08-18 * Fix a bug when series is not guessed by guessit * Fix dependencies failure when installing package * Fix encoding issues with logging * Add a script to ease subtitles download * Add possibility to choose mode of created files * Add more checks before adjusting permissions 0.2 --- **release date:** 2011-07-11 * Fix plugin configuration * Fix some encoding issues * Remove extra logging 0.1 --- **release date:** not released yet * Initial release Keywords: subtitle subtitles video movie episode tv show Platform: UNKNOWN Classifier: Development Status :: 4 - Beta Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: MIT License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.3 Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: Topic :: Multimedia :: Video subliminal-0.7.4/LICENSE0000644000175000017500000000207112233717102016000 0ustar antoineantoine00000000000000The MIT License (MIT) Copyright (c) 2013 Antoine Bertin Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. subliminal-0.7.4/subliminal/0000755000175000017500000000000012271547017017142 5ustar antoineantoine00000000000000subliminal-0.7.4/subliminal/converters/0000755000175000017500000000000012271547017021334 5ustar antoineantoine00000000000000subliminal-0.7.4/subliminal/converters/addic7ed.py0000644000175000017500000000336112271546332023354 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals from babelfish import LanguageReverseConverter, get_language_converter class Addic7edConverter(LanguageReverseConverter): def __init__(self): self.name_converter = get_language_converter('name') self.from_addic7ed = {'CatalĂ ': ('cat',), 'Chinese (Simplified)': ('zho',), 'Chinese (Traditional)': ('zho',), 'Euskera': ('eus',), 'Galego': ('glg',), 'Greek': ('ell',), 'Malay': ('msa',), 'Portuguese (Brazilian)': ('por', 'BR'), 'Serbian (Cyrillic)': ('srp', None, 'Cyrl'), 'Serbian (Latin)': ('srp',), 'Spanish (Latin America)': ('spa',), 'Spanish (Spain)': ('spa',)} self.to_addic7ed = {('cat',): 'CatalĂ ', ('zho',): 'Chinese (Simplified)', ('eus',): 'Euskera', ('glg',): 'Galego', ('ell',): 'Greek', ('msa',): 'Malay', ('por', 'BR'): 'Portuguese (Brazilian)', ('srp', None, 'Cyrl'): 'Serbian (Cyrillic)'} self.codes = self.name_converter.codes | set(self.from_addic7ed.keys()) def convert(self, alpha3, country=None, script=None): if (alpha3, country, script) in self.to_addic7ed: return self.to_addic7ed[(alpha3, country, script)] if (alpha3, country) in self.to_addic7ed: return self.to_addic7ed[(alpha3, country)] if (alpha3,) in self.to_addic7ed: return self.to_addic7ed[(alpha3,)] return self.name_converter.convert(alpha3, country, script) def reverse(self, addic7ed): if addic7ed in self.from_addic7ed: return self.from_addic7ed[addic7ed] return self.name_converter.reverse(addic7ed) subliminal-0.7.4/subliminal/converters/__init__.py0000644000175000017500000000000012233717102023423 0ustar antoineantoine00000000000000subliminal-0.7.4/subliminal/converters/tvsubtitles.py0000644000175000017500000000216012271546332024274 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals from babelfish import LanguageReverseConverter, get_language_converter class TVsubtitlesConverter(LanguageReverseConverter): def __init__(self): self.alpha2_converter = get_language_converter('alpha2') self.from_tvsubtitles = {'br': ('por', 'BR'), 'ua': ('ukr',), 'gr': ('ell',), 'cn': ('zho',), 'jp': ('jpn',), 'cz': ('ces',)} self.to_tvsubtitles = {v: k for k, v in self.from_tvsubtitles} self.codes = self.alpha2_converter.codes | set(self.from_tvsubtitles.keys()) def convert(self, alpha3, country=None, script=None): if (alpha3, country) in self.to_tvsubtitles: return self.to_tvsubtitles[(alpha3, country)] if (alpha3,) in self.to_tvsubtitles: return self.to_tvsubtitles[(alpha3,)] return self.alpha2_converter.convert(alpha3, country, script) def reverse(self, tvsubtitles): if tvsubtitles in self.from_tvsubtitles: return self.from_tvsubtitles[tvsubtitles] return self.alpha2_converter.reverse(tvsubtitles) subliminal-0.7.4/subliminal/converters/podnapisi.py0000644000175000017500000000365012271170654023677 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals from babelfish import LanguageReverseConverter, LanguageConvertError, LanguageReverseError class PodnapisiConverter(LanguageReverseConverter): def __init__(self): self.from_podnapisi = {2: ('eng',), 28: ('spa',), 26: ('pol',), 36: ('srp',), 1: ('slv',), 38: ('hrv',), 9: ('ita',), 8: ('fra',), 48: ('por', 'BR'), 23: ('nld',), 12: ('ara',), 13: ('ron',), 33: ('bul',), 32: ('por',), 16: ('ell',), 15: ('hun',), 31: ('fin',), 30: ('tur',), 7: ('ces',), 25: ('swe',), 27: ('rus',), 24: ('dan',), 22: ('heb',), 51: ('vie',), 52: ('fas',), 5: ('deu',), 14: ('spa', 'AR'), 54: ('ind',), 47: ('srp', None, 'Cyrl'), 3: ('nor',), 20: ('est',), 10: ('bos',), 17: ('zho',), 37: ('slk',), 35: ('mkd',), 11: ('jpn',), 4: ('kor',), 29: ('sqi',), 6: ('isl',), 19: ('lit',), 46: ('ukr',), 44: ('tha',), 53: ('cat',), 56: ('sin',), 21: ('lav',), 40: ('cmn',), 55: ('msa',), 42: ('hin',), 50: ('bel',)} self.to_podnapisi = {v: k for k, v in self.from_podnapisi.items()} self.codes = set(self.from_podnapisi.keys()) def convert(self, alpha3, country=None, script=None): if (alpha3,) in self.to_podnapisi: return self.to_podnapisi[(alpha3,)] if (alpha3, country) in self.to_podnapisi: return self.to_podnapisi[(alpha3, country)] if (alpha3, country, script) in self.to_podnapisi: return self.to_podnapisi[(alpha3, country, script)] raise LanguageConvertError(alpha3, country, script) def reverse(self, podnapisi): if podnapisi not in self.from_podnapisi: raise LanguageReverseError(podnapisi) return self.from_podnapisi[podnapisi] subliminal-0.7.4/subliminal/__init__.py0000644000175000017500000000121212271546735021255 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- __title__ = 'subliminal' __version__ = '0.7.4' __author__ = 'Antoine Bertin' __license__ = 'MIT' __copyright__ = 'Copyright 2013 Antoine Bertin' import logging from .api import PROVIDERS_ENTRY_POINT, list_subtitles, download_subtitles, download_best_subtitles from .cache import MutexLock, region as cache_region from .exceptions import Error, ProviderError, ProviderConfigurationError, ProviderNotAvailable, InvalidSubtitle from .subtitle import Subtitle from .video import VIDEO_EXTENSIONS, SUBTITLE_EXTENSIONS, Video, Episode, Movie, scan_videos, scan_video logging.getLogger(__name__).addHandler(logging.NullHandler()) subliminal-0.7.4/subliminal/providers/0000755000175000017500000000000012271547017021157 5ustar antoineantoine00000000000000subliminal-0.7.4/subliminal/providers/addic7ed.py0000644000175000017500000001750212271546332023201 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals import logging import babelfish import bs4 import charade import requests from . import Provider from .. import __version__ from ..cache import region from ..exceptions import ProviderConfigurationError, ProviderNotAvailable, InvalidSubtitle from ..subtitle import Subtitle, is_valid_subtitle from ..video import Episode logger = logging.getLogger(__name__) class Addic7edSubtitle(Subtitle): provider_name = 'addic7ed' def __init__(self, language, series, season, episode, title, version, hearing_impaired, download_link, referer): super(Addic7edSubtitle, self).__init__(language, hearing_impaired) self.series = series self.season = season self.episode = episode self.title = title self.version = version self.download_link = download_link self.referer = referer def compute_matches(self, video): matches = set() # series if video.series and self.series == video.series: matches.add('series') # season if video.season and self.season == video.season: matches.add('season') # episode if video.episode and self.episode == video.episode: matches.add('episode') # title if video.title and self.title.lower() == video.title.lower(): matches.add('title') # release_group if video.release_group and self.version and video.release_group.lower() in self.version.lower(): matches.add('release_group') # resolution if video.resolution and self.version and video.resolution in self.version.lower(): matches.add('resolution') return matches class Addic7edProvider(Provider): languages = {babelfish.Language('por', 'BR')} | {babelfish.Language(l) for l in ['ara', 'aze', 'ben', 'bos', 'bul', 'cat', 'ces', 'dan', 'deu', 'ell', 'eng', 'eus', 'fas', 'fin', 'fra', 'glg', 'heb', 'hrv', 'hun', 'hye', 'ind', 'ita', 'jpn', 'kor', 'mkd', 'msa', 'nld', 'nor', 'pol', 'por', 'ron', 'rus', 'slk', 'slv', 'spa', 'sqi', 'srp', 'swe', 'tha', 'tur', 'ukr', 'vie', 'zho']} video_types = (Episode,) server = 'http://www.addic7ed.com' def __init__(self, username=None, password=None): if username is not None and password is None or username is None and password is not None: raise ProviderConfigurationError('Username and password must be specified') self.username = username self.password = password self.logged_in = False def initialize(self): self.session = requests.Session() self.session.headers = {'User-Agent': 'Subliminal/%s' % __version__} # login if self.username is not None and self.password is not None: logger.debug('Logging in') data = {'username': self.username, 'password': self.password, 'Submit': 'Log in'} try: r = self.session.post(self.server + '/dologin.php', data, timeout=10, allow_redirects=False) except requests.Timeout: raise ProviderNotAvailable('Timeout after 10 seconds') if r.status_code == 302: logger.info('Logged in') self.logged_in = True else: logger.error('Failed to login') def terminate(self): # logout if self.logged_in: try: r = self.session.get(self.server + '/logout.php', timeout=10) logger.info('Logged out') except requests.Timeout: raise ProviderNotAvailable('Timeout after 10 seconds') if r.status_code != 200: raise ProviderNotAvailable('Request failed with status code %d' % r.status_code) self.session.close() def get(self, url, params=None): """Make a GET request on `url` with the given parameters :param string url: part of the URL to reach with the leading slash :param params: params of the request :return: the response :rtype: :class:`bs4.BeautifulSoup` :raise: :class:`~subliminal.exceptions.ProviderNotAvailable` """ try: r = self.session.get(self.server + url, params=params, timeout=10) except requests.Timeout: raise ProviderNotAvailable('Timeout after 10 seconds') if r.status_code != 200: raise ProviderNotAvailable('Request failed with status code %d' % r.status_code) return bs4.BeautifulSoup(r.content, ['permissive']) @region.cache_on_arguments() def get_show_ids(self): """Load the shows page with default series to show ids mapping :return: series to show ids :rtype: dict """ soup = self.get('/shows.php') show_ids = {} for html_show in soup.select('td.version > h3 > a[href^="/show/"]'): show_ids[html_show.string.lower()] = int(html_show['href'][6:]) return show_ids @region.cache_on_arguments() def find_show_id(self, series): """Find a show id from the series Use this only if the series is not in the dict returned by :meth:`get_show_ids` :param string series: series of the episode :return: the show id, if any :rtype: int or None """ params = {'search': series, 'Submit': 'Search'} logger.debug('Searching series %r', params) suggested_shows = self.get('/search.php', params).select('span.titulo > a[href^="/show/"]') if not suggested_shows: logger.info('Series %r not found', series) return None return int(suggested_shows[0]['href'][6:]) def query(self, series, season): show_ids = self.get_show_ids() if series.lower() in show_ids: show_id = show_ids[series.lower()] else: show_id = self.find_show_id(series.lower()) if show_id is None: return [] params = {'show_id': show_id, 'season': season} logger.debug('Searching subtitles %r', params) link = '/show/{show_id}&season={season}'.format(**params) soup = self.get(link) subtitles = [] for row in soup('tr', class_='epeven completed'): cells = row('td') if cells[5].string != 'Completed': logger.debug('Skipping incomplete subtitle') continue if not cells[3].string: logger.debug('Skipping empty language') continue subtitles.append(Addic7edSubtitle(babelfish.Language.fromaddic7ed(cells[3].string), series, season, int(cells[1].string), cells[2].string, cells[4].string, bool(cells[6].string), cells[9].a['href'], link)) return subtitles def list_subtitles(self, video, languages): return [s for s in self.query(video.series, video.season) if s.language in languages and s.episode == video.episode] def download_subtitle(self, subtitle): try: r = self.session.get(self.server + subtitle.download_link, timeout=10, headers={'Referer': self.server + subtitle.referer}) except requests.Timeout: raise ProviderNotAvailable('Timeout after 10 seconds') if r.status_code != 200: raise ProviderNotAvailable('Request failed with status code %d' % r.status_code) if r.headers['Content-Type'] == 'text/html': raise ProviderNotAvailable('Download limit exceeded') subtitle_text = r.content.decode(charade.detect(r.content)['encoding'], 'replace') if not is_valid_subtitle(subtitle_text): raise InvalidSubtitle return subtitle_text subliminal-0.7.4/subliminal/providers/__init__.py0000644000175000017500000001101412271546332023264 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals import babelfish from ..video import Episode, Movie class Provider(object): """Base class for providers If any configuration is possible for the provider, like credentials, it must take place during instantiation :param \*\*kwargs: configuration :raise: :class:`~subliminal.exceptions.ProviderConfigurationError` if there is a configuration error """ #: Supported BabelFish languages languages = set() #: Supported video types video_types = (Episode, Movie) #: Required hash, if any required_hash = None def __init__(self, **kwargs): pass def __enter__(self): self.initialize() return self def __exit__(self, *args): self.terminate() def initialize(self): """Initialize the provider Must be called when starting to work with the provider. This is the place for network initialization or login operations. .. note: This is called automatically if you use the :keyword:`with` statement :raise: :class:`~subliminal.exceptions.ProviderNotAvailable` if the provider is unavailable """ pass def terminate(self): """Terminate the provider Must be called when done with the provider. This is the place for network shutdown or logout operations. .. note: This is called automatically if you use the :keyword:`with` statement :raise: :class:`~subliminal.exceptions.ProviderNotAvailable` if the provider is unavailable """ pass @classmethod def check(cls, video): """Check if the `video` can be processed The video is considered invalid if not an instance of :attr:`video_types` or if the :attr:`required_hash` is not present in :attr:`~subliminal.video.Video`'s `hashes` attribute. :param video: the video to check :type video: :class:`~subliminal.video.Video` :return: `True` if the `video` and `languages` are valid, `False` otherwise :rtype: bool """ if not isinstance(video, cls.video_types): return False if cls.required_hash is not None and cls.required_hash not in video.hashes: return False return True def query(self, languages, *args, **kwargs): """Query the provider for subtitles This method arguments match as much as possible the actual parameters for querying the provider :param languages: languages to search for :type languages: set of :class:`babelfish.Language` :param \*args: other required arguments :param \*\*kwargs: other optional arguments :return: the subtitles :rtype: list of :class:`~subliminal.subtitle.Subtitle` :raise: :class:`~subliminal.exceptions.ProviderNotAvailable` if the provider is unavailable :raise: :class:`~subliminal.exceptions.ProviderError` if something unexpected occured """ raise NotImplementedError def list_subtitles(self, video, languages): """List subtitles for the `video` with the given `languages` This is a proxy for the :meth:`query` method. The parameters passed to the :meth:`query` method may vary depending on the amount of information available in the `video` :param video: video to list subtitles for :type video: :class:`~subliminal.video.Video` :param languages: languages to search for :type languages: set of :class:`babelfish.Language` :return: the subtitles :rtype: list of :class:`~subliminal.subtitle.Subtitle` :raise: :class:`~subliminal.exceptions.ProviderNotAvailable` if the provider is unavailable :raise: :class:`~subliminal.exceptions.ProviderError` if something unexpected occured """ raise NotImplementedError def download_subtitle(self, subtitle): """Download the `subtitle` :param subtitle: subtitle to download :type subtitle: :class:`~subliminal.subtitle.Subtitle` :return: the subtitle text :rtype: string :raise: :class:`~subliminal.exceptions.ProviderNotAvailable` if the provider is unavailable :raise: :class:`~subliminal.exceptions.InvalidSubtitle` if the downloaded subtitle is invalid :raise: :class:`~subliminal.exceptions.ProviderError` if something unexpected occured """ raise NotImplementedError def __repr__(self): return '<%s [%r]>' % (self.__class__.__name__, self.video_types) subliminal-0.7.4/subliminal/providers/opensubtitles.py0000644000175000017500000001525312271546332024436 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals import base64 import logging import os import re import xmlrpclib import zlib import babelfish import charade import guessit from . import Provider from .. import __version__ from ..exceptions import ProviderError, ProviderNotAvailable, InvalidSubtitle from ..subtitle import Subtitle, is_valid_subtitle, compute_guess_matches from ..video import Episode, Movie logger = logging.getLogger(__name__) class OpenSubtitlesSubtitle(Subtitle): provider_name = 'opensubtitles' series_re = re.compile('^"(?P.*)" (?P.*)$') def __init__(self, language, hearing_impaired, id, matched_by, movie_kind, hash, movie_name, movie_release_name, # @ReservedAssignment movie_year, movie_imdb_id, series_season, series_episode): super(OpenSubtitlesSubtitle, self).__init__(language, hearing_impaired) self.id = id self.matched_by = matched_by self.movie_kind = movie_kind self.hash = hash self.movie_name = movie_name self.movie_release_name = movie_release_name self.movie_year = movie_year self.movie_imdb_id = movie_imdb_id self.series_season = series_season self.series_episode = series_episode @property def series_name(self): return self.series_re.match(self.movie_name).group('series_name') @property def series_title(self): return self.series_re.match(self.movie_name).group('series_title') def compute_matches(self, video): matches = set() # episode if isinstance(video, Episode) and self.movie_kind == 'episode': # series if video.series and self.series_name.lower() == video.series.lower(): matches.add('series') # season if video.season and self.series_season == video.season: matches.add('season') # episode if video.episode and self.series_episode == video.episode: matches.add('episode') # guess matches |= compute_guess_matches(video, guessit.guess_episode_info(self.movie_release_name + '.mkv')) # movie elif isinstance(video, Movie) and self.movie_kind == 'movie': # year if video.year and self.movie_year == video.year: matches.add('year') # guess matches |= compute_guess_matches(video, guessit.guess_movie_info(self.movie_release_name + '.mkv')) else: logger.info('%r is not a valid movie_kind for %r', self.movie_kind, video) return matches # hash if 'opensubtitles' in video.hashes and self.hash == video.hashes['opensubtitles']: matches.add('hash') # imdb_id if video.imdb_id and self.movie_imdb_id == video.imdb_id: matches.add('imdb_id') # title if video.title and self.movie_name.lower() == video.title.lower(): matches.add('title') return matches class OpenSubtitlesProvider(Provider): languages = {babelfish.Language.fromopensubtitles(l) for l in babelfish.get_language_converter('opensubtitles').codes} def __init__(self): self.server = xmlrpclib.ServerProxy('http://api.opensubtitles.org/xml-rpc') self.token = None def initialize(self): try: response = self.server.LogIn('', '', 'eng', 'subliminal v%s' % __version__) except xmlrpclib.ProtocolError: raise ProviderNotAvailable if response['status'] != '200 OK': raise ProviderError('Login failed with status %r' % response['status']) self.token = response['token'] def terminate(self): try: response = self.server.LogOut(self.token) except xmlrpclib.ProtocolError: raise ProviderNotAvailable if response['status'] != '200 OK': raise ProviderError('Logout failed with status %r' % response['status']) def query(self, languages, hash=None, size=None, imdb_id=None, query=None): # @ReservedAssignment searches = [] if hash and size: searches.append({'moviehash': hash, 'moviebytesize': str(size)}) if imdb_id: searches.append({'imdbid': imdb_id}) if query: searches.append({'query': query}) if not searches: raise ValueError('One or more parameter missing') for search in searches: search['sublanguageid'] = ','.join(l.opensubtitles for l in languages) logger.debug('Searching subtitles %r', searches) try: response = self.server.SearchSubtitles(self.token, searches) except xmlrpclib.ProtocolError: raise ProviderNotAvailable if response['status'] != '200 OK': raise ProviderError('Search failed with status %r' % response['status']) if not response['data']: logger.debug('No subtitle found') return [] return [OpenSubtitlesSubtitle(babelfish.Language.fromopensubtitles(r['SubLanguageID']), bool(int(r['SubHearingImpaired'])), r['IDSubtitleFile'], r['MatchedBy'], r['MovieKind'], r['MovieHash'], r['MovieName'], r['MovieReleaseName'], int(r['MovieYear']) if r['MovieYear'] else None, int(r['IDMovieImdb']), int(r['SeriesSeason']) if r['SeriesSeason'] else None, int(r['SeriesEpisode']) if r['SeriesEpisode'] else None) for r in response['data']] def list_subtitles(self, video, languages): query = None if ('opensubtitles' not in video.hashes or not video.size) and not video.imdb_id: query = video.name.split(os.sep)[-1] return self.query(languages, hash=video.hashes.get('opensubtitles'), size=video.size, imdb_id=video.imdb_id, query=query) def download_subtitle(self, subtitle): try: response = self.server.DownloadSubtitles(self.token, [subtitle.id]) except xmlrpclib.ProtocolError: raise ProviderNotAvailable if response['status'] != '200 OK': raise ProviderError('Download failed with status %r' % response['status']) if not response['data']: raise ProviderError('Nothing to download') subtitle_bytes = zlib.decompress(base64.b64decode(response['data'][0]['data']), 47) subtitle_text = subtitle_bytes.decode(charade.detect(subtitle_bytes)['encoding'], 'replace') if not is_valid_subtitle(subtitle_text): raise InvalidSubtitle return subtitle_text subliminal-0.7.4/subliminal/providers/tvsubtitles.py0000644000175000017500000001602712271546332024126 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals import io import logging import re import zipfile import babelfish import bs4 import charade import requests from . import Provider from .. import __version__ from ..cache import region from ..exceptions import InvalidSubtitle, ProviderNotAvailable, ProviderError from ..subtitle import Subtitle, is_valid_subtitle from ..video import Episode logger = logging.getLogger(__name__) class TVsubtitlesSubtitle(Subtitle): provider_name = 'tvsubtitles' def __init__(self, language, series, season, episode, id, rip, release): # @ReservedAssignment super(TVsubtitlesSubtitle, self).__init__(language) self.series = series self.season = season self.episode = episode self.id = id self.rip = rip self.release = release def compute_matches(self, video): matches = set() # series if video.series and self.series == video.series: matches.add('series') # season if video.season and self.season == video.season: matches.add('season') # episode if video.episode and self.episode == video.episode: matches.add('episode') # release_group if video.release_group and self.release and video.release_group.lower() in self.release.lower(): matches.add('release_group') # video_codec if video.video_codec and self.release and (video.video_codec in self.release.lower() or video.video_codec == 'h264' and 'x264' in self.release.lower()): matches.add('video_codec') # resolution if video.resolution and self.rip and video.resolution in self.rip.lower(): matches.add('resolution') return matches class TVsubtitlesProvider(Provider): languages = {babelfish.Language('por', 'BR')} | {babelfish.Language(l) for l in ['ara', 'bul', 'ces', 'dan', 'deu', 'ell', 'eng', 'fin', 'fra', 'hun', 'ita', 'jpn', 'kor', 'nld', 'pol', 'por', 'ron', 'rus', 'spa', 'swe', 'tur', 'ukr', 'zho']} video_types = (Episode,) server = 'http://www.tvsubtitles.net' episode_id_re = re.compile('^episode-\d+\.html$') subtitle_re = re.compile('^\/subtitle-\d+\.html$') link_re = re.compile('^(?P.+) \(\d{4}-\d{4}\)$') def initialize(self): self.session = requests.Session() self.session.headers = {'User-Agent': 'Subliminal/%s' % __version__} def terminate(self): self.session.close() def request(self, url, params=None, data=None, method='GET'): """Make a `method` request on `url` with the given parameters :param string url: part of the URL to reach with the leading slash :param dict params: params of the request :param dict data: data of the request :param string method: method of the request :return: the response :rtype: :class:`bs4.BeautifulSoup` :raise: :class:`~subliminal.exceptions.ProviderNotAvailable` """ try: r = self.session.request(method, self.server + url, params=params, data=data, timeout=10) except requests.Timeout: raise ProviderNotAvailable('Timeout after 10 seconds') if r.status_code != 200: raise ProviderNotAvailable('Request failed with status code %d' % r.status_code) return bs4.BeautifulSoup(r.content, ['permissive']) @region.cache_on_arguments() def find_show_id(self, series): """Find a show id from the series :param string series: series of the episode :return: the show id, if any :rtype: int or None """ data = {'q': series} logger.debug('Searching series %r', data) soup = self.request('/search.php', data=data, method='POST') links = soup.select('div.left li div a[href^="/tvshow-"]') if not links: logger.info('Series %r not found', series) return None for link in links: match = self.link_re.match(link.string) if not match: logger.warning('Could not parse %r', link.string) continue if match.group('series').lower().replace('.', ' ').strip() == series.lower(): return int(link['href'][8:-5]) return int(links[0]['href'][8:-5]) @region.cache_on_arguments() def find_episode_ids(self, show_id, season): """Find episode ids from the show id and the season :param int show_id: show id :param int season: season of the episode :return: episode ids per episode number :rtype: dict """ params = {'show_id': show_id, 'season': season} logger.debug('Searching episodes %r', params) soup = self.request('/tvshow-{show_id}-{season}.html'.format(**params)) episode_ids = {} for row in soup.select('table#table5 tr'): if not row('a', href=self.episode_id_re): continue cells = row('td') episode_ids[int(cells[0].string.split('x')[1])] = int(cells[1].a['href'][8:-5]) return episode_ids def query(self, series, season, episode): show_id = self.find_show_id(series.lower()) if show_id is None: return [] episode_ids = self.find_episode_ids(show_id, season) if episode not in episode_ids: logger.info('Episode %d not found', episode) return [] params = {'episode_id': episode_ids[episode]} logger.debug('Searching episode %r', params) soup = self.request('/episode-{episode_id}.html'.format(**params)) return [TVsubtitlesSubtitle(babelfish.Language.fromtvsubtitles(row.h5.img['src'][13:-4]), series, season, episode, row['href'][10:-5], row.find('p', title='rip').text.strip() or None, row.find('p', title='release').text.strip() or None) for row in soup('a', href=self.subtitle_re)] def list_subtitles(self, video, languages): return [s for s in self.query(video.series, video.season, video.episode) if s.language in languages] def download_subtitle(self, subtitle): try: r = self.session.get(self.server + '/download-{subtitle_id}.html'.format(subtitle_id=subtitle.id), timeout=10) except requests.Timeout: raise ProviderNotAvailable('Timeout after 10 seconds') if r.status_code != 200: raise ProviderNotAvailable('Request failed with status code %d' % r.status_code) with zipfile.ZipFile(io.BytesIO(r.content)) as zf: if len(zf.namelist()) > 1: raise ProviderError('More than one file to unzip') subtitle_bytes = zf.read(zf.namelist()[0]) subtitle_text = subtitle_bytes.decode(charade.detect(subtitle_bytes)['encoding'], 'replace') if not is_valid_subtitle(subtitle_text): raise InvalidSubtitle return subtitle_text subliminal-0.7.4/subliminal/providers/podnapisi.py0000644000175000017500000001556112271546332023526 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals import io import logging import re import xml.etree.ElementTree import zipfile import babelfish import bs4 import charade import guessit import requests from . import Provider from .. import __version__ from ..exceptions import InvalidSubtitle, ProviderNotAvailable, ProviderError from ..subtitle import Subtitle, is_valid_subtitle, compute_guess_matches from ..video import Episode, Movie logger = logging.getLogger(__name__) class PodnapisiSubtitle(Subtitle): provider_name = 'podnapisi' def __init__(self, language, id, releases, hearing_impaired, link, series=None, season=None, episode=None, # @ReservedAssignment title=None, year=None): super(PodnapisiSubtitle, self).__init__(language, hearing_impaired) self.id = id self.releases = releases self.hearing_impaired = hearing_impaired self.link = link self.series = series self.season = season self.episode = episode self.title = title self.year = year def compute_matches(self, video): matches = set() # episode if isinstance(video, Episode): # series if video.series and self.series.lower() == video.series.lower(): matches.add('series') # season if video.season and self.season == video.season: matches.add('season') # episode if video.episode and self.episode == video.episode: matches.add('episode') # guess for release in self.releases: matches |= compute_guess_matches(video, guessit.guess_episode_info(release + '.mkv')) # movie elif isinstance(video, Movie): # title if video.title and self.title.lower() == video.title.lower(): matches.add('title') # year if video.year and self.year == video.year: matches.add('year') # guess for release in self.releases: matches |= compute_guess_matches(video, guessit.guess_movie_info(release + '.mkv')) return matches class PodnapisiProvider(Provider): languages = {babelfish.Language.frompodnapisi(l) for l in babelfish.get_language_converter('podnapisi').codes} video_types = (Episode, Movie) server = 'http://simple.podnapisi.net' link_re = re.compile('^.*(?P/ppodnapisi/download/i/\d+/k/.*$)') def initialize(self): self.session = requests.Session() self.session.headers = {'User-Agent': 'Subliminal/%s' % __version__} def terminate(self): self.session.close() def get(self, url, params=None, is_xml=True): """Make a GET request on `url` with the given parameters :param string url: part of the URL to reach with the leading slash :param dict params: params of the request :param bool xml: whether the response content is XML or not :return: the response :rtype: :class:`xml.etree.ElementTree.Element` or :class:`bs4.BeautifulSoup` :raise: :class:`~subliminal.exceptions.ProviderNotAvailable` """ try: r = self.session.get(self.server + '/ppodnapisi' + url, params=params, timeout=10) except requests.Timeout: raise ProviderNotAvailable('Timeout after 10 seconds') if r.status_code != 200: raise ProviderNotAvailable('Request failed with status code %d' % r.status_code) if is_xml: return xml.etree.ElementTree.fromstring(r.content) else: return bs4.BeautifulSoup(r.content, ['permissive']) def query(self, language, series=None, season=None, episode=None, title=None, year=None): params = {'sXML': 1, 'sJ': language.podnapisi} if series and season and episode: params['sK'] = series params['sTS'] = season params['sTE'] = episode elif title: params['sK'] = title if year: params['sY'] = year else: raise ValueError('Missing parameters series and season and episode or title') logger.debug('Searching episode %r', params) subtitles = [] while True: root = self.get('/search', params) if not int(root.find('pagination/results').text): logger.debug('No subtitle found') break if series and season and episode: subtitles.extend([PodnapisiSubtitle(language, int(s.find('id').text), s.find('release').text.split(), 'h' in (s.find('flags').text or ''), s.find('url').text[38:], series=series, season=season, episode=episode) for s in root.findall('subtitle')]) elif title: subtitles.extend([PodnapisiSubtitle(language, int(s.find('id').text), s.find('release').text.split(), 'h' in (s.find('flags').text or ''), s.find('url').text[38:], title=title, year=year) for s in root.findall('subtitle')]) if int(root.find('pagination/current').text) >= int(root.find('pagination/count').text): break params['page'] = int(root.find('pagination/current').text) + 1 return subtitles def list_subtitles(self, video, languages): if isinstance(video, Episode): return [s for l in languages for s in self.query(l, series=video.series, season=video.season, episode=video.episode)] elif isinstance(video, Movie): return [s for l in languages for s in self.query(l, title=video.title, year=video.year)] def download_subtitle(self, subtitle): soup = self.get(subtitle.link, is_xml=False) link = soup.find('a', href=self.link_re) if not link: raise ProviderError('Cannot find the download link') try: r = self.session.get(self.server + self.link_re.match(link['href']).group('link'), timeout=10) except requests.Timeout: raise ProviderNotAvailable('Timeout after 10 seconds') if r.status_code != 200: raise ProviderNotAvailable('Request failed with status code %d' % r.status_code) with zipfile.ZipFile(io.BytesIO(r.content)) as zf: if len(zf.namelist()) > 1: raise ProviderError('More than one file to unzip') subtitle_bytes = zf.read(zf.namelist()[0]) subtitle_text = subtitle_bytes.decode(charade.detect(subtitle_bytes)['encoding'], 'replace') if not is_valid_subtitle(subtitle_text): raise InvalidSubtitle return subtitle_text subliminal-0.7.4/subliminal/providers/thesubdb.py0000644000175000017500000000561012271546332023332 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals import logging import babelfish import charade import requests from . import Provider from .. import __version__ from ..exceptions import InvalidSubtitle, ProviderNotAvailable, ProviderError from ..subtitle import Subtitle, is_valid_subtitle logger = logging.getLogger(__name__) class TheSubDBSubtitle(Subtitle): provider_name = 'thesubdb' def __init__(self, language, hash): # @ReservedAssignment super(TheSubDBSubtitle, self).__init__(language) self.hash = hash def compute_matches(self, video): matches = set() # hash if 'thesubdb' in video.hashes and video.hashes['thesubdb'] == self.hash: matches.add('hash') return matches class TheSubDBProvider(Provider): languages = {babelfish.Language.fromalpha2(l) for l in ['en', 'es', 'fr', 'it', 'nl', 'pl', 'pt', 'ro', 'sv', 'tr']} required_hash = 'thesubdb' def initialize(self): self.session = requests.Session() self.session.headers = {'User-Agent': 'SubDB/1.0 (subliminal/%s; https://github.com/Diaoul/subliminal)' % __version__} def terminate(self): self.session.close() def get(self, params): """Make a GET request on the server with the given parameters :param params: params of the request :return: the response :rtype: :class:`requests.Response` :raise: :class:`~subliminal.exceptions.ProviderNotAvailable` """ try: r = self.session.get('http://api.thesubdb.com', params=params, timeout=10) except requests.Timeout: raise ProviderNotAvailable('Timeout after 10 seconds') return r def query(self, hash): # @ReservedAssignment params = {'action': 'search', 'hash': hash} logger.debug('Searching subtitles %r', params) r = self.get(params) if r.status_code == 404: logger.debug('No subtitle found') return [] elif r.status_code != 200: raise ProviderError('Request failed with status code %d' % r.status_code) return [TheSubDBSubtitle(language, hash) for language in {babelfish.Language.fromalpha2(l) for l in r.content.split(',')}] def list_subtitles(self, video, languages): return [s for s in self.query(video.hashes['thesubdb']) if s.language in languages] def download_subtitle(self, subtitle): params = {'action': 'download', 'hash': subtitle.hash, 'language': subtitle.language.alpha2} r = self.get(params) if r.status_code != 200: raise ProviderError('Request failed with status code %d' % r.status_code) subtitle_text = r.content.decode(charade.detect(r.content)['encoding'], 'replace') if not is_valid_subtitle(subtitle_text): raise InvalidSubtitle return subtitle_text subliminal-0.7.4/subliminal/providers/bierdopje.py0000644000175000017500000001225712271546332023502 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals import logging import urllib import babelfish import charade import guessit import requests import xml.etree.ElementTree from . import Provider from .. import __version__ from ..cache import region from ..exceptions import InvalidSubtitle, ProviderNotAvailable, ProviderError from ..subtitle import Subtitle, is_valid_subtitle, compute_guess_matches from ..video import Episode logger = logging.getLogger(__name__) class BierDopjeSubtitle(Subtitle): provider_name = 'bierdopje' def __init__(self, language, season, episode, tvdb_id, series, filename, download_link): super(BierDopjeSubtitle, self).__init__(language) self.season = season self.episode = episode self.tvdb_id = tvdb_id self.series = series self.filename = filename self.download_link = download_link def compute_matches(self, video): matches = set() # tvdb_id if video.tvdb_id and self.tvdb_id == video.tvdb_id: matches.add('tvdb_id') # series if video.series and self.series == video.series: matches.add('series') # season if video.season and self.season == video.season: matches.add('season') # episode if video.episode and self.episode == video.episode: matches.add('episode') matches |= compute_guess_matches(video, guessit.guess_episode_info(self.filename + '.mkv')) return matches class BierDopjeProvider(Provider): languages = {babelfish.Language(l) for l in ['eng', 'nld']} video_types = (Episode,) def initialize(self): self.session = requests.Session() self.session.headers = {'User-Agent': 'Subliminal/%s' % __version__} def terminate(self): self.session.close() def get(self, url, **params): """Make a GET request on the `url` formatted with `**params` :param string url: API part of the URL to reach without the leading slash :param \*\*params: format specs for the `url` :return: the response :rtype: :class:`xml.etree.ElementTree.Element` :raise: :class:`~subliminal.exceptions.ProviderNotAvailable` """ try: r = self.session.get('http://api.bierdopje.com/A2B638AC5D804C2E/' + url.format(**params), timeout=10) except requests.Timeout: raise ProviderNotAvailable('Timeout after 10 seconds') if r.status_code == 429: raise ProviderNotAvailable('Too Many Requests') elif r.status_code != 200: raise ProviderError('Request failed with status code %d' % r.status_code) return xml.etree.ElementTree.fromstring(r.content) @region.cache_on_arguments() def find_show_id(self, series): """Find the show id from series name :param string series: series of the episode :return: show id :rtype: int """ logger.debug('Searching for series %r', series) root = self.get('FindShowByName/{series}', series=urllib.quote(series)) if root.find('response/status').text == 'false': logger.info('Series %r not found', series) return None return int(root.find('response/results/result[1]/showid').text) def query(self, language, season, episode, tvdb_id=None, series=None): params = {'language': language.alpha2, 'season': season, 'episode': episode} if tvdb_id is not None: params['showid'] = tvdb_id params['istvdbid'] = 'true' elif series is not None: show_id = self.find_show_id(series) if show_id is None: return [] params['showid'] = show_id params['istvdbid'] = 'false' else: raise ValueError('Missing parameter tvdb_id or series') logger.debug('Searching subtitles %r', params) root = self.get('GetAllSubsFor/{showid}/{season}/{episode}/{language}/{istvdbid}', **params) if root.find('response/status').text == 'false': logger.debug('No subtitle found') return [] logger.debug('Found subtitles %r', root.find('response/results')) return [BierDopjeSubtitle(language, season, episode, tvdb_id, series, result.find('filename').text, result.find('downloadlink').text) for result in root.find('response/results')] def list_subtitles(self, video, languages): return [s for l in languages for s in self.query(l, video.season, video.episode, video.tvdb_id, video.series)] def download_subtitle(self, subtitle): try: r = self.session.get(subtitle.download_link, timeout=10) except requests.Timeout: raise ProviderNotAvailable('Timeout after 10 seconds') if r.status_code == 429: raise ProviderNotAvailable('Too Many Requests') elif r.status_code != 200: raise ProviderError('Request failed with status code %d' % r.status_code) subtitle_text = r.content.decode(charade.detect(r.content)['encoding'], 'replace') if not is_valid_subtitle(subtitle_text): raise InvalidSubtitle return subtitle_text subliminal-0.7.4/subliminal/tests/0000755000175000017500000000000012271547017020304 5ustar antoineantoine00000000000000subliminal-0.7.4/subliminal/tests/__init__.py0000644000175000017500000000067112243734454022423 0ustar antoineantoine00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- from __future__ import unicode_literals from unittest import TextTestRunner, TestSuite from subliminal import cache_region from . import test_providers, test_subliminal cache_region.configure('dogpile.cache.memory', expiration_time=60 * 30) # @UndefinedVariable suite = TestSuite([test_providers.suite(), test_subliminal.suite()]) if __name__ == '__main__': TextTestRunner().run(suite) subliminal-0.7.4/subliminal/tests/test_subliminal.py0000644000175000017500000001757012271546332024065 0ustar antoineantoine00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- from __future__ import unicode_literals import os import shutil from unittest import TestCase, TestSuite, TestLoader, TextTestRunner from babelfish import Language from subliminal import list_subtitles, download_subtitles, download_best_subtitles, scan_video from subliminal.tests.common import MOVIES, EPISODES TEST_DIR = 'test_data' class ApiTestCase(TestCase): def setUp(self): os.mkdir(TEST_DIR) def tearDown(self): shutil.rmtree(TEST_DIR) def test_list_subtitles_movie_0(self): videos = [MOVIES[0]] languages = {Language('eng')} subtitles = list_subtitles(videos, languages) self.assertEqual(len(subtitles), len(videos)) self.assertGreater(len(subtitles[videos[0]]), 0) def test_list_subtitles_movie_0_por_br(self): videos = [MOVIES[0]] languages = {Language('por', 'BR')} subtitles = list_subtitles(videos, languages) self.assertEqual(len(subtitles), len(videos)) self.assertGreater(len(subtitles[videos[0]]), 0) def test_list_subtitles_episodes(self): videos = [EPISODES[0], EPISODES[1]] languages = {Language('eng'), Language('fra')} subtitles = list_subtitles(videos, languages) self.assertEqual(len(subtitles), len(videos)) self.assertGreater(len(subtitles[videos[0]]), 0) def test_download_subtitles(self): videos = [EPISODES[0], EPISODES[1]] for video in videos: video.name = os.path.join(TEST_DIR, video.name.split(os.sep)[-1]) languages = {Language('eng'), Language('fra')} subtitles = list_subtitles(videos, languages) download_subtitles(subtitles) for video in videos: self.assertTrue(os.path.exists(os.path.splitext(video.name)[0] + '.en.srt')) self.assertTrue(os.path.exists(os.path.splitext(video.name)[0] + '.fr.srt')) def test_download_subtitles_single(self): videos = [EPISODES[0], EPISODES[1]] for video in videos: video.name = os.path.join(TEST_DIR, video.name.split(os.sep)[-1]) languages = {Language('eng'), Language('fra')} subtitles = list_subtitles(videos, languages) download_subtitles(subtitles, single=True) for video in videos: self.assertTrue(os.path.exists(os.path.splitext(video.name)[0] + '.srt')) def test_download_best_subtitles(self): videos = [EPISODES[0], EPISODES[1]] for video in videos: video.name = os.path.join(TEST_DIR, video.name.split(os.sep)[-1]) languages = {Language('eng'), Language('fra')} subtitles = download_best_subtitles(videos, languages) for video in videos: self.assertEqual(video in subtitles and len(subtitles[video]), 2) self.assertTrue(os.path.exists(os.path.splitext(video.name)[0] + '.en.srt')) self.assertTrue(os.path.exists(os.path.splitext(video.name)[0] + '.fr.srt')) def test_download_best_subtitles_single(self): videos = [EPISODES[0], EPISODES[1]] for video in videos: video.name = os.path.join(TEST_DIR, video.name.split(os.sep)[-1]) languages = {Language('eng'), Language('fra')} subtitles = download_best_subtitles(videos, languages, single=True) for video in videos: self.assertIn(video, subtitles) self.assertEqual(len(subtitles[video]), 1) self.assertTrue(os.path.exists(os.path.splitext(video.name)[0] + '.srt')) def test_download_best_subtitles_min_score(self): videos = [MOVIES[0]] for video in videos: video.name = os.path.join(TEST_DIR, video.name.split(os.sep)[-1]) languages = {Language('eng'), Language('fra')} subtitles = download_best_subtitles(videos, languages, min_score=1000) self.assertEqual(len(subtitles), 0) def test_download_best_subtitles_hearing_impaired(self): videos = [MOVIES[0]] for video in videos: video.name = os.path.join(TEST_DIR, video.name.split(os.sep)[-1]) languages = {Language('eng')} subtitles = download_best_subtitles(videos, languages, hearing_impaired=True) self.assertTrue(subtitles[videos[0]][0].hearing_impaired) class VideoTestCase(TestCase): def setUp(self): os.mkdir(TEST_DIR) for video in MOVIES + EPISODES: open(os.path.join(TEST_DIR, os.path.split(video.name)[1]), 'w').close() def tearDown(self): shutil.rmtree(TEST_DIR) def test_scan_video_movie(self): video = MOVIES[0] scanned_video = scan_video(os.path.join(TEST_DIR, os.path.split(video.name)[1])) self.assertEqual(scanned_video.name, os.path.join(TEST_DIR, os.path.split(video.name)[1])) self.assertEqual(scanned_video.title.lower(), video.title.lower()) self.assertEqual(scanned_video.year, video.year) self.assertEqual(scanned_video.video_codec, video.video_codec) self.assertEqual(scanned_video.resolution, video.resolution) self.assertEqual(scanned_video.release_group, video.release_group) self.assertEqual(scanned_video.subtitle_languages, set()) self.assertEqual(scanned_video.hashes, {}) self.assertIsNone(scanned_video.audio_codec) self.assertIsNone(scanned_video.imdb_id) self.assertEqual(scanned_video.size, 0) def test_scan_video_episode(self): video = EPISODES[0] scanned_video = scan_video(os.path.join(TEST_DIR, os.path.split(video.name)[1])) self.assertEqual(scanned_video.name, os.path.join(TEST_DIR, os.path.split(video.name)[1])) self.assertEqual(scanned_video.series, video.series) self.assertEqual(scanned_video.season, video.season) self.assertEqual(scanned_video.episode, video.episode) self.assertEqual(scanned_video.video_codec, video.video_codec) self.assertEqual(scanned_video.resolution, video.resolution) self.assertEqual(scanned_video.release_group, video.release_group) self.assertEqual(scanned_video.subtitle_languages, set()) self.assertEqual(scanned_video.hashes, {}) self.assertIsNone(scanned_video.title) self.assertIsNone(scanned_video.tvdb_id) self.assertIsNone(scanned_video.imdb_id) self.assertIsNone(scanned_video.audio_codec) self.assertEqual(scanned_video.size, 0) def test_scan_video_subtitle_language_und(self): video = EPISODES[0] open(os.path.join(TEST_DIR, os.path.splitext(os.path.split(video.name)[1])[0]) + '.srt', 'w').close() scanned_video = scan_video(os.path.join(TEST_DIR, os.path.split(video.name)[1])) self.assertEqual(scanned_video.subtitle_languages, {Language('und')}) def test_scan_video_subtitles_language_eng(self): video = EPISODES[0] open(os.path.join(TEST_DIR, os.path.splitext(os.path.split(video.name)[1])[0]) + '.en.srt', 'w').close() scanned_video = scan_video(os.path.join(TEST_DIR, os.path.split(video.name)[1])) self.assertEqual(scanned_video.subtitle_languages, {Language('eng')}) def test_scan_video_subtitles_languages(self): video = EPISODES[0] open(os.path.join(TEST_DIR, os.path.splitext(os.path.split(video.name)[1])[0]) + '.en.srt', 'w').close() open(os.path.join(TEST_DIR, os.path.splitext(os.path.split(video.name)[1])[0]) + '.fr.srt', 'w').close() open(os.path.join(TEST_DIR, os.path.splitext(os.path.split(video.name)[1])[0]) + '.srt', 'w').close() scanned_video = scan_video(os.path.join(TEST_DIR, os.path.split(video.name)[1])) self.assertEqual(scanned_video.subtitle_languages, {Language('eng'), Language('fra'), Language('und')}) def suite(): suite = TestSuite() suite.addTest(TestLoader().loadTestsFromTestCase(ApiTestCase)) suite.addTest(TestLoader().loadTestsFromTestCase(VideoTestCase)) return suite if __name__ == '__main__': TextTestRunner().run(suite()) subliminal-0.7.4/subliminal/tests/common.py0000644000175000017500000000255512271546332022154 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals from subliminal import Movie, Episode MOVIES = [Movie('Man of Steel (2013)/man.of.steel.2013.720p.bluray.x264-felony.mkv', 'Man of Steel', release_group='felony', resolution='720p', video_codec='h264', audio_codec='DTS', imdb_id=770828, size=7033732714, year=2013, hashes={'opensubtitles': '5b8f8f4e41ccb21e', 'thesubdb': 'ad32876133355929d814457537e12dc2'})] EPISODES = [Episode('The Big Bang Theory/Season 07/The.Big.Bang.Theory.S07E05.720p.HDTV.X264-DIMENSION.mkv', 'The Big Bang Theory', 7, 5, release_group='DIMENSION', resolution='720p', video_codec='h264', audio_codec='AC3', imdb_id=3229392, size=501910737, title='The Workplace Proximity', tvdb_id=80379, hashes={'opensubtitles': '6878b3ef7c1bd19e', 'thesubdb': '9dbbfb7ba81c9a6237237dae8589fccc'}), Episode('Game of Thrones/Season 03/Game.of.Thrones.S03E10.Mhysa.720p.WEB-DL.DD5.1.H.264-NTb.mkv', 'Game of Thrones', 3, 10, release_group='NTb', resolution='720p', video_codec='h264', audio_codec='AC3', imdb_id=2178796, size=2142810931, title='Mhysa', tvdb_id=121361, hashes={'opensubtitles': 'b850baa096976c22', 'thesubdb': 'b1f899c77f4c960b84b8dbf840d4e42d'})] subliminal-0.7.4/subliminal/tests/test_providers.py0000644000175000017500000006012712271546332023737 0ustar antoineantoine00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- from __future__ import unicode_literals import os from unittest import TestCase, TestSuite, TestLoader, TextTestRunner from babelfish import Language from pkg_resources import iter_entry_points from subliminal import PROVIDERS_ENTRY_POINT from subliminal.subtitle import is_valid_subtitle from subliminal.tests.common import MOVIES, EPISODES class ProviderTestCase(TestCase): provider_name = '' def setUp(self): for provider_entry_point in iter_entry_points(PROVIDERS_ENTRY_POINT, self.provider_name): self.Provider = provider_entry_point.load() break class Addic7edProviderTestCase(ProviderTestCase): provider_name = 'addic7ed' def test_find_show_id(self): with self.Provider() as provider: show_id = provider.find_show_id('The Big Bang') self.assertEqual(show_id, 126) def test_find_show_id_error(self): with self.Provider() as provider: show_id = provider.find_show_id('the big how i met your mother') self.assertIsNone(show_id) def test_get_show_ids(self): with self.Provider() as provider: show_ids = provider.get_show_ids() self.assertIn('the big bang theory', show_ids) self.assertEqual(show_ids['the big bang theory'], 126) def test_query_episode_0(self): video = EPISODES[0] languages = {Language('tur'), Language('rus'), Language('heb'), Language('ita'), Language('fra'), Language('ron'), Language('nld'), Language('eng'), Language('deu'), Language('ell'), Language('por', 'BR'), Language('bul')} matches = {frozenset(['episode', 'release_group', 'title', 'series', 'resolution', 'season']), frozenset(['series', 'resolution', 'season']), frozenset(['series', 'episode', 'season', 'title']), frozenset(['series', 'release_group', 'season']), frozenset(['series', 'episode', 'season', 'release_group', 'title']), frozenset(['series', 'season'])} with self.Provider() as provider: subtitles = provider.query(video.series, video.season) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_query_episode_1(self): video = EPISODES[1] languages = {Language('ind'), Language('spa'), Language('hrv'), Language('ita'), Language('fra'), Language('cat'), Language('ell'), Language('nld'), Language('eng'), Language('fas'), Language('por'), Language('nor'), Language('deu'), Language('ron'), Language('por', 'BR'), Language('bul')} matches = {frozenset(['series', 'episode', 'resolution', 'season', 'title']), frozenset(['series', 'resolution', 'season']), frozenset(['series', 'episode', 'season', 'title']), frozenset(['series', 'release_group', 'season']), frozenset(['series', 'resolution', 'release_group', 'season']), frozenset(['series', 'episode', 'season', 'release_group', 'title']), frozenset(['series', 'season'])} with self.Provider() as provider: subtitles = provider.query(video.series, video.season) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_list_subtitles(self): video = EPISODES[0] languages = {Language('eng'), Language('fra')} matches = {frozenset(['series', 'episode', 'season', 'release_group', 'title']), frozenset(['series', 'episode', 'season', 'title'])} with self.Provider() as provider: subtitles = provider.list_subtitles(video, languages) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_download_subtitle(self): video = EPISODES[0] languages = {Language('eng'), Language('fra')} with self.Provider() as provider: subtitles = provider.list_subtitles(video, languages) subtitle_text = provider.download_subtitle(subtitles[0]) self.assertTrue(is_valid_subtitle(subtitle_text)) class BierDopjeProviderTestCase(ProviderTestCase): provider_name = 'bierdopje' def test_find_show_id(self): with self.Provider() as provider: show_id = provider.find_show_id('The Big Bang') self.assertEqual(show_id, 9203) def test_find_show_id_error(self): with self.Provider() as provider: show_id = provider.find_show_id('the big how i met your mother') self.assertIsNone(show_id) def test_query_episode_0(self): video = EPISODES[0] language = Language('eng') matches = {frozenset(['series', 'video_codec', 'resolution', 'episode', 'season']), frozenset(['season', 'video_codec', 'episode', 'series']), frozenset(['episode', 'video_codec', 'season', 'series', 'resolution', 'release_group'])} with self.Provider() as provider: subtitles = provider.query(language, video.season, video.episode, series=video.series) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, {language}) def test_query_episode_1(self): video = EPISODES[1] language = Language('nld') matches = {frozenset(['series', 'video_codec', 'resolution', 'episode', 'season']), frozenset(['season', 'video_codec', 'episode', 'series']), frozenset(['series', 'episode', 'season']), frozenset(['season', 'video_codec', 'episode', 'release_group', 'series']), frozenset(['episode', 'video_codec', 'season', 'series', 'resolution', 'release_group'])} with self.Provider() as provider: subtitles = provider.query(language, video.season, video.episode, series=video.series) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, {language}) def test_query_episode_0_tvdb_id(self): video = EPISODES[0] language = Language('eng') matches = {frozenset(['video_codec', 'tvdb_id', 'episode', 'season', 'series']), frozenset(['episode', 'video_codec', 'series', 'season', 'tvdb_id', 'resolution', 'release_group']), frozenset(['episode', 'series', 'video_codec', 'tvdb_id', 'resolution', 'season'])} with self.Provider() as provider: subtitles = provider.query(language, video.season, video.episode, tvdb_id=video.tvdb_id) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, {language}) def test_list_subtitles(self): video = EPISODES[1] languages = {Language('eng'), Language('nld')} matches = {frozenset(['series', 'video_codec', 'tvdb_id', 'episode', 'season']), frozenset(['episode', 'video_codec', 'season', 'series', 'tvdb_id', 'resolution', 'release_group']), frozenset(['season', 'tvdb_id', 'episode', 'series']), frozenset(['episode', 'video_codec', 'season', 'series', 'tvdb_id', 'resolution']), frozenset(['episode', 'video_codec', 'season', 'series', 'tvdb_id', 'release_group'])} with self.Provider() as provider: subtitles = provider.list_subtitles(video, languages) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_download_subtitle(self): video = EPISODES[0] languages = {Language('eng'), Language('nld')} with self.Provider() as provider: subtitles = provider.list_subtitles(video, languages) subtitle_text = provider.download_subtitle(subtitles[0]) self.assertTrue(is_valid_subtitle(subtitle_text)) class OpenSubtitlesProviderTestCase(ProviderTestCase): provider_name = 'opensubtitles' def test_query_movie_0_query(self): video = MOVIES[0] languages = {Language('eng')} matches = {frozenset([]), frozenset(['imdb_id', 'resolution', 'title', 'year']), frozenset(['imdb_id', 'title', 'year']), frozenset(['imdb_id', 'video_codec', 'title', 'year']), frozenset(['imdb_id', 'resolution', 'title', 'video_codec', 'year']), frozenset(['imdb_id', 'title', 'year', 'video_codec', 'resolution', 'release_group'])} with self.Provider() as provider: subtitles = provider.query(languages, query=video.title) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_query_episode_0_query(self): video = EPISODES[0] languages = {Language('eng')} matches = {frozenset(['series', 'episode', 'season', 'imdb_id']), frozenset(['series', 'imdb_id', 'video_codec', 'episode', 'season']), frozenset(['episode', 'title', 'series', 'imdb_id', 'video_codec', 'season'])} with self.Provider() as provider: subtitles = provider.query(languages, query=video.name.split(os.sep)[-1]) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_query_episode_1_query(self): video = EPISODES[1] languages = {Language('eng'), Language('fra')} matches = {frozenset(['episode', 'title', 'series', 'imdb_id', 'video_codec', 'season']), frozenset(['series', 'imdb_id', 'title', 'episode', 'season']), frozenset(['series', 'imdb_id', 'video_codec', 'episode', 'season']), frozenset(['episode', 'video_codec', 'series', 'imdb_id', 'resolution', 'season']), frozenset(['series', 'imdb_id', 'resolution', 'episode', 'season']), frozenset(['series', 'episode', 'season', 'imdb_id'])} with self.Provider() as provider: subtitles = provider.query(languages, query=video.name.split(os.sep)[-1]) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_query_movie_0_imdb_id(self): video = MOVIES[0] languages = {Language('eng'), Language('fra')} matches = {frozenset(['imdb_id', 'video_codec', 'title', 'year']), frozenset(['imdb_id', 'resolution', 'title', 'video_codec', 'year']), frozenset(['imdb_id', 'title', 'year', 'video_codec', 'resolution', 'release_group']), frozenset(['imdb_id', 'title', 'year']), frozenset(['imdb_id', 'resolution', 'title', 'year'])} with self.Provider() as provider: subtitles = provider.query(languages, imdb_id=video.imdb_id) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_query_episode_0_imdb_id(self): video = EPISODES[0] languages = {Language('eng'), Language('fra')} matches = {frozenset(['series', 'episode', 'season', 'imdb_id']), frozenset(['episode', 'release_group', 'video_codec', 'series', 'imdb_id', 'resolution', 'season']), frozenset(['series', 'imdb_id', 'video_codec', 'episode', 'season']), frozenset(['episode', 'title', 'series', 'imdb_id', 'video_codec', 'season'])} with self.Provider() as provider: subtitles = provider.query(languages, imdb_id=video.imdb_id) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_query_movie_0_hash(self): video = MOVIES[0] languages = {Language('eng')} matches = {frozenset(['hash', 'title', 'video_codec', 'year', 'resolution', 'imdb_id']), frozenset(['hash', 'title', 'video_codec', 'year', 'resolution', 'release_group', 'imdb_id']), frozenset(['year', 'video_codec', 'imdb_id', 'hash', 'title']), frozenset([]), frozenset(['year', 'resolution', 'imdb_id', 'hash', 'title']), frozenset(['year', 'imdb_id', 'hash', 'title'])} with self.Provider() as provider: subtitles = provider.query(languages, hash=video.hashes['opensubtitles'], size=video.size) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_query_episode_0_hash(self): video = EPISODES[0] languages = {Language('eng')} matches = {frozenset(['series', 'hash']), frozenset(['episode', 'season', 'series', 'imdb_id', 'video_codec', 'hash']), frozenset(['series', 'episode', 'season', 'hash', 'imdb_id']), frozenset(['series', 'resolution', 'hash', 'video_codec'])} with self.Provider() as provider: subtitles = provider.query(languages, hash=video.hashes['opensubtitles'], size=video.size) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_list_subtitles(self): video = MOVIES[0] languages = {Language('eng'), Language('fra')} matches = {frozenset(['title', 'video_codec', 'year', 'resolution', 'release_group', 'imdb_id']), frozenset(['imdb_id', 'year', 'title']), frozenset(['year', 'video_codec', 'imdb_id', 'resolution', 'title']), frozenset(['hash', 'title', 'video_codec', 'year', 'resolution', 'release_group', 'imdb_id']), frozenset(['year', 'video_codec', 'imdb_id', 'hash', 'title']), frozenset([]), frozenset(['year', 'resolution', 'imdb_id', 'hash', 'title']), frozenset(['hash', 'title', 'video_codec', 'year', 'resolution', 'imdb_id']), frozenset(['year', 'imdb_id', 'hash', 'title']), frozenset(['video_codec', 'imdb_id', 'year', 'title']), frozenset(['year', 'imdb_id', 'resolution', 'title'])} with self.Provider() as provider: subtitles = provider.list_subtitles(video, languages) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_download_subtitle(self): video = MOVIES[0] languages = {Language('eng'), Language('fra')} with self.Provider() as provider: subtitles = provider.list_subtitles(video, languages) subtitle_text = provider.download_subtitle(subtitles[0]) self.assertTrue(is_valid_subtitle(subtitle_text)) class PodnapisiProviderTestCase(ProviderTestCase): provider_name = 'podnapisi' def test_query_movie_0(self): video = MOVIES[0] language = Language('eng') matches = {frozenset(['video_codec', 'title', 'resolution', 'year']), frozenset(['title', 'resolution', 'year']), frozenset(['video_codec', 'title', 'year']), frozenset(['title', 'year']), frozenset(['video_codec', 'title', 'resolution', 'release_group', 'year']), frozenset(['video_codec', 'title', 'resolution', 'audio_codec', 'year'])} with self.Provider() as provider: subtitles = provider.query(language, title=video.title, year=video.year) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, {language}) def test_query_episode_0(self): video = EPISODES[0] language = Language('eng') matches = {frozenset(['episode', 'series', 'season', 'video_codec', 'resolution', 'release_group']), frozenset(['season', 'video_codec', 'episode', 'resolution', 'series'])} with self.Provider() as provider: subtitles = provider.query(language, series=video.series, season=video.season, episode=video.episode) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, {language}) def test_list_subtitles(self): video = MOVIES[0] languages = {Language('eng'), Language('fra')} matches = {frozenset(['video_codec', 'title', 'resolution', 'year']), frozenset(['title', 'resolution', 'year']), frozenset(['video_codec', 'title', 'year']), frozenset(['title', 'year']), frozenset(['video_codec', 'title', 'resolution', 'release_group', 'year']), frozenset(['video_codec', 'title', 'resolution', 'audio_codec', 'year'])} with self.Provider() as provider: subtitles = provider.list_subtitles(video, languages) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_download_subtitle(self): video = MOVIES[0] languages = {Language('eng'), Language('fra')} with self.Provider() as provider: subtitles = provider.list_subtitles(video, languages) subtitle_text = provider.download_subtitle(subtitles[0]) self.assertTrue(is_valid_subtitle(subtitle_text)) class TheSubDBProviderTestCase(ProviderTestCase): provider_name = 'thesubdb' def test_query_episode_0(self): video = EPISODES[0] languages = {Language('eng'), Language('spa'), Language('por')} matches = {frozenset(['hash'])} with self.Provider() as provider: subtitles = provider.query(video.hashes['thesubdb']) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_query_episode_1(self): video = EPISODES[1] languages = {Language('eng'), Language('por')} matches = {frozenset(['hash'])} with self.Provider() as provider: subtitles = provider.query(video.hashes['thesubdb']) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_list_subtitles(self): video = MOVIES[0] languages = {Language('eng'), Language('por')} matches = {frozenset(['hash'])} with self.Provider() as provider: subtitles = provider.list_subtitles(video, languages) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_download_subtitle(self): video = MOVIES[0] languages = {Language('eng'), Language('por')} with self.Provider() as provider: subtitles = provider.list_subtitles(video, languages) subtitle_text = provider.download_subtitle(subtitles[0]) self.assertTrue(is_valid_subtitle(subtitle_text)) class TVsubtitlesProviderTestCase(ProviderTestCase): provider_name = 'tvsubtitles' def test_find_show_id(self): with self.Provider() as provider: show_id = provider.find_show_id('The Big Bang') self.assertEqual(show_id, 154) def test_find_show_id_ambiguous(self): with self.Provider() as provider: show_id = provider.find_show_id('New Girl') self.assertEqual(show_id, 977) def test_find_show_id_no_dots(self): with self.Provider() as provider: show_id = provider.find_show_id('Marvel\'s Agents of S H I E L D') self.assertEqual(show_id, 1340) def test_find_show_id_error(self): with self.Provider() as provider: show_id = provider.find_show_id('the big gaming') self.assertIsNone(show_id) def test_find_episode_ids(self): with self.Provider() as provider: episode_ids = provider.find_episode_ids(154, 5) self.assertEqual(set(episode_ids.keys()), set(range(1, 25))) def test_query_episode_0(self): video = EPISODES[0] languages = {Language('fra'), Language('por'), Language('hun'), Language('ron'), Language('eng')} matches = {frozenset(['series', 'episode', 'season', 'video_codec']), frozenset(['series', 'episode', 'season'])} with self.Provider() as provider: subtitles = provider.query(video.series, video.season, video.episode) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_query_episode_1(self): video = EPISODES[1] languages = {Language('fra'), Language('ell'), Language('ron'), Language('eng'), Language('hun'), Language('por'), Language('por', 'BR')} matches = {frozenset(['series', 'episode', 'resolution', 'season']), frozenset(['series', 'episode', 'season', 'video_codec']), frozenset(['series', 'episode', 'season'])} with self.Provider() as provider: subtitles = provider.query(video.series, video.season, video.episode) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_list_subtitles(self): video = EPISODES[0] languages = {Language('eng'), Language('fra')} matches = {frozenset(['series', 'episode', 'season'])} with self.Provider() as provider: subtitles = provider.list_subtitles(video, languages) self.assertEqual({frozenset(subtitle.compute_matches(video)) for subtitle in subtitles}, matches) self.assertEqual({subtitle.language for subtitle in subtitles}, languages) def test_download_subtitle(self): video = EPISODES[0] languages = {Language('hun')} with self.Provider() as provider: subtitles = provider.list_subtitles(video, languages) subtitle_text = provider.download_subtitle(subtitles[0]) self.assertTrue(is_valid_subtitle(subtitle_text)) def suite(): suite = TestSuite() suite.addTest(TestLoader().loadTestsFromTestCase(Addic7edProviderTestCase)) suite.addTest(TestLoader().loadTestsFromTestCase(BierDopjeProviderTestCase)) suite.addTest(TestLoader().loadTestsFromTestCase(OpenSubtitlesProviderTestCase)) suite.addTest(TestLoader().loadTestsFromTestCase(PodnapisiProviderTestCase)) suite.addTest(TestLoader().loadTestsFromTestCase(TheSubDBProviderTestCase)) suite.addTest(TestLoader().loadTestsFromTestCase(TVsubtitlesProviderTestCase)) return suite if __name__ == '__main__': TextTestRunner().run(suite()) subliminal-0.7.4/subliminal/subtitle.py0000644000175000017500000001311312271546332021345 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals import logging import os.path import babelfish import pysrt from .video import Episode, Movie logger = logging.getLogger(__name__) class Subtitle(object): """Base class for subtitle :param language: language of the subtitle :type language: :class:`babelfish.Language` :param bool hearing_impaired: `True` if the subtitle is hearing impaired, `False` otherwise """ def __init__(self, language, hearing_impaired=False): self.language = language self.hearing_impaired = hearing_impaired def compute_matches(self, video): """Compute the matches of the subtitle against the `video` :param video: the video to compute the matches against :type video: :class:`~subliminal.video.Video` :return: matches of the subtitle :rtype: set """ raise NotImplementedError def compute_score(self, video): """Compute the score of the subtitle against the `video` There are equivalent matches so that a provider can match one element or its equivalent. This is to give all provider a chance to have a score in the same range without hurting quality. * Matching :class:`~subliminal.video.Video`'s `hashes` is equivalent to matching everything else * Matching :class:`~subliminal.video.Episode`'s `season` and `episode` is equivalent to matching :class:`~subliminal.video.Episode`'s `title` * Matching :class:`~subliminal.video.Episode`'s `tvdb_id` is equivalent to matching :class:`~subliminal.video.Episode`'s `series` :param video: the video to compute the score against :type video: :class:`~subliminal.video.Video` :return: score of the subtitle :rtype: int """ score = 0 # compute matches initial_matches = self.compute_matches(video) matches = initial_matches.copy() # hash is the perfect match if 'hash' in matches: score = video.scores['hash'] else: # remove equivalences if isinstance(video, Episode): if 'imdb_id' in matches: matches -= {'series', 'tvdb_id', 'season', 'episode', 'title'} if 'tvdb_id' in matches: matches -= {'series'} if 'title' in matches: matches -= {'season', 'episode'} # add other scores score += sum((video.scores[match] for match in matches)) logger.info('Computed score %d with matches %r', score, initial_matches) return score def __repr__(self): return '<%s [%s]>' % (self.__class__.__name__, self.language) def get_subtitle_path(video_path, language=None): """Create the subtitle path from the given `video_path` and `language` :param string video_path: path to the video :param language: language of the subtitle to put in the path :type language: :class:`babelfish.Language` or None :return: path of the subtitle :rtype: string """ subtitle_path = os.path.splitext(video_path)[0] if language is not None: try: return subtitle_path + '.%s.%s' % (language.alpha2, 'srt') except babelfish.LanguageConvertError: return subtitle_path + '.%s.%s' % (language.alpha3, 'srt') return subtitle_path + '.srt' def is_valid_subtitle(subtitle_text): """Check if a subtitle text is a valid SubRip format :return: `True` if the subtitle is valid, `False` otherwise :rtype: bool """ try: pysrt.from_string(subtitle_text, error_handling=pysrt.ERROR_RAISE) return True except pysrt.Error as e: if e.args[0] > 80: return True except: logger.exception('Unexpected error when validating subtitle') return False def compute_guess_matches(video, guess): """Compute matches between a `video` and a `guess` :param video: the video to compute the matches on :type video: :class:`~subliminal.video.Video` :param guess: the guess to compute the matches on :type guess: :class:`guessit.Guess` :return: matches of the `guess` :rtype: set """ matches = set() if isinstance(video, Episode): # Series if video.series and 'series' in guess and guess['series'].lower() == video.series.lower(): matches.add('series') # Season if video.season and 'seasonNumber' in guess and guess['seasonNumber'] == video.season: matches.add('season') # Episode if video.episode and 'episodeNumber' in guess and guess['episodeNumber'] == video.episode: matches.add('episode') elif isinstance(video, Movie): # Year if video.year and 'year' in guess and guess['year'] == video.year: matches.add('year') # Title if video.title and 'title' in guess and guess['title'].lower() == video.title.lower(): matches.add('title') # Release group if video.release_group and 'releaseGroup' in guess and guess['releaseGroup'].lower() == video.release_group.lower(): matches.add('release_group') # Screen size if video.resolution and 'screenSize' in guess and guess['screenSize'] == video.resolution: matches.add('resolution') # Video codec if video.video_codec and 'videoCodec' in guess and guess['videoCodec'] == video.video_codec: matches.add('video_codec') # Audio codec if video.audio_codec and 'audioCodec' in guess and guess['audioCodec'] == video.audio_codec: matches.add('audio_codec') return matches subliminal-0.7.4/subliminal/exceptions.py0000644000175000017500000000111612271546332021673 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals class Error(Exception): """Base class for exceptions in subliminal""" pass class ProviderError(Error): """Exception raised by providers""" pass class ProviderConfigurationError(ProviderError): """Exception raised by providers when badly configured""" pass class ProviderNotAvailable(ProviderError): """Exception raised by providers when unavailable""" pass class InvalidSubtitle(ProviderError): """Exception raised by providers when the downloaded subtitle is invalid""" pass subliminal-0.7.4/subliminal/api.py0000644000175000017500000003435112271546332020272 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals import collections import io import logging import operator import babelfish import pkg_resources from .exceptions import ProviderNotAvailable, InvalidSubtitle from .subtitle import get_subtitle_path logger = logging.getLogger(__name__) #: Entry point for the providers PROVIDERS_ENTRY_POINT = 'subliminal.providers' def list_subtitles(videos, languages, providers=None, provider_configs=None): """List subtitles for `videos` with the given `languages` using the specified `providers` :param videos: videos to list subtitles for :type videos: set of :class:`~subliminal.video.Video` :param languages: languages of subtitles to search for :type languages: set of :class:`babelfish.Language` :param providers: providers to use for the search, if not all :type providers: list of string or None :param provider_configs: configuration for providers :type provider_configs: dict of provider name => provider constructor kwargs :return: found subtitles :rtype: dict of :class:`~subliminal.video.Video` => [:class:`~subliminal.subtitle.Subtitle`] """ provider_configs = provider_configs or {} subtitles = collections.defaultdict(list) # filter videos videos = [v for v in videos if v.subtitle_languages & languages < languages] if not videos: logger.info('No video to download subtitles for with languages %r', languages) return subtitles subtitle_languages = set.intersection(*[v.subtitle_languages for v in videos]) for provider_entry_point in pkg_resources.iter_entry_points(PROVIDERS_ENTRY_POINT): # filter and initialize provider if providers is not None and provider_entry_point.name not in providers: logger.debug('Skipping provider %r: not in the list', provider_entry_point.name) continue Provider = provider_entry_point.load() provider_languages = Provider.languages & languages - subtitle_languages if not provider_languages: logger.info('Skipping provider %r: no language to search for', provider_entry_point.name) continue provider_videos = [v for v in videos if Provider.check(v)] if not provider_videos: logger.info('Skipping provider %r: no video to search for', provider_entry_point.name) continue # list subtitles with the provider try: with Provider(**provider_configs.get(provider_entry_point.name, {})) as provider: for provider_video in provider_videos: provider_video_languages = provider_languages - provider_video.subtitle_languages if not provider_video_languages: logger.debug('Skipping provider %r: no language to search for for video %r', provider_entry_point.name, provider_video) continue logger.info('Listing subtitles with provider %r for video %r with languages %r', provider_entry_point.name, provider_video, provider_video_languages) try: provider_subtitles = provider.list_subtitles(provider_video, provider_video_languages) except ProviderNotAvailable: logger.warning('Provider %r is not available, discarding it', provider_entry_point.name) break except: logger.exception('Unexpected error in provider %r', provider_entry_point.name) continue logger.info('Found %d subtitles', len(provider_subtitles)) subtitles[provider_video].extend(provider_subtitles) except ProviderNotAvailable: logger.warning('Provider %r is not available, discarding it', provider_entry_point.name) return subtitles def download_subtitles(subtitles, provider_configs=None, single=False): """Download subtitles :param subtitles: subtitles to download :type subtitles: dict of :class:`~subliminal.video.Video` => [:class:`~subliminal.subtitle.Subtitle`] :param provider_configs: configuration for providers :type provider_configs: dict of provider name => provider constructor kwargs :param bool single: download with .srt extension if `True`, add language identifier otherwise """ provider_configs = provider_configs or {} discarded_providers = set() providers_by_name = {ep.name: ep.load() for ep in pkg_resources.iter_entry_points(PROVIDERS_ENTRY_POINT)} initialized_providers = {} try: for video, video_subtitles in subtitles.items(): languages = {subtitle.language for subtitle in video_subtitles} downloaded_languages = set() for subtitle in video_subtitles: # filter if subtitle.language in downloaded_languages: continue if subtitle.provider_name in discarded_providers: logger.debug('Skipping subtitle from discarded provider %r', subtitle.provider_name) continue # initialize provider if subtitle.provider_name in initialized_providers: provider = initialized_providers[subtitle.provider_name] else: provider = providers_by_name[subtitle.provider_name](**provider_configs.get(subtitle.provider_name, {})) try: provider.initialize() except ProviderNotAvailable: logger.warning('Provider %r is not available, discarding it', subtitle.provider_name) discarded_providers.add(subtitle.provider_name) continue initialized_providers[subtitle.provider_name] = provider # download subtitles subtitle_path = get_subtitle_path(video.name, None if single else subtitle.language) logger.info('Downloading subtitle %r into %r', subtitle, subtitle_path) try: subtitle_text = provider.download_subtitle(subtitle) except ProviderNotAvailable: logger.warning('Provider %r is not available, discarding it', subtitle.provider_name) discarded_providers.add(subtitle.provider_name) continue except InvalidSubtitle: logger.info('Invalid subtitle, skipping it') continue except: logger.exception('Unexpected error in provider %r', subtitle.provider_name) continue with io.open(subtitle_path, 'w', encoding='utf-8') as f: f.write(subtitle_text) downloaded_languages.add(subtitle.language) if single or downloaded_languages == languages: break finally: # terminate providers for (provider_name, provider) in initialized_providers.items(): try: provider.terminate() except ProviderNotAvailable: logger.warning('Provider %r is not available, unable to terminate', provider_name) except: logger.exception('Unexpected error in provider %r', provider_name) def download_best_subtitles(videos, languages, providers=None, provider_configs=None, single=False, min_score=0, hearing_impaired=False): """Download the best subtitles for `videos` with the given `languages` using the specified `providers` :param videos: videos to download subtitles for :type videos: set of :class:`~subliminal.video.Video` :param languages: languages of subtitles to download :type languages: set of :class:`babelfish.Language` :param providers: providers to use for the search, if not all :type providers: list of string or None :param provider_configs: configuration for providers :type provider_configs: dict of provider name => provider constructor kwargs :param bool single: download with .srt extension if `True`, add language identifier otherwise :param int min_score: minimum score for subtitles to download :param bool hearing_impaired: download hearing impaired subtitles """ provider_configs = provider_configs or {} discarded_providers = set() downloaded_subtitles = collections.defaultdict(list) # filter videos videos = [v for v in videos if v.subtitle_languages & languages < languages and (not single or babelfish.Language('und') not in v.subtitle_languages)] if not videos: logger.info('No video to download subtitles for with languages %r', languages) return downloaded_subtitles # filter and initialize providers subtitle_languages = set.intersection(*[v.subtitle_languages for v in videos]) initialized_providers = {} for provider_entry_point in pkg_resources.iter_entry_points(PROVIDERS_ENTRY_POINT): if providers is not None and provider_entry_point.name not in providers: logger.debug('Skipping provider %r: not in the list', provider_entry_point.name) continue Provider = provider_entry_point.load() if not Provider.languages & languages - subtitle_languages: logger.info('Skipping provider %r: no language to search for', provider_entry_point.name) continue if not [v for v in videos if Provider.check(v)]: logger.info('Skipping provider %r: no video to search for', provider_entry_point.name) continue provider = Provider(**provider_configs.get(provider_entry_point.name, {})) try: provider.initialize() except ProviderNotAvailable: logger.warning('Provider %r is not available, discarding it', provider_entry_point.name) continue initialized_providers[provider_entry_point.name] = provider try: for video in videos: # search for subtitles subtitles = [] for provider_name, provider in initialized_providers.items(): if provider.check(video): if provider_name in discarded_providers: logger.debug('Skipping discarded provider %r', provider_name) continue provider_video_languages = provider.languages & languages - video.subtitle_languages if not provider_video_languages: logger.debug('Skipping provider %r: no language to search for for video %r', provider_name, video) continue logger.info('Listing subtitles with provider %r for video %r with languages %r', provider_name, video, provider_video_languages) try: provider_subtitles = provider.list_subtitles(video, provider_video_languages) except ProviderNotAvailable: logger.warning('Provider %r is not available, discarding it', provider_name) discarded_providers.add(provider_name) continue except: logger.exception('Unexpected error in provider %r', provider_name) continue logger.info('Found %d subtitles', len(provider_subtitles)) subtitles.extend(provider_subtitles) # find the best subtitles and download them downloaded_languages = video.subtitle_languages.copy() for subtitle, score in sorted([(s, s.compute_score(video)) for s in subtitles], key=operator.itemgetter(1), reverse=True): # filter if subtitle.provider_name in discarded_providers: logger.debug('Skipping subtitle from discarded provider %r', subtitle.provider_name) continue if subtitle.hearing_impaired != hearing_impaired: logger.debug('Skipping subtitle: hearing impaired != %r', hearing_impaired) continue if score < min_score: logger.debug('Skipping subtitle: score < %d', min_score) continue if subtitle.language in downloaded_languages: logger.debug('Skipping subtitle: %r already downloaded', subtitle.language) continue # download provider = initialized_providers[subtitle.provider_name] subtitle_path = get_subtitle_path(video.name, None if single else subtitle.language) logger.info('Downloading subtitle %r with score %d into %r', subtitle, score, subtitle_path) try: subtitle_text = provider.download_subtitle(subtitle) downloaded_subtitles[video].append(subtitle) except ProviderNotAvailable: logger.warning('Provider %r is not available, discarding it', subtitle.provider_name) discarded_providers.add(subtitle.provider_name) continue except InvalidSubtitle: logger.info('Invalid subtitle, skipping it') continue except: logger.exception('Unexpected error in provider %r', subtitle.provider_name) continue with io.open(subtitle_path, 'w', encoding='utf-8') as f: f.write(subtitle_text) downloaded_languages.add(subtitle.language) if single or downloaded_languages >= languages: logger.debug('All languages downloaded') break finally: # terminate providers for (provider_name, provider) in initialized_providers.items(): try: provider.terminate() except ProviderNotAvailable: logger.warning('Provider %r is not available, unable to terminate', provider_name) except: logger.exception('Unexpected error in provider %r', provider_name) return downloaded_subtitles subliminal-0.7.4/subliminal/cli.py0000644000175000017500000002057012271546332020266 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals, print_function import argparse import datetime import logging import os import re import sys import babelfish import guessit import pkg_resources from subliminal import (__version__, PROVIDERS_ENTRY_POINT, cache_region, MutexLock, Video, Episode, Movie, scan_videos, download_best_subtitles) try: import colorlog except ImportError: colorlog = None DEFAULT_CACHE_FILE = os.path.join('~', '.config', 'subliminal.cache.dbm') def subliminal(): parser = argparse.ArgumentParser(prog='subliminal', description='Subtitles, faster than your thoughts', epilog='Suggestions and bug reports are greatly appreciated: ' 'https://github.com/Diaoul/subliminal/issues', add_help=False) # required arguments required_arguments_group = parser.add_argument_group('required arguments') required_arguments_group.add_argument('paths', nargs='+', metavar='PATH', help='path to video file or folder') required_arguments_group.add_argument('-l', '--languages', nargs='+', required=True, metavar='LANGUAGE', help='wanted languages as IETF codes e.g. fr, pt-BR, sr-Cyrl ') # configuration configuration_group = parser.add_argument_group('configuration') configuration_group.add_argument('-s', '--single', action='store_true', help='download without language code in subtitle\'s filename i.e. .srt only') configuration_group.add_argument('-c', '--cache-file', default=DEFAULT_CACHE_FILE, help='cache file (default: %(default)s)') # filtering filtering_group = parser.add_argument_group('filtering') providers = [ep.name for ep in pkg_resources.iter_entry_points(PROVIDERS_ENTRY_POINT)] filtering_group.add_argument('-p', '--providers', nargs='+', metavar='PROVIDER', help='providers to use (%s)' % ', '.join(providers)) filtering_group.add_argument('-m', '--min-score', type=int, help='minimum score for subtitles (0-%d for episodes, 0-%d for movies)' % (Episode.scores['hash'], Movie.scores['hash'])) filtering_group.add_argument('-a', '--age', help='download subtitles for videos newer than AGE e.g. 12h, 1w2d') filtering_group.add_argument('-h', '--hearing-impaired', action='store_true', help='download hearing impaired subtitles') filtering_group.add_argument('-f', '--force', action='store_true', help='force subtitle download for videos with existing subtitles') # addic7ed addic7ed_group = parser.add_argument_group('addic7ed') addic7ed_group.add_argument('--addic7ed-username', metavar='USERNAME', help='username for addic7ed provider') addic7ed_group.add_argument('--addic7ed-password', metavar='PASSWORD', help='password for addic7ed provider') # output output_group = parser.add_argument_group('output') output_exclusive_group = output_group.add_mutually_exclusive_group() output_exclusive_group.add_argument('-q', '--quiet', action='store_true', help='disable output') output_exclusive_group.add_argument('-v', '--verbose', action='store_true', help='verbose output') output_group.add_argument('--color', action='store_true', help='add color to console output (requires colorlog)') # troubleshooting troubleshooting_group = parser.add_argument_group('troubleshooting') troubleshooting_group.add_argument('--debug', action='store_true', help='debug output') troubleshooting_group.add_argument('--version', action='version', version=__version__) troubleshooting_group.add_argument('--help', action='help', help='show this help message and exit') # parse args args = parser.parse_args() # parse paths try: args.paths = [os.path.abspath(os.path.expanduser(p.decode('utf-8'))) for p in args.paths] except UnicodeDecodeError: parser.error('argument paths: encodings is not utf-8: %r' % args.paths) # parse languages try: args.languages = {babelfish.Language.fromietf(l) for l in args.languages} except babelfish.Error: parser.error('argument -l/--languages: codes are not IETF: %r' % args.languages) # parse age if args.age is not None: match = re.match(r'^(?:(?P\d+?)w)?(?:(?P\d+?)d)?(?:(?P\d+?)h)?$', args.age) if not match: parser.error('argument -a/--age: invalid age: %r' % args.age) args.age = datetime.timedelta(**{k: int(v) for k, v in match.groupdict(0).items()}) # parse cache-file args.cache_file = os.path.abspath(os.path.expanduser(args.cache_file)) if not os.path.exists(os.path.split(args.cache_file)[0]): parser.error('argument -c/--cache-file: directory %r for cache file does not exist' % os.path.split(args.cache_file)[0]) # parse provider configs provider_configs = {} if (args.addic7ed_username is not None and args.addic7ed_password is None or args.addic7ed_username is None and args.addic7ed_password is not None): parser.error('argument --addic7ed-username/--addic7ed-password: both arguments are required or none') if args.addic7ed_username is not None and args.addic7ed_password is not None: provider_configs['addic7ed'] = {'username': args.addic7ed_username, 'password': args.addic7ed_password} # parse color if args.color and colorlog is None: parser.error('argument --color: colorlog required') # setup output if args.debug: handler = logging.StreamHandler() if args.color: handler.setFormatter(colorlog.ColoredFormatter('%(log_color)s%(levelname)-8s%(reset)s [%(blue)s%(name)s-%(funcName)s:%(lineno)d%(reset)s] %(message)s', log_colors=dict(colorlog.default_log_colors.items() + [('DEBUG', 'cyan')]))) else: handler.setFormatter(logging.Formatter('%(levelname)-8s [%(name)s-%(funcName)s:%(lineno)d] %(message)s')) logging.getLogger().addHandler(handler) logging.getLogger().setLevel(logging.DEBUG) elif args.verbose: handler = logging.StreamHandler() if args.color: handler.setFormatter(colorlog.ColoredFormatter('%(log_color)s%(levelname)-8s%(reset)s [%(blue)s%(name)s%(reset)s] %(message)s')) else: handler.setFormatter(logging.Formatter('%(levelname)-8s [%(name)s] %(message)s')) logging.getLogger('subliminal').addHandler(handler) logging.getLogger('subliminal').setLevel(logging.INFO) elif not args.quiet: handler = logging.StreamHandler() if args.color: handler.setFormatter(colorlog.ColoredFormatter('[%(log_color)s%(levelname)s%(reset)s] %(message)s')) else: handler.setFormatter(logging.Formatter('%(levelname)s: %(message)s')) logging.getLogger('subliminal.api').addHandler(handler) logging.getLogger('subliminal.api').setLevel(logging.INFO) # configure cache cache_region.configure('dogpile.cache.dbm', expiration_time=datetime.timedelta(days=30), # @UndefinedVariable arguments={'filename': args.cache_file, 'lock_factory': MutexLock}) # scan videos videos = scan_videos([p for p in args.paths if os.path.exists(p)], subtitles=not args.force, embedded_subtitles=not args.force, age=args.age) # guess videos videos.extend([Video.fromguess(os.path.split(p)[1], guessit.guess_file_info(p, 'autodetect')) for p in args.paths if not os.path.exists(p)]) # download best subtitles subtitles = download_best_subtitles(videos, args.languages, providers=args.providers, provider_configs=provider_configs, single=args.single, min_score=args.min_score, hearing_impaired=args.hearing_impaired) # result output if not subtitles: if not args.quiet: sys.stderr.write('No subtitles downloaded\n') exit(1) if not args.quiet: subtitles_count = sum([len(s) for s in subtitles.values()]) if subtitles_count == 1: print('%d subtitle downloaded' % subtitles_count) else: print('%d subtitles downloaded' % subtitles_count) subliminal-0.7.4/subliminal/cache.py0000644000175000017500000000364312271546332020564 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- import inspect from dogpile.cache import make_region # @UnresolvedImport from dogpile.cache.backends.file import AbstractFileLock # @UnresolvedImport from dogpile.cache.compat import string_type # @UnresolvedImport from dogpile.core.readwrite_lock import ReadWriteMutex # @UnresolvedImport #: Subliminal's cache version CACHE_VERSION = 1 def subliminal_key_generator(namespace, fn, to_str=string_type): """Add a :data:`CACHE_VERSION` to dogpile.cache's default function_key_generator""" if namespace is None: namespace = '%d:%s:%s' % (CACHE_VERSION, fn.__module__, fn.__name__) else: namespace = '%d:%s:%s|%s' % (CACHE_VERSION, fn.__module__, fn.__name__, namespace) args = inspect.getargspec(fn) has_self = args[0] and args[0][0] in ('self', 'cls') def generate_key(*args, **kw): if kw: raise ValueError('Keyword arguments not supported') if has_self: args = args[1:] return namespace + '|' + ' '.join(map(to_str, args)) return generate_key class MutexLock(AbstractFileLock): """:class:`MutexLock` is a thread-based rw lock based on :class:`dogpile.core.ReadWriteMutex`""" def __init__(self, filename): self.mutex = ReadWriteMutex() def acquire_read_lock(self, wait): ret = self.mutex.acquire_read_lock(wait) return wait or ret def acquire_write_lock(self, wait): ret = self.mutex.acquire_write_lock(wait) return wait or ret def release_read_lock(self): return self.mutex.release_read_lock() def release_write_lock(self): return self.mutex.release_write_lock() #: The dogpile.cache region (long-lived) region = make_region(function_key_generator=subliminal_key_generator) #: The dogpile.cache region for :meth:`~subliminal.providers.Provider.query` (short-lived) query_region = make_region(function_key_generator=subliminal_key_generator) subliminal-0.7.4/subliminal/score.py0000755000175000017500000000657712271546332020650 0ustar antoineantoine00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- from __future__ import print_function, unicode_literals from sympy import Eq, symbols, solve # Symbols release_group, resolution, video_codec, audio_codec = symbols('release_group resolution video_codec audio_codec') imdb_id, hash, title, series, tvdb_id, season, episode = symbols('imdb_id hash title series tvdb_id season episode') # @ReservedAssignment year = symbols('year') def get_episode_equations(): """Get the score equations for a :class:`~subliminal.video.Episode` The equations are the following: 1. hash = resolution + video_codec + audio_codec + series + season + episode + release_group 2. series = resolution + video_codec + audio_codec + season + episode + 1 3. tvdb_id = series 4. season = resolution + video_codec + audio_codec + 1 5. imdb_id = series + season + episode 6. resolution = video_codec 7. video_codec = 2 * audio_codec 8. title = season + episode 9. season = episode 10. release_group = season 11. audio_codec = 1 :return: the score equations for an episode :rtype: list of :class:`sympy.Eq` """ equations = [] equations.append(Eq(hash, resolution + video_codec + audio_codec + series + season + episode + release_group)) equations.append(Eq(series, resolution + video_codec + audio_codec + season + episode + release_group)) equations.append(Eq(tvdb_id, series)) equations.append(Eq(season, resolution + video_codec + audio_codec + 1)) equations.append(Eq(imdb_id, series + season + episode)) equations.append(Eq(resolution, video_codec)) equations.append(Eq(video_codec, 2 * audio_codec)) equations.append(Eq(title, season + episode)) equations.append(Eq(season, episode)) equations.append(Eq(release_group, season)) equations.append(Eq(audio_codec, 1)) return equations def get_movie_equations(): """Get the score equations for a :class:`~subliminal.video.Movie` The equations are the following: 1. hash = resolution + video_codec + audio_codec + title + year + release_group 2. imdb_id = hash 3. resolution = video_codec 4. video_codec = 2 * audio_codec 5. title = resolution + video_codec + audio_codec + year + 1 6. release_group = resolution + video_codec + audio_codec + 1 7. year = release_group + 1 8. audio_codec = 1 :return: the score equations for a movie :rtype: list of :class:`sympy.Eq` """ equations = [] equations.append(Eq(hash, resolution + video_codec + audio_codec + title + year + release_group)) equations.append(Eq(imdb_id, hash)) equations.append(Eq(resolution, video_codec)) equations.append(Eq(video_codec, 2 * audio_codec)) equations.append(Eq(title, resolution + video_codec + audio_codec + year + 1)) equations.append(Eq(video_codec, 2 * audio_codec)) equations.append(Eq(release_group, resolution + video_codec + audio_codec + 1)) equations.append(Eq(year, release_group + 1)) equations.append(Eq(audio_codec, 1)) return equations if __name__ == '__main__': print(solve(get_episode_equations(), [release_group, resolution, video_codec, audio_codec, imdb_id, hash, series, tvdb_id, season, episode, title])) print(solve(get_movie_equations(), [release_group, resolution, video_codec, audio_codec, imdb_id, hash, title, year])) subliminal-0.7.4/subliminal/video.py0000644000175000017500000004101712271546332020624 0ustar antoineantoine00000000000000# -*- coding: utf-8 -*- from __future__ import unicode_literals import datetime import hashlib import logging import os import struct import babelfish import enzyme import guessit logger = logging.getLogger(__name__) #: Video extensions VIDEO_EXTENSIONS = ('.3g2', '.3gp', '.3gp2', '.3gpp', '.60d', '.ajp', '.asf', '.asx', '.avchd', '.avi', '.bik', '.bix', '.box', '.cam', '.dat', '.divx', '.dmf', '.dv', '.dvr-ms', '.evo', '.flc', '.fli', '.flic', '.flv', '.flx', '.gvi', '.gvp', '.h264', '.m1v', '.m2p', '.m2ts', '.m2v', '.m4e', '.m4v', '.mjp', '.mjpeg', '.mjpg', '.mkv', '.moov', '.mov', '.movhd', '.movie', '.movx', '.mp4', '.mpe', '.mpeg', '.mpg', '.mpv', '.mpv2', '.mxf', '.nsv', '.nut', '.ogg', '.ogm', '.omf', '.ps', '.qt', '.ram', '.rm', '.rmvb', '.swf', '.ts', '.vfw', '.vid', '.video', '.viv', '.vivo', '.vob', '.vro', '.wm', '.wmv', '.wmx', '.wrap', '.wvx', '.wx', '.x264', '.xvid') #: Subtitle extensions SUBTITLE_EXTENSIONS = ('.srt', '.sub', '.smi', '.txt', '.ssa', '.ass', '.mpl') class Video(object): """Base class for videos Represent a video, existing or not, with various properties that defines it. Each property has an associated score based on equations that are described in subclasses. :param string name: name or path of the video :param string release_group: release group of the video :param string resolution: screen size of the video stream (480p, 720p, 1080p or 1080i) :param string video_codec: codec of the video stream :param string audio_codec: codec of the main audio stream :param int imdb_id: IMDb id of the video :param dict hashes: hashes of the video file by provider names :param int size: byte size of the video file :param set subtitle_languages: existing subtitle languages """ scores = {} def __init__(self, name, release_group=None, resolution=None, video_codec=None, audio_codec=None, imdb_id=None, hashes=None, size=None, subtitle_languages=None): self.name = name self.release_group = release_group self.resolution = resolution self.video_codec = video_codec self.audio_codec = audio_codec self.imdb_id = imdb_id self.hashes = hashes or {} self.size = size self.subtitle_languages = subtitle_languages or set() @classmethod def fromguess(cls, name, guess): if guess['type'] == 'episode': return Episode.fromguess(name, guess) if guess['type'] == 'movie': return Movie.fromguess(name, guess) raise ValueError('The guess must be an episode or a movie guess') def __repr__(self): return '<%s [%r]>' % (self.__class__.__name__, self.name) def __hash__(self): return hash(self.name) class Episode(Video): """Episode :class:`Video` Scores are defined by a set of equations, see :func:`~subliminal.score.get_episode_equations` :param string series: series of the episode :param int season: season number of the episode :param int episode: episode number of the episode :param string title: title of the episode :param int tvdb_id: TheTVDB id of the episode """ scores = {'title': 12, 'video_codec': 2, 'imdb_id': 35, 'audio_codec': 1, 'tvdb_id': 23, 'resolution': 2, 'season': 6, 'release_group': 6, 'series': 23, 'episode': 6, 'hash': 46} def __init__(self, name, series, season, episode, release_group=None, resolution=None, video_codec=None, audio_codec=None, imdb_id=None, hashes=None, size=None, subtitle_languages=None, title=None, tvdb_id=None): super(Episode, self).__init__(name, release_group, resolution, video_codec, audio_codec, imdb_id, hashes, size, subtitle_languages) self.series = series self.season = season self.episode = episode self.title = title self.tvdb_id = tvdb_id @classmethod def fromguess(cls, name, guess): if guess['type'] != 'episode': raise ValueError('The guess must be an episode guess') if 'series' not in guess or 'season' not in guess or 'episodeNumber' not in guess: raise ValueError('Insufficient data to process the guess') return cls(name, guess['series'], guess['season'], guess['episodeNumber'], release_group=guess.get('releaseGroup'), resolution=guess.get('screenSize'), video_codec=guess.get('videoCodec'), audio_codec=guess.get('audioCodec'), title=guess.get('title')) def __repr__(self): return '<%s [%r, %rx%r]>' % (self.__class__.__name__, self.series, self.season, self.episode) class Movie(Video): """Movie :class:`Video` Scores are defined by a set of equations, see :func:`~subliminal.score.get_movie_equations` :param string title: title of the movie :param int year: year of the movie """ scores = {'title': 13, 'video_codec': 2, 'resolution': 2, 'audio_codec': 1, 'year': 7, 'imdb_id': 31, 'release_group': 6, 'hash': 31} def __init__(self, name, title, release_group=None, resolution=None, video_codec=None, audio_codec=None, imdb_id=None, hashes=None, size=None, subtitle_languages=None, year=None): super(Movie, self).__init__(name, release_group, resolution, video_codec, audio_codec, imdb_id, hashes, size, subtitle_languages) self.title = title self.year = year @classmethod def fromguess(cls, name, guess): if guess['type'] != 'movie': raise ValueError('The guess must be a movie guess') if 'title' not in guess: raise ValueError('Insufficient data to process the guess') return cls(name, guess['title'], release_group=guess.get('releaseGroup'), resolution=guess.get('screenSize'), video_codec=guess.get('videoCodec'), audio_codec=guess.get('audioCodec'), year=guess.get('year')) def __repr__(self): if self.year is None: return '<%s [%r]>' % (self.__class__.__name__, self.title) return '<%s [%r, %r]>' % (self.__class__.__name__, self.title, self.year) def scan_subtitle_languages(path): """Search for subtitles with alpha2 extension from a video `path` and return their language :param string path: path to the video :return: found subtitle languages :rtype: set """ language_extensions = tuple('.' + c for c in babelfish.get_language_converter('alpha2').codes) dirpath, filename = os.path.split(path) subtitles = set() for p in os.listdir(dirpath): if not isinstance(p, bytes) and p.startswith(os.path.splitext(filename)[0]) and p.endswith(SUBTITLE_EXTENSIONS): if os.path.splitext(p)[0].endswith(language_extensions): subtitles.add(babelfish.Language.fromalpha2(os.path.splitext(p)[0][-2:])) else: subtitles.add(babelfish.Language('und')) logger.debug('Found subtitles %r', subtitles) return subtitles def scan_video(path, subtitles=True, embedded_subtitles=True): """Scan a video and its subtitle languages from a video `path` :param string path: absolute path to the video :param bool subtitles: scan for subtitles with the same name :param bool embedded_subtitles: scan for embedded subtitles :return: the scanned video :rtype: :class:`Video` :raise: ValueError if cannot guess enough information from the path """ dirpath, filename = os.path.split(path) logger.info('Scanning video %r in %r', filename, dirpath) video = Video.fromguess(path, guessit.guess_file_info(path, 'autodetect')) video.size = os.path.getsize(path) if video.size > 10485760: logger.debug('Size is %d', video.size) video.hashes['opensubtitles'] = hash_opensubtitles(path) video.hashes['thesubdb'] = hash_thesubdb(path) logger.debug('Computed hashes %r', video.hashes) else: logger.warning('Size is lower than 10MB: hashes not computed') if subtitles: video.subtitle_languages |= scan_subtitle_languages(path) # enzyme try: if filename.endswith('.mkv'): with open(path, 'rb') as f: mkv = enzyme.MKV(f) if mkv.video_tracks: video_track = mkv.video_tracks[0] # resolution if video_track.height in (480, 720, 1080): if video_track.interlaced: video.resolution = '%di' % video_track.height logger.debug('Found resolution %s with enzyme', video.resolution) else: video.resolution = '%dp' % video_track.height logger.debug('Found resolution %s with enzyme', video.resolution) # video codec if video_track.codec_id == 'V_MPEG4/ISO/AVC': video.video_codec = 'h264' logger.debug('Found video_codec %s with enzyme', video.video_codec) elif video_track.codec_id == 'V_MPEG4/ISO/SP': video.video_codec = 'DivX' logger.debug('Found video_codec %s with enzyme', video.video_codec) elif video_track.codec_id == 'V_MPEG4/ISO/ASP': video.video_codec = 'XviD' logger.debug('Found video_codec %s with enzyme', video.video_codec) else: logger.warning('MKV has no video track') if mkv.audio_tracks: audio_track = mkv.audio_tracks[0] # audio codec if audio_track.codec_id == 'A_AC3': video.audio_codec = 'AC3' logger.debug('Found audio_codec %s with enzyme', video.audio_codec) elif audio_track.codec_id == 'A_DTS': video.audio_codec = 'DTS' logger.debug('Found audio_codec %s with enzyme', video.audio_codec) elif audio_track.codec_id == 'A_AAC': video.audio_codec = 'AAC' logger.debug('Found audio_codec %s with enzyme', video.audio_codec) else: logger.warning('MKV has no audio track') if mkv.subtitle_tracks: # embedded subtitles if embedded_subtitles: embedded_subtitle_languages = set() for st in mkv.subtitle_tracks: if st.language: try: embedded_subtitle_languages.add(babelfish.Language.fromalpha3b(st.language)) except babelfish.Error: logger.error('Embedded subtitle track language %r is not a valid language', st.language) embedded_subtitle_languages.add(babelfish.Language('und')) elif st.name: try: embedded_subtitle_languages.add(babelfish.Language.fromname(st.name)) except babelfish.Error: logger.error('Embedded subtitle track name %r is not a valid language', st.name) embedded_subtitle_languages.add(babelfish.Language('und')) else: embedded_subtitle_languages.add(babelfish.Language('und')) logger.debug('Found embedded subtitle %r with enzyme', embedded_subtitle_languages) video.subtitle_languages |= embedded_subtitle_languages else: logger.debug('MKV has no subtitle track') except enzyme.Error: logger.error('Parsing video metadata with enzyme failed') return video def scan_videos(paths, subtitles=True, embedded_subtitles=True, age=None): """Scan `paths` for videos and their subtitle languages :params paths: absolute paths to scan for videos :type paths: list of string :param bool subtitles: scan for subtitles with the same name :param bool embedded_subtitles: scan for embedded subtitles :param age: age of the video, if any :type age: datetime.timedelta or None :return: the scanned videos :rtype: list of :class:`Video` """ videos = [] # scan files for filepath in [p for p in paths if os.path.isfile(p)]: if age and datetime.datetime.now() - datetime.datetime.fromtimestamp(os.path.getmtime(filepath)) > age: logger.info('Skipping video %r: older than %r', filepath, age) continue try: videos.append(scan_video(filepath, subtitles, embedded_subtitles)) except ValueError as e: logger.error('Skipping video: %s', e) continue # scan directories for path in [p for p in paths if os.path.isdir(p)]: logger.info('Scanning directory %r', path) for dirpath, dirnames, filenames in os.walk(path): # skip badly encoded directories if isinstance(dirpath, bytes): logger.error('Skipping badly encoded directory %r', dirpath.decode('utf-8', errors='replace')) continue # skip badly encoded and hidden sub directories for dirname in list(dirnames): if isinstance(dirname, bytes): logger.error('Skipping badly encoded dirname %r in %r', dirname.decode('utf-8', errors='replace'), dirpath) dirnames.remove(dirname) elif dirname.startswith('.'): logger.debug('Skipping hidden dirname %r in %r', dirname, dirpath) dirnames.remove(dirname) # scan for videos for filename in filenames: # skip badly encoded files if isinstance(filename, bytes): logger.error('Skipping badly encoded filename %r in %r', filename.decode('utf-8', errors='replace'), dirpath) continue # filter videos if not filename.endswith(VIDEO_EXTENSIONS): continue # skip hidden files if filename.startswith('.'): logger.debug('Skipping hidden filename %r in %r', filename, dirpath) continue filepath = os.path.join(dirpath, filename) # skip links if os.path.islink(filepath): logger.debug('Skipping link %r in %r', filename, dirpath) continue if age and datetime.datetime.now() - datetime.datetime.fromtimestamp(os.path.getmtime(filepath)) > age: logger.info('Skipping video %r: older than %r', filepath, age) continue try: video = scan_video(filepath, subtitles, embedded_subtitles) except ValueError as e: logger.error('Skipping video: %s', e) continue videos.append(video) return videos def hash_opensubtitles(video_path): """Compute a hash using OpenSubtitles' algorithm :param string video_path: path of the video :return: the hash :rtype: string """ bytesize = struct.calcsize(b'q') with open(video_path, 'rb') as f: filesize = os.path.getsize(video_path) filehash = filesize if filesize < 65536 * 2: return None for _ in range(65536 / bytesize): filebuffer = f.read(bytesize) (l_value,) = struct.unpack(b'q', filebuffer) filehash += l_value filehash = filehash & 0xFFFFFFFFFFFFFFFF # to remain as 64bit number f.seek(max(0, filesize - 65536), 0) for _ in range(65536 / bytesize): filebuffer = f.read(bytesize) (l_value,) = struct.unpack(b'q', filebuffer) filehash += l_value filehash = filehash & 0xFFFFFFFFFFFFFFFF returnedhash = '%016x' % filehash return returnedhash def hash_thesubdb(video_path): """Compute a hash using TheSubDB's algorithm :param string video_path: path of the video :return: the hash :rtype: string """ readsize = 64 * 1024 if os.path.getsize(video_path) < readsize: return None with open(video_path, 'rb') as f: data = f.read(readsize) f.seek(-readsize, os.SEEK_END) data += f.read(readsize) return hashlib.md5(data).hexdigest().decode('ascii')