Compare commits

...

105 Commits

Author SHA1 Message Date
pannal dbd2f7d69e fix el picturo 2015-10-16 05:31:28 +02:00
panni 95ac877c08 Merge branch 'master' of github.com:pannal/Sub-Zero 2015-10-16 05:17:52 +02:00
panni 5831f19ae0 forgot constant 2015-10-16 05:17:43 +02:00
pannal 530bdc5510 Update README.md 2015-10-16 05:09:35 +02:00
panni 0c01d6989a search correctly for tv subtitles; 1.1.0.3 2015-10-16 05:08:54 +02:00
pannal 02861d01d6 Update README.md 2015-10-16 04:28:55 +02:00
pannal 668d1693fe Update Info.plist 2015-10-16 04:28:28 +02:00
panni 7a3911c837 adjust default scores 2015-10-16 04:25:55 +02:00
panni 5291cbc136 only old changes in CHANGELOG.md; update logo 2015-10-16 04:09:23 +02:00
panni c1fc68204c Merge branch 'master' of github.com:pannal/Sub-Zero 2015-10-16 04:07:23 +02:00
pannal cd8fed5c7c Update README.md 2015-10-16 04:06:02 +02:00
pannal f2506fa762 Merge pull request #43 from pannal/1.1.0.1
1.1.0.1
2015-10-16 04:04:11 +02:00
pannal 382763c89e Update README.md 2015-10-16 04:02:24 +02:00
panni b4cd1ccaa5 clarify new defaults; cleanup 2015-10-16 04:01:18 +02:00
panni b5032f457f default external folder setting: current folder 2015-10-16 03:59:18 +02:00
panni f0bb3cae90 more readme 2015-10-16 03:52:56 +02:00
panni e416e82179 readme 1.1.0.1 2015-10-16 03:45:34 +02:00
panni 552aed19a0 separate changelog from readme 2015-10-16 03:36:42 +02:00
panni 6c4cefcf25 remove only_one leftover 2015-10-16 03:33:14 +02:00
panni ac41ba699c remove obsolete only_one setting; add IETF to ISO 639-1 option; rename agents 2015-10-16 03:31:05 +02:00
panni cd64118868 update version 2015-10-16 03:19:31 +02:00
panni 735df8078f Log proxy not needed anymore 2015-10-16 03:17:41 +02:00
panni 8304f49273 incorporate localmediaextended functionality into core 2015-10-16 03:16:00 +02:00
panni 3130de3a02 move back because localmediaextended won't be needed anymore 2015-10-16 01:07:27 +02:00
panni a284ac7677 use more common agent names 2015-10-16 00:12:10 +02:00
pannal 7964fd9042 prepare for 1.1.0.1 2015-10-16 00:05:31 +02:00
panni ded012a1bc tvsubtitles: be more smart about punctuation 2015-10-15 15:00:13 +02:00
panni df3e3465f9 addic7ed: be smarter about show ids 2015-10-15 14:50:59 +02:00
pannal bed93bf928 RC5.2 info 2015-10-14 22:13:59 +02:00
pannal 7697ceffef RC 5.2 readme 2015-10-14 22:13:32 +02:00
panni 81dd24a9bd Merge branch 'detached' 2015-10-14 22:05:23 +02:00
panni 729d7d97c4 revert back from plex/localmedia/master to plex/localmedia/dist 2015-10-14 22:04:15 +02:00
pannal c7a4b3c0a4 README.md not so outdated anymore 2015-10-14 19:17:44 +02:00
pannal 3da044ada9 forgot Info.plist update 2015-10-14 19:01:32 +02:00
pannal 44bbc93dae Update README.md 2015-10-14 17:41:13 +02:00
pannal 54341a0afc RC5.1 2015-10-14 17:41:05 +02:00
pannal 599eab3e5b Merge pull request #40 from pannal/RC5
RC5.1
2015-10-14 17:33:44 +02:00
panni 9f9c875234 Merge remote-tracking branch 'origin' into RC5 2015-10-14 17:32:25 +02:00
panni 74c0ed80c5 make hearing impaired more configurable and clear 2015-10-14 17:32:06 +02:00
pannal 5ecb7aea5e update download links 2015-10-14 16:42:10 +02:00
pannal 829eacc4d6 RC5 2015-10-14 16:41:46 +02:00
pannal f7b3f924b4 Merge pull request #39 from pannal/RC5
RC5
2015-10-14 16:32:45 +02:00
panni e247bc0e59 add optional boost for addic7ed subtitles; partly fixes #8 2015-10-14 16:31:56 +02:00
panni 4158416183 hard bail-out if hearing_impaired didn't match 2015-10-14 16:30:33 +02:00
panni cf1181f2af add custom language field; fixes #27 2015-10-14 15:39:42 +02:00
panni a2d1335403 pass known video type info to guessit; fixes #38 2015-10-14 14:53:20 +02:00
panni 520cbb5189 patch subtitle repr to include download/page link; fixes #34 2015-10-14 14:37:44 +02:00
panni e8eeadb094 add colon and single quote to punctuation fix mixin; resolves #36 2015-10-14 13:57:27 +02:00
panni 92a2336dba Merge remote-tracking branch 'origin' into RC5 2015-10-14 13:56:06 +02:00
panni cbc75c8b85 update to newest LocalMediaExtended 2015-10-14 13:40:06 +02:00
panni 563973163e only pass the file name and three parent directories to guessit; should fix #38 2015-10-14 13:24:10 +02:00
panni e147a7a0ca use persistent Daemon mode; use correct bundle versioning; short: 1.0.9, build: 1.0.9.5 2015-10-14 13:16:18 +02:00
panni b494dc7bec cosmetic guessit update; add LICENSE and README 2015-10-14 12:49:10 +02:00
pannal 9ce4b02610 most likely fix punctuation issues with quotes in series names 2015-10-13 10:15:37 +02:00
pannal d0ff69d224 Update README.md 2015-10-11 04:17:56 +02:00
pannal cde09e0f56 add plex forum thread link 2015-10-11 04:17:39 +02:00
pannal 84409395d1 Update README.md 2015-10-11 03:36:40 +02:00
pannal e4e6bcfad2 Update README.md 2015-10-11 03:25:39 +02:00
panni 2103215e41 add dynamic animated logo from github 2015-10-11 03:24:17 +02:00
panni d086569f09 add correct plugin info; test animated subzero :) 2015-10-11 03:13:59 +02:00
panni 28064767ea update Info.plist 2015-10-11 02:42:53 +02:00
panni e996e4d4b6 replace default icon 2015-10-11 02:16:38 +02:00
pannal 422100f9fc Update README.md 2015-10-11 02:12:31 +02:00
pannal c9a7ffd778 Update README.md 2015-10-11 02:11:41 +02:00
pannal db009abf79 Merge pull request #30 from pannal/RC4
decouple from Subliminal.bundle
2015-10-11 02:07:24 +02:00
pannal c1cc7c98ef Update README.md 2015-10-11 02:06:31 +02:00
pannal a08b00d5c4 Update README.md 2015-10-11 02:06:17 +02:00
panni 16a22ab7b2 move more 2015-10-11 02:02:27 +02:00
panni da32ee2504 move moving 2015-10-11 02:01:36 +02:00
panni 54eaa9e695 move stuff 2015-10-11 02:00:11 +02:00
peter penis 28c1481a48 move to Sub-Zero; RC4; add LocalMediaExtended.bundle into SS 2015-10-11 01:57:48 +02:00
pannal cac340ad43 Update Info.plist 2015-10-11 01:53:05 +02:00
pannal d6994d9a60 Update README.md 2015-10-11 01:52:35 +02:00
pannal 90372ad30d Update DefaultPrefs.json 2015-10-10 14:43:12 +02:00
pannal 24fc22dbe6 Update DefaultPrefs.json 2015-10-10 14:42:39 +02:00
pannal 7b7adac774 Update README.md 2015-10-10 00:51:08 +02:00
pannal 7f0ff6ae2f Update README.md 2015-10-10 00:50:27 +02:00
pannal 1b3e58b326 Update README.md 2015-10-10 00:45:55 +02:00
pannal dc47fc60b8 Update README.md 2015-10-09 19:22:16 +02:00
pannal 6c588964a7 Update README.md 2015-10-09 02:42:20 +02:00
pannal f65b24094a Merge pull request #25 from pannal/rc3
pull RC3 into master
2015-10-09 02:36:57 +02:00
panni 6b807be0e6 opensubtitles: add optional credentials for VIPs; fixes #17 2015-10-09 02:35:33 +02:00
panni a794eb8310 providers: move punctuation fix into seperate mixins.py and use it 2015-10-09 02:08:43 +02:00
panni 8290c8a371 tvsubtitles: fix series with punctuation 2015-10-09 02:04:30 +02:00
panni 475152a7eb podnapisi: fix logging 2015-10-09 01:40:24 +02:00
panni 4e75e20ede add download retry option; fixes #24; move questionable only_one setting to the bottom 2015-10-09 01:28:56 +02:00
panni d36823c7ca better score logging; move patched providers to separate folder; better addic7ed punctuation handling in get_show_ids 2015-10-09 00:48:11 +02:00
panni 2a6b387112 addic7ed: fix series detection with punctuation; add missing self 2015-10-08 10:38:29 +02:00
panni a83822bff9 more verbose logging on subtitle download fail 2015-10-08 10:37:51 +02:00
panni 8e7538f6e6 fix broken import 2015-10-07 19:05:48 +02:00
panni 9cdb26f7cc forgot second clean_punctuation 2015-10-07 19:03:45 +02:00
panni 9659c913c4 Merge branch 'master' of github.com:pannal/Subliminal.bundle 2015-10-07 19:02:46 +02:00
panni c9506cb95e fix getting addic7ed show IDs for series with punctuation in their names 2015-10-07 19:02:33 +02:00
pannal 43e6ce3997 Update README.md 2015-10-07 05:13:36 +02:00
pannal dfd12edcb3 Update DefaultPrefs.json 2015-10-07 05:11:10 +02:00
pannal 154a8072f6 Update README.md 2015-10-07 04:07:59 +02:00
pannal 904abaf26b Update README.md 2015-10-07 02:58:32 +02:00
panni bea18a27ba set default TV score to 15; movie score to 30 2015-10-07 02:55:56 +02:00
pannal 2d998eab50 Update README.md 2015-10-07 02:47:40 +02:00
pannal a25a67572b Update README.md 2015-10-07 02:45:23 +02:00
pannal 1bdf6f9969 Merge pull request #22 from pannal/rc1-fix
RC1 fixes
2015-10-07 02:44:10 +02:00
panni 0b32892fa8 better existing subtitles debug logging 2015-10-07 02:42:14 +02:00
panni fea5b8a716 switch to tonswieb/enzyme 2015-10-07 02:06:47 +02:00
panni 90b3707409 update enzyme 2015-10-07 01:07:01 +02:00
panni 1c0224fbe7 skip empty folder creation if not subtitles found; should fix #20 2015-10-07 00:59:07 +02:00
36 changed files with 1879 additions and 219 deletions
+56
View File
@@ -0,0 +1,56 @@
RC-5.2
- revert back to /plexinc-agents/LocalMedia.bundle/tree/dist instead of /plexinc-agents/LocalMedia.bundle/tree/master, as the current public PMS version is too old for that
RC-5.1
- make hearing_impaired option more configurable and clear (see #configuration-)
RC-5
- fix wrong video type matching by hinting video type to guessit
- update to newest LocalMediaExtended.bundle (incorporated plex-inc's changes)
- show page links for subtitles in log file instead of subtitle ID
- add custom language setting in addition to the three hardcoded ones
- if a subtitle doesn't match our hearing_impaired setting, ignore it
- add an optional boost for addic7ed subtitles, if their series, season, episode, year, and format (e.g. WEB-DL) matches
RC-4
- rename project to Sub-Zero
- incorporate LocalMediaExtended.bundle
- making this a multi-bundle plugin
- update default scores
- add icon
RC-3
- addic7ed/tvsubtitles: punctuation fixes (correctly get show ids for series like "Mr. Poopster" now)
- podnapisi: fix logging
- opensubtitles: add login credentials (for VIPs)
- add retry functionality to retry failed subtitle downloads, including configurable amount of retries until discarding of provider
- move possibly not needed setting "Restrict to one language" to the bottom
- more detailed logging
- some cleanup
RC-2
- fix empty custom subtitle folder creation
- fix detection of existing embedded subtitles (switch to https://github.com/tonswieb/enzyme)
- better logging
- set default TV score to 15; movie score to 30
RC-1
- fix subliminal's logging error on min_score not met (fixes #15)
- separated tv and movies subtitle scores settings (fixes #16)
- add option to save only one subtitle per video (skipping the ".lang." naming scheme plex supports) (fixes #3)
beta5
- fix storing subtitles besides the actual video file, not subfolder (fixes #14)
- "custom folder" setting now always used if given (properly overrides "subtitle folder" setting)
- also scan (custom) given subtitle folders for existing subtitles instead of redownloading them on every refresh (fixes #9, #2)
beta4
- ~~increased score of addic7ed subtitles a bit~~ (not existing currently)
- **support for newest Subliminal ([1.0.1](27a6e51cd36ffb2910cd9a7add6d797a2c6469b7)) and guessit ([0.11.0](2814f57e8999dcc31575619f076c0c1a63ce78f2))**
- **plugin now also [works with com.plexapp.agents.thetvdbdvdorder](924470d2c0db3a71529278bce4b7247eaf2f85b8)**
- providers fixed for subliminal 1.0.1 ([at least addic7ed](131504e7eed8b3400c457fbe49beea3b115bc916))
- providers [don't simply fail and get excluded on non-detected language](1a779020792e0201ad689eefbf5a126155e89c97)
- support for addic7ed languages: [French (Canadian)](b11a051c233fd72033f0c3b5a8c1965260e7e19f)
- support for additional languages: [pt-br (Portuguese (Brasil)), fa (Persian (Farsi))](131504e7eed8b3400c457fbe49beea3b115bc916)
- support for [three (two optional) subtitle languages](e543c927cf49c264eaece36640c99d67a99c7da2)
- optionally use [random user agent for addic7ed provider](83ace14faf75fbd75313f0ceda9b78161895fbcf) (should not be needed)
+95 -27
View File
@@ -1,15 +1,22 @@
# hdbits.org
import string, os, urllib, zipfile, re, copy
from babelfish import Language
from datetime import timedelta
# coding=utf-8
import string
import os
import urllib
import zipfile
import re
import copy
import subliminal
import subliminal_patch
import subzero
import logger
from babelfish import Language
from datetime import timedelta
OS_PLEX_USERAGENT = 'plexapp.com v9.0'
DEPENDENCY_MODULE_NAMES = ['subliminal', 'subliminal_patch', 'enzyme', 'guessit', 'requests']
PERSONAL_MEDIA_IDENTIFIER = "com.plexapp.agents.none"
def Start():
HTTP.CacheTime = 0
@@ -19,8 +26,6 @@ def Start():
# configured cache to be in memory as per https://github.com/Diaoul/subliminal/issues/303
subliminal.region.configure('dogpile.cache.memory')
def ValidatePrefs():
Log.Debug("Validate Prefs called.")
return
@@ -28,12 +33,25 @@ def ValidatePrefs():
# Prepare a list of languages we want subs for
def getLangList():
langList = {Language.fromietf(Prefs["langPref1"])}
if(Prefs['subtitles.only_one']):
return langList
if(Prefs["langPref2"] != "None"):
langCustom = Prefs["langPrefCustom"].strip()
if Prefs["langPref2"] != "None":
langList.update({Language.fromietf(Prefs["langPref2"])})
if(Prefs["langPref3"] != "None"):
if Prefs["langPref3"] != "None":
langList.update({Language.fromietf(Prefs["langPref3"])})
if len(langCustom) and langCustom != "None":
for lang in langCustom.split(u","):
lang = lang.strip()
try:
real_lang = Language.fromietf(lang)
except:
try:
real_lang = Language.fromname(lang)
except:
continue
langList.update({real_lang})
return langList
@@ -48,6 +66,8 @@ def initSubliminalPatches():
# configure custom subtitle destination folders for scanning pre-existing subs
dest_folder = getSubtitleDestinationFolder()
subliminal_patch.patch_video.CUSTOM_PATHS = [dest_folder] if dest_folder else []
subliminal_patch.patch_provider_pool.DOWNLOAD_TRIES = int(Prefs['subtitles.try_downloads'])
subliminal_patch.patch_providers.addic7ed.USE_BOOST = bool(Prefs['provider.addic7ed.boost'])
def getProviders():
providers = {'opensubtitles' : Prefs['provider.opensubtitles.enabled'],
@@ -63,7 +83,10 @@ def getProviderSettings():
'password': Prefs['provider.addic7ed.password'],
'use_random_agents': Prefs['provider.addic7ed.use_random_agents'],
},
}
'opensubtitles': {'username': Prefs['provider.opensubtitles.username'],
'password': Prefs['provider.opensubtitles.password'],
},
}
return provider_settings
@@ -73,7 +96,7 @@ def scanTvMedia(media):
for episode in media.seasons[season].episodes:
for item in media.seasons[season].episodes[episode].items:
for part in item.parts:
scannedVideo = scanVideo(part)
scannedVideo = scanVideo(part, "episode")
videos[scannedVideo] = part
return videos
@@ -81,38 +104,55 @@ def scanMovieMedia(media):
videos = {}
for item in media.items:
for part in item.parts:
scannedVideo = scanVideo(part)
scannedVideo = scanVideo(part, "movie")
videos[scannedVideo] = part
return videos
def scanVideo(part):
def scanVideo(part, video_type):
embedded_subtitles = Prefs['subtitles.scan.embedded']
external_subtitles = Prefs['subtitles.scan.external']
Log.Debug("Scanning video: %s, subtitles=%s, embedded_subtitles=%s" % (part.file, external_subtitles, embedded_subtitles))
try:
return subliminal.video.scan_video(part.file, subtitles=external_subtitles, embedded_subtitles=embedded_subtitles)
return subliminal.video.scan_video(part.file, subtitles=external_subtitles, embedded_subtitles=embedded_subtitles, video_type=video_type)
except ValueError:
Log.Warn("File could not be guessed by subliminal")
def downloadBestSubtitles(videos, min_score=0):
hearing_impaired = Prefs['subtitles.search.hearingImpaired']
Log.Debug("Download best subtitles using settings: min_score: %s, hearing_impaired: %s" %(min_score, hearing_impaired))
languages = getLangList()
if not languages:
return
missing_languages = False
for video in videos:
if not (languages - video.subtitle_languages):
Log.Debug('All languages %r exist for %s', languages, video)
continue
missing_languages = True
break
if missing_languages:
Log.Debug("Download best subtitles using settings: min_score: %s, hearing_impaired: %s" %(min_score, hearing_impaired))
return subliminal.api.download_best_subtitles(videos, getLangList(), min_score, hearing_impaired, providers=getProviders(), provider_configs=getProviderSettings(), only_one=Prefs['subtitles.only_one'])
return subliminal.api.download_best_subtitles(videos, languages, min_score, hearing_impaired, providers=getProviders(), provider_configs=getProviderSettings())
Log.Debug("All languages for all requested videos exist. Doing nothing.")
def saveSubtitles(videos, subtitles):
if Prefs['subtitles.save.filesystem']:
Log.Debug("Saving subtitles to filesystem")
Log.Debug("Using filesystem as subtitle storage")
saveSubtitlesToFile(subtitles)
else:
Log.Debug("Saving subtitles as metadata")
Log.Debug("Using metadata as subtitle storage")
saveSubtitlesToMetadata(videos, subtitles)
def saveSubtitlesToFile(subtitles):
fld_custom = Prefs["subtitles.save.subFolder.Custom"].strip() if bool(Prefs["subtitles.save.subFolder.Custom"]) else None
for video, video_subtitles in subtitles.items():
if not video_subtitles:
continue
fld = None
if fld_custom or Prefs["subtitles.save.subFolder"] != "current folder":
# specific subFolder requested, create it if it doesn't exist
@@ -127,7 +167,7 @@ def saveSubtitlesToFile(subtitles):
fld = os.path.join(fld_base, Prefs["subtitles.save.subFolder"])
if not os.path.exists(fld):
os.makedirs(fld)
subliminal.api.save_subtitles(video, video_subtitles, directory=fld, single=Prefs['subtitles.only_one'])
subliminal.api.save_subtitles(video, video_subtitles, directory=fld)
def saveSubtitlesToMetadata(videos, subtitles):
for video, video_subtitles in subtitles.items():
@@ -135,8 +175,30 @@ def saveSubtitlesToMetadata(videos, subtitles):
for subtitle in video_subtitles:
mediaPart.subtitles[Locale.Language.Match(subtitle.language.alpha2)][subtitle.page_link] = Proxy.Media(subtitle.content, ext="srt")
class SubliminalSubtitlesAgentMovies(Agent.Movies):
name = 'Subliminal Movie Subtitles'
def updateLocalMedia(media, media_type="movies"):
# Look for subtitles
if media_type == "movies":
for item in media.items:
for part in item.parts:
subzero.localmedia.findSubtitles(part)
return
# Look for subtitles for each episode.
for s in media.seasons:
# If we've got a date based season, ignore it for now, otherwise it'll collide with S/E folders/XML and PMS
# prefers date-based (why?)
if int(s) < 1900 or metadata.guid.startswith(PERSONAL_MEDIA_IDENTIFIER):
for e in media.seasons[s].episodes:
for i in media.seasons[s].episodes[e].items:
# Look for subtitles.
for part in i.parts:
subzero.localmedia.findSubtitles(part)
else:
pass
class SubZeroSubtitlesAgentMovies(Agent.Movies):
name = 'Sub-Zero Subtitles (Movies)'
languages = [Locale.Language.English]
primary_provider = False
contributes_to = ['com.plexapp.agents.imdb']
@@ -150,11 +212,14 @@ class SubliminalSubtitlesAgentMovies(Agent.Movies):
initSubliminalPatches()
videos = scanMovieMedia(media)
subtitles = downloadBestSubtitles(videos.keys(), min_score=int(Prefs["subtitles.search.minimumMovieScore"]))
saveSubtitles(videos, subtitles)
if subtitles:
saveSubtitles(videos, subtitles)
class SubliminalSubtitlesAgentTvShows(Agent.TV_Shows):
updateLocalMedia(media, media_type="movies")
class SubZeroSubtitlesAgentTvShows(Agent.TV_Shows):
name = 'Subliminal TV Subtitles'
name = 'Sub-Zero Subtitles (TV)'
languages = [Locale.Language.English]
primary_provider = False
contributes_to = ['com.plexapp.agents.thetvdb', 'com.plexapp.agents.thetvdbdvdorder']
@@ -168,4 +233,7 @@ class SubliminalSubtitlesAgentTvShows(Agent.TV_Shows):
initSubliminalPatches()
videos = scanTvMedia(media)
subtitles = downloadBestSubtitles(videos.keys(), min_score=int(Prefs["subtitles.search.minimumTVScore"]))
saveSubtitles(videos, subtitles)
if subtitles:
saveSubtitles(videos, subtitles)
updateLocalMedia(media, media_type="series")
+6
View File
@@ -0,0 +1,6 @@
License for parts taken out of plexinc-agents/LocalMedia.bundle
License
-------
If the software submitted to this repository accesses or calls any software provided by Plex (“Interfacing Software”), then as a condition for receiving services from Plex in response to such accesses or calls, you agree to grant and do hereby grant to Plex and its affiliates worldwide a worldwide, nonexclusive, and royalty-free right and license to use (including testing, hosting and linking to), copy, publicly perform, publicly display, reproduce in copies for distribution, and distribute the copies of any Interfacing Software made by you or with your assistance; provided, however, that you may notify Plex at legal@plex.tv if you do not wish for Plex to use, distribute, copy, publicly perform, publicly display, reproduce in copies for distribution, or distribute copies of an Interfacing Software that was created by you, and Plex will reasonable efforts to comply with such a request within a reasonable time.
+16
View File
@@ -0,0 +1,16 @@
import sys
# thanks, https://github.com/trakt/Plex-Trakt-Scrobbler/blob/master/Trakttv.bundle/Contents/Code/core/__init__.py
import config
sys.modules["subzero.config"] = config
import helpers
sys.modules["subzero.helpers"] = helpers
import localmedia
sys.modules["subzero.localmedia"] = localmedia
import subtitlehelpers
sys.modules["subzero.subtitlehelpers"] = subtitlehelpers
+4
View File
@@ -0,0 +1,4 @@
SUBTITLE_EXTS = ['utf','utf8','utf-8','srt','smi','rt','ssa','aqt','jss','ass','idx','sub','txt', 'psb']
VIDEO_EXTS = ['3g2', '3gp', 'asf', 'asx', 'avc', 'avi', 'avs', 'bivx', 'bup', 'divx', 'dv', 'dvr-ms', 'evo', 'fli', 'flv',
'm2t', 'm2ts', 'm2v', 'm4v', 'mkv', 'mov', 'mp4', 'mpeg', 'mpg', 'mts', 'nsv', 'nuv', 'ogm', 'ogv', 'tp',
'pva', 'qt', 'rm', 'rmvb', 'sdp', 'svq3', 'strm', 'ts', 'ty', 'vdr', 'viv', 'vob', 'vp3', 'wmv', 'wpl', 'wtv', 'xsp', 'xvid', 'webm']
+36
View File
@@ -0,0 +1,36 @@
# coding=utf-8
import unicodedata
# Unicode control characters can appear in ID3v2 tags but are not legal in XML.
RE_UNICODE_CONTROL = u'([\u0000-\u0008\u000b-\u000c\u000e-\u001f\ufffe-\uffff])' + \
u'|' + \
u'([%s-%s][^%s-%s])|([^%s-%s][%s-%s])|([%s-%s]$)|(^[%s-%s])' % \
(
unichr(0xd800),unichr(0xdbff),unichr(0xdc00),unichr(0xdfff),
unichr(0xd800),unichr(0xdbff),unichr(0xdc00),unichr(0xdfff),
unichr(0xd800),unichr(0xdbff),unichr(0xdc00),unichr(0xdfff)
)
# A platform independent way to split paths which might come in with different separators.
def splitPath(str):
if str.find('\\') != -1:
return str.split('\\')
else:
return str.split('/')
def unicodize(s):
filename = s
try:
filename = unicodedata.normalize('NFC', unicode(s.decode('utf-8')))
except:
Log('Failed to unicodize: ' + filename)
try:
filename = re.sub(RE_UNICODE_CONTROL, '', filename)
except:
Log('Couldn\'t strip control characters: ' + filename)
return filename
def cleanFilename(filename):
#this will remove any whitespace and punctuation chars and replace them with spaces, strip and return as lowercase
return string.translate(filename.encode('utf-8'), string.maketrans(string.punctuation + string.whitespace, ' ' * len (string.punctuation + string.whitespace))).strip().lower()
+107
View File
@@ -0,0 +1,107 @@
# coding=utf-8
import os, unicodedata
import config
import helpers
import subtitlehelpers
def findSubtitles(part):
lang_sub_map = {}
part_filename = helpers.unicodize(part.file)
part_basename = os.path.splitext(os.path.basename(part_filename))[0]
paths = [ os.path.dirname(part_filename) ]
# Check for local subtitles subdirectory
sub_dirs_default = ["sub", "subs", "subtitle", "subtitles"]
sub_dir_base = paths[0]
sub_dir_list = []
if Prefs["subtitles.save.subFolder"] != "current folder":
# got selected subfolder
sub_dir_list.append(os.path.join(sub_dir_base, Prefs["subtitles.save.subFolder"]))
sub_dir_custom = Prefs["subtitles.save.subFolder.Custom"].strip() if bool(Prefs["subtitles.save.subFolder.Custom"]) else None
if sub_dir_custom:
# got custom subfolder
if sub_dir_custom.startswith("/"):
# absolute folder
sub_dir_list.append(sub_dir_custom)
else:
# relative folder
sub_dir_list.append(os.path.join(sub_dir_base, sub_dir_custom))
for sub_dir in sub_dir_list:
if os.path.isdir(sub_dir):
paths.append(sub_dir)
# Check for a global subtitle location
global_subtitle_folder = os.path.join(Core.app_support_path, 'Subtitles')
if os.path.exists(global_subtitle_folder):
paths.append(global_subtitle_folder)
# We start by building a dictionary of files to their absolute paths. We also need to know
# the number of media files that are actually present, in case the found local media asset
# is limited to a single instance per media file.
#
file_paths = {}
total_media_files = 0
for path in paths:
path = helpers.unicodize(path)
for file_path_listing in os.listdir(path):
# When using os.listdir with a unicode path, it will always return a string using the
# NFD form. However, we internally are using the form NFC and therefore need to convert
# it to allow correct regex / comparisons to be performed.
#
file_path_listing = helpers.unicodize(file_path_listing)
if os.path.isfile(os.path.join(path, file_path_listing)):
file_paths[file_path_listing.lower()] = os.path.join(path, file_path_listing)
# If we've found an actual media file, we should record it.
(root, ext) = os.path.splitext(file_path_listing)
if ext.lower()[1:] in config.VIDEO_EXTS:
total_media_files += 1
Log('Looking for subtitle media in %d paths with %d media files.', len(paths), total_media_files)
Log('Paths: %s', ", ".join([ helpers.unicodize(p) for p in paths ]))
for file_path in file_paths.values():
local_basename = helpers.unicodize(os.path.splitext(os.path.basename(file_path))[0])
local_basename2 = local_basename.rsplit('.', 1)[0]
filename_matches_part = local_basename == part_basename or local_basename2 == part_basename
# If the file is located within the global subtitle folder and it's name doesn't match exactly
# then we should simply ignore it.
#
if file_path.count(global_subtitle_folder) and not filename_matches_part:
continue
# If we have more than one media file within the folder and located filename doesn't match
# exactly then we should simply ignore it.
#
if total_media_files > 1 and not filename_matches_part:
continue
subtitle_helper = subtitlehelpers.SubtitleHelpers(file_path)
if subtitle_helper != None:
local_lang_map = subtitle_helper.process_subtitles(part)
for new_language, subtitles in local_lang_map.items():
# Add the possible new language along with the located subtitles so that we can validate them
# at the end...
#
if not lang_sub_map.has_key(new_language):
lang_sub_map[new_language] = []
lang_sub_map[new_language] = lang_sub_map[new_language] + subtitles
# Now whack subtitles that don't exist anymore.
for language in lang_sub_map.keys():
part.subtitles[language].validate_keys(lang_sub_map[language])
# Now whack the languages that don't exist anymore.
for language in list(set(part.subtitles.keys()) - set(lang_sub_map.keys())):
part.subtitles[language].validate_keys({})
+132
View File
@@ -0,0 +1,132 @@
# coding=utf-8
import re, unicodedata, os
import config
import helpers
class SubtitleHelper(object):
def __init__(self, filename):
self.filename = filename
def SubtitleHelpers(filename):
filename = helpers.unicodize(filename)
for cls in [ VobSubSubtitleHelper, DefaultSubtitleHelper ]:
if cls.is_helper_for(filename):
return cls(filename)
return None
#####################################################################################################################
class VobSubSubtitleHelper(SubtitleHelper):
@classmethod
def is_helper_for(cls, filename):
(file, file_extension) = os.path.splitext(filename)
# We only support idx (and maybe sub)
if not file_extension.lower() in ['.idx', '.sub']:
return False
# If we've been given a sub, we only support it if there exists a matching idx file
return os.path.exists(file + '.idx')
def process_subtitles(self, part):
lang_sub_map = {}
# We don't directly process the sub file, only the idx. Therefore if we are passed on of these files, we simply
# ignore it.
(file, ext) = os.path.splitext(self.filename)
if ext == '.sub':
return lang_sub_map
# If we have an idx file, we need to confirm there is an identically names sub file before we can proceed.
sub_filename = file + ".sub"
if os.path.exists(sub_filename) == False:
return lang_sub_map
Log('Attempting to parse VobSub file: ' + self.filename)
idx = Core.storage.load(os.path.join(self.filename))
if idx.count('VobSub index file') == 0:
Log('The idx file does not appear to be a VobSub, skipping...')
return lang_sub_map
languages = {}
language_index = 0
basename = os.path.basename(self.filename)
for language in re.findall('\nid: ([A-Za-z]{2})', idx):
if not languages.has_key(language):
languages[language] = []
Log('Found .idx subtitle file: ' + self.filename + ' language: ' + language + ' stream index: ' + str(language_index))
languages[language].append(Proxy.LocalFile(self.filename, index = str(language_index), format = "vobsub"))
language_index += 1
if not lang_sub_map.has_key(language):
lang_sub_map[language] = []
lang_sub_map[language].append(basename)
for language, subs in languages.items():
part.subtitles[language][basename] = subs
return lang_sub_map
#####################################################################################################################
class DefaultSubtitleHelper(SubtitleHelper):
@classmethod
def is_helper_for(cls, filename):
(file, file_extension) = os.path.splitext(filename)
return file_extension.lower()[1:] in config.SUBTITLE_EXTS
def process_subtitles(self, part):
lang_sub_map = {}
basename = os.path.basename(self.filename)
(file, ext) = os.path.splitext(self.filename)
# Remove the initial '.' from the extension
ext = ext[1:]
# Attempt to extract the language from the filename (e.g. Avatar (2009).eng)
language = ""
# IETF support thanks to https://github.com/hpsbranco/LocalMedia.bundle/commit/4fad9aefedece78a1fa96401304351347f644369
language_match = re.match(".+\.([^\.]+)$" if not Prefs["subtitles.language.ietf"] else ".+\.([^-.]+)(?:-[A-Za-z]+)?$", file)
if language_match and len(language_match.groups()) == 1:
language = language_match.groups()[0]
language = Locale.Language.Match(language)
codec = None
format = None
if ext in ['txt', 'sub']:
try:
file_contents = Core.storage.load(self.filename)
lines = [ line.strip() for line in file_contents.splitlines(True) ]
if re.match('^\{[0-9]+\}\{[0-9]*\}', lines[1]):
format = 'microdvd'
elif re.match('^[0-9]{1,2}:[0-9]{2}:[0-9]{2}[:=,]', lines[1]):
format = 'txt'
elif '[SUBTITLE]' in lines[1]:
format = 'subviewer'
else:
Log("The subtitle file does not have a known format, skipping... : " + self.filename)
return lang_sub_map
except:
Log("An error occurred while attempting to parse the subtitle file, skipping... : " + self.filename)
return lang_sub_map
if codec is None and ext in ['ass', 'ssa', 'smi', 'srt', 'psb']:
codec = ext.replace('ass', 'ssa')
if format is None:
format = codec
Log('Found subtitle file: ' + self.filename + ' language: ' + language + ' codec: ' + str(codec) + ' format: ' + str(format))
part.subtitles[language][basename] = Proxy.LocalFile(self.filename, codec = codec, format = format)
lang_sub_map[language] = [ basename ]
return lang_sub_map
+47 -14
View File
@@ -1,5 +1,11 @@
[
{
{ "id": "subtitles.try_downloads",
"label": "How many download tries per subtitle (on timeout or error)",
"type": "enum",
"values": ["1", "2", "3", "4"],
"default": "2"
},
{
"id": "provider.addic7ed.username",
"label": "Addic7ed Username",
"type": "text",
@@ -13,6 +19,20 @@
"default": "",
"secure": "true"
},
{
"id": "provider.opensubtitles.username",
"label": "Opensubtitles Username (VIP)",
"type": "text",
"default": ""
},
{
"id": "provider.opensubtitles.password",
"label": "Opensubtitles Password",
"type": "text",
"option": "hidden",
"default": "",
"secure": "true"
},
{
"id": "provider.addic7ed.use_random_agents",
"label": "Addic7ed: Use random user agents (should not be necessary)",
@@ -41,10 +61,10 @@
"default": "None"
},
{
"id": "subtitles.only_one",
"label": "Restrict to one language (skips adding \".lang.\" to the subtitle filename; only uses \"Subtitle Language (1)\")",
"type": "bool",
"default": "false"
"id": "langPrefCustom",
"label": "Additional Subtitle Languages (use ISO-639-1 codes; comma-separated)",
"type": "text",
"default": "None"
},
{
"id": "provider.opensubtitles.enabled",
@@ -69,6 +89,12 @@
"label": "Provider: Enable Addic7ed",
"type": "bool",
"default": "true"
},
{
"id": "provider.addic7ed.boost",
"label": "Addic7ed: boost over hash score if requirements met (prefer over other providers)",
"type": "bool",
"default": "false"
},
{
"id": "provider.tvsubtitles.enabled",
@@ -80,51 +106,58 @@
"id": "subtitles.scan.embedded",
"label": "Scan: include embedded subtitles (skip if existing)",
"type": "bool",
"default": "false"
"default": "true"
},
{
"id": "subtitles.scan.external",
"label": "Scan: include external subtitles (skip if existing)",
"type": "bool",
"default": "false"
"default": "true"
},
{
"id": "subtitles.search.minimumTVScore",
"label": "Minimum score for TV subtitles to download",
"type": "enum",
"values": ["100","95","90","85","80","75","70","65","60","55","50","45","40","35","30","25","20","15","10","5","0"],
"default": "40"
"default": "80"
},
{
"id": "subtitles.search.minimumMovieScore",
"label": "Minimum score for movie subtitles to download",
"type": "enum",
"values": ["100","95","90","85","80","75","70","65","60","55","50","45","40","35","30","25","20","15","10","5","0"],
"default": "60"
"default": "35"
},
{
"id": "subtitles.search.hearingImpaired",
"label": "Download hearing impaired subtitles.",
"type": "bool",
"default": "false"
"type": "enum",
"values": ["prefer", "don't prefer", "force HI", "force non-HI"],
"default": "don't prefer"
},
{
"id": "subtitles.save.filesystem",
"label": "Store subtitles next to media files (instead of metadata)",
"type": "bool",
"default": "false"
"default": "true"
},
{
"id": "subtitles.save.subFolder",
"label": "Subtitle Folder (\"current folder\" is the folder the current media file lives in) - needs LocalMediaExtended agent",
"label": "Subtitle Folder (\"current folder\" is the folder the current media file lives in)",
"type": "enum",
"values": ["current folder", "sub", "subs", "subtitle", "subtitles"],
"default": "current folder"
},
{
"id": "subtitles.save.subFolder.Custom",
"label": "Custom Subtitle folder (overrides \"Subtitle Folder\"; computes to real paths; use for example \"bla\" as a subfolder of the current media file folder or an absolute path) - needs LocalMediaExtended agent",
"label": "Custom Subtitle folder (overrides \"Subtitle Folder\"; computes to real paths)",
"type": "text",
"default": ""
},
{
"id": "subtitles.language.ietf",
"label": "Treat IETF language tags as ISO 639-1 (e.g. pt-BR = pt)",
"type": "bool",
"default": "true"
}
]
+20 -6
View File
@@ -4,24 +4,22 @@
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>English</string>
<key>CFBundleExecutable</key>
<string>Test Plug-in</string>
<key>CFBundleIdentifier</key>
<string>com.plexapp.agents.subliminal</string>
<string>com.plexapp.agents.subzero</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleShortVersionString</key>
<string>1.0</string>
<string>1.1.0</string>
<key>CFBundleSignature</key>
<string>????</string>
<key>CFBundleVersion</key>
<string>1.0</string>
<string>1.1.0.3</string>
<key>PlexFrameworkVersion</key>
<string>2</string>
<key>PlexPluginClass</key>
<string>Agent</string>
<key>PlexPluginMode</key>
<string>AlwaysOn</string>
<string>Daemon</string>
<key>PlexPluginConsoleLogging</key>
<string>1</string>
<key>PlexPluginDevMode</key>
@@ -29,5 +27,21 @@
<key>PlexPluginCodePolicy</key>
<!-- this allows channels to access some python methods which are otherwise blocked, as well as import external code libraries, and interact with the PMS HTTP API -->
<string>Elevated</string>
<key>PlexAgentAttributionText</key>
<string>&lt;div style=&quot;white-space: pre;&quot;&gt;&lt;img src=&quot;https://raw.githubusercontent.com/pannal/Sub-Zero/master/Contents/Resources/subzero.gif&quot; /&gt;
&lt;h1&gt;Sub-Zero for Plex&lt;/h1&gt;&lt;i&gt;Subtitles done right&lt;/i&gt;
Version 1.1.0.3
Originally based on @bramwalet's awesome &lt;a href=&quot;https://github.com/bramwalet/Subliminal.bundle&quot;&gt;Subliminal.bundle&lt;/a&gt;
&lt;strong&gt;Need help?&lt;/strong&gt;
Plex thread: &lt;a href=&quot;https://forums.plex.tv/discussion/186575&quot;>https://forums.plex.tv/discussion/186575&lt;/a&gt;
Github: &lt;a href=&quot;https://github.com/pannal/Sub-Zero&quot;&gt;https://github.com/pannal/Sub-Zero&lt;/a&gt;
panni, 2015
&lt;/div&gt;
</string>
</dict>
</plist>
@@ -8,6 +8,7 @@ __copyright__ = 'Copyright 2013 Antoine Bertin'
import logging
from .exceptions import *
from .mkv import *
from .subtitle import *
logging.getLogger(__name__).addHandler(logging.NullHandler())
+42 -18
View File
@@ -65,30 +65,53 @@ class MKV(object):
continue
if element_name == 'Info':
logger.info('Processing element %s from SeekHead at position %d', element_name, element_position)
stream.seek(element_position)
self.info = Info.fromelement(ebml.parse_element(stream, specs, True, ignore_element_names=['Void', 'CRC-32']))
element = self._load_element(stream, specs, element_position)
self.info = Info.fromelement(element)
elif element_name == 'Tracks':
logger.info('Processing element %s from SeekHead at position %d', element_name, element_position)
stream.seek(element_position)
tracks = ebml.parse_element(stream, specs, True, ignore_element_names=['Void', 'CRC-32'])
tracks = self._load_element(stream, specs, element_position)
self.video_tracks.extend([VideoTrack.fromelement(t) for t in tracks if t['TrackType'].data == VIDEO_TRACK])
self.audio_tracks.extend([AudioTrack.fromelement(t) for t in tracks if t['TrackType'].data == AUDIO_TRACK])
self.subtitle_tracks.extend([SubtitleTrack.fromelement(t) for t in tracks if t['TrackType'].data == SUBTITLE_TRACK])
elif element_name == 'Chapters':
logger.info('Processing element %s from SeekHead at position %d', element_name, element_position)
stream.seek(element_position)
self.chapters.extend([Chapter.fromelement(c) for c in ebml.parse_element(stream, specs, True, ignore_element_names=['Void', 'CRC-32'])[0] if c.name == 'ChapterAtom'])
element = self._load_element(stream, specs, element_position)
self.chapters.extend([Chapter.fromelement(c) for c in element[0] if c.name == 'ChapterAtom'])
elif element_name == 'Tags':
logger.info('Processing element %s from SeekHead at position %d', element_name, element_position)
stream.seek(element_position)
self.tags.extend([Tag.fromelement(t) for t in ebml.parse_element(stream, specs, True, ignore_element_names=['Void', 'CRC-32'])])
element = self._load_element(stream, specs, element_position)
self.tags.extend([Tag.fromelement(t) for t in element])
elif element_name == 'SeekHead' and self.recurse_seek_head:
logger.info('Processing element %s from SeekHead at position %d', element_name, element_position)
stream.seek(element_position)
self._parse_seekhead(ebml.parse_element(stream, specs, True, ignore_element_names=['Void', 'CRC-32']), segment, stream, specs)
element = self._load_element(stream, specs, element_position)
self._parse_seekhead(element, segment, stream, specs)
else:
logger.debug('Element %s ignored', element_name)
self._parsed_positions.add(element_position)
def _load_element(self,stream, specs, position):
stream.seek(position)
element = ebml.parse_element(stream,specs)
element.load(stream, specs, ignore_element_names=['Void', 'CRC-32'])
return element
def get_srt_subtitles_track_by_language(self):
"""get a dictionary of the SRT subtitles track id's indexed by language"""
subtitles = dict()
for track in self.subtitle_tracks:
logger.info("Found subtitle language %s, with codec %s and lacing %s",
track.language,track.codec_id,track.lacing)
if not track.is_srt():
logger.debug("Ignoring subtitle language %s with codec %s",track.language,track.codec_id)
elif track.lacing:
logger.info("Ignoring subtitle language %s with lacing %s",track.language,track.lacing)
else:
subtitles[track.language] = track
return subtitles
def to_dict(self):
return {'info': self.info.__dict__, 'video_tracks': [t.__dict__ for t in self.video_tracks],
@@ -103,6 +126,7 @@ class Info(object):
"""Object for the Info EBML element"""
def __init__(self, title=None, duration=None, date_utc=None, timecode_scale=None, muxing_app=None, writing_app=None):
self.title = title
self.timecode_scale = timecode_scale
self.duration = timedelta(microseconds=duration * (timecode_scale or 1000000) // 1000) if duration else None
self.date_utc = date_utc
self.muxing_app = muxing_app
@@ -119,7 +143,7 @@ class Info(object):
title = element.get('Title')
duration = element.get('Duration')
date_utc = element.get('DateUTC')
timecode_scale = element.get('TimecodeScale')
timecode_scale = element.get('TimecodeScale',1000000)
muxing_app = element.get('MuxingApp')
writing_app = element.get('WritingApp')
return cls(title, duration, date_utc, timecode_scale, muxing_app, writing_app)
@@ -133,7 +157,7 @@ class Info(object):
class Track(object):
"""Base object for the Tracks EBML element"""
def __init__(self, type=None, number=None, name=None, language=None, enabled=None, default=None, forced=None, lacing=None, # @ReservedAssignment
def __init__(self, type=None, number=None, name=None, language=None, enabled=None, default=None, forced=None, lacing=None,
codec_id=None, codec_name=None):
self.type = type
self.number = number
@@ -154,10 +178,10 @@ class Track(object):
:type element: :class:`~enzyme.parsers.ebml.Element`
"""
type = element.get('TrackType') # @ReservedAssignment
type = element.get('TrackType')
number = element.get('TrackNumber', 0)
name = element.get('Name')
language = element.get('Language')
language = element.get('Language','eng')
enabled = bool(element.get('FlagEnabled', 1))
default = bool(element.get('FlagDefault', 1))
forced = bool(element.get('FlagForced', 0))
@@ -256,8 +280,9 @@ class AudioTrack(Track):
class SubtitleTrack(Track):
"""Object for the Tracks EBML element with :data:`SUBTITLE_TRACK` TrackType"""
pass
def is_srt(self):
return self.codec_id == 'S_TEXT/UTF8'
class Tag(object):
"""Object for the Tag EBML element"""
@@ -344,8 +369,7 @@ class Chapter(object):
if chapterdisplays:
string = chapterdisplays[0].get('ChapString')
language = chapterdisplays[0].get('ChapLanguage')
return cls(start, hidden, enabled, end, string, language)
return cls(start, hidden, enabled, end)
return cls(start, hidden, enabled, end, string, language)
def __repr__(self):
return '<%s [%s, enabled=%s]>' % (self.__class__.__name__, self.start, self.enabled)
@@ -38,8 +38,15 @@ READERS = {
BINARY: read_element_binary
}
class BaseElement(object):
class Element(object):
def __init__(self, id=None, position=None, size=None, data=None):
self.id = id
self.position = position
self.size = size
self.data = data
class Element(BaseElement):
"""Base object of EBML
:param int id: id of the element, best represented as hexadecimal (0x18538067 for Matroska Segment element)
@@ -52,14 +59,11 @@ class Element(object):
:param data: data as read by the corresponding :data:`READERS`
"""
def __init__(self, id=None, type=None, name=None, level=None, position=None, size=None, data=None): # @ReservedAssignment
self.id = id
def __init__(self, id=None, type=None, name=None, level=None, position=None, size=None, data=None):
super(Element, self).__init__(id, position, size, data)
self.type = type
self.name = name
self.level = level
self.position = position
self.size = size
self.data = data
def __repr__(self):
return '<%s [%s, %r]>' % (self.__class__.__name__, self.name, self.data)
@@ -89,7 +93,7 @@ class MasterElement(Element):
Element(DocType, u'matroska')
"""
def __init__(self, id=None, name=None, level=None, position=None, size=None, data=None): # @ReservedAssignment
def __init__(self, id=None, name=None, level=None, position=None, size=None, data=None):
super(MasterElement, self).__init__(id, MASTER, name, level, position, size, data)
def load(self, stream, specs, ignore_element_types=None, ignore_element_names=None, max_level=None):
@@ -137,8 +141,7 @@ class MasterElement(Element):
def __iter__(self):
return iter(self.data)
def parse(stream, specs, size=None, ignore_element_types=None, ignore_element_names=None, max_level=None):
def parse(stream, specs, size=None, ignore_element_types=None, ignore_element_names=None, max_level=None, include_element_names=None):
"""Parse a stream for `size` bytes according to the `specs`
:param stream: file-like object from which to read
@@ -148,6 +151,7 @@ def parse(stream, specs, size=None, ignore_element_types=None, ignore_element_na
:param list ignore_element_types: list of element types to ignore
:param list ignore_element_names: list of element names to ignore
:param int max_level: maximum level of elements
:param list include_element_names: list of element names to include exclusively, so ignoring all other element names
:return: parsed data as a tree of :class:`~enzyme.parsers.ebml.core.Element`
:rtype: list
@@ -158,26 +162,32 @@ def parse(stream, specs, size=None, ignore_element_types=None, ignore_element_na
"""
ignore_element_types = ignore_element_types if ignore_element_types is not None else []
ignore_element_names = ignore_element_names if ignore_element_names is not None else []
include_element_names = include_element_names if include_element_names is not None else []
start = stream.tell()
elements = []
while size is None or stream.tell() - start < size:
try:
element = parse_element(stream, specs)
if element is None:
if element.type is None:
logger.error('Element with id 0x%x is not in the specs' % element_id)
stream.seek(element_size, 1)
continue
logger.debug('%s %s parsed', element.__class__.__name__, element.name)
if element.type in ignore_element_types or element.name in ignore_element_names:
logger.info('%s %s ignored', element.__class__.__name__, element.name)
if element.type == MASTER:
stream.seek(element.size, 1)
elif element.type in ignore_element_types or element.name in ignore_element_names:
logger.info('%s %s %s ignored', element.__class__.__name__, element.name, element.type)
stream.seek(element.size, 1)
continue
if element.type == MASTER:
elif len(include_element_names) > 0 and element.name not in include_element_names:
stream.seek(element.size, 1)
continue
elif element.type == MASTER:
if max_level is not None and element.level >= max_level:
logger.info('Maximum level %d reached for children of %s %s', max_level, element.__class__.__name__, element.name)
stream.seek(element.size, 1)
else:
logger.debug('Loading child elements for %s %s with size %d', element.__class__.__name__, element.name, element.size)
element.data = parse(stream, specs, element.size, ignore_element_types, ignore_element_names, max_level)
element.data = parse(stream, specs, element.size, ignore_element_types, ignore_element_names, max_level,include_element_names)
else:
element.data = READERS[element.type](stream, element.size)
elements.append(element)
except ReadError:
if size is not None:
@@ -186,21 +196,15 @@ def parse(stream, specs, size=None, ignore_element_types=None, ignore_element_na
return elements
def parse_element(stream, specs, load_children=False, ignore_element_types=None, ignore_element_names=None, max_level=None):
def parse_element(stream, specs):
"""Extract a single :class:`Element` from the `stream` according to the `specs`
:param stream: file-like object from which to read
:param dict specs: see :ref:`specs`
:param bool load_children: load children elements if the parsed element is a :class:`MasterElement`
:param list ignore_element_types: list of element types to ignore
:param list ignore_element_names: list of element names to ignore
:param int max_level: maximum level for children elements
:return: the parsed element
:rtype: :class:`Element`
"""
ignore_element_types = ignore_element_types if ignore_element_types is not None else []
ignore_element_names = ignore_element_names if ignore_element_names is not None else []
element_id = read_element_id(stream)
if element_id is None:
raise ReadError('Cannot read element id')
@@ -208,20 +212,14 @@ def parse_element(stream, specs, load_children=False, ignore_element_types=None,
if element_size is None:
raise ReadError('Cannot read element size')
if element_id not in specs:
logger.error('Element with id 0x%x is not in the specs' % element_id)
stream.seek(element_size, 1)
return None
return BaseElement(element_id,stream.tell(),element_size)
element_type, element_name, element_level = specs[element_id]
if element_type == MASTER:
element = MasterElement(element_id, element_name, element_level, stream.tell(), element_size)
if load_children:
element.data = parse(stream, specs, element.size, ignore_element_types, ignore_element_names, max_level)
else:
element = Element(element_id, element_type, element_name, element_level, stream.tell(), element_size)
element.data = READERS[element_type](stream, element_size)
return element
def get_matroska_specs(webm_only=False):
"""Get the Matroska specs
@@ -0,0 +1,185 @@
# -*- coding: utf-8 -*-
from .exceptions import ReadError
from .parsers import ebml
from .mkv import MKV
from .parsers import ebml
import logging
import codecs
import os
import io
__all__ = ['Subtitle']
logger = logging.getLogger(__name__)
class Subtitle(object):
"""Subtitle extractor for Matroska Video File.
Currently only SRT subtitles stored without lacing are supported
"""
def __init__(self, stream):
"""Read the available subtitles from a MKV file-like object"""
self._stream = stream
#Use the MKV class to parse the META information
mkv = MKV(stream)
self._timecode_scale = mkv.info.timecode_scale
self._subtitles = mkv.get_srt_subtitles_track_by_language()
def has_subtitle(self, language):
return language in self._subtitles
def write_subtitle_to_stream(self, language):
"""Write a single subtitle to stream or return None if language not available"""
if language in self._subtitles:
subtitle = self._subtitles[language]
return _write_track_to_srt_stream(self._stream,subtitle.number,self._timecode_scale)
logger.info("Writing subtitle for language %s to stream",language)
else:
logger.info("Subtitle for language %s not found",language)
def write_subtitles_to_stream(self):
"""Write all available subtitles as streams to a dictionary with language as the key"""
subtitles = dict()
for language in self._subtitles:
subtitles[language] = self.write_subtitle_to_stream(language)
return subtitles
def _write_track_to_srt_stream(mkv_stream, track, timecode_scale):
srt_stream = io.StringIO()
index = 0
for cluster in _parse_segment(mkv_stream,track):
for blockgroup in cluster.blockgroups:
index = index + 1
timeRange = _print_time_range(timecode_scale,cluster.timecode,blockgroup.block.timecode,blockgroup.duration)
srt_stream.write(str(index) + '\n')
srt_stream.write(timeRange + '\n')
srt_stream.write(codecs.decode(blockgroup.block.data.read(),'utf-8') + '\n')
srt_stream.write('\n')
return srt_stream
def _parse_segment(stream,track):
stream.seek(0)
specs = ebml.get_matroska_specs()
# Find all level 1 Cluster elements and its subelements. Speed up this process by excluding all other currently known level 1 elements
try:
segments = ebml.parse(stream, specs,include_element_names=['Segment','Cluster','BlockGroup','Timecode','Block','BlockDuration',],max_level=3)
except ReadError:
pass
clusters = []
for cluster in segments[0].data:
_parse_cluster(track, clusters, cluster)
return clusters
def _parse_cluster(track, clusters, cluster):
blockgroups = []
timecode = None
for child in cluster.data:
if child.name == 'BlockGroup':
_parse_blockgroup(track, blockgroups, child)
elif child.name == 'Timecode':
timecode = child.data
if len(blockgroups) > 0 and timecode != None:
clusters.append(Cluster(timecode, blockgroups))
def _parse_blockgroup(track, blockgroups, blockgroup):
block = None
duration = None
for child in blockgroup.data:
if child.name == 'Block':
block = Block.fromelement(child)
if block.track != track:
block = None
elif child.name == 'BlockDuration':
duration = child.data
if duration != None and block != None:
blockgroups.append(BlockGroup(block, duration))
def _print_time_range(timecode_scale,clusterTimecode,blockTimecode,duration):
timecode_scale_ms = timecode_scale / 1000000 #Timecode
rawTimecode = clusterTimecode + blockTimecode
startTimeMilleSeconds = (rawTimecode) * timecode_scale_ms
endTimeMilleSeconds = (rawTimecode + duration) * timecode_scale_ms
return _print_time(startTimeMilleSeconds) + " --> " + _print_time(endTimeMilleSeconds)
def _print_time(timeInMilleSeconds):
timeInSeconds, milleSeconds = divmod(timeInMilleSeconds, 1000)
timeInMinutes, seconds = divmod(timeInSeconds, 60)
hours, minutes = divmod(timeInMinutes, 60)
return '%d:%02d:%02d,%d' % (hours,minutes,seconds,milleSeconds)
class Cluster(object):
def __init__(self,timecode=None, blockgroups=[]):
self.timecode = timecode
self.blockgroups = blockgroups
class BlockGroup(object):
def __init__(self,block=None,duration=None):
self.block = block
self.duration = duration
class Block(object):
def __init__(self, track=None, timecode=None, invisible=False, lacing=None, flags=None, data=None):
self.track = track
self.timecode = timecode
self.invisible = invisible
self.lacing = lacing
self.flags = flags
self.data = data
@classmethod
def fromelement(cls,element):
stream = element.data
track = ebml.read_element_size(stream)
timecode = ebml.read_element_integer(stream,2)
flags = ord(stream.read(1))
invisible = bool(flags & 0x8)
if (flags & 0x6):
lacing = 'EBML'
elif (flags & 0x4):
lacing = 'fixed-size'
elif (flags & 0x2):
lacing = 'Xiph'
else:
lacing = None
if lacing:
raise ReadError('Laced blocks are not implemented yet')
data = ebml.read_element_binary(stream, element.size - stream.tell())
return cls(track,timecode,invisible,lacing,flags,data)
def __repr__(self):
return '<%s track=%d, timecode=%d, invisible=%d, lacing=%s>' % (self.__class__.__name__, self.track,self.timecode,self.invisible,self.lacing)
class SimpleBlock(Block):
def __init__(self, track=None, timecode=None, keyframe=False, invisible=False, lacing=None, flags=None, data=None, discardable=False):
super(SimpleBlock,self).__init__(track,timecode,invisible,lacing,flags,data)
self.keyframe = keyframe
self.discardable = discardable
def fromelement(cls,element):
simpleblock = super(SimpleBlock, cls).fromelement(element)
simpleblock.keyframe = bool(simpleblock.flags & 0x80)
simpleblock.discardable = bool(simpleblock.flags & 0x1)
return simpleblock
def __repr__(self):
return '<%s track=%d, timecode=%d, keyframe=%d, invisible=%d, lacing=%s, discardable=%d>' % (self.__class__.__name__, self.track,self.timecode,self.keyframe,self.invisible,self.lacing,self.discardable)
@@ -1,9 +1,11 @@
# -*- coding: utf-8 -*-
from . import test_mkv, test_parsers
from . import test_mkv, test_parsers, test_subtitle
import unittest
suite = unittest.TestSuite([test_mkv.suite(), test_parsers.suite()])
suite = unittest.TestSuite([test_mkv.suite(), test_parsers.suite(), test_subtitle.suite()])
if __name__ == '__main__':
@@ -193,7 +193,7 @@ class MKVTestCase(unittest.TestCase):
self.assertTrue(mkv.audio_tracks[0].type == AUDIO_TRACK)
self.assertTrue(mkv.audio_tracks[0].number == 2)
self.assertTrue(mkv.audio_tracks[0].name is None)
self.assertTrue(mkv.audio_tracks[0].language is None)
self.assertTrue(mkv.audio_tracks[0].language == 'eng')
self.assertTrue(mkv.audio_tracks[0].enabled == True)
self.assertTrue(mkv.audio_tracks[0].default == True)
self.assertTrue(mkv.audio_tracks[0].forced == False)
@@ -276,7 +276,7 @@ class MKVTestCase(unittest.TestCase):
self.assertTrue(mkv.audio_tracks[1].type == AUDIO_TRACK)
self.assertTrue(mkv.audio_tracks[1].number == 10)
self.assertTrue(mkv.audio_tracks[1].name == 'Commentary')
self.assertTrue(mkv.audio_tracks[1].language is None)
self.assertTrue(mkv.audio_tracks[1].language == 'eng')
self.assertTrue(mkv.audio_tracks[1].enabled == True)
self.assertTrue(mkv.audio_tracks[1].default == False)
self.assertTrue(mkv.audio_tracks[1].forced == False)
@@ -292,7 +292,7 @@ class MKVTestCase(unittest.TestCase):
self.assertTrue(mkv.subtitle_tracks[0].type == SUBTITLE_TRACK)
self.assertTrue(mkv.subtitle_tracks[0].number == 3)
self.assertTrue(mkv.subtitle_tracks[0].name is None)
self.assertTrue(mkv.subtitle_tracks[0].language is None)
self.assertTrue(mkv.subtitle_tracks[0].language == 'eng')
self.assertTrue(mkv.subtitle_tracks[0].enabled == True)
self.assertTrue(mkv.subtitle_tracks[0].default == True)
self.assertTrue(mkv.subtitle_tracks[0].forced == False)
@@ -33,7 +33,7 @@ class EBMLTestCase(unittest.TestCase):
self.stream.close()
def check_element(self, element_id, element_type, element_name, element_level, element_position, element_size, element_data, element,
ignore_element_types=None, ignore_element_names=None, max_level=None):
ignore_element_types=None, ignore_element_names=None, max_level=None, include_element_names=None):
"""Recursively check an element"""
# base
self.assertTrue(element.id == element_id)
@@ -53,6 +53,8 @@ class EBMLTestCase(unittest.TestCase):
element_data = [e for e in element_data if e[1] not in ignore_element_types]
if ignore_element_names is not None: # filter validation on element names
element_data = [e for e in element_data if e[2] not in ignore_element_names]
if include_element_names is not None: # filter validation on element names
element_data = [e for e in element_data if e[2] in include_element_names]
if element.level == max_level: # special check when maximum level is reached
self.assertTrue(element.data is None)
return
@@ -60,7 +62,7 @@ class EBMLTestCase(unittest.TestCase):
for i in range(len(element.data)):
self.check_element(element_data[i][0], element_data[i][1], element_data[i][2], element_data[i][3],
element_data[i][4], element_data[i][5], element_data[i][6], element.data[i], ignore_element_types,
ignore_element_names, max_level)
ignore_element_names, max_level,include_element_names)
def test_parse_full(self):
result = ebml.parse(self.stream, self.specs)
@@ -87,6 +89,15 @@ class EBMLTestCase(unittest.TestCase):
self.check_element(self.validation[i][0], self.validation[i][1], self.validation[i][2], self.validation[i][3],
self.validation[i][4], self.validation[i][5], self.validation[i][6], result[i], ignore_element_names=ignore_element_names)
def test_parse_include_element_names(self):
include_element_names = ['Segment','Cluster']
result = ebml.parse(self.stream, self.specs, include_element_names=include_element_names)
self.validation = [e for e in self.validation if e[2] in include_element_names]
self.assertTrue(len(result) == len(self.validation))
for i in range(len(self.validation)):
self.check_element(self.validation[i][0], self.validation[i][1], self.validation[i][2], self.validation[i][3],
self.validation[i][4], self.validation[i][5], self.validation[i][6], result[i], include_element_names=include_element_names)
def test_parse_max_level(self):
max_level = 3
result = ebml.parse(self.stream, self.specs, max_level=max_level)
@@ -0,0 +1,86 @@
# -*- coding: utf-8 -*-
from enzyme.subtitle import Subtitle, _print_time_range, _print_time
import unittest
import os
import io
import requests
import zipfile
import glob
# Test directory
TEST_DIR = os.path.join(os.path.dirname(__file__), os.path.splitext(__file__)[0])
def setUpModule():
if not os.path.exists(TEST_DIR):
r = requests.get('http://downloads.sourceforge.net/project/matroska/test_files/matroska_test_w1_1.zip')
with zipfile.ZipFile(io.BytesIO(r.content), 'r') as f:
f.extractall(TEST_DIR)
class SubtitleTestCase(unittest.TestCase):
@classmethod
def setUpClass(cls):
file = 'test5.mkv'
stream = io.open(os.path.join(TEST_DIR, file), 'rb')
cls.subtitle = Subtitle(stream)
def test_subtitles_found(self):
subtitles = self.subtitle._subtitles
self.assertTrue('eng' in subtitles)
self.assertTrue('hun' in subtitles)
self.assertTrue('ger' in subtitles)
self.assertTrue('fre' in subtitles)
self.assertTrue('spa' in subtitles)
self.assertTrue('ita' in subtitles)
self.assertTrue('jpn' in subtitles)
self.assertTrue('und' in subtitles)
def test_write_subtitle_to_stream(self):
subtitle_stream = self.subtitle.write_subtitle_to_stream("eng")
self.assertIsInstance(subtitle_stream,io.StringIO,"Expecting a StringIO stream")
def test_write_subtitle_to_stream(self):
subtitle_streams = self.subtitle.write_subtitles_to_stream()
self.assertIn("eng", subtitle_streams, "Expecting a subtitle stream for language eng")
self.assertIsInstance(subtitle_streams["eng"],io.StringIO,"Expecting a StringIO stream")
self.assertIn("hun", subtitle_streams, "Expecting a subtitle stream for language hun")
self.assertIsInstance(subtitle_streams["hun"],io.StringIO,"Expecting a StringIO stream")
self.assertIn("ger", subtitle_streams, "Expecting a subtitle stream for language ger")
self.assertIsInstance(subtitle_streams["ger"],io.StringIO,"Expecting a StringIO stream")
self.assertIn("fre", subtitle_streams, "Expecting a subtitle stream for language fre")
self.assertIsInstance(subtitle_streams["fre"],io.StringIO,"Expecting a StringIO stream")
self.assertIn("spa", subtitle_streams, "Expecting a subtitle stream for language spa")
self.assertIsInstance(subtitle_streams["spa"],io.StringIO,"Expecting a StringIO stream")
self.assertIn("ita", subtitle_streams, "Expecting a subtitle stream for language ita")
self.assertIsInstance(subtitle_streams["ita"],io.StringIO,"Expecting a StringIO stream")
self.assertIn("jpn", subtitle_streams, "Expecting a subtitle stream for language jpn")
self.assertIsInstance(subtitle_streams["jpn"],io.StringIO,"Expecting a StringIO stream")
def test_print_time(self):
self.assertEqual('0:00:00,0',_print_time(0))
self.assertEqual('0:00:00,1',_print_time(1))
self.assertEqual('0:00:00,999',_print_time(999))
self.assertEqual('0:00:01,0',_print_time(1000))
self.assertEqual('0:00:59,999',_print_time(1000*60-1))
self.assertEqual('0:01:00,0',_print_time(1000*60))
self.assertEqual('0:59:59,999',_print_time(1000*60*60-1))
self.assertEqual('1:00:00,0',_print_time(1000*60*60))
def test_print_time_range(self):
self.assertEqual('0:00:00,0 --> 0:00:00,0',_print_time_range(1000000,0,0,0))
self.assertEqual('0:01:00,0 --> 0:01:01,0',_print_time_range(1000000,0,60000,1000))
def suite():
suite = unittest.TestSuite()
suite.addTest(unittest.TestLoader().loadTestsFromTestCase(SubtitleTestCase))
return suite
if __name__ == '__main__':
unittest.TextTestRunner().run(suite())
+165
View File
@@ -0,0 +1,165 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.
"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".
The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.
c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.
@@ -0,0 +1,227 @@
GuessIt
=======
.. image:: http://img.shields.io/pypi/v/guessit.svg
:target: https://pypi.python.org/pypi/guessit
:alt: Latest Version
.. image:: http://img.shields.io/badge/license-LGPLv3-blue.svg
:target: https://pypi.python.org/pypi/guessit
:alt: License
.. image:: http://img.shields.io/travis/wackou/guessit.svg?branch=master
:target: http://travis-ci.org/wackou/guessit
:alt: Build Status
.. image:: http://img.shields.io/coveralls/wackou/guessit.svg?branch=master
:target: https://coveralls.io/r/wackou/guessit
:alt: Coveralls
`HuBoard <https://huboard.com/wackou/guessit>`_
GuessIt is a python library that extracts as much information as
possible from a video file.
It has a very powerful filename matcher that allows to guess a lot of
metadata from a video using its filename only. This matcher works with
both movies and tv shows episodes.
For example, GuessIt can do the following::
$ guessit "Treme.1x03.Right.Place,.Wrong.Time.HDTV.XviD-NoTV.avi"
For: Treme.1x03.Right.Place,.Wrong.Time.HDTV.XviD-NoTV.avi
GuessIt found: {
[1.00] "mimetype": "video/x-msvideo",
[0.80] "episodeNumber": 3,
[0.80] "videoCodec": "XviD",
[1.00] "container": "avi",
[1.00] "format": "HDTV",
[0.70] "series": "Treme",
[0.50] "title": "Right Place, Wrong Time",
[0.80] "releaseGroup": "NoTV",
[0.80] "season": 1,
[1.00] "type": "episode"
}
Install
-------
Installing GuessIt is simple with `pip <http://www.pip-installer.org/>`_::
$ pip install guessit
or, with `easy_install <http://pypi.python.org/pypi/setuptools>`_::
$ easy_install guessit
But, you really `shouldn't do that <http://stackoverflow.com/questions/3220404/why-use-pip-over-easy-install>`_.
You can now launch a demo::
$ guessit -d
and guess your own filename::
$ guessit "Breaking.Bad.S05E08.720p.MP4.BDRip.[KoTuWa].mkv"
For: Breaking.Bad.S05E08.720p.MP4.BDRip.[KoTuWa].mkv
GuessIt found: {
[1.00] "mimetype": "video/x-matroska",
[1.00] "episodeNumber": 8,
[0.30] "container": "mkv",
[1.00] "format": "BluRay",
[0.70] "series": "Breaking Bad",
[1.00] "releaseGroup": "KoTuWa",
[1.00] "screenSize": "720p",
[1.00] "season": 5,
[1.00] "type": "episode"
}
Filename matcher
----------------
The filename matcher is based on pattern matching and is able to recognize many properties from the filename,
like ``title``, ``year``, ``series``, ``episodeNumber``, ``seasonNumber``,
``videoCodec``, ``screenSize``, ``language``. Guessed values are cleaned up and given in a readable format
which may not match exactly the raw filename.
The full list of available properties can be seen in the
`main documentation <http://guessit.readthedocs.org/en/latest/user/properties.html>`_.
Other features
--------------
GuessIt also allows you to compute a whole lot of hashes from a file,
namely all the ones you can find in the hashlib python module (md5,
sha1, ...), but also the Media Player Classic hash that is used (amongst
others) by OpenSubtitles and SMPlayer, as well as the ed2k hash.
If you have the 'guess-language' python package installed, GuessIt can also
analyze a subtitle file's contents and detect which language it is written in.
If you have the 'enzyme' python package installed, GuessIt can also detect the
properties from the actual video file metadata.
Usage
-----
guessit can be use from command line::
$ guessit
usage: guessit [-h] [-t TYPE] [-n] [-c] [-X DISABLED_TRANSFORMERS] [-v]
[-P SHOW_PROPERTY] [-u] [-a] [-y] [-f INPUT_FILE] [-d] [-p]
[-V] [-s] [--version] [-b] [-i INFO] [-S EXPECTED_SERIES]
[-T EXPECTED_TITLE] [-Y] [-D] [-L ALLOWED_LANGUAGES] [-E]
[-C ALLOWED_COUNTRIES] [-G EXPECTED_GROUP]
[filename [filename ...]]
positional arguments:
filename Filename or release name to guess
optional arguments:
-h, --help show this help message and exit
Naming:
-t TYPE, --type TYPE The suggested file type: movie, episode. If undefined,
type will be guessed.
-n, --name-only Parse files as name only. Disable folder parsing,
extension parsing, and file content analysis.
-c, --split-camel Split camel case part of filename.
-X DISABLED_TRANSFORMERS, --disabled-transformer DISABLED_TRANSFORMERS
Transformer to disable (can be used multiple time)
-S EXPECTED_SERIES, --expected-series EXPECTED_SERIES
Expected series to parse (can be used multiple times)
-T EXPECTED_TITLE, --expected-title EXPECTED_TITLE
Expected title (can be used multiple times)
-Y, --date-year-first
If short date is found, consider the first digits as
the year.
-D, --date-day-first If short date is found, consider the second digits as
the day.
-L ALLOWED_LANGUAGES, --allowed-languages ALLOWED_LANGUAGES
Allowed language (can be used multiple times)
-E, --episode-prefer-number
Guess "serie.213.avi" as the episodeNumber 213.
Without this option, it will be guessed as season 2,
episodeNumber 13
-C ALLOWED_COUNTRIES, --allowed-country ALLOWED_COUNTRIES
Allowed country (can be used multiple times)
-G EXPECTED_GROUP, --expected-group EXPECTED_GROUP
Expected release group (can be used multiple times)
Output:
-v, --verbose Display debug output
-P SHOW_PROPERTY, --show-property SHOW_PROPERTY
Display the value of a single property (title, series,
videoCodec, year, type ...)
-u, --unidentified Display the unidentified parts.
-a, --advanced Display advanced information for filename guesses, as
json output
-y, --yaml Display information for filename guesses as yaml
output (like unit-test)
-f INPUT_FILE, --input-file INPUT_FILE
Read filenames from an input file.
-d, --demo Run a few builtin tests instead of analyzing a file
Information:
-p, --properties Display properties that can be guessed.
-V, --values Display property values that can be guessed.
-s, --transformers Display transformers that can be used.
--version Display the guessit version.
guessit.io:
-b, --bug Submit a wrong detection to the guessit.io service
Other features:
-i INFO, --info INFO The desired information type: filename, video,
hash_mpc or a hash from python's hashlib module, such
as hash_md5, hash_sha1, ...; or a list of any of them,
comma-separated
It can also be used as a python module::
>>> from guessit import guess_file_info
>>> guess_file_info('Treme.1x03.Right.Place,.Wrong.Time.HDTV.XviD-NoTV.avi')
{u'mimetype': 'video/x-msvideo', u'episodeNumber': 3, u'videoCodec': u'XviD', u'container': u'avi', u'format': u'HDTV', u'series': u'Treme', u'title': u'Right Place, Wrong Time', u'releaseGroup': u'NoTV', u'season': 1, u'type': u'episode'}
Support
-------
The project website for GuessIt is hosted at `ReadTheDocs <http://guessit.readthedocs.org/>`_.
There you will also find the User guide and Developer documentation.
This project is hosted on GitHub: `<https://github.com/wackou/guessit>`_
Please report issues and/or feature requests via the `bug tracker <https://github.com/wackou/guessit/issues>`_.
You can also report issues using the command-line tool::
$ guessit --bug "filename.that.fails.avi"
Contribute
----------
GuessIt is under active development, and contributions are more than welcome!
#. Check for open issues or open a fresh issue to start a discussion around a feature idea or a bug.
There is a Contributor Friendly tag for issues that should be ideal for people who are not very
familiar with the codebase yet.
#. Fork `the repository`_ on Github to start making your changes to the **master**
branch (or branch off of it).
#. Write a test which shows that the bug was fixed or that the feature works as expected.
#. Send a pull request and bug the maintainer until it gets merged and published. :)
.. _the repository: https://github.com/wackou/guessit
License
-------
GuessIt is licensed under the `LGPLv3 license <http://www.gnu.org/licenses/lgpl.html>`_.
@@ -17,4 +17,4 @@
# You should have received a copy of the Lesser GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
__version__ = '0.11.0'
__version__ = '0.11.1.dev0'
@@ -1,20 +1,43 @@
# coding=utf-8
from .patch_provider_pool import PatchedProviderPool
from .patch_providers import PatchedAddic7edProvider
from .patch_video import patched_search_external_subtitles
import subliminal
import babelfish
from .patch_provider_pool import PatchedProviderPool
from .patch_video import patched_search_external_subtitles, scan_video
from .patch_providers import addic7ed, podnapisi, tvsubtitles, opensubtitles
# patch subliminal's ProviderPool
subliminal.api.ProviderPool = PatchedProviderPool
# patch subliminal's Addic7edProvider
subliminal.providers.addic7ed.Addic7edProvider = PatchedAddic7edProvider
# patch subliminal's subtitle classes
def subtitleRepr(self):
link = self.page_link
# specialcasing addic7ed; eww
if self.__class__.__name__ == "Addic7edSubtitle":
link = u"http://www.addic7ed.com/%s" % self.download_link
return '<%s %r [%s]>' % (self.__class__.__name__, link, self.language)
subliminal.subtitle.Subtitle.__repr__ = subtitleRepr
# patch subliminal's providers
subliminal.providers.addic7ed.Addic7edProvider = addic7ed.PatchedAddic7edProvider
subliminal.providers.podnapisi.PodnapisiProvider = podnapisi.PatchedPodnapisiProvider
subliminal.providers.tvsubtitles.TVsubtitlesProvider = tvsubtitles.PatchedTVsubtitlesProvider
subliminal.providers.opensubtitles.OpenSubtitlesProvider = opensubtitles.PatchedOpenSubtitlesProvider
# add language converters
babelfish.language_converters.register('addic7ed = subliminal_patch.patch_language:PatchedAddic7edConverter')
babelfish.language_converters.register('tvsubtitles = subliminal.converters.tvsubtitles:TVsubtitlesConverter')
# patch subliminal's external subtitles search algorithm
subliminal.video.search_external_subtitles = patched_search_external_subtitles
subliminal.video.search_external_subtitles = patched_search_external_subtitles
# patch subliminal's scan_video function
subliminal.video.scan_video = scan_video
subliminal.video.Episode.scores["boost"] = 40
@@ -5,14 +5,20 @@ import traceback
import requests
import socket
import operator
import time
from babelfish.exceptions import LanguageReverseError
from pkg_resources import EntryPoint, iter_entry_points
from subliminal.api import ProviderPool, compute_score
from subliminal.api import ProviderPool
from subliminal_patch.patch_subtitle import compute_score
logger = logging.getLogger(__name__)
DOWNLOAD_TRIES = 0
DOWNLOAD_RETRY_SLEEP = 2
class OldToNewProvider(object):
"""
Simple proxy class to support the .plugin property which would normally exist
@@ -184,6 +190,49 @@ class PatchedProviderPool(ProviderPool):
return subtitles
def download_subtitle(self, subtitle):
"""Download `subtitle`'s :attr:`~subliminal.subtitle.Subtitle.content`.
:param subtitle: subtitle to download.
:type subtitle: :class:`~subliminal.subtitle.Subtitle`
:return: `True` if the subtitle has been successfully downloaded, `False` otherwise.
:rtype: bool
"""
# check discarded providers
if subtitle.provider_name in self.discarded_providers:
logger.warning('Provider %r is discarded', subtitle.provider_name)
return False
logger.info('Downloading subtitle %r', subtitle)
tries = 0
# retry downloading on failure until settings' download retry limit hit
while True:
tries += 1
try:
self[subtitle.provider_name].download_subtitle(subtitle)
except (requests.Timeout, socket.timeout):
logger.error('Provider %r timed out', subtitle.provider_name)
except:
logger.exception('Unexpected error in provider %r, Traceback: %s', subtitle.provider_name, traceback.format_exc())
else:
break
if tries == DOWNLOAD_TRIES:
self.discarded_providers.add(subtitle.provider_name)
logger.error('Maximum retries reached for provider %r, discarding it', subtitle.provider_name)
return False
# don't hammer the provider
logger.debug('Errors while downloading subtitle, retrying provider %r in %s seconds', subtitle.provider_name, DOWNLOAD_RETRY_SLEEP)
time.sleep(DOWNLOAD_RETRY_SLEEP)
# check subtitle validity
if not subtitle.is_valid():
logger.error('Invalid subtitle')
return False
return True
def download_best_subtitles(self, subtitles, video, languages, min_score=0, hearing_impaired=False, only_one=False,
scores=None):
"""Download the best matching subtitles.
@@ -201,14 +250,20 @@ class PatchedProviderPool(ProviderPool):
:return: downloaded subtitles.
:rtype: list of :class:`~subliminal.subtitle.Subtitle`
"""
use_hearing_impaired = hearing_impaired in ("prefer", "force HI")
# sort subtitles by score
scored_subtitles = sorted([(s, compute_score(s.get_matches(video, hearing_impaired=hearing_impaired), video,
scores=scores))
for s in subtitles], key=operator.itemgetter(1), reverse=True)
unsorted_subtitles = []
for s in subtitles:
logger.debug("Starting score computation for %s", s)
matches = s.get_matches(video, hearing_impaired=use_hearing_impaired)
unsorted_subtitles.append((s, compute_score(matches, video, scores=scores), matches))
scored_subtitles = sorted(unsorted_subtitles, key=operator.itemgetter(1), reverse=True)
# download best subtitles, falling back on the next on error
downloaded_subtitles = []
for subtitle, score in scored_subtitles:
for subtitle, score, matches in scored_subtitles:
# check score
if score < min_score:
logger.info('Score %d is below min_score (%d)', score, min_score)
@@ -219,6 +274,11 @@ class PatchedProviderPool(ProviderPool):
logger.debug('Skipping subtitle: %r already downloaded', subtitle.language)
continue
# bail out if hearing_impaired was wrong
if not "hearing_impaired" in matches and hearing_impaired in ("force HI", "force non-HI"):
logger.debug('Skipping subtitle: %r with score %d because hearing-impaired set to %s', subtitle, score, hearing_impaired)
continue
# download
logger.info('Downloading subtitle %r with score %d', subtitle, score)
if self.download_subtitle(subtitle):
@@ -1,71 +0,0 @@
# coding=utf-8
import logging
from random import randint
from subliminal.providers.addic7ed import Addic7edProvider, Addic7edSubtitle, ParserBeautifulSoup, series_year_re, Language
logger = logging.getLogger(__name__)
class PatchedAddic7edProvider(Addic7edProvider):
USE_ADDICTED_RANDOM_AGENTS = False
def __init__(self, username=None, password=None, use_random_agents=False):
super(PatchedAddic7edProvider, self).__init__(username=username, password=password)
self.USE_ADDICTED_RANDOM_AGENTS = use_random_agents
def initialize(self):
super(PatchedAddic7edProvider, self).initialize()
if self.USE_ADDICTED_RANDOM_AGENTS:
from .utils import FIRST_THOUSAND_OR_SO_USER_AGENTS as AGENT_LIST
logger.debug("addic7ed: using random user agents")
self.session.headers = {
'User-Agent': AGENT_LIST[randint(0, len(AGENT_LIST)-1)],
'Referer': self.server_url,
}
def query(self, series, season, year=None, country=None):
# get the show id
show_id = self.get_show_id(series, year, country)
if show_id is None:
logger.error('No show id found for %r (%r)', series, {'year': year, 'country': country})
return []
# get the page of the season of the show
logger.info('Getting the page of show id %d, season %d', show_id, season)
r = self.session.get(self.server_url + 'show/%d' % show_id, params={'season': season}, timeout=10)
r.raise_for_status()
soup = ParserBeautifulSoup(r.content, ['lxml', 'html.parser'])
# loop over subtitle rows
header = soup.select('#header font')
if header:
match = series_year_re.match(header[0].text.strip()[:-10])
series = match.group('series')
year = int(match.group('year')) if match.group('year') else None
subtitles = []
for row in soup.select('tr.epeven'):
cells = row('td')
# ignore incomplete subtitles
status = cells[5].text
if status != 'Completed':
logger.debug('Ignoring subtitle with status %s', status)
continue
# read the item
language = Language.fromaddic7ed(cells[3].text)
hearing_impaired = bool(cells[6].text)
page_link = self.server_url + cells[2].a['href'][1:]
season = int(cells[0].text)
episode = int(cells[1].text)
title = cells[2].text
version = cells[4].text
download_link = cells[9].a['href'][1:]
subtitle = Addic7edSubtitle(language, hearing_impaired, page_link, series, season, episode, title, year,
version, download_link)
logger.debug('Found subtitle %r', subtitle)
subtitles.append(subtitle)
return subtitles
@@ -0,0 +1,191 @@
# coding=utf-8
import logging
import re
from random import randint
from subliminal.providers.addic7ed import Addic7edProvider, Addic7edSubtitle, ParserBeautifulSoup, Language
from subliminal.cache import SHOW_EXPIRATION_TIME, region
from .mixins import PunctuationMixin
logger = logging.getLogger(__name__)
series_year_re = re.compile('^(?P<series>[ \w.:\']+)(?: \((?P<year>\d{4})\))?$')
USE_BOOST = False
class PatchedAddic7edSubtitle(Addic7edSubtitle):
def __init__(self, *args, **kwargs):
super(PatchedAddic7edSubtitle, self).__init__(*args, **kwargs)
def get_matches(self, video, hearing_impaired=False):
matches = super(PatchedAddic7edSubtitle, self).get_matches(video, hearing_impaired=hearing_impaired)
if not USE_BOOST:
return matches
if {"series", "season", "episode", "year"}.issubset(matches) and "format" in matches:
matches.add("boost")
logger.info("Boosting Addic7ed subtitle")
return matches
class PatchedAddic7edProvider(PunctuationMixin, Addic7edProvider):
USE_ADDICTED_RANDOM_AGENTS = False
def __init__(self, username=None, password=None, use_random_agents=False):
super(PatchedAddic7edProvider, self).__init__(username=username, password=password)
self.USE_ADDICTED_RANDOM_AGENTS = use_random_agents
def initialize(self):
# patch: add optional user agent randomization
super(PatchedAddic7edProvider, self).initialize()
if self.USE_ADDICTED_RANDOM_AGENTS:
from .utils import FIRST_THOUSAND_OR_SO_USER_AGENTS as AGENT_LIST
logger.debug("addic7ed: using random user agents")
self.session.headers = {
'User-Agent': AGENT_LIST[randint(0, len(AGENT_LIST)-1)],
'Referer': self.server_url,
}
@region.cache_on_arguments(expiration_time=SHOW_EXPIRATION_TIME)
def _get_show_ids(self):
"""Get the ``dict`` of show ids per series by querying the `shows.php` page.
:return: show id per series, lower case and without quotes.
:rtype: dict
# patch: add punctuation cleaning
"""
# get the show page
logger.info('Getting show ids')
r = self.session.get(self.server_url + 'shows.php', timeout=10)
r.raise_for_status()
soup = ParserBeautifulSoup(r.content, ['lxml', 'html.parser'])
# populate the show ids
show_ids = {}
for show in soup.select('td.version > h3 > a[href^="/show/"]'):
show_ids[self.clean_punctuation(show.text.lower())] = int(show['href'][6:])
logger.debug('Found %d show ids', len(show_ids))
return show_ids
def get_show_id(self, series, year=None, country_code=None):
"""Get the best matching show id for `series`, `year` and `country_code`.
First search in the result of :meth:`_get_show_ids` and fallback on a search with :meth:`_search_show_id`
:param str series: series of the episode.
:param year: year of the series, if any.
:type year: int or None
:param country_code: country code of the series, if any.
:type country_code: str or None
:return: the show id, if found.
:rtype: int or None
"""
series_clean = self.clean_punctuation(series.lower())
show_ids = self._get_show_ids()
show_id = None
# attempt with country
if not show_id and country_code:
logger.debug('Getting show id with country')
show_id = show_ids.get('%s (%s)' % (series_clean, country_code.lower()))
# attempt with year
if not show_id and year:
logger.debug('Getting show id with year')
show_id = show_ids.get('%s (%d)' % (series_clean, year))
# attempt clean
if not show_id:
logger.debug('Getting show id')
show_id = show_ids.get(series_clean)
# search as last resort
if not show_id:
logger.warning('Series not found in show ids, attempting search')
show_id = self._search_show_id(series_clean)
if not show_id:
show_id = self.search_show_id(series.lower().replace("'", ""))
return show_id
@region.cache_on_arguments(expiration_time=SHOW_EXPIRATION_TIME)
def _search_show_id(self, series, year=None):
"""Search the show id from the `series` and `year`.
:param string series: series of the episode.
:param year: year of the series, if any.
:type year: int or None
:return: the show id, if found.
:rtype: int or None
# patch: add punctuation cleaning
"""
# build the params
series_year = '%s (%d)' % (series, year) if year is not None else series
params = {'search': series_year, 'Submit': 'Search'}
# make the search
logger.info('Searching show ids with %r', params)
r = self.session.get(self.server_url + 'search.php', params=params, timeout=10)
r.raise_for_status()
soup = ParserBeautifulSoup(r.content, ['lxml', 'html.parser'])
# get the suggestion
suggestion = soup.select('span.titulo > a[href^="/show/"]')
if not suggestion:
logger.warning('Show id not found: no suggestion')
return None
if not self.clean_punctuation(suggestion[0].i.text.lower()) == self.clean_punctuation(series_year.lower()):
logger.warning('Show id not found: suggestion does not match')
return None
show_id = int(suggestion[0]['href'][6:])
logger.debug('Found show id %d', show_id)
return show_id
def query(self, series, season, year=None, country=None):
# patch: fix logging
# get the show id
show_id = self.get_show_id(series, year, country)
if show_id is None:
logger.error('No show id found for %r (%r)', series, {'year': year, 'country': country})
return []
# get the page of the season of the show
logger.info('Getting the page of show id %d, season %d', show_id, season)
r = self.session.get(self.server_url + 'show/%d' % show_id, params={'season': season}, timeout=10)
r.raise_for_status()
soup = ParserBeautifulSoup(r.content, ['lxml', 'html.parser'])
# loop over subtitle rows
header = soup.select('#header font')
if header:
match = series_year_re.match(header[0].text.strip()[:-10])
series = match.group('series')
year = int(match.group('year')) if match.group('year') else None
subtitles = []
for row in soup.select('tr.epeven'):
cells = row('td')
# ignore incomplete subtitles
status = cells[5].text
if status != 'Completed':
logger.debug('Ignoring subtitle with status %s', status)
continue
# read the item
language = Language.fromaddic7ed(cells[3].text)
hearing_impaired = bool(cells[6].text)
page_link = self.server_url + cells[2].a['href'][1:]
season = int(cells[0].text)
episode = int(cells[1].text)
title = cells[2].text
version = cells[4].text
download_link = cells[9].a['href'][1:]
subtitle = PatchedAddic7edSubtitle(language, hearing_impaired, page_link, series, season, episode, title, year,
version, download_link)
logger.debug('Found subtitle %r', subtitle)
subtitles.append(subtitle)
return subtitles
@@ -0,0 +1,12 @@
# coding=utf-8
class PunctuationMixin(object):
"""
provider mixin
fixes show ids for stuff like "Mr. Petterson", as our matcher already sees it as "Mr Petterson" but addic7ed doesn't
"""
def clean_punctuation(self, s):
return s.replace(".", "").replace(":", "").replace("'", "")
@@ -0,0 +1,23 @@
# coding=utf-8
import logging
from subliminal.providers.opensubtitles import OpenSubtitlesProvider, checked, get_version, __version__
logger = logging.getLogger(__name__)
class PatchedOpenSubtitlesProvider(OpenSubtitlesProvider):
def __init__(self, username=None, password=None):
if username is not None and password is None or username is None and password is not None:
raise ConfigurationError('Username and password must be specified')
self.username = username or ''
self.password = password or ''
super(PatchedOpenSubtitlesProvider, self).__init__()
def initialize(self):
logger.info('Logging in')
response = checked(self.server.LogIn(self.username, self.password, 'eng', 'subliminal v%s' % get_version(__version__)))
self.token = response['token']
logger.debug('Logged in with token %r', self.token)
@@ -0,0 +1,22 @@
# coding=utf-8
import logging
import io
from zipfile import ZipFile
from subliminal.providers.podnapisi import PodnapisiProvider, fix_line_ending, ProviderError
logger = logging.getLogger(__name__)
class PatchedPodnapisiProvider(PodnapisiProvider):
def download_subtitle(self, subtitle):
# download as a zip
logger.info('Downloading subtitle %r', subtitle)
r = self.session.get(self.server_url + subtitle.pid + '/download', params={'container': 'zip'}, timeout=10)
r.raise_for_status()
# open the zip
with ZipFile(io.BytesIO(r.content)) as zf:
if len(zf.namelist()) > 1:
raise ProviderError('More than one file to unzip')
subtitle.content = fix_line_ending(zf.read(zf.namelist()[0]))
@@ -0,0 +1,46 @@
# coding=utf-8
import logging
from subliminal.providers import ParserBeautifulSoup
from subliminal.cache import SHOW_EXPIRATION_TIME, region
from subliminal.providers.tvsubtitles import TVsubtitlesProvider, link_re
from .mixins import PunctuationMixin
logger = logging.getLogger(__name__)
class PatchedTVsubtitlesProvider(PunctuationMixin, TVsubtitlesProvider):
@region.cache_on_arguments(expiration_time=SHOW_EXPIRATION_TIME)
def search_show_id(self, series, year=None):
"""Search the show id from the `series` and `year`.
:param string series: series of the episode.
:param year: year of the series, if any.
:type year: int or None
:return: the show id, if any.
:rtype: int or None
"""
# make the search
series_clean = self.clean_punctuation(series).lower()
logger.info('Searching show id for %r', series_clean)
r = self.session.post(self.server_url + 'search.php', data={'q': series_clean}, timeout=10)
r.raise_for_status()
# get the series out of the suggestions
soup = ParserBeautifulSoup(r.content, ['lxml', 'html.parser'])
show_id = None
for suggestion in soup.select('div.left li div a[href^="/tvshow-"]'):
match = link_re.match(self.clean_punctuation(suggestion.text))
if not match:
logger.error('Failed to match %s', suggestion.text)
continue
if self.clean_punctuation(match.group('series')).lower() == series_clean:
if year is not None and int(match.group('first_year')) != year:
logger.debug('Year does not match')
continue
show_id = int(suggestion['href'][8:-5])
logger.debug('Found show id %d', show_id)
break
return show_id
@@ -0,0 +1,45 @@
# coding=utf-8
import logging
from subliminal.video import Episode, Movie
logger = logging.getLogger(__name__)
def compute_score(matches, video, scores=None):
"""Compute the score of the `matches` against the `video`.
Some matches count as much as a combination of others in order to level the final score:
* `hash` removes everything else
* For :class:`~subliminal.video.Episode`
* `imdb_id` removes `series`, `tvdb_id`, `season`, `episode`, `title` and `year`
* `tvdb_id` removes `series` and `year`
* `title` removes `season` and `episode`
:param video: the video to get the score with.
:type video: :class:`~subliminal.video.Video`
:param dict scores: scores to use, if `None`, the :attr:`~subliminal.video.Video.scores` from the video are used.
:return: score of the subtitle.
:rtype: int
# patch: remove score cap for enabling individual boost
"""
final_matches = matches.copy()
scores = scores or video.scores
logger.info('Computing score for matches %r and %r', matches, video)
# remove equivalent match combinations
if 'hash' in final_matches:
final_matches &= {'hash', 'hearing_impaired'}
elif isinstance(video, Episode):
if 'imdb_id' in final_matches:
final_matches -= {'series', 'tvdb_id', 'season', 'episode', 'title', 'year'}
if 'tvdb_id' in final_matches:
final_matches -= {'series', 'year'}
if 'title' in final_matches:
final_matches -= {'season', 'episode'}
# compute score
logger.debug('Final matches: %r', final_matches)
score = sum((scores[match] for match in final_matches))
logger.info('Computed score %d', score)
return score
@@ -2,7 +2,7 @@
import os
import logging
from subliminal.video import SUBTITLE_EXTENSIONS, Language
from subliminal.video import SUBTITLE_EXTENSIONS, VIDEO_EXTENSIONS, Language, Video, EnzymeError, MKV, guess_file_info, hash_opensubtitles, hash_thesubdb
logger = logging.getLogger(__name__)
@@ -59,3 +59,116 @@ def patched_search_external_subtitles(path):
logger.debug("external subs: found %s", subtitles)
return subtitles
def scan_video(path, subtitles=True, embedded_subtitles=True, video_type=None):
"""Scan a video and its subtitle languages from a video `path`.
:param str path: existing path to the video.
:param bool subtitles: scan for subtitles with the same name.
:param bool embedded_subtitles: scan for embedded subtitles.
:return: the scanned video.
:rtype: :class:`Video`
# patch: suggest video type to guessit beforehand
"""
# check for non-existing path
if not os.path.exists(path):
raise ValueError('Path does not exist')
# check video extension
if not path.endswith(VIDEO_EXTENSIONS):
raise ValueError('%s is not a valid video extension' % os.path.splitext(path)[1])
dirpath, filename = os.path.split(path)
logger.info('Scanning video (type: %s) %r in %r', video_type, filename, dirpath)
# guess
video = Video.fromguess(path, guess_file_info(path, options={"type": video_type}))
# size and hashes
video.size = os.path.getsize(path)
if video.size > 10485760:
logger.debug('Size is %d', video.size)
video.hashes['opensubtitles'] = hash_opensubtitles(path)
video.hashes['thesubdb'] = hash_thesubdb(path)
logger.debug('Computed hashes %r', video.hashes)
else:
logger.warning('Size is lower than 10MB: hashes not computed')
# external subtitles
if subtitles:
video.subtitle_languages |= set(patched_search_external_subtitles(path).values())
# video metadata with enzyme
try:
if filename.endswith('.mkv'):
with open(path, 'rb') as f:
mkv = MKV(f)
# main video track
if mkv.video_tracks:
video_track = mkv.video_tracks[0]
# resolution
if video_track.height in (480, 720, 1080):
if video_track.interlaced:
video.resolution = '%di' % video_track.height
else:
video.resolution = '%dp' % video_track.height
logger.debug('Found resolution %s with enzyme', video.resolution)
# video codec
if video_track.codec_id == 'V_MPEG4/ISO/AVC':
video.video_codec = 'h264'
logger.debug('Found video_codec %s with enzyme', video.video_codec)
elif video_track.codec_id == 'V_MPEG4/ISO/SP':
video.video_codec = 'DivX'
logger.debug('Found video_codec %s with enzyme', video.video_codec)
elif video_track.codec_id == 'V_MPEG4/ISO/ASP':
video.video_codec = 'XviD'
logger.debug('Found video_codec %s with enzyme', video.video_codec)
else:
logger.warning('MKV has no video track')
# main audio track
if mkv.audio_tracks:
audio_track = mkv.audio_tracks[0]
# audio codec
if audio_track.codec_id == 'A_AC3':
video.audio_codec = 'AC3'
logger.debug('Found audio_codec %s with enzyme', video.audio_codec)
elif audio_track.codec_id == 'A_DTS':
video.audio_codec = 'DTS'
logger.debug('Found audio_codec %s with enzyme', video.audio_codec)
elif audio_track.codec_id == 'A_AAC':
video.audio_codec = 'AAC'
logger.debug('Found audio_codec %s with enzyme', video.audio_codec)
else:
logger.warning('MKV has no audio track')
# subtitle tracks
if mkv.subtitle_tracks:
if embedded_subtitles:
embedded_subtitle_languages = set()
for st in mkv.subtitle_tracks:
if st.language:
try:
embedded_subtitle_languages.add(Language.fromalpha3b(st.language))
except BabelfishError:
logger.error('Embedded subtitle track language %r is not a valid language', st.language)
embedded_subtitle_languages.add(Language('und'))
elif st.name:
try:
embedded_subtitle_languages.add(Language.fromname(st.name))
except BabelfishError:
logger.debug('Embedded subtitle track name %r is not a valid language', st.name)
embedded_subtitle_languages.add(Language('und'))
else:
embedded_subtitle_languages.add(Language('und'))
logger.debug('Found embedded subtitle %r with enzyme', embedded_subtitle_languages)
video.subtitle_languages |= embedded_subtitle_languages
else:
logger.debug('MKV has no subtitle track')
except EnzymeError:
logger.exception('Parsing video metadata with enzyme failed')
return video
Binary file not shown.

Before

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 213 KiB

+57 -32
View File
@@ -1,29 +1,46 @@
## pannal's fork of Subliminal.bundle
#### RC-1
- fix subliminal's logging error on min_score not met (fixes #15)
- separated tv and movies subtitle scores settings (fixes #16)
- add option to save only one subtitle per video (skipping the ".lang." naming scheme plex supports) (fixes #3)
beta5
- fix storing subtitles besides the actual video file, not subfolder (fixes #14)
- "custom folder" setting now always used if given (properly overrides "subtitle folder" setting)
- also scan (custom) given subtitle folders for existing subtitles instead of redownloading them on every refresh (fixes #9, #2)
beta4
- ~~increased score of addic7ed subtitles a bit~~ (not existing currently)
- **support for newest Subliminal ([1.0.1](27a6e51cd36ffb2910cd9a7add6d797a2c6469b7)) and guessit ([0.11.0](2814f57e8999dcc31575619f076c0c1a63ce78f2))**
- **plugin now also [works with com.plexapp.agents.thetvdbdvdorder](924470d2c0db3a71529278bce4b7247eaf2f85b8)**
- providers fixed for subliminal 1.0.1 ([at least addic7ed](131504e7eed8b3400c457fbe49beea3b115bc916))
- providers [don't simply fail and get excluded on non-detected language](1a779020792e0201ad689eefbf5a126155e89c97)
- support for addic7ed languages: [French (Canadian)](b11a051c233fd72033f0c3b5a8c1965260e7e19f)
- support for additional languages: [pt-br (Portuguese (Brasil)), fa (Persian (Farsi))](131504e7eed8b3400c457fbe49beea3b115bc916)
- support for [three (two optional) subtitle languages](e543c927cf49c264eaece36640c99d67a99c7da2)
- optionally use [random user agent for addic7ed provider](83ace14faf75fbd75313f0ceda9b78161895fbcf) (should not be needed)
Subliminal.bundle
Sub-Zero for Plex, 1.1.0.3
=================
![logo](https://raw.githubusercontent.com/pannal/Sub-Zero/master/Contents/Resources/subzero.gif)
##### Subtitles done right
Originally based on @bramwalet's awesome [Subliminal.bundle](https://github.com/bramwalet/Subliminal.bundle)
Plex forum thread: https://forums.plex.tv/discussion/186575
### Installation
* go to ```Library/Application Support/Plex Media Server/Plug-ins/```
* ```rm -r Sub-Zero.bundle```
* get the release you want from *https://github.com/pannal/Sub-Zero/releases/*
* unzip the release
* more indepth: see [article](https://support.plex.tv/hc/en-us/articles/201187656-How-do-I-manually-install-a-channel-) on Plex website.
### Usage
Use the following agent order:
1. Sub-Zero TV/Movie Subtitles
2. Local Media Assets
3. anything else
### Encountered a bug?
* be sure to post your logs: ```Library/Application Support/Plex Media Server/Logs/PMS Plugin Logs/com.plexapp.agents.subzero.log```; there may be multiple logs (com.plexapp.agents.subzero.log.*) depending on the amount of Videos you're refreshing
* **Remember: before you open a bug-ticket please double-check, that you've deleted the Sub-Zero.bundle folder BEFORE every update** (to avoid .pyc leftovers)
## Changelog
1.1.0.3
- addic7ed/tvsubtitles: be way smarter about punctuation in series names (*A.G.E.N.T.S. ...*)
- ditch LocalMediaExtended and incorporate the functionality in Sub-Zero (**RC-users: delete LocalMediaExtended.bundle and re-enable LocalMedia!**)
- remove (unused) setting "Restrict to one language"
- add "Treat IETF language tags as ISO 639-1 (e.g. pt-BR = pt)" setting (default: true)
- change default external storage to "current folder" instead of "/subs"
- adjust default scores
[older changes](CHANGELOG.md)
Description
------------
Plex Metadata agent plugin based on Subliminal. This agent will search on the following sites for the best matching subtitles:
- OpenSubtitles
- TheSubDB
@@ -33,29 +50,36 @@ Plex Metadata agent plugin based on Subliminal. This agent will search on the fo
All providers can be disabled or enabled on a per provider setting. Certain preferences change the behaviour of subliminal, for instance the minimum score of subtitles to download, or whether to download hearing impaired subtitles or not. The agent stores the subtitles as metadata, but can be configured (See Configuration) to store it next to the media files.
Installation
------------
See [article](https://support.plex.tv/hc/en-us/articles/201187656-How-do-I-manually-install-a-channel-) on Plex website.
Configuration
-------------
Several options are provided in the preferences of this agent.
* Addic7ed username/password: Provide your addic7ed username here, otherwise the provider won't work. Please make sure your account is activated, before using the agent.
* Subtitle language (1)/(2): Your preferred languages to download subtitles for.
* Subtitle language (1)/(2)/(3): Your preferred languages to download subtitles for.
* Additional Subtitle Languages: Additional languages to download; comma-separated; use [ISO-639-1 codes](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes))
* Provider: Enable ...: Enable/disable this provider. Affects both movies and series.
* Addic7ed: boost over hash score if requirements met: if an Addic7ed subtitle matches the video's series, season, episode, year, and format (e.g. WEB-DL), boost its score, possibly over OpenSubtitles/TheSubDB direct hash match
* Scan: Include embedded subtitles: When enabled, subliminal finds embedded subtitles that are already present within the media file.
* Scan: Include external subtitles: When enabled, subliminal finds subtitles located near the media file on the filesystem.
* Minimum score for download: When configured, what is the minimum score for subtitles to download them? Lower scored subtitles are not downloaded.
* Download hearing impaired subtitles: When configured, hearing impaired subtitles will be downloaded.
* Download hearing impaired subtitles:
* "prefer": score subtitles for hearing impaired higher
* "don't prefer": score subtitles for hearing impaired lower
* "force HI": skip subtitles if the hearing impaired flag isn't set
* "force non-HI": skip subtitles if the hearing impaired flag is set
* Store subtitles next to media files (instead of metadata): See Store as metadata or on filesystem
* Subtitle folder: See Store as metadata or on filesystem
* Subtitle folder: (default: current media file's folder) See Store as metadata or on filesystem
* Custom Subtitle folder: See Store as metadata or on filesystem
* Treat IETF language tags as ISO 639-1: Treats subtitle files with IETF language identifiers, such as pt-BR, as their ISO 639-1 counterpart. Thus "pt-BR" will be shown as "Portuguese" instead of "Unknown"
Store as metadata or on filesystem
----------------------------------
By default, Plex stores posters, fan art and subtitles as metadata in a separate folder which is not managed by the user. This is the default behaviour of this agent. However, expert users can enable 'Store subtitles next to media files'. The agent will write the subtitle files in the media folder. The setting 'Subtitle folder' configures in which folder (current folder or other subfolder) the subtitles are stored. The expert user can also supply 'Custom Subtitle folder' which can also be an absolute path.
By default, Plex stores posters, fan art and subtitles as metadata in a separate folder which is not managed by the user.
In Sub-Zero, though, 'Store subtitles next to media files' is enabled by default.
The agent will write the subtitle files in the media folder next to the media file itself.
The setting 'Subtitle folder' configures in which folder (current folder or other subfolder) the subtitles are stored. The expert user can also supply 'Custom Subtitle folder' which can also be an absolute path.
Please note that you need a way to pick up external subtitles to show up in the Plex Media server. When the subtitles are stored next to your media folders, it is sufficient to enable Local Media agent and place it below the Subliminal agent in the agent priorities. When a subfolder (either custom or predefined) is used, you need [LocalMediaExtended](https://github.com/pannal/LocalMediaExtended.bundle).
**When a subfolder (either custom or predefined) is used, the automatic scheduled refresh of Plex won't pick up your subtitles, only a manual refresh will!**
License
-------
@@ -78,3 +102,4 @@ Uses the following libraries and their LICENSE:
- [subliminal](https://pypi.python.org/pypi/subliminal/) (MIT)
- [xdg](https://pypi.python.org/pypi/pyxdg/) (LGPLv2)
- [setuptools](https://pypi.python.org/pypi/setuptools/) (PSF ZPL)
- [plexinc-agents/LocalMedia.bundle](https://github.com/plexinc-agents/LocalMedia.bundle) (Plex)