release 2.6.5.3152

New implementation of Titlovi using API

CHANGELOG.md

@@ -1,3 +1,72 @@
+2.6.5.3124
+
+subscene, addic7ed and titlovi
+- either of those providers might impose a reCAPTCHA verification. In order to use those providers, please create an account at an AntiCaptcha service ([anti-captcha.com](http://getcaptchasolution.com/kkvviom7nh) or [deathbycaptcha.com](http://deathbycaptcha.com)), add funds, then supply your credentials/apikey in the configuration
+
+Changelog
+- core: http: fallback to default DNS when normal resolving fails; fixes #657
+- core: extract embedded/menu: fix detection of unknown streams; don't use unknown streams if a known language was previously found
+- core: language: use replacement map from bazarr
+- providers: titlovi: fix matching
+- providers: subscene: fix unknown language code error when "empty" result is returned
+- providers: subscene: add support for pt-BR (based on Diaoul/subliminal@b22cf08)
+- providers: subscene: explicitly set account filters for languages
+- providers: subscene: limit alternative searches to 3; set throttle to 8
+- providers: subscene: move login/cookies to initialization sequence
+- submod: generic: en: fix ";='s
+
+
+2.6.5.3109
+
+subscene, addic7ed and titlovi
+- either of those providers might impose a reCAPTCHA verification. In order to use those providers, please create an account at an AntiCaptcha service ([anti-captcha.com](http://getcaptchasolution.com/kkvviom7nh) or [deathbycaptcha.com](http://deathbycaptcha.com)), add funds, then supply your credentials/apikey in the configuration
+
+Changelog
+- providers: add Napisy24 (polish)
+- providers: subscene: reduce provider load by possibly half
+- providers: subscene: support logging in (username/password are now required)
+- providers: subscene: fallback to non year results if none found with year
+
+
+2.6.5.3099
+
+subscene, addic7ed and titlovi
+- either of those providers might impose a reCAPTCHA verification. In order to use those providers, please create an account at an AntiCaptcha service ([anti-captcha.com](http://getcaptchasolution.com/kkvviom7nh) or [deathbycaptcha.com](http://deathbycaptcha.com)), add funds, then supply your credentials/apikey in the configuration
+
+Changelog
+- core: allow system DNS again by putting "system" as the DNS
+- providers: subscene: fix again (subscene, contact us please, so we can end this)
+
+
+2.6.5.3092
+
+subscene, addic7ed and titlovi
+- either of those providers might impose a reCAPTCHA verification. In order to use those providers, please create an account at an AntiCaptcha service ([anti-captcha.com](http://getcaptchasolution.com/kkvviom7nh) or [deathbycaptcha.com](http://deathbycaptcha.com)), add funds, then supply your credentials/apikey in the configuration
+
+Changelog
+- providers: subscene: fix endpoint (hopefully for longer now)
+- providers: subscene: don't search for season packs (broken for now; relieves 50% of server load on provider)
+- providers: subscene: don't calculate video fn for now
+- providers: argenteam: backport fixes from bazarr
+- subtitle: try decoding with utf-16 by default as well (zho/farsi)
+- submod: HI: remove music tags by default
+- core: compat (bazarr): add env var SZ_KEEP_ENCODING to keep encoding of subtitles
+
+
+2.6.5.3074
+
+Changelog
+- core: cf: bypass cf 95% of the time without captchas
+- core: fix breaking line endings of certain languages (chinese, UTF-16); fixes #646
+- core: update pysubs2 to 0.2.3
+
+
+2.6.5.3062
+
+Changelog
+- core: cf: optimize
+- core: http: don't query DNS with IPs. thanks @fgump (fixes sonarr/radarr)
+
 
 2.6.5.3041
 

Contents/Code/__init__.py

@@ -119,19 +119,23 @@ def agent_extract_embedded(video_part_map):
             for plexapi_part in get_all_parts(plexapi_item):
                 item_count = item_count + 1
                 used_one_unknown_stream = False
+                used_one_known_stream = False
                 for requested_language in config.lang_list:
+                    skip_unknown = used_one_unknown_stream or used_one_known_stream
                     embedded_subs = stored_subs.get_by_provider(plexapi_part.id, requested_language, "embedded")
                     current = stored_subs.get_any(plexapi_part.id, requested_language) or \
                         requested_language in scanned_video.external_subtitle_languages
 
                     if not embedded_subs:
                         stream_data = get_embedded_subtitle_streams(plexapi_part, requested_language=requested_language,
-                                                                    skip_unknown=used_one_unknown_stream)
+                                                                    skip_unknown=skip_unknown)
 
                         if stream_data:
                             stream = stream_data[0]["stream"]
                             if stream_data[0]["is_unknown"]:
                                 used_one_unknown_stream = True
+                            else:
+                                used_one_known_stream = True
 
                             to_extract.append(({scanned_video: part_info}, plexapi_part, str(stream.index),
                                                str(requested_language), not current))

Contents/Code/support/config.py

@@ -761,6 +761,7 @@ class Config(object):
         return {'opensubtitles': cast_bool(Prefs['provider.opensubtitles.enabled']),
                      # 'thesubdb': Prefs['provider.thesubdb.enabled'],
                      'podnapisi': cast_bool(Prefs['provider.podnapisi.enabled']),
+                     'napisy24': cast_bool(Prefs['provider.napisy24.enabled']),
                      'titlovi': cast_bool(Prefs['provider.titlovi.enabled']),
                      'addic7ed': cast_bool(Prefs['provider.addic7ed.enabled']) and self.has_anticaptcha,
                      'tvsubtitles': cast_bool(Prefs['provider.tvsubtitles.enabled']),
@@ -801,6 +802,7 @@ class Config(object):
             providers["argenteam"] = False
             providers["assrt"] = False
             providers["subscene"] = False
+            providers["napisy24"] = False
             providers_forced_off = dict(providers)
 
         if not self.unrar and providers["legendastv"]:
@@ -841,11 +843,8 @@ class Config(object):
     providers = property(get_providers)
 
     def get_provider_settings(self):
-        os_use_https = self.advanced.providers.opensubtitles.use_https \
-            if self.advanced.providers.opensubtitles.use_https is not None else True
-
-        os_skip_wrong_fps = self.advanced.providers.opensubtitles.skip_wrong_fps \
-            if self.advanced.providers.opensubtitles.skip_wrong_fps is not None else True
+        os_use_https = self.advanced.providers.opensubtitles.get("use_https", True)
+        os_skip_wrong_fps = self.advanced.providers.opensubtitles.get("skip_wrong_fps", True)
 
         provider_settings = {'addic7ed': {'username': Prefs['provider.addic7ed.username'],
                                           'password': Prefs['provider.addic7ed.password'],
@@ -864,8 +863,18 @@ class Config(object):
                                  'only_foreign': self.forced_only,
                                  'also_foreign': self.forced_also,
                              },
+                             'titlovi': {
+                                 'username': Prefs['provider.titlovi.username'],
+                                 'password': Prefs['provider.titlovi.password'],
+                             },
+                             'napisy24': {
+                                 'username': Prefs['provider.napisy24.username'],
+                                 'password': Prefs['provider.napisy24.password'],
+                             },
                              'subscene': {
                                  'only_foreign': self.forced_only,
+                                 'username': Prefs['provider.subscene.username'],
+                                 'password': Prefs['provider.subscene.password'],
                              },
                              'legendastv': {'username': Prefs['provider.legendastv.username'],
                                             'password': Prefs['provider.legendastv.password'],
@@ -1084,11 +1093,12 @@ class Config(object):
     def parse_custom_dns(self):
         custom_dns = Prefs['use_custom_dns2'].strip()
         os.environ["dns_resolvers"] = ""
-        if custom_dns:
+
+        if custom_dns and custom_dns != "system":
             ips = filter(lambda x: x, [d.strip() for d in custom_dns.split(",")])
             if ips:
                 os.environ["dns_resolvers"] = json.dumps(ips)
-                return os.environ["dns_resolvers"]
+            return os.environ["dns_resolvers"]
 
     def init_subliminal_patches(self):
         # configure custom subtitle destination folders for scanning pre-existing subs

Contents/Code/support/download.py

@@ -33,14 +33,14 @@ def get_missing_languages(video, part):
     alpha3_map = {}
     if config.ietf_as_alpha3:
         for language in languages:
-            if language.country:
+            if language and language.country:
                 alpha3_map[language.alpha3] = language.country
                 language.country = None
 
     have_languages = video.subtitle_languages.copy()
     if config.ietf_as_alpha3:
         for language in have_languages:
-            if language.country:
+            if language and language.country:
                 alpha3_map[language.alpha3] = language.country
                 language.country = None
 
@@ -53,14 +53,14 @@ def get_missing_languages(video, part):
                 filter(lambda l: not l.forced, video.subtitle_languages)
             if langs:
                 Log.Debug("We have at least one subtitle for any configured language.")
-                return False
+                return set()
 
         elif "External subtitle" in config.any_language_is_enough:
-            langs = video.subtitle_languages if not not_in_forced else \
+            langs = video.external_subtitle_languages if not not_in_forced else \
                 filter(lambda l: not l.forced, video.external_subtitle_languages)
             if langs:
                 Log.Debug("We have at least one external subtitle for any configured language.")
-                return False
+                return set()
 
     # all languages are found if we either really have subs for all languages or we only want to have exactly one language
     # and we've only found one (the case for a selected language, Prefs['subtitles.only_one'] (one found sub matches any language))
@@ -70,7 +70,7 @@ def get_missing_languages(video, part):
             Log.Debug('Only one language was requested, and we\'ve got a subtitle for %s', video)
         else:
             Log.Debug('All languages %r exist for %s', languages, video)
-        return False
+        return set()
 
     # re-add country codes to the missing languages, in case we've removed them above
     if config.ietf_as_alpha3:
@@ -106,21 +106,22 @@ def language_hook(provider):
 def download_best_subtitles(video_part_map, min_score=0, throttle_time=None, providers=None):
     hearing_impaired = Prefs['subtitles.search.hearingImpaired']
     languages = set([Language.rebuild(l) for l in config.lang_list])
-    missing_languages = []
     if not languages:
         return
 
     use_videos = []
+    missing_languages = set()
     for video, part in video_part_map.iteritems():
         if not video.ignore_all:
-            missing_languages = get_missing_languages(video, part)
+            p_missing_languages = get_missing_languages(video, part)
         else:
-            missing_languages = languages
+            p_missing_languages = languages
 
-        if missing_languages:
-            Log.Info(u"%s has missing languages: %s", os.path.basename(video.name), missing_languages)
+        if p_missing_languages:
+            Log.Info(u"%s has missing languages: %s", os.path.basename(video.name), p_missing_languages)
             refine_video(video, refiner_settings=config.refiner_settings)
             use_videos.append(video)
+            missing_languages.update(p_missing_languages)
 
     # prepare blacklist
     blacklist = get_blacklist_from_part_map(video_part_map, languages)

Contents/Code/support/helpers.py

@@ -394,7 +394,7 @@ def get_language_from_stream(lang_code):
             return Language.fromietf(lang)
         elif lang:
             try:
-                return language_from_stream(lang)
+                return language_from_stream(lang_code)
             except LanguageError:
                 pass
 

Contents/Code/support/localmedia.py

@@ -7,6 +7,7 @@ import helpers
 import subtitlehelpers
 
 from config import config as sz_config
+from subzero.language import ENDSWITH_LANGUAGECODE_RE
 
 
 SECONDARY_TAGS = ['forced', 'normal', 'default', 'embedded', 'embedded-forced', 'custom', 'hi', 'cc', 'sdh']
@@ -125,7 +126,7 @@ def find_subtitles(part, ignore_parts_cleanup=None):
                             root = split_tag[0]
 
                         # get associated media file name without language
-                        sub_fn = subtitlehelpers.ENDSWITH_LANGUAGECODE_RE.sub("", root)
+                        sub_fn = ENDSWITH_LANGUAGECODE_RE.sub("", root)
 
                         # subtitle basename and basename without possible language tag  not found in collected
                         # media files? kill.

Contents/Code/support/plex_media.py

@@ -177,6 +177,7 @@ def get_all_parts(plex_item):
 def get_embedded_subtitle_streams(part, requested_language=None, skip_duplicate_unknown=True, skip_unknown=False):
     streams = []
     streams_unknown = []
+    all_streams = []
     has_unknown = False
     found_requested_language = False
     for stream in part.streams:
@@ -189,27 +190,40 @@ def get_embedded_subtitle_streams(part, requested_language=None, skip_duplicate_
 
             is_unknown = False
             found_requested_language = requested_language and requested_language == language
+            stream_data = None
 
-            if not language and config.treat_und_as_first:
+            if not language:
                 # only consider first unknown subtitle stream
-                if has_unknown and skip_duplicate_unknown:
-                    continue
+                if requested_language and config.treat_und_as_first:
+                    if has_unknown and skip_duplicate_unknown:
+                        Log.Debug("skipping duplicate unknown")
+                        continue
 
-                language = Language.rebuild(list(config.lang_list)[0], forced=is_forced)
+                    language = Language.rebuild(list(config.lang_list)[0], forced=is_forced)
+                else:
+                    language = Language("unk")
                 is_unknown = True
                 has_unknown = True
-                streams_unknown.append({"stream": stream, "is_unknown": is_unknown, "language": language,
-                                        "is_forced": is_forced})
+                stream_data = {"stream": stream, "is_unknown": is_unknown, "language": language,
+                               "is_forced": is_forced}
+                streams_unknown.append(stream_data)
 
             if not requested_language or found_requested_language:
-                streams.append({"stream": stream, "is_unknown": is_unknown, "language": language,
-                                "is_forced": is_forced})
+                stream_data = {"stream": stream, "is_unknown": is_unknown, "language": language,
+                               "is_forced": is_forced}
+                streams.append(stream_data)
 
                 if found_requested_language:
                     break
 
-    if streams_unknown and not found_requested_language and not skip_unknown:
-        streams = streams_unknown
+            if stream_data:
+                all_streams.append(stream_data)
+
+    if requested_language:
+        if streams_unknown and not found_requested_language and not skip_unknown:
+            streams = streams_unknown
+    else:
+        streams = all_streams
 
     return streams
 

Contents/Code/support/scanning.py

@@ -127,7 +127,8 @@ def prepare_video(pms_video_info, ignore_all=False, hints=None, rating_key=None,
             set_existing_languages(video, pms_video_info, external_subtitles=external_subtitles,
                                    embedded_subtitles=embedded_subtitles, known_embedded=known_embedded,
                                    stored_subs=stored_subs, languages=config.lang_list,
-                                   only_one=config.only_one, known_metadata_subs=known_metadata_subs)
+                                   only_one=config.only_one, known_metadata_subs=known_metadata_subs,
+                                   match_strictness=config.ext_match_strictness)
 
         # add video fps info
         video.fps = plex_part.fps

Contents/Code/support/subtitlehelpers.py

@@ -5,6 +5,7 @@ import helpers
 
 from config import config, SUBTITLE_EXTS, TEXT_SUBTITLE_EXTS
 from bs4 import UnicodeDammit
+from subzero.language import match_ietf_language
 
 
 class SubtitleHelper(object):
@@ -85,19 +86,6 @@ class VobSubSubtitleHelper(SubtitleHelper):
 #####################################################################################################################
 
 
-IETF_MATCH = ".+\.([^-.]+)(?:-[A-Za-z]+)?$"
-ENDSWITH_LANGUAGECODE_RE = re.compile("\.([^-.]{2,3})(?:-[A-Za-z]{2,})?$")
-
-
-def match_ietf_language(s):
-    language_match = re.match(".+\.([^\.]+)$" if not helpers.cast_bool(Prefs["subtitles.language.ietf_display"])
-                              else IETF_MATCH, s)
-    if language_match and len(language_match.groups()) == 1:
-        language = language_match.groups()[0]
-        return language
-    return s
-
-
 class DefaultSubtitleHelper(SubtitleHelper):
     @classmethod
     def is_helper_for(cls, filename):
@@ -133,7 +121,7 @@ class DefaultSubtitleHelper(SubtitleHelper):
         # Attempt to extract the language from the filename (e.g. Avatar (2009).eng)
         # IETF support thanks to
         # https://github.com/hpsbranco/LocalMedia.bundle/commit/4fad9aefedece78a1fa96401304351347f644369
-        lang_part = match_ietf_language(file)
+        lang_part = match_ietf_language(file, ietf=helpers.cast_bool(Prefs["subtitles.language.ietf_display"]))
         if lang_part != file:
             language = Locale.Language.Match(lang_part)
         elif config.only_one:

Contents/DefaultPrefs.json

@@ -288,7 +288,7 @@
   },
   {
     "id": "anticaptcha.service",
-    "label": "AntiCaptcha-Service (needs paid account; enables Addic7ed, titlovi)",
+    "label": "AntiCaptcha-Service (needs paid account; enables Addic7ed)",
     "type": "enum",
     "values": [
       "none",
@@ -335,6 +335,26 @@
     "type": "bool",
     "default": "true"
   },
+  {
+    "id": "provider.napisy24.enabled",
+    "label": "Provider: Enable Napisy24 (pl)",
+    "type": "bool",
+    "default": "false"
+  },
+  {
+    "id": "provider.napisy24.username",
+    "label": "Napisy24 Username",
+    "type": "text",
+    "default": ""
+  },
+  {
+    "id": "provider.napisy24.password",
+    "label": "Napisy24 Password",
+    "type": "text",
+    "option": "hidden",
+    "default": "",
+    "secure": "true"
+  },
   {
     "id": "provider.addic7ed.enabled",
     "label": "Provider: Enable Addic7ed (needs AntiCaptcha)",
@@ -389,10 +409,24 @@
   },
   {
     "id": "provider.titlovi.enabled",
-    "label": "Provider: Enable Titlovi.com (might need AntiCaptcha)",
+    "label": "Provider: Enable Titlovi.com (User and Password required)",
     "type": "bool",
     "default": "true"
   },
+  {
+    "id": "provider.titlovi.username",
+    "label": "Titlovi Username",
+    "type": "text",
+    "default": ""
+  },
+  {
+    "id": "provider.titlovi.password",
+    "label": "Titlovi Password",
+    "type": "text",
+    "option": "hidden",
+    "default": "",
+    "secure": "true"
+  },
   {
     "id": "provider.legendastv.enabled",
     "label": "Provider: Enable Legendas TV (mostly pt-BR; UNRAR NEEDED)",
@@ -431,6 +465,20 @@
     "type": "bool",
     "default": "true"
   },
+  {
+    "id": "provider.subscene.username",
+    "label": "SubScene Username",
+    "type": "text",
+    "default": ""
+  },
+  {
+    "id": "provider.subscene.password",
+    "label": "SubScene Password",
+    "type": "text",
+    "option": "hidden",
+    "default": "",
+    "secure": "true"
+  },
   {
     "id": "provider.supersubtitles.enabled",
     "label": "Provider: Enable feliratok.info (Hungarian)",
@@ -861,7 +909,7 @@
   },
   {
     "id": "use_custom_dns2",
-    "label": "Use custom DNS (IPs, comma-separated, leave empty for system DNS. Default: Google/CF)",
+    "label": "Use custom DNS (IPs, comma-separated, set to 'system' for system DNS. Default: Google/CF)",
     "type": "text",
     "default": "1.1.1.1, 8.8.8.8"
   },

Contents/Info.plist

@@ -13,7 +13,7 @@
         <key>CFBundleSignature</key>
         <string>????</string>
         <key>CFBundleVersion</key>
-        <string>2.6.5.3074</string>
+        <string>2.6.5.3152</string>
         <key>PlexFrameworkVersion</key>
         <string>2</string>
         <key>PlexPluginClass</key>
@@ -23,7 +23,7 @@
         <key>PlexPluginConsoleLogging</key>
         <string>0</string>
         <key>PlexPluginDevMode</key>
-        <string>1</string>
+        <string>0</string>
          <key>PlexPluginCodePolicy</key>
             <!-- this allows channels to access some python methods which are otherwise blocked, as well as import external code libraries, and interact with the PMS HTTP API -->
             <string>Elevated</string>
@@ -32,7 +32,7 @@
 
 &lt;h1&gt;Sub-Zero for Plex&lt;/h1&gt;&lt;i&gt;Subtitles done right&lt;/i&gt;
 
-Version 2.6.5.3074 DEV
+Version 2.6.5.3152
 
 Originally based on @bramwalet's awesome &lt;a href=&quot;https://github.com/bramwalet/Subliminal.bundle&quot;&gt;Subliminal.bundle&lt;/a&gt;
 

Contents/Libraries/Shared/subliminal_patch/converters/subscene.py

@@ -12,6 +12,13 @@ from_subscene = {
         'Malay': 'msa', 'Pashto': 'pus', 'Punjabi': 'pan', 'Swahili': 'swa'
 }
 
+from_subscene_with_country = {
+    'Brazillian Portuguese': ('por', 'BR')
+}
+
+to_subscene_with_country = {val: key for key, val in from_subscene_with_country.items()}
+
+
 to_subscene = {v: k for k, v in from_subscene.items()}
 
 exact_languages_alpha3 = [
@@ -34,12 +41,12 @@ language_ids = {
         'mkd': 48, 'mal': 64, 'mni': 65, 'mon': 72, 'pus': 67, 'pol': 31,
         'por': 32, 'pan': 66, 'rus': 34, 'srp': 35, 'sin': 58, 'slk': 36,
         'slv': 37, 'som': 70, 'tgl': 53, 'tam': 59, 'tel': 63, 'tha': 40,
-        'tur': 41, 'ukr': 56, 'urd': 42, 'yor': 71
+        'tur': 41, 'ukr': 56, 'urd': 42, 'yor': 71, 'pt-BR': 4
 }
 
 # TODO: specify codes for unspecified_languages
 unspecified_languages = [
-        'Big 5 code', 'Brazillian Portuguese', 'Bulgarian/ English',
+        'Big 5 code', 'Bulgarian/ English',
         'Chinese BG code', 'Dutch/ English', 'English/ German',
         'Hungarian/ English', 'Rohingya'
 ]
@@ -50,6 +57,8 @@ alpha3_of_code = {l.name: l.alpha3 for l in supported_languages}
 
 supported_languages.update({Language(l) for l in to_subscene})
 
+supported_languages.update({Language(lang, cr) for lang, cr in to_subscene_with_country})
+
 
 class SubsceneConverter(LanguageReverseConverter):
     codes = {l.name for l in supported_languages}
@@ -61,9 +70,15 @@ class SubsceneConverter(LanguageReverseConverter):
         if alpha3 in to_subscene:
             return to_subscene[alpha3]
 
+        if (alpha3, country) in to_subscene_with_country:
+            return to_subscene_with_country[(alpha3, country)]
+
         raise ConfigurationError('Unsupported language for subscene: %s, %s, %s' % (alpha3, country, script))
 
     def reverse(self, code):
+        if code in from_subscene_with_country:
+            return from_subscene_with_country[code]
+
         if code in from_subscene:
             return (from_subscene[code],)
 

Contents/Libraries/Shared/subliminal_patch/converters/titlovi.py

@@ -27,16 +27,6 @@ class TitloviConverter(LanguageReverseConverter):
                            }
         self.codes = set(self.from_titlovi.keys())
 
-        # temporary fix, should be removed as soon as API is used
-        self.lang_from_countrycode = {'ba': ('bos',),
-                                 'en': ('eng',),
-                                 'hr': ('hrv',),
-                                 'mk': ('mkd',),
-                                 'rs': ('srp',),
-                                 'rsc': ('srp', None, 'Cyrl'),
-                                 'si': ('slv',)
-                                 }
-
     def convert(self, alpha3, country=None, script=None):
         if (alpha3, country, script) in self.to_titlovi:
             return self.to_titlovi[(alpha3, country, script)]
@@ -49,9 +39,5 @@ class TitloviConverter(LanguageReverseConverter):
         if titlovi in self.from_titlovi:
             return self.from_titlovi[titlovi]
 
-        # temporary fix, should be removed as soon as API is used
-        if titlovi in self.lang_from_countrycode:
-            return self.lang_from_countrycode[titlovi]
-
         raise ConfigurationError('Unsupported language number for titlovi: %s' % titlovi)
 

Contents/Libraries/Shared/subliminal_patch/core.py

@@ -30,7 +30,7 @@ from subliminal.core import guessit, ProviderPool, io, is_windows_special_path,
     ThreadPoolExecutor, check_video
 from subliminal_patch.exceptions import TooManyRequests, APIThrottled
 
-from subzero.language import Language
+from subzero.language import Language, ENDSWITH_LANGUAGECODE_RE
 from scandir import scandir, scandir_generic as _scandir_generic
 
 logger = logging.getLogger(__name__)
@@ -309,7 +309,8 @@ class SZProviderPool(ProviderPool):
             logger.error('Invalid subtitle')
             return False
 
-        subtitle.normalize()
+        if not os.environ.get("SZ_KEEP_ENCODING", False):
+            subtitle.normalize()
 
         return True
 
@@ -472,7 +473,7 @@ if is_windows_special_path:
     SZAsyncProviderPool = SZProviderPool
 
 
-def scan_video(path, dont_use_actual_file=False, hints=None, providers=None, skip_hashing=False):
+def scan_video(path, dont_use_actual_file=False, hints=None, providers=None, skip_hashing=False, hash_from=None):
     """Scan a video from a `path`.
 
     patch:
@@ -537,28 +538,34 @@ def scan_video(path, dont_use_actual_file=False, hints=None, providers=None, ski
             video.alternative_titles.append(alt_guess["title"])
         logger.debug("Adding alternative title: %s", alt_guess["title"])
 
-    if dont_use_actual_file:
+    if dont_use_actual_file and not hash_from:
         return video
 
     # size and hashes
     if not skip_hashing:
-        video.size = os.path.getsize(path)
+        hash_path = hash_from or path
+        video.size = os.path.getsize(hash_path)
         if video.size > 10485760:
             logger.debug('Size is %d', video.size)
+            osub_hash = None
             if "opensubtitles" in providers:
-                video.hashes['opensubtitles'] = hash_opensubtitles(path)
+                video.hashes['opensubtitles'] = osub_hash = hash_opensubtitles(hash_path)
 
             if "shooter" in providers:
-                video.hashes['shooter'] = hash_shooter(path)
+                video.hashes['shooter'] = hash_shooter(hash_path)
 
             if "thesubdb" in providers:
-                video.hashes['thesubdb'] = hash_thesubdb(path)
+                video.hashes['thesubdb'] = hash_thesubdb(hash_path)
 
             if "napiprojekt" in providers:
                 try:
-                    video.hashes['napiprojekt'] = hash_napiprojekt(path)
+                    video.hashes['napiprojekt'] = hash_napiprojekt(hash_path)
                 except MemoryError:
-                    logger.warning(u"Couldn't compute napiprojekt hash for %s", path)
+                    logger.warning(u"Couldn't compute napiprojekt hash for %s", hash_path)
+
+            if "napisy24" in providers:
+                # Napisy24 uses the same hash as opensubtitles
+                video.hashes['napisy24'] = osub_hash or hash_opensubtitles(hash_path)
 
             logger.debug('Computed hashes %r', video.hashes)
         else:
@@ -567,14 +574,16 @@ def scan_video(path, dont_use_actual_file=False, hints=None, providers=None, ski
     return video
 
 
-def _search_external_subtitles(path, languages=None, only_one=False, scandir_generic=False):
+def _search_external_subtitles(path, languages=None, only_one=False, scandir_generic=False, match_strictness="strict"):
     dirpath, filename = os.path.split(path)
     dirpath = dirpath or '.'
-    fileroot, fileext = os.path.splitext(filename)
+    fn_no_ext, fileext = os.path.splitext(filename)
+    fn_no_ext_lower = fn_no_ext.lower()
     subtitles = {}
     _scandir = _scandir_generic if scandir_generic else scandir
+
     for entry in _scandir(dirpath):
-        if not entry.name and not scandir_generic:
+        if (not entry.name or entry.name in ('\x0c', '$', ',', '\x7f')) and not scandir_generic:
             logger.debug('Could not determine the name of the file, retrying with scandir_generic')
             return _search_external_subtitles(path, languages, only_one, True)
         if not entry.is_file(follow_symlinks=False):
@@ -583,9 +592,11 @@ def _search_external_subtitles(path, languages=None, only_one=False, scandir_gen
         p = entry.name
 
         # keep only valid subtitle filenames
-        if not p.lower().startswith(fileroot.lower()) or not p.lower().endswith(SUBTITLE_EXTENSIONS):
+        if not p.lower().endswith(SUBTITLE_EXTENSIONS):
             continue
 
+        # not p.lower().startswith(fileroot.lower()) or not
+
         p_root, p_ext = os.path.splitext(p)
         if not INCLUDE_EXOTIC_SUBS and p_ext not in (".srt", ".ass", ".ssa", ".vtt"):
             continue
@@ -604,7 +615,19 @@ def _search_external_subtitles(path, languages=None, only_one=False, scandir_gen
             forced = "forced" in adv_tag
 
         # extract the potential language code
-        language_code = p_root[len(fileroot):].replace('_', '-')[1:]
+        language_code = p_root.rsplit(".", 1)[1].replace('_', '-')
+
+        # remove possible language code for matching
+        p_root_bare = ENDSWITH_LANGUAGECODE_RE.sub("", p_root)
+
+        p_root_lower = p_root_bare.lower()
+
+        filename_matches = p_root_lower == fn_no_ext_lower
+        filename_contains = p_root_lower in fn_no_ext_lower
+
+        if not filename_matches:
+            if match_strictness == "strict" or (match_strictness == "loose" and not filename_contains):
+                continue
 
         # default language is undefined
         language = Language('und')
@@ -628,7 +651,7 @@ def _search_external_subtitles(path, languages=None, only_one=False, scandir_gen
     return subtitles
 
 
-def search_external_subtitles(path, languages=None, only_one=False):
+def search_external_subtitles(path, languages=None, only_one=False, match_strictness="strict"):
     """
     wrap original search_external_subtitles function to search multiple paths for one given video
     # todo: cleanup and merge with _search_external_subtitles
@@ -649,10 +672,11 @@ def search_external_subtitles(path, languages=None, only_one=False):
         if os.path.isdir(os.path.dirname(abspath)):
             try:
                 subtitles.update(_search_external_subtitles(abspath, languages=languages,
-                                                            only_one=only_one))
+                                                            only_one=only_one, match_strictness=match_strictness))
             except OSError:
                 subtitles.update(_search_external_subtitles(abspath, languages=languages,
-                                                            only_one=only_one, scandir_generic=True))
+                                                            only_one=only_one, match_strictness=match_strictness,
+                                                            scandir_generic=True))
     logger.debug("external subs: found %s", subtitles)
     return subtitles
 

Contents/Libraries/Shared/subliminal_patch/http.py

@@ -354,7 +354,6 @@ def patch_create_connection():
                         return _orig_create_connection((ip, port), *args, **kwargs)
                     except dns.exception.DNSException:
                         logger.warning("DNS: Couldn't resolve %s with DNS: %s", host, custom_resolver.nameservers)
-                        raise
 
         logger.debug("DNS: Falling back to default DNS or IP on %s", host)
         return _orig_create_connection((host, port), *args, **kwargs)

Contents/Libraries/Shared/subliminal_patch/providers/argenteam.py

@@ -23,9 +23,10 @@ class ArgenteamSubtitle(Subtitle):
     hearing_impaired_verifiable = False
     _release_info = None
 
-    def __init__(self, language, download_link, movie_kind, title, season, episode, year, release, version, source,
+    def __init__(self, language, page_link, download_link, movie_kind, title, season, episode, year, release, version, source,
                  video_codec, tvdb_id, imdb_id, asked_for_episode=None, asked_for_release_group=None, *args, **kwargs):
-        super(ArgenteamSubtitle, self).__init__(language, download_link, *args, **kwargs)
+        super(ArgenteamSubtitle, self).__init__(language, page_link=page_link, *args, **kwargs)
+        self.page_link = page_link
         self.download_link = download_link
         self.movie_kind = movie_kind
         self.title = title
@@ -135,7 +136,8 @@ class ArgenteamProvider(Provider, ProviderSubtitleArchiveMixin):
     provider_name = 'argenteam'
     languages = {Language.fromalpha2(l) for l in ['es']}
     video_types = (Episode, Movie)
-    API_URL = "http://argenteam.net/api/v1/"
+    BASE_URL = "http://www.argenteam.net/"
+    API_URL = BASE_URL + "api/v1/"
     subtitle_class = ArgenteamSubtitle
     hearing_impaired_verifiable = False
     language_list = list(languages)
@@ -240,12 +242,13 @@ class ArgenteamProvider(Provider, ProviderSubtitleArchiveMixin):
 
             for r in content['releases']:
                 for s in r['subtitles']:
-                    sub = ArgenteamSubtitle(language, s['uri'], "episode" if is_episode else "movie", returned_title,
+                    movie_kind = "episode" if is_episode else "movie"
+                    page_link = self.BASE_URL + movie_kind + "/" + str(aid)
+                    sub = ArgenteamSubtitle(language, page_link, s['uri'], movie_kind, returned_title,
                                             season, episode, year, r.get('team'), r.get('tags'),
                                             r.get('source'), r.get('codec'), content.get("tvdb"), imdb_id,
                                             asked_for_release_group=video.release_group,
-                                            asked_for_episode=episode
-                                            )
+                                            asked_for_episode=episode)
                     subtitles.append(sub)
 
             if has_multiple_ids:

Contents/Libraries/Shared/subliminal_patch/providers/napisy24.py

@@ -0,0 +1,124 @@
+import logging
+import os
+from io import BytesIO
+from zipfile import ZipFile
+
+from requests import Session
+
+from subliminal_patch.subtitle import Subtitle
+from subliminal_patch.providers import Provider
+from subliminal import __short_version__
+from subliminal.exceptions import AuthenticationError, ConfigurationError
+from subliminal.subtitle import fix_line_ending
+from subzero.language import Language
+
+logger = logging.getLogger(__name__)
+
+
+class Napisy24Subtitle(Subtitle):
+    '''Napisy24 Subtitle.'''
+    provider_name = 'napisy24'
+
+    def __init__(self, language, hash, imdb_id, napis_id):
+        super(Napisy24Subtitle, self).__init__(language)
+        self.hash = hash
+        self.imdb_id = imdb_id
+        self.napis_id = napis_id
+
+    @property
+    def id(self):
+        return self.hash
+
+    def get_matches(self, video):
+        matches = set()
+
+        # hash
+        if 'napisy24' in video.hashes and video.hashes['napisy24'] == self.hash:
+            matches.add('hash')
+
+        # imdb_id
+        if video.imdb_id and self.imdb_id == video.imdb_id:
+            matches.add('imdb_id')
+
+        return matches
+
+
+class Napisy24Provider(Provider):
+    '''Napisy24 Provider.'''
+    languages = {Language(l) for l in ['pol']}
+    required_hash = 'napisy24'
+    api_url = 'http://napisy24.pl/run/CheckSubAgent.php'
+
+    def __init__(self, username=None, password=None):
+        if all((username, password)):
+            self.username = username
+            self.password = password
+        else:
+            self.username = 'subliminal'
+            self.password = 'lanimilbus'
+
+        self.session = None
+
+    def initialize(self):
+        self.session = Session()
+        self.session.headers['User-Agent'] = 'Subliminal/%s' % __short_version__
+        self.session.headers['Content-Type'] = 'application/x-www-form-urlencoded'
+
+    def terminate(self):
+        self.session.close()
+
+    def query(self, language, size, name, hash):
+        params = {
+            'postAction': 'CheckSub',
+            'ua': self.username,
+            'ap': self.password,
+            'fs': size,
+            'fh': hash,
+            'fn': os.path.basename(name),
+            'n24pref': 1
+        }
+
+        response = self.session.post(self.api_url, data=params, timeout=10)
+        response.raise_for_status()
+
+        response_content = response.content.split(b'||', 1)
+        n24_data = response_content[0].decode()
+
+        if n24_data[:2] != 'OK':
+            if n24_data[:11] == 'login error':
+                raise AuthenticationError('Login failed')
+            logger.error('Unknown response: %s', response.content)
+            return None
+
+        n24_status = n24_data[:4]
+        if n24_status == 'OK-0':
+            logger.info('No subtitles found')
+            return None
+
+        subtitle_info = dict(p.split(':', 1) for p in n24_data.split('|')[1:])
+        logger.debug('Subtitle info: %s', subtitle_info)
+
+        if n24_status == 'OK-1':
+            logger.info('No subtitles found but got video info')
+            return None
+        elif n24_status == 'OK-2':
+            logger.info('Found subtitles')
+        elif n24_status == 'OK-3':
+            logger.info('Found subtitles but not from Napisy24 database')
+            return None
+        
+        subtitle_content = response_content[1]
+
+        subtitle = Napisy24Subtitle(language, hash, 'tt%s' % subtitle_info['imdb'].zfill(7), subtitle_info['napisId'])
+        with ZipFile(BytesIO(subtitle_content)) as zf:
+            subtitle.content = fix_line_ending(zf.open(zf.namelist()[0]).read())
+
+        return subtitle
+
+    def list_subtitles(self, video, languages):
+        subtitles = [self.query(l, video.size, video.name, video.hashes['napisy24']) for l in languages]
+        return [s for s in subtitles if s is not None]
+
+    def download_subtitle(self, subtitle):
+        # there is no download step, content is already filled from listing subtitles
+        pass

Contents/Libraries/Shared/subliminal_patch/providers/subscene.py

@@ -4,24 +4,34 @@ import io
 import logging
 import os
 import time
+import traceback
+
+import requests
+
 import inflect
+import re
+import json
+import HTMLParser
+import urlparse
 
 from zipfile import ZipFile
 from babelfish import language_converters
 from guessit import guessit
+from dogpile.cache.api import NO_VALUE
 from subliminal import Episode, ProviderError
+from subliminal.exceptions import ConfigurationError, ServiceUnavailable
 from subliminal.utils import sanitize_release_group
+from subliminal.cache import region
 from subliminal_patch.http import RetryingCFSession
 from subliminal_patch.providers import Provider
 from subliminal_patch.providers.mixins import ProviderSubtitleArchiveMixin
 from subliminal_patch.subtitle import Subtitle, guess_matches
 from subliminal_patch.converters.subscene import language_ids, supported_languages
-from subscene_api.subscene import search, Subtitle as APISubtitle
+from subscene_api.subscene import search, Subtitle as APISubtitle, SITE_DOMAIN
 from subzero.language import Language
 
 p = inflect.engine()
 
-
 language_converters.register('subscene = subliminal_patch.converters.subscene:SubsceneConverter')
 logger = logging.getLogger(__name__)
 
@@ -112,28 +122,106 @@ class SubsceneProvider(Provider, ProviderSubtitleArchiveMixin):
     skip_wrong_fps = False
     hearing_impaired_verifiable = True
     only_foreign = False
+    username = None
+    password = None
 
-    search_throttle = 2  # seconds
+    search_throttle = 8  # seconds
+
+    def __init__(self, only_foreign=False, username=None, password=None):
+        if not all((username, password)):
+            raise ConfigurationError('Username and password must be specified')
 
-    def __init__(self, only_foreign=False):
         self.only_foreign = only_foreign
+        self.username = username
+        self.password = password
 
     def initialize(self):
         logger.info("Creating session")
         self.session = RetryingCFSession()
 
+        prev_cookies = region.get("subscene_cookies2")
+        if prev_cookies != NO_VALUE:
+            logger.debug("Re-using old subscene cookies: %r", prev_cookies)
+            self.session.cookies.update(prev_cookies)
+
+        else:
+            logger.debug("Logging in")
+            self.login()
+
+    def login(self):
+        r = self.session.get("https://subscene.com/account/login")
+        if "Server Error" in r.content:
+            logger.error("Login unavailable; Maintenance?")
+            raise ServiceUnavailable("Login unavailable; Maintenance?")
+
+        match = re.search(r"<script id='modelJson' type='application/json'>\s*(.+)\s*</script>", r.content)
+
+        if match:
+            h = HTMLParser.HTMLParser()
+            data = json.loads(h.unescape(match.group(1)))
+            login_url = urlparse.urljoin(data["siteUrl"], data["loginUrl"])
+            time.sleep(1.0)
+
+            r = self.session.post(login_url,
+                                  {
+                                      "username": self.username,
+                                      "password": self.password,
+                                      data["antiForgery"]["name"]: data["antiForgery"]["value"]
+                                  })
+            pep_content = re.search(r"<form method=\"post\" action=\"https://subscene\.com/\">"
+                                    r".+name=\"id_token\".+?value=\"(?P<id_token>.+?)\".*?"
+                                    r"access_token\".+?value=\"(?P<access_token>.+?)\".+?"
+                                    r"token_type.+?value=\"(?P<token_type>.+?)\".+?"
+                                    r"expires_in.+?value=\"(?P<expires_in>.+?)\".+?"
+                                    r"scope.+?value=\"(?P<scope>.+?)\".+?"
+                                    r"state.+?value=\"(?P<state>.+?)\".+?"
+                                    r"session_state.+?value=\"(?P<session_state>.+?)\"",
+                                    r.content, re.MULTILINE | re.DOTALL)
+
+            if pep_content:
+                r = self.session.post(SITE_DOMAIN, pep_content.groupdict())
+                try:
+                    r.raise_for_status()
+                except Exception:
+                    raise ProviderError("Something went wrong when trying to log in: %s", traceback.format_exc())
+                else:
+                    cj = self.session.cookies.copy()
+                    store_cks = ("scene", "idsrv", "idsrv.xsrf", "idsvr.clients", "idsvr.session", "idsvr.username")
+                    for cn in self.session.cookies.iterkeys():
+                        if cn not in store_cks:
+                            del cj[cn]
+
+                    logger.debug("Storing cookies: %r", cj)
+                    region.set("subscene_cookies2", cj)
+                    return
+        raise ProviderError("Something went wrong when trying to log in #1")
+
     def terminate(self):
         logger.info("Closing session")
         self.session.close()
 
     def _create_filters(self, languages):
         self.filters = dict(HearingImpaired="2")
+        acc_filters = self.filters.copy()
         if self.only_foreign:
             self.filters["ForeignOnly"] = "True"
+            acc_filters["ForeignOnly"] = self.filters["ForeignOnly"].lower()
             logger.info("Only searching for foreign/forced subtitles")
 
-        self.filters["LanguageFilter"] = ",".join((str(language_ids[l.alpha3]) for l in languages
-                                                   if l.alpha3 in language_ids))
+        selected_ids = []
+        for l in languages:
+            lid = language_ids.get(l.basename, language_ids.get(l.alpha3, None))
+            if lid:
+                selected_ids.append(str(lid))
+
+        acc_filters["SelectedIds"] = selected_ids
+        self.filters["LanguageFilter"] = ",".join(acc_filters["SelectedIds"])
+
+        last_filters = region.get("subscene_filters")
+        if last_filters != acc_filters:
+            region.set("subscene_filters", acc_filters)
+            logger.debug("Setting account filters to %r", acc_filters)
+            self.session.post("https://u.subscene.com/filter", acc_filters, allow_redirects=False)
 
         logger.debug("Filter created: '%s'" % self.filters)
 
@@ -176,7 +264,11 @@ class SubsceneProvider(Provider, ProviderSubtitleArchiveMixin):
     def parse_results(self, video, film):
         subtitles = []
         for s in film.subtitles:
-            subtitle = SubsceneSubtitle.from_api(s)
+            try:
+                subtitle = SubsceneSubtitle.from_api(s)
+            except NotImplementedError, e:
+                logger.info(e)
+                continue
             subtitle.asked_for_release_group = video.release_group
             if isinstance(video, Episode):
                 subtitle.asked_for_episode = video.episode
@@ -189,10 +281,16 @@ class SubsceneProvider(Provider, ProviderSubtitleArchiveMixin):
 
         return subtitles
 
+    def do_search(self, *args, **kwargs):
+        try:
+            return search(*args, **kwargs)
+        except requests.HTTPError:
+            region.delete("subscene_cookies2")
+
     def query(self, video):
-        vfn = get_video_filename(video)
+        # vfn = get_video_filename(video)
         subtitles = []
-        #logger.debug(u"Searching for: %s", vfn)
+        # logger.debug(u"Searching for: %s", vfn)
         # film = search(vfn, session=self.session)
         #
         # if film and film.subtitles:
@@ -201,16 +299,17 @@ class SubsceneProvider(Provider, ProviderSubtitleArchiveMixin):
         # else:
         #     logger.debug('No release results found')
 
-        #time.sleep(self.search_throttle)
+        # time.sleep(self.search_throttle)
 
         # re-search for episodes without explicit release name
         if isinstance(video, Episode):
-            #term = u"%s S%02iE%02i" % (video.series, video.season, video.episode)
-            more_than_one = len([video.series] + video.alternative_series) > 1
-            for series in [video.series] + video.alternative_series:
+            titles = list(set([video.series] + video.alternative_series))[:2]
+            # term = u"%s S%02iE%02i" % (video.series, video.season, video.episode)
+            more_than_one = len(titles) > 1
+            for series in titles:
                 term = u"%s - %s Season" % (series, p.number_to_words("%sth" % video.season).capitalize())
                 logger.debug('Searching for alternative results: %s', term)
-                film = search(term, session=self.session, release=False, throttle=self.search_throttle)
+                film = self.do_search(term, session=self.session, release=False, throttle=self.search_throttle)
                 if film and film.subtitles:
                     logger.debug('Alternative results found: %s', len(film.subtitles))
                     subtitles += self.parse_results(video, film)
@@ -218,26 +317,27 @@ class SubsceneProvider(Provider, ProviderSubtitleArchiveMixin):
                     logger.debug('No alternative results found')
 
                 # packs
-                if video.season_fully_aired:
-                    term = u"%s S%02i" % (series, video.season)
-                    logger.debug('Searching for packs: %s', term)
-                    time.sleep(self.search_throttle)
-                    film = search(term, session=self.session, throttle=self.search_throttle)
-                    if film and film.subtitles:
-                        logger.debug('Pack results found: %s', len(film.subtitles))
-                        subtitles += self.parse_results(video, film)
-                    else:
-                        logger.debug('No pack results found')
-                else:
-                    logger.debug("Not searching for packs, because the season hasn't fully aired")
+                # if video.season_fully_aired:
+                #     term = u"%s S%02i" % (series, video.season)
+                #     logger.debug('Searching for packs: %s', term)
+                #     time.sleep(self.search_throttle)
+                #     film = search(term, session=self.session, throttle=self.search_throttle)
+                #     if film and film.subtitles:
+                #         logger.debug('Pack results found: %s', len(film.subtitles))
+                #         subtitles += self.parse_results(video, film)
+                #     else:
+                #         logger.debug('No pack results found')
+                # else:
+                #     logger.debug("Not searching for packs, because the season hasn't fully aired")
                 if more_than_one:
                     time.sleep(self.search_throttle)
         else:
-            more_than_one = len([video.title] + video.alternative_titles) > 1
-            for title in [video.title] + video.alternative_titles:
-                logger.debug('Searching for movie results: %s', title)
-                film = search(title, year=video.year, session=self.session, limit_to=None, release=False,
-                              throttle=self.search_throttle)
+            titles = list(set([video.title] + video.alternative_titles))[:2]
+            more_than_one = len(titles) > 1
+            for title in titles:
+                logger.debug('Searching for movie results: %r', title)
+                film = self.do_search(title, year=video.year, session=self.session, limit_to=None, release=False,
+                                      throttle=self.search_throttle)
                 if film and film.subtitles:
                     subtitles += self.parse_results(video, film)
                 if more_than_one:

Contents/Libraries/Shared/subliminal_patch/providers/titlovi.py

@@ -2,42 +2,35 @@
 
 import io
 import logging
-import math
 import re
-import time
+from datetime import datetime
+import dateutil.parser
 
 import rarfile
 
-from bs4 import BeautifulSoup
 from zipfile import ZipFile, is_zipfile
 from rarfile import RarFile, is_rarfile
 from babelfish import language_converters, Script
-from requests import RequestException
+from requests import RequestException, codes as request_codes
 from guessit import guessit
 from subliminal_patch.http import RetryingCFSession
 from subliminal_patch.providers import Provider
 from subliminal_patch.providers.mixins import ProviderSubtitleArchiveMixin
 from subliminal_patch.subtitle import Subtitle
 from subliminal_patch.utils import sanitize, fix_inconsistent_naming as _fix_inconsistent_naming
-from subliminal.exceptions import ProviderError
+from subliminal.exceptions import ProviderError, AuthenticationError, ConfigurationError
 from subliminal.score import get_equivalent_release_groups
 from subliminal.utils import sanitize_release_group
 from subliminal.subtitle import guess_matches
 from subliminal.video import Episode, Movie
 from subliminal.subtitle import fix_line_ending
-from subliminal_patch.pitcher import pitchers, load_verification, store_verification
-from subzero.language import Language
 
-from random import randint
-from .utils import FIRST_THOUSAND_OR_SO_USER_AGENTS as AGENT_LIST
+from subzero.language import Language
+from dogpile.cache.api import NO_VALUE
+from subliminal.cache import region
 
 # parsing regex definitions
 title_re = re.compile(r'(?P<title>(?:.+(?= [Aa][Kk][Aa] ))|.+)(?:(?:.+)(?P<altitle>(?<= [Aa][Kk][Aa] ).+))?')
-lang_re = re.compile(r'(?<=flags/)(?P<lang>.{2})(?:.)(?P<script>c?)(?:.+)')
-season_re = re.compile(r'Sezona (?P<season>\d+)')
-episode_re = re.compile(r'Epizoda (?P<episode>\d+)')
-year_re = re.compile(r'(?P<year>\d+)')
-fps_re = re.compile(r'fps: (?P<fps>.+)')
 
 
 def fix_inconsistent_naming(title):
@@ -51,6 +44,7 @@ def fix_inconsistent_naming(title):
     return _fix_inconsistent_naming(title, {"DC's Legends of Tomorrow": "Legends of Tomorrow",
                                             "Marvel's Jessica Jones": "Jessica Jones"})
 
+
 logger = logging.getLogger(__name__)
 
 # Configure :mod:`rarfile` to use the same path separator as :mod:`zipfile`
@@ -62,9 +56,9 @@ language_converters.register('titlovi = subliminal_patch.converters.titlovi:Titl
 class TitloviSubtitle(Subtitle):
     provider_name = 'titlovi'
 
-    def __init__(self, language, page_link, download_link, sid, releases, title, alt_title=None, season=None,
-                 episode=None, year=None, fps=None, asked_for_release_group=None, asked_for_episode=None):
-        super(TitloviSubtitle, self).__init__(language, page_link=page_link)
+    def __init__(self, language, download_link, sid, releases, title, alt_title=None, season=None,
+                 episode=None, year=None, rating=None, download_count=None, asked_for_release_group=None, asked_for_episode=None):
+        super(TitloviSubtitle, self).__init__(language)
         self.sid = sid
         self.releases = self.release_info = releases
         self.title = title
@@ -73,11 +67,21 @@ class TitloviSubtitle(Subtitle):
         self.episode = episode
         self.year = year
         self.download_link = download_link
-        self.fps = fps
+        self.rating = rating
+        self.download_count = download_count
         self.matches = None
         self.asked_for_release_group = asked_for_release_group
         self.asked_for_episode = asked_for_episode
 
+    def __repr__(self):
+        if self.season and self.episode:
+            return '<%s "%s (%r)" s%.2de%.2d [%s:%s] ID:%r R:%.2f D:%r>' % (
+                self.__class__.__name__, self.title, self.year, self.season, self.episode, self.language, self._guessed_encoding, self.sid,
+                self.rating, self.download_count)
+        else:
+            return '<%s "%s (%r)" [%s:%s] ID:%r R:%.2f D:%r>' % (
+                self.__class__.__name__, self.title, self.year,  self.language, self._guessed_encoding, self.sid, self.rating, self.download_count)
+
     @property
     def id(self):
         return self.sid
@@ -134,20 +138,62 @@ class TitloviSubtitle(Subtitle):
 class TitloviProvider(Provider, ProviderSubtitleArchiveMixin):
     subtitle_class = TitloviSubtitle
     languages = {Language.fromtitlovi(l) for l in language_converters['titlovi'].codes} | {Language.fromietf('sr-Latn')}
-    server_url = 'https://titlovi.com'
-    search_url = server_url + '/titlovi/?'
-    download_url = server_url + '/download/?type=1&mediaid='
+    api_url = 'https://kodi.titlovi.com/api/subtitles'
+    api_gettoken_url = api_url + '/gettoken'
+    api_search_url = api_url + '/search'
+
+    def __init__(self, username=None, password=None):
+        if not all((username, password)):
+            raise ConfigurationError('Username and password must be specified')
+
+        self.username = username
+        self.password = password
+
+        self.session = None
+
+        self.user_id = None
+        self.login_token = None
+        self.token_exp = None
 
     def initialize(self):
         self.session = RetryingCFSession()
         #load_verification("titlovi", self.session)
 
+        token = region.get("titlovi_token")
+        if token is not NO_VALUE:
+            self.user_id, self.login_token, self.token_exp = token
+            if datetime.now() > self.token_exp:
+                logger.debug('Token expired')
+                self.log_in()
+            else:
+                logger.debug('Use cached token')
+        else:
+            logger.debug('Token not found in cache')
+            self.log_in()
+
+    def log_in(self):
+        login_params = dict(username=self.username, password=self.password, json=True)
+        try:
+            response = self.session.post(self.api_gettoken_url, params=login_params)
+            if response.status_code == request_codes.ok:
+                resp_json = response.json()
+                self.login_token = resp_json.get('Token')
+                self.user_id = resp_json.get('UserId')
+                self.token_exp = dateutil.parser.parse(resp_json.get('ExpirationDate'))
+
+                region.set("titlovi_token", [self.user_id, self.login_token, self.token_exp])
+                logger.debug('New token obtained')
+
+            elif response.status_code == request_codes.unauthorized:
+                raise AuthenticationError('Login failed')
+
+        except RequestException as e:
+            logger.error(e)
     def terminate(self):
         self.session.close()
 
-    def query(self, languages, title, season=None, episode=None, year=None, video=None):
-        items_per_page = 10
-        current_page = 1
+    def query(self, languages, title, season=None, episode=None, year=None, imdb_id=None, video=None):
+        search_params = dict()
 
         used_languages = languages
         lang_strings = [str(lang) for lang in used_languages]
@@ -162,135 +208,73 @@ class TitloviProvider(Provider, ProviderSubtitleArchiveMixin):
         langs = '|'.join(map(str, [l.titlovi for l in used_languages]))
 
         # set query params
-        params = {'prijevod': title, 'jezik': langs}
+        search_params['query'] = title
+        search_params['lang'] = langs
         is_episode = False
         if season and episode:
             is_episode = True
-            params['s'] = season
-            params['e'] = episode
-        if year:
-            params['g'] = year
+            search_params['season'] = season
+            search_params['episode'] = episode
+        #if year:
+        #    search_params['year'] = year
+        if imdb_id:
+            search_params['imdbID'] = imdb_id
 
         # loop through paginated results
-        logger.info('Searching subtitles %r', params)
+        logger.info('Searching subtitles %r', search_params)
         subtitles = []
+        query_results = []
 
-        while True:
-            # query the server
-            try:
-                r = self.session.get(self.search_url, params=params, timeout=10)
-                r.raise_for_status()
-            except RequestException as e:
-                logger.exception('RequestException %s', e)
-                break
+        try:
+            search_params['token'] = self.login_token
+            search_params['userid'] = self.user_id
+            search_params['json'] = True
+
+            response = self.session.get(self.api_search_url, params=search_params)
+            resp_json = response.json()
+            if resp_json['SubtitleResults']:
+                query_results.extend(resp_json['SubtitleResults'])
+
+
+        except Exception as e:
+            logger.error(e)
+
+        for sub in query_results:
+
+            # title and alternate title
+            match = title_re.search(sub.get('Title'))
+            if match:
+                _title = match.group('title')
+                alt_title = match.group('altitle')
             else:
-                try:
-                    soup = BeautifulSoup(r.content, 'lxml')
+                continue
 
-                    # number of results
-                    result_count = int(soup.select_one('.results_count b').string)
-                except:
-                    result_count = None
+            # handle movies and series separately
+            if is_episode:
+                subtitle = self.subtitle_class(Language.fromtitlovi(sub.get('Lang')), sub.get('Link'), sub.get('Id'), sub.get('Release'), _title,
+                                               alt_title=alt_title, season=sub.get('Season'), episode=sub.get('Episode'),
+                                               year=sub.get('Year'), rating=sub.get('Rating'),
+                                               download_count=sub.get('DownloadCount'),
+                                               asked_for_release_group=video.release_group,
+                                               asked_for_episode=episode)
+            else:
+                subtitle = self.subtitle_class(Language.fromtitlovi(sub.get('Lang')), sub.get('Link'), sub.get('Id'), sub.get('Release'), _title,
+                                               alt_title=alt_title, year=sub.get('Year'), rating=sub.get('Rating'),
+                                               download_count=sub.get('DownloadCount'),
+                                               asked_for_release_group=video.release_group)
+            logger.debug('Found subtitle %r', subtitle)
 
-                # exit if no results
-                if not result_count:
-                    if not subtitles:
-                        logger.debug('No subtitles found')
-                    else:
-                        logger.debug("No more subtitles found")
-                    break
+            # prime our matches so we can use the values later
+            subtitle.get_matches(video)
 
-                # number of pages with results
-                pages = int(math.ceil(result_count / float(items_per_page)))
-
-                # get current page
-                if 'pg' in params:
-                    current_page = int(params['pg'])
-
-                try:
-                    sublist = soup.select('section.titlovi > ul.titlovi > li.subtitleContainer.canEdit')
-                    for sub in sublist:
-                        # subtitle id
-                        sid = sub.find(attrs={'data-id': True}).attrs['data-id']
-                        # get download link
-                        download_link = self.download_url + sid
-                        # title and alternate title
-                        match = title_re.search(sub.a.string)
-                        if match:
-                            _title = match.group('title')
-                            alt_title = match.group('altitle')
-                        else:
-                            continue
-
-                        # page link
-                        page_link = self.server_url + sub.a.attrs['href']
-                        # subtitle language
-                        _lang = sub.select_one('.lang')
-                        match = lang_re.search(_lang.attrs.get('src', _lang.attrs.get('cfsrc', '')))
-                        if match:
-                            try:
-                                # decode language
-                                lang = Language.fromtitlovi(match.group('lang')+match.group('script'))
-                            except ValueError:
-                                continue
-
-                        # relase year or series start year
-                        match = year_re.search(sub.find(attrs={'data-id': True}).parent.i.string)
-                        if match:
-                            r_year = int(match.group('year'))
-                        # fps
-                        match = fps_re.search(sub.select_one('.fps').string)
-                        if match:
-                            fps = match.group('fps')
-                        # releases
-                        releases = str(sub.select_one('.fps').parent.contents[0].string)
-
-                        # handle movies and series separately
-                        if is_episode:
-                            # season and episode info
-                            sxe = sub.select_one('.s0xe0y').string
-                            r_season = None
-                            r_episode = None
-                            if sxe:
-                                match = season_re.search(sxe)
-                                if match:
-                                    r_season = int(match.group('season'))
-                                match = episode_re.search(sxe)
-                                if match:
-                                    r_episode = int(match.group('episode'))
-
-                            subtitle = self.subtitle_class(lang, page_link, download_link, sid, releases, _title,
-                                                           alt_title=alt_title, season=r_season, episode=r_episode,
-                                                           year=r_year, fps=fps,
-                                                           asked_for_release_group=video.release_group,
-                                                           asked_for_episode=episode)
-                        else:
-                            subtitle = self.subtitle_class(lang, page_link, download_link, sid, releases, _title,
-                                                           alt_title=alt_title, year=r_year, fps=fps,
-                                                           asked_for_release_group=video.release_group)
-                        logger.debug('Found subtitle %r', subtitle)
-
-                        # prime our matches so we can use the values later
-                        subtitle.get_matches(video)
-
-                        # add found subtitles
-                        subtitles.append(subtitle)
-
-                finally:
-                    soup.decompose()
-
-                # stop on last page
-                if current_page >= pages:
-                    break
-
-                # increment current page
-                params['pg'] = current_page + 1
-                logger.debug('Getting page %d', params['pg'])
+            # add found subtitles
+            subtitles.append(subtitle)
 
         return subtitles
 
     def list_subtitles(self, video, languages):
         season = episode = None
+
         if isinstance(video, Episode):
             title = video.series
             season = video.season
@@ -300,6 +284,7 @@ class TitloviProvider(Provider, ProviderSubtitleArchiveMixin):
 
         return [s for s in
                 self.query(languages, fix_inconsistent_naming(title), season=season, episode=episode, year=video.year,
+                           imdb_id=video.imdb_id,
                            video=video)]
 
     def download_subtitle(self, subtitle):
@@ -337,10 +322,12 @@ class TitloviProvider(Provider, ProviderSubtitleArchiveMixin):
         sub_to_extract = None
 
         for sub_name in subs_in_archive:
-            if not ('.cyr' in sub_name or '.cir' in sub_name):
+            _sub_name = sub_name.lower()
+
+            if not ('.cyr' in _sub_name or '.cir' in _sub_name or 'cyr)' in _sub_name):
                 sr_lat_subs.append(sub_name)
 
-            if ('.cyr' in sub_name or '.cir' in sub_name) and not '.lat' in sub_name:
+            if ('.cyr' in sub_name or '.cir' in _sub_name) and not '.lat' in _sub_name.lower():
                 sr_cyr_subs.append(sub_name)
 
         if subtitle.language == 'sr':

Contents/Libraries/Shared/subliminal_patch/subtitle.py

@@ -117,14 +117,14 @@ class Subtitle(Subtitle_):
 
         logger.info('Guessing encoding for language %s', self.language)
 
-        encodings = ['utf-8', 'utf-16']
+        encodings = ['utf-8']
 
         # add language-specific encodings
         # http://scratchpad.wikia.com/wiki/Character_Encoding_Recommendation_for_Languages
 
         if self.language.alpha3 == 'zho':
             encodings.extend(['cp936', 'gb2312', 'gbk', 'gb18030', 'hz', 'iso2022_jp_2', 'cp950', 'gb18030', 'big5',
-                              'big5hkscs'])
+                              'big5hkscs', 'utf-16'])
         elif self.language.alpha3 == 'jpn':
             encodings.extend(['shift-jis', 'cp932', 'euc_jp', 'iso2022_jp', 'iso2022_jp_1', 'iso2022_jp_2',
                               'iso2022_jp_2004', 'iso2022_jp_3', 'iso2022_jp_ext', ])
@@ -133,7 +133,7 @@ class Subtitle(Subtitle_):
 
         # arabian/farsi
         elif self.language.alpha3 in ('ara', 'fas', 'per'):
-            encodings.append('windows-1256')
+            encodings.extend(['windows-1256', 'utf-16'])
         elif self.language.alpha3 == 'heb':
             encodings.extend(['windows-1255', 'iso-8859-8'])
         elif self.language.alpha3 == 'tur':
@@ -251,8 +251,7 @@ class Subtitle(Subtitle_):
                 subs = pysubs2.SSAFile.from_string(text, fps=self.plex_media_fps)
 
             unicontent = self.pysubs2_to_unicode(subs)
-            self.content = unicontent.encode("utf-8")
-            self._guessed_encoding = "utf-8"
+            self.content = unicontent.encode(self._guessed_encoding)
         except:
             logger.exception("Couldn't convert subtitle %s to .srt format: %s", self, traceback.format_exc())
             return False
@@ -320,7 +319,8 @@ class Subtitle(Subtitle_):
         :return: string 
         """
         if not self.mods:
-            return fix_text(self.content.decode("utf-8"), **ftfy_defaults).encode(encoding="utf-8")
+            return fix_text(self.content.decode(encoding=self._guessed_encoding), **ftfy_defaults).encode(
+                encoding=self._guessed_encoding)
 
         submods = SubtitleModifications(debug=debug)
         if submods.load(content=self.text, language=self.language):
@@ -329,7 +329,7 @@ class Subtitle(Subtitle_):
             self.mods = submods.mods_used
 
             content = fix_text(self.pysubs2_to_unicode(submods.f, format=format), **ftfy_defaults)\
-                .encode(encoding="utf-8")
+                .encode(encoding=self._guessed_encoding)
             submods.f = None
             del submods
             return content

Contents/Libraries/Shared/submod_test.py

@@ -21,7 +21,7 @@ if debug:
     logging.basicConfig(level=logging.DEBUG)
 
 #sub = Subtitle(Language.fromietf("eng:forced"), mods=["common", "remove_HI", "OCR_fixes", "fix_uppercase", "shift_offset(ms=-500)", "shift_offset(ms=500)", "shift_offset(s=2,ms=800)"])
-sub = Subtitle(Language.fromietf("eng:forced"), mods=["common", "remove_HI", "OCR_fixes", "fix_uppercase", "shift_offset(ms=0,s=1)"])
+sub = Subtitle(Language.fromietf("eng"), mods=["common", "remove_HI", "OCR_fixes", "fix_uppercase", "shift_offset(ms=0,s=1)"])
 sub.content = open(fn).read()
 sub.normalize()
 content = sub.get_modified_content(debug=True)

Contents/Libraries/Shared/subscene_api/subscene.py

@@ -30,6 +30,7 @@ import enum
 import sys
 import requests
 import time
+import logging
 
 is_PY2 = sys.version_info[0] < 3
 if is_PY2:
@@ -39,8 +40,13 @@ else:
     from contextlib import suppress
     from urllib2.request import Request, urlopen
 
+from dogpile.cache.api import NO_VALUE
+from subliminal.cache import region
 from bs4 import BeautifulSoup, NavigableString
 
+
+logger = logging.getLogger(__name__)
+
 # constants
 HEADERS = {
 }
@@ -50,14 +56,21 @@ DEFAULT_USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWeb"\
                      "Kit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36"
 
 
+ENDPOINT_RE = re.compile(ur'(?uis)<form.+?action="/subtitles/(.+)">.*?<input type="text"')
+
+
+class NewEndpoint(Exception):
+    pass
+
+
 # utils
-def soup_for(url, session=None, user_agent=DEFAULT_USER_AGENT):
+def soup_for(url, data=None, session=None, user_agent=DEFAULT_USER_AGENT):
     url = re.sub("\s", "+", url)
     if not session:
         r = Request(url, data=None, headers=dict(HEADERS, **{"User-Agent": user_agent}))
         html = urlopen(r).read().decode("utf-8")
     else:
-        ret = session.get(url)
+        ret = session.post(url, data=data)
         ret.raise_for_status()
         html = ret.text
     return BeautifulSoup(html, "html.parser")
@@ -112,7 +125,7 @@ class Subtitle(object):
         subtitles = []
 
         for row in rows:
-            if row.td.a is not None:
+            if row.td.a is not None and row.td.get("class", ["lazy"])[0] != "empty":
                 subtitles.append(cls.from_row(row))
 
         return subtitles
@@ -242,29 +255,42 @@ def get_first_film(soup, section, year=None, session=None):
                 url = SITE_DOMAIN + t.div.a.get("href")
                 break
         if not url:
-            return
+            # fallback to non-year results
+            logger.info("Falling back to non-year results as year wasn't found (%s)", year)
+            url = SITE_DOMAIN + tag.findNext("ul").find("li").div.a.get("href")
 
     return Film.from_url(url, session=session)
 
 
+def find_endpoint(session, content=None):
+    endpoint = region.get("subscene_endpoint2")
+    if endpoint is NO_VALUE:
+        if not content:
+            content = session.get(SITE_DOMAIN).text
+
+        m = ENDPOINT_RE.search(content)
+        if m:
+            endpoint = m.group(1).strip()
+            logger.debug("Switching main endpoint to %s", endpoint)
+            region.set("subscene_endpoint2", endpoint)
+    return endpoint
+
+
 def search(term, release=True, session=None, year=None, limit_to=SearchTypes.Exact, throttle=0):
     # note to subscene: if you actually start to randomize the endpoint, we'll have to query your server even more
-    endpoints = ["searching", "search", "srch", "find"]
-    if release:
-        endpoints = ["release"]
 
-    soup = None
-    for endpoint in endpoints:
-        try:
-            soup = soup_for("%s/subtitles/%s?q=%s" % (SITE_DOMAIN, endpoint, term),
-                            session=session)
-        except requests.HTTPError, e:
-            if e.response.status_code == 404:
-                time.sleep(throttle)
-                # fixme: detect endpoint from html
-                continue
-            raise
-        break
+    if release:
+        endpoint = "release"
+    else:
+        endpoint = find_endpoint(session)
+        time.sleep(throttle)
+
+    if not endpoint:
+        logger.error("Couldn't find endpoint, exiting")
+        return
+
+    soup = soup_for("%s/subtitles/%s" % (SITE_DOMAIN, endpoint), data={"query": term},
+                    session=session)
 
     if soup:
         if "Subtitle search by" in str(soup):

Contents/Libraries/Shared/subzero/language.py

@@ -1,5 +1,6 @@
 # coding=utf-8
 import types
+import re
 
 from babelfish.exceptions import LanguageError
 from babelfish import Language as Language_, basestr
@@ -8,6 +9,25 @@ repl_map = {
     "dk": "da",
     "nld": "nl",
     "english": "en",
+    "alb": "sq",
+    "arm": "hy",
+    "baq": "eu",
+    "bur": "my",
+    "chi": "zh",
+    "cze": "cs",
+    "dut": "nl",
+    "fre": "fr",
+    "geo": "ka",
+    "ger": "de",
+    "gre": "el",
+    "ice": "is",
+    "mac": "mk",
+    "mao": "mi",
+    "may": "ms",
+    "per": "fa",
+    "rum": "ro",
+    "slo": "sk",
+    "tib": "bo",
 }
 
 
@@ -115,3 +135,16 @@ class Language(Language_):
             return Language(*Language_.fromietf(s).__getstate__())
 
         return Language(*Language_.fromalpha3b(s).__getstate__())
+
+
+IETF_MATCH = ".+\.([^-.]+)(?:-[A-Za-z]+)?$"
+ENDSWITH_LANGUAGECODE_RE = re.compile("\.([^-.]{2,3})(?:-[A-Za-z]{2,})?$")
+
+
+def match_ietf_language(s, ietf=False):
+    language_match = re.match(".+\.([^\.]+)$" if not ietf
+                              else IETF_MATCH, s)
+    if language_match and len(language_match.groups()) == 1:
+        language = language_match.groups()[0]
+        return language
+    return s

Contents/Libraries/Shared/subzero/lib/dict.py

@@ -107,6 +107,12 @@ class Dicked(object):
         for key, value in entries.iteritems():
             self.__dict__[key] = (Dicked(**value) if isinstance(value, dict) else value)
 
+    def has(self, key):
+        return self._entries is not None and key in self._entries
+
+    def get(self, key, default=None):
+        return self._entries.get(key, default) if self._entries else default
+
     def __repr__(self):
         return str(self)
 

Contents/Libraries/Shared/subzero/modification/dictionaries/make_data.py

@@ -36,6 +36,7 @@ SZ_FIX_DATA = {
             u" l ": u" I ",
             u"'sjust": u"'s just",
             u"'tjust": u"'t just",
+            u"\";": u"'s",
         },
         "WholeWords": {
             u"I'11": u"I'll",

Contents/Libraries/Shared/subzero/video.py

@@ -17,7 +17,8 @@ def has_external_subtitle(part_id, stored_subs, language):
 
 
 def set_existing_languages(video, video_info, external_subtitles=False, embedded_subtitles=False, known_embedded=None,
-                           stored_subs=None, languages=None, only_one=False, known_metadata_subs=None):
+                           stored_subs=None, languages=None, only_one=False, known_metadata_subs=None,
+                           match_strictness="strict"):
     logger.debug(u"Determining existing subtitles for %s", video.name)
 
     external_langs_found = set()
@@ -27,7 +28,8 @@ def set_existing_languages(video, video_info, external_subtitles=False, embedded
         external_langs_found = known_metadata_subs
 
     external_langs_found.update(set(search_external_subtitles(video.name, languages=languages,
-                                                              only_one=only_one).values()))
+                                                              only_one=only_one,
+                                                              match_strictness=match_strictness).values()))
 
     # found external subtitles should be considered?
     if external_subtitles:
@@ -52,10 +54,10 @@ def set_existing_languages(video, video_info, external_subtitles=False, embedded
             video.subtitle_languages.add(language)
 
 
-def parse_video(fn, hints, skip_hashing=False, dry_run=False, providers=None):
+def parse_video(fn, hints, skip_hashing=False, dry_run=False, providers=None, hash_from=None):
     logger.debug("Parsing video: %s, hints: %s", os.path.basename(fn), hints)
     return scan_video(fn, hints=hints, dont_use_actual_file=dry_run, providers=providers,
-                      skip_hashing=skip_hashing)
+                      skip_hashing=skip_hashing, hash_from=hash_from)
 
 
 def refine_video(video, no_refining=False, refiner_settings=None):

README.md

@@ -14,13 +14,17 @@ Check out **[the Sub-Zero Wiki](https://github.com/pannal/Sub-Zero.bundle/wiki)*
 
 ---
 
+**[Kitana is now required to have a UI](https://github.com/pannal/Kitana)**
+
+---
+
 **[The future of Sub-Zero](https://www.reddit.com/r/PleX/comments/9n9qjl/subzero_the_future/)**
 
 ---
 
 ## Helping development
 
-If you like this, buy me a beer: <br>[![Donate](https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=G9VKR2B8PMNKG) <br>or become a Patreon starting at **1 $ / month** <br><a href="https://www.patreon.com/subzero_plex" target="_blank"><img src="http://www.wenspencer.com/wp-content/uploads/2017/02/patreon-button.png" height="42" /></a> <br>or use the OpenSubtitles Sub-Zero affiliate link to become VIP <br>**10€/year, ad-free subs, 1000 subs/day, no-cache *VIP* server**<br><a href="http://v.ht/osvip" target="_blank"><img src="https://static.opensubtitles.org/gfx/logo.gif" height="50" /></a> 
+If you like this, buy me a beer: <br>[![Donate](https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=G9VKR2B8PMNKG) <br>or become a Patreon starting at **1 $ / month** <br><a href="https://www.patreon.com/subzero_plex" target="_blank"><img src="https://i0.wp.com/tablecakes.com/wp-content/uploads/2018/08/become-a-patron-button.png" height="54" /></a> <br>or use the OpenSubtitles Sub-Zero affiliate link to become VIP <br>**10€/year, ad-free subs, 1000 subs/day, no-cache *VIP* server**<br><a href="http://v.ht/osvip" target="_blank"><img src="https://static.opensubtitles.org/gfx/logo.gif" height="50" /></a> 
 
 If you register with an anti-captcha service and you decide to use [Anti-Captcha.com](http://getcaptchasolution.com/kkvviom7nh), you can use [this affiliate link](http://getcaptchasolution.com/kkvviom7nh) to help development.
 
@@ -91,28 +95,24 @@ the.vbm, mmgoodnow, Vertig0ne, thliu78, tattoomees, ostman, count_confucius, ehe
 ## Changelog
 
 
-2.6.5.3074
+2.6.5.3152
 
-subscene, addic7ed and titlovi
+subscene, addic7ed
 - either of those providers might impose a reCAPTCHA verification. In order to use those providers, please create an account at an AntiCaptcha service ([anti-captcha.com](http://getcaptchasolution.com/kkvviom7nh) or [deathbycaptcha.com](http://deathbycaptcha.com)), add funds, then supply your credentials/apikey in the configuration
 
 Changelog
-- core: cf: bypass cf 95% of the time without captchas
-- core: fix breaking line endings of certain languages (chinese, UTF-16); fixes #646
-- core: update pysubs2 to 0.2.3
-
-
-2.6.5.3062
-
-Changelog
-- core: cf: optimize
-- core: http: don't query DNS with IPs. thanks @fgump (fixes sonarr/radarr)
+- core: fix core issue possibly impacting results on OpenSubtitles in certain conditions
+- core: fix default values of opensubtitles-skip-wrong-fps, use_https; fix #676
+- core: fix for determining whether to search under certain circumstances; fixes #666
+- core: #664 fix missing language processing of multiple videos refreshed at once
+- core: #661 fix match strictness when determining preexisting external subtitles
+- providers: titlovi: New implementation of Titlovi using API (thanks @viking1304)
 
 
 [older changes](CHANGELOG.md)
 
 
-Subtitles provided by [OpenSubtitles.org](http://www.opensubtitles.org/), [Podnapisi.NET](https://www.podnapisi.net/), [TVSubtitles.net](http://www.tvsubtitles.net/), [Addic7ed.com](http://www.addic7ed.com/), [Legendas TV](http://legendas.tv/), [Napi Projekt](http://www.napiprojekt.pl/), [Shooter](http://shooter.cn/), [Titlovi](http://titlovi.com), [aRGENTeaM](http://argenteam.net), [SubScene](https://subscene.com/), [Hosszupuska](http://hosszupuskasub.com/)
+Subtitles provided by [OpenSubtitles.org](http://www.opensubtitles.org/), [Podnapisi.NET](https://www.podnapisi.net/), [TVSubtitles.net](http://www.tvsubtitles.net/), [Addic7ed.com](http://www.addic7ed.com/), [Legendas TV](http://legendas.tv/), [Napi Projekt](http://www.napiprojekt.pl/), [Shooter](http://shooter.cn/), [Titlovi](http://titlovi.com), [aRGENTeaM](http://argenteam.net), [SubScene](https://subscene.com/), [Hosszupuska](http://hosszupuskasub.com/), [Napisy24](https://napisy24.pl/)
 
 [3rd party licenses](https://github.com/pannal/Sub-Zero.bundle/tree/master/Licenses)