release 2016.04.01

[cbs] improve extraction(closes #6321 )
[generic] remove sbnation test(handled by VoxMediaIE)
2026-04-27 00:00:04 -04:00 · 2016-04-01 09:07:40 +02:00 · 2016-04-01 07:33:37 +01:00 · 2016-03-31 23:50:45 +01:00 · 2016-03-31 23:33:36 +01:00 · 2016-04-01 02:24:22 +06:00
268 changed files with 2586 additions and 955 deletions
@@ -0,0 +1,58 @@
+## Please follow the guide below
+
+- You will be asked some questions and requested to provide some information, please read them **carefully** and answer honestly
+- Put an `x` into all the boxes [ ] relevant to your *issue* (like that [x])
+- Use *Preview* tab to see how your issue will actually look like
+
+---
+
+### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *2016.04.01*. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
+- [ ] I've **verified** and **I assure** that I'm running youtube-dl **2016.04.01**
+
+### Before submitting an *issue* make sure you have:
+- [ ] At least skimmed through [README](https://github.com/rg3/youtube-dl/blob/master/README.md) and **most notably** [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
+- [ ] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones
+
+### What is the purpose of your *issue*?
+- [ ] Bug report (encountered problems with youtube-dl)
+- [ ] Site support request (request for adding support for a new site)
+- [ ] Feature request (request for a new functionality)
+- [ ] Question
+- [ ] Other
+
+---
+
+### The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your *issue*
+
+---
+
+### If the purpose of this *issue* is a *bug report*, *site support request* or you are not completely sure provide the full verbose output as follows:
+
+Add `-v` flag to **your command line** you run youtube-dl with, copy the **whole** output and insert it here. It should look similar to one below (replace it with **your** log inserted between triple ```):
+```
+$ youtube-dl -v <your command line>
+[debug] System config: []
+[debug] User config: []
+[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
+[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
+[debug] youtube-dl version 2016.04.01
+[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
+[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
+[debug] Proxy map: {}
+...
+<end of log>
+```
+
+---
+
+### If the purpose of this *issue* is a *site support request* please provide all kinds of example URLs support for which should be included (replace following example URLs by **yours**):
+- Single video: https://www.youtube.com/watch?v=BaW_jenozKc
+- Single video: https://youtu.be/BaW_jenozKc
+- Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc
+
+---
+
+### Description of your *issue*, suggested solution and other information
+
+Explanation of your *issue* in arbitrary form goes here. Please make sure the [description is worded well enough to be understood](https://github.com/rg3/youtube-dl#is-the-description-of-the-issue-itself-sufficient). Provide as much context and examples as possible.
+If work on your *issue* required an account credentials please provide them or explain how one can obtain them.
@@ -0,0 +1,58 @@
+## Please follow the guide below
+
+- You will be asked some questions and requested to provide some information, please read them **carefully** and answer honestly
+- Put an `x` into all the boxes [ ] relevant to your *issue* (like that [x])
+- Use *Preview* tab to see how your issue will actually look like
+
+---
+
+### Make sure you are using the *latest* version: run `youtube-dl --version` and ensure your version is *%(version)s*. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected.
+- [ ] I've **verified** and **I assure** that I'm running youtube-dl **%(version)s**
+
+### Before submitting an *issue* make sure you have:
+- [ ] At least skimmed through [README](https://github.com/rg3/youtube-dl/blob/master/README.md) and **most notably** [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections
+- [ ] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones
+
+### What is the purpose of your *issue*?
+- [ ] Bug report (encountered problems with youtube-dl)
+- [ ] Site support request (request for adding support for a new site)
+- [ ] Feature request (request for a new functionality)
+- [ ] Question
+- [ ] Other
+
+---
+
+### The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your *issue*
+
+---
+
+### If the purpose of this *issue* is a *bug report*, *site support request* or you are not completely sure provide the full verbose output as follows:
+
+Add `-v` flag to **your command line** you run youtube-dl with, copy the **whole** output and insert it here. It should look similar to one below (replace it with **your** log inserted between triple ```):
+```
+$ youtube-dl -v <your command line>
+[debug] System config: []
+[debug] User config: []
+[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
+[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
+[debug] youtube-dl version %(version)s
+[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
+[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
+[debug] Proxy map: {}
+...
+<end of log>
+```
+
+---
+
+### If the purpose of this *issue* is a *site support request* please provide all kinds of example URLs support for which should be included (replace following example URLs by **yours**):
+- Single video: https://www.youtube.com/watch?v=BaW_jenozKc
+- Single video: https://youtu.be/BaW_jenozKc
+- Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc
+
+---
+
+### Description of your *issue*, suggested solution and other information
+
+Explanation of your *issue* in arbitrary form goes here. Please make sure the [description is worded well enough to be understood](https://github.com/rg3/youtube-dl#is-the-description-of-the-issue-itself-sufficient). Provide as much context and examples as possible.
+If work on your *issue* required an account credentials please provide them or explain how one can obtain them.
@@ -163,3 +163,7 @@ Patrick Griffis
 Aidan Rowe
 mutantmonkey
 Ben Congdon
+Kacper Michajłow
+José Joaquín Atria
+Viťas Strádal
+Kagami Hiiragi
@@ -85,7 +85,7 @@ To run the test, simply invoke your favorite test runner, or execute a test file
 If you want to create a build of youtube-dl yourself, you'll need

 * python
-* make
+* make (both GNU make and BSD make are supported)
 * pandoc
 * zip
 * nosetests
@@ -1,7 +1,7 @@
-all: youtube-dl README.md CONTRIBUTING.md README.txt youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish supportedsites
+all: youtube-dl README.md CONTRIBUTING.md ISSUE_TEMPLATE.md README.txt youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish supportedsites

 clean:
-	rm -rf youtube-dl.1.temp.md youtube-dl.1 youtube-dl.bash-completion README.txt MANIFEST build/ dist/ .coverage cover/ youtube-dl.tar.gz youtube-dl.zsh youtube-dl.fish *.dump *.part *.info.json *.mp4 *.flv *.mp3 *.avi CONTRIBUTING.md.tmp youtube-dl youtube-dl.exe
+	rm -rf youtube-dl.1.temp.md youtube-dl.1 youtube-dl.bash-completion README.txt MANIFEST build/ dist/ .coverage cover/ youtube-dl.tar.gz youtube-dl.zsh youtube-dl.fish *.dump *.part *.info.json *.mp4 *.flv *.mp3 *.avi CONTRIBUTING.md.tmp ISSUE_TEMPLATE.md.tmp youtube-dl youtube-dl.exe
 	find . -name "*.pyc" -delete
 	find . -name "*.class" -delete

@@ -12,15 +12,7 @@ SHAREDIR ?= $(PREFIX)/share
 PYTHON ?= /usr/bin/env python

 # set SYSCONFDIR to /etc if PREFIX=/usr or PREFIX=/usr/local
-ifeq ($(PREFIX),/usr)
-	SYSCONFDIR=/etc
-else
-	ifeq ($(PREFIX),/usr/local)
-		SYSCONFDIR=/etc
-	else
-		SYSCONFDIR=$(PREFIX)/etc
-	endif
-endif
+SYSCONFDIR != if [ $(PREFIX) = /usr -o $(PREFIX) = /usr/local ]; then echo /etc; else echo $(PREFIX)/etc; fi

 install: youtube-dl youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish
 	install -d $(DESTDIR)$(BINDIR)
@@ -67,6 +59,9 @@ README.md: youtube_dl/*.py youtube_dl/*/*.py
 CONTRIBUTING.md: README.md
 	$(PYTHON) devscripts/make_contributing.py README.md CONTRIBUTING.md

+ISSUE_TEMPLATE.md:
+	$(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl.md .github/ISSUE_TEMPLATE.md
+
 supportedsites:
 	$(PYTHON) devscripts/make_supportedsites.py docs/supportedsites.md

@@ -164,6 +164,8 @@ which means you can modify it, redistribute it or use it however you like.
                                     (e.g. 50K or 4.2M)
    -R, --retries RETRIES            Number of retries (default is 10), or
                                     "infinite".
+    --fragment-retries RETRIES       Number of retries for a fragment (default
+                                     is 10), or "infinite" (DASH only)
    --buffer-size SIZE               Size of download buffer (e.g. 1024 or 16K)
                                     (default is 1024)
    --no-resize-buffer               Do not automatically adjust the buffer
@@ -376,8 +378,8 @@ which means you can modify it, redistribute it or use it however you like.
    --no-post-overwrites             Do not overwrite post-processed files; the
                                     post-processed files are overwritten by
                                     default
-    --embed-subs                     Embed subtitles in the video (only for mkv
-                                     and mp4 videos)
+    --embed-subs                     Embed subtitles in the video (only for mp4,
+                                     webm and mkv videos)
    --embed-thumbnail                Embed thumbnail in the audio as cover art
    --add-metadata                   Write metadata to the video file
    --metadata-from-title FORMAT     Parse additional metadata like song title /
@@ -598,6 +600,7 @@ Also filtering work for comparisons `=` (equals), `!=` (not equals), `^=` (begin
 - `vcodec`: Name of the video codec in use
 - `container`: Name of the container format
 - `protocol`: The protocol that will be used for the actual download, lower-case. `http`, `https`, `rtsp`, `rtmp`, `rtmpe`, `m3u8`, or `m3u8_native`
+ - `format_id`: A short description of the format

 Note that none of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by video hoster.

@@ -831,7 +834,7 @@ To run the test, simply invoke your favorite test runner, or execute a test file
 If you want to create a build of youtube-dl yourself, you'll need

 * python
-* make
+* make (both GNU make and BSD make are supported)
 * pandoc
 * zip
 * nosetests
@@ -0,0 +1,29 @@
+#!/usr/bin/env python
+from __future__ import unicode_literals
+
+import io
+import optparse
+
+
+def main():
+    parser = optparse.OptionParser(usage='%prog INFILE OUTFILE')
+    options, args = parser.parse_args()
+    if len(args) != 2:
+        parser.error('Expected an input and an output filename')
+
+    infile, outfile = args
+
+    with io.open(infile, encoding='utf-8') as inf:
+        issue_template_tmpl = inf.read()
+
+    # Get the version from youtube_dl/version.py without importing the package
+    exec(compile(open('youtube_dl/version.py').read(),
+                 'youtube_dl/version.py', 'exec'))
+
+    out = issue_template_tmpl % {'version': locals()['__version__']}
+
+    with io.open(outfile, 'w', encoding='utf-8') as outf:
+        outf.write(out)
+
+if __name__ == '__main__':
+    main()
@@ -45,9 +45,9 @@ fi
 /bin/echo -e "\n### Changing version in version.py..."
 sed -i "s/__version__ = '.*'/__version__ = '$version'/" youtube_dl/version.py

-/bin/echo -e "\n### Committing documentation and youtube_dl/version.py..."
-make README.md CONTRIBUTING.md supportedsites
-git add README.md CONTRIBUTING.md docs/supportedsites.md youtube_dl/version.py
+/bin/echo -e "\n### Committing documentation, templates and youtube_dl/version.py..."
+make README.md CONTRIBUTING.md ISSUE_TEMPLATE.md supportedsites
+git add README.md CONTRIBUTING.md .github/ISSUE_TEMPLATE.md docs/supportedsites.md youtube_dl/version.py
 git commit -m "release $version"

 /bin/echo -e "\n### Now tagging, signing and pushing..."
@@ -74,6 +74,7 @@
 - **Bigflix**
 - **Bild**: Bild.de
 - **BiliBili**
+ - **BioBioChileTV**
 - **BleacherReport**
 - **BleacherReportCMS**
 - **blinkx**
@@ -100,6 +101,7 @@
 - **CBSNews**: CBS News
 - **CBSNewsLiveVideo**: CBS News Live Videos
 - **CBSSports**
+ - **CDA**
 - **CeskaTelevize**
 - **channel9**: Channel 9
 - **Chaturbate**
@@ -116,6 +118,7 @@
 - **Clubic**
 - **Clyp**
 - **cmt.com**
+ - **CNBC**
 - **CNET**
 - **CNN**
 - **CNNArticle**
@@ -132,6 +135,7 @@
 - **CrooksAndLiars**
 - **Crunchyroll**
 - **crunchyroll:playlist**
+ - **CSNNE**
 - **CSpan**: C-SPAN
 - **CtsNews**: 華視新聞
 - **culturebox.francetvinfo.fr**
@@ -244,6 +248,7 @@
 - **GPUTechConf**
 - **Groupon**
 - **Hark**
+ - **HBO**
 - **HearThisAt**
 - **Heise**
 - **HellPorno**
@@ -344,6 +349,7 @@
 - **MiTele**: mitele.es
 - **mixcloud**
 - **MLB**
+ - **Mnet**
 - **MoeVideo**: LetitBit video services: moevideo.net, playreplay.net and videochart.net
 - **Mofosex**
 - **Mojvideo**
@@ -372,7 +378,8 @@
 - **myvideo** (Currently broken)
 - **MyVidster**
 - **n-tv.de**
- - **NationalGeographic**
+ - **natgeo**
+ - **natgeo:channel**
 - **Naver**
 - **NBA**
 - **NBC**
@@ -440,6 +447,7 @@
 - **OnionStudios**
 - **Ooyala**
 - **OoyalaExternal**
+ - **Openload**
 - **OraTV**
 - **orf:fm4**: radio FM4
 - **orf:iptv**: iptv.ORF.at
@@ -525,6 +533,7 @@
 - **RUTV**: RUTV.RU
 - **Ruutu**
 - **safari**: safaribooksonline.com online video
+ - **safari:api**
 - **safari:course**: safaribooksonline.com online courses
 - **Sandia**: Sandia National Laboratories
 - **Sapo**: SAPO Vídeos
@@ -612,12 +621,12 @@
 - **Telegraaf**
 - **TeleMB**
 - **TeleTask**
- - **TenPlay**
 - **TF1**
 - **TheIntercept**
 - **TheOnion**
 - **ThePlatform**
 - **ThePlatformFeed**
+ - **TheScene**
 - **TheSixtyOne**
 - **TheStar**
 - **ThisAmericanLife**
@@ -733,6 +742,7 @@
 - **vlive**
 - **Vodlocker**
 - **VoiceRepublic**
+ - **VoxMedia**
 - **Vporn**
 - **vpro**: npo.nl and ntr.nl
 - **VRT**
@@ -786,6 +796,7 @@
 - **youtube:channel**: YouTube.com channels
 - **youtube:favorites**: YouTube.com favourite videos, ":ytfav" for short (requires authentication)
 - **youtube:history**: Youtube watch history, ":ythistory" for short (requires authentication)
+ - **youtube:live**: YouTube.com live streams
 - **youtube:playlist**: YouTube.com playlists
 - **youtube:playlists**: YouTube.com user/channel playlists
 - **youtube:recommended**: YouTube.com recommended videos, ":ytrec" for short (requires authentication)
@@ -2,5 +2,5 @@
 universal = True

 [flake8]
-exclude = youtube_dl/extractor/__init__.py,devscripts/buildserver.py,setup.py,build,.git
+exclude = youtube_dl/extractor/__init__.py,devscripts/buildserver.py,devscripts/make_issue_template.py,setup.py,build,.git
 ignore = E402,E501,E731
@@ -222,6 +222,11 @@ class TestFormatSelection(unittest.TestCase):
        downloaded = ydl.downloaded_info_dicts[0]
        self.assertEqual(downloaded['format_id'], 'dash-video-low')

+        ydl = YDL({'format': 'bestvideo[format_id^=dash][format_id$=low]'})
+        ydl.process_ie_result(info_dict.copy())
+        downloaded = ydl.downloaded_info_dicts[0]
+        self.assertEqual(downloaded['format_id'], 'dash-video-low')
+
        formats = [
            {'format_id': 'vid-vcodec-dot', 'ext': 'mp4', 'preference': 1, 'vcodec': 'avc1.123456', 'acodec': 'none', 'url': TEST_URL},
        ]
@@ -19,6 +19,7 @@ from youtube_dl.compat import (
    compat_str,
    compat_urllib_parse_unquote,
    compat_urllib_parse_unquote_plus,
+    compat_urllib_parse_urlencode,
 )


@@ -70,6 +71,12 @@ class TestCompat(unittest.TestCase):
        self.assertEqual(compat_urllib_parse_unquote_plus('abc%20def'), 'abc def')
        self.assertEqual(compat_urllib_parse_unquote_plus('%7e/abc+def'), '~/abc def')

+    def test_compat_urllib_parse_urlencode(self):
+        self.assertEqual(compat_urllib_parse_urlencode({'abc': 'def'}), 'abc=def')
+        self.assertEqual(compat_urllib_parse_urlencode({'abc': b'def'}), 'abc=def')
+        self.assertEqual(compat_urllib_parse_urlencode({b'abc': 'def'}), 'abc=def')
+        self.assertEqual(compat_urllib_parse_urlencode({b'abc': b'def'}), 'abc=def')
+
    def test_compat_shlex_split(self):
        self.assertEqual(compat_shlex_split('-option "one two"'), ['-option', 'one two'])

@@ -1,4 +1,5 @@
 #!/usr/bin/env python
+# coding: utf-8
 from __future__ import unicode_literals

 # Allow direct execution
@@ -120,5 +121,14 @@ class TestProxy(unittest.TestCase):
        response = ydl.urlopen(req).read().decode('utf-8')
        self.assertEqual(response, 'cn: {0}'.format(url))

+    def test_proxy_with_idn(self):
+        ydl = YoutubeDL({
+            'proxy': 'localhost:{0}'.format(self.port),
+        })
+        url = 'http://中文.tw/'
+        response = ydl.urlopen(url).read().decode('utf-8')
+        # b'xn--fiq228c' is '中文'.encode('idna')
+        self.assertEqual(response, 'normal: http://xn--fiq228c.tw/')
+
 if __name__ == '__main__':
    unittest.main()
@@ -577,11 +577,11 @@ class TestUtil(unittest.TestCase):
        )
        self.assertEqual(
            escape_url('http://тест.рф/фрагмент'),
-            'http://тест.рф/%D1%84%D1%80%D0%B0%D0%B3%D0%BC%D0%B5%D0%BD%D1%82'
+            'http://xn--e1aybc.xn--p1ai/%D1%84%D1%80%D0%B0%D0%B3%D0%BC%D0%B5%D0%BD%D1%82'
        )
        self.assertEqual(
            escape_url('http://тест.рф/абв?абв=абв#абв'),
-            'http://тест.рф/%D0%B0%D0%B1%D0%B2?%D0%B0%D0%B1%D0%B2=%D0%B0%D0%B1%D0%B2#%D0%B0%D0%B1%D0%B2'
+            'http://xn--e1aybc.xn--p1ai/%D0%B0%D0%B1%D0%B2?%D0%B0%D0%B1%D0%B2=%D0%B0%D0%B1%D0%B2#%D0%B0%D0%B1%D0%B2'
        )
        self.assertEqual(escape_url('http://vimeo.com/56015672#at=0'), 'http://vimeo.com/56015672#at=0')

@@ -702,6 +702,8 @@ class TestUtil(unittest.TestCase):
        self.assertEqual(parse_count('1.000'), 1000)
        self.assertEqual(parse_count('1.1k'), 1100)
        self.assertEqual(parse_count('1.1kk'), 1100000)
+        self.assertEqual(parse_count('1.1kk '), 1100000)
+        self.assertEqual(parse_count('1.1kk views'), 1100000)

    def test_version_tuple(self):
        self.assertEqual(version_tuple('1'), (1,))
@@ -8,6 +8,6 @@ deps =
 passenv = HOME
 defaultargs = test --exclude test_download.py --exclude test_age_restriction.py
    --exclude test_subtitles.py --exclude test_write_annotations.py
-    --exclude test_youtube_lists.py
+    --exclude test_youtube_lists.py --exclude test_iqiyi_sdk_interpreter.py
 commands = nosetests --verbose {posargs:{[testenv]defaultargs}}  # --with-coverage --cover-package=youtube_dl --cover-html
                                               # test.test_download:TestDownload.test_NowVideo
@@ -39,6 +39,8 @@ from .compat import (
    compat_urllib_request_DataHandler,
 )
 from .utils import (
+    age_restricted,
+    args_to_str,
    ContentTooShortError,
    date_from_str,
    DateRange,
@@ -58,13 +60,16 @@ from .utils import (
    PagedList,
    parse_filesize,
    PerRequestProxyHandler,
-    PostProcessingError,
    platform_name,
+    PostProcessingError,
    preferredencoding,
+    prepend_extension,
    render_table,
+    replace_extension,
    SameFileError,
    sanitize_filename,
    sanitize_path,
+    sanitize_url,
    sanitized_Request,
    std_headers,
    subtitles_filename,
@@ -75,10 +80,6 @@ from .utils import (
    write_string,
    YoutubeDLCookieProcessor,
    YoutubeDLHandler,
-    prepend_extension,
-    replace_extension,
-    args_to_str,
-    age_restricted,
 )
 from .cache import Cache
 from .extractor import get_info_extractor, gen_extractors
@@ -905,7 +906,7 @@ class YoutubeDL(object):
                '*=': lambda attr, value: value in attr,
            }
            str_operator_rex = re.compile(r'''(?x)
-                \s*(?P<key>ext|acodec|vcodec|container|protocol)
+                \s*(?P<key>ext|acodec|vcodec|container|protocol|format_id)
                \s*(?P<op>%s)(?P<none_inclusive>\s*\?)?
                \s*(?P<value>[a-zA-Z0-9._-]+)
                \s*$
@@ -1229,6 +1230,7 @@ class YoutubeDL(object):
                t.get('preference'), t.get('width'), t.get('height'),
                t.get('id'), t.get('url')))
            for i, t in enumerate(thumbnails):
+                t['url'] = sanitize_url(t['url'])
                if t.get('width') and t.get('height'):
                    t['resolution'] = '%dx%d' % (t['width'], t['height'])
                if t.get('id') is None:
@@ -1263,6 +1265,8 @@ class YoutubeDL(object):
        if subtitles:
            for _, subtitle in subtitles.items():
                for subtitle_format in subtitle:
+                    if subtitle_format.get('url'):
+                        subtitle_format['url'] = sanitize_url(subtitle_format['url'])
                    if 'ext' not in subtitle_format:
                        subtitle_format['ext'] = determine_ext(subtitle_format['url']).lower()

@@ -1292,6 +1296,8 @@ class YoutubeDL(object):
            if 'url' not in format:
                raise ExtractorError('Missing "url" key in result (index %d)' % i)

+            format['url'] = sanitize_url(format['url'])
+
            if format.get('format_id') is None:
                format['format_id'] = compat_str(i)
            else:
@@ -1836,7 +1842,7 @@ class YoutubeDL(object):
        if fdict.get('language'):
            if res:
                res += ' '
-            res += '[%s]' % fdict['language']
+            res += '[%s] ' % fdict['language']
        if fdict.get('format_note') is not None:
            res += fdict['format_note'] + ' '
        if fdict.get('tbr') is not None:
@@ -144,14 +144,20 @@ def _real_main(argv=None):
        if numeric_limit is None:
            parser.error('invalid max_filesize specified')
        opts.max_filesize = numeric_limit
-    if opts.retries is not None:
-        if opts.retries in ('inf', 'infinite'):
-            opts_retries = float('inf')
+
+    def parse_retries(retries):
+        if retries in ('inf', 'infinite'):
+            parsed_retries = float('inf')
        else:
            try:
-                opts_retries = int(opts.retries)
+                parsed_retries = int(retries)
            except (TypeError, ValueError):
                parser.error('invalid retry count specified')
+        return parsed_retries
+    if opts.retries is not None:
+        opts.retries = parse_retries(opts.retries)
+    if opts.fragment_retries is not None:
+        opts.fragment_retries = parse_retries(opts.fragment_retries)
    if opts.buffersize is not None:
        numeric_buffersize = FileDownloader.parse_bytes(opts.buffersize)
        if numeric_buffersize is None:
@@ -299,7 +305,8 @@ def _real_main(argv=None):
        'force_generic_extractor': opts.force_generic_extractor,
        'ratelimit': opts.ratelimit,
        'nooverwrites': opts.nooverwrites,
-        'retries': opts_retries,
+        'retries': opts.retries,
+        'fragment_retries': opts.fragment_retries,
        'buffersize': opts.buffersize,
        'noresizebuffer': opts.noresizebuffer,
        'continuedl': opts.continue_dl,
@@ -169,6 +169,31 @@ except ImportError:  # Python 2
        string = string.replace('+', ' ')
        return compat_urllib_parse_unquote(string, encoding, errors)

+try:
+    from urllib.parse import urlencode as compat_urllib_parse_urlencode
+except ImportError:  # Python 2
+    # Python 2 will choke in urlencode on mixture of byte and unicode strings.
+    # Possible solutions are to either port it from python 3 with all
+    # the friends or manually ensure input query contains only byte strings.
+    # We will stick with latter thus recursively encoding the whole query.
+    def compat_urllib_parse_urlencode(query, doseq=0, encoding='utf-8'):
+        def encode_elem(e):
+            if isinstance(e, dict):
+                e = encode_dict(e)
+            elif isinstance(e, (list, tuple,)):
+                e = encode_list(e)
+            elif isinstance(e, compat_str):
+                e = e.encode(encoding)
+            return e
+
+        def encode_dict(d):
+            return dict((encode_elem(k), encode_elem(v)) for k, v in d.items())
+
+        def encode_list(l):
+            return [encode_elem(e) for e in l]
+
+        return compat_urllib_parse.urlencode(encode_elem(query), doseq=doseq)
+
 try:
    from urllib.request import DataHandler as compat_urllib_request_DataHandler
 except ImportError:  # Python < 3.4
@@ -588,6 +613,7 @@ __all__ = [
    'compat_urllib_parse_unquote',
    'compat_urllib_parse_unquote_plus',
    'compat_urllib_parse_unquote_to_bytes',
+    'compat_urllib_parse_urlencode',
    'compat_urllib_parse_urlparse',
    'compat_urllib_request',
    'compat_urllib_request_DataHandler',
@@ -115,6 +115,10 @@ class FileDownloader(object):
            return '%10s' % '---b/s'
        return '%10s' % ('%s/s' % format_bytes(speed))

+    @staticmethod
+    def format_retries(retries):
+        return 'inf' if retries == float('inf') else '%.0f' % retries
+
    @staticmethod
    def best_block_size(elapsed_time, bytes):
        new_min = max(bytes / 2.0, 1.0)
@@ -297,7 +301,9 @@ class FileDownloader(object):

    def report_retry(self, count, retries):
        """Report retry in case of HTTP error 5xx"""
-        self.to_screen('[download] Got server HTTP error. Retrying (attempt %d of %.0f)...' % (count, retries))
+        self.to_screen(
+            '[download] Got server HTTP error. Retrying (attempt %d of %s)...'
+            % (count, self.format_retries(retries)))

    def report_file_already_downloaded(self, file_name):
        """Report file has already been fully downloaded."""
@@ -4,6 +4,7 @@ import os
 import re

 from .fragment import FragmentFD
+from ..compat import compat_urllib_error
 from ..utils import (
    sanitize_open,
    encodeFilename,
@@ -36,20 +37,41 @@ class DashSegmentsFD(FragmentFD):

        segments_filenames = []

-        def append_url_to_file(target_url, target_filename):
-            success = ctx['dl'].download(target_filename, {'url': combine_url(base_url, target_url)})
-            if not success:
+        fragment_retries = self.params.get('fragment_retries', 0)
+
+        def append_url_to_file(target_url, tmp_filename, segment_name):
+            target_filename = '%s-%s' % (tmp_filename, segment_name)
+            count = 0
+            while count <= fragment_retries:
+                try:
+                    success = ctx['dl'].download(target_filename, {'url': combine_url(base_url, target_url)})
+                    if not success:
+                        return False
+                    down, target_sanitized = sanitize_open(target_filename, 'rb')
+                    ctx['dest_stream'].write(down.read())
+                    down.close()
+                    segments_filenames.append(target_sanitized)
+                    break
+                except (compat_urllib_error.HTTPError, ) as err:
+                    # YouTube may often return 404 HTTP error for a fragment causing the
+                    # whole download to fail. However if the same fragment is immediately
+                    # retried with the same request data this usually succeeds (1-2 attemps
+                    # is usually enough) thus allowing to download the whole file successfully.
+                    # So, we will retry all fragments that fail with 404 HTTP error for now.
+                    if err.code != 404:
+                        raise
+                    # Retry fragment
+                    count += 1
+                    if count <= fragment_retries:
+                        self.report_retry_fragment(segment_name, count, fragment_retries)
+            if count > fragment_retries:
+                self.report_error('giving up after %s fragment retries' % fragment_retries)
                return False
-            down, target_sanitized = sanitize_open(target_filename, 'rb')
-            ctx['dest_stream'].write(down.read())
-            down.close()
-            segments_filenames.append(target_sanitized)

        if initialization_url:
-            append_url_to_file(initialization_url, ctx['tmpfilename'] + '-Init')
+            append_url_to_file(initialization_url, ctx['tmpfilename'], 'Init')
        for i, segment_url in enumerate(segment_urls):
-            segment_filename = '%s-Seg%d' % (ctx['tmpfilename'], i)
-            append_url_to_file(segment_url, segment_filename)
+            append_url_to_file(segment_url, ctx['tmpfilename'], 'Seg%d' % i)

        self._finish_frag_download(ctx)

@@ -223,6 +223,12 @@ def write_metadata_tag(stream, metadata):
        write_unsigned_int(stream, FLV_TAG_HEADER_LEN + len(metadata))


+def remove_encrypted_media(media):
+    return list(filter(lambda e: 'drmAdditionalHeaderId' not in e.attrib and
+                                 'drmAdditionalHeaderSetId' not in e.attrib,
+                       media))
+
+
 def _add_ns(prop):
    return '{http://ns.adobe.com/f4m/1.0}%s' % prop

@@ -244,9 +250,7 @@ class F4mFD(FragmentFD):
            # without drmAdditionalHeaderId or drmAdditionalHeaderSetId attribute
            if 'id' not in e.attrib:
                self.report_error('Missing ID in f4m DRM')
-        media = list(filter(lambda e: 'drmAdditionalHeaderId' not in e.attrib and
-                                      'drmAdditionalHeaderSetId' not in e.attrib,
-                            media))
+        media = remove_encrypted_media(media)
        if not media:
            self.report_error('Unsupported DRM')
        return media
@@ -19,8 +19,17 @@ class HttpQuietDownloader(HttpFD):
 class FragmentFD(FileDownloader):
    """
    A base file downloader class for fragmented media (e.g. f4m/m3u8 manifests).
+
+    Available options:
+
+    fragment_retries:   Number of times to retry a fragment for HTTP error (DASH only)
    """

+    def report_retry_fragment(self, fragment_name, count, retries):
+        self.to_screen(
+            '[download] Got server HTTP error. Retrying fragment %s (attempt %d of %s)...'
+            % (fragment_name, count, self.format_retries(retries)))
+
    def _prepare_and_start_frag_download(self, ctx):
        self._prepare_frag_download(ctx)
        self._start_frag_download(ctx)
@@ -72,6 +72,7 @@ from .bet import BetIE
 from .bigflix import BigflixIE
 from .bild import BildIE
 from .bilibili import BiliBiliIE
+from .biobiochiletv import BioBioChileTVIE
 from .bleacherreport import (
    BleacherReportIE,
    BleacherReportCMSIE,
@@ -108,6 +109,7 @@ from .cbsnews import (
 )
 from .cbssports import CBSSportsIE
 from .ccc import CCCIE
+from .cda import CDAIE
 from .ceskatelevize import CeskaTelevizeIE
 from .channel9 import Channel9IE
 from .chaturbate import ChaturbateIE
@@ -125,6 +127,7 @@ from .cloudy import CloudyIE
 from .clubic import ClubicIE
 from .clyp import ClypIE
 from .cmt import CMTIE
+from .cnbc import CNBCIE
 from .cnet import CNETIE
 from .cnn import (
    CNNIE,
@@ -136,6 +139,7 @@ from .collegerama import CollegeRamaIE
 from .comedycentral import ComedyCentralIE, ComedyCentralShowsIE
 from .comcarcoff import ComCarCoffIE
 from .commonmistakes import CommonMistakesIE, UnicodeBOMIE
+from .commonprotocols import RtmpIE
 from .condenast import CondeNastIE
 from .cracked import CrackedIE
 from .crackle import CrackleIE
@@ -283,6 +287,7 @@ from .goshgay import GoshgayIE
 from .gputechconf import GPUTechConfIE
 from .groupon import GrouponIE
 from .hark import HarkIE
+from .hbo import HBOIE
 from .hearthisat import HearThisAtIE
 from .heise import HeiseIE
 from .hellporno import HellPornoIE
@@ -406,6 +411,7 @@ from .mit import TechTVMITIE, MITIE, OCWMITIE
 from .mitele import MiTeleIE
 from .mixcloud import MixcloudIE
 from .mlb import MLBIE
+from .mnet import MnetIE
 from .mpora import MporaIE
 from .moevideo import MoeVideoIE
 from .mofosex import MofosexIE
@@ -432,10 +438,14 @@ from .myspass import MySpassIE
 from .myvi import MyviIE
 from .myvideo import MyVideoIE
 from .myvidster import MyVidsterIE
-from .nationalgeographic import NationalGeographicIE
+from .nationalgeographic import (
+    NationalGeographicIE,
+    NationalGeographicChannelIE,
+)
 from .naver import NaverIE
 from .nba import NBAIE
 from .nbc import (
+    CSNNEIE,
    NBCIE,
    NBCNewsIE,
    NBCSportsIE,
@@ -531,6 +541,7 @@ from .ooyala import (
    OoyalaIE,
    OoyalaExternalIE,
 )
+from .openload import OpenloadIE
 from .ora import OraTVIE
 from .orf import (
    ORFTVthekIE,
@@ -626,6 +637,7 @@ from .ruutu import RuutuIE
 from .sandia import SandiaIE
 from .safari import (
    SafariIE,
+    SafariApiIE,
    SafariCourseIE,
 )
 from .sapo import SapoIE
@@ -728,7 +740,6 @@ from .telecinco import TelecincoIE
 from .telegraaf import TelegraafIE
 from .telemb import TeleMBIE
 from .teletask import TeleTaskIE
-from .tenplay import TenPlayIE
 from .testurl import TestURLIE
 from .tf1 import TF1IE
 from .theintercept import TheInterceptIE
@@ -737,6 +748,7 @@ from .theplatform import (
    ThePlatformIE,
    ThePlatformFeedIE,
 )
+from .thescene import TheSceneIE
 from .thesixtyone import TheSixtyOneIE
 from .thestar import TheStarIE
 from .thisamericanlife import ThisAmericanLifeIE
@@ -892,6 +904,7 @@ from .vk import (
 from .vlive import VLiveIE
 from .vodlocker import VodlockerIE
 from .voicerepublic import VoiceRepublicIE
+from .voxmedia import VoxMediaIE
 from .vporn import VpornIE
 from .vrt import VRTIE
 from .vube import VubeIE
@@ -953,7 +966,9 @@ from .youtube import (
    YoutubeChannelIE,
    YoutubeFavouritesIE,
    YoutubeHistoryIE,
+    YoutubeLiveIE,
    YoutubePlaylistIE,
+    YoutubePlaylistsIE,
    YoutubeRecommendedIE,
    YoutubeSearchDateIE,
    YoutubeSearchIE,
@@ -963,7 +978,6 @@ from .youtube import (
    YoutubeTruncatedIDIE,
    YoutubeTruncatedURLIE,
    YoutubeUserIE,
-    YoutubePlaylistsIE,
    YoutubeWatchLaterIE,
 )
 from .zapiks import ZapiksIE
@@ -12,7 +12,7 @@ from ..utils import (

 class ABCIE(InfoExtractor):
    IE_NAME = 'abc.net.au'
-    _VALID_URL = r'http://www\.abc\.net\.au/news/(?:[^/]+/){1,2}(?P<id>\d+)'
+    _VALID_URL = r'https?://www\.abc\.net\.au/news/(?:[^/]+/){1,2}(?P<id>\d+)'

    _TESTS = [{
        'url': 'http://www.abc.net.au/news/2014-11-05/australia-to-staff-ebola-treatment-centre-in-sierra-leone/5868334',
@@ -44,6 +44,7 @@ class Abc7NewsIE(InfoExtractor):
            'contentURL', webpage, 'm3u8 url', fatal=True)

        formats = self._extract_m3u8_formats(m3u8, display_id, 'mp4')
+        self._sort_formats(formats)

        title = self._og_search_title(webpage).strip()
        description = self._og_search_description(webpage).strip()
@@ -6,7 +6,7 @@ from .common import InfoExtractor
 from ..compat import (
    compat_HTTPError,
    compat_str,
-    compat_urllib_parse,
+    compat_urllib_parse_urlencode,
    compat_urllib_parse_urlparse,
 )
 from ..utils import (
@@ -16,7 +16,7 @@ from ..utils import (


 class AddAnimeIE(InfoExtractor):
-    _VALID_URL = r'http://(?:\w+\.)?add-anime\.net/(?:watch_video\.php\?(?:.*?)v=|video/)(?P<id>[\w_]+)'
+    _VALID_URL = r'https?://(?:\w+\.)?add-anime\.net/(?:watch_video\.php\?(?:.*?)v=|video/)(?P<id>[\w_]+)'
    _TESTS = [{
        'url': 'http://www.add-anime.net/watch_video.php?v=24MR3YO5SAS9',
        'md5': '72954ea10bc979ab5e2eb288b21425a0',
@@ -60,7 +60,7 @@ class AddAnimeIE(InfoExtractor):
            confirm_url = (
                parsed_url.scheme + '://' + parsed_url.netloc +
                action + '?' +
-                compat_urllib_parse.urlencode({
+                compat_urllib_parse_urlencode({
                    'jschl_vc': vc, 'jschl_answer': compat_str(av_val)}))
            self._download_webpage(
                confirm_url, video_id,
@@ -6,7 +6,7 @@ from ..utils import int_or_none


 class AftonbladetIE(InfoExtractor):
-    _VALID_URL = r'http://tv\.aftonbladet\.se/abtv/articles/(?P<id>[0-9]+)'
+    _VALID_URL = r'https?://tv\.aftonbladet\.se/abtv/articles/(?P<id>[0-9]+)'
    _TEST = {
        'url': 'http://tv.aftonbladet.se/abtv/articles/36015',
        'info_dict': {
@@ -4,7 +4,7 @@ from .common import InfoExtractor


 class AlJazeeraIE(InfoExtractor):
-    _VALID_URL = r'http://www\.aljazeera\.com/programmes/.*?/(?P<id>[^/]+)\.html'
+    _VALID_URL = r'https?://www\.aljazeera\.com/programmes/.*?/(?P<id>[^/]+)\.html'

    _TEST = {
        'url': 'http://www.aljazeera.com/programmes/the-slum/2014/08/deliverance-201482883754237240.html',
@@ -69,12 +69,14 @@ class AMPIE(InfoExtractor):

        self._sort_formats(formats)

+        timestamp = parse_iso8601(item.get('pubDate'), ' ') or parse_iso8601(item.get('dc-date'))
+
        return {
            'id': video_id,
            'title': get_media_node('title'),
            'description': get_media_node('description'),
            'thumbnails': thumbnails,
-            'timestamp': parse_iso8601(item.get('pubDate'), ' '),
+            'timestamp': timestamp,
            'duration': int_or_none(media_content[0].get('@attributes', {}).get('duration')),
            'subtitles': subtitles,
            'formats': formats,
@@ -3,10 +3,13 @@ from __future__ import unicode_literals
 import re

 from .common import InfoExtractor
-from ..compat import compat_urlparse
+from ..compat import (
+    compat_urlparse,
+    compat_str,
+)
 from ..utils import (
    determine_ext,
-    encode_dict,
+    extract_attributes,
    ExtractorError,
    sanitized_Request,
    urlencode_postdata,
@@ -34,6 +37,10 @@ class AnimeOnDemandIE(InfoExtractor):
        # Episodes without titles
        'url': 'https://www.anime-on-demand.de/anime/162',
        'only_matching': True,
+    }, {
+        # ger/jap, Dub/OmU, account required
+        'url': 'https://www.anime-on-demand.de/anime/169',
+        'only_matching': True,
    }]

    def _login(self):
@@ -44,6 +51,10 @@ class AnimeOnDemandIE(InfoExtractor):
        login_page = self._download_webpage(
            self._LOGIN_URL, None, 'Downloading login page')

+        if '>Our licensing terms allow the distribution of animes only to German-speaking countries of Europe' in login_page:
+            self.raise_geo_restricted(
+                '%s is only available in German-speaking countries of Europe' % self.IE_NAME)
+
        login_form = self._form_hidden_inputs('new_user', login_page)

        login_form.update({
@@ -59,7 +70,7 @@ class AnimeOnDemandIE(InfoExtractor):
            post_url = compat_urlparse.urljoin(self._LOGIN_URL, post_url)

        request = sanitized_Request(
-            post_url, urlencode_postdata(encode_dict(login_form)))
+            post_url, urlencode_postdata(login_form))
        request.add_header('Referer', self._LOGIN_URL)

        response = self._download_webpage(
@@ -126,33 +137,86 @@ class AnimeOnDemandIE(InfoExtractor):

            formats = []

-            playlist_url = self._search_regex(
-                r'data-playlist=(["\'])(?P<url>.+?)\1',
-                episode_html, 'data playlist', default=None, group='url')
-            if playlist_url:
-                request = sanitized_Request(
-                    compat_urlparse.urljoin(url, playlist_url),
-                    headers={
-                        'X-Requested-With': 'XMLHttpRequest',
-                        'X-CSRF-Token': csrf_token,
-                        'Referer': url,
-                        'Accept': 'application/json, text/javascript, */*; q=0.01',
-                    })
+            for input_ in re.findall(
+                    r'<input[^>]+class=["\'].*?streamstarter_html5[^>]+>', episode_html):
+                attributes = extract_attributes(input_)
+                playlist_urls = []
+                for playlist_key in ('data-playlist', 'data-otherplaylist'):
+                    playlist_url = attributes.get(playlist_key)
+                    if isinstance(playlist_url, compat_str) and re.match(
+                            r'/?[\da-zA-Z]+', playlist_url):
+                        playlist_urls.append(attributes[playlist_key])
+                if not playlist_urls:
+                    continue

-                playlist = self._download_json(
-                    request, video_id, 'Downloading playlist JSON', fatal=False)
-                if playlist:
-                    playlist = playlist['playlist'][0]
-                    title = playlist['title']
+                lang = attributes.get('data-lang')
+                lang_note = attributes.get('value')
+
+                for playlist_url in playlist_urls:
+                    kind = self._search_regex(
+                        r'videomaterialurl/\d+/([^/]+)/',
+                        playlist_url, 'media kind', default=None)
+                    format_id_list = []
+                    if lang:
+                        format_id_list.append(lang)
+                    if kind:
+                        format_id_list.append(kind)
+                    if not format_id_list:
+                        format_id_list.append(compat_str(num))
+                    format_id = '-'.join(format_id_list)
+                    format_note = ', '.join(filter(None, (kind, lang_note)))
+                    request = sanitized_Request(
+                        compat_urlparse.urljoin(url, playlist_url),
+                        headers={
+                            'X-Requested-With': 'XMLHttpRequest',
+                            'X-CSRF-Token': csrf_token,
+                            'Referer': url,
+                            'Accept': 'application/json, text/javascript, */*; q=0.01',
+                        })
+                    playlist = self._download_json(
+                        request, video_id, 'Downloading %s playlist JSON' % format_id,
+                        fatal=False)
+                    if not playlist:
+                        continue
+                    start_video = playlist.get('startvideo', 0)
+                    playlist = playlist.get('playlist')
+                    if not playlist or not isinstance(playlist, list):
+                        continue
+                    playlist = playlist[start_video]
+                    title = playlist.get('title')
+                    if not title:
+                        continue
                    description = playlist.get('description')
                    for source in playlist.get('sources', []):
                        file_ = source.get('file')
-                        if file_ and determine_ext(file_) == 'm3u8':
-                            formats = self._extract_m3u8_formats(
+                        if not file_:
+                            continue
+                        ext = determine_ext(file_)
+                        format_id_list = [lang, kind]
+                        if ext == 'm3u8':
+                            format_id_list.append('hls')
+                        elif source.get('type') == 'video/dash' or ext == 'mpd':
+                            format_id_list.append('dash')
+                        format_id = '-'.join(filter(None, format_id_list))
+                        if ext == 'm3u8':
+                            file_formats = self._extract_m3u8_formats(
                                file_, video_id, 'mp4',
-                                entry_protocol='m3u8_native', m3u8_id='hls')
+                                entry_protocol='m3u8_native', m3u8_id=format_id, fatal=False)
+                        elif source.get('type') == 'video/dash' or ext == 'mpd':
+                            continue
+                            file_formats = self._extract_mpd_formats(
+                                file_, video_id, mpd_id=format_id, fatal=False)
+                        else:
+                            continue
+                        for f in file_formats:
+                            f.update({
+                                'language': lang,
+                                'format_note': format_note,
+                            })
+                        formats.extend(file_formats)

            if formats:
+                self._sort_formats(formats)
                f = common_info.copy()
                f.update({
                    'title': title,
@@ -161,16 +225,18 @@ class AnimeOnDemandIE(InfoExtractor):
                })
                entries.append(f)

-            m = re.search(
-                r'data-dialog-header=(["\'])(?P<title>.+?)\1[^>]+href=(["\'])(?P<href>.+?)\3[^>]*>Teaser<',
-                episode_html)
-            if m:
-                f = common_info.copy()
-                f.update({
-                    'id': '%s-teaser' % f['id'],
-                    'title': m.group('title'),
-                    'url': compat_urlparse.urljoin(url, m.group('href')),
-                })
-                entries.append(f)
+            # Extract teaser only when full episode is not available
+            if not formats:
+                m = re.search(
+                    r'data-dialog-header=(["\'])(?P<title>.+?)\1[^>]+href=(["\'])(?P<href>.+?)\3[^>]*>Teaser<',
+                    episode_html)
+                if m:
+                    f = common_info.copy()
+                    f.update({
+                        'id': '%s-teaser' % f['id'],
+                        'title': m.group('title'),
+                        'url': compat_urlparse.urljoin(url, m.group('href')),
+                    })
+                    entries.append(f)

        return self.playlist_result(entries, anime_id, anime_title, anime_description)
@@ -5,7 +5,7 @@ from .common import InfoExtractor

 class AolIE(InfoExtractor):
    IE_NAME = 'on.aol.com'
-    _VALID_URL = r'(?:aol-video:|http://on\.aol\.com/video/.*-)(?P<id>[0-9]+)(?:$|\?)'
+    _VALID_URL = r'(?:aol-video:|https?://on\.aol\.com/video/.*-)(?P<id>[0-9]+)(?:$|\?)'

    _TESTS = [{
        'url': 'http://on.aol.com/video/u-s--official-warns-of-largest-ever-irs-phone-scam-518167793?icid=OnHomepageC2Wide_MustSee_Img',
@@ -25,7 +25,7 @@ class AolIE(InfoExtractor):

 class AolFeaturesIE(InfoExtractor):
    IE_NAME = 'features.aol.com'
-    _VALID_URL = r'http://features\.aol\.com/video/(?P<id>[^/?#]+)'
+    _VALID_URL = r'https?://features\.aol\.com/video/(?P<id>[^/?#]+)'

    _TESTS = [{
        'url': 'http://features.aol.com/video/behind-secret-second-careers-late-night-talk-show-hosts',
@@ -23,7 +23,7 @@ from ..utils import (


 class ArteTvIE(InfoExtractor):
-    _VALID_URL = r'http://videos\.arte\.tv/(?P<lang>fr|de|en|es)/.*-(?P<id>.*?)\.html'
+    _VALID_URL = r'https?://videos\.arte\.tv/(?P<lang>fr|de|en|es)/.*-(?P<id>.*?)\.html'
    IE_NAME = 'arte.tv'

    def _real_extract(self, url):
@@ -6,16 +6,14 @@ import hashlib
 import re

 from .common import InfoExtractor
-from ..compat import (
-    compat_str,
-    compat_urllib_parse,
-)
+from ..compat import compat_str
 from ..utils import (
-    int_or_none,
-    float_or_none,
-    sanitized_Request,
-    xpath_text,
    ExtractorError,
+    float_or_none,
+    int_or_none,
+    sanitized_Request,
+    urlencode_postdata,
+    xpath_text,
 )


@@ -86,7 +84,7 @@ class AtresPlayerIE(InfoExtractor):
        }

        request = sanitized_Request(
-            self._LOGIN_URL, compat_urllib_parse.urlencode(login_form).encode('utf-8'))
+            self._LOGIN_URL, urlencode_postdata(login_form))
        request.add_header('Content-Type', 'application/x-www-form-urlencoded')
        response = self._download_webpage(
            request, None, 'Logging in as %s' % username)
@@ -98,7 +98,7 @@ class AzubuIE(InfoExtractor):


 class AzubuLiveIE(InfoExtractor):
-    _VALID_URL = r'http://www.azubu.tv/(?P<id>[^/]+)$'
+    _VALID_URL = r'https?://www.azubu.tv/(?P<id>[^/]+)$'

    _TEST = {
        'url': 'http://www.azubu.tv/MarsTVMDLen',
@@ -120,6 +120,7 @@ class AzubuLiveIE(InfoExtractor):
        bc_info = self._download_json(req, user)
        m3u8_url = next(source['src'] for source in bc_info['sources'] if source['container'] == 'M2TS')
        formats = self._extract_m3u8_formats(m3u8_url, user, ext='mp4')
+        self._sort_formats(formats)

        return {
            'id': info['id'],
@@ -9,7 +9,7 @@ from ..utils import unescapeHTML

 class BaiduVideoIE(InfoExtractor):
    IE_DESC = '百度视频'
-    _VALID_URL = r'http://v\.baidu\.com/(?P<type>[a-z]+)/(?P<id>\d+)\.htm'
+    _VALID_URL = r'https?://v\.baidu\.com/(?P<type>[a-z]+)/(?P<id>\d+)\.htm'
    _TESTS = [{
        'url': 'http://v.baidu.com/comic/1069.htm?frp=bdbrand&q=%E4%B8%AD%E5%8D%8E%E5%B0%8F%E5%BD%93%E5%AE%B6',
        'info_dict': {
@@ -4,15 +4,13 @@ import re
 import itertools

 from .common import InfoExtractor
-from ..compat import (
-    compat_urllib_parse,
-    compat_str,
-)
+from ..compat import compat_str
 from ..utils import (
    ExtractorError,
-    int_or_none,
    float_or_none,
+    int_or_none,
    sanitized_Request,
+    urlencode_postdata,
 )


@@ -58,7 +56,7 @@ class BambuserIE(InfoExtractor):
        }

        request = sanitized_Request(
-            self._LOGIN_URL, compat_urllib_parse.urlencode(login_form).encode('utf-8'))
+            self._LOGIN_URL, urlencode_postdata(login_form))
        request.add_header('Referer', self._LOGIN_URL)
        response = self._download_webpage(
            request, None, 'Logging in as %s' % username)
@@ -688,6 +688,10 @@ class BBCIE(BBCCoUkIE):
        # custom redirection to www.bbc.com
        'url': 'http://www.bbc.co.uk/news/science-environment-33661876',
        'only_matching': True,
+    }, {
+        # single video article embedded with data-media-vpid
+        'url': 'http://www.bbc.co.uk/sport/rowing/35908187',
+        'only_matching': True,
    }]

    @classmethod
@@ -817,7 +821,7 @@ class BBCIE(BBCCoUkIE):

        # single video story (e.g. http://www.bbc.com/travel/story/20150625-sri-lankas-spicy-secret)
        programme_id = self._search_regex(
-            [r'data-video-player-vpid="(%s)"' % self._ID_REGEX,
+            [r'data-(?:video-player|media)-vpid="(%s)"' % self._ID_REGEX,
             r'<param[^>]+name="externalIdentifier"[^>]+value="(%s)"' % self._ID_REGEX,
             r'videoId\s*:\s*["\'](%s)["\']' % self._ID_REGEX],
            webpage, 'vpid', default=None)
@@ -942,7 +946,7 @@ class BBCIE(BBCCoUkIE):


 class BBCCoUkArticleIE(InfoExtractor):
-    _VALID_URL = 'http://www.bbc.co.uk/programmes/articles/(?P<id>[a-zA-Z0-9]+)'
+    _VALID_URL = r'https?://www.bbc.co.uk/programmes/articles/(?P<id>[a-zA-Z0-9]+)'
    IE_NAME = 'bbc.co.uk:article'
    IE_DESC = 'BBC articles'

@@ -34,7 +34,7 @@ class BeegIE(InfoExtractor):
        video_id = self._match_id(url)

        video = self._download_json(
-            'https://api.beeg.com/api/v5/video/%s' % video_id, video_id)
+            'https://api.beeg.com/api/v6/1738/video/%s' % video_id, video_id)

        def split(o, e):
            def cut(s, x):
@@ -50,8 +50,8 @@ class BeegIE(InfoExtractor):
            return n

        def decrypt_key(key):
-            # Reverse engineered from http://static.beeg.com/cpl/1105.js
-            a = '5ShMcIQlssOd7zChAIOlmeTZDaUxULbJRnywYaiB'
+            # Reverse engineered from http://static.beeg.com/cpl/1738.js
+            a = 'GUuyodcfS8FW8gQp4OKLMsZBcX0T7B'
            e = compat_urllib_parse_unquote(key)
            o = ''.join([
                compat_chr(compat_ord(e[n]) - compat_ord(a[n % len(a)]) % 21)
@@ -8,7 +8,7 @@ from ..utils import url_basename


 class BehindKinkIE(InfoExtractor):
-    _VALID_URL = r'http://(?:www\.)?behindkink\.com/(?P<year>[0-9]{4})/(?P<month>[0-9]{2})/(?P<day>[0-9]{2})/(?P<id>[^/#?_]+)'
+    _VALID_URL = r'https?://(?:www\.)?behindkink\.com/(?P<year>[0-9]{4})/(?P<month>[0-9]{2})/(?P<day>[0-9]{2})/(?P<id>[^/#?_]+)'
    _TEST = {
        'url': 'http://www.behindkink.com/2014/12/05/what-are-you-passionate-about-marley-blaze/',
        'md5': '507b57d8fdcd75a41a9a7bdb7989c762',
@@ -94,6 +94,7 @@ class BetIE(InfoExtractor):
            xpath_with_ns('./media:thumbnail', NS_MAP)).get('url')

        formats = self._extract_smil_formats(smil_url, display_id)
+        self._sort_formats(formats)

        return {
            'id': video_id,
@@ -14,7 +14,7 @@ from ..utils import (


 class BiliBiliIE(InfoExtractor):
-    _VALID_URL = r'http://www\.bilibili\.(?:tv|com)/video/av(?P<id>\d+)(?:/index_(?P<page_num>\d+).html)?'
+    _VALID_URL = r'https?://www\.bilibili\.(?:tv|com)/video/av(?P<id>\d+)(?:/index_(?P<page_num>\d+).html)?'

    _TESTS = [{
        'url': 'http://www.bilibili.tv/video/av1074402/',
@@ -0,0 +1,86 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..utils import remove_end
+
+
+class BioBioChileTVIE(InfoExtractor):
+    _VALID_URL = r'https?://tv\.biobiochile\.cl/notas/(?:[^/]+/)+(?P<id>[^/]+)\.shtml'
+
+    _TESTS = [{
+        'url': 'http://tv.biobiochile.cl/notas/2015/10/21/sobre-camaras-y-camarillas-parlamentarias.shtml',
+        'md5': '26f51f03cf580265defefb4518faec09',
+        'info_dict': {
+            'id': 'sobre-camaras-y-camarillas-parlamentarias',
+            'ext': 'mp4',
+            'title': 'Sobre Cámaras y camarillas parlamentarias',
+            'thumbnail': 're:^https?://.*\.jpg$',
+            'uploader': 'Fernando Atria',
+        },
+    }, {
+        # different uploader layout
+        'url': 'http://tv.biobiochile.cl/notas/2016/03/18/natalia-valdebenito-repasa-a-diputado-hasbun-paso-a-la-categoria-de-hablar-brutalidades.shtml',
+        'md5': 'edc2e6b58974c46d5b047dea3c539ff3',
+        'info_dict': {
+            'id': 'natalia-valdebenito-repasa-a-diputado-hasbun-paso-a-la-categoria-de-hablar-brutalidades',
+            'ext': 'mp4',
+            'title': 'Natalia Valdebenito repasa a diputado Hasbún: Pasó a la categoría de hablar brutalidades',
+            'thumbnail': 're:^https?://.*\.jpg$',
+            'uploader': 'Piangella Obrador',
+        },
+        'params': {
+            'skip_download': True,
+        },
+    }, {
+        'url': 'http://tv.biobiochile.cl/notas/2015/10/22/ninos-transexuales-de-quien-es-la-decision.shtml',
+        'only_matching': True,
+    }, {
+        'url': 'http://tv.biobiochile.cl/notas/2015/10/21/exclusivo-hector-pinto-formador-de-chupete-revela-version-del-ex-delantero-albo.shtml',
+        'only_matching': True,
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+
+        webpage = self._download_webpage(url, video_id)
+
+        title = remove_end(self._og_search_title(webpage), ' - BioBioChile TV')
+
+        file_url = self._search_regex(
+            r'loadFWPlayerVideo\([^,]+,\s*(["\'])(?P<url>.+?)\1',
+            webpage, 'file url', group='url')
+
+        base_url = self._search_regex(
+            r'file\s*:\s*(["\'])(?P<url>.+?)\1\s*\+\s*fileURL', webpage,
+            'base url', default='http://unlimited2-cl.digitalproserver.com/bbtv/',
+            group='url')
+
+        formats = self._extract_m3u8_formats(
+            '%s%s/playlist.m3u8' % (base_url, file_url), video_id, 'mp4',
+            entry_protocol='m3u8_native', m3u8_id='hls', fatal=False)
+        f = {
+            'url': '%s%s' % (base_url, file_url),
+            'format_id': 'http',
+            'protocol': 'http',
+            'preference': 1,
+        }
+        if formats:
+            f_copy = formats[-1].copy()
+            f_copy.update(f)
+            f = f_copy
+        formats.append(f)
+        self._sort_formats(formats)
+
+        thumbnail = self._og_search_thumbnail(webpage)
+        uploader = self._html_search_regex(
+            r'<a[^>]+href=["\']https?://busca\.biobiochile\.cl/author[^>]+>(.+?)</a>',
+            webpage, 'uploader', fatal=False)
+
+        return {
+            'id': video_id,
+            'title': title,
+            'thumbnail': thumbnail,
+            'uploader': uploader,
+            'formats': formats,
+        }
@@ -33,7 +33,7 @@ class BokeCCBaseIE(InfoExtractor):

 class BokeCCIE(BokeCCBaseIE):
    _IE_DESC = 'CC视频'
-    _VALID_URL = r'http://union\.bokecc\.com/playvideo\.bo\?(?P<query>.*)'
+    _VALID_URL = r'https?://union\.bokecc\.com/playvideo\.bo\?(?P<query>.*)'

    _TESTS = [{
        'url': 'http://union.bokecc.com/playvideo.bo?vid=E44D40C15E65EA30&uid=CD0C5D3C8614B28B',
@@ -12,7 +12,7 @@ from ..utils import (

 class BpbIE(InfoExtractor):
    IE_DESC = 'Bundeszentrale für politische Bildung'
-    _VALID_URL = r'http://www\.bpb\.de/mediathek/(?P<id>[0-9]+)/'
+    _VALID_URL = r'https?://www\.bpb\.de/mediathek/(?P<id>[0-9]+)/'

    _TEST = {
        'url': 'http://www.bpb.de/mediathek/297/joachim-gauck-zu-1989-und-die-erinnerung-an-die-ddr',
@@ -11,7 +11,7 @@ from ..utils import (


 class BreakIE(InfoExtractor):
-    _VALID_URL = r'http://(?:www\.)?break\.com/video/(?:[^/]+/)*.+-(?P<id>\d+)'
+    _VALID_URL = r'https?://(?:www\.)?break\.com/video/(?:[^/]+/)*.+-(?P<id>\d+)'
    _TESTS = [{
        'url': 'http://www.break.com/video/when-girls-act-like-guys-2468056',
        'info_dict': {
@@ -136,13 +136,16 @@ class BrightcoveLegacyIE(InfoExtractor):
        else:
            flashvars = {}

+        data_url = object_doc.attrib.get('data', '')
+        data_url_params = compat_parse_qs(compat_urllib_parse_urlparse(data_url).query)
+
        def find_param(name):
            if name in flashvars:
                return flashvars[name]
            node = find_xpath_attr(object_doc, './param', 'name', name)
            if node is not None:
                return node.attrib['value']
-            return None
+            return data_url_params.get(name)

        params = {}

@@ -294,7 +297,7 @@ class BrightcoveLegacyIE(InfoExtractor):
            'uploader': video_info.get('publisherName'),
        }

-        renditions = video_info.get('renditions')
+        renditions = video_info.get('renditions', []) + video_info.get('IOSRenditions', [])
        if renditions:
            formats = []
            for rend in renditions:
@@ -316,13 +319,23 @@ class BrightcoveLegacyIE(InfoExtractor):
                if ext is None:
                    ext = determine_ext(url)
                size = rend.get('size')
-                formats.append({
+                a_format = {
                    'url': url,
                    'ext': ext,
                    'height': rend.get('frameHeight'),
                    'width': rend.get('frameWidth'),
                    'filesize': size if size != 0 else None,
-                })
+                }
+
+                # m3u8 manifests with remote == false are media playlists
+                # Not calling _extract_m3u8_formats here to save network traffic
+                if ext == 'm3u8':
+                    a_format.update({
+                        'ext': 'mp4',
+                        'protocol': 'm3u8',
+                    })
+
+                formats.append(a_format)
            self._sort_formats(formats)
            info['formats'] = formats
        elif video_info.get('FLVFullLengthURL') is not None:
@@ -426,7 +439,7 @@ class BrightcoveNewIE(InfoExtractor):
                    </video>.*?
                    <script[^>]+
                        src=["\'](?:https?:)?//players\.brightcove\.net/
-                        (\d+)/([\da-f-]+)_([^/]+)/index(?:\.min)?\.js
+                        (\d+)/([^/]+)_([^/]+)/index(?:\.min)?\.js
                ''', webpage):
            entries.append(
                'http://players.brightcove.net/%s/%s_%s/index.html?videoId=%s'
@@ -6,7 +6,7 @@ import re

 from .common import InfoExtractor
 from ..compat import (
-    compat_urllib_parse,
+    compat_urllib_parse_urlencode,
    compat_urlparse,
 )
 from ..utils import (
@@ -16,7 +16,7 @@ from ..utils import (


 class CamdemyIE(InfoExtractor):
-    _VALID_URL = r'http://(?:www\.)?camdemy\.com/media/(?P<id>\d+)'
+    _VALID_URL = r'https?://(?:www\.)?camdemy\.com/media/(?P<id>\d+)'
    _TESTS = [{
        # single file
        'url': 'http://www.camdemy.com/media/5181/',
@@ -104,7 +104,7 @@ class CamdemyIE(InfoExtractor):


 class CamdemyFolderIE(InfoExtractor):
-    _VALID_URL = r'http://www.camdemy.com/folder/(?P<id>\d+)'
+    _VALID_URL = r'https?://www.camdemy.com/folder/(?P<id>\d+)'
    _TESTS = [{
        # links with trailing slash
        'url': 'http://www.camdemy.com/folder/450',
@@ -139,7 +139,7 @@ class CamdemyFolderIE(InfoExtractor):
        parsed_url = list(compat_urlparse.urlparse(url))
        query = dict(compat_urlparse.parse_qsl(parsed_url[4]))
        query.update({'displayMode': 'list'})
-        parsed_url[4] = compat_urllib_parse.urlencode(query)
+        parsed_url[4] = compat_urllib_parse_urlencode(query)
        final_url = compat_urlparse.urlunparse(parsed_url)

        page = self._download_webpage(final_url, folder_id)
@@ -1,21 +1,24 @@
 from __future__ import unicode_literals

-from .common import InfoExtractor
+from .theplatform import ThePlatformIE
 from ..utils import (
-    sanitized_Request,
-    smuggle_url,
+    xpath_text,
+    xpath_element,
+    int_or_none,
+    ExtractorError,
+    find_xpath_attr,
 )


-class CBSIE(InfoExtractor):
+class CBSIE(ThePlatformIE):
    _VALID_URL = r'https?://(?:www\.)?(?:cbs\.com/shows/[^/]+/(?:video|artist)|colbertlateshow\.com/(?:video|podcasts))/[^/]+/(?P<id>[^/]+)'

    _TESTS = [{
        'url': 'http://www.cbs.com/shows/garth-brooks/video/_u7W953k6la293J7EPTd9oHkSPs6Xn6_/connect-chat-feat-garth-brooks/',
        'info_dict': {
-            'id': '4JUVEwq3wUT7',
+            'id': '_u7W953k6la293J7EPTd9oHkSPs6Xn6_',
            'display_id': 'connect-chat-feat-garth-brooks',
-            'ext': 'flv',
+            'ext': 'mp4',
            'title': 'Connect Chat feat. Garth Brooks',
            'description': 'Connect with country music singer Garth Brooks, as he chats with fans on Wednesday November 27, 2013. Be sure to tune in to Garth Brooks: Live from Las Vegas, Friday November 29, at 9/8c on CBS!',
            'duration': 1495,
@@ -47,22 +50,55 @@ class CBSIE(InfoExtractor):
        'url': 'http://www.colbertlateshow.com/podcasts/dYSwjqPs_X1tvbV_P2FcPWRa_qT6akTC/in-the-bad-room-with-stephen/',
        'only_matching': True,
    }]
+    TP_RELEASE_URL_TEMPLATE = 'http://link.theplatform.com/s/dJ5BDC/%s?manifest=m3u&mbr=true'
+
+    def _parse_smil_subtitles(self, smil, namespace=None, subtitles_lang='en'):
+        closed_caption_e = find_xpath_attr(smil, self._xpath_ns('.//param', namespace), 'name', 'ClosedCaptionURL')
+        return {
+            'en': [{
+                'ext': 'ttml',
+                'url': closed_caption_e.attrib['value'],
+            }]
+        } if closed_caption_e is not None and closed_caption_e.attrib.get('value') else []

    def _real_extract(self, url):
        display_id = self._match_id(url)
-        request = sanitized_Request(url)
-        # Android UA is served with higher quality (720p) streams (see
-        # https://github.com/rg3/youtube-dl/issues/7490)
-        request.add_header('User-Agent', 'Mozilla/5.0 (Linux; Android 4.4; Nexus 5)')
-        webpage = self._download_webpage(request, display_id)
-        real_id = self._search_regex(
-            [r"video\.settings\.pid\s*=\s*'([^']+)';", r"cbsplayer\.pid\s*=\s*'([^']+)';"],
-            webpage, 'real video ID')
-        return {
-            '_type': 'url_transparent',
-            'ie_key': 'ThePlatform',
-            'url': smuggle_url(
-                'http://link.theplatform.com/s/dJ5BDC/%s?mbr=true&manifest=m3u' % real_id,
-                {'force_smil_url': True}),
+        webpage = self._download_webpage(url, display_id)
+        content_id = self._search_regex(
+            [r"video\.settings\.content_id\s*=\s*'([^']+)';", r"cbsplayer\.contentId\s*=\s*'([^']+)';"],
+            webpage, 'content id')
+        items_data = self._download_xml(
+            'http://can.cbs.com/thunder/player/videoPlayerService.php',
+            content_id, query={'partner': 'cbs', 'contentId': content_id})
+        video_data = xpath_element(items_data, './/item')
+        title = xpath_text(video_data, 'videoTitle', 'title', True)
+
+        subtitles = {}
+        formats = []
+        for item in items_data.findall('.//item'):
+            pid = xpath_text(item, 'pid')
+            if not pid:
+                continue
+            try:
+                tp_formats, tp_subtitles = self._extract_theplatform_smil(
+                    self.TP_RELEASE_URL_TEMPLATE % pid, content_id, 'Downloading %s SMIL data' % pid)
+            except ExtractorError:
+                continue
+            formats.extend(tp_formats)
+            subtitles = self._merge_subtitles(subtitles, tp_subtitles)
+        self._sort_formats(formats)
+
+        info = self.get_metadata('dJ5BDC/media/guid/2198311517/%s' % content_id, content_id)
+        info.update({
+            'id': content_id,
            'display_id': display_id,
-        }
+            'title': title,
+            'series': xpath_text(video_data, 'seriesTitle'),
+            'season_number': int_or_none(xpath_text(video_data, 'seasonNumber')),
+            'episode_number': int_or_none(xpath_text(video_data, 'episodeNumber')),
+            'duration': int_or_none(xpath_text(video_data, 'videoLength'), 1000),
+            'thumbnail': xpath_text(video_data, 'previewImageURL'),
+            'formats': formats,
+            'subtitles': subtitles,
+        })
+        return info
@@ -11,7 +11,7 @@ from ..utils import (

 class CBSNewsIE(ThePlatformIE):
    IE_DESC = 'CBS News'
-    _VALID_URL = r'http://(?:www\.)?cbsnews\.com/(?:news|videos)/(?P<id>[\da-z_-]+)'
+    _VALID_URL = r'https?://(?:www\.)?cbsnews\.com/(?:news|videos)/(?P<id>[\da-z_-]+)'

    _TESTS = [
        {
@@ -96,7 +96,7 @@ class CBSNewsIE(ThePlatformIE):

 class CBSNewsLiveVideoIE(InfoExtractor):
    IE_DESC = 'CBS News Live Videos'
-    _VALID_URL = r'http://(?:www\.)?cbsnews\.com/live/video/(?P<id>[\da-z_-]+)'
+    _VALID_URL = r'https?://(?:www\.)?cbsnews\.com/live/video/(?P<id>[\da-z_-]+)'

    _TEST = {
        'url': 'http://www.cbsnews.com/live/video/clinton-sanders-prepare-to-face-off-in-nh/',
@@ -122,6 +122,7 @@ class CBSNewsLiveVideoIE(InfoExtractor):
            for entry in f4m_formats:
                # URLs without the extra param induce an 404 error
                entry.update({'extra_param_to_segment_url': hdcore_sign})
+        self._sort_formats(f4m_formats)

        return {
            'id': video_id,
@@ -6,7 +6,7 @@ from .common import InfoExtractor


 class CBSSportsIE(InfoExtractor):
-    _VALID_URL = r'http://www\.cbssports\.com/video/player/(?P<section>[^/]+)/(?P<id>[^/]+)'
+    _VALID_URL = r'https?://www\.cbssports\.com/video/player/(?P<section>[^/]+)/(?P<id>[^/]+)'

    _TEST = {
        'url': 'http://www.cbssports.com/video/player/tennis/318462531970/0/us-open-flashbacks-1990s',
@@ -0,0 +1,96 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import re
+
+from .common import InfoExtractor
+from ..utils import (
+    decode_packed_codes,
+    ExtractorError,
+    parse_duration
+)
+
+
+class CDAIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:(?:www\.)?cda\.pl/video|ebd\.cda\.pl/[0-9]+x[0-9]+)/(?P<id>[0-9a-z]+)'
+    _TESTS = [{
+        'url': 'http://www.cda.pl/video/5749950c',
+        'md5': '6f844bf51b15f31fae165365707ae970',
+        'info_dict': {
+            'id': '5749950c',
+            'ext': 'mp4',
+            'height': 720,
+            'title': 'Oto dlaczego przed zakrętem należy zwolnić.',
+            'duration': 39
+        }
+    }, {
+        'url': 'http://www.cda.pl/video/57413289',
+        'md5': 'a88828770a8310fc00be6c95faf7f4d5',
+        'info_dict': {
+            'id': '57413289',
+            'ext': 'mp4',
+            'title': 'Lądowanie na lotnisku na Maderze',
+            'duration': 137
+        }
+    }, {
+        'url': 'http://ebd.cda.pl/0x0/5749950c',
+        'only_matching': True,
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage('http://ebd.cda.pl/0x0/' + video_id, video_id)
+
+        if 'Ten film jest dostępny dla użytkowników premium' in webpage:
+            raise ExtractorError('This video is only available for premium users.', expected=True)
+
+        title = self._html_search_regex(r'<title>(.+?)</title>', webpage, 'title')
+
+        formats = []
+
+        info_dict = {
+            'id': video_id,
+            'title': title,
+            'formats': formats,
+            'duration': None,
+        }
+
+        def extract_format(page, version):
+            unpacked = decode_packed_codes(page)
+            format_url = self._search_regex(
+                r"url:\\'(.+?)\\'", unpacked, '%s url' % version, fatal=False)
+            if not format_url:
+                return
+            f = {
+                'url': format_url,
+            }
+            m = re.search(
+                r'<a[^>]+data-quality="(?P<format_id>[^"]+)"[^>]+href="[^"]+"[^>]+class="[^"]*quality-btn-active[^"]*">(?P<height>[0-9]+)p',
+                page)
+            if m:
+                f.update({
+                    'format_id': m.group('format_id'),
+                    'height': int(m.group('height')),
+                })
+            info_dict['formats'].append(f)
+            if not info_dict['duration']:
+                info_dict['duration'] = parse_duration(self._search_regex(
+                    r"duration:\\'(.+?)\\'", unpacked, 'duration', fatal=False))
+
+        extract_format(webpage, 'default')
+
+        for href, resolution in re.findall(
+                r'<a[^>]+data-quality="[^"]+"[^>]+href="([^"]+)"[^>]+class="quality-btn"[^>]*>([0-9]+p)',
+                webpage):
+            webpage = self._download_webpage(
+                href, video_id, 'Downloading %s version information' % resolution, fatal=False)
+            if not webpage:
+                # Manually report warning because empty page is returned when
+                # invalid version is requested.
+                self.report_warning('Unable to download %s version information' % resolution)
+                continue
+            extract_format(webpage, resolution)
+
+        self._sort_formats(formats)
+
+        return info_dict
@@ -5,7 +5,6 @@ import re

 from .common import InfoExtractor
 from ..compat import (
-    compat_urllib_parse,
    compat_urllib_parse_unquote,
    compat_urllib_parse_urlparse,
 )
@@ -13,6 +12,7 @@ from ..utils import (
    ExtractorError,
    float_or_none,
    sanitized_Request,
+    urlencode_postdata,
 )


@@ -102,7 +102,7 @@ class CeskaTelevizeIE(InfoExtractor):

        req = sanitized_Request(
            'http://www.ceskatelevize.cz/ivysilani/ajax/get-client-playlist',
-            data=compat_urllib_parse.urlencode(data))
+            data=urlencode_postdata(data))

        req.add_header('Content-type', 'application/x-www-form-urlencoded')
        req.add_header('x-addr', '127.0.0.1')
@@ -129,7 +129,8 @@ class CeskaTelevizeIE(InfoExtractor):
            formats = []
            for format_id, stream_url in item['streamUrls'].items():
                formats.extend(self._extract_m3u8_formats(
-                    stream_url, playlist_id, 'mp4', entry_protocol='m3u8_native'))
+                    stream_url, playlist_id, 'mp4',
+                    entry_protocol='m3u8_native', fatal=False))
            self._sort_formats(formats)

            item_id = item.get('id') or item['assetId']
@@ -48,6 +48,7 @@ class ChaturbateIE(InfoExtractor):
            raise ExtractorError('Unable to find stream URL')

        formats = self._extract_m3u8_formats(m3u8_url, video_id, ext='mp4')
+        self._sort_formats(formats)

        return {
            'id': video_id,
@@ -19,7 +19,7 @@ def _decode(s):
 class CliphunterIE(InfoExtractor):
    IE_NAME = 'cliphunter'

-    _VALID_URL = r'''(?x)http://(?:www\.)?cliphunter\.com/w/
+    _VALID_URL = r'''(?x)https?://(?:www\.)?cliphunter\.com/w/
        (?P<id>[0-9]+)/
        (?P<seo>.+?)(?:$|[#\?])
    '''
@@ -8,7 +8,7 @@ from ..utils import (


 class ClipsyndicateIE(InfoExtractor):
-    _VALID_URL = r'http://(?:chic|www)\.clipsyndicate\.com/video/play(list/\d+)?/(?P<id>\d+)'
+    _VALID_URL = r'https?://(?:chic|www)\.clipsyndicate\.com/video/play(list/\d+)?/(?P<id>\d+)'

    _TESTS = [{
        'url': 'http://www.clipsyndicate.com/video/play/4629301/brick_briscoe',
@@ -6,7 +6,7 @@ import re
 from .common import InfoExtractor
 from ..compat import (
    compat_parse_qs,
-    compat_urllib_parse,
+    compat_urllib_parse_urlencode,
    compat_HTTPError,
 )
 from ..utils import (
@@ -64,7 +64,7 @@ class CloudyIE(InfoExtractor):
                'errorUrl': error_url,
            })

-        data_url = self._API_URL % (video_host, compat_urllib_parse.urlencode(form))
+        data_url = self._API_URL % (video_host, compat_urllib_parse_urlencode(form))
        player_data = self._download_webpage(
            data_url, video_id, 'Downloading player data')
        data = compat_parse_qs(player_data)
@@ -12,7 +12,7 @@ from ..utils import (


 class ClubicIE(InfoExtractor):
-    _VALID_URL = r'http://(?:www\.)?clubic\.com/video/(?:[^/]+/)*video.*-(?P<id>[0-9]+)\.html'
+    _VALID_URL = r'https?://(?:www\.)?clubic\.com/video/(?:[^/]+/)*video.*-(?P<id>[0-9]+)\.html'

    _TESTS = [{
        'url': 'http://www.clubic.com/video/clubic-week/video-clubic-week-2-0-le-fbi-se-lance-dans-la-photo-d-identite-448474.html',
@@ -0,0 +1,33 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..utils import smuggle_url
+
+
+class CNBCIE(InfoExtractor):
+    _VALID_URL = r'https?://video\.cnbc\.com/gallery/\?video=(?P<id>[0-9]+)'
+    _TEST = {
+        'url': 'http://video.cnbc.com/gallery/?video=3000503714',
+        'info_dict': {
+            'id': '3000503714',
+            'ext': 'mp4',
+            'title': 'Fighting zombies is big business',
+            'description': 'md5:0c100d8e1a7947bd2feec9a5550e519e',
+        },
+        'params': {
+            # m3u8 download
+            'skip_download': True,
+        },
+    }
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        return {
+            '_type': 'url_transparent',
+            'ie_key': 'ThePlatform',
+            'url': smuggle_url(
+                'http://link.theplatform.com/s/gZWlPC/media/guid/2408950221/%s?mbr=true&manifest=m3u' % video_id,
+                {'force_smil_url': True}),
+            'id': video_id,
+        }
@@ -11,7 +11,7 @@ from ..utils import (


 class ComCarCoffIE(InfoExtractor):
-    _VALID_URL = r'http://(?:www\.)?comediansincarsgettingcoffee\.com/(?P<id>[a-z0-9\-]*)'
+    _VALID_URL = r'https?://(?:www\.)?comediansincarsgettingcoffee\.com/(?P<id>[a-z0-9\-]*)'
    _TESTS = [{
        'url': 'http://comediansincarsgettingcoffee.com/miranda-sings-happy-thanksgiving-miranda/',
        'info_dict': {
@@ -41,7 +41,13 @@ class ComCarCoffIE(InfoExtractor):

        display_id = full_data['activeVideo']['video']
        video_data = full_data.get('videos', {}).get(display_id) or full_data['singleshots'][display_id]
+
        video_id = compat_str(video_data['mediaId'])
+        title = video_data['title']
+        formats = self._extract_m3u8_formats(
+            video_data['mediaUrl'], video_id, 'mp4')
+        self._sort_formats(formats)
+
        thumbnails = [{
            'url': video_data['images']['thumb'],
        }, {
@@ -54,15 +60,14 @@ class ComCarCoffIE(InfoExtractor):
            video_data.get('duration'))

        return {
-            '_type': 'url_transparent',
-            'url': 'crackle:%s' % video_id,
            'id': video_id,
            'display_id': display_id,
-            'title': video_data['title'],
+            'title': title,
            'description': video_data.get('description'),
            'timestamp': timestamp,
            'duration': duration,
            'thumbnails': thumbnails,
+            'formats': formats,
            'season_number': int_or_none(video_data.get('season')),
            'episode_number': int_or_none(video_data.get('episode')),
            'webpage_url': 'http://comediansincarsgettingcoffee.com/%s' % (video_data.get('urlSlug', video_data.get('slug'))),
@@ -5,7 +5,7 @@ import re
 from .mtv import MTVServicesInfoExtractor
 from ..compat import (
    compat_str,
-    compat_urllib_parse,
+    compat_urllib_parse_urlencode,
 )
 from ..utils import (
    ExtractorError,
@@ -201,7 +201,7 @@ class ComedyCentralShowsIE(MTVServicesInfoExtractor):
        # Correct cc.com in uri
        uri = re.sub(r'(episode:[^.]+)(\.cc)?\.com', r'\1.com', uri)

-        index_url = 'http://%s.cc.com/feeds/mrss?%s' % (show_name, compat_urllib_parse.urlencode({'uri': uri}))
+        index_url = 'http://%s.cc.com/feeds/mrss?%s' % (show_name, compat_urllib_parse_urlencode({'uri': uri}))
        idoc = self._download_xml(
            index_url, epTitle,
            'Downloading show index', 'Unable to download episode index')
@@ -21,9 +21,11 @@ from ..compat import (
    compat_os_name,
    compat_str,
    compat_urllib_error,
-    compat_urllib_parse,
+    compat_urllib_parse_urlencode,
+    compat_urllib_request,
    compat_urlparse,
 )
+from ..downloader.f4m import remove_encrypted_media
 from ..utils import (
    NO_DEFAULT,
    age_restricted,
@@ -48,6 +50,7 @@ from ..utils import (
    determine_protocol,
    parse_duration,
    mimetype2ext,
+    update_Request,
    update_url_query,
 )

@@ -346,7 +349,7 @@ class InfoExtractor(object):
    def IE_NAME(self):
        return compat_str(type(self).__name__[:-2])

-    def _request_webpage(self, url_or_request, video_id, note=None, errnote=None, fatal=True, data=None, headers=None, query=None):
+    def _request_webpage(self, url_or_request, video_id, note=None, errnote=None, fatal=True, data=None, headers={}, query={}):
        """ Returns the response handle """
        if note is None:
            self.report_download_webpage(video_id)
@@ -356,11 +359,14 @@ class InfoExtractor(object):
            else:
                self.to_screen('%s: %s' % (video_id, note))
        # data, headers and query params will be ignored for `Request` objects
-        if isinstance(url_or_request, compat_str):
+        if isinstance(url_or_request, compat_urllib_request.Request):
+            url_or_request = update_Request(
+                url_or_request, data=data, headers=headers, query=query)
+        else:
            if query:
                url_or_request = update_url_query(url_or_request, query)
            if data or headers:
-                url_or_request = sanitized_Request(url_or_request, data, headers or {})
+                url_or_request = sanitized_Request(url_or_request, data, headers)
        try:
            return self._downloader.urlopen(url_or_request)
        except (compat_urllib_error.URLError, compat_http_client.HTTPException, socket.error) as err:
@@ -376,7 +382,7 @@ class InfoExtractor(object):
                self._downloader.report_warning(errmsg)
                return False

-    def _download_webpage_handle(self, url_or_request, video_id, note=None, errnote=None, fatal=True, encoding=None, data=None, headers=None, query=None):
+    def _download_webpage_handle(self, url_or_request, video_id, note=None, errnote=None, fatal=True, encoding=None, data=None, headers={}, query={}):
        """ Returns a tuple (page content as string, URL handle) """
        # Strip hashes from the URL (#1038)
        if isinstance(url_or_request, (compat_str, str)):
@@ -469,7 +475,7 @@ class InfoExtractor(object):

        return content

-    def _download_webpage(self, url_or_request, video_id, note=None, errnote=None, fatal=True, tries=1, timeout=5, encoding=None, data=None, headers=None, query=None):
+    def _download_webpage(self, url_or_request, video_id, note=None, errnote=None, fatal=True, tries=1, timeout=5, encoding=None, data=None, headers={}, query={}):
        """ Returns the data of the page as a string """
        success = False
        try_count = 0
@@ -490,7 +496,7 @@ class InfoExtractor(object):

    def _download_xml(self, url_or_request, video_id,
                      note='Downloading XML', errnote='Unable to download XML',
-                      transform_source=None, fatal=True, encoding=None, data=None, headers=None, query=None):
+                      transform_source=None, fatal=True, encoding=None, data=None, headers={}, query={}):
        """Return the xml as an xml.etree.ElementTree.Element"""
        xml_string = self._download_webpage(
            url_or_request, video_id, note, errnote, fatal=fatal, encoding=encoding, data=data, headers=headers, query=query)
@@ -504,7 +510,7 @@ class InfoExtractor(object):
                       note='Downloading JSON metadata',
                       errnote='Unable to download JSON metadata',
                       transform_source=None,
-                       fatal=True, encoding=None, data=None, headers=None, query=None):
+                       fatal=True, encoding=None, data=None, headers={}, query={}):
        json_string = self._download_webpage(
            url_or_request, video_id, note, errnote, fatal=fatal,
            encoding=encoding, data=data, headers=headers, query=query)
@@ -989,6 +995,11 @@ class InfoExtractor(object):
        if not media_nodes:
            manifest_version = '2.0'
            media_nodes = manifest.findall('{http://ns.adobe.com/f4m/2.0}media')
+        # Remove unsupported DRM protected media from final formats
+        # rendition (see https://github.com/rg3/youtube-dl/issues/8573).
+        media_nodes = remove_encrypted_media(media_nodes)
+        if not media_nodes:
+            return formats
        base_url = xpath_text(
            manifest, ['{http://ns.adobe.com/f4m/1.0}baseURL', '{http://ns.adobe.com/f4m/2.0}baseURL'],
            'base URL', default=None)
@@ -1021,8 +1032,6 @@ class InfoExtractor(object):
                'height': int_or_none(media_el.attrib.get('height')),
                'preference': preference,
            })
-        self._sort_formats(formats)
-
        return formats

    def _extract_m3u8_formats(self, m3u8_url, video_id, ext=None,
@@ -1143,7 +1152,6 @@ class InfoExtractor(object):
                    last_media = None
                formats.append(f)
                last_info = {}
-        self._sort_formats(formats)
        return formats

    @staticmethod
@@ -1300,7 +1308,7 @@ class InfoExtractor(object):
                        'plugin': 'flowplayer-3.2.0.1',
                    }
                f4m_url += '&' if '?' in f4m_url else '?'
-                f4m_url += compat_urllib_parse.urlencode(f4m_params)
+                f4m_url += compat_urllib_parse_urlencode(f4m_params)
                formats.extend(self._extract_f4m_formats(f4m_url, video_id, f4m_id='hds', fatal=False))
                continue

@@ -1317,8 +1325,6 @@ class InfoExtractor(object):
                })
                continue

-        self._sort_formats(formats)
-
        return formats

    def _parse_smil_subtitles(self, smil, namespace=None, subtitles_lang='en'):
@@ -1536,7 +1542,6 @@ class InfoExtractor(object):
                            existing_format.update(f)
                    else:
                        self.report_warning('Unknown MIME type %s in DASH manifest' % mime_type)
-        self._sort_formats(formats)
        return formats

    def _live_title(self, name):
@@ -0,0 +1,36 @@
+from __future__ import unicode_literals
+
+import os
+
+from .common import InfoExtractor
+from ..compat import (
+    compat_urllib_parse_unquote,
+    compat_urlparse,
+)
+from ..utils import url_basename
+
+
+class RtmpIE(InfoExtractor):
+    IE_DESC = False  # Do not list
+    _VALID_URL = r'(?i)rtmp[est]?://.+'
+
+    _TESTS = [{
+        'url': 'rtmp://cp44293.edgefcs.net/ondemand?auth=daEcTdydfdqcsb8cZcDbAaCbhamacbbawaS-bw7dBb-bWG-GqpGFqCpNCnGoyL&aifp=v001&slist=public/unsecure/audio/2c97899446428e4301471a8cb72b4b97--audio--pmg-20110908-0900a_flv_aac_med_int.mp4',
+        'only_matching': True,
+    }, {
+        'url': 'rtmp://edge.live.hitbox.tv/live/dimak',
+        'only_matching': True,
+    }]
+
+    def _real_extract(self, url):
+        video_id = compat_urllib_parse_unquote(os.path.splitext(url.rstrip('/').split('/')[-1])[0])
+        title = compat_urllib_parse_unquote(os.path.splitext(url_basename(url))[0])
+        return {
+            'id': video_id,
+            'title': title,
+            'formats': [{
+                'url': url,
+                'ext': 'flv',
+                'format_id': compat_urlparse.urlparse(url).scheme,
+            }],
+        }
@@ -5,7 +5,7 @@ import re

 from .common import InfoExtractor
 from ..compat import (
-    compat_urllib_parse,
+    compat_urllib_parse_urlencode,
    compat_urllib_parse_urlparse,
    compat_urlparse,
 )
@@ -45,7 +45,7 @@ class CondeNastIE(InfoExtractor):
        'wmagazine': 'W Magazine',
    }

-    _VALID_URL = r'http://(?:video|www|player)\.(?P<site>%s)\.com/(?P<type>watch|series|video|embed(?:js)?)/(?P<id>[^/?#]+)' % '|'.join(_SITES.keys())
+    _VALID_URL = r'https?://(?:video|www|player)\.(?P<site>%s)\.com/(?P<type>watch|series|video|embed(?:js)?)/(?P<id>[^/?#]+)' % '|'.join(_SITES.keys())
    IE_DESC = 'Condé Nast media group: %s' % ', '.join(sorted(_SITES.values()))

    EMBED_URL = r'(?:https?:)?//player\.(?P<site>%s)\.com/(?P<type>embed(?:js)?)/.+?' % '|'.join(_SITES.keys())
@@ -97,7 +97,7 @@ class CondeNastIE(InfoExtractor):
        video_id = self._search_regex(r'videoId: [\'"](.+?)[\'"]', params, 'video id')
        player_id = self._search_regex(r'playerId: [\'"](.+?)[\'"]', params, 'player id')
        target = self._search_regex(r'target: [\'"](.+?)[\'"]', params, 'target')
-        data = compat_urllib_parse.urlencode({'videoId': video_id,
+        data = compat_urllib_parse_urlencode({'videoId': video_id,
                                              'playerId': player_id,
                                              'target': target,
                                              })
@@ -11,8 +11,8 @@ from math import pow, sqrt, floor
 from .common import InfoExtractor
 from ..compat import (
    compat_etree_fromstring,
-    compat_urllib_parse,
    compat_urllib_parse_unquote,
+    compat_urllib_parse_urlencode,
    compat_urllib_request,
    compat_urlparse,
 )
@@ -78,7 +78,7 @@ class CrunchyrollBaseIE(InfoExtractor):
        # See https://github.com/rg3/youtube-dl/issues/7202.
        qs['skip_wall'] = ['1']
        return compat_urlparse.urlunparse(
-            parsed_url._replace(query=compat_urllib_parse.urlencode(qs, True)))
+            parsed_url._replace(query=compat_urllib_parse_urlencode(qs, True)))


 class CrunchyrollIE(CrunchyrollBaseIE):
@@ -308,7 +308,7 @@ Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text

        playerdata_url = compat_urllib_parse_unquote(self._html_search_regex(r'"config_url":"([^"]+)', webpage, 'playerdata_url'))
        playerdata_req = sanitized_Request(playerdata_url)
-        playerdata_req.data = compat_urllib_parse.urlencode({'current_page': webpage_url})
+        playerdata_req.data = urlencode_postdata({'current_page': webpage_url})
        playerdata_req.add_header('Content-Type', 'application/x-www-form-urlencoded')
        playerdata = self._download_webpage(playerdata_req, video_id, note='Downloading media info')

@@ -322,7 +322,7 @@ Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
            streamdata_req = sanitized_Request(
                'http://www.crunchyroll.com/xml/?req=RpcApiVideoPlayer_GetStandardConfig&media_id=%s&video_format=%s&video_quality=%s'
                % (stream_id, stream_format, stream_quality),
-                compat_urllib_parse.urlencode({'current_page': url}).encode('utf-8'))
+                compat_urllib_parse_urlencode({'current_page': url}).encode('utf-8'))
            streamdata_req.add_header('Content-Type', 'application/x-www-form-urlencoded')
            streamdata = self._download_xml(
                streamdata_req, video_id,
@@ -15,7 +15,7 @@ from .senateisvp import SenateISVPIE


 class CSpanIE(InfoExtractor):
-    _VALID_URL = r'http://(?:www\.)?c-span\.org/video/\?(?P<id>[0-9a-f]+)'
+    _VALID_URL = r'https?://(?:www\.)?c-span\.org/video/\?(?P<id>[0-9a-f]+)'
    IE_DESC = 'C-SPAN'
    _TESTS = [{
        'url': 'http://www.c-span.org/video/?313572-1/HolderonV',
@@ -8,7 +8,7 @@ from ..utils import parse_iso8601, ExtractorError
 class CtsNewsIE(InfoExtractor):
    IE_DESC = '華視新聞'
    # https connection failed (Connection reset)
-    _VALID_URL = r'http://news\.cts\.com\.tw/[a-z]+/[a-z]+/\d+/(?P<id>\d+)\.html'
+    _VALID_URL = r'https?://news\.cts\.com\.tw/[a-z]+/[a-z]+/\d+/(?P<id>\d+)\.html'
    _TESTS = [{
        'url': 'http://news.cts.com.tw/cts/international/201501/201501291578109.html',
        'md5': 'a9875cb790252b08431186d741beaabe',
@@ -57,6 +57,7 @@ class CWTVIE(InfoExtractor):

        formats = self._extract_m3u8_formats(
            video_data['videos']['variantplaylist']['uri'], video_id, 'mp4')
+        self._sort_formats(formats)

        thumbnails = [{
            'url': image['uri'],
@@ -8,8 +8,8 @@ import itertools
 from .common import InfoExtractor
 from ..compat import (
    compat_parse_qs,
-    compat_urllib_parse,
    compat_urllib_parse_unquote,
+    compat_urllib_parse_urlencode,
    compat_urlparse,
 )
 from ..utils import (
@@ -70,7 +70,7 @@ class DaumIE(InfoExtractor):

    def _real_extract(self, url):
        video_id = compat_urllib_parse_unquote(self._match_id(url))
-        query = compat_urllib_parse.urlencode({'vid': video_id})
+        query = compat_urllib_parse_urlencode({'vid': video_id})
        movie_data = self._download_json(
            'http://videofarm.daum.net/controller/api/closed/v1_2/IntegratedMovieData.json?' + query,
            video_id, 'Downloading video formats info')
@@ -86,7 +86,7 @@ class DaumIE(InfoExtractor):
        formats = []
        for format_el in movie_data['output_list']['output_list']:
            profile = format_el['profile']
-            format_query = compat_urllib_parse.urlencode({
+            format_query = compat_urllib_parse_urlencode({
                'vid': video_id,
                'profile': profile,
            })
@@ -6,7 +6,7 @@ import base64

 from .common import InfoExtractor
 from ..compat import (
-    compat_urllib_parse,
+    compat_urllib_parse_urlencode,
    compat_str,
 )
 from ..utils import (
@@ -15,6 +15,7 @@ from ..utils import (
    sanitized_Request,
    smuggle_url,
    unsmuggle_url,
+    urlencode_postdata,
 )


@@ -106,7 +107,7 @@ class DCNVideoIE(DCNBaseIE):

        webpage = self._download_webpage(
            'http://admin.mangomolo.com/analytics/index.php/customers/embed/video?' +
-            compat_urllib_parse.urlencode({
+            compat_urllib_parse_urlencode({
                'id': video_data['id'],
                'user_id': video_data['user_id'],
                'signature': video_data['signature'],
@@ -133,7 +134,7 @@ class DCNLiveIE(DCNBaseIE):

        webpage = self._download_webpage(
            'http://admin.mangomolo.com/analytics/index.php/customers/embed/index?' +
-            compat_urllib_parse.urlencode({
+            compat_urllib_parse_urlencode({
                'id': base64.b64encode(channel_data['user_id'].encode()).decode(),
                'channelid': base64.b64encode(channel_data['id'].encode()).decode(),
                'signature': channel_data['signature'],
@@ -174,7 +175,7 @@ class DCNSeasonIE(InfoExtractor):
        data['show_id'] = show_id
        request = sanitized_Request(
            'http://admin.mangomolo.com/analytics/index.php/plus/show',
-            compat_urllib_parse.urlencode(data),
+            urlencode_postdata(data),
            {
                'Origin': 'http://www.dcndigital.ae',
                'Content-Type': 'application/x-www-form-urlencoded'
@@ -6,7 +6,7 @@ from ..compat import compat_str


 class DctpTvIE(InfoExtractor):
-    _VALID_URL = r'http://www.dctp.tv/(#/)?filme/(?P<id>.+?)/$'
+    _VALID_URL = r'https?://www.dctp.tv/(#/)?filme/(?P<id>.+?)/$'
    _TEST = {
        'url': 'http://www.dctp.tv/filme/videoinstallation-fuer-eine-kaufhausfassade/',
        'info_dict': {
@@ -5,7 +5,7 @@ from .common import InfoExtractor

 class DefenseGouvFrIE(InfoExtractor):
    IE_NAME = 'defense.gouv.fr'
-    _VALID_URL = r'http://.*?\.defense\.gouv\.fr/layout/set/ligthboxvideo/base-de-medias/webtv/(?P<id>[^/?#]*)'
+    _VALID_URL = r'https?://.*?\.defense\.gouv\.fr/layout/set/ligthboxvideo/base-de-medias/webtv/(?P<id>[^/?#]*)'

    _TEST = {
        'url': 'http://www.defense.gouv.fr/layout/set/ligthboxvideo/base-de-medias/webtv/attaque-chimique-syrienne-du-21-aout-2013-1',
@@ -38,6 +38,7 @@ class DFBIE(InfoExtractor):
        token_el = f4m_info.find('token')
        manifest_url = token_el.attrib['url'] + '?' + 'hdnea=' + token_el.attrib['auth'] + '&hdcore=3.2.0'
        formats = self._extract_f4m_formats(manifest_url, display_id)
+        self._sort_formats(formats)

        return {
            'id': video_id,
@@ -9,7 +9,7 @@ from ..compat import compat_str


 class DiscoveryIE(InfoExtractor):
-    _VALID_URL = r'''(?x)http://(?:www\.)?(?:
+    _VALID_URL = r'''(?x)https?://(?:www\.)?(?:
            discovery|
            investigationdiscovery|
            discoverylife|
@@ -63,18 +63,23 @@ class DiscoveryIE(InfoExtractor):

        video_title = info.get('playlist_title') or info.get('video_title')

-        entries = [{
-            'id': compat_str(video_info['id']),
-            'formats': self._extract_m3u8_formats(
+        entries = []
+
+        for idx, video_info in enumerate(info['playlist']):
+            formats = self._extract_m3u8_formats(
                video_info['src'], display_id, 'mp4', 'm3u8_native', m3u8_id='hls',
-                note='Download m3u8 information for video %d' % (idx + 1)),
-            'title': video_info['title'],
-            'description': video_info.get('description'),
-            'duration': parse_duration(video_info.get('video_length')),
-            'webpage_url': video_info.get('href') or video_info.get('url'),
-            'thumbnail': video_info.get('thumbnailURL'),
-            'alt_title': video_info.get('secondary_title'),
-            'timestamp': parse_iso8601(video_info.get('publishedDate')),
-        } for idx, video_info in enumerate(info['playlist'])]
+                note='Download m3u8 information for video %d' % (idx + 1))
+            self._sort_formats(formats)
+            entries.append({
+                'id': compat_str(video_info['id']),
+                'formats': formats,
+                'title': video_info['title'],
+                'description': video_info.get('description'),
+                'duration': parse_duration(video_info.get('video_length')),
+                'webpage_url': video_info.get('href') or video_info.get('url'),
+                'thumbnail': video_info.get('thumbnailURL'),
+                'alt_title': video_info.get('secondary_title'),
+                'timestamp': parse_iso8601(video_info.get('publishedDate')),
+            })

        return self.playlist_result(entries, display_id, video_title)
@@ -10,7 +10,7 @@ from ..compat import (compat_str, compat_basestring)

 class DouyuTVIE(InfoExtractor):
    IE_DESC = '斗鱼'
-    _VALID_URL = r'http://(?:www\.)?douyutv\.com/(?P<id>[A-Za-z0-9]+)'
+    _VALID_URL = r'https?://(?:www\.)?douyu(?:tv)?\.com/(?P<id>[A-Za-z0-9]+)'
    _TESTS = [{
        'url': 'http://www.douyutv.com/iseven',
        'info_dict': {
@@ -60,6 +60,9 @@ class DouyuTVIE(InfoExtractor):
        'params': {
            'skip_download': True,
        },
+    }, {
+        'url': 'http://www.douyu.com/xiaocang',
+        'only_matching': True,
    }]

    def _real_extract(self, url):
@@ -10,7 +10,7 @@ from ..utils import int_or_none


 class DPlayIE(InfoExtractor):
-    _VALID_URL = r'http://(?P<domain>it\.dplay\.com|www\.dplay\.(?:dk|se|no))/[^/]+/(?P<id>[^/?#]+)'
+    _VALID_URL = r'https?://(?P<domain>it\.dplay\.com|www\.dplay\.(?:dk|se|no))/[^/]+/(?P<id>[^/?#]+)'

    _TESTS = [{
        'url': 'http://it.dplay.com/take-me-out/stagione-1-episodio-25/',
@@ -118,6 +118,8 @@ class DPlayIE(InfoExtractor):
                if info.get(protocol):
                    extract_formats(protocol, info[protocol])

+        self._sort_formats(formats)
+
        return {
            'id': video_id,
            'display_id': display_id,
@@ -6,7 +6,6 @@ import itertools
 from .amp import AMPIE
 from ..compat import (
    compat_HTTPError,
-    compat_urllib_parse,
    compat_urlparse,
 )
 from ..utils import (
@@ -14,6 +13,7 @@ from ..utils import (
    clean_html,
    int_or_none,
    sanitized_Request,
+    urlencode_postdata
 )


@@ -50,7 +50,7 @@ class DramaFeverBaseIE(AMPIE):
        }

        request = sanitized_Request(
-            self._LOGIN_URL, compat_urllib_parse.urlencode(login_form).encode('utf-8'))
+            self._LOGIN_URL, urlencode_postdata(login_form))
        response = self._download_webpage(
            request, None, 'Logging in as %s' % username)

@@ -7,7 +7,7 @@ from .zdf import ZDFIE

 class DreiSatIE(ZDFIE):
    IE_NAME = '3sat'
-    _VALID_URL = r'(?:http://)?(?:www\.)?3sat\.de/mediathek/(?:index\.php|mediathek\.php)?\?(?:(?:mode|display)=[^&]+&)*obj=(?P<id>[0-9]+)$'
+    _VALID_URL = r'(?:https?://)?(?:www\.)?3sat\.de/mediathek/(?:index\.php|mediathek\.php)?\?(?:(?:mode|display)=[^&]+&)*obj=(?P<id>[0-9]+)$'
    _TESTS = [
        {
            'url': 'http://www.3sat.de/mediathek/index.php?mode=play&obj=45918',
@@ -15,7 +15,7 @@ class DVTVIE(InfoExtractor):
    IE_NAME = 'dvtv'
    IE_DESC = 'http://video.aktualne.cz/'

-    _VALID_URL = r'http://video\.aktualne\.cz/(?:[^/]+/)+r~(?P<id>[0-9a-f]{32})'
+    _VALID_URL = r'https?://video\.aktualne\.cz/(?:[^/]+/)+r~(?P<id>[0-9a-f]{32})'

    _TESTS = [{
        'url': 'http://video.aktualne.cz/dvtv/vondra-o-ceskem-stoleti-pri-pohledu-na-havla-mi-bylo-trapne/r~e5efe9ca855511e4833a0025900fea04/',
@@ -39,13 +39,13 @@ class DWIE(InfoExtractor):
        hidden_inputs = self._hidden_inputs(webpage)
        title = hidden_inputs['media_title']

-        formats = []
        if hidden_inputs.get('player_type') == 'video' and hidden_inputs.get('stream_file') == '1':
            formats = self._extract_smil_formats(
                'http://www.dw.com/smil/v-%s' % media_id, media_id,
                transform_source=lambda s: s.replace(
                    'rtmp://tv-od.dw.de/flash/',
                    'http://tv-download.dw.de/dwtv_video/flv/'))
+            self._sort_formats(formats)
        else:
            formats = [{'url': hidden_inputs['file_name']}]

@@ -7,7 +7,7 @@ from .common import InfoExtractor


 class EchoMskIE(InfoExtractor):
-    _VALID_URL = r'http://(?:www\.)?echo\.msk\.ru/sounds/(?P<id>\d+)'
+    _VALID_URL = r'https?://(?:www\.)?echo\.msk\.ru/sounds/(?P<id>\d+)'
    _TEST = {
        'url': 'http://www.echo.msk.ru/sounds/1464134.html',
        'md5': '2e44b3b78daff5b458e4dbc37f191f7c',
@@ -3,7 +3,7 @@ from __future__ import unicode_literals
 import re

 from .common import InfoExtractor
-from ..compat import compat_urllib_parse
+from ..compat import compat_urllib_parse_urlencode
 from ..utils import (
    ExtractorError,
    unescapeHTML
@@ -43,7 +43,7 @@ class EroProfileIE(InfoExtractor):
        if username is None:
            return

-        query = compat_urllib_parse.urlencode({
+        query = compat_urllib_parse_urlencode({
            'username': username,
            'password': password,
            'url': 'http://www.eroprofile.com/',
@@ -8,7 +8,7 @@ from .common import InfoExtractor
 class ExfmIE(InfoExtractor):
    IE_NAME = 'exfm'
    IE_DESC = 'ex.fm'
-    _VALID_URL = r'http://(?:www\.)?ex\.fm/song/(?P<id>[^/]+)'
+    _VALID_URL = r'https?://(?:www\.)?ex\.fm/song/(?P<id>[^/]+)'
    _SOUNDCLOUD_URL = r'http://(?:www\.)?api\.soundcloud\.com/tracks/([^/]+)/stream'
    _TESTS = [
        {
@@ -5,19 +5,18 @@ import hashlib

 from .common import InfoExtractor
 from ..compat import (
-    compat_urllib_parse,
    compat_urllib_request,
    compat_urlparse,
 )
 from ..utils import (
-    encode_dict,
    ExtractorError,
    sanitized_Request,
+    urlencode_postdata,
 )


 class FC2IE(InfoExtractor):
-    _VALID_URL = r'^http://video\.fc2\.com/(?:[^/]+/)*content/(?P<id>[^/]+)'
+    _VALID_URL = r'^https?://video\.fc2\.com/(?:[^/]+/)*content/(?P<id>[^/]+)'
    IE_NAME = 'fc2'
    _NETRC_MACHINE = 'fc2'
    _TESTS = [{
@@ -57,7 +56,7 @@ class FC2IE(InfoExtractor):
            'Submit': ' Login ',
        }

-        login_data = compat_urllib_parse.urlencode(encode_dict(login_form_strs)).encode('utf-8')
+        login_data = urlencode_postdata(login_form_strs)
        request = sanitized_Request(
            'https://secure.id.fc2.com/index.php?mode=login&switch_language=en', login_data)

@@ -4,7 +4,7 @@ from .common import InfoExtractor


 class FirstpostIE(InfoExtractor):
-    _VALID_URL = r'http://(?:www\.)?firstpost\.com/[^/]+/.*-(?P<id>[0-9]+)\.html'
+    _VALID_URL = r'https?://(?:www\.)?firstpost\.com/[^/]+/.*-(?P<id>[0-9]+)\.html'

    _TEST = {
        'url': 'http://www.firstpost.com/india/india-to-launch-indigenous-aircraft-carrier-monday-1025403.html',
@@ -8,7 +8,7 @@ from ..utils import int_or_none
 class FirstTVIE(InfoExtractor):
    IE_NAME = '1tv'
    IE_DESC = 'Первый канал'
-    _VALID_URL = r'http://(?:www\.)?1tv\.ru/(?:[^/]+/)+(?P<id>.+)'
+    _VALID_URL = r'https?://(?:www\.)?1tv\.ru/(?:[^/]+/)+(?P<id>.+)'

    _TESTS = [{
        'url': 'http://www.1tv.ru/videoarchive/73390',
@@ -4,8 +4,8 @@ import re

 from .common import InfoExtractor
 from ..compat import (
-    compat_urllib_parse,
    compat_parse_qs,
+    compat_urllib_parse_urlencode,
    compat_urllib_parse_urlparse,
    compat_urlparse,
 )
@@ -109,7 +109,7 @@ class FiveMinIE(InfoExtractor):

        response = self._download_json(
            'https://syn.5min.com/handlers/SenseHandler.ashx?' +
-            compat_urllib_parse.urlencode({
+            compat_urllib_parse_urlencode({
                'func': 'GetResults',
                'playlist': video_id,
                'sid': sid,
@@ -10,7 +10,7 @@ from ..utils import (

 class FKTVIE(InfoExtractor):
    IE_NAME = 'fernsehkritik.tv'
-    _VALID_URL = r'http://(?:www\.)?fernsehkritik\.tv/folge-(?P<id>[0-9]+)(?:/.*)?'
+    _VALID_URL = r'https?://(?:www\.)?fernsehkritik\.tv/folge-(?P<id>[0-9]+)(?:/.*)?'

    _TEST = {
        'url': 'http://fernsehkritik.tv/folge-1',
@@ -1,7 +1,7 @@
 from __future__ import unicode_literals

 from .common import InfoExtractor
-from ..compat import compat_urllib_parse
+from ..compat import compat_urllib_parse_urlencode
 from ..utils import (
    ExtractorError,
    int_or_none,
@@ -42,7 +42,7 @@ class FlickrIE(InfoExtractor):
        }
        if secret:
            query['secret'] = secret
-        data = self._download_json(self._API_BASE_URL + compat_urllib_parse.urlencode(query), video_id, note)
+        data = self._download_json(self._API_BASE_URL + compat_urllib_parse_urlencode(query), video_id, note)
        if data['stat'] != 'ok':
            raise ExtractorError(data['message'])
        return data
@@ -5,7 +5,7 @@ from .common import InfoExtractor


 class FootyRoomIE(InfoExtractor):
-    _VALID_URL = r'http://footyroom\.com/(?P<id>[^/]+)'
+    _VALID_URL = r'https?://footyroom\.com/(?P<id>[^/]+)'
    _TESTS = [{
        'url': 'http://footyroom.com/schalke-04-0-2-real-madrid-2015-02/',
        'info_dict': {
@@ -4,7 +4,7 @@ from .common import InfoExtractor


 class FoxgayIE(InfoExtractor):
-    _VALID_URL = r'http://(?:www\.)?foxgay\.com/videos/(?:\S+-)?(?P<id>\d+)\.shtml'
+    _VALID_URL = r'https?://(?:www\.)?foxgay\.com/videos/(?:\S+-)?(?P<id>\d+)\.shtml'
    _TEST = {
        'url': 'http://foxgay.com/videos/fuck-turkish-style-2582.shtml',
        'md5': '80d72beab5d04e1655a56ad37afe6841',
@@ -18,8 +18,8 @@ class FoxNewsIE(AMPIE):
                'title': 'Frozen in Time',
                'description': '16-year-old girl is size of toddler',
                'duration': 265,
-                # 'timestamp': 1304411491,
-                # 'upload_date': '20110503',
+                'timestamp': 1304411491,
+                'upload_date': '20110503',
                'thumbnail': 're:^https?://.*\.jpg$',
            },
        },
@@ -32,8 +32,8 @@ class FoxNewsIE(AMPIE):
                'title': "Rep. Luis Gutierrez on if Obama's immigration plan is legal",
                'description': "Congressman discusses president's plan",
                'duration': 292,
-                # 'timestamp': 1417662047,
-                # 'upload_date': '20141204',
+                'timestamp': 1417662047,
+                'upload_date': '20141204',
                'thumbnail': 're:^https?://.*\.jpg$',
            },
            'params': {
@@ -6,7 +6,7 @@ from ..utils import int_or_none


 class FranceInterIE(InfoExtractor):
-    _VALID_URL = r'http://(?:www\.)?franceinter\.fr/player/reecouter\?play=(?P<id>[0-9]+)'
+    _VALID_URL = r'https?://(?:www\.)?franceinter\.fr/player/reecouter\?play=(?P<id>[0-9]+)'
    _TEST = {
        'url': 'http://www.franceinter.fr/player/reecouter?play=793962',
        'md5': '4764932e466e6f6c79c317d2e74f6884',
@@ -60,28 +60,31 @@ class FranceTVBaseInfoExtractor(InfoExtractor):
                    video_id, 'Downloading f4m manifest token', fatal=False)
                if f4m_url:
                    formats.extend(self._extract_f4m_formats(
-                        f4m_url + '&hdcore=3.7.0&plugin=aasp-3.7.0.39.44', video_id, 1, format_id))
+                        f4m_url + '&hdcore=3.7.0&plugin=aasp-3.7.0.39.44',
+                        video_id, f4m_id=format_id, fatal=False))
            elif ext == 'm3u8':
-                formats.extend(self._extract_m3u8_formats(video_url, video_id, 'mp4', m3u8_id=format_id))
+                formats.extend(self._extract_m3u8_formats(
+                    video_url, video_id, 'mp4', entry_protocol='m3u8_native',
+                    m3u8_id=format_id, fatal=False))
            elif video_url.startswith('rtmp'):
                formats.append({
                    'url': video_url,
                    'format_id': 'rtmp-%s' % format_id,
                    'ext': 'flv',
-                    'preference': 1,
                })
            else:
-                formats.append({
-                    'url': video_url,
-                    'format_id': format_id,
-                    'preference': -1,
-                })
+                if self._is_valid_url(video_url, video_id, format_id):
+                    formats.append({
+                        'url': video_url,
+                        'format_id': format_id,
+                    })
        self._sort_formats(formats)

        title = info['titre']
        subtitle = info.get('sous_titre')
        if subtitle:
            title += ' - %s' % subtitle
+        title = title.strip()

        subtitles = {}
        subtitles_list = [{
@@ -125,13 +128,13 @@ class PluzzIE(FranceTVBaseInfoExtractor):

 class FranceTvInfoIE(FranceTVBaseInfoExtractor):
    IE_NAME = 'francetvinfo.fr'
-    _VALID_URL = r'https?://(?:www|mobile)\.francetvinfo\.fr/.*/(?P<title>.+)\.html'
+    _VALID_URL = r'https?://(?:www|mobile|france3-regions)\.francetvinfo\.fr/.*/(?P<title>.+)\.html'

    _TESTS = [{
        'url': 'http://www.francetvinfo.fr/replay-jt/france-3/soir-3/jt-grand-soir-3-lundi-26-aout-2013_393427.html',
        'info_dict': {
            'id': '84981923',
-            'ext': 'flv',
+            'ext': 'mp4',
            'title': 'Soir 3',
            'upload_date': '20130826',
            'timestamp': 1377548400,
@@ -139,6 +142,10 @@ class FranceTvInfoIE(FranceTVBaseInfoExtractor):
                'fr': 'mincount:2',
            },
        },
+        'params': {
+            # m3u8 downloads
+            'skip_download': True,
+        },
    }, {
        'url': 'http://www.francetvinfo.fr/elections/europeennes/direct-europeennes-regardez-le-debat-entre-les-candidats-a-la-presidence-de-la-commission_600639.html',
        'info_dict': {
@@ -155,11 +162,32 @@ class FranceTvInfoIE(FranceTVBaseInfoExtractor):
        'url': 'http://www.francetvinfo.fr/economie/entreprises/les-entreprises-familiales-le-secret-de-la-reussite_933271.html',
        'md5': 'f485bda6e185e7d15dbc69b72bae993e',
        'info_dict': {
-            'id': '556e03339473995ee145930c',
+            'id': 'NI_173343',
            'ext': 'mp4',
            'title': 'Les entreprises familiales : le secret de la réussite',
            'thumbnail': 're:^https?://.*\.jpe?g$',
-        }
+            'timestamp': 1433273139,
+            'upload_date': '20150602',
+        },
+        'params': {
+            # m3u8 downloads
+            'skip_download': True,
+        },
+    }, {
+        'url': 'http://france3-regions.francetvinfo.fr/bretagne/cotes-d-armor/thalassa-echappee-breizh-ce-venredi-dans-les-cotes-d-armor-954961.html',
+        'md5': 'f485bda6e185e7d15dbc69b72bae993e',
+        'info_dict': {
+            'id': 'NI_657393',
+            'ext': 'mp4',
+            'title': 'Olivier Monthus, réalisateur de "Bretagne, le choix de l’Armor"',
+            'description': 'md5:a3264114c9d29aeca11ced113c37b16c',
+            'thumbnail': 're:^https?://.*\.jpe?g$',
+            'timestamp': 1458300695,
+            'upload_date': '20160318',
+        },
+        'params': {
+            'skip_download': True,
+        },
    }]

    def _real_extract(self, url):
@@ -172,7 +200,9 @@ class FranceTvInfoIE(FranceTVBaseInfoExtractor):
            return self.url_result(dmcloud_url, 'DailymotionCloud')

        video_id, catalogue = self._search_regex(
-            r'id-video=([^@]+@[^"]+)', webpage, 'video id').split('@')
+            (r'id-video=([^@]+@[^"]+)',
+             r'<a[^>]+href="(?:https?:)?//videos\.francetv\.fr/video/([^@]+@[^"]+)"'),
+            webpage, 'video id').split('@')
        return self._extract_video(video_id, catalogue)


@@ -5,7 +5,7 @@ from ..utils import ExtractorError


 class FreeVideoIE(InfoExtractor):
-    _VALID_URL = r'^http://www.freevideo.cz/vase-videa/(?P<id>[^.]+)\.html(?:$|[?#])'
+    _VALID_URL = r'^https?://www.freevideo.cz/vase-videa/(?P<id>[^.]+)\.html(?:$|[?#])'

    _TEST = {
        'url': 'http://www.freevideo.cz/vase-videa/vysukany-zadecek-22033.html',
@@ -5,7 +5,6 @@ from .common import InfoExtractor
 from ..utils import (
    clean_html,
    determine_ext,
-    encode_dict,
    int_or_none,
    sanitized_Request,
    ExtractorError,
@@ -54,10 +53,10 @@ class FunimationIE(InfoExtractor):
        (username, password) = self._get_login_info()
        if username is None:
            return
-        data = urlencode_postdata(encode_dict({
+        data = urlencode_postdata({
            'email_field': username,
            'password_field': password,
-        }))
+        })
        login_request = sanitized_Request('http://www.funimation.com/login', data, headers={
            'User-Agent': 'Mozilla/5.0 (Windows NT 5.2; WOW64; rv:42.0) Gecko/20100101 Firefox/42.0',
            'Content-Type': 'application/x-www-form-urlencoded'
@@ -10,7 +10,7 @@ from .youtube import YoutubeIE


 class GamekingsIE(InfoExtractor):
-    _VALID_URL = r'http://www\.gamekings\.nl/(?:videos|nieuws)/(?P<id>[^/]+)'
+    _VALID_URL = r'https?://www\.gamekings\.nl/(?:videos|nieuws)/(?P<id>[^/]+)'
    _TESTS = [{
        # YouTube embed video
        'url': 'http://www.gamekings.nl/videos/phoenix-wright-ace-attorney-dual-destinies-review/',
@@ -14,7 +14,7 @@ from ..utils import (


 class GameSpotIE(InfoExtractor):
-    _VALID_URL = r'http://(?:www\.)?gamespot\.com/.*-(?P<id>\d+)/?'
+    _VALID_URL = r'https?://(?:www\.)?gamespot\.com/.*-(?P<id>\d+)/?'
    _TESTS = [{
        'url': 'http://www.gamespot.com/videos/arma-3-community-guide-sitrep-i/2300-6410818/',
        'md5': 'b2a30deaa8654fcccd43713a6b6a4825',
@@ -13,7 +13,7 @@ from ..utils import (


 class GameStarIE(InfoExtractor):
-    _VALID_URL = r'http://www\.gamestar\.de/videos/.*,(?P<id>[0-9]+)\.html'
+    _VALID_URL = r'https?://www\.gamestar\.de/videos/.*,(?P<id>[0-9]+)\.html'
    _TEST = {
        'url': 'http://www.gamestar.de/videos/trailer,3/hobbit-3-die-schlacht-der-fuenf-heere,76110.html',
        'md5': '96974ecbb7fd8d0d20fca5a00810cea7',
@@ -9,7 +9,7 @@ from ..utils import (


 class GametrailersIE(InfoExtractor):
-    _VALID_URL = r'http://www\.gametrailers\.com/videos/view/[^/]+/(?P<id>.+)'
+    _VALID_URL = r'https?://www\.gametrailers\.com/videos/view/[^/]+/(?P<id>.+)'

    _TEST = {
        'url': 'http://www.gametrailers.com/videos/view/gametrailers-com/116437-Just-Cause-3-Review',
@@ -3,11 +3,11 @@ from __future__ import unicode_literals
 import re

 from .common import InfoExtractor
-from ..compat import compat_urllib_parse
 from ..utils import (
    remove_end,
    HEADRequest,
    sanitized_Request,
+    urlencode_postdata,
 )


@@ -123,7 +123,7 @@ class GDCVaultIE(InfoExtractor):
            'password': password,
        }

-        request = sanitized_Request(login_url, compat_urllib_parse.urlencode(login_form))
+        request = sanitized_Request(login_url, urlencode_postdata(login_form))
        request.add_header('Content-Type', 'application/x-www-form-urlencoded')
        self._download_webpage(request, display_id, 'Logging in')
        start_page = self._download_webpage(webpage_url, display_id, 'Getting authenticated video page')
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Philipp Hagemeister	1e02bc7ba2	release 2016.04.01	2016-04-01 09:07:40 +02:00
remitamine	63c55e9f22	[cbs] improve extraction(closes #6321 )	2016-04-01 07:33:37 +01:00
remitamine	f9b1529af8	[generic] remove sbnation test(handled by VoxMediaIE)	2016-03-31 23:50:45 +01:00
remitamine	961fc024d2	[voxmedia] improve sbnation support	2016-03-31 23:33:36 +01:00
Sergey M․	b53a06e3b9	[udemy:course] Use new URL format	2016-04-01 02:24:22 +06:00
remitamine	4ecc1fc638	[howstuffworks] improve extraction	2016-03-31 21:11:58 +01:00
Yen Chi Hsuan	5b012dfce8	[tudou] Improve error handling (closes #8988 )	2016-04-01 01:42:16 +08:00
remitamine	8369942773	[voxmedia] Add new extractor(closes #3182 )	2016-03-31 18:36:41 +01:00
Sergey M․	86f3b66cec	[udemy] Remove unused import	2016-03-31 23:00:11 +06:00
Sergey M․	6bb4600717	[udemy:course] Simplify course curriculum downloading	2016-03-31 22:59:19 +06:00
Sergey M․	41d06b0424	[extractor/common] Improve _request_webpage * Do not ignore data, headers and query for Requests * Default values for headers and query switched to dicts since these are used by urllib itself	2016-03-31 22:58:38 +06:00
Sergey M․	15d260ebaa	[utils] Use update_Request in http_request	2016-03-31 22:55:49 +06:00
Sergey M․	ed0291d153	[utils] Add update_Request	2016-03-31 22:55:01 +06:00
Sergey M․	81da8cbc45	[udemy] Switch to api 2.0 (Closes #9035 )	2016-03-31 22:05:25 +06:00
Sergey M․	5299bc3f91	[beeg] Switch to api v6 (Closes #9036 )	2016-03-31 20:42:41 +06:00
remitamine	c9c39c22c5	[nationalgeographic] add support for channel.nationalgeographic.com urls	2016-03-31 13:47:38 +01:00
remitamine	d84b48e3f1	[nationalgeographic] improve extraction	2016-03-31 13:44:55 +01:00
remitamine	dd17041c82	[tenplay] remove extractor(fixes #6927 )	2016-03-31 12:02:04 +01:00
remitamine	fea7295b14	[brightcove] relax embed_in_page regex	2016-03-31 10:48:22 +01:00
remitamine	9cf01f7f30	[nbc] add new extractor for csnne.com(#5432 )	2016-03-31 00:26:42 +01:00
remitamine	ce548296fe	[cnbc] fix test	2016-03-31 00:25:11 +01:00
remitamine	c02ec7d430	[cnbc] Add new extractor(closes #8012 )	2016-03-30 23:18:31 +01:00
remitamine	6b820a2376	[myspace] improve extraction	2016-03-30 21:18:07 +01:00
Yen Chi Hsuan	e621a344e6	[kwuo] Port to new API and enable --cn-verification-proxy	2016-03-31 02:27:52 +08:00
Yen Chi Hsuan	3ae6f8fec1	[kwuo] Remove _sort_formats() from KuwoBaseIE._get_formats() Following the idea proposed in `19dbaeece3`	2016-03-31 02:11:21 +08:00
Yen Chi Hsuan	597d52fadb	[kuwo:song] Correct song ID extraction (fixes #9033 ) Bug introduced in `daef04a4e7`.	2016-03-31 02:00:50 +08:00
Sergey M․	afca767d19	[tumblr] Improve _VALID_URL (Closes #9027 )	2016-03-30 22:26:43 +06:00
remitamine	6e359a1534	[comcarcoff] don not depend on crackle extractor(closes #8995 ) previously extraction has been delegated to crackle to extract more info and subtitles #6106 but some of the episodes can't be extracted using crackle #8995.	2016-03-30 12:27:00 +01:00
Sergey M․	607619bc90	Add manually generated ISSUE_TEMPLATE.md In order not to wait for the next release	2016-03-29 22:04:29 +06:00
Sergey M․	0b7bfc9422	Improve ISSUE_TEMPLATE_tmpl.md	2016-03-29 22:02:42 +06:00
Sergey M․	7168a6c874	[devscripts/make_issue_template] Fix __version__ again	2016-03-29 03:05:15 +06:00
Sergey M․	034947dd1e	Rename ISSUE_TEMPLATE.tmpl in order not to be picked up by github	2016-03-29 02:48:04 +06:00
Sergey M․	3c0de33ad7	Remove ISSUE_TEMPLATE.md	2016-03-29 02:43:48 +06:00
Sergey M․	89924f8230	[devscripts/make_issue_template] Fix NameError under python3	2016-03-29 02:41:27 +06:00
Sergey M․	a39c68f7e5	Exclude make_issue_template.py from flake8	2016-03-29 02:19:24 +06:00
Sergey M․	4a5a67ca25	[devscripts/release.sh] Make ISSUE_TEMPLATE.md and commit it	2016-03-29 02:18:52 +06:00
Sergey M․	8751da85a7	[Makefile] Fix ISSUE_TEMPLATE.md target	2016-03-29 02:17:57 +06:00
Sergey M․	3bf1df51fd	[devscripts/make_issue_template] Rework to use ISSUE_TEMPLATE.tmpl (Closes #8785 )	2016-03-29 02:16:38 +06:00
Sergey M․	3842a3e652	Add ISSUE_TEMPLATE.tmpl as template for ISSUE_TEMPLATE.md	2016-03-29 02:15:26 +06:00
Sander van den Oever	7710bdf4e8	Add initial ISSUE_TEMPLATE Add auto-updating of youtube-dl version in ISSUE_TEMPLATE Move parts of template text and adopt makefile to new format Moved the 'kind-of-issue' section and rephrased a bit Rephrased and moved Example URL section upwards Moved ISSUE_TEMPLATE inside .github folder. Update makefile to match new folderstructure	2016-03-28 22:43:13 +06:00
Sergey M	8d9dd3c34b	[README.md] Add format_id to the list of string meta fields available for use in format selection	2016-03-28 03:08:34 +05:00
Sergey M․	33f3040a3e	[YoutubeDL] Fix sanitizing subtitles' url	2016-03-28 03:13:39 +06:00
Sergey M․	03442072c0	[pornhub] Fix typo (Closes #9008 )	2016-03-28 01:21:44 +06:00
Sergey M․	c8b13fec02	[foxnews] Restore upload time fields in test	2016-03-28 01:14:12 +06:00
Sergey M․	87d105ac6c	[amp] Fix upload timestamp extraction (Closes #9007 )	2016-03-28 01:13:47 +06:00
Sergey M․	3454139576	[pornhub:uservideos] Add support for multipage videos (Closes #9006 )	2016-03-28 00:50:46 +06:00
Sergey M․	3a23bae9cc	[pornhub:playlistbase] Do not include videos not from playlist	2016-03-28 00:32:57 +06:00
Sergey M․	8f9a477e7f	[pornhub:playlistbase] Use orderedSet	2016-03-28 00:21:08 +06:00
Sergey M․	a1cf3e38a3	[bbc] Extend vpid regex (Closes #9003 )	2016-03-27 23:22:51 +06:00
Philipp Hagemeister	a122e7080b	release 2016.03.27	2016-03-27 16:56:33 +02:00
Sergey M․	b22ca76204	[extractor/common] Filter out unsupported encrypted media for f4m formats (Closes #8573 )	2016-03-27 07:42:38 +06:00
Sergey M․	f7df343b4a	[downloader/f4m] Extract routine for removing unsupported encrypted media	2016-03-27 07:41:19 +06:00
Sergey M․	19dbaeece3	Remove _sort_formats from _extract_*_formats methods Now _sort_formats should be called explicitly. _sort_formats has been added to all the necessary places in code. Closes #8051	2016-03-27 07:03:08 +06:00
Yen Chi Hsuan	395fd4b08a	[twitter] Handle another form of embedded Vine Fixes #8996	2016-03-27 04:36:02 +08:00
Sergey M․	8018028d0f	[pluralsight] Extract chapter metadata (Closes #8993 )	2016-03-27 02:10:52 +06:00
Sergey M․	00322ad4fd	[lynda] Extract chapter metadata (#8993 )	2016-03-27 02:00:36 +06:00
Sergey M․	4cf3489c6e	[vevo] Update videoservice API URL (Closes #8900 )	2016-03-27 01:11:11 +06:00
Sergey M․	b24ab3e341	[udemy] Improve paid course detection	2016-03-27 00:09:12 +06:00
Sergey M․	af4116f4f0	[udemy] Improve format_id	2016-03-27 00:02:52 +06:00
Sergey M․	f973e5d54e	[udemy] Drop outputs' formats Always results in 403	2016-03-26 23:55:07 +06:00
Sergey M․	62f55aa68a	[udemy] Add outputs metadata to view_html formats	2016-03-26 23:54:12 +06:00
Sergey M․	02d7634d24	[udemy] Fix outputs' formats format_id	2016-03-26 23:43:25 +06:00
Sergey M․	48dce58ca9	[udemy] Use custom sorting	2016-03-26 23:42:46 +06:00
Sergey M․	efcba804f6	[udemy] Extract formats from view_html (Closes #8979 )	2016-03-26 23:42:34 +06:00
Sergey M․	6dee688e6d	[youtube:playlistsbase] Restrict playlist regex (Closes #8986 )	2016-03-26 20:42:18 +06:00
Sergey M․	eedb7ba536	[YoutubeDL] Sort imports	2016-03-26 19:40:33 +06:00
Sergey M․	dcf77cf1a7	[YoutubeDL] Sanitize final URLs (Closes #8991 )	2016-03-26 19:37:41 +06:00
Sergey M․	17bcc626bf	[utils] Extract sanitize_url routine	2016-03-26 19:33:57 +06:00
Sergey M․	b5a5bbf376	[mailru] Extend _VALID_URL (Closes #8990 )	2016-03-26 19:15:32 +06:00
Yen Chi Hsuan	e68d3a010f	[twitter] Fix extraction (closes #8966 ) HLS and DASH formats are no longer appeared in test cases. I keep them for fear of triggering new errors.	2016-03-26 18:34:51 +08:00
Yen Chi Hsuan	d10fe8358c	[generic] Add a test case for brightcove embed Closes #8862	2016-03-26 18:30:43 +08:00
Yen Chi Hsuan	d6c340cae5	[brightcove] Extract more formats (#8862 )	2016-03-26 18:21:07 +08:00
Yen Chi Hsuan	5964b598ff	[brightcove] Support alternative BrightcoveExperience layout The full URL lays in the `data` attribute of <object> (#8862)	2016-03-26 17:47:32 +08:00
Philipp Hagemeister	62cdb96f51	release 2016.03.26	2016-03-26 08:58:03 +01:00
Sergey M․	e289d6d62c	[test_compat] Add tests for compat_urllib_parse_urlencode	2016-03-26 02:38:33 +06:00
Sergey M․	6e6bc8dae5	Use urlencode_postdata across the codebase	2016-03-26 02:19:24 +06:00
Sergey M․	15707c7e02	[compat] Add compat_urllib_parse_urlencode and eliminate encode_dict encode_dict functionality has been improved and moved directly into compat_urllib_parse_urlencode All occurrences of compat_urllib_parse.urlencode throughout the codebase have been replaced by compat_urllib_parse_urlencode Closes #8974	2016-03-26 01:46:57 +06:00
Sergey M․	2156f16ca7	[thescene] Fix extraction and improve style (Closes #8978 )	2016-03-25 20:14:34 +06:00
Sergey M․	4db441de72	[once] Relax _VALID_URL (Closes #8976 )	2016-03-25 19:51:28 +06:00
Philipp Hagemeister	0be8314dc8	release 2016.03.25	2016-03-25 09:27:18 +01:00
Yen Chi Hsuan	d7f62b049a	[iqiyi] Update enc_key	2016-03-25 15:45:40 +08:00
Yen Chi Hsuan	3bb3356812	[douyutv] Extend _VALID_URL	2016-03-25 15:43:29 +08:00
Sergey M․	3f15fec1d1	Credit @Kagami for mnet (#8958 )	2016-03-25 03:56:27 +06:00
Sergey M․	98e68806fb	[mnet] Improve (Closes #8958 )	2016-03-25 03:26:29 +06:00
Kagami Hiiragi	e031768666	[mnet] Add new extractor	2016-03-25 02:32:06 +06:00
Sergey M․	5eb7db4ee9	[udemy] Add support for new URL schema	2016-03-25 02:28:39 +06:00
Sergey M․	f0e83681d9	[udemy] Extract formats from outputs	2016-03-25 02:27:13 +06:00
Sergey M․	ff9d5d0938	[udemy] Improve course enrolling	2016-03-25 02:26:46 +06:00
Sergey M․	d041a73674	[extractor/__init__] Add youtube:live and sort youtube extractors alphabetically	2016-03-25 01:39:25 +06:00
Sergey M․	f07e276a04	[youtube:live] Add extractor (Closes #8959 )	2016-03-25 01:18:14 +06:00
Sergey M․	993271da0a	[nytimes] Tolerate missing metadata (Closes #8952 )	2016-03-24 23:28:24 +06:00
Sergey M․	369e7e3ff0	[iprima] Fix extraction (Closes #8953 )	2016-03-24 22:54:26 +06:00
Sergey M․	5767b4eeae	[mtv] Fix description extraction (Closes #8962 )	2016-03-24 22:23:31 +06:00
Yen Chi Hsuan	622d19160b	[utils] Clarify Python versions affected by buggy struct module	2016-03-24 18:06:15 +08:00
Yen Chi Hsuan	32d88410eb	[tumblr] Add a test with Instagram embed Closes #8817	2016-03-24 16:32:53 +08:00
Yen Chi Hsuan	5a51775a58	[generic] Extract Instagram embeds (#8817 )	2016-03-24 16:32:27 +08:00
Yen Chi Hsuan	87696e78d7	[instagram] Unescape description (#8817 )	2016-03-24 16:30:01 +08:00
Yen Chi Hsuan	c4096e8aea	[instagram] Extract embed videos (#8817 )	2016-03-24 16:29:33 +08:00
Yen Chi Hsuan	fc27ea9464	[tumblr] Support Vine embeds (#8817 )	2016-03-23 23:55:52 +08:00
Yen Chi Hsuan	088e1aac59	[generic] Support Vine embeds (#8817 )	2016-03-23 23:55:08 +08:00
Yen Chi Hsuan	81f36eba88	[test/test_utils] Update for escape_url change (again)	2016-03-23 23:23:26 +08:00
Yen Chi Hsuan	2d60465e44	[test/test_utils] Update for escape_url change	2016-03-23 23:20:28 +08:00
Sergey M	4333d56494	Merge pull request #8898 from dstftw/fragment-retries Add --fragment-retries option (Fixes #8466)	2016-03-23 20:12:32 +05:00
Sergey M․	882c699296	[tunein] Fix stream data extraction (Closes #8899 , closes #8924 )	2016-03-23 20:45:39 +06:00
Yen Chi Hsuan	efbed08dc2	[utils] Encode hostnames before passing to urllib With IDN (Internationalized Domain Name) and a proxy, non-ascii URLs are passed down to urllib/urllib2, causing UnicodeEncodeError Fixes #8890	2016-03-23 22:24:52 +08:00
Jaime Marquínez Ferrándiz	7da2c87119	Add extractor for thescene.com (closes #8929 )	2016-03-22 22:17:59 +01:00
Sergey M․	c6ca11f1b3	[once] Prevent ads from embedding into m3u8 playlists (Closes #8893 )	2016-03-22 23:48:05 +06:00
Sergey M․	2beeb286e1	[laola1tv] Add support for livestreams (Closes #8934 )	2016-03-22 22:32:59 +06:00
Sergey M․	cc7397b04d	[ceskatelevize] Make m3u8 formats extraction non fatal (Closes #8933 )	2016-03-22 21:12:29 +06:00
Sergey M․	bc5d16b302	[animeondemand] Skip dash for now	2016-03-21 23:37:39 +06:00
Sergey M․	85c637b737	[animeondemand] Extract teaser when no full episode available (#8923 )	2016-03-21 23:35:50 +06:00
Sergey M․	5c69f7a479	[animeondemand] Respect startvideo (Closes #8923 )	2016-03-21 23:31:40 +06:00
Sergey M․	ff5873b72d	[motherless] Detect friends only videos	2016-03-21 22:24:42 +06:00
Sergey M․	065c4b27bf	[xhamster:embed] Extract vars (Closes #8912 )	2016-03-21 22:07:34 +06:00
Sergey M․	1600ed1ff9	[rutv] Improve flash version pattern (Closes #8911 )	2016-03-21 21:46:49 +06:00
Sergey M․	5886b38d73	Add support for https for all extractors as preventive and future-proof measure	2016-03-21 21:36:32 +06:00
Sergey M․	0cef27ad25	Add missing r prefix for _VALID_URLs	2016-03-21 21:22:37 +06:00
Sergey M․	12af4beb3e	[mailru] Add support for https (Closes #8920 )	2016-03-21 21:17:29 +06:00
Sergey M․	9016d76f71	[YoutubeDL] Improve _format_note	2016-03-20 22:01:45 +06:00
Sergey M․	3c5d183c19	[animeondemand] Extract all formats (Closes #8906 )	2016-03-20 21:51:22 +06:00
Sergey M․	3e8bb9a972	[animeondemand] Detect geo restriction	2016-03-20 20:39:00 +06:00
Yen Chi Hsuan	daef04a4e7	[kwuo] Fix KuwoChartIE and KuwoSingerIE and accept new URL forms	2016-03-20 20:17:56 +08:00
Yen Chi Hsuan	7caae128a7	Credit @vitstradal for the key algorithm in OpenloadIE (#8489 ) [ci skip]	2016-03-20 19:12:02 +08:00
Yen Chi Hsuan	2648918c81	[vlive] Fix creator extraction (closes #8814 )	2016-03-20 18:15:53 +08:00
Jaime Marquínez Ferrándiz	920d318d3c	README: document that BSD make is also supported (#8902 )	2016-03-20 10:55:14 +01:00
Yen Chi Hsuan	9e3c2f1d74	[openload] Misc improvements * Add thumbnail * Detect errors (#6469) * Match more (#6469, #8489)	2016-03-20 16:49:44 +08:00
Yen Chi Hsuan	2bfeee69b9	[openload] Add new extractor (closes #8489 )	2016-03-20 15:54:58 +08:00
Yen Chi Hsuan	664bcd80b9	[tudou] Use InAdvancePagedList (closes #8884 )	2016-03-20 15:45:31 +08:00
Sergey M․	3c20208eff	[francetv] Improve formats extraction	2016-03-20 13:00:46 +06:00
Sergey M․	db264e3cc3	[francetvinfo] Add support for france3-regions and strip title (Closes #7673 )	2016-03-20 12:44:04 +06:00
Sergey M	d396f30467	Merge pull request #8902 from jaimeMF/bmake Makefile: make it compatible with bmake	2016-03-20 11:08:57 +05:00
Sergey M․	96a9f22d98	[discovery] Relax _VALID_URL (Closes #8903 )	2016-03-20 10:26:58 +06:00
Sergey M․	40025ee2a3	[postprocessort/ffmpeg] Allow embedding webvtt into webm (Closes #8874 )	2016-03-20 04:12:34 +06:00
Jaime Marquínez Ferrándiz	3ff63fb365	Makefile: make it compatible with bmake It's the portable version of BSD make: http://crufty.net/help/sjg/bmake.html The syntax for conditionals is different in GNU make and BSD make, so we use the shell	2016-03-19 21:51:13 +01:00
Jaime Marquínez Ferrándiz	5c7cd37ebd	tox.ini: Exclude test_iqiyi_sdk_interpreter.py	2016-03-19 21:50:16 +01:00
Sergey M․	298c04b464	[91porn] Use common messages' wording	2016-03-20 02:35:48 +06:00
Sergey M․	d95114dd83	[91porn] Unquote final URL (Closes #8881 )	2016-03-20 02:34:02 +06:00
Sergey M․	94dcade8f8	Credit @jjatria for biobiochiletv (#7314 )	2016-03-20 01:36:20 +06:00
Sergey M․	fa023ccb2c	[biobiochiletv] Fix extraction, extract m3u8 formats and overall improve (Closes #7314 )	2016-03-20 01:31:55 +06:00
jjatria	e36f4aa72b	[biobiotv] Add extractor	2016-03-20 01:29:08 +06:00
Sergey M․	9261e347cc	Credit @kasper93 for cda (#8805 )	2016-03-19 23:18:04 +06:00
Sergey M․	f1ced6df51	[cda] Improve and simplify (Closes #8805 )	2016-03-19 23:17:14 +06:00
Kacper Michajłow	8b0d7a66ef	[cda] Add new extractor for cda.pl Fixes #8760	2016-03-19 22:42:40 +06:00
Sergey M․	3aec71766d	[safari:api] Separate extractor (Closes #8871 )	2016-03-19 22:30:48 +06:00
Sergey M․	16a8b7986b	[downloader/fragment] Document fragment_retries	2016-03-19 20:54:21 +06:00
Sergey M․	617e58d850	[downloader/{common,fragment}] Fix total retries reporting on python 2.6	2016-03-19 20:51:30 +06:00
Sergey M․	e33baba0dd	[downloader/dash] Add fragment retry capability YouTube may often return 404 HTTP error for a fragment causing the whole download to fail. However if the same fragment is immediately retried with the same request data this usually succeeds (1-2 attemps is usually enough) thus allowing to download the whole file successfully. So, we will retry all fragments that fail with 404 HTTP error for now.	2016-03-19 20:42:23 +06:00
Sergey M․	721f26b821	[downloader/fragment] Add report_retry_fragment	2016-03-19 20:41:24 +06:00
Sergey M․	52bb437e41	[options] Add --fragment-retries option	2016-03-19 20:40:36 +06:00
Jaime Marquínez Ferrándiz	782b1b5bd1	[utils] lookup_unit_table: Match word boundary instead of end of string	2016-03-19 11:44:49 +01:00
Sergey M․	0d769bcb78	[extractor/generic] Fix missing byte literal prefix	2016-03-19 05:43:43 +06:00
remitamine	4cd70099ea	[hbo] Add new extractor	2016-03-18 21:18:18 +01:00
Jaime Marquínez Ferrándiz	09fc33198a	utils: lookup_unit_table: Use a stricter regex In parse_count multiple units start with the same letter, so it would match different units depending on the order they were sorted when iterating over them.	2016-03-18 19:23:06 +01:00
Sergey M․	4c3b16d5d1	[test_YoutubeDL] Add test for format_id format selection	2016-03-19 00:04:26 +06:00
John Peel	d5aacf9a90	Added format_id to the filers on -f.	2016-03-18 23:59:24 +06:00
Sergey M․	19e2617a6f	[commonprotocols] Add generic support for rtmp URLs (Closes #8488 )	2016-03-18 23:42:15 +06:00
Sergey M․	edd9b71c2c	[extractor/generic] Add a test for m3u playlist served without proper Content-Type	2016-03-18 22:49:11 +06:00
Sergey M․	5940862d5a	[extractor/generic] Detect m3u playlists served without proper Content-Type	2016-03-18 22:45:28 +06:00
Sergey M․	de6c51e88e	[extractor/generic] Fix direct link semantics	2016-03-18 22:43:07 +06:00
Sergey M․	303dcdb995	[extractor/generic] Simplify upload_date extraction	2016-03-18 22:41:16 +06:00
Sergey M․	20938f768b	[extractor/generic] Add another test for generic m3u8	2016-03-18 21:54:33 +06:00
Sergey M․	955737b2d4	[extractor/generic] Force Content-Type to lowecase	2016-03-18 21:50:44 +06:00
Sergey M․	263eff9537	[extractor/generic] Properly extract format id from Content-Type Fixes extraction for cases like: audio/x-mpegURL; charset=utf-8	2016-03-18 21:50:10 +06:00
Sergey M․	cae21032ab	[theplatform] Improve geo restriction detection	2016-03-18 21:08:25 +06:00
remitamine	6187091532	[once] check http formats availability	2016-03-18 11:51:34 +01:00