Update NEWS for v0.9.20131130

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Merge branch 'tg/next__fix_media/vimeo' into next

tests: media_vimeo: Run additional checks

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: media/vimeo: Re-enable

FIX: media/vimeo: Reimpl. support

  - Use HTTPS if the input URL was given using it [1]
  - Improve stream identification (incl. bitrate)
  - Parse media properties using LuaJSON
  - Parse data from a new location
  - Update all relevant patterns

Notes:
[1]: media streams are still provided over HTTP
[2]: http://thread.gmane.org/gmane.comp.web.flash.quvi/358

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Merge branch 'tg/next__fix_media/publicsenat' into next

Merge branch 'tg/next__101gg_fixes' into next

tests: media_101greatgoals: Update test URL

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

FIX: media/101greatgoals: redirect: Add URL scheme

Make sure the redirection URL has a scheme.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

FIX: media/publicsenat: Reimpl. support

Notable changes:
  - Fetch and parse media properties from an emission URL
  - Use LuaJSON to parse the properties ({stream,thumb} URL)

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: media_spiegel: Update test URL

The old one no longer has thumbnail available.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: media_cbsnews: Update URLs

Other:
  - Do not test the resolution properties (no longer returned)

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

FIX: media/cbsnews.lua: Adapt to website changes

Adapt to the recently updated website design.
  - Parse all media properties from video page HTML
  - Use LuaJSON to  parse the media properties

Notable changes:
  - Media streams may now contain {HTTP,RTMP,M3U8} URLs [1]
  - Presume the "best" quality to be the "desktop" stream
  - Switch to support the new video URI paths

Notes:
[1]: None of which are currently identified in any other way than
     setting the "container" property which is simply parsed from the
     URI file extension (e.g. "m3u8" or "mp4"), quvi-get(1) cannot
     currently handle the m3u8/hls (or rtmp for that matter)

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: media_canalplus: Update test URLs

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/canalplus.lua: Relicense under AGPLv3+

Notable differences to the 0.4 script:
  - Supports also d8.tv and d17.tv which share the same video database
  - Uses the JSON interface provided by Canal+

Differences (:editor:) to the original patch[1]:
  - Concatenate strings using a table, or "the Lua way"[2]
  - Remove the old script from the repo
  test:
  - Update expected media {title,ID} for the first test URL
  - Rename test function to `test_media_canalplus'

Links:
[1]: http://thread.gmane.org/gmane.comp.web.flash.quvi/364
[2]: http://www.luafaq.org/#T1.9

Signed-off-by: Mohamed El Morabity <melmorabity@fedoraproject.org>
Edited-by: Toni Gundogdu <legatvs@gmail.com>

share/README: Add an URL to "Lua Unofficial FAQ (uFAQ)"

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Merge branch 'tg/next__media_port_dorkly.lua' into next

Merge branch 'tg/next__media_port_collegehumor.lua' into next

tests: Add a test for media/dorkly.lua

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Port website/dorkly.lua from the 0.4 series

Notable changes:
* Use LuaExpat for parsing the media properties
* Apply 2-space indentation
* Relicense under AGPLv3+
* Update vim modeline

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: Add a test for media/collegehumor.lua

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/collegehumor.lua: Rewrite parse function

Rewrite the `parse' function for the libquvi 0.9+ series.
* Remove now obsolete code (e.g. self.host_id)
* Apply 2-space indentation
* Update vim modeline

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/collegehumor.lua: Remove query_formats function

Remove the now obsolete `query_formats' function which is no longer
supported by libquvi 0.9+.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/collegehumor.lua: Rewrite ident function

Rewrite the `ident' function for the libquvi 0.9+ series.
* Implement `CollegeHumor.can_parse_url' function
* Remove any now obsolete code

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/collegehumor.lua: Relicense under AGPLv3+

See also:
* http://article.gmane.org/gmane.comp.web.flash.quvi/334

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Move website/collegehumor.lua to media/

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Update NEWS for v0.9.20131104

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/vimeo.lua: Mark as FIXME

Notes:
* http://thread.gmane.org/gmane.comp.web.flash.quvi/358
* Affects: {0.4, 0.9}

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: media_liveleak: Replace URL for embedded youtube content

The previously used video was removed from the liveleak website.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/lego.lua: Update ID pattern

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Merge branch 'tg/next__media_port_videobash.lua' into next

Merge branch 'tg/next__media_port_tagtele.lua' into next

FIX: media/vimeo.lua: Vimeo.normalize: Escape pattern

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: Add a test for media/videobash.lua

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: Add a test for media/tagtele.lua

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/tagtele.lua: Rewrite parse function

Rewrite the `parse' function for the libquvi 0.9+ series.
* Remove now obsolete code (e.g. self.host_id)
* Use LuaJSON to parse the media stream URL
* Parse all available media streams
* Apply 2-space indentation
* Update vim modeline

Notable changes:
* Skip: playlist fetch: the page HTML contains the stream URL data

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/tagtele.lua: Remove query_formats function

Remove the now obsolete `query_formats' function which is no longer
supported by libquvi 0.9+.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/tagtele.lua: Rewrite ident function

Rewrite the `ident' function for the libquvi 0.9+ series.
* Implement `Tagtele.can_parse_url' function
* Remove any now obsolete code

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/tagtele.lua: Relicense under AGPLv3+

See also:
* http://article.gmane.org/gmane.comp.web.flash.quvi/340

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/videobash.lua: Rewrite parse function

Rewrite the `parse' function for the libquvi 0.9+ series.
* Remove now obsolete code (e.g. self.host_id)
* Apply 2-space indentation
* Update vim modeline
* Parse `duration_ms'

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/videobash.lua: Remove query_formats function

Remove the now obsolete `query_formats' function which is no longer
supported by libquvi 0.9+.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/videobash.lua: Rewrite ident function

Rewrite the `ident' function for the libquvi 0.9+ series.
* Implement `Videobash.can_parse_url' function
* Remove any now obsolete code

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/videobash.lua: Relicense under AGPLv3+

See also:
* http://article.gmane.org/gmane.comp.web.flash.quvi/336

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Move website/videobash.lua to media/

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Move website/tagtele.lua to media/

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Merge branch 'tg/next__media_port_metacafe.lua' into next

tests: Add test for media/metacafe.lua

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/metacafe.lua: Rewrite parse function

Rewrite the `parse' function for the libquvi 0.9+ series.
* Rewrite `Metacafe.redirectp' as `Metacafe.is_affiliate'
* Remove now obsolete code (e.g. self.host_id)
* Use LuaJSON to parse the media stream URL
* Parse all available media streams
* Apply 2-space indentation
* Parse duration (ms)
* Update vim modeline

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/metacafe.lua: Remove query_formats function

Remove the now obsolete `query_formats' function which is no longer
supported by libquvi 0.9+.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/metacafe.lua: Rewrite ident function

Rewrite the `ident' function for the libquvi 0.9+ series.
* Implement `Metacafe.can_parse_url' function
* Remove any now obsolete code

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/metacafe.lua: Relicense under AGPLv3+

See also:
* http://article.gmane.org/gmane.comp.web.flash.quvi/334

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Move website/metacafe.lua to media/

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/arte.lua: Arte.to_id: Return ID in lowercase

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/arte.lua: Use the 'a?b:c' -idiom

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Fix support for arte.tv

* Videos are now hosted on arte.tv/guide; for new videos, there's a
  redirection from videos.arte.tv to arte.tv/guide

* For a given video, all stream datas are now provided by a JSON file
  instead of a XML file as before

* In this JSON data file, an ID (versionProg) describes the language of
  each available stream:

    versionProg == 1 -> default language version, matching the video URL
                        language

    versionProg == 2 -> alternate language version

    versionProg == 3 -> original version with default language subtitles

    versionProg == 8 -> default language version with subtitle for
                        hard-of-hearing

* I took the liberty of adding the stream protocol to the generated
  stream ID, to distinguish same-quality videos (some videos are
  available through HTTP, in addition of RTMP, for some MD/LD videos,
  probably for mobile phones

* The tests were updated to reflect these changes

Signed-off-by: Mohamed El Morabity <melmorabity@fedoraproject.org>
Reviewed-by: Toni Gundogdu <legatvs@gmail.com>

cleanup: table.concat: Do not call with an empty sep value

As this is not required, nor a required arg.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Merge branch 'tg/next__add_support__media/majestyc.lua' into next

Merge branch 'tg/next__add_support__{media/scan}/vzaar.lua' into next

Merge branch 'tg/next__media_port_tapuz.lua' into next

tests: Add a test for scan/vzaar.lua

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

scan: Add support for embedded vzaar.com media URLs

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: Add a test for media/vzaar.lua

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media: Add support for vzaar.com

See also:
* https://www.mail-archive.com/quvi-devel@lists.sourceforge.net/msg00346.html

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: Add a test for media/majestyc.lua

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media: Add support for tube.majestyc.net

See also:
* https://www.mail-archive.com/quvi-devel@lists.sourceforge.net/msg00347.html

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: Add a test for media/tapuz.lua

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/tapuz.lua: Rewrite parse function

Rewrite the `parse' function for the libquvi 0.9+ series.
* Use LuaExpat to parse the media {title,stream,thumb} properties
* Remove now obsolete code (e.g. self.host_id)
* Apply 2-space indentation
* Update vim modeline
* Parse duration (ms)

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/tapuz.lua: Remove query_formats function

Remove the now obsolete `query_formats' function which is no longer
supported by libquvi 0.9+.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/tapuz.lua: Rewrite ident function

Rewrite the `ident' function for the libquvi 0.9+ series.
* Implement `Tapuz.can_parse_url' function
* Remove any now obsolete code

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/tapuz.lua: Relicense under AGPLv3+

See also:
* http://article.gmane.org/gmane.comp.web.flash.quvi/335

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Move website/tapuz.lua to media/

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/liveleak.lua: Ignore fac3book iframe src values

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Update NEWS for v0.9.20131012

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Merge branch 'tg/next__media_port_1tvru.lua' into next

tests: Add a test for media/1tvru.lua

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/1tvru.lua: Rewrite parse function

Rewrite the `parse' function for the libquvi 0.9+ series.
* Check for 'og:video' to determine if input URL contains a video
* Use LuaJSON to parse the media {title,stream,thumb} properties
* Use 'windows-1251' charset encoding with quvi.http.fetch
* Media duration (ms) is now parsed and returned
* Remove now obsolete code (e.g. self.host_id)
* Apply 2-space indentation
* Update vim modeline

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/1tvru.lua: Remove query_formats function

Remove the now obsolete `query_formats' function which is no longer
supported by libquvi 0.9+.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/1tvru.lua: Rewrite ident function

Rewrite the `ident' function for the libquvi 0.9+ series.
* Allow HTTPS input URLs (media streams will remain HTTP nonetheless)
* Allow '/sprojects_utro_video/' in addition to '/sprojects_edition/'
* Implement `OTvRu.can_parse_url' function
* Remove any now obsolete code

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/1tvru.lua: Relicense under AGPLv3+

See also:
* http://article.gmane.org/gmane.comp.web.flash.quvi/333

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

Move website/1tvru.lua to media/

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

FIX: media/clipfish.lua: Raise "no match: media stream URL"

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

quvi/youtube: normalize: Reimplement

Reimplement using helper functions. Try to improve readability. Add more
test cases.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

FIX: quvi/youtube: ident: query: "v=" may not be first param

Do not assume the URL 'v=' parameter of the 'query' element to be the
first parameter.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

quvi/youtube: normalize: Process YouTube URLs only

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

FIX: media/sevenload.lua: JSON extraction pattern

Otherwise LuaJSON would exit with an error:
../lua/5.1/json/decode/state.lua:151: Value set when one already in slot

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: media_lego: Add a test for "/videos"

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/lego.lua: Add support for "/videos?video=ID"

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: media_youtube: Add test for an embedded media URL

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

FIX: resolve_redirections.lua: Normalize YouTube input URL

libquvi will always attempt to resolve URL redirections.  This means
that quvi_media_new (for example) will always resolve first and only
then pass the input URL to the media scripts to determine the support

Problem:
  - g00gle now redirects the embedded media URLs to the API pages

  - This causes libquvi to return the "no support" because the
    destination URL looks nothing like a typical media page URL

Solution:
  - "normalize" (quvi/youtube) the input URL before we try to resolve
    URL redirections

  - "normalize" will convert the embedded media URLs to media page URLs,
    which can then be passed to libquvi/libcurl for URL resolving

Notes:
  - quvi_supports function skips resolving altogether unless online
    check is forced

  - This is the reason "normalize" must still be called in "ident"
    function of the media script, even if this script does so

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: scan_youtube: Use different shortened test URL

The old one no longer contains any embedded media URLs.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: scan_youtube: Remove obsolete test URL

The page embeds the videos using the now defunct "youtube-nocookie.com"
domain.

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

FIX: media/funnyordie.lua: Set "Accept-Encoding" HTTP header

For an unknown reason when:
1) libcurl sends "Accept-Encoding: deflate, gzip" by default
2) server returns gzipped data without "Encoding: gzip" within the reply

Causing the following:
1) libcurl skips decompression and passes the compressed data to libquvi
2) libquvi passes the compressed data to the media/funnyordie.lua
3) script fails to match any of the patterns due to the compressed data

This commit makes the script override the default "Accept-Encoding"
value with one that seems to work with the servers and libcurl.

Package: libcurl4-gnutls-dev
Version: 7.26.0-1+wheezy3

Versions of packages libcurl4-gnutls-dev depends on:
ii  libc6-dev [libc-dev]  2.13-38
ii  libcurl3-gnutls       7.26.0-1+wheezy3
ii  libgnutls-dev         2.12.20-7
ii  libidn11-dev          1.25-2
ii  libkrb5-dev           1.10.1+dfsg-5+deb7u1
ii  libldap2-dev          2.4.31-1+nmu2
ii  librtmp-dev           2.4+20111222.git4e06e21-1
ii  libssh2-1-dev         1.4.2-1.1
ii  zlib1g-dev            1:1.2.7.dfsg-1

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/liveleak.lua: Parse stream URLs from "config:"

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/liveleak.lua: Sanitize JSON

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

DOC: quvi-object-3rdparty: Add LuaJSON

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: media_liveleak: Add a multistream test

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

media/liveleak.lua: Add support for >1 streams

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

FIX: media/liveleak.lua: Sanitize input for LuaJSON

Otherwise the script will throw an error that looks like:
  /usr/share/lua/5.1/json/decode/util.lua:40: unexpected character @
  character: 733 17:16 [e] line:

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

tests: media_funnyordie: Update URL

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>

FIX: media/funnyordie.lua: Media stream URL pattern

This was fixed by 41a7516647, which removed the pattern that appears to
work again. Keep both patterns.

Other changes:
* Insert only those media URLs that match the quality-container pattern

Signed-off-by: Toni Gundogdu <legatvs@gmail.com>