4 * Fix: --format=best not working with youtube (closes issue #39)
5 # Thanks to Peter Baranyi for the fix
12 * Fix: liveleak: "error: Unsupported protocol (http/1)" (closes #36)
13 * Fix: vimeo: title parsing (Thanks to Peter Baranyi for the fix, closes #37)
14 - Fixes title issues with accented chars
15 * Fix: break: http/403 (Thanks to Werner Elsler for the fix, closes #38)
17 Developer-visible changes:
18 * t/11redtube.t: fix dead link
25 * Add: --print-fname option
26 * Fix: dailymotion ID parsing
27 * Fix: --hosts: dailymotion: spak-mini -> spark-mini
30 Developer-visible changes:
31 * Correct copyright years and notices in source code
32 * tlinks/youtube-utf8.url: fix dead link
40 # --stream-pass option (port from cclive)
46 * Thanks to Patrick Hoffmann <patrick@sobran.de> for data mining
48 # Misc. typos in manual
49 # Redtube support (videoid/title parsing issues)
50 # Dailymotion: spak-mini -> spark-mini
52 # replace dashes ("-") with underscores ("_") in video ids
54 Developer-visible changes:
66 # t/02youtube.t: test link (dead link)
67 # tlinks/ytube.url: ditto
68 # tlinks/multi.url: youtube: ditto
78 1. Support spiegel.de (closes issue #32) [thanks to anon for data]
79 2. Support golem.de (closes issue #33) [thanks to anon for data]
80 3. ~/.clive/config path support
81 4. --cookie-jar option
82 5. Support for ehrensenf.de (closes issue #34) [thanks to bjoern for data]
83 * Improve: unicode support (related to issue #29)
84 1. cctv: no longer requires -C and -R options (obsoletes -R)
85 2. --cclass: now works as expected with unicode as well
86 * Improve: backwards compatibility with clive 2.0, 2.1
87 1. Allow --format=(mp4|hd|hq|3gp) (closes Debian bug #535957)
88 2. Restore ~/.config/clive/config path support (closes Debian bug #535483)
89 3. Add --upgrade-config (2.0/2.1 config to 2.2+ format)
91 1. --hosts prints now supported formats
92 2. Pair --exec with --exec-run, the latter now triggers the former
93 2.1. Closes Debian bug #520520
95 3.1. ~/.clivelast -> ~/.cache/clive/last
96 3.2. ~/.clivecache -> ~/.cache/clive/cache
97 4. If HOME environment variable undefined, assume current workdir
98 5. --version: print locale etc.
100 1. --version: copyright years
101 2. File suffix for CCTV (.x-shockwave-flash -> .flv)
102 3. Remove "see examples" from pod markup (closes issue #30)
103 4. Prevent ".$suffix" output filenames
104 4.1. Use $id if !$title && %i was not defined in --filename-format
105 4.2. Related to issue #29
106 5. --no-proxy: really disable all proxy use (closes issue #25)
107 5.1. http_proxy setting was still used
108 6. Can't call method "filename" on unblessed reference at Exec.pm line 5x
109 6.1. e.g. clive URL --exec="ls;"
110 6.2. Closes Debian bug #535459
111 7. Dailymotion support (closes issue #35)
112 8. Redtube support (error: no match: `(?-xism:videoid=(.*?)')')
114 1. --recall, --recall-file -> --last, --last-file
115 2. CLIVE_HOME -> CLIVE_CACHE
117 1. -R, --raw (obsoleted by improved unicode support)
119 2.1. Missleading name
120 2.2. Obsoleted by --cache-file and --last-file
122 1. Add note about numeric suffix (closes issue #28)
123 2. Fix a number of typos, use better grammar
126 Developer-visible changes:
129 * clive/Host/Spiegel.pm
130 * clive/Host/Golem.pm
131 * clive/Host/Ehrensenf.pm
137 * tlinks/ehrensenf.url
138 * New prerequisite: Config::Tiny
139 1. Required by --upgrade-config
141 1. Extract VERSION from lib/clive/Config.pm
142 2. Add "use warnings/strct"
149 User-visible changes:
150 * Change: Google: --format=mp4/best
151 1. Default to flv if mp4 is not available
152 2. Fix --format=best support
154 Developer-visible changes:
155 * Add: t/$host.t: NO_INTERNET variable support (closes issue #24)
156 * Change: t/02youtube.t test link
158 * Remove: t/06lastfm.t
159 1. t/02youtube.t: includes the same test
165 User-visible changes:
166 * Add: support for Vimeo (closes issue #19)
167 * Add: return codes for each error case (closes issue #22)
168 * Fix: return code is always 0 (closes issue #20)
169 * Change: --hosts output now matches cclive output
171 Developer-visible changes:
172 * Change: Cctv, Dailymotion, Liveleak -> use clive::Utils::matchRegExps
182 This release is a major overhaul aimed to fix the previous design flaws
183 and clean up the codebase. Note that 2.2.0 breaks compatibility with
184 the earlier versions of clive.
186 Most users will not notice much differences after upgrading to 2.2.0
187 but those users who have used clive for anything more than "clive URL"
188 should read the changes carefully. The summary of changes section
189 includes more detailed changes that is recommended reading for all
190 users and maintainers.
192 2.2.0 changes the license from ISC to GPLv3. The license was last
193 changed in 2.1.0 to ISC but has now been reverted back to GPLv3 after
194 some further consideration.
196 Config::Tiny has been replaced with Getopt::ArgvFile. The latter had
197 some advantages over Config::Tiny that lead to the switch. For example,
198 instead of trying to memorize the (often confusing) config variable
199 names, users can now use command line options in the config file.
201 This also means that everytime a new feature is added to the program,
202 we are no longer required to modify the code responsible for parsing
203 the config file. Using Getopt::ArgvFile also required adding only one
204 line of code to the project whereas Config::Tiny required several.
206 cache no longer reads by default. This means that --cache-read option
207 must be invoked for clive to read previously stored video records
208 from the cache. The change was made after seeing how most users found
209 the reading from cache too confusing and frequently reported expired
210 link errors such as HTTP 403 as bugs even though the behaviour was
211 documented in the manual page. DAERTM?
213 --emit-csv now outputs very few details about the videos. Only those
214 fields that are known to be any use are printed out.
216 2.2.0 also removes a number of less used features that have lingered
217 in the project since the 1.x. Most of these features were never
218 requested and have only been burdening the program codebase since
221 Part time hackers and/or developers will notice that clive now follows
222 a new OO design -- as well as Perl5 allows it. While there are still
223 some considerations regarding the design and limitations that could not
224 quite be ported from C++ used in cclive, WYSIWYG.
226 Package/port maintainers will want to read the README file. The previously
227 used GNU Makefile is no longer used and clive installation now depends on
228 ExtUtils::MakeMaker instead.
233 User-visible changes:
237 2. Last changed in 2.1.0 (-> ISC)
239 * Slight improvements to program startup time
241 * Config file format changes (Config::Tiny -> Getopt::ArgvFile)
242 1. Command line options can now be used in config file
243 2. Breaks compatibility with previous versions
244 3. Cleaner and new format allows using cmdline options in configs
245 4. Format was last changed in 2.0.0
247 -----------------------------------------------
248 # Config::Tiny: clive 2.0 - 2.1
249 cat >> ~/.config/clive/config
251 proxy = "http://foo:1234"
253 savedir = "/home/user/videos"
255 # Getopt::ArgvFile: clive 2.2
257 --proxy="http://foo:1234"
258 --savedir="/home/user/videos"
259 -----------------------------------------------
261 * Fix: Redtube video title parsing
264 1. Cache is now passive (read:no, write:yes)
265 2. Add: --cache- option prefix
266 3. New option: --cache-read, --no-cache
267 5. Rename: misc. options (e.g. --show -> --cache-dump)
268 6. Record field changes
269 o Breaks compatibility with previous versions
270 7. New field delimiter '#'
274 1. ~/.config/clive/config -> ~/.cliverc
275 2. ~/.config/clive/recall -> ~/.clivelast
276 3. ~/.config/clive/cache -> ~/.clivecache
279 1. Add: --format=best support (closes issue #15 )
280 2. Rename: mp4 -> fmt18 (Youtube)
281 3. Rename: Dailymotion ID spark -> flv
284 1. cURL error handling
285 2. Google mp4 support
286 3. Do not strip link params (closes debian bug #530659)
287 4. --format=fmt6 (no longer supported by Youtube?)
288 5. Dailymotion ID parsing
296 6. --stop-after (closes issue #18)
298 * --emit-csv: changes to CSV fields (cleanup, print only the necessary fields)
300 page_link, video_link, filename, file_length_mb,
301 file_length_bytes, video_id, time_stamp, page_title,
302 initial_length, remaining_bytes
304 base_filename, file_length, video_link
308 2. --renew (now obsolete)
309 3. --youtube-user (broken since 2.1)
310 4. --youtube-pass (...)
322 1. Long options: aliases (e.g. --output_file | --output-file | --outputfile)
323 2. --filename-format: new specifiers, rename some of the old ones
324 3. --format: exit with an error if id is not recognized by clive
325 4. Rename: -r -> -l (--recall)
326 5. Many short options have been removed (or reused)
328 Developer-visible changes:
332 * Change: use a custom perltidyrc (closes issue #17)
333 - Thanks to Tong Sun <suntong001@gmail.com>
335 * Prerequisite changes
336 1. Written for Perl 5.10.0
337 o 5.8 probably OK although unverified
341 3. Removed prerequisites
346 4. Optional modules (no changes from 2.1.14)
350 * Change: use decode_utf8 returned html only if html is valid utf8
352 * Installation changes
353 1. Makefile.PL (ExtUtils::MakeMaker) [see README]
356 1. Makefile (obsoleted by Makefile.PL)
357 2. clive.pod (merged into bin/clive)
358 3. clive.1 (generated from bin/clive)
360 * Remove: binmode STDOUT, ":utf8"
362 * Remove: extra decode_utf8
368 User-visible changes:
369 * Add: support for youtube-nocookie.com (closes issue #12)
370 * Add: fmt35 format ID for Youtube [closes issue #10 (and #13)]
371 * Fix: liveleak id parsing
372 * Rename: --format IDs for Youtube
376 * Rename: --output-video -> --output-file
377 * Rename: output:file -> output:filename_format (config file)
378 * Remove: --overwrite option (use --output-file instead)
380 Developer-visible changes:
381 * Add: tests/ytube-nocookies.url
385 * 2.2.0 is under development, read more here:
386 http://code.google.com/p/clive/wiki/WhatsNew_2_2_0
392 Developer-visible changes:
393 * Change: use perltidy defaults for style
394 * Change: BerkeleyDB and File::Path modules are now optional
395 ** Cache features depend on these modules
396 * Moved opted_mods initilization to init_opted_mods (-> loop)
402 User-visible changes:
403 * Change: --connect-timeout, --connect-timeout-socks default -> 30s
404 * Fix: timeout after file transfer starts
405 * Fix: title parsing with custom character class (closes issue #8)
406 * Add: redtube support; thanks to anonprn2@gmail.com (closes issue #11)
407 * Fix: typo in --emit-xml --help string
413 User-visible changes:
414 * Change: --youtube-pass: prompt for password if left undefined
415 * Change: --filename-format: default -> "%t.%s"
416 * Fix: "Use of uninitialized value $opts{"ytpass"} ..."
419 ** --connect-timeout-socks
420 * Cleanup: manual page
422 Developer-visible changes:
423 * Rename: auth_youtube subroutine -> loginto_youtube
426 * Youtube login does not work (throw an error, patches welcome!)
432 NOTE: This release changes the clive status messages. This will likely
433 break compatibility with utilities that depend on these messages.
434 The changes were made to enable using abby (0.2.0+) as a front-end
437 User-visible changes:
438 * Add abby (front-end) compatibility
439 - Change output similar to cclive
440 - http://code.google.com/p/abby/
441 * Add Youtube HD support (--format=mp4_hd)
442 - Thanks to bf for the patch (via FreeBSD ports)
443 * Add tv.cctv.com support [issue #6]
444 - Thanks to suntong001 for data mining
446 - --limit-rate (config: [http]:limit_rate)
447 - --hosts (print supported hosts to stdout)
448 - --stderr (redirect all output to stderr instead of stdout)
449 * Improve file suffix parsing from content-type
450 - Rewrite the content-type handling block in query_video_length
451 - Use whatever is found after slash in the content-type string
452 - Use "flv" for exceptions (e.g. octet-*, x-flv, etc.)
453 * Replace dashes in video IDs with underscores before using them in fnames
454 * Change --version output
455 * Fix reading from stdin (closes Debian #522422)
456 - Assume reading from stdin only if queue and argv arrays are empty
457 * Fix exit with status code
458 - Exit 0 on success, and >0 if an error occurs
460 Developer-visible changes:
461 * Update examples/config with limit_rate
462 * Rename: configrc -> examples/config
463 * Replace the monstrous if-elsif-$host code block with a loop
469 User-visible changes:
470 * Fix GoogleVideo "id not found" handling
471 - Symptom: exit after "process page..." message
472 - Cause: no check for undefined id value
473 * Fix GoogleVideo support
474 - Symptom: see above fix
475 - Cause: HTML changes to the video pages
476 * Fix URI::Escape import for GoogleVideo redirects
477 - Symptom: crash with "Undefined subroutine &URI::Escape::uri_unescape"
478 - Cause: recent script start optimizations; a missing 'require'
480 - Symptom: "expected different content-type"
481 - Cause: website content-type changes
483 Developer-visible changes:
484 * Cleanup code (style-wise)
485 - perltidy -gnu -l=72
491 User-visible changes:
492 * Keep URL queue order
493 - Remove duplicate URL checking
494 * Remove --modver, change --version output
495 * Update manual page (clive.1, clive.pod)
496 - Rewrite EXAMPLES section
497 - Cleanup other sections
498 * Config file: add support for [output]:format (--format)
499 * Port dailymotion support from cclive
501 Developer-visible changes:
503 - Replace Perl module checking with for-loops (incl. optional)
504 * Optimize script start time
505 - Replace "use $module" with "require $module" where possible
506 * Update LICENSE, configrc
508 - Add [output]:format to configrc
509 * Add tests/dmotion.url
515 User-visible changes:
516 * Add --output-video option
517 * Add option groups to --help output
518 * Add --stream-exec and --stream options
520 * Config file: add support for
521 - [commands]:exec (--exec)
522 - [commands]:stream (--stream-exec)
523 - See configrc for an example
524 * Fix --exec; should no longer spew out "not found" shell errors
525 * Change default filename format to "%t_%d_%i.%s"
526 * Change to use "file is already fully retrieved" error message
527 * Port resizing progressbar from cclive
529 Developer-visible changes:
530 * Add tests/lastfm.url and tests/multi.lst
531 * Remove XML::Simple dependency
533 * Implement SIGWINCH handling
534 - Resize progressbar to match terminal width
535 - Progressbar width defaults to 80 (chars)
536 - OS must support SIGWINCH
537 - Requires Term::ReadKey module
538 - clive works without the module but will ignore any SIGWINCH events
539 * Touch cclive.1, cclive.pod, INSTALL and Makefile files
543 * Buried clive-list, grew weary of fighting spam with bare fists
544 - email the author directly or use the issue tracker, thanks
550 User-visible changes:
551 * Renamed the following options
552 - --noproxy => --no-proxy
553 - --noextract => --no-extract
554 - --nologin => --no-login
555 * Butchered --help output
556 - much less verbose now
557 - relocated manual page data to clive.pod
558 * Removed --manual option
559 * Obsoleted "play" and "rencode" config keys in "commands" section
560 * Added --exec option which resembles cclive's --exec command
561 * Removed --play, --noplay, --rencode and --norencode options
563 * Changed --paste short option from '-x' to '-p'
564 * Removed wmv format (breakcom)
565 - persistant HTTP/404 (not found) error
567 Developer-visible changes:
569 - additional INSTALL variables
571 - can be used to disable module checks
572 - removed "man" from "all"
573 - removed "clean" (useless)
579 User-visible changes:
580 * Fixed sevenload support
586 User-visible changes:
587 * Fixed Youtube login which sometimes reported failed attempts as OK
588 - Youtube defaults web interface language based on users ip
589 - clive looks for English words in the post-login page to confirm login
590 * Tweaked --version output
592 Developer-visible changes:
593 * Removed metacafe remnants
594 * Fixed bug in time2str subroutine which caused incorrect calculations
596 * removed genman script (obsoleted by "make man")
597 * Added a rudimentary (GNU) Makefile
598 * Cleaned up translate_embed subroutine
601 * Removed the clive-announce and clive-users mailing lists
602 - New release announcements will be posted via Freshmeat alone from now on
603 - Subscribe to the project at http://freshmeat.net/projects/clive
604 * Created the clive-list@googlegroups.com mailing-list
605 - For support questions, patches and bug reports
606 - Issue tracker will remain for tracking bugs
607 - Send an email to clive-list-subscribe@googlegroups.com to subscribe
613 User-visible changes:
614 * Documented missing "%t" (--filename-format) in manual page
615 * Tweaked "localfile_length == remotefile_length" message
616 * Renamed CLIVE_CONFIGDIR environment variable to CLIVE_HOME
618 Developer-visible changes:
619 * Cleaned up manual page
621 * Cleaned up FreeBSD details (INSTALL)
627 User-visible changes:
628 * Removed "See --manual" from --version output
629 * Added support for evisortv
630 - http://code.google.com/p/clive/issues/detail?id=4
631 * Fixed: reused incorrectly last http error code for remaining urls in queue
632 * Fixed: progressbar reported 100% even if error had occurred (e.g. http/403)
634 Developer-visible changes:
635 * Cleaned up subroutines
641 User-visible changes:
642 * Fixed "Requested range was not delivered by the server (http/33)" w/ -c
643 - this would occur with a batch of URLs while using the --continue option,
644 e.g. if the first file was continued, the following page fetches exited
646 * Added --savebatch option
647 * Added --overwrite option
648 * Replaced the buggy progressbar with a "lazy man's progressbar"
649 * Fixed "print() on closed filehandle STDOUT" with --grep+delete
650 * Fixed progressbar display with --continue
652 Developer-visible changes:
654 * Started to bundle src tarball with clive.1 manual page
655 * Added genman script which wraps pod2man with common options
656 * Fixed a potential div by zero bug in calc_progress
662 NOTE: Changed license GPL3 => ISC/OpenBSD.
664 User-visible changes:
665 * Added --modversion option
666 * Replaced Term::Progressbar with a built-in thermometer progressbar
667 * Removed --progress=bar:nominor support due to above changes
668 * Added liveleakcom support
669 - http://code.google.com/p/clive/issues/detail?id=3
671 Developer-visible changes:
672 * Made IO::Pager module optional
674 * Cleaned up --progress option description
675 * Changed to require WWW::Curl 4.05+
676 * Removed Term::Progress module dependency
677 * Fixed a potential bug in calc_progress (improper $rate init)
679 * Made Expect module optional
685 User-visible changes:
686 * Fixed gvideo/mp4 support (http/404 even if the extraction link existed)
687 * Changed "found redirect" message
688 * Changed --play, --rencode status messages to use file basename
690 Developer-visible changes:
691 * decode_utf8($page): removes extra steps, works better with HTML::TokeParser
692 * Added missing Expect module version to INSTALL
699 This aims to be the last beta before the release of 2.0 in December 2008.
700 Please report any remaining bugs to the issue tracker at:
701 http://code.google.com/p/clive/issues/list
703 User-visible changes:
704 * Removed metacafe support until fixed (see known issues)
705 * Changed Youtube login to report if there were "too many login failures"
706 * Fixed Youtube login support: reported failures incorrectly as OK
707 * Added support clivepass(1) utility (http://code.google.com/p/clive-utils)
708 * Added support for metacafe and sevenload embed URL translation
710 Developer-visible changes:
712 * New prerequisite module: Expect
713 * Updated FreeBSD + Ubuntu details (INSTALL)
714 * Cleaned up some code
715 * Rewrote embed URL translation
718 * Metacafe support broken (http/404)
724 User-visible changes:
725 * Fixed pasting from clipboard containing multiple URLs
726 * Fixed page fetch progress glitch that occured with 1+ batches
727 * Added --agent option
728 * Added --proxy option
729 * Added --savedir option
730 * Added --cclass option
731 * Added --filename-format option
732 * Added --show-format option
733 * Added --youtube-user and --youtube-pass options
734 * Added --emit-xml option
735 * Added --progress=[none|bar|dot]
736 * Replaced --noprogress with --progress=none
737 * Added avg. transfer rate notification
738 * Fixed filename printing for existing output files (e.g. output.flv.1)
739 * Added --play option
740 * Fixed metacafe support (error: failed to extract videoCDNURL)
741 * Added --rencode option
743 Developer-visible changes:
744 * Ported --emit-csv option from 1.x with minor changes
745 - Backwards compatible with 1.x
746 - Prints additional video details
747 - $length, $video_id, $time_stamp, $page_title, $cont_from, $remaining
748 * Changed to prefer 'dot' bar if the output is not a TTY
749 * Replaced progress bar with Term::ProgressBar (new module prerequisite)
750 * Cleaned up queue processing subroutine
756 User-visible changes:
757 * clive now ignores input lines that start with the '#' character
758 * Added support for CLIVE_CONFIGDIR environment variable
759 * Ported lastfm video support from 1.x
760 * --version now displays XML::Simple
761 * Changed "= Play:" to "=> Playing ..."
762 * Playing occurs now subsequently after going over the URL batch
764 Developer-visible changes:
765 * Removed a google video redirect (commented out, see code)
766 * Added the AUTHORS file
767 * XML::Simple was missing as a prerequisite in INSTALL
768 * Made minor syntactic tweaks to the source code
769 * Fixed missing --quiet check with the "=> Play" message
775 A complete overhaul and rewrite of 1.x.
777 User-visible changes:
778 * Users are expected to use the --continue/-c option to resume transfers
779 * Configuration file format (=> INI), path was changed (=> ~/.config/clive)
780 * All extractions now assume flv as the default download format
781 * Users are expected to understand when/how use the --format/-f option
782 * Reading input from multiple sources is possible (e.g. % clive -x URL URL)
783 * Added: --grep: used to grep and recall cached URL entries
784 * Added: --background and --output=logfile, --append=logfile options
785 * Many old cmdline and config options were removed due to new design
786 * Improved GoogleVideo support: handles redirects to other hosts
787 * Pager is used where needed (e.g. --show/-s)
788 * Changed default output file naming format to "%n-(%i)-[%d].%s"
789 * Removed: guba (deadweight), myvideo (borked), dmotion (bitches @ anon users)
791 Developer-visible changes:
793 - See the INSTALL file for prerequisites
794 * No installation: all-in-one script (excl. the 3rd party modules)
795 * Cache is now an integral part of clive, see CACHE in the manual page
796 * Embedded the --manual page (Pod)
797 * Improved Youtube login: removed is_adult HTTP get (done manually now)
798 * Grab the file suffix from the Content-Type header
799 * Improved GoogleVideo support: mp4 detection no more relies on English words
800 * Improved Metacafe support: removed unnecessary "familyfilter" HTTP get
804 - 2.x+ at http://googlecode.com/p/clive/
805 - 1.x remains at http://gna.org/projects/clive
807 - 2.x+ at http://googlecode.com/p/clive/
808 - 1.x remain at http://dl.gna.org/clive
810 - http://clive.sf.net/
811 - http://home.gna.org/clive redirects to the above
813 - 2.x+ at git://repo.or.cz/clive.git
814 - 1.x remains at svn://svn.gna.org/svn/clive
815 * Issue tracking: Moved
816 - 2.x+ at http://googlecode.com/p/clive/issues/
817 - 1.x bug tracker remains at http://gna.org/projects/clive
818 * Created http://googlegroups.com/group/clive-announce/
820 NOTE: See http://dl.gna.org/clive/CHANGES for 1.x.