5 # --stream-pass option (port from cclive)
7 * Thanks to Patrick Hoffmann <patrick@sobran.de> for data mining
9 1. Misc. typos in manual
10 2. Redtube support (videoid/title parsing issues)
18 1. Support spiegel.de (closes issue #32) [thanks to anon for data]
19 2. Support golem.de (closes issue #33) [thanks to anon for data]
20 3. ~/.clive/config path support
21 4. --cookie-jar option
22 5. Support for ehrensenf.de (closes issue #34) [thanks to bjoern for data]
23 * Improve: unicode support (related to issue #29)
24 1. cctv: no longer requires -C and -R options (obsoletes -R)
25 2. --cclass: now works as expected with unicode as well
26 * Improve: backwards compatibility with clive 2.0, 2.1
27 1. Allow --format=(mp4|hd|hq|3gp) (closes Debian bug #535957)
28 2. Restore ~/.config/clive/config path support (closes Debian bug #535483)
29 3. Add --upgrade-config (2.0/2.1 config to 2.2+ format)
31 1. --hosts prints now supported formats
32 2. Pair --exec with --exec-run, the latter now triggers the former
33 2.1. Closes Debian bug #520520
35 3.1. ~/.clivelast -> ~/.cache/clive/last
36 3.2. ~/.clivecache -> ~/.cache/clive/cache
37 4. If HOME environment variable undefined, assume current workdir
38 5. --version: print locale etc.
40 1. --version: copyright years
41 2. File suffix for CCTV (.x-shockwave-flash -> .flv)
42 3. Remove "see examples" from pod markup (closes issue #30)
43 4. Prevent ".$suffix" output filenames
44 4.1. Use $id if !$title && %i was not defined in --filename-format
45 4.2. Related to issue #29
46 5. --no-proxy: really disable all proxy use (closes issue #25)
47 5.1. http_proxy setting was still used
48 6. Can't call method "filename" on unblessed reference at Exec.pm line 5x
49 6.1. e.g. clive URL --exec="ls;"
50 6.2. Closes Debian bug #535459
51 7. Dailymotion support (closes issue #35)
52 8. Redtube support (error: no match: `(?-xism:videoid=(.*?)')')
54 1. --recall, --recall-file -> --last, --last-file
55 2. CLIVE_HOME -> CLIVE_CACHE
57 1. -R, --raw (obsoleted by improved unicode support)
60 2.2. Obsoleted by --cache-file and --last-file
62 1. Add note about numeric suffix (closes issue #28)
63 2. Fix a number of typos, use better grammar
66 Developer-visible changes:
69 * clive/Host/Spiegel.pm
71 * clive/Host/Ehrensenf.pm
77 * tlinks/ehrensenf.url
78 * New prerequisite: Config::Tiny
79 1. Required by --upgrade-config
81 1. Extract VERSION from lib/clive/Config.pm
82 2. Add "use warnings/strct"
90 * Change: Google: --format=mp4/best
91 1. Default to flv if mp4 is not available
92 2. Fix --format=best support
94 Developer-visible changes:
95 * Add: t/$host.t: NO_INTERNET variable support (closes issue #24)
96 * Change: t/02youtube.t test link
98 * Remove: t/06lastfm.t
99 1. t/02youtube.t: includes the same test
105 User-visible changes:
106 * Add: support for Vimeo (closes issue #19)
107 * Add: return codes for each error case (closes issue #22)
108 * Fix: return code is always 0 (closes issue #20)
109 * Change: --hosts output now matches cclive output
111 Developer-visible changes:
112 * Change: Cctv, Dailymotion, Liveleak -> use clive::Utils::matchRegExps
122 This release is a major overhaul aimed to fix the previous design flaws
123 and clean up the codebase. Note that 2.2.0 breaks compatibility with
124 the earlier versions of clive.
126 Most users will not notice much differences after upgrading to 2.2.0
127 but those users who have used clive for anything more than "clive URL"
128 should read the changes carefully. The summary of changes section
129 includes more detailed changes that is recommended reading for all
130 users and maintainers.
132 2.2.0 changes the license from ISC to GPLv3. The license was last
133 changed in 2.1.0 to ISC but has now been reverted back to GPLv3 after
134 some further consideration.
136 Config::Tiny has been replaced with Getopt::ArgvFile. The latter had
137 some advantages over Config::Tiny that lead to the switch. For example,
138 instead of trying to memorize the (often confusing) config variable
139 names, users can now use command line options in the config file.
141 This also means that everytime a new feature is added to the program,
142 we are no longer required to modify the code responsible for parsing
143 the config file. Using Getopt::ArgvFile also required adding only one
144 line of code to the project whereas Config::Tiny required several.
146 cache no longer reads by default. This means that --cache-read option
147 must be invoked for clive to read previously stored video records
148 from the cache. The change was made after seeing how most users found
149 the reading from cache too confusing and frequently reported expired
150 link errors such as HTTP 403 as bugs even though the behaviour was
151 documented in the manual page. DAERTM?
153 --emit-csv now outputs very few details about the videos. Only those
154 fields that are known to be any use are printed out.
156 2.2.0 also removes a number of less used features that have lingered
157 in the project since the 1.x. Most of these features were never
158 requested and have only been burdening the program codebase since
161 Part time hackers and/or developers will notice that clive now follows
162 a new OO design -- as well as Perl5 allows it. While there are still
163 some considerations regarding the design and limitations that could not
164 quite be ported from C++ used in cclive, WYSIWYG.
166 Package/port maintainers will want to read the README file. The previously
167 used GNU Makefile is no longer used and clive installation now depends on
168 ExtUtils::MakeMaker instead.
173 User-visible changes:
177 2. Last changed in 2.1.0 (-> ISC)
179 * Slight improvements to program startup time
181 * Config file format changes (Config::Tiny -> Getopt::ArgvFile)
182 1. Command line options can now be used in config file
183 2. Breaks compatibility with previous versions
184 3. Cleaner and new format allows using cmdline options in configs
185 4. Format was last changed in 2.0.0
187 -----------------------------------------------
188 # Config::Tiny: clive 2.0 - 2.1
189 cat >> ~/.config/clive/config
191 proxy = "http://foo:1234"
193 savedir = "/home/user/videos"
195 # Getopt::ArgvFile: clive 2.2
197 --proxy="http://foo:1234"
198 --savedir="/home/user/videos"
199 -----------------------------------------------
201 * Fix: Redtube video title parsing
204 1. Cache is now passive (read:no, write:yes)
205 2. Add: --cache- option prefix
206 3. New option: --cache-read, --no-cache
207 5. Rename: misc. options (e.g. --show -> --cache-dump)
208 6. Record field changes
209 o Breaks compatibility with previous versions
210 7. New field delimiter '#'
214 1. ~/.config/clive/config -> ~/.cliverc
215 2. ~/.config/clive/recall -> ~/.clivelast
216 3. ~/.config/clive/cache -> ~/.clivecache
219 1. Add: --format=best support (closes issue #15 )
220 2. Rename: mp4 -> fmt18 (Youtube)
221 3. Rename: Dailymotion ID spark -> flv
224 1. cURL error handling
225 2. Google mp4 support
226 3. Do not strip link params (closes debian bug #530659)
227 4. --format=fmt6 (no longer supported by Youtube?)
228 5. Dailymotion ID parsing
236 6. --stop-after (closes issue #18)
238 * --emit-csv: changes to CSV fields (cleanup, print only the necessary fields)
240 page_link, video_link, filename, file_length_mb,
241 file_length_bytes, video_id, time_stamp, page_title,
242 initial_length, remaining_bytes
244 base_filename, file_length, video_link
248 2. --renew (now obsolete)
249 3. --youtube-user (broken since 2.1)
250 4. --youtube-pass (...)
262 1. Long options: aliases (e.g. --output_file | --output-file | --outputfile)
263 2. --filename-format: new specifiers, rename some of the old ones
264 3. --format: exit with an error if id is not recognized by clive
265 4. Rename: -r -> -l (--recall)
266 5. Many short options have been removed (or reused)
268 Developer-visible changes:
272 * Change: use a custom perltidyrc (closes issue #17)
273 - Thanks to Tong Sun <suntong001@gmail.com>
275 * Prerequisite changes
276 1. Written for Perl 5.10.0
277 o 5.8 probably OK although unverified
281 3. Removed prerequisites
286 4. Optional modules (no changes from 2.1.14)
290 * Change: use decode_utf8 returned html only if html is valid utf8
292 * Installation changes
293 1. Makefile.PL (ExtUtils::MakeMaker) [see README]
296 1. Makefile (obsoleted by Makefile.PL)
297 2. clive.pod (merged into bin/clive)
298 3. clive.1 (generated from bin/clive)
300 * Remove: binmode STDOUT, ":utf8"
302 * Remove: extra decode_utf8
308 User-visible changes:
309 * Add: support for youtube-nocookie.com (closes issue #12)
310 * Add: fmt35 format ID for Youtube [closes issue #10 (and #13)]
311 * Fix: liveleak id parsing
312 * Rename: --format IDs for Youtube
316 * Rename: --output-video -> --output-file
317 * Rename: output:file -> output:filename_format (config file)
318 * Remove: --overwrite option (use --output-file instead)
320 Developer-visible changes:
321 * Add: tests/ytube-nocookies.url
325 * 2.2.0 is under development, read more here:
326 http://code.google.com/p/clive/wiki/WhatsNew_2_2_0
332 Developer-visible changes:
333 * Change: use perltidy defaults for style
334 * Change: BerkeleyDB and File::Path modules are now optional
335 ** Cache features depend on these modules
336 * Moved opted_mods initilization to init_opted_mods (-> loop)
342 User-visible changes:
343 * Change: --connect-timeout, --connect-timeout-socks default -> 30s
344 * Fix: timeout after file transfer starts
345 * Fix: title parsing with custom character class (closes issue #8)
346 * Add: redtube support; thanks to anonprn2@gmail.com (closes issue #11)
347 * Fix: typo in --emit-xml --help string
353 User-visible changes:
354 * Change: --youtube-pass: prompt for password if left undefined
355 * Change: --filename-format: default -> "%t.%s"
356 * Fix: "Use of uninitialized value $opts{"ytpass"} ..."
359 ** --connect-timeout-socks
360 * Cleanup: manual page
362 Developer-visible changes:
363 * Rename: auth_youtube subroutine -> loginto_youtube
366 * Youtube login does not work (throw an error, patches welcome!)
372 NOTE: This release changes the clive status messages. This will likely
373 break compatibility with utilities that depend on these messages.
374 The changes were made to enable using abby (0.2.0+) as a front-end
377 User-visible changes:
378 * Add abby (front-end) compatibility
379 - Change output similar to cclive
380 - http://code.google.com/p/abby/
381 * Add Youtube HD support (--format=mp4_hd)
382 - Thanks to bf for the patch (via FreeBSD ports)
383 * Add tv.cctv.com support [issue #6]
384 - Thanks to suntong001 for data mining
386 - --limit-rate (config: [http]:limit_rate)
387 - --hosts (print supported hosts to stdout)
388 - --stderr (redirect all output to stderr instead of stdout)
389 * Improve file suffix parsing from content-type
390 - Rewrite the content-type handling block in query_video_length
391 - Use whatever is found after slash in the content-type string
392 - Use "flv" for exceptions (e.g. octet-*, x-flv, etc.)
393 * Replace dashes in video IDs with underscores before using them in fnames
394 * Change --version output
395 * Fix reading from stdin (closes Debian #522422)
396 - Assume reading from stdin only if queue and argv arrays are empty
397 * Fix exit with status code
398 - Exit 0 on success, and >0 if an error occurs
400 Developer-visible changes:
401 * Update examples/config with limit_rate
402 * Rename: configrc -> examples/config
403 * Replace the monstrous if-elsif-$host code block with a loop
409 User-visible changes:
410 * Fix GoogleVideo "id not found" handling
411 - Symptom: exit after "process page..." message
412 - Cause: no check for undefined id value
413 * Fix GoogleVideo support
414 - Symptom: see above fix
415 - Cause: HTML changes to the video pages
416 * Fix URI::Escape import for GoogleVideo redirects
417 - Symptom: crash with "Undefined subroutine &URI::Escape::uri_unescape"
418 - Cause: recent script start optimizations; a missing 'require'
420 - Symptom: "expected different content-type"
421 - Cause: website content-type changes
423 Developer-visible changes:
424 * Cleanup code (style-wise)
425 - perltidy -gnu -l=72
431 User-visible changes:
432 * Keep URL queue order
433 - Remove duplicate URL checking
434 * Remove --modver, change --version output
435 * Update manual page (clive.1, clive.pod)
436 - Rewrite EXAMPLES section
437 - Cleanup other sections
438 * Config file: add support for [output]:format (--format)
439 * Port dailymotion support from cclive
441 Developer-visible changes:
443 - Replace Perl module checking with for-loops (incl. optional)
444 * Optimize script start time
445 - Replace "use $module" with "require $module" where possible
446 * Update LICENSE, configrc
448 - Add [output]:format to configrc
449 * Add tests/dmotion.url
455 User-visible changes:
456 * Add --output-video option
457 * Add option groups to --help output
458 * Add --stream-exec and --stream options
460 * Config file: add support for
461 - [commands]:exec (--exec)
462 - [commands]:stream (--stream-exec)
463 - See configrc for an example
464 * Fix --exec; should no longer spew out "not found" shell errors
465 * Change default filename format to "%t_%d_%i.%s"
466 * Change to use "file is already fully retrieved" error message
467 * Port resizing progressbar from cclive
469 Developer-visible changes:
470 * Add tests/lastfm.url and tests/multi.lst
471 * Remove XML::Simple dependency
473 * Implement SIGWINCH handling
474 - Resize progressbar to match terminal width
475 - Progressbar width defaults to 80 (chars)
476 - OS must support SIGWINCH
477 - Requires Term::ReadKey module
478 - clive works without the module but will ignore any SIGWINCH events
479 * Touch cclive.1, cclive.pod, INSTALL and Makefile files
483 * Buried clive-list, grew weary of fighting spam with bare fists
484 - email the author directly or use the issue tracker, thanks
490 User-visible changes:
491 * Renamed the following options
492 - --noproxy => --no-proxy
493 - --noextract => --no-extract
494 - --nologin => --no-login
495 * Butchered --help output
496 - much less verbose now
497 - relocated manual page data to clive.pod
498 * Removed --manual option
499 * Obsoleted "play" and "rencode" config keys in "commands" section
500 * Added --exec option which resembles cclive's --exec command
501 * Removed --play, --noplay, --rencode and --norencode options
503 * Changed --paste short option from '-x' to '-p'
504 * Removed wmv format (breakcom)
505 - persistant HTTP/404 (not found) error
507 Developer-visible changes:
509 - additional INSTALL variables
511 - can be used to disable module checks
512 - removed "man" from "all"
513 - removed "clean" (useless)
519 User-visible changes:
520 * Fixed sevenload support
526 User-visible changes:
527 * Fixed Youtube login which sometimes reported failed attempts as OK
528 - Youtube defaults web interface language based on users ip
529 - clive looks for English words in the post-login page to confirm login
530 * Tweaked --version output
532 Developer-visible changes:
533 * Removed metacafe remnants
534 * Fixed bug in time2str subroutine which caused incorrect calculations
536 * removed genman script (obsoleted by "make man")
537 * Added a rudimentary (GNU) Makefile
538 * Cleaned up translate_embed subroutine
541 * Removed the clive-announce and clive-users mailing lists
542 - New release announcements will be posted via Freshmeat alone from now on
543 - Subscribe to the project at http://freshmeat.net/projects/clive
544 * Created the clive-list@googlegroups.com mailing-list
545 - For support questions, patches and bug reports
546 - Issue tracker will remain for tracking bugs
547 - Send an email to clive-list-subscribe@googlegroups.com to subscribe
553 User-visible changes:
554 * Documented missing "%t" (--filename-format) in manual page
555 * Tweaked "localfile_length == remotefile_length" message
556 * Renamed CLIVE_CONFIGDIR environment variable to CLIVE_HOME
558 Developer-visible changes:
559 * Cleaned up manual page
561 * Cleaned up FreeBSD details (INSTALL)
567 User-visible changes:
568 * Removed "See --manual" from --version output
569 * Added support for evisortv
570 - http://code.google.com/p/clive/issues/detail?id=4
571 * Fixed: reused incorrectly last http error code for remaining urls in queue
572 * Fixed: progressbar reported 100% even if error had occurred (e.g. http/403)
574 Developer-visible changes:
575 * Cleaned up subroutines
581 User-visible changes:
582 * Fixed "Requested range was not delivered by the server (http/33)" w/ -c
583 - this would occur with a batch of URLs while using the --continue option,
584 e.g. if the first file was continued, the following page fetches exited
586 * Added --savebatch option
587 * Added --overwrite option
588 * Replaced the buggy progressbar with a "lazy man's progressbar"
589 * Fixed "print() on closed filehandle STDOUT" with --grep+delete
590 * Fixed progressbar display with --continue
592 Developer-visible changes:
594 * Started to bundle src tarball with clive.1 manual page
595 * Added genman script which wraps pod2man with common options
596 * Fixed a potential div by zero bug in calc_progress
602 NOTE: Changed license GPL3 => ISC/OpenBSD.
604 User-visible changes:
605 * Added --modversion option
606 * Replaced Term::Progressbar with a built-in thermometer progressbar
607 * Removed --progress=bar:nominor support due to above changes
608 * Added liveleakcom support
609 - http://code.google.com/p/clive/issues/detail?id=3
611 Developer-visible changes:
612 * Made IO::Pager module optional
614 * Cleaned up --progress option description
615 * Changed to require WWW::Curl 4.05+
616 * Removed Term::Progress module dependency
617 * Fixed a potential bug in calc_progress (improper $rate init)
619 * Made Expect module optional
625 User-visible changes:
626 * Fixed gvideo/mp4 support (http/404 even if the extraction link existed)
627 * Changed "found redirect" message
628 * Changed --play, --rencode status messages to use file basename
630 Developer-visible changes:
631 * decode_utf8($page): removes extra steps, works better with HTML::TokeParser
632 * Added missing Expect module version to INSTALL
639 This aims to be the last beta before the release of 2.0 in December 2008.
640 Please report any remaining bugs to the issue tracker at:
641 http://code.google.com/p/clive/issues/list
643 User-visible changes:
644 * Removed metacafe support until fixed (see known issues)
645 * Changed Youtube login to report if there were "too many login failures"
646 * Fixed Youtube login support: reported failures incorrectly as OK
647 * Added support clivepass(1) utility (http://code.google.com/p/clive-utils)
648 * Added support for metacafe and sevenload embed URL translation
650 Developer-visible changes:
652 * New prerequisite module: Expect
653 * Updated FreeBSD + Ubuntu details (INSTALL)
654 * Cleaned up some code
655 * Rewrote embed URL translation
658 * Metacafe support broken (http/404)
664 User-visible changes:
665 * Fixed pasting from clipboard containing multiple URLs
666 * Fixed page fetch progress glitch that occured with 1+ batches
667 * Added --agent option
668 * Added --proxy option
669 * Added --savedir option
670 * Added --cclass option
671 * Added --filename-format option
672 * Added --show-format option
673 * Added --youtube-user and --youtube-pass options
674 * Added --emit-xml option
675 * Added --progress=[none|bar|dot]
676 * Replaced --noprogress with --progress=none
677 * Added avg. transfer rate notification
678 * Fixed filename printing for existing output files (e.g. output.flv.1)
679 * Added --play option
680 * Fixed metacafe support (error: failed to extract videoCDNURL)
681 * Added --rencode option
683 Developer-visible changes:
684 * Ported --emit-csv option from 1.x with minor changes
685 - Backwards compatible with 1.x
686 - Prints additional video details
687 - $length, $video_id, $time_stamp, $page_title, $cont_from, $remaining
688 * Changed to prefer 'dot' bar if the output is not a TTY
689 * Replaced progress bar with Term::ProgressBar (new module prerequisite)
690 * Cleaned up queue processing subroutine
696 User-visible changes:
697 * clive now ignores input lines that start with the '#' character
698 * Added support for CLIVE_CONFIGDIR environment variable
699 * Ported lastfm video support from 1.x
700 * --version now displays XML::Simple
701 * Changed "= Play:" to "=> Playing ..."
702 * Playing occurs now subsequently after going over the URL batch
704 Developer-visible changes:
705 * Removed a google video redirect (commented out, see code)
706 * Added the AUTHORS file
707 * XML::Simple was missing as a prerequisite in INSTALL
708 * Made minor syntactic tweaks to the source code
709 * Fixed missing --quiet check with the "=> Play" message
715 A complete overhaul and rewrite of 1.x.
717 User-visible changes:
718 * Users are expected to use the --continue/-c option to resume transfers
719 * Configuration file format (=> INI), path was changed (=> ~/.config/clive)
720 * All extractions now assume flv as the default download format
721 * Users are expected to understand when/how use the --format/-f option
722 * Reading input from multiple sources is possible (e.g. % clive -x URL URL)
723 * Added: --grep: used to grep and recall cached URL entries
724 * Added: --background and --output=logfile, --append=logfile options
725 * Many old cmdline and config options were removed due to new design
726 * Improved GoogleVideo support: handles redirects to other hosts
727 * Pager is used where needed (e.g. --show/-s)
728 * Changed default output file naming format to "%n-(%i)-[%d].%s"
729 * Removed: guba (deadweight), myvideo (borked), dmotion (bitches @ anon users)
731 Developer-visible changes:
733 - See the INSTALL file for prerequisites
734 * No installation: all-in-one script (excl. the 3rd party modules)
735 * Cache is now an integral part of clive, see CACHE in the manual page
736 * Embedded the --manual page (Pod)
737 * Improved Youtube login: removed is_adult HTTP get (done manually now)
738 * Grab the file suffix from the Content-Type header
739 * Improved GoogleVideo support: mp4 detection no more relies on English words
740 * Improved Metacafe support: removed unnecessary "familyfilter" HTTP get
744 - 2.x+ at http://googlecode.com/p/clive/
745 - 1.x remains at http://gna.org/projects/clive
747 - 2.x+ at http://googlecode.com/p/clive/
748 - 1.x remain at http://dl.gna.org/clive
750 - http://clive.sf.net/
751 - http://home.gna.org/clive redirects to the above
753 - 2.x+ at git://repo.or.cz/clive.git
754 - 1.x remains at svn://svn.gna.org/svn/clive
755 * Issue tracking: Moved
756 - 2.x+ at http://googlecode.com/p/clive/issues/
757 - 1.x bug tracker remains at http://gna.org/projects/clive
758 * Created http://googlegroups.com/group/clive-announce/
760 NOTE: See http://dl.gna.org/clive/CHANGES for 1.x.