5 1. spiegel.de (closes issue #32)
6 2. golem.de (closes issue #33)
7 3. Thanks to anon data miner (via old bug tracker at gna.org)
8 * Add: ~/.clive/config path support
9 * Improve: unicode support (related to issue #29)
10 1. cctv: no longer requires -C and -R options (obsoletes -R)
11 2. --cclass: now works as expected with unicode as well
12 * Improve: backwards compatibility with clive 2.0, 2.1
13 1. --format: mp4, hd (mp4_hd), hq, 3gp (3gpp) [youtube]
14 1.1. closes Debian bug #535957
15 2. Restore ~/.config/clive/config path support
16 2.1. closes Debian bug #535483
17 3. Add: --upgrade-config
18 3.1. Can be used to upgrade 2.0/2.1 config to 2.2+ format
19 * Change: --hosts prints now supported formats
20 * Change: pair --exec with --exec-run, the latter now triggers the former
21 * Change: relocate files
22 1. ~/.clivelast -> ~/.cache/clive/last
23 2. ~/.clivecache -> ~/.cache/clive/cache
24 * Change: If HOME environment variable undefined, assume current workdir
25 * Tweak: --version: print locale etc.
26 * Fix: --version: copyright years
27 * Fix: file suffix for CCTV (.x-shockwave-flash -> .flv)
28 * Fix: Remove "see examples" from pod markup (closes issue #30)
29 * Fix: Prevent ".$suffix" output filenames
30 1. Use $id if $title is empty and %i was not defined in --filename-format
31 2. Related to issue #29
32 * Fix: --no-proxy: really disable all proxy use
33 1. http_proxy setting was still used if it was set
35 * Fix: Can't call method "filename" on unblessed reference at Exec.pm line 5x
36 1. e.g. clive URL --exec="ls;"
37 2. Closes Debian bug #535459
38 * Rename: --recall, --recall-file -> --last, --last-file
39 * Rename: CLIVE_HOME -> CLIVE_CACHE
40 * Remove: -R, --raw (obsoleted by improved unicode support)
41 * Remove: --home-dir option (missleading name)
43 2. Obsoleted by --cache-file and --last-file
45 1. Add note about numeric suffix (closes issue #28)
46 2. Fix a number of typos, use better grammar
48 Developer-visible changes:
49 * Add: clive/Compat.pm
50 * Add: clive/Host/Spiegel.pm
51 * New prerequisite: Config::Tiny
52 1. Required by --upgrade-config
54 1. Extract VERSION from lib/clive/Config.pm
55 2. Add "use warnings/strct"
63 * Change: Google: --format=mp4/best
64 1. Default to flv if mp4 is not available
65 2. Fix --format=best support
67 Developer-visible changes:
68 * Add: t/$host.t: NO_INTERNET variable support (closes issue #24)
69 * Change: t/02youtube.t test link
71 * Remove: t/06lastfm.t
72 1. t/02youtube.t: includes the same test
79 * Add: support for Vimeo (closes issue #19)
80 * Add: return codes for each error case (closes issue #22)
81 * Fix: return code is always 0 (closes issue #20)
82 * Change: --hosts output now matches cclive output
84 Developer-visible changes:
85 * Change: Cctv, Dailymotion, Liveleak -> use clive::Utils::matchRegExps
95 This release is a major overhaul aimed to fix the previous design flaws
96 and clean up the codebase. Note that 2.2.0 breaks compatibility with
97 the earlier versions of clive.
99 Most users will not notice much differences after upgrading to 2.2.0
100 but those users who have used clive for anything more than "clive URL"
101 should read the changes carefully. The summary of changes section
102 includes more detailed changes that is recommended reading for all
103 users and maintainers.
105 2.2.0 changes the license from ISC to GPLv3. The license was last
106 changed in 2.1.0 to ISC but has now been reverted back to GPLv3 after
107 some further consideration.
109 Config::Tiny has been replaced with Getopt::ArgvFile. The latter had
110 some advantages over Config::Tiny that lead to the switch. For example,
111 instead of trying to memorize the (often confusing) config variable
112 names, users can now use command line options in the config file.
114 This also means that everytime a new feature is added to the program,
115 we are no longer required to modify the code responsible for parsing
116 the config file. Using Getopt::ArgvFile also required adding only one
117 line of code to the project whereas Config::Tiny required several.
119 cache no longer reads by default. This means that --cache-read option
120 must be invoked for clive to read previously stored video records
121 from the cache. The change was made after seeing how most users found
122 the reading from cache too confusing and frequently reported expired
123 link errors such as HTTP 403 as bugs even though the behaviour was
124 documented in the manual page. DAERTM?
126 --emit-csv now outputs very few details about the videos. Only those
127 fields that are known to be any use are printed out.
129 2.2.0 also removes a number of less used features that have lingered
130 in the project since the 1.x. Most of these features were never
131 requested and have only been burdening the program codebase since
134 Part time hackers and/or developers will notice that clive now follows
135 a new OO design -- as well as Perl5 allows it. While there are still
136 some considerations regarding the design and limitations that could not
137 quite be ported from C++ used in cclive, WYSIWYG.
139 Package/port maintainers will want to read the README file. The previously
140 used GNU Makefile is no longer used and clive installation now depends on
141 ExtUtils::MakeMaker instead.
146 User-visible changes:
150 2. Last changed in 2.1.0 (-> ISC)
152 * Slight improvements to program startup time
154 * Config file format changes (Config::Tiny -> Getopt::ArgvFile)
155 1. Command line options can now be used in config file
156 2. Breaks compatibility with previous versions
157 3. Cleaner and new format allows using cmdline options in configs
158 4. Format was last changed in 2.0.0
160 -----------------------------------------------
161 # Config::Tiny: clive 2.0 - 2.1
162 cat >> ~/.config/clive/config
164 proxy = "http://foo:1234"
166 savedir = "/home/user/videos"
168 # Getopt::ArgvFile: clive 2.2
170 --proxy="http://foo:1234"
171 --savedir="/home/user/videos"
172 -----------------------------------------------
174 * Fix: Redtube video title parsing
177 1. Cache is now passive (read:no, write:yes)
178 2. Add: --cache- option prefix
179 3. New option: --cache-read, --no-cache
180 5. Rename: misc. options (e.g. --show -> --cache-dump)
181 6. Record field changes
182 o Breaks compatibility with previous versions
183 7. New field delimiter '#'
187 1. ~/.config/clive/config -> ~/.cliverc
188 2. ~/.config/clive/recall -> ~/.clivelast
189 3. ~/.config/clive/cache -> ~/.clivecache
192 1. Add: --format=best support (closes issue #15 )
193 2. Rename: mp4 -> fmt18 (Youtube)
194 3. Rename: Dailymotion ID spark -> flv
197 1. cURL error handling
198 2. Google mp4 support
199 3. Do not strip link params (closes debian bug #530659)
200 4. --format=fmt6 (no longer supported by Youtube?)
201 5. Dailymotion ID parsing
209 6. --stop-after (closes issue #18)
211 * --emit-csv: changes to CSV fields (cleanup, print only the necessary fields)
213 page_link, video_link, filename, file_length_mb,
214 file_length_bytes, video_id, time_stamp, page_title,
215 initial_length, remaining_bytes
217 base_filename, file_length, video_link
221 2. --renew (now obsolete)
222 3. --youtube-user (broken since 2.1)
223 4. --youtube-pass (...)
235 1. Long options: aliases (e.g. --output_file | --output-file | --outputfile)
236 2. --filename-format: new specifiers, rename some of the old ones
237 3. --format: exit with an error if id is not recognized by clive
238 4. Rename: -r -> -l (--recall)
239 5. Many short options have been removed (or reused)
241 Developer-visible changes:
245 * Change: use a custom perltidyrc (closes issue #17)
246 - Thanks to Tong Sun <suntong001@gmail.com>
248 * Prerequisite changes
249 1. Written for Perl 5.10.0
250 o 5.8 probably OK although unverified
254 3. Removed prerequisites
259 4. Optional modules (no changes from 2.1.14)
263 * Change: use decode_utf8 returned html only if html is valid utf8
265 * Installation changes
266 1. Makefile.PL (ExtUtils::MakeMaker) [see README]
269 1. Makefile (obsoleted by Makefile.PL)
270 2. clive.pod (merged into bin/clive)
271 3. clive.1 (generated from bin/clive)
273 * Remove: binmode STDOUT, ":utf8"
275 * Remove: extra decode_utf8
281 User-visible changes:
282 * Add: support for youtube-nocookie.com (closes issue #12)
283 * Add: fmt35 format ID for Youtube [closes issue #10 (and #13)]
284 * Fix: liveleak id parsing
285 * Rename: --format IDs for Youtube
289 * Rename: --output-video -> --output-file
290 * Rename: output:file -> output:filename_format (config file)
291 * Remove: --overwrite option (use --output-file instead)
293 Developer-visible changes:
294 * Add: tests/ytube-nocookies.url
298 * 2.2.0 is under development, read more here:
299 http://code.google.com/p/clive/wiki/WhatsNew_2_2_0
305 Developer-visible changes:
306 * Change: use perltidy defaults for style
307 * Change: BerkeleyDB and File::Path modules are now optional
308 ** Cache features depend on these modules
309 * Moved opted_mods initilization to init_opted_mods (-> loop)
315 User-visible changes:
316 * Change: --connect-timeout, --connect-timeout-socks default -> 30s
317 * Fix: timeout after file transfer starts
318 * Fix: title parsing with custom character class (closes issue #8)
319 * Add: redtube support; thanks to anonprn2@gmail.com (closes issue #11)
320 * Fix: typo in --emit-xml --help string
326 User-visible changes:
327 * Change: --youtube-pass: prompt for password if left undefined
328 * Change: --filename-format: default -> "%t.%s"
329 * Fix: "Use of uninitialized value $opts{"ytpass"} ..."
332 ** --connect-timeout-socks
333 * Cleanup: manual page
335 Developer-visible changes:
336 * Rename: auth_youtube subroutine -> loginto_youtube
339 * Youtube login does not work (throw an error, patches welcome!)
345 NOTE: This release changes the clive status messages. This will likely
346 break compatibility with utilities that depend on these messages.
347 The changes were made to enable using abby (0.2.0+) as a front-end
350 User-visible changes:
351 * Add abby (front-end) compatibility
352 - Change output similar to cclive
353 - http://code.google.com/p/abby/
354 * Add Youtube HD support (--format=mp4_hd)
355 - Thanks to bf for the patch (via FreeBSD ports)
356 * Add tv.cctv.com support [issue #6]
357 - Thanks to suntong001 for data mining
359 - --limit-rate (config: [http]:limit_rate)
360 - --hosts (print supported hosts to stdout)
361 - --stderr (redirect all output to stderr instead of stdout)
362 * Improve file suffix parsing from content-type
363 - Rewrite the content-type handling block in query_video_length
364 - Use whatever is found after slash in the content-type string
365 - Use "flv" for exceptions (e.g. octet-*, x-flv, etc.)
366 * Replace dashes in video IDs with underscores before using them in fnames
367 * Change --version output
368 * Fix reading from stdin (closes Debian #522422)
369 - Assume reading from stdin only if queue and argv arrays are empty
370 * Fix exit with status code
371 - Exit 0 on success, and >0 if an error occurs
373 Developer-visible changes:
374 * Update examples/config with limit_rate
375 * Rename: configrc -> examples/config
376 * Replace the monstrous if-elsif-$host code block with a loop
382 User-visible changes:
383 * Fix GoogleVideo "id not found" handling
384 - Symptom: exit after "process page..." message
385 - Cause: no check for undefined id value
386 * Fix GoogleVideo support
387 - Symptom: see above fix
388 - Cause: HTML changes to the video pages
389 * Fix URI::Escape import for GoogleVideo redirects
390 - Symptom: crash with "Undefined subroutine &URI::Escape::uri_unescape"
391 - Cause: recent script start optimizations; a missing 'require'
393 - Symptom: "expected different content-type"
394 - Cause: website content-type changes
396 Developer-visible changes:
397 * Cleanup code (style-wise)
398 - perltidy -gnu -l=72
404 User-visible changes:
405 * Keep URL queue order
406 - Remove duplicate URL checking
407 * Remove --modver, change --version output
408 * Update manual page (clive.1, clive.pod)
409 - Rewrite EXAMPLES section
410 - Cleanup other sections
411 * Config file: add support for [output]:format (--format)
412 * Port dailymotion support from cclive
414 Developer-visible changes:
416 - Replace Perl module checking with for-loops (incl. optional)
417 * Optimize script start time
418 - Replace "use $module" with "require $module" where possible
419 * Update LICENSE, configrc
421 - Add [output]:format to configrc
422 * Add tests/dmotion.url
428 User-visible changes:
429 * Add --output-video option
430 * Add option groups to --help output
431 * Add --stream-exec and --stream options
433 * Config file: add support for
434 - [commands]:exec (--exec)
435 - [commands]:stream (--stream-exec)
436 - See configrc for an example
437 * Fix --exec; should no longer spew out "not found" shell errors
438 * Change default filename format to "%t_%d_%i.%s"
439 * Change to use "file is already fully retrieved" error message
440 * Port resizing progressbar from cclive
442 Developer-visible changes:
443 * Add tests/lastfm.url and tests/multi.lst
444 * Remove XML::Simple dependency
446 * Implement SIGWINCH handling
447 - Resize progressbar to match terminal width
448 - Progressbar width defaults to 80 (chars)
449 - OS must support SIGWINCH
450 - Requires Term::ReadKey module
451 - clive works without the module but will ignore any SIGWINCH events
452 * Touch cclive.1, cclive.pod, INSTALL and Makefile files
456 * Buried clive-list, grew weary of fighting spam with bare fists
457 - email the author directly or use the issue tracker, thanks
463 User-visible changes:
464 * Renamed the following options
465 - --noproxy => --no-proxy
466 - --noextract => --no-extract
467 - --nologin => --no-login
468 * Butchered --help output
469 - much less verbose now
470 - relocated manual page data to clive.pod
471 * Removed --manual option
472 * Obsoleted "play" and "rencode" config keys in "commands" section
473 * Added --exec option which resembles cclive's --exec command
474 * Removed --play, --noplay, --rencode and --norencode options
476 * Changed --paste short option from '-x' to '-p'
477 * Removed wmv format (breakcom)
478 - persistant HTTP/404 (not found) error
480 Developer-visible changes:
482 - additional INSTALL variables
484 - can be used to disable module checks
485 - removed "man" from "all"
486 - removed "clean" (useless)
492 User-visible changes:
493 * Fixed sevenload support
499 User-visible changes:
500 * Fixed Youtube login which sometimes reported failed attempts as OK
501 - Youtube defaults web interface language based on users ip
502 - clive looks for English words in the post-login page to confirm login
503 * Tweaked --version output
505 Developer-visible changes:
506 * Removed metacafe remnants
507 * Fixed bug in time2str subroutine which caused incorrect calculations
509 * removed genman script (obsoleted by "make man")
510 * Added a rudimentary (GNU) Makefile
511 * Cleaned up translate_embed subroutine
514 * Removed the clive-announce and clive-users mailing lists
515 - New release announcements will be posted via Freshmeat alone from now on
516 - Subscribe to the project at http://freshmeat.net/projects/clive
517 * Created the clive-list@googlegroups.com mailing-list
518 - For support questions, patches and bug reports
519 - Issue tracker will remain for tracking bugs
520 - Send an email to clive-list-subscribe@googlegroups.com to subscribe
526 User-visible changes:
527 * Documented missing "%t" (--filename-format) in manual page
528 * Tweaked "localfile_length == remotefile_length" message
529 * Renamed CLIVE_CONFIGDIR environment variable to CLIVE_HOME
531 Developer-visible changes:
532 * Cleaned up manual page
534 * Cleaned up FreeBSD details (INSTALL)
540 User-visible changes:
541 * Removed "See --manual" from --version output
542 * Added support for evisortv
543 - http://code.google.com/p/clive/issues/detail?id=4
544 * Fixed: reused incorrectly last http error code for remaining urls in queue
545 * Fixed: progressbar reported 100% even if error had occurred (e.g. http/403)
547 Developer-visible changes:
548 * Cleaned up subroutines
554 User-visible changes:
555 * Fixed "Requested range was not delivered by the server (http/33)" w/ -c
556 - this would occur with a batch of URLs while using the --continue option,
557 e.g. if the first file was continued, the following page fetches exited
559 * Added --savebatch option
560 * Added --overwrite option
561 * Replaced the buggy progressbar with a "lazy man's progressbar"
562 * Fixed "print() on closed filehandle STDOUT" with --grep+delete
563 * Fixed progressbar display with --continue
565 Developer-visible changes:
567 * Started to bundle src tarball with clive.1 manual page
568 * Added genman script which wraps pod2man with common options
569 * Fixed a potential div by zero bug in calc_progress
575 NOTE: Changed license GPL3 => ISC/OpenBSD.
577 User-visible changes:
578 * Added --modversion option
579 * Replaced Term::Progressbar with a built-in thermometer progressbar
580 * Removed --progress=bar:nominor support due to above changes
581 * Added liveleakcom support
582 - http://code.google.com/p/clive/issues/detail?id=3
584 Developer-visible changes:
585 * Made IO::Pager module optional
587 * Cleaned up --progress option description
588 * Changed to require WWW::Curl 4.05+
589 * Removed Term::Progress module dependency
590 * Fixed a potential bug in calc_progress (improper $rate init)
592 * Made Expect module optional
598 User-visible changes:
599 * Fixed gvideo/mp4 support (http/404 even if the extraction link existed)
600 * Changed "found redirect" message
601 * Changed --play, --rencode status messages to use file basename
603 Developer-visible changes:
604 * decode_utf8($page): removes extra steps, works better with HTML::TokeParser
605 * Added missing Expect module version to INSTALL
612 This aims to be the last beta before the release of 2.0 in December 2008.
613 Please report any remaining bugs to the issue tracker at:
614 http://code.google.com/p/clive/issues/list
616 User-visible changes:
617 * Removed metacafe support until fixed (see known issues)
618 * Changed Youtube login to report if there were "too many login failures"
619 * Fixed Youtube login support: reported failures incorrectly as OK
620 * Added support clivepass(1) utility (http://code.google.com/p/clive-utils)
621 * Added support for metacafe and sevenload embed URL translation
623 Developer-visible changes:
625 * New prerequisite module: Expect
626 * Updated FreeBSD + Ubuntu details (INSTALL)
627 * Cleaned up some code
628 * Rewrote embed URL translation
631 * Metacafe support broken (http/404)
637 User-visible changes:
638 * Fixed pasting from clipboard containing multiple URLs
639 * Fixed page fetch progress glitch that occured with 1+ batches
640 * Added --agent option
641 * Added --proxy option
642 * Added --savedir option
643 * Added --cclass option
644 * Added --filename-format option
645 * Added --show-format option
646 * Added --youtube-user and --youtube-pass options
647 * Added --emit-xml option
648 * Added --progress=[none|bar|dot]
649 * Replaced --noprogress with --progress=none
650 * Added avg. transfer rate notification
651 * Fixed filename printing for existing output files (e.g. output.flv.1)
652 * Added --play option
653 * Fixed metacafe support (error: failed to extract videoCDNURL)
654 * Added --rencode option
656 Developer-visible changes:
657 * Ported --emit-csv option from 1.x with minor changes
658 - Backwards compatible with 1.x
659 - Prints additional video details
660 - $length, $video_id, $time_stamp, $page_title, $cont_from, $remaining
661 * Changed to prefer 'dot' bar if the output is not a TTY
662 * Replaced progress bar with Term::ProgressBar (new module prerequisite)
663 * Cleaned up queue processing subroutine
669 User-visible changes:
670 * clive now ignores input lines that start with the '#' character
671 * Added support for CLIVE_CONFIGDIR environment variable
672 * Ported lastfm video support from 1.x
673 * --version now displays XML::Simple
674 * Changed "= Play:" to "=> Playing ..."
675 * Playing occurs now subsequently after going over the URL batch
677 Developer-visible changes:
678 * Removed a google video redirect (commented out, see code)
679 * Added the AUTHORS file
680 * XML::Simple was missing as a prerequisite in INSTALL
681 * Made minor syntactic tweaks to the source code
682 * Fixed missing --quiet check with the "=> Play" message
688 A complete overhaul and rewrite of 1.x.
690 User-visible changes:
691 * Users are expected to use the --continue/-c option to resume transfers
692 * Configuration file format (=> INI), path was changed (=> ~/.config/clive)
693 * All extractions now assume flv as the default download format
694 * Users are expected to understand when/how use the --format/-f option
695 * Reading input from multiple sources is possible (e.g. % clive -x URL URL)
696 * Added: --grep: used to grep and recall cached URL entries
697 * Added: --background and --output=logfile, --append=logfile options
698 * Many old cmdline and config options were removed due to new design
699 * Improved GoogleVideo support: handles redirects to other hosts
700 * Pager is used where needed (e.g. --show/-s)
701 * Changed default output file naming format to "%n-(%i)-[%d].%s"
702 * Removed: guba (deadweight), myvideo (borked), dmotion (bitches @ anon users)
704 Developer-visible changes:
706 - See the INSTALL file for prerequisites
707 * No installation: all-in-one script (excl. the 3rd party modules)
708 * Cache is now an integral part of clive, see CACHE in the manual page
709 * Embedded the --manual page (Pod)
710 * Improved Youtube login: removed is_adult HTTP get (done manually now)
711 * Grab the file suffix from the Content-Type header
712 * Improved GoogleVideo support: mp4 detection no more relies on English words
713 * Improved Metacafe support: removed unnecessary "familyfilter" HTTP get
717 - 2.x+ at http://googlecode.com/p/clive/
718 - 1.x remains at http://gna.org/projects/clive
720 - 2.x+ at http://googlecode.com/p/clive/
721 - 1.x remain at http://dl.gna.org/clive
723 - http://clive.sf.net/
724 - http://home.gna.org/clive redirects to the above
726 - 2.x+ at git://repo.or.cz/clive.git
727 - 1.x remains at svn://svn.gna.org/svn/clive
728 * Issue tracking: Moved
729 - 2.x+ at http://googlecode.com/p/clive/issues/
730 - 1.x bug tracker remains at http://gna.org/projects/clive
731 * Created http://googlegroups.com/group/clive-announce/
733 NOTE: See http://dl.gna.org/clive/CHANGES for 1.x.