6 # fmt34 is now treated as yet another (new?) format
7 # Rewrite youtube section of the FORMATS in the manual
8 * Fix: --format=best not working with youtube (closes issue #39)
9 # Thanks to Peter Baranyi for the fix
11 Developer-visible changes:
12 * "make test": ADULT_OK env.var. now enables redtube test
22 * Fix: liveleak: "error: Unsupported protocol (http/1)" (closes #36)
23 * Fix: vimeo: title parsing (Thanks to Peter Baranyi for the fix, closes #37)
24 - Fixes title issues with accented chars
25 * Fix: break: http/403 (Thanks to Werner Elsler for the fix, closes #38)
27 Developer-visible changes:
28 * t/11redtube.t: fix dead link
35 * Add: --print-fname option
36 * Fix: dailymotion ID parsing
37 * Fix: --hosts: dailymotion: spak-mini -> spark-mini
40 Developer-visible changes:
41 * Correct copyright years and notices in source code
42 * tlinks/youtube-utf8.url: fix dead link
50 # --stream-pass option (port from cclive)
56 * Thanks to Patrick Hoffmann <patrick@sobran.de> for data mining
58 # Misc. typos in manual
59 # Redtube support (videoid/title parsing issues)
60 # Dailymotion: spak-mini -> spark-mini
62 # replace dashes ("-") with underscores ("_") in video ids
64 Developer-visible changes:
76 # t/02youtube.t: test link (dead link)
77 # tlinks/ytube.url: ditto
78 # tlinks/multi.url: youtube: ditto
88 1. Support spiegel.de (closes issue #32) [thanks to anon for data]
89 2. Support golem.de (closes issue #33) [thanks to anon for data]
90 3. ~/.clive/config path support
91 4. --cookie-jar option
92 5. Support for ehrensenf.de (closes issue #34) [thanks to bjoern for data]
93 * Improve: unicode support (related to issue #29)
94 1. cctv: no longer requires -C and -R options (obsoletes -R)
95 2. --cclass: now works as expected with unicode as well
96 * Improve: backwards compatibility with clive 2.0, 2.1
97 1. Allow --format=(mp4|hd|hq|3gp) (closes Debian bug #535957)
98 2. Restore ~/.config/clive/config path support (closes Debian bug #535483)
99 3. Add --upgrade-config (2.0/2.1 config to 2.2+ format)
101 1. --hosts prints now supported formats
102 2. Pair --exec with --exec-run, the latter now triggers the former
103 2.1. Closes Debian bug #520520
105 3.1. ~/.clivelast -> ~/.cache/clive/last
106 3.2. ~/.clivecache -> ~/.cache/clive/cache
107 4. If HOME environment variable undefined, assume current workdir
108 5. --version: print locale etc.
110 1. --version: copyright years
111 2. File suffix for CCTV (.x-shockwave-flash -> .flv)
112 3. Remove "see examples" from pod markup (closes issue #30)
113 4. Prevent ".$suffix" output filenames
114 4.1. Use $id if !$title && %i was not defined in --filename-format
115 4.2. Related to issue #29
116 5. --no-proxy: really disable all proxy use (closes issue #25)
117 5.1. http_proxy setting was still used
118 6. Can't call method "filename" on unblessed reference at Exec.pm line 5x
119 6.1. e.g. clive URL --exec="ls;"
120 6.2. Closes Debian bug #535459
121 7. Dailymotion support (closes issue #35)
122 8. Redtube support (error: no match: `(?-xism:videoid=(.*?)')')
124 1. --recall, --recall-file -> --last, --last-file
125 2. CLIVE_HOME -> CLIVE_CACHE
127 1. -R, --raw (obsoleted by improved unicode support)
129 2.1. Missleading name
130 2.2. Obsoleted by --cache-file and --last-file
132 1. Add note about numeric suffix (closes issue #28)
133 2. Fix a number of typos, use better grammar
136 Developer-visible changes:
139 * clive/Host/Spiegel.pm
140 * clive/Host/Golem.pm
141 * clive/Host/Ehrensenf.pm
147 * tlinks/ehrensenf.url
148 * New prerequisite: Config::Tiny
149 1. Required by --upgrade-config
151 1. Extract VERSION from lib/clive/Config.pm
152 2. Add "use warnings/strct"
159 User-visible changes:
160 * Change: Google: --format=mp4/best
161 1. Default to flv if mp4 is not available
162 2. Fix --format=best support
164 Developer-visible changes:
165 * Add: t/$host.t: NO_INTERNET variable support (closes issue #24)
166 * Change: t/02youtube.t test link
168 * Remove: t/06lastfm.t
169 1. t/02youtube.t: includes the same test
175 User-visible changes:
176 * Add: support for Vimeo (closes issue #19)
177 * Add: return codes for each error case (closes issue #22)
178 * Fix: return code is always 0 (closes issue #20)
179 * Change: --hosts output now matches cclive output
181 Developer-visible changes:
182 * Change: Cctv, Dailymotion, Liveleak -> use clive::Utils::matchRegExps
192 This release is a major overhaul aimed to fix the previous design flaws
193 and clean up the codebase. Note that 2.2.0 breaks compatibility with
194 the earlier versions of clive.
196 Most users will not notice much differences after upgrading to 2.2.0
197 but those users who have used clive for anything more than "clive URL"
198 should read the changes carefully. The summary of changes section
199 includes more detailed changes that is recommended reading for all
200 users and maintainers.
202 2.2.0 changes the license from ISC to GPLv3. The license was last
203 changed in 2.1.0 to ISC but has now been reverted back to GPLv3 after
204 some further consideration.
206 Config::Tiny has been replaced with Getopt::ArgvFile. The latter had
207 some advantages over Config::Tiny that lead to the switch. For example,
208 instead of trying to memorize the (often confusing) config variable
209 names, users can now use command line options in the config file.
211 This also means that everytime a new feature is added to the program,
212 we are no longer required to modify the code responsible for parsing
213 the config file. Using Getopt::ArgvFile also required adding only one
214 line of code to the project whereas Config::Tiny required several.
216 cache no longer reads by default. This means that --cache-read option
217 must be invoked for clive to read previously stored video records
218 from the cache. The change was made after seeing how most users found
219 the reading from cache too confusing and frequently reported expired
220 link errors such as HTTP 403 as bugs even though the behaviour was
221 documented in the manual page. DAERTM?
223 --emit-csv now outputs very few details about the videos. Only those
224 fields that are known to be any use are printed out.
226 2.2.0 also removes a number of less used features that have lingered
227 in the project since the 1.x. Most of these features were never
228 requested and have only been burdening the program codebase since
231 Part time hackers and/or developers will notice that clive now follows
232 a new OO design -- as well as Perl5 allows it. While there are still
233 some considerations regarding the design and limitations that could not
234 quite be ported from C++ used in cclive, WYSIWYG.
236 Package/port maintainers will want to read the README file. The previously
237 used GNU Makefile is no longer used and clive installation now depends on
238 ExtUtils::MakeMaker instead.
243 User-visible changes:
247 2. Last changed in 2.1.0 (-> ISC)
249 * Slight improvements to program startup time
251 * Config file format changes (Config::Tiny -> Getopt::ArgvFile)
252 1. Command line options can now be used in config file
253 2. Breaks compatibility with previous versions
254 3. Cleaner and new format allows using cmdline options in configs
255 4. Format was last changed in 2.0.0
257 -----------------------------------------------
258 # Config::Tiny: clive 2.0 - 2.1
259 cat >> ~/.config/clive/config
261 proxy = "http://foo:1234"
263 savedir = "/home/user/videos"
265 # Getopt::ArgvFile: clive 2.2
267 --proxy="http://foo:1234"
268 --savedir="/home/user/videos"
269 -----------------------------------------------
271 * Fix: Redtube video title parsing
274 1. Cache is now passive (read:no, write:yes)
275 2. Add: --cache- option prefix
276 3. New option: --cache-read, --no-cache
277 5. Rename: misc. options (e.g. --show -> --cache-dump)
278 6. Record field changes
279 o Breaks compatibility with previous versions
280 7. New field delimiter '#'
284 1. ~/.config/clive/config -> ~/.cliverc
285 2. ~/.config/clive/recall -> ~/.clivelast
286 3. ~/.config/clive/cache -> ~/.clivecache
289 1. Add: --format=best support (closes issue #15 )
290 2. Rename: mp4 -> fmt18 (Youtube)
291 3. Rename: Dailymotion ID spark -> flv
294 1. cURL error handling
295 2. Google mp4 support
296 3. Do not strip link params (closes debian bug #530659)
297 4. --format=fmt6 (no longer supported by Youtube?)
298 5. Dailymotion ID parsing
306 6. --stop-after (closes issue #18)
308 * --emit-csv: changes to CSV fields (cleanup, print only the necessary fields)
310 page_link, video_link, filename, file_length_mb,
311 file_length_bytes, video_id, time_stamp, page_title,
312 initial_length, remaining_bytes
314 base_filename, file_length, video_link
318 2. --renew (now obsolete)
319 3. --youtube-user (broken since 2.1)
320 4. --youtube-pass (...)
332 1. Long options: aliases (e.g. --output_file | --output-file | --outputfile)
333 2. --filename-format: new specifiers, rename some of the old ones
334 3. --format: exit with an error if id is not recognized by clive
335 4. Rename: -r -> -l (--recall)
336 5. Many short options have been removed (or reused)
338 Developer-visible changes:
342 * Change: use a custom perltidyrc (closes issue #17)
343 - Thanks to Tong Sun <suntong001@gmail.com>
345 * Prerequisite changes
346 1. Written for Perl 5.10.0
347 o 5.8 probably OK although unverified
351 3. Removed prerequisites
356 4. Optional modules (no changes from 2.1.14)
360 * Change: use decode_utf8 returned html only if html is valid utf8
362 * Installation changes
363 1. Makefile.PL (ExtUtils::MakeMaker) [see README]
366 1. Makefile (obsoleted by Makefile.PL)
367 2. clive.pod (merged into bin/clive)
368 3. clive.1 (generated from bin/clive)
370 * Remove: binmode STDOUT, ":utf8"
372 * Remove: extra decode_utf8
378 User-visible changes:
379 * Add: support for youtube-nocookie.com (closes issue #12)
380 * Add: fmt35 format ID for Youtube [closes issue #10 (and #13)]
381 * Fix: liveleak id parsing
382 * Rename: --format IDs for Youtube
386 * Rename: --output-video -> --output-file
387 * Rename: output:file -> output:filename_format (config file)
388 * Remove: --overwrite option (use --output-file instead)
390 Developer-visible changes:
391 * Add: tests/ytube-nocookies.url
395 * 2.2.0 is under development, read more here:
396 http://code.google.com/p/clive/wiki/WhatsNew_2_2_0
402 Developer-visible changes:
403 * Change: use perltidy defaults for style
404 * Change: BerkeleyDB and File::Path modules are now optional
405 ** Cache features depend on these modules
406 * Moved opted_mods initilization to init_opted_mods (-> loop)
412 User-visible changes:
413 * Change: --connect-timeout, --connect-timeout-socks default -> 30s
414 * Fix: timeout after file transfer starts
415 * Fix: title parsing with custom character class (closes issue #8)
416 * Add: redtube support; thanks to anonprn2@gmail.com (closes issue #11)
417 * Fix: typo in --emit-xml --help string
423 User-visible changes:
424 * Change: --youtube-pass: prompt for password if left undefined
425 * Change: --filename-format: default -> "%t.%s"
426 * Fix: "Use of uninitialized value $opts{"ytpass"} ..."
429 ** --connect-timeout-socks
430 * Cleanup: manual page
432 Developer-visible changes:
433 * Rename: auth_youtube subroutine -> loginto_youtube
436 * Youtube login does not work (throw an error, patches welcome!)
442 NOTE: This release changes the clive status messages. This will likely
443 break compatibility with utilities that depend on these messages.
444 The changes were made to enable using abby (0.2.0+) as a front-end
447 User-visible changes:
448 * Add abby (front-end) compatibility
449 - Change output similar to cclive
450 - http://code.google.com/p/abby/
451 * Add Youtube HD support (--format=mp4_hd)
452 - Thanks to bf for the patch (via FreeBSD ports)
453 * Add tv.cctv.com support [issue #6]
454 - Thanks to suntong001 for data mining
456 - --limit-rate (config: [http]:limit_rate)
457 - --hosts (print supported hosts to stdout)
458 - --stderr (redirect all output to stderr instead of stdout)
459 * Improve file suffix parsing from content-type
460 - Rewrite the content-type handling block in query_video_length
461 - Use whatever is found after slash in the content-type string
462 - Use "flv" for exceptions (e.g. octet-*, x-flv, etc.)
463 * Replace dashes in video IDs with underscores before using them in fnames
464 * Change --version output
465 * Fix reading from stdin (closes Debian #522422)
466 - Assume reading from stdin only if queue and argv arrays are empty
467 * Fix exit with status code
468 - Exit 0 on success, and >0 if an error occurs
470 Developer-visible changes:
471 * Update examples/config with limit_rate
472 * Rename: configrc -> examples/config
473 * Replace the monstrous if-elsif-$host code block with a loop
479 User-visible changes:
480 * Fix GoogleVideo "id not found" handling
481 - Symptom: exit after "process page..." message
482 - Cause: no check for undefined id value
483 * Fix GoogleVideo support
484 - Symptom: see above fix
485 - Cause: HTML changes to the video pages
486 * Fix URI::Escape import for GoogleVideo redirects
487 - Symptom: crash with "Undefined subroutine &URI::Escape::uri_unescape"
488 - Cause: recent script start optimizations; a missing 'require'
490 - Symptom: "expected different content-type"
491 - Cause: website content-type changes
493 Developer-visible changes:
494 * Cleanup code (style-wise)
495 - perltidy -gnu -l=72
501 User-visible changes:
502 * Keep URL queue order
503 - Remove duplicate URL checking
504 * Remove --modver, change --version output
505 * Update manual page (clive.1, clive.pod)
506 - Rewrite EXAMPLES section
507 - Cleanup other sections
508 * Config file: add support for [output]:format (--format)
509 * Port dailymotion support from cclive
511 Developer-visible changes:
513 - Replace Perl module checking with for-loops (incl. optional)
514 * Optimize script start time
515 - Replace "use $module" with "require $module" where possible
516 * Update LICENSE, configrc
518 - Add [output]:format to configrc
519 * Add tests/dmotion.url
525 User-visible changes:
526 * Add --output-video option
527 * Add option groups to --help output
528 * Add --stream-exec and --stream options
530 * Config file: add support for
531 - [commands]:exec (--exec)
532 - [commands]:stream (--stream-exec)
533 - See configrc for an example
534 * Fix --exec; should no longer spew out "not found" shell errors
535 * Change default filename format to "%t_%d_%i.%s"
536 * Change to use "file is already fully retrieved" error message
537 * Port resizing progressbar from cclive
539 Developer-visible changes:
540 * Add tests/lastfm.url and tests/multi.lst
541 * Remove XML::Simple dependency
543 * Implement SIGWINCH handling
544 - Resize progressbar to match terminal width
545 - Progressbar width defaults to 80 (chars)
546 - OS must support SIGWINCH
547 - Requires Term::ReadKey module
548 - clive works without the module but will ignore any SIGWINCH events
549 * Touch cclive.1, cclive.pod, INSTALL and Makefile files
553 * Buried clive-list, grew weary of fighting spam with bare fists
554 - email the author directly or use the issue tracker, thanks
560 User-visible changes:
561 * Renamed the following options
562 - --noproxy => --no-proxy
563 - --noextract => --no-extract
564 - --nologin => --no-login
565 * Butchered --help output
566 - much less verbose now
567 - relocated manual page data to clive.pod
568 * Removed --manual option
569 * Obsoleted "play" and "rencode" config keys in "commands" section
570 * Added --exec option which resembles cclive's --exec command
571 * Removed --play, --noplay, --rencode and --norencode options
573 * Changed --paste short option from '-x' to '-p'
574 * Removed wmv format (breakcom)
575 - persistant HTTP/404 (not found) error
577 Developer-visible changes:
579 - additional INSTALL variables
581 - can be used to disable module checks
582 - removed "man" from "all"
583 - removed "clean" (useless)
589 User-visible changes:
590 * Fixed sevenload support
596 User-visible changes:
597 * Fixed Youtube login which sometimes reported failed attempts as OK
598 - Youtube defaults web interface language based on users ip
599 - clive looks for English words in the post-login page to confirm login
600 * Tweaked --version output
602 Developer-visible changes:
603 * Removed metacafe remnants
604 * Fixed bug in time2str subroutine which caused incorrect calculations
606 * removed genman script (obsoleted by "make man")
607 * Added a rudimentary (GNU) Makefile
608 * Cleaned up translate_embed subroutine
611 * Removed the clive-announce and clive-users mailing lists
612 - New release announcements will be posted via Freshmeat alone from now on
613 - Subscribe to the project at http://freshmeat.net/projects/clive
614 * Created the clive-list@googlegroups.com mailing-list
615 - For support questions, patches and bug reports
616 - Issue tracker will remain for tracking bugs
617 - Send an email to clive-list-subscribe@googlegroups.com to subscribe
623 User-visible changes:
624 * Documented missing "%t" (--filename-format) in manual page
625 * Tweaked "localfile_length == remotefile_length" message
626 * Renamed CLIVE_CONFIGDIR environment variable to CLIVE_HOME
628 Developer-visible changes:
629 * Cleaned up manual page
631 * Cleaned up FreeBSD details (INSTALL)
637 User-visible changes:
638 * Removed "See --manual" from --version output
639 * Added support for evisortv
640 - http://code.google.com/p/clive/issues/detail?id=4
641 * Fixed: reused incorrectly last http error code for remaining urls in queue
642 * Fixed: progressbar reported 100% even if error had occurred (e.g. http/403)
644 Developer-visible changes:
645 * Cleaned up subroutines
651 User-visible changes:
652 * Fixed "Requested range was not delivered by the server (http/33)" w/ -c
653 - this would occur with a batch of URLs while using the --continue option,
654 e.g. if the first file was continued, the following page fetches exited
656 * Added --savebatch option
657 * Added --overwrite option
658 * Replaced the buggy progressbar with a "lazy man's progressbar"
659 * Fixed "print() on closed filehandle STDOUT" with --grep+delete
660 * Fixed progressbar display with --continue
662 Developer-visible changes:
664 * Started to bundle src tarball with clive.1 manual page
665 * Added genman script which wraps pod2man with common options
666 * Fixed a potential div by zero bug in calc_progress
672 NOTE: Changed license GPL3 => ISC/OpenBSD.
674 User-visible changes:
675 * Added --modversion option
676 * Replaced Term::Progressbar with a built-in thermometer progressbar
677 * Removed --progress=bar:nominor support due to above changes
678 * Added liveleakcom support
679 - http://code.google.com/p/clive/issues/detail?id=3
681 Developer-visible changes:
682 * Made IO::Pager module optional
684 * Cleaned up --progress option description
685 * Changed to require WWW::Curl 4.05+
686 * Removed Term::Progress module dependency
687 * Fixed a potential bug in calc_progress (improper $rate init)
689 * Made Expect module optional
695 User-visible changes:
696 * Fixed gvideo/mp4 support (http/404 even if the extraction link existed)
697 * Changed "found redirect" message
698 * Changed --play, --rencode status messages to use file basename
700 Developer-visible changes:
701 * decode_utf8($page): removes extra steps, works better with HTML::TokeParser
702 * Added missing Expect module version to INSTALL
709 This aims to be the last beta before the release of 2.0 in December 2008.
710 Please report any remaining bugs to the issue tracker at:
711 http://code.google.com/p/clive/issues/list
713 User-visible changes:
714 * Removed metacafe support until fixed (see known issues)
715 * Changed Youtube login to report if there were "too many login failures"
716 * Fixed Youtube login support: reported failures incorrectly as OK
717 * Added support clivepass(1) utility (http://code.google.com/p/clive-utils)
718 * Added support for metacafe and sevenload embed URL translation
720 Developer-visible changes:
722 * New prerequisite module: Expect
723 * Updated FreeBSD + Ubuntu details (INSTALL)
724 * Cleaned up some code
725 * Rewrote embed URL translation
728 * Metacafe support broken (http/404)
734 User-visible changes:
735 * Fixed pasting from clipboard containing multiple URLs
736 * Fixed page fetch progress glitch that occured with 1+ batches
737 * Added --agent option
738 * Added --proxy option
739 * Added --savedir option
740 * Added --cclass option
741 * Added --filename-format option
742 * Added --show-format option
743 * Added --youtube-user and --youtube-pass options
744 * Added --emit-xml option
745 * Added --progress=[none|bar|dot]
746 * Replaced --noprogress with --progress=none
747 * Added avg. transfer rate notification
748 * Fixed filename printing for existing output files (e.g. output.flv.1)
749 * Added --play option
750 * Fixed metacafe support (error: failed to extract videoCDNURL)
751 * Added --rencode option
753 Developer-visible changes:
754 * Ported --emit-csv option from 1.x with minor changes
755 - Backwards compatible with 1.x
756 - Prints additional video details
757 - $length, $video_id, $time_stamp, $page_title, $cont_from, $remaining
758 * Changed to prefer 'dot' bar if the output is not a TTY
759 * Replaced progress bar with Term::ProgressBar (new module prerequisite)
760 * Cleaned up queue processing subroutine
766 User-visible changes:
767 * clive now ignores input lines that start with the '#' character
768 * Added support for CLIVE_CONFIGDIR environment variable
769 * Ported lastfm video support from 1.x
770 * --version now displays XML::Simple
771 * Changed "= Play:" to "=> Playing ..."
772 * Playing occurs now subsequently after going over the URL batch
774 Developer-visible changes:
775 * Removed a google video redirect (commented out, see code)
776 * Added the AUTHORS file
777 * XML::Simple was missing as a prerequisite in INSTALL
778 * Made minor syntactic tweaks to the source code
779 * Fixed missing --quiet check with the "=> Play" message
785 A complete overhaul and rewrite of 1.x.
787 User-visible changes:
788 * Users are expected to use the --continue/-c option to resume transfers
789 * Configuration file format (=> INI), path was changed (=> ~/.config/clive)
790 * All extractions now assume flv as the default download format
791 * Users are expected to understand when/how use the --format/-f option
792 * Reading input from multiple sources is possible (e.g. % clive -x URL URL)
793 * Added: --grep: used to grep and recall cached URL entries
794 * Added: --background and --output=logfile, --append=logfile options
795 * Many old cmdline and config options were removed due to new design
796 * Improved GoogleVideo support: handles redirects to other hosts
797 * Pager is used where needed (e.g. --show/-s)
798 * Changed default output file naming format to "%n-(%i)-[%d].%s"
799 * Removed: guba (deadweight), myvideo (borked), dmotion (bitches @ anon users)
801 Developer-visible changes:
803 - See the INSTALL file for prerequisites
804 * No installation: all-in-one script (excl. the 3rd party modules)
805 * Cache is now an integral part of clive, see CACHE in the manual page
806 * Embedded the --manual page (Pod)
807 * Improved Youtube login: removed is_adult HTTP get (done manually now)
808 * Grab the file suffix from the Content-Type header
809 * Improved GoogleVideo support: mp4 detection no more relies on English words
810 * Improved Metacafe support: removed unnecessary "familyfilter" HTTP get
814 - 2.x+ at http://googlecode.com/p/clive/
815 - 1.x remains at http://gna.org/projects/clive
817 - 2.x+ at http://googlecode.com/p/clive/
818 - 1.x remain at http://dl.gna.org/clive
820 - http://clive.sf.net/
821 - http://home.gna.org/clive redirects to the above
823 - 2.x+ at git://repo.or.cz/clive.git
824 - 1.x remains at svn://svn.gna.org/svn/clive
825 * Issue tracking: Moved
826 - 2.x+ at http://googlecode.com/p/clive/issues/
827 - 1.x bug tracker remains at http://gna.org/projects/clive
828 * Created http://googlegroups.com/group/clive-announce/
830 NOTE: See http://dl.gna.org/clive/CHANGES for 1.x.