4 * Fix: dailymotion ID parsing
6 Developer-visible changes:
7 * tlinks/youtube-utf8.url: fix dead link
15 # --stream-pass option (port from cclive)
21 * Thanks to Patrick Hoffmann <patrick@sobran.de> for data mining
23 # Misc. typos in manual
24 # Redtube support (videoid/title parsing issues)
25 # Dailymotion: spak-mini -> spark-mini
27 # replace dashes ("-") with underscores ("_") in video ids
29 Developer-visible changes:
41 # t/02youtube.t: test link (dead link)
42 # tlinks/ytube.url: ditto
43 # tlinks/multi.url: youtube: ditto
53 1. Support spiegel.de (closes issue #32) [thanks to anon for data]
54 2. Support golem.de (closes issue #33) [thanks to anon for data]
55 3. ~/.clive/config path support
56 4. --cookie-jar option
57 5. Support for ehrensenf.de (closes issue #34) [thanks to bjoern for data]
58 * Improve: unicode support (related to issue #29)
59 1. cctv: no longer requires -C and -R options (obsoletes -R)
60 2. --cclass: now works as expected with unicode as well
61 * Improve: backwards compatibility with clive 2.0, 2.1
62 1. Allow --format=(mp4|hd|hq|3gp) (closes Debian bug #535957)
63 2. Restore ~/.config/clive/config path support (closes Debian bug #535483)
64 3. Add --upgrade-config (2.0/2.1 config to 2.2+ format)
66 1. --hosts prints now supported formats
67 2. Pair --exec with --exec-run, the latter now triggers the former
68 2.1. Closes Debian bug #520520
70 3.1. ~/.clivelast -> ~/.cache/clive/last
71 3.2. ~/.clivecache -> ~/.cache/clive/cache
72 4. If HOME environment variable undefined, assume current workdir
73 5. --version: print locale etc.
75 1. --version: copyright years
76 2. File suffix for CCTV (.x-shockwave-flash -> .flv)
77 3. Remove "see examples" from pod markup (closes issue #30)
78 4. Prevent ".$suffix" output filenames
79 4.1. Use $id if !$title && %i was not defined in --filename-format
80 4.2. Related to issue #29
81 5. --no-proxy: really disable all proxy use (closes issue #25)
82 5.1. http_proxy setting was still used
83 6. Can't call method "filename" on unblessed reference at Exec.pm line 5x
84 6.1. e.g. clive URL --exec="ls;"
85 6.2. Closes Debian bug #535459
86 7. Dailymotion support (closes issue #35)
87 8. Redtube support (error: no match: `(?-xism:videoid=(.*?)')')
89 1. --recall, --recall-file -> --last, --last-file
90 2. CLIVE_HOME -> CLIVE_CACHE
92 1. -R, --raw (obsoleted by improved unicode support)
95 2.2. Obsoleted by --cache-file and --last-file
97 1. Add note about numeric suffix (closes issue #28)
98 2. Fix a number of typos, use better grammar
101 Developer-visible changes:
104 * clive/Host/Spiegel.pm
105 * clive/Host/Golem.pm
106 * clive/Host/Ehrensenf.pm
112 * tlinks/ehrensenf.url
113 * New prerequisite: Config::Tiny
114 1. Required by --upgrade-config
116 1. Extract VERSION from lib/clive/Config.pm
117 2. Add "use warnings/strct"
124 User-visible changes:
125 * Change: Google: --format=mp4/best
126 1. Default to flv if mp4 is not available
127 2. Fix --format=best support
129 Developer-visible changes:
130 * Add: t/$host.t: NO_INTERNET variable support (closes issue #24)
131 * Change: t/02youtube.t test link
133 * Remove: t/06lastfm.t
134 1. t/02youtube.t: includes the same test
140 User-visible changes:
141 * Add: support for Vimeo (closes issue #19)
142 * Add: return codes for each error case (closes issue #22)
143 * Fix: return code is always 0 (closes issue #20)
144 * Change: --hosts output now matches cclive output
146 Developer-visible changes:
147 * Change: Cctv, Dailymotion, Liveleak -> use clive::Utils::matchRegExps
157 This release is a major overhaul aimed to fix the previous design flaws
158 and clean up the codebase. Note that 2.2.0 breaks compatibility with
159 the earlier versions of clive.
161 Most users will not notice much differences after upgrading to 2.2.0
162 but those users who have used clive for anything more than "clive URL"
163 should read the changes carefully. The summary of changes section
164 includes more detailed changes that is recommended reading for all
165 users and maintainers.
167 2.2.0 changes the license from ISC to GPLv3. The license was last
168 changed in 2.1.0 to ISC but has now been reverted back to GPLv3 after
169 some further consideration.
171 Config::Tiny has been replaced with Getopt::ArgvFile. The latter had
172 some advantages over Config::Tiny that lead to the switch. For example,
173 instead of trying to memorize the (often confusing) config variable
174 names, users can now use command line options in the config file.
176 This also means that everytime a new feature is added to the program,
177 we are no longer required to modify the code responsible for parsing
178 the config file. Using Getopt::ArgvFile also required adding only one
179 line of code to the project whereas Config::Tiny required several.
181 cache no longer reads by default. This means that --cache-read option
182 must be invoked for clive to read previously stored video records
183 from the cache. The change was made after seeing how most users found
184 the reading from cache too confusing and frequently reported expired
185 link errors such as HTTP 403 as bugs even though the behaviour was
186 documented in the manual page. DAERTM?
188 --emit-csv now outputs very few details about the videos. Only those
189 fields that are known to be any use are printed out.
191 2.2.0 also removes a number of less used features that have lingered
192 in the project since the 1.x. Most of these features were never
193 requested and have only been burdening the program codebase since
196 Part time hackers and/or developers will notice that clive now follows
197 a new OO design -- as well as Perl5 allows it. While there are still
198 some considerations regarding the design and limitations that could not
199 quite be ported from C++ used in cclive, WYSIWYG.
201 Package/port maintainers will want to read the README file. The previously
202 used GNU Makefile is no longer used and clive installation now depends on
203 ExtUtils::MakeMaker instead.
208 User-visible changes:
212 2. Last changed in 2.1.0 (-> ISC)
214 * Slight improvements to program startup time
216 * Config file format changes (Config::Tiny -> Getopt::ArgvFile)
217 1. Command line options can now be used in config file
218 2. Breaks compatibility with previous versions
219 3. Cleaner and new format allows using cmdline options in configs
220 4. Format was last changed in 2.0.0
222 -----------------------------------------------
223 # Config::Tiny: clive 2.0 - 2.1
224 cat >> ~/.config/clive/config
226 proxy = "http://foo:1234"
228 savedir = "/home/user/videos"
230 # Getopt::ArgvFile: clive 2.2
232 --proxy="http://foo:1234"
233 --savedir="/home/user/videos"
234 -----------------------------------------------
236 * Fix: Redtube video title parsing
239 1. Cache is now passive (read:no, write:yes)
240 2. Add: --cache- option prefix
241 3. New option: --cache-read, --no-cache
242 5. Rename: misc. options (e.g. --show -> --cache-dump)
243 6. Record field changes
244 o Breaks compatibility with previous versions
245 7. New field delimiter '#'
249 1. ~/.config/clive/config -> ~/.cliverc
250 2. ~/.config/clive/recall -> ~/.clivelast
251 3. ~/.config/clive/cache -> ~/.clivecache
254 1. Add: --format=best support (closes issue #15 )
255 2. Rename: mp4 -> fmt18 (Youtube)
256 3. Rename: Dailymotion ID spark -> flv
259 1. cURL error handling
260 2. Google mp4 support
261 3. Do not strip link params (closes debian bug #530659)
262 4. --format=fmt6 (no longer supported by Youtube?)
263 5. Dailymotion ID parsing
271 6. --stop-after (closes issue #18)
273 * --emit-csv: changes to CSV fields (cleanup, print only the necessary fields)
275 page_link, video_link, filename, file_length_mb,
276 file_length_bytes, video_id, time_stamp, page_title,
277 initial_length, remaining_bytes
279 base_filename, file_length, video_link
283 2. --renew (now obsolete)
284 3. --youtube-user (broken since 2.1)
285 4. --youtube-pass (...)
297 1. Long options: aliases (e.g. --output_file | --output-file | --outputfile)
298 2. --filename-format: new specifiers, rename some of the old ones
299 3. --format: exit with an error if id is not recognized by clive
300 4. Rename: -r -> -l (--recall)
301 5. Many short options have been removed (or reused)
303 Developer-visible changes:
307 * Change: use a custom perltidyrc (closes issue #17)
308 - Thanks to Tong Sun <suntong001@gmail.com>
310 * Prerequisite changes
311 1. Written for Perl 5.10.0
312 o 5.8 probably OK although unverified
316 3. Removed prerequisites
321 4. Optional modules (no changes from 2.1.14)
325 * Change: use decode_utf8 returned html only if html is valid utf8
327 * Installation changes
328 1. Makefile.PL (ExtUtils::MakeMaker) [see README]
331 1. Makefile (obsoleted by Makefile.PL)
332 2. clive.pod (merged into bin/clive)
333 3. clive.1 (generated from bin/clive)
335 * Remove: binmode STDOUT, ":utf8"
337 * Remove: extra decode_utf8
343 User-visible changes:
344 * Add: support for youtube-nocookie.com (closes issue #12)
345 * Add: fmt35 format ID for Youtube [closes issue #10 (and #13)]
346 * Fix: liveleak id parsing
347 * Rename: --format IDs for Youtube
351 * Rename: --output-video -> --output-file
352 * Rename: output:file -> output:filename_format (config file)
353 * Remove: --overwrite option (use --output-file instead)
355 Developer-visible changes:
356 * Add: tests/ytube-nocookies.url
360 * 2.2.0 is under development, read more here:
361 http://code.google.com/p/clive/wiki/WhatsNew_2_2_0
367 Developer-visible changes:
368 * Change: use perltidy defaults for style
369 * Change: BerkeleyDB and File::Path modules are now optional
370 ** Cache features depend on these modules
371 * Moved opted_mods initilization to init_opted_mods (-> loop)
377 User-visible changes:
378 * Change: --connect-timeout, --connect-timeout-socks default -> 30s
379 * Fix: timeout after file transfer starts
380 * Fix: title parsing with custom character class (closes issue #8)
381 * Add: redtube support; thanks to anonprn2@gmail.com (closes issue #11)
382 * Fix: typo in --emit-xml --help string
388 User-visible changes:
389 * Change: --youtube-pass: prompt for password if left undefined
390 * Change: --filename-format: default -> "%t.%s"
391 * Fix: "Use of uninitialized value $opts{"ytpass"} ..."
394 ** --connect-timeout-socks
395 * Cleanup: manual page
397 Developer-visible changes:
398 * Rename: auth_youtube subroutine -> loginto_youtube
401 * Youtube login does not work (throw an error, patches welcome!)
407 NOTE: This release changes the clive status messages. This will likely
408 break compatibility with utilities that depend on these messages.
409 The changes were made to enable using abby (0.2.0+) as a front-end
412 User-visible changes:
413 * Add abby (front-end) compatibility
414 - Change output similar to cclive
415 - http://code.google.com/p/abby/
416 * Add Youtube HD support (--format=mp4_hd)
417 - Thanks to bf for the patch (via FreeBSD ports)
418 * Add tv.cctv.com support [issue #6]
419 - Thanks to suntong001 for data mining
421 - --limit-rate (config: [http]:limit_rate)
422 - --hosts (print supported hosts to stdout)
423 - --stderr (redirect all output to stderr instead of stdout)
424 * Improve file suffix parsing from content-type
425 - Rewrite the content-type handling block in query_video_length
426 - Use whatever is found after slash in the content-type string
427 - Use "flv" for exceptions (e.g. octet-*, x-flv, etc.)
428 * Replace dashes in video IDs with underscores before using them in fnames
429 * Change --version output
430 * Fix reading from stdin (closes Debian #522422)
431 - Assume reading from stdin only if queue and argv arrays are empty
432 * Fix exit with status code
433 - Exit 0 on success, and >0 if an error occurs
435 Developer-visible changes:
436 * Update examples/config with limit_rate
437 * Rename: configrc -> examples/config
438 * Replace the monstrous if-elsif-$host code block with a loop
444 User-visible changes:
445 * Fix GoogleVideo "id not found" handling
446 - Symptom: exit after "process page..." message
447 - Cause: no check for undefined id value
448 * Fix GoogleVideo support
449 - Symptom: see above fix
450 - Cause: HTML changes to the video pages
451 * Fix URI::Escape import for GoogleVideo redirects
452 - Symptom: crash with "Undefined subroutine &URI::Escape::uri_unescape"
453 - Cause: recent script start optimizations; a missing 'require'
455 - Symptom: "expected different content-type"
456 - Cause: website content-type changes
458 Developer-visible changes:
459 * Cleanup code (style-wise)
460 - perltidy -gnu -l=72
466 User-visible changes:
467 * Keep URL queue order
468 - Remove duplicate URL checking
469 * Remove --modver, change --version output
470 * Update manual page (clive.1, clive.pod)
471 - Rewrite EXAMPLES section
472 - Cleanup other sections
473 * Config file: add support for [output]:format (--format)
474 * Port dailymotion support from cclive
476 Developer-visible changes:
478 - Replace Perl module checking with for-loops (incl. optional)
479 * Optimize script start time
480 - Replace "use $module" with "require $module" where possible
481 * Update LICENSE, configrc
483 - Add [output]:format to configrc
484 * Add tests/dmotion.url
490 User-visible changes:
491 * Add --output-video option
492 * Add option groups to --help output
493 * Add --stream-exec and --stream options
495 * Config file: add support for
496 - [commands]:exec (--exec)
497 - [commands]:stream (--stream-exec)
498 - See configrc for an example
499 * Fix --exec; should no longer spew out "not found" shell errors
500 * Change default filename format to "%t_%d_%i.%s"
501 * Change to use "file is already fully retrieved" error message
502 * Port resizing progressbar from cclive
504 Developer-visible changes:
505 * Add tests/lastfm.url and tests/multi.lst
506 * Remove XML::Simple dependency
508 * Implement SIGWINCH handling
509 - Resize progressbar to match terminal width
510 - Progressbar width defaults to 80 (chars)
511 - OS must support SIGWINCH
512 - Requires Term::ReadKey module
513 - clive works without the module but will ignore any SIGWINCH events
514 * Touch cclive.1, cclive.pod, INSTALL and Makefile files
518 * Buried clive-list, grew weary of fighting spam with bare fists
519 - email the author directly or use the issue tracker, thanks
525 User-visible changes:
526 * Renamed the following options
527 - --noproxy => --no-proxy
528 - --noextract => --no-extract
529 - --nologin => --no-login
530 * Butchered --help output
531 - much less verbose now
532 - relocated manual page data to clive.pod
533 * Removed --manual option
534 * Obsoleted "play" and "rencode" config keys in "commands" section
535 * Added --exec option which resembles cclive's --exec command
536 * Removed --play, --noplay, --rencode and --norencode options
538 * Changed --paste short option from '-x' to '-p'
539 * Removed wmv format (breakcom)
540 - persistant HTTP/404 (not found) error
542 Developer-visible changes:
544 - additional INSTALL variables
546 - can be used to disable module checks
547 - removed "man" from "all"
548 - removed "clean" (useless)
554 User-visible changes:
555 * Fixed sevenload support
561 User-visible changes:
562 * Fixed Youtube login which sometimes reported failed attempts as OK
563 - Youtube defaults web interface language based on users ip
564 - clive looks for English words in the post-login page to confirm login
565 * Tweaked --version output
567 Developer-visible changes:
568 * Removed metacafe remnants
569 * Fixed bug in time2str subroutine which caused incorrect calculations
571 * removed genman script (obsoleted by "make man")
572 * Added a rudimentary (GNU) Makefile
573 * Cleaned up translate_embed subroutine
576 * Removed the clive-announce and clive-users mailing lists
577 - New release announcements will be posted via Freshmeat alone from now on
578 - Subscribe to the project at http://freshmeat.net/projects/clive
579 * Created the clive-list@googlegroups.com mailing-list
580 - For support questions, patches and bug reports
581 - Issue tracker will remain for tracking bugs
582 - Send an email to clive-list-subscribe@googlegroups.com to subscribe
588 User-visible changes:
589 * Documented missing "%t" (--filename-format) in manual page
590 * Tweaked "localfile_length == remotefile_length" message
591 * Renamed CLIVE_CONFIGDIR environment variable to CLIVE_HOME
593 Developer-visible changes:
594 * Cleaned up manual page
596 * Cleaned up FreeBSD details (INSTALL)
602 User-visible changes:
603 * Removed "See --manual" from --version output
604 * Added support for evisortv
605 - http://code.google.com/p/clive/issues/detail?id=4
606 * Fixed: reused incorrectly last http error code for remaining urls in queue
607 * Fixed: progressbar reported 100% even if error had occurred (e.g. http/403)
609 Developer-visible changes:
610 * Cleaned up subroutines
616 User-visible changes:
617 * Fixed "Requested range was not delivered by the server (http/33)" w/ -c
618 - this would occur with a batch of URLs while using the --continue option,
619 e.g. if the first file was continued, the following page fetches exited
621 * Added --savebatch option
622 * Added --overwrite option
623 * Replaced the buggy progressbar with a "lazy man's progressbar"
624 * Fixed "print() on closed filehandle STDOUT" with --grep+delete
625 * Fixed progressbar display with --continue
627 Developer-visible changes:
629 * Started to bundle src tarball with clive.1 manual page
630 * Added genman script which wraps pod2man with common options
631 * Fixed a potential div by zero bug in calc_progress
637 NOTE: Changed license GPL3 => ISC/OpenBSD.
639 User-visible changes:
640 * Added --modversion option
641 * Replaced Term::Progressbar with a built-in thermometer progressbar
642 * Removed --progress=bar:nominor support due to above changes
643 * Added liveleakcom support
644 - http://code.google.com/p/clive/issues/detail?id=3
646 Developer-visible changes:
647 * Made IO::Pager module optional
649 * Cleaned up --progress option description
650 * Changed to require WWW::Curl 4.05+
651 * Removed Term::Progress module dependency
652 * Fixed a potential bug in calc_progress (improper $rate init)
654 * Made Expect module optional
660 User-visible changes:
661 * Fixed gvideo/mp4 support (http/404 even if the extraction link existed)
662 * Changed "found redirect" message
663 * Changed --play, --rencode status messages to use file basename
665 Developer-visible changes:
666 * decode_utf8($page): removes extra steps, works better with HTML::TokeParser
667 * Added missing Expect module version to INSTALL
674 This aims to be the last beta before the release of 2.0 in December 2008.
675 Please report any remaining bugs to the issue tracker at:
676 http://code.google.com/p/clive/issues/list
678 User-visible changes:
679 * Removed metacafe support until fixed (see known issues)
680 * Changed Youtube login to report if there were "too many login failures"
681 * Fixed Youtube login support: reported failures incorrectly as OK
682 * Added support clivepass(1) utility (http://code.google.com/p/clive-utils)
683 * Added support for metacafe and sevenload embed URL translation
685 Developer-visible changes:
687 * New prerequisite module: Expect
688 * Updated FreeBSD + Ubuntu details (INSTALL)
689 * Cleaned up some code
690 * Rewrote embed URL translation
693 * Metacafe support broken (http/404)
699 User-visible changes:
700 * Fixed pasting from clipboard containing multiple URLs
701 * Fixed page fetch progress glitch that occured with 1+ batches
702 * Added --agent option
703 * Added --proxy option
704 * Added --savedir option
705 * Added --cclass option
706 * Added --filename-format option
707 * Added --show-format option
708 * Added --youtube-user and --youtube-pass options
709 * Added --emit-xml option
710 * Added --progress=[none|bar|dot]
711 * Replaced --noprogress with --progress=none
712 * Added avg. transfer rate notification
713 * Fixed filename printing for existing output files (e.g. output.flv.1)
714 * Added --play option
715 * Fixed metacafe support (error: failed to extract videoCDNURL)
716 * Added --rencode option
718 Developer-visible changes:
719 * Ported --emit-csv option from 1.x with minor changes
720 - Backwards compatible with 1.x
721 - Prints additional video details
722 - $length, $video_id, $time_stamp, $page_title, $cont_from, $remaining
723 * Changed to prefer 'dot' bar if the output is not a TTY
724 * Replaced progress bar with Term::ProgressBar (new module prerequisite)
725 * Cleaned up queue processing subroutine
731 User-visible changes:
732 * clive now ignores input lines that start with the '#' character
733 * Added support for CLIVE_CONFIGDIR environment variable
734 * Ported lastfm video support from 1.x
735 * --version now displays XML::Simple
736 * Changed "= Play:" to "=> Playing ..."
737 * Playing occurs now subsequently after going over the URL batch
739 Developer-visible changes:
740 * Removed a google video redirect (commented out, see code)
741 * Added the AUTHORS file
742 * XML::Simple was missing as a prerequisite in INSTALL
743 * Made minor syntactic tweaks to the source code
744 * Fixed missing --quiet check with the "=> Play" message
750 A complete overhaul and rewrite of 1.x.
752 User-visible changes:
753 * Users are expected to use the --continue/-c option to resume transfers
754 * Configuration file format (=> INI), path was changed (=> ~/.config/clive)
755 * All extractions now assume flv as the default download format
756 * Users are expected to understand when/how use the --format/-f option
757 * Reading input from multiple sources is possible (e.g. % clive -x URL URL)
758 * Added: --grep: used to grep and recall cached URL entries
759 * Added: --background and --output=logfile, --append=logfile options
760 * Many old cmdline and config options were removed due to new design
761 * Improved GoogleVideo support: handles redirects to other hosts
762 * Pager is used where needed (e.g. --show/-s)
763 * Changed default output file naming format to "%n-(%i)-[%d].%s"
764 * Removed: guba (deadweight), myvideo (borked), dmotion (bitches @ anon users)
766 Developer-visible changes:
768 - See the INSTALL file for prerequisites
769 * No installation: all-in-one script (excl. the 3rd party modules)
770 * Cache is now an integral part of clive, see CACHE in the manual page
771 * Embedded the --manual page (Pod)
772 * Improved Youtube login: removed is_adult HTTP get (done manually now)
773 * Grab the file suffix from the Content-Type header
774 * Improved GoogleVideo support: mp4 detection no more relies on English words
775 * Improved Metacafe support: removed unnecessary "familyfilter" HTTP get
779 - 2.x+ at http://googlecode.com/p/clive/
780 - 1.x remains at http://gna.org/projects/clive
782 - 2.x+ at http://googlecode.com/p/clive/
783 - 1.x remain at http://dl.gna.org/clive
785 - http://clive.sf.net/
786 - http://home.gna.org/clive redirects to the above
788 - 2.x+ at git://repo.or.cz/clive.git
789 - 1.x remains at svn://svn.gna.org/svn/clive
790 * Issue tracking: Moved
791 - 2.x+ at http://googlecode.com/p/clive/issues/
792 - 1.x bug tracker remains at http://gna.org/projects/clive
793 * Created http://googlegroups.com/group/clive-announce/
795 NOTE: See http://dl.gna.org/clive/CHANGES for 1.x.