Linux Certif

Toute la documentation sur la certification Linux LPI

clive

NAME

clive - Command LIne Video Extraction tool

SYNOPSIS

clive [ OPTION ]... [ URL ]...

DESCRIPTION

clive is an open source command line tool to extract videos and to bypass the need to use Adobe Flash in order to view user-generated content available on video-sharing websites.

OPTIONS

--version

Print version to stdout and exit.

-h, --help

Print out a verbose message describing the program options and exit.

-q, --quiet

Causes clive to be less verbose.

config: enable_verbose=["YES"|"NO"]

-u, --check-update

Check for update and prompt if a new version is available.

-n, --no-extract

Do not actually download any videos. Simulate only.

config: enable_extract=["YES"|"NO"]

-d, --clear-cache

Clear URL cache. See FILES.

-D, --clear-last

Clear last recorded URL (batch). See FILES.

-w, --write-conf

Write configuration file which will contain all supported commands. Attempts to autodetect ffmpeg(1) xine(1) xclip(1) and some other player commands. See FILES.

-L, --low-quality

Explicitly extract low-quality video. clive extracts better quality by default but switches to low quality if a high-quality link is not found. See HOSTS.

config: enable_low_quality=["YES"|"NO"]

-l, --recall

Recall the URL (batch) from the last runtime. clive saves the batch by default so it can be recalled with this option without having to re-type it. Note that the data gets overwrittenon every runtime unless the --no-recall option is toggled. See FILES.

See also: --cache

-R, --no-recall

Do not save URL (batch) for the --recall option.

config: enable_recall=["YES"|"NO"]

-e, --emit-csv

Emit video details in CSV format without extracting the videos.

-F, --rss

Causes clive to treat URLs as RSS/Atom feed URLs. The feeds are fetched, parsed and displayed with found video items. For example:

% clive --rss http://youtube.com/rss/user/lisanova/videos.rss

Feed URLs can also be saved to the configuration file. For example:

...
feed_url=http://youtube.com/rss/user/lisanova/videos.rss
feed_url=http://youtube.com/rss/user/spricket24/videos.rss
feed_url=http://youtube.com/rss/user/theonion/videos.rss
feed_url=http://youtube.com/rss/user/clipcritics/videos.rss
...

Once saved, run clive with:

% clive --rss

-c, --cache

Displays URL cache and re-extract videos using the collected cache data. See FILES.

See also: --no-cache --clear-cache

-C, --no-cache

Turn off all URL caching features. clive will not read or write cache. URL caching speeds up re-extracting videos. Instead of re-fetching the video page data from a host, clive reuses the cache data. For some hosts, e.g. Youtube, the page needs to be re-fetched as the saved extraction URL expires eventually.

If you are getting HTTP/410 or HTTP/403 errors, try disabling the caching. This is often an indication of using an expired extraction URL.

Note that clive disables the caching automatically if the Python SQLite3 module is not found.

config: enable_cache=["YES"|"NO"]

-S, --scan

Causes clive to scan the URLs for video links. These video links may be either embedded or normal video page links. For example:

% clive --scan http://fakesteve.blogspot.com
% clive --scan http://youtube.com

The found links are then visited, parsed and displayed for extraction. Note that clive currently scans for Youtube and GoogleVideo links only.

-Q, --no-confirm

Do not prompt for approval before each action. For example:

% clive --scan --no-confirm http://youtube.com

The above example would cause clive to scan the front page of Youtube for videos and then extract all of them without selecting them first.

Output Options

-s, --savedir=PATH

Specifies the save directory for the extracted videos.

config: output_savedir=["PATH"]

-a, --exists=ACTION

By default, clive will attempt to continue partially downloaded videos (local_file < remote_file).If this is not possible or not preferable, this option can be used to define the action if a file exists already.

rename ( default ) causes the program to skip the already existing file and re-download the file with a slightly different name. Typically the new file will contain the original filename and a random string appended to it.

overwrite causes the program to overwrite the existing file.

config: output_if_file_exists=["RENAME"|"OVERWRITE"]

-o, --output=FILENAME

Specifies the output filename for the video.

NOTE: Do not use this option when extracting a batch of videos.

-m, --filter=ACTION

Specifies the preferred method for filtering video page title information which is used to construct the video output filename. clive extracts the video page title of each the URL that it visits. After this the program strips any host specific strings from the title (e.g. site name) and proceeds to one of the three options:

1: Default filter is very strict about the used characters, accepting only characters of A..Z, a..z and 0..9 from the original video title to be used in the filename. This option can be used to customize the regular expression used by default ("A-Za-z0-9"), write a custom method that handles the filtering instead of clive or disable the filter altogether to use the whole video page title as a video filename.

If regular expression is used for action, clive calls Python's re.sub method with it. For example, the default regexp is: "A-Za-z0-9". The method is called in the following manner:

re.sub('[^%s]' % regexp, '', title)

You may want to refer to the re.sub documentation at:

http://docs.python.org/lib/node46.html

2: Custom Python method that clive calls instead of the re.sub method is also possible. This requires some basic knowledge of the Python programming language, however. Below is a very basic example that does nothing more but returns the original page title in lower-case:

def custom_output_filter(title):

return title.lower()

The method must be saved to ~/.clive/custom.py and when you run the program you must set the filter to "custom", e.g.:

% clive --filter=custom URL

clive which uses it for naming the video (original video title in lower-case).

3: Disabling the filtering is the quickest way to keep the original video title. This can, however, result in clive crashing if any of the characters used in the video page title cannot be used in a filename. Set the action to "no" if you wish to disable the filter, e.g.:

% clive --filter=no URL

config: output_title_filter=["regexp"|"NO"|"CUSTOM"]

See also: --format

-f, --format=FORMAT

Specifies the output filename format. Used to define how clive should construct the video output filename from all the available elements.

The default format is "%t.%e". Available format specifiers are: "%t" (title), "%e" (extension), "%i" (video id) and "%h" (host).

config: output_filename_format=["FORMAT"]

Connection Options

-y, --proxy=PROXY

Use HTTP proxy. clive reads the http_proxy environment setting by default. This option overrides the http_proxy setting (if any). Authentication information may be provided using normal URL constructs, e.g.:

% clive --proxy=http://user:host@foo:1234

config: http_proxy=["PROXY"]

-N, --no-proxy

Disable use of HTTP proxy altogether.

-t, --throttle=RATE

Causes clive to throttle the bandwidth to the specified rate (KB/s). Setting this option to 0 equals to unlimited.

config: http_throttle=["RATE"]

-g, --agent=STRING

Use HTTP user-agent string. By default, the program generates a random string each runtime.

config: http_agent=["STRING"]

--youtube=LOGINFO

Username and password for Youtube in format "username:password". Used to login and extract flagged content from the host. To use encrypted passwords stored with clive-passwd(1) set this option to "passwd:entry". For example:

% clive-passwd add yt
% clive --youtube=passwd:yt

config: youtube_login=["USERNAME|passwd:PASSWORD|ENTRY"]

See also: --passwd

--dmotion=LOGINFO

Like --youtube above but for Dailymotion.

config: dmotion_login=["USERNAME|passwd:PASSWORD|ENTRY"]

See also: --passwd

-P, --passwd=PASSPHRASE

Use passphrase for ~/.clive/passwd without prompting for it.

-G, --no-login

Do not attempt to login with provided username and password before extraction.

Playback Options

--player=PATH

Specifies path to a player command with any options. The format specifier "%i" must be used in the path. For example:

% clive --player=/usr/local/bin/xine %i

config: path_player=["PATH"]

-p, --play=FORMAT

Specifies the video format to play. The format can be anything as long as ffmpeg(1) recognizes the format. The only exception is "src" which tells clive to play the extracted video without re-encoding it to another format. Examples: "src", "mpg", "avi", "mp4". The --ffmpeg option must be used if the format is anything else than "src".

% clive --play=src URL
% clive --play=mpg --ffmpeg=/usr/bin/ffmpeg -y -i %i %o URL

config: play_format=["FORMAT"]

See also: --ffmpeg --reencode

Re-encoding Options

--ffmpeg=PATH

Specifies the path to the ffmpeg(1) command with any options. For example:

% clive --ffmpeg=/usr/bin/ffmpeg -y -i %i %o URL

Note that the format specifiers "%i" and "%o" must be used in the path. Using the "-y" option is recommended to prevent any subprocess lockups which will happen if the output file exists already and ffmpeg(1) prompts to confirm to overwrite it.

config: path_ffmpeg=["PATH"]

-r, --reencode=FORMAT

Causes clive to call ffmpeg(1) to re-encode the extracted video to another format. The format can be anything as long as ffmpeg(1) recognizes it. Examples: "mpg", "avi", "mp4". The --ffmpeg option must be used together with this option.

Note that if the --play option is being used with anything else than "src", you don't need to use this option. For example:

% clive --ffmpeg=/usr/bin/ffmpeg -y -i %i %o

--play=mpg URL

This would extract the video, re-encode it to mpg format and then play the mpg.

config: reencode_format=["FORMAT"]

Clipboard Options

--xclip=PATH

Specifies the path to xclip(1) command with any options. Note that the "-o" option must be used in the path. For example:

% clive --xclip=/usr/bin/xclip -o --paste

config: path_xclip=["PATH"]

-x, --paste

Causes clive to paste the URLs from X clipboard.

config: enable_paste=["YES"|"NO"]

EXAMPLES

Using a configuration file is recommended if you are going to use ffmpeg(1) xine(1) xclip(1) commands or any other commands. The easiest way to create the file is to run:

% clive --write-conf

And then edit the ~/.clive/config file manually. Note that clive will attempt to locate the commands automatically and save the ffmpeg, xine and xclip paths with the min. req. options. So in a perfect world you could ignore the --ffmpeg --player --xclip options used in the examples below.

Extract a video from Youtube:

% clive http://youtube.com/watch?v=dr3qPRAAnOg

Extract multiple videos from Youtube using UNIX pipes:

% cat url.lst
http://youtube.com/watch?v=dr3qPRAAnOg
http://youtube.com/watch?v=VlFGTtU65Xo
% cat url.lst | clive

Same as above but paste the URL batch from X clipboard:

% xclip -o
http://youtube.com/watch?v=dr3qPRAAnOg
http://youtube.com/watch?v=VlFGTtU65Xo
% clive --xclip=/usr/bin/xclip -o --paste

Extract and play a video from Youtube:

% clive --player=/usr/bin/xine %i --play=src

http://youtube.com/watch?v=dr3qPRAAnOg

Extract, re-encode and play the re-encoded video:

% clive --ffmpeg=/usr/bin/ffmpeg -y -i %i %o

--player=/usr/bin/xine %i --play=mpg
http://youtube.com/watch?v=dr3qPRAAnOg

Parse a Youtube RSS feed:

% clive --rss http://youtube.com/rss/user/clipcritics/videos.rss

Scan Youtube front page for videos:

% clive --scan http://youtube.com

Browse existing URL cache:

% clive --cache

Advanced example - paste (-x) an URL from X clipboard, scan (-S) the URL for videos, extract as low-quality (-L) and disable URL caching (-C):

% clive -xSLC

HOSTS

Youtube.com

format:flv/mp4 resume:mp4 quality:low/high

video.google.(com|au|ca|de|es|fr|it|nl|pl|uk|cn)

format:flv/mp4 resume:mp4 quality:low/high

dailymotion.com

format:varies resume:yes quality:varies

guba.com

format:flv resume:yes quality:low

metacafe.com

format:flv resume:yes quality:low

last.fm

Jzalae: "Last.fm recently added a video feature which appears to be a simple wrapping of YouTube videos, and the YouTube id appears in the Last.fm URL. The attached patch finds it from URLs like http://www.last.fm/music/Sylver/+videos/+1-fVD2TfSGkaw"

sevenload.com

format:flv resume:yes quality:low

break.com

format:flv/wmv resume:yes quality:low/high

FILES

~/.clive/config

User configuration file which can be created with the --write-conf option.

~/.clive/cache

The URL cache file. See the --cache --no-cache options. This is a normal SQLite3 database.

~/.clive/recall

Contains the last saved URL (batch). See also the --recall --no-recall program options.

~/.clive/passwd

Stores encrypted password information for host specific logins. This file is created and maintained using the clive-passwd(1) tool.

ENVIRONMENT

http_proxy

HTTP proxy setting. Authentication information may be provided using normal URL constructs, e.g.: http://user:host@foo:1234

BUGS

Dailymotion: login verification

clive cannot currently verify if the login succeeded. The only way to know for sure, is to login and then try extract a video that requires logging in will result in "error: extraction url not found".

Youtube: partially downloaded FLV videos

If the host is requested to continue a transfer, it appears to ignore the request and begins sending the file from the first byte rather than from the requested range. This seems to cause urlgrabber, which is used to transfer files, to append the new data incorrectly at the end of the existing file causing the file to become useless.

Because known workarounds do not exist, clive will always skip the existing file creating either a new file (default) or overwriting the existing file ( --exists option).

GoogleVideo: partially downloaded FLV videos

See above.

AUTHOR

Toni Gundogdu <legatvsgmailcom>. Bugs should be reported to the clive bug tracking system: https://gna.org/bugs/?func=additem&group=clive

SEE ALSO

clive-passwd(1)

COPYRIGHT

Copyright (C) 2007-2008 Toni Gundogdu

This program comes with ABSOLUTELY NO WARRANTY. You may redistribute copies of clive under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. You should have received a copy of of the General Public License along with this program. If not, see http://www.gnu.org/licenses/.