downloader module

Download objects from several services.

downloader.curl(url, dest)[source]

Download an object using curl.

Parameters:
  • url – Parsed URL specifying an object.
  • dest – Destination path.
Returns:

Path for the downloaded file.
downloader.download(url, unzip=True)[source]

Download an object specified by a url.

Url can have a destination path. The format is

  • scheme://host/path
  • scheme://host/path:dest

where dest is the destination path.

Scheme is one of http, https, gs, dropbox.

Parameters:
  • url – An extended url specifying the url of an object and an destination path.
  • unzip – If True and the object specified by the url is zipped, unzip them.
downloader.dropbox(url, dest)[source]

Download an object from dropbox.

Parameters:
  • url – Parsed URL specifying an object.
  • dest – Destination path.
Returns:

Downloaded filename.
downloader.gsutil(url, dest)[source]

Download an object using gsutil.

Parameters:
  • url – Parsed URL specifying an object.
  • dest – Destination path.
Returns:

Path for the downloaded file.