Roadie-GCP

A helper container to execute a program on Google Cloud Platform.

Features:

• downloads necessary files from web and Google Cloud Storage,
• runs commands,
• uploads results to Google Cloud Storage.

Roadie-GCP takes one instruction file. The instruction file is simple YAML documents like

apt:
- nodejs
source: https://github.com/itslab-kyushu/youtube-comment-scraper.git
data:
- http://sample.com/run.sh
- gs://a-project/input/data:/tmp
run:
- npm install
- run.sh /tmp/data
result: gs://a-project/result/
upload:
- "*.out"

This example commands Roadie-GCP to install nodejs via apt, and download source codes from a Github repository. Then, it prepares to data from some web server and Google Cloud Storage, and run a command run.sh /tmp/data. Finally, it uploads stdout and results which have extension .out to a bucket in Google Cloud Storage. Roadie-GCP automatically shutdowns the virtual machine Roadie-GCP is running on, so you can minimize charge.

Usage

• Run
• Instruction file

API Reference

• auth module
• downloader module
• roadie module
• shutdown module

Indices and tables

• Index
• Module Index
• Search Page

License

This software is released under the MIT License.