A helper container to execute a program on Google Cloud Platform.
- downloads necessary files from web and Google Cloud Storage,
- runs commands,
- uploads results to Google Cloud Storage.
Roadie-GCP takes one instruction file. The instruction file is simple YAML documents like
apt: - nodejs source: https://github.com/itslab-kyushu/youtube-comment-scraper.git data: - http://sample.com/run.sh - gs://a-project/input/data:/tmp run: - npm install - run.sh /tmp/data result: gs://a-project/result/ upload: - "*.out"
This example commands Roadie-GCP to install
nodejs via apt, and download source codes from a Github repository. Then, it prepares to data from some web server and Google Cloud Storage, and run a command
run.sh /tmp/data. Finally, it uploads stdout and results which have extension
.out to a bucket in Google Cloud Storage. Roadie-GCP automatically shutdowns the virtual machine Roadie-GCP is running on, so you can minimize charge.
This software is released under the MIT License.