jm + workflows   4

Nextflow - A DSL for parallel and scalable computational pipelines
Data-driven computational pipelines

Nextflow enables scalable and reproducible scientific workflows using software containers. It allows the adaptation of pipelines written in the most common scripting languages.

Its fluent DSL simplifies the implementation and the deployment of complex parallel and reactive workflows on clouds and clusters.


GPLv3 licensed, open source
computation  workflows  pipelines  batch  docker  ops  open-source 
12 days ago by jm
AWS Step Functions – Build Distributed Applications Using Visual Workflows
Looks like one of the few new AWS features that's actually usable and useful -- replacement for SWF
swf  lambda  workflows  aws  architecture 
december 2016 by jm
Pinball
Pinterest's Hadoop workflow manager; 'scalable, reliable, simple, extensible' apparently. Hopefully it allows upgrades of a workflow component without breaking an existing run in progress, like LinkedIn's Azkaban does :(
python  pinterest  hadoop  workflows  ops  pinball  big-data  scheduling 
april 2015 by jm
Gerrit, Git and Jenkins
This is the future of code review. Commit directly from your git checkout to the Gerrit code-review system; change is immediately web-visible and enters the review workflow; at the same time, Jenkins checks out the proposed change and runs the test suite; once it's approved, it automatically gets checked in. Brilliant!
git  coding  code-review  workflows  jenkins  gerrit  c-i  testing  automation  from delicious
february 2011 by jm

Copy this bookmark:



description:


tags: