mapreduce   11517

« earlier    

Azkaban is a batch workflow job scheduler created at LinkedIn to run Hadoop jobs. Azkaban resolves the ordering through job dependencies and provides an easy to use web user interface to maintain and track your workflows.
java  mapreduce  integration 
27 days ago by janpeuker
GitHub - bcongdon/corral: 🐎 A serverless MapReduce framework written for AWS Lambda
GitHub is where people build software. More than 27 million people use GitHub to discover, fork, and contribute to over 80 million projects.
golang  aws  mapreduce  serverless  lambda  etl 
6 weeks ago by geetarista
Introducing Corral: A Serverless MapReduce Framework
Around the same time, AWS announced native Go support for Lambda. Go’s short startup time, ease of deployment (i.e. single-binary packages), and general speed made it a great candidate for this project.

My idea was this: use Lambda as an execution environment, much like Hadoop MapReduce uses YARN. A local driver coordinates function invocation, and S3 is used for data storage.
golang  serverless  etl  MapReduce 
7 weeks ago by euler
GitHub - bcongdon/corral: 🐎 A serverless MapReduce framework written for AWS Lambda
Corral is a MapReduce framework designed to be deployed to serverless platforms, like AWS Lambda. It presents a lightweight alternative to Hadoop MapReduce.
etl  golang  mapreduce  aws-lambda 
7 weeks ago by kangas

« earlier    

related tags

advocacy  algorithm  algorithms  amazon  analytics  apache  app  architecture  article  asynchronous_programming  avro  avrò  awesome  aws-lambda  aws  bash  bi  big-data  big  bigdata  bigquery  blockchain  business  calcite  cascading  cassandra  cli  clojure  cloud  cluster  clustering  clusters  code  coding  collectors  compsci  computerscience  computing  concurrency  containers  coreos  course  cpp  criticism  cryptography  data/science  data  database  databases  datastore  dataviz  datawarehouse  dbms  delicious-import  delicious  dev  development  distcp  distributed  distributedsystem  distsys  docker  drill  editor  emr  engine  etl  example  facebook  fileformats  filter  flume  for  forthecomments  framework  functional  functionalprogramming  gae  git  github  go  go_tr  golang  google  graph  hadoop  hbase  hdfs  history  hive  howto  important  integration  java  java8  javascript  javascript_map  jobtracker  kmeans  kubernetes  lambda  local  logfiles  machinelearning  map  map_function  memory  mesos  metadata  mobile  mr  mr1  mr2  networking  node.js  nosql  nsa  opensource  openstreetmap  optimization  package  parallel  parkour  patterns  performance  pig  pipeline  presentation  processing  programming  python  pywren  r  rdbms  realtime  recommendation  reduce  reference  relational  relationship  review  rhadoop  ruby  s3  scala  scalability  scalding  schema  science  security  serverless  shell  shellscript  slides  smartcontracts  software  spark  sql  sqoop  statistics  storage  storm  stream  streaming  sysadmin  system  teaching  tech  techtalk  tez  tuning  tutorial  twitter  typescript  udf  video  web  webdev  yarn 

Copy this bookmark: