asteroza + s3   111

Automate executing AWS Athena queries and moving the results around S3 with Airflow
Automating renaming/moving athena query results to a specific S3 bucket for later pickup
AWS  athena  apache  airflow  query  result  rename  move  S3  devops 
june 2019 by asteroza
Historical Log Analysis for Elastic Stack - CHAOSSEARCH
Get Kibana/ElasticSearch for your S3 data without using AWS elasticsearch...
AWS  S3  kibana  elasticsearch 
may 2019 by asteroza
Bucket Finder - DigiNinja
Use Bucket-Stream to locate buckets, then use Bucket-Finder to explore those buckets
AWS  S3  open  bucket  recon  security  hacking  pentesting  software 
april 2018 by asteroza
-= BuckHacker - The Bucket Search Engine for Hackers =-
amazon open S3 bucket search engine, AKA time to go bucket trawling...
security  hacking  pentesting  recon  open  S3  bucket  search  engine  OSINT 
february 2018 by asteroza
Amazon Redshift Spectrum - Run SQL queries directly against exabytes of data in Amazonn S3.
This would be for more interactive stuff or recently ingested data, whereas AWS Athena is for older data
AWS  redshift  spectrum  S3  serverless  database  service 
november 2017 by asteroza
Bucket Query - Index and search your S3 buckets with ease!
Basically like AWS Aurora, but support for more file types?
Amazon  AWS  S3  storage  bucket  file  index  query  search  service 
july 2017 by asteroza
very interesting S3-like object storage server
API  object  storage  server  software  go  opensource  S3  Delicious 
december 2016 by asteroza
Backblaze B2: The World’s Lowest Cost Cloud Storage
For nearline online storage, cheap to store/upload, gunning to be cheaper than AWS S3 and maybe glacier. If you don't need to interact with other AWS services, this might not be half bad...
backblaze  cloud  object  storage  backup  service  REST  API  file  sharing  S3  Delicious 
october 2015 by asteroza
Netflix/s3mper · GitHub
Interesting Netflix open source library for dealing with S3 storage inconsistency by caching file metadata in DynamoDB.
Netflix  opensource  amazon  AWS  S3  storage  consistency  library  software  dynamoDB  file  metadata  secondary  index  Delicious 
january 2014 by asteroza
Swiss Space Systems
S3 is partnering with EPFL for launching a space junk retrieval sat. Air launch on an A300, lifting body first stage, rocket second stage, then sat with deorbit capability.
systems  technology  launch  retrieval  space  garbage  research  Switzerland  junk  S3  swiss  debris  removal  air  TSTO  Delicious 
september 2013 by asteroza
High capacity 7000 mAhr shell case for a Galaxy S III cellphone that isn't stupidly thick.
shell  hardware  external  S3  cellphone  battery  capacity  high  case  devices  Samsung  galaxy  electronics  Delicious 
april 2013 by asteroza
Abstraction layer service that optimizes using hadoop on AWS with S3 stored datasets. Uses actual EC2 instance clusters specific to your job, rather than Elastic MapReduce. Now also has Presto, an SQL interface opensourced by Facebook for querying hadoop data directly.
Qubole  hadoop  cloud  cluster  service  platform  amazon  S3  SQL  BigData  query  SaaS  Presto  Delicious 
june 2012 by asteroza
Amazon Web Services Blog: The AWS Storage Gateway - Integrate Your Existing On-Premises Applications with AWS Cloud Storage
This might be useful, but it requires a large amount of local storage for now. The upcoming cache version might be suitable if you have reasonably fast local storage for the cache disk (SSD? SAN based even?), as an implementation of the long range monster iSCSI concept for operations. Better have a fast connection though...
AWS  amazon  EC2  S3  EBS  storage  gateway  VMware  VM  appliance  iSCSI  Delicious 
february 2012 by asteroza
IIJ、「Amazon S3相当」のAPIを備えたクラウドストレージサービスを提供開始 - ニュース:ITpro
IIJ is pimping something called GIO storage service FV/S as a domestic alternative to Amazon S3. REST API's and they have a java based SDK for you to tinker with. Still seems more expensive than Amazon.
IIJ  cloud  storage  service  japan  alternative  amazon  S3  Delicious 
june 2011 by asteroza
s3ql - a full-featured file system for online data storage - Google Project Hosting
Hrm, this seems to use characteristics of ZFS over S3, by using SQLite as an underlying filesystem engine. I wonder if it's pulling something like a ZIL write cache to protect write data in flight to S3?
S3QL  FUSE  plugin  amazon  S3  distributed  filesystem  storage  linux  software  database  cloud  file  backup  Delicious 
april 2011 by asteroza
s3fs - Trac
This is the other s3fs, using python. Note the data loss warnings.
S3FS  FUSE  plugin  amazon  S3  distributed  filesystem  storage  linux  software  python  Delicious 
april 2011 by asteroza
s3backer - FUSE-based single file backing store via Amazon S3 - Google Project Hosting
Interesting workaround method of using Amazon S3 as a large distributed virtual disk, splitting storage into filesystem block sized files served through S3. Rather creatively gets over some issues, but I wonder how much pain is involved. Apparently standard 4K block sizes can really nail you for PUT/GET costs, so maybe someting like 256K blocks? But you have to watch out for host RAM consumption due to block mapping.
s3backer  FUSE  filesystem  plugin  amazon  S3  distributed  block  storage  compression  encryption  encrypted  loopback  disk  image  mount  linux  cloud  Delicious 
april 2011 by asteroza
Product Overview - Google Storage for Developers - Google Code
The gloves are off now. Google is gunning for Amazon's S3 service by offering a storage API/service. With Google Appengine providing an abstracted virtual computing service, Amazon's last remaining card is EC2 and Dynamo, and Google could easily provide access to a restricted area of their mapreduce/hadoop system to fight Dynamo.
google  storage  alternative  amazon  S3  cloud  file  data  hosting  CDN  googleFS  service  computing  API  Delicious 
may 2010 by asteroza
Twitter / tmae: Amazon EC2のデータセンタが東京に出来るらし ...
Amazon may be FINALLY getting serious about EC2 in Japan. They set up an office just recently, and according to this rumor, they having been courting IDC for datacenter space recently. Gee, only 4 years after the first rumor started...
Amazon  S3  EC2  japan  datacenter  rumor  Delicious 
february 2010 by asteroza
Asiajin " Amazon Opens Cloud Subsidiary In Japan
So Amazon AWS/EC2/S3 is rolling out the local office and may be putting up a japanese datacenter in the future. Except one thing doesn't quite make sense. I know for a fact that Amazon was standing up a datacenter in Japan back in 2007 with the implication that it was for AWS, so back then I figured it was only a short matter of time before EC2 had asia zone possibilities. Did that not work out, or was that datacenter exclusively for Amazon Japan's web ops and not for AWS?
amazon  AWS  EC2  S3  cloud  datacenter  subsidiary  japan  Delicious 
january 2010 by asteroza
Ubuntu Enterprise Cloud | Ubuntu
Private using tools that work almost seamlessly with EC2 out on the greater internet? Sounds good...
Ubuntu  enterprise  9.04  cloud  distribution  linux  Debian  KVM  virtualization  computing  amazon  AWS  s3  EC2  compatibility  Eucalyptus  Delicious 
june 2009 by asteroza
AWS Import/Export
Can you hear Iron Mountain pooping in their pants? Sure, you gotta use your own hardware to move the data, but this basically completes the trifecta of your data being everywhere, editable at anytime, at on-demand prices. Finally harnessing the bandwidth of a station wagon full of tapes barreling down a highway, there's really not many reasons left to not get all your data into the cloud.
amazon  AWS  S3  physical  data  import  export  transfer  service  uploading  downloading  offsite  load  migration  backup  disaster  recovery  cloud  grid  computing  storage  sneakernet  upload  Delicious 
may 2009 by asteroza
Amazon Elastic MapReduce
Big number crunching now on tap. I wonder what kind of startups have datasets big enough to benefit from this?
amazon  elastic  mapreduce  service  AWS  s3  cloud  computing  Hadoop  scalability  AMR  Delicious 
april 2009 by asteroza
ongoing · The Sun Cloud
Sun wants to go up against the Amazon AWS/EC2/S3 juggernaut by building a more open cloud offering with API access
cloud  virtual  service  alternative  datacenter  grid  computing  EC2  amazon  AWS  s3  HTTP  Sun  API  json  REST  VDC  Delicious 
march 2009 by asteroza
JumpBox | Instant Infrastructure | JumpBox Inc.
Free EC2 trials of popular opensource server packages for an hour. Not shabby...
virtual  service  opensource  vendor  EC2  amazon  AWS  s3  demo  support  appliance  AMI  JumpBox  vitualization  Delicious 
march 2009 by asteroza
Amazon Web Services Developer Community : Load data into S3 via hard drives? ...
Amazon thinking about bulk data loading into S3? Hard to beat a station wagon full of tapes still.

A real bulk solution is an integrated 40 foot container. Half resembles a Sun Blackbox loaded with disgusting amounts of Thumper storage boxes and 10Gbps network connectivity. The other half of the container contains an integrated genset/UPS/chiller and fiber optic cable reels. Roll up this container on a truck in a customer parking lot, then snake the cable into the customer's datacenter. The point is to make it painfully easy for a customer to dump bulk data into a self contained/operated container. Companies already do this to a lesser extent with Iron Mountain fetching backup tapes weekly.
data  EC2  amazon  s3  upload  bulk  sneakernet  stationwagon  Delicious 
february 2009 by asteroza
« earlier      
per page:    204080120160

related tags

2.0  3GP  9.04  accelerated  access  added  addon  administration  air  airflow  ajax  algorithm  alternative  amazon  AMI  AMR  analysis  antivirus  apache  API  App  AppEngine  Appengine  appliance  architecture  archive  article  ASP  assist  AT&T  athena  automation  autoscaling  availability  avi  AWS  backblaze  backend  backup  battery  BigData  BigTable  billing  bittorent  block  blog  brackup  browser  bucket  bucknet  bulk  business  C++  CaaS  capacity  case  CDN  cellphone  certificate  CGM  chart  CLI  client  cloud  CloudBerry  cloudfront  cluster  clustered  clustering  CMS  code  cold  collaboration  command  comparison  compatibility  compression  computing  connector  consistency  console  consumer  content  control  converter  costing  could  crowdsourcing  CSA  cubesat  customer  dashboard  data  database  datacenter  Debian  debris  Delicious  delivery  demo  deploy  deployment  deprecation  design  desktop  development  device  devices  devops  DFS  disaster  disk  distributed  distribution  distro  DIVX  DNS  domain  download  downloader  downloading  drive  dropbox  duplicity  dynamo  dynamoDB  EBS  EC  EC2  economics  edge  elastic  ElasticLive  elasticsearch  electronics  encrypted  encryption  endpoint  engine  Enomalism  enterprise  enumeration  enumerator  EU  Eucalyptus  europe  export  extension  external  fact  feature  file  filesystem  filter  firefox  flash  free  frontend  ftp  FTPS  FUSE  galaxy  garbage  gateway  generated  generator  GnuPG  go  google  googleFS  GoPlan  GPG  GPGPU  GPU  grid  GUI  guide  hacking  hadoop  hard  hardware  hash  high  home  homepage  hosted  hosting  howto  HSI  HTTP  hugo  idea  IIJ  image  import  incremental  index  information  infrastructure  instance  interface  iPhone  iSCSI  item  japan  JAWS  json  JumpBox  JungleDisk  junk  Kafka  key  kibana  KVM  lake  lambda  large  launch  launcher  leak  leakage  LevelDB  library  limitation  line  linux  list  load  log  loopback  mac  machine  malware  management  mapreduce  market  mass  mathusalem  media  message  metadata  metrics  micropayment  migration  mobile  model  modeling  monitoring  mount  move  mpeg  mySQL  MySQL  name  nanosat  narabe  NAS  Netflix  network  Nirvanix  nonexistent  object  offload  offsite  ondemand  online  open  Openfount  opensource  origin  OS  OSINT  OSX  overlay  PaaS  page  parallel  partial  partition  path  pattern  payment  pays  pentesting  performance  perl  persistent  phishing  PHP  physical  platform  plugin  poll  portal  POSIX  PouchDB  predictive  Presto  pricing  primary  private  product  productivity  programming  projectmanagement  proxy  PyPI  python  Qubole  query  quicktime  rails  rdiff  rdiffdir  RDP  read  recon  recovery  redshift  reference  remote  removal  rename  replication  repository  requestor  research  reseller  REST  result  retrieval  reusable  RightScale  rocket  RoR  rsync  ruby  rumor  s3  s3backer  s3DAV  S3DFS  s3fs  S3hub  S3QL  S3SQL  SaaS  Samsung  SAN  satellite  Sawzall  scalability  scalable  Scalr  scan  scanner  SCP  screencast  SDK  search  secondary  security  select  server  serverless  service  services  setup  SFTP  sharing  shell  simple  site  SLA  small  SMEstorage  sneakernet  SOA  software  solution  space  spacecraft  spaceplane  spectrum  sql  squatting  ssh  static  stationwagon  status  storage  suborbital  subsidiary  Sun  support  swiss  Switzerland  Synaptic  sync  sysadmin  system  systems  table  Tahoe-LAFS  technology  timemachine  tips  tools  tracker  traffic  transfer  transparency  transportation  trend  trends  trial  tricks  TSTO  turnkey  tutorial  Ubuntu  upload  uploading  usage  utilities  utility  value  VAR  VDC  vehicle  vendor  video  virtual  virtualization  vitualization  VM  VMware  Voxcast  Voxel  warehouse  web  WebDAV  webDAV  webdev  webdrive  website  windows  wisdomofcrowds  Workspaces  xen 

Copy this bookmark: