jm + events   21

The log/event processing pipeline you can't have - apenwarr
So good. Apenwarr knows how to design a system.
Simple things don't break. Our friends on the "let's use structured events to make metrics" team streamed those events straight into a database, and it broke all the time, because databases have configuration options and you inevitably set those options wrong, and it'll fall over under heavy load, and you won't find out until you're right in the middle of an emergency and you really want to see those logs. Or events.
logging  scalability  klog  kernel  log-processing  events  embedded  ops 
5 days ago by jm
Osso
"A modern standard for event-oriented data". Avro schema, events have time and type, schema is external and not part of the Avro stream.

'a modern standard for representing event-oriented data in high-throughput operational systems. It uses existing open standards for schema definition and serialization, but adds semantic meaning and definition to make integration between systems easy, while still being size- and processing-efficient.

An Osso event is largely use case agnostic, and can represent a log message, stack trace, metric sample, user action taken, ad display or click, generic HTTP event, or otherwise. Every event has a set of common fields as well as optional key/value attributes that are typically event type-specific.'
osso  events  schema  data  interchange  formats  cep  event-processing  architecture 
september 2016 by jm
Hybrid Logical Clocks
neat substitute for physical-time clocks in synchronization and ordering in a distributed system, based on Lamport's Logical Clocks and Google's TrueTime.

'HLC captures the causality relationship like LC, and enables easy identification of consistent snapshots in distributed systems. Dually, HLC can be used in lieu of PT clocks since it maintains its logical clock to be always close to the PT clock.'
hlc  clocks  logical-clocks  time  synchronization  ordering  events  logs  papers  algorithms  truetime  distcomp 
june 2015 by jm
AWS Lambda Event-Driven Architecture With Amazon SNS
Any message posted to an SNS topic can trigger the execution of custom code you have written, but you don’t have to maintain any infrastructure to keep that code available to listen for those events and you don’t have to pay for any infrastructure when the code is not being run. This is, in my opinion, the first time that Amazon can truly say that AWS Lambda is event-driven, as we now have a central, independent, event management system (SNS) where any authorized entity can trigger the event (post a message to a topic) and any authorized AWS Lambda function can listen for the event, and neither has to know about the other.
aws  ec2  lambda  sns  events  cep  event-processing  coding  cloud  hacks  eric-hammond 
april 2015 by jm
Privacy Security Talk in TOG – 22nd April @ 7pm – FREE
Dublin is lucky enough to have great speakers pass through town on occasion and on Wednesday the 22nd April 2015, Runa A. Sandvik (@runasand) and Per Thorsheim (@thorsheim) have kindly offered to speak in TOG from 7pm. The format for the evening is a general meet and greet, but both speakers have offered to give a presentation on a topic of their choice. Anyone one interested in privacy, security, journalism, Tor and/or has previously attended a CryptoParty would be wise to attend. Doors are from 7pm and bring any projects with you you would like to share with other attendees. This is a free event, open to the public and no need to book. See you Wednesday.

Runa A. Sandvik is an independent privacy and security researcher, working at the intersection of technology, law and policy. She contributes to The Tor Project, writes for Forbes, and is a technical advisor to both the Freedom of the Press Foundation and the TrueCrypt Audit project.

Per Thorsheim as founder/organizer of PasswordsCon.org, his topic of choice is of course passwords, but in a much bigger context than most people imagine. Passwords, pins, biometrics, 2-factor authentication, security/usability and all the way into surveillance and protecting your health, kids and life itself.
privacy  security  runa-sandvik  per-thorsheim  passwords  tor  truecrypt  tog  via:oisin  events  dublin 
april 2015 by jm
sclasen/event-shuttle
UNIX system service [jmason: ie a sidecar] that collects events and reliably delivers them to kafka, relieving other services on the same system from having to do so. Journals events through bolt-db so that in the event of an kafka outage, events can still be accepted, and will be delivered when kafka becomes available.
kafka  messaging  ruby  go  events  fault-tolerance  queueing 
november 2014 by jm
Pub Standards to close
The next Pub Standards, on Thursday 13th November, will be the last one. When I started Pub Standards in August 2010, there wasn't very many meetups for people who build apps, interfaces and businesses. These days, there are loads! I don't feel that Pub Standards is needed anymore. It served it's purpose -- other meetups were formed, startups were founded, projects were created and people got hired. We had a good run :)
dublin  meetups  events  pub-standards  pubs  social  the-end 
october 2014 by jm
Open Invention Network Symposium on Open Source Software and Patents in Context
Dublin, 24th September 2014, hosted by Enterprise Ireland. Hosted by former Ubuntu
counsel (via gcarr)
via:gcarr  ubuntu  law  legal  open-source  floss  oss  oin  inventions  patents  swpat  software  ireland  ei  events 
september 2014 by jm
Haywire
kellabyte's hack in progress -- 'an asynchronous HTTP server framework written in C. The goal of Haywire is to learn how to create a server with a minimal feature set that can handle a high rate of requests and connections with as low of latency and resource usage as possible. Haywire uses the event loop based libuv platform layer that node.js is built on top of (also written in C). libuv abstracts IOCP on Windows and epoll/kqueue/event ports/etc. on Unix systems to provide efficient asynchronous I/O on all supported platforms.'

Outperforms libevent handily, it seems. Apache-licensed.
server  http  asynchronous  libuv  haywire  kellabyte  c  events  open-source  asl2 
april 2014 by jm
Creative Commons event in Dublin this Friday
'Maximising Digital Creativity, Sharing and Innovation', Event organised by Creative Commons Ireland and Faculty of Law, University College Cork, Lecture Theatre, National Gallery of Ireland, Clare Street entrance, Dublin 2, Friday 17 January 2014, 9.45 a.m. to 1 p.m. (via Darius Whelan)
creative-commons  ireland  dublin  events  talks  law  copyright 
january 2014 by jm
Storm at spider.io - London Storm Meetup 2013-06-18
Not just a Storm success story. Interesting slides indicating where a startup *stopped* using Storm as realtime wasn't useful to their customers
storm  realtime  hadoop  cascading  python  cep  spider.io  anti-spam  events  architecture  distcomp  low-latency  slides  rabbitmq 
october 2013 by jm
Event Bars - Craft Beer
craft beer kegs for hire in Dublin, Sligo, Limerick and Galway. Needs more Metalman, of course ;)
beer  ireland  craft-beer  keg-hire  events  parties 
february 2013 by jm
'The Unified Logging Infrastructure for Data Analytics at Twitter' [PDF]
A picture of how Twitter standardized their internal service event logging formats to allow batch analysis and analytics. They surface service metrics to dashboards from Pig jobs on a daily basis, which frankly doesn't sound too great...
twitter  analytics  event-logging  events  logging  metrics 
january 2013 by jm
John Carmack's .plan update from 10/14/98
John Carmack presciently defines the benefits of an event sourcing architecture in 1998, as a key part of Quake 3's design:

"The key point: Journaling of time along with other inputs turns a
realtime application into a batch process, with all the attendant
benefits for quality control and debugging. These problems, and
many more, just go away. With a full input trace, you can accurately
restart the session and play back to any point (conditional
breakpoint on a frame number), or let a session play back at an
arbitrarily degraded speed, but cover exactly the same code paths."

(This was the first time I'd heard of the concept, at least.)
john-carmack  design  software  coding  event-sourcing  events  quake-3 
november 2012 by jm
Data distribution in the cloud with Node.js
Very interesting presentation from ex-IONAian Darach Ennis of Push Technology on eep.js, embedded event processing in Javascript for node.js stream processing. Handles tumbling, monotonic, periodic and sliding windows at 8-40 million events per second; no multi-dimensional, infinite or predicate event-processing windows. (via Sergio Bossa)
via:sbtourist  events  event-processing  streaming  data  ex-iona  darach-ennis  push-technology  cep  javascript  node.js  streams 
october 2012 by jm
Facebook's New Realtime Analytics System: HBase to Process 20 Billion Events Per Day
Scribe logs events, "ptail" (parallel tail presumably) tails logs from Scribe stores, Puma batch-aggregates, writes to HBase.  Java and Thrift on the backend, PHP in front
facebook  hbase  scalability  performance  hadoop  scribe  events  analytics  architecture  tail  append  from delicious
march 2011 by jm
Akka
'platform for event-driven, scalable, and fault-tolerant architectures on the JVM' .. Actor-based, 'let-it-crash', Apache-licensed, Java and Scala APIs, remote Actors, transactional memory -- looks quite nice
scala  java  concurrency  scalability  apache  akka  actors  erlang  fault-tolerance  events  from delicious
march 2011 by jm
Irish Hackerspaces Week
hackerspaces in Dublin, Cork, Galway, Limerick and Belfast, running events all next week (08-14 - 08-22)
hackerspace  hacking  ireland  conferences  events  from delicious
august 2010 by jm
Radisson Blu Galway – I’d avoid for events « Damien Mulley
Damien shares his atrocious experiences with a Galway hotel, and (naturally) commenters from the hotel's IP address range pile on what looks like astroturf in the comments
comments  astroturf  radisson  galway  hotels  funny  events  customer-service  cluetrain  from delicious
april 2010 by jm

related tags

actors  akka  algorithms  analytics  animation  anti-spam  apache  append  architecture  asl2  astroturf  asynchronous  aws  beer  c  cascading  cep  clocks  cloud  cluetrain  coding  comet  comments  concurrency  conferences  copyright  craft-beer  creative-commons  customer-service  darach-ennis  data  design  distcomp  dublin  ec2  ei  embedded  eric-hammond  erlang  event-logging  event-processing  event-sourcing  events  ex-iona  facebook  fault-tolerance  floss  formats  funny  galway  gifs  go  hackerspace  hacking  hacks  hadoop  haywire  hbase  hlc  hotels  http  interchange  interviews  inventions  ireland  java  javascript  john-carmack  kafka  keg-hire  kellabyte  kernel  klog  lambda  law  legal  libuv  log-processing  logging  logical-clocks  logs  low-latency  meetups  messaging  metrics  node.js  oin  olympics  open-source  ops  ordering  oss  osso  papers  parties  passwords  patents  per-thorsheim  performance  privacy  pub-standards  pubs  push-technology  python  qcon  quake-3  queueing  rabbitmq  radisson  realtime  ruby  runa-sandvik  rx  scala  scalability  schema  scribe  security  server  slides  sns  social  software  spider.io  sports  storm  stream-processing  streaming  streams  swpat  synchronization  tail  talks  the-end  time  tog  tor  truecrypt  truetime  tv  twitter  ubuntu  via:gcarr  via:oisin  via:sbtourist 

Copy this bookmark:



description:


tags: