jm + forecasting   4

Palantir has secretly been using New Orleans to test its predictive policing technology - The Verge
Predictive policing technology has proven highly controversial wherever it is implemented, but in New Orleans, the program escaped public notice, partly because Palantir established it as a philanthropic relationship with the city through Mayor Mitch Landrieu’s signature NOLA For Life program. Thanks to its philanthropic status, as well as New Orleans’ “strong mayor” model of government, the agreement never passed through a public procurement process.

In fact, key city council members and attorneys contacted by The Verge had no idea that the city had any sort of relationship with Palantir, nor were they aware that Palantir used its program in New Orleans to market its services to another law enforcement agency for a multimillion-dollar contract.

Even James Carville, the political operative instrumental in bringing about Palantir’s collaboration with NOPD, said that the program was not public knowledge. “No one in New Orleans even knows about this, to my knowledge,” Carville said.
palantir  creepy  surveillance  crime  forecasting  precrime  new-orleans  us-politics  privacy 
18 days ago by jm
Eircom have run out of network capacity
This is due in part to huge growth in the data volumes and data traffic that is transported over our network, which has exceeded our forecasted growth. We are making a number of improvements to our international connectivity which will add significant capacity and this work will be completed in the next two or three weeks.

Guess this is what happens when Amazon poach your IP network engineers. doh!

More seriously though, if you're marketing eFibre heavily, shouldn't you be investing in the upstream capacity to go with it?
eircom  fail  internet  capacity  forecasting  networking 
november 2014 by jm
Accuweather long-range forecast accuracy questionable
"questionable" is putting it mildly:

Now to to the point: Are the 25-day forecasts any good? In a word, no. Specifically, after running this data, I would not trust a forecast high temperature more than a week out. I’d rather look at the normal (historical average) temperature for that day than the forecast. Similarly, I would not even look at a precipitation forecast more than 6 days in advance, and I wouldn’t start to trust it for anything important until about 3 days ahead of time.
accuweather  accuracy  fail  graphs  data  weather  forecasting  philadelphia 
june 2013 by jm
paperplanes. Monitoring for Humans
A good contemplation of the state of ops monitoring, post-#monitorama. At one point, he contemplates the concept of automated anomaly detection:
This leads to another interesting question: if I need to create activity to measure it, and if my monitoring system requires me to generate this activity to be able to put a graph and an alert on it, isn't my monitoring system wrong? Are all the monitoring systems wrong? [...]

We spend an eternity looking at graphs, right after an alert was triggered because a certain threshold was crossed. Does that alert even mean anything, is it important right now? It's where a human operator still has to decide if it's worth the trouble or if they should just ignore the alert. As much as I enjoy staring at graphs, I'd much rather do something more important than that.

I'd love for my monitoring system to be able to tell me that something out of the ordinary is currently happening. It has all the information at hand to make that decision at least with a reasonable probability.

I like the concept of Holt-Winters-style forecasting and confidence bands etc., but my experience is that the reality is that anomalies often aren't sufficiently bad news -- ie. when an anomalous event occurs, it may not indicate an outage. Anomaly detection is hard to turn into a reliable alarm. Having said that, I have seen it done (and indeed our team has done it!) where there is sufficiently massive volume to smooth out the "normal" anomalies, and leave real signs of impact.

Still, this is something that Baron Schwartz (ex-Percona) has been talking about too, so there are some pretty smart people thinking about it and it has a bright future.
monitoring  networks  holt-winters  forecasting  confidence-bands  anomaly-detection  ops  monitorama  baron-schwartz  false-positives 
march 2013 by jm

Copy this bookmark: