algorithms   67481

« earlier    

axiomhq/hyperloglog: HyperLogLog with lots of sugar (Sparse, LogLog-Beta bias correction and TailCut space reduction)
An improved version of HyperLogLog for the count-distinct problem, approximating the number of distinct elements in a multiset using 20-50% less space than other usual HyperLogLog implementations.

This work is based on "Better with fewer bits: Improving the performance of cardinality estimation of large data streams - Qingjun Xiao, You Zhou, Shigang Chen".


The core difference to other implementations are:

* use metro hash instead of xxhash
* sparse representation for lower cadinalities (like HyperLogLog++)
* loglog-beta for dynamic bias correction medium and high cardinalities.
* 4-bit register instead of 5 (HLL) and 6 (HLL++), but most implementations use 1 byte registers out of convenience.

In general it borrows a lot from the InfluxData's fork of Clark Duvall HyperLogLog++ implementation, but uses 50% less space.
golang  algorithms  hyperloglog  sketch  datastructure  github  approximation  cardinality 
yesterday by newtonapple
Designing an Authentication System: a Dialogue in Four Scenes
Description of how Kerberos was designed, as a dialogue between two hackers.
Algorithms  Programming  Kerberos  Crypto  Reference 
yesterday by tfortunato
The Power of Two Random Choices: A Survey of Techniques and Results
Clever technique for hash tables: use two hash functions, and always insert into the smaller of the two buckets obtained from the functions. This paper explores extensions of this technique to other collision-avoidance problems.
yesterday by alexbecker
Liblb - A suite of load balancing policies
It's an effort to implement and conduct studies on a variety of load balancing algorithms. Seeing how they work and perform, in regard of things such as Load Distribution, Cache Hits, and their Guarantees.
algorithms  networking 
yesterday by jnunemaker
Designing state machines | Drivy Engineering
State machines are a very powerful tool but are often underused in web development. The design process forces you to think hard about how you want to model your data, about the different objects lifecycles, about the way you want to expose your data and communicate with your whole team, and about the upcoming evolutions.
algorithms  articles  programming 
yesterday by maluta
Silicon Valley to Liberal Arts Majors: We Want You | Boston Review
Both The Fuzzy and the Techie and What Algorithms Want explore the broader question of work in a digital world. It is an old argument, whether technology will replace human labor or complement it, but it has been given new life by the recent dramatic developments in artificial intelligence.

In one of the better chapters in The Fuzzy and the Techie, Hartley argues that liberal arts are in general a complement to technology, not a substitute. The more tech we have, the more work there is those trained in the humanities. Hartley shows that the demand for many STEM careers is actually falling in the United States: the big growth is in areas such as health care and personal care, which are heavily social and full of non-routine, difficult-to-specify tasks—in other words, fields that are not well-suited for algorithmic solutions.

Finn is more worried that much work falls into what he calls an “implementation gap”: doing what machines cannot yet do but will do sooner or later. Sometimes the boundary between the two perspectives seems to vanish: Facebook and YouTube moderators and Uber drivers owe their jobs to technology platforms, even as they train their digital replacements by their own efforts.

There is no doubt that humans will become increasingly dependent on machines. Even when it comes to creative work, Finn describes how Netflix’s algorithms and detailed metrics frame the creative efforts of auteurs. But as they look to the future, Finn and Hartley part company on the nature of that dependence. Hartley sees technological innovation as an opportunity for the sciences and humanities to join together in harmony. Finn hopes for a more tension-filled relationship, in which “the role of the curator, the editor, and the critic is more important than ever.” I am with Finn, but then, I would be.
algorithms  culture  humanities  bias  tomslee  siliconvalley  politics  technology  education 
2 days ago by timcowlishaw

« earlier    

related tags

3dprint  @favorite  ai  algorithm  algorithmmarkets  analytics  anticipatory  api  approximation  art  articles  auditing  automation  balancing  bestpractices  bias  big_data  blockchain  buffers  c++  cardinality  career  code  coding  compression  compsci  computational_fabrication  computer-science  computer  cookbook  cpp  crypto  cryptocurrency  cs  culture  data-structures  data  datastreams  datastructure  datastructures  deeplearning  design  dj  economics  education  efficent  epistemology  ethics_of_algorithms  examples  facebook  facial  filetype:pdf  finance  funny  game  gamedev  games  geohashes  geolocating  git  github  go  golang  google  governance  hash  hashing  hashtable  higher_education  history  hll  humanities  hyperloglog  hyperspeculative  interactivepublishing  interesting  internals  interview  issue  java  jobs  jun17  k/v  kerberos  language  lauren  liesje  line-breaking  linux  load  loadbalance  loadbalancing  lookup  machinelearning  math  mathematics  media:document  mh  models  networking  newsfeed  nlp  northpointe  origami  papercraft  papers  parametrics  performance  philosophy  politics  predictive-policing  preparation  programming  python  question  questions  randomized  ratelimiting  recognition  recsys  reference  report  research  resiliency  resources  route  ruby  rust  security  service  services  siliconvalley  sketch  sketching  socialmedia  softwareengineering  speechrecognition  sports-newmedia  statistics  stock  string  study  tcpip  technology  tensorflow  tomslee  trading  trends  trust  tutorial  ux  versioncontrol  videos  visualisation  visualization  web  whitepapers  youtube 

Copy this bookmark: