jm + compression   9

lrzip
'Lrzip uses an extended version of rzip which does a first pass long distance redundancy reduction. The lrzip modifications make it scale according to memory size. [...] The unique feature of lrzip is that it tries to make the most of the available ram in your system at all times for maximum benefit. It does this by default, choosing the largest sized window possible without running out of memory.'
zip  compression  via:dakami  gzip  bzip2  archiving  benchmarks 
february 2012 by jm
snappy - A fast compressor/decompressor
'On a single core of a Core i7 processorin 64-bit mode, it compresses at about 250 MB/sec or more and decompresses atabout 500 MB/sec or more. (These numbers are for the slowest inputs in ourbenchmark suite; others are much faster.) In our tests, Snappy usuallyis faster than algorithms in the same class (e.g. LZO, LZF, FastLZ, QuickLZ,etc.) while achieving comparable compression ratios.'  Apache-licensed, from Google
snappy  google  compression  speed  from delicious
march 2011 by jm
Blosc
A high-performance compressor optimized for binary data -- 'designed to transmit data to the processor cache faster than a traditional, non-compressed, direct memory fetch via memcpy()' (via Bill de hOra)
via:dehora  compression  memcpy  caching  l1  software  memory  optimization  performance  python  pytables  from delicious
october 2010 by jm
Why Our Civilization's Video Art and Culture is Threatened by the MPEG-LA
incredible. Almost every single modern camera capable of recording video now requires that you obtain a license from MPEG-LA to use recorded footage for commercial purposes. These clauses are currently not enforced, but could be. Horrifying (via Tony Finch)
via:fanf  patents  mpeg2  codec  compression  consumer-rights  copyright  legal  law  mpeg  h264  mpegla  codecs  from delicious
may 2010 by jm
Vlnt
'A variable-length format for positive integers is defined where the high-order bit of each byte indicates whether more bytes remain to be read. The low-order seven bits are appended as increasingly more significant bits in the resulting integer value. Thus values from zero to 127 may be stored in a single byte, values from 128 to 16,383 may be stored in two bytes, and so on.' UTF8-ish compression, used in Avro
utf8  compression  utf  lucene  avro  hadoop  java  fomats  numeric  from delicious
november 2009 by jm
XZ Utils
15% smaller than bzip, 30% smaller than gzip, and now shipped with Fedora and Ubuntu. uses LZMA2
xz  xzdec  gzip  bzip  compression  lzma  via:wmf  unix  compress  from delicious
october 2009 by jm
pigz
'A parallel implementation of gzip for modern multi-processor, multi-core machines', by Mark Adler, no less
adler  pigz  gzip  compression  performance  concurrency  shell  parallel  multicore  zip  software  from delicious
october 2009 by jm

Copy this bookmark:



description:


tags: