jm + compression 9
Mailinator(tm) Blog: How Mailinator compresses email by 90%
february 2012 by jm
Quite a lot of work for an extra 5% ;)
mailinator
lcs
caching
algorithms
compression
email
february 2012 by jm
lrzip
february 2012 by jm
'Lrzip uses an extended version of rzip which does a first pass long distance redundancy reduction. The lrzip modifications make it scale according to memory size. [...] The unique feature of lrzip is that it tries to make the most of the available ram in your system at all times for maximum benefit. It does this by default, choosing the largest sized window possible without running out of memory.'
zip
compression
via:dakami
gzip
bzip2
archiving
benchmarks
february 2012 by jm
snappy - A fast compressor/decompressor
march 2011 by jm
'On a single core of a Core i7 processorin 64-bit mode, it compresses at about 250 MB/sec or more and decompresses atabout 500 MB/sec or more. (These numbers are for the slowest inputs in ourbenchmark suite; others are much faster.) In our tests, Snappy usuallyis faster than algorithms in the same class (e.g. LZO, LZF, FastLZ, QuickLZ,etc.) while achieving comparable compression ratios.' Apache-licensed, from Google
snappy
google
compression
speed
from delicious
march 2011 by jm
Hacker News thread on a new string search algorithm
february 2011 by jm
Great comments -- the Burrows-Wheeler Transform is crazy stuff
strings
search
algorithms
burrows-wheeler-transform
sequencing
genome
compression
dna
string-matching
from delicious
february 2011 by jm
Blosc
october 2010 by jm
A high-performance compressor optimized for binary data -- 'designed to transmit data to the processor cache faster than a traditional, non-compressed, direct memory fetch via memcpy()' (via Bill de hOra)
via:dehora
compression
memcpy
caching
l1
software
memory
optimization
performance
python
pytables
from delicious
october 2010 by jm
Why Our Civilization's Video Art and Culture is Threatened by the MPEG-LA
may 2010 by jm
incredible. Almost every single modern camera capable of recording video now requires that you obtain a license from MPEG-LA to use recorded footage for commercial purposes. These clauses are currently not enforced, but could be. Horrifying (via Tony Finch)
via:fanf
patents
mpeg2
codec
compression
consumer-rights
copyright
legal
law
mpeg
h264
mpegla
codecs
from delicious
may 2010 by jm
Vlnt
november 2009 by jm
'A variable-length format for positive integers is defined where the high-order bit of each byte indicates whether more bytes remain to be read. The low-order seven bits are appended as increasingly more significant bits in the resulting integer value. Thus values from zero to 127 may be stored in a single byte, values from 128 to 16,383 may be stored in two bytes, and so on.' UTF8-ish compression, used in Avro
utf8
compression
utf
lucene
avro
hadoop
java
fomats
numeric
from delicious
november 2009 by jm
XZ Utils
october 2009 by jm
15% smaller than bzip, 30% smaller than gzip, and now shipped with Fedora and Ubuntu. uses LZMA2
xz
xzdec
gzip
bzip
compression
lzma
via:wmf
unix
compress
from delicious
october 2009 by jm
pigz
october 2009 by jm
'A parallel implementation of gzip for modern multi-processor, multi-core machines', by Mark Adler, no less
adler
pigz
gzip
compression
performance
concurrency
shell
parallel
multicore
zip
software
from delicious
october 2009 by jm
related tags
adler ⊕ algorithms ⊕ archiving ⊕ avro ⊕ benchmarks ⊕ burrows-wheeler-transform ⊕ bzip ⊕ bzip2 ⊕ caching ⊕ codec ⊕ codecs ⊕ compress ⊕ compression ⊖ concurrency ⊕ consumer-rights ⊕ copyright ⊕ dna ⊕ email ⊕ fomats ⊕ genome ⊕ google ⊕ gzip ⊕ h264 ⊕ hadoop ⊕ java ⊕ l1 ⊕ law ⊕ lcs ⊕ legal ⊕ lucene ⊕ lzma ⊕ mailinator ⊕ memcpy ⊕ memory ⊕ mpeg ⊕ mpeg2 ⊕ mpegla ⊕ multicore ⊕ numeric ⊕ optimization ⊕ parallel ⊕ patents ⊕ performance ⊕ pigz ⊕ pytables ⊕ python ⊕ search ⊕ sequencing ⊕ shell ⊕ snappy ⊕ software ⊕ speed ⊕ string-matching ⊕ strings ⊕ unix ⊕ utf ⊕ utf8 ⊕ via:dakami ⊕ via:dehora ⊕ via:fanf ⊕ via:wmf ⊕ xz ⊕ xzdec ⊕ zip ⊕Copy this bookmark: