Interestingly enough, Chinchilla 70B was trained mostly on text and beat domain-specific compressors PNG and FLAC in one experiment.

https://arxiv.org/abs/2309.10668

Not saying you are wrong. I assume that newer, domain-specific algorithms would still outperform the general Chinchilla algorithm, and there can be practical downsides if they involve large memory requirements, even if they result in more efficient compression.

I can't tell if you're serious, but Ariadne is right. Simple example: Flac will losslessly compress audio better than zip or gzip will. That's why it was invented.

@ariadne @BillSaysThis @davidgerard

• This is perhaps neither here nor there, but the X account making the originally-quoted post is https://www.aibyaakash.com , "AI by Aakash" (this is linked later in the same thread). The person seems fully AI-pilled and has several AI-themed substacks
• TurboQuant, or at least the QJL bit, sounds suspiciously like Locality-Sensitive Hashing. That's a well-known technique, and it can definitely do impressive things. When I tried my hand at startups I made heavy use of it (see https://bucci.onl/notes/Legit-tech ). In my use case I could get something like a 1,000-fold compression with acceptable accuracy loss. Basically LSH can be used to turn a long vector of floats into a comparatively short bitstring without losing too much of the geometrical information in the float vectors. Even one bit packs a ton of information
• The general problem of vector search that this method aims to address is an old one, and rotating or compressing the vectors is nothing new. In old school linear algebra things like diagonalization or SVD do this, for instance. I don't know if that's what they're doing but it's a general class of technique and a straightforward thing to try
• Vector quantization is, of course, also quite old. You experience it every time you listen to an MP3.

This algorithm is somehow only applicable to AI??