minus-squareWalnutLum@lemmy.mltoTechnology@beehaw.org•To grow, we must forget… but now AI remembers everythinglinkfedilinkarrow-up1·2 days ago“Infini-attention” isn’t perfect memory, it’s highly compressed representations of the entire history. (https://arxiv.org/html/2404.07143v1). Much like quantization. The longer the context gets the worse the compression makes recall. linkfedilink
“Infini-attention” isn’t perfect memory, it’s highly compressed representations of the entire history. (https://arxiv.org/html/2404.07143v1).
Much like quantization. The longer the context gets the worse the compression makes recall.