
Genomes are long but redundant. Not great for pretraining. To fix that, we've developed a lossless tokenization strategy that shortens sentence length by >2000x. For pretraining, we achieve a 0.55 validation CE on All of Us. Public replication on 1KG linked below.




