Ways to create infinite context?

[Panakotta00]

Hey guys, are there more or less performant ways to create a infinite context?

I initially intended to first tokenize new prompt additions, compare the current size and new size, then decide to either regenerate the full context based on the new tokens and old tokens but cropped, or just add the new tokens. But it seems there is no current way to add tokens directly. Only way I've found is to tokenize the same text twice, once for the checks, and once for the infer.

I've also seen issue #77 for naive but smarter way of creating an infinite context, but I question how to do a currently even more naive approach.

Are there any other ways to create something like that?