You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently if you are not satisfied with the provided options the only way to continue generation is to increase the beam limit and regenerate, It would be nice to be able to continue generation for another cycle.
The text was updated successfully, but these errors were encountered:
@cuyler72 This is actually a really interesting thought. The original implementation of the search was depth-first (once a beam is identified we rode its splits all the way until the end, then backend up) but I've since flipped it to breadth-first (for each beam we step one token at a time and keep lots of beams running at once). I originally did this to support parallelism but it turned out to have some other nice properties like being able to support max-suggestions coherently. Now that you mention it, BFS also enables a simple "Step Next Tokens" button that can either supplement or replace the depth and breadth controls. I will look into this one!
Currently if you are not satisfied with the provided options the only way to continue generation is to increase the beam limit and regenerate, It would be nice to be able to continue generation for another cycle.
The text was updated successfully, but these errors were encountered: