12 Sep
2023
12 Sep
'23
8:53 a.m.
basically with beams you want to explore the most interesting branches of the tree of possible next tokens, and the implementor is challenged to retain all these branches so as to compare them, because their interestingness can change deep down inside them, and there are various ways to approach that [i usually implement a priority queue, and huggingface doesn't do this -- -- but something to rem [there is sense to approaches that don't do this.