Why do we need to use z as a condition to get the probability distribution of x?

[ranck626]

Why do we need to use z as a condition to get the probability distribution of x? Can't we just use z as the next token? Isn't z also a continuous tensor?

[LTH14]

If you directly regress z with, say, L2 loss, then it is hard to model the probability distribution as it always output a deterministic z. We included such an ablation in Appendix D3.