Optimization = 3 accuracy issue for nanoGPT

[yizhuoz004]

python3 examples/nanogpt/example.py --input-text "What is the answer to life, the universe, and everything?" --seed=1

opt level = 0 gives:

What is the answer to life, the universe, and everything? The answer to Aquinas, the most important thinker

opt level = 3 gives:

Free View in iTunes

176 Clean The Myth

Attaching the engines and layer info. Not successful in finding any difference, the layer norm are handled by myelin kgen, e.g.

__myl_AddCasSumMulSubMulSumMulAddDivSqrMulMulAddCas_0xd1f813b22d1e2e4c4f3e29cee9f643dd_vec

opt-0-layer-info.json

opt-3-layer-info.json