Add timestamps for pte generation in CI

[lucylq]

🚀 The feature, motivation and pitch

Regression tracking for aot export times: add some timestamps to indicate how long the llama export took:

executorch/.ci/scripts/test_llama.sh

Line 268 in d67fb52

$PYTHON_EXECUTABLE -m examples.models.llama.export_llama ${EXPORT_ARGS}

And fail if the export time is over some threshold. For now, we can set the threshold to be the time it takes to export the longest-running configuration.

The breakdown can look something like this:

1. Add timestamps for the export command in CI.
2. Observe the time it takes for each llama test in CI runs. Expect different export commands to show different export times.
3. Add a fail, if the export time exceeds the largest time observed in (2), with some buffer; I guess we expect some variability with each run.
4. Optional add-on: make (3) more fine-grained, and targeted per-command.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

[lucylq]

cc @digantdesai

[mroreo]

Hi I would love to give this PR a try I am new to executorch and would love to slowly work on it to learn more about the inner workings

[lucylq]

Hi @mroreo, thank you for your interest! Feel free to assign the issue to yourself.

If you have any questions or run into problems, you can comment here, or ask in the discord: https://discord.com/invite/Dh43CKSAdc

[mroreo]

perfect thank you! Would you be able to assign the issue to me?

[mroreo]

Okay, @lucylq can you let me know if this approach makes sense or if I am understanding the code properly.

1. benchmark_configs contains the information regarding each of the models. So ideally, we want to upload the expected export time there via this github action step https://github.com/pytorch/executorch/blob/main/.github/workflows/apple-perf.yml#L563

where the config would look like this

{
      "model": "Qwen/Qwen3-0.6B",
      "config": "hf_xnnpack_custom_spda_kv_cache_8da4w",
      "device_name": "apple_iphone_15",
      "variant": "public",
      "device_arn": "arn:aws:devicefarm:us-west-2:308535385114:devicepool:02a2cf0f-6d9b-45ee-ba1a-a086587469e6/3b5acd2e-92e2-4778-b651-7726bafe129d",
      "max_export_time": "20 minutes"
    }

2. Then when apple-perf.yaml workflow or the android-perf.yaml workflows triggers in the step export-models we would be able to retrieve the max_export_time and compare witht he current run time.