feat: Add run job traces

[bgins]

Review Type Requested (choose one):

• Logic - thorough check (from everybody doing review)

Summary

This pull request implements the following changes:

• Add OpenTelemetry apparatus
  • Add tracer provider
    • Add HTTP exporter
    • Add resource attributes, including service name, version, OS, architecture, GPU info, network (e.g. "testnet"), and address
  • Add --disable-telemetry CLI flag
  • Add telemetry_url config
    • Set endpoints for local development and production
  • Add telemetry_token config
  • Add GetOTelServiceName helper function and table test
• Add Resource Provider run job trace
  • Add span attributes with information about associated deal
  • Add span events for run job steps
  • Log deal ID
  • Debug log trace ID on starting the trace
• Disable telemetry in integration tests
• Move Version to system package to make it accessible as a resource attribute
• Redact private key from logs
• Debug log public address at initialization (all services)
• Debug log Job Offer ID in job creator (to correlate job creator run with resource provider run job trace)
• Upgrade to Go 1.22
• Add debug logs to test GitHub action workflow

Task/Issue reference

Closes: #293

Details

This pull request adds our initial tracing implementation with a single trace over resource provider job runs.

We collect the following information about resource providers:

• Operating system
• Architecture
• GPU (if any)
• Wallet address
• Network (dev, devnet, or testnet)

This information will also be collected for any service where we add tracing.

We collect the following information about jobs:

• Deal ID
• Job creator address
• Resource provider address
• Job offer CID
• Resource offer CID
• Module repo
• Module hash (the git hash, also maybe a tag)
• Result ID
• Transaction hash from adding the result on chain

The public address, job offer ID, deal ID, trace ID logs are identifiers for looking up traces.

How to test this code?

Run our forthcoming observability implementation (https://github.com/Lilypad-Tech/observability/tree/dev-ronelcanaria/feat-add-alloy):

./stack observe-up

Navigate to the Grafana Dashboard at http://localhost:3000. Start the full Lilypad stack, then run a job.

The resource provider should send the trace to the observability stack where it can be viewed in Home > Explore, select "Tempo" at the top, and search. It's possible to filter on run_job for the span name, but shouldn't be needed because we only have one trace.

[walkah]

Looks good and runs locally! 🚀

[walkah]

what's the purpose / value of having all these logs in the integration tests?

[bgins]

It's to help debug when the integration tests fail. While working on this pull request, there was an issue where there were failures in the runner environment only. Had to add logs to get to the bottom of it.

I think there's no harm in printing the logs? Makes it easier to see what may have failed.

[walkah]

this will be great to have for cross-referencing 🙏

[cla-bot]

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have the users @narbs91 on file. In order for us to review and merge your code, please open a new pull request and sign the Contributor License Agreement following the instructions in our contributing guide. Once the pull request is merged, ping a maintainer who will summon me again.