Propagate trace and span id to the yaml configuration

[7DedSins]

Here is the system: I have a airflow dag that starts a gke pod. Inside that pod, I run a repository that has the following:
sourcefile.py
otel_helper.py
otel_config.yaml

Ignore the otel_helper.py file for now. Inside the sourcefile.py, I have a piece of code with nested span as the following structure:

with tracer.start_as_current_span(span_name) as span1:
.... some code
with tracer.start_as_current_span(span_name) as span2:
... some code

The yaml file is a simple otel yaml configuration with receiver having a regex pattern that will extract the log from a log file (Full path provided), service, processor and exporter. The logs are exported to elastic backend.

The problems: the logs extracted from the yaml that are sent to elastic are devoid of any trace and span ids. I want the trace and span ids of span2 from the sourcefile.py file code to be associated with the yaml logs that are sent to the elastic.

What I tried: I tried to create an environment variable, and associated that with the yaml logs under processor as attributes, but facing 2 blockers - 1. the logs sent to elastic as being sent as labels(custom attributes) thus elastic cannot arrange them in hierarchy, 2. tried to provide dynamic trace and span ids by assigning the environment variable to the span2 trace and span ids in the code, when printing Inside the py file, the environment variables are getting updated accordingly but the yaml file seems to have been configured by the default environment variable only and not getting associated dynamically.

Kindly assist with the problem statement. Thank you.