app log entries do not have request-Id metadata to associate with a request log entry

[timc13]

wild guess, but could it be because the actual request header is X-Appengine-Request-Log-Id and the code looks like it's looking for X-Appengine-Request-Id?

[bryanmau1]

Are you referring to the logs one can see on the developers console? A code snippet or screenshot would be helpful.

[timc13]

I'm referring to the logs i see in Google Cloud Logs. I converted an app from the standard runtime to flexible vm.

Before the conversion, my app logs would show up grouped with the request log. After the conversion they are in a separate app log, which is fine, but there is no request-id in the metadata that I can use to correlate to any request log entries.

[andrewsg]

In the Cloud Logging UI, can you click on the vertical ". . ." on the right-hand side and see if there's a "see logs from same request" or similar functionality exposed? This functionality is based on the "cloud trace ID" (in the metadata of the log) and will be the focus of our log correlation efforts going forward.

[timc13]

@andrewsg - I tried a bunch of things - and still no solution.

If I click "Show entries from same request" on the request log - this gets put in the filter: label:appengine.googleapis.com/request_id:570dc25000ff0265238e01b5860001737e656d69737361727 92d73746167696e670001323031363034313274323334313332000100

If I click "Show entries from same request" on the app log - this gets put in the filter:
label:appengine.googleapis.com/thread_id:140649891129088

I see a protoPayload.traceId in the request log, but traceId doesn't exist in any metadata in the app log. The app log doesn't even have protoPayload.

Hope this helps...

[timc13]

for context, this is our app constructor

def __init__(self, *args, **kwargs):
    super(MyApplication, self).__init__(*args, **kwargs)
    self.router.set_dispatcher(self.__class__.custom_dispatcher)

here's a screenshot of the app log

[timc13]

hey just following up, are you guys able to confirm this is indeed a bug? Or is there something wrong with the way I'm doing things.

[andrewsg]

Not sure quite yet.

[theacodes]

To clarify here:

• App Engine Standard will group logs by request in the Cloud Logging UI. App Engine Standard also only shows the app/request log.
• App Engine Flex, Compute Engine, and Container Engine will not group the logs by request, instead, it services a request log, a stderr log, and a stdout log.

The primary reason for this difference that because there's so much variance in how things can be set up, so there currently isn't a simple way to do log to request correlation. We are investigating this, but we have not yet committed to feature parity.

There are a few workarounds. You can always log in JSON format and log the request ID and use bigquery to analyze by request.

I am closing this issue for the time being.

[zeldrinn]

@jonparrott how does one log the request ID that gets logged in request_log from within application-level logs? Currently, I'm creating my own request ID for each request that comes in and just logging that, but that doesn't help with correlating my stdout/stderr logs with request_log entries.

[theacodes]

@kavriskive I'm not sure. Perhaps @andrewsg knows (iirc, he's working on some of this).

[timc13]

@kavriskive I'm doing this by grabbing the header value from X-Appengine-Request-Log-Id

[andrewsg]

Currently this isn't done automatically because we're in the process of unifying on Cloud Trace IDs as the canonical ID for a request. UI work is ongoing to make this appear seamless; for now there's an awkward adolescence. X-Appengine-Request-Log-Id is available in the meantime although a future version may not include this and instead include only the Trace ID.

[czzarr]

Hello,
Is there any update on this, or at least a walkthrough on how to set use the Cloud Logging API to emulate AES behavior?
I'm using AEF with nodejs
Thanks

[duggelz]

We know that users want this feature, and we are working on improving the logging experience across all of our compute platforms. We can't presently speak to any specific plans or dates.

[suhaasprasad]

Any update on how this can be accomplished even as a workaround? Even if it's not possible to group logs visually like with AES, there should at least be a way to view all logs for a specific request on AEF right?

[duggelz]

The Google Cloud Client library for logging (https://googlecloudplatform.github.io/google-cloud-python/latest/logging/usage.html) properly labels logs for per-request aggregation. You can use that library on App Engine Flex, GKE, GCE, GCF, and non-Google platforms. Unfortunately, it is not supported on App Engine Standard yet (googleapis/google-cloud-python#1893 (comment)).

Please note that if you are using one of the "python-compat" runtimes ("runtime: python27" for Managed VMs, "runtime: python-compat" for App Engine Flex), they have been frozen except for security updates.