How to return nginx error page if upstream returns server error (http code >= 500)

[gregor-4E]

Hi folks,
We have created our own plugin and we would like to return the nginx error pages using nginx error_page directive with it, if the upstream returns a server error (basically if upstream http status is >= 500). However, there is a catch. We also use the api-breaker plugin which uses the log phase to increase its error count for enabling the breaker, so we cannot simply use core.response.exit(500) in header_filter phase, because it would circumvent the api breaker log phase.

What we have tried so far:

1. Use core.response.exit(500) in header_filterphase -> Does everything we want, but renders the api-breaker plugin useless (log phase will not run)
2. Use ngx.status = 500 and ngx.exit(0) -> Will return upstream response body and content type headers, so not what we need, we need the nginx integrated error_page directive to work
3. Manually run the log phase of all global rules and all plugins in the current ctx after core.response.exit -> Works, but feels increadibly hacky, so would like to avoid it
4. Use response-rewrite plugin -> We have template variables in our error pages (e.g. zipkin trace id), how would we be able to make use of the nginx variable interpolation for error pages here?
5. Use proxy_intercept_errors on with nginx -> We also have error_page directive enabled for other error codes < 500 (in case other plugins exit with such statuses), however, if the upstream returns such a status, we would like to return the original response of the upstream

Of course, we are unsure, if we are even on the right track, so if you have ideas on how to solve this without a custom plugin, we are happy to hear your suggestions.

Thanks and best regards
Gregor

[rohitkrishna-marneni]

Hi @gregor-4E were you able to figure this out?

[gregor-4E]

Unfortunately not.
We ended up writing a custom plugin that reads a template file and returns it in the body_filter phase, if the upstream encountered a 5XX error.
We used the resolve_var function in API apisix/core/utils.lua to help us replace nginx variables within the template.
Hope that helps a little.
I'm afraid, I can't share the plugin code, because I'm not in that project anymore and don't have access to the source code

[rohitkrishna-marneni]

Thanks one last question. How can you make another request in body_filter phase? In a normal scenario, when someone hits an endpoint(in apisix) it goes through all those phases(access, rewrite etc) and in access I believe request reaches the upstream.

However we will only know the status code in header_filter or body_filter and from here how do we "redirect" the request to another url(custom error page)? From this body_filter page we wont be able to restart the request phases(access, rewrite etc) right?