Print final response #146

Wambere · 2024-02-20T15:52:32Z

IMPORTANT: Where possible all PRs must be linked to a Github issue

Fixes #143

Engineer Checklist

I have run ./gradlew spotlessApply to check my code follows the project's style guide
I have built and run the efsity jar to verify my change fixes the issue and/or does not break the application

pld · 2024-02-20T16:16:37Z

importer/main.py

@@ -945,6 +948,9 @@ def main(
    else:
        logging.error("Empty csv file!")

+    if only_response:
+        print(final_response.text)


can't we pass this to logging instead of printing it?

@pld the idea was to have an option that only print/outputs the final response
So technically you would be able to get both logging and final response, or one or the other depending on what options passed
If we pass it to logging, then we will get everything else that generally gets logged
Unless there is a way to limit logging that I am missing?
Or maybe this is not a use case we want?

I think, but it's been a while, you can choose to send log to the screen based on the type of log message, so like if you send this to logging.info you can configure logging.info to be printed

But agree on use case, I don't understand that. Who requested this?

Here is an example of running the command with both options

$ python3 main.py --csv_file csv/locations/locations_full.csv --resource_type locations --only_response true --log_level info INFO:root:Start time: 12:00:40 INFO:root:Starting csv import... INFO:root:Reading csv file INFO:root:Returning records from csv file INFO:root:Processing locations INFO:root:Building request payload INFO:root:Posting request----------------- INFO:root:Request type: GET INFO:root:Url: https://fhir.labs.smartregister.org/fhir/Location/ba787982-b973-4bd5-854e-eacbe161e297 INFO:root:[200]: SUCCESS! INFO:root:Posting request----------------- INFO:root:Request type: GET INFO:root:Url: https://fhir.labs.smartregister.org/fhir/Location/0a04f1c2-de2a-4869-bab2-763d017e5316 INFO:root:[200]: SUCCESS! INFO:root:Posting request----------------- INFO:root:Request type: POST INFO:root:Url: https://fhir.labs.smartregister.org/fhir INFO:root:Processing complete! { "resourceType": "OperationOutcome", "text": { "status": "generated", "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><h1>Operation Outcome</h1><table border=\"0\"><tr><td style=\"font-weight: bold;\">ERROR</td><td>[]</td><td><pre>HAPI-0550: HAPI-0550: HAPI-0989: Trying to update Location/6c821162-0436-5830-9b4e-ea49027e0a7d/_history/1 but this is not the current version</pre></td>\n\t\t\t</tr>\n\t\t</table>\n\t</div>" }, "issue": [ { "severity": "error", "code": "processing", "diagnostics": "HAPI-0550: HAPI-0550: HAPI-0989: Trying to update Location/6c821162-0436-5830-9b4e-ea49027e0a7d/_history/1 but this is not the current version" } ] } INFO:root:End time: 12:00:47 INFO:root:Total time: 6.797744 seconds

So if we are to log.info the output instead of printing it, we'd just get the json response below all the other actual logging info. Let me look into filters and try to update the code

Also, side note, print is actually the recommended option to display console output according to the python docs here

This issue came from a discussion I had with @peterMuriuki, maybe he can add more context around use case

Once we embed this in fhir-web, we'll need its stdout to be in a form that can be easily parsed programmatically, like stringified json. There is a proposal here on what that could look like.

@peterMuriuki can you clarify if there is a use-case where we need only the output, or will we always need the log as well? so we can log the output together with everything else and then later on filter and get it from that?

yeah the use-case only needs the final output, I expect the final output to be a "a summary of the log" so the log is not needed.

pld · 2024-02-22T12:57:11Z

Ok then I think we'd want to return this, not print it, we can also log itOn Feb 22, 2024, at 07:41, Peter Muriuki ***@***.***> wrote: @peterMuriuki commented on this pull request. In importer/main.py:

@@ -945,6 +948,9 @@ def main(

else: logging.error("Empty csv file!") + if only_response: + print(final_response.text) yeah the use-case only needs the final output, I expect the final output to be a "a summary of the log" so the log is not needed. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

Wambere · 2024-02-23T10:57:11Z

Added a logging filter, so now to get just response:

$ python3 main.py --csv_file csv/locations/locations_full.csv --resource_type locations --only_response true
{ "final-response": {
  "resourceType": "OperationOutcome",
  "text": {
    "status": "generated",
    "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><h1>Operation Outcome</h1><table border=\"0\"><tr><td style=\"font-weight: bold;\">ERROR</td><td>[]</td><td><pre>HAPI-0550: HAPI-0550: HAPI-0989: Trying to update Location/6c821162-0436-5830-9b4e-ea49027e0a7d/_history/1 but this is not the current version</pre></td>\n\t\t\t</tr>\n\t\t</table>\n\t</div>"
  },
  "issue": [ {
    "severity": "error",
    "code": "processing",
    "diagnostics": "HAPI-0550: HAPI-0550: HAPI-0989: Trying to update Location/6c821162-0436-5830-9b4e-ea49027e0a7d/_history/1 but this is not the current version"
  } ]
}}

And to get all the logs:

$ python3 main.py --csv_file csv/locations/locations_full.csv --resource_type locations --log_level info
INFO:root:Start time: 13:55:39
INFO:root:Starting csv import...
INFO:root:Reading csv file
INFO:root:Returning records from csv file
INFO:root:Processing locations
INFO:root:Building request payload
INFO:root:Posting request-----------------
INFO:root:Request type: GET
INFO:root:Url: https://fhir.labs.smartregister.org/fhir/Location/ba787982-b973-4bd5-854e-eacbe161e297
INFO:root:[200]: SUCCESS!
INFO:root:Posting request-----------------
INFO:root:Request type: GET
INFO:root:Url: https://fhir.labs.smartregister.org/fhir/Location/0a04f1c2-de2a-4869-bab2-763d017e5316
INFO:root:[200]: SUCCESS!
INFO:root:Posting request-----------------
INFO:root:Request type: POST
INFO:root:Url: https://fhir.labs.smartregister.org/fhir
INFO:root:Processing complete!
INFO:root:{ "final-response": {
  "resourceType": "OperationOutcome",
  "text": {
    "status": "generated",
    "div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><h1>Operation Outcome</h1><table border=\"0\"><tr><td style=\"font-weight: bold;\">ERROR</td><td>[]</td><td><pre>HAPI-0550: HAPI-0550: HAPI-0989: Trying to update Location/6c821162-0436-5830-9b4e-ea49027e0a7d/_history/1 but this is not the current version</pre></td>\n\t\t\t</tr>\n\t\t</table>\n\t</div>"
  },
  "issue": [ {
    "severity": "error",
    "code": "processing",
    "diagnostics": "HAPI-0550: HAPI-0550: HAPI-0989: Trying to update Location/6c821162-0436-5830-9b4e-ea49027e0a7d/_history/1 but this is not the current version"
  } ]
}}
INFO:root:End time: 13:55:44
INFO:root:Total time: 4.120441 seconds

Wambere · 2024-02-27T13:08:52Z

@pld are there any other changes you would like added to this?

pld reviewed Feb 20, 2024

View reviewed changes

Wambere enabled auto-merge (squash) February 21, 2024 13:37

Wambere force-pushed the 143-only-response branch from c51d8f6 to ac81092 Compare February 21, 2024 13:49

Wambere force-pushed the 143-only-response branch from d1440e7 to b449222 Compare February 23, 2024 11:00

Wambere requested a review from pld February 23, 2024 11:02

Wambere added 3 commits February 27, 2024 16:03

Print final response

a2745e0

Update readme

576b98c

Add logging filter to only print final response

7b17dc2

Wambere force-pushed the 143-only-response branch from b449222 to 7b17dc2 Compare February 27, 2024 13:07

pld approved these changes Feb 27, 2024

View reviewed changes

Wambere merged commit 174645a into main Feb 27, 2024
4 checks passed

Wambere deleted the 143-only-response branch February 27, 2024 17:00

Wambere mentioned this pull request Jun 6, 2024

Change output to json #140

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Print final response #146

Print final response #146

Wambere commented Feb 20, 2024

pld Feb 20, 2024

Wambere Feb 21, 2024

pld Feb 21, 2024

Wambere Feb 22, 2024

peterMuriuki Feb 22, 2024

Wambere Feb 22, 2024 •

edited

Loading

peterMuriuki Feb 22, 2024

pld commented Feb 22, 2024 via email

Wambere commented Feb 23, 2024

Wambere commented Feb 27, 2024

Print final response #146

Print final response #146

Conversation

Wambere commented Feb 20, 2024

pld Feb 20, 2024

Choose a reason for hiding this comment

Wambere Feb 21, 2024

Choose a reason for hiding this comment

pld Feb 21, 2024

Choose a reason for hiding this comment

Wambere Feb 22, 2024

Choose a reason for hiding this comment

peterMuriuki Feb 22, 2024

Choose a reason for hiding this comment

Wambere Feb 22, 2024 • edited Loading

Choose a reason for hiding this comment

peterMuriuki Feb 22, 2024

Choose a reason for hiding this comment

pld commented Feb 22, 2024 via email

Wambere commented Feb 23, 2024

Wambere commented Feb 27, 2024

Wambere Feb 22, 2024 •

edited

Loading