-
Notifications
You must be signed in to change notification settings - Fork 4.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow for atomic appends to end of file:: Round 2 #76408
Comments
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label. |
Tagging subscribers to this area: @dotnet/area-system-io Issue DetailsLink to original proposal: #53432 (comment) I am not sure why this proposal never resulted in a provision to enable atomic appends, at least as an opt-in. The discussions on this topic are quite technical and dig deeper into the API's and OS layers than I feel comfortable explaining. But let me just explain my simple scenario and why it blocks me from deploying dotnet core to Linux, at least in my environment. EnvironmentIn our Linux environment we control the life cycle of our applications in our own way. It has been the way for deploying Java apps for some number of years. So, lets rule out containers or cloud. Logs are rotated with a utility called "logrotate". This is all orchestrated for all apps and again has worked great for years. It is natural then to start migrating dotnet core apps to Linux. Well imagine our surprise when before long apps with any kind of logging volume, during the moment logs are rotated we begin to find log files growing much larger than the limit. When we investigate the logs, we find them filled with NUL bytes. Basically, if the logger is writing 50MB deep into the file during log rotation, you will get a new file with 50MB of NUL bytes and then continue from there. It is a mess. After searching around, I found my way to the original proposal and this issue in serilog-sinks-file/issues/221. Interim SolutionOf course one could just control the log rotation themselves. But we want to keep things consistent. After all, one reason to consolidate on Linux is for consistency. So, we decided to just write to console (stdout). It is a good long term way to go after all, if we plan on going to containers. It turns out that redirecting output to a log file with ">>" solved our problem. W3CLoggerI am checking out W3CLogger . After moving off IIS and running behind Kestrel only, it was logical to investigate lighting up a w3c style log like we get from IIS. But the ASP.NET Core W3CLogger requires a file to log to. Ugh! Back to wishing I could turn on atomic appends. Please reconsider enabling atomic appends as an opt-in.
|
Link to original proposal: #53432 (comment)
I am not sure why this proposal never resulted in a provision to enable atomic appends, at least as an opt-in. The discussions on this topic are quite technical and dig deeper into the API's and OS layers than I feel comfortable explaining. But let me just explain my simple scenario and why it blocks me from deploying dotnet core to Linux, at least in my environment.
Environment
In our Linux environment we control the life cycle of our applications in our own way. It has been the way for deploying Java apps for some number of years. So, lets rule out containers or cloud. Logs are rotated with a utility called "logrotate". This is all orchestrated for all apps and again has worked great for years.
It is natural then to start migrating dotnet core apps to Linux. Well imagine our surprise when before long apps with any kind of logging volume, during the moment logs are rotated we begin to find log files growing much larger than the limit. When we investigate the logs, we find them filled with NUL bytes. Basically, if the logger is writing 50MB deep into the file during log rotation, you will get a new file with 50MB of NUL bytes and then continue from there. It is a mess. After searching around, I found my way to the original proposal and this issue in serilog-sinks-file/issues/221.
Interim Solution
Of course one could just control the log rotation themselves. But we want to keep things consistent. After all, one reason to consolidate on Linux is for consistency. So, we decided to just write to console (stdout). It is a good long term way to go after all, if we plan on going to containers. It turns out that redirecting output to a log file with ">>" solved our problem.
W3CLogger
I am checking out W3CLogger . After moving off IIS and running behind Kestrel only, it was logical to investigate lighting up a w3c style log like we get from IIS. But the ASP.NET Core W3CLogger requires a file to log to. Ugh! Back to wishing I could turn on atomic appends.
Please reconsider enabling atomic appends as an opt-in.
The text was updated successfully, but these errors were encountered: