Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve file-logs #123

Open
mark-meyer opened this issue Sep 5, 2017 · 0 comments
Open

Improve file-logs #123

mark-meyer opened this issue Sep 5, 2017 · 0 comments

Comments

@mark-meyer
Copy link
Member

The bus app currently uses lowDB to write JSON file-based logs to disk. The way lowDB works requires it to hold the entire log in memory and then overwrite the entire log file with each hit. The current db.json file is 62MB and about 2.7 million lines! That's a lot of overhead for each hit and since there are two log files the app write both with each hit.

I propose moving to something like neB https://github.com/louischatriot/nedb (or similar if anyone has another idea).

The main advantages neBD has are:

  1. It writes by appending to the file rather than rewriting the file.
  2. It writes one record per line, which will make it easier to rollover old logs by simply truncating them.

The interface is about as simple as lowDB. It still hold the db in memory, which is less than ideal, but manageable if we can automate log rollovers.

It also has indexing which might speed up the /logPlot which is currently slow (and I think it also blocks).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant