Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Implement In-Memory Caching for Faster Reads #235

Open
2 tasks done
asmit27rai opened this issue Jan 14, 2025 · 1 comment
Open
2 tasks done

Feat: Implement In-Memory Caching for Faster Reads #235

asmit27rai opened this issue Jan 14, 2025 · 1 comment

Comments

@asmit27rai
Copy link
Contributor

asmit27rai commented Jan 14, 2025

Describe the feature

Implement In-Memory Caching for Faster Reads

Description
Currently, the cache system reads from disk every time a request is made. This can be slow, especially for large directories. Implementing an in-memory cache would significantly improve performance by storing frequently accessed data in RAM.

Problem

  • Every request requires reading from the cache file, causing I/O overhead.
  • Large cache files slow down retrieval times.
  • No mechanism exists for quick lookups of previously fetched data.

Proposed Solution

  • Use lazy_static and HashMap<String, Vec<PathBuf>> to store recently accessed cache data in memory.
  • When fetching cache data:
    1. Check if the data exists in memory.
    2. If not, read from disk, store it in memory, and return the result.
    3. If yes, return directly from memory for a faster response.

Expected Benefits

  • Reduces disk reads for frequently accessed cache files
  • Improves performance for repeated requests
  • Makes cache retrieval instantaneous after the first load

Record

  • I agree to follow this project's Code of Conduct
  • I want to work on this issue
@asmit27rai
Copy link
Contributor Author

@Pranav0-0Aggarwal, Along with this, I have created two additional issues related to cache and memory optimization. I have already worked out solutions for all of them and would love the opportunity to implement these improvements.

These optimizations will significantly enhance performance, reduce disk I/O overhead, and improve the overall efficiency of the caching system. Let me know if I can proceed with these!

Looking forward to your approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant