Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After dispose and restore obj.log is growing always #882

Open
Kosmikos opened this issue Oct 11, 2023 · 2 comments
Open

After dispose and restore obj.log is growing always #882

Kosmikos opened this issue Oct 11, 2023 · 2 comments

Comments

@Kosmikos
Copy link

Hi.
I use FASTER as key-value cache with restore it after re-run application.
And I have a problem with disk space.
Steps to reproduce

  1. Create store
    var config = new FasterKVSettings<Key, Value>(baseDir)
    {
        TryRecoverLatest = true,
        RemoveOutdatedCheckpoints = true,
        MemorySize = 1L << 10,
        PageSize = 1L << 9,
        EqualityComparer = new Key.Comparer()
    };
    var store = new FasterKV<Key, Value>(config);
  1. Upsert values to store
  using var session = store.NewSession(new SimpleFunctions<Key, Value>());
    var random = new Random();
    for (var ii = 0; ii < 100; ii++)
    {
        var key = new Key(ii % keyMod);
        var value = new Value(key.key + random.Next(keyMod));
        session.Upsert(key, value);
    }
  1. Take checkpoin and dispose all
    store.TakeHybridLogCheckpointAsync(CheckpointType.Snapshot).GetAwaiter().GetResult();
    store.Dispose();
    config.Dispose();
  1. Repeat step 1->2->3.

Result
File hlog.obj.log.0 grow after each iteration.
And we confised because in each iteration we upsert same keys.

I read documentation about checkpoint and do not found how fix it.

This is .gif about problem
SimpleChecks_j9jEP3DJRR

This is repo with code with promlem
https://github.com/Kosmikos/FASTERLogGrowth

Please explain what I'm doing wrong

@maoxianfly
Copy link

You can try to use store.CheckpointManager.PurgeAll();

@Kosmikos
Copy link
Author

Hello.
Where should I add this line?
I added here

    store.CheckpointManager.PurgeAll();
    store.TakeHybridLogCheckpointAsync(CheckpointType.Snapshot).GetAwaiter().GetResult();    
    store.Dispose();

Behavior has not changed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants