Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved storage capabilities #63

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

Kariton
Copy link
Contributor

@Kariton Kariton commented Jun 2, 2024

because the current implementation of storage.read() or storage.save() do not work well when they run simultaneously i moved the _redis from storage class to redis.
now the entire redis class is available within the storage class.
furthermore storage.delete() is now a thing.

i have a command that fetches content from a website. this takes time.
to make sure a user never receives the same post twice i used to:

  1. storage.read()
  2. do stuff
  3. storage_result.append(new_id)
  4. storage.save()

if the user runs the same command twice in a row it might happen that both coroutines read the same data and only the last one is actually saved.

now this is possible:

if c.bot.storage.redis is not None: # 'None' is default if storage is not redis
    c.bot.storage.redis.smembers(storage_key) # use redis.Redis().smembers() - list the set of stored data
    c.bot.storage.redis.sadd(storage_key, item["id"]) # use redis.Redis().sadd() - add item to the end of the existing data

p.s.: sorry to spam your repo. i messed up my git settings... new pc and stuff...

@Era-Dorta
Copy link
Collaborator

Would you still like to merge this? I'm happy to review if so.

@Kariton
Copy link
Contributor Author

Kariton commented Oct 17, 2024

Hey,
yes, i heavily depend on that concurrent coroutines and need the additional functionality.

for now i depend on my own fork and think others would, if they want to, profit from the extended storage capabilities as well.

@Era-Dorta
Copy link
Collaborator

Amazing, thanks! Let me set myself as reviewer and add a few comments :)

@Era-Dorta Era-Dorta self-requested a review October 17, 2024 20:52
@@ -4,12 +4,18 @@


class Storage:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about making this class abstract to ensure the methods are implemented?

Suggested change
class Storage:
class Storage(ABC):

Comment on lines +7 to +9
def __init__(self):
self.redis = None

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why the base Storage class needs to have the redis attribute defined. If anything, for consistency with the InMemoryStorage, it should be renamed to _storage

result_str = result_bytes.decode("utf-8")
result_dict = json.loads(result_str)
return result_dict
except Exception as e:
raise StorageError(f"Redis load failed: {e}")

def delete(self, *keys: str) -> int:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally optional but how do you feel about adding some unit testing for the new function?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants