Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prefer the Redis unix socket if available #220

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

ekohl
Copy link
Member

@ekohl ekohl commented Jul 13, 2021

Unix sockets typically have lower overhead and also allows setting stricter permissions. While iptables can be used to limit access using users, file permissions are much easier to manage.

Includes #218.

Currently a draft, but I'm exploring the option of deploying a dedicated Redis instance just for Pulp (if needed). This means Redis can be tuned different. Foreman needs Redis to be in persistent mode but Pulp doesn't always:

Tasking Caching Redis mode
RQ Yes Persistent
RQ No Persistent
PG Yes Empheral
PG No Off

@ehelms
Copy link
Member

ehelms commented Jul 13, 2021

Currently a draft, but I'm exploring the option of deploying a dedicated Redis instance just for Pulp (if needed). This means Redis can be tuned different. Foreman needs Redis to be in persistent mode but Pulp doesn't always:

Since Foreman may also want to do caching potentially with a Redis, would we be looking at:

  • 3 instances, one for Dynflow, one for Pulp cache, one for Foreman cache
  • 2 instances, one for persistent workloads (e.g. Dynflow) and one for caching (Foreman or Pulp)

And does that change any of the design?

@@ -17,6 +17,15 @@
ensure_newline => true,
}

if $redis::unixsocket != '' {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be configuring this given we only include redis ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically it could have been changed via hiera. Outside of the installer this can also happen directly via manifests.

@ekohl
Copy link
Member Author

ekohl commented Jul 15, 2021

Since Foreman may also want to do caching potentially with a Redis, would we be looking at:

* 3 instances, one for Dynflow, one for Pulp cache, one for Foreman cache

* 2 instances, one for persistent workloads (e.g. Dynflow) and one for caching (Foreman or Pulp)

And does that change any of the design?

Possibly. At this point I want to find out if we can even use a unix socket in the first place. We may run into problems like permissions, maybe something inside Pulp itself and probably SELinux policies.

At the same time, should start to investigate if Foreman and Dynflow can.

@ekohl ekohl force-pushed the redis-unix-socket branch from d7dfdb7 to 27dbee0 Compare February 28, 2024 10:08
This allows users to run with a remote Redis server.
Unix sockets typically have lower overhead and also allows setting
stricter permissions. While iptables can be used to limit access using
users, file permissions are much easier to manage.
@ekohl ekohl force-pushed the redis-unix-socket branch from 27dbee0 to 73d945f Compare February 28, 2024 10:10
@ekohl
Copy link
Member Author

ekohl commented Feb 28, 2024

Rebased, but it's now split into two commits. First there's #331 which makes the Redis URL configurable (which is useful for remote Redis servers). Then this expands it to use the unix socket if available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants