How can I turn username --> email address using LDAP?

[snaildos]

I've read the documentation but can only see use cases when the email address the LDAP user uses is the domain that docker-mailserver hosts.
I however, want to be able to have a username provision to [email protected] using docker-mailserve.r
Any ideas?
Warm Regards,
Snail

[polarathene]

Postfix presently appends @example.com to the SASL login username when no domain-part (@some-value) is provided. @example.com would be used when DMS has the compose.yaml setting hostname: mail.example.com. That would only apply for Postfix ports 587 and 465.

This is due to the Postfix main.cf setting using the mydomain setting:

docker-mailserver/target/postfix/main.cf

Line 99 in e370c0c

smtpd_sasl_local_domain = $mydomain

I can't quite recall specifics with LDAP involved, Postfix would query LDAP with the username I think and if a result was returned, use the mail attribute value:

docker-mailserver/target/postfix/ldap-users.cf

Line 5 in e370c0c

result_attribute = mail

I think this had the query use the login username to query the mail address for that user lookup:

docker-mailserver/target/postfix/ldap-users.cf

Line 4 in e370c0c

query_filter = (&(mail=%s)(mailEnabled=TRUE))

So you may need to alter that via the current ENV support. I do recall that SASLAuthd complicates that a little further, and presently has a configuration that makes that a little bit inconsistent, but many users mistakenly enable SASLAuthd when you don't actually need that to use LDAP. So if you have that enabled, try without it.

On the Dovecot side for other ports that need your LDAP user, this is configured here:

docker-mailserver/target/dovecot/dovecot-ldap.conf.ext

Lines 10 to 11 in e370c0c

	user_attrs = mailHomeDirectory=home,mailUidNumber=uid,mailGidNumber=gid,mailStorageDirectory=mail
	user_filter = (&(objectClass=PostfixBookMailAccount)(uniqueIdentifier=%n))

You'd currently use the ENV to adjust that. Dovecot has separate syntax for querying with only the login username, or also including a domain-part, which in the example above is %n (just the local-part, anything before an @), while %u would be the full email address. Make sure you query on the correct attribute.

---

There is a PR I need to finish that refactors the LDAP support, along with improved LDAP docs to support that, but I've not had time to focus on that for a while: #3524

[kraghuprasad]

/tmp/docker-mailserver

This folder is not getting used, if I enable LDAP. No configuration files are available in this folder when I set various LDAP related parameters in mailserver.env file.

Due to this, we won't be able to even modify the files like /etc/postfix/ldap-*.cf manually, even if auto-generation of LDAP configuration turns out to be unreliable.

[polarathene]

This folder is not getting used, if I enable LDAP. No configuration files are available in this folder when I set various LDAP related parameters in mailserver.env file.

You misunderstood. /tmp/docker-mailserver allows you to provide these config files yourself, and they will be copied over to the internal config location. We already provide these files by default, so you don't have to provide your own, the ENV you set will modify the internal copies:

docker-mailserver/test/tests/serial/mail_with_ldap.bats

Lines 212 to 216 in e370c0c

	local LDAP_CONFIGS_POSTFIX=(
	/etc/postfix/ldap-users.cf
	/etc/postfix/ldap-groups.cf
	/etc/postfix/ldap-aliases.cf
	)

Due to this, we won't be able to even modify the files like /etc/postfix/ldap-*.cf manually, even if auto-generation of LDAP configuration turns out to be unreliable.

You modify those via providing your own configs to /tmp/docker-mailserver and/or using ENV.

[kraghuprasad]

OK, got it. Thanks for the clarifications. The confusion arose because when I use file based user-accounts, this folder had many configuration files. But as soon as I enabled LDAP, this folder was empty.

I am trying to debug why I am not able to login to an existing mail account kept in LDAP. If I want to change configurations in a running container, would it suffice to make changes in /etc/postfix/* and /etc/dovecot/* and test the setup? Or are we getting configurations from some other locations?

[polarathene]

I am trying to debug why I am not able to login to an existing mail account kept in LDAP

Ah, there are some gotchas IIRC with the LDAP implementation. If you have enabled SASLAUTHD try to disable that, you shouldn't actually need it, it was contributed by the same contributor of LDAP support for their own needs (without dovecot). Postfix and Dovecot have some related config (which I seem to have already documented above for reference).

If I want to change configurations in a running container, would it suffice to make changes in /etc/postfix/* and /etc/dovecot/* and test the setup?

Within the container you could make those changes and use postfix reload / dovecot reload commands, if those don't have effect, you can also use supervisorctl restart postfix / supervisorctl restart dovecot.

Just note if you are directly mounting files via volumes that changes will not update in the container if the inode of the file on the Docker host changes (this happens depending on how your text editor saves changes to a file), the only way to avoid that is directory mounts instead but that will replace the existing contents of the directory within the container which probably won't work here. I tend to use nano command in the container to manage files directly, or you can mount a folder somewhere else and use cp /from/path/file /to/path/file to update the real internal configuration.

DMS does have a change detection service that detects config files in the Config Volume (mounted to /tmp/docker-mailserver) but presently it's disabled when LDAP is enabled, so you can't leverage that.

[kraghuprasad]

Thanks for the info.

I am planning to do directory-mount in docker-compose.yml to /tmp/docker-mailserver and use my own *.cf files. I wanted to change configuration files in the container only for the purpose of debugging. At present I think, I have misconfigured ENVs in my docker-compose file, due to which I am unable to authenticate via my LDAP server while sending out emails to external parties.

[snaildos]

Hey @polarathene ,
Thank you for all the feedback you've given.
I've gotten everything working from the configuration you've given to me aside from sending.
When sending an email I get:
"Sender address rejected: not owned by user USERNAME;"
I have almost identical configuration to the testing documents you've given to me, aside from "DOVECOT_PASS_ATTRS" (I've removed) an additionally using LDAP_QUERY_FILTER_SENDERS=(|(mail=%s)(mail=admin@*)) (The user I am sending emails from is called 'test').
I hope to hear some insight from you. Thanks for your help prior.
Warm regards,
Snail

[polarathene]

I've gotten everything working from the configuration you've given to me aside from sending.

You might need to search our issue history (and maybe discussion history) here to try find previous LDAP issues/discussions. They'll likely have advice you're seeking on any issues you run into.

Here's a recent issue related to LDAP sending for example: #4029

---

When sending an email I get:
"Sender address rejected: not owned by user USERNAME;"

This suggests you have SPOOF_PROTECTION=1. That will restrict you to only being able to send mail with a sender address allowed for your user login name. I believe we presently have our LDAP SPOOF_PROTECTION=1 tests disabled (or at least partially) because of a bug there. I did investigate that over a year ago and it was due to the default LDAP configs IIRC.

From memory, when you connect to Postfix (ports 587 / 465) with your mail client to submit mail to send. You'll authenticate with a username, this may look like john.doe or [email protected]. If it's just john.doe then Postfix will add @example.com for the SASL login I think, and that may be part of the LDAP lookup query for ldap-users.cf, while the table for SPOOF_PROTECTION=1 is separate, which you seem to be aware of with the ENV LDAP_QUERY_FILTER_SENDERS 👍

The way this check works I think was the sender address being used would be queried, LDAP receives that query, and then the result attribute is used for any returned responses to filter which login usernames are permitted to send with that sender address. Here's the defaults we currently have:

docker-mailserver/target/postfix/ldap-senders.cf

Lines 4 to 5 in e370c0c

	query_filter = (mail=%s)
	result_attribute = mail, uid

So typically that's returning mail ([email protected]) and uid (the User ID john.doe). Your LDAP values for those attributes may not be the same though. I can't quite recall the specifics for that bug, but do recall an issue with SASLAuthD feature (it would not remove the @example.com "realm" from Postfix, unless config was updated).

If you're not using SASLAuthD anymore, you shouldn't have an issue there. I could be mistaken, but Postfix should be trying to match either john.doe or [email protected] formats for your "login username" (without LDAP this is what is provided to setup email add, which treats the email as the DMS account username that Postfix and Dovecot use). You need to make sure the LDAP query is working correctly, and then it's just making sure that Postfix can match the given client user login name with the attribute response returned from LDAP. After that it should work as expected.

Alternatively don't use SPOOF_PROECTION=1 if that's acceptable. This feature is only to restrict what sender address you can use to send mail with. Without it, you could attempt to send mail with domains you don't own ([email protected] / [email protected]), which would fail but DMS wouldn't know any better. It's also used for deployments where you have other users that shouldn't be able to send as others with your mail domain (eg: john.doe sending as [email protected]).

[snaildos]

Hello, thank you for your insight. I've gotten everything working as expected from your help - I appreciate your efforts while being low on time.
Just a quick final question, if SPOOF PROTECTION is off how will i know my custom spoof protection is working? It should just deny from sending other emails, correct? Thanks.
Thanks again for your help.
Kind regards,
Snail

[polarathene]

Just a quick final question, if SPOOF PROTECTION is off how will i know my custom spoof protection is working? It should just deny from sending other emails, correct?

SPOOF_PROTECTION=1 should prevent you from sending mail with any sender address that your account/user has not been allowed to use.

Here is the LDAP tests related to that feature:

docker-mailserver/test/tests/serial/mail_with_ldap.bats

Lines 325 to 367 in 32c0a34

	# Requires ENV `SPOOF_PROTECTION=1` for the expected assert_output
	@test "spoofing (with LDAP): rejects sender forging" {
	_wait_for_smtp_port_in_container_to_respond dms-test_ldap
	_send_email --expect-rejection \
	--port 465 -tlsc --auth PLAIN \
	--auth-user [email protected] \
	--auth-password secret \
	--ehlo mail \
	--from [email protected] \
	--data 'auth/ldap-smtp-auth-spoofed.txt'
	assert_failure
	assert_output --partial 'Sender address rejected: not owned by user'
	}
	@test "spoofing (with LDAP): accepts sending as alias" {
	_send_email \
	--port 465 -tlsc --auth PLAIN \
	--auth-user [email protected] \
	--auth-password secret \
	--ehlo mail \
	--from [email protected] \
	--to [email protected] \
	--data 'auth/ldap-smtp-auth-spoofed-alias.txt'
	assert_output --partial 'End data with'
	}
	@test "spoofing (with LDAP): uses senders filter" {
	# skip introduced with #3006, changing port 25 to 465
	# Template used has invalid AUTH: https://github.com/docker-mailserver/docker-mailserver/pull/3006#discussion_r1073321432
	skip 'TODO: This test seems to have been broken from the start (?)'
	_send_email --expect-rejection \
	--port 465 -tlsc --auth PLAIN \
	--auth-user [email protected] \
	--auth-password secret \
	--ehlo mail \
	--from [email protected] \
	--to [email protected] \
	--data 'auth/ldap-smtp-auth-spoofed-sender-with-filter-exception.txt'
	assert_failure
	assert_output --partial 'Sender address rejected: not owned by user'
	}

The custom ldap-senders.cf test at the end there is skipped, I did identify a fix to enable it again but delayed resolving it until after the LDAP refactor PR lands.

I haven't looked at the LDAP stuff for a while, but here is the related portion for configuring a custom SPOOF_PROTECTION=1 query I documented inline a while ago:

docker-mailserver/test/tests/serial/mail_with_ldap.bats

Lines 62 to 70 in 32c0a34

	# Given the sender address `%s` from Postfix, query LDAP for accounts that meet the search filter,
	# the `result_attribute` is `mail` + `uid` (`userID`) attributes for login names that are authorized to use that sender address.
	# One of the result values returned must match the login name of the user trying to send mail with that sender address.
	#
	# Filter: Search for accounts that meet any of the following conditions:
	# - Has any attribute (`mail`, `mailAlias`, `mailGroupMember`) with a value of `%s`, AND is of class `PostfixBookMailAccount` with `mailEnabled=TRUE` attribute set.
	# - Has attribute `mailAlias` with value of `%s` AND is of class PostfixBookMailForward
	# - Has the attribute `userID` with value `some.user.id`
	local QUERY_SENDERS='(| (& (|(mail=%s) (mailAlias=%s) (mailGroupMember=%s)) (&(objectClass=PostfixBookMailAccount)(mailEnabled=TRUE)) ) (&(mailAlias=%s)(objectClass=PostfixBookMailForward)) (userID=some.user.id))'

For LDAP with SPOOF_PROTECTION=1 there is presently two variants configured, when you configure a custom query like shown above, it would only use that query, not sourcing from the users/aliases/groups LDAP sources (which I think was a bad idea and will probably be dropped in future):

docker-mailserver/target/scripts/startup/setup.d/security/spoofing.sh

Lines 7 to 12 in 32c0a34

	if [[ ${ACCOUNT_PROVISIONER} == 'LDAP' ]]; then
	if [[ -z ${LDAP_QUERY_FILTER_SENDERS} ]]; then
	postconf 'smtpd_sender_login_maps = ldap:/etc/postfix/ldap-users.cf ldap:/etc/postfix/ldap-aliases.cf ldap:/etc/postfix/ldap-groups.cf'
	else
	postconf 'smtpd_sender_login_maps = ldap:/etc/postfix/ldap-senders.cf'
	fi

---

So if you want to verify the feature is working as intended. Just send mail with sender address that should be allowed, vs one that is not allowed for that account. Maybe also double check by sending from another user account just to verify that a shared sender address works, and that sender address that shouldn't work for that user only works for the account it should 😅

[snaildos]

Hello, thank you for the reply.
Everything is working well except sending outbound emails from docker-mailserver.
When sending an email to my @hotmail.com account I get "User doesn't exist:", but internal sending works (to other email addresses in the email server)
Do you have an idea on how to solve this?
Thank you again for all your previous insight regarding the other issues I've had, everything works flawlessly except for this one small issue.
Warm regards

[polarathene]

Do you have an idea on how to solve this?

You likely have an alias or account assigned with an @hotmail.com mail domain, this will tell DMS that it owns and manages mail for that mail domain, so it won't attempt to send mail outbound to another mail server. As such it tries to deliver the mail but realizes there is no account for that username, hence the "User doesn't exist".

If you don't have an account or alias with that, only the alias for a local mail domain pointing to an external one like @hotmail.com, then this may be a bug with LDAP (or rather the query filter). I believe this is the QUERY_DOMAIN like shown here:

docker-mailserver/test/tests/serial/mail_with_ldap.bats

Lines 46 to 53 in 32c0a34

	# `mailAlias`` is supported by both classes provided from the schema `postfix-book.ldif`, but `mailEnabled` is only available to `PostfixBookMailAccount` class:
	local QUERY_ALIAS='(&(mailAlias=%s) (| (objectClass=PostfixBookMailForward) (&(objectClass=PostfixBookMailAccount)(mailEnabled=TRUE)) ))'
	# Postfix does domain lookups with the domain of the recipient to check if DMS manages the mail domain.
	# For this lookup `%s` only represents the domain, not a full email address. Hence the match pattern using a wildcard prefix `*@`.
	# For a breakdown, see QUERY_SENDERS comment.
	# NOTE: Although `result_attribute = mail` will return each accounts full email address, Postfix will only compare to domain-part.
	local QUERY_DOMAIN='(| (& (|(mail=*@%s) (mailAlias=*@%s) (mailGroupMember=*@%s)) (&(objectClass=PostfixBookMailAccount)(mailEnabled=TRUE)) ) (&(mailAlias=*@%s)(objectClass=PostfixBookMailForward)) )'

docker-mailserver/test/tests/serial/mail_with_ldap.bats

Line 117 in 32c0a34

--env LDAP_QUERY_FILTER_DOMAIN="${QUERY_DOMAIN}"

The mailAlias=*@%s allows for any mailAlias with the domain match to be considered as managed by DMS, which it should be AFAIK. If you have that or the similar mail attribute returning with @hotmail.com address, that is your problem I think.

[kms-office-free]

Omg, thank you!

[polarathene]

Any part of this in particular that helped? I will be updating LDAP docs, so feedback is appreciated :)