-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add JitsiXmppStringprep #105
Merged
Merged
Changes from 8 commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
58b2245
minor: Fix log message, formatting.
bgrozev e6bd67f
Add JitsiXmppStringprep.
bgrozev 4cfc3e7
test: Move JidTest from jicofo.
bgrozev 894500b
Add tests for JIDs with _, allow in any position.
bgrozev b97e588
test: Add XmlStringBuilderPerfTest.
bgrozev 0d826aa
chore: Update to smack 4.4.8.
bgrozev caa39a9
test:Add another invalid JID.
bgrozev 6b23236
squash: Fix typo.
bgrozev fdbbe55
feat: Implement the LDHU test explicitly, test IDNs.
bgrozev 5582784
Add some more test cases.
bgrozev d14f2ee
Also accept % in domain labels.
bgrozev 9d0d4bd
Add a flag to control the use of JitsiXmppStringprep.
bgrozev 29be535
squash: Add a P to function name and log message.
bgrozev File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
/* | ||
* Copyright @ 2024 - present 8x8, Inc. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
package org.jitsi.xmpp | ||
|
||
import org.jitsi.utils.logging2.createLogger | ||
import org.jitsi.xmpp.stringprep.JitsiXmppStringprep | ||
import org.jivesoftware.smack.SmackConfiguration | ||
import org.jivesoftware.smack.parsing.ExceptionLoggingCallback | ||
import org.jivesoftware.smackx.bytestreams.socks5.Socks5Proxy | ||
import org.jxmpp.JxmppContext | ||
import org.jxmpp.jid.impl.JidCreate | ||
|
||
object Smack { | ||
val logger = createLogger() | ||
|
||
fun initialize() { | ||
logger.info("Setting XML parsing limits.") | ||
System.setProperty("jdk.xml.entityExpansionLimit", "0") | ||
System.setProperty("jdk.xml.maxOccurLimit", "0") | ||
System.setProperty("jdk.xml.elementAttributeLimit", "524288") | ||
System.setProperty("jdk.xml.totalEntitySizeLimit", "0") | ||
System.setProperty("jdk.xml.maxXMLNameLimit", "524288") | ||
System.setProperty("jdk.xml.entityReplacementLimit", "0") | ||
|
||
// Force XmppStringPrepUtil to load before we override the context, otherwise it gets reverted. | ||
// https://github.com/igniterealtime/jxmpp/pull/44 | ||
JidCreate.from("example") | ||
logger.info("Using JitsiXmppStringprep.") | ||
JxmppContext.setDefaultXmppStringprep(JitsiXmppStringprep.INSTANCE) | ||
|
||
// if there is a parsing error, do not break the connection to the server(the default behaviour) as we need | ||
// it for the other conferences. | ||
SmackConfiguration.setDefaultParsingExceptionCallback(ExceptionLoggingCallback()) | ||
Socks5Proxy.setLocalSocks5ProxyEnabled(false) | ||
} | ||
} |
75 changes: 75 additions & 0 deletions
75
src/main/kotlin/org/jitsi/xmpp/stringprep/JitsiXmppStringprep.kt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,75 @@ | ||
/* | ||
* Copyright @ 2024 - present 8x8, Inc. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
package org.jitsi.xmpp.stringprep | ||
|
||
import org.jxmpp.stringprep.XmppStringprep | ||
import org.jxmpp.stringprep.XmppStringprepException | ||
import org.jxmpp.stringprep.rocksxmppprecis.RocksXmppPrecisStringprep | ||
import rocks.xmpp.precis.PrecisProfile | ||
import java.net.IDN | ||
import java.text.Normalizer | ||
import java.util.regex.Pattern | ||
|
||
/** | ||
* Extends [RocksXmppPrecisStringprep] to allow underscores (_) in the domain part. | ||
* | ||
* This is needed because jitsi-meet URLs of the form https://domain/tenant/room get translated into a JID of the | ||
* form [email protected], and the tenant field has been allowed to use underscores for a long time (in | ||
* fact '.' in the tenant is translated into '_'). | ||
*/ | ||
class JitsiXmppStringprep : XmppStringprep by RocksXmppPrecisStringprep.INSTANCE { | ||
override fun domainprep(string: String?): String { | ||
try { | ||
return idnWithUnderscoreProfile.enforce(string) | ||
} catch (e: IllegalArgumentException) { | ||
throw XmppStringprepException(string, e) | ||
} | ||
} | ||
|
||
companion object { | ||
val INSTANCE = JitsiXmppStringprep() | ||
private val idnWithUnderscoreProfile = IDNWithUnderscoreProfile() | ||
} | ||
} | ||
|
||
/** | ||
* Based on [PrecisProfiles.IDN], but allows underscores. | ||
*/ | ||
class IDNWithUnderscoreProfile : PrecisProfile(false) { | ||
override fun prepare(input: CharSequence): String { | ||
val str = input.toString() | ||
|
||
// Throws if it contains invalid characters | ||
IDN.toASCII(str.replace("_", ""), IDN.USE_STD3_ASCII_RULES) | ||
|
||
return IDN.toUnicode(IDN.toASCII(str), IDN.USE_STD3_ASCII_RULES) | ||
} | ||
|
||
override fun applyWidthMappingRule(charSequence: CharSequence) = widthMap(charSequence) | ||
override fun applyAdditionalMappingRule(charSequence: CharSequence) = | ||
LABEL_SEPARATOR.matcher(charSequence).replaceAll(".") | ||
override fun applyCaseMappingRule(charSequence: CharSequence) = charSequence.toString().lowercase() | ||
|
||
override fun applyNormalizationRule(charSequence: CharSequence) = | ||
Normalizer.normalize(charSequence, Normalizer.Form.NFC) | ||
|
||
override fun applyDirectionalityRule(charSequence: CharSequence) = charSequence | ||
|
||
companion object { | ||
private const val DOTS: String = "[.\u3002\uFF0E\uFF61]" | ||
private val LABEL_SEPARATOR: Pattern = Pattern.compile(DOTS) | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,130 @@ | ||
/* | ||
* Jicofo, the Jitsi Conference Focus. | ||
* | ||
* Copyright @ 2024-Present 8x8, Inc. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
package org.jitsi.xmpp | ||
|
||
import io.kotest.assertions.throwables.shouldThrow | ||
import io.kotest.assertions.withClue | ||
import io.kotest.core.spec.IsolationMode | ||
import io.kotest.core.spec.style.ShouldSpec | ||
import io.kotest.core.test.TestCase | ||
import io.kotest.matchers.shouldNotBe | ||
import io.kotest.matchers.types.shouldBeInstanceOf | ||
import org.jitsi.xmpp.stringprep.JitsiXmppStringprep | ||
import org.jxmpp.JxmppContext | ||
import org.jxmpp.jid.impl.JidCreate | ||
import org.jxmpp.stringprep.XmppStringprepException | ||
|
||
/** | ||
* Test JID parsing. The lists below are based on the jxmpp corpora here, plus a couple additional ones: | ||
* https://github.com/igniterealtime/jxmpp/tree/master/jxmpp-strings-testframework/src/main/resources/xmpp-strings/jids/valid/main | ||
* https://github.com/igniterealtime/jxmpp/blob/master/jxmpp-strings-testframework/src/main/resources/xmpp-strings/jids/invalid/main | ||
*/ | ||
class JidTest : ShouldSpec() { | ||
override fun isolationMode(): IsolationMode { | ||
return IsolationMode.SingleInstance | ||
} | ||
override suspend fun beforeAny(testCase: TestCase) { | ||
super.beforeAny(testCase) | ||
Smack.initialize() | ||
} | ||
|
||
init { | ||
context("Parsing valid JIDs") { | ||
JxmppContext.getDefaultContext().xmppStringprep.shouldBeInstanceOf<JitsiXmppStringprep>() | ||
validJids.forEach { | ||
withClue(it) { | ||
JidCreate.from(it) shouldNotBe null | ||
} | ||
} | ||
} | ||
context("Parsing invalid JIDs") { | ||
JxmppContext.getDefaultContext().xmppStringprep.shouldBeInstanceOf<JitsiXmppStringprep>() | ||
invalidJids.forEach { | ||
withClue(it) { | ||
shouldThrow<XmppStringprepException> { | ||
JidCreate.from((it)) | ||
} | ||
} | ||
} | ||
} | ||
} | ||
} | ||
|
||
val validJids = listOf( | ||
"[email protected]", | ||
"[email protected]/foo", | ||
"[email protected]/foo bar", | ||
"[email protected]/foo@bar", | ||
"foo\\[email protected]", | ||
"[email protected]", | ||
"fuß[email protected]", | ||
"π@example.com", | ||
"Σ@example.com", | ||
"ς@example.com", | ||
"[email protected]/♚", | ||
"example.com", | ||
"example.com/foobar", | ||
"a.example.com/[email protected]", | ||
"server/resource@foo", | ||
"server/resource@foo/bar", | ||
"user@CaSe-InSeNsItIvE", | ||
"[email protected]", | ||
// "user@[2001:638:a000:4134::ffff:40]", | ||
// "user@[2001:638:a000:4134::ffff:40%eno1]", | ||
// "user@averylongdomainpartisstillvalideventhoughitexceedsthesixtyfourbytelimitofdnslabels", | ||
"long-conference-name-1245c711a15e466687b6333577d83e0b@" + | ||
"conference.vpaas-magic-cookie-a32a0c3311ee432eab711fa1fdf34793.8x8.vc", | ||
"[email protected]/🍺", | ||
// These are not valid according to the XMPP spec, but we accept it intentionally. | ||
"do_main.com", | ||
"u_s_e_r@_do_main_.com", | ||
"user@do_ma-in.com" | ||
) | ||
|
||
val invalidJids = listOf( | ||
"jul\[email protected]", | ||
"\"juliet\"@example.com", | ||
"foo [email protected]", | ||
// This fails due to a corner case in JidCreate when "example.com" is already cached as a DomainpartJid | ||
// "@example.com/", | ||
"henryⅣ@example.com", | ||
"♚@example.com", | ||
"juliet@", | ||
"/foobar", | ||
"node@/server", | ||
"@server", | ||
"@server/resource", | ||
"@/resource", | ||
"@/", | ||
"/", | ||
"@", | ||
"user@", | ||
"user@@", | ||
"user@@host", | ||
"user@@host/resource", | ||
"user@@host/", | ||
"[email protected]/x", | ||
"[email protected]@example.org", | ||
"foo\[email protected]", | ||
"foobar@ex\u0000ample.org", | ||
// Leading - in domain part. | ||
"[email protected]", | ||
// Trailing - in domain part. | ||
"[email protected]", | ||
"[email protected]" | ||
) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand what this will do with actual IDNs (domains that have unicode and/or the IDN encoding --xn--). Do we want to write tests for this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure what's going here, but please be careful with Java and IDN, as per this comment: dnsjava/dnsjava#207 (comment) (disregard the specifics for dnsjava).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! It does indeed convert "ß" to "ss" under the few openjdk versions I tried. I've added a test to document it, but I think it should be fine for our purpose here.
For context: we're adding stricter validation of the JIDs used in jicofo (and the other components) to prevent obviously invalid JIDs to be processed. But we've been using
_
and%
as part of the "tenant" for years and prefer to continue accepting to prevent breaking conference URLs that used to work. Unicode characters in the URL are urlencoded before they are used in JIDs, so in practice this shouldn't affect URLs that use unicode.As an example the URL
https://meet.jit.si/fuß.ball/foo
ends up using the following MUC JID:[email protected]%c3%9f_ball.meet.jit.si
. The domain part is invalid due to%
and_
, but we want to allow it anyway.