Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Encoding Error In CardDav #5

Open
ana-u opened this issue Oct 25, 2019 · 6 comments
Open

Encoding Error In CardDav #5

ana-u opened this issue Oct 25, 2019 · 6 comments

Comments

@ana-u
Copy link

ana-u commented Oct 25, 2019

If you create change or an Contact via CardDav The encoding gets mangeled.
(UTF-8 interpreted as iso-8859-1 ?)
If you change the contact in the Kopano Webgui the carracters arrive corrrectly at the CardDav clients.

Tested with:
Thunderbid + CardBook
Win10

Examples:
Expected
ÄÜÖ -> Ã�Ã�Ã�
Comércio -> Comércio
@boospy
Copy link

boospy commented Oct 25, 2019

Maybe also interesting: https://forum.kopano.io/topic/2586/kdav-does-not-sync-correctly-500-internal-server-error

@junes
Copy link

junes commented Nov 1, 2019

I can confirm the observation from @ana-u . I have the same problem:

When creating a new address book entry via CardDav client (evolution in my case), then German Umlauts are not encoded correctly when the are stored in the Kopano DB. But when I create the same entry in the Kopano Webclient then the Umlauts are correct. As you can see from the logfiles the HTML/XML content is sent correctly. So the error should be somewhere on the way to Kopano DB in my opnion.

01.11.2019 16:45:53 [23903] [DEBUG] main ------------------ Start
01.11.2019 16:45:53 [23903] [DEBUG] main PUT /addressbooks/testuser/Contacts/a801112750de50e243877a864a804763bfe95690.vcf
01.11.2019 16:45:53 [23903] [DEBUG] main KDAV version master-9f8f3e0
01.11.2019 16:45:53 [23903] [DEBUG] main SabreDAV version 3.2.2
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoSyncState->__construct(): Using db sqlite:/var/lib/kopano/kdav/syncstate.db
01.11.2019 16:45:53 [23903] [DEBUG] main KLogger->LogIncoming(): INPUT
PUT /addressbooks/testuser/Contacts/a801112750de50e243877a864a804763bfe95690.vcf HTTP/1.1
Host: kdav.xyz
User-Agent: Evolution/3.28.5
Connection: close
Cache-Control: no-cache
Pragma: no-cache
If-None-Match: *
Content-Type: text/vcard; charset="utf-8"
Accept-Language: de-de, de;q=0.9
Content-Length: 184
Authorization: Basic REDACTED

BEGIN:VCARD
VERSION:3.0
FN:Bärbel Börner
N:Börner;Bärbel;;;
X-EVOLUTION-FILE-AS:Börner\, Bärbel
X-MOZILLA-HTML:FALSE
UID:a801112750de50e243877a864a804763bfe95690
END:VCARD
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->Logon(): testuser / password
01.11.2019 16:45:53 [23903] [DEBUG]  dav KopanoDavBackend->Logon(): Auth: OK - user testuser - session Resource id #10
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetUser(): testuser
01.11.2019 16:45:53 [23903] [TRACE] card KopanoCardDavBackend->getAddressBooksForUser(): principalUri: principals/testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetFolders(): principal 'principals/testuser', classes 'Array
(
    [0] => IPF.Contact
)
'
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetFolders(): found 2 folders: Array
(
    [0] => Array
        (
            [id] => principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000
            [uri] => Contacts
            [principaluri] => principals/testuser
            [{http://sabredav.org/ns}sync-token] => 0000000000
            [{DAV:}displayname] => Contacts
            [{http://calendarserver.org/ns/}getctag] => 1572622528
        )

    [1] => Array
        (
            [id] => principals/testuser:0368e31a1663476dadcc7dd4126e5f6c5b8c04000000
            [uri] => Suggested Contacts
            [principaluri] => principals/testuser
            [{http://sabredav.org/ns}sync-token] => 0000000000
            [{DAV:}displayname] => Suggested Contacts
            [{http://calendarserver.org/ns/}getctag] => 0000000000
        )

)

01.11.2019 16:45:53 [23903] [TRACE] card KopanoCardDavBackend->getCard(): addressBookId: principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000 - cardUri: a801112750de50e243877a864a804763bfe95690.vcf
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiFolder(): Id: principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Searching for 'a801112750de50e243877a864a804763bfe95690' in 'principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000' (Resource id #15)
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Try PR_SOURCE_KEY a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Try APPTTSREF a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetCustomProperties(): Fetching properties id:principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [DEBUG]  dav KopanoDavBackend->GetMapiMessageForId(): Nothing found for a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [23903] [DEBUG] card KopanoCardDavBackend->getCard(): Object NOT FOUND
01.11.2019 16:45:53 [23903] [TRACE] card KopanoCardDavBackend->getCard(): addressBookId: principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000 - cardUri: a801112750de50e243877a864a804763bfe95690.vcf
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiFolder(): Id: principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Searching for 'a801112750de50e243877a864a804763bfe95690' in 'principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000' (Resource id #17)
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Try PR_SOURCE_KEY a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Try APPTTSREF a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [23903] [DEBUG]  dav KopanoDavBackend->GetMapiMessageForId(): Nothing found for a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [23903] [DEBUG] card KopanoCardDavBackend->getCard(): Object NOT FOUND
01.11.2019 16:45:53 [23903] [TRACE] card KopanoCardDavBackend->getCard(): addressBookId: principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000 - cardUri: a801112750de50e243877a864a804763bfe95690.vcf
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiFolder(): Id: principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Searching for 'a801112750de50e243877a864a804763bfe95690' in 'principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000' (Resource id #19)
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Try PR_SOURCE_KEY a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Try APPTTSREF a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [23903] [DEBUG]  dav KopanoDavBackend->GetMapiMessageForId(): Nothing found for a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [23903] [DEBUG] card KopanoCardDavBackend->getCard(): Object NOT FOUND
01.11.2019 16:45:53 [23903] [TRACE] card KopanoCardDavBackend->createCard(): addressBookId: principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000 - cardUri: a801112750de50e243877a864a804763bfe95690.vcf - cardData: BEGIN:VCARD
VERSION:3.0
FN:Bärbel Börner
N:Börner;Bärbel;;;
X-EVOLUTION-FILE-AS:Börner\, Bärbel
X-MOZILLA-HTML:FALSE
UID:a801112750de50e243877a864a804763bfe95690
END:VCARD
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetMapiFolder(): Id: principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [TRACE] card KopanoCardDavBackend->setData(): mapimessage: Resource id #24 - vcf: BEGIN:VCARD
VERSION:3.0
FN:Bärbel Börner
N:Börner;Bärbel;;;
X-EVOLUTION-FILE-AS:Börner\, Bärbel
X-MOZILLA-HTML:FALSE
UID:a801112750de50e243877a864a804763bfe95690
END:VCARD
01.11.2019 16:45:53 [23903] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [23903] [DEBUG] main KLogger->LogOutgoing(): OUTPUT:
HTTP/1.1 201 Created
X-KDAV-Version: master-9f8f3e0
X-Sabre-Version: 3.2.2
Content-Length: 0
ETag: "1572623153"


01.11.2019 16:45:53 [23903] [DEBUG] main httpcode='201' memory='1.09 MiB/2.00 MiB' time='0.37s'
01.11.2019 16:45:53 [23903] [DEBUG] main ------------------ End
01.11.2019 16:45:53 [25019] [DEBUG] main ------------------ Start
01.11.2019 16:45:53 [25019] [DEBUG] main GET /addressbooks/testuser/Contacts/a801112750de50e243877a864a804763bfe95690.vcf
01.11.2019 16:45:53 [25019] [DEBUG] main KDAV version master-9f8f3e0
01.11.2019 16:45:53 [25019] [DEBUG] main SabreDAV version 3.2.2
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoSyncState->__construct(): Using db sqlite:/var/lib/kopano/kdav/syncstate.db
01.11.2019 16:45:53 [25019] [DEBUG] main KLogger->LogIncoming(): INPUT
GET /addressbooks/testuser/Contacts/a801112750de50e243877a864a804763bfe95690.vcf HTTP/1.1
Host: kdav.xyz
User-Agent: Evolution/3.28.5
Connection: close
Cache-Control: no-cache
Pragma: no-cache
Accept-Language: de-de, de;q=0.9
Authorization: Basic REDACTED


01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->Logon(): testuser / password
01.11.2019 16:45:53 [25019] [DEBUG]  dav KopanoDavBackend->Logon(): Auth: OK - user testuser - session Resource id #10
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetUser(): testuser
01.11.2019 16:45:53 [25019] [TRACE] card KopanoCardDavBackend->getAddressBooksForUser(): principalUri: principals/testuser
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetFolders(): principal 'principals/testuser', classes 'Array
(
    [0] => IPF.Contact
)
'
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetFolders(): found 2 folders: Array
(
    [0] => Array
        (
            [id] => principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000
            [uri] => Contacts
            [principaluri] => principals/testuser
            [{http://sabredav.org/ns}sync-token] => 0000000000
            [{DAV:}displayname] => Contacts
            [{http://calendarserver.org/ns/}getctag] => 1572623153
        )

    [1] => Array
        (
            [id] => principals/testuser:0368e31a1663476dadcc7dd4126e5f6c5b8c04000000
            [uri] => Suggested Contacts
            [principaluri] => principals/testuser
            [{http://sabredav.org/ns}sync-token] => 0000000000
            [{DAV:}displayname] => Suggested Contacts
            [{http://calendarserver.org/ns/}getctag] => 0000000000
        )

)

01.11.2019 16:45:53 [25019] [TRACE] card KopanoCardDavBackend->getCard(): addressBookId: principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000 - cardUri: a801112750de50e243877a864a804763bfe95690.vcf
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetMapiFolder(): Id: principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Searching for 'a801112750de50e243877a864a804763bfe95690' in 'principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000' (Resource id #15)
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Try PR_SOURCE_KEY a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Try APPTTSREF a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetCustomProperties(): Fetching properties id:principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetIdOfMapiMessage(): Finding ID of Resource id #17
01.11.2019 16:45:53 [25019] [DEBUG]  dav KopanoDavBackend->GetIdOfMapiMessage(): Found PROP_APPTTSREF: a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [25019] [TRACE] card KopanoCardDavBackend->getCard(): vcf generated by mapi_mapitovcf: 
BEGIN:VCARD
VERSION:3.0
PRODID:-//Kopano//libicalmapi 8.7.5//EN
N:Börner;Bärbel
FN:Bärbel Börner
UID:a801112750de50e243877a864a804763bfe95690
REV:2019-11-01T15:45:53Z
END:VCARD


01.11.2019 16:45:53 [25019] [TRACE] card KopanoCardDavBackend->getCard(): returned data id: a801112750de50e243877a864a804763bfe95690 - size: 199 - etag: "1572623153"
01.11.2019 16:45:53 [25019] [TRACE] card KopanoCardDavBackend->getCard(): addressBookId: principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000 - cardUri: a801112750de50e243877a864a804763bfe95690.vcf
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetMapiFolder(): Id: principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Searching for 'a801112750de50e243877a864a804763bfe95690' in 'principals/testuser:0368e31a1663476dadcc7dd4126e5f6c4e8c04000000' (Resource id #19)
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Try PR_SOURCE_KEY a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:53 [25019] [TRACE]  dav KopanoDavBackend->GetMapiMessageForId(): Try APPTTSREF a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:54 [25019] [TRACE]  dav KopanoDavBackend->GetStore(): storename testuser
01.11.2019 16:45:54 [25019] [TRACE]  dav KopanoDavBackend->GetIdOfMapiMessage(): Finding ID of Resource id #21
01.11.2019 16:45:54 [25019] [DEBUG]  dav KopanoDavBackend->GetIdOfMapiMessage(): Found PROP_APPTTSREF: a801112750de50e243877a864a804763bfe95690
01.11.2019 16:45:54 [25019] [TRACE] card KopanoCardDavBackend->getCard(): vcf generated by mapi_mapitovcf: 
BEGIN:VCARD
VERSION:3.0
PRODID:-//Kopano//libicalmapi 8.7.5//EN
N:Börner;Bärbel
FN:Bärbel Börner
UID:a801112750de50e243877a864a804763bfe95690
REV:2019-11-01T15:45:53Z
END:VCARD


01.11.2019 16:45:54 [25019] [TRACE] card KopanoCardDavBackend->getCard(): returned data id: a801112750de50e243877a864a804763bfe95690 - size: 199 - etag: "1572623153"
01.11.2019 16:45:54 [25019] [DEBUG] main KLogger->LogOutgoing(): OUTPUT:
HTTP/1.1 200 OK
X-KDAV-Version: master-9f8f3e0
X-Sabre-Version: 3.2.2
Content-Type: text/vcard; charset=utf-8
Last-Modified: Fri, 01 Nov 2019 15:45:53 GMT
ETag: "1572623153"
Content-Length: 199

BEGIN:VCARD
VERSION:3.0
PRODID:-//Kopano//libicalmapi 8.7.5//EN
N:Börner;Bärbel
FN:Bärbel Börner
UID:a801112750de50e243877a864a804763bfe95690
REV:2019-11-01T15:45:53Z
END:VCARD


01.11.2019 16:45:54 [25019] [DEBUG] main httpcode='200' memory='1.09 MiB/2.00 MiB' time='0.07s'
01.11.2019 16:45:54 [25019] [DEBUG] main ------------------ End

Server:
Ubuntu 18.04 (with all recent updates)
kopano-server-packages 8.7.5 (from licensed repository)
kdav from git (today)

Client:
Ubuntu 18.04 (with all recent updates)
evolution 3.28.5-0ubuntu0.18.04.1

@zergchen
Copy link

Hello! Has there been any progress in the matter because the error persists. Are there any plans to fix the bug? CardDav is becoming more and more important if you want to manage Kopano contacts with other clients, e.g. AVM FritzBox, or macOS.

@stefan-inf
Copy link

Hello, I'm also urgently waiting for this bug to be solved in order to import my Kopano address book into my FritzBox. I think many FritzBox users (eg in Germany) also do so...

@zergchen
Copy link

Maybe this isn't priority number one, but maybe at least there is a roadmap in place to address the problem?

@raceface2nd
Copy link

I am not sure if the issue I encountered is the same or has the equal source. I am syncing a kopano account with a nextcloud account by using vdirsyncer https://github.com/pimutils/vdirsyncer

After successful sync from Kopano via kdav to nextcloud contacts German special characters are possibly re encoded at nextcloud after a while. This causes a resync to Kopano kdav with replacing all special charaters so some different characters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@junes @boospy @raceface2nd @ana-u @zergchen @stefan-inf