Add custom properties support + unit-tests, feature-tests #580

michael-koeller · 2018-11-23T10:47:50Z

I patched the implementation of @renejsum into a new fork and added the missing test cases.
Along the way I also added proper type handling for string/int/bool properties.

Regards,
Michael

renejsum · 2018-11-23T10:51:16Z

Awesome :-)

…

Ricyteach · 2018-12-10T12:10:04Z

I haven't reviewed the PR but wanted to say thanks, @michael-koeller!

jekozyra · 2019-08-23T23:38:51Z

@scanny any chance of this ever making it into a release? It would save me a lot of time.

BastienFaure · 2019-09-14T02:23:46Z

Merging this pull-request would we fantastic :) Thanks to all you guys for your job !

geobeo · 2020-01-29T09:45:58Z

This would basically be the one and only reason for me to use python-docx (a lot). I consider this a killer feature :P Please merge (or give someone write access to merge :P).

applied custom properties pr python-openxml#580 for storing private metadata within custom xml part.

docx/opc/package.py

docx/opc/customprops.py

nozokada · 2020-12-16T00:06:40Z

I'm so excited for this to be merged to master. Thank you for the work.

Ricyteach · 2020-12-16T03:08:20Z

I'm so excited for this to be merged to master. Thank you for the work.

Seconded. I intend to use the stink out of this feature. THANK YOU!!!

paweljasinski · 2021-01-15T15:06:59Z

Can this be merged?
If not, what is outstanding?
Thanks in advance

HansBambel · 2021-04-23T09:34:17Z

Would be nice to have this feature merged in :)

mohitmathew · 2021-12-14T14:55:07Z

Can we get this merged please ? this has been open for a very long time now.

hugoe · 2022-02-10T13:27:10Z

Yes, I need it too, please merge.

boomesq · 2022-03-07T01:35:00Z

I need it as well.

lsaint · 2022-05-19T06:30:42Z

It's been four years. Merge, please.

Add custom properties support + unit-tests, feature-tests #580

Apteryks · 2022-07-25T03:54:48Z

Hi, and thanks for this work!

It looks rather complete, with the acceptance tests and unit tests already in! I've taken the freedom to rebase it and get rid of the merge commits; I believe I've preserved the correct authors and the tests still passes; @michael-koeller could you hard reset your branch to https://github.com/Apteryks/python-docx/tree/add-support-for-custom-properties, after verifying that it looks alright?

After which we can look into some details.

sanzoghenzo · 2022-07-25T06:20:01Z

Hi, there is still a fix to do in the docstring that refers to pptx package instead of docx. See my review above.

I'm glad you're taking the time and effort to keep the ball rolling!

michael-koeller · 2022-07-25T07:55:02Z

@Apteryks: Nice to hear, that things might get some traction after laying sleeping for so long. 👍

I just did a rebase on my pull request branch and I also added a separate feature branch in my repo wich I didn't do originally (don't know any more why).

Apteryks · 2022-07-25T12:05:12Z

Hi @michael-koeller; for tidiness, could you please split the acceptance tests in a first commit, and everything else into a 2nd commit? Basically the way I did it here: https://github.com/Apteryks/python-docx/commits/add-support-for-custom-properties.

michael-koeller · 2022-07-25T13:53:17Z

for tidiness, could you please split the acceptance tests in a first commit, and everything else into a 2nd commit?

Yes, sure. Done.

Apteryks

Hi! This looks good; I added a few minor comments.

Testing it though, I seem to trigger a hang this way:

from docx import Document
doc = Document('./features/steps/test_files/doc-customprops.docx')
list(doc.custom_properties)

Is this expected?

Thanks.

docx/opc/customprops.py

docx/oxml/__init__.py

docx/opc/customprops.py

docx/oxml/customprops.py

tests/opc/parts/test_customprops.py

tests/opc/test_customprops.py

Apteryks · 2022-07-25T19:34:02Z

@michael-koeller One last thing: it'd be nice to add documentation for the new feature; it seems it'd go here: https://github.com/python-openxml/python-docx/blob/master/docs/api/document.rst, next to the CoreProperties section. A usage example would also be neat in https://raw.githubusercontent.com/python-openxml/python-docx/master/docs/index.rst, demonstrating how to access or set custom properties on a document. Thanks again!

mohitmathew · 2022-08-06T14:11:53Z

Almost there team can we push this through 👍 . Thanks for everyone's effort ..

R00118189 · 2022-11-06T17:30:44Z

@celkas, what is this feature's estimated merge date?

Apteryks · 2022-11-08T16:35:14Z

Could someone volunteer to test this branch? Especially w.r.t. #580 (review).

celkas · 2022-11-10T12:14:39Z

@Apteryks I tested it on Windows. TL;DR same result that You have in #580 (review)

from docx import Document

print("-------------------------------------------")
print("opening doc with custom-props => works fine")
print("-------------------------------------------")
name = r"C:\Python\python-docx-master\features\steps\test_files\doc-customprops.docx"
doc = Document(name)
print(name)
print(doc)
print(doc.custom_properties)

print("-------------------------------------------")
print("access using known values => works fine")
print("-------------------------------------------")
known_propvals = (
    ("CustomPropBool", True),
    ("CustomPropInt", 13),
    ("CustomPropString", "Test String"),
)
for name, expected_value in known_propvals:
    print(name, doc.custom_properties[name])

print("-------------------------------------------")
print("access like a list or dict => fails")
print("-------------------------------------------")
i = 0
for prop in doc.custom_properties:
    if i > 10:
        break
    print(prop)
    i += 1
list(doc.custom_properties)

print("-------------------------------------------")
print("end => never reached")
print("-------------------------------------------")

results in

C:\...>python docx-test.py
-------------------------------------------
opening doc with custom-props => works fine
-------------------------------------------
C:\Python\python-docx-master\features\steps\test_files\doc-customprops.docx
<docx.document.Document object at 0x000001E784E8FF40>
<docx.opc.customprops.CustomProperties object at 0x000001E7870DBE20>
-------------------------------------------
access using known values => works fine
-------------------------------------------
CustomPropBool True
CustomPropInt 13
CustomPropString Test String
-------------------------------------------
access like a list or dict => fails
-------------------------------------------
None
None
None
None
None
None
None
None
None
None
None
Traceback (most recent call last):
  File "C:\...\docx-test.py", line 32, in <module>
    list(doc.custom_properties)
  File "C:\Python\Python310\lib\site-packages\docx\opc\customprops.py", line 26, in __getitem__
    prop = self.lookup(item)
  File "C:\Python\Python310\lib\site-packages\docx\opc\customprops.py", line 70, in lookup
    if child.get("name") == item:
KeyboardInterrupt
^C
C:\...>

Apteryks · 2022-11-11T20:47:53Z

@Apteryks I tested it on Windows. TL;DR same result that You have in #580 (review)
Thanks for testing. So I think this should be fixed before we poke the maintainer to have a look and hopefully merge; @michael-koeller, is this something you'd be interested looking into? Giving a more dict-like interface to the properties (or storing them in a plain dict directly)

celkas · 2022-11-14T09:55:48Z

@michael-koeller, easiest way I see is adding this code to docx\opc\customprops.py:

    def __iter__(self):
        return iter(self._element)

This would make it possible to iterate all custom props in a file.

One needs to be aware, though, that the access to a custom-prop is (then) still not straight forward:

for prop in doc.custom_properties:
    #print(prop) # does not work!
    print(type(prop), prop.items(), prop.get('name'))

michael-koeller · 2022-11-15T07:24:27Z

I just added the iteration support on custom properties, and added the relevant test cases for this feature.

@celkas : Thanks for your input. My version is slightly different from yours. The iteration just returns the names of existing custom properties. To obtain the values, any client code may then explicitely access the getter method.

celkas · 2022-11-21T13:06:38Z

@Apteryks, looks good to me. Tested the branch, everything OK.

jakeformico · 2022-11-25T21:32:49Z

I might be a little out of my depth here, but I'm hoping someone can clear this up. With this, I was successfully able to get and update my custom property, which in my case was used for updating the existing header, but I noticed inside the actual word document nothing changed. The old header stayed the same. Then I checked, and it looks like I need to update the field even after I've updated the custom property itself. Does this functionality also handle updating the fields associated with the custom properties?

R00118189 · 2022-12-02T11:33:29Z

I might be a little out of my depth here, but I'm hoping someone can clear this up. With this, I was successfully able to get and update my custom property, which in my case was used for updating the existing header, but I noticed inside the actual word document nothing changed. The old header stayed the same. Then I checked, and it looks like I need to update the field even after I've updated the custom property itself. Does this functionality also handle updating the fields associated with the custom properties?

I believe you have to refresh it manually in MSWord and save it to keep it displayed or set MSWord to update fields before printing Click FILE > Options > Display, and under Printing options, select the check box for Update fields before printing. see here

Edit: Your header footers won't update unless you print the document with the option ↑. Maybe there is an option to refresh all programmatically; who knows?

R00118189 · 2022-12-02T12:18:48Z

@michael-koeller, how do we remove or delete custom properties from <class 'docx.opc.customprops.CustomProperties'> object?

michael-koeller · 2022-12-02T12:24:42Z

@michael-koeller, how do we remove or delete custom properties from <class 'docx.opc.customprops.CustomProperties'> object?

Good question. That's not covered yet, as it seems.

michael-koeller · 2022-12-02T13:30:34Z

@michael-koeller, how do we remove or delete custom properties from <class 'docx.opc.customprops.CustomProperties'> object?

OK, should be working now. 😉

I also added corresponding test cases.

R00118189 · 2022-12-02T14:35:19Z

@michael-koeller do we delete by setting the value (that does not work, as properties stay with empty values) to None or is there a way to access the delete function and get rid of the property altogether?
edit: is it del doc.custom_properties['CustomPropString']

michael-koeller · 2022-12-02T14:43:18Z

@michael-koeller do we delete by setting the value (that does not work, as properties stay with empty values) to None or is there a way to access the delete function and get rid of the property altogether?

Ah, sorry. I forgot to explain:

    custom_properties = document.custom_properties
    # delete custom property "Foo"
    del custom_properties["Foo"]

The implementation is defensive. So if there's no custom property to the given key, nothing happens.

celkas · 2022-12-21T13:36:07Z

What can be done to make this happen?

Apteryks · 2022-12-31T20:41:06Z

    custom_properties = document.custom_properties
    # delete custom property "Foo"
    del custom_properties["Foo"]

I'm wondering, is there a reason the properties can't be kept in a true Python dict object, instead of a look-alike class? I think the API would be more intuitive if this was the case (we can simply document that the properties are exposed as a dictionary, the same way the environment variables are exposed as such via os.environ). What do you think?

@michael-koeller ^

michael-koeller · 2023-01-02T07:59:35Z

I'm wondering, is there a reason the properties can't be kept in a true Python dict object, instead of a look-alike class?

That's already part of the original implementation by @renejsum.

The util class actually operates directly on the XML document tree. So while it might as well be possible to keep the custom properties in a plain dictionary, you would still need code to couple this dictionary instance to the XML elements in the tree.

Apteryks · 2023-01-02T15:16:25Z

Hi everyone, and happy New Year!

I've tested this again, and the test suite passes (============================= 1477 passed in 5.10s =============================), and the implementation seems to work alright.

My preference would be to have a better emulated dict-like interface (where you could call .values() or .items()), but that can always be added later.

@scanny in case you haven't followed this discussion, this is about adding support for custom properties; the change includes tests and has been reviewed and manually tested by myself and others.

R00118189 · 2023-01-13T14:45:35Z

Lads, what is the roadmap for merging it into the repo?

Apteryks · 2023-01-16T03:00:18Z

Lads, what is the roadmap for merging it into the repo?

@scanny as the owner of the project has the last word; it seems they haven't had the time to catch up with this thread yet; I hope they do, as it seems to be a much wanted feature :-).

jzzsxm · 2023-02-24T15:01:54Z

@michael-koeller and @Apteryks I appreciate all the hard work on this. I've just implemented this new functionality at work and it's a huge time saver. Very well done!

Kasimashi · 2023-07-07T14:44:53Z

Any news about it ?

sHermanGriffiths · 2023-07-12T20:33:58Z

@michael-koeller @scanny, any updates?

celkas · 2023-10-20T08:57:55Z

@michael-koeller it seems there are conflicts to be resolved? At least that is a new message appearing in my feed...

scanny · 2023-10-20T18:10:05Z

Hi All, apologies for the long hiatus. This PR looks potentially viable. @michael-koeller are you still interested in driving this?

I have a few comments but no sense getting into that if it's gone stale. It would need a rebase given all the changes in getting from 0.8 to 1.0 and dropping the Python 2 support.

Let me know if you're still interested and we can see where to go next.

michael-koeller · 2023-10-23T05:49:33Z

Hi Steve, great to hear from you. And yes, I’m still interested in having this feature merged into the project. I hope that I find the time today to make a rebase on my MR. If you have any further comments, please don’t hesitate to hand them to me.

…

Am 20.10.2023 um 20:10 schrieb Steve Canny ***@***.***>: Hi All, apologies for the long hiatus. This PR looks potentially viable. @michael-koeller <https://github.com/michael-koeller> are you still interested in driving this? I have a few comments but no sense getting into that if it's gone stale. I would need a rebase given all the changes in getting from 0.8 to 1.0 and dropping the Python 2 support. Let me know if you're still interested and we can see where to go next. — Reply to this email directly, view it on GitHub <#580 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALKVDLURDK276XLZAA5RJTYAK5ATAVCNFSM4GGBZMT2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZXGMYTQMRWGEZQ>. You are receiving this because you were mentioned.

michael-koeller · 2023-10-23T11:48:35Z

Oops.

While working on a rebase for this MR, I accidentally closed this MR. Sorry for any inconveniences. 😞

As a possible followup, I created a new MR on my recent branch: #1273

jghaines mentioned this pull request Dec 10, 2018

feature: add custom properties support #91

Open

geobeo approved these changes Jan 29, 2020

View reviewed changes

aleksandarbos added a commit to openlawlibrary/python-docx that referenced this pull request Feb 29, 2020

custom properties feature

421c2a1

applied custom properties pr python-openxml#580 for storing private metadata within custom xml part.

aleksandarbos mentioned this pull request Feb 29, 2020

merge custom properties openlawlibrary/python-docx#76

Merged

2 tasks

zebulon501 reviewed Mar 17, 2020

View reviewed changes

docx/opc/package.py Outdated Show resolved Hide resolved

sanzoghenzo reviewed Jul 4, 2020

View reviewed changes

docx/opc/customprops.py Outdated Show resolved Hide resolved

lsaint referenced this pull request in lsaint/python-docx-oss May 25, 2022

Merge pull request #2 from michael-koeller/master

db24f31

Add custom properties support + unit-tests, feature-tests #580

michael-koeller force-pushed the master branch from 981e38a to dd38dc8 Compare July 25, 2022 07:46

michael-koeller force-pushed the master branch from 347f2a0 to a413b72 Compare July 25, 2022 13:46

Apteryks suggested changes Jul 25, 2022

View reviewed changes

michael-koeller closed this Oct 23, 2023

michael-koeller force-pushed the master branch from 515f8ef to 8e05650 Compare October 23, 2023 09:48

michael-koeller mentioned this pull request Oct 23, 2023

Add custom properties support + unit-tests, feature-tests #1273

Open

Add custom properties support + unit-tests, feature-tests #580

Add custom properties support + unit-tests, feature-tests #580

Conversation

michael-koeller commented Nov 23, 2018

renejsum commented Nov 23, 2018 via email

Ricyteach commented Dec 10, 2018

jekozyra commented Aug 23, 2019

BastienFaure commented Sep 14, 2019

geobeo commented Jan 29, 2020

nozokada commented Dec 16, 2020

Ricyteach commented Dec 16, 2020

paweljasinski commented Jan 15, 2021

HansBambel commented Apr 23, 2021

mohitmathew commented Dec 14, 2021

hugoe commented Feb 10, 2022

boomesq commented Mar 7, 2022

lsaint commented May 19, 2022

Apteryks commented Jul 25, 2022

sanzoghenzo commented Jul 25, 2022

michael-koeller commented Jul 25, 2022

Apteryks commented Jul 25, 2022

michael-koeller commented Jul 25, 2022

Apteryks left a comment

Choose a reason for hiding this comment

Apteryks commented Jul 25, 2022

mohitmathew commented Aug 6, 2022

R00118189 commented Nov 6, 2022 • edited Loading

Apteryks commented Nov 8, 2022

celkas commented Nov 10, 2022 • edited Loading

Apteryks commented Nov 11, 2022

celkas commented Nov 14, 2022

michael-koeller commented Nov 15, 2022

celkas commented Nov 21, 2022

jakeformico commented Nov 25, 2022

R00118189 commented Dec 2, 2022 • edited Loading

R00118189 commented Dec 2, 2022

michael-koeller commented Dec 2, 2022

michael-koeller commented Dec 2, 2022

R00118189 commented Dec 2, 2022 • edited Loading

michael-koeller commented Dec 2, 2022

celkas commented Dec 21, 2022

Apteryks commented Dec 31, 2022 • edited Loading

michael-koeller commented Jan 2, 2023

Apteryks commented Jan 2, 2023

R00118189 commented Jan 13, 2023

Apteryks commented Jan 16, 2023

jzzsxm commented Feb 24, 2023

Kasimashi commented Jul 7, 2023

sHermanGriffiths commented Jul 12, 2023

celkas commented Oct 20, 2023

scanny commented Oct 20, 2023 • edited Loading

michael-koeller commented Oct 23, 2023 via email

michael-koeller commented Oct 23, 2023

R00118189 commented Nov 6, 2022 •

edited

Loading

celkas commented Nov 10, 2022 •

edited

Loading

R00118189 commented Dec 2, 2022 •

edited

Loading

R00118189 commented Dec 2, 2022 •

edited

Loading

Apteryks commented Dec 31, 2022 •

edited

Loading

scanny commented Oct 20, 2023 •

edited

Loading