DOC add example for Cramer V for column_associations

[reshamas]

add example for Cramer V

Will be rendered here in documentation:
https://skrub-data.org/stable/reference/generated/skrub.column_associations.html

Towards #1181

[jeromedockes]

Hi @reshamas thanks for opening this! such examples directly in the reference documentation are super useful to understand how to use a library.

Your example is great! I think >>> df.shape() needs to be replaced with >>> df.shape to fix the error. to run the doctests on your machine, which can be easier than waiting for the CI to run you can use the command pytest skrub/_column_associations.py

[jeromedockes]

Also I see you add the example section to the _compute_cramer function. This one is a "private" helper function used only within the _column_associations module itself. you can tell because its name starts with an underscore, and it does not have an entry in the skrub documentation. Could we move the Examples section that you wrote to the docstring of the column_associations function here? thanks!

[GaelVaroquaux]

Super good idea to have done this as a docstring example

[rcap107]

The same for this line and the one past associations

[reshamas]

added output for associations

[reshamas]

It looks like the output is too wide (> 80 chars):

       left_column_name  left_column_idx right_column_name  right_column_idx  cramer_v
    0               c_3                3             c_str                 5    0.8215
    1               c_1                1               c_4                 4    0.8215
    2               c_0                0               c_1                 1    0.8215
    3               c_2                2             c_str                 5    0.7551
    4               c_0                0             c_str                 5    0.7551
    5               c_0                0               c_3                 3    0.7551
    6               c_1                1               c_3                 3    0.6837
    7               c_0                0               c_4                 4    0.6837
    8               c_4                4             c_str                 5    0.6837
    9               c_3                3               c_4                 4    0.6053
    10              c_2                2               c_3                 3    0.6053
    11              c_1                1             c_str                 5    0.6053
    12              c_0                0               c_2                 2    0.6053
    13              c_2                2               c_4                 4    0.5169
    14              c_1                1               c_2                 2    0.4122

[rcap107]

It's something to do with different configurations for different pandas versions, tests fail on the min reqs configuration 🤔

[rcap107]

Adding >>> pd.set_option('expand_frame_repr', False) seems to print it properly, however now there's a problem with the ordering of the rows, which for whatever reason changes between pandas 1.5 and pandas 2.2.

Not sure how we can get around this, @jeromedockes any idea?

[jeromedockes]

so I think this should be ok for the display:

    >>> pd.set_option('display.width', 200)
    >>> pd.set_option('display.max_columns', 10)
    >>> pd.set_option('display.precision', 4)

and then at the end of the example

    >>> pd.reset_option('display.width')
    >>> pd.reset_option('display.max_columns')
    >>> pd.reset_option('display.precision')

[rcap107]

Hi @reshamas, thanks for contributing! I added a few comments that should fix the tests.

[rcap107]

You might also want to run pytest skrub/tests/test_docstrings.py to check everything runs properly.

[jeromedockes]

sorry for the delay I'll look at it now

[jeromedockes]

I think we should have a little function in the datasets module like toy_dataframe() that we can use in the reference examples so simplify the setup code and so we get the same every time

also we probably can set pandas display options before running the doctests and perhaps reset them before and after running each module's doctests

I don't want this stuff to block this PR though, because it really helps clarify what the output of the column_associations contains. I suggest we skip this one doctest output, as we do in a few other places where ensuring the output is exactly consistent would make the example too complicated, defeating its documentation purpose.

[jeromedockes]

thanks for adding the explanation and the link that's super useful!

[jeromedockes]

the rendred output looks great: https://output.circle-artifacts.com/output/job/c186c9ef-59a1-47b8-81f2-3b901a436f1b/artifacts/0/doc/reference/generated/skrub.column_associations.html

[jeromedockes]

Looks great!! thanks a ton @reshamas . this function was moved to the public api in the last release as we realized it can be very useful on its own ... but it was lacking a proper documentation and examples! battling with pandas output formatting details across multiple versions of pandas and python is not fun but thanks a lot for pushing through :)

[reshamas]

Thank you @jeromedockes and @rcap107. It was a very good learning experience.

[rcap107]

Thank you @jeromedockes and @rcap107. It was a very good learning experience.

Likewise, thanks for contributing!

[jeromedockes]

yep, thanks a lot! I know you had also started working on adding Pearson correlations to the output, so if you have any questions about that don't hesitate to reach us here on github or on the skrub discord server: https://discord.gg/ABaPnm7fDC