Table in a container does not display or update as expected.

[tfcroft4]

Describe the bug

possibly an extension to: #2974
A table within a container does not initially expand columns
The table does not display data updates when the underlying data is updated.
A row edit is not reflected in a change in the display
Append and Delete Rows show updated data including pending row edits.

Steps to reproduce

I attach a test program that can be run in a dev environment to demonstrate the problem.

A new table in an opcontainer is displayed with compressed columns.

To Test: Press the Run table Test button. There are 5 tests.
1 use append to add a new Row. New Row IS visible
2 use find to find row matching the given attribute and value - an exact match.
3 set row attribute using tablet.data[rowid].setattr('age',999), updated value is NOT shown
4 set row attribute, upsert function will update an existing row OR insert a new row.
new value of age = 0 expected but NO change shown
5 delete an existing row, the table display updates to not show the removed row as expected but does now show the
previously updated updated age for the last row.
[
tabletest.zip
]

Expected behavior

On a programmatic change in data e.g. made with a call to tablet.data[rowid].setattr('age',999) I would expect the underlying ListSource to update and the display to reflect the change. However although it can be shown the ListSource has changed the display does not update.
Note that the change does show following a subsequent Row Delete action

Screenshots

following a further update to age = 0 and a Row Delete:

Environment

• Operating System: Windows 10 64bit
• Python version: 3.10.5
• Software versions:
  • Briefcase-Version: 0.3.20
  • Toga: 0.4.8
  • ...

Logs

terminal putput:

> Upsert Zaphod age 0
> Looking for  Zaphod
> 0 <Row 2217b225a50 age=42 name="Arthur 'Dent">
> 1 <Row 2217b225a80 age=37 name='Ford ""Prefect'>
> 2 <Row 2217b225ae0 age=38 name='Tricia McMillan'>
> 3 <Row 2217fe6d360 age=999 name='Zaphod Beeblebrox'>
> Found Zaphod at 3
> Updating Zaphod age 0  at row  3
> Updated Zaphod age 0
> Looking for  Zaphod
> 0 <Row 2217b225a50 age=42 name="Arthur 'Dent">
> 1 <Row 2217b225a80 age=37 name='Ford ""Prefect'>
> 2 <Row 2217b225ae0 age=38 name='Tricia McMillan'>
> 3 <Row 2217fe6d360 age=0 name='Zaphod Beeblebrox'>
> Found Zaphod at 3
> Deleting Row Arthur
> Looking for  Arthur
> 0 <Row 2217b225a50 age=42 name="Arthur 'Dent">
> Found Arthur at 0

briefcase.2024_12_16-11_05_11.dev.log

[freakboy3742]

As far as I can make out, at best, this is a Windows specific issue. On macOS, I don't see any issues with the update you've flagged as a problem.

I do, however, get a hard crash during test "5". Debugging this issue is very difficult, though. Between the series of nested functions, weird workarounds for "issues" (such as the many calls to "scroll_to_row"), API usage like literal AST parsing, the large blocks of commented out code, the weird usage of Python APIs (like __setitem__ and __delitem__, instead of using [] notation), custom-reimplementations of APIs like myfind() instead of table.data.find() make it very difficult to follow what the intended behavior is. As a result, it's unclear if the problem you are seeing is because of an error of usage (or, at least, very off-book usage of APIs); an error of logic caused by confusion over what object tablex is currently referring to; or an actual bug in Toga.

When you're providing a minimal reproduction case for a bug, the critical word is minimal. You need to provide the least possible code to reproduce the problem. Having to press a button 5 times to generate a problem? Unless "do X 5 times" is the problem you're observing, your example code has just become 5 times more complicated than it needs to be. In this case, if the issue is "row doesn't update"... then the example should be "updating a row". If changing the scroll position of the list isn't a critical part of making the bug manifest - don't scroll the position of the list. Any complication above and beyond what makes the bug occur only serves to make the underlying problem harder to find.

It's also a key part of getting an issue looked at. Right now, if I want to investigate this issue, the first thing I'll need to do is spend an hour rewriting the example so that it only demonstrates the problem you're describing. That inevitably puts this bug further down my todo list, because I need to find an additional hour to work on in, in addition to any time I need to spend hunting down a fix.

[tfcroft4]

Thanks for your comments.
Unfortunately I am only able to test this in a Windows environment at present.

The reason for the what you see as complexity is that I was trying to pinpoint what did and did not work in a step by step approach rather than saying. "It does not work!"
I had seen that a data update to a ListSource were not being reflected in the Table display when the table itself was within a sub container.

The step by step test approach was taken to try and demonstrate what did work and what did not work.

• Testing the data source after an data change directly showed that the data had been updated.
• Testing scrolling to attempt a display refresh had no effect but I did not know that until I tried.
• The data change did not trigger a display update ( with or without scrolling) but a row delete does.

I appreciate that some one with more knowledge or experience may have been able to take a different approach.
Cheers
Ted

[freakboy3742]

Thanks for your comments. Unfortunately I am only able to test this in a Windows environment at present.

Sure - and that's not a problem, as long as you declare that (which you have). We don't expect every user to have access to every platform.

The reason for the what you see as complexity is that I was trying to pinpoint what did and did not work in a step by step approach rather than saying. "It does not work!" I had seen that a data update to a ListSource were not being reflected in the Table display when the table itself was within a sub container.

Sure - except that you've stopped too soon. Once you've got a broken example, you then need to continue simplifying until you can remove nothing else. For example - how is the "substring search" behavior of myfind important to the test? If it's not required - remove it. Is the "find" ever going to fail? If not, remove the error handling for that case. Anything getting in the way of demonstrating the problem is a distraction, and should be removed.

Also - you need to frame the example so that it best exhibits the problem you're trying to describe. You say that the scrolling is necessary? Make that part of the test.

To demonstrate what I mean, I've reworked your example:

import toga
from toga.style import Pack
from toga.style.pack import COLUMN, ROW
from toga.colors import CYAN


class TableTest(toga.App):
    testno = 1

    def runtest(self, testno, table):
        # use widget id
        self.widgets["testtext"].value = f"Running Test {testno}"
        match testno:
            case 1:
                # Append new data to the table
                new_row = table.data.append(("Zaphod Beeblebrox", 47))
                self.widgets["testtext"].value = f"Appended {str(new_row)}"
            case 2:
                # Find then update a row.
                row = table.data.find({"name": "Zaphod Beeblebrox"})
                row.age = 999
                self.widgets["testtext"].value = "Zaphod Beeblebrox's age set to 999"
            case 3:
                # Scroll to the row to refresh
                row = table.data.find({"name": "Zaphod Beeblebrox"})
                index = table.data.index(row)
                table.scroll_to_row(0)
                table.scroll_to_row(index)
                self.widgets["testtext"].value = "Scrolled to force an update"
            case 4:
                # drop an item using index, uses myfind
                row = table.data.find({"name": "Arthur Dent"})
                index = table.data.index(row)
                del table.data[index]
                self.widgets["testtext"].value = "Remove row"

            case _:
                self.widgets["testtext"].value = "Test complete"

   def test_table(self, widget, **kwargs):
        self.runtest(self.testno, self.table1)
        self.runtest(self.testno, self.table2)

        self.testno += 1

    def startup(self):
        """Construct and show the Toga application.
        Toga table at the Top Level
        tabel Test Class to test updates and call backs
        """
        self.table1 = toga.Table(
            headings=["Name", "Age"],
            accessors=["name", "age"],
            data=[
                    {"name": "Arthur Dent", "age": 42},
                    {"name": "Ford Prefect", "age": 37},
                    {"name": "Tricia McMillan", "age": 38},
                ]
            )

        self.table2 = toga.Table(
            headings=["Name", "Age"],
            accessors=["name", "age"],
            data=[
                {"name": "Arthur Dent", "age": 42},
                {"name": "Ford Prefect", "age": 37},
                {"name": "Tricia McMillan", "age": 38},
            ]
        )

        # boxes etc
        test_label = toga.Label(
            "Test message: ",
            style=Pack(padding=(0, 5)),
        )
        test_input = toga.TextInput(id="testtext", style=Pack(flex=1))

        button = toga.Button(
            "Run Table Test",
            on_press=self.test_table,
            style=Pack(padding=5),
        )

        test_box = toga.Box(style=Pack(direction=ROW, padding=10))
        test_box.add(test_label)
        test_box.add(test_input)
        table_box = toga.Box(
            style=Pack(background_color=CYAN, direction=COLUMN, padding=10)
        )
        table_box.add(self.table2)

        # Create tabs
        opcontainer = toga.OptionContainer(
            style=Pack(direction=COLUMN, flex=1),
            content=[
                toga.OptionItem("Table CRUD", table_box)
            ]
        )

        main_box = toga.Box(style=Pack(direction=COLUMN, padding=5))
        main_box.add(test_box)
        main_box.add(button)
        main_box.add(self.table1)
        main_box.add(opcontainer)

        self.main_window = toga.MainWindow(title=self.formal_name)
        self.main_window.content = main_box
        self.main_window.show()


def main():
    return TableTest("Table Test", "com.example.tabletest")


if __name__ == "__main__":
    main().main_loop()

In this example:

1. Adds the row. It's correctly displayed on both Windows and macOS.
2. Finds and updates the row. Zaphod's age does change on macOS, but not on Windows
3. Forces a scroll-to-refresh. This doesn't change anything on Windows.
4. Deletes a row. This removes the Arthur row, and Zaphod's age is updated.

In addition, if you manually select the Zaphod row at any point after step 2, the age is correctly rendered.

With a minimal reproduction case, I can now easily demonstrate the difference between the table update, the scroll, and the delete.

Even this isn't a 100% minimal example, because being nested in an OptionContainer doesn't change anything - you can see the same behavior in the "main container" table. However, given there's some potential crossover with #2974, and it's easy to accomodate both tables in the example, I've left it in.

But as a result of this simplified example, I can also easily conform that the Winforms table is being notified of the change of data; but this isn't triggering a redraw of the virtual table item. I'm not entirely sure why this is occurring, but it might be related to recent changes restoring keyboard navigation for tables.

[HalfWhitt]

Just chiming in that I can similarly reproduce the data display lag on Windows (with or without a box or option container), but the column widths still work correctly for me, even inside a container. Tested on Windows 11.