Add support for data xfer completion IRQs

[gsomlo]

@Dolu1990
@paulusmack

[paulusmack]

There is a subtle race introduced by commit 898fde8. Once you call complete(), it is possible that the wait_for_completion() call in litex_mmc_send_cmd() could return (on another CPU) and another command could be issued to the hardware before we get to clear the pending interrupt bits here by writing to the irq pending register. If that second command has already completed, we will then miss its interrupt. I know this is pretty unlikely on the slow CPUs that litesdcard is generally used with, but it is at least theoretically possible.

A similar race exists for the card detect interrupt, if there is any possibility that mmc_detect_change could do something that triggers an interrupt.

What I suggest instead is to write back to the pending register immediately after reading it, before taking any action based on its contents. We can actually just write back the same value that we read. Then, if anything we do causes another interrupt to arise, it will stay pending and cause us to enter litex_mmc_interrupt() again to handle it.

[paulusmack]

@gsomlo

I did some measurements with these patches applied to the kernel I'm currently using (it's based on 6.4, I'll move up to 6.6 soon). This is on a microwatt system with one core, 256MB DRAM, 100MHz CPU clock, and litesdcard connecting to a Sandisk 32GB "high endurance" micro SD card, and 25MHz maximum SD clock speed. I haven't updated the gateware yet, so I hacked the litex_mmc driver to put back the enabling/disabling of the command-done interrupt. (I don't believe that will affect the results measurably since it's only two MMIO writes per command; I will update the gateware and re-do the measurements shortly, but I though you might be interested to see these as a preliminary indication.)

Also, I used the 'time' command rather than 'date' so as to get more accurate measurements. That is, the read test is

# mount /dev/mmcblk0p1 /mnt; time cp /mnt/foo .; umount /dev/mmcblk0p1

and the write test is

# mount /dev/mmcblk0p1 /mnt; time cp foo /mnt; time umount /dev/mmcblk0p1

I use the "real" (overall elapsed time) measurement, and for the write test I add together the times for the cp and the umount. The file being copied is 32768 kiB.

Using the current driver without your changes, the read test takes 10.8 seconds and the write test takes 24.4 seconds. With your changes the times are 11.0 seconds and 26.1 seconds, so about 2% slower on reads and 7% slower on writes.

[gsomlo]

@paulusmack -- thanks for the test results and the feedback. Please see #16 (comment) for my reply (it's the re-worked for more clarity patch series, witht the same end result as this one, just better git commit "sausage-making" :) )

[gsomlo]

closing, obsoleted by #16