Low FPS? #3
Comments
|
Hi, thank you for your comment! Yes, you are correct that the screen takes 10 frames at 60Hz to respond, that's ~170ms from either direction. So if implemented in a naive way this would translate to a framerate of only 6Hz. This is far from ideal, so existing commercial implementations made a tradeoff between contrast ratio and framerate: If it doesn't drive the screen to the full white/ black, then the time could be reduced. Due to the nonlinearity of the screen, you could go pretty far with this. Many of them even exposed the control to the user (slow mode, fast mode, fast++ mode, etc) so users could choose the compromise themselves. But it doesn't have to be like this. One example would be early STN or early VA screens, which usually have a long response time of 80-300ms (STN) or 20-90ms (early VA). These are higher than the frame time of 16ms but these screens could deliver smooth (though less clear) images at 60fps. This basically comes down to 2 points to mimic these slow LCDs:
The downside of these optimizations is that they require maintaining pixel state information on a per-pixel basis, requiring a lot of memory bandwidth. Then how does this (6 Hz + optimizations) compare to commercial implementations (15 Hz but lower contrast)? Due to the 2nd optimization, it in some ways, performs the tradeoff between contrast and framerate automatically. For example, in a worst-case scenario, to stress this capability you display alternate black and white frames at 15 Hz. Commercial implementation would drive it with lower contrast. With Caster, it would try to drive to the highest contrast, but it can't: pixel changes color before reaching 10 frames count, so it cancels the current update and starts driving to another color. This results in not fully driven pixels, or, black not fully black, and white not fully white. This is essentially reduced contrast. So in such a worst-case scenario, it performs roughly the same as existing solutions. But in more realistic settings, keeps alternating between white and black is rare. If given enough time (10 frames), Caster would take advantage to drive it to the highest contrast rather than stopping at a not fully driven state. Being able to respond to every frame (optimization 1) and dynamically adjust the frame count or contrast target for driving (optimization 2) does go a long way in the final result. I think you've seen the end result in my tweet (and some YouTube videos embedded in Modos blog posts), I would call it 60 FPS instead of 6 FPS. Then regarding the maximum possible refresh rate. You are right it could go up to 60MHz and potentially achieves a 120Hz refresh rate. This would improve the grayscale display, but unfortunately not the monochrome display. Also at the expense of doubling the memory bandwidth requirement. Regarding the last point, I was curious about that as well. Unfortunately, I am not a TFT device guy or Eink material guy, so I don't know the answer for sure. But this is a straightforward experiment to carry out: if this theory is true, then, with the current screen driving circuitry design, the lower resolution, the faster it should be able to go. This is easy to try, just don't drive the full display area, and only the first 10 or 20 lines or so in each frame before sending a VSYNC, with a corresponding lower clock, so line time is greatly lengthened. However, little improvement in response time is observed, so the theory is at least flawed in some way. Maybe the pixel has some capacitance somewhere (for example inside TFT) so they only need to be charged up in a single line time and then the TFT would take care of it during the rest of the frame. Again I don't have a real answer here. If you are curious about the waveform, there are some tools in my NekoInk repo to convert waveform from i.MX .bin format to a somewhat human-readable .csv format. The grayscale ones are quite delicate, but monochrome black/ white is really just keep driving the same value. That would represent how commercial implementations drive the screen. The waveform support in Caster is still working in progress though. |
|
Hi wenting, Thanks for your incredible work as well as the very informative article in https://www.modos.tech/blog/modos-paper-monitor, it has been really helpful for me. I'm also doing some early hacking with the eink screen and I'd like to use this thread to ask some follow up questions: You mentioned that
Say, if some pixel takes 10 frames of positive signal to change from black to white. Now before the current update is finished (only 6 frames passed) and we decide to change it to black again, do we apply a full 10 frame of negative signal on it? or a reduced number of frames considering that its current state is gray? If it's the former case, does it possibly produce burn-in effect on the screen and maybe even harm it permanently? If it's the latter case, how do we know the exact waveforms to use? Another question: you also mentioned that
Could you please elaborate on the "unfortunately not the monochrome display" part? If I understand it right, it's related to the last point that the update voltage seems to be constantly maintained for the entire frame, so the display latency cannot be really improved by higher refresh rate, is that right? Thanks in advance! |
|
Hi @blahgeek,
As far as I know, using your example, signal is driven in the opposite direction for 6 + 2 frames. That is we retrace our steps to black, plus to 2 more frames for good measure. The purpose of the additional frames is to make sure that the pixel is now as black as it could be. In general it's thought that such overdriving is harmful for the microcapsules, but I think Wenting thinking is that because we are overdriving for only 2 frames it's not such a big deal. In previous conversation he told me that he experimented with deliberately overdriving some EPD display but couldn't damage. I don't know much details about that experiment, e.g. for how long he ran it(probably closer to hours than days), but it seems that damaging the display this way, while supposedly possible, takes some "determination". Moreover, I guess in the expected case a pixel would be overdriven in both 'white' and 'black' direction so possibly this charging of the particles, or whatever it is that causes the damage may cancel out. I don't think anyone in the open source eink community had run extensive experiments to determine whether, to what extent, and over what time frame the screen is damaged if driven this way. For your second question I can't say for sure what algorithm for obtaining grayscales Wenting has in mind... In general though grayscales on EPD require more elaborate waveforms, so it seems plausible that having finer grained control might be helpful. By the way after asking my question I did some Googling on that, and it seems that there are indeed per-pixel "storage" capacitors. In TFT LCD's they maintain the state of a pixel until it's scanned again in the next frame. In EPDs, which use basically the same technology, the charge is used to sustain the driving direction until the next frame. At least that's my understanding. |
|
Long time no see, Wenting. It seems like the open-source hardware you've been working on is different from the hardware shown in the videos. Will you consider updating them simultaneously in the future? Additionally, have you considered incorporating an FPGA with ARM, such as the Zynq7010, for the current Raspberry Pi MCU? |
Hi, can you clarify the simultaneously part? Do you mean hardware and video? I have considered about Zynq 7010, but it's kind of like an overkill for the application and provides little benefit. |
|
Hi @zephray , By the way do you consider Caster/Glider ready for production? I mean if someone is interested in trying them, would it be possible to order the board assembled on JLCPCB for example? Do you know what the ballpark cost for that would be? Thanks! |
I mean there are some differences between your open-source FPGA PCB and the PCB in your video. Have you tried the resolution of 2200x1650? I think the cost-effectiveness of zynq7010 will be higher |
你是中国人吗 |
Of course there are differences, it doesn't make too much sense for me to redo video whenever I make a new board revision. I don't own any 2200x1650 resolution screens, so, no that's not tested but that's on the plan. The cost-effectiveness is hard to evaluate: I know that used 7010 chips are extremely cheap, but not so much if you buy brand new chips from authorized resellers. Also keep in mind the RPi MCU I am using costs less than $1 in bulk, it basically doesn't contribute much to the overall BOM, especially when the Eink screen costs >$100. Trying to optimizing on a <$10 FPGA or a <$1 MCU is focusing on the wrong thing. There are also multiple design goals, and the cost is only being one of them, for example power consumption. It's not entirely clear between 7010 and Spartan6 which one is better (it depends on the application a lot). But, bottom line, it's all open source code. If 7010 make a lot of sense to you, go ahead and port it to the 7010, there should be nothing stopping you from doing so. I may even do that myself in coming months just to see how it performs. |
It's not exactly production ready yet, but I do have a timeline for you: it should be done before Feb 2024. Gowin-based design is an interesting one that I might explore in the future. Personally I had some bad impressions about Chinese FPGAs: I started using Anlogic FPGAs quite a few years ago because they were quite affordable. But turns out their software is buggy, hard to download. Even worse, they somehow decided they will no longer selling chips on LCSC, making it really hard for me to buy samples overseas. Gowin chips are also suffering from logic issues when utilization is high even when timing is clean (YosysHQ/apicula#169). But I have also heard good words towards their chips, so I am not sure. |
After listening to your explanation, I suddenly realized that in China, the screen price of EINK is indeed much lower. I am looking forward to your work being released soon, and I am also looking forward to your laptop project. Nice to meet you. You are truly admirable |
Your FPGA SPI interface not only connects to bit files, but also contains some other content that I currently cannot understand |
That's great to hear. Looking forward to being able to order it.
Yes, this always seems to be a concern with the smaller manufacturers. The problem is Intel and AMD don't seem interested in the low segment anymore, so it will likely end up being dominated by Lattice and the new Chinese players. Gowin's intentions look serious, but there still doesn't seem to be an easy way to buy their chips in single quantity, except on Mouser. And the price there is higher than the Sipeed's price for an entire Tang Primer 20K module which packs 128MB of DDR3.
Yes, in fact it was me who suggested to refer this to Apicula's developers, when it was first posted on r/GowinFPGA. The interesting thing about this is that no one else seems to be affected by this problem, which makes me think that there might be some peculiar feature of poster's design that triggers it. I haven't had the time to experiment with his demo design, but there are two things that stick out to me:
|
Being available on mouser is a big step forward already IMO, guess I will pick up a Tang Primer 20K to play with first.
Thanks for the insight! Glad to hear the problem is not wide-spread and the FPGA might not be to the blame. |
If you just want to familiarize yourself with Gowin's architecture and tooling Tang Nano 9K might be a better choice. As you may know already, it's a sweet, self-contained, breadboardable and fairly affordable module. It has 2x4MB PSRAM dies built-in into the FPGA package. Never had the opportunity to use Spartan 6, but I suspect you will find Gowin's FPGAs a good deal slower(they seem to strive for power efficiency). Nevertheless, I'm reasonably sure that if one really wants to, they can build an EPD monitor controller by just using the TN9K module, plus something like adv7611 and couple of voltage regulator/smps chips. |
|
别推高云了 就是垃圾器件 |
|
always @(posedge clk) begin |
Hi Wenting,
it's great to see that you are still working on this project. After a long pause I'll be starting to tinker with EPD driving again(I'm planning to perhaps do some experiments using a simple board(no video input for now) based on Tang Nano 9K). Your design provides a very helpful guidance.
I've been studying the code in the last couple of days, and one of the things that jumped at me is that apparently you've determined that at least 10 frames are needed to go from white to black and vice versa (I'm looking at FASTM_B2W_FRAMES in pixel_processing.v). You've actually revised this value up from the initial 9 frames, according to commit history, which tells me that it's likely not just an initial "just-to-be-sure" value, but you've actually determined that at least that many frames are really needed. As far as I understand(mostly from what you've written in blogs etc- haven't been able to figure out the frequency of v_pclk from which epd_clk derives from ISE's timing reports for example) the EPD pixel commands are sent at 60Hz, so 10 frames translates to exactly 6 FPS. Is this correct?
If so, why do you think this huge gap from the commercial implementations that reportedly can go to 15 FPS or even higher if what's reported on the web can be trusted(haven't tried to measure that myself yet). I wonder how do they achieve these high FPS rates... Do you think it's due to fancy waveforms, not driving fully to black and white or something else?
In the Waveshare's datasheet for their 1600x1200 13.3" panels they list the max source driver frequency as 60MHz(16.7ns period), which if I'm not mistaken translates to about 120 screen updates per second, which is twice the 60Hz you that you are currently using. Do you think higher update frequency could improve(or worsen for that matter) the FPS?
I guess for many applications such as text entry, which is arguably the most compelling use case for these monitors, the low latency(that Caster already supports) is at least as important, if not more so, as the throughput/FPS, but still ideally we'd like to have both.
There is something very fundamental about the process of driving EPDs that I don't understand. It could be tangential to the current topic, but I'd be very grateful if you could help me understand anyway... The basic process, as I understand it is: we go through each row using the gate driver signals/clock, we write the commands(2bits/pixel) for the current row into the source driver register, then we tell to source driver to start updating the row. Then we go to the next row and so on. The thing that I don't understand: do pixel update voltages for the previous row continue to be applied when the gate driver switches to the next row, or the row is updated only until the gate driver switches to the next one? The descriptions of the driving process as far as I recall seem to say that the voltages are only applied until gate driver moves on to the next row. However, wouldn't that imply that, contrary to the common belief, EPD pixels could be updated very fast, and it's only the limitations of driving circuitry and perhaps power supply that makes them slow. I mean, a typical 13.3" panel has 1200 rows, so if the rows are only updated while activated by the gate driver, that would mean that each row is only updated for less than a millisecond each second. And in this short period it must be able to change state several(at least 6 :) times. Would that imply that theoretically EPD updates can be drastically speed up, just by changes in the driving circuitry? (Theoretically, because even in the unlikely case this hypothesis is true, I don't think there would be much interest from the current manufacturers to do that). I must be missing something obvious, so I would appreciate very much if you could help me clear the confusion.
The text was updated successfully, but these errors were encountered: