Difference between the old sparkmonitor and the new one

[wqh17101]

the old monitor's feature

the new:

So where is the feature which is in old ver but not in the new one.

[krishnan-r]

• Due to the number of tasks being very large in some workloads, rate limits had to be implemented on number of messages sent to the Jupyter fronted. This means it is not possible to reliably show details about an individual task, it is only possible to plot aggregated statistics.

• The feature to proxy the Spark UI through the extension was dropped due to the complexity involved and differences in configuration when submitting to remote clusters. Some clusters also use the History Server instead of the Spark UI and along with authentication. This is too much effort to maintain for local, yarn and kubernetes clusters, and needs investigation of alternatives (Explore an alternative approach to Spark UI Proxy #3).

If you have ideas for making things more usable, contributions/ideas are welcome!

[wqh17101]

• This means it is not possible to reliably show details about an individual task

What does this mean? It will show less data or lead to page crash?