You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After reviewing the DISCO website as a new user, here is a list of things that can be fixed and improved. This issue goes in pair with #627 which addresses higher-level concerns with the website. Some changes made after #627 may render some items in this issue obsolete and both issues should be considered at the same time.
Overall
Fixes
On Safari, hovering over (any) buttons seems makes the button border bug (I noticed it does bug if I approach the cursor quickly). On the image below I'm hovering on the Next button. Works well on Firefox.
On Safari, hovering over (any) paragraphs creates a square border while the paragraphs have rounded corners (pay attention to the corners on the image. Works well on Firefox.
Hovering over the top-left disco logo creates an empty text bubble.
When hovering:
Not hovering:
Features
Make sure all links open in new tab, especially links to download datasets
The website remembers at which step you were but sometimes that should be reset. An example would be clearer: Going to Tasks, Joining the Titanic task and clicking next until the Train your Model step gets you to a particular step. Now if you use the sidebar to go back to the Tasks page, and click again on Joining the Titanic task brings you back to the Train your Model step and not the first step (Task Description). This is great but it should not always behave this way. For example, download two pre-trained models from the Evaluate page, let's say Titanic and MNIST. From the model library click on Titanic to Test it, that leads you to the Connect Your Data step. Now from the homepage click on Evaluate. That directly leads you to Connect Your Data step, implying that you are currently testing Titanic while the expected behavior is to let the user choose which model they want to evaluate (since we clicked on the noon-specific Evaluate button).
I think the Previous and Next buttons should be at the bottom of pages rather than top, to follow the user flow who fills information or reads info from top to bottom. Right now users need to scroll up after every page to get to the next step.
Homepage
Fixes
update GIF and logo quality, it is pixelated
Features
After clicking on Getting Started, one can hover on the word DISCOllaborative which displays an explanation. This is useful, but it is not clear this is possible to do. Either write the text directly on the website this is essential or at least underline the word to make it clear and make the info appear faster.
It is not clear at all that users can download pretrained models after clicking on the Evaluate button. Add it to the button description, e.g. "Evaluate your DISCO-trained model or download a pre-trained model."
Tasks
Fixes
filter by training scheme should not allow ticking both decentralized and federated filters at the same time. Since it is either one or the other filter, it should untick the previous filter automatically. Same for data types.
When the window is too narrow the “PREVIOUS” and “NEXT” buttons disappear making impossible to go to the next steps
Features
Clicking on Join button (e.g. for titanic task) should get you to the top of the next page, right now it's already scrolled down
After clicking on Join, “The Task” should be replaced by the task name
More dataset text description, add metadata
Add explanations for training parameters (on hover for example)
uploading images successfully is not explicit enough, there should be a loading indicator and “Select Files” should change to “Add more files”
Rather than clicking “Next” there should be a data upload validation button (e.g. "Submit") that takes the user to the next page (Training)
Data Format and Example Data accordion paragraphs should be toggled by clicking on the whole row (rather than only the arrow icon) and I think they should be opened by default.
Uploading Data should specify that the Data stays local and is only used for local training. Data isn't shared or uploaded to a server. Make users drag and drop their confidential data into a website is fishy. I don't see alternatives since browser can't access file systems.
Prevent users from clicking next without uploading images first. Right now, it only triggers an error at the next steps when trying to train a model.
Nothing prevents users from uploading a whole folder rather the individual images, there should be additional checks on what is uploaded.
NEXT and PREVIOUS buttons should be at the bottom of the page rather than the top
For the successive steps, color the horizontal bars in dark blue as the user progresses through the tasks and make the current active step different (e.g. bigger circle) than the previous ones
What does it mean to “Train alone” or “Train collaboratively”? It is not clear for a new user that doesn't know how disco works. Choosing either of them doesn't seem to make a difference from a user point of view
COVID LUS
Model description should include a performance comparison rather than only stating deepchest performance.
Not sure if specifying the model simplification is useful
Aggregation parameter has an empty value
We need to pre-process the dataset before uploading it? Actually not, the filename format is not clear (”patientId_*.png”)
MNIST
How to upload the data is not clear and explicit enough. We need to 1) click on the link in the previous page, 2) sign in / sign up to kaggle 3) click on download button 4) unzip archive 5) click on upload by group ) tediously upload each image group individually 6) should we upload trainingSample, trainngSet, testSample or testSet?
Ideally the dataset should already be available on disco. But only “Select labels by” should already put on GROUP rather than CSV, since the default dataset if by group.
Typo in data format: “The model taskes images of size 28x28 as input.”
TITANIC
In Example Data: “On the testing & validation page,” was does it mean?
improve text description and add link to the dataset at least
need more text information in Data format and example data.
SIMPLE FACE
Improve text description
“Simple face is a small subset of face_task” which subset?
GeoTag task removed
No mention of where to find the dataset
no example image either
CIFAR-10
No link to the dataset
Model library
Fixes
Model items do not always have the same width:
When turning off the browser model library, trying to download a trained model doesn't trigger any error/feedback, it should state that a user can't download models if the model library is turned off
Features
When the model Library is empty, it should state that users can download trained model or need to train models and link where to do so (currently users can download pretrained models in the "evaluate" page)
When deleting a model, either ask users to confirm or create an undo button. It is dangerous to be able to misclick and delete a model inadvertently.
Add hover explanations over model icons, what does the university hat mean? From the code it "loads" the model. The icon is not clear enough and there should explanations on why and when should a user load a model (what's the difference between a model saved in the library and a model loaded?). Loading an older model (e.g. from 10mins ago) duplicates it. Why this happens isn't clear.
Order models by tasks and within tasks from most recent to oldest.
Because there are some icon buttons, it is not clear at all that we can actually click on the model to Test it (on the image below, we can click anywhere in the rectangle to evaluate the model, e.g we can click on the model name). Buttons have to be coherent and use on icon buttons for example. Create an "evaluate" button rather than clicking on the rectangle. Moreover, clicking on the rectangle lets you "Test" the model while we can also "Predict" with the model. The fact that the rectangle leads to the Test rather than Predict is arbitrary and implicit.
Graphic design is not coherent with rest of the website, the blue color is a darker shade and the line thickness is thinner.
The setting to Use the model library should be accessible from the library rather than the Setting sidebar tab
Evaluate
Features
The Evaluate page should also be accessible from the sidebar menu since one can download pretrained models.
The Evaluate landing page should explain the difference between Test (Data with labels) and Predict (data without labels)
Add a loading indicator when downloading a model and a toaster message for successful download (or error)
The Evaluation page at the "Connect your data" and "Evaluate your model" should specify if we are currently Testing or Predicting
Info
Fixes
At the "Tutorial" step, bottom buttons don't have the usual graphic design, needs to have the blue background and hover effect.
Very cool graphics in the info tab, however the legends are too small to be readable
Features
At the "Fundamentals" step, remove the dashes and redact actual sentences
"Further" information tab can have better formatting and text design, hyperlink design
Settings
Features
Toggle model library should be moved to Model library
The toggle button doesn't have the usual graphic design, it is in black with double lined border when clicked on
About us
Features
About Us menu should have another icon (currently it looks like a profile icon) and the MLO can have a higher quality logo (or smaller size)
Improve the text design.
"About Us" title should have a title font and size
Make icons clickable to point to the respective links
Custom training
Fixes
Error message too technical when entering the wrong format. There should be more user-friendly type checks
"Byzantine-Robust Aggregation" for Federated learning and "Secure Aggregation" for Decentralized learning tick boxes shouldn't be centered
For Federated:
For Decentralized:
Choosing Decentralized of Federated displays different Privacy parameters (Secure aggregation for Decentralized, Byzantine aggregation for Federated). However, switching between the two doesn't always update parameters correctly. For example, choosing Decentralized and ticking Secure Aggregation displays the parameter "Maximum Value of Shares used in Secure Aggregation". Now if we change to Federated, it doesn't remove the "Maximum Value of Shares" parameter. Similarly the federated parameter "Percentile of Trusted Users for Byzantine-Robust Aggregation" isn't removed when changing to Decentralized.
In the Training Parameters, clicking on Add Element makes the button border bug:
Right after clicking on Add Element:
What it should look like:
Can't drag and drop files (and there is no hover effect). Uploading files doesn't give any user feedback, users can't know if a file is successfully uploaded). Can't clear files either without Resetting everything.
“Request help on slack” button doesn't do anything
Resetting doesn't empty “model train data” and “list of labels” list
Features
Overall, there should be more information accessible on parameters, what they do and what the expected format is (float, percentage, integer, string etc). For example, have a question mark icon next to each parameter to display more information on hover.
Give more information on what is needed to create a task and how to create an initial model, e.g. why can't we start without any model files? Not enough explanations on how to create model files. Create a Markdown doc with an example of how to create an initial model and link it on the website.
Improve UI design, smaller red cross, rename “trainingParameter” to "Training Parameter" etc
Give more explanations on how the collaboration works, how can other people join the custom task after it's created?
Selecting multiple metrics in the Metrics list menu isn't intuitive (need to use ctrl + click), clicking should toggle selection.
When creating a Decentralized task, what does “Maximum Value of Shares used in Secure Aggregation” mean in the Privacy Parameters?
The "secure" icon below file upload needs to give more information on hover or when being clicked.
submitting with empty fields should trigger an error alert with all the missing fields, or display over the Submit button, or scroll up to the first missing field
Need more explanation on model files, e.g. why can't we start without any model files? How can we create model files?
The text was updated successfully, but these errors were encountered:
After reviewing the DISCO website as a new user, here is a list of things that can be fixed and improved. This issue goes in pair with #627 which addresses higher-level concerns with the website. Some changes made after #627 may render some items in this issue obsolete and both issues should be considered at the same time.
Overall
Fixes
When hovering:
Not hovering:
Features
Homepage
Fixes
Features
Tasks
Fixes
Features
COVID LUS
We need to pre-process the dataset before uploading it?Actually not, the filename format is not clear (”patientId_*.png”)MNIST
Ideally the dataset should already be available on disco. But only “Select labels by” should already put on GROUP rather than CSV, since the default dataset if by group.
TITANIC
SKIN DISEASE
SIMPLE FACE
GeoTagtask removedCIFAR-10
Model library
Fixes
Features
Evaluate
Features
Info
Fixes
Features
Settings
Features
About us
Features
Custom training
Fixes
Error message too technical when entering the wrong format. There should be more user-friendly type checks
"Byzantine-Robust Aggregation" for Federated learning and "Secure Aggregation" for Decentralized learning tick boxes shouldn't be centered
For Federated:
For Decentralized:
Choosing Decentralized of Federated displays different Privacy parameters (Secure aggregation for Decentralized, Byzantine aggregation for Federated). However, switching between the two doesn't always update parameters correctly. For example, choosing Decentralized and ticking Secure Aggregation displays the parameter "Maximum Value of Shares used in Secure Aggregation". Now if we change to Federated, it doesn't remove the "Maximum Value of Shares" parameter. Similarly the federated parameter "Percentile of Trusted Users for Byzantine-Robust Aggregation" isn't removed when changing to Decentralized.
In the Training Parameters, clicking on Add Element makes the button border bug:
Right after clicking on Add Element:
What it should look like:
Can't drag and drop files (and there is no hover effect). Uploading files doesn't give any user feedback, users can't know if a file is successfully uploaded). Can't clear files either without Resetting everything.
“Request help on slack” button doesn't do anything
Resetting doesn't empty “model train data” and “list of labels” list
Features
The text was updated successfully, but these errors were encountered: