-
Notifications
You must be signed in to change notification settings - Fork 3
Requirements
Aleksandr Nedorezov edited this page Aug 8, 2017
·
37 revisions
Functional requirements do not affect on architecture decisions directly, but provide an evidence of what software will do and a baseline for thinking about quality attributes. For LabelThem we have decided to use a lightweight way to capture functional requirements, namely by capturing shall statements. Later, from those shall statements, we have derived a list of functional responsibilities, in order to distribute them to systems components during design iterations.
| Shall Statement | Functional Responsibility |
|---|---|
| The system shall present users with an image to be labeled | 1. Present an image from the specified source |
| The system shall allow users to draw polygons on the image | 2. Handle clicks on polygon tool button 3. Handle сlicks on the image 4. Save points coordinates 5. Display / Invalidate graphical representation of the markup according to the coordinates of currently saved points |
| The system shall allow users to assign a class to the labelled object | 6. Receive data from a remote server 7. Parse JSON with classes 8. Display classes specified for the job 9. Handle users selection of a class 10. Save users selection of a class |
| The system shall allow users to modify points of the labelled object | 11. Handle clicks on saved points over the image 12. Delete saved points 13. Save points coordinates 14. Invalidate graphical representation of the markup according to the coordinates of currently saved points |
| The system shall allow users to add parameters to the labelled object | 15. Receive data from a remote server 16. Parse JSON with parameters 17. Display parameters specified for the job 18. Handle users selection of parameters 19. Save users selection of parameters |
| The system shall allow user to configure the location of a dataset | 20. Configure dataset location |
| The system shall be able to use images from a remote dataset by accepting URL's as images sources | 21. Receive data from a remote server 22. Present an image from a remote source |
| The system shall allow users to specify classes and parameters for a job | 23. Specify classes / parameters for a job 24. Save data on the remote server |
| The system shall allow users to modify the scale of the image | 25. Handle clicks on increase / decrease image scale buttons 26. Increase / decrease images scale 27. Invalidate graphical representation of the markup according to the current scale of the image |
| The system shall allow users to select labeled object to modify / delete | 28. Handle the selection of the labelled object |
| The system shall allow users to modify class of the labeled object | 29. Delete saved users selection of a class for the labelled object 30. Handle users selection of a class 31. Save users selection of a class |
| The system shall allow users to modify parameters of the labeled object | 32. Delete saved users selection of parameters for the labelled object 33. Handle users selection of parameters 34. Save users selection of parameters |
| The system shall allow users to delete selected labeled object | 35. Delete saved points 36. Invalidate graphical representation of the markup according to the coordinates of currently saved points 37. Delete saved users selection of a class for the labelled object 38. Delete saved users selection of parameters for the labelled object |
| The system shall allow users to export markup as JSON | 39. Generate a JSON with currently saved markup data (points, classes, parameters for all objects labelled on the image, markup quality, and identification data of the user who performed image markup) 40. Handle requests to GET markup data from the system |
| The system shall allow user to modify brightness of the image | 41. Handle clicks on increase / decrease image brightness buttons 42. Increase / Decrease brightness of the image |
| The system shall support integration with third-party classificators | 43. Convert marked-up image into third-party classificator-receivable format 44. Send data to the third-party classificator 45. Receive data from the third-party classificator 46. Convert data received from the classificator to a format supported by the system 47. Save recommended classes 48. Display selection of the class suggested by a third-party classificator |
| The system shall allow users to markup images by contours (lines) recommendation | 49. Convert an image into recommendation method-receivable format 50. Execute contours recommendation 51. Convert recommendation data to a format supported by the system 52. Save points coordinates 53. Invalidate graphical representation of the markup according to the coordinates of currently saved points |
| The system shall run multiple jobs on one instance of itself | 54. Select / add / remove a job |
| The system shall allow users to markup areas on the image by scribbling on them | 55. Handle scribbling over the image 56. Convert scribbled area into a polygon 57. Save points coordinates 58. Invalidate graphical representation of the markup according to the coordinates of currently saved points |
| The system shall check the quality of images markup | 59. Convert marked-up image into quality checker-receivable format 60. Check the quality of the markup 61. Save information about the quality of the image |
| The system shall support user authorization | 62. Register a user in the system 63. Log in a user in the system 64. Log out the user from the system |
| The system shall allow users to undo previous points addition / modification | 65. Handle clicks on the undo button 66. Save actions history 67. Display actions history 68. Handle clicks on the records in the actions history 69. Delete saved points 70. Invalidate graphical representation of the markup according to the coordinates of currently saved points |
After defining the scope of the system in terms of functional requirements, we can think about quality attributes, which will shape the architecture.
For quality attributes, we are going to use ISO 25010 Quality Model as a basis for defining quality attributes. As LabelThem project goals include minimization of cost of dataset preparation, increase markup quality, providing quick on demand dataset preparation, and providing a universal (configurable for different jobs) tool for images markup, we will focus on the following quality attributes and sub-characteristics during the design and development of our application:
- Usability (Operability, Learnability, Accessibility, User Error Protection)
- Performance Efficiency (Time Behaviour)
- Reliability (Fault Tolerance)
- Maintainability (Testability, Modifiability, Reusability, Analyzability)
- Functional Suitability (Functional Correctness)
- Performance Efficiency (Resource Utilization)
- Portability (Installability)
- Security (Authenticity, Confidentiality)
- Compatibility (Interoperability)
In order to have traceability between our business goals and quality attributes, we analyzed the business goals of LabelThem project, transferring them to engineering objectives and mapping them on quality attributes. For each quality attribute we came up with quality attribute scenarios, prioritized them (first value is a business priority – how important this goal is from a business perspective, and the second one is a development priority – how critical it is to implement this engineering objective in relation to the whole system). After that, we came up with architecture tactics (Bass, L., Clements, P. & Kazman, R., 2012. Software Architecture in Practice, Addison-Wesley, p.89), which are techniques that an architect can use to achieve the required quality attributes.
The resulting table is presented below:
| Business Goal | Engineering Objective | Quality Attribute | Quality Attribute Scenario | Priority | Tactics |
|---|---|---|---|---|---|
| Reduce cost of dataset preparation | Minimize time to markup an image | Operability | Markup process must take less than 5 minutes | (Must, Low) | Maintain task model |
| User Error Protection | A user can undo addition of the previous markup point in less than 5 seconds | (Would, Medium) | Mantain records of previous markup actions | ||
| Time Behaviour | Systems webpage on which images can be labelled should opened on local host in less than 1 second | (Should, Low) | Improve resource efficiency | ||
| Learnability | A user should understand the basic workflow of the system after acquainting with a user tutorial for not more than 15 minutes | (Must, Low) | Maintain task model | ||
| Minimize system setup time | Installability | Procedure of the system installation should take less than 60 minutes. Ideally should be improved to less than 10 minutes | (Must, Low) | Minimize Platform Dependencies | |
| Ensure that system do not lose markup data | Fault Tolerance | If the backend crashes because of the arriving events overload, this fault should be detected in less than 60 seconds | (Should, Medium) | Ping/echo | |
| In case of detection of the fault of backend crashing because of the arriving events overload, a notification to a user should appear in less than 10 seconds after the fault was detected | (Should, Medium) | Exception handling | |||
| The front-end of the system should not crash in case of the fault of backend crashing because of the arriving events overload | (Should, High) | Removal from service | |||
| Replace labor with automation by supporting integration with third-party classificators | Interoperability | The system should return JSON values for all requests from third-party classificators | (Would, Low) | Tailor Interface | |
| Increase markup quality | Support markup quality control | Functional Correctness | Saved coordinates of markup points should not differ from the ones pointed out by a user more than on 5 pixels | (Would, Low) | - |
| Authenticity | All markup results should contain identification data of the user who performed image markup | (Would, Low) | Identify Actors | ||
| Provide a universal (configurable for different jobs) tool for images markup | Support configuration of datasets, jobs parameters and classes | Modifiability | A user should be able to configure classes or parameters for the job without introducing new defects to the system | (Would, Medium) | Encapsulate |
| A user should be able to configure classes or parameters for the job by editing data in no more than 1 file | (Would, Medium) | ||||
| Improve Innosoft company's reputation | Increase open-source community involvement in project development | Testability | Statement coverage of 80% is achieved | (Should, Low) | Abstract data sources |
| Limit structural complexity | |||||
| Modularity | Modification of available tools or data collection workflow will affect no more than 4 modules | (Would, High) | Split module | ||
| Increase semantic coherence | |||||
| Use an intermediary (Publish-subscribe) | |||||
| Abstract common services | |||||
| Defer binding (Publish-subscribe) |
- The system should be implemented as a web-service
- JavaScript should be used on the frontend
- Python or/and JavaScript should be used on the backend
- The system should work in Chromium Browser (ver. 55+)
- Systems users are supposed to have a computer mouse with at least two buttons and a "wheel"
- The system should be implemented as a Yandex.Toloka-hosted and integrated system
//: # (The system should support integration with the Yandex.Toloka crowdsourcing system
- The system should be released before August 9, 2017