Skip to content

The Lexos 4 Frontend

Jump to bottom Edit New page
AndrewS-GH edited this page Feb 24, 2021 · 9 revisions

The Lexos 4 Frontend

The following guide will familiarize you with the codebase of the Lexos 4 frontend and the development practices that should be followed when working with it.

Index

General

To keep the codebase consistent, there are a few general rules to follow. Use snake_case for JS and kebab-case for HTML and CSS. Prefer to use verbose names rather than terse ones; prefer to not abbreviate words. Similarly, write in complete sentences for JS comments and documentation. Use ESLint configured with the .eslintrc.json file so that your JS code conforms to standard syntax and documentation conventions.

File Structure

The frontend code is located in lexos/frontend. This contains three important subfolders: html, css, and js. Within each of these folders is a file for each webpage and a helpers subfolder which contains any code that is shared between multiple pages or otherwise benefits from being separated from the page-specific files.

The separation between HTML, CSS, and JS is necessary to keep the codebase organized and maintainable. If someone wants to change CSS or JS, they will expect to look in the css or js folder. Do not embed JS or CSS in the HTML files and only include HTML and CSS in JS files when it is necessary for dynamic content.

Each page has its own HTML, CSS, and JS file which should contain only code specific to that page. No code should be duplicated in multiple files. Instead, put the duplicate code in an appropriately named file in the helpers folder and include that file in each page that needs it.

HTML

HTML files define the elements that are initially loaded on a page.

Lexos uses Jinja in its HTML files. The main purpose for using Jinja is to allow HTML to be reused, reducing code duplication. As mentioned previously, HTML that is used by multiple files is in the helpers folder. These are then included in the files that need it using Jinja. Currently, there are three helpers or "templates":

base-template.html is the base of all other pages and templates. It provides the elements which are common to all pages such as the navbar, the footer, and essential JS and CSS includes.

visualize-template.html is the base of many visualize pages (Word Cloud, Multicloud, Bubbleviz). It inherits from the base template. Additionally it includes D3 and JS and CSS common to visualize pages as well as a graph section with options for font, term count, color, and PNG, SVG, and fullscreen buttons.

analyze-template.html is similarly the base of many analyze pages (Dendrogram, K-Means, Consensus Tree, Similarity Query, and Top Words). It inherits from the base template. Additionally it includes Plotly and JS and CSS common to analyze pages as well as an analysis options section with sections for tokenizing, normalizing, and culling.

Another use for Jinja is that it facilitates passing variables from the backend to HTML directly. One example of this is the active document count which appears at the bottom of the page. For the specifics on these features, consult the Jinja documentation.

CSS

CSS files are for modifying the layout and appearance of elements on the page.

Lexos heavily uses CSS grid and flexbox for layout. Grid and flexbox make creating dynamic layouts easy. Use grid for two-dimensional layouts and flexbox for one-dimensional layouts. These should be your go-to for centering items, creating the structure of a page, creating tables or lists, and for layout in general.

Use CSS variables for values that are going to be used multiple times. Define the variables in the root element. CSS variables make it easy to change values. With them, rather than going through every CSS file that contained the value and changing it, the value only needs to be changed in the variable definition.

Familiarize yourself with grid, flexbox, and variables before working on Lexos' CSS.

Modify an HTML tag's style directly when you want a style to apply to all elements of that type. Use classes for styles that you want to apply to multiple elements. Use IDs for unique styles that only appear once per page. Make sure your class and ID names are similar to the ones that exist in the codebase.

The CSS files in the helpers folder are either shared between multiple files or otherwise benefit from being separated out from page CSS. There are some files of special note here. common.css defines styles for tags and common elements such as buttons and tooltips, classes.css defines general-purpose classes, fonts.css defines the fonts, and variables.css defines general-purpose variables. All three of these are included in base-template.html, making them available throughout the website. The rest of the CSS files should be self-explanitory in their name. Note that some CSS helper files are for dynamic content created with JS rather than content in the HTML files.

JavaScript

JS files define the interactivity of a webpage and its elements.

Lexos heavily uses jQuery Core. Essentially, jQuery is a shorthand/syntax sugar library for JS. Generally jQuery results in more readable and maintainable code, so use of jQuery should be preferred over the equivalent plain JS except in situations where you are certain it introduces a perceivable performance bottleneck.

Familiarize yourself with the basics of jQuery Core before working on Lexos JS.

Like the HTML and CSS helpers, it is also a good idea to become familiar with the files in the JS helpers folder so that you understand how the website works and so can use them in your own JS code where appropriate. If there is already a JS file that does what you need or is close to what you need, you should use the functionality in that file or try to expand it to fit your needs rather then rewriting something similar. Below is a brief overview of the JS helper files:

Helpers Related To HTML Templates

  • base.js - The JS related to base-template.html. Contains functions for the navbar and footer.

  • popup.js - Used in base-template.html. Contains functions for creating a popup. This is used by many elements on the site.

  • utility.js - Used in base-template.html. Contains various general-purpose utility functions such as getting the mouse position, getting the size of elements, and sending AJAX requests.

  • visualize-template.js - The JS related to visualize-template.html. Contains functions for the visualize settings, including color changing and SVG and PNG download generation.

  • analyze-template.js - The JS related to analyze-template.html. Contains functions for the analyze settings, including its tooltips and input validation.

Helpers For Functionality That Is Used On Multiple Pages

  • document-previews.js - Used in cut.html and scrub.html. Contains functions for previewing changes made to the active documents.

  • graph.js - Used in dendrogram.html, k-means.html, rolling-window.html, and statistics.html. Contains functions for creating Plotly graphs, saving them, and displaying them in fullscreen.

  • table.js - Used in content-analysis.html, manage.html, similarity-query.html, statistics.html, tokenize.html, and top-words.html. Defines a table class that is used on many pages. It has sorting, pagination, download, and searching capabilities.

  • tree-options.js - Used in consensus-tree.html and dendrogram.html. Contains functions for creating a tree options popup that allows a user to select from a variety of distance and linkage methods.

Helpers For Separated Out Functionality Used On A Single Page

  • manage-table.js - Used in manage.html. Contains functions for creating and sorting the manage table and select/deselect all.

  • manage-table-selection-box.js - Used in manage.html. Contains functions for the manage table's selection box.

  • point.js - Used in manage.html. Contains a class representing a 2D point. Used for the manage table selection box.

  • message.js - Used in upload.html. Contains functions for displaying a temporary popup message.

Setting Up A Workspace

Now that you know the basics of the existing frontend codebase and the guidelines that should be followed when adding to it, it's time to set up a workspace so that you can efficiently debug and add to Lexos. This section will cover a workspace based around Visual Studio Code.

Installation

Download and install Visual Studio Code. You can use all of the defaults when installing. Next, download and install Git. During installation, when prompted for the default text editor to use, choose Visual Studio Code. Otherwise, the defaults are fine.

Cloning The Repository

Open VS Code and go to the source control button on the left side to open the Source Control pane (Shortcut: Ctrl+Shift+G). Select "Clone a Repository" and paste the Lexos "Clone with HTTPS" URL in: https://github.com/WheatonCS/Lexos.git. Press Enter and you will be prompted to select the folder to clone into. On Windows, I recommend cloning to C:/Users/your_username/Documents/GitHub. A folder named "Lexos" will be created in the selected directory and it will be populated with the Lexos codebase. A progress popup will appear on the bottom right. When it has finished, you will then be prompted to open the repository. Do this, and you can now browse the Lexos codebase using the VS Code Explorer pane. To open the Explorer pane, click the top left button (Shortcut: Ctrl+Shift+E).

Installing Dependencies

Lexos has many dependencies on other packages and libraries. These are not included in the GitHub repository in order to save space. You need to download and install these dependencies yourself after cloning the repository. To do this, you need first download and install Anaconda (used for Python dependencies) and Node (used for JS dependencies). You can use the default installation options.

First, we will install the Python dependencies. To access Anaconda and the Python dependencies, use the special Anaconda Prompt that was installed. Launch the Anaconda Prompt, navigate the the Lexos directory, and run pip install -r requirements.txt to install the Python dependencies.

Node on the other hand can be used on the terminal of your choice, so after you have installed the Python dependencies, launch a terminal and navigate to the Lexos directory. Then go to the lexos/frontend folder where package.json is located. There, run npm install to install the JS dependencies.

Note that if the dependencies change in the branch you are working in because of another person's commits, you will need to re-run the installation commands.

Linting

Linting is a way to enforce a common code style. Lexos uses ESLint for linting JS code and Flake8 for linting Python code. When you push a commit to the Lexos repository your code will be linted. If your code does not pass the linting tests, it will fail validation, so try to make sure your code has no linting errors before committing. VS Code has a wide range of extensions that you can use to make development easier, including ones that integrate ESLint and Flake8 into VS Code so that you can easily make sure your code passes validation.

First, let's install ESLint. Click the extensions button on the left side to open the Extensions pane (Shortcut: Ctrl+Shift+X), Search for "ESLint", and install it. ESLint will automatically detect the .eslint.json in lexos/frontend and display errors in VS Code for any violated rules. Because this extension depends on the ESLint Node dependency being installed, you may need to restart VS Code for it to initially detect the ESLint Node module you installed in the previous section.

Next, install the official Python extension similarly to how you installed the ESLint extension. After it is installed, click the gear button on the bottom left and select "settings" to open VS Code's settings (Shortcut: Ctrl+,). First, disable the default linter by searching for "pylint" and unchecking "Python > Linting: Enabled". Next, search for "flake8" and make sure the "Python > Linting: Flake8 Enabled" setting is checked. Then add the following argument to the "Python > Linting: Flake8 Args" option: --ignore E121,E123,E126,E226,E24,E704,W503,W504,E741. This disables some errors and warnings we do not check for. Now you will be able to see Flake8 errors in VS Code.

Running The Server Locally

You will likely want to run a local version of the server so that you can see the changes you have made and debug. With VS code it is easy to set this up.

First, open lexos/application.py using the VS Code Explorer pane (again, to open the Explorer pane, click the top left button or press Ctrl+Shift+E). Then open the Run pane (Shortcut: Ctrl+Shift+D), select "create a launch.json file", and then select "Flask" as the debug configuration. When prompted for the path to the application, enter lexos/application.py. You can close the launch.json file that gets opened as it is already configured correctly. Now the Run pane will display a green arrow which you can press to start the server.

Press the green arrow and you will see the server start up in the Terminal pane. When it has finished starting up, you can Ctrl+Click on the link it outputs to open the website (usually http://127.0.0.1:5000/). To stop the server, open the Terminal pane again (Ctrl+`) and press Ctrl+C.

One benefit of launching the server via VS Code is that it has integration which allows you to debug the backend code in the editor. Error handling is done elegantly and you can set breakpoints, view the values of variables, view calling stacks, and step through the code.

For debugging the front end code (JS/HTML/CSS) or requests, use the developer tools of your browser (for Chrome, Ctrl+Shift+I opens the developer tools pane).

Running Unit Tests

Although unit tests are have more to do with the backend, I will include how to set them up here for completeness. Unit tests are a way to make sure modification to the code does not break anything. To pass validation on GitHub, in addition to linting, the code must pass all unit tests. For this reason you may want to run and debug the unit tests locally before committing, and VS Code has a nice interface for this.

To set testing up, open the command palette (Ctrl+Shift+P), type "Python: Discover Tests", and press Enter. A popup will appear on the bottom left saying no test framework has been configured. Press the "Enable and configure a Test Framework" button on the popup and then select "pytest" on the dropdown that appears at the top center. Then enter "test" as the unit test directory and VS Code will begin discovering tests. After it finishes this, a new button will appear on the left bar which looks like a test tube. Click this to open the Test pane. Here, you can click the green arrow to run all unit tests and see their results.

Version Control

Now that you have the workspace set up, you can start contributing. To start modifying the codebase, first you should create a new branch. You can do this by opening the command palette (again Ctrl+Shift+P) and entering "Git: Create Branch". After you enter a name for the branch in the popup, the branch will be created and become active. On the bottom bar of the VS Code window, you will see the active branch switch from "master" to the branch you just created.

After you have made changes, you should commit your work. To do this, open the Source Control pane (again, Ctrl+Shift+G). There, you will see a list of the files you changed. You can click on them to view the differences. If you are happy with the changes, commit by pressing the checkmark. If you get a warning about staging, select "Always" to automatically stage changes. Then enter the commit message and press Enter to commit.

When you want to publish or synchronize changes with the repository, click the button on the bottom bar to the right of to the active branch name. You will be prompted to sign into GitHub the first time you do this. After the syncing is complete, you can then view your changes on the Lexos GitHub page.

Conclusion

Now you know the basics of the Lexos frontend codebase, the practices that should be followed when modifying it, and how to set up and use VS Code to efficiently work on Lexos. Happy contributing!

Add a custom footer
Clone this wiki locally