Skip to content

Back End Developer's Guide

Jump to bottom Edit New page
Scott Kleinman edited this page May 3, 2018 · 8 revisions

This guide gives Lexos coders an understanding of some of the Lexos Back-End code.

Introduction to the Back End

Lexos was initially developed in Python 2.7, and, as of version 3.2 has switched to Python 3.x. It uses the Flask web framework. Flask sets up a local server and handles processing tasks, including the rendering of HTML templates and the sending and receiving of http requests to and from the client.

Switching between Versions of Lexos

It is possible to run both Lexos 3.1 and Lexos 3.2 on the same machine. How you do it depends on which version of Anaconda you have downloaded. If you have downloaded the Python 2.7 version of Anaconda and wish to run Lexos 3.2, you will need to perform the following steps:

  1. Switching to the Python 3 environment.

$ conda create -n py36 python=3.6 anaconda

You can use any name you want in place of "py36".

  1. Activate the environment.

**Linux, OS X:** source activate py36

**Windows:** activate py36

To de-activeate the environment so that you can run Lexos 3.2 or greater, use deactivate py36 or source deactivate py36.

You can see what environments you have with conda info --envs. You can see what Python version the active environment is using with python --version.

You should have access to all default Anaconda packages, but you may have to install individual dependencies for the Python 3 environment.

If you are running Anaconda with Python 3.x, the procedure is the same, except that you will create an environment for Python 2.7. Adjust the commands above accordingly.

Note that Lexos creates a cache folder in your /tmp/ directory (~\AppData\Local\Temp\USERNAME in Windows). This may cause errors when you switch between versions of Lexos. Scrubber, for instance, is known to fail. You may need to delete this directory before launching Lexos when switching between versions.

Patching Lexos v.3.1.1

Occasionally it will be necessary to apply patches to the Lexos v3.1.1 release. The procedure is somewhat involved, so the steps are laid out here.

  1. On the GitHub website, take a note of the release message. Then delete the "Lexos v3.1.1" release.
  2. Checkout the 3.1_bugfix branch. Commit changes and push them to GitHub.
  3. On the command line, delete the remote and local tags with
git push --delete origin v3.1.1
git tag --delete v3.1.1
  1. Create a new tag (replacing the commit number shown below with the latest commit):
git tag -a v3.1.1 c3c946f -m "Message here"
git push --tags origin 3.1_bugfix
  1. On GitHub, click the "v3.1.1" tag and then click "Add Release Notes". Add the title "Lexos v3.1.1" and restore the old message, adding any new material. The message should specify that this will be the last release for Python 2.7. Click "Publish Release".

The Flask Session

When Lexos is first run, Flask establishes a session cookie to handle information about the user's workspace. This cookie holds values that can be accessed from both the front end and the back end by means of the session variable. Lexos uses the session cookie for the following purposes:

  • To cache user's options and other information.
  • To send default information to the front end. Lexos defaults can be found in Lexos/lexos/helpers/constants.py.

The session variable functions like a Python dictionary. Hence session["option"] will access the value of the specified option in the session.

The session can be renewed by calling session_function.init().

When a user enters information through front-end form field, this is sent to the back end through the Flask request object. Information in the request object can be accessed in a variety of ways:

  • request.method: returns methods of the request, typically POST or GET;
  • request.form: returns a dict containing the name each form field mapped to its value;
  • request.form.getlist: returns a dict containing the name each form field mapped to multiple values (if there is more than one);
  • request.file: returns a dict containing the id of each file in an HTML file input submission;
  • request.json: returns a json object typically containing the same information as request.form. It is generally sent from an Ajax request.

For a complete set of properties in the Flask request object, see Accessing Request Data in the Flask documentation.

In Lexos 2, most Lexos tools required the user to submit the form, which sent the form data to the back end and then triggered a page refresh with the back-end response. In Lexos 3, some features have been transfered to Ajax functions, which send data to return a response to the front end without the need for a page refresh.

Structure of the Flask App

The main Lexos folder contains a file (currently called lexos_core.py, but that will likely change) that runs the app with the command python lexos_core.py. All other files for the app are in the lexos folder. This contains the file application.py, which sets up the application environment. It also contain the config.cfg, which is used to set configuration settings such as LOCAL_MODE. The lexos folder contains the following subfolders:

  • /helpers: Contains Lexos constants and small general functions
  • /interfaces: Contains the main Python code for each of the Lexos tools
  • /managers: Contains Python code for the large data management features
  • /processors: Contains modular logic for each of the Lexos tools
  • /resources: Contains files used to create rule sets for Scrubber
  • /static: Contains CSS, Javascript, and fonts
  • /templates: Contains the HTML template files for displaying each of the Lexos tools

Lexos uses Flask Blueprints to modularise its funcitons. Blueprints are implemented by importing the interface in application.py as follows:

from lexos.interfaces.base_interface import base_view
from lexos.interfaces.bubble_viz_interface import viz_view
etc.

Each interface is then registered as a Blueprint:

app.register_blueprint(base_view)
app.register_blueprint(viz_view)
etc.

The interface then functions like a miniature app. To work, the interface file (e.g. interfaces/base_interface.py) must have the following elements:

from flask import session, redirect, url_for, render_template, send_file, \
    request, Blueprint
from lexos.helpers import constants as constants
from lexos.managers import utility, session_manager as session_manager

base_view = Blueprint('base', __name__)

Note that resources imported from other folders must be prefixed by lexos.. The Blueprint name defined in the last line is then used in defining Flask routes.

@base_view.route("/", methods=["GET"])
def base():
    # Do something...

Lexos employs a flat structure for its blueprints; all of the templates and static files for the interfaces are kept in the templates and static folders used by the main app, rather than allocating them to separate folders for each interface.

White Space and Unicode Issues

Lexos handles white space and Unicode pretty well, but, in case we encounter further challenges, the following may be useful.

  • [ftfy](https://ftfy.readthedocs.io/en/latest/) converts bad Unicode into good Unicode
  • [textnorm](https://github.com/isawnyu/textnorm) normalises white space and Unicode character encodings. In particular, it may help with this problem.

Illustration of character encoding problem.

Add a custom footer
Clone this wiki locally