Merge branch 'develop'

.gitignore

@@ -1,6 +1,7 @@
 build/*
 dist/*
 traces_and_results/*
+tflite_tests/*
 __pycache__
 .ipynb_checkpoints
 pyinstaller_command.txt

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) [year] [fullname]
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -4,9 +4,9 @@
 
 ### The goal
 
-GPSClean is an Open Source application developed with the goal of automatically detect and correct errors in GPS traces, using machine learning techniques and without prior geographical knowledge. It is the result of the research undertaken for my thesis project during the Master in Computational Data Science (University of Bolzano). 
+GPSClean is an Open Source application developed with the goal of automatically detect and correct errors in GPS traces, exported in the widely adopted GPX format, using machine learning techniques and without prior geographical knowledge. It is the result of the research undertaken for my thesis project during the Master in Computational Data Science (University of Bolzano). 
 
-Feel free to try the application and report any feedback here on Gitlab. 
+Feel free to try the application and report any feedback on the project's [Github page](https://github.com/sbettid/GPSClean). 
 
 ### How does it work?
 
@@ -60,18 +60,17 @@ The corrected version of the trace is then exported in the GPX format.
 
 In order to run the application, please follow the subsequent steps: 
 
-1) Clone the repository and open a shell in the main folder
-2) Execute `pip install -r requirements.txt` to install the required libraries
-3) Now you are ready to clean your first GPS trace by executing the following command: `python gpsclean.py path/to/your_trace.gpx`
-4) In the same folder where the original trace resides, you will find a cleaned version of the trace with the suffix "_cleaned"
+1) Install the package by executing the following command: `pip install -i https://test.pypi.org/simple/ gpsclean`
+2) Now you are ready to clean your first GPS trace by executing the following command: `gpsclean path/to/your_trace.gpx`
+3) In the same folder where the original trace resides, you will find a cleaned version of the trace with the suffix "_cleaned"
 
 ## Downloading a pre-built all-in-one executable
 
-On the other hand, if you prefer to download a all-in-one executable which does not require any prior software installed (not even Python), you can find pre-built all-in-one packages in the [Release section](https://github.com/sbettid/GPSClean/releases) of this repository. 
+On the other hand, if you prefer to download a all-in-one executable which does not require any prior software installed (not even Python), you can find pre-built all-in-one packages in the [Release section](https://github.com/sbettid/GPSClean/releases) of the repository. 
 
 
 ## How to manually create the all-in-one executable
 
-1) Clone the repository and open a shell in the main folder
-2) Execute the following command: `pyinstaller -F --add-data "model_42t_traces.h5;." gpsclean.py`
-3) At the end of the process, a dist folder can be found inside the main folder, containing the packaged executable. 
+1) Clone the [repository](https://github.com/sbettid/GPSClean) and open a shell in the `src/gpsclean` folder
+2) Execute the following command: `pyinstaller -F --add-data "data/model_42t_traces.h5;data" gpsclean.py`
+3) At the end of the process, a dist folder can be found inside the `src/gpsclean` folder, containing the packaged executable. 

pyproject.toml

@@ -0,0 +1,6 @@
+[build-system]
+requires = [
+    "setuptools>=42",
+    "wheel"
+]
+build-backend = "setuptools.build_meta"

setup.py

@@ -0,0 +1,46 @@
+import setuptools
+
+with open("README.md", "r", encoding="utf-8") as fh:
+    long_description = fh.read()
+
+setuptools.setup(
+    name="gpsclean",
+    version="0.2.0",
+    author="Davide Sbetti",
+    author_email="[email protected]",
+    description="An application to correct a GPS trace using machine learning techniques",
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    url="https://github.com/sbettid/GPSClean",
+    project_urls={
+        "Bug Tracker": "https://github.com/sbettid/GPSClean/issues",
+    },
+    classifiers=[
+        "Programming Language :: Python :: 3",
+        "License :: OSI Approved :: MIT License",
+        "Operating System :: OS Independent",
+    ],
+    package_dir={"": "src"},
+    packages=setuptools.find_packages(where="src"),
+    python_requires=">=3.6",
+    install_requires=[
+        "art==5.3",
+        "filterpy==1.4.5",
+        "geojson==2.5.0",
+        "gpxpy==1.4.2",
+        "tensorflow==2.7.0",
+        "pandas==0.25.3",
+        "scipy==1.6.1",
+        "pyproj==3.1.0",
+        "numpy==1.21.4",
+        "pyinstaller==4.5.1",
+    ],
+    package_data={
+         "gpsclean": ["data/*.h5"],
+    },
+    entry_points={
+    'console_scripts': [
+        'gpsclean = gpsclean.gpsclean:main',
+    ],
+},
+)

src/gpsclean.egg-info/PKG-INFO

@@ -0,0 +1,94 @@
+Metadata-Version: 2.1
+Name: gpsclean
+Version: 0.2.0
+Summary: An application to correct a GPS trace using machine learning techniques
+Home-page: https://github.com/sbettid/GPSClean
+Author: Davide Sbetti
+Author-email: [email protected]
+License: UNKNOWN
+Project-URL: Bug Tracker, https://github.com/sbettid/GPSClean/issues
+Platform: UNKNOWN
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+License-File: LICENSE
+
+# GPSClean
+
+## What is GPSClean?
+
+### The goal
+
+GPSClean is an Open Source application developed with the goal of automatically detect and correct errors in GPS traces, exported in the widely adopted GPX format, using machine learning techniques and without prior geographical knowledge. It is the result of the research undertaken for my thesis project during the Master in Computational Data Science (University of Bolzano). 
+
+Feel free to try the application and report any feedback on the project's [Github page](https://github.com/sbettid/GPSClean). 
+
+### How does it work?
+
+The activity undertaken by GPSClean can be divided in three main steps: 
+
+- Data Preprocessing
+- Error detection
+- Error correction
+
+The following sections briefly explain the three activities, for further details I invite you to consult the jupyter notebooks available at the [Transform and Model](https://gitlab.inf.unibz.it/gps-clean/transform-and-model) repository. 
+
+### Data preprocessing
+
+The first operations performed by GPSClean are related to data pre-processing. In particular, the machine learning model adopted by the application considers the differences (deltas) between the registered points, since it does not use any geographical knowledge abou the region in which the trace was registered. 
+
+For this reason, the latitude, longitude and altitude original coordinates are converted to ECEF (Earth Centered Earth Fixed) coordinates, enabling so a simpler calculation of the deltas given their cartesian nature. 
+
+The second pre-processing step is the calculation of the deltas between two consecutive points, with the result being the input that will be provided to the machine learning model. 
+
+### Error detection
+
+The error detection step is based on a previously trained machine learning model. The model, based on a neural network composed by Bidirectional LSTM (Long-Short Term Memory) cells, was trained on a set of annotated GPS traces, namely traces on which the errors were explicitly marked using an ad-hoc developed web-based tool: [Track annotation](https://api.dawnets.unibz.it/). 
+
+The considered errors are the following: 
+
+- Pauses: when the selected movements were a pause and are the result of GPS reception errors.
+- Outliers: when the point, due to a reception error, is in an incorrect position.
+- Indoor: a point collected while being indoor in a building.
+
+The result of the application of the model is an annotated trace, used then in the following step. 
+
+### Error correction
+
+#### A special type of error: pauses
+
+Firstly, it should be noted how one of the considered error categories, namely pauses, differs fromthe others, when it comes to correction. In fact, points representing pauses should appear, in the ideal case, stacked one upon the other, given that the position of the receiver did not change in the considered time frame.  However, often pauses are represented, in the GPS trace, by a cloud of points resulting from a measurement error bythe GPS sensor. For the aforementioned reasons, points identified by the model as belonging to a pause can bedirectly removed, resulting so in a more smoothed trace with no unnecessary points
+
+#### Correcting misplaced points
+
+Pauses, as explained in the previous section, can be directly removed, step that can not be performed on misplaced points, which need then to be corrected.
+
+In order to obtain the most appropriate error correction, various techniques were investigated during the thesis research. The selected approach was denoted Bidirectional Separate Kalman Filters, a variation of the widely adopted Kalman Filter. 
+
+A Kalman Filter is a structure which predicts the following observation (in our case position) in time series and joins it together with the observed value to obtain the final observation. The predicted value of the next observation, represented by a state vector, is based on the covariance and relation of the different components, combined with an external influence and uncertainty. The values for the covariance and uncertainty were derived from previous researches in the field. 
+
+The proposed variation applies two separate Kalman Filters for each detected error area, one in each direction, and takes the mean of the resulting corrected positions as the final ones, smoothing so the initial corrections exploiting the availability of the entire time series. 
+
+The corrected version of the trace is then exported in the GPX format. 
+
+## Running the application
+
+In order to run the application, please follow the subsequent steps: 
+
+1) Install the package by executing the following command: `pip install -i https://test.pypi.org/simple/ gpsclean`
+2) Now you are ready to clean your first GPS trace by executing the following command: `gpsclean path/to/your_trace.gpx`
+3) In the same folder where the original trace resides, you will find a cleaned version of the trace with the suffix "_cleaned"
+
+## Downloading a pre-built all-in-one executable
+
+On the other hand, if you prefer to download a all-in-one executable which does not require any prior software installed (not even Python), you can find pre-built all-in-one packages in the [Release section](https://github.com/sbettid/GPSClean/releases) of the repository. 
+
+
+## How to manually create the all-in-one executable
+
+1) Clone the [repository](https://github.com/sbettid/GPSClean) and open a shell in the `src/gpsclean` folder
+2) Execute the following command: `pyinstaller -F --add-data "data/model_42t_traces.h5;." gpsclean.py`
+3) At the end of the process, a dist folder can be found inside the `src/gpsclean` folder, containing the packaged executable. 
+