Merged upstream changes into current style_black branch. Includes ver…

…sion 3 overhaul changes.

.github/workflows/python-app.yml

@@ -0,0 +1,40 @@
+# This workflow will install Python dependencies, run tests and lint with a single version of Python
+# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
+
+name: Python application
+
+on:
+  push:
+    branches: [ "main" ]
+  pull_request:
+    branches: [ "main" ]
+
+permissions:
+  contents: read
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v3
+    - name: Set up Python 3.8
+      uses: actions/setup-python@v3
+      with:
+        python-version: "3.8"
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install flake8 pytest
+        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+    - name: Lint with flake8
+      run: |
+        # stop the build if there are Python syntax errors or undefined names
+        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
+        # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
+        flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
+    - name: Test with pytest
+      working-directory: tests/
+      run: |
+        python -m pytest --import-mode=append main_tests.py

.gitignore

@@ -1,5 +1,7 @@
 /notes.txt
+/temp.py
 /debug.log
+*/debug.log
 /stats.json
 /stats.json.backup
 /global_prefs_override.xml
@@ -8,3 +10,10 @@
 /venv/
 /.git-blame-ignore-revs
 scratch.py
+/user_config.py
+/tests/__pycache__
+/__pycache__
+/bin
+/lib
+/lib64
+pyvenv.cfg

CHANGELOG

@@ -1,3 +1,7 @@
+Version 3.0 release date Sep 16, 2023
+- New config file format, be sure to make a new config when upgrading to this version. Suggested to make a fresh install dir as well, as FTM's old stats file for keeping track of things may not migrate gracefully.
+- Major stability fixes, particularly around communication with the BOINC client
+- Added testing to most code functions to more readily accept pull requests and find bugs
 Version 2.0 release date Jan 30, 2023
 - FTM can now directly control BOINC to match assigned project weights, calculate profitability, and crunch based on profitability or room temperature
 Version .1 Release Date: Nov 2021

dev_notes.txt

@@ -1,3 +1,13 @@
+## URL formats
+There are several URL formats FTM uses.
+- Generally, all URLs should be in the "canonicalized" format created by resolve_url_database(). The canonical format is
+ UPPERCASE, with HTTP/HTTPS/WWW. removed and any trailing slashes removed.
+- The URL canonicalizer will also join projects which have multiple attach URLs for the same project.
+- We use non-canonical URLs when speaking with Gridcoin or BOINC RPC since they have their own formats. BOINC is
+particularly picky about capitalization. When we fetch data from these tools, we convert it into canonical format as
+soon as possible (sometimes in the fetch function itself). Use the URL converters resolve_url_boinc_rpc to turn URLs
+into URLs BOINC will like.
+
 # DATABASE is used to track various counters etc during the program's operation. It is saved at opportune times
 # in the program's execution including when user issues a ctrl + c. It should only be used for variables which are
 # needed across executions or across functions
@@ -7,10 +17,10 @@
 DATABASE format:
 'GRCPRICELASTCHECKED':datetime.datetime # last time we checked GRC's price
 'GRCPRICE':float # last known average price of GRC
-'DEVTIMECOUNTER':float # counter for # of minutes we have managed BOINC crunching and it has been actively crunching, gets lowered by crunching under dev account
-'DEVTIMETOTAL': float # total amount of time crunched for dev
-'FTMTOTAL': float # total amount of time crunched for non-dev
+'DEVTIMETOTAL': float # total amount of minutes crunched for dev
+'FTMTOTAL': float # total amount of minutes crunched for non-dev
 'LASTUPDATECHECK':datetime.datetime # last time we checked for updates from github
+'MAGLASTCHECKED': datetime.datetime # last time we queried mag amounts
 'STATSLASTCALCULATED':datetime.datetime # last time we calculated stats
 'LASTGRIDCOINSTATSPROJECTCHECK':datetime.datetime, # last time we checked gridcoinstats for projects
 'GSPROJECTLIST':List[str] # cached project list from gridcoinstats

libs/pyboinc/_raw_client.py

@@ -11,15 +11,14 @@
 import asyncio
 from socket import AF_INET
 import xml.etree.ElementTree as ET
-
+import sys
 
 GUI_RPC_DEFAULT_PORT = 31416
 REPLY_TAG = "boinc_gui_rpc_reply"
 REQUEST_TAG = "boinc_gui_rpc_request"
 END_OF_MESSAGE = b"\x03"
 BOINC_ENCODING = "ISO-8859-1"
 
-
 class _RPCClientRaw:
     """
     Connects to the RPC server and transports whichever request and reply to and from it