Skip to content

Releases: UBC-Solar/sunlink

v3.0.0: Post 2024 Comp Sunlink

09 Aug 23:57
@AarjavJain101 AarjavJain101
v3.0.0
a7221a5
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v3.0.0: Post 2024 Comp Sunlink Latest
Latest

What's New

v3.0.0 marks a milestone for sunlink's journey throughout Brightside's race at FSGP and ASC. Specifically, all the changes in this release were created during competition to better serve other Solar members using the application. The changes range from updated documentation to make sunlink more understandable and scripts to make our data more accessible.

Below are the specific changes:

  • Documentation for setting up Tailscale so that members can access the bay computer remotely and more specifically view our production Influx Buckets. See PR #114.
  • Fixing the main README.md to better explain that the setup.sh script already fully sets up sunlink and the rest of the README.md focuses on explaining the working of sunlink at a high level as well as explaining what the script is doing so that manual setup is documented. See PR #115.
  • A small fix to the requirements.txt to change the PyYAML version from 6.0.1 to 6.0.0 because 6.0.1 is not supported for Python 3.12. Note 6.0.0 is supported for 3.12 as well as previous versions. See PR #116.
  • A feature was added to the memorator upload script to support better a CLI. Specifically, instead of users having to go into the code to change the folder path constant for their memorator logs, they will now enter the full file path in the terminal when prompted by the script to do so. See PR #117.
  • Lastly, the INFLUXDB.md document was added to aid users who want to download, upload, or view data on InfluxDB. This addition also came with a script to automatically download a compressed CSV to your computer from the elec bay Influx instance. See PR #119.
    • image

Generated Release Notes:

Full Changelog: v0.2.0...v3.0.0

Contributors

AarjavJain101
Assets 2
Loading

v.2.4.0: Setup Script and Grafana Fixes

07 Jul 07:21
@AarjavJain101 AarjavJain101
v2.4.0
1009aad
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v.2.4.0: Setup Script and Grafana Fixes

What's New

This release focused on getting sunlink ready to distribute to other people in an easy way. This was done by adding further automation and UI improvements to our setup script as well as fixing encoding mistakes on Grafana dashboards.

Below are the specific changes:

  • Refactor: Improved UI and automation in the setup script (#107, #111, #112)
  • Fix: Added an AMB dashboard + fixes to the BMS dashboard for incorrect panel titles as well as removing redundant panels in the pit crew dashboard. Also fixed encoding issues on Grafana dashboards (#105, #113).
  • Feat: To address RAM issues we added a 6ms delay when uploading data from the memorator (#110).
  • Fix: Removed the certbot volume to reduce unnecessary compute usage since HTTPS is not used (#109)

What's Changed

Full Changelog: v2.3...v2.4

Contributors

ishanjoshi23 and AarjavJain101
Assets 3
Loading

v2.3.0: Quality of Life and NGINX - A Sunlink Ready for 2024 Comp!

28 Jun 23:08
@AarjavJain101 AarjavJain101
v2.3.0
1009aad
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v2.3.0: Quality of Life and NGINX - A Sunlink Ready for 2024 Comp!

What's New

The purpose of v2.3 sunlink was to take all the features we wanted and all issues we realized during driving and implement and fix them. For example, a recurring area of improvement during testing was Grafana dashboards not having sufficient panels for viewing overlays of graphs or just the new Mitsuba messages. Additionally, our DBC naming needed to be fixed to only refer to the MDI and MC not the MDU (which combines these two causing major confusion). We also realized useful features like a runtime counter to see if sunlink was working during testing and also log upload was a small but necessary feature to help with debugging. Laslty, we added NGINX and a sunlink set up script to improve the ease-of-use of sunlink for people unfamiliar with the system. Overall, all these changes pushed sunlink to be competition ready!

Major features and fixes added to this new version of sunlink include:

  • DBC fixes (#92, #92) and refactor for MDI and MC (#100). See Sunlink v2.3 versioned in history on the CAN ID Table

    • This solved the problem of referring to both the MDI and Mitsuba Motor Controller (MC) as the MDU which did not make sense when discussing origins of messages during testing. We also added VDS messages. Additionally small issues like the endianness of the satellite count message were resolved.

  • A pull request template was created to standardize the PR process and also to remind devs of testing requirements and the citations necessary to make a working PR (#96).

  • In terms of new features we added and fixed the following:

    • A new Sunlink Runtime Tracker (#98). This feature displays the number of callbacks received from the parser, the current runtime of link_telemetry.py and the current date and time. The purpose of this was so we know the performance of sunlink (processed messages rate) and also to identify if sunlink was even running or not when we turned off tables and logging. Use the -t option to force it on. Otherwise the tracker automatically appears if you are not logging or displaying tables.
      image
    • The ability to split all log files (failfiles, dbgfiles, and logfiles) when the character count exceeds a limit of 55,000,000 (#101). We noticed that on the bay PC that files nearing 110,000,00 characters would crash VSCode when opened. Thus we chose halfway to be the stable point across any device so that these files can always be viewed. After this limit, a new file is created with the same name but is appended with the next incremented index (<NAME>_0 -> <NAME>_1 -> <NAME>_2 ... etc).
    • The memorator upload script has been fully tested with vehicle driving data and correctly uploads it (with the right timestamp now) to the _log bucket (#97).

  • Grafana was also heavily revamped with a new naming convention, pit crew dashboard, and GRAFANA.md doc that explains how to set up panels, use the naming convention, and tips and tricks for making the most out of Grafana! (#102, #103)
    image

  • Lastly we added a sunlink set-up script for people using sunlink for the first time or even when you want to run sunlink from scratch to ensure breaking changes work out of the box (#89). With this we also added a certification bot that will continually renew our SSL certificate for improved security and also NGINX configurations so that we can type parser.telemetry.ubcsolar.com, grafana,telemetry.ubcsolar.com, or influxdb.telemetry.ubcsolar.com to reach our different services.

Competition Commands

  • ./link_telemetry.py -p /dev/ttyUSB0 -b 230400 --prod: Runs link_tel with no tables or logging and data is read from serial (radio) and goes to production bucket.
  • ./link_telemetry.py -u fast:Performs memorator log upload, Must have SD card plugged in and canlib installed.
  • ./link_telemetry.py --health: Checks if the parser, Grafana, and InfluxDB are up.
  • ./link_telemetry.py -r can --debug: Runs the system in randomizer mode. Good for testing.

Generated Release Notes:

New Contributors

Full Changelog: v2.2...v2.3

Contributors

ishanjoshi23, DiegoArmstrong, and AarjavJain101
Assets 3
Loading

v2.2.0 Parsing API Modifications + Debugging Features

12 Jun 16:48
@AarjavJain101 AarjavJain101
v2.2.0
1009aad
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v2.2.0 Parsing API Modifications + Debugging Features

What's New

The focus for sunlink v2.2 was to reduce the parse fails occurring from slow HTTP callbacks. To do this, major changes to the API, debugging features, and other fixes were made. All of these are detailed below:

Changes

Major API Modifications (#90)

The major change in the API was the response returned by the parser. Previously, exactly 1 valid message is sent to the parser and exactly 1 message worth of data is returned back. Now, due to packet's combined because of Nagle's Algorithm, we return a response with 1 field which is all_responses. This is a list containing the data dictionaries for each valid message in the packet. For example:

{
    "all_responses" : [
        {
            "result": "OK",
            "message": {
                "ROW": {
                    "Raw Hex": "41d99749c232e1482300000628000000000000000008"
                },
                "COL": {
                    "Hex_ID": "0x628",
                    "Source": "BMS",
                    "Class": "Module Statuses",
                    "Measurement": "Multiplexing bits",
                    "Value": "0.0",
                    "Timestamp": "2024-06-03 02:17:39.224"
                }
            },
            "logMessage": "True",
            "type": "CAN",
        },
        {
            "result": "OK",
            "message": {
                "ROW": {
                    "Raw Hex": "41d99749c232e1482300000401000000000000000008"
                },
                "COL": {
                    "Hex_ID": "0x401",
                    "Source": "MCB",
                    "Class": "PercentageOfMaxCurrent",
                    "Measurement": "MotorCurrent",
                    "Value": "0.0",
                    "Timestamp": "2024-06-03 02:17:39.224"
                }
            },
            "logMessage": "True",
            "type": "CAN",
        },
    ]
}

is returned if the request's message field is

{
    "message": "41d99749c232e14823000006280000000000000000080d41d99749c232e1482300000401000000000000000008"
}

Overall, these changes resulted in parse fails occurring 0.9% of the time. In addition, to support these changes the API.md docs are updated and link_telemetry.py's process_response function is changed to accommodate this new response.

Features and Other Changes:

  • You can now view raw data over serial with the --raw and --rawest options
    • --raw: This option prints the exact message as a hex string which will be sent to the parser.
    • --rawest: This option prints the CHUNK_SIZE chunk from the serial stream as a hex string. Note that an incomplete message may be at the end of the chunk because it did not fit within the CHUNK_SIZE specified (#91).
  • The DBC was updated to contain the MEMORATOR node which send the 0x300 RTC Timestamp message. This contains the seconds, minutes, hours, days, months, and years signals which are the exact time struct values provided by the memorator's RTC. Note that the memorator' RTC is 7 seconds BEHIND real time (#87).
  • The memorator script is simplified so that the user only needs to change the LOG_FOLDER path. Also a README.md was added to explain the process (#87).
  • Small bug fixes for the --no-write option were made when concatenated messages were sent to that endpoint (#91).

Generated Release Notes

Full Changelog: v2.1.0...v2.2

Contributors

AarjavJain101
Assets 3
Loading

v0.4.0 - Introducing Sunlink!

11 Jul 03:27
@mihirnimgade mihirnimgade
v0.4.0
1009aad
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.4.0 - Introducing Sunlink!

What's new

  • Implemented server-side Flask parser to enable support for cellular module.
  • Revamped link_telemetry.py CLI.
  • Added SIGINT signal handler to link_telemetry.py.
  • New --health, --jobs, and --frequency-hz arguments for link_telemetry.py.
  • New telemetry.toml configuration file for link_telemetry.py.
  • Added multi-threading support for HTTP requests from telemetry link to parser.
  • Improved parsed response formatting.
  • Improved error handling.
  • Added telemetry link configuration confirmation menu before main execution.
  • Docker image versions for InfluxDB and Grafana are now fixed to 2.7.1 and 9.5.2 respectively.
  • Improved InfluxDB bucket provisioning. The Telemetry and Test buckets are created automatically on InfluxDB initial startup.
  • Updated README.md setup instructions.
  • Added new documentation files: SYSTEM.md and API.md.

Pull requests

Full Changelog: v0.3...v0.4

Contributors

mihirnimgade
Assets 2
Loading

v0.3.0

25 Dec 05:58
@mihirnimgade mihirnimgade
v0.3.0
1009aad
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.3.0

What's Changed

Full Changelog: v0.2...v0.3

Contributors

mihirnimgade and DiegoArmstrong
Assets 2
Loading

v0.2.0

22 Jun 06:39
@mihirnimgade mihirnimgade
v0.2.0
1009aad
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.2.0

What's Changed

New Contributors

Full Changelog: v0.1...v0.2

Contributors

mihirnimgade
Assets 2
Loading

v2.1.0 GPS and IMU support + Debugging Features

23 May 05:46
@AarjavJain101 AarjavJain101
v2.1.0
9af803b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v2.1.0 GPS and IMU support + Debugging Features

What's new

  • The main difference between this release (v2.1) and the last release (v2.0) is generalizing sunlink's parsing ability for any data type. Using this support we were able to accommodate GPS and IMU messages not only in parsing but also for sending their data to InfluxDB and Grafana (#67).
  • In addition to GPS and IMU support, significantly more documentation was written to describe sunlink at various levels of depth (a high level overview to implementation details) so that developers can understand sunlink at any level (#67).
  • Furthermore, major debugging features such as PARSE_FAIL messages have been created so that when incoming data cannot be parsed sunlink does not just crash or give an ambiguous error message; instead sunlink will display a pretty printed message indicating the exact location and cause of the error as well as contextual information and values.
    image
  • Live streaming data also significantly improved because you can now specify what data types (CAN, IMU, GPS, etc) and/or ID's to live stream. This choice also applies to logging certain types of messages to view later (#67).
  • Printing of data tables has also changed to enable more flexibility in choosing not only what data to display but also how to display that data so that the table's maintain their easy-to-read style (#82 ).
    image
  • The brightside.dbc has also been updated to match v0.3.0 of the CAN ID table in the BOM (#83).
  • Log uploading functionality for the Kvaser Memorator's SD card is implemented with the ability to parse the .kmf files on the card and upload their data to the parser as well as clear all the logfiles after (#86). These uploaded logs currently go to the *_log buckets on InfluxDB.
  • Lastly, more Grafana dashboards were created to support multiple testing and production circumstances in which different configurations of CAN nodes are set up (testing with the battery vs the MCB vs a full integration test, for example) (#67).
  • Please see the following slideshow for details on multiple data type support. Note that there are changes in the printing of the data tables now: Support_for_IMU_GPS_Sunlink.pdf

Fundamental Changes

  • With v2.1 the way sunlink's features have been implemented are meant to reduce the time taken to make changes. In particular, using data classes to separate parsing logic from the rest of sunlink's functions is especially abstracted in this release because a developer will spend 90% of their work creating the data class for their data type and 10% of the time making minor tweaks within the parser/ folder to integrate their new data type. Essentially, implementers can follow a simple 2 step process of first creating the data type as a contained class and secondly adding a couple of lines to let sunlink know your data exists.

Useful Commands

  • ./link_telemetry.py --live-on GPS 0x403 -r all --debug: Runs link telemetry and live streams all GPS messages and CAN 0x403 messages to Grafana and writes to the debug bucket of influx. Also randomly generates all types of messages.
  • ./link_telemetry.py -u: Perfoms log uploading on the log files in the Kvaser Memorator.
  • ./link_telemetry.py -l 0x627 imu -r can imu --debug: Will run randomizer only on CAN and IMU messages and will log only 0x627 messages and IMU messages. Also writes to the debug buckets on InfluxDB.
  • ./link_telemetry.py -p /dev/ttyUSB0 -b 230400 --prod: Checks the port on which the serial converter for radio is on and specifies a baud rate of 230400 bits/s.
    ./link_telemetry.py --prod -o: Runs link_telemetry in offline mode and writes all data to the production buckets on InfluxDB.

Future Features:

Generated Release Notes Per PR:

New Contributors

Full Changelog: v2.0.0...v2.1

Contributors

mjohal67, ishanjoshi23, and AarjavJain101
Assets 3
Loading

v2.0.0

14 Mar 18:15
@ishanjoshi23 ishanjoshi23
v2.0.0
70fe78d
Compare
Choose a tag to compare
v2.0.0

What's New

  • Updated Brightside DBC
  • Offline Mode for link_telemetry.py to receive data through a physical CAN bus connection.
  • New Grafana Dashboards for Brightside

Summary of this revision

Sunlink V2.0 accepts CAN data for messages specified in the DBC files. There are 3 modes to receive/process data: through a UART connection (using a radio receiver), offline mode with a physical CAN bus connection, and random data generation.

Assets 2
Loading