T1000 Development Log
I'm going to try and put as much here as I can over time to show the development process for this iteration. Selfishly it should also make writing my ENEE499L reports much easier. During debugging & adding features I'm going to try and track issues using the issues feature in this repository and will hopefully link them here consistently.
This revision included two boards. The T1000 Digital Board and the T1000 Cutdown Power Board.
I want better names for these boards, I think I'm going to try T1000-M4F for the digital board (M4F processor). Still a WIP.
This board builds off of experience with BITSv5.x but with changes. Below I describe what's on the board and some of the why behind these selections.
-
STM32L4
- Very low power, ballpark estimate is 10mA consumption for this application
- Hardware floating point unit, will want fast floating point computations for running PI loop
- Super small package, 36 ball wafer scale package is I think the smallest I could possibly get for this board and fab process I chose
-
U-blox MIA-M10
- The smallest M10 GPS they make
- U-blox products have excellent flight heritage which justifies $30 per piece cost
-
MB85RS1MT FRAM
- Simple, non-volatile, byte-addressable memory for storing data and logging events
- More complicated than an SD card but at much lower power and size
-
MS5607 Pressure/Temperature Sensor - simple sensor that happens to be properly rated for the environment unlike Bosch offerings commonly used
-
SX1262
- Same radio used in modules used on other designs
- Integrated based on a very simple reference design, lower total output power (~17 dBm if I had to guess) due to wanting to run the system at 1.8V for absolutely minimal power consumption
- May have to relay through a more powerful radio (i.e. BITS) at times but should still be able to receive line-of-sight commands
-
Power - Not much to talk about, 1.8V from a battery and a 1S LiPo charger for convenience, produces 3.3V from USB power, both 3.3V and 1.8V are fed through and ideal ORing diode IC which just supplies the higher voltage to the system
This board was fabbed on JLCPCB's 6-layer process. It was cheap at the time of ordering because of a special offer. Critically, this process offers a free via-in-pad process which allows me to use the atrociously small STM32L4 package I picked. It's not truly necessary but it helps reduce total board size at the end of the day.
In terms of the layout, it could be more optimal and dense. My first priority with this revision was to design it fast so that I could start figuring out what worked and what didn't. Given the time I'd like to experiment with different layouts to further optimize the density.
I chose to produce this as a different board for two reasons.
- The 6-layer board process I chose was only cheap for designs under 50 x 50 mm. Integrating both of these boards in one 50 x 50 mm board would be difficult (see the inductor I chose for this board)
- I wanted to be able to change this part of the T1000 design without changing or building more Digital boards. Given the number of components on the Digital board it takes quite awhile to assemble by hand. I've never done a power stage like the one found on the Cutdown board so I split them up to make my life easier.
A benefit to this approach is that the Cudown board can be produced as a 4-layer or probably even 2-layer board.
As for what is actually on this board, there's not much. You'll note that the ICs are both Analog Devices products. Although expensive, they were chosen specifically because they are available in LTSpice. I wanted to do as much simulation of this system as possible before committing a design to a board and I'm very comfortable with LTSpice at this point.
- LTC1751 Charge Pump - attempts to produce 10V from the supplied voltage at the battery terminal. This is critical since driving power mosfets like the ones used requires voltages in the neighborhood of 10V to switch them properly. The actual amount of energy used for this process is fairly low so I chose to use a charge pump to save on space.
- LTC7061 FET Driver - is what actually drives the FETs. This makes it very easy to drive a half-bridge FET configuration with extra safety features to prevent short circuiting the voltage supply to ground.
Did first pass assembly of the digital board on 9/25/2024. Took about 4 hours which was longer than I had hoped. That being said there are 98 footprints on the board, not all of them were placed but 90 probably were.
Since I did it in Terrapin Works, I got some lovely pictures with their fancy microscope. Some touch-up work will be required.
- I'm very pleased with the alignment of U3, will need to check for bridging before I power it up but it looks good.
- USB connector bridging like it does every time, my paste application wasn't very great this time around so that might be the issue, need to investigate better methods for paste application.
Stencil from OSH Stencils, only ordered the one I needed for the Digital board since I can throw the Cutdown board together without one.
Batteries from HobbyKing. Not my favorite vendor but I needed a small capacity 1S LiPo that could dump 3A and they had some cells that claimed to do this, for much less than $30 a pop.
Everything else was from these two DigiKey lists, one for each board. There's purposeful overlap between to two where components can be the same (capacitors mostly).
https://www.digikey.com/en/mylists/list/FE2V6D2R7P
https://www.digikey.com/en/mylists/list/F9UPXKBF5G
I threw together the cutdown board as an LTSpice sim to play around with it before ordering. You'll note that it's absurdly basic.
There is no modeling of parasitics and the PI loop is essentially analog and ideal.
This is because I'm not particularly knowledge of how to model those things, and the amount of power and cost involved here means that it's faster to just try the design in the lab since it's not particularly expensive or dangerous if it fails miserably.
Nonetheless, the result from this sim shows a very damped response which is pretty much what I want since there's no load transients involved (it's just a resistor).
A sim can only be so good though so hopefully the real thing behaves how I think it will.
This is once again a new adventure. New MCU = new software :)
I won't repeat all of the info you can find in Firmware/gen_proj/README.md but I will talk about it some.
First and foremost I don't like vendor IDEs. While they sometimes include really nice conveniences and features that make development easier, I find myself often wasting time trying to understand what's happening behind the scenes when debugging something.
In true C purist fashion, just give me a Makefile
So here's what I did, I pulled the basic main.c, syscalls.c, and sysmem.c out of a generated STM32CubeIDE project and dumped them into my own project with the help of Google. I also stole the CubeIDE generated linker script.
I then downloaded the ARM CMSIS and STM32L4 CMSIS components, along with the STM32L4 HAL and Low Layer (LL) drivers. All of these things are included as submodules so that they can be updated and versioned easily.
The CMSIS components give basic startup logic and system configuration. syscalls and sysmem are needed to get compilation to work properly, I'm not entirely sure why but I don't think it matters for this application.
Most critically, the LL drivers in particular make using peripherals easier. The ST HAL is also available but it seems like more than is necessary for this application. It seems to border on being an RTOS which I don't want right now.
I then took a fancy Makefile from the internet and updated it to compile all the source files from the relevant directories and link them together into the needed .elf file.
This approaches means that I shouldn't need the CubeIDE for this project. At the worst, I made use it to generate some of the special blackmagic that is needed on some occasions. Hopefully that won't be necessary though
I have yet to really write demo software yet though because of other work.
Scratch some of the above.
I have changed my approach somewhat in that I'm now just copying the STM32CubeIDE project structure. The top level view is that the Debug folder is where builds happen. The structure of this folder mimics the top level project structure, each source directory has a .mk file that describes how the files in that directory should be built, and the top level files and makefile bring them all together. This makes adding new libraries and files relatively painless while making it easy to leverage code generated by the CubeIDE.
The first thing I have started working on is SPI because being able to log data to FRAM is crucial. I now have the beginnings of working SPI, I can read the device ID of an FRAM breakout I have. Now that I have worked out some of the issues with my development setup and familiarized myself with ST's LL drivers, the rest should go more quickly.
10/12/2024 - SPI reading and writing working correctly now, lifted code (link in function comment) from an example online to get it negotiating the buffers correctly (both tx, then rx, every access).
Originally tried to port FatFS. Unfortunately FAT requires at least 128 blocks with minimum 512 byte size. This isn't possible on the FRAM I'm going to be using so I had to pivot to littleFS.
I had originally looked into porting littleFS a while ago but something just wouldn't click in my brain with it. Wahtever that was, it's no longer an issue and the port was mostly pain free. Now the boot_count example works, so after a little more testing it looks like I'll have filesystem/logging pretty much done.
PWM is working now, on the advanced timer peripherals the MOE bit has to be set which wasn't obvious for a while.
http://efton.sk/STM32/gotcha/g3.html
Now that that's taken care of the complementary PWM generation works very nicely. I hooked it up to my power conversion board and that started working nicely as well! Have not tried to add in dead-time yet but I don't think it will be too difficult since I think it's just a struct field.
I also have an ADC demo working on the dev board. It mostly accurately reads voltages that I set, more testing needed in final form to see what the accuracy is really like since I didn't have a precision source to test against or set with. Either way it does work in it's most basic form.
However, it doesn't seem to be properly reading values in the low millivolts ranges. Again I didn't have anything very precise to use as a source but 10's of mV seemed fine, less than that seemed not so good. This could be as simple as the fact that I was breadboarding with long leads on everything so total resistance was fairly high compared to what will be in the final system but I'm not sure right now.
This was as issue when testing basic PI loop functionality since the ADC just kept reading 0 when I was needing it to read at the <10mV level. Theorectically the granularity of a 12-bit read is ~0.8mV. Either way, something needs more work and testing here. Worst case scenario is I end up scaling the values with an op-amp configuration.
I really quickly took the PI loop implementation from a Phil's Lab video to try and play with.
I got some mixed results but overall it certainly did iterate and change the PWM values according to the setpoint I provided. Since the ADC was having trouble sensing across the 5 mOhm current sense resistor I have on the power board actual feedback wasn't occurring, but the code still ran and the PWM duty cycle still changed!
Notably I had to crank the system clock up to 20 MHz to get a 1 MHz PWM out of the peripheral with a duty cycle step of 1%. I for some reason had in my head that the PWM peripheral could be clocked off of some kind of PLL loop but it seems like it is essentially right off the main system clock. Not really a problem, just an observation.
I'm going to try and follow this https://github.com/CShark/stm32usb/wiki, will see how it works.
Now that I have something assembled in a reasonable fashion I can start making sure it works at the hardware level. This stuff is all be tracked as issues so I can check them off when completed and track the steps to fix.
-
First thing I figured out that I did wrong: This SKU only has one comparator. Not the end of the world but a mistake nonetheless.
-
USB DFU does not appear to be working.
-
I got a connection with an ST-Link that appeared to be fully working for a couple of minutes. It stopped working now but I think that's because my connections are a very quick bodge job.In hindsight I think that this was actually connecting to the devboard MCU, not the one on the digital board. It shouldn't have been happening according to the jumper configuration I had but looking back at the connection details pulled by the STLink the part number isn't correct.
10/11/2024 - I made a quick board revision to clean up a few issues and switch the MCU to a QFN package. The wafer level package has merits but being able to visually verify most of the connections is more important right now.
10/17/2024 - I have all the parts now for the revision 1 Digital board, will assemble soon. Will now need to add a simple op amp configuration to the Cutdown board to properly scale the current sense signal since I made a critical oversight. it's unrealistic to expect the ADC on the STM32 to properly read values as low as 4mV which is what would be required for the current design. I think if I take an opamp and scale the signal so that it is mostly between 0.5V and 1.5V I'll have better luck.
10/20/2024 - assembled the revision and got the device to present and connect over USB DFU. Oddly it presents as an STM32L412 when it's an STM32L422. I think that's a bug in the STM32CubeProgrammer application since I seriously doubt Mouser sent me the wrong part and that the package marking is wrong. Either way the device is definitely working better on this board than the previous one. Excited to run software on it and get it hooked up to the power stage.
- PWM output works
- SPI, FRAM, and littleFS demo all work
- Working on getting I2C to work, it seems that the data line is correctly functioning but the clock line is completely unstable which means comms don't work
- ok not sure what the problem actually was, took the pressure/temp sensor off the board and it definitely works, I think the instability was just not having a good probe point (really should put more test points), haven't put it back on but polling reading data from the GPS works now
10/28/2024 - Flew flight code yesterday that I was fairly confident in and had tested prior to flight. The intention was to log GPS data to the littleFS partition on the the board once every 30s during flight. This was to confirm basic operation in a minimal structure and validate the ability to record data during flight. Unfortunately I didn't get any data back. Based on the state of the filesystem on return it does appear that the system started ~8 times during the course of about 5 hours. Restarts during flight are not unexpected but for the system to have collected no data on the ground is confusing for sure. The system was plugged in and running for ~30 minutes before release so I'm not sure why there aren't any logs from that time.
- I know that the system was plugged into the battery because the
boot_countvalue increased, also the battery voltage was about ~3.8v on return. - I know that the code on the board was valid because I tested it on the way there and tested it on return and both times it logged valid data.
My theory is essentially that somewhere under 40 degrees F that the system suffers some power or other issue due to temperature (the LSI stops operating which stops the log timer, the battery/power system voltage drops too low (~100mV), etc.).
In a related manner, if I2C comms with the GPS failed the system would hang idefinitely (a fix for this has been implemented).
I'm not satisfied with a temperature explanation though since the system did restart multiple times and it seems reasonable that it would've succeeded logging something at some point. Other than the environment the only other thing different from the rest of my testing was the single piece of duct tape hold the board to the side of the plastic container it flew in. I doubt that was the problem.
Currently my plans to try and reproduce involve buying dry ice and leaving the system running in a cooler for an hour or two while recording temperature separately.
Suffice it to say that this was a disappointing and concerning result for NS-129, hopefully NS-130 is better.
10/31/2024 - Well now the GPS is not cooperating. I got lock yesterday after maybe 20ish minutes but it was off by 30+ miles. Time was correct. Have tried reseating a different GPS from the first assembly attempt but no luck getting a lock yet.
I've also been working on a MS5607 driver. I believe the software is functioning correctly but for some reason two different pieces I tried both refuse to return data sometimes and trigger my software I2C timeout (probably over 10 ms since I can see when it happens). This is pretty frustrating since I'm not sure if it's a HW or SW problem. I don't really see it being a HW problem at 100kHz over a fairly short distance.
But I do have one success to report. I just ran the power stage off the battery for the first time and easily cut some 550 paracord.
- Open circuit batt voltage: ~3.7v
- Nichrome resistance: 0.3 Ohm
- Duty Cycle: 25%
While powered on the voltage drop of the battery compared to the open circuit config was only about 70mV, this will be higher at altitude I think due to the temperature difference but it doesn't concern me right now. I didn't have too much of a reason to be worried about this particular part of the design but I'm very glad to have confirmed that the battery can supply the requisite amount of current for a cut.
11/03/2024 - So I tried replacing both the GPS and the antenna and it still isn't getting position lock. Looks like the antenna trace impedance is more like 45 Ohms instead of 50. I still don't think this explains why it was working originally but it's the only thing I can think of as being an issue right now. Guess another revision is in the cards for that issue.
I've adjusted LPTIM1 to be much closer to a true 30 second timer. I've also added a 30 second interval count that I'm going to use as a trigger for NS-130 this week. I'm thinking that a cut attempt at 45 minutes is a good plan. I ran a test outside in the 40 something degree weather for 30 minutes and it cut right on time. Completely burned out the nichrome I was using though. I've also confirmed that the screw holes I have to hold the nichrome will work quite nicely on the V0.1 cutdown power stage I have coming in tomorrow. After that I'll have to test in the pressure chamber, cold box, and do a design review.
11/11/2024 - A few new things. NS-130 pushed back to this weekend.
- Closed loop PID works for cutdown purposes. No idea what the transient response is right now but that isn't crucial for this flight test. The point is that it nicely stabilizes to a constant current across the nichrome which is plenty sufficient for cutting.
- Cold box testing is a partial success so far. Got the system down to -49.9 C after about 20 minutes at which point it shut off. It spent overnight in the cooler and in the morning there was still dry ice left so I imagine it spent a fairly significant amount of time at that temperature. Seems to be working completely fine today so no permanent damage so far. -50 C is definitely where I would expect to see things stop working so it's good to know that the system will function and record data down to that point. Further testing needed to ensure that container insulation will allow for operation during the entire test flight and also allow me to confirm that cutdown will work down to that temperature.
- If insulation isn't enough I'll throw a chemical handwarmer in there on the launch pad.
- Still working on trying to see if I can get the system reading data from an external GPS module over USART. Transmission works fine for dumping log files but not reading for whatever reason.
- I have a board revision in that cleans up the GPS area of the board (tuning antenna line impedance and better grounding). I'd like to try and assemble it before launch but I think the timeline is too tight.
- I think my best bet is to get the external module working, then I can test geofence logic in flight, and also log data.
- The MS5607 driver works now. I want to work on it more but essentially I added 20ms delays between writing commands and reading and it's working consistently with what looks like good results. Not the cleanest solution but it works for now.
- The first overnight cold box test I did lasted about 2.5 hours with a successful cut at 45 minutes. The data from the test didn't save properly but I dumped the contents of the FRAM and was able to see most of it. For whatever reason it looks like it only got down to -2 C on that test but it's a useful confirmation of battery life either way. The string used was the nano-cord used in the current cutdown devices since it's the easiest to use right now. I'm planning on flying that for NS-130 since it not only gives the best chances for a successful cut, but is also the easiest to configure with the current layout of the cutdown power stage.
- I added internal watchdog support, should reset the chip if not reset within 32 seconds (the longest interval I could set). Not tested but hasn't been interrupting data collection at least.
- Still have not gotten to playing with the SX1262 on board. I full intend on running it through it's paces after this flight. For now I want to focus on getting something out of this flight (data for the full duration, a successful cut).