Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different voltage readings for connected voltage inputs #88

Open
mmallozzi opened this issue Feb 17, 2022 · 108 comments
Open

Different voltage readings for connected voltage inputs #88

mmallozzi opened this issue Feb 17, 2022 · 108 comments

Comments

@mmallozzi
Copy link

I'm in the US with split phase 120V/240V AC power. I just set up my first expandable meter, starting with one of my subpanels where all I want to measure right now are a few 240V HVAC appliances which seem to be balanced (all newly wired with 2 wires plus ground, so no neutrals), so I figured I'd be okay with one AC transformer. So I have not cut the jumpers to enable separate transformers (I'll end up doing that for the meter on my main panel). If I understand correctly, that means they are hardwired together, and I should get the same exact reading for V1 and V2. However, I am seeing ~117V for V1 and ~127V for V2. The voltage calibration values in EmonESP are the same for both.

@CircuitSetup
Copy link
Owner

It's very odd to see that much of a difference between the 2 voltage readings when they are coming from the same AC transformer. I would recommend calibrating the voltages separately, and changing the calibration values so they both read about the same. Otherwise the power reading is going to be a bit off on one side of the meter.

@mmallozzi
Copy link
Author

Makes sense - thanks!

@Cougar
Copy link

Cougar commented Mar 5, 2022

I'd like to second on this. It is not just a calibration issue. Maybe even another issue should be created.

I did a simple test with three meter boards, all connected to similar AC power supplies (could be a little bit different of course). I changed calibration values until I got very close readings from all meters. Values (for ESPHome config) were following:

gain_voltage: "12770"
gain_voltage: "12674"

gain_voltage: "12638"
gain_voltage: "12727"

gain_voltage: "12697"
gain_voltage: "12750"

Then I did power restart for all modules (moved them to another extension cord) and the voltage difference between the lowest and highest reading changed from less than 0.5V to around 5V! M90E32AS Vrms voltage accuracy by spec is ±0.5%

imgur-2022_02_23-154013

This means that it was just a pure luck that these calibration values gave me the same voltage readings. Now I'm troubleshooting this issue with all values 12770 and the result is the same. Every board reset makes readings different. Here is one ESPHome reset:

imgur-2022_03_05-15:44:57

For this graph I measure the voltage each second.

@Cougar
Copy link

Cougar commented Mar 5, 2022

I first suspected some sort of initialization problem because not all setup registers are written in the boot. However, based on spec it should not be a problem and defualt values are going to be used in this case. I have done many chip restarts and read all registers after that but haven't seen any difference in config registers.

Another idea came from M90E32AS datasheet 6.4 POWER ON RESET TIMING where it says:

To make sure M90E32AS is reset and can work properly, MCU must force M90E32AS into idle mode firstly and then into normal mode. In this operation, RESET is held to high in idle mode and de-asserted by delay T1 after idle-normal transition

This meter connects PM0 and PM1 to VDD which means that the chip is hardwired to Normal mode only. Could this cause a problems here? Now way to test without removing two chip pins from PCB..

@CircuitSetup
Copy link
Owner

@Cougar that is very interesting. I haven't observed that behavior myself. You may be onto something with the reset timing. Are you doing a reset via software or power cycling? If software, I'm wondering if it is not updating the calibration values properly.

You may want to try a reset via software, and compare to power cycling to see if there is a difference. I'm going to bet there is. Especially with the second IC (assuming there is a natural bit of delay in initializing the second IC) You are correct that the board puts the metering ICs in normal mode only, but I haven't seen problems with this in the past. It may be a matter of reinitializing and writing to the config register with a POR. The code for that in ESPHome is located here.

@Cougar
Copy link

Cougar commented Mar 6, 2022

@CircuitSetup My first post was power cycle. I moved the whole setup to another room and lost all my "good" calibrations.

Most resets are software resets that ESPHome does when uploading new firmware. I hope to find something and add more and more debug to this firmware. Now I dump all registers to the log after chip initialization. Only differences are actual measurement registers, all config remains unchanged.

Now I did 10 sec power cycle for all three meters and this is how it looks like:

emon-reset

"Line Voltage" is the UrmsA of the first IC and "Line2 Voltage" is second IC.

I also tried to set up all offset registers based on this code but it didn't change anything.

@CircuitSetup
Copy link
Owner

I would expect that the 3 voltage measurements per IC would be similar to each other since they have the same voltage dividers (there's a set of voltage dividers for each IC). Depending on when you received your meter those resistors are either 1% or 0.5%. More recent batches are 0.5%. I'm guessing the variance between individual IC voltage readings is either the IC itself, or the small delay in ESPHome actually taking the reading.

Either way, if you're reading power from the meter, and each voltage is calibrated, you should still get very accurate results.

FYI, I believe the offset registers only apply to the accumulated power registers. The datasheet is a bit confusing regarding them, and how they should be used.

@Cougar
Copy link

Cougar commented Mar 12, 2022

I 100% agree. I read only first voltage from every chip. I will try to read all three to see if there is any inconsistency between them too. Of course reading times are slightly different but from graph it is still visible if the difference has always the same offset.

I don't think that voltage divider accuracy is a problem right now because the change always happens with chip reset. I'll add some more knobs to ESPHome and check if also happens if I just send ATM90E32_REGISTER_SOFTRESET without any other configuration in setup phase. Or if ATM90E32_REGISTER_METEREN toggle change anything.

If I understand correctly then all config goes to flash and you don't need to resend them at all. This should be the reason for config CRC register and dedicated WarnOut pin. Isn't it how simple pulse counting meters with this chip (and no MCU) should work?

@Cougar
Copy link

Cougar commented Mar 12, 2022

I tried to reset chips during operation and this didn't change anything. However, the whole ESPHome reset almost always did.

My transformer is 9.0V 0.67A (6.0W) which should be enough for ESP32. But the secondary resistance 2.5 ohm is too big for a stable output for ESP32 current peaks which are by datasheet up to 240 mA for wifi itself. This is at least 0.6V drop already. Most probably the difference between two IC readings came from timing when ESP32 wifi is transmitting or not.

When I removed the ESP32 from PCB and powered it from USB (keep only M90E32AS power from AC), the difference between all voltage readigs is around 0.2-0.4V only.

emon3_voltage

Now I'm thinking, should I remove rectifier from PCB, keep AC only for voltage sensing and use separate USB power adapter for power the boards to get even more precise reading.

Would be great to have dedicated jumper traces for such configuration in the future PCB revision. Or even better, change the JP12 and JP13 to three position so that it is possible cut J4 AC socket from VA+ and VA- and use J3 for both ICs.

imgur-2022_03_12-21:13:54

@alexruffell
Copy link

I am not sure whether it is the same issue or not, but it looks like it may be.

I have a tiny 9V PCB mount transformer to provide reference for V2, and a PCB mounted 12V 14VA transformer to power the board and ESP32, and provide V1. While I do see a voltage drop of about 0.1V when I connect the ESP/energy monitor to the 12VAC transformer, it tends to be consistent (meaning it seems -0.1V all the time but hard to tell for sure given the fast line voltage fluctuations). I am using a 5 1/2 digit bench DMM set to slow for my measurements. I compare the DMM measurement of line voltage (which for now is powering both transformers) to the voltage reported back for V1 and V2 and calibrated them to show the same reading. Every time I go back to it after some time, the delta between V1 and V2 is off by up to 0.5V while after calibration it was within 0.0V and 0.1V. I chart the voltage readings in Grafana and it is quite visible that they grow apart.

I often noticed that when I reset the board manually or by uploading a new ESPHome firmware, the calibration change I made seemed off, or the calibration I had completed was now off again. This sounds similar to what @Cougar was reporting although with a smaller delta.

I know that in the big scheme of things a voltage measurement error up to 0.5V on a 120V line is not critical but I was wondering whether anything could be done. I just purchased my board and have v1.4 rev1. Are there any changes I can make to it to make things better (better resistors, etc)? I am comfortable with soldering and have good enough equipment to do it.

Also, is there something I can do to stabilize the secondary output powering the ESP so that V1 won't be affected by its fluctuating power draw?

@alexruffell
Copy link

alexruffell commented Aug 29, 2022

Based on the graph below, the delta may have something to do with the update delay. When the delta is set to the normal 10s delay, the delta between V1 and V2 is 0.4 ~ 0.5V but when it is set to 2s it is 0.1V. So, the calibration should be fine in my case, just a matter of timing and the measurements not being taken close enough in time. I wonder whether there is some way to improve on that without keeping them at 2s which bogs down my HA with tons of db writes and unnecessary data.

In the graph below, where the two lines grow nearer is when I uploaded the firmware changed from 10s to 2s. The big L1 dips were when I rebooted the ESP remotely which may be connected with a higher draw on the transformer while ESP32 is booting up causing a minor drop that translates to a bigger drop of this graph.

image

@Cougar
Copy link

Cougar commented Sep 24, 2022

I modified my board a little bit to remove board power from AC transformer. The easiest was to just remove a rectifier from the board and supply power to ESP and ATM90E32AS via NodeMCU micro-USB port. I bought dedicated DIN rail 5V power supply for that. I'm not sure if this is a good practice to feed 3.3V to the step down switcher but so far it works fine. Might be better to remove L1 as well to completely disconnect onboard power supply but this component is much harder to remove.

@alexruffell
Copy link

@Cougar - Thanks for the tip. For now I have a large capacitor on the 3.3V and 5V but I still have to vet if it is really helping. I opened a ticket on ESPHome github reporting a related issue. I have 2 boards... and the 1st chip of both boards alternate having a 0.5V error in the reading.

image

The arrows indicate manual restarts, no changes. For an instance they all matched but that rarely happens. I can keep restarting and the L1 V and L1 AO V just keep trading places having the 0.5V error. That is not due to the ESP32 current draw, or voltage fluctuation etc... it has to be something with how the chips are configured (?) at boot-up.

@Cougar
Copy link

Cougar commented Nov 6, 2022

My conclusion was that it doesn't matter if the problem is visible for one or both chips. These chips are polled in different times and it is possible that during one operation the ESP32 draws more current than during another. I think it is because of WiFi but I haven't tried to measure these timings to prove it. WiFi just seems most probable reason due quite high current draw.

@alexruffell
Copy link

@Cougar What I am showing above is not due to wifi as only 1 of the 2 voltage channels (chip1 on both boards) connected to the same transformer see the 0.5V shift and that shift doesn't change based on anything other than a reboot. The behavior is so repeatable that I am convinced there is a software bug somewhere during initial setup of the chips.

@Cougar
Copy link

Cougar commented Nov 7, 2022

Did you switch off the Wifi? How do you get these readings? Different boards have been read in different times. You can try to swap chips under sensor: and see if the shift moves along to another chip.

@alexruffell
Copy link

alexruffell commented Nov 13, 2022

@Cougar No, WIFI is fully operational. For testing purposes I connected the same 120V feed to both transformers and calibrated them to match what I was reading from my benchtop 5 1/2 digit DMM. That in itself was a bit hard due to the constant fluctuations of line voltage and I found that late at night it is a bit easier. I also sped up the reporting to 3s (faster would cause it all to choke up) because I figured that the 4 chips taking measurements at different times would account for some variation. Using Grafana graphs to monitor the changes helped as I focused mostly on it trending in the right ballpark and for all 4 channels to be close to each other since they were all reading the same line voltage.

image

Trying to reword what I said in my previous post, all 4 channels are now calibrated to read the correct voltage however each time I reboot the system, ch1 of board 1 or board 2 shifts up by about 0.5V. The shift is constant and comes and goes simply by rebooting the board. Most commonly they trade places... when one is shifted high, the other is correct, and after a reboot they trade places. In one instance shown in the screenshot I re-pasted below they happened to both not have the shift so all 4 channels were reporting the voltage correctly (look at signals between the red arrows). Before the left red arrow L1V (Green) is shifted high, and after rebooting twice, L1 AO V (Blue) was now shifted high. This behavior cannot be related to what the ESP is doing... so it must be some software bug in how the chips are setup at boot (or something along those lines).

image

@Cougar
Copy link

Cougar commented Nov 19, 2022

Ok, now I see that I described a little bit different situation. You have quite powerful transformer and the load of the ESP and power measurement boards don't affect the reading as much as my 6W transformer (around 2 volt or 1% power drop).

I did test with all my three boards now. EMON-1 and EMON-2 are using 9V 0.67A (6W) AC power transformer. EMON-3 has rectifier removed and external transformer doesn't supply power to the board but only to the ATM90E32AS voltage sense inputs. ESP and energy meter board get their power via ESP board 5V USB input.

This is a 30 min graph with 1 second step. I reset all boards via API in 5 min interval. First graph shows voltage reading from all boards and then there are graph for each board where you can see voltage reading difference from IC1 and IC2 in volts and in percentage.

emon_voltage

You can see that the difference between EMON-1 and EMON-2 is quite big and not very stable. This is where the ESP32 power consumption changes are visible. Fluctuation is in volts.

EMON-3 does not use power from transformer and two chips should get exactly the same voltage reading. Or with small constant offset due voltage divider resistor tolerances.

Difference between two chips is very stable. Still, every reset changes the difference and I think this is the error you are seeing too and I don't have any good explanation for that either.

@Cougar
Copy link

Cougar commented Jan 12, 2023

I have had now some time to let my test run days.

My ESPHome is still not stable enough to run it long time. Its measuring scheduler just stops after some time and only restart helps (even over API). But this is different story and I'll take it to the ESPHome development.

Back to the ATM90E32AS errors now. I still have the issue that almost every ESPHome restart changes the error margin in the range of 2V and I have no idea what is the cause.

I made a simple button where I can run IC setup any time:

button:
  - platform: template
    name: ${node_name} IC Setup
    on_press:
      then:
        lambda: !lambda |-
          id(chip1).setup();
          id(chip2).setup();

AFAIK this button runs exactly the same IC initialization code that ESPHome reset does and still only ESPHome reset changes the voltage reading error.

But what is even more interesting is that the difference does not stay constant even between restarts and its change looks cyclic. I'm totally puzzled now.

This is how it looks like for 3.4 days continuous measurements every second where I already compensated the mean difference (0.25 V). Does anyone have any idea what is going on there?

emon

I already suspected things from ESP32 board interference and room temperature to rotation of the Earth. I already separated ESP32 board from the ATM90E32AS board with 20cm cable but nothing changed. The relative difference stays between ± 0.16 % which is very good and I actually don't worry about that but it is still interesting.

@CircuitSetup
Copy link
Owner

There is a feature on the energy metering IC's that compensate for readings based on temperature (which is why you can get a temp value from them). It is set to its default settings, and shouldn't need to be adjusted though. Details are here on page 35-37
Keep in mind that each chip may be reading a slightly different temperature.

I'm guessing the differences you are seeing are a combination of this and other variables mentioned above.

@Cougar
Copy link

Cougar commented Jan 13, 2023

Yeah but the temperature is not changing like that and it is quite stable over time. This was the first thing I checked. My daily routine is not same every day either.

Still this small change is not an issue but just interesting thing that you usually don't notice unless you do a lot of measurements.

This is how the same voltage difference graph looks like now when I send ESPHome reset in every 5 min and this is much bigger issue and can't be related with Earth rotation or anything like that :)

emon

@alexruffell
Copy link

@Cougar I am certain that the issue I am seeing is not caused by any other signal or environmental factor as I believe you have eliminated as well. It is so repeatable by simply resetting the board that I'd say it is certainly something to do with the software or some 'defect' in the chip. The error in the reading seems to toggle on and off at each reset and while at first I thought it was bouncing from one board to the other, I then saw that at times the voltages match likely because both errors are on the same leg. In other words, it is something that somewhat randomly affects one leg (voltage measurements of L1 and L2) or the other with it often switching which one it affects. Hopefully this makes sense... To be clear, my setup has not been installed yet as once I do that I won't be able to troubleshoot further. Both transformers are connected to the same outlet and both voltage inputs have been calibrated. When the error is not affecting the channel, the reading is correct. Both channels read the voltage correctly unless the error is affecting that channel.

If we look at the lower level code that implements this board / chip, is there any code that could be randomly affecting just one of the 2 voltage measurement channels, or maybe both or neither, somewhat randomly?

@alexruffell
Copy link

alexruffell commented Jan 16, 2023

I made a simple button where I can run IC setup any time:

Where did you define the chip ids? I don't have any chip related ids in my YAML. I'd like to test what you did to see if I can get the error to bounce around.

EDIT: I added id: chip# right after each platform for the 4 chips. I then added the setup command for the 2 additional chips.

image

My starting point shows that L2 and L2 AO match and that L1 and L1 AO match. The chip setup appears to make no difference (assuming I implemented it correctly). At 11:21 I hit the reset button and L1 error went away and it now matches the other 3 leaving just L1 AO with the 0.5V error. At 11:26 I hit reset again and L1 got the error back...

image

@Cougar
Copy link

Cougar commented Jan 17, 2023

Here you found the same behavior that I see. Chip reset via setup hook doesn't chance anything but reset button or ESPHome restart does. Only thing that happens in latter case is the I2C setup and the data collection timing relative to AC zero crossing is probably different. Could any of them be a reason why the difference changes?

@alexruffell
Copy link

alexruffell commented Aug 4, 2023

I built a little calibration rig to try to get to the bottom of this issue. While I understand that the voltage difference is minimal and in the big scheme of things irrelevant, I wonder whether there is some way to fix it given this seems to be a software issue. I implemented a faster SPI clock (1MHz) courtesy of @descipher and while I have not noticed any drawbacks, I am inclined to think it helps given this time around I was able to narrow the gap between the voltages reported by the 2 boards. The one thing that still bugs me is why does L1 voltage reported by the main board and the addon board swap places reporting a slightly higher value nearly each time the ESP is re-flashed?

The graph below only shows L1 V and L1 AO V so it is easier to see how they swap places. I have a larger transformer feeding L1 on both boards, and a smaller transformer feeding L2 on both boards. Both transformers are currently being powered by the same mains connection (outlet) and I am monitoring the voltage with a benchtop 5 1/2 digit DMM. I am not trying to obtain absolute accuracy, but rather the 4 measurements to be as close as possible to each other.

In this thread there is discussion of the effect of the ESP power draw when using WIFI, but that would not explain why L1V and L1AOV trade places... which is the issue I am trying to fix.

image

I am also wondering whether the 4 voltage readings are actually necessary to read 6 power measurements. Would it be possible to just use the L1 and L2 voltages measurements from one of the two boards for the purposes of the entire system? I am guessing not as each chip will independently read voltage & current and calculate power by multiplying the two?

To calibrate the system I am using a 150W 120V bulb (my load) and power it through a 4 x loop so the power that the current transformers see it 4x which, given bulb wattage tolerances, translates to 4A. The clamp in the picture is what I use to read the current (reading it typically around 4.017A). The caps I soldered on the ESP are just tests to see if I can reduce fluctuations caused by the ESP draw but have no effect on what I am trying to fix.

image

@descipher
Copy link

descipher commented Aug 4, 2023

I see a number of potential reasons for the variation in L1, L2 or L3 voltages.

  1. UoffsetA/B/C in not implemented in the code and thus the variation of L1, L2 or L3 voltage circuit value/noise differences cannot be calibrated. To correctly calibrate the input to each chip both UgainA/B/C and UoffsetA/B/C must be implemented.
    Gain is not the same as offset. Gain is the factor and offset is the position to factor. Both are required to have accuracy.

See:
image

  1. The value tolerance of the quad input resistor pack R24 on the 6 channel board is 5% which could have enough difference to be observable. The other circuit resistors are not visable on the published diagram so not sure what those tolerences are.
    image

  2. With all the leads/connectors/EMI there is a high probability of inductive/noise/impedence variables which require full calibration to keep the input measurement in spec.

  3. The input sampling is also too low, a single read is not very accurate, several samples should be read and averaged.

@descipher
Copy link

Sorry, needed to include this:
image

@alexruffell
Copy link

@descipher - I realize this board, and likely the chip, are not meant for high accuracy applications and am ok with some error. Also, not all of my equipment is calibrated or at least not recently so my end goal is mostly to make the readings jive for the lack of a better word. There is a parameter I can change to "calibrate" the voltage measurements and I can get only so close because L1 on the 2 boards appear to keep swapping an error that skews the readings.

If you look at the graph in my last post, you will notice that the top trace starts green, then after a reboot trades its place with blue, and then back to green. It doesn't happen at every single reboot, and in fact I point out 3 reboots but the error moved only twice. When this error moves from one input to the other, it messes with my calibration so I find myself having to correct it in the other direction which then gets nullified when it swaps again. Since this swap happens pretty reliably, I am hoping it is a software implementation issue that can be addressed. Resistors with high tolerance values such as the 5% one you mentioned would affect the readings but not cause the error to move from one board to the other, right?

With all the leads/connectors/EMI there is a high probability of inductive/noise/impedence variables which require full >calibration to keep the input measurement in spec.

The best I can do to mitigate this issue is what I show in the picture. The CTs are labeled for the respective circuit branches they will measure and the black box with numbered 3.5mm connectors will be inside the outdoor electrical panel. The gray wires will exit the panel and enter the box you see in the picture through electrical conduit. In other words, I am trying to do a "system calibration" in my office as if it were installed because I won't be able to do one once it is installed outdoors. The gray cables will be shorter but I doubt the reduced resistance will have a significant impact.

When the ESP32 reboots, something happens to the main voltage input on one of the two boards causing them to diverge. In rare occasions whatever happens makes then "agree". Your 1MHz SPI tweak seems to have had an impact as I was never able to get the V readings so close, but many other things may have changed too, so I am not sure.

The input sampling is also too low, a single read is not very accurate, several samples should be read and averaged.

I was planning on implementing a sliding_window_moving_average filter in ESPHome as a way to reduce the impact of outlier readings but those will also be hidden by reducing the decimals to just 1 position once I did all I can do to calibrate this system.

UoffsetA/B/C in not implemented in the code and thus the variation of L1, L2 or L3 voltage circuit value/noise differences >cannot be calibrated. To correctly calibrate the input to each chip both UgainA/B/C and UoffsetA/B/C must be implemented.
Gain is not the same as offset. Gain is the factor and offset is the position to factor. Both are required to have accuracy.

This sounds like something that can be improved in the custom component that supports this chip in ESPHome. I don't have the ability to do so myself. Hopefully someone will contribute it in the future, however I am also guessing that it would not help with the error jumping from one board to the other randomly.

Below is the latest revision of the YAML I am using. The update rate is set to 3s only to make it easier to calibrate the measurements. As you mentioned in another thread I typically would keep it at a higher value but not so high I can't see a relevant change fast enough for it to be useful in troubleshooting.

In regards to the snapshot on Offsets, aren't those what I am changing? See my substitutions section.

# Example config for when jp8-jp11 are all bridged - this connects all the voltage channels and allows for power to be calculated by the meter directly.
# Boards >= v1.4 jp8-jp11 are removed and have all voltage channels connected

# Oven 40A, Dryer 30A, HVAC 25A, HVAC 40A, SubPanel 80A

substitutions:
  devicename: home-energy-meter
  devicename_no_dashes: home_energy_meter
  friendly_devicename: "Home Energy Meter"
  device_description: "Home Energy Meter"
  appliance1: "Dryer"
  appliance2: "Oven"
#  appliance3: "TBD"
  hvac1: "Downstairs HVAC"
  hvac2: "Upstairs HVAC"
# Current Transformers:
# 20A/25mA SCT-006: 11143
# 30A/1V SCT-013-030: 8650
# 50A/1V SCT-013-050: 15420
# 80A/26.6mA SCT-010: 41660
# 100A/50ma SCT-013-000: 27518
# 120A/40mA: SCT-016: 41787
# 200A/100mA SCT-024: 27518
  current_cal_ct1:  '46800' #L1 Main
  current_cal_ct2:  '42480' #L1 Dryer
  current_cal_ct3:  '42480' #L1 Oven
  current_cal_ct4:  '42565' #L2 Oven was 42075
  current_cal_ct5:  '42580' #L2 Dryer
  current_cal_ct6:  '47010' #L2 Main
  current_cal_ct7:  '41660' #L1 CT7
  current_cal_ct8:  '42355' #L1 HVAC DS
  current_cal_ct9:  '42265' #L1 HVAC US
  current_cal_ct10: '42375' #L2 HVAC US
  current_cal_ct11: '42365' #L2 HVAC DS
  current_cal_ct12: '41660' #L2 CT12
# voltage calibrations done by feeding 60Hz sine on seconday input
# and painfully on primary but they match as close as I can get it
  v1_cal: '5018' #5022 1L1
  v2_cal: '5388' #5378 1L2      V2 and V4 match when stable 60Hz sine is fed to them. Do not change independently
  v3_cal: '4998' #5004 2L1 (AO)
  v4_cal: '5362' #5352 2L2 (AO) V2 and V4 match when stable 60Hz sine is fed to them. Do not change independently
  # Interval of how often the wifi info is updated
  update_interval_wifi: "120s"
  # Interval of how often the power is updated
  update_interval_s:   "3s"
  update_interval_s_totals:   "60s"
  v_accuracy: "3"
  a_accuracy: "3"
  w_accuracy: "3"
  kwh_accuracy: "1"

  
esphome:
  name: ${devicename}
  friendly_name: "${friendly_devicename}"
  comment: ${device_description}
  platform: ESP32
  board: nodemcu-32s

external_components:
  source: github://descipher/[email protected]
  components: [ atm90e32 ]
  refresh: 0s


wifi:
  ssid: !secret iot_wifi_ssid
  password: !secret iot_wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "${devicename}"
    password: !secret iot_wifi_password

#Faster than DHCP. Also use if can't reach because of name change
  manual_ip:
    static_ip: 192.168.3.198
    gateway: 192.168.3.1
    subnet: 255.255.255.0
    dns1: 192.168.1.25
    dns2: 192.168.1.36

#Manually override what address to use to connect to the ESP.
#Defaults to auto-generated value. Example, if you have changed your
#static IP and want to flash OTA to the previously configured IP address.
  #use_address: 192.168.3.198

# Enable logging
logger:
  baud_rate: 921600

# Enable Home Assistant API
api: 
#  password: !secret api_pwd

ota:
  password: !secret ota_pwd

web_server:
  port: 80

# Sync time with Home Assistant
time:
  - platform: homeassistant
    id: ha_time

spi:
  clk_pin: 18
  miso_pin: 19
  mosi_pin: 23

text_sensor:
  - platform: wifi_info
    ip_address:
      name: "IP"
      icon: "mdi:ip-outline"
      update_interval: ${update_interval_wifi}
    ssid:
      name: "SSID"
      icon: "mdi:wifi-settings"
      update_interval: ${update_interval_wifi}
    bssid:
      name: "BSSID"
      icon: "mdi:wifi-settings"
      update_interval: ${update_interval_wifi}
    mac_address:
      name: "MAC"
      icon: "mdi:network-outline"
    scan_results:
      name: "Wifi Scan"
      icon: "mdi:wifi-refresh"
      disabled_by_default: true

sensor:
  - platform: wifi_signal
    name: "WiFi Signal"
    update_interval: ${update_interval_wifi}
    device_class: signal_strength

#IC1
  - platform: atm90e32
    id: chip1
    cs_pin: 5
    phase_a:
      voltage:
        name: "L1 V"
        id: ic1Volts
        accuracy_decimals: ${v_accuracy}
      current:
        name: "L1 A"
        id: ct1Amps
        accuracy_decimals: ${a_accuracy}
        # The max value for current that the meter can output is 65.535. If you expect to measure current over 65A, 
        # divide the gain_ct by 2 (120A CT) or 4 (200A CT) and multiply the current and power values by 2 or 4 by uncommenting the filter below
        filters:
          - multiply: 4
      power:
        name: "L1 W"
        id: ct1Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - multiply: 4
          #With no load why do I have to filter out negative W values??
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v1_cal}
      gain_ct: ${current_cal_ct1}
    phase_b:
      current:
        name: "${appliance1} L1 A" #Dryer
        id: ct2Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${appliance1} L1 W" #Dryer
        id: ct2Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v1_cal}
      gain_ct: ${current_cal_ct2}
    phase_c:
      current:
        name: "${appliance2} L1 A" #Oven
        id: ct3Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${appliance2} L1 W" #Oven
        id: ct3Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v1_cal}
      gain_ct: ${current_cal_ct3}
    frequency:
      name: "L1 Hz"
      device_class: frequency
    chip_temperature:
      name: "L1 Chip Temperature"
      id: l1_chip_temperature
      device_class: temperature
    line_frequency: 60Hz
    gain_pga: 1X
    update_interval: ${update_interval_s}
#IC2
  - platform: atm90e32
    id: chip2
    cs_pin: 4
    phase_a:
      current:
        name: "${appliance2} L2 A" #Oven
        id: ct4Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${appliance2} L2 W" #Oven
        id: ct4Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v2_cal}
      gain_ct: ${current_cal_ct4}
    phase_b:
      current:
        name: "${appliance1} L2 A" #Dryer
        id: ct5Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${appliance1} L2 W" #Dryer
        id: ct5Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v2_cal}
      gain_ct: ${current_cal_ct5}
    phase_c:
      #this voltage is only needed if monitoring 2 voltages
      voltage:
        name: "L2 V"
        id: ic2Volts
        accuracy_decimals: ${v_accuracy}
      current:
        name: "L2 A"
        id: ct6Amps
        accuracy_decimals: ${a_accuracy}
        # The max value for current that the meter can output is 65.535. If you expect to measure current over 65A, 
        # divide the gain_ct by 2 (120A CT) or 4 (200A CT) and multiply the current and power values by 2 or 4 by uncommenting the filter below
        filters:
          - multiply: 4
      power:
        name: "L2 W"
        id: ct6Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - multiply: 4
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v2_cal}
      gain_ct: ${current_cal_ct6}
    frequency:
      name: "L2 Hz"
      device_class: frequency
    chip_temperature:
      name: "L2 Chip Temperature"
      id: l2_chip_temperature
      device_class: temperature
    line_frequency: 60Hz
    gain_pga: 1X
    update_interval: ${update_interval_s}

#IC1 AddOn
  - platform: atm90e32
    id: chip3
    cs_pin: 0
    phase_a:
      voltage:
        name: "L1 AO V"
        id: ic3Volts
        accuracy_decimals: ${v_accuracy}
      current:
        name: "CT7 Amps"
        id: ct7Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "CT7 Watts"
        id: ct7Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v3_cal}
      gain_ct: ${current_cal_ct7}
    phase_b:
      current:
        name: "${hvac1} L1 A" #Downstairs
        id: ct8Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${hvac1} L1 W" #Downstairs
        id: ct8Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v3_cal}
      gain_ct: ${current_cal_ct8}
    phase_c:
      current:
        name: "${hvac2} L1 A" #Upstairs
        id: ct9Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${hvac2} L1 W" #Upstairs
        id: ct9Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v3_cal}
      gain_ct: ${current_cal_ct9}
    line_frequency: 60Hz
    gain_pga: 1X
    update_interval: ${update_interval_s}
#IC2 AddOn
  - platform: atm90e32
    id: chip4
    cs_pin: 16
    phase_a:
      current:
        name: "${hvac2} L2 A" #Upstairs
        id: ct10Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${hvac2} L2 W" #Upstairs
        id: ct10Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v4_cal}
      gain_ct: ${current_cal_ct10}
    phase_b:
      current:
        name: "${hvac1} L2 A" #Downstairs
        id: ct11Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "${hvac1} L2 W" #Downstairs
        id: ct11Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v4_cal}
      gain_ct: ${current_cal_ct11}
    phase_c:
      voltage:
        name: "L2 AO V"
        id: ic4Volts
        accuracy_decimals: ${v_accuracy}
      current:
        name: "CT12 Amps"
        id: ct12Amps
        accuracy_decimals: ${a_accuracy}
      power:
        name: "CT12 Watts"
        id: ct12Watts
        accuracy_decimals: ${w_accuracy}
        filters:
          - lambda: !lambda |-
              if (x < 0) return 0.0;
              return x;
      gain_voltage: ${v4_cal}
      gain_ct: ${current_cal_ct12}
    line_frequency: 60Hz
    gain_pga: 1X
    update_interval: ${update_interval_s}

#Appliance1 Watts (Dryer)
  - platform: template
    name: "${appliance1} W" #Dryer
    id: appliance1TotalWatts
    accuracy_decimals: ${w_accuracy}
    lambda: return id(ct2Watts).state + id(ct5Watts).state ;
    unit_of_measurement: "W"
    device_class: power
    update_interval: ${update_interval_s_totals}

#Appliance1 Amps (Dryer)   
  - platform: template
    name: "${appliance1} A" #Dryer
    id: appliance1TotalAmps
    accuracy_decimals: ${a_accuracy}
    lambda: return id(ct2Amps).state + id(ct5Amps).state ;
    unit_of_measurement: "A"
    device_class: current
    update_interval: ${update_interval_s_totals} 

#Appliance1 kWh - (Dryer)
  - platform: total_daily_energy
    name: "${appliance1} kWh" #Dryer
    power_id: appliance1TotalWatts
    accuracy_decimals: ${w_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

#Appliance2 Total Watts (Oven)
  - platform: template
    name: "${appliance2} W" #Oven
    id: appliance2TotalWatts
    accuracy_decimals: ${w_accuracy}
    lambda: return id(ct3Watts).state + id(ct4Watts).state ;
    unit_of_measurement: "W"
    device_class: power
    update_interval: ${update_interval_s_totals}

#Appliance2 Total Amps (Oven)
  - platform: template
    name: "${appliance2} A" #Oven
    id: appliance2TotalAmps
    accuracy_decimals: ${a_accuracy}
    lambda: return id(ct3Amps).state + id(ct4Amps).state ;
    unit_of_measurement: "A"
    device_class: current
    update_interval: ${update_interval_s_totals}

#Appliance2 kWh - (Oven)
  - platform: total_daily_energy
    name: "${appliance2} kWh" #Oven
    power_id: appliance2TotalWatts
    accuracy_decimals: ${kwh_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

#HVAC1 Watts (Downstairs)
  - platform: template
    name: "${hvac1} W" #Downstairs
    id: hvac1TotalWatts
    accuracy_decimals: ${w_accuracy}
    lambda: return id(ct8Watts).state + id(ct11Watts).state ;
    unit_of_measurement: "W"
    device_class: power
    update_interval: ${update_interval_s_totals}

#HVAC1 Amps (Downstairs)  
  - platform: template
    name: "${hvac1} A" #Downstairs
    id: hvac1TotalAmps
    accuracy_decimals: ${a_accuracy}
    lambda: return id(ct8Amps).state + id(ct11Amps).state ;
    unit_of_measurement: "A"
    device_class: current
    update_interval: ${update_interval_s_totals} 

#HVAC1 kWh (Downstairs)
  - platform: total_daily_energy
    name: "${hvac1} kWh" #Downstairs
    power_id: hvac1TotalWatts
    accuracy_decimals: ${w_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

#HVAC2 Watts (Upstairs)
  - platform: template
    name: "${hvac2} W" #Upstairs
    id: hvac2TotalWatts
    accuracy_decimals: ${w_accuracy}
    lambda: return id(ct9Watts).state + id(ct10Watts).state ;
    unit_of_measurement: "W"
    device_class: power
    update_interval: ${update_interval_s_totals}

#HVAC2 Amps (Upstairs)  
  - platform: template
    name: "${hvac2} A" #Upstairs
    id: hvac2TotalAmps
    accuracy_decimals: ${a_accuracy}
    lambda: return id(ct9Amps).state + id(ct10Amps).state ;
    unit_of_measurement: "A"
    device_class: current
    update_interval: ${update_interval_s_totals} 

#HVAC2 kWh (Upstairs)
  - platform: total_daily_energy
    name: "${hvac2} kWh" #Upstairs
    power_id: hvac2TotalWatts
    accuracy_decimals: ${kwh_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

#Watts - L1 & L2
  - platform: template
    name: "L1 & L2 W"
    id: l1l2Watts
    accuracy_decimals: ${w_accuracy}
    lambda: return id(ct1Watts).state + id(ct6Watts).state ;
    unit_of_measurement: "W"
    device_class: power
    update_interval: ${update_interval_s_totals}
    
#Amps - L1 & L2
  - platform: template
    name: "L1 & L2 A"
    id: l1l2Amps
    accuracy_decimals: ${a_accuracy}
    lambda: return id(ct1Amps).state + id(ct6Amps).state ;
    unit_of_measurement: "A"
    device_class: current
    update_interval: ${update_interval_s_totals}  

#Volts - L1 & L2 - Average
  - platform: template
    name: "L1 & L2 V (Avg)"
    id: l1l2Volts
    accuracy_decimals: ${v_accuracy}
    lambda: return (id(ic1Volts).state + id(ic2Volts).state)/2;
    unit_of_measurement: "V"
    device_class: voltage
    update_interval: ${update_interval_s_totals}  

#kWh - L1
  - platform: total_daily_energy
    name: "L1 kWh"
    power_id: ct1Watts
    accuracy_decimals: ${kwh_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

#kWh - L2
  - platform: total_daily_energy
    name: "L2 kWh"
    power_id: ct6Watts
    accuracy_decimals: ${kwh_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

#kWh - Whole House (L1 + L2)
  - platform: total_daily_energy
    name: "L1 & L2 kWh"
    power_id: l1l2Watts
    accuracy_decimals: ${kwh_accuracy}
    filters:
      - multiply: 0.001
    unit_of_measurement: "kWh"
    device_class: energy
    state_class: total_increasing

button:
  - platform: safe_mode
    name: "Restart (Safe Mode)"
    entity_category: diagnostic

  - platform: restart
    name: "Restart"
    entity_category: diagnostic

  - platform: template
    name: "IC Setup"
    on_press:
      then:
        lambda: !lambda |-
          id(chip1).setup();
          id(chip2).setup();
          id(chip3).setup();
          id(chip4).setup();

@descipher
Copy link

@alexruffell

There are some code issues that I suspect need to be addressed.

The code does a SoftReset reg write and fails to wait the required time b4 writing out other reg values.

  this->write16_(ATM90E32_REGISTER_SOFTRESET, 0x789A);    // Perform soft reset
  this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x55AA);  // enable register config access
  this->write16_(ATM90E32_REGISTER_METEREN, 0x0001);      // Enable Metering
  if (this->read16_(ATM90E32_REGISTER_LASTSPIDATA) != 0x0001) {
    ESP_LOGW(TAG, "Could not initialize ATM90E32 IC, check SPI settings");
    this->mark_failed();
    return;
  }

There are no checks to see if the subsequent write is ack'd after the reset. This can be why you are seeing differing results. The ATM90E32 has a verification method where all that is needed is a compare to the LastSPIData and the Buffer value. This should be changed so that the next write operation is verified after reset and it needs to wait 5ms + 1ms which is the minimum + 1 which applieas to a powered on state.

image

I have increased the SPI clock to 2MHz and see no issues.

I have also added the delay and check in the code of my branch named component.atm90e32

  this->write16_(ATM90E32_REGISTER_SOFTRESET, 0x789A);    // Perform soft reset
  delay(6);                                               // Wait the minimum time after reset
  this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x55AA);  // enable register config access
    if (this->read16_(ATM90E32_REGISTER_LASTSPIDATA) != 0x55AA) {
    ESP_LOGW(TAG, "Could not initialize ATM90E32 IC, check SPI settings");
    this->mark_failed();
    return;
  }

@CircuitSetup
Copy link
Owner

Do you think adding the ability to set the SPI clock rate from the external yaml config function has value?

I don't think this would be necessary since you'd want the fastest rate possible. Slowing it down would only produce slightly more less accurate results.

Did some more research on the calibrations when they dialogue this Ub=Uc=Un, Ua=0, Ia=0 you have to have a 0 input level for both the current and the voltage in order to set a valid voltage calibration on Ua. The design of the circuit we use does not allow this so I need to make it optional in the code for any design that could use it.

I honestly forgot about this for the voltage. The only way around it is to power the ESP32 separately on startup. I think it is more applicable to the current channels anyway.

@CircuitSetup
Copy link
Owner

@descipher circling back around to this as I have some users reporting some issues with, I think, the current offset calibration, here: #179
I can't reproduce this myself, and was wondering if you experienced anything similar with the current offset registers - rather, if they're doing anything at all? I've never been able to get them to do anything meaningful either way.

@descipher
Copy link

@CircuitSetup I have not observed that issue. However after looking at the code I'm not sure why this was overlooked but we need a bool variable for controlling the offset calibration function at will so it can be sampled and saved to be use at startup allowing the user to run a calibration cycle with no voltage input and then use it at startup vs collect it at startup, I will do a PR for that function at some point to correct it. So in this case it will not cause the effect seen on #179, I think that's the component accuracy variance of the input RMS divider resistors. The offset calibration input variance will be very small and almost not observable. It's capturing the delta values between many voltage input samples in a very short period during startup. The only way this delta could be significant is if the local voltage is subject due to very large current load changes which may drop or increase the sample voltage during that very short sample period.

The comment on the larger value input capacitor is not relevant, a larger capacity would simply smooth the RMS ripple more resulting in better accuracy vs less.

I will do some tests to see what that delta looks like with non 0 inputs, I don't expect much and it is easy to log that activity for validation purposes.

@CircuitSetup
Copy link
Owner

@CircuitSetup I have not observed that issue. However after looking at the code I'm not sure why this was overlooked but we need a bool variable for controlling the offset calibration function at will so it can be sampled and saved to be use at startup allowing the user to run a calibration cycle with no voltage input and then use it at startup vs collect it at startup, I will do a PR for that function at some point to correct it.

That's what I was thinking. It doesn't make sense to recalculate the offset at every startup. Especially if the CTs are reading something.

So in this case it will not cause the effect seen on #179, I think that's the component accuracy variance of the input RMS divider resistors.

The variance they're seeing is on the current inputs, though. It still could be the variance in components - the burden resistors, 22ohm (0.1%), and 100ohm array resistor for 2 of the 3 inputs on each chip, then single 100ohm resistors for the remaining input (both 1%).

The offset calibration input variance will be very small and almost not observable. It's capturing the delta values between many voltage input samples in a very short period during startup. The only way this delta could be significant is if the local voltage is subject due to very large current load changes which may drop or increase the sample voltage during that very short sample period.

I have yet to see the offset registers affect anything, but in theory, shouldn't it correct for non 0 values when readings should be 0? The guidance in the application note isn't very descriptive.

The comment on the larger value input capacitor is not relevant, a larger capacity would simply smooth the RMS ripple more resulting in better accuracy vs less.

That's what I figured, starting at a lower frequency.

I will do some tests to see what that delta looks like with non 0 inputs, I don't expect much and it is easy to log that activity for validation purposes.

Speaking of, I was trying to see what was getting put into the offset registers by setting the log level to VERY_VERBOSE, and watching the logs while resetting everything, but was unable to see anything coming from here: https://github.com/esphome/esphome/blob/1f3754684adccccc54a4795d8a685d13ba59e352/esphome/components/atm90e32/atm90e32.cpp#L274
Any ideas with that? Is the log initializing after the startup code?

Thanks again for your insight and help!

@SzymonSlupik
Copy link

So in this case it will not cause the effect seen on #179, I think that's the component accuracy variance of the input RMS divider resistors.

The variance they're seeing is on the current inputs, though. It still could be the variance in components - the burden resistors, 22ohm (0.1%), and 100ohm array resistor for 2 of the 3 inputs on each chip, then single 100ohm resistors for the remaining input (both 1%).

IMO this cannot be the resistors, as the residual current values which are reported change every time the meter is reset. Resistor values do not fluctuate that much. It rather looks like an uninitialized variable / register.

@descipher
Copy link

@CircuitSetup

      Any ideas with that? Is the log initializing after the startup code?

I think you are correct, the setup priority is at level "IO" so that is likely well B4 the log component is started. We can do strait printf() calls to catch that output for debugging that element.

@descipher
Copy link

descipher commented Jul 26, 2024

@SzymonSlupik

 The issue is these offsets change (are different) after each reset. So I'm not sure how CT calibration has anything to do with this, especially as the offsets fluctuate (after each reset) with no CTs plugged in.

You need to have the CTs in circuit to do any calibration or diagnosis, without them in place it behaves like an antenna, possibly injecting random noise into the ATM90 DAC's.

@SzymonSlupik
Copy link

OK I shorted all the inputs on the add-on board and it still reports non-zero current:
image
image
It still reports completely different values after reset
image

The values do not change until I reset so this is the reset / initialization procedure which introduces these ghost offsets.

@descipher
Copy link

@SzymonSlupik Looks like normal noise variance to me.

If you would like to rule out the offset calibration I have commented out that function in this branch. I do not have a test rig available from where I am ATM so feel free to run this as a test on that rig you have connected.

external_components:
  - source: github://descipher/[email protected]
    components: [ atm90e32 ]
    refresh: 0s

@SzymonSlupik
Copy link

Thanks, I’ll give it a try, probably tomorrow.

Anyway 10 Watts is hardly a noise :) It is a lot of power.

I’ll report back once I have the new results.

@descipher
Copy link

@SzymonSlupik If you calculate the accuracy of the Atmel published 0.1% accuracy you will find that its well within spec.

0-100A = 0-24000W at 240V

.001 * 24000 = 24W so basically you are seeing the noise floor levels, when we correct the offset 0 level calibration application function it will cancel that noise to some degree when its at 0 current and voltage levels. That's all the offset calibration is supposed to correct.

@SzymonSlupik
Copy link

Yes I agree with what you’re saying.

I’m looking for the truth though. Which means where these discrepancies come from.

look, this is not noise, as these values do not change, only when you reset the circuit. And some channels show pure zero while some show random (but static) numbers.

Let’s find the root cause.

I’ll have some time tomorrow to experiment more.

@descipher
Copy link

@SzymonSlupik I'm not certain what you are observing at this point.

 IMO this cannot be the resistors, as the residual current values which are reported change every time the meter is reset. 
 Resistor values do not fluctuate that much. It rather looks like an uninitialized variable / register.

If the reported values are now static then it comes back to the in circuit component tolerances. e.g. voltage divider resistors etc. as described from my original assessment however your observe defined them as changing with every restart. Is that changing output no longer the case with a shunt on the CT input circuit? Please validate and confirm what the current observe is.

Power cycles can result in 0 level changes based on the ADC + noise at the time of init within the ATM90E32 IC, restarts would not normally do that with the exception of my comment on the load based change varying the calibration sample which should be gone with the commented code during real input sampling and with the shunts however the voltage input is still an unknown.

We will need to know what phases have a voltage reference since those and the current channels should be 0 during the calibration sampling for us to confirm that offset calibration works as expected. Your screen shot shows 4 phases having a reference however I see only one AC input. Are those all shunted to the same AC input in the screen shot? If so you need to adjust your gain to bring them in sync based on an actual external voltage meter check. Based on the voltage reported it certainly has some variation in component tolerance and that's a possible reason for variance as well because each voltage phase offset calibration is done separately but the basis would be not be correct using an invalid voltage gain setting.

@CircuitSetup
Copy link
Owner

@SzymonSlupik I'm not certain what you are observing at this point.

If the reported values are now static then it comes back to the in circuit component tolerances. e.g. voltage divider resistors etc. as described from my original assessment however your observe defined them as changing with every restart. Is that changing output no longer the case with a shunt on the CT input circuit? Please validate and confirm what the current observe is.

That's the issue - he's saying regardless of the shunt, the current values, that should be 0 anyway, change on reset.

We will need to know what phases have a voltage reference since those and the current channels should be 0 during the calibration sampling for us to confirm that offset calibration works as expected. Your screen shot shows 4 phases having a reference however I see only one AC input.

On the meters, the 3 voltage channels are tied to the 1 input internally. The 4 is coming from the main board + add-on board's 4 ATM90E32's.

@SzymonSlupik
Copy link

If you would like to rule out the offset calibration I have commented out that function in this branch. I do not have a test rig available from where I am ATM so feel free to run this as a test on that rig you have connected.

external_components:
  - source: github://descipher/[email protected]
    components: [ atm90e32 ]
    refresh: 0s

Yep. It is all good now - just the noise at the 1W level. And this is regardless if the inputs are shorted or open.

image

Now we need to figure out what goes wrong in the calibration. And why....

@descipher
Copy link

descipher commented Jul 27, 2024

@SzymonSlupik Thanks for the bench work, this gives us the key piece of info needed. We have non zero voltages on all inputs at the time we are running offset calibrations. The current is zero on all inputs. When we look at the ATM90E32 application guide details, they give us some insight identifying the observed error state.

Section 4.2.5
Every phase’s voltage/current offset calibration should be preceded individually. Take phase A for example, the signal
source is: Ub=Uc=Un, Ua=0, Ia=0. The calibration flow of voltage/current offset is as below:
1.Read measurement registers (32 bits). It is suggested to read several times to get the average value;
2.Right shift the 32-bit data by 7 bits (ignore the lowest 7 bits);
3.Invert all bits and add 1 (2’s complement);
4.Write the lower 16-bit result to the offset register

Based on this formula Ub=Uc=Un, Ua=0, Ia=0, Ua = channel a voltage, Ia = channel a current
Ub = Uc = Un means all voltages must be equal to the calibration channel. So technically we could run this on a non-zero but it's not prescribed based on Ua=0

The code performs steps 1,2,3,4 correctly but we are not at 0 volts on any channel and that's the issue here. As previously indicated we must take the calibration code out of the startup area and use a bool option flag to allow any user to meet the requirements for calibration where they can power ATM90E32 separately without AC line voltage and then capture the input noise offset values during a 0 volts sampling. Then those values can be stored in flash and applied during startup if desired.

We could also just remove the capability since the application guide indicates its not essential for our use case.
I prefer the former where the capability is available but not applied in general.

@descipher
Copy link

descipher commented Jul 27, 2024

@SzymonSlupik Did you happen to do some calibration for the voltage gain for each phase? I see some variance in the screenshot. You should be able to bring those to +-0.02 volts

@SzymonSlupik
Copy link

ATM I have just one power adapter. Plan to move to three once we have the things sorted out here (which I believe now we almost have - thanks for the explanation). BTW it is a bit weird the chips show different voltages, but I understand this is due to the tolerances of the components and that is why separate calibration for each chip is needed.

@CircuitSetup
Copy link
Owner

CircuitSetup commented Jul 27, 2024

BTW it is a bit weird the chips show different voltages, but I understand this is due to the tolerances of the components and that is why separate calibration for each chip is needed.

Not just each chip, but there are 3 voltage inputs PER chip that could potentially be different. This is why in the ESPHome config there is a gain_voltage: per phase. You don't need this many voltage references, but because of how the chips are made, each may have a slightly different gain_voltage. What you're currently viewing is the phase_a of each chip. If you want to check the accuracy for this, add:

      voltage:
        name: ${disp_name} Volts [...]
        id: ic[x]Volts[A/B/C]
        accuracy_decimals: 2

for each phase, replacing things between brackets to differentiate

Luckily each voltage phase per chip uses the same on-board voltage dividers (2 sets for each board), so they should be really close, regardless. Any difference you see may just be in the timing of actually getting the data from the chip.

@SzymonSlupik
Copy link

Oh oh thanks for explaining that. I was wondering why the phase voltages were nested under each chip definition - my understanding was the phase voltages were representing the independent power supplies. Now it all makes sense. I guess I'll do that calibration once I move the meter to the electrical cabinet where I have access to all 3 phases and will implement the proper 3-phase configuration.

@descipher
Copy link

I think it would be beneficial to write up an overall calibration guide for everyone that explains how it works. I will place it on https://github.com/gelidusresearch/device.docs when completed over the next few weeks after I create a PR for the offset calibration issue. It will apply to any ATM90E32/ESPHome based devices.

@descipher
Copy link

descipher commented Aug 7, 2024

I have updated the code to handle optional offset calibration now at:

external_components:
  - source: github://pr#7228
    components:
      - atm90e32
    refresh: 0s

The code adds the ability to run a calibration or clear a calibration for both the voltage and current sensors. The calibration must be run when all inputs are 0. I have not tested it yet since I am not near my test bench to do it but will be in the coming weeks.

Feel free to test it and validate.

The calibration feature is fully optional and this is the basic YAML to use it:

substitutions:
  disp_name: PM1
  update_time: 15s
  phase_a_current_cal: '15270'
  phase_c_current_cal: '15270'
  phase_a_voltage_cal: '4470'
  phase_c_voltage_cal: '4671'

sensor:
  - platform: atm90e32
    cs_pin: 5
    id: chip1
    phase_a:
      voltage:
        name: ${disp_name} L1 Volts
        accuracy_decimals: 2
      current:
        name: ${disp_name} CT1 Amps
        id: "ct1Amps"
      power:
        name: ${disp_name} CT1 Watts
        accuracy_decimals: 1
        id: "ct1Watts"
      gain_voltage: ${phase_a_voltage_cal}
      gain_ct: ${phase_a_current_cal}
    phase_c:
      voltage:
        name: ${disp_name} L2 Volts
        accuracy_decimals: 2
      current:
        name: ${disp_name} CT2 Amps
        id: "ct2Amps"
      power:
        name: ${disp_name} CT2 Watts
        accuracy_decimals: 1
        id: "ct2Watts"
      gain_voltage: ${phase_c_voltage_cal}
      gain_ct: ${phase_c_current_cal}
    frequency:
      name: ${disp_name} Freq
    line_frequency: 60Hz
    gain_pga: 2x
    update_interval: ${update_time}
    enable_offset_calibration: True
	
  - platform: atm90e32
    cs_pin: 4
    id: chip2
    phase_a:
      voltage:
        name: ${disp_name} L3 Volts
        accuracy_decimals: 2

      current:
        name: ${disp_name} CT3 Amps
        id: "ct3Amps"
      power:
        name: ${disp_name} CT3 Watts
        accuracy_decimals: 1
        id: "ct3Watts"
      gain_voltage: ${phase_a_voltage_cal}
      gain_ct: ${phase_a_current_cal}
    phase_c:
      voltage:
        name: ${disp_name} L4 Volts
        accuracy_decimals: 2
      current:
        name: ${disp_name} CT4 Amps
        id: "ct4Amps"
      power:
        name: ${disp_name} CT4 Watts
        accuracy_decimals: 1
        id: "ct4Watts"
    line_frequency: 60Hz
    current_phases: 2
    gain_pga: 2X
    update_interval: 60s

button:
  - platform: atm90e32
    id: chip1
    run_offset_calibration:
      name: "Chip1 - Run Offset Calibration"
    clear_offset_calibration:
      name: "Chip1 - Clear Offset Calibration"

The use of id is only required when more than one instance of the atm90e32 component is defined.
With multiple chips you can have calibration enabled or not using:

#Sensor Instance for chip1
sensor:
  - platform: atm90e32
    id: chip1
    enable_offset_calibration: True

then based on the atm90e32_id you can add the run and clear buttons:

#Sensor Instance for chip1
sensor:
  - platform: atm90e32
    id: chip1
`
#Button Instance matching chip1
button:
  - platform: atm90e32
    id: chip1
    run_offset_calibration:
      name: "Chip1 - Run Offset Calibration"
    clear_offset_calibration:
      name: "Chip1 - Clear Offset Calibration"
17:28:13 | [I] | [atm90e32.button:010] | Running offset calibrations, Note: CTs and ACVs must be 0 during this process...
17:28:13 | [I] | [atm90e32:165] | PhaseA Vo=22260 PhaseB Vo=64608 PhaseC Vo=22751
17:28:13 | [I] | [atm90e32:167] | PhaseA Io=46490 PhaseB Io=12288 PhaseC Io=39117


17:31:58 | [I] | [atm90e32.button:015] | Offset calibrations cleared.
17:31:58 | [I] | [atm90e32:192] | PhaseA Vo=    0 PhaseB Vo=    0 PhaseC Vo=    0
17:31:58 | [I] | [atm90e32:194] | PhaseA Io=    0 PhaseB Io=    0 PhaseC Io=    0

@CircuitSetup
Copy link
Owner

CircuitSetup commented Aug 8, 2024

I have updated the code to handle optional offset calibration now at:

Thanks so much for doing all of this. It looks perfect!

btw, I was looking further into the calibration procedure, and noted something in the eval kit documentation (https://www.microchip.com/en-us/development-tool/ATM90E32AS-DB), located here: https://ww1.microchip.com/downloads/aemDocuments/documents/SE/ProductDocuments/BoardDesignFiles/AutoCalibration_Ver1.0.zip
Under Auto Calibration > User Guide

A few things:

  1. Offsets for voltage and current aren't even mentioned.
  2. Power offset (P & Q) is mentioned and includes: "Read phase mean active power registers (32 bits). It is suggested to read 8 times to get the averaged value and each register reading interval shall be at least 320ms (one register refresh period), 500ms recommended" The application note doesn't only says reading the register "several times", let alone evenly spaced over 4 seconds.

Note that the right shift of 7 bits isn't done with the power offsets, otherwise the process is the same.

I'm wondering if the power offset, with voltage connected and no power going through the CTs, would be more useful than the voltage and current offset, or if they're needed at all.

FYI, I looked in the demo firmware code for any kind of offset function and didn't see anything other then writing default values to the offset registers. The only other thing I saw was the phase angle offsets. This would be impossible to do without external equipment, though, as it needs a constant power factor.

@descipher
Copy link

@CircuitSetup That information is excellent and just what we needed, good find and thanks for this.
It defines everything needed to perform any type of calibration, even the code is provided with supporting comments. I wish we found this long ago. It explains how its done and with what parameters/inputs. We are currently doing only one part of the calibration and need to add the complete set for it to work as prescribed. This will take some time so we need to do it in 2 stages. 1) is to turn off the startup calibration code which is already completed in the PR. 2) is to complete the calibration functions which the code can do with changes. It's optional and requires known inputs, thus its only advisable when one needs very high accuracy that reaches comercial levels. Calibration also requires that the values are saved in flash, this is already defined in the PR and just needs more structure fields added.
@jesserocks has done some of the initial review work to bring the PR in line with the current coding practice, thanks Jesse!

Some of the methods in this code require that the manual V and I gain and scale settings we current use would be ignored when calibration is enabled. Those elements would be substituted with the calculated output of the calibration process.

Some of the methods do require special equipment so this needs to be evaluated further with regard to viability. We may only do a basic adoption based on that complexity level.

Demo code clip
FYI AFE = Analog Front End

//=========================================================
//decription	::	Calibration AFE
//function		::	Calibration_AFE
//Input			::	afe_modify
//Output		::	1:success,2:failed
//call			::	Rd_Measure_Parameter,Calculate_V_I_Mul_Scal
//					Write_Calibration_Reg,sin_angle,cos_angle
//effect		::	none
//=========================================================
uint32_t	Calibration_AFE( AFE_MODIFY_Str	afe_modify )
{
    __NOP();
    __NOP();
    afe_modify.phase_id = Phase_A;

    //----------calibration voltage----------------------------------
    switch ( afe_modify.phase_id )
    {
        case	Phase_A:
        {
            k = afe_modify.phase_a.urms;// * 256;		//1 LSB=0.01V/256
            val	= ( uint64_t )Read_AFE_1Reg( Urms );

            val = Calculate_V_I_Mul_Scal( val, afe_modify.conf.u_scale );
            gain = Read_AFE_1Reg( ( uint16_t )Ugain );
            m = ( uint64_t )k;
            m = ( ( m * gain ) / val );
            gain = ( uint32_t )( m );
            if ( Write_Calibration_Reg( Ugain, gain ) == 2 )
            {
                return ( 2 );
            }
        }
        break;
    }
    //----------calibration current----------------------------------
    switch ( afe_modify.phase_id )
    {
        case	Phase_A:
        {
            k = afe_modify.phase_a.irms;// * 256;		//1 LSB=0.001A/256
            val	= ( uint64_t )Read_AFE_1Reg( Irms );

            val = Calculate_V_I_Mul_Scal( val, afe_modify.conf.i_scale );
            gain = Read_AFE_1Reg( ( uint16_t )IgainL );
            m = k;
            m = ( ( m * gain ) / val ); //VAFE.uiscal.BIT.I_SCALE;
            gain = ( uint32_t )( m );
            if ( Write_Calibration_Reg( IgainL, gain ) == 2 )
            {
                return ( 2 );
            }
        }
        break;
    }

    //----------calibration Lgain------------------------------------
    Delay_N_ms( 1000 );
    switch ( afe_modify.phase_id )
    {
        case	Phase_A:
        {
/*
            k = afe_modify.phase_a.irms * afe_modify.phase_a.urms;      // *100*1000
            psin = ( float )afe_modify.phase_a.angle;
            psin = ( 3.14159265 * psin ) /(180 * 100);
            psin = cos(psin);
            psin = k * psin;
            val	= ( uint64_t )Read_AFE_1Reg( Pmean );
            val = Calculate_V_I_Mul_Scal( val, VAFE.ulscal.BIT.I_SCALE );
            val = Calculate_V_I_Mul_Scal( val, VAFE.ulscal.BIT.U_SCALE );
            val = ( val & 0x7FFF );
            val	*= 100000;

            l_a = ((float)val - psin) / psin;
            l_a = ( -l_a ) / ( 1 + l_a );
            l_a = l_a * 0x8000 * 8;

            gain = (int32_t)l_a;
*/
            k   = Read_AFE_1Reg( ( uint16_t )Ugain );
            k   = Calculate_V_I_Mul_Scal( k, VAFE.ulscal.BIT.U_SCALE );

            gain= Read_AFE_1Reg( ( uint16_t )IgainL );
            gain= Calculate_V_I_Mul_Scal( gain, VAFE.ulscal.BIT.I_SCALE );
            
            val = Read_AFE_1Reg(PLcosntL);
            val += (uint32_t)(Read_AFE_1Reg(PLconstH) << 16);

            l_a = ( k * gain * val)/((float)838860800 * 1406250) - 1.0;
            l_a = l_a * 0x8000;
            
            gain = (int32_t)l_a;

            if(gain & 0x8000)
            {
                gain = 1 + ~gain;
            }
            gain &= 0xFFFF;

            if ( Write_Calibration_Reg( Lgain, gain ) == 2 )
            {
                return ( 2 );
            }
        }
        break;
    }
    //----------calibration angle------------------------------------
    switch ( afe_modify.phase_id )
    {
        case	Phase_A:
        {
            psin = ( float )afe_modify.phase_a.angle;
            psin = psin / 100;
            m = ( uint64_t )Read_AFE_1Reg( Pmean );            
            if ( m & 0x8000 )
            {
                m = ~m;
                m += 1;
                m &= 0x7FFF;
            } 
            val = ( uint64_t )Read_AFE_1Reg( Smean );
            l_a = ( float )( m ) / val;
            l_a = acos( l_a );
            l_a = (l_a * 180) / 3.14159265;
            pcos = l_a;
            
            // l_a = ( l_a - psin ) * 113.778;
            l_a = (( l_a - psin ) * 60 ) / 0.52734375;

            gain = (int32_t)l_a;
            if(gain & 0x8000)
            {
                gain = 1 + ~gain;
            }
            gain &= 0xFFFF;

            if ( Write_Calibration_Reg( Lphi, gain ) == 2 )
            {
                return ( 2 );
            }
        }
        break;
    }

    gain = Read_AFE_1Reg( ( uint16_t )CS1 );
    Write_Calibration_Reg( CS1, gain );
    gain = Read_AFE_1Reg( ( uint16_t )CS2 );
    Write_Calibration_Reg( CS2, gain );

    return ( 1 );
}
//=========================================================
//decription	::	calibrate meter process
//function		::	Cal_Meter_Proc
//Input			::	regaddr,value
//Output		::	1:success,2:failed
//call			::	write_AFE_1Reg,EEPROMwriter
//effect		::	none
//=========================================================
void	Cal_Meter_Proc( void )
{
    switch ( VAFE_MODIFY.phase_id )
    {
        case	Phase_A:
        {
            VAFE_MODIFY.phase_a.urms = bcd_4byte_to_hex( VAFE_MODIFY.phase_a.urms );
            VAFE_MODIFY.phase_a.irms = bcd_4byte_to_hex( VAFE_MODIFY.phase_a.irms );
            VAFE_MODIFY.phase_a.angle = bcd_4byte_to_hex( VAFE_MODIFY.phase_a.angle );
        }
        break;
        default:
            break;
    }
    Calibration_AFE( VAFE_MODIFY );
}

//=========================================================
//decription	::	configure AFE
//function		::	Configure_AFE
//Input			::	afe_modify
//Output		::	1:success,2:failed
//call			::	Write_Calibration_Reg,Export_Calbration_Reg_To_Eprom
//effect		::	none
//=========================================================
uint32_t	Configure_AFE( CONF_Str	afe_conf )
{
    uint32_t	i;
    uint64_t	m;
    //------------configure default value----------------------------
    Write_Calibration_Reg( CalStart, Calibration_Status );
    Write_Calibration_Reg( AdjStart, Calibration_Status );

    Write_Calibration_Reg( Ugain, 0x6720 );
    Write_Calibration_Reg( IgainL, 0x7A13 );
    Write_Calibration_Reg( Lgain, 0x0000 );
    Write_Calibration_Reg( Lphi, 0x0000 );

    Write_Calibration_Reg( Uoffset, 0x0000 );
    Write_Calibration_Reg( IoffsetL, 0x0000 );

    Write_Calibration_Reg( PoffsetL, 0x0000 );
    Write_Calibration_Reg( QoffsetL, 0x0000 );

    Write_Calibration_Reg( MMode, 0x7422 );	//MMode

    //Export_Calbration_Reg_To_Eprom();		//write default to eeprom

    i = bcd_4byte_to_hex( afe_conf.pstart );
    i = i * 10000;			//1 LSB =0.00032 W
    i = i >> 5;
    Write_Calibration_Reg( PStartTH, ( uint16_t )i );	//
    i = bcd_4byte_to_hex( afe_conf.qstart );
    i = i * 10000;			//1 LSB =0.00032 W
    i = i >> 5;
    Write_Calibration_Reg( QStartTH, ( uint16_t )i );	//
/*
    i = bcd_4byte_to_hex( afe_conf.sstart );
    i = i * 10000;			//1 LSB =0.00032 W
    i = i >> 5;
    Write_Calibration_Reg( P3_SStartTh, ( uint16_t )i );	//
*/
    VAFE.ulscal.BIT.I_SCALE = ( uint16_t )( VAFE_MODIFY.conf.i_scale );
    VAFE.ulscal.BIT.U_SCALE = ( uint16_t )( VAFE_MODIFY.conf.u_scale );
    EEPROMwriter( E2_AT90E2x_UIScal, ( uint8_t * )( &( VAFE.ulscal.HFWORD ) ), 2 );

    //-----------configure plconst-----------------------------------
    i = afe_conf.mc;
    if ( i == 0 )
    {
        i = 3200;
    }
    m = (uint64_t)838860800 * 24 * 3 * 262 / 4;
    m = m / ( i * 220 * 5 );

    i = ( uint32_t )( m );
    Write_Calibration_Reg( PLconstH, ( uint16_t )( i >> 16 ) );
    Write_Calibration_Reg( PLcosntL, ( uint16_t )( i & 0xFFFF ) );

    __NOP();
    __NOP();

    return ( 1 );
}

@CircuitSetup
Copy link
Owner

@descipher no problem! I'm glad you're finding that useful.

I didn't think the calibration functions were as relevant, since they're basically doing what is outlined in the calibration instructions, but slightly more automated. If you're up for integrating them in some way, awesome! They would just need a way for the user to input constants for the voltage and current.

FYI, the functions you copied above are for the ATM90E26 single phase IC. Take a look in the \AutoCalibration_Ver1.0\Auto Calibration\Firmware\ATM90E3x+Calibration+SAM4L\src\at90e_xx folder

@descipher
Copy link

@CircuitSetup Yes totally agree the demo is targeting large volume automated calibrations which we do manually. The specific information I found useful was the clarity it provides in the code for using this noise floor offset tuning:

image

The code details calibration of the power offsets. I was doing is just the voltage/current offset to account for noise levels. This is something we can do without a special rig providing source signals. How much value it has is depends on how much DC noise is present within the AFE circuits.

The clarity is here with the conformations of what the inputs should be:
voltage and current = Ub=Uc=Un, Ua=0, Ia=0
power = Ua=Ub=Uc=Un, Ia=0

The code confirms the use of those formulas.

In our case we need only to zero all input values and read the noise offset. In other words we are doing the right calibration optionally now. We can go further for those that would like comercial accuracy levels but not in this round of changes. It's a significant effort more so on the bench validations.

@CircuitSetup
Copy link
Owner

Right, got it.

How much value it has is depends on how much DC noise is present within the AFE circuits.

I'm wondering if the power offset registers (Poffset, Qoffset) have more to do with the random low and negative values that are seen with no current load, than the voltage/current ones (Uoffset, Ioffset). Especially since the demo doc doesn't even mention them.

@descipher
Copy link

I'm wondering if the power offset registers (Poffset, Qoffset) have more to do with the random low and negative values that are seen with no current load, than the voltage/current ones (Uoffset, Ioffset). Especially since the demo doc doesn't even mention them.

Only way to know is by checking it, I can add that calibration testing code to see what impact it has when I do the bench work for the current PR.

@CircuitSetup
Copy link
Owner

CircuitSetup commented Aug 21, 2024

It looks like the fix for the offset calibration of voltage and current was just merged. Thanks @descipher!
esphome/esphome#7228

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants