Question about data sampling

[elyorkhakimov]

Hi, I read through Microchip ATM90E32AS datasheet and the respective ESPHome page, but can't seem to understand how this energy meter is sampling data. I am currently running 6 channel meter (v1.4 rev1) with only 4 channels in use (waiting for more CT Clamps for the remaining 2 channels).

Question is really about the values that are send via MQTT.
In yaml file update_interval is set to 10 seconds. Does this mean that meter reports instant value of Voltage, Wattage, Current, Power factor, etc, at a given instant "t", and then after 10 seconds reports the instant value at "t+10sec" ? What about Wh readings?

Or, rather, does the meter sample with a specific sampling rate (what is it?) values for each of the sensors and takes an average of all values over the configured 10 seconds?

If there is a better place to ask these questions, please let me know and I will move this discussion there.

[CircuitSetup]

That's a good question! The ATM90E32AS can write to registers several times a second. But from what I understand this can only happen if the register is read first. I have to double check this for the energy registers. I know this is the case for the cumulative energy registers (which aren't used in ESPHome anyway).

Either way, with a 10 second interval it would either be taking the energy reading at that moment or it would be reading the value from 10 seconds prior, then allowing a new value to be written. There is probably a way to configure an average in ESPHome if you wanted to do that, but I haven't looked into this yet.

If you want to calculate watt hours, this can be done in ESPHome as well.

[elyorkhakimov]

After more research, I came to conclusion that the ESPHome reports instantaneous sensor values via MQTT messages.

I see there are registers in esphome/components/atm90e32/atm90e32_reg.h that is forked here: esphome/esphome@dev...CircuitSetup:dev ; but the energy sensors do not seem to be implemented in ESPHome.

/* ENERGY REGISTERS */
static const uint16_t ATM90E32_REGISTER_APENERGYT = 0x80;  // Total Forward Active
static const uint16_t ATM90E32_REGISTER_APENERGYA = 0x81;  // A Forward Active
static const uint16_t ATM90E32_REGISTER_APENERGYB = 0x82;  // B Forward Active
static const uint16_t ATM90E32_REGISTER_APENERGYC = 0x83;  // C Forward Active
static const uint16_t ATM90E32_REGISTER_ANENERGYT = 0x84;  // Total Reverse Active
static const uint16_t ATM90E32_REGISTER_ANENERGYA = 0x85;  // A Reverse Active
static const uint16_t ATM90E32_REGISTER_ANENERGYB = 0x86;  // B Reverse Active
static const uint16_t ATM90E32_REGISTER_ANENERGYC = 0x87;  // C Reverse Active

@CircuitSetup , how feasible it is to implement the reading from these registers into individual ESPHome sensors we could use in yaml? I recognize the ESP32 might run out of memory, but I wonder if we could sacrifice web server component and only keep ota and mqtt. Might even sacrifice ota in favor of getting energy sensor data from chip, rather than relying on ESPHome to calculate approximated values.

[CircuitSetup]

Wow, you've done a lot of tinkering! I'm glad you're getting to know the IC!

The energy registers listed above wouldn't be hard to add to ESPHome, but they would require some additional coding to determine what the number being read means over time. I haven't experimented much with these registers, but from what I gather, as a 16 bit integer, the max they can hold is around 0.2kWh. I do know that they are reset once read though.

The Arduino library divides the output of APENERGYT by 100, then by 3200 to get kWh.

[elyorkhakimov]

Fascinating! Please don't mind my excitement. I haven't coded in over 10 years, and this whole thing makes me want to come back.

I see where that division by 100 (resolution 0.01CF) and then 3200 (MC of 3200imp/kWh) comes from. (from http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-46003-SE-M90E32AS-Datasheet.pdf)

It appears as though Arduino library only implemented registers for Total active/reactive/apparent energy values. The per-phase values are commented/not implemented.

I am now trying to understand what would it take to add the support for energy sensors in ESPHome. I can't seem to make a logical connection between "sensor" definition in YAML and how/where it gets interpreted and used to pull data from chip via atm90e32.h/atm90e32.cpp.
Where would be a good place to start research?

[elyorkhakimov]

this commit is probably all bits of data I need to add support for pulling energy data from chip.
esphome/esphome@5a67e72#diff-a545a9e0e9b82494c44c9cdadf63a9be

what IDE would you recommend using? I would like to contribute, and recognize I need lots of guidance. Thanks for your patience!

[CircuitSetup]

Yes, that commit is exactly what I did to add other registers.

For development I would recommend Atom with the platformio plugin installed. You'll want the dev build of ESPHome. Getting it to compile is a bit tricky at first because all of the dependencies have to be loaded.

[elyorkhakimov]

Thanks!
I tried installing platformio plugin for atom, but that did not seem to work. Then tried VSCode, but when compiling getting lots of errors. Did not spend time troubleshooting it.
In the meantime, forked dev branch and made several changes, but then don't seem to figure out how to upload it with "esphome xx.yaml run" command. I will troubleshoot more, but in the meantime thought I'd ask if you could try to run it and see if it works for you.
esphome/esphome@dev...elyorkhakimov:energy-add

[elyorkhakimov]

OK, I think I'm more or less done with the implementation. Folks over at ESPHome Discord helped me ascertain where I made mistakes.

I don't have a bench or some type of high-precision Wh-meter/Joule meter to estimate a Wh reading error, and therefore can't calibrate; but comparing to Kill-A-Watt (only measures to hundredth's of kWh) the reading is spot on. I tested by boiling a whole electric kettle of water several times.
I initially thought to create new unit UNIT_KWATT_HOURS, but then decided to just keep base Wh unit.

What's next? How can I get this into dev branch?
Would someone want to test this code some more?

Oh, and one more observation from the way it's implemented - upon reading the register, it is cleared. I might need to figure out a way to re-use or implement some type of sensor that "accumulates" the readings and then reports at less frequent intervals.
At the moment, with default "update_interval=10s", I just end up seeing a very small "increments" in consumed energy measured in Wh.

My use case is to log this data in influxDB, but I recognize to make it usable more broadly, the integrated value over an hour or a day might be needed

Code is in "energy-add" branch: https://github.com/elyorkhakimov/esphome/tree/energy-add

This is an excerpt from the YAML:

      power_factor:
        name: ${disp_name} CT3 Power Factor
        state_topic: ${disp_name}/ct3/power_factor
      current:
        name: ${disp_name} CT3 Amps
        id: ct3Amps
        state_topic: ${disp_name}/ct3/current
      power:
        name: ${disp_name} CT3 Watts
        id: ct3Watts
        state_topic: ${disp_name}/ct3/power
      forward_active_energy:
        name: ${disp_name} CT3 WattHours
        id: ct3FAWatts
        state_topic: ${disp_name}/ct3/forward_active_energy
      reverse_active_energy:
        name: ${disp_name} CT3 WattHours
        id: ct3RAWatts
        state_topic: ${disp_name}/ct3/reverse_active_energy
      gain_voltage: ${voltage_cal}
      gain_ct: ${current_cal}```

[CircuitSetup]

Excellent work! From here you can open a pull request with the dev branch of esphome. You'll have to do the same for the documents page, being sure to add examples of the new parameters and how they're used.

[elyorkhakimov]

I had a chance to run my setup for few weeks now, and without any calibration of CT sensors (YHDC SCT-013 100A/50mA) spliced/extended with DIY unshielded twisted copper wire extension are giving about 1.51% error. For example, Solar inverter says it generated 22.48kWh, and my setup says 22.14, and from few days ago - 24.37 from sensor vs 24.74 from solar inverter. Both CTs are connected to IC2 (ct5 and ct6).

The only global ESPHome adjustment I made was of the voltage_cal:, set it to 7280 vs stock 7305, as it seemed that reported voltage was off. Should I do any type of calibration for CTs ?
By the way, IC1 and IC2 are reporting slightly different voltages. What could be a cause of that ?

The DIY cable extensions are made using 2x 16AWG for first CT, and 2x 18AWG for the second CT, both stranded copper TFFN wires from HomeDepot. I locked two wires of 30ft length in drill chuck and then spun it. Really, eyeballed the number of twists. Then wires untwisted a bit, and I believe the effective extension length is about 28ft. I solder-spliced in this extension cable into existing 24AWG cable that was already attached to CT sensor.
I thought the noise would be unacceptable, but, it seems that there are no issues.

[CircuitSetup]

Glad to hear you're getting fairly accurate results, and that the extensions worked!

For the differences in IC1 and IC2, this is likely because of variance in the resistors. They are 1%. If you need to, you can calibrate each voltage input differently.

For the CTs you can also adjust the calibration for a more accurate result. It may be easier to do all of the CTs at once, clamped around the same wire, with the same load.

[smurfix]

@elyorkhakimov I assume you're no longer working on this?

I'd really like to log "real" energy use instead of instantaneous loads.