Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues with I2C deadlocks #2700

Open
addshore opened this issue Jan 30, 2025 · 3 comments
Open

Issues with I2C deadlocks #2700

addshore opened this issue Jan 30, 2025 · 3 comments
Assignees

Comments

@addshore
Copy link
Contributor

Case 1 & 2: I2C device has been communicated with before, but stops part way through operations for a period of time

Potential dead-lock detected:
  Process: 3
  Program: 0f3f261e-541b-c138-fef7-199b44a7a1dc
  BCI: 0x3db
  Primitive: 6:2
fatal: Potential dead-lock

abort() was called at PC 0x4202e367 on core 0
Core  0 register dump:
MEPC    : 0x40806520  RA      : 0x408142ae  SP      : 0x4082fd90  GP      : 0x40822e84
TP      : 0x4082ffe0  T0      : 0x37363534  T1      : 0x7271706f  T2      : 0x33323130
S0/FP   : 0x4082fdcc  S1      : 0x4082fdcc  A0      : 0x4082fdcc  A1      : 0x4082fdae
A2      : 0x00000000  A3      : 0x4082fdf9  A4      : 0x00000001  A5      : 0x4082b000
A6      : 0x00000000  A7      : 0x76757473  S2      : 0x4082fdb0  S3      : 0x00000001
S4      : 0x027e1356  S5      : 0x00000005  S6      : 0x40836140  S7      : 0x004c4b40
S8      : 0x00000000  S9      : 0x00000000  S10     : 0x00000000  S11     : 0x00000000
T3      : 0x6e6d6c6b  T4      : 0x6a696867  T5      : 0x66656463  T6      : 0x62613938
MSTATUS : 0x00001881  MTVEC   : 0x40800001  MCAUSE  : 0x00000007  MTVAL   : 0x00000000
MHARTID : 0x00000000
Potential dead-lock detected:
  Process: 3
  Program: 8ba5f2ca-8927-4edc-5056-5ba029f2eb90
  BCI: 0x3e5
  Primitive: 6:7
fatal: Potential dead-lock

abort() was called at PC 0x4202e367 on core 0
Core  0 register dump:
MEPC    : 0x40806520  RA      : 0x408142ae  SP      : 0x4082fd90  GP      : 0x40822e84
TP      : 0x4082ffe0  T0      : 0x37363534  T1      : 0x7271706f  T2      : 0x33323130
S0/FP   : 0x4082fdcc  S1      : 0x4082fdcc  A0      : 0x4082fdcc  A1      : 0x4082fdae
A2      : 0x00000000  A3      : 0x4082fdf9  A4      : 0x00000001  A5      : 0x4082b000
A6      : 0x00000000  A7      : 0x76757473  S2      : 0x4082fdb0  S3      : 0x00000002
S4      : 0x02c49e66  S5      : 0x00000005  S6      : 0x40836208  S7      : 0x004c4b40
S8      : 0x00000000  S9      : 0x00000000  S10     : 0x00000000  S11     : 0x00000000
T3      : 0x6e6d6c6b  T4      : 0x6a696867  T5      : 0x66656463  T6      : 0x62613938
MSTATUS : 0x00001881  MTVEC   : 0x40800001  MCAUSE  : 0x00000007  MTVAL   : 0x00000000
MHARTID : 0x00000000

Case 3: I2C device I am talking to will not respond to anything.

[jaguar] INFO: program 9d8d7d0a-1109-640c-b897-53a2b2f25735 starPotential dead-lock detected:
  Process: 3
  Program: 9d8d7d0a-1109-640c-b897-53a2b2f25735
  BCI: 0x377a
  Primitive: 6:4
fatal: Potential dead-lock

abort() was called at PC 0x4202e367 on core 0
Core  0 register dump:
MEPC    : 0x40806520  RA      : 0x408142ae  SP      : 0x4082fd90  GP      : 0x40822e84
TP      : 0x4082ffe0  T0      : 0x37363534  T1      : 0x7271706f  T2      : 0x33323130
S0/FP   : 0x4082fdcc  S1      : 0x4082fdcc  A0      : 0x4082fdcc  A1      : 0x4082fdae
A2      : 0x00000000  A3      : 0x4082fdf9  A4      : 0x00000001  A5      : 0x4082b000
A6      : 0x00000000  A7      : 0x76757473  S2      : 0x4082fdb0  S3      : 0x00000001
S4      : 0x021b3f67  S5      : 0x00000005  S6      : 0x40836140  S7      : 0x004c4b40
S8      : 0x00000000  S9      : 0x00000000  S10     : 0x00000000  S11     : 0x00000000
T3      : 0x6e6d6c6b  T4      : 0x6a696867  T5      : 0x66656463  T6      : 0x62613938
MSTATUS : 0x00001881  MTVEC   : 0x40800001  MCAUSE  : 0x00000007  MTVAL   : 0x00000000
MHARTID : 0x00000000
@floitsch
Copy link
Member

Normally, primitive operations are not allowed to block for longer periods of time.
The i2c operation is currently blocking, which usually isn't a problem, since the operation shouldn't take long. However, if the transaction takes too long (potentially because of a bad device, clock stretching, ...) a watchdog triggers and reboots the device.

We need to make the i2c operation asynchronous to fix this. I already need to update the driver to use the newer ESP-IDF APIs. When doing so, I will try to change it so it isn't blocking anymore.

@floitsch floitsch self-assigned this Jan 30, 2025
@addshore
Copy link
Contributor Author

Another one today, and I noticed its Primitive: 6:5 instead, so figured dumping it here might be useful?

Potential dead-lock detected:
  Process: 8
  Program: bae9bfbc-56ff-a982-a636-e0d62c461578
  BCI: 0x3c8d
  Primitive: 6:5
fatal: Potential dead-lock

abort() was called at PC 0x4202e001 on core 0
Core  0 register dump:
MEPC    : 0x408065de  RA      : 0x408145e8  SP      : 0x4082fd90  GP      : 0x40823204
TP      : 0x4082ffe0  T0      : 0x37363534  T1      : 0x7271706f  T2      : 0x33323130
S0/FP   : 0x4082fdcc  S1      : 0x4082fdcc  A0      : 0x4082fdcc  A1      : 0x4082fdae
A2      : 0x00000000  A3      : 0x4082fdf9  A4      : 0x00000001  A5      : 0x4082c000
A6      : 0x00000000  A7      : 0x76757473  S2      : 0x4082fdb0  S3      : 0x00000002
S4      : 0x5ce7e162  S5      : 0x00000005  S6      : 0x408316f0  S7      : 0x004c4b40
S8      : 0x00000000  S9      : 0x00000000  S10     : 0x00000000  S11     : 0x00000000
T3      : 0x6e6d6c6b  T4      : 0x6a696867  T5      : 0x66656463  T6      : 0x62613938
MSTATUS : 0x00001881  MTVEC   : 0x40800001  MCAUSE  : 0x00000007  MTVAL   : 0x00000000
MHARTID : 0x00000000

@floitsch
Copy link
Member

6:5 is I2C-read.
Almost certainly the same reason: the device pulls the clock low and delays the I2C transaction. -> we run into a timeout.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants