Skip to content

BUG: Compiler Flag Drift May Affect Pandas ABI Stability via Memory Assumptions #61452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 tasks done
BryteLite opened this issue May 18, 2025 · 3 comments
Closed
3 tasks done
Labels
Bug Build Library building on various platforms

Comments

@BryteLite
Copy link

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import numpy as np

# Create a structured array with alignment-sensitive types
dtype = np.dtype([('x', np.int64), ('y', np.float64)])
arr = np.zeros(10, dtype=dtype)

# Wrap into DataFrame
df = pd.DataFrame(arr)

# Trigger complex alignment path
try:
    # Operation that depends on consistent field layout
    df_sum = df.sum(numeric_only=True)
    print("Sum result:", df_sum)
except Exception as e:
    print("Failure during structured alignment test:", e)

Issue Description

Summary

Pandas may be vulnerable to ABI and memory alignment issues caused by C23 default behaviors in GCC 15.1. Silent adoption of padding behavior changes — particularly in union or struct definitions used in NumPy or Pandas C extensions — may lead to unpredictable runtime behavior.

This issue was originally identified in NumPy and Cython. As Pandas includes both compiled Cython code and relies on NumPy for internal memory layout, it is downstream vulnerable.

These compiled pieces are sensitive to pointer alignment, ABI expectations, or padding behaviors — especially across environments.

Reproducible Example

Please see section below

Possibly related to:

Report for more context:
Report

Expected Behavior

Recompile NumPy and Pandas with mismatched flags.

Then run the If padding bits are not cleared correctly in C structs, or if a layout mismatch occurs due to vendor/flag drift, crashes or incorrect math results may emerge.

CFLAGS="-std=c23" pip install numpy pandas --force-reinstall --no-cache-dir when #building

Installed Versions

NumPy latest 3.13 release, Pandas latest 3.13 release are suitable.

@BryteLite BryteLite added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels May 18, 2025
@BryteLite BryteLite changed the title BUG: BUG: Compiler Flag Drift May Affect Pandas ABI Stability via Memory Assumptions May 18, 2025
@BryteLite
Copy link
Author

NumPy just released 2.2.6 fixes concerning types today. Might just mask the issue however.

@eli-schwartz
Copy link
Contributor

@rhshadrach
Copy link
Member

Thanks @eli-schwartz - closing along with matplotlib/matplotlib#30064 and numpy/numpy#28953

@rhshadrach rhshadrach added Build Library building on various platforms and removed Needs Triage Issue that has not been reviewed by a pandas team member labels May 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Build Library building on various platforms
Projects
None yet
Development

No branches or pull requests

3 participants