Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem in converting FITS file tables to DataFrames #3489

Open
stefanocovino opened this issue Feb 14, 2025 · 4 comments
Open

Problem in converting FITS file tables to DataFrames #3489

stefanocovino opened this issue Feb 14, 2025 · 4 comments
Labels
bug ecosystem Issues in DataFrames.jl ecosystem

Comments

@stefanocovino
Copy link

Dear friends,

Recently, I can't import tables stored ad FITS files as DataFrame since I see errors never met before. I say "before" since it worked till a few months ago. I am using julia 11.3, on a Debian linux.

Thanks a lot,
Stefano

I do very simple things, e.g.:

using FITSIO
using DataFrames

hdu = FITS("NGC_2682.fits")
File: NGC_2682.fits
Mode: "r" (read-only)
HDUs: Num Name Type
1 Image
2 Table

hdu[2]
File: NGC_2682.fits
HDU: 2
Type: Table
Rows: 1585
Columns: Name Size Type TFORM
gaia_dr2_source.designation String 40A
gaia_dr2_source.source_id Int64 K
gaia_dr2_source.ra Float64 D
gaia_dr2_source.dec Float64 D
gaia_dr2_source.parallax Float64 D
gaia_dr2_source.parallax_error Float64 D

many more lines....

data = DataFrame(hdu[2])

[2533] signal 11 (1): Segmentation fault
in expression starting at REPL[5]:1
jl_datatype_layout at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/julia.h:1338 [inlined]
gc_mark_outrefs at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:2771 [inlined]
gc_mark_loop_serial_ at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:2943
gc_mark_loop_serial at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:2966
gc_mark_loop at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:3148 [inlined]
jl_gc_collect at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:3537
ijl_gc_collect at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:3898
maybe_collect at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:922 [inlined]
jl_gc_pool_alloc_inner at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:1324
jl_gc_pool_alloc_noinline at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:1391 [inlined]
jl_gc_alloc at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/julia_internal.h:523 [inlined]
jl_gc_alloc at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:3951
new_genericmemory at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/genericmemory.c:56 [inlined]
jl_alloc_genericmemory at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/genericmemory.c:99
GenericMemory at ./boot.jl:516 [inlined]
array_new_memory at ./array.jl:1058 [inlined]
#134 at ./array.jl:1129
_growend! at ./array.jl:1116 [inlined]
push! at ./array.jl:1261 [inlined]
compute_domtree_nodes! at ./compiler/ssair/domtree.jl:270
update_domtree! at ./compiler/ssair/domtree.jl:260 [inlined]
construct_domtree at ./compiler/ssair/domtree.jl:242
CFGTransformState! at ./compiler/ssair/ir.jl:654
IncrementalCompact at ./compiler/ssair/ir.jl:724 [inlined]
adce_pass! at ./compiler/ssair/passes.jl:2016
run_passes_ipo_safe at ./compiler/optimize.jl:994
run_passes_ipo_safe at ./compiler/optimize.jl:1009 [inlined]
optimize at ./compiler/optimize.jl:983
jfptr_optimize_42723.1 at /home/stefanocovino/.julia/juliaup/julia-1.11.3+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
finish_nocycle at ./compiler/typeinfer.jl:265
_typeinf at ./compiler/typeinfer.jl:249
typeinf at ./compiler/typeinfer.jl:215
typeinf_edge at ./compiler/typeinfer.jl:923
abstract_call_method at ./compiler/abstractinterpretation.jl:660
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:102
abstract_call_known at ./compiler/abstractinterpretation.jl:2200
abstract_call at ./compiler/abstractinterpretation.jl:2282
abstract_call at ./compiler/abstractinterpretation.jl:2275
abstract_call at ./compiler/abstractinterpretation.jl:2420
abstract_eval_call at ./compiler/abstractinterpretation.jl:2435
abstract_eval_statement_expr at ./compiler/abstractinterpretation.jl:2451
abstract_eval_statement at ./compiler/abstractinterpretation.jl:2749
abstract_eval_basic_statement at ./compiler/abstractinterpretation.jl:3065
typeinf_local at ./compiler/abstractinterpretation.jl:3319
typeinf_nocycle at ./compiler/abstractinterpretation.jl:3401
_typeinf at ./compiler/typeinfer.jl:244
typeinf at ./compiler/typeinfer.jl:215
typeinf_edge at ./compiler/typeinfer.jl:923
abstract_call_method at ./compiler/abstractinterpretation.jl:660
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:102
abstract_call_known at ./compiler/abstractinterpretation.jl:2200
abstract_call at ./compiler/abstractinterpretation.jl:2282
abstract_call at ./compiler/abstractinterpretation.jl:2275
abstract_call at ./compiler/abstractinterpretation.jl:2420
abstract_eval_call at ./compiler/abstractinterpretation.jl:2435
abstract_eval_statement_expr at ./compiler/abstractinterpretation.jl:2451
abstract_eval_statement at ./compiler/abstractinterpretation.jl:2749
abstract_eval_basic_statement at ./compiler/abstractinterpretation.jl:3065
typeinf_local at ./compiler/abstractinterpretation.jl:3319
typeinf_nocycle at ./compiler/abstractinterpretation.jl:3401
_typeinf at ./compiler/typeinfer.jl:244
typeinf at ./compiler/typeinfer.jl:215
typeinf_edge at ./compiler/typeinfer.jl:923
abstract_call_method at ./compiler/abstractinterpretation.jl:660
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:102
abstract_call_known at ./compiler/abstractinterpretation.jl:2200
abstract_call at ./compiler/abstractinterpretation.jl:2282
abstract_call at ./compiler/abstractinterpretation.jl:2275
abstract_call at ./compiler/abstractinterpretation.jl:2420
abstract_eval_call at ./compiler/abstractinterpretation.jl:2435
abstract_eval_statement_expr at ./compiler/abstractinterpretation.jl:2451
abstract_eval_statement at ./compiler/abstractinterpretation.jl:2749
abstract_eval_basic_statement at ./compiler/abstractinterpretation.jl:3065
typeinf_local at ./compiler/abstractinterpretation.jl:3319
typeinf_nocycle at ./compiler/abstractinterpretation.jl:3401
_typeinf at ./compiler/typeinfer.jl:244
typeinf at ./compiler/typeinfer.jl:215
typeinf_edge at ./compiler/typeinfer.jl:923
abstract_call_method at ./compiler/abstractinterpretation.jl:660
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:102
abstract_call_known at ./compiler/abstractinterpretation.jl:2200
abstract_call at ./compiler/abstractinterpretation.jl:2282
abstract_call at ./compiler/abstractinterpretation.jl:2275
abstract_call at ./compiler/abstractinterpretation.jl:2420
abstract_eval_call at ./compiler/abstractinterpretation.jl:2435
abstract_eval_statement_expr at ./compiler/abstractinterpretation.jl:2451
abstract_eval_statement at ./compiler/abstractinterpretation.jl:2749
abstract_eval_basic_statement at ./compiler/abstractinterpretation.jl:3065
typeinf_local at ./compiler/abstractinterpretation.jl:3319
typeinf_nocycle at ./compiler/abstractinterpretation.jl:3401
_typeinf at ./compiler/typeinfer.jl:244
typeinf at ./compiler/typeinfer.jl:215
typeinf_ext at ./compiler/typeinfer.jl:1101
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1139
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1135
jfptr_typeinf_ext_toplevel_39915.1 at /home/stefanocovino/.julia/juliaup/julia-1.11.3+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
jl_apply at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/julia.h:2157 [inlined]
jl_type_infer at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gf.c:390
jl_generate_fptr_impl at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/jitlayers.cpp:519
jl_compile_method_internal at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gf.c:2536 [inlined]
jl_compile_method_internal at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gf.c:2423
_jl_invoke at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gf.c:2940 [inlined]
ijl_apply_generic at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gf.c:3125
#52 at ./none:0
iterate at ./generator.jl:48 [inlined]
collect at ./array.jl:791
unknown function (ip: 0x7f63b531df82)
getcolumn at /home/stefanocovino/.julia/packages/FITSIO/VqV3x/src/table.jl:582
#fromcolumns#912 at /home/stefanocovino/.julia/packages/DataFrames/kcA9R/src/other/tables.jl:36
fromcolumns at /home/stefanocovino/.julia/packages/DataFrames/kcA9R/src/other/tables.jl:36 [inlined]
#DataFrame#914 at /home/stefanocovino/.julia/packages/DataFrames/kcA9R/src/other/tables.jl:59 [inlined]
DataFrame at /home/stefanocovino/.julia/packages/DataFrames/kcA9R/src/other/tables.jl:48
unknown function (ip: 0x7f63b531ae52)
jl_apply at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/julia.h:2157 [inlined]
do_call at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/interpreter.c:126
eval_value at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/interpreter.c:223
eval_stmt_value at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/interpreter.c:174 [inlined]
eval_body at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/interpreter.c:663
jl_interpret_toplevel_thunk at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/interpreter.c:821
jl_toplevel_eval_flex at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/toplevel.c:943
jl_toplevel_eval_flex at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/toplevel.c:886
ijl_toplevel_eval_in at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/toplevel.c:994
eval at ./boot.jl:430 [inlined]
eval_user_input at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:245
repl_backend_loop at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:342
#start_repl_backend#59 at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:327
start_repl_backend at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:324
#run_repl#72 at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:483
run_repl at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:469
jfptr_run_repl_10097.1 at /home/stefanocovino/.julia/juliaup/julia-1.11.3+0.x64.linux.gnu/share/julia/compiled/v1.11/REPL/u0gqU_XvZAg.so (unknown line)
#1150 at ./client.jl:446
jfptr_YY.1150_14693.1 at /home/stefanocovino/.julia/juliaup/julia-1.11.3+0.x64.linux.gnu/share/julia/compiled/v1.11/REPL/u0gqU_XvZAg.so (unknown line)
jl_apply at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/julia.h:2157 [inlined]
jl_f__call_latest at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/builtins.c:875
#invokelatest#2 at ./essentials.jl:1055 [inlined]
invokelatest at ./essentials.jl:1052 [inlined]
run_main_repl at ./client.jl:430
repl_main at ./client.jl:567 [inlined]
_start at ./client.jl:541
jfptr__start_73609.1 at /home/stefanocovino/.julia/juliaup/julia-1.11.3+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
jl_apply at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/julia.h:2157 [inlined]
true_main at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/jlapi.c:900
jl_repl_entrypoint at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/jlapi.c:1059
main at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/cli/loader_exe.c:58
unknown function (ip: 0x7f63c21aa249)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x4010b8)
Allocations: 3994572 (Pool: 3994431; Big: 141); GC: 6
Segmentation fault (core dumped)
@bkamins
Copy link
Member

bkamins commented Feb 15, 2025

The problem seems to be originating from this line:

getcolumn at /home/stefanocovino/.julia/packages/FITSIO/VqV3x/src/table.jl:582

which is in the FITSIO.jl package.

@bkamins bkamins added bug ecosystem Issues in DataFrames.jl ecosystem labels Feb 15, 2025
@stefanocovino
Copy link
Author

stefanocovino commented Feb 15, 2025 via email

@bkamins
Copy link
Member

bkamins commented Feb 15, 2025

I think it is better if you open the issue there. I do not know this package and have never used it.

@stefanocovino
Copy link
Author

stefanocovino commented Feb 15, 2025 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug ecosystem Issues in DataFrames.jl ecosystem
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@stefanocovino @bkamins