Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

issue regarding running k2sc -- having trouble reproducing the k2sc team's results #24

Open
bethbiller opened this issue Feb 17, 2020 · 23 comments
Open

issue regarding running k2sc -- having trouble reproducing the k2sc team's results #24

bethbiller opened this issue Feb 17, 2020 · 23 comments
Assignees
@hpparvi

Comments

@bethbiller
Copy link

We've been using k2sc v 1.0 (downloaded from this repository) with george v 0.3.1 to detrend the lightcurves and remove telescope systematics. Unfortunately, we're having trouble replicating the lightcurves provided by your team here:

https://archive.stsci.edu/prepds/k2sc/

I've attached an example -- this is k2 id 210327027:

lightcurves

also known as 2MASS 0355+11, an L3-L6 brown dwarf observed during campaign 4. In the attached plot, the top panel is the raw PDCSAP flux, the middle panel is our attempt to run k2sc on the PDCSAP flux (clearly something off), and the bottom panel is the result from the lightcurves provided by your team (clearly detrended). All lightcurves have been normalised by the median value, and then had 1.0 subtracted from them (so units should be fractional variation from the median).

We installed k2sc following the directions on this github repository, and with the command line:

k2sc ktwo210327027-c04_llc.fits

where ktwo210327027-c04_llc.fits is the lightcurve file downloaded directly from the MAST archive.

I've put both the input and output (EPIC_210327027_mast.fits) files up on Dropbox, along with the script used to plot the lightcurves (plot_lightcurves.py):

https://www.dropbox.com/sh/u3kgy2j7f1jh4i2/AABMSHSti1d2h6k06nNyLf6ka?dl=0

Any insight you might have into what is going on (and what we might have done wrong) would be greatly appreciated! Thanks in advance!
@benjaminpope
Copy link
Contributor

Hi Beth,

Just to update you, I have been able to reproduce your bug but haven't been able to fix it. Running it either from the command line or from the lightkurve API gives the same bug. I will keep you updated.

@hpparvi do you have any ideas?

All the best,

Ben

@hpparvi
Copy link
Contributor

hpparvi commented Feb 19, 2020

Hi Beth,

Sorry, I'm on holidays and haven't been checking my mail that frequently. I'll try to find some time to check this within the next day or two.

Cheers,
Hannu

@bethbiller
Copy link
Author

Hi Ben and Hannu --

Just checking in if there had been any resolution as to what is causing this bug? Thanks in advance!

Best Wishes,
Beth

@benjaminpope
Copy link
Contributor

benjaminpope commented Mar 13, 2020 via email

@bethbiller
Copy link
Author

Hi Ben and Hannu --

Just checking in on this. I have a new research student starting this semester who is interested in running k2sc on K2 brown dwarf lightcurves, so wanted to check if you thought it was possible for this bug to be resolved within the next month or two? If not, I will suggest a different research project for him.

Thanks!

Best Wishes,
Beth

@benjaminpope
Copy link
Contributor

benjaminpope commented Sep 18, 2020 via email

@bethbiller
Copy link
Author

Hi Ben and Hannu --

So my new student has started and is in fact very keen on working with k2sc on K2 brown dwarf lightcurves.
We've tried some of the other detrending software available (Everest, k2sff, etc.) and honestly, only k2sc does a good job detrending the light curves for these faint objects! K2sc will be quite pivotal for this project -- considering this, we're also happy to include the k2sc team on any eventual publication from this work. Thanks!

Best Wishes,
Beth

@saigrain
Copy link
Contributor

saigrain commented Sep 24, 2020 via email

@benjaminpope
Copy link
Contributor

Hi @bethbiller,

Sorry to be so lax about this. I am a little overwhelmed at the moment. @hpparvi have you plans to look at this? Or else I can.

Beth - what timescale do you need the fixes by? I have a deadline mid next week after which I may be able to get to this.

All the best,

Ben

@hpparvi
Copy link
Contributor

hpparvi commented Sep 29, 2020

Hi,

Sorry, I've been busy also for the last months, but will try to find time to fix the issue during the next two weeks.

Cheers,
Hannu

@bethbiller
Copy link
Author

Hi everyone --

A fix in the next month or so would be great.

Best Wishes,
Beth

@benjaminpope
Copy link
Contributor

Hi @hpparvi,

Ok so I'm looking at this and something is clearly wrong. Are the light curves labelled correctly or normalized in some unusual way? I can't actually remember - it was four or five years ago - but just overplotting trtime and trposi on @bethbiller's exact plots, it's clear that the corrections are much smaller than the differences.

image

Ben

@benjaminpope
Copy link
Contributor

k2sc's standalone version appears to reproduce fairly well Figure 3 from the original paper, correcting EPIC 211748059: https://arxiv.org/abs/1603.09167

image

@bethbiller
Copy link
Author

Hi everyone -- Just checking in on this. While the current standalone version doesn't work for the campaign 4 observation k2 id 210327027, there clearly was an earlier version of k2sc that did provide sensible corrections -- the lightcurve available here: https://archive.stsci.edu/prepds/k2sc/ looks well-corrected. Any chance of accessing that earlier version of k2sc for the additional K2 brown dwarf lightcurves we are looking at?

@benjaminpope
Copy link
Contributor

benjaminpope commented Nov 11, 2020 via email

@hpparvi
Copy link
Contributor

hpparvi commented Nov 11, 2020

Hi @bethbiller and @benjaminpope,

I've finally managed to find a bit of time to look into this, sorry for the long delay. I'm starting to think the issue is not with K2SC, but with the input data.

First, the raw data is a bit weird. The PDCSAP_FLUX median is around -20.7 (yes, negative), while the k2sc version in the MAST has a flux median of 659. The ktwo210327027-c04_llc.fits file has been created 2019-06-26, so it's not the same one as what was used to create the archived K2SC file. I've no idea how to access the older version of the file in MAST, but I'll see if we can find it from somewhere.

So, the difference between the archived K2SC file and the new produced file is most likely because the input photometry is very different. K2SC catches the time variability just fine, but doesn't find any position dependency.

k2sc_output

I haven't kept an eye on this, but it's possible that the official K2 pipeline has been updated (@saigrain, @benjaminpope?).

Cheers,
Hannu

@bethbiller
Copy link
Author

Hi all --

I contacted the STScI helpdesk about accessing previous versions of the 2M 0355 lightcurves. It looks like the version from Autumn 2019 was Data Release 33, here is a link to the Data Release 6 version:

https://archive.stsci.edu/missions/k2/prev_ver/lightcurves/c4/210300000/27000/ktwo210327027-c04_llc.fits

Hannu -- as you suspected, the mean values for the older version of the lightcurve are generally positive. The mean values of the Data Release 6 light curves are ~82 counts for the SAP lightcurve and ~660 counts for the PDCSAP lightcurve. Does this version of the raw lightcurve yield a reasonable result with K2SC?

I've asked the STScI helpdesk as to why the photometry has changed so much between different data releases, will let you know what they say.

Best Wishes,
Beth

@samwalker2312
Copy link

Hi all,

I'm the student that Beth has been working with on this project and referring to in the above posts. Whilst 2MASS 0355 is clearly a bad example, with its differing data quality between the various K2 data releases, we are still encountering issues when trying to replicate data from the K2SC HLSP archive by running K2SC ourselves. Attached below are a few examples of the issues we're having. You can see that the `detrended' lightcurves that we obtained by running through using the public version of K2SC (labelled as 'our' version) and what has been obtained by your team and placed on the online archive (labelled as 'their version') differs significantly, with obvious variability still present in our data that have been successfully detrended in yours. These lightcurves have all got high fluxes, and have been created using the older versions of the K2 data, before the uniform reprocessing that caused the issues with 2MASS 0355, so we should be using the exact same raw data to create each of these.

211776456_compareplot_raw
211962038_compareplot_raw
206135809_compareplot_raw
210422945_compareplot_raw

It would be great if you could help us understand what's going on here and how to fix it.

Best,
Sam

@bethbiller
Copy link
Author

Hi Sam -- Could you let the K2SC team know what the K2 id is for these three targets, and which campaign they were each observed in?

@samwalker2312
Copy link

Of course, my apologies! These are (from top to bottom): ktwo211776456 (c05), ktwo211962038 (c05), ktwo206135809 (c03) and ktwo210422945 (c04). Hopefully that helps!

@benjaminpope
Copy link
Contributor

benjaminpope commented Feb 17, 2021 via email

@samwalker2312
Copy link

samwalker2312 commented Feb 17, 2021 •
edited
Loading

We ran this through using Python 3.7.3, astropy 4.0, George 0.3.1, MPI4Py 3.0.2 (although we didn't use MPI so I imagine that shouldn't be affecting things), NumPy 1.16.2, SciPy 1.2.1, and this current version of K2SC

@hpparvi
Copy link
Contributor

hpparvi commented Mar 14, 2021

Hi Sam,

Sorry for the slow response. I'll try to have a look at what's going on within the next few days.

Cheers,
Hannu

@hpparvi hpparvi self-assigned this Mar 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hpparvi hpparvi

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@saigrain @hpparvi @benjaminpope @samwalker2312 @bethbiller