Google Summer of Code project summary: Implementing new spectral corrections in pvlib

[RDaxini]

Introduction

This issue summarises the ongoing and completed work for the GSoC 2024 programme with pvlib.

The project I am undertaking relates to an issue raised earlier this year. The official abstract for the project can be found here. This project is being carried out under the supervision of @AdamRJensen and @kandersolar

In summary, the aim of the project is to implement new spectral correction models in pvlib, as well as examples of their application and use. In addition to updating this issue as the project progresses, detailed updates on the project will also be shared in blog posts. Major milestone updates will be shared on my LinkedIn page as well.

Plan

The current overall project plan is described below.

1. Three new models are planned for development and implementation:

• Pelland 2020 PVSPEC model
• Huld 2009 JRC model
• Daxini 2023 APE/e model (stretch goal, if time permits)

2. A combined example to demonstrate the use of the new models and application in the overall modelling pipeline will be developed.

As the project develops I will link these tasks to individual issues and pull requests.

Issues

Open:

Closed:
#2065 (this one)
#2135 (create average photon energy function)
#2125 (suggestion to split mismatch.py)
#1950 (general issue, add new spectral factor models)
#2087 (add JRC spectral factor model)
#2115 (update SAPM spectral factor docs)
#2107 (add spectral factor example),
#2086 (update spectral_factor_firstsolar)

PRs

• (merged) Split test_spectrum.py Split test_spectrum.py #2151
• (merged) Function to calculate APE Create function to calculate average photon energy #2140
• (merged) Restructure pvlib/spectrum restructure pvlib/spectrum #2136
• (merged) Update First Solar model update spectral_factor_firstsolar #2100
• (merged) Spectral mismatch factor example Add spectral factor gallery example #2114
• (merged) Update SAPM docs Update spectral_factor_sapm docs #2116
• (merged) JRC model (air mass and clearsky index) Add JRC spectral factor correction #2088
• (merged) PVSPEC model (air mass and clearsky index): Add PVSPEC spectral correction factor model #2072
• (closed) Add ape_e spectral factor model Add ape_e spectral factor model #2126

---

Any feedback on the project is certainly more than welcome. Using github and contributing to open-source software is completely new to me so I am looking forward to learning about this so that I can contribute to pvlib not only through this project but also in future.

Dax

[markcampanelli]

As part of the documentation of a growing number of methods, it might be helpful if a “overview” table of spectral-correction method vs. required+optional inputs were added. This way consumers would be able to get a quicker idea of what information/data is needed to apply each correction. The timescale over which the correction is applied could also be a factor (although maybe they are all the same at this point).

[RDaxini]

@markcampanelli I think that is a good suggestion. I tried to achieve something similar with Table 10 in this study; is that the sort of thing you had in mind? Timescale is a good point too. I have had a paper examining this issue under review for a while so maybe/hopefully something will come out of that soon that I can implement into this work.

[adriesse]

@didierthevenard

[markcampanelli]

@markcampanelli I think that is a good suggestion. I tried to achieve something similar with Table 10 in this study; is that the sort of thing you had in mind? Timescale is a good point too. I have had a paper examining this issue under review for a while so maybe/hopefully something will come out of that soon that I can implement into this work.

@RDaxini Yes. Depending on your time, you could at a bare minimum reference your paper with table 10. If time permits, then you could perhaps provide a “translation” of that table into something more specific to pvlib’s variable names and functions.

BTW: Your timescale-focused paper also sounds like an interesting read 🙂. Good luck working it through the process!

[kandersolar]

@RDaxini #2088 got automatically linked to this issue based on its description, so merging that PR automatically closed this issue. Check out https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword

Try "see also #XXX" or some other phrasing to prevent the automatic linkage next time :)

[adriesse]

• Daxini 2023 APE/e model (stretch goal, if time permits)

I suggest adding the following to pvlib also:

• a function to calculate APE
• a function to calculate your "band_depth"
• make available the three SR curves that are associated with your published coefficients

[RDaxini]

@adriesse thanks for your suggestions

• a function to calculate APE
• a function to calculate your "band_depth"

I was actually just thinking about this when I opened #2126 (and #2125 to an extent). Will definitely work on this.

• make available the three SR curves that are associated with your published coefficients

Unfortunately, the SR curves for the specific modules analysed in the paper are not available. The three curves presented in Figure 7 of the paper are from a different set of devices; the figure is reproduced from the reference provided just to give the reader a rough idea of the overall curve shape for devices of the same/similar technology.

Aside: in the near future, I'll be working on some extended validation of this model using simulated spectral irradiance data and devices for which SR curves are available.

[adriesse]

• make available the three SR curves that are associated with your published coefficients

Unfortunately, the SR curves for the specific modules analysed in the paper are not available. The three curves presented in Figure 7 of the paper are from a different set of devices; the figure is reproduced from the reference provided just to give the reader a rough idea of the overall curve shape for devices of the same/similar technology.

Big thumbs down. I propose we reject all spectral mismatch models that are developed using secret spectral response curves starting today! That includes the new one from FirstSolar. I wish the journals did the same.

[RDaxini]

Big thumbs down. I propose we reject all spectral mismatch models that are developed using secret spectral response curves starting today! That includes the new one from FirstSolar. I wish the journals did the same.

@adriesse Spectral response curves were not used at any stage in the development or validation of the spectral mismatch model in the referenced paper. I think you have misunderstood the methodology.

[cwhanse]

I propose we reject all spectral mismatch models that are developed using secret spectral response curves starting today!

If you are proposing to reject spectral mismatch models if the SR for the cells/modules used for development/validation are not available, that's rather extreme. We'd probably reject all spectral models in use today, if that was a rule.

If you are saying to reject models that integrate the product of spectral irradiance and SR, but with hidden SR values, then yes I agree.

[RDaxini]

If you are proposing to reject spectral mismatch models if the SR for the cells/modules used for development/validation are not available, that's rather extreme. We'd probably reject all spectral models in use today, if that was a rule.

If you are saying to reject models that integrate the product of spectral irradiance and SR, but with hidden SR values, then yes I agree.

+1

Just for clarity here, in the APE/e model paper, spectral mismatch is represented by a normalised form of the short-circuit current (see Section 2.2), for which the data required were measured in the field and are publicly available. Similar to the representation of spectral mismatch in the SAPM model and others.

[adriesse]

@adriesse Spectral response curves were not used at any stage in the development or validation of the spectral mismatch model in the referenced paper. I think you have misunderstood the methodology.

My bad. I didn't read the paper thoroughly and it was a while back. I assumed that if you had spectra and SR curves you would have multiplied them.

The three curves presented in Figure 7 of the paper are from a different set of devices; the figure is reproduced from the reference provided just to give the reader a rough idea of the overall curve shape for devices of the same/similar technology.

I did not find any indication in the paper that these curves are only "to give a rough idea", therefore, it seems like any reader would conclude that these curves are for the actual modules or module types whose data you used.

Now that I have looked at the paper a bit more closely, I still object to the model entering pvlib for two reasons:

1. IAM effects are included in the factors you calculated and fitted.
2. The three selected modules are not representative of today's products. Silicon and CdTe cells have both evolved, and triple junction a-Si is probably no longer relevant at all--certainly not this particular one.

[RDaxini]

I understand your points @adriesse. Thanks for your input. I think limitations with the built-in coefficients could potentially impact user experience in the near term but, equally, I think this issue a) can be mitigated; b) is not of such gravity as to warrant disqualifying the model entirely from implementation.

My thoughts:

1. IAM effects are included in the factors you calculated and fitted.

Correct. No model is perfect and, in the case of this model, this IAM limitation is highlighted openly the paper. The decision was between a smaller and potentially less reliable dataset through which AOI effects could be isolated, or using a larger dataset without being able to isolate these effects. I don't think this should disqualify the model from pvlib entirely for the following reasons.

The primary objective of the paper is to develop and validate a new spectral factor model focusing on testing the use of two new variables (new as in not used in this way before), investigating different spectral bands for the second variable, and parameterising the x-y-z relation.
Excluding the AOI effects does not affect these aspects of the analysis and the final mathematical parameterisation of the spectral mismatch factor as a function of APE and the depth of a water absorption band is still valid.

Not accounting for AOI effects does, on the other hand, have a systematic impact the final module coefficients presented for these three specific devices. Seasonal bias in AOI (and other) effects is avoided by using samples of data extracted from throughout the full year for the model development and for the model validation. At the time of writing, I thought users of any model would mostly use their own module-specific coefficients, hence this is not really an issue, but it has since been brought to my attention that in pvlib at least most people probably just use the built-in coefficients for the same PV technology that they have. I understand this is therefore a limitation, but I think sufficient mitigation can be achieved since:

a) It can be highlighted in the notes and that the published coefficients already contain AOI effects. Although this does not mean the an IAM has been perfectly applied, the user would at least be aware that they should not re-apply an AOI correction.
b) Updated/new coefficients can be added as more modules are analysed.

Finally, if I have understood the papers correctly, it is not uncommon for spectral correction function literature to omit AOI effects, including some already in pvlib-python e.g. pvlib.spectrum.spectral_factor_caballero.

Summary: model concept and parameterisation are still valid, notes can inform user of the IAM limitation, user can still provide their own coefficients since the physical model is unaffected by this limitation.

2. The three selected modules are not representative of today's products. Silicon and CdTe cells have both evolved, and triple junction a-Si is probably no longer relevant at all--certainly not this particular one.

Firstly, save for the SAPM f(AMa), which I think has a (regularly?) maintained coefficient database, none of the current spectral factor models in pvlib-python (or anywhere?) use modules that are representative of today's PV technologies. Even the most recently published model in pvlib-python (and outside?), the PVSPEC model from 2020, presents coefficients for First Solar series 4 modules from around 7-8 years ago, which have been superseded by several iterations of new FS modules now. The other models were published in 2018 and prior, and use modules from years prior to publication.
I am not saying this is ideal, but just that it is not a unique limitation of this particular model.
In fact, I am unsure as to whether there is really a "representative" module of any technology whose coefficients would be universally applicable. Nowadays, with such significant variation within the traditional classification of a PV technology type --- "mSi", "CdTe", etc. --- in terms of device construction, for example different encapsulations (structural and polymeric), spectral converters and filters, AR coatings, cell architectures (IBC, TOPCon, PERC, etc.), and so on, I doubt it's really possible to provide a generic set of, for example, "mSi" coefficients. That may be a topic for another discussion but my point here is that I don't think such a heavy weighting on the importance of the built-in coefficients at this stage is sufficient to entirely disqualify the model from implementation in pvlib. This now relates back to response to point 1. regarding coefficients --- notes, scope for updates, etc. can mitigate the shortfall in this area.

[adriesse]

Unfortunately, IAM effects are generally larger than spectral effects, which undermines your main conclusions. But, I should take time out from this discussion now and let others comment.

[cwhanse]

@adriesse I got this far investigating the concern you raise. I suspect you know all this but thought I'd put my views here for others. pvlib hosts four models for the spectral mismatch factor: sapm, firstsolar, caballero, pvspec, and (proposed) Dax's model.

• sapm is a fourth-order polynomial that is intended to be fit to Isc vs. air mass, for Isc collected with the module normal to the sun, adjusted for cell temperature, and during clear sky conditions. The intended measurements avoid the conflation of spectral mismatch with reflections. The main weakness of sapm is the use of a single predictor (air mass) that neglects variation in atmosphere.
• firstsolar was developed by generating spectra (using SMARTS) for different values of atmospheric variables, calculating spectral mismatch for two assumed spectral responses, and regressing the calculated mismatch to the atmospheric variables. In the development, it is implicit that reflections are outside the calculation of spectral mismatch. The validation compared model predictions with the MPERT data from NREL. MPERT kept modules at fixed orientation and thus the measured Isc is after reflections. The authors recognized this and state that Isc as "corrected for AOI effects" presumably by increasing effective irradiance.

I've temporarily lost access to journal articles so I haven't looked at the other models.

[adriesse]

I've temporarily lost access to journal articles so I haven't looked at the other models.

Zis is a problem, but ve have vays to find zem!

[kandersolar]

It seems to me that the main concern here is regarding the technology-specific coefficient values supplied in the reference rather than the model itself. I see a few alternatives for how to proceed:

1. Include the problematic coefficient values, with a warning informing the user about how they were derived.
2. Merge only the model, making the user supply their own coefficients. If/when new coefficient values become available in some future publication, they could be incorporated into the function at that time.
3. Wait to merge the model until new coefficient values are available, and then merge the model with the new coefficients.

Option 1 seems like it is giving users a way to shoot themselves in the foot. Option 2 seems like giving users a function that the vast majority cannot use, unless they go dig up the shoot-themselves-in-the-foot coefficients for themselves. Option 3 is a bit of a shame, but it seems like the best option to me.

[RDaxini]

Considering the value and usefulness of a model in pvlib without suitable coefficients, Option 3 makes sense to me too. I will direct some of my work towards developing new coefficients for the model in the near future. I will close PR #2126 for now and return to it once a published reference with coefficients is available.
Btw, note to all, I am grateful for the critical review provided on this model. While I was proposing its implementation objectively, it is still my work at the end of the day so I do appreciate the constructive feedback. I am happy that these limitations have been found now rather than later, and it gives me direction for improving my work and contributions to this community --- pvlib and PV modelling more broadly. So, thanks!

[cwhanse]

Continuing with my investigation of models already presented in pvlib:

caballero was developed using no measured data, only SMARTS simulations. In effect it is a reduced-order model of SMARTS, which is ignorant of plane of array. It was "validated" (quotes explained in a moment) by comparing model predictions with measurements of two instruments (global total irradiance, CMP-21 pyranometer, and spectral irradiance EKO MS-700 spectroradiometer) on fixed racking_, not tracking the sun. Although I didn't digest all the details, these measurements are converted to a "measured" spectral mismatch. "validated" in quotes indicates only the fact that the predicted quantity (spectral mismatch) is not directly measured. A CMP-21 can be assumed to have negligible AOI losses, which also seems to be a reasonable assumption about EKO's spectroradiometer. So I don't think caballero mixes AOI loss with spectral mismatch.

pvspec was fit to spectral mismatch computed from measurements from three sources: CanSIM network (Spectrafy SolarSIM-G spectral irradiance sensor and a Hukseflux SR20 pyranometer); NREL's SRL; and Almeria Spain (same instruments as used for caballero). I don't see how AOI losses could meaningfully affect this model.

huld was developed by fitting measured Isc to plane-of-array broadband irradiance (instrument unspecifed) after adjusting the irradiance for AOI using Martin-Ruiz. This is a similar approach as was taken to validate first_solar.

My conclusion - pvlib doesn't host spectral mismatch models that conflates AOI loss with spectral mismatch. If I'm wrong about this I'm very happy to be corrected.

I concur with @adriesse's objection about the APE/e model.

Is it practical to consider adjusting the irradiance used to development the APE/e for AOI loss, similar to what was done for pvspec or huld?

I'm for Option 3 proposed by @kandersolar

[adriesse]

It looks like this discussion was somehow productive and led to a conclusion that has acceptance. I'm glad about that.

Personally, I really don't enjoy interrupting other people's work in progress, and would much prefer to have scientific and/or strategic assessments for pvlib occur earlier in the feature proposal/planning process.

[RDaxini]

Closing this issue now as my GSoC project is complete.
If you are interested in a summary, I have been updating the original issue to track the project progress. Although the stretch goal in the original proposal was not met, the main goals were met alongside several new goals beyond the remit of the original proposal.
Thank you to everyone for your help throughout the project, in particular @kandersolar @AdamRJensen @cwhanse @echedey-ls @IoannisSifnaios. I have learnt a lot through this project --- about pvlib, open source, and python --- and I will do my best to use these skills to continue contributing to pvlib as my career develops.