Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix arbor tests with latest arbor version #498

Merged
merged 11 commits into from
Aug 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
strategy:
matrix:
os: [ubuntu-latest]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
python-version: ["3.9", "3.10", "3.11", "3.12"]
include:
- os: macos-12
python-version: "3.10"
Expand All @@ -34,7 +34,7 @@ jobs:
run: tox

- name: "Upload coverage to Codecov"
uses: codecov/codecov-action@v2
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ News
Requirements
============

* `Python 3.8+ <https://www.python.org/downloads/release/python-380/>`_
* `Python 3.9+ <https://www.python.org/downloads/release/python-390/>`_
* `Pip <https://pip.pypa.io>`_ (installed by default in newer versions of Python)
* `Neuron 7.4+ <http://neuron.yale.edu/>`_ (compiled with Python support)
* `eFEL eFeature Extraction Library <https://github.com/BlueBrain/eFEL>`_ (automatically installed by pip)
Expand Down
2 changes: 2 additions & 0 deletions bluepyopt/ephys/morphologies.py
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,8 @@ def load(morpho_filename, replace_axon):
morpho = arbor.load_component(morpho_filename).component
elif morpho_suffix == '.swc':
morpho = arbor.load_swc_arbor(morpho_filename)
# turn loaded_morphology into morphology type
morpho = morpho.morphology
elif morpho_suffix == '.asc':
morpho = arbor.load_asc(morpho_filename).morphology
else:
Expand Down
11 changes: 9 additions & 2 deletions bluepyopt/ephys/protocols.py
Original file line number Diff line number Diff line change
Expand Up @@ -602,7 +602,12 @@ def instantiate_iclamp_stimuli(self, decor, use_labels=False):
for i, stim in enumerate(self.stimuli):
if not isinstance(stim, stimuli.SynapticStimulus):
if hasattr(stim, 'envelope'):
arb_iclamp = arbor.iclamp(stim.envelope())
envelope = stim.envelope()
envelope = [
(t * arbor.units.ms, curr * arbor.units.nA)
for (t, curr) in envelope
]
arb_iclamp = arbor.iclamp(envelope)
else:
raise ValueError('Stimulus must provide envelope method '
' or be of type NrnNetStimStimulus to be'
Expand Down Expand Up @@ -648,7 +653,9 @@ def instantiate_recordings(self, cell_model, use_labels=False):

cell_model.probe('voltage',
arb_loc.ref if use_labels else arb_loc.loc,
frequency=10) # could be a parameter
f"probe-{i}",
# frequency could be a parameter
frequency=10 * arbor.units.kHz)

return cell_model

Expand Down
6 changes: 4 additions & 2 deletions bluepyopt/ephys/simulators.py
Original file line number Diff line number Diff line change
Expand Up @@ -372,9 +372,11 @@ def run(self, arb_cell_model, tstop=None, dt=None):
dt = dt if dt is not None else self.dt

if dt is not None:
return arb_cell_model.run(tfinal=tstop, dt=dt)
return arb_cell_model.run(
tfinal=tstop * arbor.units.ms, dt=dt * arbor.units.ms
)
else:
return arb_cell_model.run(tfinal=tstop)
return arb_cell_model.run(tfinal=tstop * arbor.units.ms)


class ArbSimulatorException(Exception):
Expand Down
8 changes: 4 additions & 4 deletions bluepyopt/ephys/templates/acc/decor_acc_template.jinja2

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As said before using Arbor to write these might be an easier option.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a function that can write these kind of files?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there's versions for full cable cells or just parts thereof.
https://docs.arbor-sim.org/en/latest/fileformat/cable_cell.html#parsable-arbor-components-and-meta-data
In Python, you'd construct a cable cell (or the parts you're interested in) and call arbor.write_component(thing, filename)
https://docs.arbor-sim.org/en/latest/python/cable_cell_format.html#reading-and-writing-arbor-components

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would basically mean to turn this template into code at the end of the ACC exporter by instantiating Arbor objects that are then serialized again. Could be an option - I guess it comes down to how much control you need over the export vs. learning the s-expression syntax. The original design of BluePyOpt was to make model exporting configurable (overridable by the user) through templates, hence I chose this option.

In my view, most of the complexity in the ACC exporter comes from converting BluePyOpt's frontend representation to one suitable for Arbor. That last step of dumping the decor is less complicated, but requires some understanding of the format.

I don't want to take any particular side, though. Would only suggest that if you plan to make major changes to the ACC exporter, maybe a good idea would be to do that in a separate PR.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'd support splitting this. One straight port that can be verified and a lateral move to another scheme.

The original design of BluePyOpt was to make model exporting configurable (overridable by the user) through templates, hence I chose this option.

This is why I as an outsider should have an opinion here :D

Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
(arbor-component
(meta-data (version "0.1-dev"))
(meta-data (version "0.9-dev"))
(decor
{%- for mech, params in global_mechs.items() %}
{%- if mech is not none %}
Expand All @@ -10,7 +10,7 @@
{%- endif %}
{%- else %}
{%- for param in params %}
(default ({{ param.name }} {{ param.value }}))
(default ({{ param.name }} {{ param.value }} (scalar 1.0)))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't check all of those in detail, but this is correct for a fixed value.
If the tests are green, I'd assume all are correct.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests are green

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend to additionally check the major examples' notebooks qualitatively by inspection (not everything is run as part of the test suite).

{%- endfor %}
{%- endif %}
{%- endfor %}
Expand All @@ -27,13 +27,13 @@
{%- endif %}
{%- else %}
{%- for param in params %}
(paint {{loc.ref}} ({{ param.name }} {{ param.value }}))
(paint {{loc.ref}} ({{ param.name }} {{ param.value }} (scalar 1.0)))
{%- endfor %}
{%- endif %}
{%- endfor %}

{%- for synapse_name, mech_params in pprocess_mechs[loc].items() %}
(place {{loc.ref}} (synapse (mechanism "{{ mech_params.mech }}" {%- for param in mech_params.params %} ("{{ param.name }}" {{ param.value }}){%- endfor %})) "{{ synapse_name }}")
(place {{loc.ref}} (synapse (mechanism "{{ mech_params.mech }}" {%- for param in mech_params.params %} ("{{ param.name }}" {{ param.value }} (scalar 1.0)){%- endfor %})) "{{ synapse_name }}")
{%- endfor %}

{%- endfor %}))
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
(arbor-component
(meta-data (version "0.1-dev"))
(meta-data (version "0.9-dev"))
(label-dict
{%- for loc, label in label_dict.items() %}
{{ label.defn }}
Expand Down
6 changes: 5 additions & 1 deletion bluepyopt/tests/test_ephys/test_create_acc.py
Original file line number Diff line number Diff line change
Expand Up @@ -537,7 +537,7 @@ def run_short_sim(cable_cell):
arb_cell_model.properties.catalogue.extend(arbor.bbp_catalogue(), "BBP::")

# Run a very short simulation to test mechanism instantiation
arb_cell_model.run(tfinal=0.1)
arb_cell_model.run(tfinal=0.1 * arbor.units.ms)


@pytest.mark.unit
Expand Down Expand Up @@ -665,6 +665,10 @@ def check_acc_dir(test_dir, ref_dir):
with open(ref_dir_file / file) as f:
ref_file = f.read()
assert ref_file == test_file
# check that load_component is not raising any error here
fpath = pathlib.Path(test_dir) / file
if fpath.suffix == "acc":
arbor.load_component(fpath).component


@pytest.mark.unit
Expand Down
34 changes: 33 additions & 1 deletion bluepyopt/tests/test_ephys/test_parameterscalers.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
"""Test ephys.parameterscalers"""

import json

import pathlib
import tempfile
import arbor

import pytest

Expand Down Expand Up @@ -143,3 +145,33 @@ def test_parameterscalers_iexpr_generator_unsupported_attr():
'unsupported attribute tau.'):
iexpr = value_scaler.acc_scale_iexpr(
value=value, constant_formatter=lambda v: '%.9g' % v)


@pytest.mark.unit
def test_parameterscalers_iexpr():
"""ephys.parameterscalers: Test iexpr"""
# iexpr from bluepyopt/tests/test_ephys/test_parameterscalers.py
iexpr = '(sub (scalar 0.62109375) ' \
'(mul (log (pi) ) ' \
'(exp (div (distance (region "soma")) ' \
'(scalar 0.421875) ) ) ) )'

# modified decor as in
# bluepyopt/tests/test_ephys/testdata/acc/simplecell/simple_cell_decor.acc
simple_cell_decor_with_iexpr = \
'(arbor-component\n' \
' (meta-data (version "0.9-dev"))\n' \
' (decor\n' \
' (paint (region "soma") ' \
'(membrane-capacitance 0.01 (scalar 1.0)))\n' \
' (paint (region "soma") ' \
'(scaled-mechanism (density (mechanism "default::hh" ' \
'("gnabar" 0.10299326453483033) ("gkbar" 0.027124836082684685))) ' \
f'("gkbar" {iexpr})))))'

with tempfile.TemporaryDirectory() as test_dir:
decor_filename = pathlib.Path(test_dir).joinpath("decor.acc")
with open(decor_filename, "w") as f:
f.write(simple_cell_decor_with_iexpr)
# check that load_component is not raising any error here
arbor.load_component(decor_filename).component
12 changes: 6 additions & 6 deletions bluepyopt/tests/test_ephys/testdata/acc/CCell/CCell_decor.acc
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
(arbor-component
(meta-data (version "0.1-dev"))
(meta-data (version "0.9-dev"))
(decor
(default (gSKv3_1bar_SKv3_1 65))
(paint (region "soma") (gSKv3_1bar_SKv3_1 65))
(paint (region "soma") (gSKv3_1bar_SKv3_1 65))
(default (gSKv3_1bar_SKv3_1 65 (scalar 1.0)))
(paint (region "soma") (gSKv3_1bar_SKv3_1 65 (scalar 1.0)))
(paint (region "soma") (gSKv3_1bar_SKv3_1 65 (scalar 1.0)))
(paint (region "dend") (density (mechanism "BBP::Ih")))
(paint (region "apic") (gSKv3_1bar_SKv3_1 65))
(paint (region "apic") (gSKv3_1bar_SKv3_1 65))
(paint (region "apic") (gSKv3_1bar_SKv3_1 65 (scalar 1.0)))
(paint (region "apic") (gSKv3_1bar_SKv3_1 65 (scalar 1.0)))
(paint (region "apic") (density (mechanism "BBP::Ih")))))
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
(arbor-component
(meta-data (version "0.1-dev"))
(meta-data (version "0.9-dev"))
(label-dict
(region-def "all" (all))
(region-def "apic" (tag 4))
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
(arbor-component
(meta-data
(version "0.1-dev"))
(version "0.9-dev"))
(morphology
(branch 0 -1
(segment 0
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
(arbor-component
(meta-data (version "0.1-dev"))
(meta-data (version "0.9-dev"))
(decor
(paint (region "soma") (membrane-capacitance 0.01))
(paint (region "soma") (membrane-capacitance 0.01 (scalar 1.0)))
(paint (region "soma") (density (mechanism "default::pas")))
(place (locset "somacenter") (synapse (mechanism "default::expsyn" ("tau" 10))) "expsyn")))
(place (locset "somacenter") (synapse (mechanism "default::expsyn" ("tau" 10 (scalar 1.0)))) "expsyn")))
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
(arbor-component
(meta-data (version "0.1-dev"))
(meta-data (version "0.9-dev"))
(label-dict
(region-def "all" (all))
(region-def "soma" (tag 1))
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
(arbor-component
(meta-data
(version "0.1-dev"))
(version "0.9-dev"))
(morphology
(branch 0 -1
(segment 0
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
(arbor-component
(meta-data
(version "0.1-dev"))
(version "0.9-dev"))
(morphology
(branch 0 -1
(segment 0
Expand Down
26 changes: 13 additions & 13 deletions bluepyopt/tests/test_ephys/testdata/acc/l5pc/l5pc_decor.acc
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
(arbor-component
(meta-data (version "0.1-dev"))
(meta-data (version "0.9-dev"))
(decor
(default (membrane-potential -65))
(default (temperature-kelvin 307.14999999999998))
(default (membrane-capacitance 0.01))
(default (axial-resistivity 100))
(default (membrane-potential -65 (scalar 1.0)))
(default (temperature-kelvin 307.14999999999998 (scalar 1.0)))
(default (membrane-capacitance 0.01 (scalar 1.0)))
(default (axial-resistivity 100 (scalar 1.0)))
(paint (region "all") (density (mechanism "default::pas/e=-75" ("g" 3.0000000000000001e-05))))
(paint (region "soma") (ion-reversal-potential "na" 50))
(paint (region "soma") (ion-reversal-potential "k" -85))
(paint (region "soma") (ion-reversal-potential "na" 50 (scalar 1.0)))
(paint (region "soma") (ion-reversal-potential "k" -85 (scalar 1.0)))
(paint (region "soma") (density (mechanism "BBP::NaTs2_t" ("gNaTs2_tbar" 0.98395500000000002))))
(paint (region "soma") (density (mechanism "BBP::SKv3_1" ("gSKv3_1bar" 0.30347200000000002))))
(paint (region "soma") (density (mechanism "BBP::SK_E2" ("gSK_E2bar" 0.0084069999999999995))))
(paint (region "soma") (density (mechanism "BBP::Ca_HVA" ("gCa_HVAbar" 0.00099400000000000009))))
(paint (region "soma") (density (mechanism "BBP::Ca_LVAst" ("gCa_LVAstbar" 0.00033300000000000002))))
(paint (region "soma") (density (mechanism "BBP::CaDynamics_E2" ("gamma" 0.00060899999999999995) ("decay" 210.48528400000001))))
(paint (region "soma") (density (mechanism "BBP::Ih" ("gIhbar" 8.0000000000000007e-05))))
(paint (region "axon") (ion-reversal-potential "na" 50))
(paint (region "axon") (ion-reversal-potential "k" -85))
(paint (region "axon") (ion-reversal-potential "na" 50 (scalar 1.0)))
(paint (region "axon") (ion-reversal-potential "k" -85 (scalar 1.0)))
(paint (region "axon") (density (mechanism "BBP::NaTa_t" ("gNaTa_tbar" 3.1379679999999999))))
(paint (region "axon") (density (mechanism "BBP::Nap_Et2" ("gNap_Et2bar" 0.0068269999999999997))))
(paint (region "axon") (density (mechanism "BBP::K_Pst" ("gK_Pstbar" 0.97353800000000001))))
Expand All @@ -26,11 +26,11 @@
(paint (region "axon") (density (mechanism "BBP::Ca_HVA" ("gCa_HVAbar" 0.00098999999999999999))))
(paint (region "axon") (density (mechanism "BBP::Ca_LVAst" ("gCa_LVAstbar" 0.0087519999999999994))))
(paint (region "axon") (density (mechanism "BBP::CaDynamics_E2" ("gamma" 0.0029099999999999998) ("decay" 287.19873100000001))))
(paint (region "dend") (membrane-capacitance 0.02))
(paint (region "dend") (membrane-capacitance 0.02 (scalar 1.0)))
(paint (region "dend") (density (mechanism "BBP::Ih" ("gIhbar" 8.0000000000000007e-05))))
(paint (region "apic") (ion-reversal-potential "na" 50))
(paint (region "apic") (ion-reversal-potential "k" -85))
(paint (region "apic") (membrane-capacitance 0.02))
(paint (region "apic") (ion-reversal-potential "na" 50 (scalar 1.0)))
(paint (region "apic") (ion-reversal-potential "k" -85 (scalar 1.0)))
(paint (region "apic") (membrane-capacitance 0.02 (scalar 1.0)))
(paint (region "apic") (density (mechanism "BBP::NaTs2_t" ("gNaTs2_tbar" 0.026145000000000002))))
(paint (region "apic") (density (mechanism "BBP::SKv3_1" ("gSKv3_1bar" 0.0042259999999999997))))
(paint (region "apic") (density (mechanism "BBP::Im" ("gImbar" 0.00014300000000000001))))
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
(arbor-component
(meta-data (version "0.1-dev"))
(meta-data (version "0.9-dev"))
(label-dict
(region-def "all" (all))
(region-def "soma" (tag 1))
Expand Down
26 changes: 13 additions & 13 deletions bluepyopt/tests/test_ephys/testdata/acc/l5pc_py37/l5pc_decor.acc
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
(arbor-component
(meta-data (version "0.1-dev"))
(meta-data (version "0.9-dev"))
(decor
(default (membrane-potential -65))
(default (temperature-kelvin 307.14999999999998))
(default (membrane-capacitance 0.01))
(default (axial-resistivity 100))
(default (membrane-potential -65 (scalar 1.0)))
(default (temperature-kelvin 307.14999999999998 (scalar 1.0)))
(default (membrane-capacitance 0.01 (scalar 1.0)))
(default (axial-resistivity 100 (scalar 1.0)))
(paint (region "all") (density (mechanism "default::pas/e=-75" ("g" 3.0000000000000001e-05))))
(paint (region "soma") (ion-reversal-potential "na" 50))
(paint (region "soma") (ion-reversal-potential "k" -85))
(paint (region "soma") (ion-reversal-potential "na" 50 (scalar 1.0)))
(paint (region "soma") (ion-reversal-potential "k" -85 (scalar 1.0)))
(paint (region "soma") (density (mechanism "BBP::NaTs2_t" ("gNaTs2_tbar" 0.98395500000000002))))
(paint (region "soma") (density (mechanism "BBP::SKv3_1" ("gSKv3_1bar" 0.30347200000000002))))
(paint (region "soma") (density (mechanism "BBP::SK_E2" ("gSK_E2bar" 0.0084069999999999995))))
(paint (region "soma") (density (mechanism "BBP::Ca_HVA" ("gCa_HVAbar" 0.00099400000000000009))))
(paint (region "soma") (density (mechanism "BBP::Ca_LVAst" ("gCa_LVAstbar" 0.00033300000000000002))))
(paint (region "soma") (density (mechanism "BBP::CaDynamics_E2" ("gamma" 0.00060899999999999995) ("decay" 210.48528400000001))))
(paint (region "soma") (density (mechanism "BBP::Ih" ("gIhbar" 8.0000000000000007e-05))))
(paint (region "axon") (ion-reversal-potential "na" 50))
(paint (region "axon") (ion-reversal-potential "k" -85))
(paint (region "axon") (ion-reversal-potential "na" 50 (scalar 1.0)))
(paint (region "axon") (ion-reversal-potential "k" -85 (scalar 1.0)))
(paint (region "axon") (density (mechanism "BBP::NaTa_t" ("gNaTa_tbar" 3.1379679999999999))))
(paint (region "axon") (density (mechanism "BBP::Nap_Et2" ("gNap_Et2bar" 0.0068269999999999997))))
(paint (region "axon") (density (mechanism "BBP::K_Pst" ("gK_Pstbar" 0.97353800000000001))))
Expand All @@ -26,11 +26,11 @@
(paint (region "axon") (density (mechanism "BBP::Ca_HVA" ("gCa_HVAbar" 0.00098999999999999999))))
(paint (region "axon") (density (mechanism "BBP::Ca_LVAst" ("gCa_LVAstbar" 0.0087519999999999994))))
(paint (region "axon") (density (mechanism "BBP::CaDynamics_E2" ("gamma" 0.0029099999999999998) ("decay" 287.19873100000001))))
(paint (region "dend") (membrane-capacitance 0.02))
(paint (region "dend") (membrane-capacitance 0.02 (scalar 1.0)))
(paint (region "dend") (density (mechanism "BBP::Ih" ("gIhbar" 8.0000000000000007e-05))))
(paint (region "apic") (ion-reversal-potential "na" 50))
(paint (region "apic") (ion-reversal-potential "k" -85))
(paint (region "apic") (membrane-capacitance 0.02))
(paint (region "apic") (ion-reversal-potential "na" 50 (scalar 1.0)))
(paint (region "apic") (ion-reversal-potential "k" -85 (scalar 1.0)))
(paint (region "apic") (membrane-capacitance 0.02 (scalar 1.0)))
(paint (region "apic") (density (mechanism "BBP::NaTs2_t" ("gNaTs2_tbar" 0.026145000000000002))))
(paint (region "apic") (density (mechanism "BBP::SKv3_1" ("gSKv3_1bar" 0.0042259999999999997))))
(paint (region "apic") (density (mechanism "BBP::Im" ("gImbar" 0.00014300000000000001))))
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
(arbor-component
(meta-data
(version "0.1-dev"))
(version "0.9-dev"))
(morphology
(branch 0 -1
(segment 0
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
(arbor-component
(meta-data (version "0.1-dev"))
(meta-data (version "0.9-dev"))
(decor
(paint (region "soma") (membrane-capacitance 0.01))
(paint (region "soma") (membrane-capacitance 0.01 (scalar 1.0)))
(paint (region "soma") (density (mechanism "default::hh" ("gnabar" 0.10299326453483033) ("gkbar" 0.027124836082684685))))))
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
(arbor-component
(meta-data (version "0.1-dev"))
(meta-data (version "0.9-dev"))
(label-dict
(region-def "all" (all))
(region-def "soma" (tag 1))
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
(arbor-component
(meta-data
(version "0.1-dev"))
(version "0.9-dev"))
(morphology
(branch 0 -1
(segment 0
Expand Down
Loading
Loading