Skip to content

Commit 061e77b

Browse files
Tom-WillemsenTom-Willemsen
authored
Merge pull request #84 from ISISComputingGroup/Ticket8614_mercury_IPS_magnet_supply
Ticket8614 mercury ips magnet supply
2 parents d332a93 + 909e98f commit 061e77b

File tree

3 files changed

+233
-13
lines changed

3 files changed

+233
-13
lines changed

‎doc/specific_iocs/cryogenics/Oxford-Instruments-IPS.md‎

Lines changed: 43 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,40 @@ The IOC implements the following state machine in SNL:
6464
* Set the IPS activity to "Ramp to Zero" and wait for the supplied current to drop to zero
6565
* At the end of this state, the magnet is set to "persistent" mode.
6666

67+
The SNL state machine is also described in the following diagram:
68+
```{mermaid}
69+
---
70+
title: IPS State Machine
71+
---
72+
stateDiagram-v2
73+
direction TB
74+
[*] --> initial
75+
at_field: At Field<br>statemachine_pv=1
76+
set_psu_to_match_magnet: Set PSU to match magnet<br>statemachine_pv=2
77+
wait_for_voltage_to_stabilise: Wait for voltage to stabilise<br>statemachine_pv=3
78+
ensure_switch_heater_warm: Ensure switch heater warm<br>statemachine_pv=4
79+
set_psu_output: Set PSU output<br>statemachine_pv=5
80+
switch_off_heater: Switch off heater<br>statemachine_pv=6
81+
ramp_down_psu: Ramp down PSU<br>statemachine_pv=7
82+
initial --> at_field: field_setpoint_alarm == 0
83+
at_field --> set_psu_to_match_magnet: field_setpoint != field_setpoint_readback<br>&& field_setpoint_alarm == 0<br> && (heater == 0 || heater == 2)
84+
at_field --> set_psu_output: field_setpoint != field_setpoint_readback<br>&& field_setpoint_alarm == 0<br>&& heater == 1
85+
set_psu_to_match_magnet --> wait_for_voltage_to_stabilise: field_setpoint_readback == magnet_field<br>&& psu_field == magnet_field<br>"&& ((sweepmode == SWEEP_MODE)<br> || (sweepmode == SWEEP_MODE_ALT))<br>&& (sweep_active == 0)
86+
set_psu_to_match_magnet --> set_psu_to_match_magnet: timeout 300s
87+
wait_for_voltage_to_stabilise --> ensure_switch_heater_warm: supplyvoltage_stable == 1
88+
wait_for_voltage_to_stabilise --> set_psu_to_match_magnet: field_setpoint_readback != field_setpoint_raw
89+
wait_for_voltage_to_stabilise --> set_psu_to_match_magnet: timeout 300s
90+
ensure_switch_heater_warm --> set_psu_output: activity == ACTIVITY_HOLD<br>&& heater == 1<br> && heater_ontime_ok == 1
91+
ensure_switch_heater_warm --> ensure_switch_heater_warm: timeout heater_wait_time + 30.0s
92+
set_psu_output --> switch_off_heater: field_setpoint_readback == field_setpoint<br>&& (field_setpoint == psu_field<br>&& persistent == 1<br>&& (sweep_active == 0)<br>
93+
set_psu_output --> at_field: field_setpoint_readback == field_setpoint<br>&& field_setpoint == psu_field<br>&& persistent == 0<br>&& (sweep_active == 0)<br>
94+
set_psu_output --> set_psu_output: timeout 300s
95+
switch_off_heater --> ramp_down_psu: heater == 0 || heater == 2<br>
96+
switch_off_heater --> switch_off_heater: timeout 30s
97+
ramp_down_psu --> at_field: psu_field == 0<br>&& (sweep_active == 0) <br>
98+
ramp_down_psu --> ramp_down_psu: timeout 300s
99+
```
100+
67101
## IOC notes
68102

69103
- The IOC does not enter it's state machine until a setpoint is set.
@@ -76,11 +110,11 @@ The IOC implements the following state machine in SNL:
76110

77111
There are various IPS units here is a list of the ones we know the settings for (Lab view settings are in `...LabVIEW Modules\Oxford Software\TESLTRON.INI`):
78112

79-
Setting | Description | IN LabVIEW | 7T Magnet | 9T Chopper Magnet (LET)
80-
------- | ----------- |---------------------------------------------|------------ | ---------------
81-
`STABILITY_VOLTAGE` | 5 voltages are compared and they must differ by less than this to be declared stable | hard coded | 0.1 | 0.1
82-
`MAX_FIELD` | Maximum field | `PS0MaxField(2.2K)` and `PS0MaxField(4.2K)` | 7.0 [1] | 8.8 [2]
83-
`MAX_SWEEP_RATE` | Maximum rate of change of the field in B/min | `PS0MaxSweep(4.2K)` and `PS0MaxSweep(2.2K)` | 0.399 | 0.3990
113+
| Setting | Description | IN LabVIEW | 7T Magnet | 9T Chopper Magnet (LET)|
114+
|---------------------| ----------- |---------------------------------------------|------------ | ---------------|
115+
| `STABILITY_VOLTAGE` | 5 voltages are compared and they must differ by less than this to be declared stable | hard coded | 0.1 | 0.1|
116+
| `MAX_FIELD` | Maximum field | `PS0MaxField(2.2K)` and `PS0MaxField(4.2K)` | 7.0 [1] | 8.8 [2]|
117+
| `MAX_SWEEP_RATE` | Maximum rate of change of the field in B/min | `PS0MaxSweep(4.2K)` and `PS0MaxSweep(2.2K)` | 0.399 | 0.3990|
84118

85119
[1] - The limit may be lower on some beamlines due to the construction of their sample area. Below are known cases:
86120

@@ -109,4 +143,7 @@ Check remote mode is enabled.
109143

110144
### Magnet system status reports something other than "normal" - e.g. "quenched" or "fault"
111145

112-
This indicates a hardware fault. Inform instrument scientists that the magnet has a problem and has likely gone to zero field (even if the IPS still claims to be producing a non-zero field). Consult cryogenics for help.
146+
This indicates a hardware fault. Inform instrument scientists that the magnet has a problem and has likely gone to zero field (even if the IPS still claims to be producing a non-zero field). Consult cryogenics for help.
147+
148+
## Related Documentation:
149+
[Oxford Instruments Mercury IPS](https://isiscomputinggroup.github.io/ibex_developers_manual/specific_iocs/cryogenics/Oxford-Instruments-Mercury-IPS.html)

‎doc/specific_iocs/cryogenics/Oxford-Instruments-Mercury-IPS.md‎

Lines changed: 187 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,192 @@
11
# Mercury IPS
22

3-
The Oxford Instruments Mercury IPS is a superconducting magnet power supply. It is the successor to the [IPS](Oxford-Instruments-IPS). Much of the information on the IPS wiki page also applies to the Mercury IPS.
3+
The Oxford Instruments Mercury IPS is a superconducting magnet power supply. It is the successor to
4+
the [IPS](Oxford-Instruments-IPS). Much of the information on the IPS wiki page also applies to the Mercury IPS.
45

5-
Note: although the Mercury IPS is the successor to the "old" IPS, cryogenics prefer to run the older IPS units as they are more reliable.
66

7-
## Hardware quirks
7+
## SCPI Protocol
8+
9+
The IPS IOC now supports the SCPI protocol, which is more feature rich than Legacy mode.
10+
11+
SCPI mode keeps the same PV interface used by the older IPS units wherever possible, to minimise the
12+
changes required to instrument python scripts when swapping between old & new controllers.
13+
The SNL state machine logic is identical between SCPI & legacy mode.
14+
15+
As already mentioned, SCPI mode provides additional status reporting, much of which is based on the
16+
return string from the `READ:SYS:ALRM` command, which is poorly documented in the supplier's
17+
documentation. The status strings are assumed to all conform to the "Directory of Alarms" section
18+
(17.3) of the Operator's Manual (Issue 20, July 2018).
19+
Whilst many of the system alarms that we could test, mostly conformed, some differences were
20+
noticed, along with additional, undocumented status, such as "Magnet Safety".
21+
It has not been possible to test all alarm scenarios with the IPS unit. Some messages are
22+
undocumented and were 'discovered'. Our best guess is that the IPS Manual presents some level of
23+
truth, but as such it has not been possible to fully ascertain that all the expected message
24+
strings are correct - they may need to be adjusted later on, if/when they arise.
25+
26+
Tested and verified to date:
27+
- PSU board open circuit
28+
- PSU board short circuit
29+
- Temperature board open circuit
30+
- Temperature board short circuit
31+
32+
The support module exports an aSub record subroutine to facilitate handling of the responses to
33+
`READ:SYS:ALRM`, which is not feasible with a StreamDevice protocol handler.
34+
See the section on [System Alarms](#system-alarms) below for more details.
35+
36+
### CONTROL and CONTROL:SP
37+
These records have been removed from the SCPI variant database
38+
as the SCPI command set does not manage the panel lock in the same was as legacy and
39+
Magnet Group advised to remove this feature, as they always want to be able to control
40+
the IPS via the front panel.
41+
42+
### SYSTEM:HWFAULT status is derived from the status bits via a calc record.
43+
This collates the various possible hardware faults into a single record,
44+
which in the legacy protocol is a single value (`Xm` bit 4).
45+
46+
### SWEEPMODE:PARAMS and SWEEPMODE:SWEEP
47+
These records have been removed as they are meaningless and underivable in SCPI protocol.
48+
49+
### _SWEEPMODE:SWEEP
50+
In the legacy version, this used to be the readback from `n` of `Mmn` part of Examine command
51+
return.
52+
- 0 output constant
53+
- 1, 2, 3 output changing
54+
55+
The SCPI protocol doesn't directly offer this, so it has to be derived via the `ACTIVITY` record
56+
(`DEV:GRPZ:PSU:ACTN`)\
57+
Direction from Alex Jones: "A response of `HOLD` or `CLMP` would be equivalent to `n=0` in the
58+
response of the `X` command in the old protocol. Responses of `RTOS` or `RTOZ` would be equivalent to
59+
`n=1`.
60+
There is no equivalent for the `m=0,1` fast/slow ramps, but we do not use this feature anyway."
61+
62+
### System Alarms
63+
System alarms are derived by interrogating the IPS with the `READ:SYS:ALRM?` SCPI command.
64+
This returns a comma-separated string of active alarms, or an empty string if no alarms are present.
65+
The string is parsed by an aSub record, which sets the relevant bits in a binary register.
66+
The bits are mapped to individual status records, which are then used to set the relevant alarms.
67+
The precise protocol was determined empirically by sending the command and observing the response
68+
directly from the instrument.
69+
Discrete records are generated for each board type and each alarm by the
70+
`scpi_system_alarms_discrete.template` and substitutions files.
71+
72+
The base response is: `READ:SYS:ALRM:` which may (or may not) be followed by a board identifier,
73+
a tab character (9) an alarm string and a semicolon.
74+
75+
In ABNF:<br />
76+
```
77+
response = "READ:SYS:ALRM:" *(error)
78+
error = board_id TAB error_message SEMICOLON
79+
80+
board_id = 1*(ALPHA / DIGIT / ".") ; e.g. MB1.T1
81+
error_message = 1*(ALPHA / DIGIT / ".") ; e.g. "Open circuit"
82+
TAB = %x09 ; tab character
83+
SEMICOLON = %x3B ; semicolon ";"
84+
```
85+
86+
The board identifiers are provided as macros:
87+
88+
| Macro | Default |
89+
|-------|----------|
90+
| BOARDID_MAG | MB1.T1 |
91+
| BOARDID_10TMAG | DB8.T1 |
92+
| BOARDID_PRESS | DB5.P1 |
93+
| BOARDID_LEVEL | DB1.L1 |
94+
95+
## Development Notes:
96+
Alex Jones has looked at some of the differences between the SCPI and legacy command set and has
97+
summarised some useful information, as quoted below:
98+
For the quench and overheat status, these (and many other issues) are dealt with by “Alarms” for
99+
the SCPI protocol. See the Magnet board section on p. 162 in Issue 20. These are read by
100+
`READ:DEV::PSU:STAT` in some undefined hex format for just the magnet-related alarms, and
101+
`READ:SYS:ALRM` for a list of everything.
102+
103+
### Some further information re UID naming:
104+
The UIDs can be either the nickname for the card (which we can set to anything we want) or related
105+
to the slot number and signal type. From the spreadsheet, the positional UID for the magnet
106+
temperature sensor will be `MB1.T1`, the UID for the level meter will be `DB1.L1` and the UID for
107+
the magnet supply will be `GRPZ` for all 4 of the systems.
108+
The 10T system will have an additional temperature sensor `DB8.T1` and a pressure sensor `DB5.P1`.
109+
110+
Devices connected to the motherboard are prefixed: `MB1`\
111+
Devices connected to a daughter-board are prefixed: `DB<slot #>`
112+
113+
### A comparison of availability of legacy vs SCPI features
114+
| Function | Legacy | SCPI |
115+
|---------------------------------|--------|---------------|
116+
| Trip current readback | `F17` | Not available |
117+
| Trip field readback | `F19` | Not available |
118+
| Ramp mode reporting (fast/slow) | Yes | Not available |
119+
| Status reporting | Yes | Detailed |
120+
| He Level reporting | No | Yes |
121+
| N2 Level reporting | No | Yes |
122+
| Pressure reporting | No | Yes |
123+
| Front panel control lock | Yes | No |
124+
125+
126+
127+
### PSU Status:
128+
Using the SCPI command: `READ:DEV:<UID>:PSU:STAT`\
129+
It is important to note that the STATus word should be examined at the group device level,
130+
not the individual PSU level.\
131+
It is also very important to mask out (ignore) all other bits in this 32-bit word (i.e. ones
132+
not defined in the list given below):\
133+
134+
| Status | Bit value |
135+
| ------ | --------- |
136+
| Switch Heater Mismatch | 00000001 |
137+
| Over Temperature [Sense Resistor] | 00000004 |
138+
| Over Temperature [Rundown Resistors] | 00000002 |
139+
| Over Temperature [PCB] | 00000008 |
140+
| Calibration Failure | 00000010 |
141+
| MSP430 Firmware Error | 00000020 |
142+
| Rundown Resistors Failed | 00000040 |
143+
| MSP430 RS-485 Failure | 00000080 |
144+
| Quench detected | 00000100 |
145+
| Catch detected | 00000200 |
146+
| Over Temperature [Sense Amplifier] | 00001000 |
147+
| Over Temperature [Amplifier 1] | 00002000 |
148+
| Over Temperature [Amplifier 2] | 00004000 |
149+
| PWM Cutoff | 00008000 |
150+
| Voltage ADC error | 00010000 |
151+
| Current ADC error | 00020000 |
152+
153+
154+
### Compromises with SCPI command set and EPICS
155+
`STS:SYSTEM:LIMIT`\
156+
Is an `mbbi` and has has 5 possible values:\
157+
- 0: "Normal"
158+
- 1: "On +ve V Limit"
159+
- 2: "On -ve V Limit"
160+
- 4: "Current too -ve"
161+
- 8: "Current too +ve"
162+
163+
The limit flags came from the legacy command: `Xmn`
164+
with the index denoted by the `n` value.
165+
SCPI does not provide this information.
166+
167+
## IOC Test Framework:
168+
With support for the new SCPI based IPS command set, there are now two sets of StreamDevice
169+
protocols. The appropriate protocol is implemented by use of a macro
170+
(`PROTOCOL` = `SCPI` | `LEGACY`) defined prior to running the IOC.
171+
172+
The test framework has been adapted by splitting the existing legacy tests into common tests
173+
and tests specific to either control interface. For instance, the legacy command set knows
174+
nothing about cryogen levels, which the SCPI command set does.
175+
176+
The IOC support provides two protocol files:
177+
- Legacy: OxInstIPS.protocol
178+
- SCPI: OxInstIPS_SCPI.protocol
179+
180+
The lewis emulator and IOC test framework for the Mercury IPS is located within the IPS support
181+
module. Manually running the tests is achievable using the following approach:
182+
1. `cd` to `C:\Instrument\Apps\EPICS\support\IPS\master\system_tests`
183+
2. `run_tests.bat -t ips_scpi -a -f` - this will run all the SCPI specific tests
184+
3. `run_tests.bat -t ips -a -f` - this will run all the legacy specific tests
185+
4. To run a specific test, use something like: `run_tests.bat -t ips_scpi.test_WHEN_inductance_set_via_backdoor_THEN_value_in_ioc_updates_0__0_12345 -a -f`
186+
- Note that the -a flag simply prompts as to whether to run tests or simply run the emulator.
187+
- The -f flag forces a fast fail on error and no further tests are run.
188+
189+
190+
## Related Documentation:
191+
[Oxford Instruments IPS](https://isiscomputinggroup.github.io/ibex_developers_manual/specific_iocs/cryogenics/Oxford-Instruments-IPS.html)
8192

9-
The following faults can be seen when operating the magnet fully from the front panel, but it is likely that software will also run into the same conditions:
10-
* The firmware will sometimes crash/freeze. To reset it, the whole power supply needs to be power-cycled. This is obviously undesirable for a magnet power supply, and cryogenics are chasing OI about this issue. It's not clear whether this issue is general to all Mercury IPS units or whether we have one faulty unit.
11-
* The switch heater occasionally reports that it's ON when it's actually OFF
12-
* The power supply reports a voltage of ~9000V which is incorrect (a sensible voltage for this power supply would be around ~8V while ramping)

‎doc/spelling_wordlist.txt‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -348,6 +348,7 @@ idn
348348
ie
349349
IFR
350350
ilm
351+
iLM200
351352
ImageJ
352353
ini
353354
inits
@@ -368,6 +369,7 @@ iocs
368369
iocsh
369370
ioLogik
370371
ip
372+
iPS
371373
IPv
372374
ipython
373375
ish
@@ -883,6 +885,7 @@ uncheck
883885
unclamped
884886
uncomment
885887
uncommented
888+
underivable
886889
unintuitive
887890
uno
888891
unpadded

0 commit comments

Comments
 (0)