Add ivscc_apfrequency operation

[MichaelHuth]

close #2581

• Set the opacity of individual traces to 0.2. Leave average trace opacity at 1.
• adapt xvalues operation to extract also from xvalues meta data
• add support to plotter for opacity meta data
• Support fitting the data (use the error bars of the average trace to weight the fit). Consider implementing a new operation that's passed as an optional argument into ivscc_APfrequency.
• find out how to get automatically determined xOffset constant from e.g. exp_XOffset integrated fit func after fit -> W_fitConstants
• ??add argument to ivscc_apfrequency that enables that all experiments/cells are fitted.
• add argument to preparefit to limit the fitting range. For flexibility and simplicity I suggest to add this through a mask wave in fit2 then.
• Make the fit parameters accessible with a related operation
• ??the fit result may be based on the acquisition/sweep sequence rather than an ordered list of current intensity (x-scale values) -> 3rd avg mode?
• add a way to expose the number and type of required coefficients to the user. Perhaps a link to the relevant Igor documentation, and/or a pop-up? -> add to help of preparefit
• Another piece of data I'd like the operation to return is the individual and average DA scales at which a negative slope was first observed in the FI data.
• add argument to the operation to limit the current range of the avg trace points that are fitted. The current is in pA and refers to an x-axis range. The operation crops the point range that goes into the fit automatically. The argument is like [start_pA, end_pA].
• ?? for the single experiment f-I plots should the f-I pairs be sorted by I before displaying (prevents zig-zag connecting lines)

TBs new todos:

• Data on the FTP for regression tests: pr-2599/inputRegressionTest
• C&P into Excel does not work as multiple entries are added into one cell
• Fix else // SF_OP_IVSCCAPFREQUENCY_NONE branches and add fatal error
• Allow to add tags to experiments in the analysis browser
  - Tags are simple strings (igor object names) separated by e.g. comma: Add input functionality to tag experiments in AB. Tags are not saved to disk (except experiment is saved), but only there while the window is open.
  - Show them in an analysis browser column
  - We expect 10 or more experiments per tag, so convenient GUI is important
  - Add select variant to select data by one of multiple tags: This new select filter should work with select itself and as additional optional argument for ivscc_apfrequency. seltag([tag1, tag2, ...]) For ivscc_apfrequency we support that the seltag is given as array [seltag([tag1, tag2, ...]), seltag([tag1, tag2, ...]), ...]. ivscc_apfrequency should create one group for every seltag in the array.
  - If no seltag is given then ivscc_apfrequency should default to create one group for every list of tags present in the experiments, create one group for experiments without tags.
  - Mention list of tags in the legend.
  - When used with ivscc_apfrequency the following special analysis is applied:
  - Support seltag in ivscc_apfrequency` so that each group of experiments from the same seltag generates:
  - single experiment f-I plot (same as now)
  - one additional f-I plot where all f-I pairs from the single experiments are concatenated (and sorted?)
  - average as before (just including data for this seltag)
  - fit as before
• Add argument option for ivscc_apfrequency to turn off the traces for single experiment plots
  (Original text :one f-I plot in addition to the single experiment plots. Generate one average for each group and no global (over all data across groups) average anymore. Allow to turn off the single experiment plots.)
• IVSCC apfrequency: Prep 3 of x #2706
• getmeta
  • Tests
  • Docu
• fit2
  • Tests
  • Docu
• preparefit
  • Tests
  • Docu
• ivscc_apfrequency
  • Tests
  • Docu
• ihf docu

close #2628

[MichaelHuth]

@timjarsky This is a first version to play around. There are still a few things I have to add, that I discuss farther below.

ivscc_apfrequency([xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage]) where [] denotes that the parameters are optional.

xaxisOffset and yaxisOffset are strings that can be min, max and none.

General Plotting Behavior

The operation itself returns internally a full plotting specification that is inserted by the formula plotter at the location where the operation appears in the notebook code.

The operation creates only traces that are separated by with. This means preceding and succeeding formulas that are also separated with with go to the same subwindow.

xAxisPercentage and yAxisPercentage are treated as properties for the plot by the plotter. Specifically plot means in this context the subwindow where the traces go. Plot properties from the last formula in the with chain are applied. (i.e. there is currently no gathering of plot properties).

Thus, the plotter applies the 10% for x and y-axis when used like this, where the formula setting the plot properties is last in the chain:

1
with
ivscc_apfrequency(min, min, 10, 10)

but not for this:

ivscc_apfrequency(min, min, 10, 10)
with
1

because 1 does not include any plot properties.

Operation Arguments

Currently:

ivscc_apfrequency([xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage])

with xaxisOffset and yaxisOffset as min, max, none and xAxisPercentage, yAxisPercentage a number between 0 and 100.

The (later) final arguments should also expose arguments from apfrequency, so it will change to ivscc_apfrequency([xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage, method, level, resultType, normalize, xAxisType).

The default for xaxisOffset and yaxisOffset is min.

On the basis of the experiment avgMethodTesting2.pxp the generated code is:

sel = select(selsweeps(), selstimset("*rheo*", "*supra*"), selvis(all))
selexpAD0 = select(selexp("C57BL6J-734969.15.10A.01.nwb"), $sel, selchannels(AD0), selrange(E1))
selexpDA0 = select(selexp("C57BL6J-734969.15.10A.01.nwb"), $sel, selchannels(DA0), selrange(E1))
freq0 = apfrequency(data($selexpAD0))
current0 = max(data($selexpDA0))
currentNorm0 = $current0 - extract($current0, 0) #*1
selexpAD1 = select(selexp("C57BL6J-734969.15.10B.01.nwb"), $sel, selchannels(AD0), selrange(E1))
selexpDA1 = select(selexp("C57BL6J-734969.15.10B.01.nwb"), $sel, selchannels(DA0), selrange(E1))
freq1 = apfrequency(data($selexpAD1))
current1 = max(data($selexpDA1))
currentNorm1 = $current1 - extract($current1, 0) #*2
ivsccavg = avg([$freq0,$freq1], group)
ivscccurrentavg = avg([$currentNorm0,$currentNorm1], group)

$freq0 - extract($freq0, 0) #*3
vs
$currentNorm0
with
$freq1 - extract($freq1, 0) #*4
vs
$currentNorm1
with
$ivsccavg - extract($ivsccavg, 0) #*5
vs
$ivscccurrentavg - extract($ivscccurrentavg, 0) #*6

I added a #* to the formulas that change depending on the min, max, none setting.

#*1, #*2 and #*6 depend on the xaxisOffset argument with the following logic:

min: currentNormX = $currentX - extract($currentX, 0)
max: currentNormX = $currentX - max(flatten($currentX))
none: currentNormX = $currentX

for #*6 it is:
min: $ivscccurrentavg - extract($ivscccurrentavg, 0)
max: $ivscccurrentavg - max(flatten($ivscccurrentavg))
none: $ivscccurrentavg

#*3, #*4 and #*5 depend on the yaxisOffset argument with the following logic:

min: $freqX - extract($freqX, 0)
max: $freqX - max(flatten($freqX))
none: $freqX

for #*5 it is:
min: $ivsccavg - extract($ivsccavg, 0)
max: $ivsccavg - max(flatten($ivsccavg))
none: $ivsccavg

I need to add a flatten operation because the result in e.g. $freqX are single values in 7 datasets (for avgMethodTesting2.pxp). Thus, our max() operation would determine the max in each dataset individually, but that is not what is wanted. The flatten operation should change n datasets with a single data point to an array with n elements.

Therefore, the max argument is currently not implemented yet until I have the flatten operation implemented.

An additional task from the issue is to add a variable that contains the names of the experiments. I can create this variable in the operation and add it to the variable storage of the formula notebook. It would be available then after the operation ran.
For now I thought of $ivscc_apfrequency_explist as name and it would be a string array.

[timjarsky]

@MichaelHuth

Thanks for handling the metadata management for mismatched sweep numbers across experiments.

A few points for discussion:

1. The marker coloring makes it hard for me to evaluate the output (see plot image below). Can we use a single color for each experiment? Can the data be a single 1D wave instead of multiple x-y waves (this will enable fitting)?

2. There are many zero-frequency measurements spread across the x-axis.

3. The average seems more variable than the input data, perhaps because there are too many data points?

4. I'm not sure about the utility of the negative current values with "min" and "max" (second image). The two axes options that I'm sure are needed are none (where each FI curve starts at zero current and zero frequency) and the absolute current values.

ivscc_apfrequency(none, none)

ivscc_apfrequency()

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[MichaelHuth]

@timjarsky

1. I changed that. There are now experiment + 1 traces with markers. The +1 trace is the average in the average color. The experiment traces use the common table for trace colors, with one color per experiment. I also added code to show the experiment name in the legend for these.
2. I still have to look into this, why a lot of the results return a zero. Currently the standard setting of apfrequency for the internal apfrequency call are used.
3. Yes, it looks a bit like that. I did not see that in my local testing with the three experiments though. I guess in the graphs you pasted there is for some of the average points only a single sweep for an index in the groups.
   e.g. first exp has 10 and second exp has 11 sweeps selected, then sweep data for index 10 is only present from the second experiment -> only one sweep goes into the average and then the trace point from the experiment and the trace point from the average are equal. The average is shown in front and covers the point from the experiment. You may check for the average points on the very top if there is a data point from an experiment underneath.
4. I think it is the other way around, with none no offset is applied. min moves the trace to zero and max moves the trace such that the former maximum point is at zero.
   What I think might be unexpected with the initial formula construct is that there is an offset used per trace. Thus, when changing none -> min then each trace start is moved to zero separately and the visual relation between the traces shifts. (same applies for max)
   This per-trace behavior is the same for the xaxisOffset and yaxisOffset.

[MichaelHuth]

@timjarsky
Regarding 2.: The default level for apfrequency is 0. This results in zero peaks found for sweeps where E1 does not cross the zero line and for this apfrequency returns 0 as result with the default arguments.

I have added support for the apfrequency argument block after the first four argument for ivscc_apfrequency. The arguments are now:
ivscc_apfrequency([xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage, method, level, resultType, normalize, xAxisType])

The last four arguments are "forwarded" to apfrequency.

[timjarsky]

@MichaelHuth, Are failing sweeps included or filtered out? If included, please update to use only passing sweeps.

[t-b]

@MichaelHuth I'm currently uploading test data to the FTP, folder /issue-2656 which we used for adaptive testing. According to Tim this also works here. Should be finished in a bit.

[MichaelHuth]

@timjarsky New version, with support for seltag.

// with avgMode bins (default)
// ivscc_apfrequency([seltag, showSingleExp, xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage, prepareFit([...]), avgMode, binRange, binWidth, method, level, timeFreq, normalize, xAxisType])
//
// for avgMode: bins2
// ivscc_apfrequency([seltag, showSingleExp, xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage, prepareFit([...]), avgMode, method, level, timeFreq, normalize, xAxisType])

// seltag: optional, single or array of seltag
// showSingleExp: optional, on or off [default: on]
// xaxisOffset: first, min, max, none [default: min]
// yaxisOffset: first, min, max, none [default: min]
// xAxisPercentage: [0, 100]
// yAxisPercentage: [0, 100]
// prepareFit: See prepareFit([...]) operation, use prepareFit() to not make any fitting
// avgMode: bins, bins2 [default: bins]
// binRange: default: [0, 1000]
// binWidth: default: 100
//
// method, level, timeFreq, normalize, xAxisType are from apfrequency() with same defaults

In AB add tags, then load sweeps in SB.

The first agument seltag select exactly that tags give:
ivscc_apfrequency(seltag(tag1)) // selects experiments with exactly and only "tag1"
ivscc_apfrequency(seltag(tag1,tag2)) // selects experiments with exactly and only "tag1, tag2"
ivscc_apfrequency([seltag(tag1), seltag(tag2)]) // selects two groups of experiments, first with exactly and only "tag1", second with exactly and only "tag2"
ivscc_apfrequency(seltag("")) // select only experiments with no tag

When seltag argument is omitted, then it looks through all experiments and creates a group for all different tags lists. So if you have "tag1", "tag2", "tag1,tag2", "" then four groups of experiments are created.

To omit seltag you start the arguments for ivscc_apfrequency with showSingleExp, e.g. ivscc_apfrequency(off, ...).

If a resulting group has only one experiment, then no average trace and no fit from the average is created.

Slight adaptation in the calculation:

• the f-I pairs of single experiments are sorted ascending by I for the trace, that prevents the previous seen zig-zagging of the lines. Now connecting the dots with a line makes sense from an analytical point
• when the inflection point of the frequency is determined, some experiments yield no result (i.e. the frequency slope never changes to negative). In that case the averaging yielded NaN. I zap the NaNs now and average only the results from the experiments that show the slope change.

The new concatenated data zig-zags between the result points from the single experiment., which is to be expected. I sorted these f-I pairs as well.

Here are five experiments loaded, where the first two are tagged with "tag1" and the other 3 with "tag2", the formula is ivscc_apfrequency():

The dashed line are the concatenated points.

THe output variable names have changes, as there is now a set for each group. The variables have now the tag list of the group suffixed. So for a group with tags "a,b" the variables are:

ivscc_apfrequency_explist_a_b
ivscc_apfrequency_fit_a_b

The , get replaced by _. If there are experiments with no tags then the suffix for that group is X.

[claude]

Claude Code Review

Claude Code Review is paused for this repository. To reconnect it, an admin of this repository's GitHub organization (or the account owner, for personal repositories) who can also manage your Claude organization's Code Review settings needs to re-link GitHub in Code Review settings. This is a one-time step.

_{Tip: disable this comment in your organization's Code Review settings.}

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 8 comments.

[timjarsky]

Hi @MichaelHuth , I'm having difficulty getting the latest version to work. Can you share an example formula? It looks like I'm making a basic syntax mistake:

[MichaelHuth]

Hi @MichaelHuth , I'm having difficulty getting the latest version to work. Can you share an example formula? It looks like I'm making a basic syntax mistake:

The seltag opertion takes a single argument, so it is either seltag([control, test]) with a text array or [seltag([control), seltag(test)] as array of seltags. The first selects experiments that have exactly the tags "control" and "test" and creates one group for these. The second variant creates two groups, one with experiments that have exactly and only the tag "control" and a second with exactly and only the tag "test".

The second argument (ShowSingleExp) is not an operation, but either "on" or "off" as simple text argument.

I think you want something like this:
ivscc_apfrequency([seltag(control), seltag(test)], on, $xAxisOffset, $yAxisOffset, $xAxisPercent, $yAxisPercent, $pfit, bins2)

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 7 comments.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 7 comments.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (6)

Packages/MIES/MIES_SweepFormula_Operations.ipf:3104

• ivscc_apfrequency argument parsing currently always treats the first non-seltag argument as showSingleExp (allowed: on/off). That makes calls like ivscc_apfrequency(none, none, 100, ...) fail because none is parsed as showSingleExp instead of xaxisOffset. The tests/docs in this PR use none as the first argument, so this needs disambiguation (e.g., only consume showSingleExp if the next token is on/off; otherwise default to on and treat the token as xaxisOffset).

	args.showSingleExp   = SFH_GetArgumentAsText(exd, opShort, argOffset + 0, defValue = SF_OP_IVSCCAPFREQUENCY_SHOWSINGLE_ON, allowedValues = {SF_OP_IVSCCAPFREQUENCY_SHOWSINGLE_ON, SF_OP_IVSCCAPFREQUENCY_SHOWSINGLE_OFF})
	args.xaxisOffset     = SFH_GetArgumentAsText(exd, opShort, argOffset + 1, defValue = SF_OP_IVSCCAPFREQUENCY_MIN, allowedValues = {SF_OP_IVSCCAPFREQUENCY_FIRST, SF_OP_IVSCCAPFREQUENCY_MIN, SF_OP_IVSCCAPFREQUENCY_MAX, SF_OP_IVSCCAPFREQUENCY_NONE})
	args.yaxisOffset     = SFH_GetArgumentAsText(exd, opShort, argOffset + 2, defValue = SF_OP_IVSCCAPFREQUENCY_MIN, allowedValues = {SF_OP_IVSCCAPFREQUENCY_FIRST, SF_OP_IVSCCAPFREQUENCY_MIN, SF_OP_IVSCCAPFREQUENCY_MAX, SF_OP_IVSCCAPFREQUENCY_NONE})
	args.xAxisPercentage = SFH_GetArgumentAsNumeric(exd, opShort, argOffset + 3, defValue = 100, checkFunc = BetweenZeroAndOneHoundred)
	args.yAxisPercentage = SFH_GetArgumentAsNumeric(exd, opShort, argOffset + 4, defValue = 100, checkFunc = BetweenZeroAndOneHoundred)

Packages/MIES/MIES_SweepFormula_Operations.ipf:3309

• In bins2 mode, xaxisOffset=first currently does not apply an offset (it sets ivsccavg_norm_x to the unmodified $ivsccavg_xvalues) even though the docs describe first as subtracting the first x-value. This makes first behave like none for the averaged x-values.

		if(!CmpStr(args.xaxisOffset, SF_OP_IVSCCAPFREQUENCY_FIRST))
			// TODO what should we do here, the obvious thing below does not make sense
			//	expr = "ivsccavg_norm_x = merge($ivscccurrentavg - extract($ivscccurrentavg, 0, 0))"
			expr = "ivsccavg_norm_x = $ivsccavg_xvalues"

Packages/tests/HistoricData/UTF_HistoricSweepFormula.ipf:261

• Same tolerance issue here: use DEFAULT_TOL instead of a hardcoded 1E-12 so comparisons stay consistent across the test suite.

	CHECK_EQUAL_WAVES(fitData, refData[%fitdata], mode = WAVE_DATA, tol = 1E-12)
	WAVE xFitData = XWaveRefFromTrace(wName2, traceList2[5])
	CHECK_EQUAL_WAVES(xFitData, refData[%fitxdata], mode = WAVE_DATA, tol = 1E-12)

Packages/doc/SweepFormula.rst:2549

• The documentation says the global inflection-point average is plotted as a "red filled caro". Besides the typo, the code sets the marker (18) and uses the standard average trace color, not explicitly red. Consider describing this trace in a way that matches the actual styling (or update the styling to match the docs).

The over all experiments averages current/frequency pair where the slope of the frequency changes from positive to negative are plotted as red filled caro.

Packages/doc/SweepFormula.rst:2569

• Docs for ivscc_apfrequency don’t mention the optional seltag(...) / tag grouping behavior or the showSingleExp on/off flag, and the documented “implicitly created variables” (ivscc_apfrequency_explist, ivscc_apfrequency_fit, ivscc_apfrequency_inflection_*) don’t match what the implementation currently persists (it restores the var storage and only adds suffixed ..._explist_<tagSuffix> / ..._fit_<tagSuffix>, and doesn’t store inflection arrays). Please align the docs with the actual API/variable naming, or adjust the implementation to match the documented variables.

   # averaging mode: bins
   ivscc_apfrequency([xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage, prepareFitSpec, avgMode, binRange, binWidth, method, level, timeFreq, normalize, xAxisType])
   # averaging mode: bins2
   ivscc_apfrequency([xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage, prepareFitSpec, avgMode, method, level, timeFreq, normalize, xAxisType])

The argument `xaxisOffset` sets an offset for the x-values (current) from the apfrequency analysis that is applied to
the result from each selected experiment as well as to the averaged data. Default is `min`.

+-------------------+--------------------------------------------------------+
| xaxisOffset       | description                                            |
+===================+========================================================+
| `first`           | The subtracted offset is the first x-value (current)   |
+-------------------+--------------------------------------------------------+
| `min`             | The subtracted offset is the minimum x-value (current) |
+-------------------+--------------------------------------------------------+
| `max`             | The subtracted offset is the maximum x-value (current) |
+-------------------+--------------------------------------------------------+
| `none`            | The x-values (current) are kept as is                  |
+-------------------+--------------------------------------------------------+

The argument `yaxisOffset` sets an offset for the y-values (apfrequency result) from the apfrequency analysis that is applied to
the result from each selected experiment as well as to the averaged data. Default is `min`.

+-------------------+-------------------------------------------------------------------+
| yaxisOffset       | description                                                       |
+===================+===================================================================+
| `first`           | The subtracted offset is the first y-value (apfrequency result)   |
+-------------------+-------------------------------------------------------------------+
| `min`             | The subtracted offset is the minimum y-value (apfrequency result) |
+-------------------+-------------------------------------------------------------------+
| `max`             | The subtracted offset is the maximum y-value (apfrequency result) |
+-------------------+-------------------------------------------------------------------+
| `none`            | The y-values (apfrequency result) are kept as is                  |
+-------------------+-------------------------------------------------------------------+

The argument `xAxisPercentage` sets which percentage in x-direction of the full plot area is used by the plot. Default is `100`.

The argument `yAxisPercentage` sets which percentage in y-direction of the full plot area is used by the plot. Default is `100`.

The argument `prepareFitSpec` is a fitting specification that has to be created through the `preparefit` operation. It defines how the
results from the averaging are fitted. The default is `preparefit()`, that disables the fit.

The argument `avgMode` specifies the averaging mode that is applied to average the apfrequency results from the sweeps of the input experiments.
The modes `bins` and `bins2` are allowed. Default is `bins`. See documentation for the operation `avg` for the specifics of these modes.

When `avgMode` is `bins` the following two arguments set the bin range with the `binRange` argument and the bin width with the `binWidth` argument.
These two arguments are not optional. The argument `binRange` must be a two element array in the form `[start, end]`. The number of bins results
automatically from the given range and width of the bins.

The following arguments specify the settings for the apfrequency analysis and are the same as for the operation `apfrequency`.
The arguments are `method`, `level`, `timeFreq`, `normalize` and `xAxisType`. The defaults are the same as for the `apfrequency` operation.

Details of the `ivscc_apfrequency` analysis:
First the user has to have one or more experiments with data loaded in the sweep browser.
For each experiment the sweeps with the following properties are selected:

* the stimset name contains `LP_Rheo` or `supra`
* the sweep passed the sweepqc check

For each experiment apfrequency analysis is run on all selected sweeps on epoch `E1` with the apfrequency settings specified.
The result is paired with an x-value that is the maximum current (pA) output in the same epoch (`E1`).
Per experiment the apfrequency and current results yield multiple datasets depending on the number of sweeps in the selection.

All x-values (current) are offsetted by the specified `xaxisOffset`.

In average mode `bins`:
The apfrequency results are averaged over each experiment with the given `binRange` and `binWidth` setting. The offsetted current is used for the binning.
The currents are averaged over each experiment with the given `binRange` and `binWidth` setting. The offsetted current is used for the binning.
This results in multiple datasets (one for each experiment) with a single result value each.
The averaged currents are offsetted by the `xaxisOffset` specification and merged into a single dataset with number of points equal to the number of bins.

In average mode `bins2`:
The apfrequency results are averaged over each experiment. The offsetted current is used for the binning.
The attached x-values (current) are extracted and merged to an own dataset with number of points equal to the number of bins.
The averaged currents are offsetted by the `xaxisOffset` specification.

The averaged apfrequency results are offsetted by the `yaxisOffset` specification and merged to a single dataset with points equal to the number of bins.

For each experiment the merged and offsetted currents and frequencies value pairs are sorted by current.
If an experiment has multiple results where frequencies were calculated from the same current (e.g. from different sweeps) the frequencies are averaged.
Then the current is determined where the slope changes from positive to negative for the first time. If no such inflection point exists then NaN is returned for that experiment.
This results in current/frequency pairs from each experiment where the slope changed.
Additionally these get averaged for a global current/frequency pair from all experiments.

The following traces are output:
For each experiment the apfrequency result offsetted by `yaxisOffset` specification vs. the current offsetted by the `xaxisOffset` specification.
These traces are colored according to the headstage at 20% opacity and use lines and filled circles as markers.

The current/frequency pairs where the slope of the frequency changes from positive to negative are plotted as red markers with diagonal cross.

The over all experiments averages current/frequency pair where the slope of the frequency changes from positive to negative are plotted as red filled caro.

The offsetted averaged apfrequency results are plotted vs. the offsetted averaged currents.

When a valid fitting specification was given through `prepareFitSpec` then the offsetted averaged apfrequency results are fitted (with the offsetted averaged currents as x-values).
The x- and y-errors from the averaging (standard deviation) are used in the fit as weights.
Thus, with x-errors orthogonal distance regression is applied in the fit. The fit result is plotted as trace.

Implicitly created variables:
The operation `ivscc_apfrequency` creates variables in the sweepformula variable storage. These variables are available after the `ivscc_apfrequency` call.
If a variable with that name existed previously it is overwritten.

+----------------------------------------+------------------------------------------------------------------------------------------------------------------------+
| variable                               | description                                                                                                            |
+========================================+========================================================================================================================+
| `ivscc_apfrequency_explist`            | A string array containing the experiment list `ivscc_apfrequency` worked on                                            |
+----------------------------------------+------------------------------------------------------------------------------------------------------------------------+
| `ivscc_apfrequency_fit`                | The result of the fit. The result can be used e.g. with `getmeta` to retrieve more information from the fit.           |
|                                        | See also operation `fit2`. Note: Even if no fit is displayed as trace that can happen if the fit resulted in an error, |
|                                        | meta information can still be retrieved.                                                                               |
+----------------------------------------+------------------------------------------------------------------------------------------------------------------------+

Packages/tests/HistoricData/UTF_HistoricSweepFormula.ipf:171

• Avoid hardcoding numeric tolerances in tests; the codebase uses DEFAULT_TOL as the shared absolute tolerance for comparisons. Using a single constant keeps tolerances consistent and makes it easier to adjust globally if needed.

This issue also appears on line 259 of the same file.

	CHECK_EQUAL_WAVES(avgData, refData[%avgbinsdata], mode = WAVE_DATA, tol = 1E-12)
	CHECK_EQUAL_WAVES(xAvgData, refData[%xavgbinsdata], mode = WAVE_DATA, tol = 1E-12)

	WAVE fitData = TraceNameToWaveRef(wName, traceList[5])
	CHECK_EQUAL_WAVES(fitData, refData[%fitbinsdata], mode = WAVE_DATA, tol = 1E-12)
	WAVE xFitData = XWaveRefFromTrace(wName, traceList[5])
	CHECK_EQUAL_WAVES(xFitData, refData[%xavgbinsdata], mode = WAVE_DATA, tol = 1E-12)

[t-b]

@MichaelHuth What is the status here? Can Tim already playaround some more?

[MichaelHuth]

Yes, he can still play around. I just rebased, such that the new tag control dialog is there and the few fixed for seltag are in.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

Packages/MIES/MIES_SweepFormula_Operations.ipf:3104

• ivscc_apfrequency currently parses showSingleExp as the first non-seltag argument (allowed: on/off). However, the new documentation and tests call the operation with xaxisOffset as the first argument (e.g. ivscc_apfrequency(none, none, 100, 100, ...)), which would make this implementation reject valid calls ("none" is not on/off). Please align the argument parsing with the intended public syntax (either move showSingleExp to a trailing optional argument, or implement a disambiguation: if the first arg is on/off treat it as showSingleExp, otherwise treat it as xaxisOffset and shift the remaining arguments).

	args.showSingleExp   = SFH_GetArgumentAsText(exd, opShort, argOffset + 0, defValue = SF_OP_IVSCCAPFREQUENCY_SHOWSINGLE_ON, allowedValues = {SF_OP_IVSCCAPFREQUENCY_SHOWSINGLE_ON, SF_OP_IVSCCAPFREQUENCY_SHOWSINGLE_OFF})
	args.xaxisOffset     = SFH_GetArgumentAsText(exd, opShort, argOffset + 1, defValue = SF_OP_IVSCCAPFREQUENCY_MIN, allowedValues = {SF_OP_IVSCCAPFREQUENCY_FIRST, SF_OP_IVSCCAPFREQUENCY_MIN, SF_OP_IVSCCAPFREQUENCY_MAX, SF_OP_IVSCCAPFREQUENCY_NONE})
	args.yaxisOffset     = SFH_GetArgumentAsText(exd, opShort, argOffset + 2, defValue = SF_OP_IVSCCAPFREQUENCY_MIN, allowedValues = {SF_OP_IVSCCAPFREQUENCY_FIRST, SF_OP_IVSCCAPFREQUENCY_MIN, SF_OP_IVSCCAPFREQUENCY_MAX, SF_OP_IVSCCAPFREQUENCY_NONE})
	args.xAxisPercentage = SFH_GetArgumentAsNumeric(exd, opShort, argOffset + 3, defValue = 100, checkFunc = BetweenZeroAndOneHoundred)
	args.yAxisPercentage = SFH_GetArgumentAsNumeric(exd, opShort, argOffset + 4, defValue = 100, checkFunc = BetweenZeroAndOneHoundred)

Packages/doc/SweepFormula.rst:2463

• The ivscc_apfrequency documentation lists the call signature starting with xaxisOffset/yaxisOffset, but the implementation also supports a leading optional seltag argument and a showSingleExp on/off switch, and its current parsing order differs. Please update this section to reflect the actual supported syntax (including seltag and showSingleExp, their defaults, and their position) so users can call the operation correctly.

   # averaging mode: bins
   ivscc_apfrequency([xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage, prepareFitSpec, avgMode, binRange, binWidth, method, level, timeFreq, normalize, xAxisType])
   # averaging mode: bins2
   ivscc_apfrequency([xaxisOffset, yaxisOffset, xAxisPercentage, yAxisPercentage, prepareFitSpec, avgMode, method, level, timeFreq, normalize, xAxisType])

Packages/MIES/MIES_SweepFormula_Operations.ipf:3310

• In bins2 mode, the xaxisOffset=first branch currently does not apply any offset (it behaves like none) and leaves a TODO. This makes first inconsistent with the documented semantics and with the bins implementation. You can implement first for bins2 by subtracting the first xvalue from ivsccavg_xvalues.

		if(!CmpStr(args.xaxisOffset, SF_OP_IVSCCAPFREQUENCY_FIRST))
			// TODO what should we do here, the obvious thing below does not make sense
			//	expr = "ivsccavg_norm_x = merge($ivscccurrentavg - extract($ivscccurrentavg, 0, 0))"
			expr = "ivsccavg_norm_x = $ivsccavg_xvalues"
		elseif(!CmpStr(args.xaxisOffset, SF_OP_IVSCCAPFREQUENCY_MIN))

[t-b]

@timjarsky Ready to playaround, the merge conflicts are harmless.

[timjarsky]

@MichaelHuth I uploaded a BIG file (Dravet _sst_07-29-2026.pxp) onto the FTP that has the following error:

  MIES BUG_TS: Encountered pending RTE: 1321, a wave read; Index out of range for wave "merge_output_61".

This error is associated with failing to plot 1 of the 3 FI curves the formula calls for.

A few other nitpicks:

1. Can the confidence bands around the fit be made semi-transparent?
2. DA scale values from individual experiments should be taken at the max frequency for the FI curve, for an FI curve that finishes with a negative slope. This is to avoid the occasional DA_scale value that's drawn from the curve just prior to the first negative slope.

3. Make the marker colors for the different tag groupings different colors.