Validation · Data and Studio
How we tested the numbers in Data and Studio
Data and Studio do a lot of maths in your browser. They fit peaks and curves, run statistical tests and calculate method results such as band gaps, crystallite sizes, diffusion coefficients and activation energies. If one of these numbers is wrong, it can end up in a thesis or a paper. So we wrote tests that compare our results with sources that scientists already trust: R, SciPy, statsmodels, worked examples from textbooks and papers, and exact formulas. On this page we explain what we tested, what we compared it with and how close our results were.
Summary
.mpr files. Every one matched the text file that EC-Lab exported from the same run.Please keep in mind: these tests show that the calculations are done correctly. They cannot tell you whether a method is the right choice for your sample. Also, many of our examples use synthetic (made-up) data, which is much cleaner than a real measurement. We mention it every time this is the case.
Our approach
We did not write a separate copy of the maths for testing. The tests load the same code that runs on the website and feed it data where the right answer is already known. Then they compare the answer with the reference. If the difference is bigger than the allowed tolerance, the test fails and prints both numbers.
| Code under test | The website's own TypeScript files (src/lib/**), loaded in Node 24 with Vite |
|---|---|
| Reference software | R 4.x results from well-known example data sets (sleep, PlantGrowth, warpbreaks, anscombe, aml and the wilcox.test help page). SciPy 1.13, statsmodels 0.14 and NumPy 1.26 results made with scripts/labstats-reference.py and saved in scripts/labstats-reference.json |
| Other references | Worked examples from the literature (for example Bates and Watts 1988 for puromycin, Lakowicz for Förster distances, Bard and Faulkner for Randles–Ševčík, Dunnett's 1955 tables), exact formulas, counting every possible case by brute force, and simulations where we chose the true values ourselves |
| How close is close enough | For statistics, a relative difference of 10⁻⁶ or less by default. For test statistics we ask for 10⁻⁹ and for P values 10⁻⁷ when the reference is exact. For method results on noisy synthetic data, the result must be within a stated range of the true value (for example 1.5 percentage points for helix content in CD, or 3 % for crystallite size) |
| When the tests run | With npm test before each release. The numbers on this page come from the run on 20 October 2026 |
| Test script | What it tests | Checks |
|---|---|---|
check-significance.mjs | Every P value, interval, band and test that appears on a figure, compared with R | 207 |
check-labstats.mjs | Normality, outliers, survival, Deming and logistic regression, power, Prism equations and signal tools, compared with SciPy and statsmodels | 354 |
check-methods-root.mjs | The 38 method analyses and their 43 examples | 419 |
check-figures.mjs | Kaplan–Meier, Q–Q, density, contour and beeswarm plots, P-value wording, axis labels and journal presets | 255 |
check-integrity.mjs | What the figure editor is and is not allowed to change | 85 |
check-assistant.mjs | The spectra assistant: band assignment, reading peaks from a spectrum, library search and advice rules | 41 |
check-instruments.mjs | Readers for instrument and software files, including four real BioLogic files | 33 larger |
check-fitting.mjs | The Levenberg–Marquardt fitting engine, peak fitting, baselines, derivatives and interpolation | 20 larger |
check-curves.mjs, check-stats.mjs, check-parse.mjs, check-analysis-root.mjs | Trend lines, descriptive statistics and areas, compressed JCAMP-DX files, FFT, formulas, interpolation, regression and worksheet tools | 33 larger |
| Total | 1,447 |
A "larger" check tests several values at once but is counted only once, so the real number of compared values is higher than the total.
Statistics
Tests shown on figures, compared with R
For these tests we used classic data sets that come with R, because their correct results are printed in R's own help pages and in many textbooks.
| Topic | Reference | What we compared |
|---|---|---|
| Two groups | R t.test and wilcox.test on sleep, and the wilcox.test help-page example | Welch, Student, paired and one-sample t tests; Mann–Whitney (exact, and the normal approximation with ties and continuity correction); Wilcoxon signed rank (zeros dropped, exact P = 2 × 10/512). We also checked the exact Mann–Whitney P by listing all 3,003 possible rank splits |
| Three or more groups | aov, TukeyHSD, kruskal.test and oneway.test on PlantGrowth; Dunnett from multcomp | One-way ANOVA, Tukey HSD, Kruskal–Wallis, Welch ANOVA, Dunnett (as close as R's own random method allows, ±10⁻³) and Dunn with Holm correction. We also checked which pairs get brackets and stars |
| Two factors | aov(breaks ~ wool * tension) on warpbreaks | Two-way ANOVA, and Type III F for unbalanced data against the formula for unweighted means |
| Multiple comparisons | p.adjust | Holm, Benjamini–Hochberg, Bonferroni, Šidák and Holm–Šidák. A correction is used by default whenever a figure shows more than one comparison |
| Correlation and tables | cor.test on anscombe, fisher.test, chisq.test, and SciPy's Shapiro–Wilk example | Pearson r with its confidence interval, Spearman, Fisher's exact test (34/70), χ² with Yates correction, and Shapiro–Wilk (to the precision SciPy gives, as it works in float32) |
| Regression | summary(lm(y1 ~ x1, anscombe)) and predict(..., interval = ...) | Slope, intercept, standard errors, R², confidence and prediction bands, and weighted fits (1/σ²). The fitting engine also gives the same uncertainties as exact least squares |
| Distributions | Exact formulas | t, F, χ², normal, incomplete beta (checked with binomial tails), studentized range (for k = 2 it equals √2·|t|), Cauchy, t₂, χ²₂, F(2, ν) and hypergeometric |
| Wording and data safety | Prism and APA styles | Stars and P wording never round a significant P up to 0.05. Sorting, cutting and pasting never shift an error column away from its data, and making a chart never changes the values |
Lab statistics, compared with SciPy 1.13 and statsmodels 0.14
| Method | Reference function | Test data |
|---|---|---|
| D'Agostino–Pearson, with skewness and kurtosis z | scipy.stats.normaltest, skewtest, kurtosistest | 6 samples with n from 8 to 150 (normal, log-normal, uniform and exponential) |
| Anderson–Darling and Kolmogorov–Smirnov (Lilliefors P) | statsmodels normal_ad, lilliefors | The same 6 samples |
| Two-sample Kolmogorov–Smirnov (exact) | scipy.stats.ks_2samp | Small samples, samples with ties and larger samples |
| Grubbs' outlier test | Published two-sided critical values (NIST/SEMATECH) | Critical values for many n, and a data set with one clear outlier |
| Ratio paired t test and Fisher's LSD | scipy.stats.ttest_rel on logs, and t tests with the pooled MSE | Paired data and three-group data |
| Log-rank, Gehan–Breslow–Wilcoxon, hazard ratio and Cox regression (Breslow and Efron ties) | statsmodels survdiff, PHReg | R's aml leukaemia data |
| Deming regression (jackknife SE) and logistic regression (odds ratios, likelihood-ratio test) | scipy.odr and statsmodels Logit | Error ratios of 1 and 4; 120 simulated yes/no results with two predictors |
| Power and sample size (t tests and one-way ANOVA) | statsmodels TTestIndPower, TTestPower, and scipy.stats.ncf | A grid of effect sizes, sample sizes and α values |
| 19 Prism nonlinear equations, model comparison (AICc and extra sum of squares) and reading unknowns from a standard curve | scipy.optimize.curve_fit with tight tolerances | Binding, enzyme kinetics, exponential, growth, Gaussian, log-normal, Boltzmann, 4PL and sine |
| LOWESS, convolution, cross-correlation and Hilbert envelope | statsmodels' reference LOWESS, numpy.convolve, correlate and scipy.signal.hilbert | Noisy curves and pulses |
Two notes on the references. For ANOVA power, the statsmodels function uses a less precise non-central F distribution (off by about 10⁻⁵), so we used SciPy's ncf instead. We also ran curve_fit and odr with tight tolerances, because with the default settings they can stop before reaching the best fit.
Curve fitting, peaks and data processing
Here we made data with known true values, added noise in some cases, and checked if the fit finds the values again.
| What we tested | True value | Our result |
|---|---|---|
| Straight line with known noise (Levenberg–Marquardt) | slope 2, intercept 1 | 2.0003 ± 0.0047, 0.986 ± 0.028 |
| Two overlapping Gaussian peaks (centre / height / width) | 8 / 10 / 2.4 and 11 / 6 / 1 | 7.999 / 9.996 / 2.396 and 11.000 / 6.004 / 0.999 |
| Three peaks on a sloped, shifted background (like an FTIR spectrum) | 1100, 1655, 3300 cm⁻¹ | 1099.8, 1654.9, 3299.9 |
| One shared peak width over several spectra (joint fit compared with averaging separate fits) | FWHM 2 | 2.0013 vs 2.0167 |
| Exponential with an offset | 5e−0.2x + 0.5 | 5e−0.2x + 0.501 |
| Exponential with noise near zero (proper least squares instead of fitting ln y, which is biased) | 4e−0.3x | 4.01e−0.301x |
| Peak fit on 1,000,000 points | no crash, same accuracy | 134 ms |
| Asymmetric least squares and arPLS baselines | known baseline | largest error 0.147 and 0.030 |
| Whittaker smoother, (W + λDᵀD) z = W y | exact solution | residual 7.3 × 10⁻¹⁰ |
| Trend lines: linear, exponential, logarithmic, power, quadratic, saturation and sigmoid | data without noise | R² = 1 and the exact equations |
| FFT (lengths that are and are not powers of two) | direct DFT sum | amplitude 3.000000 |
We also checked that Lorentzian and pseudo-Voigt fits give different answers from a Gaussian fit (as they should), that limits and fixed parameters are respected, that derivatives are dy/dx in both x directions, that interpolation works on a descending wavenumber axis, that despiking does not touch the neighbouring points, that zoomed plots only thin out the visible part, and that spline and Akima interpolation, multiple regression and Set Column Values formulas give the right numbers.
Method analyses (419 checks)
We tested each method in three ways. First, the formulas are compared with a hand calculation or a published value. Second, the fit is run on data without noise, where it must return exactly the values used to make the data. Third, every example in the app is run and compared with the true values it was made from. Each example must also give a results table, a TSV export and a methods sentence, and it must say whether its data are synthetic or published.
| Technique | What we checked | Reference |
|---|---|---|
| Circular dichroism | mdeg to mean residue ellipticity and Δε; helix content (Morrow; Chen–Yang–Chau); basis-spectrum fit (helix 45 %, sheet 25 %, coil 30 %); thermal melt Tm 62 °C and van 't Hoff ΔH; chemical denaturation ΔG(H₂O), m value and Cm | Kelly et al. 2005, hand formulas, and a fit on data without noise |
| X-ray diffraction | Bragg d; Scherrer size (hand value 16.908 nm); Gaussian and Lorentzian instrument correction; Williamson–Hall size 20 nm and strain 0.002; cubic lattice a = 4.0786 Å, and face-centred cubic found from the missing reflections | Hand calculation, and a synthetic pattern with known size and strain |
| UV-Vis | Calibration slope and intercept; ICH LOD (3.3 sy/b) and LOQ; standard error of a predicted concentration; Beer–Lambert ε of 15,000; an unknown of 33 µM; Tauc band gap 3.30 eV (direct) and 3.20 eV (indirect, Kubelka–Munk) | Miller and Miller eq. 5.9, and ICH Q2 |
| Cyclic voltammetry and corrosion | Randles–Ševčík (and its 2.69 × 10⁵ constant); simulated reversible peak current 0.4463 nFAC√(nFνD/RT); Epa − E½ = 1.109 RT/F; Laviron ks; diffusion coefficient 7.6 × 10⁻⁶ cm²/s; b value; Tafel slopes (118.3 mV/dec) and j₀ of 1 µA/cm²; Stern–Geary | Bard and Faulkner, Nicholson, and a convolution simulator |
| Batteries, supercapacitors and semiconductors | Charge–discharge capacitance 150 F/g with IR-drop correction, ESR 5 Ω, coulombic efficiency and energy ½CΔV²; Mott–Schottky ND and flat-band potential | Synthetic curves with known values |
| Impedance (EIS) | R(RC) exact at ωRC = 1; Randles fit for Rs, Rct, Cdl and Warburg σ; AICc picks the right circuit; coating CPE α; Lin-KK passes good data (residual under 1.2 %) and flags drifting data (over 3 %) | Exact impedance formulas and simulated spectra |
| Thermal analysis | TGA mass-loss steps of 8 % and 55 %, residue, onsets at 94 and 350 °C, and the DTG peak; Kissinger, KAS, Friedman and Flynn–Wall–Ozawa Ea of 150 kJ/mol; DSC Tg, ΔCp, crystallinity 10.7 %, and cold-crystallisation and melting ΔH (hand check 6 J/g) | Simulated data at several heating rates, and hand integrals |
| Surfaces and particles | BET area 435.3 m²/g with the Rouquerol criteria; the right range for a microporous sample (the classic range gives C < 0); DLS Z-average 100 nm and PDI 0.05; zeta potential (Smoluchowski hand value 31.9 mV, Ohshima limits 1.5 and 1); log-normal median 18 nm | Rouquerol et al., Stokes–Einstein, and hand values |
| Chromatography | HPLC plate numbers (EP) 3,211 and 1,764, resolution 2.20 (EP and USP), tailing 1.00 for a Gaussian peak, area % and retention factor; GPC Mn 17,778 and Mw 22,500 (by hand) and dispersity 1.127 | European and US Pharmacopoeia formulas |
| Fluorescence | Stern–Volmer KSV 30 and kq 6 × 10⁹; combined static and dynamic quenching; relative quantum yield 0.300; FRET efficiency and distance; overlap integral and R₀ (Lakowicz value 13.8); lifetimes 1.2 and 5.0 ns and the average ⟨τ⟩; Stokes shift; emission FWHM | Lakowicz, Principles of Fluorescence Spectroscopy |
| NMR and ITC | Referencing, chemical shifts, multiplicities (q, s, t), J couplings and integrals 2 : 3 : 3; ITC Kd 1 µM, ΔH −40 kJ/mol, heat of the first injection and ΔG = −RT ln K | An ethyl acetate spectrum and a one-site binding model |
| Kinetics and enzymes | Michaelis–Menten on the puromycin data: Vmax 212.68 ± 6.947 and Km 0.06412 ± 0.008281; the IC50 95 % CI contains the true value; reaction order chosen by AICc; Arrhenius and Eyring; Langmuir | Bates and Watts 1988 (published estimates and standard errors) |
| Other spectroscopy and devices | Raman crystallite size La (Cançado); XPS atomic %; EPR g = 2.0036 and ΔBpp; solar cell Jsc, shunt resistance and efficiency | Cançado et al. 2006, and synthetic spectra |
| Units and readings | "nm" is not read as nanomolar and "1 A/g" is not read as a current; scan rates are read from file names; EPR g is taken at the zero crossing; capacitance uses the discharge branch | Test cases for mistakes we want to avoid |
Instrument and software files
BioLogic EC-Lab and BT-Lab binary files (.mpr), tested with real data
BioLogic does not publish how .mpr files are built. We wrote our reader based on the description from the open-source galvani project. To test it, we used four real runs from the galvani test data (CC BY 4.0) and compared every value with the text file (.mpt) that EC-Lab itself exported from the same run. If the reader meets a column it does not know, it stops and shows a message. It never guesses.
| File | Experiment | Points × columns | Values that match the .mpt |
|---|---|---|---|
bio_logic2.mpr | Cyclic voltammetry | 61 × 11 | 671 |
bio_logic5.mpr | Cyclic voltammetry | 249 × 12 | 2,987 |
121_CA_455nm_6V_30min_C01.mpr | Chronoamperometry | 1,804 × 13 | 23,452 |
020-formation_CB5.mpr | Battery formation (BT-Lab) | 1,323 × 23 | 30,429 |
| Total | 57,519 |
Some text exports print time with only 2 decimals, so time is compared to 2 decimals. The other columns are compared to about 7 significant digits, which is what the text export gives. One cell was exported by EC-Lab as "XXX", so it had nothing to compare with and was skipped.
Other file readers
| Formats | How we tested them |
|---|---|
XRD: Bruker .raw (three versions) and .brml; Rigaku .rasx, .ras, and MiniFlex and SmartLab .raw; PANalytical .xrdml; .uxd, GSAS .gsa, .cif, .chi, .asc and .xye | Small test files written in each format. The angles and counts must come back exactly. A 2-D detector image or an unreadable file must give a clear message |
Electrochemistry: BioLogic .mpt and Gamry .DTA | The same way. We also ran the .mpt reader on the four real exports above, and this found several bugs (see below) |
| Spectroscopy: JCAMP-DX (compressed SQZ, DIF and DUP forms, and peak tables) | Examples from the JCAMP-DX 4.24 standard, including the check values across lines |
Other software: Origin .opj and .opju, GraphPad .pzfx and .prism, and CSV | Test files with known content. Values, error columns and empty cells must be kept |
Renishaw .wdf files cannot be read yet. For now the app asks you to export them as text.
Figures and scientific integrity
- Maths behind the figures (255 checks). Kaplan–Meier times, numbers at risk, medians and censoring ticks were compared with R's
survivalpackage on theamldata. We also checked log-rank degrees of freedom, Q–Q positions against R'sppointsandqqnorm, density bandwidth againstbw.nrd0, Bland–Altman formulas, contour levels, beeswarm placement, the wording of P values and stars (exact, APA and threshold styles), scientific and engineering tick labels, and journal presets. - Integrity (85 checks). In Studio and Data you can move annotations and edit your own text, but nothing else. Data points, tick numbers and calculated P values or captions cannot be dragged, deleted or retyped. When traces are shifted, the figure says "offset for clarity". Axis breaks always show their marks, and if a log axis or a manual axis limit hides some points, the figure says so.
Spectra assistant
The spectra assistant works with fixed rules, not with a language model, and nothing is sent anywhere. It assigns bands using published correlation tables and searches the Open Specy reference library by correlation. We tested it in three steps.
| Test | FTIR | Raman |
|---|---|---|
| Assigning bands from positions in the literature (19 materials): right at first guess / right within the top three | 30/34 · 32/34 | 37/40 · 38/40 |
| Full run on synthetic spectra: bands found / assigned to the right functional group | 33/33 · 30/33 | 36/36 · 33/36 |
| Library search on spectra that were kept out of the library: right first hit / right within the top five (no extra noise) | 75.9 % · 93.8 % (n = 370) | 61.9 % · 73.7 % (n = 281) |
| The same with added noise (σ = 0.05) | 76.2 % · 94.3 % | 60.5 % · 74.0 % |
The band lists come from Socrates 2001, Silverstein et al. 2014, Larkin 2011, Ferrari and Robertson 2000 and other sources listed in docs/ASSISTANT-VALIDATION.md. Real samples (mixtures, shifted or weak bands) will give lower scores than these textbook cases. All 8 advice rules passed on synthetic data.
Problems we found and fixed
Testing is only useful if it finds mistakes, so we want to be open about them. These are the problems our tests found. All of them are fixed now, and each one has its own test so it cannot come back unnoticed.
- Peak fit on a sloped background. A 40 cm⁻¹ wide FTIR band came back with a width of 7,200 cm⁻¹, because the peak was soaking up the background offset. Now the background is fitted together with the peaks.
- BioLogic
.mptimport on real files. The reader skipped the row with the column names (so the x axis became the "mode" column), refused BT-Lab battery files, dropped rows that had one "XXX" cell, and had no default axes for potential-only and battery runs. - "0.125" read as 125. BioView read a number with one dot and three decimals as a thousands separator. Now both studios use the same number reader, which has 13 test cases.
- Error columns after Cut. Cutting a range of data could shift the error column away from its points. Now the errors always stay with their points.
- Spectra assistant. A "broad O–H" partner band was counted even when the band in the spectrum was narrow, and amide I was accepted without amide II.
Limits and what is still missing
- Most method examples and many file tests use synthetic data with known answers. The real measured data are the four BioLogic files, the puromycin data, R's classic data sets (
sleep,PlantGrowth,warpbreaks,aml) and the Open Specy library spectra. - The R results are taken from R's documented examples and written into the tests. R itself does not run during testing.
- Some methods from GraphPad Prism are still missing: ROUT outlier removal, nested t test and nested ANOVA, mixed-effects and three-way ANOVA, Passing–Bablok regression and Dunnett T3. From Origin, peak deconvolution and 2-D surface fitting are still missing.
- These files cannot be read yet: Renishaw
.wdf, Thermo SPC and OMNIC, Bruker OPUS, VAMAS, AIA/ANDI chromatography and Bruker NMR. Please export them as text or CSV for now. - XRD phase matching only uses lists you give it, or Bragg positions for a few cubic phases (we do not have ICDD data), so we did not score it.
How to repeat our tests
- In the project folder, run
npm test. It runs all 12 scripts inscripts/and prints how many checks passed. If something does not match, it prints our value, the reference value and the tolerance, and stops. - To make the SciPy and statsmodels reference values again, run
python -I scripts/labstats-reference.pywith SciPy 1.13, statsmodels 0.14 and NumPy 1.26 installed. - To write the assistant report again, run
node scripts/check-assistant.mjs --write. It updatesdocs/ASSISTANT-VALIDATION.md. - The four BioLogic test files, their exports and their licence notice are in
scripts/fixtures/biologic/.