Calibration Data Flow

Calibration is the single most important step for accurate VNA measurements. But what actually happens inside the firmware when you press OPEN, SHORT, LOAD, and DONE? This page traces the complete data flow from raw ADC samples through error correction, showing exactly how each calibration standard contributes to measurement accuracy.

Tip

If you want step-by-step instructions for performing a calibration, see Full 2-Port Calibration. This page explains what the firmware does behind the scenes.

The Big Picture

Every measurement the NanoVNA makes passes through this pipeline:

flowchart TD
  A["Si5351 generates<br/>stimulus frequency"] --> B["RF signal passes<br/>through DUT"]
  B --> C["TLV320 codec<br/>captures response<br/>(I2S @ 192 kHz)"]
  C --> D["FFT extracts<br/>magnitude + phase<br/>(48 samples/point)"]
  D --> E{"Calibration<br/>active?"}
  E -- Yes --> F["Interpolate cal data<br/>to current frequency"]
  F --> G["Apply CH0 error term<br/>(S11 correction)"]
  G --> H["Apply CH1 error term<br/>(S21 correction)"]
  H --> I["Corrected measurement<br/>stored in measured[ ]"]
  E -- No --> I
  I --> J["Display / export"]

  style A fill:#3d2a1a,stroke:#c87533,color:#f5f2ee
  style D fill:#3d2a1a,stroke:#c87533,color:#f5f2ee
  style G fill:#2a3d1a,stroke:#7bc87b,color:#f5f2ee
  style H fill:#2a3d1a,stroke:#7bc87b,color:#f5f2ee
  style I fill:#1a2a3d,stroke:#5b8fc8,color:#f5f2ee

The green boxes are where calibration does its work. Without them, you see raw hardware response including cable loss, connector mismatch, and mixer artifacts.

Calibration Collection: What Happens When You Press OPEN

When you press a calibration button (OPEN, SHORT, LOAD, THRU, or ISOLN), the firmware calls cal_collect(). Here is what happens:

sequenceDiagram
  participant User
  participant UI as UI Thread
  participant Sweep as Sweep Engine
  participant Cal as cal_data[ ]
  participant Status as cal_status

  User->>UI: Press "OPEN"
  UI->>Sweep: cal_collect(CAL_OPEN)

  Note over Sweep: Check if freq range<br/>changed since last cal
  alt Frequency range changed
      Sweep->>Status: Reset cal_status = 0
      Note over Status: All previous cal<br/>measurements invalidated
  end

  Sweep->>Status: Set CALSTAT_OPEN flag
  Sweep->>Status: Clear CALSTAT_ES, ER, APPLY

  Note over Sweep: Force bandwidth ≥ 100 Hz<br/>(more averaging)

  Sweep->>Sweep: Run full sweep<br/>(measure CH0)
  Sweep->>Cal: Copy measured[0] → cal_data[CAL_OPEN]

  Note over Cal: Raw OPEN measurement<br/>stored for later math

  UI->>UI: Update status display

Each standard measures a different channel:

Standard	Source Channel	Stored In
OPEN	measured[0] (S11)	cal_data[CAL_OPEN]
SHORT	measured[0] (S11)	cal_data[CAL_SHORT]
LOAD	measured[0] (S11)	cal_data[CAL_LOAD]
THRU	measured[1] (S21)	cal_data[CAL_THRU]
ISOLN	measured[1] (S21)	cal_data[CAL_ISOLN]

Order matters (sort of)

You can measure OPEN, SHORT, and LOAD in any order. But if you change the frequency range between measurements, all previous calibration data is erased and you start over. The firmware detects this by comparing frequency0, frequency1, and sweep_points against the stored calibration range.

Calibration Status Flags

The firmware tracks progress using a bitfield. Each flag represents either a raw measurement or a calculated error term:

stateDiagram-v2
  direction LR

  [*] --> Empty: Power on

  state "Raw Measurements" as raw {
      Empty --> HasLoad: cal load
      HasLoad --> HasSOL: cal open + short
      HasSOL --> HasSOLT: cal thru
      HasSOLT --> HasAll: cal isoln
  }

  state "Error Terms" as calc {
      HasAll --> Calculated: cal done
      HasSOLT --> Calculated: cal done
      HasSOL --> Calculated: cal done
  }

  state "Active" as active {
      Calculated --> Applied: CALSTAT_APPLY set
  }

  Applied --> Interpolated: Sweep range ≠ cal range

  note right of Empty
      cal_status = 0x000
  end note

  note right of HasLoad
      CALSTAT_LOAD (bit 0)
  end note

  note right of HasSOL
      LOAD + OPEN + SHORT
      (bits 0-2 = 0x07)
  end note

  note right of Calculated
      ED, ES, ER calculated
      ET, EX if THRU/ISOLN
  end note

  note right of Applied
      CALSTAT_APPLY (bit 8)
      Corrections active
  end note

  note right of Interpolated
      CALSTAT_INTERPOLATED
      (bit 9)
  end note

Flag Reference

Bit	Flag	Set When
0	CALSTAT_LOAD	LOAD standard measured
1	CALSTAT_OPEN	OPEN standard measured
2	CALSTAT_SHORT	SHORT standard measured
3	CALSTAT_THRU	THRU standard measured
4	CALSTAT_ISOLN	Isolation measured
5	CALSTAT_ES	Source match calculated (replaces OPEN)
6	CALSTAT_ER	Reflection tracking calculated (replaces SHORT)
7	CALSTAT_ET	Transmission tracking calculated (replaces THRU)
8	CALSTAT_APPLY	Error correction is active
9	CALSTAT_INTERPOLATED	Using interpolated cal data

The cal done Calculation

When you press DONE, the firmware converts raw standard measurements into error terms. This is the mathematical heart of calibration:

flowchart TD
  subgraph inputs["Raw Standard Data"]
      LOAD["cal_data[LOAD]<br/>= S11m_load"]
      OPEN["cal_data[OPEN]<br/>= S11m_open"]
      SHORT["cal_data[SHORT]<br/>= S11m_short"]
      THRU["cal_data[THRU]<br/>= S21m_thru"]
      ISOLN["cal_data[ISOLN]<br/>= S21m_isoln"]
  end

  subgraph step1["Step 1: Directivity"]
      ED["ED = LOAD measurement<br/>(perfect load reflects nothing,<br/>so whatever we see is error)"]
  end

  subgraph step2["Step 2: Source Match"]
      ES["ES = f(OPEN', SHORT', ED)<br/><br/>ES = (SHORT' + OPEN'/s11ao)<br/>     ÷ (OPEN' - SHORT')<br/><br/>where X' = X - ED"]
  end

  subgraph step3["Step 3: Reflection Tracking"]
      ER["ER = -(1 + ES) × SHORT'<br/><br/>Combines source match with<br/>short circuit response"]
  end

  subgraph step4["Step 4: Isolation"]
      EX["EX = ISOLN measurement<br/>(crosstalk with ports isolated)"]
  end

  subgraph step5["Step 5: Transmission Tracking"]
      ET["ET = 1 / (THRU - EX)<br/><br/>Stored INVERTED for<br/>multiply efficiency"]
  end

  LOAD --> ED
  OPEN --> ES
  SHORT --> ES
  ED --> ES
  SHORT --> ER
  ED --> ER
  ES --> ER
  ISOLN --> EX
  THRU --> ET
  EX --> ET

  style inputs fill:#2a2420,stroke:#8a7e74,color:#f5f2ee
  style step1 fill:#1a2a1a,stroke:#5b8f5b,color:#f5f2ee
  style step2 fill:#1a2a1a,stroke:#5b8f5b,color:#f5f2ee
  style step3 fill:#1a2a1a,stroke:#5b8f5b,color:#f5f2ee
  style step4 fill:#1a1a2a,stroke:#5b5b8f,color:#f5f2ee
  style step5 fill:#1a1a2a,stroke:#5b5b8f,color:#f5f2ee

Partial Calibration Paths

You don’t always need all five standards. The firmware handles several partial calibration scenarios:

flowchart TD
  START["cal done called"]

  START --> ChkLoad{"LOAD<br/>measured?"}
  ChkLoad -- Yes --> UseLoad["ED = LOAD data"]
  ChkLoad -- No --> DefLoad["ED = 0 + 0j<br/>(assume perfect directivity)"]

  START --> ChkIsoln{"ISOLN<br/>measured?"}
  ChkIsoln -- Yes --> UseIsoln["EX = ISOLN data"]
  ChkIsoln -- No --> DefIsoln["EX = 0 + 0j<br/>(assume no crosstalk)"]

  START --> ChkBoth{"OPEN and SHORT<br/>both measured?"}
  ChkBoth -- Yes --> CalcFull["Calculate ES from<br/>OPEN + SHORT + ED<br/><br/>Calculate ER from<br/>SHORT + ED + ES"]
  ChkBoth -- No --> ChkOpen{"Only OPEN<br/>measured?"}
  ChkOpen -- Yes --> CalcOpenOnly["Copy OPEN → SHORT slot<br/>ES = 0 + 0j<br/>Calculate ER with sign=+1"]
  ChkOpen -- No --> ChkShort{"Only SHORT<br/>measured?"}
  ChkShort -- Yes --> CalcShortOnly["ES = 0 + 0j<br/>Calculate ER with sign=-1"]
  ChkShort -- No --> DefEsEr["ES = 0, ER = 1<br/>(no reflection correction)"]

  START --> ChkThru{"THRU<br/>measured?"}
  ChkThru -- Yes --> CalcEt["Calculate ET = 1/(THRU - EX)<br/>(stored inverted)"]
  ChkThru -- No --> DefEt["ET = 1 + 0j<br/>(no transmission correction)"]

  CalcFull --> Apply["Set CALSTAT_APPLY<br/>Calibration active"]
  CalcOpenOnly --> Apply
  CalcShortOnly --> Apply
  DefEsEr --> Apply
  CalcEt --> Apply
  DefEt --> Apply
  UseLoad --> Apply
  DefLoad --> Apply
  UseIsoln --> Apply
  DefIsoln --> Apply

  style CalcFull fill:#1a2a1a,stroke:#5b8f5b,color:#f5f2ee
  style Apply fill:#2a1a1a,stroke:#c87533,color:#f5f2ee

Minimum viable calibration

The absolute minimum is one standard — even just SHORT gives you some reflection correction. But for accurate measurements, full SOL (Short-Open-Load) for S11, or SOLT for both S11 and S21, is strongly recommended.

Error Correction at Measurement Time

During every sweep, the firmware applies error correction to each measurement point. This happens per-point, using the error terms calculated during cal done:

S11 Correction (CH0)

flowchart LR
  subgraph input["Raw Measurement"]
      S11m["S11m<br/>(measured)"]
  end

  subgraph correct["Error Correction"]
      Sub["S11m' = S11m - ED<br/>(remove directivity)"]
      Div["S11a = S11m' ÷ (ER + ES × S11m')<br/>(remove source match<br/>and tracking)"]
  end

  subgraph output["Result"]
      S11a["S11a<br/>(actual)"]
  end

  S11m --> Sub --> Div --> S11a

  style input fill:#2a2420,stroke:#8a7e74,color:#f5f2ee
  style correct fill:#1a2a1a,stroke:#5b8f5b,color:#f5f2ee
  style output fill:#1a1a2a,stroke:#5b5b8f,color:#f5f2ee

The division is complex division — both numerator and denominator are complex numbers with real and imaginary parts.

S21 Correction (CH1)

flowchart LR
  subgraph input["Raw Measurement"]
      S21m["S21m<br/>(measured)"]
  end

  subgraph correct["Error Correction"]
      Sub2["S21m' = S21m - EX<br/>(remove crosstalk)"]
      Mul["S21a = S21m' × ET⁻¹<br/>(remove tracking)<br/><br/>ET stored inverted<br/>so multiply is faster"]
  end

  subgraph enhance["Enhanced Response"]
      Enh["S21a × (1 - ES × S11a)<br/><br/>Corrects for source match<br/>using S11 result"]
  end

  subgraph output["Result"]
      S21a["S21a<br/>(actual)"]
  end

  S21m --> Sub2 --> Mul --> Enh --> S21a

  style input fill:#2a2420,stroke:#8a7e74,color:#f5f2ee
  style correct fill:#1a2a1a,stroke:#5b8f5b,color:#f5f2ee
  style enhance fill:#2a2a1a,stroke:#8f8f5b,color:#f5f2ee
  style output fill:#1a1a2a,stroke:#5b5b8f,color:#f5f2ee

Why store ET inverted?

The firmware stores ET = 1/(S21_thru - EX) rather than ET = S21_thru - EX. This means the correction step uses multiplication instead of division — saving CPU cycles on the Cortex-M0 which has no hardware divider. A small optimization that matters when correcting hundreds of points per sweep.

Calibration Interpolation

When the current sweep range differs from the calibration range, the firmware interpolates error terms to match. This happens automatically when CALSTAT_INTERPOLATED is set.

flowchart TD
  subgraph stored["Stored Calibration"]
      CalFreqs["Cal frequencies:<br/>cal_frequency0 to cal_frequency1<br/>(cal_sweep_points)"]
      CalData["cal_data[5][points][2]<br/>(5 error terms × points × re/im)"]
  end

  subgraph current["Current Sweep"]
      SweepFreqs["Sweep frequencies:<br/>frequency0 to frequency1<br/>(sweep_points)"]
  end

  subgraph interp["Interpolation"]
      Check{"Sweep range =<br/>cal range?"}
      Check -- Yes --> Direct["Direct lookup:<br/>cal_data[term][i]"]
      Check -- No --> Calc["For each sweep point f:<br/>1. Find bracketing cal points<br/>2. Calculate k = (f - f0) / (f1 - f0)<br/>3. Linear interpolate all 5 terms"]
  end

  subgraph harmonic["Harmonic Boundary Handling"]
      HCheck["If bracketing points span<br/>a harmonic boundary:"]
      HFix["Extrapolate from same-side<br/>points instead of interpolating<br/>across the boundary"]
  end

  CalFreqs --> Check
  SweepFreqs --> Check
  CalData --> Direct
  CalData --> Calc
  Calc --> HCheck --> HFix

  style stored fill:#2a2420,stroke:#8a7e74,color:#f5f2ee
  style current fill:#1a2a1a,stroke:#5b8f5b,color:#f5f2ee
  style harmonic fill:#2a1a1a,stroke:#c87533,color:#f5f2ee

The harmonic boundary handling is noteworthy: at the ~290 MHz threshold where the Si5351 switches from fundamental to 3rd harmonic output, the calibration data can change abruptly. The firmware detects when two adjacent calibration points straddle this boundary and extrapolates from same-side points instead, avoiding glitches in the interpolated result.

Edge Cases

Condition	Behavior
Sweep frequency below cal range	Uses first cal point (no extrapolation)
Sweep frequency above cal range	Uses last cal point (no extrapolation)
Exact match with cal frequency	Direct copy (no interpolation needed)
Across harmonic boundary	Extrapolate from same-harmonic side

Memory Layout

Calibration data is the largest data structure in the firmware:

block-beta
  columns 5

  block:header:5
      columns 5
      h1["ETERM_ED<br/>(Directivity)"]
      h2["ETERM_ES<br/>(Source Match)"]
      h3["ETERM_ER<br/>(Refl. Tracking)"]
      h4["ETERM_ET<br/>(Trans. Tracking)"]
      h5["ETERM_EX<br/>(Isolation)"]
  end

  block:data:5
      columns 5
      d1["Point 0: re, im<br/>Point 1: re, im<br/>...<br/>Point N: re, im"]
      d2["Point 0: re, im<br/>Point 1: re, im<br/>...<br/>Point N: re, im"]
      d3["Point 0: re, im<br/>Point 1: re, im<br/>...<br/>Point N: re, im"]
      d4["Point 0: re, im<br/>Point 1: re, im<br/>...<br/>Point N: re, im"]
      d5["Point 0: re, im<br/>Point 1: re, im<br/>...<br/>Point N: re, im"]
  end

cal_data[5][SWEEP_POINTS_MAX][2]   // [error_term][point][re/im]

Target	Points Max	Size per Term	Total Cal Data
F072	101	808 bytes	4,040 bytes
F303	401	3,208 bytes	16,040 bytes

Each save slot in flash stores the complete properties_t structure which includes all 5 cal_data arrays plus sweep settings, traces, and markers.

Save and Recall Flow

sequenceDiagram
  participant User
  participant FW as Firmware
  participant Flash as Flash Memory
  participant Cal as cal_data[ ]

  Note over User,Flash: Saving calibration

  User->>FW: save 0
  FW->>FW: Build properties_t struct<br/>(cal_data + sweep + traces + markers)
  FW->>FW: Calculate CRC checksum
  FW->>Flash: Write to slot 0<br/>(F072: 0x1800 bytes, F303: 0x4000 bytes)
  FW->>FW: Set lastsaveid = 0

  Note over User,Flash: Recalling calibration

  User->>FW: recall 0
  FW->>Flash: Read slot 0
  FW->>FW: Verify CRC checksum
  alt CRC valid
      FW->>Cal: Load cal_data from properties
      FW->>FW: Restore sweep settings
      FW->>FW: Restore trace/marker config
      FW->>FW: Set CALSTAT_APPLY
      Note over FW: Check if sweep range matches<br/>cal range → set INTERPOLATED if not
  else CRC invalid
      FW->>FW: Report error, keep current state
  end

Complete Example: What Happens During a Full SOLT Calibration

Here is the entire flow for a typical calibration session:

1. Set sweep range

   ch> sweep 1M 100M 101

   Firmware stores frequency0=1000000, frequency1=100000000, sweep_points=101.

2. Measure LOAD

   ch> cal load

   • cal_collect(CAL_LOAD) runs a sweep measuring S11
   • Copies measured[0] → cal_data[CAL_LOAD]
   • Sets CALSTAT_LOAD flag
   • This raw measurement becomes ED directly (no further calculation needed)

3. Measure OPEN

   ch> cal open

   • cal_collect(CAL_OPEN) runs sweep, copies → cal_data[CAL_OPEN]
   • Sets CALSTAT_OPEN, clears CALSTAT_ES, CALSTAT_ER, CALSTAT_APPLY

4. Measure SHORT

   ch> cal short

   • cal_collect(CAL_SHORT) runs sweep, copies → cal_data[CAL_SHORT]
   • Sets CALSTAT_SHORT, clears CALSTAT_ES, CALSTAT_ER, CALSTAT_APPLY

5. Measure THRU (connect Port 1 to Port 2)

   ch> cal thru

   • cal_collect(CAL_THRU) runs sweep measuring S21 (measured[1])
   • Copies measured[1] → cal_data[CAL_THRU]
   • Sets CALSTAT_THRU, clears CALSTAT_ET, CALSTAT_APPLY

6. Calculate and apply

   ch> cal done

   cal_done() executes:

   • ED ← LOAD data (already stored)
   • EX ← defaults to 0+0j (ISOLN not measured)
   • ES ← calculated from OPEN + SHORT + ED (eterm_calc_es)
   • ER ← calculated from SHORT + ED + ES (eterm_calc_er)
   • ET ← calculated as 1/(THRU - EX) (eterm_calc_et)
   • Sets CALSTAT_APPLY — correction now active for all measurements

7. Save for later

   ch> save 0

   Writes entire properties_t to flash slot 0 with CRC checksum.

Status Display Indicators

The calibration status appears in the lower-left corner of the screen:

Display	Meaning
C0 – C6 (green)	Full cal active from slot N
D0 – D6	Cal active but interpolated (different freq range)
c (lowercase, red)	Cal data exists but not applied
(blank)	No calibration data

Related Pages

Error Model

The math behind ED, ES, ER, ET, EX — what each error term corrects.

Read more →

SOLT Standards

Why SHORT = -1, OPEN = +1, LOAD = 0 — the physical basis.

Read more →

Interpolation

How calibration data is stretched to different frequency ranges.

Read more →

Full Calibration Tutorial

Step-by-step guide to performing calibration via the menu.

Read more →