Calibration Error Model

No VNA is perfect. Cables have loss, connectors have mismatch, and internal circuitry introduces errors. Calibration measures these systematic errors so the firmware can mathematically remove them. The NanoVNA-H uses a 3-term model for S11 (reflection) and extends it to a 5-term model when S21 (transmission) is also calibrated.

Why Calibration Matters

Without calibration, your measurements include errors from:

• Directivity leakage: Signal that appears reflected even with a perfect load
• Source mismatch: The VNA’s output impedance is not exactly 50 ohms
• Frequency response: Cables and mixers have frequency-dependent behavior
• Crosstalk: Signal leaking from Port 1 to Port 2 without going through the DUT

These errors can completely dominate your measurement. A 40 dB return loss device might measure as 20 dB without calibration.

The 3-Term Error Model (S11)

For reflection measurements (S11), the NanoVNA uses three error terms:

flowchart LR
  A[Actual S11] --> B{Error Box}
  B --> C[Measured S11m]

  subgraph Error Terms
      ED[ED: Directivity]
      ES[ES: Source Match]
      ER[ER: Reflection Tracking]
  end

Error Terms Defined

Term	Name	What It Represents
ED	Directivity	Signal appearing as reflection even with perfect load
ES	Source Match	Mismatch at VNA output (re-reflections)
ER	Reflection Tracking	Overall frequency response of reflection path

The Math

The measured S11 (S11m) relates to actual S11 (S11a) by:

S11m = ED + (ER * S11a) / (1 - ES * S11a)

Solving for the actual value:

S11a = (S11m - ED) / (ER + ES * (S11m - ED))

The firmware implements this correction in main.c:

static void apply_CH0_error_term(float data[4], float c_data[CAL_TYPE_COUNT][2])
{
  // S11m' = S11m - Ed
  // S11a = S11m' / (Er + Es * S11m')
  float s11mr = data[0] - c_data[ETERM_ED][0];
  float s11mi = data[1] - c_data[ETERM_ED][1];

  float err = c_data[ETERM_ER][0] + s11mr * c_data[ETERM_ES][0] - s11mi * c_data[ETERM_ES][1];
  float eri = c_data[ETERM_ER][1] + s11mr * c_data[ETERM_ES][1] + s11mi * c_data[ETERM_ES][0];

  float sq = err*err + eri*eri;
  data[0] = (s11mr * err + s11mi * eri) / sq;
  data[1] = (s11mi * err - s11mr * eri) / sq;
}

Complex Math

All error terms are complex numbers (magnitude and phase). The firmware performs complex division to correct both the magnitude and phase of your measurement.

Calculating Error Terms from Standards

The three SOL (Short-Open-Load) measurements provide enough equations to solve for ED, ES, and ER:

From LOAD Measurement

A perfect 50 ohm load has S11 = 0. Whatever we measure is pure error:

ED = S11m_load

From SHORT Measurement

A short circuit has S11 = -1 (total reflection, inverted phase):

S11m_short = ED + ER*(-1) / (1 - ES*(-1))

From OPEN Measurement

An open circuit has S11 = +1 (total reflection, same phase):

S11m_open = ED + ER*(+1) / (1 - ES*(+1))

Solving these three equations simultaneously gives ED, ES, and ER.

The 5-Term Error Model (S21)

For transmission measurements, two additional terms are needed:

Term	Name	What It Represents
ET	Transmission Tracking	Frequency response of transmission path
EX	Isolation/Crosstalk	Signal leaking from Port 1 to Port 2 directly

The Correction

S21a = (S21m - EX) / ET

With optional enhancement for source match:

S21a = (S21m - EX) / ET * (1 - ES * S11a)

The firmware implementation:

static void apply_CH1_error_term(float data[4], float c_data[CAL_TYPE_COUNT][2])
{
  // S21a = (S21m - Ex) * Et`  (Et is stored inverted for efficiency)
  float s21mr = data[2] - c_data[ETERM_EX][0];
  float s21mi = data[3] - c_data[ETERM_EX][1];

  data[2] = s21mr * c_data[ETERM_ET][0] - s21mi * c_data[ETERM_ET][1];
  data[3] = s21mi * c_data[ETERM_ET][0] + s21mr * c_data[ETERM_ET][1];

  // Enhanced response correction using S11
  if (cal_status & CALSTAT_ENHANCED_RESPONSE) {
    float esr = 1.0f - (c_data[ETERM_ES][0] * data[0] - c_data[ETERM_ES][1] * data[1]);
    float esi = 0.0f - (c_data[ETERM_ES][1] * data[0] + c_data[ETERM_ES][0] * data[1]);
    float re = data[2];
    float im = data[3];
    data[2] = esr * re - esi * im;
    data[3] = esi * re + esr * im;
  }
}

Calculating ET and EX

From ISOLATION Measurement (nothing connected, or loads on both ports):

EX = S21m_isolation

From THRU Measurement (Port 1 connected directly to Port 2):

S21m_thru = EX + ET * 1 / (1 - ES * 0)
ET = S21m_thru - EX

Calibration Status Flags

The firmware tracks which calibration steps have been performed:

#define CALSTAT_LOAD  (1<<0)   // Load measured
#define CALSTAT_OPEN  (1<<1)   // Open measured
#define CALSTAT_SHORT (1<<2)   // Short measured
#define CALSTAT_THRU  (1<<3)   // Thru measured
#define CALSTAT_ISOLN (1<<4)   // Isolation measured

#define CALSTAT_ED    CALSTAT_LOAD   // ED calculated
#define CALSTAT_ES    (1<<5)         // ES calculated
#define CALSTAT_ER    (1<<6)         // ER calculated
#define CALSTAT_ET    (1<<7)         // ET calculated
#define CALSTAT_EX    CALSTAT_ISOLN  // EX calculated

#define CALSTAT_APPLY (1<<8)         // Calibration active
#define CALSTAT_INTERPOLATED (1<<9)  // Using interpolated cal data

Error Term Storage

The NanoVNA stores calibration data as arrays of complex values at each sweep point:

#define CAL_TYPE_COUNT  5
#define CAL_LOAD        0   // ETERM_ED
#define CAL_OPEN        1   // Used to calculate ES, ER
#define CAL_SHORT       2   // Used to calculate ES, ER
#define CAL_THRU        3   // ETERM_ET
#define CAL_ISOLN       4   // ETERM_EX

float cal_data[CAL_TYPE_COUNT][SWEEP_POINTS_MAX][2];  // [type][point][re/im]

Memory Usage

Calibration data requires significant memory: 5 types x 401 points x 2 floats x 4 bytes = 16 KB on the NanoVNA-H4. This is stored in flash memory.

Understanding Each Error Source

Directivity (ED)

Cause: Imperfect directional coupler or bridge Effect: Limits how low you can measure return loss Typical: -40 to -50 dB before cal After Cal: Removed mathematically

Source Match (ES)

Cause: VNA output not exactly 50 ohms Effect: Re-reflections between VNA and DUT Typical: -20 to -30 dB After Cal: Corrected by SOL

Reflection Tracking (ER)

Cause: Cable loss, connector mismatch, mixer response Effect: Magnitude and phase errors vs frequency Typical: Varies with frequency After Cal: Normalized to unity

Transmission Tracking (ET)

Cause: Cable loss, path differences Effect: Insertion loss errors Typical: Increases with frequency After Cal: Normalized to thru

Calibration Quality

After calibration, verify by re-measuring standards:

Standard	Ideal	Good Cal	Poor Cal
Load	S11 < -40 dB	S11 < -35 dB	S11 > -25 dB
Open	S11 = 0 dB, phase = 0	Within 0.5 dB, 5 deg	Large deviation
Short	S11 = 0 dB, phase = 180	Within 0.5 dB, 5 deg	Large deviation
Thru	S21 = 0 dB	S21 within 0.1 dB	S21 error > 0.5 dB

Sanity Check

After calibration, measure a known component (like a 10 dB attenuator). If the result is wildly wrong, your calibration has a problem.

Summary

Model	Terms	Standards Needed	Corrects
1-Port (S11)	ED, ES, ER	Short, Open, Load	Reflection measurements
2-Port (S11 + S21)	ED, ES, ER, ET, EX	SOL + Thru + Isolation	Both reflection and transmission

Next Steps

Learn what makes good calibration standards in SOLT Standards, and how the firmware handles measurements between calibration points in Calibration Interpolation.