IAE Academy

Digital instruments formulas

Digital voltmeters, multimeters and energy meters. 24 formulas for GATE EE, IN Electrical and Electronic Measurements, each with when to use it. Each is explained in the topic notes.

All 24 Digital instruments formulas

RESOLUTION OF AN N½-DIGIT DISPLAY

N is the number of full digits, so for 3½ digits N = 3 and R = 0.001. R is the smallest step as a fraction of the decade range. The half digit adds overrange, not resolution.

COUNTS OF AN N½-DIGIT DISPLAY

3½ digit: 2000 counts, maximum display 1999. 4½ digit: 20000 counts, maximum 19999. A plain N-digit display has counts.

VALUE OF ONE COUNT (ONE 'DIGIT') ON A RANGE

This is the place value of the last displayed digit. Use the first form when ranges are named by decade (1 V, 10 V, ...). Use the second when they are named by full count (2 V, 20 V, ... on a 2000-count meter).

SENSITIVITY OF A DVM

is the full scale of the lowest range, taken at its decade value. For example, a range that reads up to 199.9 mV counts as 100 mV. S is the smallest change in input the meter can display at all, and it equals the place value of the last digit on the lowest range.

LIMITING ERROR FOR ±(A% OF READING + M COUNTS)

is the value of one count on the range in use. The two terms are added because this is a worst-case bound.

LIMITING ERROR AS A PERCENTAGE OF READING

The count term is fixed in volts, so the percentage error rises as the reading falls.

SINGLE-SLOPE RAMP DVM

is the ramp slope in V/s and is the clock frequency. Both must be accurate and stable, because nothing cancels them and nothing averages out noise.

DUAL-SLOPE BALANCE

Use magnitudes; and have opposite polarity. For an input that varies during run-up, here is its average over . R, C and cancel as long as they stay constant for one conversion, so the only component value the reading depends on is . The integrator, switches and comparator must still behave ideally (linear and free of offset), and the result is quantised to ±1 count.

PEAK INTEGRATOR OUTPUT

Assumes a dc input and an integrator that starts from 0 V. At full-scale input this must stay inside the op-amp's output swing, which sets the smallest usable RC.

DUAL-SLOPE CONVERSION TIME

For a dc input. Add any auto-zero time on top. Integrating DVMs are slow compared with successive-approximation and flash converters, because is usually one or more periods of the power-line frequency.

NORMAL-MODE REJECTION OF AN INTEGRATING DVM

Assumes a sinusoidal interference of frequency f added to the input, and a reading equal to the average of the input over . NMRR is the ratio of the peak interference to the worst-case error it causes in that average. It is infinite (ideal rejection) when is a whole number, that is, when is an integer multiple of the interference period.

ADC RESOLUTION (SIZE OF 1 LSB)

is the full-scale input range. Some treatments use because they define the top code as equal to ; use whichever definition the question gives. An ideal rounding converter has quantisation error ±½ LSB.

SAR CONVERSION TIME

This is fixed and does not depend on . Some designs add start or end-of-conversion clock cycles on top.

STAIRCASE (COUNTER) RAMP: WORST-CASE CONVERSION TIME

The conversion time varies with . A full-scale input takes the longest.

FLASH ADC HARDWARE

All comparisons happen in parallel, so a conversion takes one clock period. An 8-bit flash converter needs 255 comparators.

LOADING ERROR ON THE DC VOLTAGE FUNCTION

The source is modelled as its Thevenin equivalent (, ), and is the DMM's input resistance. The error is taken relative to the true (unloaded) voltage . It is negative: the meter always reads low.

AVERAGE-RESPONDING METER, CALIBRATED IN RMS OF A SINE WAVE

is the full-wave rectified average of the actual waveform. The indication equals the true rms only for a sinusoid, because 1.11 is the sine form factor. A half-wave average-responding meter instead multiplies the half-wave average by ; it too is correct only for a sine.

WAVEFORM ERROR OF AN AVERAGE-RESPONDING METER

is the true rms of the actual waveform, so this error is relative to the true value. is 1.11 for a sine, 1 for a square wave and for a triangle wave. So a square wave reads 11.1% high and a triangle wave reads about 3.8% low.

TOTAL RMS OF A WAVEFORM WITH A DC COMPONENT

An ac-coupled true-rms meter shows only . To get the total rms, combine it with the dc reading as shown.

ACTIVE POWER FROM SAMPLES

The samples must span a whole number of cycles; otherwise the reading ripples. It stays correct when harmonics are present, provided the sampling rate satisfies .

ENERGY ACCUMULATION

is the sampling interval. Energy is the running sum of instantaneous power × time.

METER CONSTANT: PULSES AND ENERGY PER PULSE

C is the meter constant in impulses per kWh (imp/kWh). For example, C = 3200 imp/kWh gives 0.3125 Wh (1125 J) per pulse.

PULSE RATE AT CONSTANT POWER

P is the active power in kW and C is in imp/kWh.

PERCENTAGE ERROR OF AN ENERGY METER

Error is taken relative to the true energy. It is positive when the meter over-registers. in kWh.

Where this was asked: 2023 (1), 2025 (1). See the 2 previous-year questions on Digital instruments. This shows what was asked, not what will be asked.