When evaluating the gain of an FIR II IP core or a user-designed fixed-point FIR filter by comparing the input and output amplitudes, the gain may differ from what is expected or anticipated.
The following three (or a combination of the three) are the most likely causes.
Misunderstanding of the decimal point position of fixed-point numbers
Gain calculation and display function of the tool
Misunderstanding of filter characteristics
Misunderstanding of decimal point position for fixed-point numbers
If the decimal point position is wrong for either input or output due to a simple careless mistake, a value like 2^N multiplied by 2^N will be generated as a gain difference. (N is the difference as the number of binary digits, both plus and minus are possible.)
This can be resolved by modifying the decimal point position (equivalent to a shift operation).
However, if the decimal point is incorrectly positioned in some values during the calculation, such as in user design, it will result in a deviation that will not be multiplied by 2^N. It is necessary to correct the wrong part strictly.
By the gain calculation and display function of the tool
In the Coefficient tab of the FIR II IP core, the frequency response (amplitude) characteristics are displayed graphically, and since the passband is almost "0dB", it appears that gain 1 has been achieved. However, this is only a relative gain, and gain 1 in the passband is not always achieved.
In particular, when "Coefficient Scaling = Auto" is set, auto-scaling is performed to increase calculation accuracy, and the coefficients used in the calculation may be scaled by a multiplier other than 2^N times from the value given when generating the IP. In this case, correcting the decimal point position will not resolve the problem.
The above auto scaling function can be disabled by setting "Coefficient Scaling = off".
If it is deactivated, the coefficients at the time of IP generation will be used as they are, and depending on the expression method and values, they may not be recognized properly, so rechecking and readjustment are necessary.
Even with tools used by users to prepare coefficients, there is a possibility that the relative gain display may be misunderstood as described above, or that the gain may be different from what was expected as a result of not paying enough attention to the gain.
Misunderstanding of filter characteristics
Depending on the coefficient design method, accuracy (number of bits), and number of taps, even when the passband gain is close to flat, there may be a slight ripple (error). Also, the gain in the suppression band usually has a very large ripple below the target gain.
Depending on the design method, the treatment and definition (gain value) of characteristic parameters such as cutoff frequency may differ.
Applications requiring gain adjustment
In the wireless and telecommunication fields, where FIR filters are often used, the absolute amplitude is not very important because the waveform contains the transmission information.
In measurement and other applications where amplitude and power are important, auto-scaling is not necessary to accurately transmit information.
In such cases, there is no problem if the given coefficients realize the expected gain, but the absolute gain may be different from the expected gain due to the above problem even if the relative gain is OK, and adjustment is necessary.
Gain Calculation Formula
The formula for gain adjustment is shown here. (For details, please refer to a textbook on digital signal processing.
Transfer function of FIR filter: H(z) = Σ (k=0 to N-1)h(kT)z^(-k)
where T is the sampling period, h(kT)=h(k) is the FIR tap coefficient (real number), and "^" denotes a power.
DC gain of FIR filter: |H(0)| = Σ (k=0 to N-1)h(k) <sum of coefficients>.
The output after a sufficient time of unit step input, i.e., the sum of the coefficients, is relevant.
In the case of an LPF, this value is the target of adjustment, which makes the calculation easier. (Note the ripple in the passband)
FIR filter gain (amplitude-frequency response): |H(F)| = √( {HRe (F)}^2 + {HIm (F)}^2 )
where F is the ratio of the input frequency f to the sampling frequency Fs (normalized frequency) F = f/Fs
HRe (F) = Σ (k=0 to N-1)h(k)cos(2πkF)
HIm (F) = -Σ (k=0 to N-1)h(k)sin(2πkF)
Apply this transformation to the equation: z=e^(jωT), ω=2πf, e^(jθ) = cosθ + j*sinθ where j is the imaginary unit.
Hints for actual calculation for gain adjustment
If the gain is actually calculated using the above formula and differs from the target value ( Htargert, e.g., 1), it is generally adjusted by multiplying by the ratio α. Ratio α = Htargert/|H (F)| (scaling by user)
One option is to leave the FIR filter part as it is and multiply the output by α to make the overall gain Htargert
When adjusting the FIR filter, α is multiplied uniformly across all coefficients, and h'(k) = α*h(k) is the new set of coefficients.
However, it is recommended to use a spreadsheet tool such as Excel, because the process of obtaining the sum and updating it by multiplying the ratio by all the coefficients is a tedious work for the number of Taps. The sum can be obtained relatively easily, and uniform updating of coefficients can be handled by copying the multiplication equation.
When adjusting the gain near the cutoff frequency or in the passband of a BPF or HPF, a calculation of |H(F)| is required, and a spreadsheet tool such as Excel is recommended. As for functions, PI( ), SIN( ), COS( ), SUMPRODUCT( ), and SQRT( ) can be used.
Excel has a graph display function that allows you to plot the gain change around the frequency you wish to adjust and check it.
It is possible to increase the accuracy of the target gain considerably (e.g., Htarget = 1.0000...) by using Excel or other software. However, since the number of bits in the coefficients is finite, no matter how much α is multiplied uniformly, there may be errors that cannot be removed. In such cases, the decision is made to compromise, or to make fine adjustments by increasing or decreasing some coefficients by about LSB, allowing for changes in characteristics, although this is a departure from the principle of multiplying all coefficients by α.