Next, we take what coefficients we have left and apply quantization to them. We define the step size to be the following:
The two variables you see are called the exponent and mantissa pair and are determined during the construction of the header markers. The denominator of the mantissa is 211. Then, each of the coefficients ab(u,v) in the sub-band is quantized to the value qb(u,v) with the following formula:
If we are doing a lossless compression, then the step size is set to one, and we do not apply the floor function. The type of compression we are doing is also stored in the main marker header.