• Martin Storsjo's avatar
    Extend the GetInvInt table to 80 elements · 9a323405
    Martin Storsjo authored
    This is a bug fix patch from Fraunhofer, amending/replacing/extending
    the previous commit which only extended the table to 55 elements.
    (In my previous fix, 55 seemed to be enough for the issue at hand at
    least. A few of the coefficients had slightly different rounding than
    what Fraunhofer had used though).
    
    This will show up in AOSP in the next code drop as well.
    9a323405
fixpoint_math.h 14.8 KB