Mapping a smaller area of the touchpad to my screen really makes it impossible to be precise. I suggest to implement an option to set how much averaging the program would do. I did some research and an exponential averaging technique looks best for this. I played around with the code and a weight between 0.3 and 0.4 is a decent range such that the latency and cursor instability are at equal trade-off.
const double weight = 0.4;
coordsAvg.x = weight * (double)coords.x + (1.0 - weight) * (double)coordsAvg.x;
coordsAvg.y = weight * (double)coords.y + (1.0 - weight) * (double)coordsAvg.y;