I've found it with all of my rifles. There's the issue of the inertia of bits of metal moving around, how square I am to the rifle to obtain the best straight through pull without applying an off-axis squeeze, heartbeat, lightness of grip, the forward motion of the firing pin and contact with the FP shoulder itself plus the slight bolt handle lift on dry firing. That's a lot of things going on!
I have found that altering body position can help reduce it a little (fractionally) hinting that perhaps my pull isn't as axial as it ought to be (got to work on that). The lift on the bolt handle may also be causing a small reaction.
I think there may be more damage done to groups over-analysing this though than simply settling into the position that seems to give the steadiest dry firing position, and returning to basics seems to work every time for me. If the cross-hairs are not bang-smack centre when looking away and then looking back through the scope, the grouping will be off. If I haven't got the right LOP (adjustable), my groupings will be off. If cheek position isn't correct, groupings will be off. There's so much else to worry about.