The butterfly effect (which we are talking about here) is that small perturbations to the state of a dynamical system grow exponentially. More precisely and formally: If the states of two systems differ by $ε$ at a certain time $t_0$, at a later time $t_1$ they will differ by $ε·e^{λ (t_1-t_0)}$ (on average, and for sufficiently small $ε$). For most real systems, the exponent $λ$ (the largest Lyapunov exponent) is positive – tiny perturbations grow, and we have the butterfly effect.
As the solar system has a positive largest Lyapunov exponent, any perturbation will eventually grow so huge that it thwarts long-term prediction. This includes gravitational waves, as they obviously perturb the system (otherwise we couldn’t measure them). The only question is how long it will take for the perturbation to grow that large. If we know the Lyapunov exponent of the system, we can roughly estimate this. This is only roughly since the effect of a perturbation also depends on the nature of the perturbation – remember that the Lyapunov exponent is only quantifying the average.
Now the solar system is extremely complex, with each planet being its own chaotic subsystem. For example, we know that earth’s climate has a much higher Lyapunov exponents than the solar system and thus it is difficult to estimate the exact prediction horizon and impact of a particular perturbation. Therefore it is difficult to say whether the shifted pencil or the gravitational wave has a larger effect. If I had to wager a guess, I would say that the direct effects (such as the gravitational force between the pencil and Jupiter) are negligible, while the indirect effect via large-scale chaotic systems, such as the weather, dominates. As both perturbations grow to totally change these large-scale systems within an astronomically short time (say a year), I expect their effect on astronomical predictions to be comparable.