Even if only 1 were on it, that's 3%. Misery gives you the whole thing immediately, so I don't see what that has to do with anything. In any case, in combination, that's an 8% damage increase just by putting up SW:P; I don't see why you'd want to miss out on that.
I misread Misery then. Also, Shadow Weaving is being nerfed next patch to 2/4/6/8/10%.
I usually would go for things that would keep my downtime to a minimum versus getting a small damage boost while grinding. It seems to be similar to how pointless Impale is for Warriors if you're Dual Wielding.
Threaded Mode