Lord of the Sabbath

In the first part of Matthew chapter 12, Jesus is speaking to those around Him.  He points out that human need is greater than rituals.  That mercy is greater than rules, and mercy is also greater than fear.  Jesus shows mercy for those around Him, He heals them He feeds them.  Yet some of those around Him plot to kill Him.  What was the driving factor for the hatred that was so prominent at that time?