Your first mistake was that you didnt have enough units. You had enough food and gold, you should have made more rifles. They are good vs grunts and riders. He had few wind riders and rilfes can own them too, the same with bats. Griffons are good but usually to counter taurens. With beastery it could be expected. You needed 2 or 3 sorc and if you had more rifles, upgraded with lvl 1 powder and even with long rifles. Just put the elemental, focus on wind riders, slowed grunts and wolf riders can just attack your elemental and MK but you had priests. No need of griffons here.
