The defendant shall not be liable for such annual value for any longer time than five years before the suit, or for damages for any such waste or other injury done before such five years, except when he claims for improvements as aforesaid.