We consider a class of systems with time-varying parameters, which are represented in the form of linear regressions with bounded disturbances. The task is to estimate such parameters under the condition that the regressor is finitely exciting (FE). Considering such a problem statement, a new robust method is proposed to identify the time-varying parameters with bounded error, which could be reduced to the finite limit. For this purpose, unknown time-varying parameters are expanded into the Taylor series in order to transform the task into the estimation of the piecewise-constant parameters. This results in the increase of the dimensionality of the identification problem. Then, the I-DREM procedure with exponential forgetting, resetting, and normalization of the regressor, which has been proposed earlier by the authors, is applied to the obtained regression. In contrast to the known solutions, this allows one to get the dimensionality of the problem back to the initial one and provide the required exponential convergence of the parameter error to a bounded set with adjustable bound under the FE condition. In addition, proposed method guarantees that the parameter error is bounded beyond the regressor excitation interval. The above properties are proved analytically and demonstrated via numerical experiments.