Let $X$ be the fractional Brownian motion of any Hurst index $H\in (0,1)$ (resp. a semimartingale) and set $\alpha=H$ (resp. $\alpha=\frac{1}{2}$). If $Y$ is a continuous process and if $m$ is a positive integer, we study the existence of the limit, as $\varepsilon\rightarrow 0$, of the approximations $$ I_{\varepsilon}(Y,X) :=\left\{\int_{0}^{t}Y_{s}\left(\frac{X_{s+\varepsilon}-X_{s}}{\varepsilon^{\alpha}}\right)^{m}ds,\,t\geq 0\right\} $$ of $m$-order integral of $Y$ with respect to $X$. For these two choices of $X$, we prove that the limits are almost sure, uniformly on each compact interval, and are in terms of the $m$-th moment of the Gaussian standard random variable. In particular, if $m$ is an odd integer, the limit equals to zero. In this case, the convergence in distribution, as $\varepsilon\rightarrow 0$, of $\varepsilon^{-\frac{1}{2}} I_{\varepsilon}(1,X)$ is studied. We prove that the limit is a Brownian motion when $X$ is the fractional Brownian motion of index $H\in (0,\frac{1}{2}]$, and it is in term of a two dimensional standard Brownian motion when $X$ is a semimartingale.