© Published under licence by IOP Publishing Ltd. The method for numerical integration of Cauchy problems for ODEs with blow-up solutions is described. It is based on introducing a new non-local variable that reduces a single nth-order ODE to a system of first-order coupled ODEs. This method leads to problems whose solutions are presented in parametric form and do not have blowing-up singular points; therefore the standard fixed-step numerical methods can be applied. The efficiency of the proposed method is illustrated with two test problems. It is shown that the first Painlevé equation with suitable initial conditions have non-monotonic blow-up solutions.