Stochastic thermodynamics and the associated fluctuation relations provide the means to extend the fundamental laws of thermodynamics to small scales and systems out of equilibrium. The fluctuating thermodynamic variables are usually treated in the context of either isolated Hamiltonian evolution, or Markovian dynamics in open systems. However, there is no reason a priori why the Markovian approximation should be valid in driven systems under nonequilibrium conditions. In this work, we introduce an explicitly non-Markovian model of dynamics of an open system, where the correlations between the system and the environment drive a subset of the environment out of equilibrium. Such an environment gives rise to a new type of non-Markovian entropy production term. Such non-Markovian components must be taken into account in order to recover the fluctuation relations for entropy. As a concrete example, we explicitly derive such modified fluctuation relations for the case of an overheated single electron box.