We consider a hybrid single-electron transistor constituted by a gate-controlled normal-metal island (N) connected to two voltage-biased superconducting leads (S) by means of two tunnel junctions (SINIS), operated as a turnstile. We show that the exchange of photons between this system and the high-temperature electromagnetic environment where it is embedded enhances Andreev reflection, thereby limiting the single-electron tunneling accuracy.