A hybrid concept for an Inverse Compton Scattering (ICS) based compact ultrafast X-ray pulse source, relying on the combination of a conventional S-band electron gun with a dielectric-loaded waveguide driven by a multicycle THz pulse (THz linac), is under investigation by the authors. The target of this hybrid concept is to generate pC-class ultrashort electron bunches (≤ 1 fs rms), at moderate energies (15 to 20 MeV) and focused to a transverse size below 10 μm rms while keeping a compact beamline (≤ 2 m). This contribution first presents the optimization of the concept through beam dynamics simulations and the simulation of the properties of the X-rays generated via ICS. The main part is then dedicated to two aspects. First, a description of the main technical requirements for the concept and the associated challenges is given. Finally, a tolerance study deals with the influence of various jitters and beamline imperfections on the achievable electron bunch properties.