The Inert Doublet Model (IDM) is a simple extension of the Standard Model, introducing an additional Higgs doublet that brings in four new scalar particles. The lightest of the IDM scalars is stable and is a good candidate for a dark matter particle. The potential of discovering the IDM scalars in the experiment at the Compact Linear Collider (CLIC), an e$^+$e$^-$ collider proposed as the next generation infrastructure at CERN, has been tested for two high-energy running stages, at 1.5 TeV and 3 TeV centre-of-mass energy. The CLIC sensitivity to pair-production of the charged IDM scalars was studied using the full detector simulation for selected high-mass IDM benchmark scenarios and the semi-leptonic final state. To extrapolate the results to a wider range of IDM benchmark scenarios, the CLIC detector model in DELPHES was modified to take into account the $\gamma\gamma\to$ had. beam-induced background. Results of the study indicate that heavy charged IDM scalars can be discovered at CLIC for most of the considered benchmark scenarios, up to masses of the order of 1 TeV.