AI models are likely to become useful tools to support clinical decision-making especially in high mortality diseases such as lung cancer. However, the black-box nature of these models remains nowadays the main challenge to be addressed to employ AI in the clinic. This work describes the preliminary stages and results obtained by implementing a novel explainable approach in radiomics pipeline for local recurrence prediction of lung cancer and its technical validation relying on the high energy physics domain.