GraphNEx will contribute a graph-based framework for developing inherently explainable AI. Unlike current AI systems that utilise complex networks to learn high-dimensional, abstract representations of data, GraphNEx embeds symbolic meaning within AI frameworks. We will combine semantic reasoning over knowledge bases with simple modular learning on new data observations, to adaptively evolve the graphical knowledge base. The core concept is to decompose the monolithic block of highly connected layers of deep learning architectures into many smaller, simpler units. Units perform a precise task with an interpretable output, associating a value or vector to nodes. Nodes will be connected depending on their (learned) similarity, correlation in the data or closeness in some space. We will employ Graph Neural Networks to extrapolate semantic concepts and meaningful relationships from sub-graphs (concepts) within the knowledge base that can be used for semantic reasoning. By enforcing sparsity between concepts we will promote human interpretability. The integration of game-based user feedback will ensure that explanations (and therefore core mechanisms of the AI system) are understandable and relevant to humans. We will validate the GraphNEx framework, including both model performance and the performance of the explanations, with two application scenarios in which transparency and trust are critical for the adoption of AI-based systems: Clinical Genomics to assist the discovery of concepts on large, multimodal biological datasets and Activity Recognition in videos while protecting the privacy of individuals from unwanted non-essential inferences. These applications, for which understanding the decision process is paramount for trust by the general public, will demonstrate the utility of the GraphNEx framework to adapt to contexts where data are heterogeneous, incomplete, noisy and time-varying, providing new inherently explainable AI models, and a means to explain existing AI systems via post-hoc analyses.