During the past two decades, philosophers, psychologists, cognitive scientists, clinicians and neuroscientists strived to provide authoritative definitions of consciousness within a neurobiological framework. Engineers have more recently joined this quest by developing neuromorphic VLSI circuits for emulating biological functions. Yet, to date artificial systems have not been able to faithfully recreate natural attributes such as true processing locality (memory and computation) and complexity (1010 synapses per cm2), preventing the achievement of a long-term goal: the creation of autonomous cognitive systems. This project aspires to develop experimental platforms capable of perceiving, learning and adapting to stimuli by leveraging on the latest developments of five leading European institutions in neuroscience, nanotechnology, modeling and circuit design.
During the first year of the project, the non-linear dynamics as well as the plasticity of the newly discovered memristor were shown to support Spike-based- and Spike-Timing-Dependent-Plasticity (STDP), making this extremely compact device an excellent candidate for realizing large-scale self-adaptive circuits; a step towards “autonomous cognitive systems”. The intrinsic properties of real neurons and synapses as well as their organization in forming neural circuits are currently exploited for optimizing CMOS-based neurons, memristive grids and the integration of the two into real-time biophysically realistic neuromorphic systems. The overall aim here is to exploit distinct memristive experimental platforms with conventional as well as abstract methods to evaluate the technology and its computation capacity.
To facilitate this research, three scientific fields (neuroscience, electron devices and theory of computation) are effectively coupled, with our approach leveraging the diverse merits arising from the distinct interactions of the individual groups for further developing the supporting scientific foundation. This approach provides the means to study the processing and encoding of spatio-temporal information in neuro-synaptic networks, to advance the current state-of-theart in computation and lead to breakthroughs in emerging applications on healthcare and automation. The anticipated interactions between the relevant disciplines are shown in Figure 1, delineating the main objectives and the employed scientific approaches:
- Development of biophysically mimetic synapses (Memristor-‐based) and neurons (CMOS-based circuitry).
- Modelling and implementation of neural circuits.
- Development and establishment of novel brain-inspired computational principles.
- Optimisation of device and computational models.
- Practical system-level application of novel computational algorithms.
- Set-up of real-time experiments and benchmarks.
- Development of system-level hybrid CMOS/Memristor platform to demonstrate autonomous behaviour.
To date, we have succeeded to realistically emulate synaptic plasticity with memristive nanodevices. This is the prime objective of PNEUMA as it serves as proof that memristors can indeed be employed as fundamental blocks for establishing unconventional (neuromorphic) computation hardware. To achieve this, we first reviewed the neuro- and neuromorphic computing approaches that can best exploit the properties of memristor and nanoscale devices, and then devised a novel hybrid memristor-CMOS neuromorphic circuit that represents a radical departure from conventional neuro-computing approaches, as it uses memristors to directly emulate the biophysics and temporal dynamics of real synapses. We have pointed out the differences between the use of memristors in conventional neuro-computing architectures and the hybrid memristor-CMOS circuit proposed, and argued how this circuit represents an ideal building block for implementing brain-inspired probabilistic computing paradigms that are robust to variability and fault tolerant by design.
More specifically, our achievements to date have addressed all the following points, as per our original application:
- Developed and tested memristive devices and evaluated existing analytical models (D1.1 and D1.2).
- Established appropriate instrumentation techniques for evaluating and altering the memory state of individual devices as well as networked devices. Moreover this was extended on designing an individual neuron both with discrete components as well as in a CMOS technology (D2.1 and D2.2).
- Performed a detailed study of the biophysical mechanisms involved in plasticity and learning that is mapped to sub-‐threshold analog VLSI and memristor device physics (D3.1).
- Designed hybrid memristor/CMOS neuromorphic circuits that can operate using the same principles observed in detailed studies of the biophysics of synaptic plasticity in real neurons (D3.2).
- Exploited some interfacing strategies (monolithic and hybrid) of memristors with CMOS technologies (D5.1). Interestingly, while integrating the first devices onto CMOS platforms (D5.2), we discovered a promising technique that allows for implementing memristors in unmodified CMOS technologies by leveraging the intrinsic process flowchart of the technology.
All objectives of this project were aimed at demonstrating and exploiting the correlation of memristors plasticity with long-term plasticity of biological synapses. Besides this, we were also able to establish striking correlations with memristors’ volatility and short-term synaptic dynamics that open up new avenues for exploitation, as metastable effects leading into rate-limiting state restoration are currently shown to play an important role in computation. Other technological constraints were also identified and studied (e.g. the co-existence of bipolar and unipolar switching mechanism and/or the probabilistic nature of resistive switching). These effects are surprisingly shared among the biological and electronic medium, engaging the TUG partner earlier from what we originally planned. Based on these constraints, the consortium decided to utilize a probabilistic neural architecture with local spike-timing-dependent plasticity (STDP) for synaptic updates. This architecture was improved and extended in several ways by partner TUG in order to develop the final model to be implemented in hardware. The constraints of the PNEUMA hardware, in particular the memristive synapses, demand a detailed analysis of the behaviour of the neural model under suboptimal conditions. The investigations of SOTON partner showed that we can utilize rather large memristive arrays, with each memristor assuming only two or a few memristive states. Partner TUG therefore developed a theory of Spike-based Expectation Maximization (SEM) learning with binary synapses and stochastic weight updates. Preliminary computer simulations revealed that the model still produces excellent learning results.