Wetware AI Project Status
Updated: February 2023
Biological Neuroplatform Up and Running
FinalSpark’s Biological Neuroplatform (BNP) has reached the production stages. We were able to streamline our entire experimental process over the last year and achieve the following milestones:
- Reliable neurosphere production and preservation (thousands are available now in our lab)
- Reliable microfluidics enable months of continuous measurements
- Full 24/7 monitoring of life-critical parameters (temperature, CO2, O2, N2, humidity, pH, gas bubbles, medium flows, medium levels, etc.)
- Integrated Python scripting for reading neuron signals and neuron stimulations
- Fully integrated experiment documentation: We can easily trace back the whole history of the neurosphere, beginning with the thawing step, for any stimulation experiment.
The Biological Neuroplatform allows us to collect large amounts of neurophysiological in vitro data in an efficient manner. This is all performed under controlled and repeatable conditions.
Each experiment starts by placing the neurosphere on the electrodes of the multielectrode array (MEA). The neurosphere is then stimulated using a computer system that reads electrophysiological signals. The illustration below depicts a simplified version of an experiment. The illustration below depicts a simplified version of an experiment.
After placing the neurosphere on the MEA, we maintain the environmental conditions (microfluidics, temperature, CO2, etc) to keep the neurosphere alive for several weeks or months. We have conducted over 170 experiments so far. The graph below shows the number of days we kept our neurosphere alive for each experiment, from 2019 to 2022:
Thousands of stimulations and readings are performed during each experiment, and all data is continuously recorded 24/7. Here you can see a live feed from each of our experiments. We are the only organization in the world that provides live electrophysiological measurement data on the web in real-time.
The aim of our study is to find ways to modify the network’s behavior so that it produces the expected output for given inputs (it is actually the exact same purpose as for deep learning, except that we do not have direct access to connection weights). To achieve this goal, we employ new algorithms to determine where and when to perform electrical stimulations. The algorithms are fundamentally based on machine-learning approaches and, by nature, require a lot of data. Our neurophysiological data increased by approximately 7 Terabytes from March 2022 to January 2023, as shown in the graph below.
So far, we have been able to reliably modify the network responses to store one bit of information. This is comparable to the performance of the first quantum computers, which could store 1 “Qbit” (quantum bit) of data. As a result, we have our first “Bbit” (biological bit) here.
This is just the beginning of the journey. We now investigate the latest machine learning algorithms, such as those available in Pytorch, to determine the optimal reading and stimulation strategies. This approach potentially encompasses all 30 years of literature in neuroplasticity by electrophysiology.
Why? Consider the diagram below. It shows some specific examples of reading and stimulation strategies that are current in the field of in vitro electrophysiology. We are now running 24/7 machine learning approaches, and each example is only a particular case of more complex strategies.
We are not aware of any similar attempts to solve the fascinating problem of neuroplasticity and hope that this will shed some new light on this topic.
Updated: May 2022
A CLOSER LOOK AT OUR NEUROSPHERES
We have recently been working a lot on the characterization of our neurospheres. In particular, we acquired our first electron microscopy (ESEM-Quanta) images our neurospheres . Several samples were first stabilized in glutaraldehyde and then prepared for observation by our supplier. During several hours we had the chance to interactively explore the incredibly complex microstructure of our mini brains. The three images below show one entire neurosphere (left) and closeups of structures on the surface (middle and right), showing neurites and somas.
In addition to visual inspection, we also performed our first PCR!
This is a particularly critical step since we are differentiating neuro-progenitors in our own lab and it is important to verify exactly which type of cells we are generating.
We performed an RNA extraction using 30 neurospheres split in 3 sets. RNA was first checked for quality using electrophoresis. The picture below shows the result for those samples (and the reference ladder on first line). We have the expected ribosomal 18S and 28S RNA bands for eukaryotes, both bands are well defined and the computation shows also a correct 2:1 ratio between their respective intensities. This confirms a successful RNA extraction.
Our samples could then be analyzed by qRT-PCR. 14 different markers were used. The analysis of the results showed the presence of GABAergic, Glutamatergic and Cholinergic neurons. In addition to neurons, we also found glial cells like Astrocytes and Oligodendrocytes.
The conclusion was that our neurospheres seem to feature the expected cell types laying out the basis for a functional neural network!
We also continued our activity on electrophysiology. The core of our approach is based on interactions with the neurospheres using MEAs (Multi Electrode Array). In practice, this interaction is performed by stimulating electrically some neurons and measuring their responses. We already succeeded last year in eliciting reliable spike activity, however it required statistical analysis over seconds or minutes of activity.
We are proud to announce that we are now able to get reliable answers more than 10 times faster, typically within 50 milliseconds! This required a fundamental redesign of our hardware and software system.
The image below illustrates this behavior by showing:
- Horizontally, the experiment Id, each corresponding to 1 stimulation and a 100 ms recording of the response. In total, 600 stimulations have been performed every 6 seconds during one hour
- Vertically, the 100 ms recording showing the delay between answer and stimulation for each experiment (ms)
- Each dot corresponds to a spike
It can be noticed that the network responds extremely quickly to each stimulation (less than 20 ms), and the responses are very reliably (for 94% of the stimulations).
Moreover, by overlaying the 600 responses, we can confirm that this is a well-formed typical action potential recording:
In parallel, we also worked on the microfluidics with two objectives in mind:
- After breaking our lifetime record of 120 days last year, we are now aiming to reach over 1 year of continuous recording of a single neurosphere. For such long durations, we need more durable pumps than our current syringe pushers and peristaltic pumps. For this purpose, we have started testing a new generation of mechanical pump since several weeks. The preliminary results (after many tunings) seem very promising but still need confirmation in the coming months.
- The capability to inject specific molecules was planned already two years ago and will open an entire new set of possibilities, in particular for pharmacological tests. Indeed, the state of the art provides a handful of molecules that can be used to characterize the neurons and to validate the presence of various synaptic receptors. Such tests, when coupled with electrostimulation, also enable to provide information about the internal connectivity of the network (antidromic or synaptic connections for instance). These is precious information if we want to be able to train our networks to perform specific tasks, which remains our goal.
EXPERIMENTAL R&D PLATFORM FULLY OPERATIONAL
The 2nd and 3rd quarter of 2021 have been partly devoted to solving recurring issues which were drastically limiting our ability to do good science.
The first issue was related to the micro-fluidic setup where we faced multiple challenges. After exploring and combining different approaches we found ways to consistently avoid both bubbles (which kill a neurosphere in 15 minutes) and perfusion overflows (which take-off neurospheres from the electrodes). The final solution consists in a well-balanced combination of micro-fluidic pump technologies (peristaltic/syringe), time-dependent flow rate strategies, gravitational strategic positioning of Falcon 50, and tubing. The micro-fluidic setup now allows us to keep our neuronal cell cultures alive 24/7 without the need to manually take care of medium changes.
Our latest neurosphere lifetime record is an astonishing 87 days!
The second issue was electric noise on our electrodes which hindered us from performing consistent signal recording and analysis. The noise problem has been partially solved through rewiring a part of the AC power lines and optimizing the placement of the electrical equipment in the lab. As an example, the freezer which was relocated into a separate room at a fair distance of the lab.
We have now 4 MEA (Multi-Electrode Arrays) connected to the system allowing us to run simultaneous experiments on up to 16 neurospheres. This setup generates a substantial amount of data (100 GB/day). The data contains mainly voltage recordings of the 128 electrodes and pictures taken by the various microscopes from the lab and the incubator cameras.
In addition to the neurophysiological data, we also record 24/7 a myriad of environmental parameters like the pressure (lab+incubators), temperature, humidity, microfluidic flow rates, medium color, etc. The data is not only recorded, but also processed and stored 24/7 for analysis. Processed data includes for instance average spiking frequencies, timestamps, ISI (Inter Spike Interval), etc.
In order to use the data efficiently we have for 18 months been refining what can now be called a Bioelectronic Neuroscience Platform (BNP). The platform is basically an integrated set of connected web applications that provide an efficient way to manage and use all this wealth of information. Besides analytical tasks the platform allows to trigger automatic alarms (incubator door opening or temperature deviation for instance) and remotely control many of the day-to-day operations (taking pictures, pump control, remote activation of water bath pre-heating , …).
The diagram below shows all the applications that compose the BNP. In one single place it enables to do absolutely everything we need, for instance:
- Automatic computation and archiving of metrics for spiking activity analysi
- Camera application for monitoring and automatic analysis
- Micro-fluidic pumps programming or controlling (peristaltic and syringe )
- Real-time camera view for positioning of neurosphere in laminar flow hood
- Real-time display and archiving of environmental sensor data
- Graphic interface for direct access to time series database of measurements
- Control of microscopes for archiving and automatic analysis
- Automatic stimulation and analysis using integrated Python language
- Integrated lab notes system regarding all experiments and maintenance tasks
The image below shows a screenshot of each of the 20 web applications that constitutes our BNP.
Functionally the BNP can be divided into 4 categories:
- Display:like the real-time activity view, the dashboard or the camera hood.
- Programming: like the picture capture interval app or peristaltic pumping frequency and rate app.
- Interactive: like the metric computation or the microscope picture analysis by artificial neural networks (confluence, neurosphere counting, etc)
- Editing: like the tool for lab notes or python interface, those are the only applications that are not custom made: we use oneNote and JupyterLab.
On the more scientific, and less engineering, side, one of the important results that we obtained was the proof that we can reliably generate and record electrical responses to an electrical stimulation. While this observation may sound obvious given the state of the art in electrophysiology, it is for us of significant value because:
- There is only little state of the art regarding the electro-stimulation of hIPSC neurospheres
- We performed it in our own lab with our specific set of tools (knowing that the reproducibility of known results is not always given for granted in biology…) on long living neurospheres
- The electrical answer to the stimuli were strong, statistically reliable and reproducible
For illustrative purposes, the diagram below summaries one experiment on a specific electrode connected to a neurosphere:
- The horizontal axes represents the time with a total of 15 seconds of recording time
- each small vertical line represents a time stamp at which the electrode voltage exceeded 5 times the standard deviation of the signal (computed on a sliding window of 3 seconds)
- each row represents a different stimulation experiment (60 stimulations were performed here)
- the vertical red dashed line shows the time of stimulation (2 stimulations at 50mS interval of 100nA)
In the diagram we can observe that the stimulation clearly induces a statistically significant response. It is commonly believed that this type of response is a required condition for obtaining LTP (Long Term Potentiation), which remains our primary objective for the time being. As a next step we will reproduce the stimulation protocols described in literature (for example the works of Odawara[1] and Maccione[2] regarding hIPSC stimulations and Larson[3] for theta-burst stimulations) and hope that it leads us to consistent and reproducible LTP.
[1] Odawara et al., “Induction of long-term potentiation and depression phenomena in human induced pluripotent stem cell-derived cortical neurons”, Biochemical and Biophysical Research Communications, 2016.
[2] Maccione et al., “Electrical Responses and Spontaneous Activity of Human iPS-Derived Neuronal Networks Characterized for 3-month Culture
with 4096-Electrode Arrays”, Frontiers in Neuroscience, 2016.
[3] Larson et al., “Theta-burst LTP”, National Library of Medicine, 2014.
1 Year of Neuronal Activity
After 1 year of activity, our lab is now fully operational. We started by mastering the art of culturing neurospheres starting from human progenitors. The beautiful picture below shows a collection of our neurospheres after differentiation.
We have now hundreds of them in nursery, sitting orbital shaker. The oldest were made in July 2020 and are still living.
We then acquired the hardware required for electrical activity measurement using discrete electrodes. The picture below shows 4 neurospheres, measured in parallel using 8 electrodes each. Several peristaltic pumps are in use to power a microfluidic system capable of perfusing the medium 24/7. This allowed us to break our record with 5 weeks of constant monitoring of the same neurospheres. We also installed microscopic imaging systems which allows to monitor the microfluidics status. The image below shows a Multi Electrode Array (MEA) from HEPIA with neurospheres in place.
In addition, the spiking activity is recorded 24/7 (but also many other environmental parameters), resulting in gigabytes of data that accumulated over 1 year. The neural activity is constantly broadcasted in live to our public Bioserver.net website, so anyone can check the activity of our neurospheres in real-time.
We then scaled up this system and made 4 of them.
The picture below shows a screenshot of the live webcast for one of the MEAs.
We then started using optimization algorithms approaches in order to manipulate the spiking activity. More precisely, the objective is to induce a Long Term Potentiation (LTP) behavior that we characterize with several metrics, including for instance the firing frequency of spikes. The picture below shows an example of an increased spiking activity after we performed our electrical stimulation paradigm using the Van Pelt diagram approach [1][2].
More precisely, the scatter plots represent average spiking activity recorded on each single electrode on MEA during a time interval of 30 min (on the left) and 1 hour (on the right); x and y axis indicate the mean firing rate [cnts/min]. Dashed lines delineate Poisson distribution of the spike train. If the data points are within the area indicated by the dashed lines, the occurrence of spikes is not statistically significantly different between two time intervals which are compared.
On the left, we can see that average spiking activity does not change significantly when measured at different time intervals, without any external electrical stimulation. On the right, we can see measurements of neuronal spiking activity performed 1 hour after external electrical stimulation (y axis) versus the measurements performed before the external electrical simulation (x axis). Here we can see that the firing rates between these two time intervals are statistically significantly different.
All the results were announced in a press release in September 2020.
We now have an operational and scaled-up infrastructure in place as depicted in the diagram below. But it is still far from perfect and we are constantly improving it with a special attention to the micro-fluidics since our primary focus is to reach long term viability. The next big challenge is now to make progresses towards an automated training system based on machine learning approaches.
[1] Ildikó Vajda, Jaap van Pelt, Pieter Wolters, Michela Chiappalone, Sergio Martinoia, Eus van Someren, Arjen van Ooyen (2008). Low-frequency stimulation induces stable transitions in stereotypical activity in cortical networks. Biophys. J. 94 (2008) 5028-5039.
[2] Linda van der Heiden, Ildikó Vajda, Jaap van Pelt and Arjen van Ooyen. (2011). Stimulation induced transitions in spontaneous firing rates in cultured neuronal networks also occur in the presence of synaptic plasticity blocker KN93. In: Advances in Cognitive Neurodynamics (II) – Proceedings ICCN2009. R. Wang, F. Gu (Eds.), Springer, pp. 151-155.
Updated: December 2019
We finally built our own lab!
The above pictures show our installation. Basically, this is all what is needed for basic cell culture. We obviously have all the consumables required, either hardware (tips, petri, etc) or consumables (media, additive, cells, etc). Here is a video showing how we make use of this amazing laboratory (here the execution of a cell thawing protocol).
Since we will spend hours in this environment, we also took great care of creating a pleasant and comfortable place with:
- a good AC system
- silent equipment (laminar hood, freezers, fridge, incubator)
- entirely new ground, wall and ceiling finishing (including acoustic isolation)
- good lighting control (totally obscure for fluorescence and MEA measurements, large windows for naked eye observations and uniform LED lighting in the ceiling for night work)
- face recognition access system (works better than fingerprint recognition while wearing gloves…)
After several months of practice in our new lab, we achieved a number of successful experiments using rat neural stem cells, here they are:
- thawing with well over 50% of survival rate (although cells were kept frozen at -80C and not liquid nitrogen)
- freezing with successful thawing a few days later
- 3-months of lifetime using adherent culture on plastic wells
- well over 1-month of lifetime with neurospheres
- over 40 days of lifetime with adherent culture on MEA
- expansion of neural stem cells by a factor of 100, from 400’000 to 40 million cells
- detection of peaks on an adherent culture
This last success was probably the most exciting for us. The picture below shows the electric signals detected over a 40 milliseconds period.
We noticed that:
- electrodes recording signals are in the same region (which is always a good indication that signal is not noise)
- the shape of signal is typical of an action potential
- the negative part has a large amplitude, well over 200uV
- signals between electrodes seem correlated with always the same firing pattern
- the firing pattern happens spontaneously every ~10s but can also be triggered using the excitation electrodes (we used 10 spikes at 0.5Hz).
Now, our main goal is to progressively move to a situation where we are able to train those neurons. For this, we still need to achieve a number of additional steps:
- replicate our experiment that lead to peaks detection
- increase lifetime on MEA and get more spikes, for this we will follow this strategy:
- do more experiments in parallel
- test alternatives to adherent cells culture on MEA (neurospheres, membranes, etc)
- test different MEA systems (we are currently testing 3Brain, Maxwell and MCS)
- implement a real-time 24/7 MEA infrastructure to train the biological neural network
Updated: February 2019
About carbon based life and neurons
We started to work on our biological neural networks. For this application, living neural cells are needed. There are 2 ways to achieve this:
- Buy neural cells and start interacting with them
- Buy neural progenitors and start multiplying them, then only start the interaction
We chose the second option as this gave us the possibility to scale freely the number of neurons and number of experiments. We used fetal neuron stem cells from rat as well as human neurons iPSC. We applied the protocols provided by the cells suppliers (ThermoFicher, RndSystems, Axol) for thawing, culturing, passaging, freezing, and MEA coating.
One of the vial containing stem cells that we received.
We quickly noticed that a critical factor for growing cells is the so-called confluence level, which is basically equivalent to the surface fraction that the cells cover. This parameter is critical, below 30% cells will progressively die over the following 10 days, above 95% they may start to differentiate into neurons, which is not desirable at this stage. For measuring this confluence reliably, we actually trained an artificial neural network (U-Net type) in order to automatically label neurons. Our software can then use this label image in order to automatically compute the confluence value. We used this method to monitor systematically the confluence for each medium change.
Our webapp showing a picture of our cells, the processed image using our artificial neural network and the resulting confluence computation.
After several months of work, we were able to successfully obtain more than 10 million stem cells.
Cells are visible with the naked eye after centrifugation.
Most cells were frozen for later use and a fraction of them were put over 8 Multi Electrode Arrays from different suppliers.
Change of medium on one of our MEA.
We then stopped including Epidermal Growth Factor and Fibroblast Growth Factor in order to start the differentiation into neurons, astrocytes and oligodendrocytes.
Picture showing an MEA and our neural cells.
We are now starting the measurements in order to assess the electrical activity or our cells. If those experiments are successful, we will then start using electrical excitations and assess how the network reacts. This will involve C#/C++ programming using the API provided by the MEA suppliers. At this stage, it will be possible to start implementing digital machine learning algorithms in order to achieve the training of our biological networks.
Updated: December 2018
From Digital to Biology
Since its inception, FinalSpark has been actively doing fundamental research using digital computers. We now believe relying exclusively on digital computing to reach a General Artificial Intelligence is a dead end, and here is why:
From the beginning of the research in AI, say 50 years ago, the leading research strategy has fundamentally relied on digital computing (and so we did). Very few alternate approaches have been actively pursued. Among those alternates, one could cite analog electronic circuits implementations of ANN, liquid state machine (or reservoir computing) with real water (or with biological neurons…).
Three basic reasons can be given to explain why AI research mainly relies on digital computers:
- Flexibility, ease of designs of experiments, actually an effective tool if the goal is to write publications
- Proven track record: digital computing has proven to be effective in many automation tasks, and cognitive processes can be considered as yet another automation task
- Understandability: the computer engineer understands what is going on when he writes and uses a computer program. However, this particular point can be debated. Indeed, understandability has been lost for some particular classes of approaches. For instance, a deep learning artificial neural network is usually seen as a black box since its internal computing processes are too difficult to conceive for a human being. Another example is genetic programming where the algorithm itself is invented by the computer and can sometimes hardly be understood by a human being
This research provided a number of useful tools like Bayesian networks, Fuzzy logic or Artificial Neural Networks, to name a few… but these are just this: tools. Intelligence seems elusive: each time one successfully automates a cognitive process (like playing chess, go and recognizing pictures), we realize we did not get any closer to intelligence.
Let’s consider the most successful tool used in AI: artificial neural networks: they do not appear to anyone serious in the field as a realistic approach to build a GAI.
Why is that?
We believe there are two unsolved problems with ANN:
Lack of training algorithm: yes, we can simulate 100 billion neurons, but we don’t know how to connect them to achieve any high level cognitive processes. Even worst, it is not even clear what ANN models should be used, so we are left with many alternatives that researchers have been testing for decades now:
- what training strategy? (global, local, supervised, error based, Wissner-Gross like, etc.)
- what connectivity models? (electrical, chemical, quantum, etc.)
- what neuron model? (sigmoid activation, spiking, biologically realistic, etc.)
Actually, it is not even known if choosing one or another of those models simply matters at all! (at least we know that neurons need to exhibit a non-linear behavior).
Worst of all, brain dynamic is so mysterious that 30 years of work on neuro-degenerative diseases have basically lead to nothing. Situation is such that some neurobiologists even start to wonder if the importance of neurons was not overlooked (…and neurons are the focal point of AI since the beginning of connectionism) and that maybe we should rather focus on glial cells… (see for instance “l’homme glial” of Yves Agid and Pierre Magistretti).
Computer power available today is just not sufficient: the problem with this argument is it was the same 30 years ago, and in-between computer power has increased by several orders of magnitude. Based on our own experience at FinalSpark, the computing power is a constant issue: even with a simple neuron model and simple training rules, as soon as you start doing recursive topologies (which at least we can assume are more biologically plausible than feedforwards…) and looping on a few parameters, you end-up waiting a lot (we are using our 5kW HP server of 16 blades) for ridiculously small networks of a few thousands neurons.
So given all that, we decided to find a radical solution that solves simultaneously point a and b (while instantly creating a myriad of new issues…): instead of simulating artificial neural network, let’s culture biological neural networks and interact with them using electrophysiology technologies. This way we hope to get for free the built-in training capabilities and a realistic system with virtually boundless scalable computing power: remember dear reader, your 100 billion neurons are consuming only 20W when reading those lines…
Updated: September 2018
PONG – A Path Entropy Inspired Solution
Most of the machine learning tasks are using a concept of “error” (also called loss in deep learning or fitness in genetic programming or reward in reinforcement learning) in order to quantify the difference between the expected and actual performance of an AI system. For instance, if an artificial neural network \(F\) is expected to output 1 for the input \({ i }_{ 1 }\) and 0 for input \({ i }_{ 2 }\), the error \(e\) over the 2 samples can be defined using the L1 norm: \( e=\left| F\left( { i }_{ 1 }-1 \right) \right| +\left| F\left( { i }_{ 2 }-0 \right) \right| \). This so-called “supervised learning” approach is used in the vast majority of deep learning systems and has proven to be very effective. Nevertheless, it has one major drawback: it requires to explicitly provide to the system a number of examples where the input and output are known. As a result, in the field of deep learning, this requirement has led to the creation of an entire new industry and new job profile (“AI trainer”) only dedicated to the generation of those training data.
In practice, there are many cases where this training data is unavailable, for instance, how do you define the training data for a “thinking machine”? One approach consists in using a more general behavioral metric. For example, consider the case of the Pong game where the intelligent agent must control a bouncing ball with a pad. The traditional approach would be to train a neural network with a number of examples of input images of the pad game and the corresponding move of the pad. A less supervised approach was proposed by Wissner Gross. He says that an intelligent behavior consists in maximizing its future options. For this, he introduces the concept of “path entropy” which in this case would translate into “At one time step, how should I move the pad so that I have the most options for my next pad move?”. In practice this is not easy to compute, so we used an approximation which consists in saying that the pad should move in such a way that the ball is equally present over the entire pong game. We are actually replacing the path-entropy by a pure entropy \(E\) which can be easily computed using the ball probability presence \(P\) at the location \(\left( { x }_{ n }^{ t },{ y }_{ n }^{ t } \right)\) for the bin \(n\) over \(t\) time steps using \(E=\sum _{ n,t }^{ }{ P\left( { x }_{ n }^{ t },{ y }_{ n }^{ t } \right) } \times \ln { \left( P\left( { x }_{ n }^{ t },{ y }_{ n }^{ t } \right) \right) } \).
Now that we have a non-supervised (or at least “less-supervised”) performance metric, we still need to define an “engine” for our smart pong player. Based on our experience in this field, we chose to use a Linear Genetic Programming approach. With this approach, the source code of the software used to move the pad is written automatically using operators inspired by genetic selection. The software has access to the position of the ball and to the position of the pad, it returns the displacement of the pad. The source code (so-called “individual”) is generated using an intermediate language and converted in x64 machine code using our custom-made real-time compiler. We used individuals of up to 300 lines, with 256 populations (one per core) of 50 individuals, each population running on a HP BladeCenter system populated with 16 Proliant blades. The source code was evolved using mutation, crossover and migration operators. After 10 millions assessed individuals, we ended up with a fascinating solution, shown in the animation. After simplification of the source code, it corresponds to moving the pad at each time step according to the equation:
\(\Delta p=\frac { 1 }{ x } +2-\frac { p+y }{ x } +x-p-y\),
where \(\Delta p\) is the displacement to apply to the pad, \(\left( x,y \right) \) is the position of the ball and \(p\) is the position of the pad. Even knowing this equation, it is not obvious to understand why this solution works. By the way, this complexity demonstrates the power of using machine learning in general and genetic programming in particular. Indeed, it enables to develop system which complexity exceeds our understanding capabilities.
To conclude, we successfully solved Pong without explicitly indicating to the system that the pad should not miss the ball. Instead, we used the variability of the ball position across the board as the sole indication of performance. It is an interesting result since this approach can be used each time that it is impossible to express numerically a performance, which is typically the case of any high-level cognitive activity.
Updated: August 2018
FinalSpark Webcast on “FREEDOM OF ACTION”, June 27th, 2018
SinoSwiss Artificial Intelligence Event 2017 EPFL Switzerland
Updated: November 29th, 2016
A new programming language
Initially we started by designing a new class of programming language that we thought was more appropriate to implement Linear Genetic Programming. This new language has the property that each instruction can be transformed, up to a certain extent, continuously into a different instruction. A scalar factor alpha controls the transformation. We coined this new programming language CPL (Continuous Programming Language). As an example, consider the instruction x=1. With alpha = 0, x is left unchanged (equivalent to the NOP instruction). For alpha=0.5, x is the mean between the current value of x and 1. Finally for alpha=1, x is equal to 1.
Hypersurface error
Linear Genetic Programming
We then refined our genetic engine (mutations, crossover, migrations). To make sure that we have already reached state of the art performance, we benchmarked our engine against a standard classification problem (see Brameier and Banzhaf “A Comparison of Linear Genetic Programming and Neural Networks in Medical Data Mining”, 2001) and ran our genetic engine on the medical data set. The performance testing against the benchmark improved the genetic engine in many ways. We were able to use metagenetic methods to tune the many genetic parameters (mutation rate, population sizes, etc) . We were also able to visualize the populations to get a sense of what was going on as they evolved.
The picture below is a graphic representation of an entire population: each column represents a program, each color represents a different class of instruction (arithmetic, memory, flow control, etc) and the individuals are sorted based on their fitness, with the best one on the full left. We can see for instance that in this population the best individuals are similar (horizontal lines of a single color on the left indicate that the individuals, i.e. vertical lines, are similar) and that individuals with low fitness are tend to fewer number of instructions as indicated by the black spikes on the right side of the plot.
Graphical representation of program instructions against fitness
At this stage one of the open questions was: how likely is it that LGP (Linear Genetic Programming) will evolve code implementing the concept of neurons and getting us to an artificial intelligence? From our experience, we realized that this was extremely unlikely. We therefore decided to help the system by embedding the concept of neurons into the basic system. For this purpose, we introduced the concept of ”metaneuron”, a structural element whose behavior (meaning its relaxation and learning properties) would be defined by an LGP evolved program. The concepts of metaneurons is commonly used in neural networks for artificial intelligence. However, in our case the concept is much more elaborate as it is combined with LGP.
As a first check, we tested the genericity of the approach by training a multilayer network on the XOR problem. Although we were able to find solutions for the XOR problems, we also observed that the metaneurons were not collaborating as in a neural network to solve problems. Several improvements were necessary, such as imposing a grid topology (see figure below) with 8 neighbor connections (this assures that information has to travel into and through the network to reach the output).
Next, we checked for cases of chaotic behavior, situations where changing boundary conditions, for instance one single connection weight, would dramatically increase the system error. Given that healthy biological neural networks are very robust to changes that naturally occur in all physical systems we want our system to have the same characteristic. The next requirement was defined as the capacity of “true” learning. This, although obvious, is extremely important as in many instances we found that the LGP programs just secretly stored several internal states useful when executing the program. Instead of learning, the programs just tested the various sets of pre-stored states until the error was good.
Example of a grid topology needed to implement metaneuron behavior
To avoid this pitfall effectively we used a training set and a distinct test set. Consequently we designed the training set with 8 Boolean 2-inputs functions f1(i,j), f2(i,j)… f8(i,j). For instance, f1 could be the XOR problem. The genetic engine has then to find a code such that the metaneurons can learn any of those functions. Once the learning program is found, its learning performance is then assessed using a test set made of 8 other Boolean functions f9,..f16. The number of training sets and training set size has of course a direct impact on computational complexity and required us to increase the computation speed by several orders of magnitude.
Basic computational speedup was reached by hardcoding the relaxation function in C++ (using integrate and fire) and only evolved the learning program with LGP. Using several hours of computing on over 100 cores, the system found learning algorithms for different network sizes (3×3, 4×4, and 5×5). The found learning programs vastly outperformed state of the art optimization techniques. They were 2,300 times faster than the genetic algorithmic search and 100,000 times faster than a random search.
Performance of LGP approach against Random Search and Genetic Algorithms
Is our approach to reach artificial intelligence disruptive? Yes, it is. Are our results disruptive? Yes, they are. We are therefore motivated to take this to the next level. Future works will focus on extending this approach to more complex problems by using more neurons, more complex topology and constructing more powerful instructions for our LGP language. At this stage we are still far from having a thinking machine in our hands, however, we feel that the our tools become more and more powerful and that we are on the right track to tackle the problem of artificial intelligence.
CONTACT US
We're not around right now. But you can send us an email and we'll get back to you, asap.