A strange carnival with automobile-animal hybrids. Michael Tyka/ Google
Researchers at Google have found a way to make their server computers create astonishingly beautiful images by feeding them information at different levels of their identification network. One of the more fascinating aspects is that the images look strangely similar to the fractalized visions in altered states of consciousness. In this essay, I speculate that this technology might be a first attempt at visually capturing experience in a cognitive system, thus giving a glimpse of a solution to the problem of qualia.
Here we have images, created by artificial intelligence (the Google server computers) as it is identifying asked for objects by searching within it’s own bank of information. The normal identification process of the server computers is that an image passes through several layers of artificial neural networks, dedicated to recognition of certain features. The lower levels of this network are dealing with rough contrasting structures like edges and corners. At intermediate-level layers, individual object-like features are interpreted like a door or a leaf. In the final layers the computer interprets “the bigger picture” giving an illustrated output of what is asked for (a picture of a house for example). However, the above image is the product of “turn[ing] the network upside down” and feeding arbitrary visual input through selected layers while asking the computer to recognize objects by it’s own interpretations.
Drawing upon the interpretations of the previous layers the computer outputs a subjective representation through feedback loops of certain important aspects of the identified object, elaborating recognized features through iteration. If some aspect of the asked for object looks like something else, it will generate more of it, and the higher up the neural chain the input is added, the more detailed the iteration and meaningful the elaboration. The end product is a surrealistic and dreamlike depiction made up by countless fractal patterns of various intermingled images from the server bank. Doesn’t it look kind of familiar? Kind of like something you would expect from a vivid dream or the closed eye visions in an intense psychedelic experience? I kind of think so. And apparently some other people think so too.
I argue that not only does it look similar, these images are created by the same kind of neural processing of information as the brain (even if in a very simplified way). In normal waking consciousness, we, as the computers operate at the lower levels of the neural system, feeding information “upward”. We identify the world around us by filtering out unnecessary information in favor of a coherent experience. In altered states of conscious – trance, dreams and psychedelic experiences – the brain overrides these lower-level layers by increasing entropy of the neural network. It tweaks the system, sending incoming information through novel pathways, giving rise to more free associative thinking and perception.
While it certainly can feel very otherworldly, the hallucinations in the psychedelic state are (probably) not visions of transcendent dimensions, but actual and immediate psychological responses to external and internal stimuli. Multi-sensory input mingling with memories of past experiences and future planing, neurons firing in all directions producing fantastic visions and experiences which are often presented in a fractalized fashion. Very much like “the dreams” of our artificial friends.
So here’s a bold suggestion. If we can perceive and experience it, and the machines (semi-) independently can replicate it by visual representations, this could mean that it is the first empirical evidence of mind and matter being of the same substance. That qualia is not a hidden dimension, but something that is actually manifest in the material world. These artificially created images suggests that we now can record and measure how associative cognitive networks create experiences by interpreting stimuli inputs. At least in the visual domain. It is important to note that what we are looking at is not just random noise interpreted by us as meaningful, but an output of a cognitive system finding meaningful interpretations in random noise. This process is not so different from that of a human artist, and the artificial renderings are as real a depiction of experience as any painting, and in a sense even more truthful.
In the case of the painter painting a painting, whether it is an image of a landscape or the surreal abstraction of a feeling, it is the experience of stimuli that has passed through the sensory modalities that is depicted on the canvas. But, the painter is always limited by the human inability to accurately convey our conscious experience. Elapsed time and fading memories, change of context and limited artistic skills are all factors that skew the portrait of the original experience. However, one could argue that the act of painting in itself is a temporal event and the cognitive process a continuous flow of interaction. The experience is thus slowly manifesting itself in the layers of the painting. Still, it lacks the precision of a truthful depiction (and of course, this is often not even the intended purpose of most art).
The computers used for the artificial renderings are also producing their images in a self-updating continuous event. But unlike the human painter who is divorced from experience by space and time – also filtering out irrelevant information – the computer is accurately recording every instance of the process. But why is this different from that of any other recording device? The key difference between these artificially created images and those of an ordinary camera is that the images are the products of creation by meaningful interpretations rather than the arbitrary capture of photons. A still picture of a dog is nothing but random visual noise stuck on paper until a cognitive system interprets that picture as meaningful. It is the observer who creates the dog.
The higher the level of neural layers in charge of performing the interpretation of the sensory input, the more abstract the depiction. This corresponds with the information processing during normal vs. altered states of consciousness. Where normal consciousness have firm and solid renderings to optimize precision performance, the altered states invoke meaning in the details, overriding the usual cognitive filtering. The deeper one goes into the altered states, the higher the resolution of the details which in turn feeds back into the system and fosters further interpretations. At high doses of psychedelic drugs or very deep states of meditation, the level of abstraction reaches a peak where the comprehension of the experience breaks down. This process is reflected in the computer generated images produced by adding the input at the highest levels of the artificial neural layers.
Naturally, at this stage the correlation is mere speculation. But if we can assume that these images are an accurate depiction of the cognitive process of meaningful interpretation of stimuli; it is not so far fetched to assume that our brains work in a similar fashion, but more complex (adding a multitude of other sensory modalities). Thus, some instances of qualia can be captured and in this case, visually represented. And if something can represented it is measurable, and if it is measurable it exists within the world of matter. Sure, it is an analogous leap, we still can’t see what a pure biologically produced experience look/feel/taste like. However, more complex artificial intelligence is on the rise, and soon we might be able to record the same kind of cognitive processing of other artificially created sensory modalities like sound, smell or touch. One day, we might even be able to integrate these with virtual reality technologies – where we can share experiences as if they were our own. If this is a future possibility or pure fiction only time will tell. Either way, what the ability to record first hand experience is telling us, is that consciousness exists here, in this world and not in some far off transcendent dimension. Information is substance. The Word has become flesh.