Synthetic Sensing: On Experiencing Time and Space Without Leaving the Room

The second of three speculative design concepts from VIS 161, spring 2022. The term "synthetic sensing" is mine. The ideas came first; the rationale followed, as it often does.

The premise was straightforward enough to sketch in a few sentences, which is usually a sign that the idea is either very simple or very large. This one turned out to be the latter.

Synthetic sensing, as I conceived it, is the use of AR and VR technology to extend human perception beyond its current physical constraints — not as entertainment, not as escapism, but as a genuine sensory and intellectual interface with time and space. The system I was imagining had two axes, and they pulled in very different directions while sharing the same underlying logic.

The first axis was time. The idea was simple: what if you could inhabit a historical moment rather than simply read about it? Not a passive recreation — not a documentary or a diorama — but an interactive environment in which you could move through a period, engage with its architecture, its objects, its texture. The educational implications are obvious and have been gestured at before, but what interested me more was the documentary potential. If the technology existed to render current events as immersive environments in real time, history could be stored experientially rather than just textually. Future generations would not read about 2022; they would be able to walk through it. The question of who controls that archive — who decides which moments get rendered, which get preserved, which get access — is one the concept raises without resolving.

The second axis was space. The premise here was that synthetic sensing could simulate other planets or deep space environments with enough fidelity to be scientifically useful — allowing researchers to conduct experiments without the resource cost of physical space travel, or students to explore environments that are otherwise entirely inaccessible. The more speculative end of this axis was the one I found most interesting: if we ever reach a point of contact with other planetary beings, a synthetic sensing interface could become the medium through which that communication happens. A shared environment, neither here nor there, constructed from the overlap of two entirely different sensory and cognitive systems trying to find common ground. Even short of that — astronauts using the system to interact with people on Earth in something closer to presence than a video call — the spatial possibilities were generative.

The multi-setting nature of the concept was deliberate. A museum, a classroom, a research lab, a space agency, a communication terminal — the system would function differently in each context, shaped by who was using it and toward what end. That flexibility felt important. The worst version of a technology like this is one designed for a single use case that then gets retrofitted into everything else. The better version understands from the start that synthetic sensing is an infrastructure, not a product — a new layer of perceptual possibility that different institutions and individuals would inhabit in entirely different ways.

I came up with both axes more or less simultaneously, without a particular logic connecting them beyond the intuition that time and space are the two dimensions of human experience most stubbornly resistant to direct access — and therefore the most interesting candidates for synthetic extension. Whether that intuition holds up to scrutiny is a separate question. In a speculative design class, the point is to follow the idea far enough to see what it reveals, not to fully resolve it.

What it revealed, in this case, was a technology that is simultaneously an educational tool, a scientific instrument, a documentary archive, and a potential diplomatic interface. That range is either the concept's strength or its weakness, depending on how seriously you take the word "synthetic" — whether you read it as artificial and therefore suspect, or as constructed and therefore intentional.

I meant it as the latter.