What does a collection of stories about isolation and social distancing look like? What do we gain when we experience those stories from a cosmic perspective?
Introduction
The overview effect is a cognitive shift observed in astronauts who see the entire Earth from space. There is something about experiencing the whole planet from a cosmic perspective that strengthens the connection to humanity, life, and peace.
In the current climate of social distancing and isolation, can we use this inherently optimistic cognitive shift to help build empathy and connect to others through a shared experience.
Overview is a web-based meditation experience that explores its namesake effect in the context of the global COVID-19 crisis, specifically through the lens of stories about the day-to-day isolation experience.
Context
The original version of this project focused on an immersive VR experience to explore the overview effect in an Earth-bound audience.
NYU going remote created logistic problems in that I lost access to VR equipment, but more so it highlighted how inaccessible the original project was. My pivot to a web-based project was both pragmatic but also ideological, since I wanted to share the stories of as many people as possible, not just those with fancy headsets.
Much of the research I had done – into perspective and scale in VR, shaders, what experiencing the world from above felt like – still applied on a screen, though to lesser degrees.
The original context of the piece was experience-driven: what happens when we experience the world from space. Influences included VR pieces about scale and perspective and media about the overview effect. The context in which the project has landed though is a story-driven one; the most compelling part is listening to the stories of others and building a world-scale perspective of personal isolation life.
Technical Overview
The project is built in Unity and runs on the Unity WebGL engine. The WebGL application talks to both a browser application that handles audio recording, and an Apache server which stores the application and message data.
The Earth is rendered with a custom shader that uses a noisy UV map to sample a medium-resolution texture of the globe (and clouds). This results in a globe that looks decently realistic from afar but becomes painterly stylistic when close. This was my solution to avoid a pixelated experience without requiring hundreds of gigabytes of textures.
Here is a longer post on some of the graphics technicalities
ITP Thesis Week Presentation
Transcript of the final presentation during ITP Thesis week 2020.
The Overview Effect
The overview effect is a cognitive shift observed in astronauts who see the entire Earth from space. There is something about experiencing the whole planet from a cosmic perspective that strengthens the connection to humanity, life, and peace. In short, that is the premise of my original thesis plan.
In light of… [gesture vaguely at everything] recent events, I’ve added the following addendum:
In the current climate of social distancing and isolation, can we use this inherently optimistic cognitive shift to help build empathy and connect to others through a shared experience.
Project Description
Overview is a web-based meditation experience exploring its namesake effect in the context of the COVID-19 pandemic, specifically through the lens of stories about the day-to-day isolation experience.
Visitors are presented a bare-bones interface to first listen to the isolation stories of others, then optionally leave their own.
Goals and Intentions
Before Covid struck, I would have stood here and spoken in abstract about my fascination with space, dreaming of the stars, the ideology of a borderless world, and the strange little fact that the first thing any astronaut does in space is look back at home.
The global pandemic has shifted my perspective a little, or at least grounded these ideals in a more concrete situation. And so I have shifted my project from one about the unfamiliar isolation of astronauts in space to the now familiar, everyday isolation of the Covid-19 lockdowns.
Overview is a reflection on isolation, a platform to broadcast your own story and listen to others’.
Or at least, it tries to be. In its current form it’s very sparsely populated, and the stories of 6 people are, though interesting, not exactly a compendium.
On a technical level my goal was to get this prototype platform to work, which it mostly does.
On a creative level the goal is to collate different experiences and glean whether there is some overarching whole that is greater than the sum of its parts; a sense of emotional perspective that connects you, the listener, to others experiencing isolation; whether in similar ways or not.
And of course, if it’s something people respond well to then that would be fantastic.
So, to which people am I referring.
In a strange way the pandemic has helped me solidify my idea of a target audience. Pre covid-19, it was sort of an unresolved question, am I targeting people who want to go to space but can’t, or a generic stargazing community, or what? And if so am I not just preaching to the choir?
The pandemic has presented a very large group of people who are experiencing isolation in one way or another. And that’s who this is for. Whether you’re curious about the experience of others, miss hearing people’s voices, or just have something to get off your chest, this could be for you.
Context
For over three quarters of the thesis process, including the entire research period, I was collecting resources under the assumption I would be creating a high fidelity VR space experience. That was fun and interesting to me, and in many ways directly useless for what the project has ended up as. Indirectly, though, the musings of Carl Sagan and the Apollo astronauts, the work done by various space VR applications, and investigations into the Overview effect have informed both the aesthetics and mechanics of where things have ended up.
In terms of where things have ended up, there are many projects I’ve discovered along the way with similar intentions or implementations. Some different in context, such as the paper planes web app that lets you throw a message across the world, and some similar, such as quarantinestories.today, which asks the simple question “what do you see outside your window today”. These experiences have helped shape the questions I ask and the way I ask them.
Implementation
Enough preamble! What is it that I have actually done? In short, the pandemic and going remote shifted this project from a graphics-heavy Virtual Reality simulation to a web-based experience.
Much of my attention throughout the process was focused on experimenting with how to successfully render the Earth. The following is a general timeline of some of the experiments, earlier tests above to where I’ve ended up below. This process involved exploring scale, shaders, renderers, and interactivity.
The journey involved both learning and adaptation, as some of the processes I relied on earlier when using high powered computing – such as volumetric scattering – are infeasible for web-based graphics.
Early Earth render in Blender3D, exploring cloud cover,
Stylised Earth rendered in Realtime in Blender, exploring procedural clouds and atmospheric scattering.
VR Interaction experiment in Unity, exploring newly-released Oculus Quest hand tracking
Final web-based mouse interaction rendered using Unity and WebGL
More on Web Graphics
Here is a separate post on some of the graphics work on the project
One of the challenges I had when switching to the web was vastly reducing the size of the textures I could use. For both budget and bandwidth reasons, I had to stick to a free medium-resolution texture of the earth. This works fine for a ‘full’ view of the globe, but as the viewer gets close the pixilation becomes very apparent. My solution was to, instead of sampling that texture with a standard UV map, apply some noise to the UV map before sampling. Combined with a few other tricks this creates a sort of painterly effect, hiding the blurry pixilation behind stylisation. From afar the earth looks realistically rendered, and as you get closer it becomes more of a stylised aesthetic.
Reflection & Next Steps
At the end of it all I have a prototype that outlines the experience and aesthetic that I envision. I think my technical goal has been somewhat achieved and the learning journey has certainly been fulfilling.
Although there have been some positive responses, I am not yet sure how people will in general respond, and thus not yet sure how successful this prototype is. I anticipate that there is a critical mass of about 20 or 30 before a real sense of perspective sets in. This is a bit of speculation, which I’m trying to avoid before more voices are added.
One of the immediate next steps I have is getting a few more eyes (and voices) on the experience. If you have a moment after all these presentations, please go to bit.ly/overview-broadcast to share your story! Obviously things will continue to change in the near future so I appreciate your patience with bugs, crashes, and interesting creative choices.
Final Thoughts
It feels strange to end on such a droll technical note, so instead I’ll leave you with something that has been with me throughout this process, a guiding light of sorts.
Michael Collins, who stayed behind in the lunar module while Armstrong and Aldrin bounced around on the moon, is suggested to have been, for a brief moment, the most isolated person in the history of the world. As the lunar module orbited behind the moon he was entirely cut off from even radio connection to humanity. Upon returning, he said
“The thing that really surprised me was that Earth projected an air of fragility. And why, I don’t know. I don’t know to this day. I had a feeling it’s tiny, it’s beautiful, it’s home, and it’s fragile.”
