Practical insight from first metaverse founder

In 2003 Philip Rosedale founded Second Life as a way for people to build, interact – and live – virtually. Since then, he’s developed the platform, worked through numerous technological challenges, and seen a true community come to life. Discover the lessons learned from the first true metaverse and what they reveal about our collective future.

Below is a transcript of this podcast. Some parts have been edited for clarity.

Michael Hainsworth: We’re building two metaverses. One for society and one for the corporate world. But in reality, the metaverse has existed since 2003 when Second Life founder, Philip Rosedale decided he didn't just want to just make another video game. He wanted to create a world where we all could come together. 19 years later, I want to uncover what lessons the creator of the world's first metaverse learned so that we can apply to the social and corporate metaverses that we are building today.

Philip Rosedale: Second Life was unique. I think it was hard to describe at the time. It was probably hard to describe to investors, hard to describe to people that were hearing about it for the first time, so perhaps more of a challenge then than today, but Second Life was a pretty different idea. It came from the dream that I had as a kid not to play a video game, or have any particular kind of experience, but instead to basically simulate the laws of physics inside one or more computers, and then create a blank space that people could build in. This idea of allowing people to collaboratively make things in a world that they were sharing together that had its own laws of physics and was its own universe, if you will, that was what really motivated me, and that was what I created with Second Life.

MH: Tell me about the difference between a lean back experience and a lean forward experience. You didn't want to just make another video game.

PR: Right. Exactly. And I think for lots of good reasons many people that have worked in this space and are working in it today were inspired by video games. I felt like the thing that we needed to do, and I think some of this was the timing when I was born, computers were available to me, but the internet was not as a teenager. For me, the internet was the big transformation. And what I wanted to do was this lean in experience of allowing people to engage with each other, as much as possible, the same way that they do in the real world.

And contrast that to the entertainment, and a lot of even multiplayer game experiences are a lean back; where you are relaxing at the end of the day and don't want to create new relationships with new people. You're just trying to relax a little bit, maybe play a game together. Of course, there are other people on that game, but that's a very different thing than this idea of really recreating reality and then expecting, and allowing, and enabling people to do everything that they do in their waking life in a virtual world.

MH: So how do you do that? It's often referred to as a sandbox environment. You create a series of tools. You give them enough sand and they can make castles.

PR: That's a great way of putting it, a sandbox, precisely. I mean, if you imagine creating a simulated world that has got rules that apply to everybody when they're there, the question is then how do you fairly distribute resources amongst us? I mean, we as humans have an innate sense of fairness and it's very important. We'll only engage in things that seem fair. And a lot of the challenge was creating a sandbox where everybody had access to the same amount of sand, if you will.

And then there's another challenge, I think that I learned more about this as time went by, as people come into that world they're going to engage with each other in every creative and experimental way they can, and some of those are going to create conflict. And so how do you also enable a civil society to exist, again, in much the same way that we see in the real world?

So those were the big pieces and there are big components to it. I mentioned the laws of physics, a common set of rules that everybody understands. I can build. I can move this to the left or right. I can change the color of this thing over here. And then another one is economy. If you think about it much, you realize that everybody's going to want to specialize and engage in trade, just like we do in the real world. In a virtual world to enable that, and this was especially true in 2003, you must have some sort of a mechanism for an economy. And so that led to a lot of work in Second Life and then finally moderation. What are the rules by which people engage with each other? How do they form groups? How do they form the same political and social and economic structures that we form in the real world? So, lots of work there.

MH: Well, tell me more about some of those lessons that we learned as a society from Second Life in its early days, and maybe to a degree, those massively multiplayer online games as well. Things that created their own communities, these lessons that we can now apply to metaverse 2.0 if you were metaverse 1.0. Talk to me about it from that sociological perspective of what we learned.

PR: Gosh, there's so many things. I think one thing we learned was that people, we are social animals. We are successful as human beings mostly because we've come up with these different ways of helping, collaborating with, working together on things. And I think we really saw that in Second Life. People that were coming in immediately formed groups, found friends, and engaged with each other in a way that I didn't suspect. I always thought that some of the people, perhaps even ones like myself that came in, say, with an engineering background would work in there alone in the same way that you might program at your desktop.

And yet when we talked to them after a year that they'd been there, or something, they all said, "What keeps me here is the connections with other people and the specific relationships, collaborative projects, and everything that I have with others." I think one thing we learned is that we're social and this is a message of hope for the world right now because we need each other and we like each other, and we're mostly, most of the time, good to each other. I think we really saw that and continue to see that inside Second Life.

MH: Second Life peaked at a million participants in 2009. It hasn't really grown beyond that level. It's like there's a ceiling to it. What does that tell you about the likely adoption of the metaverse as a second reality for day-to-day life?

PR: Right. So, everybody showed up in Second Life around 2006 and in about 2008 or 2009, Second Life reached a million people using it. And that's about where it is today. It leveled off. It's grown some during COVID. It's grown well over the last few years, as you can imagine, but what I think I've learned is that virtual worlds are not for everyone. And that is still true today. It's still true with VR headsets and it's going to be true for this next version of the metaverse that people are trying to start right now. When I started Second Life, and when I talked about it in 2006, I said, "We're all going to spend some of our lives as avatars, and some of our lives as real people." And 20 years on from that now, or almost 20 years, I think what we've learned is that that's not true for most of us.  

Most of us have an ability to invest in one primary identity, one avatar, if you will. And that avatar is our physical body. And we engage almost entirely with the people that are physically around us in the real world. Now, there are a bunch of different reasons why the people that are in Second Life choose to be there. And there's a very diverse set of reasons that brings them there, but they've all made a very interesting choice. They've decided to make that one representation a digital one. They've decided to in many ways give up or make very secondary the experience of a physical body and instead really invest themselves in a digital avatar. I think that the challenges and the reasons why that happened they're still here today. We still face them and metaverse creators in the future will face them and we'll have to navigate them, as the science gets solved, that I think might get us past that.

MH: Do you think that the architects of the modern metaverse understand that most people don't want to live a double life?

PR: I don't think so. I think that we are jumping into this, frankly, in much the same way as we did in the early 2000s with Second Life, without a real understanding of what it's going to take to get everybody on board. To get, for example, adults that want to go into a business meeting, and maybe meeting new people who are accepting of doing that meeting as an avatar. I think there are some really big challenges to that. I think the companies that are staking claims in the space today are largely less aware of the problems than, for example, we have been at Second Life.

MH: There seems to be, from a non-social perspective and more from a corporate perspective, two avenues that we're going down on the metaverse. We're going down the let's all have a corporate meeting in the metaverse, which some might argue, "Oh my God, this meeting could have been an email. And now it's not only a meeting, but it's a meeting in the metaverse." And then there's the other side of, "Well, we are a sneaker company and we need to make sure that we've staked our claim in the metaverse for sneakers." Tell me about that bifurcation on the corporate side of things because we had a similar land rush in the corporate world on Second Life as well.

PR: Yeah, we did see, exactly, the same thing people coming into Second Life. The companies coming into Second Life did so with great enthusiasm, they wanted to stake out, and in many cases, quite literally a space in this emerging virtual world. And they were early in doing that because we did reach this as we just discussed, we reached this cap of a smaller number of people spending time there than we thought we were going to, so, yeah.  

And there's two different pieces to that that you just mentioned. On the brand side and on the presenting ideas to people, marketing, selling things, explaining who you are as a company, there's a set of challenges that are, I think, more dependent on the size of the audience, and the diversity and expanse of that audience. We've just got to get 100 million, say, adult consumers into virtual worlds before they can become that promise, but I do think the early activities made by brands and companies in many cases are good experiments that are going to teach them a lot. Look at something like IBM and Second Life who tremendously invested in things like corporate brainstorming sessions in the virtual world, and that was amazing to see it in 2006. And I think they understand that space a lot better now than a lot of companies, so I think there's value in doing that.  

As you touched on the idea of doing a company meeting is a whole different set of problems and tickles some of the biggest science problems that we have with virtual worlds, and with virtual reality. People that are, say, going into a business meeting where they're the complex dynamics of, say, a new person in the meeting, your boss in the meeting, a lot going on, they are going to demand things like perfect spatial audio and nonverbal cues being communicated with low latency. Those things are incredibly important, and across the board we're just not there yet with delivering that to an experience. I think that's why there's a little bit of pushback on some of the presentations we've seen, say, from Facebook recently about putting on a VR headset and going to a meeting. There's a lot of challenges with that. I think people broadly get that we may be farther away from that future than we think.

MH: What are some of those other big science problems that need to be addressed if we're going to see a critical mass adoption of the metaverse?

PR: Well, the most practical one and the one that I think most everybody would cite is just comfort. To put a headset on your face today and have an experience that lasts more than 30 minutes is almost impossible for most people. The weight of the device on the bridge of your nose is too high right now. It's about 500 grams and it needs to be about, say, 200 grams, the weight of a pair of ski goggles, or something like that for you to be comfortable having, say, an hour-long meeting, so that's a practical problem that can be solved by making these devices lighter.

A more subtle problem, and one that we explored richly at High Fidelity, where we built a whole open-source VR headset enabled virtual world is that different people have a very different willingness to wear these headsets. Another way of putting it is right now, putting on a VR headset is a lot like putting on a blindfold, and your willingness to put a VR headset on, therefore, is a lot like your willingness to blindfold yourself in a room where there may be other people. Everybody's got a little bit of discomfort about doing that, but some people have a lot more discomfort than others. What that ends up doing is creating an early VR headset world that is not a diverse and inclusive set of people. As somebody trying to build a social world where people could meet anybody, they wanted to there that was a real problem, and continues to be.

MH: There are some hardware limitations that play a role in the metaverse adoption. I wonder if it's not just beyond the weight. I've worn a Meta Quest 2 headset and watched a Netflix movie, and about a good 30 minutes in I'm finding myself holding it up with my hands or lying on my back so that the weight is more evenly distributed.

PR: Exactly.

MH: I wonder if maybe because VR is expected to be anywhere from 10 to 20% of the overall market, whereas augmented reality is expected to be the lion's share of revenue in the future, does this tell us that the metaverse isn't a VR experience, it's an AR experience?

PR: I think it's a little hard to say. There's so much at play there. One thing is that the AR experience is even harder from a science perspective to get finished because we must have lighter devices. We must be able to see people's eyes through those devices while they're seeing things as a result of the projections, and that's really difficult. We haven't folded up optics yet and made them light enough to have good AR glasses, but I do agree that once we get to that, which I think is going to be, say, 10 years compared to maybe five years for some of the virtual world experiences we're going to see grow.  

Once we get to those devices, I would tend to agree with you that some amount of pass-through, some feeling of safety, awareness of the real world around you may be a more dominant usage mode. Although, I would say having worked on it for a long time that fully immersive VR worlds, once we get the problems solved for them, are going to be unbelievably compelling as well. Maybe I say that with a bias being the Second Life guy, but I do like the idea of complete immersion.

MH: Well, have you seen the movie, The Kingsman where they all sit around at the boardroom table?

PR: Sure.

MH: And only one of them is there and the rest are virtual avatars that are holograms more than anything else.

The Kingsman Movie:
Lancelot was an outstanding agent, and a true Kingsman. He will be sorely missed. To Lancelot. To Lancelot.

MH: That feels like a more effective metaverse in the long run, but if that's a decade away, tell me about the next five years because it sounds like the idea of not wanting to be blindfolded and the fears associated with putting on a VR headset, weight issues aside, is that a generational thing? Is that the boomers don't want to do that because, what kind of crazy nonsense is that, but the kids today they'd be into it?  

PR: That's an interesting one for me. I'm fortunate enough to have four kids all of whom are Generation Z kids range right now from 15 to 22, two boys and two girls, in fact, so I've really gotten a diverse set of inputs on this stuff. And what I've seen is the opposite, which is fascinating. That is the boomers, and the Generation X and the people at work today that are making decisions, and in many cases managing or building projects like these presume that the Generation Z kids are going to have a greater willingness to be avatars.  

I've actually found the reverse to be a little bit true. Kids are so immersed today in a lot of information, much of which is performative or misinformation or disinformation that what I've seen when I've talked to Generation Z participants is a greater desire for this video nonverbal information. When I've watched kids do live gatherings, they are typically doing them with audio, good audio often using tools like Discord, for example, to get together and talk to each other, or they're actually using things like FaceTime, where they're just demanding that video authenticity from each other, and really demanding a lot of presence. I think that we may have, funnily enough, we may have a bit of a steeper hill to get over with Generation Z than with Generation, say, X.

MH: That's interesting. I would have assumed that the kids would have been more dialed into the world of technology and more willing to adopt new technologies than older generations.  

PR: They're using it asynchronously. Kids are using things like filters in TikTok or Instagram to create content but remember that that content is asynchronous. It's typically posted and consumed later by an audience, but it's the live case that I'm talking about. And it's the live case that's going to drive VR and AR. That Kingsman meeting you mentioned is the fantasy that we want to get to, but there's still a lot of challenges there. Is there a little drone floating in front of each one of those person's faces that's doing that crazy volumetric capture that we saw in the film? How are we going to get that out there? And let me tell you that hanging a camera in front of somebody's face like you see in the movies that's not going to happen for people going to business meetings.

MH: So what are some of the other issues that the corporate world when building the metaverse will have to address to ensure there is the adoption, not just because they're throwing a ton of money at it, but because it is an effective means of communication? I like to say we wouldn't be going down this path if there wasn't a value-add at the end of the day, how do we ensure there's value-add and how do we get there?  

PR: Well, some of the corporate uses of VR and AR, are already showing a lot of opportunity. When there's an information rich display, like examining a 3D work in progress, or looking at the engine of an airplane, or something like that there's already a great business case for using both VR and AR devices to do that, and we're seeing a lot of that. However, I think what you're talking about is really this bigger future of everybody being in social spaces, or workspaces, or larger meetings, or things like that. Some of the other things that need to be captured, first, larger meetings.  

Most of the things that we want to do as human beings socially involve a lot of other people. When you're in a social environment whether it's, live music, or a large meeting, a town hall meeting, something like that, a sporting event, you are surrounded by tons of other people and believe me, and I've worked on it for a decade now with High Fidelity, getting a ton of people digitally in a room is both critically important and very hard. There's a lot of cloud streaming, edge, endpoint type challenges associated with providing high bandwidth and low latency to a lot of people at the same time, that both the corporations and the carriers and service providers that are connecting them to people are going to have to figure out. Scale is one really big thing.

And then another thing is this issue of moderation. How do we all get along in a virtual space? We've seen moderation on asynchronous things like Instagram, and even those things, they've been done very centrally, and they've been done in a way that I think many would argue are totally inadequate for human needs. As we move into live spaces where there are thousands of people together at the same time the centralized after the fact moderation isn't, in my opinion, going to work. We're going to need to build tools for what I always like to call fair fighting, a term I borrowed from a noted thinker in this space, Danielle Allen who writes about politics and civic spaces.  

Fair fighting is the ground rules of the space, it should enable people, as much as possible, to resolve their own disputes. I think that is a very complex topic, and one that some of the people listening have probably explored, or seen a little bit with things like how do we put bubbles around each other to protect ourselves in VR, and problems like that? So there's a big ‘how do we all get along’ challenge, which I personally find very inspiring to work on and I think it’s doable, but it's like everything else, it's going to be a few years' work.

MH: Yeah, tell me about that because I know from my own experience with Meta Quest's Horizons, that they recently added the ability to put a four foot barrier between you and someone who was not a friend of yours to avoid the trolls coming right up to your virtual face and literally and metaphorically getting up in on your grill type of thing. And just the ability to click a button and go, "That person doesn't exist in my virtual world anymore." Seems like an easy solve.  

PR: But let's just talk about that for a second because I can show you where you get into trouble there. Clicking a button and saying, "That person doesn't exist in my world anymore," is good. It's an immediate safe zone. You're immediately safe, and that's great. And the reason for that is that having somebody within reach of you basically is called the peripersonal space in neurobiology. And we all as living things have an exquisite sense of that peripersonal space. So just like you said, "When somebody gets up in your face," it means within arms’ reach of you and it can be a very alarming and upsetting thing, whether you're virtual or real. So that's important, but just disappearing somebody isn't necessarily, or across a lot of people going to be the right answer.  

And the reason for that is if there's three of us standing here and I make you disappear, and the other person makes me disappear, we've got a really bizarre, almost psychedelic hallucinogenic situation where we're each experiencing a different reality that we've customized with our bubbles, and we know where that went with things like Facebook groups. We know that allowing people to filter out their own experience creates this weird dystopia where everybody is living in a different world. And we don't want that to happen. The reason why Second Life causes people to behave quite well, almost uniformly to each other, is not that they're able to do that but that they're able to abide by the ground rules of the space that they're in and all experience the same thing.

MH: Let's bring this back to the corporate meeting with the boss in the metaverse, particularly, work from home environment. First, do you feel like the corporate metaverse is getting a lot of traction because COVID happened, and that it told us that we could work from anywhere?  

PR: Well, COVID was the starting gun. I would say broadly, I think of the metaverse idea as being the 2D to 3D transition, but also the idea of a live internet. And I think COVID suddenly said, "Hey, all of these things that you used to do live in the real world, you're not going to do anymore. So, you got to find online alternatives." And so, yes, it's been absolutely fascinating, but what have we learned from COVID? Well, we've learned that video is important and it's very difficult to get beyond it as yet.

We've also learned that video communication is inadequate, particularly for large groups. It's extremely fatiguing to participate in a group meeting. You can't hear people. That's what we work on with spatial audio at High Fidelity. You can't hear people that talk at the same time, and you don't get, perhaps most importantly, you don't get all this body language and gaze signaling that we just have come to, yeah, sometimes we forget we're doing it all the time. And COVID really showed us what a hard situation that is, and how much more work there is to be done before we can have, say, a big company get together online.

MH: So it sounds like there's a hardware solution to that as well with the next generation VR headsets that will have inward facing cameras that will be able to replicate our body language.  

PR: I think so. I would note, though, that in addition to those inward cameras seeing our faces, and by the way, they really need to see our mouths. We know what happened in COVID when we covered our mouths, it was surprisingly upsetting, which is fascinating in itself. I found that part particularly interesting, but we also have these critical pieces of body language, like how my shoulders are turned.

Am I turning my shoulders to let the person that just walked up join the conversation, or am I staying pointing at the person I'm talking to give that subtle stay away I'm busy now signal? We can't convey that in virtual reality yet because we don't know where a person's shoulders are. Similarly, leaning forward like we talked about earlier and leaning back is a very, very important piece of signaling. And without knowing where your hips are, we can't do that with your avatars. And, of course, our avatars don't have hips and they don't even have legs some of them. You're not able to convey some of these critical pieces of nonverbal cuing.

MH: That's a very interesting point. When Mark Zuckerberg showed off the latest iteration of his avatar, one of the big complaints of it was that it looked like 2010's Nintendo Wii. There was nothing below the belt. It was all very cartoon-like. Do you see that as an impediment, this cartoon-like nature of the avatar as we know it today?

PR: I think that the rendering and the graphic detail and the textures and whatnot and how cool my hair looks, or whatever, that's going to matter, but I actually think the nonverbal cues matter more. In other words, my guess is that if we had an avatar that looked like those Mark Zuckerberg avatars, we've all been seeing that moved its face and leaned and moved its body just exactly as his did. In fact, so well that, for example, if Mark Zuckerberg walked into a room and you knew him and worked with him every day wearing an avatar that you didn't recognize and didn't speak yet, but just moved his body, you'd say, "I think that's Mark." And I think that technical challenge conveying those nonverbal cues well enough to identify a friend immediately is a big deal that we've got to get solved first. I think that will be more impactful than improvements to the render quality.

MH: So when it comes to your advice to the telecom companies, the CSPs who are providing the backbone for the technology behind the metaverse, you've brought this up a couple of times already, graphic fidelity is all well and good, but it sounds like the ultra-low latency that comes with 5G, and related technologies, that's of a greater priority.  

PR: Exactly. When I first heard about 5G, I was excited not because of capacity, but because of latency. Human beings when they communicate together, move their bodies in synchrony, for example. If you delay hearing and seeing somebody by more than about two-tenths of a second, by about 180 milliseconds, you stop liking each other. You stop being able to get in sync with somebody, as we say sometimes, literally. Latency is critical and the good news is if you use 5G, and if you use the right edge deployment strategy of this technology, you can get to an experience where everybody in the world can be communicating with each other with about 100 milliseconds of latency. There is really a big adventure and a huge opportunity in creating 100 millisecond latency between any two people in the world.

MH: And in addition to 5G, I suppose the other critical infrastructure technology behind that is the near edge cloud.  

PR: Right. And there, there's all kinds of stuff, cloud streaming. There are so many experiments going on. I love this stuff. I'm not even sure what's going to win out as a strategy, but there's pushing the rendering to the edge of the cloud streaming to the devices. I do think there's a lot of merit to that. It's funny that we've seen so many misfires with it in gaming, but I think that it's a big idea and a big deal just deploying in such a way that you really do find those nearest endpoints. And you do dynamic routing, maybe peer-to-peer, point-to-point when you need to. There's a lot of opportunity there.

MH: We've talked about a lot of hurdles that need to be overcome for mass adoption of the metaverse. Are you confident that we will see that critical mass?  

PR: Well, I think we're going to see bits and pieces get done in different ways. Let me say I'm profoundly an optimist. I'm not yet fully confident because these are science problems. They are problems which are not problems of scaling, or deployment. They are problems of basic R&D. We don't know yet how to make that avatar where when your avatar walks into a room, I know it's you. I do think we'll get there, but I think there are unanswered tech questions to getting there. That said, I think there are going to be areas like let's take education, for example, going to school with a group of people and being face-to-face with a teacher, being able to type and take notes while you're doing it, which by the way, is the big problem with existing technology, that is going to unlock opportunity, and is going to be minimally acceptable to people, and have an enormous impact as it gets figured out.

<< Go to previous episode