The metaverse didn’t die. It moved inside Microsoft Flight Simulator 2024 | Jorg Neumann interview

The scale of the ambition of Microsoft Flight Simulator 2024 is pretty astounding. Made with 800 game developers over four years, the title has a seriously impressive set of numbers.

I got a big download of the ambition at a preview event at the Grand Canyon, where the game makers flew us over the canyon and compared it to the simulation. The flight sim of all flight sims comes out on November 19 on the PC, Xbox Series X/S and GamePass on day one.

One of the most interesting feats is that Microsoft shifted the game’s computing from your local PC to the cloud, said Jorg Neumann, head of Microsoft Flight Simulator, in an interview. The massive amounts of data are computed in the internet-connected data centers and then streaming in real-time to the user’s machine, where the simulation is visualized onscreen.

In the 2020 version, Microsoft had a hybrid structure that streamed data from the cloud and also used the local compute resources on the user’s own machine. That resulted in downloads to your PC of up to half a terabyte, far more than the 23 gigabytes for this year’s game.

Microsoft Flight Simulator 2024 is also bringing massive enhancements to the simulated Earth by increasing the detail of its virtual environment by a factor of 4,000. The team built a “digital twin” of the Earth, much like would-be metaverse companies want to do. But this world was built with realistic physics and a huge level of accuracy. It has systems for all things that can affect flight, from ground activity to extreme weather, fuel and cargo, and turbulence. The hot air balloons in the game are simulated across 6,400 surfaces giving a realistic reaction to heat density — when you turn on the heater, the air will heat up, and it’s going to inflate the massive balloon.

The Earth in the flight simulation is really as close to a digital twin of the real planet as has ever been built, Neumann said. I heard a lot about digital twins from Nvidia — it supplies the chips to run simulations that let BMW build a digital twin factory to perfect the design before it builds the factory in real life. And Nvidia ambitiously is building Earth-2, a simulation of the entire world so accurate that it may one day be used to predict climate change for decades to come.

Overhyped and then hated, the metaverse went into hiding, and it’s lurking inside digital twins like BMW factories and Microsoft Flight Simulator 2024. In fact, Neumann said the company got a lot of the data for the photogrammetry of its planet-sized simulation from other enterprises that are digitizing the Earth.

Enhanced digital elevation maps use more than 100,000 square kilometers of countryside photogrammetry to enable visually stunning digital twin experiences. More than 150 airports, 2,000 glider airports, 10,000 heliports, 2,000 points of interest, and 900 oil rigs have been carefully hand-crafted while a procedural system generates all 40,000 airports, 80,000 helipads, 1.5 billion buildings, and nearly 3 trillion trees our planet.

Since the game journalists outnumbered the flight sim leaders, I paired up with Samuel Stone of Den of Geeks to talk with Neumann. Here’s an edited transcript of our interview.

GamesBeat: Did this have to start a long time ago in order to get that plane in the game and plan this whole event?

Jorg Neumann: You mean the real-world thing? No, actually not. The CEO of Cirrus, his name is Zean Nielsen. I call him an innovator. He wants to revolutionize how planes are perceived. Most people think of planes as scary things. They’re too far away from their lives. When you look at Cirrus’s commercials, as you’re driving up to an airplane–have you seen these things? Mom and Dad come out, a boy and a girl, and a dog. Then it says, “Here’s your weekend getaway private jet.” Okay, cool? The tone is a very playful, friendly tone. He’s a big believer in Flight Sim.

GamesBeat: I’m astounded that our pilot let Charlie take over.

Neumann: Because they want to show that it’s not scary. In very many ways, it’s like driving a car. It has all these security features. It’s super stable. You flew it. You saw it. It’s super reactive. You really feel in control.

He looks at the world of aviation through the lens of, we need to get more people comfortable with aviation. It has a lot to do with history, specifically in this country. Aviation was a family tradition. Often it was people from the Greatest Generation coming back from the war, becoming crop dusters and things like that. Having private planes, getting their grandkids into private planes, that sparked them to become pilots. That’s fading a little. Getting people back into the dream of aviation and flying is their thing.

I get phone calls from literally every manufacturer on the planet. “You have to help us with recruitment. There aren’t enough pilots.” The commercial aviation space is lacking 800,000. We know that. There’s not enough transport pilots, not enough passenger pilots. There’s a crisis coming because they’re all aging out. The Level D simulators cost $40 million. There are very few of them. They’re all looking for ways to get people into aviation faster. Then they look at us with 15 million people playing. The quality is good. This is the best recruiting tool ever. They support us however they can. Our relationship with the manufacturers, typically–if I ask them for something, they say, “How else can we help you?”

Samuel Stone: When Flight Sim came back in 2020 it came back bigger than ever, with all of that third-party aviation support. Taking all of that data, all of that feedback, how did that inform the direction you wanted to take 2024?

Neumann: It absolutely informed it. We almost completely reversed the typical way of making a game. Typically you sit there with a bunch of designers in the room and decide stuff. In this case we said, “What do people want? What are their problems? What are their needs?” Our design priorities came from the community. We have our own ideas. Nobody said, “Jorg, put giraffes in the game.” That’s a me thing. But all the serious fundamental stuff came from consumer needs. I feel great about that.

The whole process is healthier, I think. You can easily respond to people, because you already have common ground. They’ve told you what the problems are. We can propose solutions. They give us feedback on those solutions. As we implement we go through with what they actually need. I’ve been making games for 30 years. I’ve never done it this way, and it’s better. I’d never go back.

GamesBeat: I was curious about how you came to embrace digital twins. Nvidia wants to build something to predict climate change in the years to come. They need meter-level accuracy of the earth in order to do that, so they have to build a digital twin of everything. They have their own purposes, but how did you become convinced that this would lead to a better game?

Neumann: The impetus for starting Flight Sim in the first place, back in 2016 when I kicked this off with Phil–I had worked on something called World Explorer on HoloLens. Nobody ever played that because HoloLens is really expensive. But the experience was great. We did Rome. There’s a digital twin of Rome. For that we needed photogrammetry of the city. You could land in the Colosseum and those sorts of things. I was also working on Machu Picchu. We didn’t have a scan for that. It’s complicated. Everything is rounded. A complicated space.

We got to a point where we got those places right. San Francisco was another one. We did about 12 places around the earth. The real impetus was, can we do this on a worldwide level? I remember getting the Seattle scan. I stuck it into the engine. We got a Cessna 172 from Flight Sim 10 and jammed it in. It felt great. I showed it to Phil. We flew over our offices in Redmond. He said, “Why are you showing me this video?” I said, “It’s not a video.” I turned the plane. Yep, it’s real. That showed us it was possible.

The next place we tried was actually the Grand Canyon. We had problems with the digital elevation map. There was popping with the shadows everywhere. The resolution wasn’t good enough. But the reason why I thought Flight Sim was the right vessel for that idea, at the core of it all Flight Sim was always a full representation of the earth. Even if it was just a rectangle and one tower representing Chicago, it wanted to be that.

For any kind of software, when you ask that question with a digital twin–it needs a purpose. A consumer need has to be fulfilled. We have a consumer need. Flight simmers want this. I’m building this digital twin for the flight simmers. Does that mean it’s limited to flight simming? No. But there’s always a need. Now that they can land a helicopter anywhere and walk around, we needed to make it look at least as good as a first-person shooter or something. How do we do that? Again, there’s a need that drives innovation forward.

GamesBeat: It’s interesting that you’re finding more accurate information than anyone else.

Neumann: We’re pretty relentless at it. When you have 15 million people playing something, that’s a pretty big motivator. We’ll just keep chipping at it.

Stone: Flight Sim isn’t a sprint. It’s a marathon. There’s so much post-release support and content. How is it–not just mapping out what it will be at launch, but what’s it like looking at the future and that post-launch support?

Neumann: We have most of the world stuff. I just got an email that said Tallinn and Riga are ready. I had to set that up two years ago. We had to get a bunch of permissions. We had to convince a flyer to go close to a war zone. It was complicated. Do I know when this stuff will ship? No. It’s the world. The world has its own clock. They don’t wait for Flight Sim. A lot of the release plan has to do with data availability.

Obviously we listen to the community. What does the community say? “Stop doing North America and Europe. What about Brazil?” We’ve been talking to the ATC controller over Sao Paulo for two years, because they control the airspace. If they don’t want to let you fly you won’t fly there. We convinced them. We showed them what we’re doing, why it’s good for society. At some point they say, “Okay, here are your permissions.” We flew Sao Paulo three or four months. It took a month, because it’s huge. Then we got the data. Now we have to process the data, edit the data. At some point we’ll do a world update for Brazil.

That’s how you can think about it. I can’t just snap my fingers and say, “Give me Asia!” I have to talk to a whole bunch of people. And I am talking to them, a shocking number of people who have nothing to do with gaming at all.

GamesBeat: Once you have satellite data, don’t you have all the data you need?

Neumann: The satellite data is a sort of middle ground. With the cities–think about an airplane that flies pretty low over the houses. You have every angle on every house as the airplane passes by. They fly in strips. This is an airplane that flies higher. You get fewer samples. What happens is, some of the back sides, especially depending on the time of day–the back sides aren’t lit well. We don’t have enough data to show what the back wall of something like this looks like. Satellites, given how high they are–I showed it earlier, the Kilimanjaro thing. Kilimanjaro is a nice shape for that. The moment you have overhangs, it’s not so good. Cities, you can’t do that at all.

That’s why I specifically mentioned Kazakhstan. There’s no way to get into Kazakhstan. Won’t happen for years and years. Too much geopolitical stuff in the way. But people might want to fly there. Flight Sim is free. Open skies. For that I’d go with satellite data. Sometimes you just need to find the right satellite. They fly in these weird patterns.

GamesBeat: So the default is satellite data, but then you fill that in with more detail.

Neumann: Exactly. The satellite data is not strong 3D data. There’s some 3D data, but it’s not very good.

Stone: You talked about consumer need and desire. One of the more ambitious things about 2024 is the addition of all these activities and the career mode. How did you find a balance between gamification and the grounded authenticity that Flight Sim is known for?

Neumann: It’s difficult. Honestly, I’ll wait for the judgment of the court, of the people. The people will be right. All we can do is engage. For example, I lived in Seattle. The Coast Guard is close. They called me and said, “Hey, Flight Sim is awesome. Can we deploy it in our stations? We want people to train up.” Why not? “If you want to do Coast Guard missions, let us know.” We took them on. Why not? Free help. Same with the big center for firefighting in Europe. They helped us.

Some other things we probably didn’t spend as much time on, like VIP. The Asobo guys know a pilot, a VIP pilot. All he does is fly business jets around for famous rich people. But is that all that different from flying alone? Not really. We spent most of our time on the very on-the-ground things. Agricultural aviation, those types of things. Did we get it perfect? I don’t know. We’ll see. And we’ll make it better. If we get feedback and see that we didn’t get it quite right, that’s okay. We’re here to learn.

GamesBeat: The more you put things on the ground, is it conceivable that you could get help from Ubisoft’s developers making Paris, or Call of Duty making Washington D.C.?

Neumann: It goes the other way. I get called a fair bit now that we’ve merged with Activision Blizzard. I get a lot of phone calls from people who want to sim New York. We do have a brand new model of New York, it turns out. But a lot of games don’t really want real-world scale. They want a spatially optimized version. Otherwise it’s too big. It’s boring. You don’t want that. That’s where this particular team–we said, “Here’s our alpha model of New York.” Then they can take this section and that section and glue it all together. It’s up to them. We can give them the data.

In the countryside, we’re way ahead of everyone else. We have so many connections now. I used to do this by myself. The first two years it was just me doing this. That wasn’t a good solution. Now there are four guys doing nothing but talking to governments, geographical institutes, drilling companies. Anybody who does anything around the world, we try to get their data and fit it in there. It’s getting better all the time. It’s not perfect. There are still areas that are almost terra incognita, where we barely have something reasonable. All we can do is try hard. Go to Zimbabwe and try to get good data. But that’s the reality.

We need Jordan. If you saw Dune, that’s all Jordan. It’s pretty nice. I had a lot of fun working on that. There are awesome rock formations in that area. You can’t get that from the satellite data at all. It just looks like a pancake. I’m determined to fly planes over Jordan. I talked to Patrice Vermette, the creative director of the Dune movie. I met him in Budapest. We filmed a little vignette there, doing this whole thing with the ornithopter. I told him the story. I want to get this stuff. He says, “Okay, I know all the people in the Jordanian government.” Now I’m writing the emails. “Hey, I’m Jorg, I work on Flight Sim. I’d like to get this and this.” You have to engage with people. It’s just the way it is.

Stone: 2020, in addition to the PC, was released for Xbox Series X|S. Having taken that experience–I play Flight Sim with an Xbox controller. What is it like taking that and improving on that experience for 2024?

Neumann: First of all, I’d say Flight Sim was pretty good on Xbox. The key binding–you need to talk to David about this. The key binding thing, there are so many functions. Getting that right, it’s like sign language. It’s a totally different alphabet. He’s the spearhead on that. I use it, but I’m not the designer. He is. He knows everything. I’d encourage you to talk to him.

GamesBeat: Where did you get the confidence to conclude that all cloud computing would work this time, versus part local and part cloud like last time?

Neumann: Sometimes you just have to believe. Even when 2020 came out–I said, “Hey, I found two petabytes of data.” People said, “Cool, and…?” “We’re gonna stream all that!” “Come again?” 2016, 2017, when we started, the internet ping time in, say, western Australia was horrible. There was no way you could stream this game. Then more and more data centers were built. As we were working on this product, they built data centers all over the place. That enabled the product. The infrastructure of the world caught up, and thank God they did all that. Otherwise, I don’t know.

That just continued. You’ve seen the data. Everyone reads tech news. You see the explosion of where this is all going. We got lucky. Sometimes it takes that.

GamesBeat: I remember the San Mateo Bridge by my house. Half of it popped in, and then the other half. Oh, there’s the rest of it, from the cloud.

Neumann: I believe in technology making human life better. I’ve grown up like this. I believe we’ll keep investing in making that better. It makes elements of our lives better. It has some downsides, no doubt. But I think this product will exist because of an overall need. I do think we’re a helpful product. I see what people are trying to do with it. Greenpeace uses this. Amnesty International uses this. Local governments trying to figure out how to make a train line disturb as few people as possible, they use our stuff. It has real world applications. In the right hands, it’s good for people. That makes me more proud than anything, that we’ve done something beyond just another game. It’s transcended, just a little bit.

I grew up with atlases and globes. I have daughters, and when I ask them what’s the biggest country in Africa, they say, “Africa?” They don’t even understand what the countries are. The hell? You’re smart. You’re well-educated. What the heck? But their curiosity about the world, the geography of the world–I’m from Germany. Geography is mandated. You spend 12 years in school with geography. You have to learn it. In America my daughters don’t ever need to study it. They know nothing about the planet. It’s weird to me.

Stone: The X factor for flight sim is that attention to detail, to authenticity. When you’re literally working with petabytes of data, how do you sift through that and focus on what matters for the experience? How do you narrow down and hone in and optimize the Flight Sim experience with everything that you work with?

Neumann: The petabytes of data mostly sit on the ground. You can take that as far as you want. People say, “Did you get everything you want?” No. The cut list is much longer than the stuff we actually used. I wanted to do butterfly collection, for what it’s worth. I wanted to have insects. You could have a net and go make a collection. Or collect seashells.

GamesBeat: I love the sheep herding with the helicopter.

Neumann: That’s awesome. Isn’t that cool? Now, is it critical for flight simming? No. It’s critical for the authenticity of the planet, the emotional connection you have with it. The way our brains work, it’s in layers. I don’t know where you’re from. I’m from Germany, though. Say you show me the Rock of Gibraltar. Do I have an emotional connection to it? Not really. I know about it. But say I visit it someday, and I find out that the rock is full of monkeys. If the monkeys aren’t in the Rock, it diminishes the emotional reaction you have. That’s why I would say it’s important to have monkeys. That’s what the Rock is.

That’s how I typically feel. Is it actually relevant to what we do? The butterfly collection, is that important to anything? It really isn’t, until you make it important. Then it’s very important to the people who like collecting butterflies. I get up in the morning and read what other people have written. I try to understand the underlying thoughts behind it. Then I try to tackle that.

GamesBeat: Do you believe in the butterfly effect?

Right now, what’s more important: butterflies, or turbulence over the Atlantic? Turbulence over the Atlantic, no doubt. It affects flight. The AITA, the organization that collects that stuff, the pilots call back and say, “I just ran into turbulence.” They have a database and a map. They said, “Jorg, you can have our map. Do you want to put it into Flight Sim?” I do. We just didn’t get to it. But then you’ll get, in real time, the right rumbles at the right altitude over the Atlantic. Is that critical to flight simming? No, but it’s real. If you want to be a trans-Atlantic pilot, you’ll run into this. People will appreciate it.

Maybe we’re already into diminishing returns, but I don’t think about it that way. I think we’ll keep trying to make this as real as it gets.

Disclosure: Microsoft paid my way to the Grand Canyon. Our coverage remains objective.