One of the most beautiful qualities of virtual worlds is that they are perceptually immersive.
By simulating environments that mirror reality, virtual worlds can evoke in us a surprisingly strong sense of place and space. We feel like we’re “really there.”
But one thing that is sometimes overlooked in the design of virtual worlds is the fact that we perceive the world not just visually but also through our sense of hearing.
The ambient sound of wind and bird chatter in a forest. The subtle creak of a wooden floor as we walk across a room. The cracking of a fire in the distance.
Ambient and contextually relevant sounds can make any virtual world a much more immersive place. We often don’t notice them consciously. But when such sounds are absent, we usually feel like something is missing.
My primary interest in virtual worlds is immersive education. Teaching through storytelling that naturally unfolds as a person explores the world around them. Encouraging people to discover new things by creating environments that engage them. And a big part of creating any immersive learning experience is making sure that it is perceptually immersive in as many ways as possible.
I’m currently spending a lot of time learning how to create multiuser virtual worlds on the web using Jibe and Unity3d. I’ve been building out my own Jibe world and testing different ways to make it more engaging.
In my experience, the best way to encourage exploration in a virtual world is to simply reward people for exploring it.
I’ve been doing this by placing a few interesting little scenes to discover in my virtual world. Some of them you will just stumble upon as you walk around. But others call attention to themselves in subtle ways.
Visual cues, especially movements, are very useful ways to indicate “hey, come over here and check this out!” But never forget the additional power of sounds.
What was that? Did you hear something over there?
You may notice that I’ve got a lot of animals in my Jibe world. This isn’t just because I like animals. I’ve placed animals everywhere simply because animals both move and make sounds. And those characteristics in a virtual world make it feel more alive.
Worlds that feel alive are worlds people love to spend time in and explore.
How to add sounds in Jibe and Unity3d
In Jibe and Unity3d, you can embed sound files in any object in the world. These sounds sources exist in 3d, which means they can be perceived the same way we perceive sounds in the physical world. Sounds get louder the closer you get to them. And you hear things spatially when using stereo speakers or headphones.
You can learn all the details of how sounds work in Unity3d by checking out the Unity3d manual entry on sound. But here’s a quick crash course on how to add a basic looping sound to a specific object.
Crash Course in Sound Design: The Chicken
Chickens are awesome. Visually, they move in very interesting ways. And they make great sounds. So I definitely wanted some chickens in my Jibe world.
1) Find a nice animated 3d model of a chicken.
I searched the Unity Asset Store but alas, no chickens. However, I found a great animated chicken model for sale on the web. I bought it, downloaded it, and placed it in my Jibe world using the Unity editor.
I made her super-sized because I wanted folks to easily see the model detail. But my chicken lacked sounds. Bummer.
2) Find a chicken sound.
If you’re looking for sounds that are free and Creative Commons licensed, I highly recommend checking out FreeSound. I searched FreeSound for chicken sounds and found a nice WAV file of a chicken clucking.
Just one problem. The sound file was too long and large. Remember, when you are deploying a Jibe world, size matters. The more you add to your world in the Unity editor, the slower your Jibe world will load on the web. Remember this fact when you are adding sound files. You’ll always want them to be as short and small as possible.
3) Edit the chicken sound.
I love Audacity. It’s a free and open source program for recording and editing sound files. So I opened up my chicken WAV file in Audacity and cut it down to a very short clip that looped nicely. I also saved it in Ogg Vorbis format so it was compressed and as small as possible.
4) Get the chicken sound into my Jibe project file in Unity.
I opened up my Jibe project file in the Unity editor, went the Assets menu at the top of the screen, and chose Import New Asset. Then I found my chicken sound on my hard drive and selected it.
You can now see the short chicken sound file sitting in my Project folder, and the details of the sound file in the Inspector window.
5) Get the chicken sound into the chicken.
I dragged the short chicken sound file from the Project folder onto the chicken model in my Scene window. Then I viewed the new details of the chicken model in the Inspector window.
There’s the sound file in my chicken. I turned on the loop function, and I was done.
6) Enjoy and learn more!
You can visit my chicken in my Jibe world on the web. And if you’d like to join a community of people interested in creating virtual worlds using Jibe and Unity3d, please check out my Jibe and Unity3 Google Group.
If you are designing a virtual world, spend some time not only thinking about how it should look but also how it should sound. Got a fire? Make sure it has a nice crackling sound. Building a train station? Add ambient sounds of crowds and trains. You’ll be giving your users a much richer experience. And maybe you’ll learn some new tricks along the way.
-John “Pathfinder” Lester
Just don’t make that chikkin breedable, Path. Excellent post. My Usher build came right to life with the files from Freesound that I’ve added.
I might script the chickens with some code that will let them wander around and maybe follow people. But yeah, I’m not going to breed them. ;P
Great to hear you’ve found Freesound useful. I love exploring the sounds that folks upload there.
I’d be very interested if you get the scripted movement working for your chickens. I need the same sort of thing on the rats, but haven’t had any satisfactory results yet.
I’ll let you know what I learn. Absolutely!
TY for an excellent post! I explain to many people that hearing is our second-oldest sense (after smell) and contributes enormously to our sense of location and “environment volume scanning” in RL; it is crucial to the making of a “world” and sense of immersion. When I demonstrate the soundscape possible in my Garden of Sound in SL, people are just awed at the way it “opens up” their sense of the Garden as a place, deep and broad and high.
One of the saddest things in a virtual world is sitting by a beautiful seashore with amazing prim wave action… and no sound. Add a few differing surf tracks, some whales and the occasional seagull, ship’s bells and boat horns and you suddenly have a much richer and immersive environment.
“Garden of Sound.” What a beautiful name.
Do you have a SLURL to it? I’d love to visit.
The Garden of Sound & Shinden no Kagetsu
Excellent post, and excellent resource!
Sound played a huge part in the WW1 sim in second life, not only the interviews and poetry, but particularly the ambient sound. The intention was to make the atmosphere more and more intense as you got nearer the front line. I’m using the same approach in the Unity version. Fortunately I still have most of the original source files for the ambient audio as getting sound out of SL is not easy (not impossible, but very very kludgy)
Pingback: Interactivity in Virtual Worlds: Using Triggers in Jibe and Unity3d | Be Cunning and Full of Tricks