Interactive Storytelling – Virtual Reality and 360 Film

I’ve been waiting for a while to write something more on the interactive storytelling front and VR was the obvious choice. However, it’s taken a while for the dust to settle from Facebook buying Oculus etc, but now I think it’s time to take a look at how VR is going to influence innovation and creativity in storytelling.

Let’s start with 360 video, as it’s likely to be intrinsically linked to live action and VR. Filming something with a 360 camera and letting people use VR glasses, or a VR headset does not make that film VR. It’s not VR. Sorry, just wanted to get that off my chest first. Now that’s out of the way…For anyone involved in live action film, whether that be interactive or otherwise, 360 brings all sorts of new challenges. Firstly there’s the issue of the 360 view itself. Where do you place the camera? How do you hide the lighting, the boom and other equipment? What type of actor do you need?

For camera, lighting, sound etc. it’s a good bet that the location will be the guide. Take the movie Ex Machina. Director Alex Garland filmed some of the scenes in 360 and they were clever when designing the set, lighting, sound etc. They designed everything so all of the equipment was hidden. The lighting was setup and tested before the set was fully designed, meaning they could get the perfect lighting for the shots, which meant that they could incorporate the lighting as part of the set. This is a lot easier than trying to fix or remove anything in post production. Should the location be too open to hide equipment it may well end up being the story that can help incorporate everything, rather than the set. For the short film “Morrison’s Birthday Party” – which I co-wrote and was the AD – the story, as you’d have guessed, was based around a birthday party. We filmed live with multiple cameras filming simultaneously in a large garden. How did we hide the cameras, sound etc? We didn’t. We used a single jib for the centre shots, while working around the garden with the other cameras. The story itself eluded to the fact that they was a crew there to film the party and it was fully incorporated into the script. The outcome? It worked and the film won best short at the NY independent film and video festival.

These are two examples of how you can creatively deal with equipment whilst filming 360 and it’ll be interesting to see more and more solutions to the issue. However, that’s not where the challenges end. What type of actor do you need for 360 and VR? Are actors going to need to become more stage influenced and become more and more able to perform live? If you’re filming 360 – without trying to splice together different shots to make it look like it was filmed that way – you need everyone to be acting at all times, as they’re being filmed at all times.

This is all due to the viewer being able to interact. Even for a more passive 360 film, the viewer can still turn their heads and see everything that’s going on ‘around them’. If the movie itself is a narrative, there’s nothing to stop the viewer from missing elements that the writer and/or director have deliberately included as a vital element of the story. A turn of the head at the wrong time and you’ve missed it.

So, for the writer and director of a 360 movie, there’s that additional challenge. You will have to consider putting visible and audible triggers in the movie, so as to guide the viewer to those moments. These are going to have to be subtle and not so obvious that it ruins the experience. The viewer needs to feel that they are part of the action and allowed to look wherever they want, but still enjoy the narrative.

What about interactivity and a more flexible approach to really make a film VR? How about the writer includes multiple choices for the viewer throughout the movie? That would make it way more interesting, right? Allow the viewer to interact and to be able to choose where the story goes. A lot of writers and directors will not like this idea, as they want to tell a specific story, but it should be looked at as an interesting challenge, as opposed to a limiter.

With all this considered, what will it do to the movie watching experience? Part of the enjoyment of certain movie genres, is sharing the experience with one or more people. Sure at the cinema, you want to sit silently and watch, but the when at home there’s that element of sharing. If you’re sat with a VR headset on, how do you share your emotions, facial expressions, comments on what you’re seeing (Especially the latter as you maybe looking in a different direction to someone sat near you, if you’re wearing the headsets)? Just take a look at the image at the top of this article (An image that I loved, by the way), how are those guys able to interact with each other?

This is where we’ll hopefully see ‘real’ VR and real interaction for movies. If you’re able to be totally immersed, as a digital version of you, or an avatar, you can have the ability to interact with each other within the movie. Sure, this has and is being done with games, but that’s surely the next step for VR and movie experiences. Great challenge for writers and directors, right?

These are just a few of my thoughts, whilst trying to keep this fairly short and it’s just food for thought. Can’t wait to see what people are going to do with VR and interactive storytelling going forward.

 

Bill Gates – The Return?

A lot has been said since Steve Ballmer stepped down and I’m not just talking about the similarity in looks to actor Dean Norris (He’s got to play him in the inevitable “Gates” movie, right?). The main source of interest for me, is the rumor that Gates will step back into a more hands on role at Microsoft. Gates took a big step away, back in 2008 and since then, it’s debatable whether Microsoft has been able to cling onto those dizzy heights of dominance they once had in the computer software industry.

[ Steve Ballmer…or Dean Norris? ]

We’ve fairly recently lost Steve Jobs, who was an incredible inspiration to many of us and a great innovator, leader and speaker. When Jobs returned to Apple, having been famously ousted from his own company – as we’re all very aware by now – he turned Apple around. He gave Apple back that drive for innovation, creativity and he also gave them back the figurehead that they’d been missing in his absence.

I think Microsoft have got that opportunity now. Of course, I’m not saying that Microsoft are in dire straits, but they’ve got the chance to be a company that grabs hold of the industry again. Right now there’s kind of a flux. Apple are, if you believe the press and critics, starting to slump and have been since Jobs passed. Google are doing well and driving innovation throughout their product set, but still can’t take control. Adobe are still pushing, but with the imminent death of Flash, they’ll be relying on Photoshop and After Effects keeping their dream alive. Now let’s look at Microsoft.

A big game-changer for me, was going to be Microsoft Surface. I’d taken the home computing step to Apple, back in 2007 and marvelled at the power and design of their laptops, especially for film editing and rendering. Along with the iPhone and iPad, Apple were changing things dramatically. It took a long time, but Microsoft stepped into the Mobile world with Nokia and I thought they may start pushing. Integration with desktop, mobile and gaming was on the cards, surely? However, with a less than satisfactory OS and a lack of innovative thinking – in my opinion – they missed an opportunity. However, when Microsoft announced surface, I was interested again. A tablet machine, with a decent portable keyboard, at a non-restrictive price? Sounded perfect for me and I eagerly awaited the release, so I could get my hands on one. What happened? They released it as an overpriced tablet, with a so-so OS and a separate keyboard that we would have to buy. Disappointment. I wanted to buy one, I really wanted to buy one, but when they were released I lost all interest. Yes, they’ve dropped the price a little and patched the software, but they may well have missed their ‘window’.

However, there could be one thing that could get them quickly back in the game. Bill Gates. Having the founder step back in as CEO and start driving the company forward again, could have both consumers and shareholders excited again. You can see that Microsoft want a fully integrated product set, desktop, laptop, tablet, phone and games console, it’s there waiting and they have the technology, especially with the buyout of Nokia Mobile. It’s time for Gates to step back in. It’s time for him to stand up, talk about the future of computing and Microsoft and about how they’ll be creating an innovative and exciting new environment for us all to participate in. Bill Gates could put a spark back into the industry again. He could make the other’s realize that competition is out there and the game has just been re-started. Wouldn’t it be great to be in the position of seeing companies having to fight tooth and nail to become leaders in the field? I think it’s all a little too comfortable right now, with technology companies tweaking existing hardware and software, rather than leaping ahead.

On a wider scale, company founders are the company’s life source, or at least that’s the way I see it. They are the guiding light, the energy and the DNA of the company – we should never forget that. Let’s bring back the leader, the innovator, the CEO to the world. Let them take the companies by the reins and take risks, break down barriers and push for new and exciting developments. Time for us all to get excited again.

Modern Mobile

Modern Mobile

Here are some quick thoughts on mobile. I know a lot of this will be preaching to the choir, but it’s something that is worth a reminder. I’m going to hit briefly on a couple of things, the first being the difference between mobile friendly, optimized and responsive and the second being native apps vs HTML5.

Are you friendly, optimized, or responsive?

First point on this subject is that people need to understand what the difference is. Let’s take ‘mobile friendly’. Mobile friendly is the most simple of the three. Make your website for desktop, but make sure it’s fully usable and viewable on mobile. So, this means nothing like Flash can be used, as an example. The other thing with mobile friendly sites is that they use the same navigation and display as the desktop version. What this means for the user on mobile, is that they’ll need to do things like zoom to read text, or use menu items – especially if you’ve not got the most thing on fingers – and scroll a lot, once zoomed. This isn’t the best solution, but it works and the user gets the benefit of the full desktop website.

If you want a website mobile optimized, it’s a different thing. The idea of a mobile optimized website, is that of designing the site in a way that is comfortable and easy to use on a mobile device. What does this mean for the user? It means that they do not have the cumbersome experience, of zooming, pinching, scrolling etc, that you would have from a friendly version.

The third solution is to have a responsive (or adaptive) design for you website. How is this different to an optimized version? Well the main element here, is that mobile devices come in all shapes and sizes, from tablets to smartphones and we need to cater for all. This means having multiple templates for the design, both for portrait and landscape. One should remember that one of those templates should be designed for desktop, as we don’t have a separate desktop version for a responsive site. This will mean that the user will have a similar, if not the same experience from desktop to tablet, for example and for smartphones all of the site will be available in a familiar design to the desktop version that they may have experienced at home, or at work.

There is now one additional thing for everyone to consider and that’s the news that Google have changed their mobile search algorithm.  The new algorithm takes the three different mobile website types into consideration for ranking. From now on, if you search through Google on your mobile device, any site that has been indexed and is not optimized, or responsive, will be demoted in the search results and ranking. This means that there is a danger that your site will drop off people’s radars, if it’s not pre-prepared.

Another point of interest is the growing use of mobile usage. A lot and in some cases the majority, of website use is now from mobile devices. This varies from country to country, but this growing trend means that you cannot risk losing website visitors, due to the fact that you’ve not considered mobile. In addition, platforms such as Facebook and YouTube are heavily promoting their responsive capabilities for apps and gadgets.

‘Mobile First’ has been a phrase that’s been mentioned a lot over the last few years, but it’s now a vital part of design and build. Designers will need to be fully clued up on optimized and responsive design and companies will need to bite the bullet and spend the money for website builds. Let’s make it clear, it is a longer UX, design and build process to create a responsive website, but in the long term, it’s worth the commitment and investment up front.

Are you a native?

 

This is another interesting debate, whether mobile apps should be native or web based. For a native app, the development process will use a programming language that is native to the device, so for example, for Apple devices using iOS, objective C will be the programming language used. For web-based apps, the choice of the moment is HTML5, which is more platform agnostic, than using objective C for iOS, or JAVA for Android.

HTML5 sounds really interesting on the surface, as it’s flexible enough for an app to be built once, for multiple platforms. If you build an app using objective C and want to use the app on an android device, you’re going to have to rebuild it. A HTML5 app is easier to distribute – for the most part – than using the stores available or mobile apps.

However, there are disadvantages for the HTML5 app developer. Much of the functionality of a device is not usable from HML5, as the APIs are just not complete. That means for those wanting to use things like the camera, GPS and accelerator, there’s no option aside from developing a native version.

So which way do you go, right now? Well I think it’s a big clue that people like Mark Zuckerburg have said that Facebook wasted two years on HTML5, before switching to native, although he did also say that he’s still excited about HTML5, but it’s just not there yet. My opinion is, as it stands, that native is the way to create the most engaging mobile apps, at least for now and the near future.

Interactive Storytelling – Immersive Film and Gaming

Interactive Storytelling – Immersive Film and Gaming

Last year I wrote an article on interactive storytelling and the future agency. Within the article, I hit on an idea for immersive gaming and how that could interact with film. I’ve added to that idea here, expanding on it and adding thoughts on projects out there, that are already heading in that direction. In addition, I’ll be concentrating on gaming and film, rather than including the usage in the advertising world.

For those of you that have already read my aforementioned article, you’ll recognize some of this, but keep reading, as there’s new stuff.

Interactive movies are becoming more popular. This includes both online and also offline applications, such as trials for interactive cinema. How does this affect storytelling, both on and off-screen?

How do we now write for the potential multiple story arcs and how do we start to embrace real interactivity for marketing clients? How do hardware elements such as MS Kinect and web cams change the way we look at interaction? Remember, interactive storytelling in this case is not limited to the digital world.

With the drive in story telling and technology, we can now see a clear view of how we can push towards a more immersive experience. This doesn’t have to stop at film and games; in fact they become the same thing. Isn’t it a possibility that we can now start really getting people to interact with products? What’s stopping us from creating full web series or stories based on a product, or product set?

With the connectivity of mobile, online, social and indeed above the line, there is no reason not to use all of these in partnership to create something fully immersive. The possibility for games, movies and campaigns to become interactive through any medium, is something that we should be looking at.

Perhaps it’s time for film to take the next step into fully interactive and even virtual territories.  We’ve seen this in film for a number of years. Movies like ‘The Matrix’ and ‘Johnny Mnemonic’ and novels by the likes of William Gibson and W.T. Quick, have given us a teasing look into what these virtual worlds within ‘cyberspace’ could be like. TV shows, like ‘Caprica’ have given us a view of how we could interact with others through our own avatars, in a virtual world. The fact of the matter is most, if not all of the technology, is now available for this next step.

This can all come together with an immersive platform. We can start to give real interaction to people, both online and in the non-digital world. Let’s take a look at some of the more practical elements first. Interaction has always been there, whether it be face-to-face, through social networks, or through games it doesn’t really matter, it’s all around us. Essentially it’s nothing new to any of us. An immersive platform would be a facilitator, a connecting bridge between all of the touch points.

Where are we now?

Let’s see how the idea is progressing, by looking at an interesting example. ‘Defiance’ is a new TV show, that the creators have linked up with a video game release. The basis for both is a sci-fi story, based in the year 2046, where aliens and humans are living together. That aside, the game and the show live together as well and in parts will affect each other.

Sounds very interesting and we’ll see how that works out soon, as the show begins on April 15th. One of the interesting things that the creators have done, is put together a great cast. With established actors, such as Grant Bowler, Jaime Murray, Julie Benz, Mia Kirshner, Stephanie Leonidas, Nicole Munoz, Graham Greene and Tony Curran, there’s already a large fan base available and plenty of chance of that growing through the gaming world. The game itself looks superbly done, from what I’ve seen. Graphically it does a great job of capturing the world and the characters, which will add, no doubt, to the in-game excitement and enjoyment. Sci-fi is a great genre to enter this joint-media idea. Fans are loyal and with the Cosplayers becoming more prevalent in the convention world and online, there are many avenues for fantasy and sci-fi stories. To take a look at the ‘Defiance’ experience, visit the website (Not now…later!) http://www.defiance.com

These are great steps towards an immersive environment, for both media. However, hopefully it’s only the beginning. From here, how can we expand the idea further, to completely immerse the consumer? How about social networks? How about mobile?

We are Detective

Let’s take an example platform, so that we can trace the path of immersion, if you like. We start of course, with a story. With the immersive platform, you don’t necessarily need to stick to a particular genre. Let’s assume, for the sake of argument and entertainment, ours is a detective story. We have our hero, the experienced and clever detective, who has a difficult case to solve. We introduce the viewer to our lead and his conundrum, through a piece of video on our platform. We realize by the end of the piece, that they need help in solving the mystery and we can be the ones to help them.

How could we help? Well the first thing a good detective would do is look at the evidence and look for clues. We’ve been given the details of the key players; so let’s take a look at what we have. We have the video and its contents. We have the names of the players. We have access to the local newspaper (Fictional and attached to the platform). Now aside from any local news, where else would we look? Well, where would we go on the Internet? What sites and services would we use? We search on Facebook, Twitter and YouTube and we find clues, we find accounts used by the victims, by the villains, by the suspects and we pick up detail that helps us give direction to the story.

We head back to our platform with a new wealth of knowledge and choose our path, based on what we’ve found. Have we found enough? We don’t know right now and nor would our hero. We watch the next piece of film, moving us along the story. But… There’s always a ‘but’, right? We need to go, we can’t stay at home on our desktop, but we want to continue the experience. The platform allows us to take it with us. We use our mobile devices and we continue to work on our evidence and in fact on our story. It’s our story too now, as we’re making decisions along the way.

The Mobile World

We’re lucky at the moment with today’s technology, because we are becoming more and more mobile. From mobile phones to tablets, we have the opportunity to embrace our online lives outside of our desktop environment. This means that we can now converse whilst on the move, with a richer interface and experience.

Any interaction we have with a movie, or game could be extended to the mobile world, augmenting the stories through AR applications, or extensions.

A good example would be to use an AR piece, as an extension of our immersive experience. We could be given a co-ordinance in our local area, where we go to find some information. Once there, we could trigger the AR piece, to show us the next piece of data we need. This could be through heads-up style display, or even in the form of an AR game. For an advertising agency, this is a prime opportunity for engagement. The location could be a client store, or something similar. Perhaps something that the consumer will see, or interact with.

The AR piece can be triggered with a simple photo application within the game, or even a QR code, to get the user moving to the next stage. AR experiences have grown a lot over the past few years and we’ve seen stars like Robert Downey Jr. get involved with a piece for Esquire magazine and Rihanna for NIVEA. Over the next year or so, we’re about to see the next stage for augmented reality.

Google have taken a big step in this direction with their ‘project glass’, as it’s now known. During 2013 we’ll be able to buy the Google glasses that will enable us to view a real heads-up display of maps and other data. There’s no reason why we shouldn’t look to integrate this into the experience and indeed, there are already many clones announced that will hopefully give the product a wider usage.

The final step for our experience is in this strange ‘real world’ that we just entered. Yes, for those with a blinkered view of online life, we should allow real interaction. Part of the immersive experience could led by events, perhaps for the finale of a game, the last scene of a film. The consumers could meet and even sit down in an interactive theater to watch it all play out. In Korean several interactive theaters are being tested. We’re not talking about a Disney 4D, where the audience are sprayed with water, for example – Lord knows what they’d do if they produced a 4D Star Wars film! Doesn’t bear thinking about, really. But I digress – What we are talking about is having the audience participate in the movies with mobile devices, to make decisions in the films. Perfect for the immersive platform.

Where do we go from here?

So what’s the future for engagement and interaction? Many years ago, we experimented with virtual reality and how we could become ‘part’ of an experience, in a virtual environment.  This was pretty much brushed to the wayside, as the technology just wasn’t good enough to fulfill people’s ideals. However, things have moved on and there’s a good possibility that full interactive and virtual experience are on the way.

Universities have now started 3D avatar modeling of people. How have they taken influence from movies and how will this change story telling across all media in the future? It’s always an interesting thing to consider. The futures portrayed in the movies are becoming closer and closer. Well some of it. I still look out of the window Christmas morning, hoping to see my new flying car. No luck there, yet.

The fact is, there are a multitude of technologies waiting to be exploited by us, so why not use them whether they’re on or offline. We should be looking toward connecting the consumer to everything, to the client, to each other and to the stories that we can tell.

The future agency will be more of an overall facilitator, rather than just a marketing and advertising outlet. They will become more fully ingrained in the client’s world and be a source of knowledge and inspiration to those who hire them.

Let’s take our earlier example to run with and imagine that we’re a company. Let’s imagine that our company has the idea of a future immersive gaming and film platform. Where would we begin? We would begin with research. Research across the world of film and gaming, which incidentally, are two of the richest businesses in the world. Research would be completed for each and every element, from genre popularity to sales. The outcome of the research would be the determination of which direction the new platform would head and if the platform was indeed viable.

Once we’ve established that we’re correct in assuming that the platform is something that is viable, we would take a look at the technology involved and the platforms used. Where do people interact and consume? What do they do, when they’re there? The reason these questions need to be asked, is that the future of interaction and entertainment may well be something much more ‘involved’ on our new platform than just playing an interactive game, as an example. People will be able to interact with product, they will be able to interact with other people, and they will be able to consume information, all whilst interacting with the game and movie itself.

Consumer interaction is now wide spread across many platforms, as we know. Facebook, Twitter and other social networking sites have definitely changed the way that we interact with each other, but we also have to consider where we interact. Realistically our interactions spread across many platforms, from verbal communication to mobile and those social networks. Why not create a platform, so that consumers can interact across all of those? Some games are already doing this to a certain degree; games such as World of Warcraft and Call of Duty give the user the ability to interact with each other during the game play. This goes further, though. We would look at creating a platform that was portable. Something that the consumer could use via desktop, mobile, in live situations and even virtually. We’ve discussed this already, so we won’t dwell on the detail, however who better placed to build and implement these phases, than the future agency?

To be, or not to be…Open Source

The next phase in to establish the hardware and OS needed. With the hardware, we’re going to be looking at multiple devices, but with the option to use only one, or a combination, if need be. Would we want to create a physical product? Would we want to give the consumer the ability to own a piece of kit that would give them all of the functionality in one unit, or would we look to provide a platform that products could use? Would we want to create a proprietary product?

One of the most successful companies in this arena is Apple. However, Apple had something that most new companies don’t, which is the long time experience of creating product and the software that it runs. They come from a design and build background, from creating machines with parts in a garage, using software or an OS that already existed. That changed when they wanted the OS to do more, so they wrote their own, to match their products. With console gaming, each company creates their own hardware to run the games on and then relies on gaming manufacturers to write their games for that console. Each console is limited then, by the development of those games. In addition, the proprietary console is limited in its usage across other platforms. Microsoft have had some success with XBOX Live, where gamers can interact on the internet and MS promised the additional ability to carry those games, or at least part of them, to mobile devices. Unfortunately, we haven’t really seen this happen. Both Microsoft and Nintendo have extended gaming out to the physical world with Kinect and Wii respectively, but we’re all waiting for the next stage of interaction.

For a number of years, there has been a big push for open source. This could be the deciding factor in how our immersive platform would exist. A platform that is designed in such a way, that it can be hosted in one place, but allow manufacturers to get involved to create their own products for use in the new environment. It could allow storytellers, filmmakers and game developers alike, to create their own worlds for the consumer to interact with. For these storytellers, it could provide an outlet, not only for their work to be viewed, but to be fully engaged with and extended. It would give them the opportunity for their work to really ‘live’.

The time is now

So what are we waiting for? Let’s get moving and create an immersive environment that not only encompasses film and gaming, but also moves into social, mobile and real life. The technology is out there and it’s a logical direction for us to go. The time is now.