While most attention was focussed on the red carpet and the critical reception for movies in competition at the Cannes Film Festival some eyes were looking toward the future of film.
At Hewlett Packard this starts from envisaging how the world will look – and thus what computing needs might be - not just a few years hence but decades down the line.
HP works with sociologists and futurologists to predict the future based on macro socio-politico-economic trends and then uses these anchor points to guide HP’s long term roadmap.
Leading the whole effort is CTO Shane Wall who terms the sum of this vision ‘blended reality’. “The goal is to create experiences and applications that work seamlessly and where the technology disappears into the background.”
HP trains this vision on healthcare - for example in the potential to have an AI assisted brain command over prosthetic limbs for amputees; and for end to end digital manufacturing, notably just-in-time 3D printing using polymers and then metals.
The future of film and entertainment is a small subset of these global problem solving goals but nonetheless an intriguing one.
“Rapid urbanisation which will see over 60 megacities by 2050, will profoundly change where we live and how we live,” says Wall. “There will be demands on resources, how we get rid of waste. It will force us to look at problems and experiences in a different way.”
There will no longer be mega theatre complexes containing a dozen cinema screens, he predicted. Instead one might find single cinemas focused on the experiential shared experiences.
This was a theme taken up by Anish Mulani, President and COO of VFX and animation facility based Prana VFX house Prana Studios. Prana plans to reinvent the theatrical experience with massive screens encompassing entire theatres, animatronic dinosaurs, and actors performing live on stage.
“The whole canvas of the theatre will be used to tell a story,” he explained. “The moment you walk in, every wall will be covered with giant projection. There will be moving seating and sensory effects like wind and heat. Life size animatronic creatures and characters relevant to the story such as pirates, aliens and dinosaurs will be there with you.”
The giant size of the auditoria, with panoramic screens in excess of 40 metres, and display resolutions up to 24K - or 12 times that of 2K conventional exhibition - are also intended to attract audiences.
Prana Studio is also developing dome-style theatres featuring 180-degree field of view and reclining seats. “The seats would be able to change angle so that you could view a stage and see actors performing part of the story live as part of the overall experience,” he said.
“We imagine 25 or more sites worldwide within the next decade,” he said. “The main issue is the cost of rendering images at such extreme resolutions and in 3D that content will initially be short form.”
The proposals build on existing theme park projects at Universal Orlando’s Islands of Adventure ride, Skull Island: Reign of Kong – for which Prana created the six-minute movie in 24K resolution – and the Chimelong Ocean Kingdom attraction in China which features a curved screen 88 metres wide and 18 meters high – the world’s largest film screen. Prana created the short 5D experience that accompanies it.
Ultra large scale exhibition
“Cinema exhibition is moving toward ultra-scale large format experiences while conventional movie releases will be streamed for projection within people’s houses,” Mulani predicted.
Interactivity is a clear theme of media and entertainment 2050. “The emergence of the Gen Z population which are the first born who know nothing but being able to interact with content on devices seamlessly will impact how we make and view entertainment,” Wall said.
Stories might exist as transmedia – with different parts of the same story experienced on different platforms and devices – often simultaneously.
Wall also pointed to “accelerated innovation” or a speeding up of Moore’s Law. “If smartphones today are 30 times more powerful than PCs were a decade ago then they will be 30 billion time more powerful still in three decade’s time.
“With VR we are operating at the equivalent of DOS 3.0 or punch cards,” he continued. “VR has limitations today. It separates us from what is out there. But the potential is for total body, total sensory immersion. We will be able to craft images using lightfields that capture every angle and nuance of light in a scene for us to reconstruct holographic moving images. These are the technologies which will become our clay [for creating art and new media].”
The VR short ‘Tree’ is an example of how artists and filmmakers are pushing boundaries with the technology. It puts the user in the position of a tree which grows from a seed to become one of the tallest in the forest. “For us, smell is super important,” explains co-creator Winslow Porter of New Reality. “We developed custom scent tracks that are sequenced with the visuals to enhance the experience. It’s strange we are using organic molecules alongside such hi-tech. Scent is associated with memories which we can trigger by tapping into the olfactory sense.
“VR can be impactful in ways we don’t fully understand right now but it will be able to change hearts and minds,” he added. “There is an obsession with simulating reality in VR when we should be looking to surreality. We can be a person from the past, or a lion or a pyramid.”
Winslow also forecast the end of linear narrative. “A director might be able to frame the shots and a cinematographer light them but we are entering an age when the viewer becomes the editor and the protagonist,” he said. “When we have thousands of people participating in the experience what happens to narrative then – they could be building the world as they play and explore within it.”
That’s the near-future scenario sketched out in the novel Ready Player One and being adapted by Steven Spielberg in his forthcoming feature.
Keys to immersive content
The story’s concept of game-playing and education in a multi-verse of virtual worlds has become a must-read text at the Technicolor Experience Center where Marcie Jastrow, senior vice president, immersive media and head of the centre is helping to build a pipeline for immersive AR/VR and mixed reality content. There are six keys, she outlined. These include the need for high quality content, for social sharing experiences and for the user experience to be intuitive. Also on her list is “episodic cadence” which means a way to keep people coming back for more. “It has to be disruptive enough that it is different from current media and most important of all is the story. It has to entertain.”
Jastrow suggested that studios could soon sell VFX assets like characters and computer generated worlds as products to consumers for VR experiences. “Often when studios build these assets for a film they are simply archived and never used again,” she said. “What Augmented Reality and Virtual Reality offers is the potential to use these assets again. What would it be like if rather than playing with a fluffy toy, you could gift your children one of those assets to play with and interact with in a VR or AR environment? It is all about extending the experience of the film through interaction.”
HP at the movies
HP tracks its association with the movie industry to 1938 when audio oscillators built by David Packard and Bill Hewlett were used by Walt Disney to produce stereophonic sound for Fantasia.
The compute brand has sponsored the Cannes Film Festival for several years but more importantly its hardware is installed at movie studios and VFX facilities including at Technicolor and Dreamworks.
“Roughly 50% of materials and devices in the world are designed on a HP work station,” claimed HP president personal systems, Ron Coughlin. “That’s why our motto is ‘keep reinventing’.”
HP’s EMEA director Nick Lazaridis candidly admitted that gaining visibility was the principal reason to have a presence at Cannes – but then that’s why everyone else comes here.
Its technology is onboard the International Space Station, Studio Liebeskind used its workstations to architect the Freedom Tower in New York and HP claims to be the number one computer provider for education around the world.
It launched the Sprout which chimes with its idea of blended reality. Combining a scanner, depth sensor, hi-resolution camera and projector into a single device, Sprout allows users to take physical items and merge them into a digital workspace. The system also delivers a collaboration platform, allowing users in multiple locations to collaborate on and manipulate a single piece of digital content in real-time.
The firm is also one of a number of companies making mixed reality headsets for Microsoft with Windows as the operating system.
Dreamworks VR tool
Dreamworks views VR as a production tool just now rather than a content media. “It used to be that storyboards were the only way we could vizualise films,” says Kate Swanborg, head of technology communications and strategic alliances, DreamWorks Animation. “Now we are using multiple different techniques including motion capture and VR – living storyboards – which allow directors to iterate in a 3D space.”
More prosaically, like other animation houses Dreamworks masters its films for distribution in 2K simply because doing so in 4K is still too expensive.
“Unlike live action, every single pixel is digitally rendered and there’s no real way to upscale it. It’s a huge time and money expense and it’s not clear we can could recoup the ROI on that.”
For just one feature animation, Dreamworks creates 350 TB of data which is managed on HP workstations souped up with multi-cores and Nvidia Quadra graphics processors and DreamColor displays. To highlight the challenge, takes 80 million CPU hours to render one film which comprises half a billion files or 25 billion pixels. That doesn’t include archives.
As much as Dreamworks’ films are essentially data, each production team also physically prints every single asset including characters, environments and storyboard. “We pin them on the walls of our studio so that artists can immerse themselves in the tactile images,” she said. “That’s part of our process to determine if a scene is working.”
Artificial Intelligence (AI) is another key area of interest for the future of film. Mulani suggested that AI would be used in Hollywood to read scripts. “AI will read, doctor and polish scripts. When AI becomes as powerful or more powerful than the human brain it will be able to analyse a script within minutes and give you ten different analysis of where it could be altered,” Mulani says.