The world is not enough

Prime Focus' Merzin Tavaria has set his sights on reinventing the visual entertainment business
The Legend of Hercules
The Legend of Hercules
Sin City.
Sin City.
World Warz.
World Warz.


After taking Hollywood by storm, Prime Focus’ Merzin Tavaria has set his sights on reinventing the global visual entertainment business. By Nirmal Menon

For a young computer graphics artist (CG) in India, the greatest difference between now and 20 years ago may be that today he is spoilt for career choice. Two decades ago, there were few avenues to learn and fewer prospects of meaningful work. Typically he would do piecemeal jobs for television shows and commercials. At best, he would become an instructor in a training institute.

It was around that time a young instructor in his late teens started teaching basics of computer graphics in Mumbai. Little did he know that fate had more than just a ‘good’ career planned for him.

It was during one of his lectures, a student proposed an idea of starting a company that could offer editing services for the burgeoning television and advertising industry. Any complacent scout would have sidestepped the idea, but this young man chose to run the risk.

Twenty years later, Merzin Tavaria, the boy who chose to stick his neck out, is the co-founder and creative force behind the world’s largest media services powerhouse, Prime Focus Ltd. Though the growth of the company was exemplary in comparison to other media and entertainment firms, its beginnings were humble.

In 1995, Tavaria along with the founder of Prime Focus Namit Malhotra — the teenager who offered him his first brush with entrepreneurship — and co-founders Prakash Kurup and Huzefa Lokhandwala started an editing firm called Video Workshop in Mumbai.

Within two years, the company became one of the leading television post-production studios, facilitating around 23 shows per week on broadcast channels. What sparked this rapid growth is the hunger to strike it big and the ability to harness the right technology in milking the opportunity.

When Video Workshop was set up, Tavaria and his team chose to install Media 100 non-linear editing systems specifically to cater to the lower-end television shows. In 1997, Prime Focus was established, and the company had become the “go-to” facility for television commercials and music videos.

Also, what made this one-time instructor a name to reckon with in the global visual effects space is his ability to get things done.

The role of a creative leader is not to have all the ideas; it’s to create a culture where everyone can have ideas and direct them towards achieving the larger goal. It is this tenacity that led to the development of one of most enviable digital pipelines in the world and invited the attention of the biggest production houses in Hollywood and Bollywood.

Tavaria delivered some of the notable Hollywood and Bollywood projects including Gravity, Guzaarish, White House Down, Mirror Mirror, Scream 4, Houseful 2, Blue, and Ghajini. In 2010, he built a team of 2,000 artist and professionals to establish Prime Focus’ View-D stereo conversion pipeline in India. The pipeline has been delivering 3D films to Hollywood and the globe ever since.

In fact, last year was a watershed year for Prime Focus as the company delivered stereo conversion for marquee projects such as The Legend of Hercules, The Amazing Spider-Man 2, Maleficent, Edge of Tomorrow, Transformers: Age of Extinction, Guardians of Galaxy, Teenage Mutant Ninja Turtles and Sin City – A Dame to Kill For. In between these projects, the company also delivered visual effects for The Expendables 3, Cosmos: A Space Time Odyssey, Indigenous and Into The Storm to name a few.

Article continues on next page ...

“Last year has been particularly good. We delivered some of the biggest Hollywood projects. We also collaborated with some of the best VFX companies and were able to blend our pipelines. More than anything else, we learned to roll with the punches while dealing with the rigours of delivering truly global projects,” says Tavaria, co-founder and chief creative officer of Prime Focus, the largest visual entertainment company that employs over 5,500 professionals in 16 cities across four continents and seven time zones.

While every project presented its own set of challenges, Robert Rodrigues’ and Frank Miller’s Sin City – A Dame to Kill For was the most prodigious one, as Prime Focus not only became an equity partner for the project but also the sole provider of visual effects and stereo conversion work.

The movie was particularly challenging because it was entirely shot using live-action photography against greenscreen and then treated with a mix of black and white, high contrast and colouring to narrate the story.

“We knew it was going to be a tight timeline. As soon as we were green lit, the teams quickly started sorting the two big issues: people and technology. We were going to work in two locations separated by time zones. So design, asset creation and layout were assigned to Mumbai, while shot execution was shared between Vancouver and Mumbai,” says Tavaria.

For this project, his team delivered 2,282 VFX shots, using a team of 702 VFX artists, out of which 171 were based in Vancouver and 531 in Mumbai. The project also comprised a stereo team of 1,502 artists.

Sin City 2 – A Dame to Kill For was extraordinary in every sense. Typically, a cinematographer looks at a set, designs the frame, locks his camera angle and places the characters.

In this case, the actors were shot on green screen in the beginning, and the visual effects team had to position the layout of the scene in accordance with the characters and create a universe around them. This meant every location, prop and asset had to be digitally crafted. Every shot needed layout, post-visualisation, animation and compositing.

“In case of live-action movies, we’re just tracking the camera and putting it into a scene, but here we had to take up the role of a cinematographer and actually design the whole sequence. There were sequences where the entire shot were rendered in CG, including that of Mickey Rourke,” says Tavaria, while adding that his team created a total of 65 CG locations and 1,572 assets.

In order to manage the teams, visual effects supervisors Jon Cowley and Tim McGovern were made in charge of Vancouver and Mumbai operations, respectively. Cowley in the past has delivered projects such as The Great Gatsby, Men in Black 3 and X Men: First Class.

McGovern, on the other hand, is known for his work in As Good As it Gets, Total Recall and Last Action Hero. Stefen Fangmeier, who has supervised blockbusters such as Saving Private Ryan and Terminator-2, also stepped in as overall visual effects supervisor for the project.

The project was also a huge challenge in terms of technology because the assets and databases used in the project had to synchronise between two locations. It also meant that all the assets including their multiple versions of render passes and composites had to be accessible in a systematic and organised manner, which further meant that pipeline dependency had to be on top priority at all times.

“We had to make sure that if an asset was generated in Mumbai, it was automatically available to the Vancouver team the next day. Similarly, if a concept was developed in Vancouver, the latest version had to be published in Shotgun and made available for the teams in Mumbai,” adds Tavaria.

Article continues on next page ...

For the uninitiated, Shotgun is a project management tool that provides complete tracking of shots, scheduling of tasks with their respective start and end dates, overviews of tasks allocated to each location and daily reports on time log information.

Prime Focus also developed special tools to support specific tasks. For colour correction, the R&D team ensured that more than 50% of the black-and-white look was already built into the system and ready for the compositors to work on.

This saved a lot of time as the artists could now cut down the effort involved in the desaturation process. For layout, the technology team developed a tool that could pull in all the cameras of a scene into one Maya file, thereby making it easy for the supervisor to judge the layout of shots in relation to others as opposed to the traditional approach of working one shot at a time.

Prime Focus’ main toolset included Maya for CG work, V-Ray for rendering and Nuke for compositing. Its approach to the stereo was to ensure it was actually integral to the visual effects process. The VFX artists worked in mono so they could work the way they were familiar with. Then internally the stereo team would take the composites straight from the Nuke files and render any stereo passes they needed from CG. So it was a brilliant hybrid approach of integrating two teams together.

Reviews were done using RV sync and Cinesync. This helped the supervisors in Vancouver and Mumbai communicate with and annotate on the visuals to brief the teams.

“The amount of reference available for the show was unprecedented. Double Negative took reference images of all the characters, from the principal actors down to the extras, to capture specific textures that the VFX team would use while modelling the characters in CG. Actors were photographed in a variety of full battle gear, and several of the actors were captured using geometry caches. Our team then used these assets to build digital doubles, and for crowd simulations,” says Tavaria.

The crowd simulations began with some basic motion capture to develop automated simulations called procedurals. Since the armies would be clearly visible in close-ups, doubling one agent over and over appeared to be a futile process.

To avoid this, the Prime Focus VFX team built 10 different basic agents for the armies that were multiplied throughout the crowd using Massive software. As a result, whenever there was a close shot of the army, the audience could see Cinesync. This helped the supervisors in Vancouver and Mumbai communicate with and annotate on the visuals to brief the teams.

It allowed the teams to access any file on the fly, making playlists while a session was on. Team members could pull up a concept for a shot, or reference, compare it with a version or two versions before, all live. In fact, the teams in both locations used the same toolset to engage with director Robert Rodriguez.

Some of the other interesting visual effects projects that Prime Focus delivered last year included Brett Ratner’s Hercules. The company was called in to provide 224 VFX shots for the film, working alongside lead visual effects house Double Negative.

The company was tasked with delivering the ‘Bessi’ battle sequence – a complex scene involving heavy CG, crowd work, environment extensions, digital doubles, animation and some full CG shots with a full environment, consisting of villages, woods and trees. A variety of faces, bodies and clothing.

Prime Focus was also instrumental in the stereo conversion of Edge of Tomorrow – a move that reunited the award-winning team that delivered the stereo conversion work for Alfonso Cuaron’s Gravity.

Article continues on next page ...

Prime Focus senior stereo supervisor Richard Baker and production head Matthew Bristowe worked closely with Edge of Tomorrow stereo supervisor Chris Parks to deliver 113 riveting minutes of the action-packed film. The project saw Prime Focus employing its geometry-mapping techniques for stars Tom Cruise and Emily Blunt as well as for certain key environments, to ensure consistency and accuracy of depth amidst all the action.

“This being Tom Cruise’s first 3D film, we had to ensure he looked fantastic in stereo. This tied in well with our use of head geometry. We were able to show Chris, Doug [Liman] and the studio some early shots of Tom in 3D, which looked great – and consequently everybody was on board with the 3D conversion process. To meet the challenge, Prime Focus employed the full range of its latest creative and technical conversion techniques, including auto stereo camera generation for VFX integration,” says Tavaria.

There is a sequence in the bunker in which the characters are interacting with a hologram. Semi-transparent holograms can be difficult to convert; it’s always preferable to get a stereo render if you can, so it was a perfect opportunity to use the stereo camera generation technology the company had developed for Gravity.

The stereo camera generation tool is part of Prime Focus’ proprietary Hybrid Stereo Pipeline, wherein a stereo camera pair is generated from hand-sculpted disparity maps to produce a virtual rig that will work in any CG or compositing environment, allowing the VFX vendor to render CG assets with exactly the right amount of depth for a given slice of the scene.

In fact, the Hybrid Stereo Pipeline is the heart of its stereo conversion process. It originated as a roto-based process, and has evolved to also incorporate the use of geometry – specifically for lead character’s heads and for key environments, where there is a need for a high degree of consistency and accuracy.

The Hybrid Stereo Pipeline allows artists to map between the non-linear depth of our converted scenes and the linear depth of the geometry created from cyber-scans of the lead characters’ heads or LIDAR (light detecting and ranging) of the environments.

This hybrid geometry/ roto approach was first used on Gravity and developed through Prime Focus’ work on World War Z (Brad Pitt’s first 3D film) and formalised for work on Edge of Tomorrow (Tom Cruise’s first 3D film). The references to the stars on these movies were important as the studios were cautious about depicting their stars on screen, and ensuring accuracy and consistency of their appearance in stereo.

“UV Tracker is one of the tools we developed to assist this process. The artist can paint a single stroke on either a linear or non-linear disparity map to sculpt the depth adjustments and that paint stroke will automatically deform to follow the movement of the head throughout the scene, using bakedin coordinates,” points out Tavaria.

This was used last year in Maleficent, starring Angelina Jolie. The use of head geometry was particularly important to preserve the relationship of her horns in depth as her head turned, thereby ensuring the accuracy and consistency of her face, with its high, angular cheekbones.

Another interesting aspect of the Hybrid Stereo Pipeline is Auto Colour Match. A common problem in stereo is that there may be colour differences between two otherwise similar plates. Prime Focus developed a couple of unique tools that address this issue.

The first tool takes two similar plates and automatically grades them so that they are matching, creating a onedimensional LUT. The second is a very exact 3D LUT generator, which can reconstruct all of the colour operations in a shot automatically. Both tools operate in ACES space. Auto Colour Match was recently used on Prime Focus World’s conversion work on Noah.

The Depth Retrieval toolset of View-D also allows artists to take two natively captured stereo images and correctly realign them, then make further depth adjustments where required.

Mapping and roto-segmentation allows them to apply non-linear depth with a displacement tool – Depth Shift – so that they can correct errors and reset the depth for parts of the shot with great ease. Related tools also include Stereo Alignment Toolset and Stereo Correspondence Toolset, which are applied to derive an accurate and detailed disparity map. These tools have been recently used in Transformers: Age of Extinction.

“We have always focused on improving our technology and reinvesting the experience and knowledge in developing our processes and pipelines. Unfortunately, that’s where most visual effects studios falter.

Most of the studios barely manage to invest on a pipeline resource. Earlier, we had one technologist dedicated for pipeline development. Now we have 15 pipeline developers in India alone, and with Double Negative we are talking about a talent pool of 50 full-time pipeline developers,” adds Tavaria, who has not only taken Hollywood by storm but has also set his sight on reinventing the global visual entertainment business.

Not bad for someone who started out as an instructor 20 years ago. Looking back, Tavaria has much to thank the student who casually tossed the idea to start a company offering editing services for the industry at a lecture he was delivering. It was an idea that changed the course of his life and the landscape of VFX in India and abroad.
If anything, Prime Focus’ global success story is the result of the clear focus Tavaria and his team have had from the word go. Moving on, the writing is on the wall: The World is Not Enough.

Article continues on next page ...

Stereo Toolbox
Here are some of the proprietary tools that make up the Hybrid Stereo Pipeline.

Auto Stereo Camera Generator
In order to incorporate linear CG assets into non-linear conversion sequences, Prime Focus needed to automatically create a virtual stereo camera pair that would render CG assets with exactly the right amount of depth for a given slice of scene. This virtual rig had to be dynamic, allowing for changes to stereo values over the duration of the scene and in accordance to the object’s movement through the scene.

UV Tracker
UVTracker is one of the tools Prime Focus developed to assist the hybrid geometric/roto process. The artist can paint a single stroke on either a linear or non-linear disparity map to sculpt the depth adjustments, and that paint stroke will automatically deform to follow the movement of the head throughout the scene, using baked-in coordinates.

Auto Colour Match
A common problem in stereo is that there may be colour differences between two otherwise similar plates. Prime Focus developed a couple of unique tools that address this issue, collectively known as Auto Colour Match. The first tool takes two similar plates and automatically grades them so that they are matching, creating a one-dimensional LUT. The second is an exact 3D LUT generator, which can reconstruct all of the colour operations in a shot automatically.

HistoCurve analyses a scene, then builds a histogram, allowing the generation of an expression to depict how the scene is dispersed across the scene and across frames. An artist-sculpted scene can be sampled, producing a population density and allowing us to quickly see how it looks against native capture or a computer driven model.

Depth Retrieval
The Depth Retrieval toolset allows artists to take two natively captured stereo images and correctly realign them, then make further depth adjustments wherever required.

- 224 vfx shots were provided by prime focus for ‘sin city - a dame to die for’
- Hybrid stereo pipeline allows the vfx vendor to render cg assets with the right depth

Most Popular

Editor's Choice

StarzPlay’s tech powers India’s Lionsgate Play
Provides managed services, developed by StarzPlay, to mark successful launch of Lionsgate Play ...
Subscribe to Digital Studio Middle East
The latest editor of Digital Studio Middle East, the foremost and most respected platform for ...
Dubai's Faisal Hashmi working on feature film
10 short films in 10 years has earned Faisal Hashmi international plaudits at film festival, ...

Don't Miss a Story