In the dark basement of a once-glorious Hollywood dance hall, Greg Mueth?s camera team prepares to resume rolling on a new reality show from Discovery Studios. The old dance floor is now flanked by four cameras shooting compressed HD. Three more cameras keep running non-stop in the makeup trailer on the curb. Sitting at a folding table thirty feet away from where the show?s host cracks jokes in the spotlight, Mueth points out two squat silver boxes, each the size of a short stack of paperbacks and adorned with ?G? logo on its side. Connected to each box via a Thunderbolt? cable, two MacBook? Pros bottom-light his stubbled cheeks and enthusiastic grin.
?You don?t understand how different this is from how it?s been,? Mueth whispers as he nods at the computing gear. ?We shoot four episodes a week, a full show per day. A finished show runs between 24 and 28 minutes. That?s edited down from an average of 220GB of footage across each of seven cameras ? four down here and another three getting candid conversation in the makeup rooms. We?re a small operation, but I?m sending up to 2TB of data to production each day. I?m the tech supervisor, so all the hours and dollars wrapped up in managing and processing that data? That?s on me. And if you would?ve told me a few years ago that we could handle that kind of data load on-set, I would?ve said you were crazy.?
Crazy is a matter of perspective. To prove the point, external storage manufacturer G-Technology offered to provide about a backpack?s worth of equipment to Discovery Studios. The proposal almost read like a studio pitch: Can just a bit of the right storage gear change a reality show?s workflow and bottom line?
Business as (Un)Usual
Greg Mueth has a background in IT. He understands the need for fast, dependable storage, data protection, and quality components that solve more problems than they create. Unfortunately, the people who provide budget and materials for shows often lack his experience. Too often, producers would hand him a stack of drives pulled off a shelf at the nearest consumer electronics shop. Buyers looked for high capacity and low price. On a good day, the drives might match. On an exceptional day, they might even sport a moderately fast interface, such as FireWire?. But usually, Mueth would receive a box filled with USB drives or, even worse, bare desktop SATA drives and a cheap USB adapter.
Such equipment might be tolerable in situations where time wasn?t money and environments weren?t cramped and chaotic. In show production, neither condition applies. Drives get knocked around in trailers. The vibrations of footsteps rattle loose power and data cables. Because desktop systems aren?t feasible in such quarters, crews have to use laptops, many of which only offer one or two high-speed ports, so there?s less ability to process external drives.
Mueth recalls one particularly grim assignment that showcased everything wrong with how many shows treat storage. It was in rugged Utah, 45 minutes from the nearest city. Producers gave him piles of bare 2TB hard drives ? in the dry desert ? and eSATA caddies for connecting to a PC. Mueth?s team bought cases and went through antistatic bags like plastic wrap at a picnic. Still, results were abysmal.
?We were on pins and needles and had to keep double masters of everything,? says Mueth. ?Of course, that meant double the cost. We had a PC that did nothing but make these copies ? and for good reason. There were constant drive failures. A couple went down from static. A couple got cracked PCBs. It was insane. We started off with 60 drives. We finished with 35.?
Fortunately, this Hollywood gig is a better job with better gear. In a Pelican case by Mueth?s feet, black foam encases storage assets that would have filled a data center rack ten years ago. The interior foam is cut for Mueth?s two G-DOCK ev? with Thunderbolt? enclosures, the small boxes now connected via Thunderbolt? interface to each of his two MacBook? Pros. Alongside these rest a line of thirty G-DRIVE? ev cartridges, each not much larger than a pack of cards, that slip into either of each dock?s two slots. Six of the rugged aluminum cartridges house 512GB solid state drives (SSD). The rest are high-speed 1TB hard drives. All of them can be USB 3.0 bus-powered.
Mueth grabs a couple of the ev hard drives and sets them atop the nearest G-DOCK ev? with Thunderbolt?, waiting for the first set of CF cards to come off the cameras with this morning?s compressed HD footage. The aesthetic match between ev gear and Apple?s notebook is not accidental, but Mueth wants ev storage more for its reliable function than its stylish form. Rather than replace dozens of drives across weeks of shooting, he will reuse the ev equipment, which was designed specifically for demanding creative professionals.
?Should I use a lower-end product and spend extra time or get the faster product because it saves money?? Mueth asks. ?Do we buy the cheapest thing to get through the show or something that will become inventory for many shows? Now we have the dock, and that?s not going anywhere. When the 2TB or 3TB G-DRIVE? ev SSD drive comes out, it?ll still work.?
Being a creative professional should not mean having to find creative work-arounds for technical problems. Before the Evolution Series, though, that was a large part of Greg Mueth?s job when managing storage. Laptop ports might not match his drive interfaces. He couldn?t daisy chain like he can with the G-DOCK ev? with Thunderbolt? interface. USB would run one way and FireWire? another. Bottlenecks, incompatibilities, and errors multiplied like paparazzi at an A-lister sighting ? never wanted and always in the way.In a sense, reality show workflow is dictated by data bottlenecks, a factthat becomes apparent when one follows footage around the job site. The only reason these bottlenecks and drive disasters don?t completely torpedo Mueth?s work is because of a system he and his crew use based on colored masking tape, camera coding, Sharpie markers, and a transfer log maintained by Mueth and his assistant. The system starts when CF cards or SSDs get loaded into a camera. (The four floor cameras use flash cards while the three trailer cameras feed to Sound Devices recorders using SSD media.) When a drive plugs into a camera, the operator covers the media with the appropriately colored and coded tape. When the drive comes out, the operator sticks the tape on the drive, where it stays all the way through logging during copying onto backup drives at Mueth?s laptop.
These data dumps happen periodically throughout the day. CF cards, each containing 30 to 40GB of footage, land on Mueth?s desk at the end of the morning?s shooting and copy out to two hard drives. One drive becomes a master that he keeps and the other a copy that gets hand-delivered to post. The G-DRIVE? ev SSD drives, which have been recording non-stop all morning, come in at the end of lunch. In order to process the trailer?s larger data set in a
shorter time, Mueth?s team copies these three to G-DRIVE? ev SSD backups. Until very recently, Mueth anticipated that time constraints would force him to send his G-DRIVE? ev SSD masters off to post without making a backup, a dangerous prospect that was ?a nightmare waiting to happen.?
Getting morning drives to post quickly is important, because it gives that group an extra half-day to process the content. Another set of drives comes in at the day?s end, and the process repeats.
By the fourth day of the week?s shooting, post should be done with the first day?s drives, allowing that set of drives to get wiped and recycled. Mueth notes that many outfits are now pushing for a two-day turn-around to keep inventory costs lower. Ironically, this trend conflicts with the purchasing of cheap drives, which typically use 5400RPM disks and slower interfaces, so transfers take considerably longer and delay end-to-end workflow. While bosses want quicker turn-around, Mueth says he?s seeing cycle times actually stretch longer with cheap gear, sometimes up to a week.
G-Technology?s Evolution Series has the potential to remedys much of this traditional waste. With 7200RPM disks in the G-DRIVE? ev modules able to transfer at up to 136MB/s (many conventional USB drives transfer at half of this rate or less), and the SSDs in the G-DRIVE? ev SSD modules able to transfer data at up to 480MB/s, the ev system makes quick work of what used to be tedious waiting. Mueth can have his SSD backups with almost no cost in management time.
?The ev rig is cutting our old workflow in half,? he says. ?This is a situation where every minute counts. Money aside, I still need those minutes to regroup my thoughts or deal with someone else?s fire.?
Some of these workflow savings are intangible. With fewer drive failures, there?s less wrangling over budget and less waste in sending staff to the closest store for a drive that doesn?t match the current setup. In other words, there?s less stress around the set. The Evolution Series cartridge system simplifies drive management and enables the entire production crew to standardize on processes that always look, feel, and operate the same. The cartridge that leaves Mueth?s G-DOCK ev? with Thunderbolt? plugs effortlessly into production?s identical G-DOCK ev? with Thunderbolt?. Almost immediately, this translates into less confusion and fewer mistakes. Of course, Mueth gets to measure the tangible savings of his new workflow every day.
The Big Reveal
?Cut!? calls the director at mid-day. ?Let?s break for lunch!? Greg Mueth nods as camera techs deposit their CF cards beside his MacBook Pro. ?Ready for a little before and after?? he asks. ?Keep in mind, before the G-DOCK ev? with Thunderbolt?, I?d see CF transfers average anywhere from thirty to sixty minutes. In a setup like ours, I?d have a DIT sit here for three to five hours doing nothing but transfer morning footage. Now...watch.?
To begin the copying process, Mueth inserts the CF card bearing the letter A on a strip of yellow tape into his laptop?s USB 3.0-based flash card reader. Imagine Products? ShotPut Pro software registers the new volume as containing 20.09GB. He then pops two ev hard drive cartridges into the G-DOCK ev? with Thunderbolt? he uses for this initial transfer. Each is labeled with a yellow piece of tape, one reading 1A and the other Master #4. ShotPut Pro recognizes both new drives. Mueth names each volume according to his content logging system and orders the software to start copying the CF card to both ev hard drives. He notes the process in his notepad log. The copying completes in 5 minutes 31 seconds. Out goes the first CF card and in goes the next. This one shows 24.5GB of data. It finishes writing to both drives in 6 minutes 1 second.
He fiddles with a calculator for a moment. ?That?s about 65MB/s. If I disabled the checksum, it would get a lot closer to its real maximum, but you always, always do the checksum. Those log files are critical in making sure that everyone in the production chain gets the exact data that was originally created. Nothing gets corrupted. Post can never say they didn?t get something. But if you can choose to deal with checksum overhead on either a buggy or a race car, why not pick the race car?? Driving his point home, one of the SSD packs lands on his desk. He clicks out the two hard drive cartridges from his G-DOCK ev? with Thunderbolt? and plugs two G-DRIVE? ev SSD drives in their place. The 139GB source volume transfers in 14 minutes flat, just over 165 MB/s, including the checksum.
Often, the camera techs need to run both MacBook Pro/G-DOCK ev setups simultaneously, particularly when making copies of the three recorder SSDs, so they can minimize the turnaround time.
Mueth?s expression takes on a tired gravity. ?When they call camera wrap at the end of the day, the media manager might wind up with six cards in his hand. That would mean two to three hours of overtime, every day. And if the media manager is stuck here, so am I. And the location manager. And the site rep. And security. If the lights are running from a generator, then we have to keep an electrician here. That?s double-time hours. Weekends are double-time all day. If we have a hard out on a location and run over, we have to try and buy out the people scheduled after us. Most days, I was paying around $2,000 in extra costs, all because we were chained to that god-awful workflow.? The irrepressible smile returns. ?Now
guess how much overtime I?ve paid since we put in the G-Tech ev system.?
Mueth holds up a hand with his thumb and index finger, forming a perfect zero.
?This is my dream rig,? he adds. ?I never knew that I could do this. This is goin
to permanently change how we produce reality shows.?