G-Tech White Paper: Lucas Gilman: How a Pro Manages Storage Workflow

There’s only one thing Lucas Gilman dreamed of becoming more than a world-renowned extreme sports photographer and filmmaker: He wanted to be a father. In recent years, he’s become both, but balancing the two remains his biggest challenge yet. Gilman logs more than 200 days each year on the road, and every day away is a day missed in 3-year-old Landon’s life. No one appreciates the direct correlation between workflow and family more than Gilman. Every hour not spent mired in redundant steps and poky transfers, or recovering from an unexpected problem, is an hour gained with his family.

Clearly, there are elements of Gilman’s craft that can only be refined through practice, study, and innate talent. The way he embraces light or turns motion into texture or finds ways to make certain colors burst from images – these things contribute to Gilman’s art. But the business of being Lucas Gilman, or any creative professional, lives or dies based on workflow.

Too many amateurs learn about workflow the hard way. Hard-won content vanishes in a moment of data loss or device failure that could have been prevented with better backup practices. Editing drags on for days upon days because of sluggish hardware, antiquated applications, and paltry pipelines. Fortunately, improved storage can remedy many of the flaws in a poor creative workflow.

Surrounded by one of the first pre-launch Mac Pro workstations seeded into the professional community plus a slew of G-Technology storage devices, Gilman sat down with us to detail his own workflow processes so that others might skip a few of the heartaches he faced along the way.

G-DOCK ev
2TB Thunderbolt RAID
configured with 2 1TB G-DRIVE ev

Special Limited time offer! Buy 1 G-DOCK ev and get two extra G-DRIVE ev drives FREE. That's 4 drives total. Extended thru 4/15/14!

The G-DOCK ev with Thunderbolt storage system comes fully loaded with 2 stand-alone USB 3.0 G-DRIVE ev portable drives. And for a limited time you can get 2 additional G-DRIVE ev drives FREE. That’s right. Double your G-DRIVE ev investment at NO EXTRA COST - at your time of your purchase. (That’s a retail value of $1,149 for just $749!)

$749.95
with FREE Thunderbolt Cable

G-Technology G-DRIVE ev
USB 3.0 Drive, 1TB
For use as a spare drive for G-DOCK ev or as a standalone USB 3.0 storage solution
$199.95

G-RAID with Thunderbolt

G-RAID with Thunderbolt, designed specifically for professional content creation applications, features the most powerful, flexible I/O technology ever. At double the data transfer rates of USB 3.0 and over 12 times faster than Firewire 800, no other PC I/O interface can match the raw speed of Thunderbolt now available in G-Technology's leading RAID solution.

4TB $599.95 with FREE Thunderbolt Cable

8TB $799.95 with FREE Thunderbolt Cable

Now Shipping! G-DRIVE ProThunderbolt storage system with extreme transfer rates up to 480MB/s

G-DRIVE PRO
with Thunderbolt

G-Technology's G-DRIVE PRO allows you to finally experience the full speed of Thunderbolt technology. Sustained transfer rates are now up to 480MB/s! With dual Thunderbolt ports, this storage solution is designed for the most demanding applications and easily supports multi-stream workflows in all major formats that's 2K* in real-time out of a desktop box!

G-Technology has built its reputation and business on serving creative professionals and prosumers that demand performance, capacity and quality. To continue its tradition of serving these markets, G-Technology has optimized Thunderbolt technology on its product line. Leveraging the strengths of our Enterprise storage experience, our AV expertise, and the capabilities of Thunderbolt, G-Technology is delivering the future of storage now.

Zones and Stages

While some of the following may seem obvious to experienced shooters, Lucas Gilman emphasizes that it’s important to identify the major components of creative workflow in order to map the right tools to the functions within that workflow. Consumers have a bit more leeway here. For example, casual photographers may be satisfied with their computer being the live working drive for editing images, with one external drive serving as their backup target. The “prosumers” and professionals we’re addressing in this article should more cleanly compartmentalize their workflow into zones, for reasons that will soon be obvious.

By zones, Gilman means “in the field” versus “studio,” and the distinction helps him establish priorities in his equipment.

“If I’m trekking through the jungle for days, getting eaten alive by bugs, or hauling a crew across Iceland in the sub-zero wind, I need small, light portability a lot more than top capacity,” Gilman says. “Save capacity for the studio. out in the field, I want something that weighs nothing but will take a beating. For a long time, I used G-DRIVE® minis and slims for the speed of USB 3.0 and because they only weigh a few ounces. Now I use the [G-DRIVE®] ev units in the field for the same reasons, but they add the convenience of the dock later in the workflow.”

The three basic stages of Gilman’s workflow span both zones, and every stage requires careful consideration.

Stage 1: Capture & Field Backup

Just as implied, capture encompasses recording live images as both raw video and still photographs. Gilman encodes to SanDisk Extreme Pro SD and CF cards. While on-set or at the event scene, he takes the first opportunity to copy files through his MacBook Air or MacBook Pro and out to two external drives — the previously mentioned G-DRIVE ev cartridges — via USB 3.0 or a G-DoCk ev™ with Thunderbolt™. This copying to a hard drive happens throughout the shoot, not just at the end of the day or the event. If those drives fill up, he brings in another two and keeps going.

“Each member of the drive pair is then geographically separated at the end of every day. I’ll usually give one to my assistant. If we go to dinner, I get robbed, all my stuff’s gone, at least my assistant has the extra set. They’re basically never in the same place at the same time. Meanwhile, I also have the original master on the CF or SD cards as a third backup.”

One advantage to using the G-DoCk ev system in the field is that it can be accompanied by automated copying software, such as Gilman’s currently favored ShotPut Pro. With this setup, there’s little to do besides connect the camera’s storage media to the system and get busy with other tasks. Software automatically and simultaneously clones the cards’ contents to two ev drives, at Thunderbolt-driven data rates. Because the G-DoCk ev requires AC power, it may not be appropriate while you’re neck-deep in nature, but the compact dock can help make short work of in-the-field backup in a hotel room or even in the shooting-crew vehicle.

Stage 2: Ingest & Backup

Ingest refers to importing media content into the primary workstation for subsequent editing. In some cases, ingest may occur in the field, as when a client requires shots or clips ready for use the next orning. More often, though, ingest occurs back in the studio or office. With potentially terabytes of HD video crossing the wires, this step can take several hours.

“When I ingest, I want to do it as quickly as possible so I can get to the editing,” Gilman says. “The more time it takes me to get everything into the mothership — the Mac Pro, right now — the longer I’m not having coffee with my wife or playing with my kid.”

To shrink his waiting time, he makes the G-DoCk ev part of his ingest process. In fact, his ingest involves two G-DoCk enclosures, through which he can pour the contents of four ev field drives into his Mac Pro workstation via the Thunderbolt interface. Cloning software then duplicates all the ev drive data to three sites, which we’ll describe according to their function:

Live work. This drive houses the media files referenced continuously by the workstation’s editing software (Final Cut Pro, Aperture, Photoshop, et al.). The current Mac Pro model only integrates up to 1TB of internal SSD storage, which doesn’t come close to being sufficient for HD video work or large photo libraries, especially after accounting for space lost to applications, system overhead, cached data, and so on.

The trend in storage is for "scrath" (live work) capacity to be on external drives, which is why the MAc Pro offers six Thunderbolt 2 ports. Until very recently, Gilman used a pari of 8TB Thunderbolt-enables G-RAID drives striped together (RAID 0) for his live work volume.With striping, he could achieve 660MB/s of thoughput, which he needs for handling high=-bandwidth video, such as multiple HD, 2K or 4K streams.As of this writing, Gilman has started to evaluate the new G-Drive PRO Thunderbolt which will reach up to 480MB/s as a single unit. He expects the PRO units to become his new live work target going forward.

Live work backup. Any work done on the live work drive must be backed up. Any creative professional knows projects can require anywhere from hours to years of ongoing edits, and if the editor decides that a version from three changes ago is preferable to the current version, that past iteration needs to still be available.

Performance is less critical than capacity in live work backup storage. Rather than striping two drives in RAID 0, for instance, one might mirror two drives into a RAID 1 or opt for a configurable two-drive enclosure, such as G-Technology’s G-RAID® with Thunderbolt™. As a general rule, Gilman recommends having a live work backup volume with at least twice the capacity of the live work volume. This will provide more space with which to maintain historical file iterations. He also encourages users to invest in a capable synchronization application, such as ChronoSync, in order to keep the data in live work and live work backup constantly in tandem. At a minimum, Gilman has ChronoSync run every night.

Long-term storage. No matter how much precaution is taken, there is always a risk of data becoming corrupted or lost. Many professionals, including Gilman, like to keep a RAID-5-protected, completely hands-off backup of their original source content as a long-term storage solution. With this approach, there is always a pristine copy of all content close at hand in case, as Gilman says, “everything done after ingest somehow goes south.” Incorporating RAID 5 provides a balance of performance and data protection, so backup operations remain quick and data remains safe against the chance of internal drive failure over time. Gilman uses a four-bay G-Technology G-SPEED® eS PRo stocked with 4TB drives, providing 12TB of data-redundant storage. The mini-SAS-based enclosure attaches to his Mac Pro workstation via an ATTo ThunderStream storage controller.

Stage 3: Distribution

With ingest done, the project proceeds through editing until at last it’s ready to go out into the world. This could be in the form of final proofs for the client, or dailies for a director. No matter what format the project is in, it must be distributed to others outside the studio.

“At this point,” Gilman says, “for video, I’m going to want to output a ProRes 4444 master along with an H.264 for client review, and possibly other formats for different devices, such as a tablet or a phone. I’ll usually output four different file sizes. You do the same sort of thing with photos, providing high-resolution TIFFs and a couple sizes of JPEGs.”

While storage does get used as an output target during this stage, most of the bandwidth burden falls on the primary computing resources, especially the central processor and RAM. This is the other category in which the Mac Pro shines brightest, fueled with CPU options ranging up to the 12-core Intel Xeon E5 2.7GHz. The faster the workstation, the quicker those transcoded media files will be ready for sending to clients.

“Big pipes in storage and processing allow for seamless edits,” Gilman says. “No slow-down, no dropped frames, no waiting to render, no go get a cup of coffee and come back in 15 minutes every time you do a minor change on the timeline. That includes everything from color correction to special transitions to speeding up or slowing down clips in the timeline.”

Some photo projects are small enough to be sent over FTP or a cloud-based file service, but Gilman almost always sends videos out on a G-DRIVE ev or G-DRIVE mini. After so much work, the last thing the wants is for the drive he sends to not create a positive aesthetic impression, or arrive damaged.

From Flow to Finish

Because an increasing amount of Lucas Gilman’s business revolves around HD video, his storage workflow reflects a priority on very high throughput. Users who concentrate more on still photography, for example, would not need to stripe live-work drives; one target drive would usually suffice. An exception might be in an environment where multiple photographers are shoveling content into an editing pool throughout the day, such as a newspaper editorial office.

Another facet of Gilman’s workflow involves cloud-based backup. With terabytes of data piling up in his studio, it would be impractical to try saving everything across the Internet. Instead, Gilman views the cloud as basically a “best of” repository where he keeps his portfolios, recent takes, and choicest clips. While this approach isn’t as comprehensive as rotating out a periodic, complete backup to an off-site location, duplicating the top 5 or 10 percent of content to the cloud has the advantages of easy convenience and fairly low cost.

Off-site storage remains a troublesome workflow bottleneck for professionals such as Gilman, but at least it sits at the periphery of everyday operations. The core bottlenecks in traditional project pipelines, typically focused around ingest and having sufficient bandwidth for processing multiple high-def streams on a single timeline, can now be greatly minimized.

“With the Mac Pro and these latest G-Technology Thunderbolt-enabled drives,” Gilman says, “I can do things that simply weren’t possible before. When I would go out with two or three guys on a job, we didn’t have spare staff to dedicate to standing around for things like transcoding. I couldn’t have managed the load of multiple 4k cameras out in the field. But now I can. This new workflow opens up loads of new opportunities — not just in what I can offer to clients, but in the ways I can be creative and widen my artistic vision.”

That’s Lucas Gilman the professional talking. On the other side of the coin, though, storage workflow also impacts Gilman the family man. Yes, his business focuses on capturing the unique moments in life, but at some point he needs to transition from capturing moments to participating in them. Minutes matter. By constantly striving to improve his workflow, Gilman gets more of those minutes back and regains some of those 200 days for sharing with the people he loves.