|
Ash Nallawalla talks about some massive Network Storage gear — this is equipment
the big players use. |
Network Appliance (NetApp) pioneered the concept of the "network appliance", an
extension of the industry trend towards dedicated, specialised products that
perform a single function. NetApp storage and content delivery platforms (Filers
and NetCache appliances) are coupled with powerful content distribution and
reporting software.
NetApp filers are large network-attached storage (NAS) devices that have been
used for ambitious, data-rich projects like the NASA Chandra X-Ray project,
which includes the orbiting telescope, observatory and data centre. The
observatory accumulates massive volumes of data with each experiment, so the
multiterabyte NetApp F840 was chosen to support the complex data used for
scientific research.
Key features of NetApp storage appliances include:
- Built-in RAID for protection from data loss due to disk failures.
- Hot spare disks for fast failure recovery.
- Redundant hot-pluggable power supplies and cooling fans.
- Battery-backed non-volatile RAM for guaranteed writes.
Lord of the Rings
Modern, special-effect movies are using increasing amounts of computing power.
Weta Digital is the visual effects facility of Weta Ltd for Peter Jackson's
three-film screen adaptation of J.R.R. Tolkien's The Lord of The Rings for New
Line Cinema. See http://www.lordoftherings.net.
|

Lord of the Rings — where the storage requirements are
measured in millions of Megabytes. |
Weta Digital uses SGI IRIX OS-based Octane visual workstations, Onyx2
visualisation systems, Origin family servers, and SGI Linux OS-based visual
workstations and servers. Weta's two divisions worked closely together to
provide characters, creatures and environments of Middle Earth. Large volumes of
data require very fast (Gigabit) networks, so Weta uses Network Foundry's
Bigiron line of switches.
Two parts of the trilogy have been released and the third film, The Return of
the King, is expected in December 2003. I asked Harry Christian, Marketing
Manager, Network Appliance ANZ to give me some "numbers" so that we could get a
feel for the size of the task. He provided the following answer.
Each frame of the movie requires 12.5 MB of storage and there are 24 frames for
each second of the film. For a three-hour film, that equates to 3,240,000 MB of
storage. Weta used two Imagica scanners to scan in each film image at 200
horizontal lines of resolution, and scanned 1.5 million frames (or about 18 TB
of images) as live frames prior to adding special effects. NetApp filers store
the prework files, which are then made available to artists at Weta Digital. 300
artists then use the filers to add special effects for environments, creatures,
and other effects (such as motion).
Each shot can take from two to eight weeks
to complete and the shots stay live while they're being worked on. Weta Digital
uses anywhere from 1400 to 1600 processors in the facility to complete their
work (Intel Xeon 2.2 GHz) and to do rendering day and night every day of the
week. It took 40,000 vendor hours per day or 4 million work hours total to
render the special effects for the Two Towers film, giving the NetApp systems a
huge and intense workout. Scott Houston (Weta CTO) noted that he needed not only
huge data capacity, but also massive processing power.
Primary storage for the solution was NetApp filers - three F840s and five F880s,
with one R100 used as a buffer store for the first scans. The total storage
infrastructure also uses SGI Origin 2000 systems with 3 TB of storage and tape
libraries, as well, and is about 240 TB; 120 TB being made redundant for safety.
Only 20 TB of data were available to artists at any one time. Currently, Weta
Digital is managing 40 million files for both films 1 and 2, but the file
management is handled by SGI DMF software.
More information about NetApp filers is available at
http://www.netapp.com. |
 |
About the Author
Ash Nallawalla is the Honorary Editor of PC Update and consults on CRM and lead
management topics.