Houdini: Magic Doesn't Just Happen
We all know about those movies with the great special effects: The Matrix, Armageddon, The X-Files Movie, Godzilla, Lost In Space. The list goes on and on. Up until now, most of those effects have been done on SGI workstations. SGI has been a leader in this field for many years. However, in the past few years, a migration to low-end, low-cost, Intel-based PC hardware has occurred. This migration has included Windows, the only OS considered to work well on those Intel desktop computers. That situation is about to change.
Side Effects Software (http://www.sidefx.com/), a graphics software company founded in 1987 in Toronto, Canada, is porting its only product, Houdini, to Linux. This is big news in the graphics world. Houdini is the software used to create many of the stunning effects in the movies I just mentioned. To those in the graphics arts world, this is the equivalent of Oracle, Corel or Lotus porting to Linux. It gives credibility to the platform in an area where it had been mostly a curiosity.
Linux has had a major role in a few movies in the past. Darryl Strauss' article (LJ, February 1998) about Digital Domain's Linux render farm for Titanic proved that, but Linux hasn't been used as the primary graphic artist's platform for these movies. Like so many other industries, the film and video industries have been using Linux as a server, a box off in the corner happily crunching numbers or dishing out files over high-speed networks. Now, Linux gets to take center stage on the desktop.
Side Effects' Houdini product is a complete 3-D solution, encompassing modeling, compositing, lighting, particle systems, texture management, rendering and animation features. Currently in its third release, Houdini was one of the first modeling and animation products to adopt a procedural approach to 3-D. Recently, Houdini received an Academy Award for “Technical Achievement”, presented to four of the company's original developers. Used by companies like Digital Domain, Blue Sky/VIFX Studios and Centropolis Effects, Houdini is a heavyweight in the film industry.
Previously, Houdini has been supported only on SGI Irix workstation class systems. They recently announced support for Windows NT. The port to Linux comes at a time when many hardware vendors are looking for reasons to bring their high-end graphics cards into the Linux fold. Paul Salvini, Director of R&D at Side Effects Software, talks of it as the “chicken and egg” problem:
Doing a product like this for Linux required hardware acceleration to make it really viable, but hardware acceleration often requires applications in order to warrant drivers to be written. From a graphics workstation point of view, Linux isn't ready. There haven't been that many proper drivers for hardware acceleration for OpenGL under Linux. There are a number under development, but as far as state of the art, there aren't that many in production. The reason is that there aren't any applications pushing the need for these drivers. The applications want the drivers; the drivers are looking for applications.
Linux is a viable, popular, rendering platform. Many houses that formerly used SGI servers for rendering are moving to, or at least considering, low-cost Linux platforms for their rendering farms. This might be due to their familiarity with UNIX or concerns with Windows NT stability or even if they just prefer not to have a mixed UNIX/NT environment.
With Linux, most of the current software development is driven from the bottom: what happens in the kernel drives what happens with applications. Side Effects thinks, for the high-end graphics market at least, this needs to be a bit different. Houdini is an application, with low-level needs that aren't quite available the way Side Effects wants them. Salvini said:
What will happen is the missing pieces underneath will just naturally fill in—development is being driven from the top of the software chain. This is an exciting change in the way development happens for Linux.
Side Effects wanted to do their part to move things forward by providing the application. By providing a Linux solution, they felt they could provide their customers with the best of both worlds, low-cost Intel-based platforms and the familiar UNIX environment. Plus, having the kernel source offers the effects houses, which tend to have very technical needs, the opportunity to make modifications they might not otherwise be able to perform.
Part of the reason they considered doing the port was the interest shown by a number of effects houses for a Linux-based Houdini. The interest was “provisional”, considering the lack of workstation-class hardware support (i.e., the OpenGL hardware acceleration), but Side Effects took these requests to heart and felt they could help move the process along by providing the application side of the equation.
Although they didn't have any actual customers signed up for the port—just customers showing interest in its possibility—and even though no satisfactory 3-D hardware acceleration was available, Side Effects felt they had to move forward with the project. Mr. Salvini told me:
We did this as much because we felt it was the right thing to do as anything else. To sit around complaining that no one has developed proper hardware acceleration is kind of hypocritical if you're not providing the application that will take advantage of it. You've got to put your money where your mouth is and do the work.
Before starting work, Salvini said Side Effects spoke to Darryl Strauss at Digital Domain, Xi Graphics and Metro Link about the hardware support that might be forthcoming from the X server vendors. Initially, they started with XFree86, then moved to Xi's AcceleratedX to make use of support for overlay planes, something neither XFree86 nor Metro Link supported at the time. The overlay planes allowed them to speed displays by not having to continually redraw sections of the screen. The dependency on Xi's server is temporary, however, since both the other vendors are working on overlay planes, plus Houdini doesn't have a huge need for them. In fact, Side Effects may provide Houdini in a version that works without overlay planes. The drawback will be a hit in performance, but the program will still be fully compatible with the version supporting overlay planes.
Salvini couldn't talk about which 3-D cards they were looking at, due to non-disclosure agreements. He did say that on the NT port they had worked with IBM and Intergraph, 3-D Labs, Diamond Multimedia and Accel Graphics. Intergraph provides their Intense3D, also known as the Wildcat 4000 card, for the IBM Intellistation. He said response from all these vendors was very good for that port. Since all of these card manufacturers have shown varying interest in the Linux environment, one can only imagine what, if any, their involvement with upcoming hardware announcements for Linux might be.
Beyond hardware acceleration, other issues which the porting team had to deal with are easily recognizable to anyone who has done UNIX-to-UNIX porting in the past: sockets, shared memory work, clone vs. fork/vfork. These are the typical issues encountered when porting applications from IRIX to another UNIX platform (or vice versa). Linux, for all practical purposes, fits into the UNIX marketplace quite well in this respect.
As for doing the actual work, Side Effects was able to make use of all the tools they already had. Salvini told me:
The good thing about Linux was that the tools we did need—for debugging and memory checking, for example—were available for free and we already had a good idea what was going on there.
Even so, it apparently took quite a while to get all the graphics pieces working together, since they “were always downloading the latest bit of this or that.”
Salvini wouldn't term it “an easy port” from the SGI version, due to some tricky things they do concerning performance issues, but he did say it took two engineers about five weeks to do the work: “less time than moving to NT, but it was still longer than we had hoped.”
The port was actually done as a fun thing at first, and they'd figured it might be something they could finish in a weekend. We both laughed when I pointed out that, even with Linux, such a thing as a “weekend port” was still a bit of a Holy Grail for most packages.
At this point the main graphics work has been completed, but audio and MIDI support has not been done. MIDI is a fairly small portion of the product, but they'd still like to finish it sometime soon. Unfortunately, UNIX audio interfaces are fairly non-standard. The developers are looking at the Open Sound System as a possible way to standardize audio ports, but no decision in that arena has yet been made.
The Linux port is actually ready to deliver, minus the small audio issues, but they're waiting on a proper license manager. Houdini will go to beta when that tool is ready. Due to the bleeding-edge nature of this project, they don't feel they can use any of the available license managers, so they may go with one built in-house.
The lack of hardware acceleration on the Linux box means it does not compare to the IRIX boxes for performance. In terms of everything else—rendering, simple shading, wire frame preview—Linux stacks up fairly well. Salvini said:
It's surprisingly good without that one piece. I can't wait for that final chapter to close, for then it will be an amazing platform.
Houdini pushes OpenGL pretty heavily, and as Paul pointed out:
We'll be a pretty good test of OpenGL for vendors on Linux.
Many special effects shops use multiple packages for their work, one for modeling, another for animation, another for effects and another for compositing. Ultimately, you'd want to see all of those tools for Linux. The toughest one will be something like 3D Studio Max, because it has been designed specifically for the Windows platform, unlike many other tools that have their roots in the UNIX world. However, tools like 3D Studio Max are important for the graphic arts world on Linux because they're mid-range tools, where many people get their start. SoftImage and Alias/Wavefront will be easier ports, because they are used to the UNIX and X environment. SoftImage is owned by Avid, but neither the SoftImage nor the Alias products have been talked about lately for the Linux platform.
Many hardware vendors are aware of Side Effects' port, but Paul is under non-disclosure with those vendors and couldn't talk about their plans. Most of the software vendors who have talked to Side Effects have asked them about the port—they want to know what Side Effects knows that they don't. Most of this has to do with questions relating to hardware acceleration. That's the key for which many application vendors are waiting. Like the public, those other vendors will have to wait until the workstation, graphics card and X server vendors start to make their own announcements.
The developers and management at Side Effects have talked about open source quite a bit, says Salvini:
There was a great article on it by someone at Netscape about the business case for Open Source and how you might make money with that.
They had never had Open Source, but this got them tossing the idea around a bit, albeit without releasing all the bits and pieces of Houdini. The concept raised a lot of “what if” questions, but so far it hasn't gotten much past that point. They are obviously intrigued with the idea, however.
Side Effects does offer source access in a traditional sense with a developer's kit allowing access to header files and libraries, but the core remains proprietary, partly due to their concerns about maintaining product consistency. They do believe they offer extensibility with the plug-in interface. Salvini says that with high-level applications like Houdini, access to the core code is not as vital as access to the operating system source.
Paul doesn't think anyone at Side Effects actually uses Linux at home, although there may be one guy who has it. In fact, he says most of their developers don't even have systems at home. Apparently, they have lives when they leave work. I had to wonder a bit what that might be like.
Although the Houdini port to Linux is big news for graphics nuts like me, don't expect to run out and pick up a copy of Houdini off the shelf at CompUSA. A single-seat license for Houdini runs about $17,000 US, with the developer's kit running an additional $4000 US. This is very high-end software for professional organizations.
Considering this high price, I asked Paul about his take on a more affordable modeling package that has developed a big following—Blender. He replied:
You can't compare the two. They really aren't in the same class of product. I didn't play with [Blender] long enough to do it justice, but obviously its price point is such that people will be able to take advantage of it just from a personal development level, i.e., an experiment in playing with graphics tools. You don't do that with a $17,000 product.
Unlike Blender, Houdini is a product that needs a bit of serious commitment from the user in order to warrant its expense.
However, the news of the Houdini port is important more for what it represents, the credibility it gives to Linux in the desktop graphics marketplace, than the product being produced. Salvini was right when he told me,
This article won't be about trying to sell Houdini to the readers—it's not priced for the average user—it's about how Linux is developing in the high-end, graphics workstation market. Side Effects has been pushing the hardware vendors in this direction quite a bit and I feel we'll be seeing some great success in the next year, to bring Linux into the graphics workstation market, which it really hasn't been to date.
He's right. Now that Houdini has made the jump, things truly are about to change.
