It doesn't take too much work to remember the dot-com era, back when you could get financing for a company that did almost anything. At the beginning of that period, when the Internet was becoming a mainstream medium, there was a lot of talk about on-line communities. These sorts of communities weren't new to internet veterans, who had been participating in Usenet long before the Web appeared on the scene. But they looked like an excellent opportunity to the venture capitalists and entrepreneurs, who saw on-line communities as a potentially huge market.

As was the case with many such ideas, everything was great except for the business model. Many thousands of on-line communities exist today, all of which make it possible for people from around the world to share information and ideas on a topic. Although few sites have managed to build businesses around such communities, there is no doubt that such software is a vital part of today's web infrastructure.

Creating an on-line community using a relational database and a programming language is not difficult—but creating your tenth web/database user-management system in as many months is annoying for the developer and expensive for clients. Moreover, what happens when a site wants to add new functionality? It would be nice if the new features and fixed bugs could be reflected in all of your sites, rather than only the one on which you made the changes.

Philip Greenspun, author of the wonderful Philip and Alex's Guide to Web Publishing, realized all of this back in the mid-1990s, when marrying the Web and databases was still a relatively new idea. His solution was to create a set of applications that took into account the needs of as many on-line communities as possible. The software toolkit he created formed the basis of his doctoral thesis at MIT. It was also the basis for ArsDigita, the web consulting company he founded. When the ArsDigita Community System (ACS) was finally released, Greenspun made it available under the GNU General Public License (GPL).

Like many web consulting firms, ArsDigita never quite lived up to its promise. After several years of meteoric success, investors were brought in to expand the company further. Greenspun was forced out; the company released two half-baked versions of ACS (including one in Java that essentially rewrote the entire system); most of the staff was laid off, and finally, Red Hat (which was backed by the same investors as ArsDigita) hired a handful of programmers and bought the company's few remaining assets.

If ArsDigita had been a proprietary software company, then this would have been the end of the story. But because ACS was licensed under the GPL, the community took over where the company left off. More significantly, the community already had been working on a version of ACS, known as OpenACS, that would use the PostgreSQL relational database rather than the ACS default, Oracle. (This article assumes that you will want to use PostgreSQL; the instructions are only slightly different if you wish to use Oracle.)

OpenACS 4.5, as the first production release was labeled, was released in June of this year. It was renamed the “Open Architecture Community System” to reflect the fact that ArsDigita is no more. But Greenspun's influence is profoundly felt in the OpenACS community, and the wealth of code and documentation written by ArsDigita employees have helped propel the project forward.

This month, we begin an extended look at OpenACS, which is one of the more powerful (if relatively unknown) open-source web toolkits available today. In coming months, we will look at how to install OpenACS, how to use its templating system to create dynamic pages and how to create sophisticated on-line communities with limited code and administration.

It's easy to say that OpenACS is a toolkit for creating on-line communities. But what does that mean? For starters, it means that OpenACS comes with working versions of most of the applications you're likely to want on a community web site. It handles user registration and administration, forums, FAQs, groups (including a rich permission scheme), news updates, file storage and distribution, personal home pages, surveys and a web-based calendar. As you might expect from a modern system, administration of the applications is done almost completely through the Web, with only a few configuration files.

An experienced developer probably could write some or all of these applications within a few weeks or months. But why reinvent the wheel? Moreover, OpenACS is built on collective experience gained from building such communities, which is reflected in the sophistication of the data model and applications.

From a developer's perspective, OpenACS provides a database designed for the creation of new, integrated applications. This data model actually is the most important part of OpenACS. Although you could rewrite the software for another database (as has been done with PostgreSQL) and even use a language other than the default Tcl, the data model is where most of the system's smarts lie. OpenACS provides Tcl and Pl/PgSQL procedures that make it easy to work with the data model.

Because OpenACS relies so heavily on a relational database, it is important that access to the database be efficient and flexible. For this reason, OpenACS installations almost always use AOLserver (introduced in last month's installment of At the Forge), instead of the more popular Apache. Because AOLserver uses multiple threads inside of a single server process, it can provide a shared pool of database connections. (Although there is fairly strong allegiance to AOLserver within the OpenACS community, I would not be surprised if the introduction of multithreading Apache 2.0 eventually leads the project in that direction.) And while AOLserver provides its own database API, OpenACS provides a number of higher-level procedures that make it extremely easy to work with a database.

If you plan to work with only a single brand of database, then you can use these procedures directly inside your Tcl programs to store and retrieve information. But to ensure that all applications will work on all platforms, OpenACS encourages developers to place all of their database queries inside specially formatted XML files (with an .xql extension), with each file corresponding to a single database. When a Tcl program invokes a procedure to send a query to the database, the OpenACS “query dispatcher” opens the XML file for the currently configured database, reads the query and sends it to the database. An OpenACS system written in this way should be able to switch from PostgreSQL to Oracle merely by changing the top-level configuration file,

As we saw last month, AOLserver comes with its own templating system, known as ADP, that makes it easy to mix server-side programs with static HTML on a single page. Of course, this means that designers and programmers often are fighting for control of a single file, so designers must know which sections of a page to avoid. OpenACS thus includes a new, more advanced templating system that breaks each page into two parts: a Tcl page that sets variables and an ADP page that retrieves those variable values. This approach is similar in some ways to Zope Page Templates (ZPT) and Enhydra's XMLC.

OpenACS may sound extremely complex, but it actually consists of only four parts: 1) an installed PostgreSQL or Oracle server; 2) AOLserver, compiled with nsxml (the XML-parsing module for AOLserver), the PostgreSQL and/or Oracle driver and an appropriate configuration file; 3) the OpenACS data model; and 4) OpenACS Tcl libraries, Tcl pages and ADP pages.

Until version 4.x, installing OpenACS was straightforward. You installed PostgreSQL and AOLserver, loaded the OpenACS data model using the psql command-line client, copied the OpenACS libraries, Tcl pages and ADP pages into the appropriate directory and began to use the system.

But this approach led to a number of problems, most of which stemmed from the installation's inflexibility. What if I want forums under two different URLs, rather than the default /bboard? What if I want to install only two or three packages, rather than all 40? What if I want to upgrade the e-commerce solution, without also upgrading the FAQ system?

The solution to this problem is the ArsDigita Package Manager, introduced in ArsDigita's ACS 4.x and adopted by OpenACS 4.x. Each application has been renamed a “package” and consists of a data model, .xql files, Tcl libraries, Tcl pages and ADP pages, as well as documentation. Each package can be installed at any number of URLs on the system and can have arbitrary permissions associated with it (using the system of users and groups that are central to OpenACS). Each package also may define one or more parameters, giving custom information each time it is instantiated.

If you install OpenACS in /web/openacs4, then the www directory contains all of the top-level Tcl and ADP pages, the tcl directory contains the top-level Tcl libraries, and the packages directory contains each of the packages loaded onto the system.

Once a package is in the filesystem, you can use the OpenACS web-based installer program to create the package-specific data model in the database. Finally, the site administrator makes the package available by mounting it (one or more times) via the administrative site map. Once mounted, an application is available via the named URL.

Now that I've explained much of the theory behind OpenACS, we're ready to install it. Note that installation is relatively complex, because it involves a number of packages that must be installed with particular ownerships and permissions. The OpenACS 4.5 installation process is smoother and easier than previous versions, but it is still surprisingly easy to make a mistake along the way.

Before beginning, make sure that PostgreSQL 7.1.3 is installed, including the server, client and development libraries. The newest version of PostgreSQL (7.2) has a few subtle incompatibilities with older versions that can make it difficult to install OpenACS. While there are certainly a number of improvements in 7.2, you should be fine using 7.1.3.

Also, make sure you have installed libxml 2.x; on Red Hat systems, you should make sure the libxml2 and libxml2-devel RPMs are both installed. Without these, OpenACS won't be able to open the .info package associated with each file, as well as the .xql files used by the query dispatcher.

Next, install AOLserver, ensuring that /usr/local/aolserver is owned by the nsadmin user:

# mkdir /usr/local/aolserver
# useradd nsadmin
# chown -R nsadmin /usr/local/aolserver

Rather than installing the stock version of AOLserver, applying the patches that ArsDigita and the OpenACS crew have written over the years and separately downloading the PostgreSQL and XML-parsing modules, you should download the all-in-one version known as “Matt's AOLserver distribution” (see Resources).

Next, create a PostgreSQL user named openacs4, giving it full privileges. (PostgreSQL has its own list of users that is independent of the UNIX list.) In general, such activities must be performed as the postgres user, rather than as root:

# su postgres
# createuser openacs4
  Shall the new user be allowed to create
    databases? (y/n) y
  Shall the new user be allowed to create
    more new users? (y/n) y
  CREATE USER

Now, use this new PostgreSQL user to create a new database, which we will call openacs4:

# createdb -U openacs4 openacs4
  CREATE DATABASE

# mkdir /web
# cd /web
# tar -zxvf /tmp/openacs-4-5-release.tgz
# mv /web/openacs-4 /web/openacs4
# chown -R nsadmin.nsadmin /web

Once you have made these changes, you should be ready to start the system. As root, start the server in foreground mode:

# cd /usr/local/aolserver
# ./bin/nsd -f -u nsadmin -g nsadmin \
  -t openacs4.tcl

You should see a lot of debugging information on your screen, scrolling by faster than you can possibly read. When the scrolling stops, you should see something toward the end that looks like this:

[22/Jul/2002:15:13:41][23316.1024][-main-]
  Notice: nssock: listening on 127.0.0.1:8000
[22/Jul/2002:15:13:41][23316.8201][-nssock-]
  Notice: nssock: starting
[22/Jul/2002:15:13:41][23316.8201][-nssock-]
  Notice: nssock: accepting connections

The final page welcomes you to OpenACS and informs you that AOLserver has been terminated. (This is necessary, so as to load all of the newly installed Tcl library files into AOLserver's memory.) Restart the server, point your browser at http://localhost:8000/ once again, and OpenACS will be ready and waiting for you.

At this point, OpenACS is fully functional, but given that none of the packages have been installed, it won't do very much for you. In next month's At the Forge, we'll take a look at how to install and manage OpenACS packages.

There's no getting around it: OpenACS is a complex beast. Although the software is generally excellent, it requires an experienced UNIX/web/database hacker to use and modify it. Even the installation procedure is long and complicated, and I can assure you from personal experience that it's often hard to understand where you have made a mistake. The documentation is improving, but there are many gaping holes and difficult-to-understand table structures that can be confusing.

As if that weren't enough, the code isn't completely finished in many places. Yes, the fact that OpenACS is open source does mean you can fix things yourselves. And the community is generally quite open and generous, giving help to those who are trying to get started with it. But it's frustrating to hear constantly that the packages you need are almost ready or that someone expects to finish with them at some point in the future. I'm not averse to helping improve open-source projects, especially when it benefits me (and my clients) directly, but many small annoyances can add up quickly.

Given these complaints, it might seem absurd to think that I endorse OpenACS at all. And indeed, it probably will take some time for the dust to settle and for all of the necessary improvements to be made. But there's no getting around the fact that OpenACS provides a much richer infrastructure for creating on-line communities than anything else I've seen. The included applications might not work completely, but they work pretty darned well overall, and provide most of the functionality that my clients need, right out of the box. Finally, a number of commercial consulting companies, several universities and one or two dozen independent consultants are working on improvements and extensions to OpenACS that promise to make it more robust and featurefull than it is today.

If you're creating an on-line community, and you're not afraid to get your hands dirty with Tcl and SQL code, then you should take a serious look at OpenACS. This month, we considered the overall structure of OpenACS and saw how to install its various elements. Next month, we'll look at how to install and manage the various packages that come with OpenACS, so that we can put together a custom community site that includes only those programs that we really need.

Resources