You're the Boss with UBOS

by Doc Searls

on February 3, 2015

UBOS is a new Linux distro that I like for two reasons. One is that it works toward making it easy for muggles to set up their own fully independent personal home servers with little or no help from wizards. The other is that it comes from my friend Johannes Ernst.

It's in beta at the time of this writing, and it runs on PCs, Raspberry Pi models B and B+, and on Macs in VirtualBox. It's been tested with ownCloud, Known, WordPress, Mediawiki, Selfoss, Shaarli and Jenkins and plugins for a number of those.

Says Johannes:

My goal is to make the administration of personal servers 10x easier for users, and also to make it much easier for developers to create "personal data" Web applications that don't spy on us and get them deployed. Over time, we will get UBOS to as many pieces of hardware as possible, and pre-install lots of "personal data"-related middleware. We're already working with two Internet of Things projects to get UBOS to be the OS they run as the default on their hardware. Imagine if all the IoT products you bought were "indie" and not tied to some corporate overlord's take-over-the-world strategy? To do that, we need to make administration easier.

Here are some of the ways that's done:

Single-command deployment of Web apps, with automatic database provisioning, Web server configuration and so on—including SSL setup.
Full virtual hosting—for example, you can run two instances of WordPress with different plugins and one of ownCloud at http://personal.example.com/blog, http://home.example.com/news and http://home.example.net/owncloud on the same host.
Single-command undeployment.
Single-command full system upgrade, which backs up all your data, upgrades all code from the operating system over middleware to applications, runs whatever data migrations might be necessary and redeploys all your apps.
Single-command backup and restore of all or part of the apps installed on the same host.

Phil Windley is the creator of picos (persistent compute objects, or virtual things) and CloudOS (an operating system for picos, the subject of "The First Personal Platform—for Everything" in our October 2013 issue), and my guru on what I'll call the Internet of Personal Things. When I asked him for his take on UBOS, he wrote this back:

There's a healthy sorting out happening in the "personal cloud" space, around devices, sensors and other "things" that people will use—and around personal control and ownership as well. Picos/CloudOS are about providing specific services in a lightweight way for things that are all under a user's control. Picos allow personal data to remain personal. So the first way picos and systems like UBOS are related is in the heads of people realizing they need to be in control of everything in their life: words, personal data, things or whatever.

More technically, UBOS and picos could interact in the following ways:

1) You could host your picos on UBOS on your own server.

2) You could monitor and control them from such a platform.

3) UBOS would be a great place to run your personal (UMA-based?) authorization server.

To unpack this the rest of the way, I interviewed Johannes.

DS: According to DistroWatch, there are hundreds of Linux distros already. Why one more?

JE: There is a hole in the market, and it's at home. UBOS focuses on making the administration of home servers much simpler, which is not something most distros pay much attention to.

The timing is also right. First, hardware has become cheap and powerful enough to run Web servers at home. Many people now keep their calendars, personal photos and other important private data away from the major cloud providers, because they would rather store it on hardware they have control over. Second, developers of cool IoT projects almost always need a Web server, so they can open their garage door, set their thermostat or water their plants over the Web. While many of those are $35 Raspberry Pis, they still require the same amount of maintenance as any big Linux server used for business purposes.

With UBOS, we set out to make administration of these kinds of personally owned servers at least 10x faster and easier, so more people can run their own servers at home, build more cool IoT projects and keep their data private and at home, without turning home server maintenance into a full-time job.

DS: "10 times" faster system administration is a big claim. How do you back this up?

JE: Think of what you need to do to install a Web app, such as ownCloud, on your server. This usually involves:

Finding and downloading a tar file.
Full virtual hosting. For example, you can run two instances of WordPress with different plugins and one of ownCloud at http://personal.example.com/blog, http://home.example.com/news and http://home.example.com/owncloud.
Provisioning a MySQL database.
Editing application configuration files and fixing permissions.
Setting up logging.
Restarting servers and so on. Installation of a single app can easily take a few hours and usually requires real Linux admin knowledge. We think it should be easier, so more people can do it, faster, and more reliably.

In UBOS, you say sudo ubos-admin createsite. UBOS will ask you a few questions, like the desired virtual hostname or whether you want SSL, and then UBOS does all these steps for you. Total time consumed: maybe a minute.

This already works for a number of Web apps, including ownCloud, WordPress, Mediawiki, Selfoss, Shaarli, Known and even Jenkins where UBOS automatically sets up an Apache reverse proxy to Tomcat.

When it is time to upgrade, server administration is even more time-consuming, and don't we all want to keep our Web sites patched for security reasons! You need to do all of the above, but also back up data before (don't forget any place the app might have written valuable files!), and perhaps manually run data migrations. Some apps like WordPress have made it easy, but most have not. In UBOS, it's simply: sudo ubos-admin update. And you can relax while UBOS does all of the above, for all the apps you have installed at all of the virtual hosts on your personal server.

DS: What changes did you need to make to the Web apps you list so they could be administered by UBOS?

JE: So far, we haven't had to fork any of them, so it's been pleasantly simple. It's mostly adding some JSON metadata to the application package and parameterizing configuration files so UBOS can "edit" them correctly. Sometimes some additional scripts are required, but rarely. All the current apps are on GitHub at https://github.com/indiebox.

DS: Wasn't one of the existing distros close to being what you want to give the world with UBOS?

JE: UBOS is a derivative of Arch Linux, which we rely on heavily. You can think of UBOS as a subset of Arch, plus UBOS administration tools and the additional metadata and conventions required to make ubos-admin work.

We inherit many of the great qualities of Arch—for example, it is a rolling release distro, so we never have to impose major upgrades on the user; packages on Arch are never very much out of date, because we do want to run the latest and greatest, and the Arch community and its wiki are simply amazing. UBOS takes many of the Arch packages even without recompiling.

We started out by simply adding our administration packages to Arch, but because we want to make sure that apps always install correctly and upgrade correctly, we need to control when upgraded packages are made available to servers. So it was the needs of the quality assurance process that caused us to create a new distro with separate repositories.

DS: How, besides this interview, are you going to attract hackers to jump in and improve the code?

JE: We just published the very first UBOS beta, for x86_64 and the Raspberry Pi, so we are only at the beginning of this. The next steps are the inevitable bug fixes, easier networking setup, perhaps more hardware platforms if users want that and more apps.

But then, we're looking to work with developers of Web applications who want to make it easier for their users to install their apps, and to developers of Internet-of-Things hardware who want to make it easier for developers to get Web applications running and maintained on their hardware. We all have the same goal here—make home servers easier—and we invite anybody who shares this goal to help make it so.

You might also be a really frustrated home user who's had it with with fixing /etc/i/forget/what Saturday afternoons and would much rather help fix the problem once and for all.

DS: What do you mean by personal servers? I can guess, but I'd rather have you tell me. And how are these different from personal clouds, stores, lockers and vaults, except in the sense of where they live?

JE: All these, in my mind, describe a different part of mostly the same elephant. A personal server, for me, is a computer that is primarily accessed over the network, but it is one that I have root access to and nobody else does. (Or at least that I can have root access to if I want to, and I can take away root access from anybody else.) Such a personal server can be a physical piece of hardware or a virtualized one in the cloud. It's different from a non-personal server in that I use this server for my own, personal purposes and for those of my family, but not for setting up a Web site for the general public.

A personal cloud, a personal data locker or vault would be one kind of use of that personal server. Another use would be to control one's home appliances. So many kinds of personal applications have their natural home on a personal server, because by its very definition, nobody gets to touch the data on that server other than the owner.

DS: Why just Web apps?

JE: Package managers are great at helping administer most code on a Linux box, and UBOS heavily relies on pacman. But they are not able to administer Web apps because they do not (and cannot) help with Web server configuration, database configuration and so forth. ubos-admin sits on top of the pacman package manager and adds that additional layer of automation.

DS: What kind of "intelligent Internet of Things devices" are you talking about?

JE: There seem to be four types of IoT "devices" emerging:

Sensors, for example, thermometers, often with very low power requirements.
Actuators, for example, relays to switch on something.
Cloud servers.
Higher-powered devices in the home that connect the Internet to the sensors and the actuators. For many hobbyists, that often is a Raspberry Pi.

This last category of devices is often powerful enough to run Web applications. It turns out that for many home IoT applications, there is no need to run Web applications in the cloud at all. So why not turn these devices into devices that run the Web apps directly?

This has the added advantage that our personal IoT data stays private in the home, and can't be abused by a cloud provider.

DS: What's the end state of the "Internet of our own things"? Or at least a state to which Linux Journal readers can aspire—and weigh in with some code and product?

JE: As geeks, and as an industry, we have a choice to make here about how we'd like the Internet of Things to look ten years from now.

There's the "NEST way", which is totally closed devices 100% dependent on being tethered to some corporate overlord. Google in this case. Some people consider these beautiful-looking devices to be nothing else than surveillance devices first and thermostats second. I guess they have a point. In any case, we have no control whatsoever over their functioning and terms of service, nor what they do with our data.

And there's the user-owned, free/libre, "indie" way, where every device is—at least in principle—hackable. Where data stays home, or at least is only shared with anybody because the owner of the device decided that that is what they wanted to do. Where we can run the code we want on whatever device we buy or build. Where interoperability is not subject to whether some big companies love or hate each other this week.

Personally, I'm horrified at the first of these scenarios, and I know many other geeks are too, because all these great IoT projects keep popping up everywhere. The next step is to make this all reliable and maintainable enough that it's not just the geeks who can control their own homes. UBOS is one step in that direction, but many others are required. In fact, we might need a real barn-raising effort to together create an "Internet of our own things". Anybody up for that? I am!

LJ: Us too.