App-Wrappers Yet Again

From jeske@... Fri Aug 17 16:06:52 2001
Date: Fri, 17 Aug 2001 17:42:08 -0700
From: David Jeske 
To: =?iso-8859-1?Q?Stig_Hackv=E4n?= 
Subject: Re: lynx vs links
Message-ID: <20010817174208.H17932@mozart.chat.net>
References: <20010817153532.A5002@hackvan.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
X-Mailer: Mutt 1.0pre3i
In-Reply-To: <20010817153532.A5002@hackvan.com>
Status: RO
Content-Length: 13555
Lines: 263

On Fri, Aug 17, 2001 at 03:35:32PM -0700, Stig Hackvän wrote:

unrelated parts of this email deleted

> i've been having a lively email exchange about the rampant cluelessness in
> linuxland today...with the author of
> http://hackvan.com/pub/stig/etext/thomas-scoville--the-elements-of-UNIX-style.txt
> which i suppose you've probably already seen...

I hadn't seen it. Interesting read. Whomever said to him "It's arcane
and needlessly complex." was in complete agreement with me, but not in
the way he proposes in the article. 

In fact, it ended up inspiring me to write this monster editorial on
the subject. Let me know what you think. Perhaps I should try and get
it posted on Slashdot.


UNIX is needlessly complex when it comes to configuration of
everything. Right down to the basic /etc/fstab, everything in unix is
configured with an ad-hoc text file usable only by knowing a
smattering of different 'conventions' for different unicies. It's
/etc/vfstab here, /etc/fstab there. Sometimes it has 8 arguments per
line, sometimes six. The source devices for the line are often as
simple as /dev/hda1, but sometimes as complex as /dev/sd/0/0a/01. All
just to figure out that the same partition the BIOS loaded the kernel
off of is in fact supposed to be the root partition.

However, all of UNIX's power and all of UNIX's warts are all nothing
compared to the mistake that is killing UNIX's chance at ever becoming
a desktop rival for Windows. UNIX has always been based on the idea of
a single centralized authority distributing a set of software bound
together by THEIR decided conventions. From when the first Linux
distribution rolled off the block, to today's RedHat, you pretty much
don't install an application on UNIX unless it is either (a) designed
for the specific distribution you are using, or (b) you compile from
source. The reason this model is such a killer, is that IT DOES NOT
SCALE. 

To contrast. Any old fool can use a compiler for windows, zip up the
result, and hand it to anyone with a windows box. Chances are, it'll
run on Win95,98,NT,2k, and XP. The irony is, the biggest reason
something like that dosn't normally work on UNIX is NOT because UNIX
is missing the hundreds of thousands of man hours of compatibility
test that Windows has, although it is. The biggest reason that's
almost never possible on UNIX is because somewhere deep in the heart
of that first experience with /etc/fstab, UNIX teaches you that it's
condoned, in fact, it's correct, for you to compile that static
/usr/local/lib/myapplication.datafile path right into your executable.

The free-UNIX community had a chance to fix this very recently and
they blew it. Just like the original UNIX wrote the blueprint for tens
(or hundreds) of thousands of open-source developers to churn out
Linux, FreeBSD, OpenBSD, and who knows what other full-on-clones of
UNIX -- Nextstep recently solved this problem of merging the UNIX
philosophy of hardcoding paths with the need for 3rd party software
kits, and they solved in a most beautiful way. It was there for the
taking. All the Gnome or KDE people had to do was repeat the same
success Linus did by following the path, instead of building their
own, like they both so fatefully did. The amazingly elegant solution
tha those crafty NeXT engineers came up with is commonly called
"app-wrappers" and it's fortunatly alive and kicking as part of the
the new MacOS X.

NeXTStep, like today's Linux or FreeBSD systems, is a combination of a
graphical desktop with a UNIX core. Sure, it's Display PostScript
instead of X. Sure, it was a Mach/BSD kernel instead of Linux or
FreeBSD. However, none of that is terribly important. 

The developers of NeXT decided to keep UNIX underneath the way it was
meant to be underneath. The core system was UNIX. There was a command
line terminal to make even the most dedicated of xterm devotees
proud. However, they needed to make the system ready for third-party
developers. Those third party developers would most likely be
developing graphical applications. However, more importantly those
third party developers would be shipping their applications DIRECTLY
to customers. They wouldn't be handing their application source to a
handful of representatives, each from a different camp of
'distribution builders'.

The first thing they did was build an amazing development environment,
cornerstoned on the famous "InterfaceBuilder". IB was far more than an
old-school resource-editor. However, it did generate resource-like
files which contained information about the GUI windows of the
application. In the early days these were stuck right into the
executable themselves. In fact, the application icon, supported
filetypes, and any icons for supported filetypes were all right in the
executable. Whenever your app needed something, it just use a library
call to find its own resources. Beautifully, this library call did NOT
take a path of any kind. This assured that everything your app needed
was rolled up all in one location-independent file. The library dealt
with figuring out where your application had been launched from, and
digging the resources out of it.

*** I REPEAT ***

There were no hard-coded /usr/local/lib/myapp directories to deal
with. There were no ${GNOME_ENV}/share/myapp.ico files. If you had
images, or window resources, or anything they rolled it right up into
the executable. Whenever you wanted something, you just asked the
resource finding library call for it.

Sure, you're saying, that dosn't work for big apps. If there is a big
app, it would end up with a 200 meg executable. You're right, putting
everything right into the executable image dosn't scale up. However,
they got the philosophy correct right from the outset. They knew that
an end user should be able to drop an executable on a machine,
ANYWHERE on a machine, double click the sucker and be off and
running. The application icon should just work, the app should just
work.

In case you are wondering, the answer is "no", these were not the
famed app-wrappers. We're not quite there yet. Just like UNIX set down
a foundation of philosophies with /dev entries, command line tools,
and /etc configuration files, it's important to understand how
Nextstep laid a new philosophy of it's own. The philosophy started
before app-wrappers and was demonstrated in a much simpler way with
Next-defaults. We'll get back to app-wrappers in a minute.

So now you've got your application and icons all wrapped up in a
single executable. However, every application needs to store settings
and preference data. On Windows at the time it was going in
c:\windows\myapp.ini. On the Mac it was going in the system folder. On
UNIX, applications were free to combine some smattering of /etc,
/usr/local/etc, and dot-files in the user's home
directory. Next-defaults replaced all that, and did it by bringing
together the best ideas from all of those solutions.

Next-defaults was a simple API which allowed you to read and write
key/value preference data. There are implementations of similar APIs
available today for UNIX, such as libPropList. However, the genius in
Next-defaults was that it too had no hard-coded paths. NONE.

When an application asked for preference information, it always
provided a default value itself to the library call. This assured
there was no need to install /usr/local/etc/myapp.base.config. When an
application needed to store a new user-choosen preference,
Next-defaults squirreled it away in a dot-directory in the user's home
directory. If a system administrator wanted to provide a default value
for all users, he need only enter that value into the Next-defaults
for the root user on the machine (and make sure they were publicly
readable). Beautiful, no hardcoded paths, no mess. Pretty much like
today's open-source configuration systems for Gnome and KDE right?
WRONG.

No part of the Next-defaults api ever let you provide a path. They
were saying, "Hey, on NeXT, you don't need to worry about that, we'll
take care of that path stuff for you". Both Gnome and KDE allow the
application to provide directory the pathname of the preference
information directly. Both Gnome and KDE need default preference files
information installed into the system-wide install directories. I can
not grab a Gnome or KDE application, untar it into my home directory,
and expect it to work.

Next-defaults was a simple little system, but it was the philosophy
that it demonstrated that made NeXTstep so elegant. That philosophy
said "applications shall be completely location independent". I hear
some of you in the peanut gallery saying that Gnome and KDE have
already done this. Again, WRONG. 

Gnome and KDE both have the concept of changing the base location of a
KDE or Gnome install by setting an environment variable. That is VERY
different from building applications to be _completely location
independent_. On NeXt, one could just drop an application in their
home directory, find it in the file browser, and poof, the application
icon was there -- double click on it, and poof, it's running. As long
as the system could find it, it's filetypes, icons for filetypes, and
application icon all worked correctly because those were all contained
WITHIN the app.

Now we're back to the 200meg executable problem and the magic of
app-wrappers. Developers started to write larger and larger apps. In
fact, apps being written by the developers at NeXT started to get
larger and larger. They wanted to handle internatilization. They
wanted big, beautiful, 24-bit color icons all over the place. The
"everything piled into the executable solution" had to go. Thus, they
came up with app-wrappers. 

An "app-wrapper" is a directory, labeled with the extension ".app",
which has an executable inside it with the same name. That's it. 

Whenever the file-manager saw one of these, it transparently hid the
contents of the app-wrapper directory. It found the executable inside,
and opened it up to find the application icon, still stored directly
in the COFF executable format. It also could dig in and find the
supported mimetypes, icons for each supported mimetype, and a list of
"Services" supported by the application. [You'll have to go dig
elsewhere to learn about NeXT Application Services] All of the
window-resources, text-strings for all languages, everything was all
wrapped right up in the app-wrapper.

Now the philosophy of NeXT is really beginning to shine. You know how
it was the application got at all that data which was buried inside
the application directory? THE SAME WAY that it had got at it when it
was inside the executable directly. By asking the resource finding
library call to dig up a given resource. Right there all this magic of
not letting applications suppily paths to these calls starts to pay
off. 

NeXT also allowed shared-libraries with associated data-files to be
wrapped up into a similar directory wrapper called a
"bundle". Assuring that things such as screensavers, image format
handlers, and other plug-in code could have all the same wonderful
location indepdent properties.

A few OS releases later, NeXT even extended wrappers to the
installation of "library plus header" development kits by making a
wrappered item called a "framework". If it had existed at the time,
all you would have to do was drop "Gnome.framework" into the right
directory, and all the necessary development headers and libraries
would be found by the compiler. 

Seems somewhat anti-climactic dosn't it? Seems like we should just rip
this concept off right now. Seems like it would pave the way for
applications to work easily with both KDE and Gnome, by having their
filetypes and icons inside themselves in a standard format. Seems like
it would pave the way for third parties to ship GUI applications for
Linux without worrying about whether you're RedHat or SUSE, Gnome or
KDE. Seems like we should all follow Linus' example, clone something
that already demonstrated how to make this stuff work, so we can move
onto the next problem. We should.

For more information about app-wrappers, or how we could start having
KDE and Gnome use them right now, check out:

 http://www.chat.net/~jeske/unsolicitedDave/next_wrapper_description.html
 http://www.chat.net/~jeske/unsolicitedDave/kde_encap_ideas.html

-- 
David Jeske (N9LCA) + http://www.chat.net/~jeske/ + jeske@...