VSTa related app encap discussion
These are more of my thoughts about app-encap that started during a discussion
on the VSTa mailing list.
From jeske@... Wed Dec 2 10:58:34 1998
Date: Wed, 2 Dec 1998 10:58:34 -0800
From: David Jeske
To: vsta@zendo.com
Subject: App installation and encapsulation (Was: Tandem Expand)
Message-ID: <19981202105834.N28651@home.chat.net>
Mail-Followup-To: vsta@zendo.com
References: <19981202000003.A21860@gerf.org> <000f01be1e01$d8d37590$577670c6@endymion.eps.inso.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.94.13i
In-Reply-To: <000f01be1e01$d8d37590$577670c6@endymion.eps.inso.com>; from Gavin Thomas Nicol on Wed, Dec 02, 1998 at 09:41:29AM -0500
Status: RO
Content-Length: 6594
This email has turned into yet another 'os philosophy' discussion,
more than a discussion about VSTa specifics. However, so far it seems
that people here are interested in this sort of thing.
On Wed, Dec 02, 1998 at 09:41:29AM -0500, Gavin Thomas Nicol wrote:
> > I have a qustion. Should we have some sort of VSTa name service? I mean,
> > a lot of the problems involved in using purely symbolic names for things
> > is getting a consistant and expectable result every time.
>
> Well, I was just installing Office 2000 yesterday, and what David
> Jeske said hit me in spades as I watched the install process.
I'm glad you agree.
> We all know the problems caused by windows install programs that
> install things all over the system because they need to live in
> "special" places in order for things to work well (just try
> installing things in non-default places and watch what
> breaks). Worse, uninstall doesn't always clear everything that was
> installed off the machine, so you often get to a point where your
> disk is half full of crud that you don't need, but are not sure that
> you can delete.
Yes, but I think you should ask "why are their special places?". IMO,
the problem is that current single heirarchy filesystems are a blank
untyped slate within which applications create their own add-hoc
storage conventions.
> I think this is a clear example of why we need per-process maps and
> mount overlays... with these, the programs, at starup, would simply
> make the namespace looks like they need it to be, but have the
> actual *installation* of the program be purely local. Uninstall
> would be a snap.
I believe we need to think long and hard about this. How is the
application going to create the per-process maps? If the application
is just accessing absolute names in the namer, and mounting them into
it's pathspace, we have not improved anything.
The idea of putting devices and files into a heirarchy is (IMO) mostly
a convinence to human users. There are much more powerful storage
mechanisms which applications could use just as easily to store data
(i.e. databases, or even flat resource stores with unique
names). However, these storage mechanisms don't translate as well into
something a human user can 'browse'.
A better solution (IMO) is to recognize that the heirarchy (and mount
tables) need to exist for the human interactive mode, and perhaps for
little scripts the operator writes, but that the resources themselves
should exist in a different shape.
For example, we might let an application get an opaque handle to the
'root' of it's files but not connect that to other files via some
add-hoc heirarchy. If a human user wants to muck with the internal
files of an application, let him mount it into his add-hoc pathname
space to browse it.
For example, I assert that it's a better solution for an app to do
something like:
int node_id = open_my_instance_resource(PRIMARY_DATA);
int my_icon = open_resource(node_id,"myicon.tiff");
int my_data = open_resource(node_id,"somedata/my_data.dat");
Than to do something like:
mount_my_instance(PRIMARY_DATA,"/appdata")
int my_icon = open("/appdata/myicon.tiff");
int my_data = open("/appdata/somedata/my_data.dat");
However... either is a much better solution than 99% of operating
systems are doing today, where app resources are installed into add-hoc
locations at install time.
====
I'm going to go into more detail about the suggestion above that I
don't like (i.e. mounting an apps primary data into some fixed
location) both because it's the less radical one, and because it'll
highlight some of the problems to be solved.
We can improve app installation with this mode, but the single
heirachy filesystem gets in the way. For example, we could make an
add-hoc convention for use of the raw FS. This would have applications
installed into instance IDs instead of some developer dictated install
location:
/@apps/0/app_export # a text file describing the app info, name, etc
/myicon.tiff # app icon
/somedata/my_data.dat # app data
/my_app # app binary
/@apps/1/app_export
/bin-i386/netscape # i386 binary
/classes.zip # default java classes file
/icons/netscape.ico # netscape icon
/icons/html.ico # html file icon
/icons/tplain.ico # text/plain icon
/icons/jpeg.ico # jpeg file icon
/users/jeske/@apps/0/... # private app for user jeske
The app_export would contain information like:
# app_export for netscape
APP_ICON = netscape.ico
APP_BINARY{i386} = bin-i386/netscape
APP_FILETYPES = { { type = "text/html", icon = "icons/html.ico",
actions = { "open", "edit" } },
{ type = "text/plain", icon = "icons/tplain.ico",
actions = { "open" } },
{ type = "img/jpeg", icon = "icons/jpeg.ico",
actions = { "open" } }
}
Then when an app (or a command line tool) was launched, we would mount
it's datafiles into a standard location (like /__appdata), and it
could access them all from there.
The problem here is that the "@apps" stuff is just add-hoc convention
stuffed into an untyped heirarchy. (that is, if someone just created a
directory called "@apps", it would be indistinguishable from our apps
directory above) In addition, if we are storing data in the above
manner into the raw-filesystem, then we have to write code to special
case things like the private user app repository, or a group
app-repository. Of course we can use something like union mounts to
push this into a process local pathnamespace, but any decision about
which mounts to make are again, just add-hoc.
The world is a much nicer place if we have a multi-heirarchy, where we
can express the physical storage location as one heirarchy, the
ownership as another, and the identity information as another, and all
of them can be rigidly typed items:
physical location ownership identity class instance
fs/root:home/jeske/my_app users.jeske app/my_app 00
fs/root:home/jeske/netscape users.jeske app/netscape4.05 01
fs/root:apps/gcc users app/gcc2.7.0 02
net/share:apps/gimp users app/gimp 03
That way, when we want to do something like "build the list of
installed applications, we merely list the "identify class" = "app/*".
I think I've gone on long enough about this, thoughts?
--
David Jeske (N9LCA) + http://www.chat.net/~jeske/ + jeske@...
From jeske@... Thu Dec 3 01:39:39 1998
Return-Path:
Delivered-To: jeske@...
Date: Thu, 3 Dec 1998 01:23:50 -0800
From: David Jeske
To: vsta@...
Subject: Re: App installation and encapsulation (Was: Tandem Expand)
Message-ID: <19981203012350.P28651@home.chat.net>
Mail-Followup-To: vsta@...
References: <19981202000003.A21860@gerf.org> <000f01be1e01$d8d37590$577670c6@endymion.eps.inso.com> <19981202105834.N28651@home.chat.net> <19981202184842.A565@gerf.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.94.13i
In-Reply-To: <19981202184842.A565@gerf.org>; from The Doctor What on Wed, Dec 02, 1998 at 06:48:42PM -0600
Status: RO
Content-Length: 5436
Lines: 113
On Wed, Dec 02, 1998 at 06:48:42PM -0600, The Doctor What wrote:
> > For example, I assert that it's a better solution for an app to do
> > something like:
> >
> > int node_id = open_my_instance_resource(PRIMARY_DATA);
> > int my_icon = open_resource(node_id,"myicon.tiff");
> > int my_data = open_resource(node_id,"somedata/my_data.dat");
> >
> > Than to do something like:
> >
> > mount_my_instance(PRIMARY_DATA,"/appdata")
> > int my_icon = open("/appdata/myicon.tiff");
> > int my_data = open("/appdata/somedata/my_data.dat");
>
> I have a question then, does the first example imply that
> "my_instance_resource" is some sort of registry? I'm not keen on that
> unless this registry is very dynamic, strongly typed, network transparent
> and has various data integrety verification methods.
The implementation sitting on the other side of the API is
irrelevant. The important part is that the application isn't
responsible for composing the path, because if it is, then it just
encodes more information which only has meaning by 'convention'.
I assume by registry you are making reference to something similar to
the windows registry. My biggest problem with the windows registry is
that items in the registry have no connection with their 'owners' on
disk. Such that you can lose registry information, or lose file
information, and there is no way to know that you lost one and not the
other. I think we are both thinking along the same lines about where
this kind of information needs to be stored.
> > However... either is a much better solution than 99% of operating
> > systems are doing today, where app resources are installed into add-hoc
> > locations at install time.
>
> So I have a question: To what level do we want a user to be able to
> browse all the structures? Should there be something like a mount point
> (aliased server, namer location, whatever) that can give you access to all
> the functions, from the shell (i.e. from the VFS)?
> VSTa$ /lib/printf ( "%s", "Narf?" )
It's interesting that you bring this up. Most of my thoughts have been
to go away from the non-typed path space and instead rely on stronger
typed structures.
Picture this stark difference between binding done with shlibs, and
binding done with file resources on UNIX.
With an shlib call, the app explains which version of the shlib it
expects the call to come from. Even if you don't have the exact
instance he wanted, you know what he was expecting and can provide a
work-alike.
When a file resource (say /etc/fstab), you don't know which version of
/etc/fstab the app was expecting to see. If somewhere in between when
the app was compiled and now we changed the layout of /etc/fstab, one
would think it wouldn't be too hard to just emulate the old layout for
the old apps. Except that with open("/etc/fstab"), no information is
conveyed about which "/etc/fstab" version the application expected to
see.
So to answer your question, binding libs (or anything) into the
pathname space, is fine if you have some reason you want to do so, as
long as when you write some perl script or something you have the
system preserve information about exactly what instances/version you
were expecting to be bound into the pathname space so that later it
can be recreated without cracking open the head of the person who
wrote the script.
> Frankly, I love it. Having a strongly typed FS/resources would be a
> beautiful thing (IMO). In addition, to owner, there should be an ACL too.
> This would make life really nice.
I prefer to just let you store arbitrary 'dimensions' of information
on top of resources. I think it's important to establish a system of
ownership, unique naming, and translation between these extended
attributes, but to dictate them ahead of time just walls us into more
static solutions.
> Imagine taking a scsi disk from one scsi controller and putting on it
> another (i.e. change it's physical location) and have the whole thing just
> move cleanly (after all, its identity "storage/Chris' scsi root" would be
> unchange!).
Exactly.. or better yet... imagine installing a 1gb disk and having
the system just say "you have 1gb more space" instead of requiring you
to deal with manually installing items in different physical
locations. Of course information about what physical location(s)
something is stored in should be available, but it isn't necessarily what I want to center around all the time.
> I think process space should be broken in this way too, after all,
> everything in a microkernel is a userspace process. That'd allow
> something like Tandem's Expand.
I don't understand what you mean here... can you elaborate?
> Oooh.... I like. I must be my C++ programming roots saying "Whoohoo!
> Strong Typing!"
I'm just as much a dynamic typer at heart. However, I recognize that
some items inherently are static and some items are inherently
dynamic, regardless of whether you treat them as such.
Much of my desire to treat static requirements as static typed items
is that I don't like the idea that every open("/etc/footab") function
in a program is a point of failure. I'd prefer to be able to static
check all the static requirements of a program beforehand to assure it
can run. That way, the program only has to do complex error checking
and handling for those things which it ADMITS are dynamic cases.
--
David Jeske (N9LCA) + http://www.chat.net/~jeske/ + jeske@...
From jeske@... Thu Dec 3 10:18:17 1998
Return-Path:
Delivered-To: jeske@...
Date: Thu, 3 Dec 1998 09:59:52 -0800
From: David Jeske
To: vsta@zendo.com
Subject: Re: App installation and encapsulation (Was: Tandem Expand)
Message-ID: <19981203095951.S28651@home.chat.net>
Mail-Followup-To: vsta@zendo.com
References: <19981203010644.O28651@home.chat.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Mailer: Mutt 0.94.13i
In-Reply-To: ; from Gavin Thomas Nicol on Thu, Dec 03, 1998 at 06:31:07AM -0500
Status: RO
Content-Length: 1957
Lines: 55
On Thu, Dec 03, 1998 at 06:31:07AM -0500, Gavin Thomas Nicol wrote:
> Ok.This is a bit different to what I was talking about. What you are
> proposing here is more or less that resource resolution be a function,
> rather than being statically resolved. I agree.
Correct.
> However, in the above function, you *still* have to bind to something,
> and have a way for naming that something, and again, I contend that
>
> /app_resources/ja/ui/menu
>
> is as good as a GUID.
If you want a standard heirarchy to store your applications private
information, that's fine. I don't think we should dictate this for all
apps, because some apps will want RDBMS storage, or simple record
storage, or BSP, or some other storage type.
However, I still contend that we're better off naming the "app
resource instance" by using a collection of fields in a flat table,
and using something analagous to a GUID for uniqueness properties.
For example, in this case we'd be storing a flat collection of
heirarchial "mini-spaces", one per app. So instead of letting the app
arbitrarily place itself inside the physical namespace:
/usr/local/netscape/ja/ui/menu
We place it's "mini-space" inside the flat logical namespace:
Instance ID Category ID Name Ver Mini-Space Name/type
I-1 C-1 Netscape 4.03 MS-1/HEIRARCHY
I-2 C-1 Netscape 4.05 MS-2/HEIRARCHY
I-3 C-2 Yellow Pages v1.0 MS-3/DB
And it's local resources can be stored within the mini-space with a
standard heirarchy if the author so choose like:
Mini-space contents: MS-1/HEIRARCHY
ja/ui/menu
en/ui/menu
Mini-space contents: MS-2/HEIRARCHY
ja/ui/menu
en/ui/menu
Or if it suited the author's purpose better purpose better:
Mini-space contents: MS-3/DB
table "listing" columns = name, address, phone
--
David Jeske (N9LCA) + http://www.chat.net/~jeske/ + jeske@...