James:
I think the points raised in this case, that the SQLite community has
a good track record of interface stability is a good argument for
increasing its stability level.
That said, I have concerns with the suggestions that people are making
that *all* FOSS interfaces with the intent of stability should be at
least Uncommitted. Or, as David Comay said previously in this thread:
> But in general, *if* the intent is to provide a library which other
> consumers can use, IMO it should be Uncommitted or higher.
When we were struggling to determine proper interface stability levels
for the GNOME stack, we worked closely with Joe Kowalski to determine
our plan. With Joe's help, we decided that we needed the minimum set of
interfaces needed to write a good GNOME application needed to be
Committed. We also decided that other interfaces, even if they are in
the GNOME Platform and have ABI compatibility guarantees, should be
Volatile unless there is real evidence that users need or depend on
them. We determined the list by reviewing what actual 3rd party ISV's
(firefox, thunderbird, Adobe Reader, RealPlayer, gimp, etc) actually
depend upon. You can refer to the following case, where you can see
the following text in section 3:
This interface table was compiled with the assistance and approval
of Joe Kowalski.
http://sac.eng.sun.com/arc/LSARC/2005/734/opinion.txt
So, since then, our stance has been that interfaces should be
Volatile unless there is a real use case that indicates users
should require the interface.
If we are saying that SQLite has a great interface stability track
record and such use cases, and therefore should not be Volatile, then
this makes sense to me. However, if we are saying that all FOSS
interfaces with an intent to be stable should be Uncommitted or higher,
then this is a pretty big change in the way the JDS consolidation
approaches ARC. I guess we may need to discuss this further when the
GNOME 2.22 ARC case is submitted.
>>> You don't need to promise to boil away the ocean to see if there might
>>> be rusty nails and other bugs at the bottom.
>> I'm not sure it always makes sense to mark a Solaris interface as
>> Committed or Uncommitted just because an external community has the
>> "intent" for stability.
>
> So what do you make of the Sun projects that use these same commitment
> levels? Would you expect a Sun team to disavow support because
> they're publishing Volatile interfaces? Or that we provide some sort
> of special premium level of support for Committed interfaces?
I think Sun's stability levels are important. Sun adds value by
standing behind its interfaces.
That said, I think Sun needs to be careful about what interfaces it
marks as Committed. I think it makes the most sense for interfaces
that customers and users depend upon to be Committed. I do not think
whether or not the external module maintainers intend Stability should
be the driving reason.
> In fact, the two are just plain unrelated. The stability level
> communicates intent: it tells people when we will likely break things,
> and when we intend not to do so. We can't possibly prevent all human
> mistakes, and we can't predict the future, so it's not possible for us
> to say anything more than *intent."
In a sense I agree with you. In this discussion "track record" has
also been cited as a factor that should affect stability levels. A
project's future "track record" is also important, and it can be hard to
predict the future. Even projects with good track records can stumble.
Some modules are more risky than others, and some mistakes are
more expensive to fix than others. In addition to communicating
stability levels, ARC is about managing these risks.
> Whether and how a customer gets support is a separate business
> decision and a matter of appropriate documentation. It could be
> through some Sun-provided on-site team. It could be an on-demand
> service. It could be a third party with whom Sun contracts. It could
> even be some open source forum. Perhaps even "no support at all."
>
> I'm not buying the argument that "Volatile" can or should mean "we
> won't try very hard to support this." The ARC doesn't make or
> evaluate support commitments.
I agree with you. Using "Volatile" to mean "no support" is bad
practice. In much the same way as using "External" used to be
problematic. However, higher interface stability levels does affect
resourcing and our ability to support interfaces.
While it may not make sense, from an architectural perspective, to
worry about support commitments, I am sure you can understand why a
project team with limited resources might want to be conservative
about the interface stability levels it signs up to. I also think
it is perfectly reasonable for ARC to challenge proposed stability
levels when they are inappropriate.
>>> Or you (the consumer) may well be assuming a higher stability level.
>>> In other words, you know that SQLite is stable, and Sun has applied
>>> the wrong stability level. You chuckle at the "Volatile" label and
>>> plow on.
>> If I were a customer, I would understand that Volatile means that
>> Sun isn't going to support me if I have problems. If I know the
>> SQLite community will take care of me, then I might be agreeable
>> to not worrying about the lack of Sun support.
>
> That doesn't make sense here. Sun is providing the SQLite packages.
> There's no way that community can "take care of" anybody; it doesn't
> produce the signed SUNW* packages or patches. Sun itself will have to
> do that.
When I said "take care of", I meant the customer was placing their
trust in the external communities stability guarantee and track
record rather than in Sun's stability promise.
>> Most people who use free software interfaces, however, aren't looking
>> for this level of stability.
>
> I think that's a mistake. Nobody likes breakage, particularly in
> patches, and even with free software.
>
> I've yet to run into someone who was even slightly thrilled to find
> that something stopped working right after an upgrade.
Oh, I agree. I think we are simply talking about the tension between
keeping up with the latest & greatest and keeping things stable. I
do not think it is possible to have both all the time. I believe there
are likely some Volatile interfaces that are needed in order to keep
up with the bleeding edge. I think what we try to do in ARC is to
make sure that any such Volatile interfaces are not likely to create
undue problems for our users. This can be achieved, I think, by good
documentation, and by ensuring that end-users can write the programs
they need without depending on any Volatile interfaces. Such Volatile
interfaces should be clearly marked as being for bleeding edge users
only.
>> Perhaps that is changing? In the past when I have suggested that
>> ARC get more involved with external communities to promote better
>> interface stability within those projects, the responses seem to be
>> somewhere between ambivalent and negative.
>
> I'm not sure how the OpenSolaris architectural review mechanism works
> for some non-OpenSolaris project. If those teams wanted to do so, I
> don't see a problem with those folks deciding to participate in
> OpenSolaris and getting their reviews here -- that'd certainly work --
> but reaching out and saying in effect "we'd like to tell you how to
> produce stable software" seems quite strange to me.
You misunderstand my suggestion. I have seen numerous FOSS-related ARC
cases where all exported interfaces are Volatile be approved without
much question. While it may be too much to expect ARC to be directly
involved with external communities, I think it would be helpful if ARC
were to ask questions to determine the degree the project team is
involved with the external community. If this level is found to be
lacking, then ARC could recommend that this is important and give
ideas (perhaps best practices) to help the project team engage
effectively.
For example, good documentation (manpage/API docs) is an aspect of
interface stablity. If a project is missing manpages or API docs,
then this could raise a red flag. Perhaps ARC should encourage
the project team to work with the external community to help fix
such documentation problems.
> I don't think this ARC is the only way to produce stable software, or
> that we're the only font of architectural information. I think it's
> entirely possible for other software projects to do a decent job of
> engineering without necessarily needing (or even wanting) our help to
> do it.
Exactly. For example, there is FreeDesktop.org, which relates to
desktop specifications. I would think that when ARC sees a desktop
related FOSS case, that asking questions about how the project works
with FreeDesktop would be a good question. If the answer is "What
is FreeDesktop.org" then perhaps ARC should recommend getting more
involved or engaged with that community, etc.
> Thus, in my opinion, equating all external efforts that produce
> essentially equivalent results with our lowest possible stability
> level does them (and us) a substantial disservice. They just can't be
> that bad, and we can't possibly be that good.
My suggestions that ARC be more aware and supportive of external
interface stability efforts have been unpopular. However, it is
hard for me to understand how this does anybody a disservice.
Encouraging project teams to be more involved with the external
projects they work with just seems like common sense to me.
Perhaps it is not ARC's job to do such encouraging, but people seem
to agree that it benefits ARC when project teams do get so involved.
John Plocher, for example, suggested that FOSS project teams should
get more involved with ARC. In my opinion, it make most sense for ARC
to encourage project teams as well. In my opinion, we probably need
a holistic attitude where both ARC and Project Teams change they way
they approach things, rather than just expecting Project Teams to be
more involved.
>>> When we know we're abusing the taxonomy, we should stop. Even if
>>> we're surrounded by others who are busily digging the same hole ever
>>> deeper.
>> I'm not sure that we are abusing the taxonomy. It depends what our
>> priorities are. Are we working to build an ironclad stable software
>> platform based on FOSS components, or are we trying to provide a
>> development platform that is competitive with Linux, or what? How
>> we classify FOSS interfaces should depend on what we are trying to
>> achieve.
>
> We're trying to build a system. A big one. Doing that requires
> software architecture.
>
> Part of software architecture is analyzing the dependencies and
> stability levels *advertised* by the components. If Jane says she'll
> break her commitments in the next patch, but Sarah says she's building
> on top of Jane's work, then we've got a problem. It matters not
> whether Jane promises to provide support, or Sarah says she'll
> recompile every other Tuesday. What matters is finding some way to
> stabilize that dependency so that users of the software are not likely
> to see the problems.
>
> I don't think anyone here is suggesting that we ought not be
> competitive with Linux. And I agree completely that classifying FOSS
> interfaces (and, actually, classifying *ALL* interfaces) should depend
> on what we're trying to achieve.
>
> If what we're trying to achieve is software that either breaks
> fundamental rules of good design (by depending on things that change
> in uncoordinated ways) or is useless once customers look behind the
> glossy marketing materials (it has SQLite ... but Sun promises to
> break it in a patch), then, by all means, lump it all in as "Volatile"
> and don't worry about it. I'd actually advocate skipping
> architectural review entirely if that's what we want to achieve, as
> it's a large expense with zero return for merely Volatile bits.
>
> That's what was wrong with the "External" mess, and that's what we
> were trying to avoid repeating. FOSS != External and FOSS !=
> Volatile.
I agree with you. That said, I think the right solution is somewhere
in between making all FOSS interfaces Volatile and making all interfaces
"other consumers can use, IMO it should be Uncommitted or higher."
> Brian Cameron writes:
>> In the GNOME Project, the ORBIT2 and bonobo interfaces are stable and a
>> part of the GNOME Platform, and covered under ABI guarantees. In our
>> Solaris distro they are "Volatile".
>>
>> I can point you towards many references where the GNOME community
>> discourages people from using these interfaces, and the GNOME community
>> has announced that they will, someday, deprecate the interfaces (though
>> they plan to continue supporting them in a stable fashion).
>
> That sounds like "Obsolete Committed" to me. It's an interface
> they're not going to change, but that nobody new should be using
> because they're planning to remove them eventually in the future.
Right, but one complication is that (for some tasks) there is no other
interface to use. For example, Accessibility programs will continue
to require ORBit2/bonbobo until the port to using D-Bus is finished.
No visibility on how long that will take. Perhaps this is not a
problem, though.
> Yes, an interface can be declared "Obsolete" when it's first
> published.
>
> (I seriously doubt that there are significant cases in the GNOME or
> any other open source community that fail to fit nicely in the
> existing taxonomy. If there are, we'd certainly make room for them,
> but it's not as if those stability levels were developed in a
> technical vacuum.)
Perhaps not in GNOME, which has a somewhat reasonable interface
stability story. However, with the Indiana project's desire to make
an OS that is more developer friendly, I think we will likely see a
rash of ARC cases over the next few months that will be a challenge
to properly classify. I think we're already seeing some tension, for
example, with the FOSS C++ and garbage collection ARC cases that are
starting to show up via Indiana.
> Brian Cameron writes:
>> It seems that to bring a new, external interface in as Volatile and
>> then upgrade them to more stable classifications when there are clear
>> users is not out-of-line with the evolutionary nature of how ARC is
>> supposed to work.
>
> It's not ... but the feedback you're getting is that when we know that
> the interface is not as risky as advertised, then we're not doing
> favors by claiming it is.
In terms of SQLite, I agree 100%. However, I have concerns with some
of the recent suggestions that all public facing interfaces need higher
stability levels. But, we can discuss these concerns in other ARC
cases where this issues are more relevant, such as the upcoming GNOME
2.22 ARC case.
Brian