[ogb-discuss] test resource sponsors

[Jim Walker]

Sorry of the lengthy reply...

James C. McPherson wrote:
> Hi Jim,
> I really don't understand why you think it is
> necessary that people who want to use the test
> farm must have a sponsor.
> 
> Granted, we don't have unlimited resources, but
> surely when we've got the good people of Solaris
> QE involved we could have something very close
> to the DIY-PIT system that runs internally?

Self-Service Testing combines ON-PIT DIY and
Perf-PIT Self-Test into one interface for
external OpenSolaris developers, so in some
ways it's even better than we have internally.

> If access is limited to people who have signed
> and submitted the SCA, surely that should be
> more than sufficient for gaining access?

Here's some more food for thought...

- I personally don't like limiting anything, but
sometimes tough decisions have to be made.

- Self-Service Testing may be able to operate
with a larger field of potential users, since
to use it you have to be able to build a BFU
archive, which very few external users can or
want to do. So, I'm not as concerned about
getting overloaded here, short term, but, I
would rather start with access controls instead
of having to add them later after people are use
to no controls.

- In the future, we plan to support packages and
other approaches that define the bits to be tested,
which will make it easier for people to use
Self-Service Testing, so usage could grow.

- In Sun we may have something like 2 or 3+ test systems
per Solaris engineer, and depending on what system you
want, it's still hard to find a free one. Right now we
have 14 machines in the test farm, so that's about 1 test
system for every 10 contributors. And, near term, that
ratio will stay about the same.

- You may be able to calculate better percentages, but
maybe 25% of the contributors have been active in terms
of putbacks and only 1% are really active in terms of
multiple putbacks and bigger projects. Since we have
limited resources, I was thinking it would be best to
have an approach that favors the 25% group and
especially the 1% group, and scales with a focus
on these active contributors.

- If we just authorize all contributors, and have a
free-for-all approach, I would think the active
contributors would get blocked, and we could give
people a reason to sign the SCA for the sole
purpose of getting access to nice test machines,
with no intention of ever contributing to OpenSolaris
in anyway.

- It seems like the OpenSolaris Sponsors (or future RTI
advocates) and Project Leaders would be the best people
to determine who should get access. The support staff
for these resources, aren't the right people to decide
who gets access and who doesn't, plus they already have
a full plate.

- In the future, I was planning to target OpenSolaris
application developers, in addition to providing support
for OS developers, but we would still need a authorization
process so the people that really needed access can get
access.

- Once I add virtualization (zones, LDOMs and VMs) we will
get more test "systems" and utilize the hardware better,
but there will still be limits.

- What ever approach we arrive at, I would like
it to be consistent and something we can use as
a model for future shared resources on opensolaris.org.

- After we launch the first instance of the test farm, I'm
planning to focus more on establishing additional test farm
sites around the world both inside and outside Sun. Discussions
have already begun, so we need a process that scales well.

- If everything contributors are working on is tracked by
bug or rfe, it might be possible to enter a CR id as a method
of auto approving access to these resources. People entering
fake bugs just to get access would be quickly discovered.

In summary...

I would like an authorization process that:

- is simple, fair and consistent
- gives test resources to active OpenSolaris contributors
- filters out people who don't contribute to OpenSolaris
- spreads the load, so one person isn't an approval bottleneck
- is easy to implement and automate
- scales across multiple sites

Cheers,
Jim

-- 
Jim Walker, http://blogs.sun.com/jwalker
Sun Microsystems, Software, Solaris QE
x77744, 500 Eldorado Blvd, Broomfield CO 80021