Crusty Blaa

Laurence Miao did some great work recently to port cfg from optparse to argparse.

The only significant API impact was that we could no longer have CONF() return unparsed command line arguments.

We chose to offer two alternatives - firstly, support for positional arguments:

>>> CONF.register_cli_opt(MultiStrOpt('bar', positional=True))
True >>> CONF(['a', 'b'])
>>> CONF.bar
['a', 'b']

and, secondly, integration with argparse's sub-commands:

>>> def add_parsers(subparsers):
... list_action = subparsers.add_parser('list')
... list_action.add_argument('id')
...
>>> CONF.register_cli_opt(SubCommandOpt('action', handler=add_parsers))
True
>>> CONF(['list', '10'])
>>> CONF.action.name, CONF.action.id
('list', '10')

After porting the CLIs in Nova (nova-manage), Glance (glance-control and glance-manage) and Keystone (keystone-manage) over to this sub-parsers stuff, it looks like it's going to work out quite nicely.

We're hoping to publish Nova, Glance, Keystone, Quantum, Cinder and Horizon 2012.2.1 next week (Nov 29).

The list of issues fixed so far can be seen here:

• https://launchpad.net/nova/+milestone/2012.2.1
• https://launchpad.net/glance/+milestone/2012.2.1
• https://launchpad.net/keystone/+milestone/2012.2.1
• https://launchpad.net/quantum/+milestone/2012.2.1
• https://launchpad.net/cinder/+milestone/2012.2.1
• https://launchpad.net/horizon/+milestone/2012.2.1

That's roughly 80 bugs.

We'd appreciate anyone who could give the candidate tarballs a whirl:

• http://tarballs.openstack.org/nova/nova-stable-folsom.tar.gz
• http://tarballs.openstack.org/glance/glance-stable-folsom.tar.gz
• http://tarballs.openstack.org/keystone/keystone-stable-folsom.tar.gz
• http://tarballs.openstack.org/quantum/quantum-stable-folsom.tar.gz
• http://tarballs.openstack.org/cinder/cinder-stable-folsom.tar.gz
• http://tarballs.openstack.org/horizon/horizon-stable-folsom.tar.gz

We've also started drafting release notes here. Contributions to those release notes are very welcome.

The OpenStack Technical Committee and the OpenStack Foundation Board of Directors have pretty separate sets of responsibilities and can get on with their work independently. One exception to that is the inclusion of new projects in OpenStack.

In the coming weeks, members of the two bodies will decide how to clarify confusion around the term "core project" and what exactly happens projects who graduate through OpenStack's Incubation process.

A thread on the openstack-dev mailing list is ongoing and is a great example of how a mailing list discussion can actually help to drive a rough consensus while still giving everyone an opportunity to express their views.

The TC is attempting to agree on a rough direction that represents the views of the TC before meeting with the Foundation Board. There are currently three distinct views. Firstly, this:

The concepts of "what is core" and "what is in OpenStack" have been conflated until now. The TC cares far more about the process for new projects to be included in the coordinated release than it cares about which projects are required to be used by providers in order to access the trademark.

We would like to take an inclusive but measured approach to accepting new OpenStack projects. We should evaluate any given proposed project on a well defined set of criteria like whether it embraces our values and processes, is useful to OpenStack users, well integrated with other projects and represents a sensible broadening of the scope of OpenStack.

We see Incubation as a trial period where promising projects have the opportunity to demonstrate their suitability for inclusion in our coordinated releases.

We see the term "Core OpenStack Project" in section 4.1.b of the bylaws as being solely related to trademark guidelines. The Foundation should simply maintain a list of projects required for trademark usage. We would be happy for that list to be called "Core Projects" or for a new name to be chosen to describe that list.

Secondly, Anne Gentle's variation which I'd summarize as allowing two groups of projects be accepted into OpenStack - "nuclear" projects which are the current group of "core" service projects and "core" projects which are everything else, for example Horizon, Ceilometer or Heat.

Thirdly, John Dickinson's variation which I'd summarize as only accepting projects into OpenStack which "solve IaaS problems" or support those projects in some way.

The way I'm thinking of these three approaches is how we want the project to treat proposals for new projects: "inclusive", "inclusive but two-tier" or "exclusive".

As I did for Essex, I've used Jonathan Corbet's gitdm to do some analysis of the commits to the core projects during Folsom.

Here's the top 20 committers across Nova, Glance, Swift, Keystone, Horizon, Quantum and Cinder:

Processed 3110 csets from 291 developers 132 employers found A total of 350046 lines added, 275491 removed (delta 74555)

Developers with the most changesets
Russell Bryant 160 (5.1%)
Brian Waldon 160 (5.1%)
Dan Prince 154 (5.0%)
Gabriel Hurley 124 (4.0%)
Mark McLoughlin 118 (3.8%)
Johannes Erdfelt 113 (3.6%)
Vishvananda Ishaya 92 (3.0%)
Joe Gordon 85 (2.7%)
Michael Still 71 (2.3%)
Eoghan Glynn 70 (2.3%)
Rick Harris 59 (1.9%)
Gary Kotton 58 (1.9%)
Dolph Mathews 55 (1.8%)
Yun Mao 50 (1.6%)
John Griffith 45 (1.4%)
Daniel P. Berrange 45 (1.4%)
Dan Wendlandt 43 (1.4%)
gholt 40 (1.3%)
Rongze Zhu 39 (1.3%)
Alex Meade 39 (1.3%)
Covers 52.090032% of changesets

Congrats and thanks to everyone involved in this release!

There are more raw stats here showing stats for each project individually and also statistics for gerrit reviewers and launchpad bug fixers.

Of course, this is nowhere near as polished as Bitergia's awesome report with pretty graphs and detailed analysis.

My mate Derek was giving me grief about not testing his OpenStack deployment in our lab at Red Hat. Friday seemed like a good day to give it a shot for a few minutes.

First problem - I'm one of the weird people at Red Hat who eschews the VPN in favour of SSH tunnels. At first, I figured I'd tunnel directly to the various OpenStack API services but that didn't work because the endpoint URLs returned by keystone obviously wouldn't point to my tunnelled connections.

Ok, let's just use a HTTP proxy, that should be fine. But no, not on yak shaving day. For some reason, I was getting 403 Forbidden errors.

To cut a long story short, it turns out:

• httplib2 always uses HTTP CONNECT tunneling rather than just sending the requests directly to the proxy
• squid by default and, indeed, our corporate proxy defaults to rejecting CONNECT for ports other than 443
• The recently released httplib2 0.7.5 has a PROXY_TYPE_HTTP_NO_TUNNEL which only uses CONNECT tunnelling for port 443, but it doesn't use this type when you configure your proxy via http_proxy in the environment

Not content with shaving the yak once, I shaved her thrice:

• An upstream patch to use HTTP_NO_TUNNEL when using http_proxy env
• A request to get the patch into Fedora
• A patch to work around it in python-novaclient

One other troubling conclusion is that if you're exposing the services over HTTPS, you really should use port 443 for everything or clients won't be able to connect over many proxies.