Crusty Blaa - OpenStackhttps://crustyblaa.com/2017-09-14T11:00:00+01:00September 10th OpenStack Foundation Board Meeting2017-09-14T11:00:00+01:002017-09-14T11:00:00+01:00markmctag:crustyblaa.com,2017-09-14:/september-10-2017-openstack-foundation-board-meeting.html<p>The <a href="https://wiki.openstack.org/wiki/Governance/Foundation/10Sep2017BoardMeeting">OpenStack Foundation met in Denver on September 10th for a Joint
Leadership
Meeting</a>
involving the foundation Board of Directors, the Technical Committee,
and the User Committee.</p>
<p>The usual disclaimer applies - this my informal recollection of the
meeting. It’s not an official record.</p>
<h2>Foundation Events Update</h2>
<p>We began with …</p><p>The <a href="https://wiki.openstack.org/wiki/Governance/Foundation/10Sep2017BoardMeeting">OpenStack Foundation met in Denver on September 10th for a Joint
Leadership
Meeting</a>
involving the foundation Board of Directors, the Technical Committee,
and the User Committee.</p>
<p>The usual disclaimer applies - this my informal recollection of the
meeting. It’s not an official record.</p>
<h2>Foundation Events Update</h2>
<p>We began with an update from Lauren, Jonathan, and Mark on the events
that have happened so far this year, the Project Teams Gathering (PTG)
in Denver this week, and the coming OpenStack Summit in Sydney.</p>
<p>Lauren outlined some details of the recent Pike release, emphasizing
the positive media coverage of the release, with the <a href="https://www.openstack.org/news/view/340/openstack-pike-delivers-composable-infrastructure-services-and-improved-lifecycle-management">"composable
infrastructure
services"</a>
messaging resonating.</p>
<p>Jonathan talked about the many OpenStack Days events that happened
over the summer, including Melbourne, Tel Aviv, Budapest, Korea,
Taiwan, Japan, and China. Jonathan has attended all of these, covering
13 countries since the OpenStack Summit in Boston and he spoke about
the many new users and new use cases that he learned about over the
course of these events. <a href="https://www.openstack.org/community/events/openstackdays">More OpenStack Days are
coming this
year</a>
including Benelux, UK, Italy, Turkey, Nordic, Canada, France, and
Germany.</p>
<p>Mark spoke about the <a href="http://www.opendevconf.com">OpenDev event</a> held
the previous week in San Francisco. The goal was to bring in people
who are experts in different domains, and the important and emerging
use case of "Edge Computing" was chosen for this first event. The
keynote from <a href="https://www.cs.cmu.edu/~satya/">Dr Satya of Carnegie Mellon
University</a> was mentioned as one
particularly inspiring contribution.</p>
<p>An particularly interesting conclusion from <a href="https://etherpad.openstack.org/p/Deployment_Considerations__Hardware_options">one of the
sessions</a>
was a simple definition of what Edge Computing actually is:</p>
<blockquote>
<p>Edge is the furthest boundary that separates application-agnostic
scheduled computing workloads within the same operator's domain of
control, from applications or devices that can't schedule workloads,
and are outside the same operator's control.</p>
</blockquote>
<p>(Thanks to Dan Sneddon for pointing this out!)</p>
<p>Another interesting development is <a href="https://etherpad.openstack.org/p/opendev-sf-use-cases">the collection of edge use
cases</a> which
will be published to <a href="https://www.openstack.org/edge">Edge Computing mini-site on
openstack.org</a>.</p>
<p>The <a href="https://www.openstack.org/ptg/">PTG</a> was touched on next - more
than 400 contributors in attendance from 35 project teams, with the
first two days focused on the strategic goals of simplification,
adjacent technologies, onboarding new contributors, etc.</p>
<p>Jonathan also talked about the coming <a href="https://www.openstack.org/summit/sydney-2017/">OpenStack Summit in
Sydney</a>. We are aiming
for 2500+ attendees, and the amazing work by the program committee to
work the 1100 speaking submissions into an awesome three day schedule
has been completed. There will be <a href="http://hackathon.openstack.org.au/">Hackathon focused on Cloud
Applications</a> the weekend before
the event.</p>
<p>Finally, we looked forward to the OpenStack Summits in <a href="https://www.openstack.org/summit/vancouver-2018/">Vancouver in
May</a>, and <a href="https://www.openstack.org/summit/berlin-2018/">Berlin the
following November</a>.</p>
<h2>The Strategic Focus Areas</h2>
<p>Back in March, at the <a href="https://crustyblaa.com/march-8-2017-openstack-foundation-strategic-planning-workshop.html">Strategic Planning Workshop in
Boston</a>,
we developed a set of 5 strategic focus areas and formed working
groups around each of these. For each of those focus areas, the
working group presented their findings and progress, followed by some
discussion.</p>
<h3>Better communicate about OpenStack</h3>
<p>Thierry Carrez and Lauren Sell lead the discussion of this topic with
<a href="https://docs.google.com/presentation/d/1qQ1oCjhEE3bjtjkPydGhyeR6vFyuyFeOhbwc-WueddI">a set of
slides</a>.</p>
<p>We began by discussing progresson developing a map of OpenStack
deliverables. The idea is for the map to make it easy for users of the
software to make sense of what OpenStack has to offer, and one key
part of this mapping effort is to categorize deliverables into
buckets:</p>
<ul>
<li>openstack-user: Things an end user installs to consume the IaaS
stack</li>
<li>openstack-iaas: Primary compute, storage & networking services</li>
<li>openstack-operations: Things an operator uses to manage an openstack
cloud once installed</li>
<li>openstack-lifecyclemanagement: Things that help deploy/upgrade
OpenStack or standalone components</li>
<li>openstack-adjacentenablers: Things that other infrastructure stacks
can use to leverage individual OpenStack components</li>
</ul>
<p>Some of the outstanding questions include how to represent projects
which are coming down the line, where various types of plugins should
live, and whether Glance is tied to Compute or should be represented
as a Shared Service.</p>
<p>After the meeting, <a href="http://lists.openstack.org/pipermail/foundation-board/2017-September/000376.html">Lauren sent out a
request</a>
for everyone to <a href="https://etherpad.openstack.org/p/map-feedback">contribute their
feedback</a> on the draft
of the map. Please do join in!</p>
<p>Next, we discussed at some length how OpenStack has been affected by
"Big Tent" concept where we welcome collaboration, experimentation,
and innovation on "infrastructure things" beyond the core OpenStack
technology. We've know that users have found it difficult to make
sense of the breadth of project teams, and we have created further
confusion around "what is OpenStack".</p>
<p>Our discussion on this revolved around the idea of separating the
technologies directly related to the deliverables map above (which we
could call "OpenStack IaaS and friends"), the <a href="https://docs.openstack.org/infra/">"software forge"
infrastructure project</a>, and the
free-for-all project hosting area previously known as
Stackforge. There was broad consensus that we should give each of
those its own identity, which is particularly exciting when you think
of the potential for "Infra" to have an identity that isn't so closely
tied with OpenStack. We also discussed the potential to extend this
model to other projects in the future, but also our desire to not
become a "Foundation of Foundations" or a collection of entirely
unrelated projects.</p>
<h3>Requirements: Close the feedback loop</h3>
<p>Melvin Hillsman and Thierry Carrez talked us through the <a href="https://docs.google.com/presentation/d/1vjGm5OzjVvtbhf_WUyKN3nvYGpB_hDTJEzMAwvgLd1I">unanswered
requirements strategic focus
area</a>.</p>
<p>The focus of this discussion was on the creation of <a href="https://wiki.openstack.org/wiki/OpenStack_SIGs">OpenStack Special
Interest Groups
(SIGs)</a> as a mechanism
to have cross-community collaboration on a given topic, without the
work being under the umbrella of any one governance body.</p>
<p>The SIGs created so far are:</p>
<ul>
<li>a Meta SIG to discuss how to improve SIG processes</li>
<li>an API SIG, which is an evolution of the <a href="https://wiki.openstack.org/wiki/API_Working_Group">API Working
Group</a> already
formed, and</li>
<li>a still-forming Ansible SIG, with the goal of facilitating
collaboration between Ansible and OpenStack projects.</li>
</ul>
<h3>Community Health</h3>
<p>On this topic, Steve Dake talked us through some efforts to help grow
the next generation of leaders in the OpenStack community, supporting
people who wish to become a core contributor or PTL. Steve
particularly highlighted efforts along these lines within the Kolla
project.</p>
<h3>Increase complementary with adjacent technologies</h3>
<p>Steve Dake again took the lead on <a href="https://docs.google.com/presentation/d/1iF09E1LTil55sLzncWIyS9WvRK-L_4Gp975mWQ53sis">presenting this
topic</a>,
focusing on success stories of collaboration between OpenStack and
other communities - Ansible and Helm, in particular.</p>
<p>For Ansible, it was observed that OpenStack has built upon Ansible's
highly reusable technology in many ways, and OpenStack members have
contributed significantly to the Ansible modules for OpenStack based
on Shade. The conclusion was that the success was down to (a)
building releationships between the communities, (b) leadership
endorsement, and (c) the simplified collaboration process adopted by
Ansible.</p>
<p>For Helm, the collaboration has been focused in areas where Helm is
being used to deploy OpenStack services on Kubernetes.</p>
<p>Finally, Dims gave a read-out on collaboration on OpenStack within the
Kubernetes community, mostly with work in the <a href="https://github.com/kubernetes/community/blob/master/sig-openstack/README.md">OpenStack
SIG</a>
focused on <a href="https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/openstack/openstack.go">the OpenStack cloud
provider</a>.</p>
<h3>Technology changes: Simplify OpenStack</h3>
<p>For our final strategic focus area, <a href="http://www.scaryland.net/complicated.pdf">Mike Perez gave an
update</a> on progress. He
described projects which have recently been retired, the OpenStack
manuals project migration, and the status of a number of projects who
are seeing low levels of contribution activity.</p>
<h2>Clarifying and communicating where help is needed</h2>
<p>Next up, Thierry walked us through the TC's mechanism for exposing
areas where help is needed in the community. We talked through this
<a href="https://governance.openstack.org/tc/reference/top-5-help-wanted.html">"top 5 help wanted
list"</a>
and had a good discussion on the two items currently on the list -
Documentation and Glance.</p>
<h2>Interoperability Working Group</h2>
<p>As our final topic, Egle Sigler gave an update from <a href="https://docs.google.com/document/d/1CZxF-DL9CaxX9ab3Rd_IAqDzV-R_M8TCm1RwmQ_Emd4/">Interoperability
Working
Group</a>.</p>
<p>The first item of business was to <a href="https://review.openstack.org/502328">approve the 2017.09
guideline</a>. Both the compute and
object components gained some new capabilities in this update.</p>
<p><a href="https://crustyblaa.com/june-20-openstack-foundation-board-meeting.html">As discussed in the previous
meeting</a>,
the working group proposed the creation of "add-on" programs which
would focus on interoperability between different implementations of a
given service, without having to add that service as a requirement in
the core OpenStack Powered programs. As a starting point, it was
proposed to create advisory add-ons for <a href="https://review.openstack.org/492635">DNS
(Designate)</a> and <a href="https://review.openstack.org/#/c/490648/">Orchestration
(Heat)</a>. After some
discussions on the implications of these additions, they were formally
approved by the board.</p>
<h2>Next Meeting</h2>
<p>The board's next meeting is a 2 hour conference call on <a href="https://wiki.openstack.org/wiki/Governance/Foundation/10Oct2017BoardMeeting">Tuesday,
October
10</a>. Our
next in-person meeting will be in <a href="https://wiki.openstack.org/wiki/Governance/Foundation/5Nov2017BoardMeeting">Sydney on Sunday, November
5</a>.</p>June 20th OpenStack Foundation Board Meeting2017-06-20T11:00:00+01:002017-06-20T11:00:00+01:00markmctag:crustyblaa.com,2017-06-20:/june-20-openstack-foundation-board-meeting.html<p>The <a href="https://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a>
met for a <a href="https://wiki.openstack.org/wiki/Governance/Foundation/20June2017BoardMeeting">two hour conference call on
Tuesday</a>. The
usual disclaimer applies - this my informal recollection of the
meeting. It’s not an official record. Jonathan Bryce has <a href="http://lists.openstack.org/pipermail/foundation/2017-June/002495.html">posted an
official summary of the
meeting</a>.</p>
<h2>Interoperability Working Group Update</h2>
<p>After the usual formalities …</p><p>The <a href="https://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a>
met for a <a href="https://wiki.openstack.org/wiki/Governance/Foundation/20June2017BoardMeeting">two hour conference call on
Tuesday</a>. The
usual disclaimer applies - this my informal recollection of the
meeting. It’s not an official record. Jonathan Bryce has <a href="http://lists.openstack.org/pipermail/foundation/2017-June/002495.html">posted an
official summary of the
meeting</a>.</p>
<h2>Interoperability Working Group Update</h2>
<p>After the usual formalities of a roll call and approving <a href="https://wiki.openstack.org/wiki/Governance/Foundation/7May2017BoardMinutes">the previous
meeting
minutes</a>,
the board heard from Egle Sigler, Mark Voelker, and Chris Hoge of the
<a href="https://wiki.openstack.org/wiki/Governance/InteropWG">Interoperability Working
Group</a>.</p>
<p>The topics of the discussion are laid out in <a href="https://docs.google.com/document/d/1Ftw2tHn9l3oOlBhYMRrdVWcRfyk9tqGaTkfX3FVHdfo">the working group's
board
report</a>
with Egle covering the upcoming 2017.08 guideline, Mark covering the
proposal for extension programs, and Chris covering version 2.0 of the
interop schema.</p>
<p>The <a href="https://review.openstack.org/472785">extension programs proposal</a>
resulted in the most discussion. Mark described how the proposal
explains how the OpenStack Powered trademark programs work today, the
history of those programs, and how the additional programs would work.</p>
<p>The first type of program is a "vertical" program - examples given
include "OpenStack Powered for NFV" or "OpenStack Powered for
Containers". These would add requirements for additional capabilities
specific to these use cases, provided those capabilities are already
widely deployed in the context of those use cases.</p>
<p>The second type of program is an "add-on" program - for example,
"OpenStack Powered DNS". This would require capabilities specific to
that service, and ensure interoperability between implementations of a
given service. It is anticipated that individual project teams would
be responsible for definining the interop requirements.</p>
<p>Anni asked how these programs would relate to the idea of
"constellations" as a way of describing OpenStack components, but the
working group didn't see any immediate overlap with that idea.</p>
<p>I raised a concern that if obscure projects are free to define add-on
programs of their own, it could dilute the value of the OpenStack
Powered programs overall. However, it was clarified that, while the
individual project teams could define interop requirements, each
individual new program would require board approval.</p>
<p>Anni also raised a concern that vertical programs should not be
exclusive - i.e. it should be possible for a single product to qualify
for all vertical programs at once, so these programs need to not have
conflicting requirements. The working group agreed with this, and
explained that they had already taken this feedback into account.</p>
<p>Finally, the working group explained that their goal is for the board
to approve this framework at our Fall meeting.</p>
<h2>Membership Changes</h2>
<p>The last topic for the board to consider was some membership changes
and applications. Put simply, Canonical wished to move from being a
Platinum member to being a Gold member, and Ericsson wished to apply
for Canonical's Platinum member slot.</p>
<p>Chris Price presented Ericsson's case for Platinum
membership. Interestingly, this was the second time that Ericsson had
applied for Platinum membership in the past year. The previous time,
at <a href="https://crustyblaa.com/march-9-2017-openstack-foundation-board-meeting.html">the March 9 board
meeting</a>,
Ericsson and Huawei applied for the slot left vacant by HPE. Huawei
was successful with their application that time around.</p>
<p>Chris explained Ericsson's vision for OpenStack, and how they plan to
continue developing and driving forward the OpenStack ecosystem. He
also explained Ericsson's role in working with adjacent communities
like OpenDaylight and OPNFV, as well as their role in related
standard's bodies.</p>
<p>Next up, Mark Baker described how Canonical felt that with several
"industry giants" looking to take up Platinum membership, that the
right thing for a smaller entity like Canonical to do from a community
perspective, was to take a step back and allow others with greater
resources to take their Platinum member slot. However, he also
emphasized how OpenStack remains at the core of Canonical's business.</p>
<p>After some brief questions, the board went into executive session and,
on return, both applications were approved.</p>
<h2>Next Meeting</h2>
<p>The board's next meeting is a 2 hour conference call on Tuesday,
August 22. Our next in-person meeting will be in Denver on Sunday,
September 10.</p>March 9, 2017 OpenStack Foundation Board Meeting2017-03-14T11:00:00+00:002017-03-14T11:00:00+00:00markmctag:crustyblaa.com,2017-03-14:/march-9-2017-openstack-foundation-board-meeting.html<p>The OpenStack Foundation Board of Directors <a href="https://wiki.openstack.org/wiki/Governance/Foundation/8Mar2017BoardMeeting">met in-person for two
days</a>
in Boston last week.</p>
<p>The first day was a <a href="https://crustyblaa.com/march-8-2017-openstack-foundation-strategic-planning-workshop.html">strategic planning
workshop</a>
and the second day was a regular board meeting.</p>
<h2>Executive Director Update</h2>
<p>After a roll call and approving the minutes of the previous meeting,
<a href="http://lists.openstack.org/pipermail/foundation/attachments/20170313/73838d4b/attachment-0002.pdf">Jonathan gave a …</a></p><p>The OpenStack Foundation Board of Directors <a href="https://wiki.openstack.org/wiki/Governance/Foundation/8Mar2017BoardMeeting">met in-person for two
days</a>
in Boston last week.</p>
<p>The first day was a <a href="https://crustyblaa.com/march-8-2017-openstack-foundation-strategic-planning-workshop.html">strategic planning
workshop</a>
and the second day was a regular board meeting.</p>
<h2>Executive Director Update</h2>
<p>After a roll call and approving the minutes of the previous meeting,
<a href="http://lists.openstack.org/pipermail/foundation/attachments/20170313/73838d4b/attachment-0002.pdf">Jonathan gave a presentation outlining his perspective on where
OpenStack is
at</a>.</p>
<p>He talked about perception challenges that the Foundation is working
to address, emphasizing the messages of "costs less, does more" and
"all apps need Open Infrastructure".</p>
<p>He described how the Foundation has been iterating on a "pitch deck"
and had completed twenty or so interviews with journalists where they
presented this deck in a concerted effort to "change the
narrative". This begins by talking about some recent industry trends,
including a slowdown in the rate of public cloud growth and the
coverage of Snap's significant spending on public cloud services
compared to their revenues.</p>
<p>It goes on to compare "Gen 1" and "Gen 2" private clouds, with the
challenges moving from technology and people to culture and
processes. The advantage of a multi-cloud strategy, with sophisticated
workload placement was discussed as well as some detailed information
backing up the claim that companies are achieving cost savings with
OpenStack.</p>
<p>Jonathan described how this presentation had been very well received,
and had resulted in some <a href="https://techcrunch.com/2017/02/22/openstacks-latest-release-focuses-on-containers/">very positive
coverage</a>.</p>
<p>Jonathan briefly touched on key improvements in Ocata and information
about the profile of our contributors. This resulted in some debate
about the value of "vanity metrics" and there was general agreement
that while there continues to be demand for this information, we we
should all be cautious in over-emphasizing it.</p>
<p>Finally, Jonathan reflected on feedback from the Project Team
Gathering in Atlanta. Attendance was strong and attendees reported
feeling less distractions and a greater ability to have important
discussions because of the smaller venue and attendee count. While it
was mostly seen as a great success, there are areas for improvement
including the approach to choosing hotels and committing to hotel room
bookings, how scheduling is organized, the size of some rooms,
etc. Jonathan mentioned the success of the Forum and the OpenStack
Summit in Boston will be key.</p>
<h2>User Committee and Compensation Committee</h2>
<p>The board spent a fairly short time discussing two matters from its
committees.</p>
<p>Firstly, a proposal from the User Committee Product Working Group for
facilitating the organization of the schedule for The Forum in Boston
was well received, but since the board encouraged all interested
parties to work with the Foundation staff who are coordinating the
planning, particularly Tom Fifield and Thierry Carrez.</p>
<p>Secondly, the board approved Jonathan's goals for 2017, which had been
prepared by the Compensation Committee and sent to board members
earlier for review. The board only sets Jonathan's goals, and empower
him to set the goals for the rest of the staff. Related to Jonathan's
goals, the board also had some discussion later around documenting the
goals of the board itself.</p>
<h2>Membership Applications</h2>
<p>Significant time during the day was given over to considering some
corporate membership applications.</p>
<p>With HPE resigning its Platinum membership, we had invited
applications to take over its slot and received applications from
Ericsson and Huawei. Both companies had previously applied in
November, 2014 at the OpenStack Summit in Paris to replace Nebula as
Platinum member, but Intel had been successful that time.</p>
<p>Anni Lai presented for <a href="http://www.huawei.com/en/">Huawei</a> and Chris
Price presented for <a href="https://www.ericsson.com/">Ericsson</a>. Both
described their employer's vision for OpenStack, and their wide range
of contributions to date. Both also talked about their position in the
market, their key customers, and how they are growing the OpenStack
ecosystem. The presentations were well received and, after an
executive session, the board voted to approve Huawei as a Platinum
member. The board expressed their gratitude for Ericsson's interest
and preparation, observing that having multiple companies interested
in Platinum membership opportunities is a sign of the strength of our
community.</p>
<p>Representatives from <a href="http://www.h3c.com/en/">H3C</a> also presented
their application for Gold membership. H3C is a prominent IT vendor in
China's enterprise IT market, distributes an OpenStack based cloud
product, has several very large reference customers, and is
establishing itself as a technical contributor to the project. After
executive session, the board voted to approve H3C as a Gold member.</p>
<h2>Wrapping Up</h2>
<p>One piece of good news wasn't public during the meeting, but has since
been <a href="http://lists.openstack.org/pipermail/foundation/2017-March/002477.html">announced by
Jonathan</a>:</p>
<blockquote>
<p>The Board also approved the promotion of Thierry Carrez to VP of
Engineering for the OpenStack Foundation. Thierry has been a leader
in the technical community since the beginning of OpenStack and has
also built a team within the Foundation focused on upstream
collaboration.</p>
</blockquote>
<p>I think it's safe to say the board were warmly supportive of this
change, wished Thierry every success in this new role, and looked
forward to working more closely with Thierry than ever before.</p>
<p>And, with that, the board dispersed feeling pretty fried after an
intense couple of days of discussions!</p>March 8, 2017 OpenStack Foundation Strategic Planning Workshop2017-03-14T07:00:00+00:002017-03-14T07:00:00+00:00markmctag:crustyblaa.com,2017-03-14:/march-8-2017-openstack-foundation-strategic-planning-workshop.html<p>The OpenStack Foundation Board of Directors <a href="https://wiki.openstack.org/wiki/Governance/Foundation/8Mar2017BoardMeeting">met in-person for two
days</a>
in Boston last week.</p>
<p>The first day was a joint workshop with the Technical Committee, User
Committee, and Foundation staff. The workshop was planned in response
to the "OpenStack Futures" discussion at our three previous board
conference calls in …</p><p>The OpenStack Foundation Board of Directors <a href="https://wiki.openstack.org/wiki/Governance/Foundation/8Mar2017BoardMeeting">met in-person for two
days</a>
in Boston last week.</p>
<p>The first day was a joint workshop with the Technical Committee, User
Committee, and Foundation staff. The workshop was planned in response
to the "OpenStack Futures" discussion at our three previous board
conference calls in
<a href="https://crustyblaa.com/november-17-openstack-foundation-board-meeting.html">November</a>,
<a href="https://crustyblaa.com/december-6-openstack-foundation-board-meeting.html">December</a>,
and January.</p>
<h2>Introductions</h2>
<p>We began the workshop with very brief personal introductions, followed
by Alan, Thierry, and Edgar giving an overview of the roles and
responsibilities of the Board of Directors, the <a href="https://docs.google.com/presentation/d/17U3Lf83nKrhfqT0HLr3gq5xqQmwUFcW28OcfUrS1n3E">Technical
Committee</a>,
and the <a href="https://docs.google.com/presentation/d/1dsRa35TaqeiSrB-n76pj2jONBnWW_-L1UryEbT1vqNE">User
Committee</a>.</p>
<h2>State of the Union</h2>
<p>Next, <a href="http://lists.openstack.org/pipermail/foundation/attachments/20170313/73838d4b/attachment-0003.pdf">Mark Collier presented his view of the state of
OpenSack</a>,
with a particular emphasis on <a href="https://etherpad.openstack.org/p/ocata-strategic-review-board">the four areas planned for discussion
during the
day</a>. Mark
began by talking about the exciting opportunity we had by having such
breadth and depth of expertise in the room, and appealed to everyone
to put aside their particular roles and work together as a single
leadership team to talk about the work.</p>
<h3>Evolving The Architecture</h3>
<p>Mark spent some time trying to demystify the Big Tent change,
describing how the previous Stackforge/Incubation/Integrated stages
have been replaced by almost any project being welcome to use
OpenStack infrastructure with the TC responsible for reviewing
applications to join the set of official OpenStack projects known as
"The Big Tent".</p>
<p>Mark then described one of the key changes happening in OpenStack
right now as the containerization of the control plane, with projects
like Kolla, openstack-helm, and TripleO all tackling this area. He
also talked about the work happening around running containers on
OpenStack itself with projects like Kuryr, Fuxi, Magnum, and Zun, but
he also wondered aloud whether we're addressing all the right
integration points. He also described some of the ongoing debates
about the scope of OpenStack and our technology choices, with topics
like the use of golang, the Gluon project, whether we welcome
competition within the Big Tent, and community-wide goals.</p>
<p>Finally, Mark gave us a preview of the work happening around version 2
of the <a href="https://www.openstack.org/software/project-navigator/">OpenStack Project
Navigator</a> and
talked about how this will play a key role in helping people
understand what OpenStack provides and how it can be used.</p>
<h3>Unanswered Requirements</h3>
<p>Mark talked briefly about the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/UserCommittee">working groups under the User
Committee</a>
and the transition from Design Summit to <a href="https://www.openstack.org/ptg/">Project Team
Gathering</a> and
<a href="https://wiki.openstack.org/wiki/Forum">Forum</a> formats. These concepts
are all important in understanding how OpenStack thinks about our
requirements gathering, strategic long-term planning, and
implementation planning.</p>
<p>Mark also gave a preview of some detractor quotes from our user
survey, and emphasized a common theme - the perceived and actual
complexity of OpenStack, both in terms of understanding and operating
the software.</p>
<h3>Adjacent Communities</h3>
<p>Mark classified the various sets of adjacent communities that we are
particularly interested in developing strong relationships
with. Container technologies like Kubernetes, Docker, and Mesos. PaaS
technologies like CloudFoundry and OpenShift. NFV projects like OPNFV
and Cloudify. Provisioning technologies like Terraform, Puppet, and
Saltstack. And specific ecosystem relationships, with companies like
CoreOS.</p>
<p>Mark described the change in the Foundation's event strategy,
targeting events like KubeCon, DockerCon, CoreOS Fest, etc. as key
events where we should be positioning the OpenStack brand and
developing relationships.</p>
<p>He also described particular focus areas of individual staff members
which are relevant to the topic - Chris Hoge working with upstream
Kubernetes and running OpenStack SIG meetings, David Flanders working
on a report around the gaps when running platforms on OpenStack (like
Cloud Foundry, OpenShift, Kubernetes, and Terraform), and how Ildiko
Vancsa and Kathy Cacciatore are both working closely with OPNFV.</p>
<p>Finally, Mark talked about the Open Source Days event at the OpenStack
Summit in Boston, as well as some very early stage discussions for an
OpenDev event which would be a small, focused event around improving
the integration between applications frameworks and open
infrastructure.</p>
<h3>Community Health</h3>
<p>The final area of discussion was the subject of community health, and
Mark first put out some statistics that he felt painted a very
reassuring picture of the community's health. In 2016, we had 3,500
unique contributors, 1,850 of which were retained from 2015. In Ocata,
we had fewer developers than Newton, most likely because it was a
shorter cycle.</p>
<p>Mark contrasted challenges with projects like Trove and Designate
losing contributors, while projects like Kuryr, Kolla, and Zun seeing
the greatest number of new contributors.</p>
<p>Similarly, Mark talked about HPE laying off upstream developers, Cisco
killing off intercloud, a small slowdown in Summit sponsorships, while
we have also added 7 more Gold members, and many first-time corportate
members and Summit sponsors.</p>
<h2>Strategic Planning Exercise</h2>
<p>The rest of the day was given over to a multi-stage strategic planning
exercise prepared by Allison and Alan. The idea was to discuss these
focus areas, <a href="https://photos.google.com/share/AF1QipM10xRXuSCufsV8g8O60nnWgAW0n3yDteHqM1BPSgUfxNNZ6fZulsJ8BnYtcY5pNA?key=UEk5SXlaYlRSdllyYXZDVTlvN2EwR1NidjhjUGl3">gather everyone's ideas for
improvement</a>,
summarize and categorize these ideas, vote on ideas in each focus
area, and finally agree on how to proceed with concrete goals for the
next 6-9 months.</p>
<h3>Discussion</h3>
<p>The initial discussion covered a lot of ground. Allison introduced
each focus area by describing the input we gathered via the etherpads
and input she gathered through 1:1 interviews with a variety of
people.</p>
<p>One topic of discussion related to how OpenStack can simplify how we
describe OpenStack, particularly to reduce confusion introduced with
the Big Tent change. Various ideas around categorization, tagging,
vertical definitions, a concept of constellations, maturity ratings,
and much more, were discussed.</p>
<p>We talked about the promise for the future that OpenStack
provides. That there will be evolution over time, that we deliver the
cloud solutions of today and will deliver the solutions of
tomorrow. That the challenge of smooth upgrades is part of our
challenge in delivering "future proof infrastructure".</p>
<p>We talked about the challenges of scalability, manageability, and
complexity. The theme of containerized deployments, the need for
vertically focused views of OpenStack, for example for Telco users. We
discussed the need for OpenStack to be able to evolve over time, with
refactoring or rewriting components being only one of the possible
approaches we may see over time.</p>
<p>We talked at great length about how OpenStack could work more closely
with adjacent communities. How the relationship with these communities
should bring value to both communities. We particularly emphasized the
need for a closer relationship with the CNCF and the Kubernetes
community.</p>
<h3>Gathering Ideas</h3>
<p>Over lunch, everyone wrote their concrete, actionable ideas for
improvement on sticky notes and put them on <a href="https://photos.google.com/share/AF1QipM10xRXuSCufsV8g8O60nnWgAW0n3yDteHqM1BPSgUfxNNZ6fZulsJ8BnYtcY5pNA?key=UEk5SXlaYlRSdllyYXZDVTlvN2EwR1NidjhjUGl3">flipcharts for each of the
areas of
discussion</a>. Later,
Jonathan volunteered to group the ideas into themes, and summarized
these themes for the group, facilitating further discussion before
voting on which theme in each area we should particularly focus on.</p>
<p>On the subject of communicating about "what is OpenStack", the main
themese were marketing activities, various categorization ideas, and
idea Allison talk about earlier referred to as "constellations". We
later voted to focus on the categorization area and formed a group of
interested parties:</p>
<blockquote>
<p>Communicate about OpenStack: Categorize (objective data) and map
(subjective approach) OpenStack projects as base versus optional
(within a specific use case), integrated versus independent release,
emerging versus mature, stability, adoption metrics, what works
together, services versus consumption (operational tools/client
libraries), and other criteria</p>
<p>Names: Thierry Carrez [lead], Alan Clark, Allison Randal, Jon
Proulx, Melvin Hillsman, Lauren Sell, Tim Bell, Mark Baker, Kenji
Kaneshige</p>
</blockquote>
<p>For unanswered requirements, we discussed how to prioritize, ideas
around a solution focus, scalability challenges, and a list of
specific features that people felt were important. A counter-point was
made that rather than focusing on any of these ideas, perhaps the
focus should be on working with adjacent communities. Later, we
discussed the need to grow the connection between the Product Working
Group, the TC, and individual projects. The outcome and group for this
was:</p>
<blockquote>
<p>Requirements: Bring different groups (UC/technical/etc) together at
Forum to collaborate/communicate aroud user stories, gap analysis,
what fits in the current state of tech, prioritize what would have
the greatest impact in reducing pain for users.</p>
<p>Names: Melvin Hillsman [lead], Yih Leong Sun, Jon Proulx, Rob Esker,
Emilien Macchi, Doug Hellmann, Tim Bell, Shamail Tahir</p>
</blockquote>
<p>On the topic of adjacent communities, we observed that by far the most
dominant area of discussion was the need to create better connection
with the Kubernetes community. The themes were community engagement,
technical engagement, OpenStack consuming technology from the
Kubernetes and containers world, and making OpenStack technology more
consumable by Kubernetes. In the end, there was strong consensus to
focus on the consumability of OpenStack technologies:</p>
<blockquote>
<p>Adjacent Technologies: Make our technology more consumable
(independently) by other communities/projects.</p>
<p>Names: Chris Price [lead], Alan Clark, Dims, Rob Esker, Mark
Collier, Steven Dake, Mark McLoughlin, Shamail Tahir</p>
</blockquote>
<p>For changes to the technology, we discussed simplifications, making
containers first class citizens, recording tribal knowlede, culling
failed efforts, converging deployment tools, and welcoming emerging or
competing projects. The theme we voted to focus on was:</p>
<blockquote>
<p>Changes to the Technology: Workstream to simplify existing projects,
reduce dependency options, reduce config options.</p>
<p>Names: Mike Perez [lead], --> TC project</p>
</blockquote>
<p>Finally, on the subject of community health, we talked about
onboarding contributors, reworking our processes, community tools,
growing leaders, corporate involvement in the project, and recognizing
work with adjacent communities. We voted to focus on the leadership
theme:</p>
<blockquote>
<p>Community Health: Grow next generation of
leadership/experts/cross-project devs within the community</p>
<p>Names: Steven Dake [lead], Chris Price, Jeremy Stanley, Dims,
AlanClark, Joseph Wang</p>
</blockquote>
<h3>Next Steps</h3>
<p>For each of these focus areas, the lead person in the group committed
to organizing a kick-off meeting by March 22nd. The real work will
begin there!</p>January 24th OpenStack Foundation Board Meeting2017-01-27T11:00:00+00:002017-01-27T11:00:00+00:00markmctag:crustyblaa.com,2017-01-27:/january-24-openstack-foundation-board-meeting.html<p>The <a href="https://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a>
met for a <a href="https://wiki.openstack.org/wiki/Governance/Foundation/24Jan2017BoardMeeting">two hour conference call on
Tuesday</a>. The
usual disclaimer applies - this my informal recollection of the
meeting. It’s not an official record. Jonathan Bryce has <a href="http://lists.openstack.org/pipermail/foundation/2017-January/002464.html">posted an
official summary of the
meeting</a>.</p>
<h2>The 2017 Board</h2>
<p>This was the first meeting …</p><p>The <a href="https://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a>
met for a <a href="https://wiki.openstack.org/wiki/Governance/Foundation/24Jan2017BoardMeeting">two hour conference call on
Tuesday</a>. The
usual disclaimer applies - this my informal recollection of the
meeting. It’s not an official record. Jonathan Bryce has <a href="http://lists.openstack.org/pipermail/foundation/2017-January/002464.html">posted an
official summary of the
meeting</a>.</p>
<h2>The 2017 Board</h2>
<p>This was the first meeting of the board since the recent <a href="http://lists.openstack.org/pipermail/foundation/2017-January/002449.html">Individual
and Gold member director
elections</a>. As
such, Alan asked each of our new directors to introduce themselves:</p>
<ul>
<li>Brad Topol</li>
<li>Brian Stein</li>
<li>ChangBo Guo</li>
<li>Christopher Price</li>
<li>Joseph Wang</li>
<li>Junwei Liu</li>
<li>Russell Bryant</li>
<li>Steven Dake</li>
</ul>
<p>Alan also thanked our outgoing board members for their contributions:</p>
<ul>
<li>Alex Freedland</li>
<li>Edgar Magana</li>
<li>Manju Ramanathpura</li>
<li>Monty Taylor</li>
<li>Roland Chan</li>
<li>Raj Patel</li>
<li>Todd Moore</li>
<li>Van Lindberg</li>
</ul>
<p>Since it's a new year, we took the opportunity to review the various
policies which apply to board members:</p>
<ol>
<li><a href="https://wiki.openstack.org/wiki/Governance/Foundation/AntitrustPolicy">Antitrust policy</a></li>
<li><a href="https://wiki.openstack.org/wiki/Governance/Foundation/CodeOfConduct">Code of Conduct</a></li>
<li><a href="http://www.openstack.org/legal/transparency-policy/">Transparency policy</a></li>
<li>Board communication channels, including webex, foundation mailing
... list, foundation board open and confidential mailing lists, IRC,
... etherpad, and wiki.</li>
</ol>
<p>We also took the time to review the list of board committees and
invite board members to volunteer to join these via <a href="https://etherpad.openstack.org/p/UnofficialBoardNotes-Jan24-2017">the
etherpad</a>.</p>
<p>Finally, we reviewed the board meeting schedule for the year,
including 6 conference calls and 4 in-person meetings.</p>
<h2>Staff Update</h2>
<p>Mark Collier then presented an update to the board. Reflecting on our
strategy, he talked about three forces that need to be in alignment -
the ecosystem, the software, and our users. The <a href="http://lists.openstack.org/pipermail/foundation/attachments/20170125/86672a90/attachment-0001.pdf">slide
deck</a>
gives a good view into the discussion.</p>
<h2>Commitee Actions and Updates</h2>
<p>Next up, the compensation gave a brief summary of their review of
Jonathan's performance in 2016, which had been shared in advance with
the board.</p>
<p>The Interop Working Group also talked through <a href="https://docs.google.com/document/d/1MFotmauuwZXmmhiTj_iyzUIOcTZA1V8JXgPk8Lqdc-4">their January 2017
report</a>
and the board voted to approve the <a href="https://github.com/openstack/defcore/blob/master/doc/source/guidelines/2017.01.rst">2017.01
guideline</a>.</p>
<h2>Strategic Discussions</h2>
<p>Finally, the board spent some time discussing the strategic planning
discussions that started at <a href="https://crustyblaa.com/november-17-openstack-foundation-board-meeting.html">the November
meeting</a>
and continued during <a href="https://crustyblaa.com/december-6-openstack-foundation-board-meeting.html">the December
meeting</a>. Our
hope is to have a joint meeting with the Technical Committee in March
to spend significant time on the topic. Allison Randal discussed
various options to prepare for the meeting, including conducting
one-to-one interviews or a public survey.</p>December 6th OpenStack Foundation Board Meeting2016-12-06T11:00:00+00:002016-12-06T11:00:00+00:00markmctag:crustyblaa.com,2016-12-06:/december-6-openstack-foundation-board-meeting.html<p>The <a href="https://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a>
met for a two hour conference call yesterday. The usual disclaimer
applies - this my informal recollection of the meeting. It’s not an
official record. Jonathan Bryce has <a href="http://lists.openstack.org/pipermail/foundation/2016-December/002443.html">posted an official summary of the
meeting</a>.</p>
<h2>Executive Director Update - 2017 Budget</h2>
<p>After taking a roll …</p><p>The <a href="https://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a>
met for a two hour conference call yesterday. The usual disclaimer
applies - this my informal recollection of the meeting. It’s not an
official record. Jonathan Bryce has <a href="http://lists.openstack.org/pipermail/foundation/2016-December/002443.html">posted an official summary of the
meeting</a>.</p>
<h2>Executive Director Update - 2017 Budget</h2>
<p>After taking a roll call and approving minutes from two previous
meetings, Jonathan gave a presentation on the 2017 Foundation budget.</p>
<p>Jonathan described a number of goals under a "One Platform" theme for
2017. He talked about how OpenStack should (a) make sure containers
are first class citizens within OpenStack, (b) ensure workload
portability across interoperable public and private OpenStack clouds,
and (c) reinforce OpenStack position as the standard platform for
NFV.</p>
<p>Jonathan went on to remind the board that in 2016 we had planned for a
2016 loss, the idea being to invest some of the cash reserves that the
Foundation had built up in order that we could take on some new
activities. In 2017, the proposal is to aim for a break even budget so
as to avoid not further depleting the reserves. However, with the
addition of new Gold Members, we will be able to maintain the
increased investment from 2016 while still breaking even. Mark Collier
noted that China is a key growth area, given that much of this new
funding is from new Gold Members from China.</p>
<p>Jonathan then presented the budget itself, showing quarter-by-quarter
income and expenses in a number of high level buckets like "Corporate
Fees", "Events", "G&A", "Community Development", and "Community
Infrastructure". One particularly striking aspect of OpenStack
Foundation budgets is that much of the financial activity comes in Q2
and Q4 with the OpenStack Summit events in those quarters, accounting
for $10.5M and $6.5M activity in Q2 and Q3 respectively out of the
total $22M budget.</p>
<p>A discussion followed the presentation around what level of financial
detail the board would like and expect to have available. Roland Chan
expressed a concern that he was unable to tell how the strategic
decisions the board had made were impacting the budget. For example,
you could compare the 2016 and 2017 high-level budgets and make a good
guess as to the impact caused by the addition of the PTG, but it would
be nice to understand that in more detail. Some debate ensued as to
whether this was information that was already available to the Finance
Committee, whether the board should be supplied with a more detailed
budget breakdown, or whether we instead need some analysis and
commentary about the financial impact of key changes.</p>
<p>Jonathan took an action to prepare some further information which
should help address the question. Alan also took an action for the
Finance Committee to consider how to address this more
structurally. Finally, the 2017 budget was approved by a vote of board
members.</p>
<h2>User Committee Proposal</h2>
<p>Next up, Edgar Magana presented some <a href="https://docs.google.com/document/d/1QmLOeseAkjBWM_TXsUeKBErNaSHnuZp81II0T71ARfo">proposed by-laws changes around
the structure of the User
Committee</a>
and related <a href="https://review.openstack.org/404318">changes to the UC
charter</a>.</p>
<p>Since these changes had been previously discussed by the board, they
were put to a vote after some short discussion. The board approved a
motion to replace section 4.14 of the by-laws with the text above,
along with adding a new UC member policy appendix.</p>
<h2>OpenStack Futures</h2>
<p>The board then turned its attention to a discussion started at the
<a href="https://crustyblaa.com/november-17-openstack-foundation-board-meeting.html">previous board
meeting</a>
about the future of the project and some key questions it was felt the
board should be considering.</p>
<p>Allison Randal lead the discussion by talking through some of the
information the board had gathered in an etherpad since the previous
board meeting. We had condensed the questions and concerns into four
distinct areas:</p>
<ol>
<li>OpenStack and its evolving relationship with <em>adjacent
technologies</em>.</li>
<li><em>Unanswered requirements</em>, and how they will be address by the
project over time.</li>
<li>Whether OpenStack is sufficiently able to <em>evolve architecturally</em>
architecture to meet certain needs.</li>
<li>Community health indicators the board and the wider community
should be paying attention to.</li>
</ol>
<p>It was felt that we would only be able to discuss these in a useful
way with an in-person board meeting, and the hope is that we will be
able to hold this meeting alongside the PTG with involvement from the
Technical Committee.</p>
<p>In order to help frame the discussion at that in-person meeting,
Allison proposed that board members consider a series of 12 strategic
questions as homework for the next board meeting. These questions are
based on the 12 "boundary questions" proposed by <a href="https://en.wikipedia.org/wiki/Werner_Ulrich">Werner
Ulrich</a> in his <a href="http://www.wulrich.com/downloads/ulrich_2005f.pdf">work on
Critical Systems Heuristics
(CSH)</a>. There was
some discussion as to whether these questions could, in future, be
posed to the wider community via a survey.</p>
<p>There was some brief discussion about how this relates to
<a href="http://governance.openstack.org/goals/index.html">"OpenStack-wide Goals"
mechanism</a> recently
introduced by the TC. Toby Ford gave the concrete example of "1000s of
nodes under the management of a single control plane", and wondered
whether that would qualify as an OpenStack-wide goal. Doug Hellmann
happened to be on the call and was able to explain that the goals
mechanism is used to achieve changes that address concerns across all
projects and which can be completed in a single cycle.</p>
<p>The discussion ended with the board agreeing to complete the 12
questions prepared by Allison for an in-person meeting in Atlanta.</p>
<h2>Gold Membership Expansion</h2>
<p>The final topic of discussion was raised by Shane Wang. Now that the
24 Gold Member slots have been filled, an obvious question remains
... should the Foundation increase the number of Gold Member slots?</p>
<p>Shane made the case that more companies want to join as Gold Members,
particularly in China and gave the example of an other major Chinese
telecommunications operator that is eager to join.</p>
<p>Roland explained that the topic had been discussed at a recent - but
poorly attended - meeting of the Gold Members. It was felt at that
meeting that more options need to be considered than simply increasing
the number of Gold Member slots - for example, increasing the number
of Platinum Member slots so that perhaps some Gold Members could move
to Platinum, or the introduction of a Silver Member tier.</p>
<p>However, it was also felt that we should carefully consider our goals
even before jumping into considering specific options. Anni made the
point that we're trying to achieve a balance between the risk of
devaluing the membership status versus wanting to welcome more
participation in the Foundation.</p>
<p>All felt that this was an important topic that warranted further
discussion and consideration.</p>
<h2>Looking Ahead to 2017</h2>
<p>And so, the boards work for 2016 has come to an end. The <a href="https://wiki.openstack.org/wiki/Governance/Foundation#OpenStack_Board_of_Director_Meetings">schedule of
meetings for
2017</a>
has not yet been finalized, but the next two meetings are likely to be
a two hour conference call on January 31 and an all day in-person
meeting in Atlanta on February 24.</p>Sustainable Investment in Open Source2016-11-22T11:00:00+00:002016-11-22T11:00:00+00:00markmctag:crustyblaa.com,2016-11-22:/sustainable-investment-in-open-source.html<p><em>This is the prose version of a talk I gave today at OpenStack Day,
France. The <a href="https://redhat.slides.com/markmc-redhat/sustainable-investment-in-open-source">slides are available
here</a>.</em></p>
<p>Today, I want to speak about a somewhat subtle, meta-topic. I hope to
share some insight on the question of how a company should think about
its investment in any …</p><p><em>This is the prose version of a talk I gave today at OpenStack Day,
France. The <a href="https://redhat.slides.com/markmc-redhat/sustainable-investment-in-open-source">slides are available
here</a>.</em></p>
<p>Today, I want to speak about a somewhat subtle, meta-topic. I hope to
share some insight on the question of how a company should think about
its investment in any given open-source project. The reason I think
the subject is important right now is that, as OpenStack follows its
inevitable path through the “hype curve”, we are seeing some major
players in the OpenStack ecosystem have recently re-shaped their
investment in the project. I think there is plenty of opportunity for
companies to take a much more realistic view on this subject, and my
hope is that this will lead to a much more sustainable level of
investment in the project.</p>
<h2>My Story</h2>
<p>My views on this question are so heavily influenced by an early
personal experience, that I’m tempted to talk at length about my own
story. But time is short. Very quickly, though, as I was finishing
university, I realized I wanted to focus my career on open-source, and
I was faced with the obvious question of who was going to pay me to
work on open-source? What value would be open-source work bring to my
employer? I spent a lot of time thinking about this, even after I
found myself employed to work full time on open-source.</p>
<h2>My Employer</h2>
<p>An interesting thing happened. Because I had spent a good deal of time
thinking about the value of my own open-source work, I was well-placed
to help my employer think about how, where, and why to invest in
open-source projects. It quickly became apparent to me then - and is
still apparent to me now - that this fundamental question is fraught
with difficulty, and most people struggle greatly to answer it.</p>
<h2>Investment vs Business Needs</h2>
<p>Over the years, I’ve boiled my answer to this simple-sounding question
to something equally simple - your investment in a project should be
directly related to what the business needs from the project. I think
it’s important to frame it this way, because if the business doesn’t
have a good have a good understanding in the value of an investment,
the business is going to be quick to discontinue that investment when
it is looking at how to make the best use of its resources.</p>
<h2>Anti-Patterns</h2>
<p>What do I mean about an investment that isn’t directly related to the
needs of the business? Let me dig into that a little bit with some
anti-patterns.</p>
<h3>Focus on Features</h3>
<p>First, features. Or, <a href="https://fnords.wordpress.com/2011/09/28/the-next-step-for-openstack/">as Thierry Carrez puts it, “tactical
contributions”</a>. Companies
often look at the “what do we need from this project” question through
the lens of feature analysis - what are my requirements? What is missing?</p>
<p>The good thing about contributing features is that it is directly
related to business needs. The thing about your primary focus being on
new feature development is it misses the bigger picture. A community
of developers that is focused only on their own new features is not
going to get anywhere. Who is thinking about the long-term future of
the project? Who is looking at feedback from users? Who is merging the
code for these new features?</p>
<p>All of these types of activities are the basic necessities of any
software project. They are the basis through which new features get
added. You must invest in ensuring that this is happening in this
project whose future you care about.</p>
<h3>The Donation</h3>
<p>This anti-pattern is about our choice of language, and how it affects
our thinking. People often talk about “donating” code to a
project. Calling it a donation suggests you have a feeling that
the return on your investment is quite intangible. How long will you
continue with these “donations”?</p>
<h3>For Recognition</h3>
<p>Related, it’s often quite clear that a major motivation for companies
investing in open-source is the recognition they receive in doing
so. Certainly, in OpenStack, with Stackalytics we have spawned an
impressively pointless competition between companies for ranking in
contributor statistics. If you give your employees the goal of
achieving a particular ranking in the contributor statistics, you may
achieve this, but what does that really achieve?</p>
<p>The type of business value that it delivers is essentially short-term
marketing value. We don’t want investment in open-source projects to
be made out marketing budgets, since that’s possibly the least
reliable source of funding!</p>
<h3>"100% Upstream"</h3>
<p>Not so long ago, when companies were falling over themselves to
demonstrate their commitment to OpenStack, we saw an interesting
phenomenon - the “100% dedicated upstream resource”. These were
individuals and teams at various companies who were employed to
contribute to the project but - as far as I can tell - were
self-directed and deliberately kept isolated from any “downstream”
work.</p>
<p>This is an incredibly alluring opportunity for those involved! The
company is saying that the project is critically important to the
business and whatever you or your team does to help the project be
successful, that’s valuable to the company! Unfortunately, we can see
how these things go in cycles and, at some point, the company will
take a harder look at what this person or team is doing. When that
happens, the likelihood is that the company has very little visibility
into - or understanding of - the work being done and, worse, the
person or team has no experience articulating how the value of the
work is meaningful to the business!</p>
<p>There are some exceptions to this, even within Red Hat. But as a
systematic way of investing in a project … it’s unlikely to be
sustainable.</p>
<h3>Non-Profit Staff</h3>
<p>Finally, the culmination of a number of these anti-patterns - the idea
that companies should "donate" to a non-profit organization like the
OpenStack Foundation, have that organization use the donations to
employ staff who are "100% dedicated upstream resources", and the
value to companies ...? The recognition it receives for being such
generous benefactors?</p>
<p>Having some technical staff employed by the non-profit is fine. Some
of my favorite people work for the OpenStack Foundation! My preference
would be for Foundation staff to be in facilitation roles. But
certainly, you would do well to avoid this pattern for the majority of
the contributors on a project.</p>
<p>One example of this is the <a href="https://www.coreinfrastructure.org/">Linux Foundation's Core Infrastructure
Initiative</a>. In the wake of the
OpenSSL Heartbleed vulnerability, the Linux Foundation brought
together funding from a number of companies to invest resources for
key projects like OpenSSL. It's fascinating because here is a project
like OpenSSL that many, many businesses depended on, but few invested
in. What the Linux Foundation has done is positive, but I can't help
feel we have failed as a community if this is the only way to sustain
these projects. You'll notice that Red Hat hasn't joined this
initiative, despite us being supportive of it - we have always
invested in these projects directly, and think that is actually a
healthier model for the future.</p>
<h2>Think Strategically</h2>
<p>Ok, so that's a bunch of anti-patterns ... what you generally
shouldn't do. What should you do? Well, you need to think
strategically. You are choosing to have your business depend on
something you don't control, but you should not allow yourself to feel
powerless about how successful that choice will be.</p>
<h3>The Future</h3>
<p>Think about the future, the long-term. Given the business choice you
are making, for how long will your business success depend on the
success of the project? Indefinitely? What are you doing to ensure
that success?</p>
<p>One way to think about that would be a worst-case scenario. The
unimaginable happens, and the rest of our community disappears. You
are left almost completely alone maintaining the project. What you
focus on? Obviously, you wouldn't choose to depend on a project where
there is a possibility of that happening, but the thought exercise
does help you think about your priorities.</p>
<h3>Influence</h3>
<p>If you are determined to ensure the success of the project, you'll
have your own view of what that success looks like. Are you happy to
hope the community will find a direction that fits your needs, without
any input from community leaders that understand the needs of your
business?</p>
<p>At Red Hat, we talk about wearing "two hats". Given a particular
problem, a particular question of direction, you should have the
ability to understand what's good for the project overall and what's
good for Red Hat. And crucially, you should have a way to reconcile
those two views. This is not a zero sum game. Almost always, you can
find a solution that is good for both the project and Red Hat. Why
would Red Hat want a solution that is harmful to the project?</p>
<h3>Expertise</h3>
<p>In order to be successful with any technology, you need to have access
to expertise that understands the technology. There is no better
expertise than the authors or maintainers of the project. These are
the people who understand not just how the technology works now, but
how it evolved there, and how things are likely to change in the
future. They understand the pitfalls, the know the cool new tricks.</p>
<p>You can have access to that expertise by having those experts join
your team. They stop being experts pretty quickly if they stop having
time to work on the project, so their upstream responsibilities should
continue to be a significant proportion of their time. But you'd be
amazed at how their presence on the team can help the whole team be
successful.</p>
<h2>Red Hat's Model</h2>
<p>As somewhat of an aside, since this presentation is not a sales pitch,
think about Red Hat's business model, and our proposition to our
customers. Certainly part of the model is that, through our product
subscriptions, the customer is investing in the projects involved and,
by proxy, is safeguarding the future of the project, gaining a level
of influence in the project, and has access to expertise relating to
the project.</p>
<h2>A Measurable Goal</h2>
<p>Recently, on Red Hat's OpenStack team, and thanks to the influence of
<a href="http://alexis.monville.com/">Alexis Monville</a>, we've been using the
<a href="https://en.wikipedia.org/wiki/OKR">"Objectives and Key Results"</a>
framework for using well-defined, measurable goals to guide our
teams. We started brainstorming on these OKRs almost a year ago, and
from those early discussions, I wondered “how do we frame an
measurable goal around our investment in upstream?”. What indicator
could we use to know whether we were on the right track, and to ensure
we’d stay on the right track?</p>
<h3>Our Vision</h3>
<p>Our first thoughts on this was to look at data like our position in
the contributor statistics, or the number of core contributors, PTLs,
or TC members on our team. None of this sat well with us, though,
because we have seen that these type of goals don’t drive the right
behaviors. When we started drafting a vision statement for each goal,
I wrote:</p>
<p>“Teams of engineers with the required motivation, an understanding of
Red Hat's vision, and empowered with the necessary time,
encouragement, and recognition, working upstream to drive and
influence many diverse parts of OpenStack.”</p>
<p>And there it sat for a while, us all lacking ideas on how to measure
this. Quite recently, Russell Bryant and Doug Hellmann hit on a
promising solution. Survey the team with a small set of questions and
use that sentiment of our measure of success with this goal.</p>
<h3>The Questions</h3>
<p>The questions that Russell and Doug developed are:</p>
<ol>
<li>Does your team have effective input into the features accepted and design decisions made upstream?</li>
<li>Does your team have effective input into bug fixes and backports made upstream?</li>
<li>Does your team have effective input into discussions related to processes, schedules, requirements, infrastructure management, and other decisions made by the upstream OpenStack community in general?</li>
<li>What level of investment does your team make in upstream work?</li>
</ol>
<p>The allowed answers are “not enough”, “just right”, “too much”, and
“don’t know”. We also had a free-form comments section which is
helping us gain insight into the results.</p>
<p>Notice one important aspect of this - we are asking individuals about
the effectiveness of the investment their team is making
upstream. That reflects the vision above.</p>
<h3>The Results</h3>
<p>We only recently started running this survey, and we will do one every
release cycle. So far, we’ve had over 70 responses, so that’s a pretty
good start.</p>
<p><img alt="Level of Invesment" src="https://chart.apis.google.com/chart?cht=p&chs=500x250&chdl=Not+enough%7CAbout+right%7CToo+much%7CDon%27t+know&chl=16.7%25%7C79.2%25&chco=3366CC%7CDC3912%7CFF9900%7C109618&chp=0.436326388889&chtt=Level+of+Investment&chts=000000,24&chd=t:12,57,1,2" title="Level of Investment"></p>
<p>The really interesting judgment call we need to make is what
percentage of “just right” answers do we want to aim for? It would
seem misguided to aim for 100% - do we really think that it’s
important that every individual feels we’re striking the right
balance? We’ve arbitrarily chosen 80% as our target, which means we
feel pretty good about where we’re at, but there’s still opportunities
for improvement.</p>
<h2>Sustainability</h2>
<p>One thing I’m loving about my job these days is the direct exposure I
have to Red Hat customers who are choosing to invest in
OpenStack. Deciding to build a significant cloud for production use is
a huge decision, and no-one takes it lightly. It’s exciting, and
companies doing this can look forward to a true transformation in how
they think about IT resources. The truly daunting part is the sense of
responsibility that comes with it.</p>
<p>These are long-term choices being made, and they’re being made because
people believe in the future of this project. Red Hat takes the
responsibility seriously by making choices that we hope will ensure
the project has a long, sustainable future.</p>November 17th OpenStack Foundation Board Meeting2016-11-21T09:00:00+00:002016-11-21T09:00:00+00:00markmctag:crustyblaa.com,2016-11-21:/november-17-openstack-foundation-board-meeting.html<p>I had <a href="https://crustyblaa.com/tag/openstack-foundation-board-meeting.html">previously posted summaries of most board
meetings</a>,
but had stopped doing this when I was appointed as Red Hat's
representative on the board. Lately, I've sensed folks at Red Hat
becoming more interested in the workings of the Foundation, so I
figured it might be useful to start …</p><p>I had <a href="https://crustyblaa.com/tag/openstack-foundation-board-meeting.html">previously posted summaries of most board
meetings</a>,
but had stopped doing this when I was appointed as Red Hat's
representative on the board. Lately, I've sensed folks at Red Hat
becoming more interested in the workings of the Foundation, so I
figured it might be useful to start doing this again.</p>
<p>The <a href="https://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a>
met for a two hour conference call last week. The usual disclaimer
applies - this my informal recollection of the meeting. It’s not an
official record.</p>
<h3>Gold Member Applications</h3>
<p>The first half of the meeting was given over to an Executive Session,
after which the board voted to <a href="http://www.openstack.org/news/view/284/investment-in-openstack-accelerates-as-foundation-welcomes-china-telecom-inspur-and-zte-as-gold-members">approve China Telecom, Inspur, and ZTE
as Gold
Members</a>
of the Foundation.</p>
<p>This completes the handling of the 7 Gold Member applications that
were presented to the board at the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/24Oct2016BoardMeeting">October 24
meeting</a>
in Barcelona. 99Cloud, China Mobile, City Networks, and Deutsche
Telecom were approved at that meeting.</p>
<p>For the first time, the Foundation now has reached the maximum of <a href="https://www.openstack.org/foundation/companies">24
Gold Members</a>. We will
only be able to consider applications for new Gold Members if an
existing member departs, or if we go through a difficult process to
change the limit in <a href="http://www.openstack.org/legal/bylaws-of-the-openstack-foundation/">our
bylaws</a>.</p>
<h3>User Committee Proposal</h3>
<p>Next up, Edgar Magana updated the board on some <a href="https://docs.google.com/document/d/1QmLOeseAkjBWM_TXsUeKBErNaSHnuZp81II0T71ARfo">proposed changes to
the User
Committee</a>
that was first discussed at the October meeting.</p>
<p>Currently the bylaws describe the User Committee is an "advisory
committee" of three members appointed by the Board and TC, which
prepares regular reports for the Board and TC. The idea with this
proposal is to make the User Committee a parallel entity to the TC,
with its members chosen through an election of Active User
Contributors (AUC).</p>
<p>The bylaws changes outline how the User Committee has at least five
members, that elections of Active User Contributors (AUCs) are held
every six months, an affiliation limit for members of the UC, how AUC
status is determined, and more.</p>
<p>The hope is that feedback will be collected as comments in the
<a href="https://docs.google.com/document/d/1QmLOeseAkjBWM_TXsUeKBErNaSHnuZp81II0T71ARfo">proposal
document</a>
and that the Board will vote on the proposal during our December 6
meeting. A vote of the Board is sufficient to enact the changes.</p>
<p>One point of discussion was whether bylaws changes are necessary at
all. The hope when the bylaws were originally drafted was that the
User Committee would have plenty of lattitude to evolve via changes to
<a href="http://www.openstack.org/legal/bylaws-of-the-openstack-foundation/">the UC
charter</a>. Edgar
made the point that the key change is for the UC to become a parallel
entity to the TC rather than an advisory committee to the Board and
TC.</p>
<h3>"Futures" Discussion</h3>
<p>Next, Toby Ford gave a presentation on a topic he had proposed along
with Imad Sousou, Allison Randal, and Mark Baker. Toby introduced the
topic by saying that as any project evolves and matures, it is
important to reflect on competitive threats and the need to evolve the
project.</p>
<p>Toby talked about a wide set of competitive threats to OpenStack from
the container ecosystem (including Kubernetes and Mesos), to other
storage implementations, to various projects in the Telco world
(<a href="https://github.com/nfvlabs/openvim">OpenVIM</a>,
<a href="https://blog.acolyer.org/2016/06/17/e2-a-framework-for-nfv-applications/">E2</a>,
<a href="http://www.3gpp.org/">3GPPP</a>), and the obvious challenge of AWS,
Azure, and Google with AWS revenue over $10B.</p>
<p>Toby moved on to talk about "the good and the bad of the Big Tent",
describing the need for a balance between diversity and innovation
versus consolidation and refactoring. Toby made the case that we're
seeing a lot of expansion in the Big Tent, but not consolidation and
refactoring. He expressed particular concern about how and whether
core projects will be able to evolve, especially if the Big Tent does
not allow for competitors to existing projects.</p>
<p>Toby then put on his "AT&T hat" and talked about the challenges for
OpenStack from an AT&T perspective - the need to get to a control
plane that scales to 10k servers, the problem of managing over 100
different production sites, the challenge of more and more happening
"at the edge" with 5G, innovation happening in the networking space
and how it relates to OpenStack, and the unsolved problem of keeping a
production environment current with latest OpenStack releases.</p>
<p>To wrap up his presentation, Toby listed a few "possible
recommendations" the Board could make to the technical community - (1)
allow core projects to have competition within the Big Tent, (2) a
mechanism to incubate new approaches, and (3) a mechanism to
reationalize, clean up, or refactor.</p>
<p>What followed was a pretty lively discussion that covered much ground,
over and beyond the concerns raised by Toby. While the success of
OpenStack in bringing together such a diverse set of interests was
acknowledged, there was a definite frustration that some form of
necessary change is failing to emerge naturally, and whether it falls
on the Board to try to at least try to clearly articulate the problem
at a strategic level.</p>
<p>Jonathan tried to focus the conversation by giving his perspective
that the Big Tent concerns had tied down until "recent comments in the
media", and that is probably a distraction from the concerns people
are expressing about core projects like Nova and Neutron. He was at
pains to say that this isn't about project teams doing bad work - we
continue to make huge progress in each release, and valuable work is
being done. However, we're definitely seeing frustration from some
quarters that it is difficult to influence the community with their
ideas.</p>
<p>Jonathan warned that talking too much in the abstract creates the risk
that the discussion will go around in circles. Despite this, the
discussion never did go into any particular detail on where we've
witnessed the problems we were discussing. From my own perspective, it
was clear that much of frustration was as a result of how the
<a href="https://clearlinux.org/ciao">Ciao</a> and
<a href="https://wiki.openstack.org/wiki/Gluon">Gluon</a> projects have been
received, but we're failing to learn anything significant from those
experiences by not talking about them in detail.</p>
<p>By the end of the discussion, we had agree to collaborate on a
concrete description of specific problem areas, the questions they
raise, and some possible solutions. The hope is that we would complete
this before our December meeting, and we may then plan a longer
meeting (possibly face-to-face, possibly with the TC) to dig into
this.</p>RDO and Upstream Packaging2015-06-12T12:27:00+01:002015-06-12T12:27:00+01:00markmctag:crustyblaa.com,2015-06-12:/rdo-and-upstream-packaging.html<p>Derek mentioned "upstream packaging" on <a href="http://meetbot.fedoraproject.org/rdo/2015-06-10/rdo.2015-06-10-15.00.html">this week's packaging
meeting</a>
and asked RDO packagers to participate in the upstream discussions. I
thought some more context might be useful.</p>
<p>First, a little history ...</p>
<p>When I first started contributing to OpenStack, it
<a href="https://review.openstack.org/708">briefly</a> looked like I would need to
make some Ubuntu packaging …</p><p>Derek mentioned "upstream packaging" on <a href="http://meetbot.fedoraproject.org/rdo/2015-06-10/rdo.2015-06-10-15.00.html">this week's packaging
meeting</a>
and asked RDO packagers to participate in the upstream discussions. I
thought some more context might be useful.</p>
<p>First, a little history ...</p>
<p>When I first started contributing to OpenStack, it
<a href="https://review.openstack.org/708">briefly</a> looked like I would need to
make some Ubuntu packaging updates in order to get a Nova patch landed.
At the Essex design summit a few weeks later, I raged at Monty Taylor
how ridiculous it would be to require a Fedora packager to fix Ubuntu
packaging in order to contribute a patch. I was making the point that
upstream projects should leave packaging to the downstream packaging
maintainers. Upstream CI quickly moved away from using packages after
that summit, and I've heard Monty cite that conversation several times
as why upstream should not get into packaging.</p>
<p>Meanwhile, Dan Prince was running the Smokestack CI system at the time,
which effectively was being treated as OpenStack's first "third party
CI". Interestingly, Smokestack was using packages to do its deployment,
and for a long time Dan was successfully keeping packaging up to date
such that Smokestack could build packages for patches proposed in
gerrit.</p>
<p>And then there's been the persistent interest in "chasing trunk".
Operators who want to practice Continuous Deployment of OpenStack from
trunk. How does packaging fit in that world? Well, the DevOps mantra of
doing development and CI in environments that model your production
environment applies. You should be using packaging as early on in your
pipeline as possible.</p>
<p>My conclusion from all of that is:</p>
<ol>
<li>A key part in building a Continuous Delivery pipeline for OpenStack
is to practice continuous package maintenance. You can glibly say
this is "applying a DevOps mindset to package maintenance".</li>
<li>How awesome would it be if OpenStack had "upstream infrastructure
for downstream package maintainers". In other words, if downstream
package maintainer teams could do their work close to the upstream
project, using upstream infrastructure, without disrupting
upstream development.</li>
</ol>
<p>I think the work that Derek, Alan, Dan, John, and everyone else has been
doing on Delorean is really helping RDO maintainers figure out how to
practice (1). I first started maintaining Fedora packages for Fedora
Core 2, so IMO what RDO is doing here is really dramatic. It's a very
different way of thinking about package maintenance.</p>
<p>As for (2), this where we get back on topic ...</p>
<p>At a <a href="https://etherpad.openstack.org/p/YVR-ops-packaging">Design Summit session in
Vancouver</a>, the idea
of maintaining packaging using upstream infra really took hold. Thomas
Goirand (aka zigo) <a href="https://review.openstack.org/185187">proposed the creation of a "distribution packaging"
team</a> and this triggered a <a href="http://lists.openstack.org/pipermail/openstack-dev/2015-June/thread.html#65545">healthy
debate on
openstack-dev</a>.
Derek has since pushed <a href="https://review.openstack.org/189497">a WIP patch showing how RDO packaging could be
imported</a>.</p>
<p>There's a clear desire on the part of the Debian and Ubuntu package
maintainers to collaborate on shared packaging, and it sounds like this
goal of further collaboration is one of the primary motivators for
moving their packaging upstream. This makes a lot of sense, given the
shared heritage of Debian and Ubuntu.</p>
<p>The RDO team is enthusiastic about adopting this sort of upstream
workflow, but the Debian/Ubuntu collaboration has added an entirely new
aspect to the conversation. Despite the fact that RDO and SUSE platforms
have little in the way of shared heritage, shouldn't the RDO and SUSE
packaging teams also collaborate, since they both use the RPM format?
And perhaps deb and rpm maintainers should also collaborate to ensure
consistency?</p>
<p>To my mind, the goal here should be to encourage downstream packaging
teams to work closer to the upstream project, and have downstream
packaging teams collaborate more with upstream developers. This is about
upstream infrastructure for downstream teams, rather than a way to force
collaboration between downstream teams, simply because forced
collaboration rarely works.</p>
<p>For me, what's hugely exciting about all of this is the future prospect
of the package maintainers for different platforms adopting a
"continuous packaging" workflow and working closely with project
developers, to the extent that packaging changes could even be
coordinated with code changes. With its amazing infrastructure,
OpenStack has broken new ground for how open-source projects can
operate. This could be yet another breakthrough, this time demonstrating
how a project's infrastructure can be used to enable an entirely new
level of collaboration between package maintainers and project
developers.</p>Network Function Virtualization - The Opportunity for OpenStack and Open Source2014-10-02T20:28:00+01:002014-10-02T20:28:00+01:00markmctag:crustyblaa.com,2014-10-02:/network-function-virtualization-the-opportunity-for-openstack-and-open-source.html<p>This week's <a href="https://www.openstack.org/blog/2014/09/telcos-mobilizing-to-drive-nfv-adoption/">launch of
OPNFV</a>
is a good opportunity to think about a simmering debate in the OpenStack
developer community for a while now - what exactly does NFV have to do
with OpenStack, and is it a good thing?</p>
<p>My own “journey” on this started exactly one year ago today …</p><p>This week's <a href="https://www.openstack.org/blog/2014/09/telcos-mobilizing-to-drive-nfv-adoption/">launch of
OPNFV</a>
is a good opportunity to think about a simmering debate in the OpenStack
developer community for a while now - what exactly does NFV have to do
with OpenStack, and is it a good thing?</p>
<p>My own “journey” on this started exactly one year ago today when I
visited a local Red Hat partner to talk about OpenStack and, towards the
end of our Q&A, I was asked something like “will OpenStack support
NFV?”. I’d never heard of the term and, when the general idea was
explained, I gave a less than coherent version of “OpenStack implements
an elastic cloud for cattle; this sounds like pets. Sorry”. After the
meeting, the person who asked the question forwarded me an NFV
whitepaper from October 2012 and, glancing through it, most of it went
right over my head and I didn’t see what it had to do with OpenStack.</p>
<p>Since then, Chris Wright has been patiently talking me through this
space and gently trying to get me over my initial skepticism. Chris
would say that our conversations has helped him refine how he explains
the concepts to open-source developers, and I think he really nailed it
in his keynote at the Linux Foundation’s Collaboration summit in April.</p>
<p>[embed]https://www.youtube.com/watch?v=SAimeBttapA[/embed]</p>
<p>In his keynote, Chris talks about the benefits of collaboration in
open-source and walks through all of the various aspects of how the
networking industry is changing, and how open-source is playing a key
part in all of those changes. He covers, and simplifies:</p>
<ul>
<li>Taking the current architecture of proprietary, expensive, complex,
difficult to manage forwarding devices (like routers) and how SDN
(Software Defined Networking) aims to “put an API on it”. This is
what’s meant by “disaggregation of the control plane and data
plane” - that forwarding devices become devices which are controlled
by open standards, and allows your distributed system of forwarding
devices to be controlled and automated.</li>
<li>NFV (Network Function Virtualization) as a shift in the telco
data-center world which embraces many of the lessons that the
elastic infrastructure cloud has taught the IT industry. More on
that below.</li>
<li>Changes in the “data plane” world, where we’re starting to see the
network device market mimic the x86 server market such that these
devices can be “white box” servers running open-source software.
Again that disaggregation word, but this time it’s about
“disaggregation of hardware and software” and how the software part
can be open-source implementations of optimized packet-forwarding
capabilities which we’re used to seeing implemented in expensive and
proprietary hardware appliances.</li>
</ul>
<p>But let’s focus here on NFV.</p>
<p>I real don’t know much about the telco industry, but what Chris has me
imagining now is data-centers full of proprietary, black-box hardware
appliances which are collectively know as “network functions” or “middle
boxes”. These boxes are used for everything from firewalls, NAT, deep
packet inspect (DPI), the mobile packet core, etc. These are software
applications trapped in hardware. They’re expensive, proprietary, slow
to roll-out, don’t always scale well and are hindering telco service
providers as they attempt to react to a rapidly changing market.</p>
<p>NFV is about completely re-thinking the architecture of these
data-centers. This is the telco industry re-imaging their data centers
as elastic infrastructure clouds running their “network functions” as
virtualized, horizontally scalable applications on these clouds. The
exciting - simply stunning - aspect of all of this for me as an
open-source advocate, is that the telco industry is settling on a
consensus around an architecture involving open-source generally and
OpenStack specifically.</p>
<p>Say that again? These huge telcos want to rebuild their entire data
centers with OpenStack and open-source? Yes.</p>
<p>If, like me, you want to see open-source change the IT world into one
where we all embrace the opportunity to collaborate in the open, while
still successfully building building businesses that serve our users’
needs … then this sounds pretty cool, right?</p>
<p>If, like me, you want to see OpenStack as the standard platform from
which many of the worlds’ elastic infrastructure clouds are built ...
then this sounds like a no-brainer, right?</p>
<p>Well, the thing we need to bear in mind is that these applications (i.e.
network functions) are pretty darn specialized. They need to have a high
level of performance, determinism and reliability. But that does not
necessarily mean they are “pets” and missing one of the key points of an
elastic cloud.</p>
<p>Let’s take the reliability requirement - when these network functions
are implemented as horizontal scale-out applications, they will look to
achieve high levels of reliability in the same way that typical cloud
applications do - with each tier of the application spread across
multiple failure domains, and by spreading application load
horizontally. Telcos will just want to take this further, with faster
and more deterministic response to failures, while also avoiding any
compromise to application's performance. For example, you’ll see a lot
of interest in how instances are scheduled to take to into account
affinity and anti-affinity within an instance group.</p>
<p>The performance requirement is largely about high-performance packet
processing. How to get a packet off the network, into a VM, processed
quickly and back out again on the network. One of the techniques being
pursued is to give VMs direct physical access to the network via SR-IOV
which, in turn, means the compute scheduler needs to know which physical
networks the NICs on each compute node has access to.</p>
<p>The deterministic requirement is about predictable performance. How to
avoid the vagaries of the hypervisor and host OS scheduler affecting
these performance-sensitive applications? You’ll see work around
allowing operators to define flavors, and application owners to define
image properties, which between them control things like vCPU topology,
vCPU to pCPU pinning, the placement of applications in relation to NUMA
nodes and making huge pages available to the applications. Compare to
Amazon’s memory-optimized and compute-optimized flavors, and imagine
this being taken a step further.</p>
<p>Oh, and another requirement you’ll see come up in this space a lot is …
IPv6 everywhere! I’m certainly down with that.</p>
<p>Want to learn more about the work involved? See <a href="https://wiki.openstack.org/wiki/Teams/NFV">the OpenStack NFV
team's amazing wiki page</a>
which goes into excruciating detail.</p>
<p>The more you dig into the specifics of what we’re talking about here,
start breaking this down into tangible concepts without all the acronyms
and buzzwords, you start to realize that this really is the telco world
embracing everything that OpenStack is all about, but just pushing the
envelope a bit with some requirements which are a pretty natural
evolution for us, but we might not otherwise have expected to come about
for some time yet.</p>
<p>I guess the summary here is that if you're skeptical, that's cool ...
you're not alone. But please do take the time to see through the
complexity and confusion to the simple fact we're poised to be a key
part in turning the telco data-center, and how this is just another
exciting part of <a href="https://wiki.openstack.org">our goal to "to produce the ubiquitous Open Source
Cloud Computing platform"</a>.</p>An Ideal OpenStack Developer2014-06-06T14:49:00+01:002014-06-06T14:49:00+01:00markmctag:crustyblaa.com,2014-06-06:/an-ideal-openstack-developer.html<p><em>(This is a prose version of a talk I gave at OpenStack meetups in
Israel and London recently. Apologies for the wordiness.)</em></p>
<p>In a recent update Jonathan gave to the Board of Directors, we described
how OpenStack has had 2,130 contributors to date and 466 of those are
active …</p><p><em>(This is a prose version of a talk I gave at OpenStack meetups in
Israel and London recently. Apologies for the wordiness.)</em></p>
<p>In a recent update Jonathan gave to the Board of Directors, we described
how OpenStack has had 2,130 contributors to date and 466 of those are
active on a monthly basis. That’s an incredible statistic. There’s no
doubt OpenStack has managed to attract an unusual number of contributors
and, for such a complex project, made it relatively easy for them to
contribute.</p>
<p>However, this isn’t just a numbers game. I often hear mutterings that a
much smaller, focused group could achieve the same velocity that
OpenStack is achieving. In some sense that’s true, but I think that the
diversity of interests and priorities is the energy that a community
like OpenStack thrives on.</p>
<p>The question then is how to improve the overall quality of our large
number of contributors. In order to do that, we need to be able to set
expectations. What do we expect and value from our contributors?</p>
<p>What I’m going to attempt to do here is define The Prototypical
OpenStack Developer. The ideal that we should aspire to. The standard
that all contributors should be held to.</p>
<p>(But … bear with me here. I’m being a little tongue-and-cheek.)</p>
<p>Ok. Where do we start? How do we begin to forge this hero from the raw
resources we are presented with?</p>
<p>Let’s start with the basics. The breadth and depth of knowledge you need
on a variety of computing topics.</p>
<p>On <strong>virtualization</strong>, you could start with KVM. You should know about
CPU extensions such as Intel’s VT-x and I/O virtualization with VT-d and
PCI SR-IOV. Some knowledge of the history of software based
virtualization and paravirtualization would be nice context too. Now
understand the responsibilities of the KVM kernel module versus the
userspace component, qemu. How does qemu emulate various devices? How
does live migration work? How does a hypervisor use page table flags to
track dirty pages during a migration?</p>
<p>And there’s probably little point in understanding all of this without
understanding the x86 architecture in some detail. Understanding how it
compares to RISC architectures would be no harm. Memory segmentation,
MMUs, page tables are all fun topics. You really can’t get into this
without learning a bit of assembly, at least the basic idea. The history
of x86, from real/protected mode to modern day PAE or x86-64 are all
important to understand. Ignore Itanium, though. It’s not enough to just
understand the CPU, though, you need to go beyond and think about how
that CPU interacts with peripherals using DMA and buses like PCI.</p>
<p>And, honestly, if you go this far you may as well understand basic
digital systems theory, like how you can construct a counter or register
from a set of basic logic gates ...</p>
<p>Woah, I think I’ve digressed a little. That’s virtualization. Do the
same for <strong>storage and networking</strong>. I’ll leave that as an exercise for
the reader.</p>
<p>That’s just the concept behind the basic resources managed by OpenStack,
though. It’s a pretty complicated <strong>distributed system</strong>, so it’s pretty
essential you do some reading on that topic. What do terms like “quorum”
and “consensus” mean? Why do people describe the Paxos algorithm as
“quicksort of distributed systems”? What do people mean when they
describe OpenStack as a “shared nothing” architecture, and are they
crazy? How would you describe OpenStack’s approach to fault tolerance?</p>
<p>And obviously related to all of this is the need for deep knowledge of
<strong>databases and messaging systems</strong>. We seem to have a large number of
ex-MySQL consultants on this project, but don’t let that be an excuse.
You know what foreign keys and cross-table joins are, right? And you
really need to know the kind of operations which will simply lock
individual rows rather than an entire table. For messaging, there’s a
little research you can do there. We’re all about AMQP in OpenStack, but
there’s been a few other messaging protocols in the past. My personal
favorite is CORBA. What’s the difference between a broker, router and
peer-to-peer based architecture? What’s this “fanout” and “topic” things
we talk about in messaging? Incidentally, you know that we’re not
actually using the standard AMQP protocol in OpenStack, right?</p>
<p>You needn’t have touched a line of code at this point. But, if you’re
going to contribute to OpenStack, you need to code, right? Almost
certainly in <strong>Python</strong>, but we like ourselves a little Bash too. With
Python, it’s important to understand not just the syntax from the most
basic to the more advanced topics like iterators, decorators, context
managers and metaclasses. You also need to have a good knowledge of the
huge number of python libraries out there, inside and outside the core
Python distribution. We need true Pythonistas. Oh, and we’re in the
process of porting to Python 3, so make sure you understand the
differences between Python 2 and 3.</p>
<p>But wait, wait. That’s no good. You can’t just dive straight into
Python. You need to start with C. Allocate and free your own memory,
damnit. You can’t go through life without learning about pointers. Now
learn how to use threads and the various synchronization primitives out
there, and why it’s all just terrible. Now learn about asynchronous I/O
techniques; what an event loop is, how you use select() and non-blocking
sockets to write a single-threaded server which processes requests from
multiple clients. Oh, Richard Stevens. My hero. Don’t be afraid to read
a few RFCs.</p>
<p>Speaking of authors, we forgot algorithms. Yes, those. Just carefully
study all three volumes of Knuth.</p>
<p>Now, before returning to Python, perhaps you should implement a REST API
in Java using JAX-RS and a web UI using Ruby on Rails. Hang out with the
cool kids and port your UI to Sinatra, before realizing that’s not cool
anymore and switching to Node.js.</p>
<p>You might be ready to contribute some code to OpenStack at this point.
But, I hate to think of anyone writing software without having a full
appreciation of the <strong>user experience design</strong> we’re driving towards. We
don’t want the inmates running the asylum, do we? Which “personas” are
we designing for? “As a web developer, I want to launch a virtual
machine to test my code in.”</p>
<p>Wait, we forgot <strong>tools</strong>. You can’t get anything done without knowing
your tools. You’re going to do all of your work on Linux, whether that
be in VMs or by running Linux on your main machine. If you’re a serious
person, you need to learn emacs. You’re going to become very close
friends with grep and sed, so learn yourself regular expressions. Lazy
and greedy regexs, both. You know how to do a HTTP POST with curl,
right?</p>
<p>Ah, git! Oh, the glorious <strong>git</strong>! You can never learn too much about
git. It’s the gift that keeps on giving. If you think I’m joking, spend
some time getting to know interactive rebasing. Reordering, editing,
squashing and splitting commits! Re-writable history! Where have you
been all my life? No git detail is too obscure to ignore. Learn how a
tilde is different from a caret in revision parameters. How you can
delete branches by leaving out the first part of a refspec in a
git-push. Force override, exciting! Is your mind blown yet? No? Find out
how git’s reflog is a history of history!</p>
<p>(Give me a second to calm down, here)</p>
<p>Now, you’ve got to realize something. Based on everything you’ve learned
so far, you could probably write OpenStack on your own. But that’s not
what’s going on here. You’re <strong>collaborating</strong>. You’re following a
process. How we collaborate and why we follow certain processes is a
more complex, involved and undocumented topic than anything you’ve
learned so far.</p>
<p>To really understand how we get stuff done in OpenStack, you need to be
steeped in open source culture. Understand what we mean when we say
things like “rough consensus and running code” or “do-acry”.</p>
<p>Perhaps start by following the linux-kernel mailing list for a few
months, watching how controversial discussions are worked through and
the subtleties that determine who holds the balance of power and
influence. Don’t worry if you’re shocked and appalled by how unfriendly
it all seems, you’re not the first. If that’s your one take-away from
the kernel, that was time well spent. Now seek out friendlier
communities and understand how they get stuff done. Compare them to
OpenStack and ask yourself questions like “how does our reliance on
voting to make decisions compare to other communities?” or “why does
there seem to be less flamewars in OpenStack than elsewhere?”.</p>
<p>The history of open source is important, will inform how you engage with
OpenStack and that, in turn, will influence how OpenStack evolves. Learn
about the “free software” versus “open source” camps, and how those
<strong>philosophies</strong> relate to the choice of copyleft licenses like the GPL
versus permissive licenses like Apache, MIT or BSD. Are you in this for
the freedom of users of your code, or are you in it to build
collaborative software development communities? That contributor
agreement you were asked to sign before you contributed to OpenStack -
how do you feel about that?</p>
<p>Think about the different <strong>governance models</strong> that open-source
communities adopt. Learn about benevolent dictators, project management
committees, “commit bit”, consensus based decision making and the pros
and cons of our representative democracy model.</p>
<p>Learn about the <strong>release processes</strong> various projects use. Time based
versus feature based. Rapid release cycles with merge windows. Planning
periods, feature freezes, release candidates, stable branches. How do
different distros do this when there are so many maintainers and
packages involved? We use Python a lot, how do they coordinate their
release cycles?</p>
<p>That’s all very well, but it’s important not to be blind to the <strong>world
outside</strong> open source. Understand how extreme programming and agile
software development evolved. Read the Agile Manifesto. Understand how
this all relates to Continuous Integration, Continuous Delivery and
DevOps. We’re operating in a much different context, but is code review
our variant of XP’s pair programming? Is our gated master superior to
traditional post-commit CI?</p>
<p>You can now consider educated to a basic level. But is that enough to be
an effective contributor? Do you now have everything you need to make an
impact? No, far from it. The hardest part is learning to be a <strong>good
human</strong>. You need to have superb communication skills, in English of
course, mostly written communication skills for mailing list, gerrit and
IRC discussions. We do meet twice a year in design summits, so you need
to be able to present and defend your ideas in person too. You need to
work on that Irish mumble of yours.</p>
<p>More than that, though, you need to understand people. You need to know
when to be empathetic, when to be pragmatic and when you be dogmatic.
When is someone’s -1 on your patch likely to be an intractable veto and
when is it simply a take-it-or-leave-it suggestion? What fights are
worth fighting? How can you build up kudos points by assisting your
fellow contributors and when is the right time to call in some favours
and spend those kudos points?</p>
<p>Ok, we’re ready to go! How do we <strong>put all of this into practice</strong>?</p>
<p>Probably the best way to start contributing to the project is by doing
<strong>code reviews</strong>. You should probably be spending at least a couple of
hours on code review every day. Not just because the number of code
reviewers on a project has the greatest influence on its velocity, but
also because its the best way to start building trust with your fellow
contributors. If you can show yourself as thoughtful, committed and
diligent through your code reviews, then other code reviewers will be
much more inclined to prioritize your patches and less carefully
scrutinize your work.</p>
<p>A good code reviewer manages to simultaneously focus on the little
details while also considering the big picture. Try not to just leave +1
on patches, but instead a little commentary that shows the kind of
things you’ve taken into consideration. Why should anyone trust that
your +1 was the result of 2 hours of careful analysis, research and
testing rather than just 2 minutes of coding style checking?</p>
<p>Also, think about who you are building up trust with. As a new code
reviewer it’s probably more fruitful to provide helpful input on some
meaty patches from some of the lead developers on the project. Then
again, patch triage can be hugely helpful too - catch obvious problems
in patches before the core reviewers ever get to the patch. Don’t forget
to mentor new contributors as a code reviewer, though. Code review is
the face of the project to these contributors and its your opportunity
to show how you can lead by example.</p>
<p>Now, you obviously want to <strong>contribute code</strong>. Find some gnarly bug to
fix, perhaps some race condition only rarely seen during automated
tests. With all the code reviewing you’ve been doing, you’ve acquired
excellent taste in coding and your work will no doubt live up to those
standards. Don’t forget to write a detailed, helpful commit message and
include a unit test which would catch any regression of the issue. If
this is a more substantial change, you must split your change into
smaller chunks where each patch represents a logical step in your
progression towards the final result.</p>
<p>If you’re making a substantial addition like a new feature or a
re-architecture, you need to document your design in some detail in a
blueprint. Make sure someone reading the spec can quickly understand the
problem you’re trying to solve, why it’s important and the general idea
behind your solution. Then make sure there’s enough background
information included that a reviewers work is made easy. Include the use
cases, any relevant history, related discussions or bugs, alternative
approaches considered and rejected and any security, upgrade,
performance or deployer impact. Describe how your work will be tested
and what documentation changes will be required.</p>
<p>While we’re on the subject of blueprints, don’t forget that these too
need reviewers. Most projects now review the specs associated with
blueprints using gerrit and so this is a way for you to demonstrate your
design skills and catch things which no-one else has yet considered.</p>
<p>Back to code, though. Yes, it’s important to contribute to the various
integrated service projects like Nova, Neutron, Swift and whatnot.
However, there are a bunch of other areas where code contributions are
always needed. For a start, the client projects are always forgotten.
Then there’s the cross-project technical debt that the Oslo program is
hard at work cleaning up. We’re also gradually porting all of OpenStack
to Python 3, and this is going to be a multi year effort requiring the
help of many.</p>
<p>We also place a huge emphasis on automated testing in OpenStack, and the
awesome CI system we have doesn’t come from nowhere. You should always
be ready to jump in a contribute to the infrastructure itself, tools
like devstack-gate, zuul, nodepool or elastic-recheck. And, last but not
least, our functional test suite, Tempest, is always desperately in need
of more contributions to increase our test coverage.</p>
<p><strong>Security</strong> is critical in a public-facing service like OpenStack, and
there are several ways you should contribute in this area. Firstly,
there is a small vulnerability management team which collaborates with
each project’s -coresec team to handle privately reported security bugs,
ensuring a fix is prepared for each supported branch before a
coordinated, responsible disclosure of the issue first to vendors and
then the wider world. Important work is this. There’s also a security
group which is trying to bring together the efforts of interested
parties to prepare official notices on security issues that aren’t
actual vulnerabilities, develop a threat analysis process for OpenStack
and maintain the OpenStack Security Guide. They need your help! Most
importantly, though, you need to be security conscious as you write and
review code. There’s a good chance you’ll find and report an existing
vulnerability during the course of your work if you keep your eyes open!</p>
<p>And then there’s <strong>docs</strong>, always the poor forgotten child of any open
source project. Yet OpenStack has some relatively awesome docs and a
great team developing them. They can never hope to cope with the
workload themselves, though, so they need you to pitch in and help
perfect those docs in your area of expertise.</p>
<p>I mentioned <strong>bugs</strong>. We must not forget the bugs! Bugs are one way
users can provide valuable contributions to the project, and we must
ensure these contributions are valued so that users will continue to
file bugs. With over 700 configuration options in Nova alone, the
project can’t possibly test all possible combinations by itself so we
rely on our users to test their own use cases and report any issues as
bugs. You should help out here by setting aside some time every day to
triage new bugs, making sure enough information has been provided and
the bug has been appropriately tagged, categorized and prioritized.</p>
<p>Along those same lines, <strong>users</strong> often struggle with issues with aren’t
obviously or necessarily bugs. You should also pay attention to forums
like ask.openstack.org or the openstack-operators mailing list. Any
outreach you can do to help users be successful with OpenStack will pay
massive dividends in the long run, even just in terms of your
understanding which issues are most important to real users. This
outreach should extend to your attending OpenStack meetups, giving
presentations on your work and listening to what users have to say.</p>
<p>Speaking of mailing lists, we have a hugely active <strong>openstack-dev
mailing list</strong>, with over 2500 emails in April alone. This is the center
of all activity happening in OpenStack at any time. You really must
track what’s happening there and engage where you can help move things
forward positively. It’s a struggle to keep up, but it really isn’t an
option.</p>
<p>However, one of the side effects of openstack-dev being overloaded is
that many important conversations now happen <strong>IRC</strong>. You can’t expect
to be around for all of those, so make sure to remain connected and log
all channels so you can catch up later.</p>
<p>Because conversations can be spread around multiple places, it can be
helpful to link all of these conversations with little breadcrumbs. A
mailing list thread might reference a gerrit review, which might
reference a log of an IRC conversation, which might reference a blog
post, which might reference a bug, which might reference a previous
commit message which referenced a previous mailing list thread.</p>
<p>Don’t be fooled into thinking IRC is all about the serious stuff,
though. It’s also a place where you can get to know your fellow
contributors on a personal level and build up yet more of that all
important trust. You will make friends working on OpenStack and some of
those <strong>friendships</strong> will last longer than your involvement in
OpenStack itself. That’s a hugely positive sign in any community. Beware
of forming cliques, however. We need this community to be open to the
most diverse set of contributors, and not all of those will buy into
US-centric young white male geek humour, for example.</p>
<p>Speaking of cliques, it’s popular to accuse OpenStack developers on
being so self-absorbed that the needs of real operators and users are
ignored. That OpenStack developers aren’t held responsible for the real
world consequences of the decisions they make. “You write code
differently when you carry a pager”. Lorin Hochstein proposed an “Adopt
a Dev” program where operators could invite individual developers to
shadow them for a few days and share their experience in the terms of a
summary, bug reports and blueprints. Basically, you should take any
opportunity you can to get your hands dirty and help <strong>operate a
production OpenStack service</strong>.</p>
<p>Related to the needs of operators are the deployment, configuration and
<strong>operational tools</strong> out there which desperately need contributions
with people more familiar with the dirty details of how the software
works. Many developers use devstack to deploy their development clouds,
but there’s huge benefit in occasionally deploying something more
production-like and contributing to whatever tool you used. TripleO is a
great deployment effort to contribute to because it’s attempting to
create a space where everyone interested in deployment can collaborate,
but also because it closely tracks the development branch of OpenStack.</p>
<p>Once you have succeeded at making an impact as an individual
contributor, you should look to extend your <strong>leadership</strong> efforts
beyond simply leading by example. Naturally, you’ll tend towards
volunteering for the responsibility of the PTL position on whichever
program you contribute most to. To demonstrate your willingness and
trustworthiness for the position, perhaps you’ll suggest the PTL
delegate some of their responsibilities to you.</p>
<p>Your leadership interests should extend beyond a single project too. In
some ways, the kind of cross-project issues considered by the
<strong>Technical Committee</strong> are as important as the per-project
responsibilities of PTLs. Do you have strong opinions on how, why and
when should add new programs or Integrated projects. If not, why not?</p>
<p>The governance of OpenStack and the shared responsibility for the future
direction of OpenStack extends beyond the TC and PTL’s governance of the
project itself, to the role of the <strong>Foundation Board of Directors</strong> in
protecting, empowering and promoting the project as well as ensuring
there’s a healthy commercial and non-commercial ecosystem around the
project. Do you care how the TC and board divide their responsibilities?
Or how much explicit corporate influence is appropriate in the technical
decision making of the project? Or how the board makes legal decisions
which directly impact the project? Or how individual members elect their
representatives on the board? You should.</p>
<p>Wait, wait, I’m forgetting a bunch of stuff. You should care deeply
about bringing contributors on board and participate in the awesome OPW
and GSoC programs. It’s important to keep track of how the project is
perceived, so you should read any articles published about the project
and follow even our worst detractors on twitter. Watch carefully how our
major competitors like AWS and GCE are evolving. Make sure to keep on
relevant new developments like NFV or Docker. Keep an eye on new
projects on Stackforge to track how they develop.</p>
<p>Huh, wait. You’re probably <strong>employed</strong> to work full time on the
project, right? Well, you really need to learn how to wear upstream and
downstream “hats”. You need to understand how you can help your employer
be successful with their objectives around the project. You need to be
able to reconcile any apparent conflicts between your employers’ needs
and the best interests of the project. This is not a zero sum game. Meet
with your employer’s customers and partners, help deliver what OpenStack
product or service your employer is providing, mentor colleagues on how
to successfully engage with the project and be the bridge over the
upstream and downstream gap.</p>
<p>Above all, through all of this, be nice to everyone you encounter and
wear a smile.</p>
<p><strong>BZZZT … BURNOUT ALERT</strong></p>
<p>I’m obviously being facetious, right? There’s no way anyone can possibly
live up to those expectations and live to tell the tale?</p>
<p>It’s pretty obvious when you put it all together like this that these
are unreasonable expectations. The hero of this tale does not exist.
Many of us have tried to be this person, but it’s just not possible.
Read into this, if you like, a very personal tale of burnout caused by
unreasonable self-imposed expectations.</p>
<p>But really, what I want to get across today is that you don’t need to be
this hero in order to contribute. Far from being too many active monthly
contributors, five hundred is just the tip of the iceberg. Why shouldn’t
every attendee of every OpenStack meetup be able to contribute in some
small way?</p>
<p>When mentoring new Red Hat engineers, my basic advice is always “find
your niche”. Find something that takes your interest and that you can
see an obvious path towards making a significant impact, and go deep!
Ignore pretty much everything else and do your thing. Maybe after a
while you’ll have got the ball rolling of its own accord and that there
are other areas you can now make an equally big impact on. Or perhaps
you’ll stick with this niche and continue to make an impact doing it
over the longer term.</p>
<p>One of my favorite examples of a less likely niche is bug triage. Back
in the summer of 2001 when I started seriously contributing the GNOME
project and became a maintainer of its CORBA ORB, ORBit, another new
contributor to the project called <a href="https://mail.gnome.org/archives/gnome-bugsquad/2001-June/msg00004.html">Luis Villa posted this
email</a>:</p>
<blockquote>
<p>Hey, everybody. By way of introduction: I'm the new bugmaster at
Ximian. As some of you may have noticed, I'm slowly moving towards
cleaning out evo and RC bugs from bugzilla.gnome and into
bugzilla.ximian.</p>
</blockquote>
<p>Luis went on to breath new life into GNOME’s “bugsquad”, helped put in
place a highly effective bug triage process and taught the GNOME
community how to truly value and celebrate the contributions of both bug
reporters and bug triagers. If you want to make fame and fortune in the
open source world, how many people would pick bug triage as the place to
start? Well, Luis did and made a huge impact, before moving on to
engineering management and then giving it all up to go to law school. He
is now Assistant General Counsel for the Wikimedia Foundation.</p>
<p>There’s a real “find your niche” lesson in that story, but also a lesson
that we as a community need to learn to truly value and celebrate all of
the myriad of different ways that contributors can help the project.
Rather than judge others based on how they’re not contributing, rather
than feel exasperated when so few others share your passion for a
particular niche no matter how important it seems to you personally, we
as a community need to acquire a greater level of empathy for our fellow
contributors.</p>
<p>We also need to experiment with ways of running the project so that
different roles and niches are appropriately recognized. Does the focus
we put on PTLs detract from the valuable project management
contributions others make? Are official programs the only way of
recognizing the importance of particular areas? If programs are the only
way, do we need to be more open to creating programs wherever a group of
people have coalesced around some particular effort? Do we need to
explicitly raise the profiles of those contributors doing hard
behind-the-scenes work in areas that we don’t typically recognize? Are
we building a culture that places too much emphasis on recognition and
instead roll back some of the ways we recognize people now?</p>
<p>Lot’s of questions, few answers. But hopefully this can get the
conversation started.</p>May 11 OpenStack Foundation Board Meeting2014-05-17T14:52:00+01:002014-05-17T14:52:00+01:00markmctag:crustyblaa.com,2014-05-17:/may-11-openstack-foundation-board-meeting.html<p>The OpenStack Foundation Board of Directors <a href="https://wiki.openstack.org/wiki/Governance/Foundation/11May2014BoardMeeting">met
in-person</a>
in advance of the OpenStack Summit in Atlanta. This my informal
recollection of the meeting. It’s not an official record, etc.</p>
<p>Unlike previous meetings held in advance of summits, this meeting only
ran from 09:00 to 14.30 at which …</p><p>The OpenStack Foundation Board of Directors <a href="https://wiki.openstack.org/wiki/Governance/Foundation/11May2014BoardMeeting">met
in-person</a>
in advance of the OpenStack Summit in Atlanta. This my informal
recollection of the meeting. It’s not an official record, etc.</p>
<p>Unlike previous meetings held in advance of summits, this meeting only
ran from 09:00 to 14.30 at which time we switched venue for the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/11May2014JointMeeting">first
ever joint board of directors and technical committee
meeting</a>.</p>
<p>I'm about to head off on vacation for a week, so I figured I'd do my
best to briefly cover some of the topics covered during the meeting.</p>
<h3>Jonathan's Update</h3>
<p>After the usual preliminaries, we began the meeting with Jonathan Bryce
(in his role as Executive Director) giving the board an update from the
Foundation's perspective.</p>
<p>One of the more interesting slides in Jonathan's updates is always the
latest statistics showing community and ecosystem growth. We now have
over 355 companies supporting the foundation, over two thousand total
contributors and almost five hundred active contributors every month.
Over 17,000 commits were merged during the Icehouse release cycle, an
increase of 25% from Havana. This level growth is just phenomenal.</p>
<p>Jonathan also talked about the growth in visitors to the openstack.org
website and made some interesting observations about the geographical
spread of the visitors. The top 4 countries seen in the stats are the
U.S., India, China and France. That France figures so highly in the
stats is a good sign in advance of the summit in Paris in November.</p>
<p>Next, Jonathan moved on to talk about the week ahead in Atlanta. Once
again, we're seeing a huge increase in the level of interest in the
event with over 4,500 attendees compared to the roughly 3,000 attendees
in Hong Kong. Running an even of this size is a massive undertaking and
Jonathan mentioned one crazy statistic - the foundation had over 23,000
pieces printed for the event and had to spread those orders over three
printing companies in order to be able to do it.</p>
<p>A big emphasis for the week was an increased focus on users and
operators. And, interestingly, there were roughly 800 developers and 700
operators signed up for the event. All were agreed that it's a very
healthy sign to see so many operators attend.</p>
<p>One comment from Jonathan triggered some debate - that the event was
turning into a broader cloud industry event rather than strictly limited
to just OpenStack. Some board members raised a concern that the event
shouldn't become completely generic and the focus should always be on
OpenStack and its ecosystem. Jonathan clarified that this is the intent.</p>
<p>Jonathan also talked about the geographical diversity of attendees at
the summit. People were coming from over 55 countries, but 81% attendees
were from the U.S. In contrast, in Hong Kong, the percentage of US
attendees were more like 40% and Jonathan felt that this showed the
importance of regularly holding summits outside of the U.S.</p>
<h3>Finances</h3>
<p>Jonathan also walked the board through and update on the foundation's
financial position. Operating income was 3% above their predictions and
expenses was down 7%. This has left the foundation with \$7.8M in the
bank, as part of Jonathan's goal to build up a substantial war-chest to
ensure the foundation's stability even in the event of unforseeable
events.</p>
<p>The summit in Atlanta was predicted to make a loss of \$50k but was on
track to make a profit. And yet, while it was predicted to be a \$2.7M
event, it was turning out to be a \$4M event. The situation will be very
different in Paris because of different cost structures and the event is
expected to make a loss. While on the topic, some board members
requested that the board be more closely involved in choosing the
location of future summits. Jonathan was happy to facilitate that and
expected to be able to give the board an update in July.</p>
<p>Jonathan next gave a detailed update on the foundation's application for
US federal tax exempt status. He explained that while we are Delaware
incorporated, non-stock, non-profit foundation we have not yet been
granted 501(c)(6) status by the IRS. After providing the IRS with
additional information in November, the IRS returned an initial denial
in March and the foundation filed a protest in April.</p>
<p>The objections from the IRS boil down to their feeling (a) that the
foundation is producing software and, as such, is "carrying on a normal
line of business", (b) that the foundation isn't improving conditions
for the entire industry and (c) that the foundation is performing
services for its members. Jonathan explained why the foundation feels
those objections aren't warranted and that the OpenStack foundation is
fundamentally no different from other similar 501(c)(6) organizations
like the Linux Foundation. He explained that other similar organizations
were going through similar difficulties and he feels it is incumbent on
the foundation to continue to challenge this in order to avoid a
precedent being set for other organizations in the future. Overall,
Jonathan seemed confident about our position while also feeling that the
outcome is hard to predict with complete certainty. This conversation
continued for some time and, because of the interest, the board moved to
establish a committee to track the issue consisting of the existing
members of the finance committee along with Eileen, Todd and Sean.</p>
<h3>Trademark Framework</h3>
<p>Jonathan moved on to give an update on some changes the foundation have
made around the trademark programs in place for commercial uses of the
mark. The six logos previously used were causing too much confusion so
the foundation has merged these into "Powered By OpenStack" and
"OpenStack Compatible" marks.</p>
<p>There followed some debate and clarifications were given, before some
members expressed concern that the board had not been adequately
consulted on the change. That objection seemed unwarranted to me given
that Jonathan had briefed the board on the change in advance of
implementing it.</p>
<p>Staying on topic of trademark programs, Boris took the floor and gave an
update on the DriverLog work his team has been working on. He request
the board use the output of DriverLog to enforce quality standards for
the use of the OpenStack compatible mark in conjunction with Nova,
Neutron and Cinder drivers. There was rather heated debate on the
implications of this, particularly around whether drivers would be
required to be open-source and/or merged in trunk.</p>
<p>Several board members objected to the fact that this proposal wasn't on
the agenda and the board hadn't been provided with supporting materials
in advance of the meeting. Boris committed to providing said material to
the board before revisiting the issue.</p>
<h3>Defcore</h3>
<p>Next up, Rob and Josh gave an update on the progress of their DefCore
initiative. Rather than attempt to repeat the background here, it's
probably best to <a href="http://robhirschfeld.com/category/clouds/openstack/defcore/">read Rob's own
words</a>.</p>
<p>Once the background was covered, the board spent some time considering
<a href="https://docs.google.com/spreadsheet/ccc?key=0Av62KoL8f9kAdFo4V1ZLUFM0OHlrRnFpQUkxSHJ5QWc&usp=drive_web#gid=6">the capabilities scoring
matrix</a>
where each capability (concretely, capabilities are groups of Tempest
tests) is scored against 12 selection criteria. This allows the
capabilities to be ranked so that the board can make an objective
judgment on which capabilities should be considered "must have". There
appeared to be generally good consensus around the approach, but a
suggestion was made to consider more graduated scoring of the criteria
(e.g. 1-5 rather than 0 or 1).</p>
<p>The conversation moved on to the subject of "designated sections".
During the conversation, the example of Swift was used and Josh felt the
technical committee's feedback indicated that either Swift in its
entirety should be a designated section or that none of Swift would be
considered binary. Josh also felt that the technical community (either
the TC or PTLs) should be responsible for such decisions but I felt that
while the TC can provide input, trademark policy decisions must
ultimately be made by the board lest we taint the technical communities
technical decision making by requiring significant political and
business implications to be considered.</p>
<p>One element of clarity that emerged from the discussion was the simple
point that "must have" tests were intended to drive interoperability
while designated sections were intended to help build our community by
requiring vendors to ship/deploy certain parts of the codebase and, by
implication, contribute to those parts of the codebase.</p>
<p>As time ran short, the board voted to approve the selection criteria
used by the DefCore committee. A straw-poll was also held to get a feel
for whether board members saw the need for an "OpenStack compatible"
mark in addition to the "OpenStack powered" mark. All but three of the
board members (Monty, Todd and Josh) indicated their support for an
additional "OpenStack compatible" mark.</p>
<h3>Win The Enterprise</h3>
<p>Briefly, Imad introduced the "Win the Enterprise" he an his team were
kicking off with a session during the summit. The goal is to drive
adoption of OpenStack in the enterprise by analyzing the technical and
business gaps that may be hindering such adoption and coming up with an
action plan to address them.</p>
<p>Feedback from board members was quite positive, with the discussion
centered around how the group would measure their success and how they
would ensure they operated in the most open and transparent way
possible.</p>
<p>There was also some discussion about the need for more product
management input to the project along with an additional focused effort
on end-users of OpenStack clouds.</p>
<h3>Wrapping Up</h3>
<p>After the meeting drew to a close, board members joined members of the
technical committee for a joint meeting. I'm hoping one of the awesome
individuals on the technical committee will write a summary of that
meeting!</p>
<p>This was a hugely draining week for many of us at Red Hat. As I prepare
to completely switch off for a week, allow me to pass on <a href="https://twitter.com/robynbergeron/status/466678776168865792">this sage
advise from Robyn
Bergeron</a>:</p>
<p><a href="http://www.keepcalm-o-matic.co.uk/p/keep-calm-and-ride-the-drama-llama/"><img alt="Keep Calm and Ride The Drama
Llama" src="https://sd.keepcalm-o-matic.co.uk/i/keep-calm-and-ride-the-drama-llama.png"></a></p>Heartbleed2014-04-10T10:04:00+01:002014-04-10T10:04:00+01:00markmctag:crustyblaa.com,2014-04-10:/heartbleed.html<p>Watching <a href="http://heartbleed.com/">#heartbleed</a> (aka
<a href="http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-0160">CVE-2014-0160</a>)
fly by in my twitter stream this week, I keep wishing we could all just
pause time for a couple of weeks and properly reflect on all the angles
here.</p>
<p>Some of the things I'd love to have more time to dig into:</p>
<ul>
<li>Obviously, <a href="http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=96db902">the bug …</a></li></ul><p>Watching <a href="http://heartbleed.com/">#heartbleed</a> (aka
<a href="http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-0160">CVE-2014-0160</a>)
fly by in my twitter stream this week, I keep wishing we could all just
pause time for a couple of weeks and properly reflect on all the angles
here.</p>
<p>Some of the things I'd love to have more time to dig into:</p>
<ul>
<li>Obviously, <a href="http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=96db902">the bug itself and its
fix</a></li>
<li>Why countermeasures in the kernel, libc or the compiler didn't
prevent this, or how static analysis tools didn't catch it. (Update:
see
<a href="http://www.tedunangst.com/flak/post/heartbleed-vs-mallocconf">here</a>
and <a href="http://article.gmane.org/gmane.os.openbsd.misc/211963">here</a>)</li>
<li>The protocol design - some wonder why a seemingly application-level
feature is included in such a critical security protocol and why the
feature is available before authentication has completed.
Interesting to note that one of the stated purposes of the feature
is <a href="http://tools.ietf.org/html/draft-ietf-tls-dtls-heartbeat-04">PMTU discovery for
DTLS</a>
yet it's enabled for plain TLS too</li>
<li>That it's not only servers which are vulnerable - <a href="http://security.stackexchange.com/questions/55119/does-the-heartbleed-vulnerability-affect-clients-as-severely">clients equally
vulnerable</a></li>
<li>The fact that attacks can't be detected because there is no logging
of this failure condition</li>
<li>The health of the OpenSSL project, <a href="http://article.gmane.org/gmane.os.openbsd.misc/211963">talk of its developers being
irresponsible</a>
when it comes to their development practice</li>
<li><a href="http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=4817504">How the bug was
introduced</a>,
how much code review or testing was involved. (Update: <a href="http://www.smh.com.au/it-pro/security-it/man-who-introduced-serious-heartbleed-security-flaw-denies-he-inserted-it-deliberately-20140410-zqta1.html">comments
from the original author of the
code</a>)</li>
<li>The realization that few in our industry are actively investing in
this project which so many place so much trust in</li>
<li>The question of whether OpenSSL is the library everyone should be
relying on, whether it should be e.g. NSS or GnuTLS</li>
<li>How something like OpenSSL can be a critical part of a web service's
architecture <a href="http://www.troyhunt.com/2014/04/everything-you-need-to-know-about.html">even where e.g. the service is implemented with
technologies like
.NET</a></li>
<li>The question of whether we're putting all our eggs in one basket if
an SSL vulnerability can have such far reaching implications</li>
<li>Just how much was exposed by the vulnerability - e.g. private keys,
usernames and passwords, other sensitive data - and how much data
could have been compromised by a determined, well-resourced attacker
in possession of this knowledge before it was patched</li>
<li>The <a href="http://www.quora.com/OCSP-Online-Certificate-Status-Protocol/Is-it-better-to-enable-OCSP-and-leak-personal-information-or-disable-OCSP-and-risk-trusting-a-revoked-certificate/answer/Robert-Love-1?srid=cwn&share=1">efficacy of certificate revocation
measures</a>
in protecting users from potentially compromised certificates</li>
<li>How <a href="https://www.eff.org/deeplinks/2014/04/why-web-needs-perfect-forward-secrecy">Perfect Forward Security is an important mitigation
measure</a>
for vulnerabilities like this</li>
<li>The <a href="http://www.seacat.mobi/blog/heartbleed">claim by one site that they could detect attacks in their
logs</a>, that there was
someone exploiting the vulnerability two weeks before it was
disclosed publicly. (Thierry Carrez points out the update to this
post which says "other tools [..] can produce the same pattern in
the SeaCat server log")</li>
<li>How <a href="http://oneverythings.blogspot.ie/2014/04/i-heartbleed-nsa.html">our perception of a vulnerability like this has
changed</a>
now that we know just how aggressive intelligence agencies (the NSA
and others) are in their approach to population surveillance
and monitoring. <a href="http://ftp.belnet.be/FOSDEM/2014/Janson/Sunday/NSA_operation_ORCHESTRA_Annual_Status_Report.webm">This FOSDEM talk is rather
prescient</a>
in describing OpenSSL as the "crown jewels" for the NSA. (Update:
<a href="http://www.bloomberg.com/news/2014-04-11/nsa-said-to-have-used-heartbleed-bug-exposing-consumers.html">claims that the NSA knew of the bug soon after its introduction and
exploited
it</a>)</li>
<li>The approach that was taken to disclosing the vulnerability, how
responsible a disclosure process it was and the pros/cons of
delaying disclosure further. <a href="http://www.eweek.com/security/heartbeat-ssl-flaw-puts-linux-distros-at-risk.html">Apparently CloudFlare was disclosed
before the public disclosure, whereas vendors like Red Hat
weren't</a>.
(Update: <a href="http://www.openwall.com/lists/oss-security/2014/04/08/10">more details on the disclosure timeline from Kurt
Seifried</a>)
(Another update: <a href="https://plus.google.com/+MarkJCox/posts/TmCbp3BhJma">OpenSSL's official view of the
timeline</a>) (Yet
another update: <a href="http://www.vocativ.com/tech/hacking/behind-scenes-crazy-72-hours-leading-heartbleed-discovery/">interview with Codenomicon's David
Chartier</a>)</li>
<li>How effective entities like Linux distros were in helping getting
the vulnerability quickly patched in the wild (see e.g. that <a href="https://stripe.com/blog/heartbleed">Stripe
went and built their own patched
package</a> rather than wait for an
Ubuntu package)</li>
<li>How much more difficult patching this was due to applications
bundling OpenSSL and how much worse it could have been if the
practice of bundling was more widespread</li>
<li>The impact that the <a href="http://www.kalzumeus.com/2014/04/09/what-heartbleed-can-teach-the-oss-community-about-marketing/">impressive
marketing</a> -
i.e. a cool name vs simply a CVE number, a logo, clear technical
writing which speaks to the business impact, etc. - had on the speed
with which the patch was deployed</li>
</ul>Mar 4 OpenStack Foundation Board Meeting2014-03-12T00:52:00+00:002014-03-12T00:52:00+00:00markmctag:crustyblaa.com,2014-03-12:/mar-4-openstack-foundation-board-meeting.html<p>On March 4th, the <a href="https://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a> met
for an <a href="https://wiki.openstack.org/wiki/Governance/Foundation/4Mar2014BoardMeeting">all-day, in-person
meeting</a>
at DLA Piper's office in Palo Alto, California. This my informal
recollection of the meeting. It’s not an official record, etc.</p>
<p>Some 20 of the 24 board members managed to make the meeting in …</p><p>On March 4th, the <a href="https://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a> met
for an <a href="https://wiki.openstack.org/wiki/Governance/Foundation/4Mar2014BoardMeeting">all-day, in-person
meeting</a>
at DLA Piper's office in Palo Alto, California. This my informal
recollection of the meeting. It’s not an official record, etc.</p>
<p>Some 20 of the 24 board members managed to make the meeting in person
with Todd and Tristan joining over the phone. Yujie Du and Chris Kemp
were unable to attend.</p>
<p>As usual our teleconferencing capabilities were woefully inadequate for
those hoping to contribute remotely. However, this time Rob and Lew
joined a Google Hangout with video cameras trained on the meeting. One
would hope that made it a little easier to engage with the meeting, but
we didn't really have any feedback on that.</p>
<p>Before we got started properly, Alan took some time to recommend
<a href="http://www.amazon.com/Startup-Boards-Getting-Board-Directors/dp/1118443667">"Startup Boards: Getting the Most Out of Your Board of
Directors"</a>
to the directors (based, in turn, on Mark Radcliffe's recommendation) as
a book which could provide some useful background on the
responsibilities of directors and what it takes to make a successful
board.</p>
<p>[Update: Josh points out we also approved the minutes of the previous
meeting]</p>
<h3>Executive Director Update</h3>
<p>Our first meaty topic was one of Jonathan's regular updates.</p>
<p>Jonathan talked about the continued tremendous growth in interest around
the project will all the foundation staff's key metrics (e.g. website,
developers, twitter, youtube, etc.) at lease doubling in 2013. One
interesting aspect of this growth is that the China, India and Japan
regions all grew their share of website traffic and this lead to a
discussion around whether having the last summit in Hong Kong directly
contributed to this shift.</p>
<p>Our community is growing too. We now have over 15,000 individual members
of the foundation, over 2,000 contributors to the project over its
lifetime and over 400 unique contributors to the project each month.</p>
<p>The mention of 15,000 individual members lead to a somewhat lengthy
discussion about the fact that <a href="http://www.openstack.org/legal/individual-member-policy/">individual membership may be
terminated</a>
under the following clause of the bylaws:</p>
<blockquote>
<p>failure to vote in at least 50% of the votes for Individual Members
within the prior twenty-four months unless the person does not respond
within thirty (30) days of notice of such termination that the person
wishes to continue to be an Individual Member</p>
</blockquote>
<p>Jonathan explained how shortly after the foundation was launched, over
6,000 people signed up as individual members. Only 1,500 or so of those
initial members have since voted in elections, so we could potentially
be looking at removing somewhere in the region of 6,000 members in 2014.
This reduced membership will facilitate bylaws changes by making it
easier (or even possible) to reach the <a href="http://www.openstack.org/legal/bylaws-of-the-openstack-foundation/">quorum necessary under clause
9.2(a)</a>:</p>
<blockquote>
<p>requires an affirmative vote of a majority of the Individual Members
voting as provided in Article III, but only if at least 25% of the
Individual Members vote</p>
</blockquote>
<p>Some discussion points around this included whether a future bylaws
change should reduce the quorum requirement to something like 10%, that
terminated members can re-register but would then have to wait 180 days
in order to be eligible for voting and that project contributors need to
be foundation members but some contributors may not be in the habit of
voting and may have their membership terminated.</p>
<p>Jonathan moved on with his slide deck and briefly mentioned some of the
foundation's new supports like Oracle and Parallels. He also talked
about OpenStack is increasingly fulfilling its role as a platform and is
being put to work for many diverse use cases. He also included a slide
with many positive media and analyst quotes about the project like
"Industry support has coalesced around OpenStack".</p>
<p>Jonathan then moved on to the foundation's budget, describing it as an
\$8M budget which turned out to be \$11M. Income was up, but expenses
were kept in line such that \$2.5M could be put in the bank. He
expressed particular pride that 18 months ago the foundation was just
starting with no money and already had built up a significant buffer
which would allow us all to feel confident about the foundation's
future, even in more turbulent or unpredictable times.</p>
<p>Finally, Jonathan review the foundation staff's priorities for 2014:</p>
<ol>
<li>Improve the software - whether that be continued investment by the
foundation in the software development process or organizing
activities which bring user feedback into the project</li>
<li>Improve interoperability between OpenStack-powered products and
services</li>
<li>Grow the service provider global footprint, with a specific mention
for interest from telco operators at Mobile World Congress around
OpenStack and NFV</li>
</ol>
<h3>DefCore Update</h3>
<p>Next up, Rob and Josh provided an update on the progress of the DefCore
committee, requesting a checkpoint from the board as to whether there
was consensus that the current approach should continue to be pursued.</p>
<p>Rob started by reviewing <a href="https://wiki.openstack.org/wiki/Governance/CoreDefinition">the purpose of DefCore and the approach taken
to date</a>. He
explained that the committee is mandated to look at ways of governing
commercial use of the OpenStack trademark and that some issues are
deliberately being punted on for now, e.g. an API interoperability
trademark and changes to the bylaws.</p>
<p>Josh took over and reviewed the <a href="https://etherpad.openstack.org/p/DefCoreTestCriteria">currently agreed-upon
criteria</a> that
will be used when evaluating whether a given capability will be required
in order to use the trademark, e.g.</p>
<ol>
<li>Stable - required to be stable for >2 releases</li>
<li>Complete - should be parity in capability tested across extension
implementations</li>
<li>Discoverable - e.g. can be found in Keystone and via service
introspection</li>
<li>Widely Deployed - favor capabilities that are supported by multiple
public cloud providers and private cloud products</li>
<li>Tools - supported by common tools</li>
<li>Clients - part of common libraries</li>
<li>Foundation - required by other must-have capabilities</li>
<li>TC Future Direction - reflects future technical direction</li>
<li>Documented - expected behaviour well documented</li>
<li>Legacy - previously considered must-have</li>
<li>Cluster - tests are available for this capability?</li>
<li>Atomic - unique capability that cannot be built out of other
must-have capabilities</li>
<li>Non-Admin - capability does not require administrative rights</li>
</ol>
<p>Next, Rob, Josh and Troy walked the board through a <a href="https://docs.google.com/spreadsheet/ccc?key=0Av62KoL8f9kAdFo4V1ZLUFM0OHlrRnFpQUkxSHJ5QWc&usp=drive_web#gid=6">draft spreadsheet
evaluating potential capabilities against those
criteria</a>.</p>
<p>Much of the subsequent discussion revolved around various board members
being very eager to get wider feedback on the ramifications of this
process, particularly around identifying the most thorny and
controversial results. Concerns were expressed that the process has been
so involved and detailed that few people are well appraised of where
this is headed and may find some of the results very surprising.</p>
<p>Rob & Josh felt that this spreadsheet approach means that we can solicit
much more targeted and useful feedback from stakeholders. For example,
if a project feels one of its capabilities should be must-have, or a
cloud provider is surprised that a capability it doesn't yet provide is
seen to be must-have, then the discussion can be around that specific
capability, whether the set of criteria and weighting used for
evaluation are appropriate, and whether the capability has been
correctly evaluated against those criteria.</p>
<p>Finally, Rob & Josh explained the proposed approach for collecting the
test results which would be used for evaluating trademark use
applications. The idea (known as TCUP, or "tea-cup", for "test collect,
upload and publish") currently being developed in the <a href="https://github.com/stackforge/refstack">refstack repo on
stackforge</a> would allow people
to download a docker container image, add your cloud credentials and
endpoint URL, run the container which would then execute the tests
against the endpoint and upload the results.</p>
<p>[Update: Josh points out that data uploaded via TCUP will "be treated
as confidential for the time being"]</p>
<h3>Driver Testing</h3>
<p>Next, Boris Renski took the floor to talk about Nova, Neutron and Cinder
driver testing particularly with a view to how it might relate to
trademark usage. This relates to <a href="http://www.mirantis.com/blog/openstack-will-open-source-vendor-certifications/">his blog post on the
topic</a>
some weeks ago.</p>
<p>There were two main observations about changes happening in the
technical community - (1) that projects were demanding that vendor
maintainers provide reliable third-party automated testing feedback in
gerrit patch reviews and (2) that manually maintained, often
out-of-date, "driver compatibility matrices" in the OpenStack wiki may
soon be replaced by dashboards showing the results of these third-party
automated testing systems.</p>
<p>Boris wished to leave that technical work aside (since it is not the
board's domain) and focus the discussion on whether the board would
consider a new trademark program such that vendors who's drivers pass
this automated testing would be allowed the use of a trademark such as
"Built for OpenStack".</p>
<p>The debate quickly got heated and went off in several different
directions.</p>
<p>Part of the discussion revolved around the automated testing
requirements projects were placing on projects, how that worked in
practice, the ramifications of that, how deprecating drivers would work,
whether a driver being in-tree implied a certain level of quality, etc.
I felt the board was really off in the weeds on a topic that is under
the authority of the individual projects and the TC. For example, it was
easy to forget during the discussion that PTLs ultimately had the
discretion to waive testing requirements for individual drivers.</p>
<p>Another surprising element to this was parallels being drawn with the
overlap between the <a href="http://activity.openstack.org/dash/browser/">OpenStack Activity
Board</a> and
<a href="http://www.stackalytics.com">Stackalytics</a>. Some board members felt
that Mirantis's work on Stackalytics had deliberately duplicated and
undermined the Activity Board effort, and that the same thing was
happening here because this driver testing dashboard naturally
(according to those board members) belongs under the DefCore/Refstack
efforts. Boris acknowledged he "had his wrist slapped over Stackalytics"
and was attempting to do the right thing here by getting advance buy-in.
Others felt that the two efforts were either unrelated or that competing
efforts can ultimately lead to a better end result.</p>
<p>Another thread of the discussion was that Boris's use of, or allusion
to, the special term "certification" automatically strayed this topic
into the area of the OpenStack brand and that it was inappropriate to
speculate that the foundation would embark on such a program before the
board had discussed it.</p>
<p>In the end, the board directed that Jonathan should work with Boris and
Rob on a plan to collect any automated test results out there and,
secondly, work with the DefCore legal subcommittee to explore the
possible use of the trademark in this context.</p>
<h3>Operator's Feedback</h3>
<p>Tim Bell took the floor next to talk the board through <a href="https://etherpad.openstack.org/p/operators-feedback-mar14">feedback from
OpenStack
operators</a>
collected at the OpenStack Operators Mini Summit the previous day.</p>
<p>The etherpad linked above perhaps provides a better summary than I can
give, but some of the highlights include:</p>
<ul>
<li>The notion that operators should be able to provide feedback on
blueprints as they are drafted, to help get operational insights to
developers early on in the development process</li>
<li>Some observations on the stability (or lack thereof) of some of the
core OpenStack components</li>
<li>The importance of a solid upgrades story was re-iterated</li>
<li>Some great feedback on TripleO</li>
<li>The split between teams doing CI/CD and those consuming releases</li>
<li>How to encourage operators to file more bugs upstream</li>
<li>Lots, lots more</li>
</ul>
<p>This feedback was well received by the board and triggered a bunch of
discussions and questions.</p>
<p>As a final point, Josh raised some concerns about how the invitee list
was drawn up and how he felt it would have been appropriate for vendors
(like Piston) to recommend some of their customers to be invited. Tim
felt this was an unfair criticism and that the user committee had worked
hard to seed the limited seating event with a diverse set of invitees
before opening it up to the public.</p>
<h3>Emerging Use Cases - NFV</h3>
<p>Finally, with limited time remaining, Toby Ford from AT&T briefed the
board on Network Function Virtualization (NFV) as an emerging use case
for OpenStack which is heavily tied to SDN (Software Defined Network).
He described how AT&T have set themselves a mission to:</p>
<blockquote>
<p>Simplify, open up, and scale our Network to be more agile, elastic,
secure, cost effective and fast by (a) moving from hardware centric to
software centric, (b) separating the control plane and data plane and
(c) making the network more programmable and open</p>
</blockquote>
<p>Toby did a great job of walking through this complex area, leaving me
with the understanding that there is a massive shift in the networking
industry from hardware appliances to scale-out software appliances
running on virtualized commodity hardware.</p>
<p>There appears to be consensus in the networking industry that OpenStack
will be the management and orchestration platform for this new world
order, but that there is a serious need for telcos and networking
vendors to engage more closely with OpenStack in order to make this
happen.</p>
<h3>Wrap Up and Evening Event</h3>
<p>Alan then wrapped up the meeting a little early after talking through
the schedule for our next meetings with a conf call on April 3 and an
in-person meeting on May 11.</p>
<p>The board then moved on to a local restaurant for dinner. Before and
after dinner, I had some great conversations with Tim, Monty, Van and
Troy. Funnily enough, because of the layout of the tables and the noise
in the restaurant, it was only really possible to talk to the person
sitting directly opposite you and so I found myself having an exclusive
2 hour dinner date with Boris! At one point, after Boris knocked a glass
of wine over me, I joked that I should tweet "Red Hat and Mirantis
tensions finally bubble over to physical violence". But, in all honesty,
these in-person, informal conversations around the board meetings are
often far more effective at enabling shared understanding and real
collaboration than the 20+ person meetings themselves. Such is the
nature of the beast, I guess.</p>Naked Pings2014-02-20T14:56:00+00:002014-02-20T14:56:00+00:00markmctag:crustyblaa.com,2014-02-20:/naked-pings.html<p>Back in November 2009, <a href="http://people.freedesktop.org/~ajax/">ajax</a> sent
an email on IRC etiquette to Red Hat's company-wide mailing list. I've
had to refer several people to it over the years, so I asked ajax for
permission to publish it. He agreed. Here it is in all its glory.</p>
<blockquote>
<p>From: Adam Jackson<br>
To …</p></blockquote><p>Back in November 2009, <a href="http://people.freedesktop.org/~ajax/">ajax</a> sent
an email on IRC etiquette to Red Hat's company-wide mailing list. I've
had to refer several people to it over the years, so I asked ajax for
permission to publish it. He agreed. Here it is in all its glory.</p>
<blockquote>
<p>From: Adam Jackson<br>
To: memo-list<br>
Subject: On "ping" etiquitte<br>
Date: Tue, 17 Nov 2009 12:21:30 -0500</p>
<p>IRC has developed a "ping" convention for getting someone's attention.
It works because most clients will highlight channels in which your
name has been mentioned, so something like</p>
<p>ajax: ping</p>
<p>will make that channel show up pink instead of white for me [1].</p>
<p>I wish to correct, or at least amend, this behaviour. The naked ping
should be Considered Harmful, for at least two reasons. The first is
that it conveys no information. The recipient of your ping, like you,
is a Busy Person. They may be in the middle of something requiring
intricate thought, and should not be interrupted for anything less
than fire, flood, or six figures of revenue. Worse, _you_ may forget
why you pinged someone; when, four hours later, your victim gets back
to IRC and responds to you, _you_ will be disrupted in turn trying
to remember what was on your mind in the first place.</p>
<p>The second, more subtle reason proceeds from the first. A ping with no
data is essentially a command. It's passive-aggressive; it implies
that the recipient's time is less valuable than yours. [2] The
pingee will respond in one (or both) of two ways. Either they will
experience increased stress due to increased unpredictable demands on
their time, or they will simply ignore naked pings.</p>
<p>The fundamental issue here is a misunderstanding of the medium. IRC is
not a telephone. It's volatile storage. The whole reason the ping
works is because the client remembers seeing the ping, and can save it
in your history buffer so you can see who was talking to you and why.</p>
<p>The naked ping removes this context.</p>
<p>Please. Save your time. Save my time. Make all of our lives more
efficient and less stressful. Ping with data. At a minimum:</p>
<p>ajax: ping re bz 534027</p>
<p>See the difference? Now you've turned slow, lockstep, PIO-like
interaction into smooth pipelineable DMA. It's good for your hardware,
and it's good for you.</p>
<p>[1] - irssi 4 life.</p>
<p>[2] - Their time may well be less valuable than yours. That's not
the<br>
point.</p>
<ul>
<li>ajax</li>
</ul>
</blockquote>Jan 30 OpenStack Foundation Board Meeting2014-02-07T16:41:00+00:002014-02-07T16:41:00+00:00markmctag:crustyblaa.com,2014-02-07:/jan-30-openstack-foundation-board-meeting.html<p>The <a href="https://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a> met
for a two hour conference call last week. This was the first meeting of
the board since the recent
<a href="http://lists.openstack.org/pipermail/foundation/2014-January/001616.html">Individual</a>
and
<a href="http://lists.openstack.org/pipermail/foundation/2014-January/001615.html">Gold</a>
member director elections.</p>
<p>As usual, this my informal recollection of the meeting. It’s not an
official record, etc.</p>
<p>Your trusty …</p><p>The <a href="https://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a> met
for a two hour conference call last week. This was the first meeting of
the board since the recent
<a href="http://lists.openstack.org/pipermail/foundation/2014-January/001616.html">Individual</a>
and
<a href="http://lists.openstack.org/pipermail/foundation/2014-January/001615.html">Gold</a>
member director elections.</p>
<p>As usual, this my informal recollection of the meeting. It’s not an
official record, etc.</p>
<p>Your trusty reporter had just arrived in Brussels for FOSDEM and got to
stay in his hotel room for this meeting rather than sampling Belgian's
fine beers. Oh, the sacrifices we make! :-P</p>
<h3>Preliminaries</h3>
<p>As usual, calling the meeting to order was a challenge and it was at
least 15 minutes after the scheduled start before we completed the roll
call.</p>
<p>Next, Alan welcomed our new directors:</p>
<ul>
<li>Yujie Du</li>
<li>Alex Freedland</li>
<li>Vish Ishaya</li>
<li>Imad Sousou</li>
</ul>
<p>and thanked our outgoing directors:</p>
<ul>
<li>Nick Barcet</li>
<li>Hui Cheng</li>
<li>Joseph George</li>
<li>Lauren Sell</li>
</ul>
<p>as well as thanking those directors who served on the board for part of
2013:</p>
<ul>
<li>Devin Carlen</li>
<li>Jim Curry</li>
<li>John Igoe</li>
<li>Kyle MacDonald</li>
<li>Jon Mittelhauser</li>
</ul>
<h3>Policies, Communication Channels and Meetings Schedule</h3>
<p>Since it's a new year, we took the opportunity to review the various
policies which apply to board members.</p>
<p>Josh went over our <a href="http://www.openstack.org/legal/transparency-policy/">transparency
policy</a> mentioning
that the board endeavours to be as transparent as possible, with board
meetings open to the public, a summary of meetings posted to the
foundation list and directors encouraged to use the foundation mailing
list for discussions. Sub-committees of the board are expected to be
similarly transparent, with wiki pages and public mailing lists. Some
caveats to this policy are that board members are not allowed to make
public comments about the board meeting until after Jonathan has posted
his summary (or 72 hours have passed), members should not discuss
executive sessions and the distribution of some non-public documents may
have to be limited to directors.</p>
<p>Alan also mentioned our <a href="https://wiki.openstack.org/wiki/Governance/Foundation/CodeOfConduct">Code of
Conduct</a>
and encouraged directors to read it carefully. Finally, Jeff from DLA
Piper walked us through our <a href="https://wiki.openstack.org/wiki/Governance/Foundation/AntitrustPolicy">antitrust
policy</a>
where he emphasised the importance of avoiding even the perception that
board members are coming together to advance the interests of some
companies over others. Members should restrict themselves to
pro-competitive collaboration.</p>
<p>Next, we quickly reviewed the various channels for communication that
directors need to be aware of - webex for conf calls, the foundation and
foundation-board mailing lists, the #openstack-board and
#openstack-foundation IRC channels, informal etherpads that we use
during board meetings and the various committee mailing lists.</p>
<p>Finally, we discussed the upcoming board meetings - an all-day
face-to-face meeting in Palo Alto on March 4, a 2 hour conference call
on April 3, an all-day face-to-face meeting in Atlanta on May 11 in
advance of the Atlanta summit and another face-to-face on July 21 at
OSCON.</p>
<p>The subject of the timing of the Atlanta face-to-face was raised again.
May 11 is also Mother's Day (in the US and some other countries) which
is a nasty conflict for many board members. However, a poll amongst
board members had already established that no better time around the
summit could be found, so we are proceeding with the meeting on May 11.
The question was raised about whether future board meetings should be
scheduled to not align with our summits, but the objection to this idea
was that it puts too much of a time and budget strain on those members
who have to travel a long distance for the meetings.</p>
<h3>Status Reports From Committees</h3>
<p>Finally, time to move on to some more meaty topics! A member of each
committee of the board was asked to provide a status update and plans
for the year ahead.</p>
<p>Alan first described the work of the <a href="https://wiki.openstack.org/wiki/Governance/CompensationCommittee">compensation
committee</a>
who are responsible for defining and evaluating the goals and
performance of the Executive Director. In summary, the committee
concluded that Jonathan met his 2013 goals and new goals have been set
for 2014.</p>
<p>Next up, Sean Roberts talked about the finance committee. This committee
works with the foundation staff on financial budgeting and accounting.
Sean described the foundation's IRS filing and that the foundation's
2012 financial audit has been completed and deemed clean (with a note
that the foundation is "operating on a cash basis"). The foundation's
application for 501(c)(6) is progressing with the IRS asking for some
clarifications which were returned to them in December. The committee
meets monthly to review any discrepancies above 10%, but there have been
no such issues so far. Essentially, everything is in excellent shape.</p>
<p>Tim Bell talked through the latest from the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/UserCommittee">user
committee</a>.
Tim mentioned the user survey that was published at the Hong Kong summit
and how the committee has asked the TC for input on the kind of feedback
that would be useful for developers. The committee is preparing to run
another survey in advance of the Atlanta summit. Tim also mentioned that
the user committee is running a couple of small, focused "operator
mini-summits" over the next few months to bring operators together to
share their feedback. Tim described the challenge running the committee
with a small number of core volunteer members so as to ensure the
privacy of survey results while also encouraging volunteers to help with
tasks like turning survey feedback into blueprints for new features.</p>
<p>Van Lindberg gave an update on the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/LegalAffairsCommittee">legal affairs
committee</a>.
He emphasised that the committee is not counsel for the foundation or
the board, but rather a group which makes recommendations to the board
on IP policy. He recapped on some of the patent policy recommendations
from last year, for example that the foundation should join OIN. There
was a brief mention of the fact that all the committee members are
currently lawyers and the by-laws limits the number of members to five.
He also mentioned that the DefCore committee has a related sub-committee
examining possible by-laws changes.</p>
<p>Todd described the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/ElectionsCommittee">elections
committee</a>,
that is was formed in February 2013 with 8 members and the goal to
consider possible changes to the individual member election process. The
committee is currently considering proposing a change to either
Condorcet or STV and held a town-hall meeting in Hong Kong on the
subject. Todd noted that the meeting was lightly attended and there
generally has been rather low participation in the process. The main
hurdle to getting such a change passed is that a majority of at least
25% of our individual members would need to vote for the by-laws change
and the turnout for the previous election was only 17%. However, in July
we will be able to begin making inactive members ineligible for voting
and this should help us achieve the required turnout.</p>
<p>Rob gave an update from the DefCore committee[5] which is considering
changes to the requirements for commercial implementations of OpenStack
who wish to use the OpenStack trademark. The committee is currently
working to identify a set of must-pass tests and the functional
capabilities which these correspond to. Rob mentioned that some projects
currently have no or minimal test coverage and, as a result, their
capabilities could not be considered for inclusion in the requirements.
Rob also mentioned a "programs vs projects" issue which had been
identified during the committee discussions and that a meeting with the
TC would be required to resolve the issue. I proposed that Rob and Josh
could join the TC's IRC meeting to discuss the issue.</p>
<p>Finally, Simon gave a brief overview of the work of the gold member
application committee. This committee helps prospective gold members
prepare their application such that it fully anticipates all the
questions and concerns the board may have about the application.</p>
<h3>Wrapping Up</h3>
<p>While there were some small number of other items on the agenda, we had
run out of time at this point. In the short time available, we had
covered a broad range of topics but hadn't really covered new ground.
This meeting was mostly about rebooting the board for 2014.</p>OpenStack, Meritocracy and Diversity2014-01-12T22:20:00+00:002014-01-12T22:20:00+00:00markmctag:crustyblaa.com,2014-01-12:/openstack-meritocracy-and-diversity.html<p>These days, any time I reach for the word "meritocracy" when I want to
explain something about OpenStack's technical community and its
governance, I give pause.</p>
<p>Clearly, <a href="http://www.aaronsw.com/weblog/meritocracy">in some circles</a>,
the concept of "meritocracy" has been seriously discredited and
represents a system whereby elites perpetuate their power by tilting the …</p><p>These days, any time I reach for the word "meritocracy" when I want to
explain something about OpenStack's technical community and its
governance, I give pause.</p>
<p>Clearly, <a href="http://www.aaronsw.com/weblog/meritocracy">in some circles</a>,
the concept of "meritocracy" has been seriously discredited and
represents a system whereby elites perpetuate their power by tilting the
rules in favour of themselves.</p>
<p>I'm not much of a political thinker and my understanding of internal
American politics is pretty limited (think watching The West Wing and
vaguely following the spectacle of a presidential election) so the first
time I really encountered the term was in the context of the GNOME
project. From the <a href="https://wiki.gnome.org/action/show/FoundationBoard/Resources/Charter">GNOME Foundation
Charter</a>:</p>
<blockquote>
<p>GNOME is a Meritocracy</p>
<p>A corporation, organization or individual should not be granted a
place in the foundation unless its presence is justified by the merits
of its contribution. Money cannot buy influence in the GNOME project:
show us the code (or documentation, or translations, or leadership, or
webmastering...).</p>
</blockquote>
<p>and, subsequently, other projects like the ASF. From <a href="http://www.apache.org/foundation/how-it-works.html#meritocracy">How The ASF
Works</a>:</p>
<blockquote>
<p>When the group felt that the person had "earned" the merit to be part
of the development community, they granted direct access to the code
repository, thus increasing the group and increasing the ability of
the group to develop the program, and to maintain and develop it more
effectively.</p>
<p>We call this basic principle "meritocracy": literally, government by
merit.</p>
<p>What is interesting to note is that the process scaled very well
without creating friction, because unlike in other situations where
power is a scarce and conservative resource, in the apache group
newcomers were seen as volunteers that wanted to help, rather than
people that wanted to steal a position.</p>
<p>Being no conservative resource at stake (money, energy, time), the
group was happy to have new people coming in and help, they were only
filtering the people that they believed committed enough for the task
and matched the human attitudes required to work well with others,
especially in disagreement.</p>
</blockquote>
<p>To me, the "power" we're talking about here is the ability, permission
or empowerment to get stuff done which advances the project. In some
projects that means commit access, but ultimately it means building up
the respect and trust of the other contributors to the project such that
you can more easily influence and drive the direction of the project.
You achieve that "power" by getting useful stuff done (defined broadly -
code, documentation, translations, leadership, marketing, advocacy,
etc.) and all it grants you is the ability to get more useful stuff
done. In a healthy project, we want to give that power to more and more
people rather than concentrating it in a small elite.</p>
<p>This is what we mean when we say "OpenStack is a technical meritocracy".
I hate to think of those well-meaning principles of project governance
being sullied by "meritocracy" being used to explain away the social
inequities in U.S. politics. I also don't like to think of us seeing
these principles as some sort of platonic ideal that don't require us to
constantly evaluate how we empower people to help advance OpenStack.</p>
<p>One hint that all is not perfect is the level of diversity within the
project. Yes, we have diversity of opinions and a diversity of
sponsoring organizations, but we don't have an impressive level of
gender, race or cultural geography.</p>
<p>My good friend from GNOME days, Daniel Veillard, <a href="http://www.youtube.com/watch?v=-wslrgu324M&t=1960">asked this question of
the Technical Committee in Hong
Kong</a>:</p>
<blockquote>
<p>We are in China. There is no Asian on the podium. What can you do to
actually try to improve the situation?</p>
</blockquote>
<p>Yes, we have a meritocracy and anyone can advance to leadership
positions within the project, but we need to recognize that there are
extremely difficult language and cultural hurdles in front of many.</p>
<p>An example of these barriers is how we often conduct our Design Summit
sessions. Quite regularly - especially when you get a large number of
the more established contributors in the room together, folks who are
good friends who understand each other well - the discussion can often
devolve into a punchy flow of casual in-joke ridden sound-bites. I'm as
much to blame for that as anyone, but sometimes I think back and shudder
at how hard it must be for someone outside of the "in group" to join
that discussion.</p>
<p>I've seen a number of examples where a new non-native English speaker
has paired with an existing contributor to lead a design summit session
about their work. What can work really well is that the existing
contributor can help to engage the attendees, slow down the conversation
and ensure the new contributor understands the feedback being given ...
without attempting to take credit for the work of the new contributor.
This is just one technique we could use to empower new contributors.</p>
<p>Anyway, in summary - I think OpenStack's "meritocracy" is a well-meaning
model for empowering contributors (and celebrating their contributions)
but we should all be on the lookout for ways that we can make a special
effort to empower contributors from groups which are not already well
represented in the leadership of the project.</p>The 2014 OpenStack Individual Member Director Elections and Red Hat2013-12-11T17:59:00+00:002013-12-11T17:59:00+00:00markmctag:crustyblaa.com,2013-12-11:/the-2014-openstack-individual-member-director-elections-and-red-hat.html<p>tl;dr - the affiliation limit means that at most one of the two Red Hat
affiliated candidates can be elected. The cumulative voting system makes
it likely that both of us running seriously damages both of our chances
of being elected. A preferential voting system like
<a href="https://wiki.openstack.org/w/images/4/41/Condorcet-briefing_10_16_2013.pdf">Condorcet</a>
or
<a href="https://wiki.openstack.org/w/images/f/f1/OpenStack_Board_Briefing_on_STV_10_16_2013.pdf">STV</a>
would …</p><p>tl;dr - the affiliation limit means that at most one of the two Red Hat
affiliated candidates can be elected. The cumulative voting system makes
it likely that both of us running seriously damages both of our chances
of being elected. A preferential voting system like
<a href="https://wiki.openstack.org/w/images/4/41/Condorcet-briefing_10_16_2013.pdf">Condorcet</a>
or
<a href="https://wiki.openstack.org/w/images/f/f1/OpenStack_Board_Briefing_on_STV_10_16_2013.pdf">STV</a>
would not have this problem.</p>
<p>--</p>
<p>At Red Hat, those of us who contribute to OpenStack take very seriously
our responsibility to put what's good for the project first and foremost
in our minds - to wear our "upstream hat", as we like to say. That's
especially true for me and Russell Bryant.</p>
<p>However, now that the <a href="https://www.openstack.org/election/2014-individual-director-election/CandidateList">candidate list for the 2014 OpenStack Individual
Member Director
Elections</a>
has been finalized, we find ourselves wrestling with the fact that
Russell and I are both running as candidates. Two aspects of our
election system make this a problem. First, the cumulative voting system
means that those who would be happy to vote for either me or Russell are
forced to choose between us - essentially, we are damaging each others'
chances of being elected. Secondly, the affiliation limit means that
even if we were both lucky enough to receive enough votes to be elected,
one of us would be eliminated by the limit.</p>
<p>The combination of these two issues means that we have to factor our
affiliation into our decision. The rules place affiliation front and
centre in election system, even though Individual Member Directors are
not elected to represent their employer.</p>
<p>Now, I'm personally guilty of not pushing this election system issue
hard enough over this past year. At one point <a href="http://blogs.gnome.org/markmc/2013/10/03/oct-3rd-openstack-foundation-board-meeting/">I favoured experimenting
with a tweak to the cumulative
system</a>
over a more dramatic change because I found the prospect of getting a
majority of over 25% of our enormous electorate to vote in favour of a
change so daunting. I want to be completely open about this decision we
now face because I want to help raise awareness about how important an
issue it is.</p>
<p>Given that Red Hat is a Platinum Member and has a automatic seat on the
board, the options we're weighing up are:</p>
<ol>
<li>Continue with both Russell and me on the ballot, accepting the risk
that we're damaging each others chances.</li>
<li>I or Russell remove ourselves from the ballot, giving the other of
us the best possible chance of being elected.</li>
<li>Brian Stevens steps down from the board and I or Russell takes his
place, giving whichever of us remains on the ballot the best
possible chance of being elected.</li>
</ol>
<p>It's not an easy decision. We both feel we have something to offer on
the board. Both of us would be very proud to be elected to represent the
Individual Members. Both of us feel that Brian Stevens (our CTO who we
greatly respect) is the best possible representative for Red Hat on the
board.</p>
<p>We will make a decision on this before the election, but right now we
don't see any of the options as being particularly better than the
others. But, at the very least, I hope everyone will find this useful as
a concrete example of why our election system needs to change.</p>OpenStack "Core" and Interoperability2013-10-30T21:49:00+00:002013-10-30T21:49:00+00:00markmctag:crustyblaa.com,2013-10-30:/openstack-core-and-interoperability.html<p>I've been following the <a href="http://robhirschfeld.com/2013/07/22/kicking-off-core/">"what is core?"
conversation</a>
since the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/IncubationUpdate2013">"Incubation Update"
committee</a>
completed its work some time ago. I'm really happy to see Rob, Alan and
others put so much work into moving this along, but each time I try to
catch up on progress I find myself a …</p><p>I've been following the <a href="http://robhirschfeld.com/2013/07/22/kicking-off-core/">"what is core?"
conversation</a>
since the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/IncubationUpdate2013">"Incubation Update"
committee</a>
completed its work some time ago. I'm really happy to see Rob, Alan and
others put so much work into moving this along, but each time I try to
catch up on progress I find myself a little bewildered.</p>
<p>Since it's going to be a big topic during <a href="https://wiki.openstack.org/wiki/Governance/Foundation/4Nov2013BoardMeeting">next week's board
meeting</a>,
I figured I'd try to get my thoughts together here.</p>
<h3>What's it all About?</h3>
<p>Even the question bothers me - "what is core?". Why is that an important
question?</p>
<p>One reason for its importance is that the bylaws say:</p>
<blockquote>
<p>The Core OpenStack Project means the software modules [...] for
which an OpenStack trademark may be used</p>
</blockquote>
<p>In other words, Heat can't call itself "OpenStack Orchestration" unless
the Board of Directors approve Heat as part of "The Core OpenStack
Project". If that was all this was about, I'd be like "hell yes! Heat
should be known as OpenStack Orchestration!". But, it's clearly not just
about this, so I tend to ignore this aspect. I don't actually think much
of what is being discussed is all that relevant to this particular
issue.</p>
<p>So, what else is this all about? Well, one clue is the emphasis on
testing in <a href="http://robhirschfeld.com/2013/08/13/openstack-core-positions/">Rob's list of "10 core
positions"</a>.</p>
<blockquote>
<p>OpenStack Core means passing all “must-pass” tests</p>
</blockquote>
<p>This is an example of where I get really confused. Maybe I'm just
getting hung up on language. I think we're talking about cloud
providers, distributions, vendors, deployers, etc. self-certifying
themselves using a test suite in order that they can use the OpenStack
trademark. But are we talking about labelling these self-certified
clouds as "Core"? Aren't we making this all very confusing by using the
term "Core" both to describe the self-certified clouds/products and also
to describe the subset of OpenStack they are required to include? I'd
phrase this as:</p>
<blockquote>
<p>The OpenStack Core definition includes a list of tests which certified
clouds must pass</p>
</blockquote>
<p>Anyway, I'm not going to nitpick my way through all of this. I <em>think</em> I
understand the intent here, but I like to approach this from an entirely
different angle.</p>
<h3>A Market of Interoperable OpenStack Providers</h3>
<p>(Yes, that is Simon Wardley's terminology)</p>
<p>We all know that one of the basic goals of OpenStack is for there to be
a bunch of public clouds available to users around the world and that
interoperability between these public OpenStack clouds will make it
easier for users to switch cloud providers and encourage competition
between these providers.</p>
<p>To my mind, that's what this whole "what is core?" conversation is
really about. We want to use an OpenStack trademark program to help
build this marketplace by enforcing interoperability between clouds
which use the OpenStack trademark.</p>
<p>(And yes, I understand that the "what is core?" discussion is also
related to questions like what is required of an OpenStack distribution.
I'm focusing on the question of public cloud interoperability because I
think it's the most important. IMHO, the whole conversation is pointless
unless it moves this particular topic along quickly.)</p>
<p>To do that, we need to define which APIs these clouds must expose, or as
Rob puts it - which use cases these clouds must support. To take a
simple example, should these clouds be required to expose the Glance API
for uploading images?</p>
<p>And here's where I get confused again. Why are we talking about "what is
core?" when we could simply say "which APIs are required to be exposed
by certified OpenStack clouds?". Again, I'm trying not to be a pedant,
but the way the question is framed leaves me unsure whether we're really
all talking about the same thing.</p>
<p>I think the focus should be on immediate baby-steps towards
kick-starting this marketplace. One simple question - if and when we
certify the first batch of interoperable clouds, would we rather have a
smaller number of big clouds included or a large number of smaller
clouds? In terms of resource capacity provided by this marketplace, I
guess it's more-or-less the same thing.</p>
<p>Let's assume we absolutely want (at least) a small number of the bigger
providers included in the program at launch. Let's say "small number"
equals "Rackspace and HP". And let's assume both of these providers are
very keen to help get this program started. Isn't the obvious next
baby-step to get representatives of those two providers to figure out
exactly what level of interoperability they already have and also what
improvements to that they can make in the short term?</p>
<p>If we had that report, we could next do a quick "sniff test" comparing
this to many of the other OpenStack clouds out there to make sure we
haven't just picked two clouds with an unusual set of compatible APIs.
Then the board could make a call on whether this represents a reasonable
starting point for the requirements of OpenStack clouds.</p>
<p>No, this isn't perfect. But it would be a genuine step forward towards
being able to certify some clouds. We would have some data on
commonalities and have made a policy decision on what commonalities are
required. It would be a starting point.</p>
<h3>OpenStack Compatible?</h3>
<p>The next big decision for the board is whether a cloud which uses the
OpenStack trademark must actually be a deployment of OpenStack's code.
Is this a question of "OpenStack clouds" or "OpenStack compatible
clouds"?</p>
<p>I do think it would be damaging to OpenStack if this marketplace took
off and was dominated by providers which don't use or contribute to
OpenStack. But, <a href="http://robhirschfeld.com/2013/07/22/making-openstack-meaningful/">as Rob
says</a>,
the trademark isn't the only way of avoiding this situation - there's
also our "velocity and culture".</p>
<p>I struggle to see how trademark requirements around the use of OpenStack
code would work. How do you define which code must be used? Must that
code be used unmodified? If not, how much can you change? What does it
even mean to "use" a piece of code? That user requests must be executed
using that code? Maybe this would just be a "yes, we use and contribute
to OpenStack" good faith statement which the Foundation would make a
judgment call on? If so, how do we ensure transparency and fairness
around how that call is made?</p>
<p>If this question proves to be a stumbling block, I'd prefer to see an
"OpenStack compatible cloud" trademark program established quickly.
Getting these interoperability guarantees in place for our users takes
priority over ensuring that certified providers actually use and
contribute to OpenStack.</p>
<h3>Conclusion of Sorts</h3>
<p>I think our immediate concern should be kicking off a trademark program
for certifying interoperability between OpenStack clouds. I'm frustrated
whenever I think this "what is core?" discussion is tackling questions
which aren't immediate blockers to making progress on this program.</p>
<p>The board has to make (or oversee the making of) some policy questions.</p>
<p>Firstly, what APIs are required in OpenStack™ clouds? I'd favour
starting to answer this by first looking at the current interoperability
levels between existing clouds.</p>
<p>Secondly, whether and how we insist on the use of OpenStack code in
OpenStack™ clouds. If this turns out to be a difficult problem, then I
think we should just start with an "OpenStack™ compatible cloud"
program.</p>A New OpenStack Technical Committee2013-10-18T10:24:00+01:002013-10-18T10:24:00+01:00markmctag:crustyblaa.com,2013-10-18:/a-new-openstack-technical-committee.html<p>The <a href="http://lists.openstack.org/pipermail/openstack-announce/2013-October/000152.html">results of the OpenStack Technical
Committee</a>
have just been announced. I'm very grateful to everyone who voted for
me. Thanks!</p>
<p>This is going to be a very different committee than previously. For the
first time, PTLs of integrated projects do not have an automatic seat. I
think this will …</p><p>The <a href="http://lists.openstack.org/pipermail/openstack-announce/2013-October/000152.html">results of the OpenStack Technical
Committee</a>
have just been announced. I'm very grateful to everyone who voted for
me. Thanks!</p>
<p>This is going to be a very different committee than previously. For the
first time, PTLs of integrated projects do not have an automatic seat. I
think this will be a positive change and see TC members generally take
more of an interest in cross-project issues. That said, we need to be
careful to explicitly include PTLs in decision making which affect their
projects.</p>
<p>One thing I'm curious about is how this new TC makeup will affect
discussions about the increase in OpenStack's technical scope. I went
through each of candidate's nomination emails and pulled out some quotes
below. To me, this represents a high level of consensus towards a
continued cautious and measured growth in the project's scope ... but
make your own mind up! I'm actually quite surprised how many candidates
felt it was important to give their views on this.</p>
<p><a href="http://lists.openstack.org/pipermail/openstack-dev/2013-October/016048.html">Monty
Taylor</a></p>
<blockquote>
<p>I have an expansive view of the scope of OpenStack. I do not think
that 'pure IaaS' as a limiting factor in the definition serves or will
serve any of our users. I think that instead of trying to come up with
random or theoretical labels and then keeping ourselves inside of the
pre-defined label, we should focus on writing software that solves
problems for users.</p>
</blockquote>
<p><a href="http://lists.openstack.org/pipermail/openstack-dev/2013-October/016139.html">Russell
Bryant</a></p>
<blockquote>
<p>One of the most exciting things for me over the last year has been
helping to expand OpenStack by including a number of new projects. I
think this growth is important for OpenStack's success. I would like
to continue to help guide this growth and to help figure out how
OpenStack can scale as an organization and still be effective.</p>
</blockquote>
<p><a href="http://lists.openstack.org/pipermail/openstack-dev/2013-October/016124.html">Anne
Gentle</a></p>
<blockquote>
<p>My goal is to provide an inclusive and supportive environment for
projects while making OpenStack better for users and admins all the
time. We are so fortunate to have the explosive growth and interest in
OpenStack, and I want it to continue. We have built upon incredible
ideas and I want us to be empowered to innovate.</p>
</blockquote>
<p><a href="http://lists.openstack.org/pipermail/openstack-dev/2013-October/016123.html">Mark
McLoughlin</a></p>
<blockquote>
<p>We welcomed Heat, Ceilometer, Trove, Savannah, Marconi into the
OpenStack family either as integrated or incubating projects. The TC
carefully considered each of these applications and my own rule of
thumb was "does it have a healthy contributor community and is it a
sensible growth of OpenStack's scope?". I love to see this sustainable
growth in our project and community.</p>
</blockquote>
<p><a href="http://lists.openstack.org/pipermail/openstack-dev/2013-October/016163.html">Doug
Hellmann</a></p>
<blockquote>
<p>I share the view of many of the other candidates that OpenStack should
not limit itself to today's definition of IaaS. The history of
computing is a progression of different levels of abstraction, and
what we consider "platform" today may become "infrastructure"
tomorrow.</p>
</blockquote>
<p><a href="http://lists.openstack.org/pipermail/openstack-dev/2013-October/016107.html">Sean
Dague</a></p>
<blockquote>
<p>I'm incredibly excited by OpenStack's growth (in people, code, scope),
which I attribute to an incredibly welcoming and constructive
community, and the velocity we get out of our preemptive integration
system. As a TC member I'd do my best to ensure those conditions
remain. I think we've only just begun to see what OpenStack will
become [..]</p>
</blockquote>
<p><a href="http://lists.openstack.org/pipermail/openstack-dev/2013-October/016067.html">James E.
Blair</a></p>
<blockquote>
<p>As a significant OpenStack user, I'm excited about the direction that
OpenStack is heading. I'm glad that we're accepting new programs that
expand the scope of our project to make it more useful for everyone. I
believe a major function of the Technical Committee is to curate and
shepherd new programs through the incubation process. However, I
believe that it should be more involved than it has been. We have been
very quick to promote out of integration some exciting new projects
that may not have been fully integrated. As a member of the TC, I
support our continued growth, and I want to make sure that the ties
that hold our collection of projects together are strong, and more
than just a marketing umbrella.</p>
</blockquote>
<p><a href="http://lists.openstack.org/pipermail/openstack-dev/2013-October/016081.html">Michael
Still</a></p>
<blockquote>
<p>First off, the TC has incubated a number of projects in the Havana
release, and I'd like to see that continue. I think its important that
we build a platform that includes the services that a deployer would<br>
need to build a cloud and that those platform elements work well
together. Now, its clear that not everyone will deploy all of the
projects we are incubating, but I think its still important that they
play well together and have a consistent look and feel.</p>
</blockquote>
<p><a href="http://lists.openstack.org/pipermail/openstack-dev/2013-October/016146.html">John
Griffith</a></p>
<blockquote>
<p>New projects and growth are important to OpenStack however I don't
think that uncontrolled and disjointed growth in the form of new
projects is a good thing, in fact I think it's detrimental to
OpenStack as a whole. I personally would like to see the TC have more
involvement in terms of recommending/investigating new projects before
they're proposed or started by others. By the same token, I'd also
like to see the TC take a more active role in the projects we
currently have and how they all tie<br>
together. I personally believe that having 10 or so individual
projects operating in their own silos is not the right direction. My
opinion here does NOT equate to "more control", but instead should
equate to being more helpful. With the continued growth of OpenStack I
believe it's critical to have some sort of vision and some resources
that have a deep understanding of the entire eco-system.</p>
</blockquote>
<p><a href="http://lists.openstack.org/pipermail/openstack-dev/2013-October/016381.html">Mark
McClain</a></p>
<blockquote>
<p>The issue of scope was a recurring theme during my recent term on the
TC. As the OpenStack ecosystem grows beyond Infrastructure as a
Service, the committee needs to more clearly define the criteria used
to determine the kind of projects and programs that fit within the
scope of integrated releases and how they move through the progression
of incubation to graduation. In addition to defining the criteria, the
Technical Committee should to work develop policies and procedures to
provide some guidance to projects which are outside of the scope an
integrated release, but valuable to our community.</p>
</blockquote>
<p><a href="http://lists.openstack.org/pipermail/openstack-dev/2013-October/016324.html">Robert
Collins</a></p>
<p>(I don't see a directly relevant quote from Robert)</p>Oct 3rd OpenStack Foundation Board Meeting2013-10-03T21:36:00+01:002013-10-03T21:36:00+01:00markmctag:crustyblaa.com,2013-10-03:/oct-3rd-openstack-foundation-board-meeting.html<p>On Oct 3rd, the OpenStack Foundation Board held a two hour conference
call meeting which was open to anyone to join. <a href="https://wiki.openstack.org/wiki/Governance/Foundation/3Oct2013BoardMeeting">The agenda was published
in
advance.</a></p>
<p>As usual, this my informal recollection of the meeting. It’s not an
official record, etc. <a href="http://lists.openstack.org/pipermail/foundation/2013-October/001481.html">Jonathan published an official summary on the …</a></p><p>On Oct 3rd, the OpenStack Foundation Board held a two hour conference
call meeting which was open to anyone to join. <a href="https://wiki.openstack.org/wiki/Governance/Foundation/3Oct2013BoardMeeting">The agenda was published
in
advance.</a></p>
<p>As usual, this my informal recollection of the meeting. It’s not an
official record, etc. <a href="http://lists.openstack.org/pipermail/foundation/2013-October/001481.html">Jonathan published an official summary on the
foundation mailing
list</a>.</p>
<h3>Preliminaries</h3>
<p>Our chairman, Alan Clark began the meeting by holding a roll call. If my
count was correct, we had 21 of the 24 members of the board on the call,
which is a pretty good turnout.</p>
<p>Alan introduced Chris Kemp as a replacement for Devin Carlen who had
been representing Nebula and the Gold Members. Chris briefly explained
that changes in his role at Nebula has left him more time for "strategic
stuff" and is delighted to spend more time with the OpenStack community
and join the board. He thanked everyone for their work in progressing
the foundation to date.</p>
<p>Before getting down to business, we quickly voted to approve <a href="https://wiki.openstack.org/wiki/Governance/Foundation/6Aug2013BoardMeeting">the
minutes of the previous
meeting</a>.</p>
<h3>Executive Director Report</h3>
<p>Jonathan Bryce took the floor and gave a really excellent update on the
Foundation's progress.</p>
<p>First off, he described the launch of the <a href="http://www.openstack.org/marketplace/training/">OpenStack Foundation Training
Marketplace</a> at LinuxCon
in New Orleans. The site now has 12 companies listing training events at
over 30 cities around the world and has seen over 10,000 unique visitors
over the past 2 weeks. He reiterated how one of the Foundation's stated
priorities for this year is to help training providers get the word out
on their education programs and thereby expand the pool of people with
OpenStack skills.</p>
<p>Next up, Jonathan gave an update on preparations for the <a href="http://www.openstack.org/summit/openstack-summit-hong-kong-2013/">OpenStack
Summit in Hong
Kong</a>.
Everything is coming together nicely and the finalized speaking agenda
has now been posted. He explained that, while they had received some 250
submissions for the summit in Portland, there were over 600 submissions
for Hong Kong and yet not many extra speaking slots. The track chairs
had a tough job whittling down the submissions but he thinks the end
result is an agenda packed with great sessions. Jonathan warned that,
unlike previous summits, we have a hard limit on the number of attendees
and the "full access" passes will sell out in the next week or so.</p>
<p>Eileen asked exactly how many full access passes were issued and
Jonathan explained there are 2,000 but that there are many more passes
for the general sessions and expo hall. Joseph asked for some insight
into the demographics of the registered attendees and how it compares to
previous summits. Jonathan felt that we would see a large number of
attendees from the region and that, while some companies may have
reduced their numbers travelling from further afield, there would still
be plenty of representation from all of the companies we are used to
seeing at summits.</p>
<p>The discussion then moved on to looking ahead to 2014 and a sneak peak
of next year's budget. Jonathan explained how 2013 was a great year
financially - particularly because the summits were larger than expected
- and the Foundation goes into 2014 with a healthy budget surplus of
over \$1M. We expect to continue to see large summits and this will lead
to increased revenues and expenses. Jonathan expects the Foundation to
continue to invest in technology and infrastructure over 2014. He
described the Foundation funded projects in 2013 to improve the member
database and tie it to the contributor agree process and an improved
system for handling summit speaker submissions.</p>
<p>Rob Hirschfeld asked whether the Foundation had budgeted for
certification testing infrastructure being proposed as part of the "what
is core" discussion. Josh felt this it not be necessary for the
Foundation to fund this, and there was a brief data tracking and
authentication systems.</p>
<p>Jonathan continued by describing the rapid growth in Foundation staff
during 2013, but that he expects ongoing staffing costs to have mostly
levelled off for 2014. He went on to talk about how he sees the role of
the Foundation over the next year to be around connecting different
parts of the community, helping to gather data on users and deployments
and sharing information with the world about new features in each
release.</p>
<p>One Jonathan had finished, Lew Tucker congratulated Jonathan on the
progress and asked whether a written summary could be prepared for board
members so that we could help get the word out. Josh pointed out that we
were soon due an annual report which would contain much of this
information.</p>
<p>Josh then asked a question about the project budget for gold member fees
and how many new members this assumed would be added in 2014. There was
some confusion on the issue because gold member fees are based on the
member's revenue, but the conclusion was that the addition of 4 new
members was projected. This would bring the total number of gold members
up to 20 out of a total possible 24 members.</p>
<h3>Election Committee Report</h3>
<p>Next up, Todd Moore delivered an excellent summary of the discussions
and conclusions of the committee established to consider whether any
changes were need for the individual and gold member director election
systems and what those changes might be.</p>
<p>Todd's report - which will be published in full soon - described how we
considered three separate questions. Firstly, whether and how concerns
about the fairness of the individual member director election system
(particularly the issue of large blocks of affiliated voters) should be
addressed, secondly whether we should consider any rule changes around
foundation employees eligibility for the individual member director
election and, finally, whether the eligibility rules for the 2013 gold
member director elections had been properly enforced.</p>
<p>The last topic was easily dispensed with - Todd and the committee worked
with the foundation staff to verify that all eligibility rules (i.e.
eligibility as a candidate and eligibility to vote) had been properly
enforced.</p>
<p>The first topic proved to be substantially more contentious. The pros
and cons of changing to an STV based election system or tweaking the
rules of the current system to halve (from 8 to 4) the number of votes
which you could allocate to a single candidate were explained in great
detail. Todd explained how the committee felt that changes were not
necessarily warranted but that it would be wise to seriously consider
the option of the "max 4 votes per candidate" tweak.</p>
<p>A motion was proposed to not fundamentally change the voting system and
was almost voted for until it became apparent that Monty wanted to speak
but was unable to come off mute. Once that was sorted out, Monty stated
that the cumulative system was utterly broken and that block voting did,
in fact, influence the outcome of the election. He felt strongly that a
change to STV was needed. The debate then took off in earnest with many
well put points made by various members. I explained how I agreed fully
with Monty but that I was convinced by the "it's working reasonably
well; this doesn't warrant a the disruption of a bylaws change" but that
we needed to be alert to any renewed concern from members after the next
election. In the end the vote was passed with no abstentions and Monty
voting against.</p>
<p>The discussion then moved on to the question of the "only 4 votes per
candidate" tweak. Much of the discussion hinged on whether this tweak
required a bylaws change. The election committee and Jonathan felt that
it was compatible with the "cumulative voting system" language in in
<a href="https://wiki.openstack.org/wiki/Governance/Foundation/Bylaws#ARTICLE_III._MEMBERSHIP_MEETINGS">section 3.9a of the
bylaws</a>
but legal counsel for the foundation (Mark Radcliffe) disagreed strongly
and felt "cumulative voting" was "binary in the eyes of the law". Since
the committee had suggested this tweak on the understanding that it was
easy to implement, the idea was dropped.</p>
<p>Next, we discussed the second issue - the question of foundation
employees on the board. The committee did not have a consensus
recommendation on this but Todd did summarize the different viewpoints
and arguments. One approach proposed that the executive director would
be granted a new, automatic seat on the board and (because there is a
limit of two members per organization), only one additional foundation
employee could be elected to the board rather than the current limit of
two employees. There was lots of debate and I felt Jonathan's input was
particularly interesting - that he did not see the need for an automatic
executive director seat and he had reservations about restricting
employees from being directors since they could bring invaluable and
passionate input, should the electorate choose to elect them. In the
end, it was felt there was not sufficient cause to warrant a disruptive
bylaws change and the motion was withdrawn.</p>
<p>So, in summary, there will be no election system changes proposed to the
electorate during this cycle. However, the board again re-stated the
importance that all members adhere to the <a href="http://www.openstack.org/legal/community-code-of-conduct/">community code of
conduct</a>
which states:</p>
<blockquote>
<p>Respect the election process. Members should not attempt to manipulate
election results. Open debate is welcome, but vote trading, ballot
stuffing and other forms of abuse are not acceptable.</p>
</blockquote>
<p>Finally, the committee took an action item from the board to analyze the
results of the next election and provide the board with a report on the
potential effect of any of the options proposed.</p>
<h3>Wrapping Up</h3>
<p>Because of the lengthy debate on the election system, we had run out of
time and the rest of the agenda had to be postponed until the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/4Nov2013BoardMeeting">in person
board meeting in Hong
Kong</a>.</p>
<h3>Edits</h3>
<ul>
<li>Changed from "legal counsel for the board" to "legal counsel for
the foundation". Thanks to Richard Fontana for the correction.</li>
<li>Included a reference to the code of comment. Thanks to Rob Esker for
his question.</li>
</ul>Aug 6th OpenStack Foundation Board Meeting2013-08-08T10:25:00+01:002013-08-08T10:25:00+01:00markmctag:crustyblaa.com,2013-08-08:/aug-6th-openstack-foundation-board-meeting.html<p>On Aug 6th, the OpenStack Foundation Board held a two hour conference
call meeting which was open to anyone to join. <a href="https://wiki.openstack.org/wiki/Governance/Foundation/6Aug2013BoardMeeting">The agenda was published
in
advance</a>.</p>
<p>As usual, this my informal recollection of the meeting. It's not an
official record, etc. Jonathan published an <a href="http://lists.openstack.org/pipermail/foundation/2013-August/001454.html">official
summary</a>
on the foundation …</p><p>On Aug 6th, the OpenStack Foundation Board held a two hour conference
call meeting which was open to anyone to join. <a href="https://wiki.openstack.org/wiki/Governance/Foundation/6Aug2013BoardMeeting">The agenda was published
in
advance</a>.</p>
<p>As usual, this my informal recollection of the meeting. It's not an
official record, etc. Jonathan published an <a href="http://lists.openstack.org/pipermail/foundation/2013-August/001454.html">official
summary</a>
on the foundation mailing list.</p>
<h3>Preliminaries</h3>
<p>Alan Clark chaired the meeting and spent the first few minutes reviewing
the agenda and holding the roll call. We had more than 50% of the board
members, and so had sufficient quorum.</p>
<p>We then reviewed the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/27June2013BoardMinutes">minutes of the previous
meeting</a>
and approved them by vote. Todd, Troy, Nick, Devin and Jim all abstained
because they weren't present at that meeting.</p>
<h3>Patent Cooperation</h3>
<p>Next up was a follow-on from the patent cooperation discussion at our
<a href="http://blogs.gnome.org/markmc/2013/06/29/june-27th-openstack-foundation-board-meeting/">previous
meeting</a>.</p>
<p>The Legal Affairs Committee had previously <a href="https://wiki.openstack.org/w/images/2/24/OSLAC_Update_for_Board_062713.pdf">proposed three
options</a>
that we make no change, that we implement a Google-style patent pledge
or that we introduce an OIN-style cross licensing agreement.</p>
<p>Keith Bergelt, CEO of the Open Invention Network, attended the meeting
and gave a
<a href="https://wiki.openstack.org/w/images/1/10/20130806OIN_Presentation_To_The_Open_Stack_Board.pdf">presentation</a>
describing the OIN and how it could play a key role in OpenStack's
strategy to create an environment of patent non-aggression around
OpenStack. Keith has been actively working with the Legal Affairs
Committee to develop a fourth proposal for the board which would make
the best possible use of the OIN.</p>
<p>Keith described how the OIN was formed 8 years ago by IBM, NEC, Novell,
Philips, Red Hat and Sony to ensure "freedom of action, freedom to
operate" around Linux. It has some 560 member companies who sign up to a
broad cross-licensing agreement whereby any patents they hold which read
on Linux is licensed to all other member companies.</p>
<p>The patents which are covered by the OIN agreement are those which read
on the software packages listed in the <a href="http://www.openinventionnetwork.com/pat_linuxdef.php">OIN's Linux System
Definition</a>. Over
time, this definition has grown to include the likes of Android, WebOS
and, more recently, OpenStack. The OIN views OpenStack has one of the
most import projects after Linux and feels that including OpenStack in
its defintiion is an obvious step.</p>
<p>That OpenStack is included in the OIN system definition means that there
are already 560 licensees committed to non-aggression around OpenStack.
Keith feels that this goes a long way to achieving the OpenStack
Foundation's goals in this area and that we should seek to make the most
effective use of the existence of this agreement rather than reinvent
the wheel.</p>
<p>Keith proposes a more formalized relationship with the Foundation, that
members would be encouraged to join the OIN, that we'd do joint press
releases and that there would be an ongoing direct relationship with the
TC to ensure the system definition is regularly updated to include all
OpenStack projects.</p>
<h3>Legal Affairs Committee</h3>
<p>Van Lindberg was up next to provde an update from the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/LegalAffairsCommittee">Legal Affairs
Committee</a>
but this turned out to be a continuation of the previous discussion on
patent cooperation.</p>
<p>Van summed up this fourth proposal succinctly - that OpenStack members
get behind the OIN. He went on to describe how a difficulty with this
approach is that some members of OpenStack will never be members of the
OIN because the OIN fundamentally see those companies as a threat to
Linux. He didn't name these companies.</p>
<p>So, the question becomes how we can ensure the ability of those
OpenStack members who aren't members of the OIN to join a patent
cooperation initiative specifically around OpenStack.</p>
<p>Van first described how OpenStack LLC (the Rackspace owned entity which
held OpenStack's before the Foundation was formed) actually joined the
OIN some time ago because of concerns about the <a href="http://www.virtualizationpractice.com/doj-blocks-emcvmware-from-acquiring-virtualization-patents-from-novell-10368/">"CPTN
transaction"</a>.
The OpenStack Foundation could quite easily take over this agreement
and, by virtue of the fact that the OpenStack Foundation is in some
sense the entity licensing the OpenStack software to everyone, the
benefits of the OIN cross-license would flow to all OpenStack users.</p>
<p>For OIN non-members, the Foundation could create a separate, limited
license agreement whereby the OpenStack related patents held by such
licensors would be licensed to the Foundation and the benefits of that
agreement would also flow to all OpenStack users.</p>
<p>Van went on to talk a little about patent trolls <a href="http://www.rackspace.com/blog/tag/patent-trolls/">(a subject close to
his heart)</a>, how this
is a serious issue for OpenStack and how, even now, there is a troll
going after Rackspace and HP over a number issues including some related
to OpenStack. Van feels that the OIN does not actually protect against
the threat of trolls very well, because (somewhat obviously) such trolls
would not be members of the OIN.</p>
<p>Van raised again (this has been discussed before) the idea of a patent
indemnity program whereby OpenStack members (or even just the
Foundation) would contribute to a fund which could be used for the legal
defence (not any settlements themselves) of any OpenStack members
threatened by patent trolls for the use of OpenStack. Van feels that
patent trolls often go after some weaker companies, obtain a small
settlement because that company feels it's cheaper to settle than to
defend and that settlement sets a problematic precedent that the troll
can then build upon. An indemnity program would encourage members to
fight patent trolls and avoid precedents being established which would
set the entire community up for trouble.</p>
<p>Eileen made the point that the OIN was created because the GPL lacks a
patent license. We're not in the same position, since OpenStack is
licensed under the Apache License which includes a broad patent license.
She raised concerns about the idea of an indemnity program, how it's
scope could turn out to be very large and expensive and disagreed that
trolls always go after the smaller players.</p>
<p>The conclusion was ... no decisions today, more dialogue needed.</p>
<h3>OpenStack Training</h3>
<p>Next up, Jonathan revisited the topic of how the Foundation can
encourage and regularize the market forming around OpenStack training.</p>
<p>This topic was also covered in the June meeting and Jonathan felt there
was good support for some of the proposals but board members had
concerns and questions about other aspects. Since then, Jonathan has
been talking to several board members including Brian Stevens and Boris
Renski and feels he can provide some clarifications and updates. The
Foundation would like to move forward with two proposals - a training
licensing program and a baseline certification test.</p>
<p>The training licensing program would replace the use by training
providers of the Built For OpenStack mark. This will allow the
Foundation to have a closer relationship with training providers and
help raise awareness about the availability of OpenStack training. There
is broad support for this proposal<br>
and the Foundation have a draft agreement ready to circulate to training
providers.</p>
<p>The baseline certification test proposal was the proposal that has
required more discussion but it is better understood now. The idea is
that the Foundation would define a common baseline set of training which
all licensed OpenStack providers must include and certify against, but
that the Foundation also fully expects providers to expand upon this
content greatly and compete on the unique aspects of their training.</p>
<p>The vast majority of what would be included in this baseline training is
already covered by all training courses out there. The Foundation would
not seek to force content and topics on training providers but would
instead seek to add new content when it sees that training providers are
already covering these new topics. It is expected that the Foundation
staff and others it engages will take 6-9 months to prepare this
program.</p>
<p>Jonathan feels a baseline OpenStack certification would ensure that
there is OpenStack training which globally available, not
cost-prohibitive for students in regions all around the world and
provides a target for people who want to become knowledge around
OpenStack.</p>
<p>The board moved to support the Foundation staff in its efforts to create
a baseline certification testing program for OpenStack. The motion was
passed unanimously.</p>
<h3>User Committee Update</h3>
<p>Tim Bell took the floor next to raise an issue the <a href="http://www.openstack.org/foundation/user-committee/">User
Committee</a> has been
discussing in recent times.</p>
<p>The committee currently consists of Tim, Ryan Lane and JC Martin. While
the committee is doing a great job, they feel that their limited numbers
(and their limited time) is constraining their ability to achieve some
of their goals.</p>
<p>The obvious solution is to add more members to the committee, but
there's a catch. Committee members sign an NDA and have access to some
sensitive data about OpenStack deployments which are obtained through
surveys with the understanding that the information is confidential.
Adding significantly more members could compromise the confidence that
survey responders would have in the confidentiality of their responses.</p>
<p>Tim has been talking with Tom Fifield about what role Tom could take (as
a Foundation staff member) on the committee in order to help out,
without Tom having to sign the NDA and gain access to the confidential
survey results. The idea is that Tom can take on some of the
responsibility to act as a go-between who ensures that issues which are
identified by the committee are raised as appropriate with the developer
community.</p>
<p>In a similar vein, the committee is exploring the idea of a structure
whereby only the "core" committee members need to sign the NDA and there
would be sub-groups of the committee who can get involved without
gaining direct access to the survey results.</p>
<h3>What is Core</h3>
<p>Rob Hirschfeld gave a quick update on the "what is core" discussions he
is<br>
leading. The discussion was initially limited to the board, more
recently<br>
<a href="http://lists.openstack.org/pipermail/openstack-tc/2013-July/thread.html#308">broadened to include the
TC</a>
and Rob now plans to broaden it to include the<br>
wider community.</p>
<p>Rob mentioned a
<a href="http://lists.openstack.org/pipermail/foundation/2013-August/001453.html">flowchart</a>
he has prepared to describe some of the proposed process. He also
discussed how the initial position on requiring "plugins" has evolved
into talk of designating some parts of projects as "required
implementations" and allowing other parts to have "alternate
implementations".</p>
<h3>Other Updates</h3>
<p>Todd and Rob met with the Foundation's legal team to discuss what
changes to the Individual Directors election process would be allowed
under Delaware law. He will present some thoughts on this at the next
board meeting.</p>
<p>Simon has been working with Mark Collier and Heidi Bretz to ensure that
companies who have expressed an interest in applying to be a Gold Member
are aware of the board's revised expectations around new members. Simon
and the membership committee still plan on re-drafting the wiki page
describing the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/PotentialMemberCriteria">criteria for new
members</a>.</p>
<p>Alan mentioned that the Compensation Committee completed a mid-year
review of Jonathan's goals for the year and concluded things are very
much on track. Full details were posted to the foundation-board mailing
list.</p>
<p>The next board meeting is scheduled for October 3rd at 9am PST. There
will also be an all day board meeting in Hong Kong on November 4th.</p>Python Exception Chaining2013-07-11T09:28:00+01:002013-07-11T09:28:00+01:00markmctag:crustyblaa.com,2013-07-11:/python-exception-chaining.html<p>OpenStack has an awful lot of developers writing Python code and many of
us wouldn't consider ourselves true "pythonistas". This means we wind up
having a bunch of interesting discussions about e.g. <a href="http://lists.openstack.org/pipermail/openstack-dev/2013-July/thread.html#11635">EABL vs
LBYL</a>.</p>
<p>A particular bugbear of mine is exception handling. I'm convinced that
very, very few …</p><p>OpenStack has an awful lot of developers writing Python code and many of
us wouldn't consider ourselves true "pythonistas". This means we wind up
having a bunch of interesting discussions about e.g. <a href="http://lists.openstack.org/pipermail/openstack-dev/2013-July/thread.html#11635">EABL vs
LBYL</a>.</p>
<p>A particular bugbear of mine is exception handling. I'm convinced that
very, very few developers think hard about their error handling strategy
and that the problem is even more serious with exception based
languages.</p>
<p>So, I really like getting into error handling discussions. This morning,
we have a great one - <a href="http://lists.openstack.org/pipermail/openstack-dev/2013-July/011702.html">how to do exception chaining in Python, and
whether that's even something you want to
do</a>.</p>June 27th OpenStack Foundation Board Meeting2013-06-29T16:04:00+01:002013-06-29T16:04:00+01:00markmctag:crustyblaa.com,2013-06-29:/june-27th-openstack-foundation-board-meeting.html<p>On June 27, the OpenStack Foundation Board of Directors met for two
hours via conference call. As usual, the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/27June2013BoardMeeting">agenda was published in
advance</a>
and the meeting was open to anyone who wanted to observe the
proceedings.</p>
<p>These notes are my perspective of the meeting. Jonathan Bryce <a href="http://lists.openstack.org/pipermail/foundation/2013-June/001435.html">published
an official …</a></p><p>On June 27, the OpenStack Foundation Board of Directors met for two
hours via conference call. As usual, the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/27June2013BoardMeeting">agenda was published in
advance</a>
and the meeting was open to anyone who wanted to observe the
proceedings.</p>
<p>These notes are my perspective of the meeting. Jonathan Bryce <a href="http://lists.openstack.org/pipermail/foundation/2013-June/001435.html">published
an official
summary</a>
and official minutes will be posted in due course.</p>
<h3>Roll Call and Meeting Confidentiality Policy</h3>
<p>As you can imagine, a conference call with 20-30 attendees takes a while
to get going. We began with a roll call and, after perhaps 15 minutes,
were ready to get started.</p>
<p>First item on the agenda was a review of our meeting confidentiality
policy. Directors agree to refrain from commenting on the meeting
proceedings until Jonathan posts his summary. The only official record
of the meeting is Jonathan's summary and the official minutes. Anything
discussed during executive session is confidential. Nothing new here.</p>
<h3>Amendment to our Certificate of Incorporation</h3>
<p>Next up was a motion to approve a relatively minor amendment to the
foundation's certificate of incorporation.</p>
<p>The details of the amendment is fairly obscure, but essentially the
Foundation is applying for <a href="https://en.wikipedia.org/wiki/501%28c%29_organization">U.S. 501(c)
status</a> which
means it will be a tax-exempt, non-profit organization. There are
various different organization types, but the<br>
two most relevant are 501(c)(3) and 501(c)(6). OpenStack is filing for
501(c)(6) status.</p>
<p>Jonathan explained that, while preparing for this filing, it was noted
that the original certificate of incorporation only allows (on winding
up of the foundation) the foundation's assets to be transferred to a
501(c)(3) organization. This amendment simply allows for the possibility
of transferring assets to 501(c)(6) organizations.</p>
<p>There was some brief discussion clarifying exactly which status we were
filing for and the motion was passed unanimously.</p>
<h3>Transparency Policy</h3>
<p>Next up, Lauren Sell explained the context for a formal transparency
policy document which had been circulated to the board members and would
require further discussion before being approved.</p>
<p>Lauren reminded us that at our meeting in April, the Transparency
Working Group presented the principles for a transparency policy. Lauren
had since worked with legal counsel to draft a more formal policy based
on those original principles.</p>
<p>The main question Lauren felt needed to be cleared up was the issue of
document management. The board has (and will continue to have) documents
which must remain confidential. At our April meeting we had agreed that
the OpenStack Infrastructure team would investigate hosting an OwnCloud
instance which would act as our document store. While this was still on
the team's to-do list, it had not been prioritized over other work.</p>
<p>I suggested that, in the team time, we create a new mailing list for the
the board (e.g. foundation-directors?) which would be open to everyone
for read-only subscription and we would use the current mailing list to
share confidential documents. Once the document management system is in
place, we could then shut down the private foundation-board mailing
list.</p>
<p>Rather than discuss in any great detail, it was agreed the Lauren would
start a discussion on the foundation mailing list.</p>
<h3>Summits & Marketing</h3>
<p>Next up, Mark Collier and Lauren gave us an update on the Marketing
front and how summit planning is progressing.</p>
<p>Lauren first talked through some excellent slides with a tonne of
details about the Icehouse Design Summit in Hong Kong from Nov 5-8,
2013. This is the first time that the summit will be held at an
"international venue" (an amusing term if you're not U.S. based :) and
we again expect record attendance.</p>
<p>Included in Lauren's slides were some really helpful maps and aerial
shots showing the venue, the geography of Hong Kong and the location of
the recommended hotels. The venue is located near the airport which is a
25 minute train journey from down town Hong Kong. There are a couple of
hotels adjacent to the venue and most of the other recommended hotels
are down town. The foundation staff have worked hard to come up with a
good range of hotel options, including hotels with a rate of under \$150
per night.</p>
<p>In terms of travel advice, it was noted that visitors must have a
passport valid for at least one month after their planned stay and that
flights from SFO to HKG are currently averaging between \$1000 and
\$1400. Jonathan recommends that people book their flights early,
because fares will increase very significantly closer to the event.
Lauren also pointed out that it's sensible to make hotel books now,
since the hotels closest to the venue are already selling out.</p>
<p>Lauren then talked through the planned format for the summit, which has
been heavily influenced by feedback received through the <a href="http://www.openstack.org/blog/2013/06/openstack-summit-survey-results/">survey results
from the previous
summit</a>.</p>
<p>This time around there will be two types of passes. A more affordable
limited access pass will give access to the expo hall, general sessions
and a single track of breakout sessions on Tuesday and Wednesday. The
hope is that this will help control the numbers at the breakout
sessions, but also make the event more accessible to folks who just want
to come along for the first time and learn about OpenStack.</p>
<p>The primary language of the event will be English, but there will be
simultaneous translation into Mandarin in the main hall.</p>
<p>The call for sponsors is already open and we have 21 sponsors to date.
The headline sponsorship sold out in an astonishing 7 minutes.</p>
<p>For the first time, there will be a travel support program designed to
ensure that lack funding won't prevent key technical contributors from
attending the event. Details of this will be announced very soon. We had
a brief discussion about how this program should be run and it was
pointed out that we could learn from similar programs for PyCon and UDS.</p>
<p>In terms of learnings from the previous summit, some of the things the
team will be working hard to improve is the quality of network
connectivity, the size breakout rooms and the variety of beverages and
snacks.</p>
<p>It was noted that feedback from ATCs who completed the survey was 2.5:1
in favour of keeping the design summit collocated with the conference.
In Hong Kong, the design summit rooms will be well separate from the
breakout session rooms, ATC status will be properly indicated on name
badges and it will be much more clear on the schedule which sessions are
part of the design summit and which are breakout sessions.</p>
<p>After some interesting discussion about the plans for Hong Kong, Lauren
gave a brief overview of how plans are proceeding for the 2014 summits.
The spring summit is planned for the week of May 14 with Atlanta and
Chicago under consideration. The autumn/fall summit will be one of the
first two weeks of November with Paris and Berlin currently under
consideration. Decisions on the venues for both these summits are
expected to be made soon.</p>
<p>Finally, Lauren ran through some updates on the progress of the
<a href="http://wiki.openstack.org/Governance/Foundation/Marketing">marketing
community</a>
more generally. Version 1 of the OpenStack <a href="http://www.openstack.org/marketing">marketing
portal</a> has been made available. The
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/marketing">mailing
list</a> is
gaining new subscribers all the time. The monthly meetings are also
seeing growing numbers attending.</p>
<h3>Patent Cooperation Proposal</h3>
<p>Next on the agenda was a presentation from several members of the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/LegalAffairsCommittee">Legal
Affairs
Committee</a>
on the three options they recommend the foundation should consider for
increased cooperation on patent sharing or cross-licensing between
foundation members.</p>
<p>Frankly, I don't really have the energy to try and summarise these
proposals in great detail, so this is short ... however, this is
certainly a complex and important topic.</p>
<p>The Apache License v2.0 has a patent provision which means you grant a
license to any of your patents which are infringed upon by any
contributions you make. If any licensee of the software claims that the
software infringes on their patent, then they lose any patent rights
granted to them under the license.</p>
<p>Two options were presented to the board for how we might encourage
further sharing of patents related to OpenStack between the companies
involved. The idea is that we could put OpenStack in a better defensive
position by sharing a wider range of applicable patents.</p>
<p>The first option proposed was to closely copy <a href="http://www.google.com/patents/opnpledge/pledge/">Google's Open Patent
Non-Assertion Pledge</a>.
The idea is that companies involved in OpenStack would pledge to not
assert specific sets of relevant patents against OpenStack users.</p>
<p>The alternative option proposed was the adoption of an <a href="http://www.openinventionnetwork.com/patents.php">OIN-style patent
cross-licensing
scheme</a>. The primary
difference of this scheme is that an actual patent license is granted to
users, rather than just a non-assertion pledge.</p>
<p>The slides outlining these options will be posed to the foundation wiki
page. It is hoped the board will be in a position to come to a decision
on this in November.</p>
<h3>Closing Topics</h3>
<p>Alice King is stepping down from her role on the Legal Affairs
Committee, so the board voted to approve a motion to appoint Van
Lindberg to the committee.</p>
<p>Rob Hirschfeld gave an update on his work to bring about a productive
discussion on the question of "what is core?". Rob has held a couple of
meetings with other board members and drafted six position statements
which he hopes will drive the discussion towards a consensus-based
decision. Rob wishes the board to come to a good level of consensus on
these position statements before opening the discussion up to the rest
of the community.</p>
<p>Finally, there was a brief discussion about training and how members of
the User Committee are actively working on training materials.</p>May 30th OpenStack Foundation Board Meeting2013-06-05T22:16:00+01:002013-06-05T22:16:00+01:00markmctag:crustyblaa.com,2013-06-05:/may-30th-openstack-foundation-board-meeting.html<p>Last Thursday, on May 30, the OpenStack Foundation Board met over the
phone for two hours to discuss a number of topics. The date for the
meeting was set well in advance, the call was open to anyone who cared
to listen in and agenda was posted in advance to …</p><p>Last Thursday, on May 30, the OpenStack Foundation Board met over the
phone for two hours to discuss a number of topics. The date for the
meeting was set well in advance, the call was open to anyone who cared
to listen in and agenda was posted in advance to <a href="https://wiki.openstack.org/wiki/Governance/Foundation/30May2013BoardMeeting">our wiki
page</a>.</p>
<p>Below is my summary of our discussions. The official summary was <a href="http://lists.openstack.org/pipermail/foundation/2013-May/001428.html">posted
earlier by Jonathan
Bryce</a>.</p>
<p>For this meeting, Lew Tucker acted as chairman in place of Alan Clark.</p>
<h3>Training</h3>
<p>The first big topic on the agenda was an update from Mark Collier on the
Foundation's work-in-progress plans for official OpenStack training and
certification programs.</p>
<p>The idea is that, with the huge interest globally in OpenStack, we're
hearing consistently that there is a shortage of people with OpenStack
expertise and training. The Foundation would like to address this in a
way that adds new Openstack experts, grows our community and establishes
a base knowledge set that everyone is united around.</p>
<p>The OpenStack ecosystem is already ramping up its training offerings and
classes are available today from a number of companies all over the
world. The Foundation wants to encourage this and help accelerate the
availability of training.</p>
<p>Crucially, the Foundation proposes to introduce a new trademark program
which would require all OpenStack training and credentialling providers
to include a base set of training material and tests which would be
developed by the Foundation itself. The hope is that we would protect
the OpenStack brand by ensuring that all official training courses would
have the same basic content and quality levels.</p>
<p>This proposal of a new trademark program triggered a significant amount
of debate which continued on over email after the call. On the call, Jim
Curry and Boris Renski kicked off the discussion by expressing similar
concerns about whether the trademark program would actually hinder the
growth of training offerings in the ecosystem. Jonathan clarified that
intent isn't to prevent training programs from competing with additional
content, but rather to ensure all programs have a common baseline.
Others like Nick Barcet and Todd Moore chimed in with the view that
OpenStack really needs this and Nick drew an analogy with Linux
Professional Institute certification.</p>
<p>After much discussion, the conclusion was that the topic needed further
discussion before any concrete steps could be taken. Mark Collier closed
out the topic by making the point that the "Built for OpenStack"
trademark was currently being used for OpenStack training and this would
continue until an alternative plan was put in place.</p>
<p>Expect to see plenty more discussion about this soon on <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation">the foundation
mailing
list</a>!</p>
<h3>Next Board Meeting</h3>
<p>We had some discussion about when our next meeting should be held and,
based on the availability of board members, it looks like it will be
held on Thursday, June 27 between 9am and 11am Pacific time.</p>
<h3>Gold Member Committee</h3>
<p>Next up, Simon Anderson discussed a new committee that we agreed to set
up at our previous meeting - the Prospective Gold Member Committee.</p>
<p>The idea with this committee is that when companies approach the
Foundation and express an interest in becoming a Gold Member, the
Committee will work with the Foundation staff and the prospective member
to ensure their application is properly prepared before it comes before
the board.</p>
<p>The committee will essentially act as a mentor for prospective new
members and help them understand what is expected of Gold Members. The
hope is that this will result in applicants being better prepared than
we've seen previously.</p>
<p>One concern expressed by Lauren Sell and others is that the committee
shouldn't become a vetting committee. The committee doesn't have the
mandate to turn away unsuitable candidates. If a candidate chooses to
ignore the committee's advice and mentorship, they should still be able
to have their application heard by the board even this ultimately means
the application is likely to be received unfavourably by the board. This
point was accepted and everyone was in agreement on the mandate for the
committee.</p>
<p>The members of the committee will be Simon Anderson, Devin Carlen, Rob
Hirschfeld, Joseph George, Sean Roberts and Nick Barcet.</p>
<h3>Update from the Executive Director</h3>
<p>Next, Jonathan gave the board a quick update.</p>
<p>Jonathan talked about his attending a number of OpenStack events
internationally recently and how there is still tremendous opportunity
to grow the OpenStack community and engage new people. He also talked
about how he met with a number of significant OpenStack users and hopes
to be able to use their stories to illustrate the truly global nature of
the OpenStack user community.</p>
<p>He also talked about the success of the OpenStack Summit in Portland and
how we had 2600 attendees compared to the 1300 attendees six months
previously. Feedback on the summit has been overwhelmingly positive with
the most common negative comments related to the Wi-Fi network and how
some of the breakout sessions were massively over-subscribed. The
Foundation will continue to invest significantly in networking
infrastructure at the conference and there is work underway to
restructure some of the room layouts at the upcoming Summit in Hong Kong
based on the feedback from Portland.</p>
<p>Jonathan also talked about how our financials are in good shape. We had
a surplus from the Portland Summit which will allow us to make the Hong
Kong Summit a kick-ass event. The Foundation is also well advanced in
completing its first audit and we expect to see fully audited financials
for 2012 published in August. Apparently this will be a good milestone
in our progress towards non-profit status.</p>
<h3>Role of Core</h3>
<p>After Jonathan's update, board members had an opportunity to briefly
raise topics of interest.</p>
<p>First of those was Rob Hirschfeld who wants to bring together a group of
board members to discuss the what "Core" project status means and should
mean in future.</p>
<p>Rob talked about how he feels that the issue of whether core projects
need a plugin architecture will be the key to unlocking the discussion
and making progress.</p>
<h3>Working With Standards Bodies</h3>
<p>Randy Bias mentioned that he had been approached by someone from the
IEEE who talked about the possibility of the IEEE working together with
OpenStack on interoperability and standardization issues. Randy was
mostly just passing on the message but also made the point that we can
probably use the the experience of other bodies to help us ensure
interoperability between OpenStack clouds.</p>
<p>Joshua McKenty quickly took a firm and contrary view to Randy's - that
standardization bodies are typically pretty ineffective and would
actually slow down our progress on OpenStack interoperability.</p>
<p>The discussion concluded with general agreement that while individuals
are welcome to talk to and learn from whoever they wish as part of their
efforts to help make progress on OpenStack interoperability. Josh also
agreed to provide the board with an update on his work on refstack.</p>
</h3>
<p>
Meeting Summaries
</h3>
The final brief topic was a joy indeed. As [had already been discussed
on the mailing
list](http://lists.openstack.org/pipermail/foundation/2013-May/001419.html),
Josh felt that [my previous meeting
summary](http://blogs.gnome.org/markmc/2013/05/25/april-14th-openstack-foundation-board-meeting/)
breached a policy agreed by the board (before my joining) that directors
would make no public comment board meetings until after Jonathan had
published an official summary of the meeting. Josh also felt that my
reference to the agenda of the executive session had breached the
confidentiality of the session.
Jonathan and I repeated the point that Jonathan had not gotten around to
doing an official summary in a reasonable time and explicitly given me
the go-ahead to post my summary. In some follow-up emails, I also made
the point that - in this case - we actually made no effort to keep the
agenda of the executive session private during the public part of the
meeting so it was completely appropriate to mention it in my summary.Async I/O and Python2013-06-04T16:28:00+01:002013-06-04T16:28:00+01:00markmctag:crustyblaa.com,2013-06-04:/async-io-and-python.html<p>When you're working on OpenStack, you'll probably hear a lot of
references to 'async I/O' and how eventlet is the library we use for
this in OpenStack.</p>
<p>But, well ... what exactly is this mysterious 'asynchronous I/O' thing?</p>
<p>The first thing to think about is what happens when a …</p><p>When you're working on OpenStack, you'll probably hear a lot of
references to 'async I/O' and how eventlet is the library we use for
this in OpenStack.</p>
<p>But, well ... what exactly is this mysterious 'asynchronous I/O' thing?</p>
<p>The first thing to think about is what happens when a process calls a
system call like write(). If there's room in the write buffer, then the
data gets copied into kernel space and the system call returns
immediately.</p>
<p>But if there isn't room in the write buffer, what happens then? The
default behaviour is that the kernel will put the process to sleep until
there is room available. In the case of sockets and pipes, space in the
buffer usually becomes available when the other side reads the data
you've sent.</p>
<p>The trouble with this is that we usually would prefer the process to be
doing something useful while waiting for space to become available,
rather than just sleeping. Maybe this is an API server and there are new
connections waiting to be accepted. How can we process those new
connections rather than sleeping?</p>
<p>One answer is to use multiple threads or processes - maybe it doesn't
matter if a single thread or process is blocked on some I/O if you have
lots of other threads or processes doing work in parallel.</p>
<p>But, actually, the most common answer is to use non-blocking I/O
operations. The idea is that rather than having the kernel put the
process to sleep when no space is available in the write buffer, the
kernel should just return a "try again later" error. We then using the
select() system call to find out when space has become available and the
file is writable again.</p>
<p>Below are a number of examples of how to implement a non-blocking write.
For each example, you can run a simple socket server on a remote machine
to test against:</p>
<div class="highlight"><pre><span></span><code>$> ssh -L 1234:localhost:1234 some.remote.host 'ncat -l 1234 | dd of=/dev/null'
</code></pre></div>
<p>The way this works is that the client connects to port 1234 on the local
machine, the connection is forwarded over SSH to port 1234 on
some.remote.host where ncat reads the input, writes the output over a
pipe to dd which, in turn, writes the output to /dev/null. I use dd to
give us some information about how much data was received when the
connection closes. Using a distant some.remote.host will help illustrate
the blocking behaviour because data clearly can't be transferred as
quickly as the client can copy it into the kernel.</p>
<h3>Blocking I/O</h3>
<p>To start with, let's look at the example of using straightforward
blocking I/O:</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">socket</span>
<span class="n">sock</span> <span class="o">=</span> <span class="n">socket</span><span class="o">.</span><span class="n">socket</span><span class="p">()</span>
<span class="n">sock</span><span class="o">.</span><span class="n">connect</span><span class="p">((</span><span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">1234</span><span class="p">))</span>
<span class="n">sock</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="s1">'foo</span><span class="se">\n</span><span class="s1">'</span> <span class="o">*</span> <span class="mi">10</span> <span class="o">*</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">1024</span><span class="p">)</span>
</code></pre></div>
<p>This is really nice and straightforward, but the point is that this
process will spend a tonne of time sleeping while the send() method
completes transferring all of the data.</p>
<h3>Non-Blocking I/O</h3>
<p>In order to avoid this blocking behaviour, we can set the socket to
non-blocking and use select() to find out when the socket is writable:</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">errno</span>
<span class="kn">import</span> <span class="nn">select</span>
<span class="kn">import</span> <span class="nn">socket</span>
<span class="n">sock</span> <span class="o">=</span> <span class="n">socket</span><span class="o">.</span><span class="n">socket</span><span class="p">()</span>
<span class="n">sock</span><span class="o">.</span><span class="n">connect</span><span class="p">((</span><span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">1234</span><span class="p">))</span>
<span class="n">sock</span><span class="o">.</span><span class="n">setblocking</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="n">buf</span> <span class="o">=</span> <span class="n">buffer</span><span class="p">(</span><span class="s1">'foo</span><span class="se">\n</span><span class="s1">'</span> <span class="o">*</span> <span class="mi">10</span> <span class="o">*</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">1024</span><span class="p">)</span>
<span class="nb">print</span> <span class="s2">"starting"</span>
<span class="k">while</span> <span class="nb">len</span><span class="p">(</span><span class="n">buf</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">buf</span> <span class="o">=</span> <span class="n">buf</span><span class="p">[</span><span class="n">sock</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="n">buf</span><span class="p">):]</span>
<span class="k">except</span> <span class="n">socket</span><span class="o">.</span><span class="n">error</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span>
<span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">errno</span> <span class="o">!=</span> <span class="n">errno</span><span class="o">.</span><span class="n">EAGAIN</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">e</span>
<span class="nb">print</span> <span class="s2">"blocking with"</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">buf</span><span class="p">),</span> <span class="s2">"remaining"</span>
<span class="n">select</span><span class="o">.</span><span class="n">select</span><span class="p">([],</span> <span class="p">[</span><span class="n">sock</span><span class="p">],</span> <span class="p">[])</span>
<span class="nb">print</span> <span class="s2">"unblocked"</span>
<span class="nb">print</span> <span class="s2">"finished"</span>
</code></pre></div>
<p>As you can see, when send() returns an EAGAIN error, we call select()
and will sleep until the socket is writable. This is a basic example of
an event loop. It's obviously a loop, but the "event" part refers to our
waiting on the "socket is writable" event.</p>
<p>This example doesn't look terribly useful because we're still spending
the same amount of time sleeping but we could in fact be doing useful
rather than sleeping in select(). For example, if we had a listening
socket, we could also pass it to select() and select() would tell us
when a new connection is available. That way we could easily alternate
between handling new connections and writing data to our socket.</p>
<p>To prove this "do something useful while we're waiting" idea, how about
we add a little busy loop to the I/O loop:</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="n">e</span><span class="p">.</span><span class="n">errno</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="n">errno</span><span class="p">.</span><span class="nl">EAGAIN</span><span class="p">:</span>
<span class="w"> </span><span class="n">raise</span><span class="w"> </span><span class="n">e</span>
<span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span>
<span class="w"> </span><span class="k">while</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="mi">5000000</span><span class="err">:</span>
<span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="mi">1</span>
<span class="w"> </span><span class="k">print</span><span class="w"> </span><span class="ss">"blocking with"</span><span class="p">,</span><span class="w"> </span><span class="nf">len</span><span class="p">(</span><span class="n">buf</span><span class="p">),</span><span class="w"> </span><span class="ss">"remaining"</span>
<span class="w"> </span><span class="k">select</span><span class="p">.</span><span class="k">select</span><span class="p">(</span><span class="err">[]</span><span class="p">,</span><span class="w"> </span><span class="o">[</span><span class="n">sock</span><span class="o">]</span><span class="p">,</span><span class="w"> </span><span class="err">[]</span><span class="p">,</span><span class="w"> </span><span class="mi">0</span><span class="p">)</span>
<span class="w"> </span><span class="k">print</span><span class="w"> </span><span class="ss">"unblocked"</span>
</code></pre></div>
<p>The difference is we've passed a timeout of zero to select() - this
means select() never actually block - and any time send() would have
blocked, we do a bunch of computation in user-space. If we run this
using the 'time' command you'll see something like:</p>
<div class="highlight"><pre><span></span><code>$> time python ./test-nonblocking-write.py
starting
blocking with 8028160 remaining
unblocked
blocking with 5259264 remaining
unblocked
blocking with 4456448 remaining
unblocked
blocking with 3915776 remaining
unblocked
blocking with 3768320 remaining
unblocked
blocking with 3768320 remaining
unblocked
blocking with 3670016 remaining
unblocked
blocking with 3670016 remaining
...
real 0m10.901s
user 0m10.465s
sys 0m0.016s
</code></pre></div>
<p>The fact that there's very little difference between the 'real' and
'user' times means we spent very little time sleeping. We can also see
that sometimes we get to run the busy loop multiple times while waiting
for the socket to become writable.</p>
<h3>Eventlet</h3>
<p>Ok, so how about eventlet? Presumably eventlet makes it a lot easier to
implement non-blocking I/O than the above example? Here's what it looks
like with eventlet:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">eventlet.green</span> <span class="kn">import</span> <span class="n">socket</span>
<span class="n">sock</span> <span class="o">=</span> <span class="n">socket</span><span class="o">.</span><span class="n">socket</span><span class="p">()</span>
<span class="n">sock</span><span class="o">.</span><span class="n">connect</span><span class="p">((</span><span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">1234</span><span class="p">))</span>
<span class="n">sock</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="s1">'foo</span><span class="se">\n</span><span class="s1">'</span> <span class="o">*</span> <span class="mi">10</span> <span class="o">*</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">1024</span><span class="p">)</span>
</code></pre></div>
<p>Yes, that does look very like the first example. What has happened here
is that by creating the socket using eventlet.green.socket.socket() we
have put the socket into non-blocking mode and when the write to the
socket blocks, eventlet will schedule any other work that might be
pending. Hitting Ctrl-C while this<br>
is running is actually pretty instructive:</p>
<div class="highlight"><pre><span></span><code>$<span class="o">></span><span class="w"> </span><span class="nv">python</span><span class="w"> </span><span class="nv">test</span><span class="o">-</span><span class="nv">eventlet</span><span class="o">-</span><span class="nv">write</span>.<span class="nv">py</span><span class="w"> </span>
<span class="o">^</span><span class="nv">CTraceback</span><span class="w"> </span><span class="ss">(</span><span class="nv">most</span><span class="w"> </span><span class="nv">recent</span><span class="w"> </span><span class="k">call</span><span class="w"> </span><span class="nl">last</span><span class="ss">)</span>:
<span class="w"> </span><span class="nv">File</span><span class="w"> </span><span class="s2">"test-eventlet-write.py"</span>,<span class="w"> </span><span class="nv">line</span><span class="w"> </span><span class="mi">6</span>,<span class="w"> </span><span class="nv">in</span><span class="w"> </span>
<span class="w"> </span><span class="nv">sock</span>.<span class="k">send</span><span class="ss">(</span><span class="s1">'foo\n'</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">10</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">1024</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">1024</span><span class="ss">)</span>
<span class="w"> </span><span class="nv">File</span><span class="w"> </span><span class="s2">".../eventlet/greenio.py"</span>,<span class="w"> </span><span class="nv">line</span><span class="w"> </span><span class="mi">289</span>,<span class="w"> </span><span class="nv">in</span><span class="w"> </span><span class="k">send</span>
<span class="w"> </span><span class="nv">timeout_exc</span><span class="o">=</span><span class="nv">socket</span>.<span class="nb">timeout</span><span class="ss">(</span><span class="s2">"timed out"</span><span class="ss">))</span>
<span class="w"> </span><span class="nv">File</span><span class="w"> </span><span class="s2">".../eventlet/hubs/__init__.py"</span>,<span class="w"> </span><span class="nv">line</span><span class="w"> </span><span class="mi">121</span>,<span class="w"> </span><span class="nv">in</span><span class="w"> </span><span class="nv">trampoline</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nv">hub</span>.<span class="nv">switch</span><span class="ss">()</span>
<span class="w"> </span><span class="nv">File</span><span class="w"> </span><span class="s2">".../eventlet/hubs/hub.py"</span>,<span class="w"> </span><span class="nv">line</span><span class="w"> </span><span class="mi">187</span>,<span class="w"> </span><span class="nv">in</span><span class="w"> </span><span class="nv">switch</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nv">self</span>.<span class="nv">greenlet</span>.<span class="nv">switch</span><span class="ss">()</span>
<span class="w"> </span><span class="nv">File</span><span class="w"> </span><span class="s2">".../eventlet/hubs/hub.py"</span>,<span class="w"> </span><span class="nv">line</span><span class="w"> </span><span class="mi">236</span>,<span class="w"> </span><span class="nv">in</span><span class="w"> </span><span class="nv">run</span>
<span class="w"> </span><span class="nv">self</span>.<span class="k">wait</span><span class="ss">(</span><span class="nv">sleep_time</span><span class="ss">)</span>
<span class="w"> </span><span class="nv">File</span><span class="w"> </span><span class="s2">".../eventlet/hubs/poll.py"</span>,<span class="w"> </span><span class="nv">line</span><span class="w"> </span><span class="mi">84</span>,<span class="w"> </span><span class="nv">in</span><span class="w"> </span><span class="k">wait</span>
<span class="w"> </span><span class="nv">presult</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">self</span>.<span class="nv">do_poll</span><span class="ss">(</span><span class="nv">seconds</span><span class="ss">)</span>
<span class="w"> </span><span class="nv">File</span><span class="w"> </span><span class="s2">".../eventlet/hubs/epolls.py"</span>,<span class="w"> </span><span class="nv">line</span><span class="w"> </span><span class="mi">61</span>,<span class="w"> </span><span class="nv">in</span><span class="w"> </span><span class="nv">do_poll</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nv">self</span>.<span class="nv">poll</span>.<span class="nv">poll</span><span class="ss">(</span><span class="nv">seconds</span><span class="ss">)</span>
<span class="nv">KeyboardInterrupt</span>
</code></pre></div>
<p>Yes, indeed, there's a whole lot going on behind that innocuous looking
send() call. You see mention of a 'hub' which is eventlet's name for an
event loop. You also see this trampoline() call which means "put the
current code to sleep until the socket is writable". And, there at the
very end, we're still sleeping in a call to poll() which is basically
the same thing as select().</p>
<p>To show the example of doing some "useful" work rather than sleeping all
the time we run a busy loop greenthread:</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">eventlet</span>
<span class="kn">from</span> <span class="nn">eventlet.green</span> <span class="kn">import</span> <span class="n">socket</span>
<span class="k">def</span> <span class="nf">busy_loop</span><span class="p">():</span>
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
<span class="n">i</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">while</span> <span class="n">i</span> <span class="o"><</span> <span class="mi">5000000</span><span class="p">:</span>
<span class="n">i</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="nb">print</span> <span class="s2">"yielding"</span>
<span class="n">eventlet</span><span class="o">.</span><span class="n">sleep</span><span class="p">()</span>
<span class="n">eventlet</span><span class="o">.</span><span class="n">spawn</span><span class="p">(</span><span class="n">busy_loop</span><span class="p">)</span>
<span class="n">sock</span> <span class="o">=</span> <span class="n">socket</span><span class="o">.</span><span class="n">socket</span><span class="p">()</span>
<span class="n">sock</span><span class="o">.</span><span class="n">connect</span><span class="p">((</span><span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">1234</span><span class="p">))</span>
<span class="n">sock</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="s1">'foo</span><span class="se">\n</span><span class="s1">'</span> <span class="o">*</span> <span class="mi">10</span> <span class="o">*</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">1024</span><span class="p">)</span>
</code></pre></div>
<p>Now every time the socket isn't writable, we switch to the busy_loop()
greenthread and do some work. Greenthreads must cooperatively yield to
one another so we call eventlet.sleep() in busy_loop() to once again
poll the socket to see if its writable. Again, if we use the 'time'
command to run this:</p>
<div class="highlight"><pre><span></span><code>$> time python ./test-eventlet-write.py
yielding
yielding
yielding
...
real 0m5.386s
user 0m5.081s
sys 0m0.088s
</code></pre></div>
<p>you can see we're spending very little time sleeping.</p>
<p>(As an aside, I was going to take a look at gevent, but it doesn't seem
fundamentally different from eventlet. Am I wrong?)</p>
<h3>Twisted</h3>
<p>Long, long ago, in times of old, <a href="https://code.launchpad.net/~termie/nova/eventlet_merge/+merge/43383">Nova switched from twisted to
eventlet</a>
so it makes sense to take a quick look at twisted:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">twisted.internet</span> <span class="kn">import</span> <span class="n">protocol</span>
<span class="kn">from</span> <span class="nn">twisted.internet</span> <span class="kn">import</span> <span class="n">reactor</span>
<span class="k">class</span> <span class="nc">Test</span><span class="p">(</span><span class="n">protocol</span><span class="o">.</span><span class="n">Protocol</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">connectionMade</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">transport</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s1">'foo</span><span class="se">\n</span><span class="s1">'</span> <span class="o">*</span> <span class="mi">2</span> <span class="o">*</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">1024</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">TestClientFactory</span><span class="p">(</span><span class="n">protocol</span><span class="o">.</span><span class="n">ClientFactory</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">buildProtocol</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">addr</span><span class="p">):</span>
<span class="k">return</span> <span class="n">Test</span><span class="p">()</span>
<span class="n">reactor</span><span class="o">.</span><span class="n">connectTCP</span><span class="p">(</span><span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">1234</span><span class="p">,</span> <span class="n">TestClientFactory</span><span class="p">())</span>
<span class="n">reactor</span><span class="o">.</span><span class="n">run</span><span class="p">()</span>
</code></pre></div>
<p>What complicates the example most is twisted protocol abstraction which
we need to use simply to write to the socket. The 'reactor' abstraction
is simply twisted's name for an event loop. So, we create a on-blocking
socket, block in the event loop (using e.g. select()) until the
connection completes and then<br>
write to the socket. The transport.write() call will actually queue a
writer in the reactor, return immediately and whenever the socket is
writable, the writer will continue its work.</p>
<p>To show how you can run something in parallel, here's how to run some
code in a deferred callback:</p>
<div class="highlight"><pre><span></span><code><span class="nv">def</span><span class="w"> </span><span class="nv">busy_loop</span><span class="ss">()</span>:
<span class="w"> </span><span class="nv">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span>
<span class="w"> </span><span class="k">while</span><span class="w"> </span><span class="nv">i</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="mi">5000000</span>:
<span class="w"> </span><span class="nv">i</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="mi">1</span>
<span class="w"> </span><span class="nv">reactor</span>.<span class="nv">callLater</span><span class="ss">(</span><span class="mi">0</span>,<span class="w"> </span><span class="nv">busy_loop</span><span class="ss">)</span>
<span class="nv">reactor</span>.<span class="nv">connectTCP</span><span class="ss">(</span>...<span class="ss">)</span>
<span class="nv">reactor</span>.<span class="nv">callLater</span><span class="ss">(</span><span class="mi">0</span>,<span class="w"> </span><span class="nv">busy_loop</span><span class="ss">)</span>
<span class="nv">reactor</span>.<span class="nv">run</span><span class="ss">()</span>
</code></pre></div>
<p>I'm using a timeout of zero here and it shows up a weakness in both
twisted and eventlet - we want this busy_loop() code to only run when
the socket isn't writeable. In other words, we want the task to have a
lower priority than the writer task. In both twisted and eventlet, the
timed tasks are run before the<br>
I/O tasks and there is no way to add a task which is only run if there
are no runnable I/O tasks.</p>
<h3>GLib</h3>
<p>My introduction to async I/O was back when I was working on GNOME
(beginning with GNOME's CORBA ORB, called ORBit) so I can't help
comparing the above abstractions to GLib's main loop. Here's some
equivalent code:</p>
<div class="highlight"><pre><span></span><code><span class="cm">/* build with gcc -g -O0 -Wall $(pkg-config --libs --cflags glib-2.0) test-glib-write.c -o test-glib-write */</span>
<span class="n">#include</span><span class="w"> </span><span class="o"><</span><span class="n">errno</span><span class="p">.</span><span class="n">h</span><span class="o">></span>
<span class="n">#include</span><span class="w"> </span><span class="o"><</span><span class="n">fcntl</span><span class="p">.</span><span class="n">h</span><span class="o">></span>
<span class="n">#include</span><span class="w"> </span><span class="o"><</span><span class="n">stdio</span><span class="p">.</span><span class="n">h</span><span class="o">></span>
<span class="n">#include</span><span class="w"> </span><span class="o"><</span><span class="n">string</span><span class="p">.</span><span class="n">h</span><span class="o">></span>
<span class="n">#include</span><span class="w"> </span><span class="o"><</span><span class="n">unistd</span><span class="p">.</span><span class="n">h</span><span class="o">></span>
<span class="n">#include</span><span class="w"> </span><span class="o"><</span><span class="n">sys</span><span class="o">/</span><span class="n">types</span><span class="p">.</span><span class="n">h</span><span class="o">></span>
<span class="n">#include</span><span class="w"> </span><span class="o"><</span><span class="n">sys</span><span class="o">/</span><span class="n">socket</span><span class="p">.</span><span class="n">h</span><span class="o">></span>
<span class="n">#include</span><span class="w"> </span><span class="o"><</span><span class="n">netinet</span><span class="o">/</span><span class="ow">in</span><span class="p">.</span><span class="n">h</span><span class="o">></span>
<span class="n">#include</span><span class="w"> </span><span class="o"><</span><span class="n">glib</span><span class="p">.</span><span class="n">h</span><span class="o">></span>
<span class="n">GMainLoop</span><span class="w"> </span><span class="o">*</span><span class="n">main_loop</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">NULL</span><span class="p">;</span>
<span class="k">static</span><span class="w"> </span><span class="n">gchar</span><span class="w"> </span><span class="o">*</span><span class="n">strv</span><span class="o">[</span><span class="n">10 * 1024 * 1024</span><span class="o">]</span><span class="p">;</span>
<span class="k">static</span><span class="w"> </span><span class="n">gchar</span><span class="w"> </span><span class="o">*</span><span class="k">data</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">NULL</span><span class="p">;</span>
<span class="nc">int</span><span class="w"> </span><span class="n">remaining</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">-</span><span class="mi">1</span><span class="p">;</span>
<span class="k">static</span><span class="w"> </span><span class="n">gboolean</span>
<span class="n">socket_writable</span><span class="p">(</span><span class="n">GIOChannel</span><span class="w"> </span><span class="o">*</span><span class="n">source</span><span class="p">,</span>
<span class="w"> </span><span class="n">GIOCondition</span><span class="w"> </span><span class="k">condition</span><span class="p">,</span>
<span class="w"> </span><span class="n">gpointer</span><span class="w"> </span><span class="n">user_data</span><span class="p">)</span>
<span class="err">{</span>
<span class="w"> </span><span class="nc">int</span><span class="w"> </span><span class="n">fd</span><span class="p">,</span><span class="w"> </span><span class="n">sent</span><span class="p">;</span>
<span class="w"> </span><span class="n">fd</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">g_io_channel_unix_get_fd</span><span class="p">(</span><span class="n">source</span><span class="p">);</span>
<span class="w"> </span><span class="n">do</span>
<span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">sent</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">write</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span><span class="w"> </span><span class="k">data</span><span class="p">,</span><span class="w"> </span><span class="n">remaining</span><span class="p">);</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">sent</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
<span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">errno</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="n">EAGAIN</span><span class="p">)</span>
<span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">fprintf</span><span class="p">(</span><span class="n">stderr</span><span class="p">,</span><span class="w"> </span><span class="ss">"Write error: %s\n"</span><span class="p">,</span><span class="w"> </span><span class="n">strerror</span><span class="p">(</span><span class="n">errno</span><span class="p">));</span>
<span class="w"> </span><span class="k">goto</span><span class="w"> </span><span class="nl">finished</span><span class="p">;</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">TRUE</span><span class="p">;</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="k">data</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="o">&</span><span class="k">data</span><span class="o">[</span><span class="n">sent</span><span class="o">]</span><span class="p">;</span>
<span class="w"> </span><span class="n">remaining</span><span class="w"> </span><span class="o">-=</span><span class="w"> </span><span class="n">sent</span><span class="p">;</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="k">while</span><span class="w"> </span><span class="p">(</span><span class="n">sent</span><span class="w"> </span><span class="o">></span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span><span class="n">remaining</span><span class="w"> </span><span class="o">></span><span class="w"> </span><span class="mi">0</span><span class="p">);</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="n">remaining</span><span class="w"> </span><span class="o"><=</span><span class="w"> </span><span class="mi">0</span><span class="p">)</span>
<span class="w"> </span><span class="k">goto</span><span class="w"> </span><span class="nl">finished</span><span class="p">;</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">TRUE</span><span class="p">;</span>
<span class="w"> </span><span class="nl">finished</span><span class="p">:</span>
<span class="w"> </span><span class="n">g_main_loop_quit</span><span class="p">(</span><span class="n">main_loop</span><span class="p">);</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">FALSE</span><span class="p">;</span>
<span class="err">}</span>
<span class="k">static</span><span class="w"> </span><span class="n">gboolean</span>
<span class="n">busy_loop</span><span class="p">(</span><span class="n">gpointer</span><span class="w"> </span><span class="k">data</span><span class="p">)</span>
<span class="err">{</span>
<span class="w"> </span><span class="nc">int</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span>
<span class="w"> </span><span class="k">while</span><span class="w"> </span><span class="p">(</span><span class="n">i</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="mi">5000000</span><span class="p">)</span>
<span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="mi">1</span><span class="p">;</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="k">TRUE</span><span class="p">;</span>
<span class="err">}</span>
<span class="nc">int</span>
<span class="n">main</span><span class="p">(</span><span class="nc">int</span><span class="w"> </span><span class="n">argc</span><span class="p">,</span><span class="w"> </span><span class="nc">char</span><span class="w"> </span><span class="o">**</span><span class="n">argv</span><span class="p">)</span>
<span class="err">{</span>
<span class="w"> </span><span class="n">GIOChannel</span><span class="w"> </span><span class="o">*</span><span class="n">io_channel</span><span class="p">;</span>
<span class="w"> </span><span class="n">guint</span><span class="w"> </span><span class="n">io_watch</span><span class="p">;</span>
<span class="w"> </span><span class="nc">int</span><span class="w"> </span><span class="n">fd</span><span class="p">;</span>
<span class="w"> </span><span class="n">struct</span><span class="w"> </span><span class="n">sockaddr_in</span><span class="w"> </span><span class="n">addr</span><span class="p">;</span>
<span class="w"> </span><span class="nc">int</span><span class="w"> </span><span class="n">i</span><span class="p">;</span>
<span class="w"> </span><span class="n">gchar</span><span class="w"> </span><span class="o">*</span><span class="n">to_free</span><span class="p">;</span>
<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="n">G_N_ELEMENTS</span><span class="p">(</span><span class="n">strv</span><span class="p">)</span><span class="o">-</span><span class="mi">1</span><span class="p">;</span><span class="w"> </span><span class="n">i</span><span class="o">++</span><span class="p">)</span>
<span class="w"> </span><span class="n">strv</span><span class="o">[</span><span class="n">i</span><span class="o">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"foo\n"</span><span class="p">;</span>
<span class="w"> </span><span class="n">strv</span><span class="o">[</span><span class="n">G_N_ELEMENTS(strv)-1</span><span class="o">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">NULL</span><span class="p">;</span>
<span class="w"> </span><span class="k">data</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">to_free</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">g_strjoinv</span><span class="p">(</span><span class="k">NULL</span><span class="p">,</span><span class="w"> </span><span class="n">strv</span><span class="p">);</span>
<span class="w"> </span><span class="n">remaining</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">strlen</span><span class="p">(</span><span class="k">data</span><span class="p">);</span>
<span class="w"> </span><span class="n">fd</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">socket</span><span class="p">(</span><span class="n">AF_INET</span><span class="p">,</span><span class="w"> </span><span class="n">SOCK_STREAM</span><span class="p">,</span><span class="w"> </span><span class="mi">0</span><span class="p">);</span>
<span class="w"> </span><span class="n">memset</span><span class="p">(</span><span class="o">&</span><span class="n">addr</span><span class="p">,</span><span class="w"> </span><span class="mi">0</span><span class="p">,</span><span class="w"> </span><span class="n">sizeof</span><span class="p">(</span><span class="n">struct</span><span class="w"> </span><span class="n">sockaddr_in</span><span class="p">));</span>
<span class="w"> </span><span class="n">addr</span><span class="p">.</span><span class="n">sin_family</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">AF_INET</span><span class="p">;</span>
<span class="w"> </span><span class="n">addr</span><span class="p">.</span><span class="n">sin_port</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">htons</span><span class="p">(</span><span class="mi">1234</span><span class="p">);</span>
<span class="w"> </span><span class="n">addr</span><span class="p">.</span><span class="n">sin_addr</span><span class="p">.</span><span class="n">s_addr</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">htonl</span><span class="p">(</span><span class="n">INADDR_LOOPBACK</span><span class="p">);</span>
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="k">connect</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span><span class="w"> </span><span class="p">(</span><span class="n">struct</span><span class="w"> </span><span class="n">sockaddr</span><span class="w"> </span><span class="o">*</span><span class="p">)</span><span class="o">&</span><span class="n">addr</span><span class="p">,</span><span class="w"> </span><span class="n">sizeof</span><span class="p">(</span><span class="n">addr</span><span class="p">))</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
<span class="w"> </span><span class="err">{</span>
<span class="w"> </span><span class="n">fprintf</span><span class="p">(</span><span class="n">stderr</span><span class="p">,</span><span class="w"> </span><span class="ss">"Error connecting to server: %s\n"</span><span class="p">,</span><span class="w"> </span><span class="n">strerror</span><span class="p">(</span><span class="n">errno</span><span class="p">));</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="mi">1</span><span class="p">;</span>
<span class="w"> </span><span class="err">}</span>
<span class="w"> </span><span class="n">fcntl</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span><span class="w"> </span><span class="n">F_SETFL</span><span class="p">,</span><span class="w"> </span><span class="n">O_NONBLOCK</span><span class="p">);</span>
<span class="w"> </span><span class="n">io_channel</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">g_io_channel_unix_new</span><span class="p">(</span><span class="n">fd</span><span class="p">);</span>
<span class="w"> </span><span class="n">io_watch</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">g_io_add_watch</span><span class="p">(</span><span class="n">io_channel</span><span class="p">,</span>
<span class="w"> </span><span class="n">G_IO_OUT</span><span class="p">,</span>
<span class="w"> </span><span class="p">(</span><span class="n">GIOFunc</span><span class="p">)</span><span class="n">socket_writable</span><span class="p">,</span>
<span class="w"> </span><span class="n">GINT_TO_POINTER</span><span class="p">(</span><span class="n">fd</span><span class="p">));</span>
<span class="w"> </span><span class="n">g_idle_add</span><span class="p">(</span><span class="n">busy_loop</span><span class="p">,</span><span class="w"> </span><span class="k">NULL</span><span class="p">);</span>
<span class="w"> </span><span class="n">main_loop</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">g_main_loop_new</span><span class="p">(</span><span class="k">NULL</span><span class="p">,</span><span class="w"> </span><span class="k">FALSE</span><span class="p">);</span>
<span class="w"> </span><span class="n">g_main_loop_run</span><span class="p">(</span><span class="n">main_loop</span><span class="p">);</span>
<span class="w"> </span><span class="n">g_main_loop_unref</span><span class="p">(</span><span class="n">main_loop</span><span class="p">);</span>
<span class="w"> </span><span class="n">g_source_remove</span><span class="p">(</span><span class="n">io_watch</span><span class="p">);</span>
<span class="w"> </span><span class="n">g_io_channel_unref</span><span class="p">(</span><span class="n">io_channel</span><span class="p">);</span>
<span class="w"> </span><span class="k">close</span><span class="p">(</span><span class="n">fd</span><span class="p">);</span>
<span class="w"> </span><span class="n">g_free</span><span class="p">(</span><span class="n">to_free</span><span class="p">);</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span>
<span class="err">}</span>
</code></pre></div>
<p>Here I create a non-blocking socket, set up an 'I/O watch' to tell me
when the socket is writable and, when it is, I keep blasting data into
the socket until I get an EAGAIN. This is the point at which write()
would block if it was a blocking socket and I return TRUE from the
callback to say "call me again when the socket is writable". Only when
I've finished writing all of the data do I return FALSE and quit the
main loop causing the g_main_loop_run() call to return.</p>
<p>The point about task priorities is illustrated nicely here. GLib does
have the concept of priorities and has a "idle callback" facility you
can use to run some code when no higher priority task is waiting to run.
In this case, the busy_loop() function will *only* run when the
socket is not writable.</p>
<h3>Tulip</h3>
<p>There's a lot of talk lately about <a href="http://www.python.org/dev/peps/pep-3156/">Guido's Asynchronous IO Support
Rebooted (PEP3156)</a> efforts
so, of course, we've got to have a look at that.</p>
<p>One interesting aspect of this effort is that it aims to support both
the coroutine and callbacks style programming models. We'll try out both
models below.</p>
<p>Tulip, of course, has an event loop, time-based callbacks, I/O callbacks
and I/O helper functions. We can build a simple variant of our
non-blocking I/O example above using tulip's event loop and I/O
callback:</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">errno</span>
<span class="kn">import</span> <span class="nn">select</span>
<span class="kn">import</span> <span class="nn">socket</span>
<span class="kn">import</span> <span class="nn">tulip</span>
<span class="n">sock</span> <span class="o">=</span> <span class="n">socket</span><span class="o">.</span><span class="n">socket</span><span class="p">()</span>
<span class="n">sock</span><span class="o">.</span><span class="n">connect</span><span class="p">((</span><span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">1234</span><span class="p">))</span>
<span class="n">sock</span><span class="o">.</span><span class="n">setblocking</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="n">buf</span> <span class="o">=</span> <span class="nb">memoryview</span><span class="p">(</span><span class="nb">str</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">'foo</span><span class="se">\n</span><span class="s1">'</span> <span class="o">*</span> <span class="mi">2</span> <span class="o">*</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">1024</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">do_write</span><span class="p">():</span>
<span class="k">global</span> <span class="n">buf</span>
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">buf</span> <span class="o">=</span> <span class="n">buf</span><span class="p">[</span><span class="n">sock</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="n">buf</span><span class="p">):]</span>
<span class="k">except</span> <span class="n">socket</span><span class="o">.</span><span class="n">error</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
<span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">errno</span> <span class="o">!=</span> <span class="n">errno</span><span class="o">.</span><span class="n">EAGAIN</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">e</span>
<span class="k">return</span>
<span class="k">def</span> <span class="nf">busy_loop</span><span class="p">():</span>
<span class="n">i</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">while</span> <span class="n">i</span> <span class="o"><</span> <span class="mi">5000000</span><span class="p">:</span>
<span class="n">i</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="n">event_loop</span><span class="o">.</span><span class="n">call_soon</span><span class="p">(</span><span class="n">busy_loop</span><span class="p">)</span>
<span class="n">event_loop</span> <span class="o">=</span> <span class="n">tulip</span><span class="o">.</span><span class="n">get_event_loop</span><span class="p">()</span>
<span class="n">event_loop</span><span class="o">.</span><span class="n">add_writer</span><span class="p">(</span><span class="n">sock</span><span class="p">,</span> <span class="n">do_write</span><span class="p">)</span>
<span class="n">event_loop</span><span class="o">.</span><span class="n">call_soon</span><span class="p">(</span><span class="n">busy_loop</span><span class="p">)</span>
<span class="n">event_loop</span><span class="o">.</span><span class="n">run_forever</span><span class="p">()</span>
</code></pre></div>
<p>We can go a step further and use tulip's Protocol abstraction and
connection helper:</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">errno</span>
<span class="kn">import</span> <span class="nn">select</span>
<span class="kn">import</span> <span class="nn">socket</span>
<span class="kn">import</span> <span class="nn">tulip</span>
<span class="k">class</span> <span class="nc">Protocol</span><span class="p">(</span><span class="n">tulip</span><span class="o">.</span><span class="n">Protocol</span><span class="p">):</span>
<span class="n">buf</span> <span class="o">=</span> <span class="sa">b</span><span class="s1">'foo</span><span class="se">\n</span><span class="s1">'</span> <span class="o">*</span> <span class="mi">10</span> <span class="o">*</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">1024</span>
<span class="k">def</span> <span class="nf">connection_made</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">transport</span><span class="p">):</span>
<span class="n">event_loop</span><span class="o">.</span><span class="n">call_soon</span><span class="p">(</span><span class="n">busy_loop</span><span class="p">)</span>
<span class="n">transport</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">buf</span><span class="p">)</span>
<span class="n">transport</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">connection_lost</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">exc</span><span class="p">):</span>
<span class="n">event_loop</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">busy_loop</span><span class="p">():</span>
<span class="n">i</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">while</span> <span class="n">i</span> <span class="o"><</span> <span class="mi">5000000</span><span class="p">:</span>
<span class="n">i</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="n">event_loop</span><span class="o">.</span><span class="n">call_soon</span><span class="p">(</span><span class="n">busy_loop</span><span class="p">)</span>
<span class="n">event_loop</span> <span class="o">=</span> <span class="n">tulip</span><span class="o">.</span><span class="n">get_event_loop</span><span class="p">()</span>
<span class="n">tulip</span><span class="o">.</span><span class="n">Task</span><span class="p">(</span><span class="n">event_loop</span><span class="o">.</span><span class="n">create_connection</span><span class="p">(</span><span class="n">Protocol</span><span class="p">,</span> <span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">1234</span><span class="p">))</span>
<span class="n">event_loop</span><span class="o">.</span><span class="n">run_forever</span><span class="p">()</span>
</code></pre></div>
<p>This is pretty similar to the twisted example and shows up yet another
example of the lack of task prioritization being an issue. If we added
the busy loop to the event loop before the connection completed, the
scheduler would run the busy loop every time the connection task yields.</p>
<h3>Coroutines, Generators and Subgenerators</h3>
<p>Under the hood, tulip depends heavily on generators to implement
coroutines. It's worth digging into that concept a bit to understand
what's going on.</p>
<p>Firstly, remind yourself how a generator works:</p>
<div class="highlight"><pre><span></span><code><span class="nv">def</span><span class="w"> </span><span class="nv">gen</span><span class="ss">()</span>:
<span class="w"> </span><span class="nv">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">0</span>
<span class="w"> </span><span class="k">while</span><span class="w"> </span><span class="nv">i</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="mi">2</span>:
<span class="w"> </span><span class="nv">print</span><span class="ss">(</span><span class="nv">i</span><span class="ss">)</span>
<span class="w"> </span><span class="nv">yield</span>
<span class="w"> </span><span class="nv">i</span><span class="w"> </span><span class="o">+=</span><span class="w"> </span><span class="mi">1</span>
<span class="nv">i</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">gen</span><span class="ss">()</span>
<span class="nv">print</span><span class="ss">(</span><span class="s2">"yo!"</span><span class="ss">)</span>
<span class="k">next</span><span class="ss">(</span><span class="nv">i</span><span class="ss">)</span>
<span class="nv">print</span><span class="ss">(</span><span class="s2">"hello!"</span><span class="ss">)</span>
<span class="k">next</span><span class="ss">(</span><span class="nv">i</span><span class="ss">)</span>
<span class="nv">print</span><span class="ss">(</span><span class="s2">"bye!"</span><span class="ss">)</span>
<span class="nv">try</span>:
<span class="w"> </span><span class="k">next</span><span class="ss">(</span><span class="nv">i</span><span class="ss">)</span>
<span class="nv">except</span><span class="w"> </span><span class="nv">StopIteration</span>:
<span class="w"> </span><span class="nv">print</span><span class="ss">(</span><span class="s2">"stopped"</span><span class="ss">)</span>
</code></pre></div>
<p>This will print:</p>
<div class="highlight"><pre><span></span><code>yo!
0
hello!
1
bye!
stopped
</code></pre></div>
<p>Now imagine a generator function which writes to a non-blocking socket
and calls yield every time the write would block. You have the
beginnings of coroutine based async I/O. To flesh out the idea, here's
our familiar example with some generator based infrastructure around it:</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">collections</span>
<span class="kn">import</span> <span class="nn">errno</span>
<span class="kn">import</span> <span class="nn">select</span>
<span class="kn">import</span> <span class="nn">socket</span>
<span class="n">sock</span> <span class="o">=</span> <span class="n">socket</span><span class="o">.</span><span class="n">socket</span><span class="p">()</span>
<span class="n">sock</span><span class="o">.</span><span class="n">connect</span><span class="p">((</span><span class="s1">'localhost'</span><span class="p">,</span> <span class="mi">1234</span><span class="p">))</span>
<span class="n">sock</span><span class="o">.</span><span class="n">setblocking</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">busy_loop</span><span class="p">():</span>
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
<span class="n">i</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">while</span> <span class="n">i</span> <span class="o"><</span> <span class="mi">5000000</span><span class="p">:</span>
<span class="n">i</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">yield</span>
<span class="k">def</span> <span class="nf">write</span><span class="p">():</span>
<span class="n">buf</span> <span class="o">=</span> <span class="nb">memoryview</span><span class="p">(</span><span class="sa">b</span><span class="s1">'foo</span><span class="se">\n</span><span class="s1">'</span> <span class="o">*</span> <span class="mi">2</span> <span class="o">*</span> <span class="mi">1024</span> <span class="o">*</span> <span class="mi">1024</span><span class="p">)</span>
<span class="k">while</span> <span class="nb">len</span><span class="p">(</span><span class="n">buf</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">buf</span> <span class="o">=</span> <span class="n">buf</span><span class="p">[</span><span class="n">sock</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="n">buf</span><span class="p">):]</span>
<span class="k">except</span> <span class="n">socket</span><span class="o">.</span><span class="n">error</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
<span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">errno</span> <span class="o">!=</span> <span class="n">errno</span><span class="o">.</span><span class="n">EAGAIN</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">e</span>
<span class="k">yield</span>
<span class="n">quit</span><span class="p">()</span>
<span class="n">Task</span> <span class="o">=</span> <span class="n">collections</span><span class="o">.</span><span class="n">namedtuple</span><span class="p">(</span><span class="s1">'Task'</span><span class="p">,</span> <span class="p">[</span><span class="s1">'generator'</span><span class="p">,</span> <span class="s1">'wfd'</span><span class="p">,</span> <span class="s1">'idle'</span><span class="p">])</span>
<span class="n">tasks</span> <span class="o">=</span> <span class="p">[</span>
<span class="n">Task</span><span class="p">(</span><span class="n">busy_loop</span><span class="p">(),</span> <span class="n">wfd</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">idle</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
<span class="n">Task</span><span class="p">(</span><span class="n">write</span><span class="p">(),</span> <span class="n">wfd</span><span class="o">=</span><span class="n">sock</span><span class="p">,</span> <span class="n">idle</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
<span class="p">]</span>
<span class="n">running</span> <span class="o">=</span> <span class="kc">True</span>
<span class="k">def</span> <span class="nf">quit</span><span class="p">():</span>
<span class="k">global</span> <span class="n">running</span>
<span class="n">running</span> <span class="o">=</span> <span class="kc">False</span>
<span class="k">while</span> <span class="n">running</span><span class="p">:</span>
<span class="n">finished</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">n</span><span class="p">,</span> <span class="n">t</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">tasks</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="nb">next</span><span class="p">(</span><span class="n">t</span><span class="o">.</span><span class="n">generator</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">StopIteration</span><span class="p">:</span>
<span class="n">finished</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">n</span><span class="p">)</span>
<span class="nb">map</span><span class="p">(</span><span class="n">tasks</span><span class="o">.</span><span class="n">pop</span><span class="p">,</span> <span class="n">finished</span><span class="p">)</span>
<span class="n">wfds</span> <span class="o">=</span> <span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">wfd</span> <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">tasks</span> <span class="k">if</span> <span class="n">t</span><span class="o">.</span><span class="n">wfd</span><span class="p">]</span>
<span class="n">timeout</span> <span class="o">=</span> <span class="mi">0</span> <span class="k">if</span> <span class="p">[</span><span class="n">t</span> <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">tasks</span> <span class="k">if</span> <span class="n">t</span><span class="o">.</span><span class="n">idle</span><span class="p">]</span> <span class="k">else</span> <span class="kc">None</span>
<span class="n">select</span><span class="o">.</span><span class="n">select</span><span class="p">([],</span> <span class="n">wfds</span><span class="p">,</span> <span class="p">[],</span> <span class="n">timeout</span><span class="p">)</span>
</code></pre></div>
<p>You can see how the generator-based write() and busy_loop() coroutines
are cooperatively yielding to one another just like greenthreads in
eventlet would do. But, there's a pretty fundamental flaw here - if we
wanted to refactor the code above to re-use that write() method to e.g.
call it multiple times with<br>
different input, we'd need to do something like:</p>
<div class="highlight"><pre><span></span><code><span class="nv">def</span><span class="w"> </span><span class="nv">write_stuff</span><span class="ss">()</span>:
<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="nv">i</span><span class="w"> </span><span class="nv">in</span><span class="w"> </span><span class="nv">write</span><span class="ss">(</span><span class="nv">b</span><span class="s1">'foo'</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">10</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">1024</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">1024</span><span class="ss">)</span>:
<span class="w"> </span><span class="nv">yield</span>
<span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="nv">i</span><span class="w"> </span><span class="nv">in</span><span class="w"> </span><span class="nv">write</span><span class="ss">(</span><span class="nv">b</span><span class="s1">'bar'</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">10</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">1024</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">1024</span><span class="ss">)</span>:
<span class="w"> </span><span class="nv">yield</span>
</code></pre></div>
<p>but that's pretty darn nasty! Well, that's the whole idea behind <a href="http://www.python.org/dev/peps/pep-0380/">Syntax
for Delegating to a Subgenerator
(PEP380)</a>. Since python 3.3, a
generator can now yield to another generator using the 'yield from'
syntax. This allows us to do:</p>
<div class="highlight"><pre><span></span><code>...
<span class="nv">def</span><span class="w"> </span><span class="nv">write</span><span class="ss">(</span><span class="nv">data</span><span class="ss">)</span>:
<span class="w"> </span><span class="nv">buf</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">memoryview</span><span class="ss">(</span><span class="nv">data</span><span class="ss">)</span>
<span class="w"> </span><span class="k">while</span><span class="w"> </span><span class="nv">len</span><span class="ss">(</span><span class="nv">buf</span><span class="ss">)</span>:
<span class="w"> </span><span class="nv">try</span>:
<span class="w"> </span><span class="nv">buf</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">buf</span>[<span class="nv">sock</span>.<span class="k">send</span><span class="ss">(</span><span class="nv">buf</span><span class="ss">)</span>:]
<span class="w"> </span><span class="nv">except</span><span class="w"> </span><span class="nv">socket</span>.<span class="nv">error</span><span class="w"> </span><span class="nv">as</span><span class="w"> </span><span class="nv">e</span>:
<span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="nv">e</span>.<span class="nv">errno</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="nv">errno</span>.<span class="nv">EAGAIN</span>:
<span class="w"> </span><span class="nv">raise</span><span class="w"> </span><span class="nv">e</span>
<span class="w"> </span><span class="nv">yield</span>
<span class="nv">def</span><span class="w"> </span><span class="nv">write_stuff</span><span class="ss">()</span>:
<span class="w"> </span><span class="nv">yield</span><span class="w"> </span><span class="nv">from</span><span class="w"> </span><span class="nv">write</span><span class="ss">(</span><span class="nv">b</span><span class="s1">'foo\n'</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">1024</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">1024</span><span class="ss">)</span>
<span class="w"> </span><span class="nv">yield</span><span class="w"> </span><span class="nv">from</span><span class="w"> </span><span class="nv">write</span><span class="ss">(</span><span class="nv">b</span><span class="s1">'bar\n'</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">1024</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="mi">1024</span><span class="ss">)</span>
<span class="w"> </span><span class="nv">quit</span><span class="ss">()</span>
<span class="nv">Task</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">collections</span>.<span class="nv">namedtuple</span><span class="ss">(</span><span class="s1">'Task'</span>,<span class="w"> </span>[<span class="s1">'generator'</span>,<span class="w"> </span><span class="s1">'wfd'</span>,<span class="w"> </span><span class="s1">'idle'</span>]<span class="ss">)</span>
<span class="nv">tasks</span><span class="w"> </span><span class="o">=</span><span class="w"> </span>[
<span class="w"> </span><span class="nv">Task</span><span class="ss">(</span><span class="nv">busy_loop</span><span class="ss">()</span>,<span class="w"> </span><span class="nv">wfd</span><span class="o">=</span><span class="nv">None</span>,<span class="w"> </span><span class="nv">idle</span><span class="o">=</span><span class="nv">True</span><span class="ss">)</span>,
<span class="w"> </span><span class="nv">Task</span><span class="ss">(</span><span class="nv">write_stuff</span><span class="ss">()</span>,<span class="w"> </span><span class="nv">wfd</span><span class="o">=</span><span class="nv">sock</span>,<span class="w"> </span><span class="nv">idle</span><span class="o">=</span><span class="nv">False</span><span class="ss">)</span>
]
...
</code></pre></div>
<h3>Conclusions?</h3>
<p>Yeah, this is the point where I've figured out what we should do in
OpenStack. Or not.</p>
<p>I really like the explicit nature of Tulip's model - for each async
task, you explicitly decide whether to block the current coroutine on
its completion (or put another way, yield to another coroutine until the
task has completed) or you register a callback to be notified of the
tasks completion. I'd much prefer this to rather cavalier "don't worry
your little head" approach of hiding the async nature of what's going
on.</p>
<p>However, the prospect of porting something like Nova to this model is
more than a little dauting. If you think about the call stack of an REST
API request being handled and ultimately doing an rpc.cast() and that
the entire call stack would need to be ported to 'yield from' in order
for us to yield and handle another API request while waiting for the
result of rpc.cast() .... as I said, daunting.</p>
<p>What I'm most interested in is how to <a href="https://wiki.openstack.org/wiki/Oslo/Messaging">design our new messaging
API</a> to be able to
<a href="http://lists.openstack.org/pipermail/openstack-dev/2013-May/009784.html">support any and all of these models in
future</a>.
I haven't quite figured that out either, but it feels pretty doable.</p>April 14th OpenStack Foundation Board Meeting2013-05-25T14:44:00+01:002013-05-25T14:44:00+01:00markmctag:crustyblaa.com,2013-05-25:/april-14th-openstack-foundation-board-meeting.html<p>On April 14, the day before the OpenStack Summit in Portland began, the
OpenStack Foundation Board held an all-day, in-person meeting. Like all
of our board meetings, the
<a href="https://wiki.openstack.org/wiki/Governance/Foundation/14Apr2013BoardMeeting">agenda</a>
was packed solid.</p>
<p>After our meetings, the goal is for Jonathan to post a summary to the
mailing list (like <a href="http://lists.openstack.org/pipermail/foundation/2013-February/001364.html">this …</a></p><p>On April 14, the day before the OpenStack Summit in Portland began, the
OpenStack Foundation Board held an all-day, in-person meeting. Like all
of our board meetings, the
<a href="https://wiki.openstack.org/wiki/Governance/Foundation/14Apr2013BoardMeeting">agenda</a>
was packed solid.</p>
<p>After our meetings, the goal is for Jonathan to post a summary to the
mailing list (like <a href="http://lists.openstack.org/pipermail/foundation/2013-February/001364.html">this
one</a>)
and I try and follow up with a longer blog post with a bit more
commentary. For a bunch of reasons, we've let things slide this time,
but here's my attempt at a summary anyway. It's been a while since the
meeting, so a lot of the details are pretty vague at this point.</p>
<p>One thing I really liked about the meeting was that we had chairs set up
so that anyone who wished to attend the meeting and listen were welcome
to do so. With so many stackers already in town for the summit, it
really was a great opportunity to open the workings of the board to
anyone interested.</p>
<h3>Transparency Committee</h3>
<p>The first meaty item on the agenda was an update from Joshua McKenty on
the progress of the board's <a href="http://lists.openstack.org/pipermail/foundation/2013-February/001362.html">Transparency
Committee</a>.</p>
<p>We began by discussing Nick Barcet's efforts to choose and implement a
document management system for the board. OwnCloud, Google Drive,
Dropbox and github have all been considered. Google Drive and Dropbox
would not be accessible by all board members. It was noted that the
Foundation staff will also require a document management system and we
might be able to use the same system for both use cases. Conclusion was
for Nick and Monty to see if the OpenStack Infrastructure team could
host OwnCloud for this purpose.</p>
<p>Next we discussed the transparency policy which Lauren Sell and Mark
Radcliffe have been working on. The core concern is how to balance the
need for having a culture of transparency and openness with the need for
some information (like legal or personnel matters) to be kept
confidential. The general approach discussed was having the board's
mailing list open, but confidential information would be shared via
private documents in a document management system that could be referred
to in mailing list posts. The distinction between confidential and
embargoed information was discussed and it was agreed that embargoed
information should have a disclosure date associated with it.</p>
<p>There was also a brief discussion on the possibility of having a
transparency ombudsman. This would be a person trusted by the community
with whom complaints related to transparency could be raised. It was
felt this could be a duty of a foundation employee, if that individual
had already established the reputation for trustworthiness and
independence required.</p>
<h3>2013 Individual Director Elections</h3>
<p>Next up, Rob Hirschfeld led a discussion on possible changes to the
election process in time for the 2013 Individual Director elections.</p>
<p>We began by discussing how we would go about making any required changes
to <a href="https://wiki.openstack.org/wiki/Governance/Foundation/Bylaws">the
bylaws</a>.
It was recognized that we would need a firm proposal to be made to the
board in August so that notice could be given in September of a vote in
October in time for the elections in November.</p>
<p>It was also noted that a bylaws change would require a majority vote of
the individual members with a voter turnout of 25%. For reference, we
had a 28% turnout at the <a href="https://www.bigpulse.com/pollresults?code=2888d7aPUFe5Euveb5kiYNGT">previous
election</a>.</p>
<p>We also briefly discussed <a href="https://etherpad.openstack.org/2013-Individual-Elections">the potential
objectives</a>
for any changes to the elections process but avoided going deep on this
because of the limited time available on the agenda.</p>
<h3>Legal Affairs Committee</h3>
<p>Next on the agenda was a discussion (led by me) about the Legal Affairs
Committee.</p>
<p>The main concern I raise was the project (or the "Technical Community
and Committee", as I called it) needed the support of subject matter
experts when dealing with the kind of legal issues regularly encountered
by all open-source projects, particularly in the area of copyright and
licensing. I proposed Richard Fontana's idea that, rather than depending
exclusively on the Legal Affairs Committee or the Foundation's legal
counsel for help with such matters, we would create a legal-discuss
mailing list where any subject matter experts could contribute to
resolving issues raise by the technical community. The idea was that we
would also have a FAQ wiki page to document any conclusions reache on
the list. There was some discussion about whether opinions expressed on
the list could be construed as legal advice and it was agreed to put in
place disclaimers to avoid that perception.</p>
<p>(For the record, we've since created <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/legal-discuss">the legal-discuss mailing
list</a>
and <a href="https://wiki.openstack.org/wiki/LegalIssuesFAQ">the legal issues
FAQ</a>.)</p>
<p>We also discussed whether the description of scope of the Legal Affairs
Committee in the bylaws could be improved and whether the strict five
member limit in the bylaws was really such a good idea. It was agreed
that I would work with Alice King to see if we can address those issues.</p>
<p>(Again for the record, Alice and I had a productive discussion and we
put together <a href="https://wiki.openstack.org/wiki/Governance/Foundation/LegalAffairsCommittee">the Legal Affairs Committee wiki
page</a>
to at least ensure the committee's scope, membership and progress could
be properly documented.)</p>
<h3>Gold Member Applications</h3>
<p>Next up, we had presentations from
<a href="https://wiki.openstack.org/w/images/4/4a/JuniperGoldApplication.pdf">Juniper</a>
and
<a href="https://wiki.openstack.org/w/images/9/94/Ericsson_application_openstack.pdf">Ericsson</a>
on their applications for Gold Membership. Once the presentations had
been completed, we convened an executive session to discuss the
applications in private.</p>
<p>During the executive session, the board reviewed each application using
the
[https://wiki.openstack.org/wiki/Governance/Foundation/PotentialMemberCriteria
previously agreed criteria].</p>
<p>Once the executive session was completed, the board reconvened the
meeting and carried out the official, public vote on the new member
applications. The applications of both Ericsson and Juniper were
approved.</p>
<p>As part of the voting process, some directors chose to publicly restate
their concerns with the applications that had been discussed during the
executive session. One concern raised by Rob Hirschfeld and Joshua
McKenty was that, while both applicants shared their plans for
increasing their OpenStack engagement, it could be argued that future
plans aren't the same as the "demonstrated commitment" talked about in
the membership criteria.</p>
<p>Another concern discussed at length was that applicants could provide
much more in the way of written detail about how they meet the
membership criteria. One take on this was that applicants should be
expected to provide a resume of their involvement with, and commitment,
to OpenStack. Simon Anderson proposed that we form a membership
committee which would mentor applicants and advise them on how to
prepare their application.</p>
<p>In total, discussion of these applications and the process for new gold
members took up 4 hours of the board's time. While this may seem
excessive, it was very apparent that everyone on the board was keen for
the Foundation to have gold members that would help drive OpenStack's
success. With the addition of Ericsson and Juniper, we now have 16 of
the 24 gold member slots filled. As the number of remaining slots
shrink, I think we will see the board having higher expectations of
applications.</p>
<h3>Programs</h3>
<p>At this point, the board seemed collectively quite drained of energy and
the agenda had to be significantly reworked because there wasn't much
time remaining.</p>
<p>Monty took the opportunity to briefly discuss an idea he had to
introduce the concept of "programs" in addition to our current concept
of projects. We would use this to recognize efforts like documentation
and CI on equal footing with the integrated projects. Other efforts like
Oslo, TripleO and puppet recipes were also mentioned as potential
programs.</p>
<p>Rather than go into too detailed discussion on the topic, we generally
expressed support for the concept but agreed it needed further
discussion.</p>
<h3>Joint Session</h3>
<p>Part of the agenda for the meeting was that members of the Technical
Committee were invited to join the meeting for a joint session. Monty
and I were obviously already present, but Michael Still and Thierry
Carrez also joined. Other TC members had not yet arrived in Portland or
weren't fully aware of the joint session. We had some discussion about
how to have a joint session with better attendance from both groups and
the idea of a joint design summit session at the next summit was mooted.</p>
<h3>User Committee</h3>
<p>Tim Bell stepped up next to give an overview of the results of the User
Committee's recently completed survey of OpenStack users.</p>
<p>Tim is presenting these exciting results at the summit, so I won't
repeat them here. However, we did have an interesting discussion about
how representative the survey was of OpenStack deployments. Tim offered
50-75% as his gut feeling for the percentage of deployments covered but
Joshua McKenty reckoned it was probably less than 25%. There was some
discussion about future surveys and the potential for an opt-in usage
stats tracking tool to increase the percentage of deployments we know
about.</p>
<h3>Incubation Update Committee</h3>
<p>Next up, Alan Clark presented the <a href="https://wiki.openstack.org/wiki/Governance/Foundation/IncubationUpdate2013">final report of the Incubation Update
Committee</a>.</p>
<p>The board was generally very supportive of the work of the committee,
but it was clear further discussion was required on the topic of Core
status and the board's trademark usage program. Much discussion was had
around the desire for projects like Heat and Ceilometer to be able to
adopt official OpenStack project names (like "OpenStack Measurement" for
Ceilometer) and whether this was all that would be implied by Core
status. However, there is also the question of whether clouds which wish
to use the OpenStack brand should be required to use all Core projects
or whether a trademark program around more targeted interoperability
testing made more sense.</p>
<p>It was agreed that this was an incredibly important and urgent topic but
that we had run out of time at this meeting to make progress on it.</p>February 12th OpenStack Foundation Board Meeting2013-02-15T17:06:00+00:002013-02-15T17:06:00+00:00markmctag:crustyblaa.com,2013-02-15:/february-12th-openstack-foundation-board-meeting.html<p>Yesterday was the first in-person meeting of the 2013 Foundation Board.
We met at Rackspace's offices in San Francisco at Jim Curry's
invitation. The meeting also coincided with the Foundation staff all
getting together in-person for the first time. With so many OpenStack
people converging on the one place, it …</p><p>Yesterday was the first in-person meeting of the 2013 Foundation Board.
We met at Rackspace's offices in San Francisco at Jim Curry's
invitation. The meeting also coincided with the Foundation staff all
getting together in-person for the first time. With so many OpenStack
people converging on the one place, it was quite funny to randomly run
into various people at the Courtyard Marriott around the corner.</p>
<p>The meeting kicked off at 10am after some informal introductions. For
me, it was a case of playing a "hey, are you Tristan?"/"No, I'm Sean"
guessing game. Monty brought his usual wackiness to the ocassion by
handing out beads and wearing a luminous, sparkling orange fedora with
bright blue flashing LEDs. It's Mardi Gras time!</p>
<h3>Formalities</h3>
<p>The first item on the agenda was a formal role call. 5 board members
couldn't attend and Tim Bell was on the phone. Also attending were
Jonathan Bryce and Mark from the Foundation staff. Mark Radcliffe, our
outside council, was on the phone.</p>
<p>From that start, it was obvious that it was going to be extremely
difficult for those dialled into the meeting to be able to participate
effectively. This is definitely something we will continue to struggle
with and, as one of the few who had to travel halfway around the world
for the meeting, I can certainly sympathise with those on the phone.</p>
<p>As a further formality, Alan Clark reviewed some of the existing board
policies:</p>
<ul>
<li>that observers are allowed at board meetings, except in the case of
executive sessions</li>
<li>that board members wouldn't blog or tweet until after an official
summary of the meeting had been posted to the foundation list</li>
</ul>
<p>There was some discussion about how executive discussions should be
handled. How we can describe what the topic of any executive sessions
were and perhaps also publicly hold any votes arising from those
sessions.</p>
<p>Rob Hirschfeld proposed that our general decision making process should
involve first coming to an agreement on the criteria for making the
decision and then applying the criteria. The idea was that this process
would avoid some circular debates and save time.</p>
<h3>User Committee</h3>
<p>Next up, Ryan Lane presented an update on the progress of the User
Committee. He described the mandate of the committee as "fighting for
the users". The initial goals of the committee are to define their
charter and a set of categories of users. Ryan encouraged the board to
review and <a href="https://docs.google.com/document/d/1yD8TfqUik2dt5xo_jMVHMl7tw9oIJnEndLK8YqEToyo/edit">comment on the Google Doc they had
prepared</a>.</p>
<p>Much discussion was had about how to gather data about our users.
Options included a CRM system, user polls, anonymous usage reporting
tools, aggregating statistics to protect the innocent and the like. This
is clearly going to be a very hot topic for the committee.</p>
<p>During the discussion, Josh McKenty posted <a href="https://blueprints.launchpad.net/oslo/+spec/opt-in-stats-tracking">a blueprint for how a opt-in
tracking
system</a>
might work.</p>
<p>We also had an interesting debate about the committee should move
forward with adding more members to the committee. The committee's own
proposal for this was to be democratic and have elections for
representatives from different geographies or categories of users. Quite
a few board members raised concerns about this approach and asked the
committee members to press forward quickly and appoint a diverse set of
members with minimal bureaucracy.</p>
<p>Once Ryan had finished, the board expressed their thanks and support for
the efforts of the committee. Ryan was asked to stay and observe the
board meeting, but Ryan preferred to go and get some "real work" done.
That's the spirit!</p>
<p>If you want to follow the progress of the user committee, the best way
is to <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee">subscribe to user-committee@lists.openstack.org mailing
list</a>.
Ryan would be happy to share his slide deck if you email him.</p>
<h3>Legal Affairs Committee</h3>
<p>Alice King and Nissa Strottman took the floor and presented the work of
the Legal Affairs Committee on the whole area of patents and risk
mitigation.</p>
<p>Alice and Nissa first talked the board through some background on
patents. They described the "risk landscape" in terms of competing
companies suing and counter-suing each other for infringement (Alice had
an awesome diagram of <a href="http://www.textually.org/textually/archives/2010/10/06/mobilelawsuits.png">"who's suing who in the mobile
industry"</a>
which got a good laugh from everyone) but also, perhaps more
importantly, the threat of Non Practicing Entities or "patent trolls".</p>
<p>We talked in some detail about the Patent Grant in the Apache License
and how it does a lot to mitigate the risks but doesn't completely
eliminate them. In particular, it does little to help with the threat of
NPEs. An interesting debate followed about various nuances of the Apache
License definitions and how they relate to OpenStack.</p>
<p>Alice and Nissa described approaches taken by other communities - e.g.
OIN, GPLv3, Open Compute and Eclipse - and also how an additional
contributor agreement could be adopted by the project to further help.
Brian Stevens talked about how the OIN works and how it isn't strictly
limited to Linux. It was noted that many of the larger members of the
Foundation are also members of OIN. Eileen Evans described how the
adoption of a new contributor agreement would be a massive bureaucratic
challenge for some of the larger companies involved in the project.</p>
<p>A point on one of the slides that a more robust patent policy would
remove a barrier to OpenStack's adoption triggered some discussion about
whether patent risk is in fact hindering OpenStack's adoption. The
consensus seemed to be that this wasn't seen as a huge issue and how, in
fact, OpenStack is in as good a position as most any other open-source
project. While it's important for the Foundation to do due diligence on
this matter, it's also important that we don't overstate the actual
risk.</p>
<p>Another topic discussed was the question of "defensive publication" of
ideas generated by the developer community so that they are properly
documented in a way that is accessible to lawyers who wish to
demonstrate prior art. Various ideas were suggested about how blueprints
could form the basis for such an approach and how to do this without
placing the burden on the developers.</p>
<p>It's amusing the way we wind ourselves in knots over these things and
generally come to the conclusion that the issues are difficult, but that
things are working pretty well as they currently stand.</p>
<p>Finally, with limited time available, we picked up again on some of the
discussion from the previous board meeting about the scope, name and
makeup of the committee. Some board members seemed fine with how things
currently stand while others reiterated the issues previously discussed.</p>
<p>Anyone interested in the legal affairs committee should talk to Alice
King about how to get involved. She would be happy to share her slides
with those interested.</p>
<h3>Lunch</h3>
<p>At this point, it was time to break for lunch. Most board members stayed
for the lunch provided on-site and had productive, informal discussions
while eating.</p>
<p>Monty, Nick Barcet and I attended a Technical Committee meeting over IRC
while also eating and chatting with the rest of the board. That's
multi-taksing! Amazingly every single member of the TC attended the
meeting and we voted unamimously to <a href="http://lists.openstack.org/pipermail/openstack-tc/2013-February/000119.html">update the Incubation Process as
recommended by the IncUp
committee</a>.
Such a level of TC consensus is completely unprecedented and took a
tonne of work to achieve. Kudos to Thierry for herding the cats on this.</p>
<h3>Financial Report</h3>
<p>Sean Roberts presented a report on the work of the Financial Oversight
committee and we discussed the progress on completing the Foundation's
first financial audit and the updated financial forecast for the year.</p>
<p>The topic was so uncontroversial (you could even say boring) that we
didn't even take much in the way of notes on it. There was unanimous
agreement that this is how it should be and we'd be doing something
wrong if the topic was "interesting". The board praised the committee
and the staff for their dilligence in doing everything to make sure the
Foundation's financial affairs were beyond question.</p>
<h3>Transparency</h3>
<p>Josh McKenty was next up presenting <a href="https://etherpad.openstack.org/TransparencyPolicy">his proposal for a Transparency
Policy</a> for the board
and his wanting volunteers to form a committee to finalize the details.</p>
<p>He described the committee's goal as:</p>
<blockquote>
<p>To improve transparency and foster collaboration between the
foundation members and members of the board, technical committee, user
committee and other committees. Specifically, to draft statements and
prototype systems changes for board review and approval.</p>
</blockquote>
<p>A large part of the discussion was around trying to quantify the problem
we'retrying to solve and understanding how we'd achieve closure on it.
The board doesn't want to be consumed with this indefinitely, so how
will we know when it's simply a case of "you can't please everyone".</p>
<p>We wrapped this up by gathering volunteers for the committee:</p>
<ul>
<li>Nick Barcet</li>
<li>Jonathan Bryce</li>
<li>Alan Clark</li>
<li>Eileen Evans</li>
<li>Tristan Goode</li>
<li>Rob Hirschfeld</li>
<li>Kyle MacDonald</li>
<li>Joshua McKenty</li>
<li>Mark McLoughlin</li>
<li>Lauren Sell</li>
</ul>
<h3>Director's Report</h3>
<p>Next up, Jonathan provided an awesome update on the Foundation's
progress and plans. It's clear an awful lot of time and thought went
into this super-helpful and encouraging update.</p>
<p>One of Jonathan's slides showed some mind-blowing statistics detailing
the growth of the developer community, user community and ecosystem. The
statistics on the number of patches merged every month were really
stunning and the board praised the success of the project's
infrastructure team in enabling this level of activity.</p>
<p>Jonathan talked to his hiring plan and introduced the latest Foundation
employees. We're executing almost to the plan and there are two
positions still to fill - another infrastructure engineer and another
community manager.</p>
<p>There was a brief discussion of the updated budget. The summary is that
slightly more money is both coming in and going out than planned, giving
a positive net effect. A motion was passed to approve the new budget.</p>
<p>The discussion moved more to event planning and marketing matters and
Lauren Sell filled the board in on a lot of the details in this area.</p>
<p>Planning of the Havana summit was well in hand and it was noted that the
choice of venue means that costs will rise more linearly with attendance
than previous summits. Lauren discussed the venue selection process for
the October 2013 summit which should come to a conclusion soon. Future
summits were discussed with general agreement that a two year cycle
should be rougly East Coast US, Asia, West Coast US and Europe. Several
board members expressed a desire to get started on the selection process
for the October 2014 summit since the size of the event means that
suitable locations get booked up very far in advance.</p>
<p>Lauren also <a href="http://wiki.openstack.org/Governance/Foundation/Marketing">described her efforts to bring the various marketers
at</a> dozens of
OpenStack companies together with some of the same principles that drive
the collaborative software development process. She has created a
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/marketing">mailing
list</a>
which already has almost 100 members and holds a regular phone meeting
of the group. The goal is to have all those involved in marketing
OpenStack to be highly co-ordinated and "on message".</p>
<p>Finally, Lauren quickly presented an independent study she had just
received on OpenStack's marketing impact relative to other open-source
projects and industry players. The results of the study were simply
stunning and most of us appeared to be struggling to believe them.
However, even if the statistics on OpenStack are massively inflated,
we're still hugely being successful. Lauren, Mark and Jonathan have
already worked with the authors of the report to find any data that
could undermine the conclusions and will continue to do so.</p>
<p>Jonathan and Lauren can be contacted directly for the materials they
presented to the board.</p>
<h3>Strategy Session</h3>
<p>Jonathan moved on to kick off a strategy session where board members
would have an opportunity to put their thinking hats on and brainstorm
together. He teed up the discussion by presenting his thesis that
OpenStack was building a "platform ecosystem" with huge potential for
network effects that would result in OpenStack being the dominant cloud
platform in the market place.</p>
<p>We then moved onto having a short exercise where we used sticky-notes to
throw out our ideas in three areas - "interoperability", "reference
architectures" and "engaging users". Once the ideas had been gathered,
we split into breakout groups to discuss ideas for each of those areas.</p>
<p>Rather than trying to capture all of the ideas and action items from the
discussions here, it's perhaps easier to call out some highlights:</p>
<ul>
<li>We're going to explore the use of Tempest as an API compatibility
testing tool which can be pointed at an OpenStack instance. The idea
is that anyone who wants to license the OpenStack trademark would
request a test run which would result in a scorecard. At every
release, the board would update the<br>
results required for each OpenStack mark so that obtaining a
trademark license simply becomes a matter of fixing any failing
tests which are required for the desired mark.</li>
<li>We're going to explore the definition of reference architecture
"flavours" and how these reference architectures could be defined
and maintained. The initial idea is that Heat templates could be
used as a deployable reference architecture definitions.</li>
<li>In terms of engaging users, we decided that trystack.org and
deployment tracking tools were crucial.</li>
</ul>
<h3>Election Process Committee</h3>
<p>Todd Moore raised the question of whether allowing Foundation staff to
run for the Individual Member Director elections was a waste of a board
seat since staff members are so involved in the Foundation anyway. The
question of potential conflicts of interest was also raised. There was
no time to debate this question so the board resolved to reconstitute
the Election Process Committee to consider this question and the general
question of the eligibility to run and vote for these board seats.</p>
<p>The committee will consist of Todd as chair and the 8 individual board
members.</p>
<h3>Evening Event</h3>
<p>We wrapped up at 6pm sharp and headed over to a nearby restaurant with
all of the Foundation staff members. Far from simply being a social
event, it was incredible to see 30+ focused and committed folks spend
over 4 hours going from conversation to conversation about how to
continue OpenStack's success into the future. For me personally, those
conversations alone were enough to justify the effort it took to get to
the meeting.</p>First Board Meeting2013-01-31T23:17:00+00:002013-01-31T23:17:00+00:00markmctag:crustyblaa.com,2013-01-31:/first-board-meeting.html<p>Disclaimer: apparently board members aren't supposed to comment on board
meetings until the official minutes have been published (up to 2 weeks
after the meeting). However, Jonathan does publish timely summaries of
the meetings to bridge that gap. This post isn't intended as an official
record of the meeting. Read …</p><p>Disclaimer: apparently board members aren't supposed to comment on board
meetings until the official minutes have been published (up to 2 weeks
after the meeting). However, Jonathan does publish timely summaries of
the meetings to bridge that gap. This post isn't intended as an official
record of the meeting. Read it as if it was a summary by a
non-board-member listening to the public part of the meeting.</p>
<p>I attended my <a href="http://wiki.openstack.org/Governance/Foundation/31Jan2013BoardMeeting">first OpenStack Foundation Board
meeting</a>
today.</p>
<p>When I was observing the board from the outside (yet not listening in on
their calls), I found the meeting minutes a fairly colourless way of
following what was going on. I'm going to attempt to write up some
thoughts after each meeting, but no promises :-)</p>
<p>The first thing that struck me is that we spend a fair amount of time
messing about with the conference call system, taking a roll call,
debating proper procedure and the like. My guess is that over the course
of the 150 minute call, we spent 30 minutes on this stuff. Still, given
that the Foundation is still relatively new and there are 24 board
members attending along with some Foundation staff members, it actually
wasn't terrible.</p>
<p>Another thing is that I'd say roughly only half of the folks on the call
actively contributed to the discussions. I'm curious whether that was
because it's difficult to jump into a discussion on such a big call or
just that all points were being covered satisfactorily by those
contributing. I myself often sit through conference calls without saying
a word for the latter reason. Not today, though.</p>
<p>The first meaty item on the agenda was Jonathan giving a nice 30 minute
summary of how the Foundation has progressed since it was formed. At one
point, Jonathan had too keep going while some hold music played in the
background. It was all really positive stuff, nicely putting our
achievements into context. I hope the slide deck will be forwarded to
the <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation">foundation mailing
list</a>.</p>
<p>Next up was Alice King with a proposed charter for the Legal Affairs
Committee (which doesn't have a page on our <a href="http://wiki.openstack.org/Governance/Foundation">governance
page</a> - we should fix
that). The committee was formed under the bylaws:</p>
<blockquote>
<p>4.15 Legal Affairs Committee. The Legal Affairs Committee shall be an
advisory committee to the Board of Directors and shall be comprised of
no more than five (5) members. The Board of Directors shall appoint
the members of Legal Affairs Committee. The Legal Affairs Committee
shall advise the Board of Directors on the management of: (i)
compliance with and enforcement of the Trademark Policy, (ii)
strategies to promote the efficient intellectual property protection
of the OpenStack Project, including without limitation, the resolution
of patent and other intellectual property issues and disputes related
to the Members’ use of the OpenStack Project, and (iii) all programs
that the Board of Directors is considering relating to intellectual
property management and protection.</p>
</blockquote>
<p>but this proposed charter attempted to elaborate that the role of the
committee would be as policy advisor in the area of Intellectual Policy.
This seemed to catch some board members by surprise (including me) and
there were a bunch of really good points like whether a name like "IP
Policy Committee" would be more appropriate and that more diversity
(including non-laywers) beyond the existing 5 members is needed for such
a charter. I'm very happy to see the level of interest in getting this
right, because it's hugely important. In the end, we agreed to the
charter as propose but without the list of specific projects the
committee might undertake. The whole topic will be revisited in more
detail at our next meeting.</p>
<p>Next up, I was presenting as a <a href="http://wiki.openstack.org/Governance/Foundation/TechnicalCommittee">Technical
Committee</a>
representative about the progress of the <a href="http://wiki.openstack.org/Governance/Foundation/IncubationUpdate2013">Incubation and Core Update
Committee</a>.
More specifically, I was briefing the board on how the TC intends to
proceed with the incubation process for Ceilometer and Heat based on our
understanding of the TC's existing mandate. This is all covered by <a href="https://docs.google.com/drawings/d/1oLo1ETnRpNSgDj_m7p6o6tF7HHA2a-3XeKa-QLMBcRc/edit">an
excellent diagram from
Thierry</a>
and <a href="https://etherpad.openstack.org/IncubationAndCoreInterimSummary">a wordy
etherpad</a>.
This seemed to be fairly well received, barring some confusion about
whether a vote was required.</p>
<p>At this point, our scheduled time was almost up but we still had two
agenda items that really needed covering. Both needed to be discussed in
private (for good reasons IMHO), however one of them will be adequetly
summarised in the minutes while the other will be open to public
discussion very soon.</p>
<p>All in all, I have to say I enjoyed the meeting and look forward to the
<a href="http://wiki.openstack.org/Governance/Foundation/12Feb2013BoardMeeting">full day meeting on Feb 12 in San
Francisco</a>.</p>Image Building Service Demo2012-12-12T11:42:00+00:002012-12-12T11:42:00+00:00markmctag:crustyblaa.com,2012-12-12:/image-building-service-demo.html<p>Martyn Taylor and Steven Hardy have done an awesome job of <a href="http://lists.openstack.org/pipermail/openstack-dev/2012-December/003901.html">demoing an
Image Building Service for
OpenStack</a>:</p>
<p>http://www.youtube.com/watch?v=MdBM4HA3QUk</p>
<p>I think this has huge potential. Imagine an OpenStack API to which you
could send a request for a fresh image build of any OS …</p><p>Martyn Taylor and Steven Hardy have done an awesome job of <a href="http://lists.openstack.org/pipermail/openstack-dev/2012-December/003901.html">demoing an
Image Building Service for
OpenStack</a>:</p>
<p>http://www.youtube.com/watch?v=MdBM4HA3QUk</p>
<p>I think this has huge potential. Imagine an OpenStack API to which you
could send a request for a fresh image build of any OS, request specific
packages, software or other content to be included and have that image
be uploaded to Glance or Cinder once it's built. The image gets built in
the cloud on a Nova instance/VM and the cloud provider bills you for the
compute and I/O resources needed to complete the build.</p>cfg and argparse sub-commands2012-11-26T16:03:00+00:002012-11-26T16:03:00+00:00markmctag:crustyblaa.com,2012-11-26:/cfg-and-argparse-sub-commands.html<p><a href="https://launchpad.net/~laurence-miao">Laurence Miao</a> did some great
work recently to <a href="https://blueprints.launchpad.net/oslo/+spec/cfg-argparse">port cfg from optparse to
argparse</a>.</p>
<p>The only significant API impact was that we could no longer have
<code>CONF()</code> return unparsed command line arguments.</p>
<p>We chose to offer two alternatives - firstly, support for positional
arguments:</p>
<blockquote>
<p>>>> CONF.register_cli_opt(MultiStrOpt('bar',
positional=True))<br>
True …</p></blockquote><p><a href="https://launchpad.net/~laurence-miao">Laurence Miao</a> did some great
work recently to <a href="https://blueprints.launchpad.net/oslo/+spec/cfg-argparse">port cfg from optparse to
argparse</a>.</p>
<p>The only significant API impact was that we could no longer have
<code>CONF()</code> return unparsed command line arguments.</p>
<p>We chose to offer two alternatives - firstly, support for positional
arguments:</p>
<blockquote>
<p>>>> CONF.register_cli_opt(MultiStrOpt('bar',
positional=True))<br>
True >>> CONF(['a', 'b'])<br>
>>> CONF.bar<br>
['a', 'b']</p>
</blockquote>
<p>and, secondly, integration with <a href="http://docs.python.org/dev/library/argparse.html#sub-commands">argparse's
sub-commands</a>:</p>
<blockquote>
<p>>>> def add_parsers(subparsers):<br>
... list_action = subparsers.add_parser('list')<br>
... list_action.add_argument('id')<br>
...<br>
>>> CONF.register_cli_opt(SubCommandOpt('action',
handler=add_parsers))<br>
True<br>
>>> CONF(['list', '10'])<br>
>>> CONF.action.name, CONF.action.id<br>
('list', '10')</p>
</blockquote>
<p>After porting the CLIs in <a href="https://review.openstack.org/16881">Nova</a>
(nova-manage), <a href="https://review.openstack.org/16900">Glance</a>
(glance-control and glance-manage) and
<a href="https://review.openstack.org/16899">Keystone</a> (keystone-manage) over to
this sub-parsers stuff, it looks like it's going to work out quite
nicely.</p>Call for testing : 2012.2.1 tarballs2012-11-20T21:56:00+00:002012-11-20T21:56:00+00:00markmctag:crustyblaa.com,2012-11-20:/call-for-testing-2012-2-1-tarballs.html<p>We're hoping to publish Nova, Glance, Keystone, Quantum, Cinder and
Horizon 2012.2.1 next week (Nov 29).</p>
<p>The list of issues fixed so far can be seen here:</p>
<ul>
<li><a href="https://launchpad.net/nova/+milestone/2012.2.1">https://launchpad.net/nova/+milestone/2012.2.1</a></li>
<li><a href="https://launchpad.net/glance/+milestone/2012.2.1">https://launchpad.net/glance/+milestone/2012.2.1</a></li>
<li><a href="https://launchpad.net/keystone/+milestone/2012.2.1">https://launchpad.net/keystone/+milestone …</a></li></ul><p>We're hoping to publish Nova, Glance, Keystone, Quantum, Cinder and
Horizon 2012.2.1 next week (Nov 29).</p>
<p>The list of issues fixed so far can be seen here:</p>
<ul>
<li><a href="https://launchpad.net/nova/+milestone/2012.2.1">https://launchpad.net/nova/+milestone/2012.2.1</a></li>
<li><a href="https://launchpad.net/glance/+milestone/2012.2.1">https://launchpad.net/glance/+milestone/2012.2.1</a></li>
<li><a href="https://launchpad.net/keystone/+milestone/2012.2.1">https://launchpad.net/keystone/+milestone/2012.2.1</a></li>
<li><a href="https://launchpad.net/quantum/+milestone/2012.2.1">https://launchpad.net/quantum/+milestone/2012.2.1</a></li>
<li><a href="https://launchpad.net/cinder/+milestone/2012.2.1">https://launchpad.net/cinder/+milestone/2012.2.1</a></li>
<li><a href="https://launchpad.net/horizon/+milestone/2012.2.1">https://launchpad.net/horizon/+milestone/2012.2.1</a></li>
</ul>
<p>That's roughly 80 bugs.</p>
<p>We'd appreciate anyone who could give the candidate tarballs a whirl:</p>
<ul>
<li><a href="http://tarballs.openstack.org/nova/nova-stable-folsom.tar.gz">http://tarballs.openstack.org/nova/nova-stable-folsom.tar.gz</a></li>
<li><a href="http://tarballs.openstack.org/glance/glance-stable-folsom.tar.gz">http://tarballs.openstack.org/glance/glance-stable-folsom.tar.gz</a></li>
<li><a href="http://tarballs.openstack.org/keystone/keystone-stable-folsom.tar.gz">http://tarballs.openstack.org/keystone/keystone-stable-folsom.tar.gz</a></li>
<li><a href="http://tarballs.openstack.org/quantum/quantum-stable-folsom.tar.gz">http://tarballs.openstack.org/quantum/quantum-stable-folsom.tar.gz</a></li>
<li><a href="http://tarballs.openstack.org/cinder/cinder-stable-folsom.tar.gz">http://tarballs.openstack.org/cinder/cinder-stable-folsom.tar.gz</a></li>
<li><a href="http://tarballs.openstack.org/horizon/horizon-stable-folsom.tar.gz">http://tarballs.openstack.org/horizon/horizon-stable-folsom.tar.gz</a></li>
</ul>
<p>We've also started drafting release notes
<a href="http://wiki.openstack.org/ReleaseNotes/2012.2.1">here</a>. Contributions
to those release notes are very welcome.</p>The Future of Incubation and Core2012-11-17T17:53:00+00:002012-11-17T17:53:00+00:00markmctag:crustyblaa.com,2012-11-17:/the-future-of-incubation-and-core.html<p>The <a href="http://www.openstack.org/foundation/technical-committee/">OpenStack Technical
Committee</a> and
the <a href="http://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a> have
pretty separate sets of responsibilities and can get on with their work
independently. One exception to that is the inclusion of new projects in
OpenStack.</p>
<p>In the coming weeks, members of the two bodies will decide how to …</p><p>The <a href="http://www.openstack.org/foundation/technical-committee/">OpenStack Technical
Committee</a> and
the <a href="http://www.openstack.org/foundation/board-of-directors/">OpenStack Foundation Board of
Directors</a> have
pretty separate sets of responsibilities and can get on with their work
independently. One exception to that is the inclusion of new projects in
OpenStack.</p>
<p>In the coming weeks, members of the two bodies will decide how to
clarify confusion around the term "core project" and what exactly
happens projects who graduate through OpenStack's Incubation process.</p>
<p>A <a href="http://lists.openstack.org/pipermail/openstack-dev/2012-November/thread.html#2387">thread on the openstack-dev mailing
list</a>
is ongoing and is a great example of how a mailing list discussion can
actually help to drive a rough consensus while still giving everyone an
opportunity to express their views.</p>
<p>The TC is attempting to agree on a rough direction that represents the
views of the TC before meeting with the Foundation Board. There are
currently three distinct views. Firstly,
<a href="http://lists.openstack.org/pipermail/openstack-dev/2012-November/002771.html">this</a>:</p>
<blockquote>
<p>The concepts of "what is core" and "what is in OpenStack" have been
conflated until now. The TC cares far more about the process for new
projects to be included in the coordinated release than it cares about
which projects are required to be used by providers in order to access
the trademark.</p>
<p>We would like to take an inclusive but measured approach to accepting
new OpenStack projects. We should evaluate any given proposed project
on a well defined set of criteria like whether it embraces our values
and processes, is useful to OpenStack users, well integrated with
other projects and represents a sensible broadening of the scope of
OpenStack.</p>
<p>We see Incubation as a trial period where promising projects have the
opportunity to demonstrate their suitability for inclusion in our
coordinated releases.</p>
<p>We see the term "Core OpenStack Project" in section 4.1.b of the
bylaws as being solely related to trademark guidelines. The Foundation
should simply maintain a list of projects required for trademark
usage. We would be happy for that list to be called "Core Projects" or
for a new name to be chosen to describe that list.</p>
</blockquote>
<p>Secondly, <a href="http://lists.openstack.org/pipermail/openstack-dev/2012-November/002956.html">Anne Gentle's
variation</a>
which I'd summarize as allowing two groups of projects be accepted into
OpenStack - "nuclear" projects which are the current group of "core"
service projects and "core" projects which are everything else, for
example Horizon, Ceilometer or Heat.</p>
<p>Thirdly, <a href="http://lists.openstack.org/pipermail/openstack-dev/2012-November/002954.html">John Dickinson's
variation</a>
which I'd summarize as only accepting projects into OpenStack which
"solve IaaS problems" or support those projects in some way.</p>
<p>The way I'm thinking of these three approaches is how we want the
project to treat proposals for new projects: "inclusive", "inclusive but
two-tier" or "exclusive".</p>Who Wrote Folsom2012-09-28T07:19:00+01:002012-09-28T07:19:00+01:00markmctag:crustyblaa.com,2012-09-28:/who-wrote-folsom.html<p>As <a href="https://lists.launchpad.net/openstack/msg09650.html">I did for
Essex</a>, I've used
<a href="http://lwn.net/Articles/290957/">Jonathan Corbet's gitdm</a> to do some
analysis of the commits to the core projects during Folsom.</p>
<p>Here's the top 20 committers across Nova, Glance, Swift, Keystone,
Horizon, Quantum and Cinder:</p>
<p><code>Processed 3110 csets from 291 developers 132 employers found A total of 350046 …</code></p><p>As <a href="https://lists.launchpad.net/openstack/msg09650.html">I did for
Essex</a>, I've used
<a href="http://lwn.net/Articles/290957/">Jonathan Corbet's gitdm</a> to do some
analysis of the commits to the core projects during Folsom.</p>
<p>Here's the top 20 committers across Nova, Glance, Swift, Keystone,
Horizon, Quantum and Cinder:</p>
<p><code>Processed 3110 csets from 291 developers 132 employers found A total of 350046 lines added, 275491 removed (delta 74555)</code></p>
<p>Developers with the most changesets<br>
Russell Bryant 160 (5.1%)<br>
Brian Waldon 160 (5.1%)<br>
Dan Prince 154 (5.0%)<br>
Gabriel Hurley 124 (4.0%)<br>
Mark McLoughlin 118 (3.8%)<br>
Johannes Erdfelt 113 (3.6%)<br>
Vishvananda Ishaya 92 (3.0%)<br>
Joe Gordon 85 (2.7%)<br>
Michael Still 71 (2.3%)<br>
Eoghan Glynn 70 (2.3%)<br>
Rick Harris 59 (1.9%)<br>
Gary Kotton 58 (1.9%)<br>
Dolph Mathews 55 (1.8%)<br>
Yun Mao 50 (1.6%)<br>
John Griffith 45 (1.4%)<br>
Daniel P. Berrange 45 (1.4%)<br>
Dan Wendlandt 43 (1.4%)<br>
gholt 40 (1.3%)<br>
Rongze Zhu 39 (1.3%)<br>
Alex Meade 39 (1.3%)<br>
Covers 52.090032% of changesets<br>
</code></p>
<p>Congrats and thanks to everyone involved in this release!</p>
<p>There are <a href="https://github.com/markmc/openstack-gitdm/tree/results/folsom">more raw stats
here</a>
showing stats for each project individually and also statistics for
gerrit reviewers and launchpad bug fixers.</p>
<p>Of course, this is nowhere near as polished as <a href="http://bitergia.com/public/reports/openstack/2012_09_folsom/">Bitergia's awesome
report</a>
with pretty graphs and detailed analysis.</p>Friday is for Yak Shaving2012-09-14T17:53:00+01:002012-09-14T17:53:00+01:00markmctag:crustyblaa.com,2012-09-14:/friday-is-for-yak-shaving.html<p>My mate <a href="http://goodsquishy.com/">Derek</a> was giving me grief about not
testing his OpenStack deployment in our lab at Red Hat. Friday seemed
like a good day to give it a shot for a few minutes.</p>
<p>First problem - I'm one of the weird people at Red Hat who eschews the
VPN in …</p><p>My mate <a href="http://goodsquishy.com/">Derek</a> was giving me grief about not
testing his OpenStack deployment in our lab at Red Hat. Friday seemed
like a good day to give it a shot for a few minutes.</p>
<p>First problem - I'm one of the weird people at Red Hat who eschews the
VPN in favour of SSH tunnels. At first, I figured I'd tunnel directly to
the various OpenStack API services but that didn't work because the
endpoint URLs returned by keystone obviously wouldn't point to my
tunnelled connections.</p>
<p>Ok, let's just use a HTTP proxy, that should be fine. But no, not on yak
shaving day. For some reason, I was getting 403 Forbidden errors.</p>
<p>To cut a long story short, it turns out:</p>
<ul>
<li>httplib2 always uses HTTP CONNECT tunneling rather than just sending
the requests directly to the proxy</li>
<li>squid by default and, indeed, our corporate proxy defaults to
rejecting CONNECT for ports other than 443</li>
<li>The recently released httplib2 0.7.5 has a
PROXY_TYPE_HTTP_NO_TUNNEL which only uses CONNECT tunnelling for
port 443, but it doesn't use this type when you configure your proxy
via http_proxy in the environment</li>
</ul>
<p>Not content with shaving the yak once, I shaved her thrice:</p>
<ul>
<li>An upstream <a href="http://code.google.com/p/httplib2/issues/detail?id=228">patch to use
HTTP_NO_TUNNEL</a>
when using http_proxy env</li>
<li>A <a href="https://bugzilla.redhat.com/857514">request</a> to get the patch
into Fedora</li>
<li>A <a href="https://bugs.launchpad.net/python-novaclient/+bug/1051007">patch to work around
it</a> in
python-novaclient</li>
</ul>
<p>One other troubling conclusion is that if you're exposing the services
over HTTPS, you really should use port 443 for everything or clients
won't be able to connect over many proxies.</p>Submitting new features to Nova2012-08-20T11:27:00+01:002012-08-20T11:27:00+01:00markmctag:crustyblaa.com,2012-08-20:/submitting-new-features-to-nova.html<p>I just wrote down a few pieces advice for someone submitting a large
feature patch to Nova, so I thought I'd re-post it here:</p>
<ul>
<li>Think about what it is like to be a nova-core reviewer looking at a
list of 40 to 60 reviews and having maybe 2 hours today …</li></ul><p>I just wrote down a few pieces advice for someone submitting a large
feature patch to Nova, so I thought I'd re-post it here:</p>
<ul>
<li>Think about what it is like to be a nova-core reviewer looking at a
list of 40 to 60 reviews and having maybe 2 hours today to
do reviews. Think about how to make it more likely that such a
reviewer will choose your code to review.</li>
<li>Small patches are easier to consume. The smaller you make the patch,
the more likely it is that it will get reviewed quickly.</li>
<li>Break your changes into a series of <a href="http://wiki.openstack.org/GitCommitMessages#Structural_split_of_changes">small, self-contained
changes</a>.</li>
<li>The earlier in the release cycle you can begin submitting some of
your changes the better. Don't wait until all of your changes are
finished before submitting.</li>
<li>Do as much of your design discussion in the open, preferably on the
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">openstack-dev mailing
list</a>.
If you have discussions on the phone or IRC, try and post a summary
of those discussions to the<br>
mailing list.</li>
<li>Holding a design summit session to discuss your changes in advance
is a great idea, but don't assume that everybody who may have an
opinion on your changes is present. Also, bear in mind that
someone's quick opinion offered at a design summit session may be
very different from their considered opinion after reviewing the
code in detail.</li>
<li>Finally, try and participate in the project beyond just making the
changes you need. Review other peoples' changes in gerrit and offer
your opinions, participate in design discussions on the mailing
lists, fix bugs you come across, triage bug reports, etc. etc. All
of this will allow other developers to get to know you, trust your
judgement and review your changes more quickly. You will also learn
more by interacting with other developers.</li>
</ul>My OpenStack Foundation Board Candidacy2012-08-20T09:45:00+01:002012-08-20T09:45:00+01:00markmctag:crustyblaa.com,2012-08-20:/my-openstack-foundation-board-candidacy.html<p>The OpenStack Foundation Board election has begun with <a href="http://www.openstack.org/election/2012-board-election/candidates/">39 excellent
candidates</a>
for only 8 seats!</p>
<p>Each candidate was asked to answer a number of questions to give an
insight into why they are running for the Board. My answers are below.</p>
<p>If you'd like to know some more about my …</p><p>The OpenStack Foundation Board election has begun with <a href="http://www.openstack.org/election/2012-board-election/candidates/">39 excellent
candidates</a>
for only 8 seats!</p>
<p>Each candidate was asked to answer a number of questions to give an
insight into why they are running for the Board. My answers are below.</p>
<p>If you'd like to know some more about my involvement in OpenStack,
Rackspace recently posted <a href="www.rackspace.com/blog/how-i-contribute-to-openstack-red-hats-mark-mcloughlin/">a nice interview with
me</a>
as part of their "How I Contribute To OpenStack" series.</p>
<p><strong>What is your relationship to OpenStack, and why is its success
important to you and/or your company? What would you say is your biggest
contribution to OpenStack's success to date?<br>
</strong></p>
<p>I've been involved with OpenStack for just over a year now. I started
out packaging OpenStack for Fedora, but have concentrated mostly on
contributing to the core projects in various ways since then.</p>
<p>I'm a member of the nova-core team and spend quite a bit of time
reviewing proposed changes in gerrit. I'm also the openstack-common PTL
and have been working to kickstart this effort to create a shared
library for all OpenStack projects. My role as PTL recently means that
I'm now a PPB/TC member, but I haven't contributed much there yet.
Finally, I co-ordinate the efforts of the stable-maint and release teams
to maintain a stable branch for OpenStack's core projects and publish
regular bugfix releases.</p>
<p>It's thanks to Red Hat that I have time to make these contributions to
the project and I am the technical lead of Red Hat's OpenStack team.</p>
<p>Red Hat's mission statement is all about being a catalyst in communities
and, while credit must go to others for initially catalysing the
OpenStack community, we recognize the massive strength of OpenStack's
diverse community. This strength is the foundation that will ensure that
OpenStack continues to grow, continues to improve and will be around for
a long time to win. Red Hat is doing everything it can to help make that
happen.</p>
<p><strong>Describe your experience with other non profits or serving as a board
member. How does your experience prepare you for the role of a board
member?<br>
</strong><br>
I've been heavily involved in various open-source projects for over a
decade now. Observing and pondering the role of non-profit organizations
in open-source projects has prepared me well to participate in the
OpenStack Foundation and its efforts to determine its own particular
role.</p>
<p>Indeed, I participated significantly in the early discussions leading up
to the formation of the Foundation and even proposed a <a href="http://wiki.openstack.org/StrawmanFoundationStructure">strawman
Foundation
structure</a> based
on my experiences with other similar organizations.</p>
<p><strong>What do you see as the Board's role in OpenStack's success?</strong></p>
<p>More importantly, I think, is the role the Foundation will play in
OpenStack's success. I'm a big fan of the Foundation's simple mission
statement - to "protect, empower and promote" OpenStack. It perfectly
encapsulates the role of the Foundation.</p>
<p>The "protecting" and "promoting" elements of the mission are hugely
important, but what I find myself wondering most about is how the
Foundation can "empower" the project. The Foundation can certainly use
some of its budget to fill gaps in the resources which are available to
the project. But, to me, "empowering" the project is mostly about
re-inforcing the understanding that the project's future and direction
lies in the hand of its meritocractic community of contributors (in the
broadest possible sense) and not the Foundation itself.</p>
<p>I think that if the Board delivers on the promise of a Foundation which
"protects, empowers and promotes", we will find ourselves in the
enviable situation of a project with a large and diverse community of
contributors supported by a well-financed organization acting at all
times in the interests of that community.</p>
<p><strong>What do you think the top priority of the Board should be during the
OpenStack Foundation's first year?<br>
</strong><br>
I think it's the transitioning of responsibilities from Rackspace to the
Foundation will take much of the focus of the Foundation and the Board
in the coming year. The Board will need to appoint an Executive Director
and, in turn, oversee the Executive Director's work to transition the
likes of community management, release management and event
co-ordination to employees of the Foundation.</p>
<p>A huge part of OpenStack's success to date is that functions such as
these have been executed extremely well under Rackspace's management.
The transition to a Foundation is a significant challenge in its own
right and we cannot afford any missteps here. As part of this
transition, I look forward to more open discussion leading up to
decisions made by the Foundation.</p>
<p>I also expect the Board to quickly establish the User Committee and
Legal Affairs Committee. The User Committee will have a hugely important
role to help advocate specific needs of operators and users. The Legal
Affairs Committee will be needed to help us make progress on reforming
our trademark and contributor agreements.</p>
<p>Finally, the Board will have its work cut out finding its feet as the
official voice of OpenStack. The Board will be expected to speak on
behalf of us all and, as such, will need to work hard to build and
reflect consensus within the project.</p>Nova DB Poking2012-07-31T13:37:00+01:002012-07-31T13:37:00+01:00markmctag:crustyblaa.com,2012-07-31:/nova-db-poking.html<p>So, you want to play with some sqlalchemy queries against the Nova DB in
an interactive python console?</p>
<div class="highlight"><pre><span></span><code><span class="err">$</span><span class="o">></span> <span class="n">sudo</span> <span class="n">nova</span><span class="o">-</span><span class="n">manage</span> <span class="n">shell</span> <span class="n">python</span>
<span class="o">...</span>
<span class="o">>>></span> <span class="kn">from</span> <span class="nn">nova.db.sqlalchemy</span> <span class="kn">import</span> <span class="n">api</span>
<span class="o">>>></span> <span class="kn">from</span> <span class="nn">nova.db.sqlalchemy</span> <span class="kn">import</span> <span class="n">models</span>
<span class="o">>>></span> <span class="kn">from</span> <span class="nn">nova</span> <span class="kn">import</span> <span class="n">context</span>
<span class="o">>>></span> <span class="n">ctxt</span> <span class="o">=</span> <span class="n">context</span><span class="o">.</span><span class="n">get_admin_context</span><span class="p">()</span>
<span class="o">>>></span> <span class="nb">vars</span><span class="p">(</span><span class="n">api</span><span class="o">.</span><span class="n">model_query</span><span class="p">(</span><span class="n">ctxt</span><span class="p">,</span> <span class="n">models</span><span class="o">.</span><span class="n">Service</span><span class="p">)</span><span class="o">.</span><span class="n">all</span><span class="p">()[</span><span class="mi">0 …</span></code></pre></div><p>So, you want to play with some sqlalchemy queries against the Nova DB in
an interactive python console?</p>
<div class="highlight"><pre><span></span><code><span class="err">$</span><span class="o">></span> <span class="n">sudo</span> <span class="n">nova</span><span class="o">-</span><span class="n">manage</span> <span class="n">shell</span> <span class="n">python</span>
<span class="o">...</span>
<span class="o">>>></span> <span class="kn">from</span> <span class="nn">nova.db.sqlalchemy</span> <span class="kn">import</span> <span class="n">api</span>
<span class="o">>>></span> <span class="kn">from</span> <span class="nn">nova.db.sqlalchemy</span> <span class="kn">import</span> <span class="n">models</span>
<span class="o">>>></span> <span class="kn">from</span> <span class="nn">nova</span> <span class="kn">import</span> <span class="n">context</span>
<span class="o">>>></span> <span class="n">ctxt</span> <span class="o">=</span> <span class="n">context</span><span class="o">.</span><span class="n">get_admin_context</span><span class="p">()</span>
<span class="o">>>></span> <span class="nb">vars</span><span class="p">(</span><span class="n">api</span><span class="o">.</span><span class="n">model_query</span><span class="p">(</span><span class="n">ctxt</span><span class="p">,</span> <span class="n">models</span><span class="o">.</span><span class="n">Service</span><span class="p">)</span><span class="o">.</span><span class="n">all</span><span class="p">()[</span><span class="mi">0</span><span class="p">])</span>
<span class="p">{</span><span class="s1">'binary'</span><span class="p">:</span> <span class="sa">u</span><span class="s1">'nova-compute'</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="s1">'topic'</span><span class="p">:</span> <span class="sa">u</span><span class="s1">'compute'</span><span class="p">,</span> <span class="s1">'host'</span><span class="p">:</span> <span class="sa">u</span><span class="s1">'f16'</span><span class="p">,</span> <span class="s1">'disabled'</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span> <span class="s1">'deleted_at'</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span> <span class="s1">'id'</span><span class="p">:</span> <span class="mi">1</span><span class="n">L</span><span class="p">}</span>
</code></pre></div>Ancient History2012-07-27T11:54:00+01:002012-07-27T11:54:00+01:00markmctag:crustyblaa.com,2012-07-27:/ancient-history.html<p>In OpenStack, we have a particular problem where much of the early
development on the project was done using bzr and launchpad. All this
history is in git, but it can be difficult to find the bzr merge
proposal in launchpad which caused a given commit to be merged.</p>
<p>Here's …</p><p>In OpenStack, we have a particular problem where much of the early
development on the project was done using bzr and launchpad. All this
history is in git, but it can be difficult to find the bzr merge
proposal in launchpad which caused a given commit to be merged.</p>
<p>Here's an example of how I did it yesterday.</p>
<p>We're interested in <a href="https://github.com/openstack/nova/commit/8aea573">commit
8aea573</a>:</p>
<div class="highlight"><pre><span></span><code>commit 8aea573bd2e44e152fb4ef1627640bab1818dede
Author: Trey Morris ...
Date: Tue Dec 28 23:55:58 2010 -0600
initial lock functionality commit
</code></pre></div>
<p>To trace back to the merge commit which merged this into master, I did:</p>
<div class="highlight"><pre><span></span><code><span class="err">$</span><span class="p">></span><span class="w"> </span><span class="nx">git</span><span class="w"> </span><span class="nx">log</span><span class="w"> </span><span class="o">--</span><span class="nx">graph</span><span class="w"> </span><span class="o">--</span><span class="nx">topo</span><span class="o">-</span><span class="nx">order</span><span class="w"> </span><span class="o">--</span><span class="nx">ancestry</span><span class="o">-</span><span class="nx">path</span><span class="w"> </span><span class="o">--</span><span class="nx">merges</span><span class="w"> </span><span class="mi">8</span><span class="nx">aea573bd2e44e152fb4ef1627640bab1818dede</span><span class="p">..</span><span class="nx">HEAD</span>
<span class="o">*</span><span class="w"> </span><span class="nx">commit</span><span class="w"> </span><span class="nx">ae5dbe2b5d4871d3e26e859c03feab705c9c59ea</span>
<span class="w"> </span><span class="nx">Merge</span><span class="p">:</span><span class="w"> </span><span class="mi">9</span><span class="nx">eca4d5</span><span class="w"> </span><span class="mi">76</span><span class="nx">e3923</span>
<span class="w"> </span><span class="nx">Author</span><span class="p">:</span><span class="w"> </span><span class="nx">Trey</span><span class="w"> </span><span class="nx">Morris</span><span class="w"> </span><span class="o">...</span>
<span class="w"> </span><span class="nx">Date</span><span class="p">:</span><span class="w"> </span><span class="nx">Fri</span><span class="w"> </span><span class="nx">Jan</span><span class="w"> </span><span class="mi">7</span><span class="w"> </span><span class="mi">00</span><span class="p">:</span><span class="mi">49</span><span class="p">:</span><span class="mi">30</span><span class="w"> </span><span class="mi">2011</span><span class="w"> </span><span class="o">+</span><span class="mi">0000</span>
<span class="w"> </span><span class="nx">This</span><span class="w"> </span><span class="nx">branch</span><span class="w"> </span><span class="nx">implements</span><span class="w"> </span><span class="nx">lock</span><span class="w"> </span><span class="nx">functionality</span><span class="p">.</span><span class="w"> </span><span class="nx">The</span><span class="w"> </span><span class="nx">lock</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="nx">stored</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">compute</span><span class="w"> </span><span class="nx">worker</span><span class="w"> </span><span class="nx">database</span><span class="p">.</span><span class="w"> </span><span class="nx">Decorators</span><span class="w"> </span><span class="nx">have</span><span class="w"> </span><span class="nx">been</span><span class="w"> </span><span class="nx">added</span><span class="w"> </span><span class="nx">to</span><span class="w"> </span><span class="nx">the</span><span class="w"> </span><span class="nx">opens</span>
<span class="o">*</span><span class="w"> </span><span class="nx">commit</span><span class="w"> </span><span class="nx">f9c33f4ba09e02f8668bdd655b7acba15984838c</span>
<span class="w"> </span><span class="nx">Merge</span><span class="p">:</span><span class="w"> </span><span class="nx">ba245da</span><span class="w"> </span><span class="mi">9</span><span class="nx">eca4d5</span>
<span class="w"> </span><span class="nx">Author</span><span class="p">:</span><span class="w"> </span><span class="nx">Trey</span><span class="w"> </span><span class="nx">Morris</span><span class="w"> </span><span class="o">...</span>
<span class="w"> </span><span class="nx">Date</span><span class="p">:</span><span class="w"> </span><span class="nx">Thu</span><span class="w"> </span><span class="nx">Jan</span><span class="w"> </span><span class="mi">6</span><span class="w"> </span><span class="mi">16</span><span class="p">:</span><span class="mi">35</span><span class="p">:</span><span class="mi">48</span><span class="w"> </span><span class="mi">2011</span><span class="w"> </span><span class="o">-</span><span class="mi">0600</span>
<span class="w"> </span><span class="nx">merged</span><span class="w"> </span><span class="nx">trunk</span>
<span class="o">*</span><span class="w"> </span><span class="nx">commit</span><span class="w"> </span><span class="nx">f09d1ce4d38f3a8ef72566e95cde38f1dc1b8bed</span>
<span class="w"> </span><span class="nx">Merge</span><span class="p">:</span><span class="w"> </span><span class="mi">9</span><span class="nx">b9b5fe</span><span class="w"> </span><span class="mi">9</span><span class="nx">a84a2b</span>
<span class="w"> </span><span class="nx">Author</span><span class="p">:</span><span class="w"> </span><span class="nx">Trey</span><span class="w"> </span><span class="nx">Morris</span><span class="w"> </span><span class="o">...</span>
<span class="w"> </span><span class="nx">Date</span><span class="p">:</span><span class="w"> </span><span class="nx">Wed</span><span class="w"> </span><span class="nx">Dec</span><span class="w"> </span><span class="mi">29</span><span class="w"> </span><span class="mi">15</span><span class="p">:</span><span class="mi">13</span><span class="p">:</span><span class="mi">24</span><span class="w"> </span><span class="mi">2010</span><span class="w"> </span><span class="o">-</span><span class="mi">0600</span>
<span class="w"> </span><span class="nx">fixed</span><span class="w"> </span><span class="nx">merge</span><span class="w"> </span><span class="nx">conflict</span><span class="w"> </span><span class="nx">with</span><span class="w"> </span><span class="nx">trunk</span>
</code></pre></div>
<p>Double check that by looking at exactly what was merged in:</p>
<div class="highlight"><pre><span></span><code>$<span class="o">></span><span class="w"> </span><span class="nv">git</span><span class="w"> </span><span class="nv">diff</span><span class="w"> </span><span class="mi">9</span><span class="nv">eca4d5</span>..<span class="nv">ae5dbe2</span>
<span class="nv">diff</span><span class="w"> </span><span class="o">--</span><span class="nv">git</span><span class="w"> </span><span class="nv">a</span><span class="o">/</span><span class="nv">nova</span><span class="o">/</span><span class="nv">api</span><span class="o">/</span><span class="nv">openstack</span><span class="o">/</span><span class="nv">servers</span>.<span class="nv">py</span><span class="w"> </span><span class="nv">b</span><span class="o">/</span><span class="nv">nova</span><span class="o">/</span><span class="nv">api</span><span class="o">/</span><span class="nv">openstack</span><span class="o">/</span><span class="nv">servers</span>.<span class="nv">py</span>
<span class="nv">index</span><span class="w"> </span><span class="nv">ce64ac7</span>..<span class="nv">f8d5e76</span><span class="w"> </span><span class="mi">100644</span>
<span class="o">---</span><span class="w"> </span><span class="nv">a</span><span class="o">/</span><span class="nv">nova</span><span class="o">/</span><span class="nv">api</span><span class="o">/</span><span class="nv">openstack</span><span class="o">/</span><span class="nv">servers</span>.<span class="nv">py</span>
<span class="o">+++</span><span class="w"> </span><span class="nv">b</span><span class="o">/</span><span class="nv">nova</span><span class="o">/</span><span class="nv">api</span><span class="o">/</span><span class="nv">openstack</span><span class="o">/</span><span class="nv">servers</span>.<span class="nv">py</span>
@@<span class="w"> </span><span class="o">-</span><span class="mi">170</span>,<span class="mi">6</span><span class="w"> </span><span class="o">+</span><span class="mi">170</span>,<span class="mi">50</span><span class="w"> </span>@@<span class="w"> </span><span class="nv">class</span><span class="w"> </span><span class="nv">Controller</span><span class="ss">(</span><span class="nv">wsgi</span>.<span class="nv">Controller</span><span class="ss">)</span>:
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nv">faults</span>.<span class="nv">Fault</span><span class="ss">(</span><span class="nv">exc</span>.<span class="nv">HTTPUnprocessableEntity</span><span class="ss">())</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nv">exc</span>.<span class="nv">HTTPAccepted</span><span class="ss">()</span>
<span class="o">+</span><span class="w"> </span><span class="nv">def</span><span class="w"> </span><span class="nv">lock</span><span class="ss">(</span><span class="nv">self</span>,<span class="w"> </span><span class="nv">req</span>,<span class="w"> </span><span class="nv">id</span><span class="ss">)</span>:
...
</code></pre></div>
<p>That's the one!</p>
<p>Now how to find the merge proposal? Simply googling for "This branch
implements lock functionality" quickly lead me to the <a href="https://code.launchpad.net/~tr3buchet/nova/lock/+merge/44874">correct merge
prop</a>, but
better ideas welcome :)</p>Fedora 16 OpenStack Test Day2011-10-20T06:03:00+01:002011-10-20T06:03:00+01:00markmctag:crustyblaa.com,2011-10-20:/fedora-16-openstack-test-day.html<p>We're holding a test day for <a href="http://fedoraproject.org/wiki/Test_Day:2011-10-20_OpenStack_Test_Day">Fedora 16
OpenStack</a>
today on #fedora-test-day.</p>Gerrit Patch Review From The Command Line2011-09-25T14:28:00+01:002011-09-25T14:28:00+01:00markmctag:crustyblaa.com,2011-09-25:/gerrit-patch-review-from-the-command-line.html<p><a href="http://nova.openstack.org/">OpenStack Nova</a> switched from bzr and
launchpad to github and <a href="http://code.google.com/p/gerrit/">gerrit</a> on
Friday. While I'm delighted the project is using git now, I've always
found the gerrit UI to be a bit of a pain.</p>
<p>On IRC, <a href="http://inaugust.com/~mordred">Monty Taylor</a> mentioned the
<a href="https://review.openstack.org/Documentation/cmd-index.html">gerrit command line
interface</a>
which looked fairly interesting. Sure …</p><p><a href="http://nova.openstack.org/">OpenStack Nova</a> switched from bzr and
launchpad to github and <a href="http://code.google.com/p/gerrit/">gerrit</a> on
Friday. While I'm delighted the project is using git now, I've always
found the gerrit UI to be a bit of a pain.</p>
<p>On IRC, <a href="http://inaugust.com/~mordred">Monty Taylor</a> mentioned the
<a href="https://review.openstack.org/Documentation/cmd-index.html">gerrit command line
interface</a>
which looked fairly interesting. Sure enough, you can actually review
and approve a patch using this without ever touching the web UI. Below
is an example of reviewing a <a href="http://glance.openstack.org/">Glance</a>
patch, but the same thing would work for Nova.</p>
<p>First, you obviously need to clone the repo:</p>
<div class="highlight"><pre><span></span><code>$> git clone git://github.com/openstack/glance.git
$> cd glance
</code></pre></div>
<p>To make life a little easier, you can add a host alias to your SSH
config:</p>
<div class="highlight"><pre><span></span><code>$> cat >> ~/.ssh/config <<EOF
Host review
Hostname review.openstack.org
Port 29418
User markmc
EOF
</code></pre></div>
<p>Then add the gerrit server as a git remote:</p>
<div class="highlight"><pre><span></span><code><span class="err">$</span><span class="o">></span><span class="w"> </span><span class="n">git</span><span class="w"> </span><span class="n">remote</span><span class="w"> </span><span class="k">add</span><span class="w"> </span><span class="o">-</span><span class="n">f</span><span class="w"> </span><span class="n">gerrit</span><span class="w"> </span><span class="nl">ssh</span><span class="p">:</span><span class="o">//</span><span class="n">markmc</span><span class="nv">@review</span><span class="o">/</span><span class="n">openstack</span><span class="o">/</span><span class="n">glance</span><span class="p">.</span><span class="n">git</span>
</code></pre></div>
<p>Okay, now browse the patches needing review:</p>
<div class="highlight"><pre><span></span><code>$> ssh review gerrit query status:open project:openstack/glance | less
</code></pre></div>
<p>Once you've picked a patch, take it's Change-Id: and look at its patch
sets and reviews:</p>
<div class="highlight"><pre><span></span><code>$> ssh review gerrit query status:open project:openstack/glance
change:I27bb6b3951422ad32e5e0225765b1056c5b3ffc5
--current-patch-set --all-approvals | less
</code></pre></div>
<p>Then, using the 'ref' in the output, you can fetch the patches into your
repo and review them:</p>
<div class="highlight"><pre><span></span><code>$> git fetch gerrit refs/changes/36/636/2
$> git checkout -b git-authors FETCH_HEAD
</code></pre></div>
<p>Once you're ready to submit your review, you can do:</p>
<div class="highlight"><pre><span></span><code>$> git checkout master
$> git branch -D git-authors
$> ssh review gerrit review --code-review +1 -m "'Looks good to me.'" cd9b3a0f2fb91d0d01606ef4bbd90cf8f29267da
</code></pre></div>
<p>That's all pretty neat, but I'm missing how to go about doing a detailed
review with comments inline with quoted sections of code. Perhaps if
'gerrit review' could take the review comments over stdin?</p>Our Bizarre Posting Conventions2011-02-17T16:09:00+00:002011-02-17T16:09:00+00:00markmctag:crustyblaa.com,2011-02-17:/our-bizarre-posting-conventions.html<p>I've been around open source mailing lists for so long now that I tend
to forget that our conventions around sending plain text emails, quoting
replies, threading, etc. aren't necessarily obvious to everyone out
there.</p>
<p>That's especially true for some of the folks working on RHEV-M, many of
whom have …</p><p>I've been around open source mailing lists for so long now that I tend
to forget that our conventions around sending plain text emails, quoting
replies, threading, etc. aren't necessarily obvious to everyone out
there.</p>
<p>That's especially true for some of the folks working on RHEV-M, many of
whom have come from a Windows development background. I eventually
realized that saying "just do what everyone else does" wasn't really
good enough and spent some time documenting what all those little
conventions are.</p>
<p>I've posted <a href="https://fedorahosted.org/rhevm-api/wiki/Email_Guidelines">some
guidelines</a> to
the rhevm-api wiki. What am I missing?</p>