Crusty Blaa

The status quo around RHEL source code availability has changed, as has Red Hat’s stance on RHEL rebuilds. For example, we now say:

Ultimately, we do not find value in a RHEL rebuild and we are not under any obligation to make things easier for rebuilders; this is our call to make. ... Simply rebuilding code, without adding value or changing it in any way, represents a real threat to open source companies everywhere.

If the status quo has changed, doesn’t that mean that our values have changed? I believe the opposite is true - this is Red Hat fully confronting its values, taking uncomfortable steps to make our values more clear, and moving beyond an uneasy ambiguity that I think has existed for too long. And that’s not a convenient, reverse-engineered conclusion to justify the change after-the-fact. I can actually pinpoint the discussion in 2011 where our values on this became clear to me. Let me explain.

I joined Red Hat 20 years ago via the GNOME project. Any personal values I had around software freedom were not particularly strongly held, and GNOME itself had a sometimes fractious relationship with the FSF. But I was drawn in and excited by the potential of developing software in the open, with a community of like-minded contributors with diverse motivations for contributing. This was a time of open-source vs free software debates, and I found myself observing, fascinated, and leaning towards the open-source perspective. To pick one random example of the sort of argument that appealed to me (while absolutely rejected by other friends in the community):

Richard thinks there is a moral imperative underlying the free redistribution of software, and now, by extension, other information. Richard feels that since there isn't any physical cost associated with copying software, limiting free redistribution is a form of extortion. I on the other hand feel that it's immoral to try to compel someone else to give you something they've created without compensating them in some way. That is, when software is freed, it is a gift, not the result of an obligation...

When I joined Red Hat, I again found myself observing the perspectives of my colleagues and the decisions we were making. I guess I was trying to identify Red Hat's values on this knotty subject. It's a funny thing trying to pin down a group’s values - you're unlikely to find a consensus view, and neither are you simply looking for the view of a set of senior leaders.

Over time, we developed a mission statement - “To be the catalyst in communities of customers, contributors, and partners creating better technology the open source way” - and a set of behavioral values - freedom, courage, commitment, and accountability. I found the omission of anything about software freedom is telling. My sense has always been that we are reluctant to open a potentially divisive debate on this, perhaps indicating a nervousness that there is an underlying and unspoken division between community-oriented technical leaders and business-oriented executive leaders.

I happened to be in the Westford office when Oracle announced its RHEL rebuild - Unbreakable Linux - in 2006. Sentiments such as “With this move, Oracle simply rips off Red Hat's mind-share, while promising a cheaper price” were widely shared. There was a sense of existential threat, and a need to rally and respond that RHEL was “Unfakeable Linux”. I 100% bought into this, but I also wondered… isn't this software freedom in action? If our customers aren’t paying for the bits, what's the problem? And yet we did have a problem with this move. It stung. It was unfair. It wasn't in the spirit of this industry movement we thought we were all a part of. And so, it seemed pretty clear to me - this sort of software freedom was not one of our values, and if we would have no qualms about limiting this particular freedom.

Some five years later in 2011, I was involved in deciding which license we would use for an exciting new open-source project we were launching based on a codebase from an acquisition. Given a blank slate in terms of license choice, we spent some time discussing the merits of copyleft vs permissive licenses. In that discussion, we seemed to have no interest in preventing any downstream user from incorporating the codebase into a proprietary product - that was fine by us. But we did agonize over the best strategy for building a community, a community where we might hope one day to not be the largest contributor. We chose the Apache License (v2), and the discussion made something clear to me - we saw no “moral imperative” around software freedom, but we were absolutely committed to building in the open, and forming or joining healthy communities - “the open source way” was an core, unshakeable value.

And so I'm glad we're openly confronting our values on this now, even if this move disappoints any of our friends in the open-source world who have different - but hugely overlapping - values. My personal view is that we are staying true to our values even as we limit the redistribution of our products - while respecting the applicable upstream licenses.

Especially when it is in response to a move - by those software vendors or larger enterprise users who do have the resources to pay for RHEL or to create their own derivative of a community distro - to benefit or profit from the value of RHEL without paying Red Hat.

However, Red Hat remains exceptionally committed to the open-source development model, following the principle of "upstream first", developing our products in the open, building communities, and embracing the diversity of users downstream of any of these open-source projects. CentOS Stream - the recently created upstream of RHEL, a project and community that we now use to develop RHEL - shows an additional level of commitment to open-source community building that I would not have predicted 10 years ago when RHEL development (then downstream of Fedora) was entirely behind Red Hat’s firewall.

I can understand the negative reaction to this change. However, a lot of the commentary is pretty muddled, and I would be happy to see a deeper discussion of open-source values. I think that Red Hatters should be proud of our values, and embrace the debate that we have now triggered.

On March 17, shortly after schools and creches closed in Ireland in an effort to contain Covid-19, I wrote a post about how my wife and I were attempting to adjust to cope with the need to fit full-time childcare into our already busy schedules.

After the first 30 days (by mid-April), we needed a serious reset of our nutrition, exercise, and sleep habits. This stopped the sense of our lives spiralling out of control. However, the Irish government's May 1 announcement of its "roadmap to easing Covid-19 restrictions" put the whole situation in a new light - we potentially had to continue adapting to a lack of childcare support for months to come.

As we spent the weekend thinking through our situation, it was clear to us that the previous plan - working 6 days per week, sharing full-time childcare - just wasn't sustainable. And, as a business owner, my wife was under especially intense pressure. Even though I also had a lot going in work, with somewhat of a change in role in progress, it seemed like the right time for me to take some time off. Especially when we imagined looking back on this period in 5 years from now. We decided we must somehow carve out some more time to focus on our family.

In Ireland, we have a Parental leave entitlement - you can take up 22 weeks unpaid leave for each child before they are 12 years old. And so, working with my manager, I put together a plan for me to take 2 weeks leave in every 3 week period until mid August. We spent some time discussing how to fully offload my responsibilities and set things up so I could effectively contribute on my weeks working. I put together a document with a detailed plan including my schedule for the months ahead, and shared it with close colleages. Everyone was super supportive. My out of office would read "out of office, returning on August 17th" but those close colleagues would understand that I would be returning to work regularly, looking for ways to contribute.

By the start of June, I had completed my first 2 week period of leave and 1 super effective week back at work. The plan was working! And so too was Ireland's containment of Covid-19 working! We were due to move to "Phase 2 plus" on June 8! I took this as sign that we could start thinking about re-hiring a nanny for the kids and, sure enough, we found someone suitable and very local quite quickly.

This meant that one month after turning on my out-of-office, and 2 months before I was due to return to work full-time, I somewhat embarassingly was able to revert the parental leave plan and life at home was able to return more-or-less to normal. And life has been manageable since then.

I decided I should write this update because it's clear that many others are struggling to understand how to make their situation sustainable where - elsewhere in the world - they may be facing the prospect of a return to lockdown restrictions and schools/creches not reopening for months to come.

There are no easy answers to the conundrum of how to sustainably look after young children while continuing to work full time. Extended periods of unpaid leave is a luxury that is not available to everyone. Employing a fulltime childcare worker in your home is a luxury that is not available to everyone, or may pose too great an infection risk for some.

My one piece of advice - use a long-term view to put your situation in perspective, and be honest with yourself and your colleagues about how sustainable that situation really is. Certainly, at Red Hat, I felt confident that everyone around me would instinctively understand the need to think this way - no matter how pressing our short-term priorities might be, none of us would want a colleague to put their family under undue stress, damage their health, or risk burning themselves out. If you work in a healthy organization, then you can be sure that the organization can adjust quickly to your absence if needs be. Trust this fact, lean on it, and talk to your manager about how you can prioritize yourself and your family.

I've seen all sorts of humor, tips, and ideas from folks as we all learn to cope with these strange times, but somehow I don't feel like I've had a lot of insight into the practicalities of family life for folks in a similar situation to us. So, five days into our new situation, I thought I'd share.

Who are we? We're a family of two adults with full-time jobs, two young girls (9yo and 4yo), and a recent addition of a 4 month old Miniature Schnauzer. We live in Dublin, in the coastal village of Raheny.

Usually, our kids are looked after Monday-Friday between 8.30am and 6.30pm each day (school, creche, childminder at home) and we have some outside help with housework. Lucky us! Starting Friday, March 13, we're learning to cope without any of that help, while continuing to work our two busy and responsible jobs from home.

So, last Thursday evening, we found ourselves in front of a whiteboard trying to come up with a plan. Our basic idea was:

• We needed a predictable daily routine to keep everyone sane
• We'd experiment with keeping that routine 7 days per week, and not adjust much at weekends - this was to avoid putting ourselves on too much daily work pressure with a strict 5 day work week
• Each of us would get a uninterrupted chunk of 4-5 hours work time every day, either in the morning or afternoon
• One parent at a time looking after the kids, not attempting to get any work done at the same time
• Start every day with one of us leaving the house by 9.30am with the kids and dog, to visit any one of the local parks and beaches
• Early to bed, early to rise - no compromising on sleep!
• We'd carve out uninterrupted time alone for short at-home workouts

The key question each evening for the next day is which of is working in the afternoon (Parent A), and which of us is working in the morning (Parent B). With that decided, we know the routine.

	Parent A	Parent B
6.00	Wake up, coffee, email
6.30
7.00
7.30	Kids up, prep breakfast	Work
8.00	Breakfast together
8.30	Get everyone ready, housework	Work
9.00
9.30	Adventure with kids and dog at a park or beach
10.00
10.30
11.00
11.30	Prep lunch
12.00	Lunch together
12.30	Work	Housework, free time for kids, ideally 4yo naps!
13.00
13.30		School time
14.00
14.30
15.00
15.30		Arts & Crafts
16.00
16.30
17.00		Prep dinner
17.30	Dinner together
18.00	Housework	Walk dog
18.30	Family play time
19.00	Kids: screen time, and off to bed Parents: email on the couch?
19.30
20.00
20.30
21.00	National evening news
21.30	Chillax
22.00	Bed!

This has been working reasonably well, with some caveats:

• The "uninterrupted" chunk of work time is your main opportunity to do anything that would be difficult to do with the kids - e.g. workouts, grocery shopping, etc.
• We're not robots, so the schedule does flex - e.g. the snooze button is as tempting as ever, and I'm probably playing too much Minecraft with my eldest!
• It can be hard to organize all work meetings in your chunk of work time, so we do adjust a bit for super important meetings that don't fit
• It's only day five!

Reading back through this, it's clear this will be quite a grind if it continues for more than a few weeks. But we have so much to be grateful for, and there will be many others with far worse to deal with. Take care of yourselves!

As part of switching gears at work from a hectic management role back to an "individual contributor" role as a software engineer, I've read a couple of books based on recommendations from friends. First, It Doesn't Have to be Crazy at work by Jason Fried and David Heinemeier Hansson (thanks Paschal!) and, second, Deep Work by Cal Newport (thanks Fred!). Below is my attempt to summarize some key points I'm taking away from these books, if only to prove to myself that I was concentrating!

Principles

Why "deep work"?

• The ability to quickly master hard things, and to perform at an elite level in terms of qulaity and speed, will set you apart

On choosing what to work on:

• Deliberately work on getting better at picking what to do
• Don't simply work on things that are easiest in the moment
• Understanding what really matters - embrace "good enough" for the less important things
• Drucker - "There is nothing so useless as doing efficiently that which should not be done at all"

On achieving intense focus:

• The rate that high quality work is produced is proportional to the intensity of your focus - or "Work Accomplished = Time Spent x Intensity"
• Aim for "Rooseveltian" style "blistering intensity"
• You can't achieve such intensity with "residue slathering interruptions" from constant context switching
• Spend your time and attention "in large bills, not in small change"
• Don't let your hours become fractured with "calendar tetris" (see also "Maker's schedule")
• Careful management of your attention as the key to a happy and fulfilling life
• "Ruthlessly culling the shallow and painstakingly cultivating the intensity of my depth"

On a cadence for your work:

• Work on projects in iterations - go deep, then take time to decompress
• Finish what you started, and let new ideas wait

Embrace real downtime:

• Downtime aids insights - Unconscious Thought Theory (UTT) establishes that some decisions are best left to your unconscious mind
• Downtime helps recharge the energy needed for deep work - Attentional Restoration Theory (ART) establishes that - like willpower - we have a limited store of "directed attention" that needs to be recharged
• The work that would replace downtime tends to be shallow and unimportant - if you have the capacity for, say, a maximum of 4 hours per day of intense concentration, you're not going to squeeze more focused time out of downtime

On the type of co-worker you aspire to being:

• Aim to leave a lasting positive impression on people
• Be a good person that others can rely on and enjoy working with
• Set an example for others to follow

On living a good life though depth:

• Aspire to a craftsman's work ethic - finding a connection between deep work and meaning in your life, as a path to living a good life
• Medieval quarry workers creed from The Pragmatic Programmer
• "We who cut mere stones must always be envisioning cathedrals"
• The best moments occur when a person's body or mind is stretched to its limit to achieve something meaningful
• Humans are a species that flourishes in depth and wallows in shallowness

Tactics

Schedule your time:

• Schedule large blocks of time for deep work
• Allow the schedule to flex - the goal is not coerce your work into a rigid schedule, but rather to be thoughtful about what you are spending your time on

Work environment:

• Work in a private office, and close the door when working deeply
• Or, in an open office, try to set the expectation of "library rules"

Work deeply:

• Develop your own philosophy for integrating deep work in your life - Newport describes "monastic", "bi-modal", "rhythmic", and "journalistic" styles
• Add routines and rituals that minimize the willpower required to enter a state of unbroken concentration
• Consider the use of "grand gestures" to help motivate you into deeper intensity of focus - I loved the story of JK Rowling finishing the last Harry Potter book in an extravagant suite in The Balmoral
• Don't work alone, but separate the pursuit of serendipitous encounters from your efforts to think deeply
• Try self-imposing an extremely tight deadline for some projects to force yourself into working with "blistering intensity"

The 4 disciplines of execution (4DX):

• Focus on the wildly important - a small number of ambitious goals
• Act on "lead measures" - new behaviours that will drive success, ultimately reflected in "lag measures"
• Keep a compelling score card - a visualization of your progress
• Establish a cadence of accountability - e.g. a weekly or monthly review where you examine your progress and make a plan for the coming weeks
• Compare 4DX to OKRs to SMART goals

Embracing downtime:

• Add a "shutdown complete" ritual to end your workday
• Put a hard constraint on your work day - e.g. finish consistently at 5.30pm

Managing distraction, and thoughts on social media:

• Make a daily mental practice of weaning your mind from a dependence on distraction - got a free moment? Don't pick up your phone, essentially begging it to distract you
• Take a complete break from social media (or "distracting services", more generally) and only re-enable them as you find you need them
• Avoid taking an "any benefit" approach to how you consider the value of these services - consider the opportunity cost, what else you could be doing instead
• Schedule the use of these distracting services
• Networking tools are just tools - they can be used to enhance your professional work, but you should assess their impact in terms of a small number of goals and activities in the vein of "the vital few" or the 80/20 Pareto principle

Meditate productively:

• Try bringing your attention back again and again to a pressing problem as you are occupied physically - this requires practice
• Avoid distractions from the problem at hand, and avoid looping on information you already know

Structure your deep thinking:

• Identify the variables, identify the next step question, and then consolidate your gains

Set yourself a "shallow work budget":

• What proportion of your time is acceptable to spend on "shallow work"? 20%? 50%? 80%?
• This gives you a heuristic for saying "no" to shallow work
• Evaluate the shallowness of your activities by how long it would take to train a bright, recent college graduate to do it
• You should aim to reduce the amount of time spent not using your expertise

Deadlines/projects:

• Fixed deadlines, but allow the scope to be reduced - "narrow as you go"
• Break big projects up into smaller chunks - "scope hammer"
• Use time budgets, not time estimates - "what's the best widget we can build in 2 weeks", not "how long will it take to build a widget?"

Instant messaging:

• Realtime sometimes, async most of the time
• If it's important, slow down
• If everyone needs to see it, write it down

Become hard to reach:

• Make people who send you email do more work
• Avoid making it easy for people to rob big chunks of your time with little investment of their time
• "Process-centric approach to email" - with your reply, consider what will bring the project represented by the email to a conclusion. Avoid bouncing back and forth
• Develop the habit of accepting that small, bad things happen, for example if you don't reply to email. If you don't allow small, bad things to happen, then you won't leave room for good, big things

In Summary

Reject crazy, distracted, and shallow. Embrace calm, focused, and deep work.

Deep work is important simply because it enables you to get useful things done.

"I'll live the focused life because it's the best kind there is".

Disclaimer: Relative to my expertise in something like OpenStack, or relative to the expertise of folks who are working daily on driving advancements in the fundamentals of Blockchain and related technologies, I know next to nothing about Blockchain. However, relative to the average member of the public, I believe I have enough understanding as a technologist to help demystify the subject for other non-experts.

The context for this post is that I attended the recent Climate Innovation Summit in Dublin, which primarily focused on the challenge of financing solutions in the area of Climate Change mitigation and adaptation. To my surprise, the subject of Blockchain seemed to be the main pure technology topic that came up again and again.

Before this event, and without exploring the area too deeply, I had a suspicion that Blockchain is all too often "a solution in search of a problem". This all too common tendency of technologists makes me grumble at the best of times, but when the subject is a technology that is impenetrable to many people, and the problem space is what I believe to be an existential threat to our way of life ... I have grave reservations, to put it mildly.

Much credit goes to EIT Climate-KIC for publishing a report titled Distributed Ledger Technology for Climate Action Assessment which I will refer to heavily below. In large part, my goal here is to highlight the key points and conclusions I took away from that report.

What Problem Does Blockchain Solve?

Blockchain is designed to be an alternative to centralized systems. What does that mean? Well imagine:

• A database with no central administrator
• A currency with no central bank
• A contract that is enforceable without recourse to a judiciary

and you start to get a sense of this problem space. This is quite appealing, on the face of it. A decentralized system should be more fault tolerant and attack resistant because there is no single point of failure, and it should also be more trustworthy and resistant to abuses of power because of the transparency it offers.

And then there's the idea of "social scalability", as researcher Nick Szabo is quoted in the Climate-KIC report:

Scaling human traditional institutions in a reliable and secure manner requires increasing [the number of] accountants, lawyers, regulators, and police, along with the increase in bureaucracy, risk, and stress that such institutions entail."

Solving problems without needing any more lawyers? Hurrah!

Why is Decentralization a Hard Problem?

The Climate-KIC report does a nice job of explaining the 5 components of this solution to the decentralization challenge:

1. Cryptography - public key encryption and hashing functions allows protecting the privacy, integrity, and even the anonymity of data in the system. This is familiar technology used in secure websites with HTTPS, for example.
2. Hash tree - this is simply a list of records whose integrity is protected by a cryptographic hash on each record which combines the hash of the previous record, a timestamp, and the transaction data. This is a well-understood concept that is used to good effect in systems like the Git Version Control System (VCS).
3. Peer-to-peer networks - this is allows you to build resilient storage by having peers in a network replicate data to other peers. Heard of Napster and Bittorrent?
4. Consensus mechanism - how can you authenticate and validate a transaction in a peer-to-peer network without appealing to a central authority? Take a look at the Byzantine General's Problem. It's difficult.
5. Sybil control mechanism - how do you protect against an actor in the system generating a large number of fake identities, known as a Sybil attack. This too is hard. Bitcoin's answer is to make it prohibitively expense to abuse the system this way, with its "Proof of Work" system known as mining.

The first three components - cryptography, hash trees, and peer-to-peer networks - are well established, and it would be possible to use them to build an effective, highly-distributed database with the aid of a central authority (to e.g. decide who can be trusted to write to the database).

However, to make that database fully decentralized - especially with anonymous actors writing to the database - some really difficult challenges are introduced, and that's where the consensus and Sybil control mechanisms come in. These are much more challenging problems, and Blockchain is one relatively recent foray into this area.

Note - in the report, the "Do you need a Blockchain?" flowchart highlights the idea of a "Permissioned Blockchain" suitable for a case where all actors are known but untrusted. That does eliminate some of the difficult problems in this space, but catering for untrusted writers is where much of the complexity remains.

Note also - the report also references an excellent article called The Truth About Smart Contracts by Jimmy Song which highlights the challenge with attempting to specify and enforce the rules of a contract using (potentially buggy) computer code without allowing for appeals to a central authority to interpret the "spirit" of the contract. And also the challenges of attempting to link such decentralized contracts with the physical world.

Blockchain Inefficiency

Given the challenges that full decentralization brings, it's not surprising that Blockchain has some drawbacks. To quote some of the points that jumped out at me in the Climate-KIC report:

DLT is a highly inefficient database technology that is up to 1 million times less efficient than a centralised database, which in turn leads to much higher energy consumption and GHG emissions.

Given that we're talking about Blockchain in the context of Climate Change mitigation, that seems important to consider!

the trade-off between low energy efficiency and higher levels of decentralisation is the key question in the DLT for climate action ecosystem and needs to be addressed for each potential DLT solution

Blockchain's inefficiency and higher emissions should be weighed up against a need for decentralization.

To determine whether DLT is the right tool to solve a given problem, it should be validated that DLT is the only solution to the given problem. If DLT is not the only solution, also a more efficient centralised database can solve the problem.

In other words, you should only consider Blockchain if you've already eliminated the possibility that a centralized system could be sufficient.

A more entertaining take on whether Blockchain is a good fit for your application, from James Mickens:

How Important is Decentralization To Addressing Climate Change?

This is the key point. As Kirsten Dunlop said in her keynote at the Climate Innovation Summit:

We have 12 years to set in place profound structural change in almost every system of cause and effect in our society

Some of "Climate Action" use cases and examples for Blockchain listed in the Climate-KIC report includes:

1. Energy - peer-to-peer energy trading
2. Supply chain management - reduce paperwork, fraud, and errors
3. Carbon trading - a more transparent carbon marketplace
4. Transportation - decentralized ride sharing
5. Open government - increased transparency and accountability in government
6. Measurement Reporting and Verification (MRV) - more transparency in carbon tracking
7. Finance - new ways of financing climate projects

More or less implied here is that the problems associated with centralized systems are on the critical path to effectively addressing these use cases.

The Climate-KIC report says:

While climate change is a truly global problem, it is well recognised that it requires a decentralised, multi-stakeholder, bottom-up approach to be solved

And that's hard to argue with! I'm a big proponent of bottom-up innovation guided by a unifying mission, and certainly such a huge problem space can't be tackled top-down. Mariana Mazzucato's report on Mission-oriented research & innovation in the European Union looks like an excellent framework. And when it comes to Climate Change, we are all stakeholders.

But ... does that really imply that the technology platforms we use must be decentralized? For example, a crowd-funding platform hosted by a single legal entity can still be effective, without itself being decentralized. Can a sufficiently high level of trust and transparency be achieved without fully decentralizing the platform? I believe that is often the case.

To quote the Climate-KIC report again:

DLT’s main power lies in decentralisation. It currently is unclear how the physical centralised world can be decentralised. [...] As many climate action solutions are more valuable when synchronised with the physical world, this barrier is a key barrier to overcome.

Decentralization and anonymity brings up some pretty fundamental questions about how human society is organized. And undoubtedly, tackling Climate Change effectively is going to require fundamental changes in our society. As Naomi Klein put it, This Changes Everything.

But are the problems that Blockchain claims to solve really the key problems getting in the way of effective Climate Change mitigation and adaptation efforts? Even if they are, are there other "good enough" solutions without the drawbacks of Blockchain that can be deployed more rapidly?

What I really worry about here is the danger that Climate Change will be perceived by some as a smokescreen for people pushing pre-existing agendas that aren't strictly related to the challenges posed by Climate Change. To take a more extreme example, how likely are we to mobilize the sort of action needed if people sense they are also being asked to buy into Anarchism as a political philosophy?

Conclusion

There is no doubt that Blockchain is a super interesting technology, and it has opened the door to exciting advances in some truly difficult computer science problems.

If you are motivated to explore how technology can be used to move towards a more decentralized society, by all means you should go down the Blockchain rabbit hole!

However, if you are primarily motivated to rapidly implement Climate Change mitigation and adaptation solutions in the real world, I would suggest that you can safely focus your limited resources on technologies other than Blockchain.