Twitter github

CNCF Annual Report for 2017 and Kubernetes Graduation

We recently published the first annual report for the Cloud Native Computing Foundation (CNCF) which encompassed our community’s work in 2017:

The CNCF is technically a little over two years old and it was about time we start publishing annual reports based on our progress. This is a well treaded path by other open source foundations out there like the Eclipse Foundation and Mozilla so we thank them for inspiration to be more transparent.

Another thing that we launched this week was the Cloud Native Landscape (interactive edition) and more importantly, the Cloud Native Trailmap which guides you through the journey of becoming cloud native by adopting different projects in the foundation.

Finally, it was fantastic for Kubernetes to be the first project to graduate from the CNCF.  What does this exactly mean? This is very akin to graduation in other open source foundations such as the ASF. Graduation here is really about confidence in CNCF development processes and really a stamp from the CNCF Technical Oversight Committee (TOC) on what is a sustainable, production ready and mature open source project  you can bet your business on. As projects mature in the CNCF in terms of following solid open source governance processes and become widely adopted, expect to see more projects graduating in the future.

Cloud Native and Serverless Landscape

For the last year or so, the CNCF has been exploring the intersection of cloud native and serverless through the CNCF Serverless WG:

As the first artifacts of the working group, we are happy to announce a whitepaper and landscape to bring some clarity to this early and evolving technology space. The CNCF Serverless WG is also working on a draft specification for describing event data in a common way to ease event declaration and delivery, focused on the serverless space. The goal is to eventually present this project to the CNCF TOC to formalize it as an official CNCF project:

We’re still early days, but in my opinion, serverless is one application/programming built on cloud native technology. There are some open source efforts out there for serverless but they tend to be focused on specific projects (e.g., OpenFaaS, kubeless) versus collaboration across cloud providers and startups. The CNCF is looking to enable collaboration/projects in this space that adhere to our values. What are our values? See these from our charter:

  • Fast is better than slow. The foundation enables projects to progress at high velocity to support aggressive adoption by users.
  • Open. The foundation is open and accessible, and operates independently of specific partisan interests. The foundation accepts all contributors based on the merit of their contributions, and the foundation’s technology must be available to all according to open source values. The technical community and its decisions shall be transparent.
  • Fair. The foundation will avoid undue influence, bad behavior or “pay-to-play” decision-making.
  • Strong technical identity. The foundation will achieve and maintain a high degree of its own technical identify that is shared across the projects.
  • Clear boundaries. The foundation shall establish clear goals, and in some cases, what the non-goals of the foundation are to allow projects to effectively co-exist, and to help the ecosystem understand where to focus for new innovation.
  • Scalable. Ability to support all scales of deployment, from small developer centric environments to the scale of enterprises and service providers. This implies that in some deployments some optional components may not be deployed, but the overall design and architecture should still be applicable.
  • Platform agnostic. The specifications developed will not be platform specific such that they can be implemented on a variety of architectures and operating systems.

Anyways, if you’re interested in this space, I highly recommend you attend the CNCF Serverless WG meetings which are public and currently happen on a weekly basis.

Starting an open source program office?

To make good on my new years resolutions of writing more, I recently wrote an article for opensource.com on starting an open source program for your company:

Please check it out and let me know if you have any comments. I’d really like to see us build a future where more companies have formal open source programs, that’s a key path towards making open source sustainable for everyone.

Winding Down an Open Source Project

For 2018, I’ve made a commitment to myself to simply WRITE AND SHARE more. I used to be really good at cranking out posts but I’ve been so heads down in running and building out open source foundations that I’ve neglected sharing what I’ve learned over the years. I recently wrote a post about the process of starting an open source program for your company:

Also yesterday we posted a new open source program guide in the TODO Group about what to do when you unfortunately wind down an open source project:

While some open source foundations have well defined project lifecycles with notions of an “attic” or “archive” – many companies who open source projects generally do not.

Anyways, I hope these articles are useful to you and you learn something new 🙂

KubeCon CloudNativeCon 2017 Takeaways + 2018 Predictions

It was a crazy 2017 for me with 300,000 miles of business travel, but it was all worth it to experience every major cloud provider adopt Kubernetes in some fashion and grow our community to 14 projects total! Also, it was amazing to help host 4000+ people in Austin for KubeCon/CloudNativeCon, where it actually snowed!

I’d like to share some personal take aways I had from the conference (of course with accompanying tweets) that will serve as predictions for 2018:

Exciting Times for Boring Container Infrastructure!

One of the themes from the conference was that the Kubernetes community was working hard to make infrastructure boring. In my humble opinion, Kubernetes becomes something like “POSIX of the cloud” or “Linux of the Cloud” where Kubernetes is solidifying kernel space but the excitement should be in user space.

The Open Container Initiative (OCI) community also held a meeting where it celebrated its v1.0 release to make containers a bit more boring and standardized.

In 2018, look for the boring infrastructure pattern to continue, the OCI community is planning to make distribution a bit more boring via a proposed distribution API specification. I also predict that some of the specialized/boutique cloud providers who haven’t offered Kubernetes integration will do so finally in 2018.

CNCF + KubeCon and CloudNativeCon: Home of Open Infrastructure

CNCF has a community of independently governed projects, as of today which there are 14 of covering all parts of cloud native. There’s Prometheus which integrates beautifully with Kubernetes but also brings modern monitoring practices to environments outside of cloud native land too! There’s Envoy which is a cloud native edge and proxy, that integrates with Kubernetes through projects like Contour or Istio, however, Envoy can be used in any environment where a reverse proxy is used. gRPC is a universal RPC framework that can help you build services that run on Kubernetes or any environment for that matter! There are many other CNCF projects that have use cases outside of just purely a cloud native environment and we will see more of that usage grow over time to help companies in transition to a cloud native world.

In 2018, look for CNCF conferences continue to grow, expand locations (hello China) and truly become the main event for open source infrastructure. In Austin it was incredible to have talks and people from the Ansible to Eclipse to JVM to Openstack to Zephyr communities (and more). I can’t think of any other event that brings together open source infrastructure across all layers of the cloud native landscape.

Moving up the Stack: 2018 is Year of Service Meshes

Service meshes are fairly a new concept (I highly recommend this blog post by Matt Klein if you’re new to the concept) and will become the middleware of the cloud native world. In CNCF, we currently host Envoy and linkerd which helped poineer this space. In 2018, I expect more service mesh related projects to be open sourced along with more participation from traditional networking vendors. We will also see some of the projects in this space to mature with real production usage.

Cloud Native AI + Machine Learning: Kubeflow

In 2018, ML focused workloads and projects will find ways to integrate with Kubernetes to help scale and encourage portability of infrastructure. Just take a look at the kubeflow project which aims to make ML with Kubernetes easy, portable and scalable. Note, this doesn’t mean that AI/ML folks will have to become Kubernetes experts, all this means is that Kubernetes will be powering more AI/ML workloads (and potentially even sharing their existing cloud native infrastructure). I expect more startups to form in this space (see RiseML as an example), look to see a “cloud native” AI movement that focuses on portability of workloads.

Developer Experience Focus and Cloud Native Tooling

One of my favorite keynotes from KubeCon was Brendan Burns speaking about metaparticle.io, a standard library for cloud native applications. I completely agree with his premise that we need to democratize distributed systems development. Not everyone developer needs to know about Kubernetes the same way not every developer needs to understand POSIX. In 2018, we are going to see an explosion of open source “cloud native languages” that will offer multiple approaches to democratizing distributed systems development.

Also in 2018, I expect us to see growth in cloud native development environments (IDEs) to provide better developer experience. As an example, for those that were wondering why there was an Eclipse Foundation booth at KubeCon, they were demoing a technology called Eclipse Che which is a cloud native IDE framework (your workspace is composed of docker/container images). Che is a framework that helps you build Cloud Native IDEs too, for example, OpenShift.io is OpenShift integrated with Che to provide you a fully blown online development experience.

Finally in 2018, I expect the developer experience of installing Kubernetes applications improved, including the underlying technology for doing so. For example, the Service Catalog work and websites like kubeapps.com showcase what is possible in making it easier for people to install Kubernetes app/integrations, we’ll see this grow significantly in 2018. Also I predict that the Helm community will grow faster than it has before.

Diversity and Inclusion

One of my favorite take aways from the conference was the focus on diversity and inclusion within our community:

https://twitter.com/CloudNativeFdn/status/938435795022745600

We (thank you amazing diversity committee) raised $250,000 and helped over 100 diversity scholarship recipients attend KubeCon/CloudNativeCon in Austin. In 2018, I predict and truly hope some other event will match or beat this.

Anyways, after a crazy 2017, I can’t wait to grow our communities in 2018.

 

 

Cloud Native Computing Foundation 2 Years Later

A little over two years ago after five years of service at Twitter, I took the opportunity to build an open source foundation from scratch using some of the computing techniques we experimented with at Twitter:

https://twitter.com/cra/status/937460286210048000

I was initially excited about the idea because of my experience with open source foundations in previous lives, from being involved with the Eclipse Foundation, Linux Foundation, Apache Foundation plus part of the early discussions around OpenStack governance formation. I viewed this as an opportunity to learn from the lessons of other foundations and do something new and modern in the GitHub era, along with of course making our own mistakes. You really don’t get many opportunities to start an open source foundation from scratch that will impact the whole industry.

Stepping back, the original idea behind the Cloud Native Computing Foundation (CNCF) was to promote a method of computing (we call it cloud native) pioneered by internet scale giants such as Google, Twitter, Facebook and so on and bring it to the rest of the industry. If you looked inside these companies, you can see they were running services at scale, packaged in containers, orchestrated by some central system.

The first mission was to provide a neutral home for every major cloud provider supporting Kubernetes natively, kubernetes is truly the lingua franca of the cloud.

We still have a long way to go within CNCF to truly making cloud native computing ubiquitous across the industry, but I’m excited to see so many companies and individuals come together under CNCF to make this happen, especially as we have our largest annual gathering this week, KubeCon/CloudNativeCon. Personally, I’m nothing but thrilled to what the future holds and truly lucky to be serving our community under the auspices of the foundation.

A special thank you to Craig McLuckie, Sarah Novotny, Todd Moore, Ken Owens, Peixin Hou, Doug Davis, Jeffrey Borek, Jonathan Donaldson, Carl Trieloff, Chris Wright and many other folks that were at that CNCF first board meeting two years ago bootstrapping the foundation.

Top 6 Public Cloud Providers in CNCF

Last week, I had the pleasure of welcoming Oracle into the Cloud Native Computing Foundation (CNCF). This now marks the top 6 leading public cloud providers in the world are now part of the CNCF:

This also marks for the first time in the history of our industry that these leading cloud providers are working together in the same open source focused foundation to move the state of the art infrastructure forward.

Also last week I had the opportunity to bring in two new high quality cloud native projects into CNCF. Envoy is a high-performance open source edge and service proxy that makes the network transparent to applications. Jaeger is an open source distributed tracing system inspired by Google Dapper paper and OpenZipkin community. It can be used for tracing microservice-based architectures. Uber began deploying Jaeger internally in 2015. It is now integrated into thousands of microservices and recording thousands of traces every second.

 

Anyways, this is one of the reasons I enjoy working in open source today, bringing together diverse (and even competing) companies to build a better world by collaborating in the open!

Launching TODO Group Guides

In my participation with the TODO Group, we recently published a set of open source guides (under CC-BY-SA 4.0) dedicated to help companies build out open source programs:

In the last year or so, we have seen companies like AWS build out an open source program via @AWSOpen and even companies like VMWare hired their first Chief Open Source Officer.

We’ve had many organizations approach TODO Group members asking for advice on how to get started with an open source program and these guides are a reaction to that. It is our hope that these are living guides and evolve with community input over time.

Shipping OCI v1.0.0

Last month after nearly 2 years, the Open Container Initiative (OCI) community shipped v1.0.0 (see release notes) of two container standard specifications:

Outside of relevant container standards finalized, another thing that was enabled by this v1.0.0 release was an interesting IP Policy (OWFa v1.0) embedded in the OCI charter. Essentially, there is patent non-assertion and royalty free patent grant made by all OCI members against the entire 1.0.0 specification implementations, not just their contributions!

https://twitter.com/cra/status/899668238824591361

This is simply good news for the industry when it comes to what I call establishing an “IP no fly zone” for core container technology, which should spur further container adoption. Also, standardization also helps implementors and tool builders feel safe that things won’t break, which has been a bit of a challenge given how fast container technology has been moving.

Finally, one more thing to highlight is that OCI is also helping sponsor a diversity scholarship at DockerCon EU in Copenhagen:

Please encourage qualified folks to apply, the deadline is September 5th.

AWS Joining CNCF

It’s been a little over a year and a half since I started to help build the Cloud Native Computing Foundation (CNCF) from scratch. One of our original goals was to build a modern open source foundation focused on a new form of cloud computing called “cloud native” (essentially think of microservices that run in containers that are orchestrated) and to get all the major cloud providers at the table to adopt this form of computing.

Last week, we were happy to welcome AWS to CNCF as a member and that now brings us to having the top five cloud providers in the world at the table committing to adopting and promoting cloud native computing:

First off, it’s always great to see an original vision of when we started CNCF come into reality, you can read more from Adrian Cockcroft why they decided to join CNCF and support cloud native computing.

Second, I think it’s great to see a company like Amazon expanding its open source efforts as they where one of the last large companies without a formal open source program. They recently started an official open source program office @AWSOpen under the leadership of Adrian Cockroft and Zaheda Borat and it’s been great to have them participate in the TODO Group too!

Anyways, always great to see large and impactful companies increase their commitment to open source. Now it’s interesting to think what large companies out there don’t have an official open source program or strategy (I’ll leave this as an exercise to the reader).