Co-authors: Tania Allard, Brian Skinn, Dharhas Pothina
We’re excited to announce a new chapter for Nebari, formerly known as QHub, as
it evolves into an independent, community-led project, with a fresh new look!
Since its launch nearly two years ago, Quansight has been incubating and
developing Nebari—a fully open source collaboration platform for data science
and research teams. What started as a proof-of-concept has become a trusted
platform by many teams, both within Quansight and in the broader community.
In this blog post, we share Nebari’s journey, what this milestone involves, and
why it is important to us.
What Is Nebari?
Nebari is a quick-to-setup JupyterHub-based platform, where data science teams
can work and collaborate effectively.
You can set up (both configure and deploy) Nebari for your team with a single
YAML file, on any major public cloud provider or your
local HPC cluster. Nebari has been developed with a
DevOps-first mindset, and it comes with
several tools & integrations pre-configured,
including Dask for scalability, reproducible environments, and features for
sharing dashboards, so that your team can use it out-of-the-box.
So why do we need yet another JupyterHub distribution? Deploying and maintaining
data science and research infrastructure remains a challenging task, especially
for small teams without dedicated infrastructure staff. Nebari aims to remove
some of this deployment overhead while integrating DevOps best practices,
scalable distributed computing, and reliable environment and dependency
management out-of-the-box. We spent the last few years solving these challenges
to bring Nebari to its current state, and we hope the community can build from
this foundation to make Nebari even better!
Company-Backed to Community-Driven
Incubating Nebari at Quansight for the past two years allowed us to iterate on
new features and improve the project quickly. We used Nebari for a number of
client projects and internally at Quansight (as a means of real-life
dogfooding), which helped us catch and fix issues,
experiment with several integrations for a diverse range of use cases, and
configure what we like to call “sane defaults.”
Nebari has been completely open source since day one, though to date it has been
primarily developed by Quansight. We believe it has reached a maturity level
where a community-first approach to Nebari’s development and governance is
essential for its long-term sustainability. This is a core part of Quansight’s
mission of “Building a Sustainable Open Source Ecosystem.”
Governance That Supports Our Community
The first step in moving to a community-driven model is adopting a governance
structure and a license that puts our community first. It’s important to
explicitly state and codify our community values and practices to truly embrace
the spirit of open source software. We are working on a light, yet sustainable
set of policies to foster a healthy community around Nebari.
By doing so, we will enable a diverse set of stakeholders and community
participants to engage with and set the direction for the project. Also, having
a more transparent and community-first governance model will allow us to better
interact with other projects in the open source ecosystem, and ensure
improvements are regularly contributed to the upstream (all those open source
projects upon which Nebari is built on).
We care deeply about accessibility and inclusion: our roadmap highlights
initiatives in these areas, and we have a strong Code of Conduct to ensure these
standards are upheld. We understand that open source contributions go beyond
writing code, and our contributing guidelines accordingly include paths for all
forms of contributions. As one example, we’re documenting and updating our
day-to-day workflows, from decision-making to infrastructure details to allow
contributors to get involved in all aspects of the project. Our policy documents
encourage contributing back to upstream libraries whenever possible, reinforcing
our commitment to the open source community.
We’re grateful for the opportunity to learn from other open source software
(OSS) communities and their governance policies to create a strong foundation
for Nebari’s next chapter. We took great inspiration from the JupyterHub, NumPy,
Kubernetes, Bokeh, and Gatsby JS communities, to name but a few.
With this new chapter, we realized that the name “QHub” no longer captured the
essence of the project. We selected the name “Nebari,” which is a Japanese word
that describes the roots of a Bonsai tree that are visible above the surface of
the soil.
Bonsai is really an art and lifestyle for some people. Growing a Bonsai
requires knowledge, patience, time, and care. It's a long-term relationship
you need to have with your tree. During its long life, you will add features
(pruning, orientation, dead wood, change branch orientations, etc.) and
oversee its growth on a daily basis. All this to have a strong and healthy
tree. I think this is what we can wish for a product, API, etc. Grow, yes, but
with planning/vision/users in mind which all take time to reflect upon. Then
you will have a strong product which corresponds to something, someone.
~ Pamphile Roy, Software Engineer at Quansight, and core developer of SciPy
Nebari captured everything we wanted the project to be: opinionated, yet
flexible, reliable, and ever-growing. Nurturing an OSS project is pretty much
like gardening, so the team loved this name and concept straight away.
The next step was to define a visual identity to capture this concept. Here are
some early concept ideas for the new logo, all of which try to bridge the ideas
of Bonsai and clouds (a nod to cloud computing):
The amazing Irina Fumarel took these ideas and created the
current Nebari logo:
The Nebari colors and fonts were chosen to be accessible
to a wide audience and to stay consistent across various mediums like data
visualizations, documentation themes, graphics, and more. All our design assets
are publicly available at nebari-dev/nebari-design, and we
will soon add the design guidelines and a broader color palette!
How Far We Have Come, and What’s Next
Nebari has come a long way since its official launch in 2020! It currently supports all major
cloud providers: Amazon Web Services (AWS), Google Cloud Provider (GCP), Azure,
and Digital Ocean; as well as local HPC clusters.
In preparation for the rename and change of governance, we spent the last few
months focusing on improving the user and developer experiences. For example, we
did a complete documentation re-write and re-structure. Nebari now follows the
Diátaxis framework, and has a new website built using
Docusaurus and hosted at nebari.dev.
We updated Nebari’s command line interface (CLI) using Typer and Rich for a
better user experience. We also created a new “Guided Init” wizard for new users
to deploy Nebari more quickly and with less friction. You can learn more about
these changes in the blog post
Developing a Typer CLI for Nebari by Asmi Jafar, a
Quansight Labs intern, who worked on this feature.
We improved integrations like Keycloak for better access and resource management
and added new ones like Visual Studio Code and Grafana monitoring for
convenience.
In the coming months, we will continue to focus on Nebari’s community and
governance documents. We want to add more guidelines for maintainers and more
contributing paths that involve little to no code. We are actively improving and
expanding the documentation, and making our integrations more robust.
You can keep up with Nebari’s progress by keeping track of our
detailed roadmap on GitHub!
Join Our Community!
This milestone wouldn’t be possible without numerous contributors who worked on
Nebari over the years, continued support from the Quansight team, and feedback
from the broader open source community. We are immensely grateful to you!
We’ve started hosting bi-weekly meetings to
give all community members, including users, contributors, and maintainers, a
chance to collaborate and share ideas. We invite you to come by and say hi.
Try out Nebari, and let us know how it goes. We look forward to seeing you in
our community spaces!