Andrey Petrov (@shazow) spoke at the Sourcegraph office about lessons learned from his successes and “many failures” (his phrase) creating open source projects. Andrey is author of the popular urllib3 library (explore urllib3 on Sourcegraph) and several other popular open source libraries. During the day, he works at Briefmetrics and he’s also a YC alum, ex-Googler, and a cat person. He writes mostly in Python, JavaScript, and Go.
There are many benefits to open-sourcing your code. Here are just a few:
you've invested a lot of time and effort into your code. Open-sourcing it makes it available and reusable. Employees come and go, as do companies, but open source code will always be available to you and others who find it useful.
Open-sourcing has many benefits, but there's a lot of open source code out there already. How do you make your project stand out? Here’s Andrey’s 4 steps you can use to build great open source software.
Read on for more, or watch the video and slides.
Andrey is the creator of urllib3, a HTTP library written in Python that is used by millions of people everyday, has 785 stars on GitHub, and has been growing since its creation in 2008. Andrey has another project, ssh-chat, that has 1560 stars on GitHub, but is only used by ~15 people today. So which project is more “successful?” In order to answer that question, Andrey described how defining success at the beginning of project will help you understand if it’s successful or not.
The beginning of every open source project looks like this:
The story of urllib3 begins with the problem of trying to upload billions of images to S3 in 2008. Using existing Python libraries, this would have taken 3+ weeks because there was no good concurrency support. You could use s3funnel, a multithreaded S3 client, but managing threads is painful. You could use a worker pool in s3funnel, instead, but existing HTTP libraries didn't re-use connections and most solutions weren't threadsafe or lacked multipart (filepost) encoding. Enter urllib3.
In Andrey’s words, “The solution space [in engineering] is about building something and solving a problem. For urllib3, the space was really small and the solution to other things were really useful, and that’s what made it successful.”
When defining a project’s goals, think about how solving a small problem can make a large impact.
Another form of “success” in open source is exemplified by ssh-chat, an encrypted chat-over-SSH program written in Go. Andrey started ssh-chat as a weekend project. One of the main factors in the project’s success was having a thorough README from the start. As Andrey puts it, “Having a great README is basically 80% of the work to success. You need to be able to answer three questions for your contributors: who else uses it, for what, and where I can get more help?”
But obviously, just creating an awesome README wasn’t enough to ensure the impressive growth of ssh-chat. To make it thrive, it needed more traction.
“If you build it… they may not come” should be the rallying cry of all open source work. As proof of this, Andrey embedded a bounty in the code of one of his projects, promising $5 to anyone who issued a simple pull request to remove the file. As of today, no one has yet claimed the free money.
To spread the word about ssh-chat, Andrey started asking for help, asking for improvements, and finding ways to bring more people into the project. In order to help build interest, Andrey reached out to people on Twitter and offered free Go programming lessons in exchange for opening pull requests. This overcame the initial inertia of getting a few people involved and interested in the project. These early contributors eventually became champions of ssh-chat and started answering questions on Stack Overflow.
Another component of building and maintaining open source projects is learning to be inclusive. “Accepting pull requests very generously and very graciously” was a key step toward building more community interest and is something that many open source authors miss in the beginning. And asking specific people to make a pull request was much more effective than making more generalized asks of his Twitter followers.
In general, programmers tend not to be self-promoting types, so actively marketing and promoting an open source project doesn’t come naturally. With this in mind, Andrey identified ways that programmers can promote their projects without being self-aggrandizing.
One approach is to write interesting blog posts about your projects to provide clearer context of the project’s story and mission. Medium is a great platform for sharing in the technology community and has more potential readers than a self-hosted blog.
Other opportunities for connecting with contributors that worked for urllib3 and ssh-chat included:
“We want to be like Docker!” is a common refrain you hear nowadays. Well don’t we all?
Two things helped Docker develop into a widely adopted open source project:
One of the key insights behind Docker’s success is that they didn't over-focus on the business model until the OSS project was insanely popular. So how can you replicate this? Enter the “corp’en source model” — where en source work becomes the core competency and business of a corporation.
The Corp’en source model
Some of the most well-known companies that embrace corp-en source include:
So how do you monetize your contributions as a free agent?
Well, you don’t. Not directly, at least.
Despite widespread adoption of urllib3, Andrey shared that he only received $5 from a single user (which he appreciates!), proving that widespread adoption of a project does not necessarily provide a sustainable revenue model to justify the time and effort.
Some companies, like Stripe and Sourcegraph are sponsoring open source work.
Open source is incredibly valuable and rewarding, but to do it successfully, you gotta hustle.
In this case, much of the work that went into building urlib3 meant focusing on a small, but common challenge, writing a great README, reaching out to people to contribute, and promoting the project through blog posts. It’s also worth mentioning that it took some time for each of these projects to gain traction and Andrey’s commitment to improving the project over time helped build his reputation and respect within the programming community.
In short, here’s how to make an open source project thrive:
For more from Andrey, follow him @shazow on Twitter or check out his GitHub profile shazow, and be sure to check out his awesome projects, urllib3 and ssh-chat.
Special thanks to Andrey and his wife, Tracy Osborn, for making the long drive to give this talk! Check out Tracy’s book on Django development, Hello Web App.
