Development at PKB: managing your environment in a remote team

This post is part of a series about software development at Patients Know Best.

Our company is fully distributed, which means we don’t have an office. We are working from various parts of the world — at home, in co-working spaces, sometimes on the beach (be careful with the sand and the heat though). This approach has many benefits as you can imagine, but it also raises some questions: how can “enterprise” software development work without an office LAN, through domestic internet connections and without dedicated on-site technical support staff.

Goals

We at PKB would like to

• maximize developer freedom — everyone has different preferences when it comes to tools, OSs and so on
• decrease time needed for new devs to come on board and be productive
• make debugging easy — if something seems to be broken somewhere then it should be easy to reproduce and diagnose it elsewhere
• enable everyone to work anytime, anywhere

What you absolutely need to have

We have pinned down a couple of things that a developer needs to write code for PKB:

• Docker — you should be able to pull images and run containers
• git — we use GitHub, so it’s kind of difficult to get the code without git 🙂
• JDK — in theory one could get away without this by running javac in a container, but we’re not there yet (and this is not practical if you are on MacOS or Windows)
• 8 GB RAM available to Docker
• disk encryption turned on just in case you lose your device

That’s all.

What you don’t need to have

Maybe even more interesting is stuff that is not required.

• You can run any OS you prefer. Half of the team is using Linux, some people use Docker for Windows or Docker for Mac or even roll their own Linux VM under VirtualBox.
• You can use any hardware that has at least 16G RAM and an SSD. Some of us work on self-built water-cooled desktops, others use subnotebooks or Macbook Pros.
• IDE of your choice, though most of us work with IntelliJ and VS Code.
• You don’t need an always-on internet connection. Yes, you heard that correctly — it is possible to run a replica of our whole system including all the microservices on a laptop, develop locally, run automated end-to-end and integration tests or verify your changes manually when you’re on an plane. A good network connection is important for other reasons — communicating with the rest of the team.

The rule is that you can use any setup that is convenient to you — if you have no interest in spending your time supporting an esoteric setup, we’ll point you towards one of the common options, so it’s fast & easy to get quick help from others.. For example, some developers using Docker for Windows struggled for a few days with intermittent problems when running automated tests, because a new Windows upgrade had slightly changed the port reservation strategy of Hyper-V, and that clashed with our test environment.

What is expected from all of us

Working remotely requires proactivity and ability to work independently (not alone!). Installing and setting up your own box should not be a problem — if something breaks we can’t easily give you another machine in a few hours. In some way, all of us are sysadmins of our own PCs, and can solve moderately complicated issues around our own tools.

What makes this possible?

Everything is containerized

Any service, even ad-hoc jobs and database server processes run in Docker containers. Local environments use docker-compose for orchestration (we are actively looking into replacing this with local Docker Swarm installations). Somewhat interestingly containerization is much more important for us from the aspect of development than operations and scaling.

Rich and representative test data

We have various database snapshots to help testing and development — firing up an initialized, fresh system with many registered users, sample data, etc. is just a docker-compose up away.

Sane defaults and supportive culture

Devs have the freedom to choose their tools, but that does not preclude providing advice, support and automation to help each other. New joiners who have come on board in 2018 were able to build and run the full working system on their own PC on their first day.

We always pay extra attention to

• actively unblocking and helping each other, when someone is fighting with a problem that seems to happen only on her box
• keeping the requirements about the dev environments to the minimum. If you are the only developer using Windows, Vagrant and Ubuntu 14.04 then we will try to keep the system and the process working on your setup. (And sadly, in some cases, we’ll say that you need to move on to something else because it would take too much effort and specialization to keep our process simple).

Automated and quick cloud-based test environments

If you have changes that you would like to demo to someone before raising a pull request, you can deploy a full system (including your changes) to a shared cloud-based server with a demo dataset with a single command. DNS, reverse proxying, HTTPS, etc. are all taken care of. You just specify the branch name you are working on, and in a few minutes there will be a complete demo environment with all the (micro)services available on your chosen subdomain.

Stuff that didn’t work or we haven’t implemented yet

Cloud workstations

It would be cool and not that hard to provision a remote workspace on demand so that we don’t need to maintain and use a relatively powerful PC at home. We might implement this in the future, but so far the team doesn’t want it. When this becomes painful is if e.g. someone has a hardware issue that’s hard or impossible to fix and can’t get a replacement quickly. To avoid this we generally recommend that developers buy hardware with on-site, next business day support.

Performance on MacOS and Windows

Using Docker for Windows and Docker for Mac means that you’re probably sharing files between a virtualized Linux instance and the host OS. That’s mostly fine for stuff like front-end development, but it can become annoyingly slow when launching half a dozen microservices.

POSIX permissions on Windows

If you check code out under Windows then you won’t really be able to commit files with proper permissions.

What’s next

In this post we have briefly explained how simplicity and automation can help keep friction in daily development to a minimum in a distributed company. We will continue talking about remote work — explaining how the flexibility of remote work can allow different lifestyles, exploring how we talk about and review code, how do we maintain quality and more.