We have a great team of developers all over the world constantly improving Harvest in the background—optimizing the infrastructure, pushing version updates, tweaking the database, etc. These things can’t easily be shown with flashy animated GIFs like when we release new features, but they’re essential in keeping our products secure, reliable, and fast. In this post I’d like to talk about one of these projects.

I’ve spent a year and a half of my career on this new system we call Harvest ID, and it’s been one of the most rewarding and challenging projects I’ve ever worked on. It’s also a very important step for us technically, a stepping stone that let us clean up a lot of technical debt and paved the way for a lot of features that wouldn’t have been possible without it.

We migrated almost one million users to this new system, we built and improved features where security is absolutely essential, like password resets and invitations, and we had endless discussions about the impact on usability for the various flows that Harvest ID provides. And we did all this without a single second of downtime.

I hope you enjoy this read as much as I enjoyed working on this project.

Authentication is very important. It’s the key that gives you access to doing anything at all with Harvest, and it involves much more than just entering your email and password on a sign-in page.

More than ten years ago, when Harvest was launched, you could sign in using your email address and password. Some time later, we started offering an API that let you pass your credentials with Basic authentication. When the iPhone started to be a thing, we built our first version of Harvest for iPhone. Some time later, we implemented the new standard OAuth2, making third-party apps easier and more secure to build. In 2014 we launched another product, Forecast, which integrates very tightly with Harvest.

All of these different features have one thing in common: they provide access to your Harvest accounts, and that access has privacy and security implications that we take very seriously.

Harvest started with a very simple solution for the needs it had ten years ago, but with time, authentication became more and more complex to manage. This complexity wasn’t intentional; it was the outcome of many years of changes and improvements. This is perfectly normal in the world of software development, where it’s important to not just extend and add new features but also take a step back and clean up anything that has become too complex.

This slide from an internal presentation two years ago made it clear that it was time for us to take that step back (don’t worry—a lot has changed since then!):

Talk Over the Internet – Authentication

Harvest ID

One of the ways we’ve solved these problems in the past is to rebuild small sections from scratch, giving us new, cleaner code. It’s also possible to take this one step further, though, and build a whole new app. This has the added benefit of removing the complexity from the original codebase and having a very small new application that’s a lot easier to maintain.

We unveiled Forecast almost two years ago, when we had already learned from our past and decided that authentication was something that could be done in a different app. We tried out our proof of concept and found that it worked well for Forecast, so the next step was to expand on the idea and get Harvest to use it, too. We named the new app Harvest ID.

Beyond cleaning up our authentication code, we knew that investing in Harvest ID would have many other benefits, some obvious, like easily switching between Harvest and Forecast accounts, and others less visible, like improving how we implement some parts of Harvest that really need some love. We’ll explain some of these more in depth in a future blog post, but security has always been at the forefront of this project, and that’s what I want to dig into now.

Security

Security is a complex concept. No one can ever say with 100% confidence that a system is completely secure; what’s considered secure changes with time, as hackers get craftier and new best practices emerge. Security is a spectrum, and we’re constantly trying to move towards the more secure end of it. Ensuring security involves many different areas and practices, from keeping our servers and packages up to date and storing private information securely to using mathematically proven secure algorithms and nudging our customers towards stronger passwords or informing them when something might be amiss.

With Harvest ID, our products are now several steps closer to the place we’d like to be with regard to security:

  • We’re now based on tokens with limited lifetimes. Someone hacking an account has a limited time during which to do any damage, and we can revoke access at any point.
  • It’s much more flexible than before. We’ll be able to build new features in the future that would’ve been very complicated to build in the days before Harvest ID.
  • Harvest ID is a very small application, which makes it a lot easier to maintain. We can run the whole test suite in ten seconds, as opposed to the ten minutes it takes to run Harvest’s. A small application has fewer bugs and lets us be way more thorough with QA. This is very important when we’re talking about such a key part of a system.
  • Harvest and Forecast can join efforts in this area. Any improvements or vulnerability fixes in our authentication code will automatically apply to both products.
  • There’s a good chunk of functionality that doesn’t live in Harvest anymore—signing in, password resets, invitations, etc. That means Harvest got simpler, and we know simpler apps are more secure, right?

Tokens

Harvest ID is, at its core, a token generator that can work as an OAuth2 provider. When accessing any of our products, we always expect an access token to be in your cookies. If it’s not there, we just tell you to go to Harvest ID and sign in to get a new one.

If you type the correct email and password into Harvest ID, we issue a new access token with a limited lifetime. That token stays in your cookies, signed and encrypted to make it a little bit more annoying for Evil Hackers to play with it, and once it expires, it stops working and you need a new one to access our apps. Signing out also invalidates it, making sure hackers can’t do anything else with it.

Our own mobile apps use Harvest ID as a good old OAuth2 provider. They use a password grant to get a pair of access and refresh tokens, just like they used to, with the added benefit that you can use these tokens for all your accounts, making it easier than ever to switch accounts with our mobile apps—something that wasn’t possible before. Spoiler alert: The Harvest for Mac app will support this very soon, too!

We foresee a future where all API access to Harvest will also benefit from Harvest ID so third-party apps can provide new, better ways to work with Harvest and, eventually, Forecast.

I’m so glad to have been part of this amazing journey. We officially started the project to integrate Harvest ID with Harvest in October 2014 and silently released the first batch of changes internally in early 2015 before slowly migrating all features from Harvest into Harvest ID. We migrated everyone with a single Harvest account during March 2015 and everyone else in the next few months. We re-built most of the sign-up code at the same time, and by the end of 2015, everyone was accessing Harvest via Harvest ID.

We haven’t been idle since then. We’ve continued to improve Harvest ID and will keep doing so in the months to come. I’m really looking forward to the new feature announcements we’ll be making in the near future.

This ambitious project wouldn’t have been possible without the fantastic team here, from my fellow developers to our amazing DevOps, tireless QA, and the nicest team of Experts around. I especially couldn’t have done it without Lorenzo, one of our security experts and an incredible resource and cheerleader during this transition.

Stay tuned for upcoming news about Harvest ID!