Monorepo Practice

Monorepo Practice

Background

When dealing with large projects containing several shared libraries, it’s usually tricky to contribute code across multiple repositories. Things even become harder when you have to handle both public and private repositories that have dependent relationships, or those dependent repositories are on different platforms. Below is a solution based on the Monorepo approach and git subrepo, which is trying to make it easier to manipulate across multiple repositories.

Convention

Folder Structure

In order to keep consistent with other Monorepos, it’s recommended to follow the designated folder structure.

.
├── LICENSE
├── README.md
├── lerna.json
├── package.json
├── packages
│   ├── main-repo
│   ├── ringcentral-integration
│   └── ringcentral-widgets
└── yarn.lock

All subrepos should be placed in pakages folder, including the original mainrepo. In this way, it’s much more straightforward to manage.

Git Subrepo

It’s highly recommended to use git subrepo to take control of your subrepo. git-subrepo is an open source tool, which leverages git hash objects to manage subrepos. It’s much easier to getting along with than git subtree, if follow workflows below, you will never run into unexpected circumstances.

Lerna

Since there will be serveral subrepo projects inside current main repo, it becomes harder to operate across subrepos, here is where Lerna comes into play. By using Lerna, you can easily run any shell commands and npm scripts in subrepos, Lerna also presents bunch of useful features that can help you out.

Setup

  1. Install git subrepo, if on Mac, you can install git-subrepo by running
    brew install git-subrepo

Otherwise, follow installation instructions on Git Subrep docs.

  1. Clone remote project into subfolder, run
    git subrepo clone <remote-url> packages/<subdir>

  2. Move files in main repo to a subfolder.

  3. Put configuration files needed in the root folder, update legacy configurations, e.g. .gitignore.
  4. Set up Lerna, create bunch of npm scripts to run boostrap, build, test etc.
  5. Set up development env by using alias, so that you can develop directly across multiple projects.
  6. Finally, we get there!

Workflow

After everything is set up, it’s now ready to get to day-to-day work. Since we have to manipulate several subrepos coordinately, the git flow would be a bit tricky, so the following graph shows a recommended subrepo workflow:

Monorepo Structure

This is an overview of Repository Relationships and overall action path, the graph below describes 3 action paths:

Monorepo Flow

Before Sprint

You usually need to pull changes from subrepo remote to keep subrepo up to date. Follow Pull Subrepo instruction.

During Sprint

  • Daily work
    Follow Contribute instruction.

  • Sync up with remote subrepo
    Sometimes you want to pull or push subrepo changes to perform special operations, in this case, you can follow Pull Subrepo and Push Subrepo instruction.

After Sprint

Push subrepo changes to subrepo remote. Follow Push Subrepo instruction.

Note: Subrepo push is a bit different, you need to use git subrepo push <subdir> -r <remote> -b <remote-branch> to push commits to your own fork instead of pushing directly to remote main repo.

Precautions

  • Do pull from remote repo after pushed to ensure tree hash is exactly the same as remote one, this is really essential, otherwise next time when you do git subrepo push, git subrepo can not find correct commit ancestor, which will mess up your commits by including too many legacy commits.
  • Do NOT merge code to main repo when someone is syncing code with subrepo remote, otherwise it will cause git-subrepo to find a wrong common ancestors, which will include dirty commits next time you push.
  • Do ensure that the tree hash is correct after pull from remote repo. You can use git subrepo branch to check the tree hash.
  • Better Do NOT pull from remote when current branch is not clean, because it will make tree hash different from remote at this point and a bit hard to figure out what’s going on here, but it won’t actually prevent you from pushing code back. The recommended way to pull from subrepo remote is to do it in a new clean branch.

References