Sharpening your axe

By

Originally posted on the LEGO® Engineering Blog

Setting up a developer environment for contributing to the JavaScript Open Source Ecosystem

“Give me six hours to chop down a tree and I will spend the first four sharpening the axe.”

Contributing to an assortment of projects is a non-trivial task.

This post will provide actionable pointers for refining a developer environment with three areas of focus:

[A] Desktop Environment (macOS)

  1. The Search Keyboard Shortcut → Simplicity
  2. Window / Tiling Manager → Speed
  3. Multiple Desktops → Compartmentalisation

[B] Sandbox Environments

  1. The Recipe for a JavaScript Sandbox → Command Line
  2. Additional steps for a React Sandbox → Next.js
  3. Unit tests in a Sandbox → Jest
  4. Codesandbox.io → Sharing reproducible examples

[C] Developer Tools

  1. Git → Leveraging Versioning
  2. Node → Running project with earlier versions of Node
  3. Shell → Writing executable scripts
  4. npm → Inspecting locally compiled binaries

There's no need to try out everything expanded upon in this post, and there's definitely more than one way of setting up an environment than the steps described here. The intention is to share a practical and comprehensive guide on the most useful information that I'm aware of.


[A] Desktop Environment (macOS)

A desktop environment is your interface to interacting with a machine. Creating for yourself an environment which is predictable and joyful to use is a great foundation from which to solve complex problems. If it's an option, I'd personally encourage investing the time into replacing tasks typically done with a mouse in favour of finding the keyboard equivalents, to reduce the cognitive overhead of context switching.

Here is a sample of some of the keyboard-based tools for macOS which I leverage:

I. The Search Keyboard Shortcut → Simplicity

If I could pick just one thing from this article which will speed up working with macOS, it's this:


⌘⇧/

Try it now!

This keyboard shortcut opens up the 'Help' bar in macOS. For example, type 'Zoom' into the Help bar, navigate down with the arrow keys to select the 'Zoom In' menu item. Pressing enter will then execute the option (Esc to quit).

A time where I realised the power of this command was using it to open 'Extensions' and then 'Settings' in VSCode (demonstration below) - the person I was pairing with was a bit taken aback that I was able to accomplish this from the keyboard alone in a very short space of time.

This keyboard shortcut is powerful because it eliminates the need to remember all of the specific, individual keyboard commands or where options live in menus. It's like having a search engine for the functionality which can be accomplished from a keyboard for every single individual program.

II. Window / Tiling Manager → Speed

Navigating, moving and resizing the windows on your machine without using a mouse will significantly increase the speed with which you can interact with a computer.

2020 Update: I've recently switched to Magnets - I found myself wrestling too often with automatic tiling.

I'd recommend an open source window/tiling manager called Amethyst, (similar to xmonad for Linux).

It comes with about 50 keyboard shortcuts; adopting one new command a week has been an incremental and sustainable way to increase the speed of my workflow. Having windows 'snap' into place is a more predictable environment from which to work, and it can even support moving windows between multiple monitors from the keyboard.

III. Multiple Desktops → Compartmentalisation

Compartmentalising different workspaces and environments will reduce your cognitive overhead with regards to context switching, helping to increase the speed with which you can accomplish tasks.



To configure keyboard shortcuts for different desktops, go to System Preferences > Keyboard > Shortcuts > Mission Control and then tick the relevant "Switch to Desktop <numbers>". This allows for jumping and swapping between different environments quickly.



Amethyst also has the functionality to move different workspaces from just the keyboard. The combination of these two tools has created for me a developer environment which I don't have to 'think' about using, in the best possible sense.


[B] Sandbox Environments

Being able to reproduce an error in the source code in a minimal example is a useful skill as a developer. I’ve found that knowing how to create a sandbox environment in the terminal by heart has increased my propensity for being able to iterate on a problem:

I. The Recipe for a JavaScript Sandbox → Command line

This is my recipe for making delicious, home baked JavaScript Sandboxes solely using the command line:

  1. In the terminal, create a blank directory and navigate to it: 
    mkdir sandbox
    cd sandbox
  2. Initialise a git repository within the sandbox directory:
    git init
  3. Create a 
    .gitignore
    . There are some good examples of things to add here - https://github.com/github/gitignore. My minimal 
    .gitignore
    is:
    # Javascript
    node_modules
    .next
    
    # Mac
    .DS_Store
    
  4. Initialise the JavaScript npm project: 
    npm init -y
  5. Install dependencies and dev dependencies: 
    npm install --save <dependency>
    npm install --save-dev <dev-dependency>
  6. If necessary, modify the
    scripts
    in
    package.json
     - for an example, see Step 6 of the II. React Sandbox / Next.js setup
  7. Create an initial git commit with:
    git add -A
    git commit

Bon Appétit!

II. Additional steps for a React Sandbox → Next.js

If you want to use React within your Sandbox, I recommend using the Next.js framework, as it allows for React development with a minimal setup.

To use Next.js for a React Project:

  1. Follow the steps above for creating a JavaScript sandbox
  2. Install Next.js, React and react-dom using the following command in the terminal:
    npm install --save next react react-dom
  3. Create a pages sub-directory within the root directory of the sandbox:
    mkdir pages
  4. Make an
    index.js
    file within the created
    pages
    directory, and populate it with the following contents:
    const Main = () => (
        <div>Main page!</div>
    );
    
    export default Main;
  5. Create any additional pages if necessary. They are going to be available on their corresponding routes: for example,
    index.js
    becomes accessible on
    localhost:3000/
    and
    about.js
    is going to be accessible on
    localhost:3000/about
    .
  6. In the code editor, add the following entries to
    scripts
    in
    package.json
    :
    {
        "scripts": {
          "dev": "next",
          "build": "next build",
          "start": "next start"
        }
      }
    
  7. Start the project in the terminal with
    npm run dev

This is a local React sandbox which is now ready to be iterated upon!

Here's what it looks like in practice:

III. Unit tests in a Sandbox → Jest

Having unit tests set-up within your Sandbox is useful for:

Starting from plain JavaScript sandbox or the React sandbox described above to set-up Jest follow the steps below.

  1. Install Jest testing library:
    npm install --save-dev jest
  2. Create
    sum.js
    file in the root of the project, with the following contents:
    const sum = require('./sum');
    
    test('adds 1 + 2 to equal 3', () => {
        expect(sum(1, 2)).toBe(3);
    });
  3. Create a testing file
    sum.test.js
    in the same directory as
    sum.js
    containing:
    const sum = require('./sum');
    
    test('adds 1 + 2 to equal 3', () => {
        expect(sum(1, 2)).toBe(3);
    });
  4. Add this line to
    scripts
    in
    package.json
    :
    {
      "scripts": {
        "test": "jest"
      }
    }
  5. Run tests in the terminal with:
    npm run test

IV. Codesandbox.io → Sharing reproducible examples

As well as allowing you to iterate on a problem, Sandboxes are a great communication tool for exposing the problem to the other contributors. Submitting a reproducible minimal example / sandbox is often a standard requirement when filing an issue with an open source project.

Codesandbox.io is a fantastic online service for hosting these environments.

For example, an entry point to adding to the conversation in open-source issues may be to create a reproducible example in Codesandbox.io to better explain an issue. Here's an example:

For this issue in the React repository:

https://github.com/facebook/react/issues/13956

The issue poster provided a CodeSandbox example for explaining the behaviour as it currently stands in React. As an entry-point to understanding the issue, I wrote a complimentary CodeSandbox example to illustrate the behaviour in pure HTML/CSS to replicate my understanding of the problem:

and left it as a comment, which may be a starting point for further discussion, or provide another contributor with a reference for solving a subjective issue.


[C] Developer Tools

Developer tools are usually characterised by having many more subcommands and options than developers use in their day-to-day work. However, some of that uncommonly used functionality can be extremely useful, especially when doing open-source work.

I’m going to shine some light on some of these:

I. Git → Leveraging Versioning

A well known feature of Git is jumping between different versions of the project that we're contributing to:

It's also useful to keep track of the changes to the sandbox code (Sandbox Environments are described in further depth above in Section B of this post).

A huge benefit of versioning with Git and having a mindful

.gitignore
setup is that removing
node_modules
and other ephemeral files in the terminal becomes simpler using:

git clean -fxd

In addition

git clean -fxd -- <path>
allows for the removal of untracked files only within a given path.

git stash

Useful for developing and jumping between different versions of the repo if you have temporary changes that you don't want to commit, for example:

Usage:

git stash

"stashes" away the temporary changes, for example, when moving to a different commit. Applying the

-u
flag includes untracked files.

git stash apply

applies the temporary stash

git stash drop

deletes the temporary stash

git stash pop

executes

git stash apply
and
git stash drop
in a single step

Advanced strategies:
  1. Interactive
    git rebase
     
    - helpful for reordering or modifying your commit history in some way i.e. using
    squash
    for maintaining a semantic version history.
  2. git bisect

    - very useful when tracking down regressions.

The number of the commits that we need to check this way is a logarithm of the number of commits between commits marked "good" and "bad", because binary search is used.

In Open Source, bugs whose description contain "this has worked in version X and doesn't work from version Y onwards" have the potential to be solved very quickly using bisect. (additional benefit: you don't need to know (almost) anything about the codebase in order to employ this). Typically these sorts of issues can be found under the label "Needs Investigation", i.e.:

https://github.com/zeit/next.js/issues/7050#issuecomment-491616993

Steps:

  1. git bisect start
  2. git bisect good version-X
  3. git bisect bad version-Y
    # test whether the commit that bisect jumps to is good or bad using your sandbox
  4. git bisect good
     # (if the desired functionality is present in that commit)
  5. git bisect bad
    # (if the functionality is broken in that commit)
  6. (rinse and repeat)

II. Node → Running project with earlier versions of Node

Be cognisant that sometimes it's necessary to run the project with an earlier version of Node. Having multiple versions of Node.js on your local machine is not as hard as you might think. Below is the way that I manage multiple Node versions on my machine manually:

  1. Download appropriate
    tar.gz
    file from https://nodejs.org/dist/ i.e. https://nodejs.org/dist/latest-v9.x/node-v9.11.2-darwin-x64.tar.gz(this is the macOS package for the v9.11.2 version of Node.js)
  2. Unpack it into a well known location on your machine (for this example I'll use the
    Downloads
    directory)
  3. Create a script called
    ~Downloads/node-v9.11.2
    using a code editor of your choice and make the script executable using:
    #!/usr/bin/env bash 

    # node, npm and npx become accessible
    export PATH=~/Downloads/node-v9.11.2-darwin-x64/bin:$PATH

    # all binaries installed with npm install -g become accessible export
    PATH=~/Downloads/node-v9.11.2-darwin-x64-node-modules/bin:$PATH

    # custom path for where to install packages export
    PREFIX=~/Downloads/node-v9.11.2-darwin-x64-node-modules $@
  4. Make the script executable with
    chmod a+x ~/Downloads/node-v9.11.2
  5. To use a completely isolated version of Node, prefix all commands with
    ~/Downloads/node-v9.11.2
    , as shown below:
    ~/Downloads/node-v9.11.2 which node
    ~/Downloads/node-v9.11.2 node --version
    ~/Downloads/node-v9.11.2 which npm
    ~/Downloads/node-v9.11.2 npm install -g yarn
    ~/Downloads/node-v9.11.2 which yarn
    ~/Downloads/node-v9.11.2 npm list -g

Node versions installed using these steps are completely isolated. You can have any number of Node versions installed on your system.

Potentially an easier way way for managing versions of Node is nvm: https://github.com/nvm-sh/nvm.

Combining these two tools - 

git bisect
and installing different versions of Node - enabled me to pinpoint exactly which line change introduced a three year old bug into React:

https://github.com/zeit/next.js/issues/7050#issuecomment-491616993

Whilst it wasn't possible in this instance, due to a large scale refactor of the React codebase in the last three years - it is not out of the bounds of reason that it would be possible to fix a regression within an open-source codebase using these two tools alone, without any prior knowledge of a project.

III. Shell → Writing executable scripts

Writing shell scripts is an endeavour which does not need to be as intimidating as the rumours would have you believe. They're a tool which can help to verify ideas, justify time spent on a particular problem, or automate processes.

In an attempt to demystify the process of writing shell scripts, here's the step-by-step process I follow:

  1. Put all of the commands which I'm writing into the terminal into a script (the filename extension for scripts is 
    .sh
    ).
  2. Useful commands are typically:
    ls
    ,
    pwd
    ,
    cd
    ,
    cat
    ,
    echo
    ,
    curl/httpie
  3. Make the script executable by putting
    #!/usr/bin/env bash
    at the top of the file and running
    chmod a+x <file-name>
    . It's usually good to add comments at the top of the script that document what the script does.
  4. Iterate and add more commands until the terminal outputs what I need.

    For reference, here's an example script I've written as a first step to verify if a problem I'm trying to solve and eventually open source will have a measurable impact on performance:
    const SIZE_OF_STRING_IN_URL = `#!/usr/bin/env bash
    set -eu -o pipefail
    
    # Script expects these binaries to be present:
    
    # - wget
    
    # - node
    
    # Tested on Mac.
    
    # Example:
    
    #
    
    #   bash sizeOfStringInUrl.sh nextjs.org "\bcreateElement\b"
    
    main() {
      url="$1"
      pattern="${2:-\bcreateElement\b}"
    
      if [ -e "$url" ]; then
    
        echo ">> Directory $url already exists... EXITING"
        exit 1
    
      fi
    
      echo ">> Downloading $url"
      mkdir "$url"
    
      cd "$url"
      wget -pk "$url"
    
      count_occurrences=$(find . -name "*.js" -exec grep -o createElement {} \; | wc -l)
      echo ">> Number of occurrences of \"$pattern\": $count_occurrences"
    
      total_size_occurrences_with_newlines=$(find . -name "*.js" -exec grep -o createElement {} \; | wc -c)
      total_size_occurrences=$(node -pe "$total_size_occurrences_with_newlines - $count_occurrences")
      echo ">> Total size of occurrences in characters \"$pattern\": $total_size_occurrences"
      echo ">> Assuming 1 character == 1 byte"
    
      total_size_of_js_files_in_bytes=$(\
    
        find . -iname "*.js" | \
        xargs stat -f%z | \
        awk '{ s+=$1 } END { print s }'\
    
      )
      echo ">> Total size of the JavaScript files in bytes: $total_size_of_js_files_in_bytes"
    
      percentage=$(node -pe "100.0 * $total_size_occurrences / $total_size_of_js_files_in_bytes")
      echo ">> Percentage size of \"$pattern\" in \"$url\": ${percentage}%"
    }
    
    main "$@"`; 
    
  5. Running a script on your machine: In this example, to run the above script on your local machine, save it to a file called

    sizeOfStringInUrl.sh
    and run in the terminal
    bash sizeOfStringInUrl.sh nextjs.org "next.js"
    .

I've included a quick example of the above script running below:

IV. npm → Inspecting locally compiled binaries

A npm feature I've found extremely useful when trying to contribute to open-source projects is

npm > package.json > Local Paths
https://docs.npmjs.com/files/package.json#local-paths

This feature allows for the installation of a dependency from the local directory instead of retrieving it from npm. This is helpful because you can compile locally a chosen version of the project you want to contribute to (let's say React), and maybe even make some modifications to it and test its behaviour within your Sandbox.

A good way to get started with this is to edit the

package.json
and make a modification similar to:

{
  "name": "npm-package-name",
      "dependencies": {
          "packageDependency": "file:/packageDirectory/packageName"
      }
}

Having added this to

package.json

, all other dependencies in the open source project are going to be retrieved from npm registry, while the

packageDependency
in the above example gist is going to be installed from the locally compiled version.

npm packages often come with some useful executables. I recommend trying out these executables before you add them to the

scripts
section in
package.json

To display the path into which the npm installed executables are stored within a project, run:

 npm bin 

To list all of the executables available within the project run:

ls $(npm bin)

Running an executable, for example,

next
, without needing to modify
package.json
's scripts section can be done with:

 $(npm bin)/next

This is powerful, as it enables you to query the executable in the command line using

--help
(i.e.
$(npm bin)/next --help
and then pass through various options. I recommend adding the executables to
scripts
in
package.json
only after figuring out which exact subcommand and which arguments you want to pass to them, using
--help
in the terminal.


To reiterate, the intention of this article has been to: