sourcegraph / handbook Goto Github PK

📘 The Sourcegraph handbook

Home Page: https://handbook.sourcegraph.com

License: Apache License 2.0

JavaScript 28.61% TypeScript 47.76% PowerShell 2.68% SCSS 20.85% Dockerfile 0.10%

handbook's Introduction

📘 Sourcegraph handbook

The Sourcegraph handbook describes how we (Sourcegraph teammates) work. It’s publicly visible because we are an open company.

The handbook is a living document and we expect every teammate to propose improvements, changes, additions, and fixes to keep it continuously up-to-date and accurate.

All content is in Markdown files under the 📁 content folder.

Need help editing?

Ask in the #handbook channel (for Sourcegraph team members), and/or post an issue.

Run or develop locally

Setup

Install asdf
Run asdf plugin add nodejs && asdf plugin add pnpm && asdf install

Running the website locally

Run:

pnpm install
pnpm dev

Then open http://localhost:3000 in your web browser.

Development notes

Autogenerated content

There are special tokens within some markdown pages ({{generator:*}}) that are filled at build time from the YAML files in the data folder. The code which does this the filling is in [src/lib/generatedMarkdown.ts], and these are called as part of the markdown pipeline in src/lib/markdownToHtml.ts.

Check links locally

We use markdown-link-check for link checking at build time in the link-check GitHub action. If you want to run it locally, from the root of the repository you can run this command:

pnpm check-links

This can be slow, so you can also check a single file by running this command, replacing path_to_file with the file you want to validate:

pnpm markdown-link-check <path_to_file>

Note that this will also check external links, which the GitHub action ignores. If you wish to ignore those, add -c .github/workflows/link-check-internal.json to the command.

Build

During deployment, the netlify-build script gets executed. To simulate the build process, you can run it locally:

pnpm netlify-build

The output will be in the out directory.

Deployment to production

The repository is configured to automatically deploy the main branch to production on Netlify.

handbook's People

Contributors

Stargazers

Watchers

Forkers

deethompson rothfels desenesterling ambersmo varungandhi-src ankhequinox wanderer163 glitchwizard jfeierabend gracebohl monkbysea simonpai docadam isuru-f 00mjk bostick julioleiva strogo brady-garrett ferozsalam isabella232 ana-lavongtheung theevilskeleton jadegreatking ryankscott rhythm02041999 dorje111 madisongclark bartgroenendal toddherskovitz r-stott ismaelpadilla 0xperko jamesghaedi alok-desai traceyjohnson00 frecks wooodhead markkennedy corruptcookies socheatsok78-archived amath00 learn2grid zchoo2001 mplacona tompinckney jamielndsy greggstone johnabroad42 kipit teacherstrange pterameta tdmosk deppp galindo-j rsaporta srmandell khawajaalidmple danielmarquespt enriquegh andrewgobran pimenovoleg nielthiart michael-dahl lauren-ross nikkigolding bconglet felixfbecker lloydchang devios01 a-aznar casperbakker shurpop frankie-b markhorsfield toolmantim gudata luis-sousa-pinto danielmartin sai-navana alexb3939 chenkc805 drewhinkson beso000 indradhanush csells davejrt markwharton mz0in streakycobra bevzzz rxminus borg-org ccll1 autarch rinckd dearborn-open-ai deepak2431 anoopelias hitesh-1997

handbook's Issues

Migrate contracts out of Dropbox

See sourcegraph/about#327 for context

Design QA

I'm going to add design QA items here, with the idea that some things may already be in progress/resolved with other work, and we can break out anything that's needed into a separate issue.

Text content uses color: #24292e; inherited from .markdown-body instead of the --text-color var defined in body.
Separators / in breadcrumbs should be gray-06.
Spacing between header bar and breadcrumbs should equal 40px / 2.5rem, and spacing between header bar and in-page navigation should increase to match.

Add information about auto-merge feature

Add documentation about the benefits of the auto-merge feature in the handbook repo

Many links on org chart page are broken

A lot of the links on https://handbook.sourcegraph.com/company/team/org_chart are broken - possibly something to do with the auto-generated redirects and nested lists? For instance, the "Code graph" link goes to https://handbook.sourcegraph.com/company/team/code-graph but should be https://handbook.sourcegraph.com/engineering/code-graph.

dx: document build issue pinging script

Update handbook usage page

Simplify and update handbook usage page. It's long and a bit cumbersome, and should reflect the views of the current handbook team + product maturity.

Reduce logspam of link checker

It's currently hard to find an error in the output of the dead link checker.

We can fix this by using the programmatic API from our own small JS script. This would also be necessary anyway for more improvements, like #82 and making sure all pages have inlinks.

Document the options that non-US employees have for structuring their work relationship with Sourcegraph

Every time a candidate asks, I have to spend time talking about something I am not an expert in. A lot of candidates read our handbook thoroughly so if we had this documented it would save me time and I think it would give confidence to candidates that we know what we are doing here.

Limit right sidebar to h2 headings

The Table of contents in the right sidebar of the Handbook should only include h2 level headings as prescribed in the design here.

Add related content module

Swiftype has a related content module I'd like to try out. I've configured it here.

Error: Could not find a production build in the '/Users/nick/dev/handbook/.next' directory

I followed the instructions in the readme to run locally on a fresh checkout and got the following error:

nick@Nicks-MBP dev % git clone [email protected]:sourcegraph/handbook.git
Cloning into 'handbook'...
remote: Enumerating objects: 26624, done.
remote: Counting objects: 100% (1702/1702), done.
remote: Compressing objects: 100% (882/882), done.
remote: Total 26624 (delta 994), reused 1268 (delta 784), pack-reused 24922
Receiving objects: 100% (26624/26624), 7.81 MiB | 12.29 MiB/s, done.
Resolving deltas: 100% (15579/15579), done.
nick@Nicks-MBP dev % cd handbook
nick@Nicks-MBP handbook % yarn
yarn install v1.22.11
[1/4] 🔍  Resolving packages...
warning Resolution field "[email protected]" is incompatible with requested version "unist-util-visit-parents@^4.0.0"
warning Resolution field "[email protected]" is incompatible with requested version "unist-util-is@^4.0.0"
warning Resolution field "[email protected]" is incompatible with requested version "unist-util-visit-parents@^3.0.0"
warning Resolution field "[email protected]" is incompatible with requested version "unist-util-visit-parents@^4.0.0"
warning Resolution field "[email protected]" is incompatible with requested version "unist-util-is@^4.0.0"
warning Resolution field "[email protected]" is incompatible with requested version "unist-util-visit-parents@^4.0.0"
[2/4] 🚚  Fetching packages...
[###############################################################################################################################################################-] 797/[###############################################################################################################################################################-] 798/[#############info @next/[email protected]: The CPU architecture "x64" is incompatible with this module.
info "@next/[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
info @next/[email protected]: The platform "darwin" is incompatible with this module.
info "@next/[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
info @next/[email protected]: The platform "darwin" is incompatible with this module.
info "@next/[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] 🔗  Linking dependencies...
warning " > [email protected]" has unmet peer dependency "@popperjs/core@^2.10.1".
warning "next > styled-jsx > @babel/[email protected]" has unmet peer dependency "@babel/core@^7.0.0-0".
warning " > [email protected]" has unmet peer dependency "@types/node@*".
[4/4] 🔨  Building fresh packages...
✨  Done in 15.69s.
nick@Nicks-MBP handbook % yarn start
yarn run v1.22.11
$ next start
ready - started server on 0.0.0.0:3000, url: http://localhost:3000
Error: Could not find a production build in the '/Users/nick/dev/handbook/.next' directory. Try building your app with 'next build' before starting the production server. https://nextjs.org/docs/messages/production-start-no-build-id
    at Server.readBuildId (/Users/nick/dev/handbook/node_modules/next/dist/server/next-server.js:1573:23)
    at new Server (/Users/nick/dev/handbook/node_modules/next/dist/server/next-server.js:92:29)
    at NextServer.createServer (/Users/nick/dev/handbook/node_modules/next/dist/server/next.js:107:16)
    at async /Users/nick/dev/handbook/node_modules/next/dist/server/next.js:119:31
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Document how we handle medical and dental benefits for non-US employees

Ability to include a section of content from another page

I would love the ability to include a section of content from another page in the handbook. This would allow us to inline valuable pieces of information in multiple places in the handbook, while preserving a single source of truth in the source markdown.

Examples of pieces of information that are valuable scoped to a specific team/org, but also valuable to see as a department/company:

Org chart (see #61)
OKRs
Roadmaps

Handbook pages need Open Graph meta tag for social sharing

Add Open Graph meta tags to handbook template headers.
Allow users to overwrite meta tags for a specific page i.e. be able to specify another image instead of using a default image such as the Sourcegraph logo.

Cache contributors from GitHub API

We are getting rate-limited for our requests to the GitHub API, resulting in the contributors section not showing up often.

Possible solution: Use https://www.npmjs.com/package/netlify-plugin-cache-nextjs, which may result in NextJS simply not rebuilding unchanged pages and therefor also not triggering new API requests. I'm not sure on that though, seems worth trying our first, could be beneficial in general.

If that still results in pages to be rebuilt, we can:

Before we make a GitHub API request, exec a quick git log $PATH --pretty=%H --max-count=1 locally to get the commit SHA of the last change to the file
Check a gitignored directory for a JSON file containing the commit SHA+API result for that file
If the commit SHA is the same, use that result (will be cache hit in 99% of cases)
If it's not, make a GitHub API request for the contributors
Store the results (just the aggregated avatar URLs etc) in a JSON file in the cache folder
Use netlify-plugin-cache to store and restore that folder https://www.npmjs.com/package/netlify-plugin-cache

Video is missing captions on "Using relative paths to link handbook pages" page

The video at https://handbook.sourcegraph.com/editing/linking-within-handbook is missing captions.

Captions can be added to this video following the guide at https://handbook.sourcegraph.com/marketing/adding_screenshots_screen_recording#adding-captions-for-your-voice-over-to-your-video

Implement new search modal design

Design: https://www.figma.com/file/ze8C5qjzSTadKv2LYmON1T/Handbook-Wizardry?node-id=1144%3A3334

Add syntax highlighting for Markdown code blocks

We haven't added a rehype plugin for syntax highlighting + CSS theme file yet.
Not the highest priority since we don't have as many code blocks in the handbook, but we do have some.

There are multiple options available, might be worth a quick research which is the most modern/considered best nowadays: https://github.com/rehypejs/rehype/blob/main/doc/plugins.md

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository. View logs.

WARN: Fallback to renovate.json file as a preset is deprecated, please use a default.json file instead.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

chore(deps): update dependency @sourcegraph/eslint-config to ^0.37.1
chore(deps): update dependency eslint to v9

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

chore(deps): update actions/checkout action to v4
chore(deps): update actions/setup-node action to v4
chore(deps): update actions/stale action to v9
Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

asdf

.tool-versions

node 18.9.0

pnpm 8.1.1

dockerfile

Dockerfile

node latest

github-actions

.github/workflows/build-handbook.yml

actions/checkout v3

actions/setup-node v3

.github/workflows/codenotify.yml

actions/checkout v3

sourcegraph/codenotify v0.4

.github/workflows/data-validation.yml

actions/checkout v3

actions/setup-node v3

.github/workflows/large-files.yml

actions/checkout v3

Amadevus/pwsh-script v2

macos 11

.github/workflows/link-check.yml

actions/checkout v3

actions/setup-node v3

.github/workflows/pr-title.yml

morrisoncole/pr-lint-action v1.5.1

.github/workflows/prettier.yml

actions/checkout v3

actions/setup-node v3

creyD/prettier_action v3.3

.github/workflows/stale.yml

actions/stale v4

.github/workflows/update-branches.yml

actions/checkout v3

npm

package.json

@agentofuser/rehype-section ^1.0.5

@stefanprobst/rehype-extract-toc ^2.1.1

@types/js-yaml 4.0.5

@types/mkdirp 1.0.2

@types/unist 2.0.6

bootstrap ^5.1.3

chrono-node ^2.3.2

date-fns ^2.25.0

execa ^5.1.1

gitlog ^4.0.4

gray-matter ^4.0.3

hast-util-find-and-replace ^4.0.0

hast-util-is-element ^2.1.1

hast-util-to-string ^2.0.0

hastscript ^7.1.0

highlight.js ^11.3.1

js-yaml ^4.1.0

mdi-react ^8.1.0

mkdirp ^1.0.4

next ^12.0.2

next-seo ^4.28.1

react ^17.0.2

react-dom ^17.0.2

rehype-autolink-headings ^6.1.0

rehype-highlight ^6.0.0

rehype-raw ^6.1.0

rehype-slug ^5.0.0

rehype-stringify ^9.0.2

rehype-url-inspector ^2.0.2

relateurl ^0.2.7

remark-extract-toc ^1.1.0

remark-gfm ^3.0.1

remark-github ^11.2.1

remark-parse ^10.0.0

retext ^8.1.0

retext-smartypants ^5.1.0

tippy.js ^6.3.5

ts-node ^10.4.0

unified ^10.1.0

unist-util-visit ^4.1.0

vfile ^5.2.0

@actions/core ^1.6.0

@sourcegraph/eslint-config ^0.26.0

@sourcegraph/prettierrc ^3.0.3

@sourcegraph/tsconfig ^4.0.1

@types/hast ^2.3.4

@types/line-column 1.0.0

@types/mdast ^3.0.10

@types/node ^18.11.18

@types/react 18.0.0

@types/relateurl 0.2.29

ajv-cli ^5.0.0

chalk ^4.1.2

eslint ^7.32.0

globby ^11.1.0

hast ^1.0.0

hast-util-select ^5.0.5

jest ^27.4.5

line-column ^1.0.2

markdown-link-check ^3.8.7

mdast ^3.0.0

next-sitemap ^3.1.16

prettier ^2.8.3

rehype ^12.0.1

remark-rehype ^10.1.0

sass ^1.43.4

strip-ansi ^7.0.1

typescript ^4.4.4

node ^v18.9.0

pnpm ^8

unist-util-is 5.1.1

unist-util-visit-parents 5.1.0

pnpm 8.1.1

Check this box to trigger a request for Renovate to run again on this repository

Slashes in headings removed instead of converted into hyphens in anchors

Slashes / are removed rather than converted into - in anchors.

For example, on pages like the Teammates page, the she/her is converted into sheher instead of the previous she-her, which breaks existing links to team members using this format.

Document hiring budgets

For sponsoring events and advertising our jobs.

Set up GitHub Actions

We should set up GitHub Action checks like in the about repo.

Check for broken links
Check for Prettier, TypeScript, ESLint on the TS code
Automatic Prettier fixes (which will now not break anything thanks to a compliant parser!)
Stale bot
BONUS: Static deploy preview like with Netlify

Make large file check also check for large plain text files containing long Base64 strings

The large file check currently rejects all large files that are over 100KB and binary (e.g. PNG, JPG, MP4).
However, sometimes plain text files include large binary content in a sneaky way that makes them equally/even larger than the equivalent binary file. E.g. an SVG that actually includes an <image> with a Base64 data: URI encoding a JPG, or even including an entire font (this actually existed in the handbook prior to the migration). The check currently doesn't catch these.

What it could do is in addition to binary files, match the content for long Base64 strings with a regex (e.g. \w{100,}). If the file is larger than 100KB and not binary, but contains any such strings (or passes a length threshold of the sum of the strings), it should also be rejected.

Some images do not load (were deleted in migration)

Examples:

Design team logo on this page: https://handbook.sourcegraph.com/product/design
Code insights screenshot atop this page: https://handbook.sourcegraph.com/engineering/code-graph/code-insights

Some do work:

Batch changes logo: https://handbook.sourcegraph.com/engineering/code-graph/batch-changes

Perhaps we missed a folder or file type, looking at the old paths? Example: https://sourcegraph.com/github.com/sourcegraph/about/-/commit/3f75bff1c2eb936cb8ab2066b79ff27796107b12?visible=1

Migrate handbook content over including Git history

This would be a big rebase and we would freeze the handbook in the meantime on the about page.

Document how/when we review compensation

Improve org chart page

We currently render the org chart by compiling all team pages' "Members" sections.

That is not a bad approach per se, but we do it all in the client at load time, which is noticeable and breaks often. For example, we currently seem to be adding a footer in the middle of the page, and date/time hover tooltips don't work in the dynamic content. The content can also not be searched I assume.

If we want to continue with this, we should look into doing this at build time. But also we should think about what the "ideal" org chart would look like for our handbook – perhaps data taken from the BambooHR API. Perhaps even a real chart (although those don't necessarily scale better, rather the opposite).

Document in the handbook the CI process in case of failures

Add check that relative links are used (no absolute links to handbook.sourcegraph.com)

Using relative links is critical to make sure that the handbook can be browsed on deploy previews in PRs and on old revisions. It can be unintuitive for people not familiar with the concept though, which is why we have this handbook page: https://handbook.sourcegraph.com/editing/linking-within-handbook

Our link checker cannot check for this directly, but we could add a step that simply does a grep for links looking like [...](https://handbook.sourcegraph.com/...) in the content/ directory.

Expand strategy

From today's kickoff meeting:

The strategy seems to be all about us, which isn't necessarily bad, but can we add sample problems that exist in each section and how either we actively are working on them with customers now, or how we plan on tackling these in the future?

This entails expanding the https://about.sourcegraph.com/company/strategy page.

Document creating a new directory + page

We have documentation for creating a new page, but not for creating a directory. Will add this to existing "creating a new page" documentation.

Add smartypants remark plugin

Our content guidelines say to always use typographic quotation marks.
This would be best facilitated by the smartypants plugin, which does this automatically.
There is a retext plugin which should be possible to be used with rehype-retext (which allows to use retext plugins), but when I tried it, I got an error that it expects a Parser to be passed. I have a hunch that it is not compatible with the newest remark versions and needs some updates (probably super minor ones).

Add MDX support to handbook

We have infrastructure in the handbook that emits markdown at build time (yarn generated-pages) but it would be cleaner if we supported MDX to fill the content from the data files, as well as bring the possibility of other automation to the handbook.

The biggest immediate benefit is that it would allow content fills to be used on-page anywhere, mixing markdown and generated content, instead of only allowing for entirely generated pages.

It would also allow for possibilities of improvements like #649.

Implement the heading ID syntax

There's a "set heading ID" syntax from docsite, which is used in several places:

https://sourcegraph.com/search?q=context:global+repo:%5Egithub%5C.com/sourcegraph/handbook%24+%7B%23.*%7D&patternType=regexp

The expected behavior is to set the id (also known as the heading slug, or the hash fragment that points to it) for that element.

The actual behavior right now is that this is accidentally being parsed by the Slack channel plugin.

Add remark-footnotes Markdown support, now supported by GitHub

Expose file where link is broken

Before the recent changes to the Handbook link checks (which were a big improvement!), a user could see which file contained the broken link right there within the check.

Now, the error tells the user which link is broken, but not where that link is:

The user can find the page with the broken link by looking at the "files changed" tab on their PR. Is it possible to re-expose that information on the handbook check failure? It's a little confusing as is.

Context in this thread.

Backlinks

Add the ability to see backlinks for each page. In other words, "What links to this page?" (Locally, that is. From within the handbook, not links from external sites.)

prettier and markdown-link-check can interact in unexpected ways

We discovered this issue in #244: prettier can change the Markdown syntax in ways that markdown-link-check seems to be unable to handle, especially when there are image or regular links with parentheses (or, I presume, other characters that cause issues in regular Markdown link syntax). Here's the original Slack thread if extra context is needed.

As a simple example, let's say we're linking to https://example.com/image(1).png, and we have this Markdown:

![my awesome image](https://example.com/image(1).png)

Prettier will then use angled brackets to escape the link:

![my awesome image](<https://example.com/image(1).png>)

At which point markdown-link-check doesn't know how to handle that syntax, returns a 400, and the test fails. Oops.

I'm not really familiar enough with this to have a good idea on the right place to change this: if Prettier has an option to URL encode instead of surrounding with angled brackets, that would probably work. Or maybe there's an option or fix for markdown-link-check to make this work.

Continuous deployment of handbook

When I edit the handbook it is unclear how or when the change will be deployed. The ideal state is that a green build on the main branch means that the commit has been deployed.

Searching in the previewed builds on notify sends back to the main website.

This is very minor, but when you use the search bar on a preview build on Netlify, the links in the results are sending back to the production website.

That can be easily overlooked if you don't pay attention though!

Document key qualification questions and responses for sales calls

Somewhat related to objection handling, but not exactly the same. Discussed this with @chrisdelane today after a call. I have a lot of qualification questions in my head, and what the pitch is depending on the result, and I (and @sqs ) should document these. E.g.:

Does your org use microservices or a monorepo?
- If microservices, talk about cross-repo references, searching for API usages and API deprecation/updates/etc., ensuring changes get propagated across all repos that use a given library/API, etc.
- If monorepo, talk about how Sourcegraph can help answer questions without cloning down the latest version (and losing your local changes) every time you want to search
What languages does your org use?
- If Python, share our Python 2-->3 migration case study (still WIP)
- If Go, talk about how we use it and precise code intelligence
- If X, talk about Y
Is your team all co-located or remote?
- If X, then Y

Etc.

document swag package is priority

"If there is a 1% chance that a swag package will help close a deal, that is the number 1 priority" - should be documented in the handbook

Document incident ownership transmission

https://about.sourcegraph.com/handbook/engineering/incidents

Show breadcrumbs

Depends on #1

Implement contributors section

Can read from the git history and display GitHub avatars.

Design: https://www.figma.com/file/ze8C5qjzSTadKv2LYmON1T/Handbook-Wizardry?node-id=1144%3A3095

Support nested folders and index.md files

Flexible relative links for generated content

The createRelativeProductionLink function does what it says, and makes an absolute link for a page (coming from the .yaml files) relative to the product home page.

handbook/src/lib/generatedMarkdown.js

Line 13 in 406e0c3

function createRelativeProductLink(link) {

This works fine for content which is included from the product home page, but if you were to include content from anywhere else that needs relative links, it will break. It isn't breaking anything yet, because nobody is doing that.

This function should be made generic to take an absolute link, the absolute path to the current page, and generate a relative link on the fly.