gregoor / tofu

Experiment in structural code editing

Tofu: post-stone age coding

Tofu is an experimental VSCode extension for structured-yet-fluid code writing with JavaScript and TypeScript.

Its goal is to provide more semantically meaningful code transformations while also offering escape hatches for direct literal code editing. A secondary goal is to minimize moments of broken syntax.

Caveats

Given my limited resources of time, I sacrifice efficiency at the altar of iterating on the idea. Tofu re-parses the whole file on every edit, using Babel and formats it with Prettier after most actions. This is not incremental, and costly for large files. In the short-term I make it work by not having large file, which I tend to favor anyway.
In the longer term, I'd love for Tofu to use an incremental parser like TreeSitter instead.

In terms of experience, depending on where you are coming from, this might be a radical change to how you usually code. Forget thinking about code style or managing syntax and slide into the world of literal structural editing.

Actions

The keymap is designed to be familiar and literal. These are the basic actions which are usable in most contexts:

Action	Key(s)
Cursor
Move	← ↑ → ↓
Jump	⌥ Alt ← ⌥ Alt →
Selection	⇧ Shift +
Extend	+ ↑
Shrink	+ ↓
Neighbors	+ ← + →
Add new line	⏎ Enter (below) ⇧ Shift ⏎ Enter (above)
Wrap	⌥ Alt ↓ (Select) ( (Parenthesize) { (Object) [ (Array) > (Function) < (JSX)
Unwrap	⌥ Alt ↑
Move Node	⇧ Shift ⌥ Alt ← ⇧ Shift ⌥ Alt →
Toggle Tofu (Escape hatch)	Esc

On top of this there are around 50 contextual actions (and counting). A few examples:

• e at the end of an if-block will add an else branch, placing the cursor within the new block
• l at the start of a variable declaration will replace the kind (e.g. const) with let
• many are small in nature such as [ at the end of an expression inserting [0] instead to minimize broken syntactical states

I am sure there are many I have not thought of, so contributions in the forms of new issues describing your expectations and wishes are greatly appreciated.

The opposite is also true, Tofu makes assumptions about what the right action is in which context. Please let me know where I have baked in assumptions which do not jive with your style.

I will stop thinking of Tofu as an experiment once I and other users deem it a clear workflow improvement and the project has found a path towards monetary sustainability.

tofu's Issues

It's too easy to break wrapping

Change move lines shortcut

Apparently some editors allow you to navigate by word with alt+left/alt+right, which might make it confusing that tofu uses those keys for moving statements. So maybe a different shortcut would be more appropriate.

Open questions:

• which editors behave like that?
• what's the most common move line/statement shortcut?

Adding a new element to a collection can prepend a new line

Input

const a = [
  veryLongElementInThisArrayLeadingToLineBreakAndThusTriggeringTheBug,
];
asd(23|);

Action

, - Add argument

Output

const a = [
  veryLongElementInThisArrayLeadingToLineBreakAndThusTriggeringTheBug,
];

asd(23, |null|);

Expected

```javascript
const a = [
  veryLongElementInThisArrayLeadingToLineBreakAndThusTriggeringTheBug,
];
asd(23, |null|);

What's wrong (in words)?

An empty new line gets added

Putting a block inside an arrow fn

atm if you have

() => ≤null≥

and press {, you get

() => ({})

Thus there is no way to put a block in an arrow function.

I see a few possible solutions:

1. Make it actually insert a block. If you want to turn it into an object, you'd have to select the hole empty block and wrap it in().
2. Overload Enter to add a block when null is selected in the arrow fn body. This would set a precedent and imo should only be done if it could be useful in other contexts.
3. ; is not used for anything yet, since we completely leave semi-colonizing to prettier. Since it symbolizes statements it would actually fit quite well. It also sets a precedent, so it'd be great to find other reasons to use it.

Both 2. & 3. also both lack the WYSIWYGness that I strive for with Tofu. So I think it should be 1 after all.

Template Strings

Backspace to reverse all kinds of actions

In the rush of adding tests I was about to add, and consequently fix this one:

it("merges array elements", async () => {
  const hook = renderHook(() => useHistory("[a, b];\n"));

  await queueMoveCursorFor(hook, new Range(4));
  await queueActionFor(
    hook,
    new KeyboardEvent("keydown", { code: "Backspace" })
  );
  expect(hook.result.current[0].code.source).toBe("[ab];\n");
  expect(hook.result.current[0].cursor.start).toBe(2);
});

After such a change tofu would turn [a, |b] (| denoting the cursor) into [ab] when pressing Backspace. While this is fine and dandy, naively implemented, it might also turn [a(), |b] into [a()b] which is not so dandy.
I was motivated to make this change because I introduced the reverse action, of splitting identifiers with a comma, when the cursor is inside, e.g. [a|b] => [a, b]. While typing this I realized I can make Backspace only ever do what I described above, when both elements are an identifier (for function calls it could also work, but for destructuing, default parameters, yadayada, things will get complex).

Okay typing is thinking, some more thoughts developed from this, which might become a new design principle: Every additive, single keystroke action, should be revertible by pressing Backspace. The idea is here, once again, to piggyback on intuition around regular old text editing, which has this neat property of Backspace undoing the additive things one just has done.

Infinite loops

You can write infinite loops in the editor which freeze the whole page (and probably also are persisted in localStorage).

Tabbing

> works no more

Select code with mouse

Should probably only select full nodes (unless inside of a literal), like keyboard selection. Current behavior is confusing, since the select preview suggests mouse selection might work.

Underly aggressive selection

Input

test(|a, b|);

Action
Shift + ArrowRight

Output

test(|a, b|);

Expected

|test(a, b)|;

Wrap expression when inserting Paran/brace

Focus operator after entering it

Input

asd|asd

Action
&

Output

asd & |asd

Expected

asd |&| asd

JSX

Shortcut for creating an arrow function

Input

reduce(|);

Action
(

Current output

reduce((|));

Expected output

reduce((|) => null);

Moving nodes leads to many empty lines being scattered around

Disable JSX keys/parsing when run outside of React-mode

No idea yet how to separate that out

Cursor position for return type annotation is being skipped

Comments

Delete Call Expression

Input

call(|arg1, arg2);

Action
Backspace - Delete

Current output

call|arg1, arg2);

Expected output

call|;

Destructuring

Overly aggressive smart select when jumping from character selection to node selection

Input

|asd|.efg;
asd;

Action
Shift + ArrowRight - Extend selection

Output

|asd.efg;
asd;|

Expected

|asd.efg|;
asd;

Find an appropriate cursor position after pasting

Delete Array

Input

let a = [|asd, asd];

Action
Backspace

Expected output

let a;

---

Input

fn([|asd, asd]);

Action
Backspace

Expected output

fn(|);

Enter likes to break lines

Especially in function headers?

PopAround

A combination of unwrap, extend selection and wrap with what was previously removed by unwrapping. Afaik Paredit has a similar feature, sth like "barfing". Should look into it. Some examples

Examples

Ternary

In

a ? fn(≤b≥) : c

Out

fn(≤a ? b : c≥)

Array

In

{ a: [≤b≥], c }

Out

[≤{ a: b, c }≥]

Function Call

In

a ? f(≤b≥, c) : d

Out

f(a ? b : d, c) 

In

a ? f(b, ≤c≥) : d

Out

f(≤b, a ? c : d≥)

JSX

In

<ErrorBoundary
  fallback={<>An error occurred while searching. We are looking into it!</>}
>
  {term && results && ≤<SearchResults {...{ term, results }} />≥}
</ErrorBoundary>;

Out

{
  term && results && (
    <ErrorBoundary
      fallback={<>An error occurred while searching. We are looking into it!</>}
    >
      <SearchResults {...{ term, results }} />
    </ErrorBoundary>
  );
}

 ¿Double Pop?

This one is a bit more involved, and maybe it's better solved with multi-selections, which Tofu can't do yet.

In

({ a: t ? ≤1≥ : 2, b: t ? ≤"y"≥ : "n" });

Out

t ? { a: 1, b: "y" } : { a: 2, b: "n" };

Move Result/Output below/into Editor

Stop the editor from jumping around, what did I even think, it's annoying, like srsly

Shortcut for initializing variable

Input

let n|;

Action
Space

Current output

let n |;

Expected output

let n = |null|;

Move statements between scopes

≤a≥;
{
  b;
}

with moveNode(RIGHT) should yield

{
  ≤a≥;
  b;
}

Show modifier keys for actions without variants

When initializing let extra key needed

After hitting space the variable should be initialized, but now I need to manually hit a random key to finish this process.

Input

let n|

Action
space

Output

let n |

Expected

let n = |null|

Btw really like your work.

can't navigate to beginning of parens

Cut/Paste don't work anymore

Range Select Previous/Next for operands in logical/binary expressions

Input

asd || (|asd| && asd);

Action
Shift + ArrowLeft - Range Select Previous

Current output

|asd || (asd && asd);|

Expected output

|asd || (asd| && asd);

long document: focus is not where the cursor is

Credits to Prettier

I really should credit them somewhere, that tool made this whole thing so much easier

delete wrapping expression / unnest

Description

Deleting a brace / paranthese should delete the wrapping expression and put its child in its place.

Triage:

• Function calls/arrays with multiple args
  • flatten when grandparent slot is collection?

Examples:

Array

Input

const a = [|23];

Action
Backspace

Output

const a = |23;

Array as arg

Input

fn([|23, 42]);

Action
Backspace

Output

fn([23, 42);

Fn call

Input

const a = fn(|23);

Action
Backspace

Output

const a = |23;

Destructure inside function params

Currently spawns expression-protected-parens (i.e. ({})) which is invalid

use project's prettier config

to keep resizability (i.e. printWidth changes), tofu would need to re-format when saving

When deleting an operand, delete the whole operation

Input

a * b|

Action
Backspace

Current output

a * |

Expected output

a

Quick Pick improvements

• use node-kinds correctly. It's still built with the old binary exp/stmt distinction, thus it also does not support pattern-wrapping
• use separators and section labels to communicate which part of the code will be affected
• preview what wrapping will change when a selection is active

Use active+anchor for setting selections

Atm active is always set to the end of a selection which makes range selects in Tofu look different from the native ones, which also obscures the effect of selecting next/prev within an identifier/string.

cut/copy/paste seems pretty broken

what happened there buddy?

Shift + Home/End

Should select whatever AST nodes are in that range

Format worker speed-up opportunities

some ideas

• abort current formatting when a new request comes in
• only send diffs to the worker to minimize the amount of copied data

I tried the former at one point, but realized that I just added complexity and bugs and ultimately reduced throughput. So another attempt should be more principled.

In collections pressing comma inside identifier should split it

When coming from the right, cursor doesn't go into empty function body

Input

() => {};|

Action
ArrowLeft - Navigate left

Output

(|) => {};

Expected

() => {|};

Class

Allow navigation to end of all blocks (not just if)

Currently it's complicated to add statements after a block that's inside another block

Input

for (const item of iterable) {
  for (const item of iterable) {|
  }
}

Action
ArrowRight - Navigate right

Output

for (const item of iterable) {
  for (const item of iterable) {
  }
}
|

Expected

for (const item of iterable) {
  for (const item of iterable) {
  }|
}

dark mode