Similar to https://github.com/r3bl-org/r3bl_rs_utils/blob/main/docs/dd_editor_component.md create a new DD for render pipline and painting:

1. how does the render pipeline work w/ renderop
2. how are different terminal backends implemented
3. how is the cursor data managed by TWData
4. how is ansi text content clipped
5. how is plain text content clipped
6. how is unicode "jumping" managed to provide the same experience across various terminals & os matrix *using unicode_width as the source of truth for how (display) wide a grapheme cluster is

How it works today

Currently the style! proc macro only works w/ Color enums or Rgb values. The expression that is passed into the proc macro has to evaluate into a Color. However, it does not support passing a variable that contains a Color.

Here's how you can use it today:

use crossterm::style::*;
use r3bl_rs_utils::*;

fn make_a_style_declaratively(id: &str) -> Style {
  style! {
    id: style2
    attrib: [dim, bold]
    margin: 1
    color_fg: Color::Red
    color_bg: Color::Rgb { r: 0, g: 0, b: 0 }
  }
}

How we would like to improve on this

We would like to be able to pass a variable that holds a Color in addition to just passing an expression which evaluates to a Color value.

use crossterm::style::*;
use r3bl_rs_utils::*;

fn make_a_style_declaratively(id: &str) -> Style {
  let black = Color::Rgb { r: 0, g: 0, b: 0 };
  let red = Color::Red;

  style! {
    id: style2
    attrib: [dim, bold]
    margin: 1
    color_fg: red
    color_bg: black
  }
}

After the fix, we should be able to pass all 3 types into the color_fg or color_bg fields:

1. Color enum
2. Rgb value
3. Variable that holds either of the two above

Related

1. Article: https://developerlife.com/2022/08/04/rust-dsl-part-1/
2. Test: https://github.com/r3bl-org/r3bl-rs-utils/blob/main/tests/test_make_style_macro.rs
3. Code: https://github.com/r3bl-org/r3bl-rs-utils/blob/main/macro/src/make_style/entry_point.rs

Currently r3bl-cmdr crate is just a set of demos / examples for r3bl_rs_utils.

1. Move main.rs & examples folders from cmdr -> rs
2. rs will be a bin crate now and cargo run will run the demos / examples
3. cmdr will get the actual TUI app that it has always intended to be (knowledge base product)

More about rust examples http://xion.io/post/code/rust-examples.html

1. add another developerlife.com article on completing the DSL w/ stylesheet!. Also point out the style! changes
2. add an intro article on RSX and the initial implementation of the layout and styling engine

also update dl fonts to use victor mono. here are helpful links:

1. https://stackoverflow.com/questions/30521623/the-right-way-to-include-multiple-fonts-in-css
2. https://rubjo.github.io/victor-mono/

cut & publish release

readme & docs:

update src/tui/readme.md:

• new screen recording video (https://user-images.githubusercontent.com/2966499/200138653-c0cf925f-2c91-4908-9ed5-1e216b5dd547.webm)
• make sure to call out the dep on r3bl_rs_utils_core (tui_core module)
• make it clear that this crate is performance (only render what has changed)
• make it clear that there's styling, layout, and rsx support (evolving)
• make it clear
  • what the r3bl_tui crate is and how is it different from others
  • what the lifecycle of things are (the SVG diagrams)
  • how components render and handle user input
• update docs.rs things

crates.io:

• rev crate versions & cut release tags for & publish on crates.io
  1. core -> 0.8.2
  2. macro -> 0.8.2
  3. redux -> 0.1.2
  4. tui -> 0.1.2
  5. public -> 0.9.1

publicity:

• publicize it on reddit https://www.reddit.com/r/rust/comments/yn8en8/media_tui_framework_fans_heres_an_update_to/
• on linkedin: https://www.linkedin.com/feed/update/urn:li:ugcPost:6994789148867919872/

Problem:

Currently, auto clipping to display width bounds is not supported for ANSI formatted text:

• RenderOp::PrintTextWithAttributes and
• its crossterm impl crossterm_backend

This means that ANSI formatted text is not clipped automatically: TruncationPolicy.

Solution:

• By using this crate ansi-parser, combining it w/ unicode_width, this can be solved.
• Here's the repo w/ the POC: https://github.com/nazmulidris/ansi-parser-fork/blob/main/src/parsers/tests.rs

Currently, the rendering engine uses unicode_width crate to determine display width (in cols) of strings. UnicodeString handles this in the platform.

However, there's a disparity in how different terminals and OSes paint various compound complex grapheme clusters.

Here's an example 🙏🏽😀░.

1. The first emoji is a compound emoji. unicode_width reports its width to be larger than the display width in kitty + linux or terminal + macos.
2. Some terminals + OS combinations show them correctly (eg: kitty on linux) but this breaks what unicode_width reports.

A solution is to make all terminals "break" in the same way. This has already been staged in a sense. SharedTWData gets piped into the rendering_pipeline and the terminal_backend.

1. This allows the framework to track the location of the cursor! 🎉 Independent from crossterm!
2. The crossterm command to save and restore cursor is UNUSABLY SLOW. This already remedies that problem.

This solution is implemented by "jumping" 🦘 the cursor to the width (display cols) reported by unicode_width. The UnicodeString has all the information to make this work when combined with SharedTWData and its tracking of the cursor (independent from crossterm!).

In the future, since there's z-ordering support in the framework, more awesome stuff like this will be possible! 🎉

Currently it is a lot of work to describe what exit_keys look like. The code is something like:

    let exit_keys: Vec<TWInputEvent> = vec![TWInputEvent::DisplayableKeypress('x')];

It would be nicer to replace the TWInputEvent::DisplayableKeypress('x') w/ a macro. And even provide the ability to handle multiple TWInputEvents. This is what that DSL syntax might look like:

   let exit_keys = tw_input_events! {
     tw_input_event { Displayable x },
     tw_input_event { Displayable X },
     tw_input_event { NonDisplayable Ctrl X }
     tw_input_event { NonDisplayable Ctrl Shift X }
   };

Currently layouts in apps are done imperatively. Here's an example https://github.com/r3bl-org/r3bl-cmdr/blob/main/src/ex_app_with_layout/app_with_layout.rs.

This issue will be updated w/ more details over time. The idea would be to build something like React JSX, so that this layout code could be expressed declaratively.

Here's a diagram of the layout code for an application. It's a 2 column responsive layout. https://app.diagrams.net/

Here's a simple block of layout code that be a good first candidate to macro-fy the code from here.

tw_surface.box_start(TWBoxProps {
  styles: tw_surface.stylesheet.find_styles_by_ids(vec!["style1"]),
  id: COL_1_ID.into(),
  dir: Direction::Vertical,
  req_size: (50, 100).try_into()?,
})?;

// OPTIMIZE: macro?
if let Some(shared_component) = self.component_registry.get(COL_1_ID) {
  let current_box = tw_surface.current_box()?;
  let queue = shared_component
    .write()
    .await
    .render(&self.has_focus, current_box, state, shared_state)
    .await?;
  tw_surface.render_buffer += queue;
}

tw_surface.box_end()?;

Here are some thoughts on the area of improvements in this block of text:

1. The stylesheet lookup by vec of string is very clunky. That should be streamlined using macros in stylesheet.rs.
2. Rendering a component needs to be turned into a macro as well.
3. For tw_surface, the act of calling box_start, render, box_end seems pretty repetitive, and might be good for a macro.

1. checkbox (multi and single select)
2. simple form
3. tree component
4. tabs
5. scroll view

Define what the demo for cmdr should contain (rs will have the examples / demo from cmdr).

📌 The latest version of this design doc is in the repo here

https://github.com/r3bl-org/r3bl_rs_utils/blob/main/src/tui/rsx/render_macros.rs#L18-L19

In r3bl-cmdr repo, when trying to change the exit key from Ctrl+q to x
it was not possible.

This is because a logic error made a KeyEvent which has a modifier key
set, to be NonDisplayable.

Create a simple modal dialog that looks like the diagram below. The dialog should only be rendered if it has gained focus, resulting from the end user pressing a keyboard shortcut.

  DialogComponent                      
  +------------------------------+ <- DialogEngine renders this in ZOrder::Glass
  |  TITLE                       |    (in DialogComponent struct)
  | | xxxxx                    | | <- DialogBuffer (in State struct / from Store)
  +------------------------------+     holds: buffer, title, is_focused_in_dialog
                                   <- Need handlers for onYes, onNo

/tui/examples/ex_editor/

• App

  • component_registry::populate()
    • create DialogComponent file
    • attach a OnDialogPressFn handler fn to the component
      • should dispatch Action::SetDialog if DialogResponse::Yes(String) or No
    • save it to component_registry
  • impl_app::app_handle_event()
    • if InputEvent matches keyboard shortcut Ctrl+l -> give the dialog focus 👀
      • save the id w/ focus to the DialogComponent, before giving focus to the dialog
      • dispatch action SetDialog (DialogBuffer: title, buffer) -> will trigger a render 🖌️
      • return Consumed (don't return ConsumedRerender as this will trigger a render 🖌️)

• Redux state and actions

  • add actions to modify the state w/ DialogBuffer: InitDialog
  • impl trait HasDialogBuffer (defined in component) for State
  • update reducer (handle init & update of dialog buffer in state)

/src/tui/dialog/

• DialogBuffer

  • create separate file; stored in redux store's state
  • struct contains:
    • buffer: EditorBuffer
    • title: String

• DialogComponent

  • create separate file
  • struct contains
    • DialogEngine
    • id: String
    • on_dialog_press_handler: OnDialogPressFn<S,A>
  • impl Component
    • handle_event() <- 👀 must already have focus for this to be called
      • call dialog engine's apply_event(input_event), deal with ApplyResponse<DialogResponse>
        return type:
        • match Applied(DialogResponse)
          • call on_dialog_press_handler fn (which will deal w/ Yes(DialogBuffer) or No)
          • restore focus via component_registry.has_focus.set_id($id) where $id is the
            previously saved in DialogComponent
          • return EventPropagation::ConsumedRerender
        • else return EventPropagation::Propagate
    • render()
      • use DialogEngine to render
  • DialogResponse enum (define in component)
    • variants: Yes(String), No
  • OnDialogPressFn<S,A> (define in component)
    • fn(&SharedStore<S,A>, DialogResponse, ComponentRegistry, String /* prev focus id */)
  • HasDialogBuffer trait (define in component): get_dialog_buffer()

• DialogEngine

  • create separate file
  • struct contains:
    • EditorEngine
    • prev_focus_id: String: last component that had focus prior to dialog being shown

• DialogEngineApi

  • create separate file
  • apply_event(): return ApplyResponse<DialogResponse>
    • check whether enter pressed -> means yes & return
      ApplyResponse::Applied<DialogResponse::Yes<String>>
    • check whether esc pressed -> means no & return
      ApplyResponse::Applied<DialogResponse::No>
    • pass the event thru to the editor engine api & return
      ApplyResponse::NotApplied
  • render(): return RenderPipeline
    • DialogBuffer holds buffer and is_focused_in_dialog enum
    • render dialog area, yes button, no button to ZOrder::Glass layer
    • render single line editor using EditorEngingeRenderApi (buffer comes from DialogBuffer)
      • calculate the size & pos for the editor engine and use that to render the editor content
      • move all the ZOrder::Normal RenderOps to Glass in the RenderPipline that's returned

need to figure out

• how to handle multiple caret displays (normal and glass layers)

wrap up before merge to main

• how does style get passed into the dialog?
• write unit tests for render pipeline (hoist)
• write unit tests for dialog engine & buffer & api

Problem

Currently it is left up to the application or component to clip to the bounds of the displayable area (in the terminal window / surface).

This should really be enforced by TWCommandQueue when it executes the stream of TWCommands.

1. If there are some commands that are out of bounds, then they can be turned into no-ops or smart truncation / clipping can be performed automatically.
2. This will limit overdraw and also remove the burden of doing this away from the component / app writer.

There may be a need to use unsafe in order to mutate a static that holds the size of the terminal window itself in order to implement these checks.

Source code entry points:

1. Here's an example of code that can be simplified w/ the framework handling bounds enforcement: https://github.com/r3bl-org/r3bl_rs_utils/blob/main/src/tui/ed/editor_engine.rs#L111

2. Here's an example of code that uses unsafe global mutable static variable: https://github.com/r3bl-org/r3bl_rs_utils/blob/main/src/utils/file_logging.rs#L96

3. Here is the method that sets the size of the terminal window (good hook to update global state variable containing the size): https://github.com/r3bl-org/r3bl_rs_utils/blob/main/src/tui/terminal_window/main_event_loop.rs#L44

See: #29

Solution part 1

Assume that the 1st arg to PrintWithAttributes is plain text ... make this clear in the docs ... calculate the available width using TWUtils.get_current_position() and the length of the text to figure out where to clip it.

See: https://github.com/r3bl-org/r3bl_rs_utils/blob/async-unsafe-size/src/tui/crossterm_helpers/tw_command.rs#L528

Added 2 sets of paint text commands in TWCommand to handle this:

Also added terminal_window_static_data that holds the following (atomics, lock-free):

1. terminal window size
2. cursor position

Solution part 2

Remove the use of static and unsafe code (https://gist.github.com/nazmulidris/78d7f64164db6cfb33eed5efd2b339c9) and replace it w/ TWData that contains the size & position.

1. add support for simple and autocomplete dialog box
2. add demo for it

Goals:

1. Add support for termion (in addition to crossterm).
2. Make TWCommand & Keypress compatible w/ both.
3. Only use crossterm if windows is detected, otherwise use termion

Note - termion is created by popos swes who work for system76

WIP

1. Here's my fork of termion w/ added examples: https://github.com/nazmulidris/termion
2. this commit 2abb831 gets main_event_loop ready to handle blocking reads from termion (if that is implemented)

Current codebase:

1. Currently only crossterm is supported and insulated so that no deps to it are exposed by r3bl_rs_utils.

Compatibility notes:

1. From what I understand so far, termion shortcuts are limited to Alt + char and Ctrl + char combos. Need to verify this. If true that means that we have to be careful about creating shortcuts that won't be recognizable by termion (but might be by crossterm).
2. Color seems to be the same between them, and so does alternate screen, and raw mode, so that should be seamless.

Starting points for adding support:

1. https://github.com/r3bl-org/r3bl_rs_utils/blob/main/src/tui/crossterm_helpers/tw_command.rs#L287
2. https://github.com/r3bl-org/r3bl_rs_utils/blob/main/src/tui/crossterm_helpers/keypress.rs#L1

More info on termion:

1. https://docs.rs/termion/1.5.6/termion/cursor/index.html
2. https://github.com/redox-os/termion

Modules that are affected (for crossterm & termion support):

• impl DebugFormatRenderOp
• impl PaintRenderOp
• async input / event stream / reader
• keypress conversions from "whatever key event for the underlying lib"
• color conversions from TWColor to underlying lib

Repro steps:

1. run r3bl-cmdr
2. select ex 1 or 2 or 3 (the problem shows up in all of them)
3. when state is at 0, press up (it won't be rendered! but state will increment)
4. press up again. it will now render 2.
5. when pressing down, same thing, the state is decremented but not rendered
6. press down again, the state is decremented & rendered

Initial thoughts:

1. I made a change to detect when state hasn't changed even after an action is dispatched (in this case, subscribers are not run). This might be where the issue lies.

PTAL at https://github.com/r3bl-org/r3bl_rs_utils/blob/main/core/src/tui_core/graphemes/mod.rs#L1

• Repo to test out all this stuff syn-hi
  • extract frontmatter from MD file
  • parse yaml frontmatter (in MD file)
  • parse json frontmatter (in MD file)
  • md to AST to do some processing?
  • syntax highlight each line in MD file
  • create r3bl theme
• Integrate into r3bl_rs_utils
  • impl From<Vec<Style, String>> for StyledTexts
  • test it
  • add syntax_set and theme to EditorEngine (make it part of the construction)
  • update EditorEngineConfigOptions to provide control over syntax highlighting
  • update EditorBuffer to hold meta data about what file extension ("md", "rs") etc that will be used to get a SyntaxReference (from the SyntaxSet in the engine)
  • use the highlighter to render content in EditorEngineRenderApi::render_content() (make note of enable/disable syntax highlighting in EditorEngineConfigOptions.
  • fix bugs
    • backspace, delete & multiple lines
    • emoji at end of clipped lines
  • write tests (test_styled_text.rs)
• Definitions
  • frontmatter
  • commonmark
• Crates
  • 🍴md parser
  • 🍴syntax highlighting
  • 🍴md to ast
  • 🔎frontmatter
• Example of rust md CLI formatter (using both crates above, but only view-mode, no edit-mode)
  • 🍴mdcat
• Parsing pedagogical resources
  • https://iximiuz.com/en/posts/rust-writing-parsers-with-nom/
  • https://blog.logrocket.com/parsing-in-rust-with-nom/
  • https://naiveai.hashnode.dev/practical-parsing-nom

Plan some work to be done

Create a set of small issues that Shawn can work on. These issues must fit the following template:

1. Put a microscope on a small function
2. Write a test for it and then getting that function to work as expected.
3. It must not require an understanding or familiarity with the entire codebase. And the workflow should be similar to something that is done in Rust playground.

Here's the list so far

1. #14
2. https://github.com/r3bl-org/r3bl-cmdr/issues/15
3. https://github.com/r3bl-org/r3bl-cmdr/issues/14

Install miri and create fish scripts to run the binary & tests using it.
https://github.com/rust-lang/miri

Also install this on all dev machines & update ~/local-backup-restore/1-install.fish

See: #28

write developerlife.com article on how to reason about tuis & create tui apps

• write the article
• publicize it on reddit
• new fonts: victor mono, mabry pro

fyi

can i use ttf fonts

• yes: https://caniuse.com/ttf

css for fonts

@font-face {
    font-family: Mabry Pro;
    src: url(MabryPro.woff2)
}

@font-face {
    font-family: Mabry Pro;
    src: url(MabryProItalic.woff2);
    font-style: italic
}

@font-face {
    font-family: Mabry Pro;
    src: url(MabryProItalic.woff2);
    font-weight: 700
}

@font-face {
    font-family: Mabry Pro;
    src: url(MabryProBoldItalic.woff2);
    font-weight: 700;
    font-style: italic
}

Related to #20

Once that is fixed, cut a new breaking release of:

1. r3bl_rs_utils (public, macro, and core)
2. and r3bl-cmdr
3. Also start creating git tags for each release on each repo from now on.

In crossterm, the following KeyEvent never happens.

let never = KeyEvent {
  code: KeyCode::Char('x'),
  modifiers: KeyModifiers::SHIFT,
}

When "X" is pressed, the following KeyEvent is sent:

let never = KeyEvent {
  code: KeyCode::Char('X'),
  modifiers: KeyModifiers::SHIFT,
}

And not:

let never = KeyEvent {
  code: KeyCode::Char('x'),
  modifiers: KeyModifiers::SHIFT,
}

Assignees: @nazmulidris @shayes12111

Currently, TWCommandQueue is a simple queue. The ordering in which commands are executed to paint is first in, first out. This can lead to some unintended consequences when painting UI since things might get clobbered unintentionally. Also, it is important to have some idea of priority when painting since a lot of UI gets rendered in layers. Eg: in an editor you have the following layers:

1. base layer that renders the content
2. layer above that which renders carets (local & remote)
3. layer above that which might render a status bar
4. layer above that which might have a dialog box

Currently there is no easy way to manage this. I propose that a z-index number be added to TWCommandQueue which shards the single queue into a bunch of different queue which map to the z-index number. So this is what the data structure might look like.

command queue
   + z-index: 0 => queue of commands
   + z-index: 1 => queue of commands
   + z-index: 2 => queue of commands  
   .. etc

enum ZOrder {
  Normal, // 0
  High,   // 1
  Caret,  // 2
  Glass   // 3
}

This can be accomplished by simply adding a number argument to TWCommand itself. And then the queue can behave like a priority queue when performing its actual painting (taking the commands produced by the render phase and actually executing them against stdout).

a
░
bcd

Observed behavior: If you place the care where ░ is, then press Right arrow. Nothing happens.
Desired behavior: It should move down to the next line.

1. Mimic the <span> tag in HTML and create a struct that can support vecs of styled text (String, Style).
2. Make it compatible w/ UnicodeStringExt.

Repro steps:

This bug surfaced when creating a simple layout which contains just 1 box in a surface. This box ignores the computed style margin. In Surface the add_root_box() is not taking computed style into account in the same was as add_box().

Desired behavior:

Even if a surface has a single box which is the root, the styling should be applied.

Let us know about features you really want to see in r3bl_rs_utils library.

References

If the feature you are interested in exists in other CLI or TUI apps, please share links to documentation or screenshots to easily communicate the desired behavior!

• rwf2/Rocket#1735 (comment)
• https://notado.substack.com/p/do-you-really-need-a-message-queue?sd=pf
• https://github.com/smallnest/concurrency-programming-via-rust

Currently if you run r3bl-cmdr and make the terminal window very small then it looks bad. Non deterministic even.

It also caused this panic fixed by this commit bbecf32

Desired behavior should be

1. the main_event_loop needs to be told what the min size is (just like it needs to be told what exit_keys are
2. if the size drops below the min, then only a warning should be displayed like this "min size of $col x $row is needed. Press $exit_keys to exit."

Here is a non exhaustive list of things that require some min size:

• DialogEngineApi::flex_box_from(..)

Currently the Stylesheet struct has to be build imperatively. Along the lines of introducing style! DSL for Style struct, we need a stylesheet! macro for Stylesheet struct.

Here's the imperative style of code.

use crossterm::event::*;
use r3bl_rs_utils::*;

fn create_stylesheet(&mut self) -> CommonResult<Stylesheet> {
  // Turquoise:  Color::Rgb { r: 51, g: 255, b: 255 }
  // Pink:       Color::Rgb { r: 252, g: 157, b: 248 }
  // Blue:       Color::Rgb { r: 55, g: 55, b: 248 }
  // Faded blue: Color::Rgb { r: 85, g: 85, b: 255 }
  throws_with_return!({
    let mut stylesheet = Stylesheet::new();

    stylesheet.add_styles(vec![
      style! {
        id: style1
        margin: 1
        color_bg: Color::Rgb { r: 55, g: 55, b: 248 }
      },
      style! {
        id: style2
        margin: 1
        color_bg: Color::Rgb { r: 85, g: 85, b: 255 }
      },
    ])?;

    stylesheet
  })
}

So the declarative version might look like.

use crossterm::event::*;
use r3bl_rs_utils::*;

fn create_stylesheet(&mut self) -> CommonResult<Stylesheet> {
  // Turquoise:  Color::Rgb { r: 51, g: 255, b: 255 }
  // Pink:       Color::Rgb { r: 252, g: 157, b: 248 }
  // Blue:       Color::Rgb { r: 55, g: 55, b: 248 }
  // Faded blue: Color::Rgb { r: 85, g: 85, b: 255 }
  throws_with_return!({
    stylesheet!(
        style! {
          id: style1
          margin: 1
          color_bg: Color::Rgb { r: 55, g: 55, b: 248 }
        },
        style! {
          id: style2
          margin: 1
          color_bg: Color::Rgb { r: 85, g: 85, b: 255 }
        }
    )
  })
}

Related:

• See: #1
• Source code: https://github.com/r3bl-org/r3bl-rs-utils/blob/main/core/src/tui_core/styles/stylesheet.rs
• Test: https://github.com/r3bl-org/r3bl-rs-utils/blob/main/tests/test_style.rs#L158

Problem - Currently RenderOps and RenderPipline end up generating crossterm commands which are transmitted to stdout

• With this approach approach ZOrder can't be handled efficiently.
• It is currently handled by fully flushing crossterm commands at a higher ZOrder followed by a lower ZOrder and so on.
• Take an ex of a modal dialog, which requires some inefficient painting to stdout for each ZOrder that has to be rendered.
• To make things worse, crossterm is slow, and when using bg color styled text, or lots of fg_colors (eg: lolcat) it just falls on its face and flickers like crazy and this is unacceptable. The compositor should fix this by only painting diffs.
• Finally, crossterm or termion, there's the issue of painting -> clearing -> painting, which causes bad flickering. The compositor should eliminate this by handling all the intermediate drawing, clearing, redrawing offscreen then putting only the diffs onscreen.

Solution - create a compositor that takes the RenderPipeline and translates it into an intermediate representation (IR) which is quite simply a 2d array of PixelChar

PixelChar is a struct that is defined as:

1. GraphemeSegment
2. style information (from style: fg_color, bg_color)
3. attrib information (from style: bold, italic, etc)

There are as many PixelChars as there are window_size.col * window_size.row.

So the RenderPipeline has to be rasterized into a grid of PixelChars. Then this grid can be painted using whatever output device & library:

1. crossterm
2. termion
3. webgl
4. icedrs

One other thing that changes between these different outputs is the way input is captured. However, all that input can be normalized to a generic InputEvent type that currently exists today.

Notes

Might also need to use a diffing library like imara-diff to calculate the diffs between a sequence of PixelChar grids.

• https://www.reddit.com/r/rust/comments/ydi7xu/announcing_imaradiff_a_reliably_performant/
• https://github.com/pascalkuthe/imara-diff

Here's another rasterizer built atop termion

• https://github.com/nazmulidris/rust-sloth

Add tests for render_pipeline.rs (in test_render_pipeline.rs):

1. test pipeline painting w/ unicode
2. test pipeline painting w/ ANSI text
3. routing to the appropriate backend (currently only crossterm)

Currently the crate unicode-width is used to calculate the displayed width of unicode characters.

These calculations are mostly correct for:

1. simple emoji like: 😀. Display width and unicode width is 2.
2. It of course works on normal ASCII characters like: ░. Display width and unicode width is 1.
3. It fails on compound emoji like: 🙏🏽. Display width is 2, but unicode width is greater.

Here's where it gets interesting:

1. It "works" on tilix, gnome terminal, etc. since they don't support complex emoji combinations.
2. It does (ironically) NOT work on kitty, which correctly supports complex emoji combinations!

Here are some approaches to fix this:

1. In keypress.rs
   1. In keypress.rs which converts incoming KeyEvent into Keypress, simply filter out the complex characters that would "mess up" unicode width. This means that a simple grapheme cluster like 👍 would be allowed, but 👍🏽 would be (SILENTLY) rejected. This would entail creating a blacklist of unicode characters or just a whitelist (and reject all others).
   2. This would ensure that a subset of emoji would work on kitty and tilix, and even on macos.
2. In editor_buffer.rs
   1. Once the content has been added to the editor buffer it would be simple to strip out grapheme clusters that are wider than some threshold (eg: unicode_width > 4 or something).

Source files to work on:

1. https://github.com/r3bl-org/r3bl_rs_utils/blob/main/src/tui/crossterm_helpers/keypress.rs
2. https://github.com/r3bl-org/r3bl_rs_utils/blob/main/core/src/tui_core/graphemes/unicode_string_ext.rs

The semantics of style! margin is really padding. Best to rename all usages to the new field name & update style! & stylesheet! macros.

Usecase

As a developer, I was trying to understand the project and found myself wondering and confused by how the utils and cmdr repo were named and what their responsibilities are. After cloning the project and taking a closer look it seemed like r3bl_utils contained the core code of this awesome library and the cmdr repo contained examples.

Confusing Aspect

• Utils to me means a utility, or something extra, its a bunch of helper functions/structs. Thus by that token, I was not expecting it to contain the core logic of this project and hence why the confusion arose.
• Making some configuration changes/code changes in one project doesn't affect the other projects. It invokes duplication in maintaining related projects in separate repos.

Proposal

Something like:

r3bl-org (organisation)
    - r3bl (workspace)
             -  r3bl-core (crate)
             -  r3bl-macro (crate)
             -  r3bl-cmdr (crate)

Single Workspace

I have seen many projects keeping the related code bases inside a single workspace. It is very common in the rust ecosystem to keep code related to a single project close to each other and within the same workspace. This provides many benefits such as:

• Single CI
• Single project configuration (rustfmt, clippy, vscode, etc.)
• Single issue, PR boards.
• Single landing page for users to land on.
• Easier to start coding and testing (at the moment the cmdr project refers to a dependency to utils by local path but one needs to assure it exists first)
• Workspace-wide configurations

How

The following steps would have to be taken

• Rename this repo from r3bl_rs_utils to r3bl, this is the project name and hence a natural fit for this repo. This project already has 22 stars so would be nice to reuse this repo and just rename.
• Move a clone of r3bl_rs_utils to a new repository
  • Update the README of this new repo with bold letters DEPRECATED
  • Release a new cargo version of this r3bl_rs_utils crate that shows it is deprecated.
• Move cmdr into the r3bl repo and create a workspace
• Depreciate the cmdr repository (or delete it as it likely doesnt have many users yet)
• Update the Cargo.toml and rename the main project to r3bl
• Release r3bl.

Discussion

You would end up having to deprecate r3bl_rs_utils but since it isn't used that much yet it should not be a problem. Those kinds of changes is better early than too late. Just publish a new version with a README saying it's deprecated and link to the new project, this repo, under its new name.

What do you think?

Currently, Component is not reusable. Make it so that it is, and make EditorComponent reusable as well.

Related to #23

Change the meaning of RenderPipeline

It is a collection of atomic paint operations (aka RenderOps at various ZOrders); each RenderOps is made up of a vec of RenderOp. It contains Map<ZOrder, Vec<RenderOps>>, eg:

• ZOrder::Normal => vec![ResetColor, MoveCursorPositionAbs(..), PrintTextWithAttributes(..)]
• ZOrder::Glass => vec![ResetColor, MoveCursorPositionAbs(..), PrintTextWithAttributes(..)]
• etc.

What is an atomic paint operation?

1. It moves the cursor using:
   1. RenderOp::MoveCursorPositionAbs
   2. RenderOp::MoveCursorPositionRelTo
2. And it does not assume that the cursor is in the correct position from some other previously executed operation!
3. So there are no side effects when re-ordering or omitting painting an atomic paint operation (eg in the case where it has already been painted before).

Paint optimization

In order to ensure that things on the terminal screen aren't being needlessly drawn (when they
have already been drawn before and are on screen), it is important to keep track of the position
and bounds of each [RenderOps]. This allows [optimized_paint::clear_flex_box] to perform its
magic by clearing out the space for a [RenderOps] before it is painted. This is needed for
managing cursor movement (cursors are painted when moved, but the old cursor doesn't get
cleared).

Tasks

• Change RenderPipeline to hold a Vec<RenderOps> for each ZOrder
• Update the docs for TWData to make it explicit when the cursor_position is safe to use.
• Copy the content of this comment into RenderOps
• Implement "dirty paint" algorithm
  • Save the rendered pipeline in TWData
  • re-render when the # of render ops is different in each layer
  • other cases for not re-rendering

A more comprehensive fix to this will arrive when this issue is fixed #46

Provide a constructor where the following is specified

1. the initial seed
2. the speed at which the colorwheel changes

Provide a method to manipulate the seed easily

1. have meaningful enums for rapid or slow change of seed

In the dialog editor make sure to have a lolcat init'd w/ the same seed so that it doesn't change every time the app is run

dialog module:

• handle style locat is true

jsx module:

• rename jsx macro names (layout and render macros)

Replace the use of the following structs w/ the following macros (which are more readable & easier for developers to use, ie, more ergonomic):

Remove crossterm deps from TWInputEvent:

• Currently the TWInputEvent enum has a variant that uses KeyEvent. Remove this dependency & replace it w/ something along the lines of DisplayableKeypress variant.

Clean up API:

• Move the does_input_event_match_exit_keys function into a method for TWInputEvent & update usages.
• Replace usages of crossterm Color w/ TWColor.
• Remove usage of KeyEvent & KeyCode w/ KeypressEvent.

From: https://discord.com/channels/273534239310479360/943315667430563862/1019937035462647840

1. https://wezfurlong.org/wezterm/install/linux.html
2. https://crates.io/crates/portable-pty/0.7.0

C++ impl of a pseudo terminal to allow terminals to be embedded in other apps:
https://github.com/FredrikAleksander/HexeTerminal

Currently EditorBuffer::lines represents the contents of the editor as a Vec<String>. While this might accurately represent a buffer when it is saved to disk, this might not be the best data model. The editor really treats content as a Vec<UnicodeString> and it has to convert back and forth between display_col and vec index. This is not really ergonomic and the logic suffers due to this.

A better approach might just to be rewrite the EditorBuffer using Vec<UnicodeString> rather than Vec<String>.

Currently this r3bl_rs_utils crate is made up of the following crates (deps):

1. r3bl_macro (depends on r3bl_core)
2. r3bl_core

The idea is to break up this crate into even smaller ones:

1. r3bl_redux (depends on r3bl_macro, r3bl_core)
   1. Move tests from tests/ into whatever module it is relevant for
2. r3bl_tui (depends on r3bl_macro, r3bl_core, r3bl_redux)
   1. The examples folder will have to be moved into here

Once these are moved out, whatever functionality remains will be r3bl_rs_utils

1. Upgrade to crossterm 0.25.0 https://docs.rs/crossterm/0.25.0/src/crossterm/event.rs.html which changes KeyEvent and a lot of other things.