Add type-safe intermediate step for node names about sapling

[kneasle]

Ok right here's what I think should be done. Currently we use chars as node specifiers right down until a new Node is created, which has been causing loads of issues because char has no type safety. So what we'd like to do is to catch the invalid nodes as early as possible so that from then on, we don't have to worry about creating nodes from invalid chars.

So what I'm thinking is that we allow Asts to specify an associated type for node types:

pub trait NodeClass: Copy + Debug + Eq + Hash {
    /// Gets the [`char`] that would have been used to create this value
    fn to_char(self) -> char;

    /// Returns the name of this value
    fn name(self) -> &'static str;

    // This is called as soon as possible when performing an edit, generating
    //  a `EditErr::CharNotANode` if this fails
    fn from_char(c: char) -> Option<Self>;
}

pub trait Ast<'arena>: ... {
    type Class: NodeClass;

    // This is used where `from_char` is used currently.  Notice how this can't fail, 
    // because the validness is enforced by the type system.
    fn from_class(node_type: Self::Class) -> Self;
}

Now, instead of using char as far as possible, we do the following:

• Call Node::Class::from_char() as soon as possible to get a type-safe value which must be a valid type, generating a EditErr::CharNotANode on failure
• Perform all the other checks using Node::Class instead of Char
• Use Node::from_class instead of Node::from_char to create the node

[stokhos]

It seems like you had a question and then deleted it... I guess it's been resolved? If you have questions or are confused, then feel free to ask :).

Yes. I have kind of solved it.

[kneasle]

trait Ast<'a>: Sized {
    fn node_function() -> Box<dyn Fn(char) -> Option<Self> + 'a>;
}

I think this whole example would work a lot better if this were fn node_function(char) -> Option<Self>. You can call this using Json::node_function('t'), without needing to store anything inside Dag.

There are several parts that I dont understand:

1. In impl<'a, Node: Ast<'a> + 'a> Editor<'a, Node> is the second 'a in Node: Ast<'a> + 'a> is necessary in here?

It shouldn't be, but I think it is a bi-product of the strange function magic that's going on. The + 'a trait bound is telling Rust the lifetime of the datatype Node rather than the lifetime of the data referred to by Node. I'm pretty sure that Sapling's code somehow avoids this...

2. Why I'm having the error: tree dropped here while still borrowed?

I don't know - everything should be Dropped in the reverse order to the way they were defined, so that should be fine.

3. Can we have Sized trait on Ast? I haven't read much over sized trait, but i feel we can add this trait bound.

Hmmm... the Sized trait always confused me a little because sometimes the compiler adds it for you and sometimes it doesn't. Either way, the Arena won't work unless Ast: Sized so I thought it was explicit. We should probably add Sized as an explicit bound.

4. Why arena is a immutable? I thought it should be mutable, because we add nodes to it. I must missed something over here, can you explain this to me?
   

   sapling/src/main.rs

   Line 34 in 241a945

   let arena = Arena::new();

I believe that the Arena type is using something called 'interior mutablity'. Basically, using unsafe you can set up a situation where a datatype can mutate its own data without a mutable reference (I think Cell and RefCell are the most common examples) - it's pretty bizarre but very useful for things like allocators which need to be passed everywhere.

[kneasle]

This bit of the code is currently hard to understand, and will probably get replaced in future.

1. I'm confused, which node will be the new root? 'B'. If it is 'B', how does Dag link 'B' to 'P'? Node 'A' and its children 'C1' 'C2' are still stored in Dag. this is why you mentioned to use 'Rc' in the other issue?

Sort of. A, C1 and C2 are both still needed in case the user performs an undo - so should be kept in the Arena. But if the user 'throws away' edit history by undoing and performing edits, then there are nodes that won't be used - and in this case, Rc would be useful.

Here's an example of what will and won't be cloned. So supposing we have the following tree (with the cursor represented by <...>):

root
  child1
  child2
    sibling_prev
    <cursor>
    sibling_next
  child3

and we replace <cursor> with <cursor'>. Sapling can't change any existing nodes (because they're immutable), and so every ancestor of the <cursor> (child2 and root) will have to be cloned. So the new tree will be the following (notice how every node that isn't in the path root -> child2 -> cursor is not cloned):

root'
  child1
  child2'
    sibling_prev
    <cursor'>
    sibling_next
  child3

Therefore, each node in the tree can have multiple parents from different 'revisions' of the tree (as an example, both root and root' are parents of child1).

I'm not sure if this makes sense... if it doesn't just say 😁.

2. If I want to see what is stored in Dag.root_history, is there a way to check this when sapling is running?

Not easily. You could debug print Dag.root_history, but it wouldn't show up anywhere because Sapling is using the whole terminal. I made #13 for this reason, but it hasn't been implemented yet.

[stokhos]

It whole idea of using Dag makes a lot sense to me now! Thank you very much!

[kneasle]

How does terminal know which tree to render?

Once you have a tree root, then generating the tree is a matter of following child pointers down the DAG of nodes. Therefore Dag stores the undo history as a sequence of tree roots in Dag::root_history. When asked by Editor for the current tree with Dag::root, the Dag uses history_index to pick the correct tree root (and the rest of the nodes follow automatically).

This means that undo/redo are just subtracting and adding one to this counter, without having to actually change any tree nodes.

[stokhos]

Yes I'd like to work on this
@kneasle

[kneasle]

It seems like you had a question and then deleted it... I guess it's been resolved? If you have questions or are confused, then feel free to ask :).

[kneasle]

In fact, thinking about it, I think that the NodeClass trait should be called either AstClass (to be consistent with the trait Ast) or simply Class (to be succinct). I personally have a slight preference towards Class, but it's largely up to you what you want to call it.

[stokhos]

I have a question regarding the NodeClass, after defining the trait.
Do I need to define a struct and implement the NodeClass on it? Or implement NodeClass on Json?

[kneasle]

Do I need to define a struct and implement the NodeClass on it?

This (except I'd strongly recommend using an enum) 😁. The whole point of Class/NodeClass is to allow Ast implementations to specify another datatype which serves as a type-safe alternative for char.

So Json would look something like this:

src/ast/json.rs

// -- snip --

#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
pub enum Class {
    True,
    False,
    Null,
    Str,
    Array,
    Object,
}

impl NodeClass for Class {
    // For you to do :wink:
}

// -- snip --

impl Ast for Json {
    // The first 'Class' refers to Ast::Class, whereas the second is the enum we just defined
    type Class = Class;

    ...
}

Each implementer of Ast will specify its own type for Ast::Class which it wants incoming chars to map to. The important thing about Ast::Class is that it must always be possible to turn an Ast::Class into an Ast node (whereas with chars this would sometimes fail, causing some hard-to-track-down bugs).

Does this make more sense? - if not then don't hesitate to ask more questions 😃... the thing is that I don't want to always explain everything because I think it would come across as a bit patronising. But that means that sometimes I leave things up to confusion so if that happens please ask more questions 😁.

[stokhos]

Does this make more sense? - if not then don't hesitate to ask more questions smiley... the thing is that I don't want to always explain everything because I think it would come across as a bit patronising. But that means that sometimes I leave things up to confusion so if that happens please ask more questions grin.

Yes. It make sense now.

Does this make more sense? - if not then don't hesitate to ask more questions smiley... the thing is that I don't want to always explain everything because I think it would come across as a bit patronising. But that means that sometimes I leave things up to confusion so if that happens please ask more questions grin.

Actually, I don't think its your fault. I think it my problem, that I wasn't trained to be a programmer. And what is straight forward to you may not be so straight forward to me. But I'm trying ; )

[stokhos]

Since Class and Json would have almost same items, how about we implement NodeClass for Json.

The code would change to:

ast/mod.rs

impl NodeClass for Json {
    fn from_char(c: char) -> Option<Self> {
        match c {
            CHAR_TRUE => Some(Json::True),
            CHAR_FALSE => Some(Json::False),
            CHAR_NULL => Some(Json::Null),
            CHAR_ARRAY => Some(Json::Array(vec![])),
            CHAR_OBJECT => Some(Json::Object(vec![])),
            CHAR_STRING => Some(Json::Str("".to_string())),
            _ => None,
        }
    }
}

And we check validate incoming char before self.perform_edit.
editor/dag.rs

    // check the validity of `char` first.
    fn insert_child(&mut self, c: char) -> EditResult {
        Json::from_char(c).ok_or(CharNotANode(char))?;
        self.perform_edit(
            |this: &mut Self,
             _parent_and_index: Option<(&'arena Node, usize)>,
             cursor: &'arena Node| {}

Then I think inside is_valid_child(index:usize, child: Json) would be something like
ast/json.rs

impl<'arena> Ast<'arena> for Json<'arena> {
    fn is_valid_child(&self, index: usize, child: Self) -> bool {
        match self {
            // values like 'true' and 'false' can never have children
            Json::True | Json::False | Json::Str(_) | Json::Null => false,
            // arrays and objects can have any children (except `field` inside `array`, which can't be inserted)
            Json::Array(_) | Json::Object(_) => true,
            // fields must have their left hand side be a string
            Json::Field(_) => {
                if index == 0 {
                    // check child is Json::Str
                    child.is(Self::Str(String))
                } else {
                    true
                }
            }
        }
    }
}

[stokhos]

I guess the reason we define a new enum Class is because we need to use Class in dag.rs, and we want it to be more general

[kneasle]

Since Class and Json would have almost same items, how about we implement NodeClass for Json.

Yeah I'm not the biggest fan of that duplication, but I think having a Class enum is going to be useful. For example, let's say that we copied a node and want to paste it in - we need to check that the node is of the correct type without cloneing it, and in this case it would be much better to have an enum.

And we check validate incoming char before self.perform_edit.

This is a good idea - I hadn't thought of it 😆. But I think having another enum will be useful for the error messages (at the moment we're returning loads of (char, String) pairs which could just become a Class.

At any rate, if it turns out that this is the wrong solution we can just go back to the old one - it feels like 'wasted' effort but it's pretty common to have to try several ways of solving a problem.

[stokhos]

fn main() {
    let v = vec![Json::True, Json::Obj(vec![])];
    let mut tree = Dag::new(&v);

    let e = Editor::new(&mut tree);
    let t = (e.tree.node)('t');
    println!("{:?}", t.unwrap());
}

trait Ast<'a>: Sized {
    fn node_function() -> Box<dyn Fn(char) -> Option<Self> + 'a>;
}

struct Dag<'a, Node: Ast<'a>> {
    x: &'a Vec<Node>,
    node: Box<dyn Fn(char) -> Option<Node> + 'a>,
}

impl<'a, Node: Ast<'a>> Dag<'a, Node> {
    fn new(x: &'a Vec<Node>) -> Self {
        Dag {
            x,
            node: Node::node_function(),
        }
    }
}

#[derive(Debug)]
enum Json<'a> {
    True,
    False,
    Obj(Vec<&'a Json<'a>>),
}

impl<'a> Ast<'a> for Json<'a> {
    fn node_function() -> Box<dyn Fn(char) -> Option<Self> + 'a> {
        Box::new(move |x| match x {
            't' => Some(Json::True),
            'f' => Some(Json::False),
            'o' => Some(Json::Obj(vec![])),
            _ => None,
        })
    }
}

struct Editor<'a, Node: Ast<'a>> {
    tree: &'a mut Dag<'a, Node>,
}

impl<'a, Node: Ast<'a> + 'a> Editor<'a, Node> {
    fn new(tree: &'a mut Dag<'a, Node>) -> Editor<'a, Node> {
        Editor { tree }
    }
}

This is one of my experiments on this issue. You can access the code from here. I don't know why it doesn't work with the Editor.

There are several parts that I dont understand:

1. In impl<'a, Node: Ast<'a> + 'a> Editor<'a, Node> is the second 'a in Node: Ast<'a> + 'a> is necessary in here?
2. Why I'm having the error: tree dropped here while still borrowed?
3. Can we have Sized trait on Ast? I haven't read much over sized trait, but i feel we can add this trait bound.
4. Why arena is a immutable? I thought it should be mutable, because we add nodes to it. I must missed something over here, can you explain this to me?
   

   sapling/src/main.rs

   Line 34 in 241a945

   let arena = Arena::new();

[stokhos]

For a tree with root 'R', If I replace a node 'A' (with parent P, children 'S_prev', 'A', 'S2_next'), node 'A' has children 'C1', 'C2', how does Dag deal with this situation?
From my understanding, It clones the tree down to node 'A'.

sapling/src/editor/dag.rs

Lines 328 to 331 in 241a945

	// Because AST nodes are immutable, we make changes to nodes by entirely cloning the path
	// down to the node under the cursor. We do this starting at the node under the cursor and
	// work our way up parent by parent until we reach the root of the tree. At that point,
	// this node becomes the root of the new tree.

1. I'm confused, which node will be the new root? 'B'. If it is 'B', how does Dag link 'B' to 'P'? Node 'A' and its children 'C1' 'C2' are still stored in Dag. this is why you mentioned to use 'Rc' in the other issue?
2. If I want to see what is stored in Dag.root_history, is there a way to check this when sapling is running?

[stokhos]

Therefore, each node in the tree can have multiple parents from different 'revisions' of the tree (as an example, both root and root' are parents of child1).

How does terminal know which tree to render?