[Feature Request][Table] Lifetime support? Let table take K instead of &K? about move

[wrwg]

Bummer. This is a result of the over-approximation of the borrowing graph in Move. For the call let r = Table::borrow(&vec.buckets, &bucket_idx), Move assumes that r is borrowing from both input parameters, because in contrast to Rust, there are no borrow annotations which could tell it that it borrows only from the first parameter.

I don't know whether there is a possible workaround by hacking the compiler to make it aware of more fine-grained borrow information for certain native functions,

Otherwise, the only option is to make table keys K:copy.

This is just another instance of that Move's borrow semantics is unfortunately not always working as expected, because of its simplifications compared to Rust.

[tnowacki]

A quick response, and I can give a more detailed one later (traveling today).

• Lifetime support is not easy to add in the current system. There is too much ambiguity in the borrow graph in relation to equivalent references.
• @sblackshear and I have been working a long time on a new borrow checker formalization (and in turn implementation). This set-based approach will be more amenable to lifetime support.
• I can try to add this new set-based approach in the next month or so. (hopefully)
• From there, I can describe how one might implement lifetime support, but will likely not have time to implement it myself. I think it should be relatively straightforward though, but it will be a MASSIVE change, as it will touch every part of Move

[sblackshear]

If tables are the main driver for this change, another option is to consider one of the older proposals for "static" tables (vs the dynamic variant that we have implemented). These require somewhat different design patterns, but will not require borrow checker changes.

There was a detailed design doc for static tables somewhere in FB google docs, but unfortunately I don't think there's a public copy I can point to.

[jolestar]

Maybe these are two issues:

1. Table API, I still insist that we should make table keys K:copy and use values as keys other than references, as we discussed at diem/move#150.

2. Lifetime, It should be a long time plan.

[wrwg]

If tables are the main driver for this change, another option is to consider one of the older proposals for "static" tables (vs the dynamic variant that we have implemented). These require somewhat different design patterns, but will not require borrow checker changes.

There was a detailed design doc for static tables somewhere in FB google docs, but unfortunately I don't think there's a public copy I can point to.

Check out this PR discussion why the borrow semantics problem is more fundamental issue than just tables. The Move language is lacking compositionally. IMO the borrow semantics in Move is incomplete and therefore disallows simple patterns like foo(&r1, &r2). With borrow(&table, &key) we see only one instance of this. Moving forward, the suggestion is we adapt tables using a copyable key, which is unfortunate, but the best pragmatic way for now.

[wrwg]

Maybe these are two issues:

1. Table API, I still insist that we should make table keys K:copy and use values as keys other than references,

You don't mean insist but suggest? We already had longish discussions about this and a majority decision for &K. We would like to have &K. But we can't for now because of the incomplete Move borrow semantics.

2. Lifetime, It should be a long time plan.

[jolestar]

You don't mean insist but suggest? We already had longish discussions about this and a majority decision for &K. We would like to have &K. But we can't for now because of the incomplete Move borrow semantics.

Yes, even if we have lifetime support, I still suggest key:K and make K:copy.

I can not imagine any possible application scenario where we need key:&K. The most common types of keys are address, number, and hash.

Yes, if we have the Lifetime support, we can do it, but I think we don't need to do it.

[wrwg]

On Fri, May 6, 2022 at 6:36 PM jolestar ***@***.***> wrote: I can not imagine any possible application scenario where we need key:&K. The most common types of keys are address, number, and hash.

A 32 byte hash is a good example. Why would you want to pass this by value and make an unnecessary copy? Another example would be `ASCII::String`. Passing a copy of strings is just awkwardly inefficient and unnecessary. For comparison, in Rust *all* functions to access a collection by key take a reference to the key. In Move, we can't implement any function where the main argument we borrow from is a reference, and where any auxiliary data passed in addition is also a reference.

…

Yes, if we have the Lifetime support, we can do it, but I think we don't need to do it. — Reply to this email directly, view it on GitHub <#95 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC2MSOXJMMPUAYRDJ3V2RILVIXCIHANCNFSM5VANGXAA> . You are receiving this because you were assigned.Message ID: ***@***.***>

[jolestar]

On Fri, May 6, 2022 at 6:36 PM jolestar @.***> wrote: I can not imagine any possible application scenario where we need key:&K. The most common types of keys are address, number, and hash.
A 32 byte hash is a good example. Why would you want to pass this by value and make an unnecessary copy? Another example would be ASCII::String. Passing a copy of strings is just awkwardly inefficient and unnecessary. For comparison, in Rust all functions to access a collection by key take a reference to the key. In Move, we can't implement any function where the main argument we borrow from is a reference, and where any auxiliary data passed in addition is also a reference.

Sorry, I have mixed up two scenarios, add and borrow.

In the add scenario, I expect the argument to be key:K.
In the borrow scenario, you are right, I also want key:&K.

But I think K: copy is right in both.

[emmazzz]

Accidentally closed it

[zekun000]

not sure if this is a bug or feature. this snippet actually compiles (borrow_mut instead of borrow on the table)

    public fun borrow_with_idx<T: store>(vec: &mut BucketedVector<T>, idx: u64): &T {
        let bucket_idx = idx / vec.bucket_size;
        let item_idx = idx % vec.bucket_size;

        let bucket = Table::borrow_mut(&mut vec.buckets, &bucket_idx);
        Vector::borrow(bucket, item_idx)
    }

[tnowacki]

not sure if this is a bug or feature. this snippet actually compiles (borrow_mut instead of borrow on the table)

    public fun borrow_with_idx<T: store>(vec: &mut BucketedVector<T>, idx: u64): &T {
        let bucket_idx = idx / vec.bucket_size;
        let item_idx = idx % vec.bucket_size;

        let bucket = Table::borrow_mut(&mut vec.buckets, &bucket_idx);
        Vector::borrow(bucket, item_idx)
    }

A feature of sorts. It is because you are using Table::borrow_mut.

Lets assume a signature of Table::borrow_mut(table: &mut Table<K, V>, key: &K): &mut V

The borrow checker knows that the return value for of type &mut V cannot possibly be extending &K because there is no way to extend an immutable ref & and get a mutable ref &mut. In other words, the borrow checker safely knows that a mutable reference return value does not extend any immutable reference arguments.

[msmouse]

We ended up converting the Table interfaces to pass by value.
#109