Support for 0-Dimensional Tensors in Burn about burn

[antimora]

@nathanielsimard and I had an offline conversation.

Here's a summary of the conversation for others:

We discussed the need to support scalar tensors in the Burn deep learning framework. While scalar values can be encoded as rank-1 tensors, the main issue is the lack of an automatic broadcasting API in Rust stable due to limitations with const generics.

As a better long-term solution, we proposed introducing a new Scalar type, which would be an enum that can hold either a native value (e.g., f32) or a rank-1 tensor. This explicit Scalar type would provide more security and avoid unnecessary broadcast operations. It would also be beneficial for exporting to other formats like ONNX, since all operation can be tracked in a computation graph.

We plan to modify the burn_tensor module to include this Scalar type, with variants like Scalar<Int>, Scalar<Float>, and Scalar<Bool>. This change would not introduce any breaking changes to the existing API.

Overall, while the naming and exact implementation details still need to be finalized, we agreed that introducing a dedicated Scalar type is a good idea to handle scalar values properly in the Burn framework.

[antimora]

CC @nathanielsimard , @laggui , @louisfd

[antimora]

@LaurentMazare has confirmed Candle supports 0D tensors:

Zermelo Fraenkel: Scalars values (tensors with 0 dimension) should be supported. Empty tensors (multiple dimensions but with one of them being zero) should also be supported but only to some extent. Certainly interested if you find places where this doesn't work properly.

PyTorch supports 0D tensors

[antimora]

cc @ashdtu

[antimora]

@laggui found that Ndarray supports 0dim arrays: https://docs.rs/ndarray/latest/ndarray/type.Array0.html