tokio-rs / valuable Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
The current error variant in Value::Error
is a plain &dyn Error
, which unfortunately doesn't support downcasting. It might be worth changing this to &(dyn Error + 'static)
, which seems to be a reasonably accepted restriction on the type to support any scenario a consumer might have.
AFAIK, in the current API, Valuable
cannot be implemented well if I need a temporarily owned value for serialization.
struct PathAsString(std::path::PathBuf);
impl Valuable for PathAsString {
fn as_value(&self) -> Value<'_> {
/*
error[E0515]: cannot return value referencing temporary value
--> lib.rs:74:9
|
74 | Value::String(&self.0.display().to_string())
| ^^^^^^^^^^^^^^^----------------------------^
| | |
| | temporary value created here
| returns a value referencing data owned by the current function
error: aborting due to previous error
*/
Value::String(&self.0.display().to_string())
}
fn visit(&self, v: &mut dyn Visit) {
v.visit_value(Value::String(&self.0.display().to_string()))
}
}
Is there a way to handle such cases well?
(I was thinking a bit about removing the Valuable::as_value
, but not sure if that would be a good idea.)
This should line up better w/ HashMap.
For example, we could a type's module path.
My motivating use-case is to write down in tracing 0.2
a visitor that is able to borrow Event
’s message
. Doing so today is impossible even though lifetimes should conceptually mostly work out. This problem extends to valuable::Value
and its Visit
trait as well, so the remaining of the issue will focus on valuable
.
So lets say we begin by writing a function as such (any mistakes mine, I basically sketched this out in a github comment field):
fn extract_message<'val>(value: &'val valuable::Value<'val>) -> &'val str {
struct Visitor<'val> {
message: &'val str
}
impl<'val> valuable::Visit for Visitor<'val> {
fn visit_entry(&mut self, key: valuable::Value<'_>, value: valuable::Value<'_>) {
match (key, value) {
(valuable::Value::String("message"), valuable::Value::String(v)) => {
self.message = v; // ERROR: `'_` does not outlive `'val`.
}
_ => {}
}
}
}
let mut visitor = Visitor<'val> { message: "" };
value.visit(&mut visitor);
visitor.message
}
Rust today provides no way to tell that, '_
will outlive 'val
here, and so users are forced to allocate in the implementation to extract the string data here.
The proposed solution here is to introduce a lifetime to Visit
such that Visit
becomes like this:
pub trait Visit<'a> {
fn visit_value(&mut self, value: Value<'a>);
fn visit_named_fields(&mut self, named_values: &NamedValues<'a>) { ... }
fn visit_unnamed_fields(&mut self, values: &[Value<'a>]) { ... }
fn visit_primitive_slice(&mut self, slice: Slice<'a>) { ... }
fn visit_entry(&mut self, key: Value<'a>, value: Value<'a>) { ... }
}
This gives implementers an optional opportunity to tie the lifetime of values to the lifetime of the visitors contents. Any implementors wanting the current behaviour can write the implementation as such:
// Unfortunately a breaking change – you cannot leave out the `<'_>`.
impl Visit<'_> for SomeVisitor { ... }
While implementations such as one presented in the motivating use-case above are also made possible:
impl<'val> valuable::Visit<'val> for Visitor<'val> {
fn visit_entry(&mut self, key: valuable::Value<'val>, value: valuable::Value<'val>) {
match (key, value) {
(valuable::Value::String("message"), valuable::Value::String(v)) => {
self.message = v; // A-OK!
}
_ => {}
}
}
}
The coerce
actor crate uses the valuable
crate. In my personal project, which uses both tracing
and coerce
, I'm unable to build the coerce
("full" features) dependency (i.e., cargo fails trying to check/build coerce v0.8.5
before it starts on my project). Oddly, I'm able to build and fully test the coerce
library itself, which suggests something is missing in my project/dependencies definition, although I have strived to match coerce
's dependencies. I have posted an issue in the coerce
project, and I'm posting here in here there's something in using valuable
I missed in the documentation. I appreciate any guidance you may be able to provide. My project can be found at https://github.com/dmrolfs/coerce-cqrs-rs.
The issue I see is:
cargo c
...
Checking coerce v0.8.5
error[E0277]: the trait bound `valuable::Value<'_>: tracing::Value` is not satisfied
--> /Users/rolfs/.cargo/registry/src/github.com-1ecc6299db9ec823/coerce-0.8.5/src/actor/lifecycle.rs:88:28
|
88 | let span = tracing::info_span!(
| ____________________________^
89 | | "actor.recv",
90 | | ctx = log.as_value(),
91 | | message_type = msg.name(),
92 | | );
| |_________________^ the trait `tracing::Value` is not implemented for `valuable::Value<'_>`
|
= help: the following other types implement trait `tracing::Value`:
&'a T
&'a mut T
(dyn StdError + 'static)
(dyn StdError + Sync + 'static)
(dyn StdError + std::marker::Send + 'static)
(dyn StdError + std::marker::Send + Sync + 'static)
Arguments<'a>
Box<T>
and 34 others
= note: required for the cast from `valuable::Value<'_>` to the object type `dyn tracing::Value`
= note: this error originates in the macro `$crate::valueset` which comes from the expansion of the macro `tracing::info_span` (in Nightly builds, run with -Z macro-backtrace for more info)
...
I have a use case where I need to "deserialize" Value
s into instances of struct Foo
.
This feels related to #17. Perhaps the impedance mismatch between valuable::Visit and serde::de::Visit is slight enough to be overcome with a generic implementation that allows deserializing a valuable::Value
into anything that implements serde::de::Deserialize
?
A while ago, I decided to start on an implementation that relies on a FromValue
trait and a corresponding proc-macro, which can fallibly convert valuable::Value
to any type that it's implemented for. I guess at the time, I must not have considered the serde solution. That implementation is available here.
Is this kind of functionality something that the team would be interested in incorporating into valuable
or valuable-serde
? If so, what should be the shape of that solution? Alternatively, if something seems fundamentally wrong about my use of tokio tracing, I would appreciate your feedback. Thanks very much!
I believe both types should just "flatten" the values instead of implementing Enumerable
.
So, Some(u32)
=> Value::U32
or Value::Unit
.
Plugin log messages should be correlated for initialization.
Suggest that a span is created around the call to plugin init that contains the following attributes:
Now that there are tests, we should setup the repo with Github Action for CI.
Currently, tuples do not implement Valuable
. How should they be represented? Probably Structable
but w/o a name? Alternatively, we would need to add a Tuple
variant to Value.
trait Tuplable
? 😬
The current signature of Fields::Named
hold a borrowed slice of NamedField
. As far as I can tell, it's designed for 'static
definition of rust structs. This makes it difficult to define dynamic Structable
for a map like we talked in #53 .
Perhaps we can make it a Cow
for the dynamic scenario.
At the moment tracing-opentelemetry
does a somewhat of a hack to know what fields are metrics. It requires the field name to be prefixed
To publish a new metric, add a key-value pair to your tracing::Event that contains following prefixes:
- monotonic_counter. (non-negative numbers): Used when the counter should only ever increase
- counter.: Used when the counter can go up or down
- value.: Used for discrete data points (i.e., summing them does not make semantic sense)
async fn current_example() {
tracing::info!(histogram.racecar.speed = 80_f64);
}
Instead if attributes were exposed it would be possible to do something like this:
/// How fast the racecar is going
#[derive(Valuable)]
#[histogram(key="racecar.speed", unit="mph")]
struct RacecarSpeed(f64);
#[tokio::test]
async fn proposed_example() {
tracing::info!("Going at {}mph", speed=RacecarSpeed(777.0012_f64));
}
I think there's two approaches here: expose the attributes directly, or expose the typeid and require subscribers to map that typeid to the attribute
It looks like most of the work to support visiting borrowed data for some concrete lifetime (instead of just for any lifetime) is already done, the Visit
trait just doesn't surface a lifetime that implementors can use to stash values across calls.
I at least find introducing lifetimes late in an API design can get a bit hairy so it might something to explore sooner rather than later 🙂
For example in templating engines, people will frequently print the value at the index n
of an array. Right now keeping track of the indices of a Listable
being visited is not triviall and as in the template engine case, we don't even want to visit all of them, just one.
Something like Listable::visit_slice(range: Range<usize>, &mut dyn Visit)
to allow for more flexibility would be great to have. A .get(usize)
would be nice too if possible
So valuable integration has started in tracing ( tokio-rs/tracing#1608 ) but a blocker for merging it is that valuable needs a release on crates.io. I'm aware there's a milestone for features for the 0.1 release so if that's not ready potentially an alpha release? Or depending on the milestone features punt them to a patch or the next version
The Visit
trait has a visit_entry
method that accepts a key-value pair. In cases where you’re bridging different APIs you might not be able to guarantee both the key and value are present at the same time to pass to this method. For instance, serde::SerializeMap
treats entries as an optimization and requires serializers support receiving keys and values independently. We ended up adding these separate methods to the standard library’s fmt
builders for this reason.
We could consider making visit_entry
forward to some new visit_key
and visit_value
methods.
We've published valuable
and valuable-derive
, but the serde
integration crate remains unpublished. We should probably get that ready to release as well?
It should be possible to implement Serde's Serialize
trait for valuable::Value
. This should enable using Serde's existing serializer ecosystem with valuable.
The Enumerable
trait and associated visit fns should be sketched out enough to implement derive for them.
Hi! 👋
Over in uuid-rs/uuid#583 we've been talking about adding unstable valuable
support to Uuid
and run into a bit of a design issue I wanted to get some input on.
It looks like we've got two possible directions we can go in:
Uuid
as its text-based format and use Value::Displayable
(#86). This means end-users in the diagnostics use-case will see something resembling a UUID rather than a blob of bytes without needing to match on strings (assuming tracing
will eventually run all values through valuable
), but it means reconstructing a Uuid
from that dyn Display
will require more work.Uuid
as its raw byte format and use Value::Listable
. This is less work than 1, and a more accurate representation of the UUIDs internal structure, but if we want a text-based UUID we need to look at its stringly-typed name and parse it.This seems like a general question that authors of primitive value types like UUIDs and IP addresses are going to need to answer, so was looking for some input on what you all think is the direction that best aligns with the goals of the valuable
project.
This is a list of cases where valuable-serde
behaves differently from using serde
directly.
valuable/valuable-serde/tests/test.rs
Lines 69 to 71 in 7a22d9d
valuable/valuable-serde/tests/test.rs
Lines 85 to 87 in 7a22d9d
valuable/valuable-serde/tests/test.rs
Lines 103 to 105 in 7a22d9d
Given that a big usecase is for generating UIs, the natural usecase for a doc comment is to act as a tooltip. It's surprisingly hard to find any derive macro that exposes doc comments, so I'm wondering if it's possible to do or add here. Maybe there's some limitation that makes it impossible I'm not aware of?
NamedValues::get
has a subtle footgun: The given reference to a NamedField
must be pointer-equal to a NamedField
in NamedValues
. But, because NamedField
is Copy
/Clone
, it's trivial to construct programs that subtly violate this requirement; e.g., this program panics:
fn main() {
use valuable::{NamedField, NamedValues, Value};
let fields = [
NamedField::new("foo"),
NamedField::new("bar")
];
let values = [
Value::U32(123),
Value::U32(456),
];
let named_values = NamedValues::new(&fields, &values);
let field = fields[0];
assert_eq!(
named_values.get(&field).and_then(Value::as_u32),
Some(123)
);
}
...but this program is fine:
fn main() {
use valuable::{NamedField, NamedValues, Value};
let fields = [
NamedField::new("foo"),
NamedField::new("bar")
];
let values = [
Value::U32(123),
Value::U32(456),
];
let named_values = NamedValues::new(&fields, &values);
let field = &fields[0];
assert_eq!(
named_values.get(field).and_then(Value::as_u32),
Some(123)
);
}
I was expecting valuable to work out of the box with tracing's macros (info
, debug
etc). However, I get an issue saying Value
is not implemented for my type (i.e. there is no blanket implementation covering all types that implement Valuable). This currently seems like the only way to make tracing happy:
for (service, old_state) in removed {
info!(service = &service as &dyn Valuable,"Service unit removed");
self.service_state.remove(&service);
}
Please let me know if this was an oversight, or if there's a fundamental limitation preventing the compiler from auto-coercing my type (RemoteServiceId
) that implements Valuable to Value?
As requested on the discord, posting a detailed issue about trying valuable
for Tera.
I am trying to see if valuable could replace serde_json::Value
for Tera.
The goal in a template engine is to find the value the user wants to use in a given context.
Right now, the context in Tera is just serde_json::Value
, which means we serialise everything even though we don’t really need it. See https://docs.rs/tera/1.10.0/tera/struct.Context.html for examples of Context.
In Tera, you need to evaluate expressions based on the context. For example:
{{ get_metadata(path=page.path ~ “image.jpg”) }}
: this concatenates into a single string which is then passed as an argument to a function{% if user.is_admin or data.public %}
: this evaluates the fields truthiness{{ item.price * current_rate | format_currency }}
: this multiplies the value found in the context and then applies a filter on the output{% for result in data.results[user.birth_year] %}
There are more examples on the Tera documentation: https://tera.netlify.app/docs/
If you are familiar with Django/Jinja2/Twig/Liquid this is essentially the same thing.
The way it currently work is the following:
user.birth_year
in the example. If the value is an integer, we assume it’s the index of the array so we don’t change anythingValue
kind: if we’re doing a math operation like *
, we need to make sure both operands resolve to a Value::Number
for example, string concatenation with ~
only work on Value::String
and so onAs mentioned, it currently serialises the whole context which can be very costly. For example, rendering a section in a Zola site will clone the content of all the sub-pages if pagination isn’t set up, which can quickly become slow.
The whole Tera cloning the context is actually one of the biggest current perf bottleneck in Zola as you need to clone the content every time you render the page, which can be pretty big considering it’s potentially thousands of HTML pages.
What I’d like to accomplish is be able to visit the context and extract the Value
for each identifier. Let’s use https://github.com/tokio-rs/valuable/blob/master/valuable/examples/print.rs as an example data and say that the person
variable is our context.
{{ name }}
The path is [“name”]
so we only need to visit Structable and Mappable and look for that key
{{ addresses.1.city }}
The path is [“addresses”, 1, “city”]
. We visit Structable/Mappable because it begins with a string and visit the field where it is equal to addresses
. This is an array so we visit Listable. We know we want to only care about the element at index 1 so we could only visit that one ideally (see #52). Once there, there is only city
left to match so we visit in Structable/Mappable again.
{% for address in addresses %}
Here we want to iterate on the Vec<Address>
so I think we just need to visit the addresses and get a Vec<Structable>
which we visit in its entirety one by one (since you can break
from the loop).
In all cases above we want to get the Value
corresponding to the path out of the visit
and pattern match later. Since visit
doesn’t return anything, I need to store it on the visitor struct. The Value
found could be a primitive, a Mappable/Structable (for loops on key values) or not found at all.
However I get a lifetime error if I try to assign a Value
in a small toy example: https://gist.github.com/Keats/150acda601910c5f8a6bd9aceba2ddfd so I’m not sure how I would accomplish that. Is there an example of a visitor holding a Value somewhere?
The article at https://tokio.rs/blog/2021-05-valuable specifically mention template engine so I’m wondering if the idea was limited to writing the value to a string buffer or whether it is too limited for Tera.
This argument would inform the visitor of the current container being visited (enum, struct, tuple, ...).
For the short term, I wanted to make myself a wrapper for anyhow::Error
, to use the unstable support in tracing
and log the values as structured.
What happened is that my manual implementation - although seemingly correct, fails to serialize. A different implementation, which stringifies early and uses#[derive(Valuable)]
works just fine.
Because the code is a bit bigger than fits into an issue, I have created a reproduction in a separate repository, with both versions.
cc @hawkw
Until Valuable reaches 1.0, http will most likely not depend on valuable, so we should provide a bridge. This crate would provide wrappers for types in http
that implement Valuable
.
Then, users can do:
use valuable::Valuable;
use valuable_http::Request;
Request::from(my_request).as_value()
One open question: what argument should the Request
type take?
http::Request
&http::Request
impl AsRef<http::Request>
In some cases, it may be desirable for types implementing Valuable
to record themselves as a pre-formatted human-readable text representation. Currently, this is only possible with the Value::String
variant, which takes a &'a str
.
For types that want to record themselves using a Display
(or Debug
) implementation, though, Value::String
isn't really sufficient. Because it takes a borrowed string slice, those types would have to allocate ahead of time and store their own formatted representation, so that the borrowed string slice can be handed out in their Valuable
impls. This is not ideal.
Therefore, we should probably add &dyn fmt::Display
as a primitive Value
variant.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.