Rojo unexpectedly trims trailing newline characters when syncing to Studio. about rojo

[Dekkonot]

Just to clarify, are you running Rojo 7.4.1? If so, would it be possible for you to check if this happens on 7.4.0?

I think this is probably related to #854, but it's surprising behavior for the implementation we're using so I'd like to make sure.

[kennethloeffler]

This is indeed related to a detail of Lines:

rojo/crates/memofs/src/lib.rs

Lines 299 to 304 in b2f133e

	pub fn read_to_string_lf_normalized<P: AsRef<Path>>(&self, path: P) -> io::Result<Arc<String>> {
	let path = path.as_ref();
	let contents = self.inner.lock().unwrap().read_to_string(path)?;
	Ok(contents.lines().collect::<Vec<&str>>().join("\n").into())
	}

When the iterator encounters a lone newline, it returns an empty string: Rust playground.

We can probably resolve this by doing something like

 pub fn read_to_string_lf_normalized<P: AsRef<Path>>(&self, path: P) -> io::Result<Arc<String>> { 
     let path = path.as_ref(); 
     let contents = self.inner.lock().unwrap().read_to_string(path)?; 
  
-    Ok(contents.lines().collect::<Vec<&str>>().join("\n").into())
+    Ok(contents.replace("\r\n", "\n").into())
 } 

[Noobot9k]

Just to clarify, are you running Rojo 7.4.1? If so, would it be possible for you to check if this happens on 7.4.0?

I think this is probably related to #854, but it's surprising behavior for the implementation we're using so I'd like to make sure.

Yes, I'm using 7.4.1. I'm pretty new to Rojo so I'm not sure how to downgrade. Could you link me to a resource to learn how?

[Dekkonot]

Yes, I'm using 7.4.1. I'm pretty new to Rojo so I'm not sure how to downgrade. Could you link me to a resource to learn how?

There's no need to do so now (Ken has identified that this is indeed an issue with our implementation).

For future reference though, it depends upon how you installed Rojo. If you used a toolchain manager like Aftman or Foreman, you'd just change the version in the relevant configuration file. Otherwise you'd have to go to the releases page and download an earlier version from there.

[Dekkonot]

We can probably resolve this by doing something like

Sounds like a good idea to me, though I'd be worried about performance. The underlying implementation for String::replace seems to use String::push_str with no preallocation done, which seems... not ideal. Wonder if it's at all possible to implement it more efficiently.

[kennethloeffler]

We can probably resolve this by doing something like

Sounds like a good idea to me, though I'd be worried about performance. The underlying implementation for String::replace seems to use String::push_str with no preallocation done, which seems... not ideal. Wonder if it's at all possible to implement it more efficiently.

I benchmarked the two approaches, and on my machine, the str::lines approach is actually slower than str::replace, and when str::replace is given LF input (i.e. there are no matches), very significantly so:

Here is the code I used to benchmark:

use criterion::{criterion_group, criterion_main, BatchSize, BenchmarkId, Criterion};

pub fn benchmark_string_normalization(c: &mut Criterion) {
    let mut group = c.benchmark_group("Line Ending Normalization");

    for i in [
        ("lf_input", "Oh My Goodness\n".repeat(2000).as_str()),
        ("crlf_input", "Oh My Goodness\r\n".repeat(2000).as_str()),
    ] {
        group.bench_with_input(BenchmarkId::new("string_replace", i.0), i.1, |b, s| {
            b.iter_batched(
                || s.to_string(),
                |s| {
                    let _ = s.replace("\r\n", "\n");
                },
                BatchSize::SmallInput,
            )
        });

        group.bench_with_input(BenchmarkId::new("string_lines", i.0), i.1, |b, s| {
            b.iter_batched(
                || s.to_string(),
                |s| {
                    s.lines().collect::<Vec<&str>>().join("\n");
                },
                BatchSize::SmallInput,
            )
        });
    }

    group.finish();
}

criterion_group!(benches, benchmark_string_normalization);
criterion_main!(benches);

[Dekkonot]

I guess that actually makes sense, but it's good to see the data on it. In that case I have no qualms about just using String::replace.