Can't handle invalid encoding about lsd

[Peltoche]

I think this can be fixed by using .to_string_lossy() everywhere.

[jfdhuiz]

I went to investigate a broken symlink pointing to content in /home/user/.steam

$ pwd                          
/home/user/.steam

$ env RUST_BACKTRACE=1 /home/user/.cargo/bin/lsd
thread 'main' panicked at 'failed to encode file name', src/libcore/option.rs:1034:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::continue_panic_fmt
   6: rust_begin_unwind
   7: core::panicking::panic_fmt
   8: core::option::expect_failed
   9: lsd::meta::name::Name::new
  10: lsd::meta::Meta::from_path
  11: lsd::meta::Meta::recurse_into
  12: lsd::core::Core::run
  13: lsd::main
  14: std::rt::lang_start::{{closure}}
  15: std::panicking::try::do_call
  16: __rust_maybe_catch_panic
  17: std::rt::lang_start_internal
  18: main
  19: __libc_start_main

$ env RUST_BACKTRACE=1 /snap/bin/lsd
thread 'main' panicked at 'failed to encode file name', src/libcore/option.rs:1034:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:59
             at src/libstd/panicking.rs:197
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:211
   4: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:474
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:381
   6: rust_begin_unwind
             at src/libstd/panicking.rs:308
   7: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
   8: core::option::expect_failed
             at src/libcore/option.rs:1034
   9: lsd::meta::name::Name::new
  10: lsd::meta::Meta::from_path
  11: lsd::meta::Meta::recurse_into
  12: lsd::core::Core::run
  13: lsd::main
  14: std::rt::lang_start::{{closure}}
  15: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:293
  16: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:87
  17: std::rt::lang_start_internal
             at src/libstd/panicking.rs:272
             at src/libstd/panic.rs:388
             at src/libstd/rt.rs:48
  18: main
  19: __libc_start_main
  20: _start

$ /home/user/.cargo/bin/lsd --version       
lsd 0.15.1

$ /snap/bin/lsd --version
lsd 0.15.1

$ /bin/ls
'97a1.icoW'$'\236\006\344\324\333''J'$'\360\320''M'$'\025\342''P'$'\366''F'$'\360\240\247\271\355'   controller_base           package         steam.pipe
'H'$'\222\305\344\345\332''u'$'\360\020''Y'$'\333\322\377\377\377\377''T'$'\b'                 d3a0                public      steam.sh
'H'$'\242\037\343\345''*'$'\235\360\340\004\261\315\260\246''O'$'\323''T'$'\022\342\375'       error.log               registry.vdf    steam_msg.sh
 ThirdPartyLegalNotices.css                                        fontconfig              resource        steam_subscriber_agreement.txt
 ThirdPartyLegalNotices.doc                                        fossilize_engine_filters.json   root        steamdeps.txt
 ThirdPartyLegalNotices.html                                       friends                 sdk32       steamui
 bin                                                   graphics                sdk64       tenfoot
 bin32                                                     html5app                servers         ubuntu12_32
 bin64                                                     installscriptevalutor_log.txt   singleapplauncher   ubuntu12_64
 bin_steam.sh                                              legacycompat            skins      ''$'\340\302\247\342'
 bin_steamdeps.py                                          linux32                 steam
 clientui                                              linux64                 steam.pid

Addendum

I moved the 4 offending files

'97a1.icoW'$'\236\006\344\324\333''J'$'\360\320''M'$'\025\342''P'$'\366''F'$'\360\240\247\271\355'
'H'$'\222\305\344\345\332''u'$'\360\020''Y'$'\333\322\377\377\377\377''T'$'\b'
'H'$'\242\037\343\345''*'$'\235\360\340\004\261\315\260\246''O'$'\323''T'$'\022\342\375'
''$'\340\302\247\342'

to a dir called garbage. Then, both installed versions work on both the .steam directory as well as .steam/garbage. I figured the offending file is '97a1.icoW'$'\236\006\344\324\333''J'$'\360\320''M'$'\025\342''P'$'\366''F'$'\360\240\247\271\355'

$ /home/user/.cargo/bin/lsd
  H���*����Ͱ�O�T��    H�����u�Y������    �§�
$ /snap/bin/lsd                                                                                                                                                        
  H���*����Ͱ�O�T��    H�����u�Y������    �§�
$ /home/user/.cargo/bin/lsd 97a1.icoW\X9e\cf\Xe4\Xd4\XdbJ\Xf0\Xd0M\cu\Xe2P\Xf6F𠧹                                                                                  
thread 'main' panicked at 'unexpected invalid UTF-8 code point', src/libcore/option.rs:1036:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
$ /snap/bin/lsd 97a1.icoW\X9e\cf\Xe4\Xd4\XdbJ\Xf0\Xd0M\cu\Xe2P\Xf6F𠧹                        
thread 'main' panicked at 'unexpected invalid UTF-8 code point', src/libcore/option.rs:1034:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

[iirelu]

Getting the same thing, in the same folder even. Who knows why steam generates a garbled mess like that.

I'll note that exa handles the mangled file name fine.

[ipkiss42]

Similar issue, but with a different backtrace. The file name is Gâteau-invisible.odt encoded using the latin1 encoding.

Compare ls and lsd:

>>> ls --version
ls (GNU coreutils) 8.32
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Richard M. Stallman and David MacKenzie.
>>> ls 
'G'$'\342''teau-invisible.odt'
>>> ls G$'\342'teau-invisible.odt
'G'$'\342''teau-invisible.odt'
>>> lsd --version
lsd 0.17.1-pre
>>> lsd
 G�teau-invisible.odt
>>> lsd G$'\342'teau-invisible.odt
thread 'main' panicked at 'unexpected invalid UTF-8 code point', /home/ipkiss/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.33.0/src/args/arg_matches.rs:213:53
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Backtrace:

>>> RUST_BACKTRACE=1 lsd G$'\342'teau-invisible.odt
thread 'main' panicked at 'unexpected invalid UTF-8 code point', /home/ipkiss/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.33.0/src/args/arg_matches.rs:213:53
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/libunwind.rs:86
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:78
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:59
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1063
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1426
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:62
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:204
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:224
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:470
  11: rust_begin_unwind
             at src/libstd/panicking.rs:378
  12: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
  13: core::option::expect_failed
             at src/libcore/option.rs:1203
  14: core::option::Option<T>::expect
             at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libcore/option.rs:347
  15: clap::args::arg_matches::ArgMatches::values_of::to_str_slice
             at /home/ipkiss/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.33.0/src/args/arg_matches.rs:213
  16: core::ops::function::FnMut::call_mut
             at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libcore/ops/function.rs:154
  17: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &mut F>::call_once
             at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libcore/ops/function.rs:285
  18: core::option::Option<T>::map
             at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libcore/option.rs:456
  19: <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::next
             at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libcore/iter/adapters/mod.rs:791
  20: <clap::args::arg_matches::Values as core::iter::traits::iterator::Iterator>::next
             at /home/ipkiss/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.33.0/src/args/arg_matches.rs:797
  21: <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::next
             at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libcore/iter/adapters/mod.rs:791
  22: <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter
             at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/liballoc/vec.rs:2014
  23: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
             at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/liballoc/vec.rs:1926
  24: core::iter::traits::iterator::Iterator::collect
             at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libcore/iter/traits/iterator.rs:1659
  25: lsd::main
             at src/main.rs:91
  26: std::rt::lang_start::{{closure}}
             at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/rt.rs:67
  27: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:52
  28: std::panicking::try::do_call
             at src/libstd/panicking.rs:303
  29: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:86
  30: std::panicking::try
             at src/libstd/panicking.rs:281
  31: std::panic::catch_unwind
             at src/libstd/panic.rs:394
  32: std::rt::lang_start_internal
             at src/libstd/rt.rs:51
  33: std::rt::lang_start
             at /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/rt.rs:67
  34: main
  35: __libc_start_main
  36: _start
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

[meain]

This should be fixed with #425 . Closing this for now, feel free to reopen if this still exists.