GithubHelp home page GithubHelp logo

willcrichton / flowistry Goto Github PK

View Code? Open in Web Editor NEW
1.8K 15.0 40.0 71.62 MB

Flowistry is an IDE plugin for Rust that helps you focus on relevant code.

Home Page: https://marketplace.visualstudio.com/items?itemName=wcrichton.flowistry

License: MIT License

TypeScript 23.90% Rust 73.63% JavaScript 0.48% SCSS 0.18% Python 1.26% Shell 0.05% HTML 0.50%
rust vscode dataflow static-analysis

flowistry's Introduction

Flowistry: Information Flow for Rust

tests crates.io docs

Flowistry is a tool that analyzes the information flow of Rust programs. Flowistry understands whether it's possible for one piece of code to affect another. Flowistry integrates into the IDE to provide a "focus mode" which helps you focus on the code that's related to your current task.

For example, this GIF shows the focus mode when reading a function that unions two sets together:



When the user clicks a given variable or expression, Flowistry fades out all code that does not influence that code, and is not influenced by that code. For example, orig_len is not influenced by the for-loop, while set.len() is.

Flowistry can be helpful when you're reading a function with a lot of code. For example, this GIF shows a real function in the Rust compiler. If you want to understand the role of a specific argument to the function, then Flowistry can filter out most of the code as irrelevant:



Table of contents

Installation

IDE plugin

Flowistry is available as a VSCode plugin. You can install Flowistry from the Visual Studio Marketplace or the Open VSX Registry. In VSCode:

  • Go to the Extensions pane by clicking this button in the left margin: Screen Shot 2021-09-20 at 9 30 43 AM
  • Search for "Flowistry" and then click "Install".
  • Open a Rust workspace and wait for the tool to finish installing.

Note on platform support: Flowistry does not yet support NixOS. Flowistry cannot provide pre-built binaries for ARM targets like M1 Macs, so Flowistry must be installed from scratch on these targets (this is done for you, but will take a few more minutes than usual).

Alternatively, you can install it from source:

# Install flowistry binaries
git clone https://github.com/willcrichton/flowistry
cd flowistry
cargo install --path crates/flowistry_ide

# Install vscode extension
cd ide
npm install
npm run build
ln -s $(pwd) ~/.vscode/extensions/flowistry

Rustc plugin

If you are interested in the underlying analysis, you can use the flowistry crate published to crates.io: https://crates.io/crates/flowistry

The documentation is published here: https://willcrichton.net/flowistry/flowistry/

Note: Docs.rs doesn't support documentation for crates that use #![feature(rustc_private)] so we have to host it ourselves.

Usage

Note that the latest Flowistry has a Maximum Supported Rust Version of Rust 1.73. Flowistry is not guaranteed to work with features implemented after 1.73.

Startup

Once you have installed Flowistry, open a Rust workspace in VSCode. You should see this icon in the bottom toolbar:

Screen Shot 2022-02-22 at 11 46 12 AM

Flowistry starts up by type-checking your codebase. This may take a few minutes if you have many dependencies.

Note: Flowistry type-checking results are cached in the target/flowistry directory. If you delete this folder, Flowistry will have to recompute types. Also for a large codebase this directory may take up a fair amount of disk space.

Entering focus mode

Once Flowistry has booted up, the loading icon will disappear. Then you can enter focus mode by running the "Toggle focus mode" command. By default the keyboard shortcut is Ctrl+R Ctrl+A (⌘+R ⌘+A on Mac), or you can use the Flowistry context menu:

Screen Shot 2022-02-22 at 11 52 39 AM

In focus mode, Flowistry will automatically compute the information flow within a given function once you put your cursor there. Once Flowistry has finished analysis, the status bar will look like this:

Screen Shot 2022-02-22 at 11 55 36 AM

Note: Flowistry can be a bit slow for larger functions. It may take up to 15 seconds to finish the analysis.

Flowistry infers what you want to focus on based on your cursor. So if you click on a variable, you should see the focus region of that variable. Flowistry will highlight the focused code in gray, and then fade out code outside the focus region. For example, because the user's cursor is on view_projection, that variable is highlighted in gray, and its focus region is shown.

Screen Shot 2022-02-22 at 12 00 22 PM

Setting a mark

Sometimes you want to keep the focus region where it is, and click on other code to inspect it without changing focus. For this purpose, Flowistry has a concept of a "mark". Once you have selected code to focus on, you can run the "Set mark" command (Ctrl+R Ctrl+S / ⌘+R ⌘+S). Then a mark is set at your cursor's current position, and the focus will stay there until you run the "Unset mark" command (Ctrl+R Ctrl+D / ⌘+R ⌘+D).

Selecting the focus region

If you want to modify all the code in the focus region, e.g. to comment it out or copy it, then you can run the "Select focused region" command (Ctrl+R Ctrl+T / ⌘+R ⌘+T). This will add the entire focus region into your editor's selection.

Limitations

Flowistry is an active research project into the applications of information flow analysis for Rust. It is continually evolving as we experiment with analysis techniques and interaction paradigms. So it's not quite as polished or efficient as tools like Rust Analyzer, but we hope you can still find it useful! Nevertheless, there are a number of important limitations you should understand when using Flowistry to avoid being surprised.

If you have questions or issues, please file a Github issue, join our Discord, or DM @wcrichton on Twitter.

Flowistry does not completely handle interior mutability

When your code has references, Flowistry needs to understand what that reference points-to. Flowistry uses Rust's lifetime information to determine points-to information. However, data structures that use interior mutability such as Arc<Mutex<T>> explicitly do not share lifetimes between pointers to the same data. For example, in this snippet:

let x = Arc::new(Mutex::new(0));
let y = x.clone();
*x.lock().unwrap() = 1;
println!("{}", y.lock().unwrap());

Flowistry can determine that *x.lock().unwrap() = 1 is a mutation to x, but it can not determine that it is a mutation to y. So if you focus on y, the assignment to 1 would be faded out, even though it is relevant to the value of y.

We are researching methods to overcome this limitation, but for now just be aware that this is the main case where Flowistry is known to provide an incorrect answer.

A focus region may include more code than you expect

Flowistry's analysis tries to include all code that could have an influence on a focal point. This analysis makes a number of assumptions for both practical and fundamental reasons. For example, in this snippet:

let mut v = vec![1, 2, 3];
let x = v.get_mut(0);
println!("{:?} {}", v, x);

If you focus on v on line 3, it will include v.get_mut(0) as an operation that could have modified v. The reason is that Flowistry does not actually analyze the bodies of called functions, but rather approximates based on their type signatures. Because get_mut takes &mut self as input, it assumes that the vector could be modified.

In general, you should use focus mode as a pruning tool. If code is faded out, then you don't have to read it (minus the limitation mentioned above!). If it isn't faded out, then it might be relevant to your task.

Not all code is selectable

Flowistry works by analyzing the MIR graph for a given function using the Rust compiler's API. Then the IDE extension lifts the analysis results from the MIR level back to the source level. However, a lot of information about the program is lost in the journey from source code to MIR.

For example, if the source contains an expression foo.whomp.bar().baz(), it's possible that a temporary variable is only generated for the expression foo.whomp.bar(). So if the user selects foo, Flowistry may not be able to determine that this corresponds to the MIR place that represents foo.

This is why the IDE extension highlights the focused code in gray, so you can understand what your cursor's selection actually maps to.

Nested functions cannot be analyzed together (including closures and async)

Flowistry analyzes a single function at a time. If a function contains other functions, e.g. fn definitions, or closures, or implicitly via async, then Flowistry will only show you focus regions within the smallest function body containing your cursor. This is usually well defined for function definitions and closures, but may be confusing for async since that depends on how rustc decides to carve up your async function.

FAQ

rustup fails on installation

If rustup fails, especially with an error like "could not rename downloaded file", this is probably because Flowistry is running rustup concurrently with another tool (like rust-analyzer). Until rustup#988 is resolved, there is unfortunately no automated way around this.

To solve the issue, go to the command line and run:

rustup toolchain install nightly-2023-08-25 -c rust-src -c rustc-dev -c llvm-tools-preview

Then go back to VSCode and click "Continue" to let Flowistry continue installing.

Why isn't Flowistry part of Rust Analyzer?

Rust Analyzer does not support MIR and the borrow checker, which are essential parts of Flowistry's analysis. That fact is unlikely to change for a long time, so Flowistry is a standalone tool.

Why does Flowistry highlight (or not) this code?

See Limitations for known issues. If that doesn't explain what you're seeing, please post it in the unexpected highlights issue or ask on Discord.

flowistry's People

Contributors

connorff avatar epompeii avatar jplatte avatar justusadam avatar kikkon avatar pigeonf avatar vakaras avatar willcrichton avatar zeramorphic avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

flowistry's Issues

Cannot find rustup when launching VS Code from the dock/finder

When I launch VS Code from the dock (on macOS), and then try to install the extension, I get an error saying that the extension could not find Rustup. I assume it was not on VS Code's PATH at this point. Alas I didn't think to copy the error message at the time, so I can't remember what it was exactly, but it was along the lines of: "could not find rustup (/bin/sh)"

The fix was to use the code command to launch VS Code from the terminal, then the installation of flowistry proceeded as expected when prompted. Not sure what the normal solution to this - I assume there are some janky ways around it people employ, but I thought I'd let you know rather than suffering in silence!

Add more tests and benchmarks

  • test calling driver in complex Cargo workspaces -- nested modules, bin, examples, workspace members, ...
  • test VSCode extension startup, installation, basic UI (added in #24)
  • test utility functions rather than just entire slicing infra
  • benchmarks for flow perf

The Flowistry Todo List

Features

  • Windows support
    • Fix span bugs? is this related to unicode? document.offsetAt seems character based and not byte-based
    • Add windows target to release builds
    • Should fix #27
  • Editor support
    • Vim
    • IntelliJ
    • Emacs
  • IDE extension options
    • Disable prefetch at startup
  • Source mapping improvements
    • fn <name> should not be greyed out
    • param: T should not grey out : T when param is relevant
    • return should not be greyed out when return expression is relevant
  • Tests
    • Add integration tests that run Flowistry on real repos (i.e. git clone > cargo flowistry ...)
  • Add feature to fold all code not in a focus region

Bugs

  • #31
  • #32
  • #34
  • Cancellation isn't actually stopping flowistry-driver

Performance

For reference, here's a subset of slow calls when running the command in rust-lang/rust:

focus compiler/rustc_typeck/src/check/op.rs 21361

Screen Shot 2022-02-16 at 11 13 25 AM

  • Create a benchmark to track Flowistry performance. Should include granular information on which pieces are bottlenecks.
    • Program with many lifetimes (in inputs / in body) to stress alias analysis
    • Program with many instructions to stress infoflow analysis
    • Program with giant data structures to test any code relying on Place::interior_*
  • Possible opportunities for optimization:
    • Loading TyCtxt can take a while, e.g. 2.5s for rustc_typeck. What's causing this? Can it be made more incremental?
    • Polonius is slow, are there any low hanging fruit we can upstream into datafrog?

sccache causes flowistry-driver to fail

When trying to use flowistry for the first time, I get this error message:

Flowistry could not run because your project failed to build with error:
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `sccache /Users/domenukk/.cargo/bin/flowistry-driver rustc - --crate-name ___ --print=file-names --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit status: 2)
  --- stderr
  sccache: error: failed to execute compile
  sccache: caused by: Compiler not supported: "error: expected one of `!` or `[`, found keyword `if`\n --> /var/folders/9b/2lxhyc8567g980pxjn_t3qy40000gn/T/sccachewzZPue/testfile.c:1:2\n  |\n1 | #if defined(__NVCC__)\n  |  ^^ expected one of `!` or `[`\n\nerror: aborting due to previous error\n\n"

Could this be realted to sccache somehow? I'll try without next, but in any case it would be good to find another solution.

Flowistry 0.5.7 fails to build in VSCode on macOS (M1).

Problem

Flowistry fails to build in VSCode.

Logs

OS: darwin (21.3.0) (M1 aarch64)
VSCode: 1.64.2
Error message

Updating crates.io index
  Installing flowistry_ide v0.5.6
    Blocking waiting for file lock on package cache
 Downloading crates ...
  Downloaded flowistry v0.5.7
    Blocking waiting for file lock on package cache
   Compiling cfg-if v1.0.0
   Compiling autocfg v1.1.0
   Compiling proc-macro2 v1.0.36
   Compiling unicode-xid v0.2.2
   Compiling libc v0.2.119
   Compiling syn v1.0.86
   Compiling serde_derive v1.0.136
   Compiling serde v1.0.136
   Compiling lazy_static v1.4.0
   Compiling crossbeam-utils v0.8.7
   Compiling crossbeam-epoch v0.9.7
   Compiling log v0.4.14
   Compiling scopeguard v1.1.0
   Compiling rayon-core v1.9.1
   Compiling anyhow v1.0.55
   Compiling semver v1.0.6
   Compiling camino v1.0.7
   Compiling serde_json v1.0.79
   Compiling ppv-lite86 v0.2.16
   Compiling hashbrown v0.11.2
   Compiling unicode-width v0.1.9
   Compiling either v1.6.1
   Compiling utf8-width v0.1.5
   Compiling ryu v1.0.9
   Compiling itoa v1.0.1
   Compiling unicode-segmentation v1.9.0
   Compiling fixedbitset v0.4.1
   Compiling bitflags v1.3.2
   Compiling textwrap v0.14.2
   Compiling fluid-let v1.0.0
   Compiling textwrap v0.11.0
   Compiling itertools v0.10.3
   Compiling html-escape v0.2.9
   Compiling memoffset v0.6.5
   Compiling indexmap v1.8.0
   Compiling rayon v1.5.1
   Compiling clap v2.34.0
   Compiling crossbeam-channel v0.5.2
   Compiling quote v1.0.15
   Compiling getrandom v0.2.5
   Compiling num_cpus v1.13.1
   Compiling env_logger v0.9.0
   Compiling rand_core v0.6.3
   Compiling flowistry v0.5.7
   Compiling rand_chacha v0.3.1
   Compiling rand v0.8.5
   Compiling crossbeam-deque v0.8.1
   Compiling petgraph v0.6.0
   Compiling cargo-platform v0.1.2
   Compiling cargo_metadata v0.14.2
   Compiling flowistry_ide v0.5.6
error[E0560]: struct `flowistry::range::Range` has no field named `start`
  --> /Users/nsabovic/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry_ide-0.5.6/src/spans.rs:28:9
   |
28 |         start: 0,
   |         ^^^^^ `flowistry::range::Range` does not have this field
   |
   = note: available fields are: `char_start`, `char_end`, `byte_start`, `byte_end`, `filename`

error[E0560]: struct `flowistry::range::Range` has no field named `end`
  --> /Users/nsabovic/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry_ide-0.5.6/src/spans.rs:29:9
   |
29 |         end: 0,
   |         ^^^ `flowistry::range::Range` does not have this field
   |
   = note: available fields are: `char_start`, `char_end`, `byte_start`, `byte_end`, `filename`

For more information about this error, try `rustc --explain E0560`.
error: could not compile `flowistry_ide` due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
error: failed to compile `flowistry_ide v0.5.6`, intermediate artifacts can be found at `/var/folders/_w/137c04_d12n4qf4009jq_5rw0000gn/T/cargo-install8mFSRm`

Caused by:
  build failed

Full log: https://paste.rs/Xwi

Flowistry does not understand inline assembly

For the following function:

pub unsafe fn load_u8(p: *const u8) -> u8 {
    let mut ret: u8;
    std::arch::asm!(
        "mov {ret}, byte ptr [{p}]",
        p = in(reg) p,
        ret = lateout(reg_byte) ret, 
        options(nostack)
    );
    ret
}

Flowistry does not highlight the use of "p" in the asm! invocation:
imagen
It also does not highlight that ret is written to by it:
imagen

Since in(reg) p and lateout(reg_byte) ret explicitly specify those 2 things, it should be possible to recognize these. The rust reference has a page explaining the asm! syntax: https://doc.rust-lang.org/nightly/reference/inline-assembly.html

Tested only with the vscode marketplace extension on windows 10 21H2

Can't run cargo-flowistry on NixOS

I installed flowistry via the vscode extension. After patching the rpath of the cargo-flowistry and flowistry-driver binaries, I still get this error when running a flowistry command in vscode:

Flowistry could not run because your project failed to build with error:
error: could not execute process `/home/rdp/.cargo/bin/cargo-flowistry flowistry effects /home/rdp/src/downloader/src/lib.rs 2453` (never executed)

Caused by:
  No such file or directory (os error 2)

Invoking cargo flowistry gives:

$ cargo flowistry
error: could not execute process `/home/rdp/.cargo/bin/cargo-flowistry flowistry` (never executed)

Caused by:
  No such file or directory (os error 2)

Invoking the cargo-flowistry binary directly gives:

$ cargo-flowistry
bash: /home/rdp/.cargo/bin/cargo-flowistry: No such file or directory

Both cargo-flowistry and flowistry-driver exist:

$ which cargo-flowistry
/home/rdp/.cargo/bin/cargo-flowistry
$ which flowistry-driver
/home/rdp/.cargo/bin/flowistry-driver

The usual suspect for these errors on NixOS is a missing dynamic library. But I have patched the binaries so that the libraries are found:

$ ldd /home/rdp/.cargo/bin/cargo-flowistry
        linux-vdso.so.1 (0x00007ffc02f14000)
	libLLVM-13-rust-1.57.0-nightly.so => /home/rdp/.rustup/toolchains/nightly-2021-10-08-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-13-rust-1.57.0-nightly.so (0x00007f0cee73e000)
	libgcc_s.so.1 => /nix/store/mij848h2x5wiqkwhg027byvmf9x3gx7y-glibc-2.33-50/lib/libgcc_s.so.1 (0x00007f0cee724000)
	libpthread.so.0 => /nix/store/mij848h2x5wiqkwhg027byvmf9x3gx7y-glibc-2.33-50/lib/libpthread.so.0 (0x00007f0cee704000)
	libm.so.6 => /nix/store/mij848h2x5wiqkwhg027byvmf9x3gx7y-glibc-2.33-50/lib/libm.so.6 (0x00007f0cee5c3000)
	libdl.so.2 => /nix/store/mij848h2x5wiqkwhg027byvmf9x3gx7y-glibc-2.33-50/lib/libdl.so.2 (0x00007f0cee5be000)
$ ldd /home/rdp/.cargo/bin/flowistry-driver
        linux-vdso.so.1 (0x00007ffd395d9000)
	librustc_driver-2cdae8a62fc3a991.so => /home/rdp/.rustup/toolchains/nightly-2021-10-08-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_driver-2cdae8a62fc3a991.so (0x00007f14c1dfa000)
	libstd-8adcca4f1427867b.so => /home/rdp/.rustup/toolchains/nightly-2021-10-08-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-8adcca4f1427867b.so (0x00007f14c1a4e000)
	libgcc_s.so.1 => /nix/store/mij848h2x5wiqkwhg027byvmf9x3gx7y-glibc-2.33-50/lib/libgcc_s.so.1 (0x00007f14c1a34000)
	libpthread.so.0 => /nix/store/mij848h2x5wiqkwhg027byvmf9x3gx7y-glibc-2.33-50/lib/libpthread.so.0 (0x00007f14c1a14000)
	libc.so.6 => /nix/store/mij848h2x5wiqkwhg027byvmf9x3gx7y-glibc-2.33-50/lib/libc.so.6 (0x00007f14c184f000)

cargo config not used/respected

I use the following .cargo\config.toml

[build]
target-dir = "\dir\outside\project"

This config is not being picked up by flowistry. Can i configure flowistry in this way?

Kind regards

Flowistry fails to install

cargo install flowistry_ide on my computer (M1 macOS 12.2.1 aarch64) fails with 700+ errors, I've attached the first few. This probably isn't meant to be installed this way, but:

  1. cargo update will try to install it and fail, breaking standard upgrade process. For instance, topgrade will error out.
  2. Installing from VSCode also fails on my computer, with a different error.
   Compiling flowistry v0.5.7
error[E0463]: can't find crate for `either`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:53:1
   |
53 | extern crate either;
   | ^^^^^^^^^^^^^^^^^^^^ can't find crate

error[E0463]: can't find crate for `rustc_borrowck`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:54:1
   |
54 | extern crate rustc_borrowck;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_data_structures`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:55:1
   |
55 | extern crate rustc_data_structures;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_driver`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:56:1
   |
56 | extern crate rustc_driver;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_graphviz`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:57:1
   |
57 | extern crate rustc_graphviz;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_hir`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:58:1
   |
58 | extern crate rustc_hir;
   | ^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_hir_pretty`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:59:1
   |
59 | extern crate rustc_hir_pretty;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_index`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:60:1
   |
60 | extern crate rustc_index;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_infer`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:61:1
   |
61 | extern crate rustc_infer;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_interface`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:62:1
   |
62 | extern crate rustc_interface;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_macros`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:63:1
   |
63 | extern crate rustc_macros;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_middle`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:64:1
   |
64 | extern crate rustc_middle;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_mir_dataflow`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:65:1
   |
65 | extern crate rustc_mir_dataflow;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_mir_transform`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:66:1
   |
66 | extern crate rustc_mir_transform;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

error[E0463]: can't find crate for `rustc_serialize`
  --> /Users/johndoe/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.7/src/lib.rs:67:1
   |
67 | extern crate rustc_serialize;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

Panic on slicing const string

ex:

fn foo() { 
  const FOO: &str = "bar";
}

select FOO, get Unknown arg span for owner_node

Issue is that const FOO: .. is treated as a standalone item, not a variable declaration within foo, (e.g. similar if you nested fn bar() {} inside of foo). So the broader issue is attempting to slice items that aren't functions.

But also: is it possible to be able to slice FOO? Even if this doesn't panic, it's confusing for a user.

Flowistry outputs nonsense on Windows

Problem

after I download and install successfully. VS gave me the error.

Logs

OS: win32 (10.0.19043)
VSCode: 1.61.2
Error message

'D:\Program' �����ڲ����ⲿ���Ҳ���ǿ����еij���
���������ļ���

Full log: https://paste.rs/P3z

Flowistry for JavaScript / TypeScript?

Hi there, first of all, thanks for creating Flowistry, really cool idea!

Was wondering if it would be possible / simple to apply this to other languages like JavaScript and TypeScript (or if another extension already exists for JS/TS).

Would be amazing to be able to use this, also in teaching!

build 0.5.5 fails with missing imports

Seems to be a lot of missing imports

....
error[E0433]: failed to resolve: use of undeclared type `HashSet`
   --> /Users/_/.cargo/registry/src/github.com-1ecc6299db9ec823/flowistry-0.5.5/src/test_utils.rs:345:27
    |
345 |                   None => HashSet::default(),
    |                           ^^^^^^^ not found in this scope
    |
help: consider importing this struct
    |
3   | use std::collections::HashSet;
    |

Some errors have detailed explanations: E0405, E0412, E0416, E0422, E0425, E0432, E0433, E0463, E0531...
For more information about an error, try `rustc --explain E0405`.
error: could not compile `flowistry` due to 771 previous errors
warning: build failed, waiting for other jobs to finish...

Flowistry panics if you open any .rs file in vscode not in SourceFiles

If you have a .rs file that's not reachable/compiled for the target flowistry uses (I assume the host?), flowistry opens a window with this error message in vscode:

Flowistry could not run because your project failed to build with error:
thread 'rustc' panicked at 'called `Result::unwrap()` on an `Err` value: Could not find SourceFile for path: d:\dev\rs_playground\src\test.rs. Available SourceFiles were: [src\main.rs, src\other.rs, C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.116\src\lib.rs, C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.116\src\macros.rs, C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.116\src\fixed_width_ints.rs, C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.116\src\windows\mod.rs, C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.116\src\windows\msvc\mod.rs]', D:\.cargo\registry\src\github.com-1ecc6299db9ec823\flowistry_ide-0.5.18\src\spans.rs:35:8
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: could not compile `rs_playground`
Backtrace:
Stack backtrace:
   0: std::backtrace::Backtrace::disabled
   1: std::backtrace::Backtrace::capture
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <rustc_lint::BuiltinCombinedPreExpansionLintPass as rustc_lint::passes::EarlyLintPass>::check_foreign_item_post
   7: <rustc_driver::args::Error as core::fmt::Debug>::fmt
   8: <rustc_lint::BuiltinCombinedPreExpansionLintPass as rustc_lint::passes::EarlyLintPass>::check_foreign_item_post
   9: <rustc_lint::BuiltinCombinedPreExpansionLintPass as rustc_lint::passes::EarlyLintPass>::check_foreign_item_post
  10: rustc_driver::pretty::print_after_hir_lowering
  11: std::sys::windows::thread::Thread::new
  12: BaseThreadInitThunk
  13: RtlUserThreadStart', D:\.cargo\registry\src\github.com-1ecc6299db9ec823\flowistry_ide-0.5.18\src\spans.rs:35:8

It should be reproducible with any project, just create an new empty .rs file in vscode, open it, and turn flowistry focus mode on and off. Note that if the file actually contains rust code, flowistry will also panic if you open that file while in focus mode and place the cursor somewhere such that it tries to analyze it.

This can be annoying in a codebase using conditional compilation for different modules/files, for example:

#[cfg(target_arch = "x86_64")]
mod x86_64;
#[cfg(target_arch = "x86_64")]
use x86_64 as arch;

#[cfg(target_arch = "aarch64")]
mod aarch64;
#[cfg(target_arch = "aarch64")]
use aarch64 as arch;

fn main() {}

Where both x86_64.rs and aarch64.rs contain some valid rust code:

fn f() -> () {
    ()
}

Opening aarch64.rs and placing the cursor on the return type panics with the same general error, on my x86_64 machine.

Tested only with the vscode marketplace extension on windows 10 21H2

Materializing entire PlaceDomain is bad with huge types

Try running effects in AnalysisItemVisitor::visit_nested_body in analysis.rs. PlaceDomain has 200,000+ elements presumably from TyCtxt.

We need to go back to computing exactly the set of places needed for the analysis, then only build PlaceDomain over those places.

Crash on any file that is not main.rs

Opens window with crash info when toggled in a file that is not main.rs, but will work in main.rs.

Example output:

Flowistry could not run because your project failed to build with error:
thread 'main' panicked at 'Could not find target for path: /home/philip/src/BoilR/src/ui/uiapp.rs', crates/flowistry_ide/src/bin/cargo-flowistry.rs:180:10
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

This is the project I have tested on:
https://github.com/PhilipK/BoilR

Flowistry doesn't work on proc-macro crates

As in title, I'm getting an error:

Flowistry could not run because your project failed to build with error:
error: Found argument '--proc-macro' which wasn't expected, or isn't valid in this context

USAGE:
    cargo rustc --package <SPEC> --profile <PROFILE-NAME> --quiet

For more information try --help

To reproduce:

Extend flowistry to types with interior mutability

You mentioned two things during your thesis defense:

  1. There are two analyses that flowistry supports: whole-program analysis, and (very accurate) heuristics using the lifetime annotations people already provide for the borrow checker.
  2. The lifetime annotations only work for exterior mutability; things like AtomicUsize::set are not considered by flowistry to affect the data flow of the program.

I think it would be possible to extend this to interior mutability by using the intra-procedural analysis that looks into dependencies, but only for types which have interior mutability. The compiler already knows statically which types have interior mutability, because they have to contain an UnsafeCell (anything else is already undefined behavior).

Trouble using

Problem

This is probably part of #15 but didn't see how to get the logs out without filing a new issue. This happens when I choose "backwards highlight" as suggested by the notification. (That said, I unset the config for this directory so it should fallback to this default:

stable-x86_64-unknown-linux-gnu (default)
rustc 1.55.0 (c8dfcfe04 2021-09-06)

)

FWIW, I also use Fish shell so might all be related to #9

Logs

OS: linux (5.10.0-0.bpo.8-amd64)
VSCode: 1.60.1
Error message

error: the 'cargo' binary, normally provided by the 'cargo' component, is not applicable to the 'nightly-2021-09-23-x86_64-unknown-linux-gnu' toolchain

Full log: https://paste.rs/wXq

Error with syncing channel updates

Problem

Right after installation, I got the following error:

Logs

OS: macOS (10.15.7)
VSCode: 1.60.2
Rust: 1.54.0 (a178d0322 2021-07-26)
Error message

info: syncing channel updates for 'nightly-2021-09-23-x86_64-apple-darwin'
info: latest update on 2021-09-23, rust version 1.57.0-nightly (308dffd25 2021-09-22)
error: component 'rust-src,rustc-dev,llvm-tools-preview' for target 'x86_64-apple-darwin' is unavailable for download for channel nightly-2021-09-23
Sometimes not all components are available in any given nightly.

Full log: https://paste.rs/tPe

install script failed with fish shell

let shell: boolean | string = process.env.SHELL || true;

I just got an error that seems to stem from the install script failing in fish shell. I think || true should just ignore the value of process.env.SHELL

I'm not familiar with child_process here, but the script has a bash hashbang etc, so I'm guessing it's something about how the script is being invoked that's an issue

Allow setting rustflags required to build codebase

If a codebase requires a rustflag to be able to build (e.g., tokio_unstable), then flowistry won't be able to do it. It would be nice if flowistry picked up .cargo/config or accepted a set of rustflags.

Installing Flowistry failed when the specific Rust toolchain isn't installed

Problem

Installing Flowistry failed. Looks like Flowistry wants a specific version of the toolchain, but that fails to install via Flowistry.

Installing it separately works as expected.

Logs

OS: darwin (20.6.0)
VSCode: 1.60.2
Error message

error: component download failed for cargo-x86_64-apple-darwin: could not rename downloaded file from '/Users/dacut/.rustup/downloads/a8674a2e1c94fe147f0fb05303de67130dc6dfc9b33f2eb9a07194a5b85ee2ec.partial' to '/Users/dacut/.rustup/downloads/a8674a2e1c94fe147f0fb05303de67130dc6dfc9b33f2eb9a07194a5b85ee2ec'
**Full log:** https://paste.rs/8md
Activating...
Workspace root	/Users/dacut/projects/WCP
Running command: 	cargo +nightly-2021-09-23 flowistry -V
error: toolchain 'nightly-2021-09-23-x86_64-apple-darwin' is not installed

Running command: 	rustup toolchain install nightly-2021-09-23 -c rust-src,rustc-dev,llvm-tools-preview
info: syncing channel updates for 'nightly-2021-09-23-x86_64-apple-darwin'
info: latest update on 2021-09-23, rust version 1.57.0-nightly (308dffd25 2021-09-22)
info: downloading component 'cargo'
error: component download failed for cargo-x86_64-apple-darwin: could not rename downloaded file from '/Users/dacut/.rustup/downloads/a8674a2e1c94fe147f0fb05303de67130dc6dfc9b33f2eb9a07194a5b85ee2ec.partial' to '/Users/dacut/.rustup/downloads/a8674a2e1c94fe147f0fb05303de67130dc6dfc9b33f2eb9a07194a5b85ee2ec'
flowistry is activated

Crate install failed

Problem

I tried installing the plugin. It installed then said that the crate needed to be installed, I clicked the button to do that. It sat there for a while then showed the error below.

Screenshot from 2021-09-23 10-20-12

Host

OS: Arch Linux x86
Rust: rustc 1.54.0 (a178d0322 2021-07-26) managed with rustup
VS Code:

Version: 1.60.1
Commit: 83bd43bc519d15e50c4272c6cf5c1479df196a4d
Date: 2021-09-10T17:06:57.931Z
Electron: 13.1.8
Chrome: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Linux x64 5.14.6-arch1-1

Error message:

error: could not find `flowistry` in registry `crates-io` with version `=0.3.6`

Full log: https://paste.rs/vUr

Bug: installation error "could not rename downloaded file"

Problem

When I try to install flowistry for the first time via the Code marketplace, on current Code and current macOS, then installation fails.

Logs

OS: darwin (20.5.0)
VSCode: 1.60.2
Error message

info: syncing channel updates for 'nightly-2021-09-23-x86_64-apple-darwin'
info: latest update on 2021-09-23, rust version 1.57.0-nightly (308dffd25 2021-09-22)
info: downloading component 'cargo'
error: component download failed for cargo-x86_64-apple-darwin: could not rename downloaded file from '/Users/joel/.rustup/downloads/a8674a2e1c94fe147f0fb05303de67130dc6dfc9b33f2eb9a07194a5b85ee2ec.partial' to '/Users/joel/.rustup/downloads/a8674a2e1c94fe147f0fb05303de67130dc6dfc9b33f2eb9a07194a5b85ee2ec'

Full log: https://paste.rs/Uf1

Unexpected highlights from Flowistry

If you use Flowistry and it highlights something unexpectedly -- either it misses something you expect to be highlighted, or it highlights something that seems irrelevant -- please post a screenshot here!

Build failed

Problem

The build failed while compiling the extension

Logs

OS: darwin (20.6.0)
VSCode: 1.61.0-insider
Error message

  build failed

Full log: https://paste.rs/XHb

Static methods aren't selectable

Example:

fn foo() {
  let x = String::new();
  let y = x;
  
  let x = 1;
  let y = x;
}

You can't select String::new(), but you can select 1 to ask for a slice of these values. This is probably an issue with span_to_places.

Install error

Problem

Just tried to install the extension with a workspace open to some rust code. Debian buster, I think I had nightly set as default for the current project? I normally default to stable.

Logs

OS: linux (5.10.0-0.bpo.8-amd64)
VSCode: 1.60.1
Error message

error: failed to install component: 'cargo-x86_64-unknown-linux-gnu', detected conflict: 'etc/bash_completion.d/cargo'

Full log: https://paste.rs/o22

Improve error messages

I work in a project with some specific rust-toolchain, currently: nightly-2020-08-13.
When I tried to use flowistry, but it failed with an unhelpful error message of some build error.

It would be nice to have some more informative error message, specially in the case of a system configuration issue.
Following I added some more details of my issue.

After investigating the command line tool, I found that it was not finding librustc-driver-....
I "fixed" this issue adding the lib path to ldconfig and fter that I got this message from the command line tool:

error[E0514]: found crate `std` compiled by an incompatible version of rustc
  |
  = help: please recompile that crate using this compiler (rustc 1.57.0-nightly (aa8f2d432 2021-09-18))
  = note: the following crate versions were found:
          crate `std` compiled by rustc 1.47.0-nightly (576d27c5a 2020-08-12): /home/user/.rustup/toolchains/nightly-2020-08-13-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-afd4b9aba051f3ad.rlib

error: aborting due to previous error

During all this process the error message in VSCode was always the same of some build error.

Prebuilt Flowistry binaries have incorrect rustc library paths on ARM targets

Hi,

The extension seems to install fine (there's the Flowistry context menu), but choosing any action opens the pane with:

Flowistry could not run because your project failed to build with error:
error: failed to run `rustc` to learn about target-specific information

Caused by:
process didn't exit successfully: `/Users/me/.cargo/bin/flowistry-driver rustc - --crate-name ___ --print=file-names --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (signal: 6, SIGABRT: process abort signal)
--- stderr
dyld: Library not loaded: @rpath/librustc_driver-ac2715475dad8249.dylib
    Referenced from: /Users/me/.cargo/bin/flowistry-driver
    Reason: image not found

OS: macOS 11.6 (Darwin arm64 20.6.0)
VSCode: 1.62.0-insider

I searched through closed isues and couldn't find one like this. Thanks in advance!

Panic on Vec::sort_by_key()

In the following program

fn main() {
    let mut v: Vec<&str> = vec![];
    v.sort_by_key(|x| x.len());
}

clicking on the sort_by_key method causes a panic with flowistry v0.5.25

Flowistry could not run because your project failed to build with error:
thread 'rustc' panicked at 'internal error: entered unreachable code: ReLateBound(DebruijnIndex(0), BoundRegion { var: 0, kind: BrAnon(0) })', crates/flowistry/src/mir/aliases.rs:645:12
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: could not compile `rust-playground`
With backtrace enabled:
Flowistry could not run because your project failed to build with error:
thread 'rustc' panicked at 'internal error: entered unreachable code: ReLateBound(DebruijnIndex(0), BoundRegion { var: 0, kind: BrAnon(0) })', crates/flowistry/src/mir/aliases.rs:645:12
stack backtrace:
   0: rust_begin_unwind
             at /rustc/b2eed72a6fbf254e7d44942eaa121fcbed05d3fb/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/b2eed72a6fbf254e7d44942eaa121fcbed05d3fb/library/core/src/panicking.rs:142:14
   2: ::visit_region
   3: rustc_middle::ty::structural_impls::::super_visit_with
   4: ::visit_ty
   5: rustc_middle::ty::structural_impls::::super_visit_with
   6: ::visit_ty
   7: rustc_middle::ty::structural_impls::::super_visit_with
   8: ::visit_ty
   9: rustc_middle::ty::structural_impls::::super_visit_with
  10: ::visit_ty
  11: flowistry::mir::aliases::Aliases::collect_loans
  12: flowistry::cached::Cache::get
  13: flowistry::infoflow::analysis::FlowAnalysis::transfer_function::{{closure}}
  14: flowistry::infoflow::analysis::FlowAnalysis::transfer_function
  15: rustc_middle::mir::visit::Visitor::visit_statement
  16: flowistry::mir::engine::iterate_to_fixpoint
  17: std::thread::local::LocalKey::with
  18: flowistry::infoflow::compute_flow
  19: ::analyze
  20: rustc_interface::passes::QueryContext::enter
  21:  as rustc_driver::Callbacks>::after_parsing
  22: ::enter::, rustc_errors::ErrorGuaranteed>>
  23: rustc_span::with_source_map::, rustc_interface::interface::create_compiler_and_run, rustc_driver::run_compiler::{closure#1}>::{closure#1}>
  24: rustc_interface::interface::create_compiler_and_run::, rustc_driver::run_compiler::{closure#1}>
  25: >::set::, rustc_driver::run_compiler::{closure#1}>::{closure#0}, core::result::Result<(), rustc_errors::ErrorGuaranteed>>
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: could not compile `rust-playground`

Flowistry (in vscode) doesn't seem to like trait default implementations

I've been poking around an existing project in flowistry, and finding places where it seems to fail to do anything.

One such place: It seems to be blind to the bodies of default method implementations in traits.

When I click in one, the output window doesn't show anything (it doesn't trigger a run of "flowistry focus", and the highlighting in the window doesn't change (it stays on whatever function I was previously looking at.) This seems to be the same behavior as when I click on a line that doesn't contain a function at all.

Error when path contains a space

Problem

Flowistry displays an error message when the rust c path contains a space.

Logs

OS: win32 (10.0.19044)
VSCode: 1.67.2
Error message

'C:\Users\The' is not recognized as an internal or external command,
operable program or batch file.

Full log: https://paste.rs/fKC

Windows support

  • add windows target to release builds
  • fix span bugs? is this related to unicode? document.offsetAt seems character based and not byte-based

Compatibility with stable rustc

Flowistry should work with projects using stable, not just nightly. It seems like there are two ways to do this:

  1. Become a component, like Clippy
  2. Reimplement the compiler, like rust-analyzer

(2) would be too much work. We could become a rust-analyzer plugin, but this would require RA to support borrow checking / MIR, and right now they only lower to HIR.

(1) would be ideal, but would probably require a lot of buy-in from the community? Not sure what the threshold is for becoming a component.

A third option would be to mimic being a component by releasing with the specific nightly used to compile stable releases. I think? Not sure if any tools take this approach or how viable it is.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.