r-lib / progress Goto Github PK
View Code? Open in Web Editor NEWProgress bar in your R terminal
Home Page: http://r-lib.github.io/progress/
License: Other
Progress bar in your R terminal
Home Page: http://r-lib.github.io/progress/
License: Other
From the cli package
E.g.
with_bar(
for (index in sequence) block
)
and
lapply_with_bar(list, func)
Implementation is easy. The for
loops need some non-standard evaluation.
Needed for performance.
It just stops them from being updated. Is this the expected behaviour?
For example on my machine, this shows a progress bar with 1 tick, waits 1s and then starts updating it.
pb <- progress_bar$new(total = 100, show_after=1)
for (i in 1:100) {
pb$tick()
Sys.sleep(2 / 100)
}
Or here the progress bar is shown, when I would not expect it to be shown.
pb <- progress_bar$new(total = 100, show_after=1)
for (i in 1:100) {
pb$tick()
Sys.sleep(0.01 / 100)
}
I see that commit 532715f added a relevant test, but the expectation does not seem to match the description "very quick loops, only the result is shown"
. Thanks for clarifying!
Is the progress package also suitable for the doMC package?
Not sure how, though.
They can all handle \r and that's all we need.
It'd be nice to have a tick_rate
parameter that gives the estimated number of ticks per second. Even better, you'd have some way to control the format of this value so you could add commas etc, when needed.
I'd like this for bigrquery where it's most natural to express the speed in the number of rows downloaded.
library(progress)
pb = progress_bar$new(total = 10)
pb$update(ratio = 0.2)
I get:
Error in pb_update(self, private, ratio, tokens) :
attempt to apply non-function
If your progress bar or message is larger than the width it messes up the output because the extra characters are not blanked out. Maybe these inputs could be trimmed to the width ala glue::collapse()
glue::collapse("123456", width = 5)
#> 12...
It would also be nice if progress was color aware in general, using crayon::col_nchar()
.
How to properly document R6 classes? By hand?
What do you think about having a throttle
argument, which controls how often the progress bar is updated on the screen?
Some tracked operations might occur very quickly, which could result in overly-fast updated to the progress bar on the screen. It would be nice to be able to say something like, 'only re-draw the progress bar every 0.1 seconds', or something similar.
In rncl, I use the C++ API of progress, and I got an email from CRAN saying they see the following warnings:
with the clang compiler
/home/hornik/tmp/R.check/r-devel-clang/Work/build/Packages/progress/include/RProgress.h:172:13: warning: variable length arrays are a C99 feature [-Wvla-extension]
/home/hornik/tmp/R.check/r-devel-clang/Work/build/Packages/progress/include/RProgress.h:220:16: warning: variable length arrays are a C99 feature [-Wvla-extension]
on Windows:
d:/RCompile/CRANpkg/lib/3.4/progress/include/RProgress.h:172:27: warning: ISO C++ forbids variable length array 'bar' [-Wvla]
d:/RCompile/CRANpkg/lib/3.4/progress/include/RProgress.h:220:26: warning: ISO C++ forbids variable length array 'spaces' [-Wvla]
library(progress)
pb <- progress_bar$new(format = "processing [:bar] :percent :elapsed | :eta",
total = 100, width = 80)
for (i in 1:100) {
pb$tick()
Sys.sleep(1 / 100)
}
If I run the code in RStudio, there's no problem at all. But if I run the code with rscript
in terminal, then the progress bar is repeatedly printed out like
As long as I remove width
argument, the problem is gone.
Here's my session info:
R version 3.2.0 (2015-04-16)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 8 x64 (build 9200)
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] progress_1.0.1
loaded via a namespace (and not attached):
[1] magrittr_1.5 assertthat_0.1 R6_2.0.1 rversions_1.0.0 prettyunits_1.0.0
[6] tools_3.2.0 rstudioapi_0.3.1 RCurl_1.95-4.6 memoise_0.2.1 git2r_0.10.1
[11] digest_0.6.8 bitops_1.0-6 devtools_1.8.0 XML_3.98-1.2
I'd like to use this, but am concerned that it might be a bit slow for general use. A really simple profile:
Rprof()
for (i in 1:10) {
pb <- progress_bar$new(total = 100)
for (j in 1:100) {
pb$tick()
}
}
Rprof(NULL)
I see about 5s to to the thousand ticks here, which seems far too much. The top few culprits for total time:
total.time total.pct self.time self.pct
"<Anonymous>" 5.00 100.0 0.02 0.4
"pb_tick" 5.00 100.0 0.00 0.0
"pb_render" 4.92 98.4 0.00 0.0
"%>%" 2.46 49.2 0.04 0.8
"isatty" 2.28 45.6 2.28 45.6
"is_supported" 2.28 45.6 0.00 0.0
"cursor_to_start" 2.12 42.4 0.00 0.0
"eval" 1.34 26.8 0.16 3.2
"split_chain" 1.26 25.2 0.16 3.2
"freduce" 1.04 20.8 0.04 0.8
"withVisible" 1.04 20.8 0.02 0.4
"_fseq" 1.04 20.8 0.00 0.0
"Recall" 1.00 20.0 0.00 0.0
"as.list" 0.72 14.4 0.48 9.6
"%s%" 0.66 13.2 0.00 0.0
"pretty_bytes" 0.62 12.4 0.02 0.4
"is_first" 0.52 10.4 0.02 0.4
presumably is_supported
is used for can be done at progress bar creation and cached. It looks like the overhead from chaining is also consuming a decent amount of time.
For progress bars with an unknown number of total ticks, it'd be nice to be able to express that directly by setting total
to some proxy value. (Of course, one alternative is to just pick a very large total
, e.g. setting it to 1E7)
In a similar vein, it might be nice to have a 'finalize' method, to be called on the progress bar to indicate that whatever operation you were tracking is done. It would be semantically equivalent to update(1)
, I think?
Hi Gabor,
just found this package โ great!
I thought it might be nice to add an example of making a plyr compatible progress bar. Here is my first pass.
Best, Greg
# progress bar creator for plyr
progress_progress<-function(...){
pb <- NULL
list(init = function(x, ...) {
pb<<-progress_bar$new(total = x, ...)
}, step = function() {
pb$tick()
}, term = function() NULL)
}
plyr::l_ply(1:100, .fun=function(...) Sys.sleep(0.01), .progress='progress')
Two, probably related, issues with custom token handling here. Can separate these into two issues if you'd like, in which case treat this as the first issue:
The full line is not correctly erased when a custom token does not
keep to a constant width.
tt <- seq(10, 0, by=-.5)
p <- progress::progress_bar$new("giving up in :remaining s", total=length(tt))
for (t in tt) {
p$tick(tokens=list(remaining=t))
Sys.sleep(.2)
}
which results in artefacts like
giving up in 9 s s
being printed. I can work around this with formatC
but it would be nice if the line erasing logic dealt with this directly.
The other issue is that the whitespace handling at the end of custom tokens seems to be dealt with differently than other tokens -- compare :remining(s) with :current(s)
tt <- seq(10, 0, by=-.5)
p <- progress::progress_bar$new("giving up in :remainings :currents",
total=length(tt))
for (t in tt) {
p$tick(tokens=list(remaining=t))
Sys.sleep(.1)
}
It does not make much sense to show it for things shorter than a second, usually.
We can specify the complete
and incomplete
characters in the progress bar constructor, but it would also be nice if we could specify the 'end' of the progress bar, so that we could draw e.g.
[=====>.....]
^ ~~ I want to be able to draw the right arrow here
It would be really helpful to have an example like the following in the readme for purrr:
library(purrr)
library(dplyr)
# dummy list of 10 elements with random numbers
dummy_list <- rerun(10, runif(5))
# create the progress bar with a dplyr function.
pb <- progress_estimated(length(dummy_list))
res <- dummy_list %>%
map(~{
# update the progress bar (tick()) and print progress (print())
pb$tick()$print()
Sys.sleep(0.5)
sum(.x)
})
Do you think this is a bit hacky code, as you are sleeping every .5 seconds?
Would be great. E.g. if there is a progress bar within a progress bar...
I have a use case combined with custom tags for advancing a progress bar by zero units, but this is currently disallowed
pb <- progress::progress_bar$new()
pb$tick(0)
I tried removing the assertion but ran into another bit of trouble with the time estimation code, even though I was not using estimation (sorry don't have the traceback handy).
In case you're wondering, I want to have a small ascii spinning wheel tick over periodically when there are no results retuned. However, I can imagine more general cases where allowing negative increments could actually make sense...
Hi, thanks for the useful package and it works fine on my mac for both the R version and Cpp version. However, the cpp version of progress bar fails to display on windows. Here's a minimal repro (using the code from your test package):
library(Rcpp)
sourceCpp(code = '
#include <Rcpp.h>
#include <RProgress.h>
#include <unistd.h>
// [[Rcpp::depends("progress")]]
// [[Rcpp::export]]
Rcpp::CharacterVector test_progress(Rcpp::CharacterVector formatSEXP =
"[:bar] :percent ") {
BEGIN_RCPP
const char *format = formatSEXP[0];
RProgress::RProgress pb(format);
pb.tick(0);
for (int i = 0; i < 100; i++) {
usleep(2.0 / 100 * 1000000);
pb.tick();
}
Rcpp::CharacterVector result(1);
result[0] = "DONE";
return result;
END_RCPP
}'
)
test_progress()
> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
Matrix products: default
locale:
[1] LC_COLLATE=Chinese (Simplified)_People's Republic of China.936
[2] LC_CTYPE=Chinese (Simplified)_People's Republic of China.936
[3] LC_MONETARY=Chinese (Simplified)_People's Republic of China.936
[4] LC_NUMERIC=C
[5] LC_TIME=Chinese (Simplified)_People's Republic of China.936
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] Rcpp_0.12.13
loaded via a namespace (and not attached):
[1] compiler_3.4.1 magrittr_1.5 assertthat_0.2.0 R6_2.2.2 prettyunits_1.0.2 tools_3.4.1
[7] yaml_2.1.14 progress_1.1.2
When I use progress and some logging at the same time, the bar gets messed up by the text bing printed. Would that be possible to have the bar always 1 line bellow so that you can log stuff if needs be but still have the nice progress bar ?
Primarily for extra fast progress bars. They can be pre-coded, so no template parsing is needed. Maybe these styles:
:spinnerbar
(just a spinner, really minimal)
:counterbar
current/total, quite minimal as well
:etabar
similar to the current dplyr style:
|=================================== | 66% ~2 s remaining
:downloadbar
Sg for downloads, e.g. 150.4 kB / 100MB
Using https://github.com/gaborcsardi/pkgconfig
This will be especially handy for #27.
and active bindings in an environment for the tokens.
To display downloads, using :rate and :bytes.
progress in speedy letters and a text-like progress bar below it.
According to the documentation,
The progress bar is displayed after the first 'tick' command. This might not be desirable for long computations, because nothing is shown before the first tick. It is good practice to call 'tick(0)' at the beginning of the computation or download, which shows the progress bar immediately.
But actually it's not printed on first tick:
tick <- progress::progress_bar$new()$tick
invisible(tick(0))
invisible(tick())
#> [=----------------------------------------------------------------------] 1%
Neither on first update:
update <- progress::progress_bar$new()$update
invisible(update(0))
invisible(update(0.01))
#> [=----------------------------------------------------------------------] 1%
When compiling progresstest, the following warning appears:
test.cpp:53:62: warning: overflow in implicit constant conversion [-Woverflow]
result[19] = RProgress::RProgress::pretty_bytes(13333333337);
Running the my_test_progress() function results in a pause and printing of "DONE", but no progressbar is shown.
It probably does not make sense to create an R6 object from C++. IF the progress bar is only used from C++, then we would rather have
Challenges:
stdout()
and stderr()
available in C++?More bits:
Here is the result.
==> Rcmd.exe INSTALL --no-multiarch --with-keep.source progresstest
* installing to library 'C:/Users/outwen/R/win-library/3.2'
* installing *source* package 'progresstest' ...
g++ -m64 -I"C:/PROGRA~1/R/R-32~1.0/include" -I"C:/Users/outwen/R/win-library/3.2/progress/include" -I"C:/Users/outwen/R/win-library/3.2/Rcpp/include" -I"d:/RCompile/r-compiling/local/local320/include" -O3 -Wall -mtune=core2 -c test.cpp -o test.o
** libs
In file included from test.cpp:3:0:
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:104:8: error: declaration of '__iob_func' as array of references
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:208:24: error: declaration of '__iob_func' as array of references
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:208:30: error: expected ')' before ',' token
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:208:32: error: expected unqualified-id before 'int'
test.cpp:58:1: error: expected '}' at end of input
In file included from test.cpp:3:0:
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h: In constructor 'RProgress::RProgress::RProgress(std::string, double, int, char, char, bool, double)':
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:108:10: warning: 'RProgress::RProgress::show_after' will be initialized after [-Wreorder]
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:98:8: warning: 'bool RProgress::RProgress::first' [-Wreorder]
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:41:3: warning: when initialized here [-Wreorder]
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:54:30: error: 'is_supported' was not declared in this scope
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:55:29: error: 'default_stderr' was not declared in this scope
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h: In member function 'void RProgress::RProgress::tick(double)':
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:74:35: error: 'time_now' was not declared in this scope
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:79:37: error: 'time_now' was not declared in this scope
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h: In member function 'void RProgress::RProgress::render()':
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:126:46: error: 'replace_all' was not declared in this scope
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:130:36: error: 'time_now' was not declared in this scope
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:131:48: error: 'vague_dt' was not declared in this scope
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:135:22: error: 'round' is not a member of 'std'
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:135:22: note: suggested alternatives:
c:\rtools\gcc-4.6.3\bin\../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/include/math.h:740:25: note: 'round'
C:/Users/outwen/R/win-library/3.2/Rcpp/include/Rcpp/sugar/functions/math.h:69:1: note: 'Rcpp::round'
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:143:28: error: 'round' is not a member of 'std'
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:143:28: note: suggested alternatives:
c:\rtools\gcc-4.6.3\bin\../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/include/math.h:740:25: note: 'round'
C:/Users/outwen/R/win-library/3.2/Rcpp/include/Rcpp/sugar/functions/math.h:69:1: note: 'Rcpp::round'
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:143:48: error: 'pretty_bytes' was not declared in this scope
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:177:29: error: 'cursor_to_start' was not declared in this scope
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h: In member function 'void RProgress::RProgress::terminate()':
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:191:29: error: 'cursor_to_start' was not declared in this scope
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h: At global scope:
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:206:3: error: expected unqualified-id at end of input
C:/Users/outwen/R/win-library/3.2/progress/include/RProgress.h:206:3: error: expected '}' at end of input
make: *** [test.o] Error 1
> sessionInfo()
R version 3.2.0 (2015-04-16)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 8 x64 (build 9200)
locale:
[1] LC_COLLATE=Chinese (Simplified)_China.936
[2] LC_CTYPE=Chinese (Simplified)_China.936
[3] LC_MONETARY=Chinese (Simplified)_China.936
[4] LC_NUMERIC=C
[5] LC_TIME=Chinese (Simplified)_China.936
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] tools_3.2.0
Because of the embedded ANSI codes, width calculation for progress bars that use crayon (or unicode characters probably) is off:
fmt <- sprintf("[%s] %s", crayon::red(":bar"), crayon::blue(":percent"))
pb <- progress::progress_bar$new(fmt, total = 30, clear = FALSE, width = 80)
for (i in 1:30) {
pb$tick()
Sys.sleep(3 / 100)
}
message(paste0(rep("-", 80), collapse=""))
Not sure what the best fix is here. If the ANSI codes are in the format, it could be unescaped and the difference between and apparent and true with calculated and added to width. If custom tokens contain escapes they'd need treating on every tick.
In the docs you say:
callback Callback function to call when the progress bar finishes.
The progress bar object itself is passed to it as the single
parameter.
and in the tests you do use a function of the form function(self) ...
but when callback is called by progress bar it's called as self$callback()
not self$callback(self)
Hi,
Reading this:
progress/inst/include/RProgress.h
Line 256 in 6c98723
I wonder: is the bug in R or in Rstudio console ?
I kind of remember I got problems too with RcppProgress...
... when preceded by a call to message()
if the appendLF = FALSE
argument is used. Instead of the spinner repainting, it appends, like this:
Running job \: 1s:Running job |: 2s:Running job /: 2s:Running job -: 2s:Running job \: 3s:Running job |: 3s:Running job /: 3s:Running job -: 4s:Running job \: 4s:Running job |: 4s:Running job /: 4s:Running job -: 5s:Running job \: 5s:46.1 gigabytes processed ...
First noticed in the most recent version of bigrquery.
Max ~ 5 times a second, or maybe even less. If last update is recent, then do nothing.
Not sure, how, though.
How to show the progress bar styles?
It'd be useful if there was a way to force the progress bar off โ often the parent function will have a quiet
argument, and it would be nice if there was a clean way to pass this responsibility on to progress.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.