yihui / knitr

A general-purpose tool for dynamic report generation in R

knitr

The R package knitr is a general-purpose literate programming engine, with lightweight API's designed to give users full control of the output without heavy coding work. It combines many features into one package with slight tweaks motivated from my everyday use of Sweave. See the package homepage for details and examples. See FAQ's for a list of frequently asked questions (including where to ask questions).

Installation

You can install the stable version on CRAN:

install.packages('knitr')

You can also install the development version (hourly build) from https://yihui.r-universe.dev:

options(repos = c(
  yihui = 'https://yihui.r-universe.dev',
  CRAN = 'https://cloud.r-project.org'
))

install.packages('knitr')

Motivation

While Sweave and related add-on packages like cacheSweave and pgfSweave are fairly good engines for literate programming in R, I often feel my hands are tied. For example:

• I stared at the source code of Sweave and wished for hundreds of times, if only I could easily insert [width=.8\textwidth] between \includegraphics and {my-plot.pdf}. (The official way in Sweave is \setkeys{Gin} but it is setting a global width, which is unrealistic since we often have to set widths individually; yes, you can use \setkeys{Gin} for many times, but why not just provide an option for each chunk?)
• I wished for many times, if only I could use graphics devices other than PDF and postscript; now the dream has come true in the official R, but what I was hoping for was an option as simple as dev = 'png' or dev = 'CairoJPEG'.
• I wished multiple plots in a code chunk could be recorded instead of only the last one.
• I wished there was a way to round the numbers in \Sexpr{} other than writing expressions like \Sexpr{round(x, 3)} for each single \Sexpr{}
• I wished I did not have to print() plots from. ggplot2 and a simple qplot(x, y) would just give me a plot in Sweave.
• I wished users would never need instructions on Sweave.sty or run into troubles due to the fact that LaTeX cannot find Sweave.sty.
• I wished cacheSweave could print the results of a code chunk even if it was cached.
• I wished brew could support graphics.
• I wished R2HTML could support R code syntax highlighting.
• ...

The package knitr was designed to give the user access to every part of the process of dealing with a literate programming document, so there is no need to hack at any core components if you want more freedom. I have gone through the source code of pgfSweave and cacheSweave for a couple of times and I often feel uncomfortable with the large amount of code copied from official R, especially when R has a new version released (I will begin to worry if the add-on packages are still up-to-date with the official Sweave).

Usage

library(knitr)
?knit
knit(input)

If options are not explicitly specified, knitr will try to guess reasonable default settings. A few manuals are available such as the main manual, and the graphics manual. For a more organized reference, see the knitr book.

License

This package is free and open source software, licensed under GPL.

knitr's Issues

help('knitr')

Very small suggestion, maybe you can add 'knitr' as alias to 'knit' so the help page shows up when using help('knitr')

Image-snippet in minimal example fails in Rstudio

I just installed knitr and tried the included minimal example. The snippet that should produce the image however, fails in a Rstudio session (Win7, R 2.14) with the following error:

  |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>         |  86%
  boring-plots: fig=TRUE, dev=pdf, width=5, height=5, out.width=.45\linewidth 

Error in gzfile(file, "wb") : cannot open the connection
In addition: Warning message:
In gzfile(file, "wb") :
  cannot open compressed file 'C:/Users/MaHermans/AppData/Local/RStudio-Desktop/graphics/c4be2257-
bbc2-4c51-8650-42b5f89e36f1.snapshot', probable reason 'No such file or directory'
Graphics error: Error in gzfile(file, "wb") : cannot open the connection

Perhaps Rstudio places images in a different location then knitr expects?

possible omitted import lead to failure on basic Rnw file.

After install through R CMD build && R CMD INSTALL with package pulled from github on 11/23/2011. I tried the following.
First knit a basic document with no code chunks, which succeeded.
Second I tried a basic code chunk with

<<first>>=
1+1
@

then ran knit("test.Rnw") and received the following error.

Error in match(data$token, grammar_symbols$token) : 
  object 'grammar_symbols' not found
In addition: Warning message:
In block_exec(code = "1+1", eval = TRUE, echo = TRUE, results = "markup",  :
  failed to tidy R code in chunk <first>
reason: Error in tidy.source(text = code, output = FALSE) : 
  could not find function "parser"

This might be an omitted import or somethings along those lines.

Specifying options at knit time

Sweave packages can specify options at compile time for certain options that the package supports, such as

> Sweave("test.Rnw", prefix.string="figures/fig")
Writing to file test.tex
Processing code chunks with options ...
 1 : echo term verbatim (label = first)
 2 : echo term verbatim (label = cachesomething)
 3 : term verbatim pdf (label = plotit)

You can now run (pdf)latex on 'test.tex'

creates a tex document and places all figures in the figures directory with fig prefixed to the figure. I see no way to specify this in knitr

> knit("test.Rnw", prefix.string='figures/fig-')
Error in knit("test.Rnw", prefix.string = "figures/fig-") : 
  unused argument(s) (prefix.string = "figures/fig-")

knit maybe could accept extra arguments in the form of knit(input, output, pattern, ...) where ... would be used to specify defaults that are then written over for file and chunk options.

This is extremely useful for specifying with a Makefile

%.tex:%Rnw
     Rscript -e "library(knitr);knit('$^","$@", prefix.string='figures/$*-')"

Which creates a tex file but could also puts graphics in a specific graphics folder which helps with cleanup, and prefixes graphics with the filename stem to prevent naming collisions, very useful with automated reporting.

Preferably, I would like to see knitr accept anything as an option and either silently or with warning discard those not valid or not used by specific blocks.

Also as a note. Original Sweave allowed for options set through SWEAVE_OPTS environment variable, although not all extension packages use this, nor is is used very much be people in general I believe.

use pdf(file = NULL) in R > 2.14.0

Paul has added support to file = NULL in the pdf() device in rev.57541, which acts like a null device. This will make knitr even faster. Testing code:

x = rnorm(1000)
system.time({
    .Call("R_GD_nullDevice", PACKAGE = "grDevices")
    replicate(500, plot(x, pch = 1:21))
    dev.off()
})
system.time({
    pdf(file.path(tempdir(), "Rplots.pdf"))
    replicate(500, plot(x, pch = 1:21))
    dev.off()
})
system.time({
    png(file.path(tempdir(), "Rplots.png"))
    replicate(500, plot(x, pch = 1:21))
    dev.off()
})
system.time({
    pdf(file = NULL)
    replicate(500, plot(x, pch = 1:21))
    dev.off()
})

internally compile tikz to pdf to fulfill externalization of tikz graphics

may not need the approach used pgfSweave; that is way too complicated

take a closer look at tikz(..., standalone = TRUE); the key is to extract the preamble out of the source document

smaller options('width') and options('digits') by default

if we do not set width, it is very likely to exceed the page margin; default digits is too big (7), who the heck cares about all 7 digits?

Error with dev = tikz, external = T

I'm in the process of converting from pgfSweave to knitr -- a great package!

The following basic code chunks (adapted from the minimal example)

<< setup,echo=FALSE,results=hide,message=FALSE >>=
options(replace.assign=TRUE,width=90,tikzDefaultEngine=pdfTex)
knit_hooks$set(fig=function(before, options, envir){if (before) par(mar=c(4,4,.1,.1),cex.lab=.95,cex.axis=.9,mgp=c(2,.7,0),tcl=-.3)})
@

<< boring-plots,fig=TRUE,dev=tikz,width=5,height=5,out.width=.45\linewidth, external=T >>=
par(las=1) # tick labels direction
boxplot(x)
@

gave the following error

This is pdfTeX, Version 3.1415926-2.3-1.40.12 (MiKTeX 2.9)
entering extended mode
! I can't find file `'miminal-boring-plots.tikz''.
<*> 'miminal-boring-plots.tikz'

Please type another input file name: 
! Emergency stop.
<*> 'miminal-boring-plots.tikz'

!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on texput.log.
Error: failed to compile figure/miminal-boring-plots.tikz to PDF
In addition: Warning message:
running command 'C:\PROGRA~2\MIKTEX~1.9\miktex\bin\pdflatex.exe 'miminal-boring-plots.tikz'' had status 1 

The original dev = pdf works fine, as does external = F.

My R session is below:

library(tikzDevice)
> getOption("tikzDefaultEngine")
[1] "pdftex"
> sessionInfo()
R version 2.14.0 (2011-10-31)
Platform: i386-pc-mingw32/i386 (32-bit)

locale:
[1] LC_COLLATE=English_Australia.1252  LC_CTYPE=English_Australia.1252    LC_MONETARY=English_Australia.1252
[4] LC_NUMERIC=C                       LC_TIME=English_Australia.1252    

attached base packages:
 [1] splines   grid      tools     stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] tikzDevice_0.6.2   pls_2.3-0          knitr_0.0.3        devtools_0.4       Cubist_0.0.8       reshape2_1.2      
 [7] lattice_0.20-0     Hmisc_3.9-0        survival_2.36-10   munsell_0.3        gridExtra_0.8.5    treemap_0.10      
[13] RColorBrewer_1.0-5 ggplot2_0.8.9      proto_0.3-9.2      reshape_0.8.4      plyr_1.6           signal_0.7-1      
[19] MASS_7.3-16        pgfSweave_1.2.1    cacheSweave_0.6    formatR_0.3-4      optparse_0.9.4     getopt_1.17       
[25] highlight_0.2-5    parser_0.0-14      Rcpp_0.9.7         codetools_0.2-8    stashR_0.3-4       filehash_2.2      

loaded via a namespace (and not attached):
[1] cluster_1.14.1   colorspace_1.1-0 digest_0.5.1     evaluate_0.4     RCurl_1.8-0.1    stringr_0.5   

code reference to an external R script

this will be handy

is the framed package included in most LaTeX systems?

I want to use the framed package to decorate chunks but it does not seem to be a core package of LaTeX, so need to check MikTeX under Windows later; Ubuntu should not be a problem; what about Mac?

kpsewhich framed.sty

\usepackage{framed}
\definecolor{shadecolor}{rgb}{.95, .95, .95}

\begin{shaded}
\begin{minipage}[t]{1\columnwidth}%

\end{minipage}
\end{shaded}

Caching on another drive will fail

This looks like it's new in 0.0.4, but cache is concatenated with current directory, so when specifying a drive for cache when working from z:\ drive, will try to cache in z:\d:\cache instead of d:\cache. Is it because of 8ffb1aa ?

minimal example for HTML

may spend some time working on highlighting

a size option to control the font size of highlighted code

Option to omit progress bars

Maybe I just didn't see it but is there an option for kint() to omit the progress bars? If I run a simple wrapper bash script in emacs it looks abit weird:

improve rgl hook to accommodate html output

instead of hard-coded \includegraphics{}

Font size on website is too small

And it makes the font look a big ugly because of anti-aliasing. If you're going to use a complex serif font, I'd recommend boosting the size up a bit (and don't make list items smaller).

implement split=TRUE/FALSE

very easy; do it in the chunk hook function

replace \SweaveOpts{} by empty string

since \SweaveOpts is not valid LaTeX command

demos in the website

currently the example files are messy; need a clear guide (source and output)

from Tal: http://yihui.name/en/2011/12/knitr-elegant-flexible-and-fast-dynamic-report-generation-with-r/#comment-11045

need to tweak the default color theme of highlight

it is too colorful to read

\\newcommand{\\hlstd}[1]{\\textcolor[rgb]{0,0,0}{#1}}%
\\newcommand{\\hlnum}[1]{\\textcolor[rgb]{0.16,0.16,1}{#1}}
\\newcommand{\\hlesc}[1]{\\textcolor[rgb]{1,0,1}{#1}}
\\newcommand{\\hlstr}[1]{\\textcolor[rgb]{1,0,0}{#1}}
\\newcommand{\\hldstr}[1]{\\textcolor[rgb]{0.51,0.51,0}{#1}}
\\newcommand{\\hlslc}[1]{\\textcolor[rgb]{0.51,0.51,0.51}{\\it{#1}}}
\\newcommand{\\hlcom}[1]{\\textcolor[rgb]{0.51,0.51,0.51}{\\it{#1}}}
\\newcommand{\\hldir}[1]{\\textcolor[rgb]{0,0.51,0}{#1}}
\\newcommand{\\hlsym}[1]{\\textcolor[rgb]{0,0,0}{#1}}
\\newcommand{\\hlline}[1]{\\textcolor[rgb]{0.33,0.33,0.33}{#1}}
\\newcommand{\\hlkwa}[1]{\\textcolor[rgb]{0,0,0}{\\bf{#1}}}
\\newcommand{\\hlkwb}[1]{\\textcolor[rgb]{0.51,0,0}{#1}}
\\newcommand{\\hlkwc}[1]{\\textcolor[rgb]{0,0,0}{\\bf{#1}}}
\\newcommand{\\hlkwd}[1]{\\textcolor[rgb]{0,0,0.51}{#1}}

Specifying a cache directory

Related to the prefix.string alluded to in issue #19 I see no way to specify a cache directory. with both cacheSweave and pgfSweave this is accomplished by setCacheDir("cache"). This fails on knitr. This is useful for easing cleanup with a makefile as well as keeping a directory clean as well. It could possibly be specified as a global option like in issue #19.

options animate and interval

to insert animations

compatibility with Sweave.sty, listings...

some people have hacked Sweave.sty, e.g. http://biostat.mc.vanderbilt.edu/wiki/pub/Main/SweaveTemplate/Sweavel.sty

so need a set of hooks to switch to Sweave immediately

show some example code inline in the website

to make it easier for beginners to read and understand the process

rename theme_xxx() to something else

Ramnath is absolutely right -- these names are misleading, since they do not actually mean themes.

create a tag wiki page on StackOverflow

I do not have edit privileges at the moment, but may do this in the future when there are enough knitr users: http://stackoverflow.com/edit-tag-wiki/75535

Error using dev=tikz

I'm doing my first job with knitr (thank you and congratulations for the package). I did a very basic Rnw file containing this chunk

<<cars-demo, fig=TRUE, dev=tikz, highlight=FALSE>>=
fit=lm(dist~speed,data=cars) # linear regression
par(mar=c(4, 4, 1, .1), mgp=c(2,1,0))
with(cars,plot(speed,dist,panel.last=abline(fit)))
text(10,100,'$Y = \\beta_0 + \\beta_1x + \\epsilon$')
@

When compiling I get this error message

Erro em switch(getOption("tikzDefaultEngine"), pdftex = getOption("tikzLatexPackages"),  : 
  EXPR deve ser um vetor de comprimento 1

I don't know how solve this. When I used dev=pdf, the compilation was successiful. Can you help me?

include the rgl hook in knitr

since it might be useful to the general audience

rethink results=tex, verbatim, hide

it might not be a good idea to change it to a TRUE/FALSE option, but the problem is that Sweave restricted itself to LaTeX in the beginning, so the option tex is not a good name

Parse markdown report and save it to R object

I'm still a newbie for your package, and I'm trying to set my mind into place, since I've been working on some kind of templating engine too. Your approach, however is way different (and better) than mine, so sorry if I'm asking something trivial. AFAICS, hooks are very versatile, and you can tell them to do all kinds of stuff with chunks. Is it possible to parse template in such way, that you can save it's contents to R objects - e.g. store each chunk/block to a list element?

P.S.
Note that I'm interested only in markdown format, it's OK if it's not Sweave/HTML compatible.

\SweaveInput

Is there anything analogous to \SweaveInput in knitr? \SweaveInput can be used to split a Rnw file into different files; similar to \input{file} in latex, which can help organise larger Rnw files. Or does everything need to be in a single file?

latex tables

Could you provide an example, maybe in the manual, of making a table? I'm struggling to work out how to use either xtable or latex (Hmisc) to produce a nicely formatted table.

results = tex, which is the usual route does not work and I am not sure what does...

Thanks

auto-generation of bib citations after knitting a document

use the tweakr package; this should be very useful

when knitting a document, record all the packages used in this document, and create a bib file using tweak_citation() in tweakr

Changing working directory will impact the location of output

This is a problem with sweave as well, but when you change a directory
setwd("aplace")
this will output the figures and the .tex in ~ directory. It can even lead to having stuff really scattered.
setwd("here") plot(something) setwd("there") plot(something) ...
It can even bring LyX to crash.

turn an R script to a report

take an R script, knit it into a tex document or html, etc

put some templates under ./misc/, contains <<ref=the.input.R.script,fig=TRUE,...>>=

a function report(scripts, templates)

any suggestions on the function name? report() seems to be dull

Allow user to specify colors by name

It would be useful to allow users to specify the background color by name, instead of having to enter rgb values. A simple internal function can handle the conversion.

col2knit <- function(color){
  paste(round(col2rgb(color)/255, 2), collapse = ";")
}

col2knit('springgreen3')
[1] "0;0.8;0.4"

\SweaveOpts ignored and retained in document

File level options specified through the \SweaveOpts{} as mentioned in the vignette are not removed from the document. When compiling with pdflatex I get the error.

! Undefined control sequence.
l.387 \SweaveOpts
                 {cache=T)

It appears that these options are both ignored and retained when knitting the document.

pdfcrop

This looks absolutely marvellous. The issues you have tried to overcome ring true with my experience.

One feature I would like to see is a way for automatically cropping pdfs using the pdfcrop tool that comes with latex distributions. I have a bash script that does this and it results in much neater reports and gets rid of a lot of unnecessary white space. This is especially useful in something like sweave because you generally don't always make the plots interactively and have a feel for the right dimension - and this can results in extra white space.

Anyway - I shall follow progress with this project with interest.

Great work!

David

Getting uncommented R output

Well, the title is pretty much self-explanatory. So far I managed to run .md example successfully, but what if I want to add some tables to a report? I'd like to use ascii package, and to export the output in say, pandoc format, like this:

<!--begin.rcode echo=FALSE
a <- ascii(mtcars)
print(a, type = "pandoc")
end.rcode-->

And what I get is correct output (a pandoc table), but with ## inserted at the beginning of each line. How to avoid this?

line numbering with listings or fancyvrb

moved from #29 suggested by @walmes

encoding problem

remember to switch to Windows to check if there are encoding problems (test with Chinese); this sounds pretty difficult in Sweave, otherwise why they used so complicated approaches to detect encodings?

Box width too narrow and Reference Classes not supported?

Hi Yihui,

kudos for knitr, this thing ROCKS!!

I just played around a bit with it and it looks great!
Came across two issues though:

1. The boxes that contain R code are just a tiny bit too narrow on the right hand side (just try making (x <- rnorm(20)) a bit larger in your minimal example and you'll see)
2. Code for reference classes does not seem to work (setRefClass etcl).

It'd be great if you could take a look at this once you get a chance!

Thanks a lot,
Janko

PS: I'm on Windows XP SP2, R 2.14.0, LyX 2.0.2, MikTeX 2.9.

tangle R code

two options

• just tangle R code
• tangle and run (like tidy.eval() in formatR)

Allow users to specify theme for syntax highlighting

First off, excellent work on the package. Sweave is a part of my daily workflow, and this package provides most of the features that I wished Sweave would. I have a couple of suggestions, which I will try to post from time to time. My first suggestion is to add a feature that allows users to specify the theme for syntax highlighting.

I see that you are using the highlight package, which uses a css file located in the stylesheet folder to control the highlighting. It would be nice to allow users to specify the highlighting theme using a css file. In the long run, it would be possible to convert standard themes available in highlight or textmate to css files, allowing the user to choose the style in SweaveOpts.

Let me know if you are open to this suggestion. I have browsed through your code and see that currently, the style definitions are in the variable .header.hi.tex. I can work with you in creating the mechanism that would allow user specified styles, if you would like me to.

Clarify how to specify package options

You pointed me to http://yihui.github.com/knitr/options#package_options in your comment on issue #19, however it is not at all clear how to specify package-options such as progress and theme. The website could be improved by a clear example of setting package options.

Tutorial for beginners

I never did sweave, but I did ok in latex and R: I couldn't grasp how the two worlds/workflows of analysis (in R or SPSS) and writing in TexnicCenter could come together. I would love to see a beginners tutorial showing this workflow when preparing a real paper:

Do an analysis (for example a regression, two models)
write a text(in an latex editor? -> syntax highlighting)
integrate a reference (biblatex -> I can do that in the regular editor, but how does that change with knitr?)
integrate the results

• p-values, whatever, nicely formatted/rounded in the regular text
• table of the regression result, formated in booktab style, properly described and numbered
• plot (regression diagnostics,...)
  compile the resulting pdf

(let the reader marvel at the productivty gained over the old workflow using tables cut and pasted from SPSS to excel, exported to latex and "postprocessed" ... and then done again because the dataset changed)

Thanks a lot!

updating stringr to v0.6 breaks knitr

have not figured out the reason yet; seems to be due to str_match

Many images in a single chunk might not plot

If I run e.g.
<<fig, fig = TRUE, width=4, height=4>>= m <- matrix(1:100, ncol = 10) image(m) image(m*2) @
I get a single image. I guess depending on the configuration of the page, it might be produced by running for(i in 1:6) image(m). But at a certain point it won't add de figures, nor will it produce them.

However,
<<fig, fig = TRUE, width=4, height=4>>= plot(rnorm(100)) image(m*2) @
is diplaying two figures.

knitr version 0.0.4, R 2.14.0

Overriding File Level Options.

Since Knitr is not exactly the same as Sweave it might be preferable to have an \KnitrOpts for tex documents rather than relying on the \SweaveOpts. Ideally, \SweaveOpts options would translate to knitr options, and would do things like translate pdf=T into dev=pdf sort of options and \KnitrOpts would have strict option checking. This would ease the transition into knitr for people, although the transition has been very easy so far. Again very nice work.

scientific notation for \Sexpr{}

may want this in the inline hook: http://stackoverflow.com/q/8366324