pmassicotte / gtrendsr

R functions to perform and display Google Trends queries

gtrendsR

gtrendsR provides an interface for retrieving and displaying Google Trends information.

Trends (number of hits) over time as well as geographic representation of the results can be displayed.

Example

In this simple example, trends for keywords nhl, nba are retrieved for Canada and USA and then plotted from R.

library(gtrendsR)

res <- gtrends(c("nhl", "nba"), geo = c("CA", "US"))
plot(res)

Installation

Since release 1.3.0, the package is on CRAN and can be installed via

install.packages("gtrendsR")

Release-candidate packages are available in the ghrr drat repository and can installed via

install.packages("drat")       # easier repo access + creation
drat:::add("ghrr")             # make it known
install.packages("gtrendsR")   # install it

Development version (which may be less stable) can be installed directly from this repository via

if (!require("devtools")) install.packages("devtools")
devtools::install_github("PMassicotte/gtrendsR")

Using gtrendsR behind a PROXY.

If gtrendsR should be used behind a proxy, especially with NTLM authentication mode, you need to set the proxy parameters using "setHandleParameters" function

Example

library(gtrendsR)

setHandleParameters(user = "xxxx", password = "*******", domain = "mydomain", proxyhost = "10.111.124.113", proxyport = 8080)
res <- gtrends(c("nhl", "nba"), geo = c("CA", "US"))

Additional information

• Limits and quotas on API requests

Authors

Philippe Massicotte and Dirk Eddelbuettel

License

GPL (>= 2)

gtrendsr's Issues

Daily data breaks normal use

Check

plot(gtrends("NBA"))

Add support for multiple countries

https://www.google.com/trends/explore#q=nhl&geo=US%2C%20BR%2C%20JP%2C%20IN%2C%20IT&cmpt=geo&tz=Etc%2FGMT%2B6

https://www.google.com/trends/explore#geo=US,+BR,+JP,+IN,+IT&cmpt=geo&q=nhl&cat=0-18

Geocode more precise

I think it would be great to be able to get results for a precise part of a country if one knows the code for this region. E.g. "IN" is India and "IN-WB" is West Bengali, one could be interested in finding the results for "IN-WB".

Strange error: "Error in charToDate(x) :character string is not in a standard unambiguous format"

I was about to analyse the search history for "Bataclan", but I recieved an error.

trend <- gtrends("Bataclan")
# Error in charToDate(x) : 
#   character string is not in a standard unambiguous format

I then tried some searches on combinations of the substring "Bata" and different letters, which either (a) result in the same error, (b) work fine, or (c) give a more intutive gtrend style error.

# "Bata" + a, works
trend <- gtrends("Bataa")

# "Bata" + b, c, d or e, all the same error
trend <- gtrends("Batab")
# Error in charToDate(x) : 
#  character string is not in a standard unambiguous format

# trend <- gtrends("Batac")
# Error in charToDate(x) : 
#  character string is not in a standard unambiguous format

# and same for "Batad" and "Batae"

Then suddenly "Bataf" gives a more gtrend-oid error:

trend <- gtrends("Bataf")
# Error: Not enough search volume. Please change your search terms
# and the same for "Batag" and "Batah"

Could it be related to the (now closed) issue 24, where at least part of the same error message was mentioned (character string is not in a standard unambiguous format)?

---

R version 3.2.2 (2015-08-14)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=Norwegian (Bokmål)_Norway.1252 LC_CTYPE=Norwegian (Bokmål)_Norway.1252
[3] LC_MONETARY=Norwegian (Bokmål)_Norway.1252 LC_NUMERIC=C
[5] LC_TIME=Norwegian (Bokmål)_Norway.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] gtrendsR_1.3.0

loaded via a namespace (and not attached):
[1] Rcpp_0.12.2 lattice_0.20-33 zoo_1.7-12 digest_0.6.8 MASS_7.3-45
[6] bitops_1.0-6 grid_3.2.2 plyr_1.8.3 gtable_0.1.2 magrittr_1.5
[11] scales_0.3.0 ggplot2_1.0.1 stringi_1.0-1 reshape2_1.4.1 googleVis_0.5.10
[16] proto_0.3-10 RColorBrewer_1.1-2 RJSONIO_1.3-0 tools_3.2.2 stringr_1.0.0
[21] munsell_0.4.2 RCurl_1.95-4.7 colorspace_1.2-6

CRAN?

What do you think should be done before we send it to CRAN?

Only README.md needed

We don't usually need the .Rmd to generate the .md.

Return ggplot2 object

Just had a decent and very simple idea -- we can also return the ggplot2 object for further processing:

R> library(gtrendsR)
R> dp <- gtrends("Donald Drumpf", res="7d")
R> p <- plot(dp) 
R> library(ggplot2)   # to get the ggplot2 functions into the current session
R> p + ggtitle("The Drumpf") + theme(legend.position="none")

et voila, we get a nicer (?) plot which maximizes the data-ink-ratio a little more:

PR forthcoming.

Multiple keyword

Might be more elegant to use a vector of keywords or ... Instead of a simple string when doing queries.

Example:

gtrends(ch, "NHL", "NBA")

Compared to:

gtrends(ch, "nhl, nba, nfl")

"Google Login Failed"

I've seen some previous issues on this get closed due to lack of reproducibility, so I've created an account that anyone can use (below). I'm working on Windows 7, but I would be surprised if the change in platform would fix this.

library(gtrendsR)
usr = "[email protected]"
psw = "gtrendsR123"
gconnect(usr,psw)

I've tried logging in with many different accounts and none of them seem to work. I always get presented with a login challenge by Google during the authenticatePage <- postForm(authenticateURL, .params = formparams, curl = ch) operation in gconnect.

Is there any setting within Google that I can set to allow this kind of login? Is the only alternative to use Google's OAuth2 API? I'd really like to use this package, but I haven't even been able to get off the ground due to the login issues. I'd love to know if anyone has rolled their own OAuth patch that I can use to get by here. Thanks!

Default connection object

We could go from

    gtrends <- function(ch, query, geo = "all", cat = "0", ...) {

to something like

    gtrends <- function(query, geo="all", cat="0", ..., ch=.getDefaultConnection()) {

where .getDefaultConnection() would recall the ch object stored by gconnect() inside a package-local environment.

Time range comparison

Hi again,
Is time range comparison supported? I cannot get it to work. I have tried:
gtrends("NHL", geo="CA", start_date = c(as.Date("2004-01-01"),as.Date("2004-03-01")), end_date = c(as.Date("2004-03-01"),as.Date("2004-05-01")))
Given the simple url structure, why don't you ask for the starting month and the number of months as in date=1%2F2004 5m?
and in case of a number of date ranges you simply have, for example, date=1%2F2004 3m%2C 3%2F2004 3m%2C 5%2F2004 3m%2C 7%2F2004 3m%2C 9%2F2004 3m (and cmpt=date)

How the cat parameter works?

I was trying one simple query in R & Web with cat parameter, to compare results.

in web:

https://www.google.com/trends/explore#cat=0-20&q=NHL&date=1%2F2015%2012m&cmpt=q&tz=Etc%2FGMT-1

in R (Cat parameter "0-20" was in url):

query <- "NHL"
testMix <- gtrends(query, cat="0-20",start_date = as.Date("2015-01-01"), end_date=as.Date("2015-12-31"))

When I checked last 3 numbers, it seems, that trend data are slightly different.
Is something wrong with my query?

Minor niggles

• README can now get a CRAN badge
• README uses relative URL to chart which fails eg here on CRAN; suggest to use this raw URL instead
• The manual pages for gtrends has the example twice; suggest to remove one of the two Roxygen entries

Error in `*tmp*`[[i]] : subscript out of bounds

If I try to run the example code, I always get the error message "Error in *tmp*[[i]] : subscript out of bounds" when running the sport_trend <- gtrends(c("nhl", "nba", "nfl"))-Line.

There is no error when connecting to google or anything. Normally this occurs in loops, but I can't figure out, where the mistake exactly is. Anyone else with this problem?

Shiny Integration

Thanks for the package! I'm not facing any problems. I just wanted to give you feedback and show how I integrated your package in a shiny application: https://github.com/jburkhardt/GoogleTrendsDashboard

Maximum 5 keywords

Google does not allow more than 5 keywords. This is a quick fixe.

Auto-connect ?

I am lazy and in another package requiring a handle I connect on package load if and only if another global is set.

So to go along with user and password, I suggest google.autoconnect that, if TRUE, will have us call gconnect() at package load. If you like the idea I can add that in a branch in no time.

and/or query

I Tried gtrendsR and I found it very usefull. but I don't know how to do an and/or query.
For example with this syntax:

library(gtrendsR)
library("ggthemes")
gconnect(usr = "myaddress", psw = "mypassword", verbose = TRUE)
dp2 <- gtrends(c("Petronas", "Castrol", "Fuchs"), geo = c("IT") , start_date = "2013-06-17", end_date = "2016-03-17")
plot(dp2) + ggplot2::ggtitle("Castrol vs Fuchs vs Petronas in Italy") +
scale_colour_solarized("blue") +
theme_fivethirtyeight() +  geom_line(size = 1)

...How can I search Petronas and/or Selenia (they are the same brand of lubricant) but not "Petronas Towers" (not pertinent)? In Google trends I would write "Petronas"+"Selenia"-"Petronas Towers". Is it possible also in gtrendsR?

thanks

Query data for multiple keywords

Dear all,
I have dificulty to get the exact google trends query data that I retrieved from https://www.google.com/trends with gtrendsR package for multiple search phrases.
Here is an example: I would like to get GT trends data for the search phrase "inflation rate"

head(gtrends("inflation rate", geo="US")$trend)
start end United.States
1 2004-01-04 2004-01-10 59
2 2004-01-11 2004-01-17 89
3 2004-01-18 2004-01-24 64
4 2004-01-25 2004-01-31 84
5 2004-02-01 2004-02-07 68
6 2004-02-08 2004-02-14 81

On the other hand, https://www.google.com/trends website provides totally different trends data.

when I run the following:

I get the data as follows:

What am I doing wrong?

Add sample data

Let's add a sample dataset returned by gtrends() in the data folder. This will be useful for examples documentation.

Hebrew fonts

I have difficulty retrieving Google Trends queries in Hebrew.
Any suggestion is welcome.
library(gtrendsR)
ch <- gconnect("user", "password",verbose = FALSE)
gtrends("זול", geo="IL", cat="all", ch, res = "week",start_date = as.Date("2004-01-01"),end_date =as.Date("2015-11-01"))
Error: Not enough search volume. Please change your search terms.

Message if weekly data are not available

In order to get only weekly data and discard monthly ones, I initially recognized the latter ones relying on the error message that appeared with the old versions of the package (I used TryCatch).
Now in the newest version that message seems to not appear anymore. From now on, Is there a simple way to identify when trends are not in a week resolution?

RCurl -> curl ?

When we started this, @jeroenooms had not yet written curl. Now it exists, and is probably a lot better. I just had my laptop fail mysteriously with RCurl. Works on my work and home machines so I suspect libssl choices...

keywords setup

HI,
I would like to ask some questions about the keyword setup.
Is there any (encoding) problems on keywords setup, when I type Google Trends query keywords in different languages ?

gtrends(ch,"アップル",geo="JP") => it is fine in Japanese language.

gtrends(ch,"蘋果",geo="TW") => However, it can't in Chinese.
[1] "No Trends data for 蘋果 - substituting NA series..."

But when I go to the Google Trends web, there has time series data on "蘋果" .

Thank you very much.

Query data for Honk Kong?

I have difficulty to retrieve any query data for Hong Kong.
Any advise?

library(gtrendsR)
ch <- gconnect("[email protected]", "password",verbose = FALSE)
gtrends("gold", geo="HK")
Error in read.table(file = file, header = header, sep = sep, quote = quote, :
no lines available in input

Error while execution

Hello.

Can you explain this error:
Error in read.table(file = file, header = header, sep = sep, quote = quote, :
incomplete final line found by readTableHeader on 'strsplit(vec[2], "\n")[[1]]'

The code:
library(gtrendsR)
gconnect(usr = '++++',psw = '++++')
lang_trend <- gtrends(c("data is", "data are"), res="week")

Bug when using space in keywords

Example:

res <- gtrends(c("NVivo", "atlas.ti", "maxqda", "qda miner"))

Missing trimws

Hi!
Have installed gtrendsR from CRAN into R3.1.3 (which may cause the problem), and I manage to connect via my gmail account, but when I try to run any query I've seen in the documentation, I get the following error message:

Error: could not find function "trimws"

Honk Kong is not listed on data(countries) on gtrendsR 1.3.2

I try to gather Google Trends data for Hong Kong with the latest release, gtrendsR 1.3.2, but get the following error code.

gtrends("money", geo="HK")
Error: Country code not valid. Please use 'data(countries)' to retreive valid codes.

Error in `tmp*`[[i]] : subscript out of bounds (2 ways authentication)

Hi,
Thanks for what looks to be very useful. However, I can't manage to use the gtrends() function, even with the provided examples:

gconnect("mygoogleaccount ","**_")
res <- gtrends(c("nhl", "nba", "nfl"))
Error in _tmp*[[i]] : subscript out of bounds

I'm sure this must be on my hand but I can't find the reason...

Best regards,
Julien

sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.1 (El Capitan)

locale:
[1] fr_FR.UTF-8/fr_FR.UTF-8/fr_FR.UTF-8/C/fr_FR.UTF-8/fr_FR.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] gtrendsR_1.3.0 RCurl_1.95-4.7 bitops_1.0-6

loaded via a namespace (and not attached):
[1] Rcpp_0.12.2 lattice_0.20-33 zoo_1.7-12 digest_0.6.8 MASS_7.3-45 grid_3.2.2
[7] plyr_1.8.3 gtable_0.1.2 magrittr_1.5 scales_0.3.0 ggplot2_1.0.1 stringi_1.0-1
[13] reshape2_1.4.1 googleVis_0.5.10 proto_0.3-10 RColorBrewer_1.1-2 RJSONIO_1.3-0 tools_3.2.2
[19] stringr_1.0.0 munsell_0.4.2 colorspace_1.2-6

Error in read.table

Hello!
Help me pleace
I'm facing with strange issue, when runing code:

library(tseries)
library(forecast)

if (!require("devtools")) install.packages("devtools")
devtools::install_github("PMassicotte/gtrendsR")
library(devtools)
library(gtrendsR)

usr <- "??????????????"
psw <- "??????????????"
gconnect(usr, psw) 

res <- gtrends(c("nhl", "nba", "nfl"))

Error:

Error in read.table(file = file, header = header, sep = sep, quote = quote,  : 
  incomplete final line found by readTableHeader on 'strsplit(vec[2], "\\\n")[[1]]'

Google Trends Category Search

Google trends has the option of choosing categories of search, rather than only terms. For example, I can choose to view "Shopping" category without entering any term to see the trend of all shopping related terms searched on google. How do I retrieve this series via the gtrendsR API? Furthermore, is there documentation for which number corresponds to which category on Google Trends?

Google Login problem in Rstudio Server

Hi!

I'm having trouble connecting to my Google acc on gtrendsR on my RStudio Server. I run Rstudio locally in a windows machine mirrorwing the packages on my server, and gconnect only works on my windows machine. My acc is exclusive to trends and has 2-factor off.

The following message shows on my Rstudio server:

"Google login failed! Check your login information."

But in the windows machine it runs smoothly.

I saw a similar question some time ago but I was close due to being unable to replicate the problem. I'm willing to help anyway I can.

Thanks,

Special characters in search field

Hi,
It appears that special characters inside of the search field, such as + and - (include and exclude characters) and/or characters from other alphabets are not treated by the package in the same way as by Google trends directly.
CSV files saved directly from Google Trends contain different values compared to the output of gtrendsR. (I have long search queries with pluses and minuses as well as words using characters from outside of English language, Russian, German, Norwegian and French words, for example. Numbers come out quite different. I previously used Python and was getting identical results).
Regards,
Barbara

Force error message when cant connect

Maybe force the display of error message even if verbose is set to FALSE in gconnect.

Support for UTF-8 queries (Arabic, Chinese, Japanese, Hindi, etc.)

Currently submitting search terms encoded in UTF-8 (if checking non-latinate language search terms) returns no results. This appears to be on account of a line in the gtrends.default function that converts latin2 queries to UTF-8 queries; queries already formatted in UTF-8 end up garbled.

I suggest adding a check around that line so that if the query is already UTF-8 then the conversion doesn't happen, like so:

Change encoding to utf-8

if (!(Encoding(query) == "UTF-8")) {
query <- iconv(query, "latin1", "utf-8", sub = "byte")
}

UK searches not working

data <- gtrends(query="2011", geo="UK")

results in "Error: Not enough search volume. Please change your search terms."

Other counties work fine though:
data <- gtrends(query="2011", geo="US")
data <- gtrends(query="2011", geo="CN")

So is the code behind the scenes incorrect? Should the geo code actually be "GB"? https://www.google.com/trends/explore#q=2011&geo=GB&cmpt=q&tz=Etc%2FGMT

Daily data regardless of "res" parameter set

Daily data regardless of "res" parameter set

Summary now (partially) borked

edd@max:~$ r -lgtrendsR -e'print(summary(gtrends(c("NBA","NFL"))))'             
Google Trends results for:
NA
Requested at: NA

Summary of trend:
     Index                 NBA             NFL      
 Min.   :2004-01-10   Min.   : 10.0   Min.   : 6.0  
 1st Qu.:2006-12-30   1st Qu.: 20.0   1st Qu.:11.0  
 Median :2009-12-19   Median : 30.0   Median :22.0  
 Mean   :2009-12-19   Mean   : 30.4   Mean   :27.1  
 3rd Qu.:2012-12-08   3rd Qu.: 37.0   3rd Qu.:38.0  
 Max.   :2015-11-28   Max.   :100.0   Max.   :89.0  
NULL
edd@max:~$ 

The query and date used to be written to the object so that summary() can return. Something changed...

Add support for categories

I will try to work on that this week.

Search term character limit

Hi,

I'm trying to use gtrendsR to complete multiple searches of Google Trends (GT) for two search terms in the UK.

I have a group of search terms like this:

keyword<- c("ant", "cat", "dog", "horse", "mouse", "bird")

I have written a for loop that takes a pair of the search terms, runs them through GT and then lists the output trend data in an object.

keyword<- c("ant", "cat", "dog", "horse", "mouse", "bird")

KeywordData<- NULL

for (j in c(1:5)){
DATA<- gtrends(query = c(keyword[j], keyword[j+1]), geo="GB")
DATATREND<- DATA$trend
keywordDATA<- c(keywordDATA, DATATREND)
}
This code works successfully for these short length keywords. i.e. cat only has three letters.

However when I try a search term over 13 letters long I get the following error message:

Error in names(trend) <- make.names(paste(kw, geo)) :
'names' attribute [2] must be the same length as the vector [1]

I have used the animals keyword list as an example to show the code does work when the search terms are short. However my project needs to search for terms such as "Flood Risk Management" a much longer character string.

Please could you advise on how to overcome this problem?

Many thanks.

T

plot idea

This is an idea from @eddelbuettel

plot googleVis in PDF. Hint: cat(unlist(G1$html), file="tmp.html") -----> print to PDF using system(...)

Cannot login from RStudio Server

Tried to connect from RStudio Server but login failed. Do I need to be on a local machine for correct authentication?

> library(gtrendsR)
> usr <- "[email protected]"
> psw <- "mypass"
> gconnect(usr, psw)
Google login failed! Check your login information.NULL

Can't run searches. Return: Not enough search volume. Please change your search terms.

Hello,

I've tried the package today but when I tried to run some searches I received this:

Not enough search volume. Please change your search terms.

This is my code:

retail <- gtrends(query = c("black friday", "cyber monday"))

Downloaded the dev version using:

devtools::install_github("PMassicotte/gtrendsR")

Thanks!

Authentification problem

Some users have this problem

http://pmassicotte.github.io/2015-11-17-gtrendsR/

Probably a something 2-steps authentication. Need better error message when that happen because it now says "not enough volume to ..."

Daily data

Dear Philippe, Dirk,

Thanks for the great package.

Would be great to have the option to have daily granularity in search. I know it opens quite a bit of issues with regards to scaling the search interest over time. Eric Salina has had a go at it with Python: https://github.com/ecsalina/gtrends. Not sure if that is of help? Will have a go at it myself.

Melk

GTrend CSV Download

@PMassicotte @eddelbuettel @bquast @jlewis91
Is there any way we can download Google Trend CSV file with this script??

not enough search volume

Hi. After successfully logging-in every query I try, even the sport_trend example, returns the same error message:
Error: Not enough search volume. Please change your search terms.

New Structure of Google Trends incompatible with gtrendsR

It seems that the querry through gtrends function has become broken due to new protocols instituted at Google Trends. Please check it out.

It seems that Google rolled out an addition of some

More README stuff

Thanks for editing in more content--was about to suggest that.

We can place the png file somewhere else, even on a gh-pages branch, and I don't think we need .Rmd and .md.

Have a look at some my README.md files. I tend to place the badges into a single row. Better?

We could also put tarballs into a drat repository if you see fit. Easy enough to set up.