Comments (55)
I don't get this issue. Can you run
clear all
sysuse auto , clear
cap prog drop pargegen
prog define pargegen, rclass
version 13
syntax varlist [if]
marksample touse
disp "`level', $_level"
gegen test = sum(price)
disp "`level', $_level"
reg `varlist' if `touse'
drop test
end
pargegen price weight foreign rep78
I don't think gtools ever sets a local called "level" but I could be wrong.
from stata-gtools.
disp
gives no output, just the usual error message.
I don't know if this is helpful, but I noted the strange behavior that after a run with gegen
, running the test program with egen
again returned the same error message. Only after restarting Stata the egen version runs fine again.
Even more strange, on Win 10 the program runs fine ?!
from stata-gtools.
As I noted, I couldn't reproduce the issue, which is why I was trying to figure out if level
really was defined by gegen. Ah, ok, I think this, perhaps:
clear all
log using test, text
sysuse auto , clear
cap prog drop pargegen
prog define pargegen, rclass
version 13
syntax varlist [if]
marksample touse
disp "`level', $_level"
cap noi gegen test = sum(price)
disp "`level', $_level"
exit `=_rc'
reg `varlist' if `touse'
drop test
end
pargegen price weight foreign rep78
log close _all
And send the log file? You say you are using parallel
, right? Is the first snippet you posted the exact code you were running?
from stata-gtools.
What is the OS where it gave the error? OSX? Win7?
from stata-gtools.
running on arch linux. This is the log file:
test.log
from stata-gtools.
I'm on arch as well, so I'm not sure what's failing. gegen
definitely doesn't do anything to level, which is what I thought, but that was your original error...
Can you run
version
which gegen
which gtools
?
from stata-gtools.
. version
version 15.1
. which gegen
/home/pp/ado/plus/g/gegen.ado
*! version 0.13.1 02May2018 Mauricio Caceres Bravo, [email protected]
*! implementation -egen- using C for faster processing
. which gtools
/home/pp/ado/plus/g/gtools.ado
*! version 0.13.1 02May2018 Mauricio Caceres Bravo, [email protected]
*! Program for managing the gtools package installation
The starting point of this issue was gvegayon/parallel#66
from stata-gtools.
Thanks! I'll look over that issue thread and see if there's any clues there. I upgraded to parallel 0.19 and still no issue. Will keep you posted.
from stata-gtools.
I've tried it on 2 other Linux systems w/o error. It's really strange to me because we are using the same OS and gtools version anyway... Can you run:
clear all
log using test, text
which reg
which regress
which _get_diopts
sysuse auto , clear
cap prog drop pargegen
prog define pargegen, rclass
version 13
syntax varlist [if]
marksample touse
egen test = sum(price)
reg `varlist' if `touse'
gegen test = sum(price), replace
reg `varlist' if `touse'
drop test
end
pargegen price weight foreign rep78
log close _all
Sorry I'm basically going blind here, since I don't get the same issue you do. Hopefully we can find the root cause of the problem, though.
from stata-gtools.
Thanks for all your efforts. I noticed that I can reproduce the error even with this code:
clear all
log using test, text
which reg
which regress
which _get_diopts
sysuse auto , clear
egen test = mean(price)
reg price weight
gegen test2 = mean(price)
reg price weight
log close _all
log: test.log
Removing all ados in PLUS and reinstalling only gtools didn't help either. Might want to try reinstalling Stata itself?
from stata-gtools.
Does this happen with other gtools commands or is it just gegen
? So if you run gunique price
or gtop price
does it also give an error with reg? (Note: Maybe try regress
and _regress
instead of reg
?)
from stata-gtools.
gtop
and gunique
produce the same error, also with regress
or _regress
.
from stata-gtools.
Can I have the log for
findfile gtools_unix.plugin
!ldd `r(fn)'
clear all
sysuse auto
global GTOOLS_CALLER gegen
_gtools_internal price, gfunction(hash) gen(test) debug(9) v bench(3)
_regress price weight
? I expect you will get the same error. Hopefully the log has a clue...
from stata-gtools.
here is the log: test.log
from stata-gtools.
log for
clear all
sysuse auto
global GTOOLS_CALLER gegen
_gtools_internal price, gfunction(hash) gen(test)
set trace on
_get_diopts diopts options,
?
from stata-gtools.
from stata-gtools.
This is very strange; there is nothing in the source file that suggests where level gets assigned to its default value of 95. It should be blank throughout... Could you edit _get_diopts.ado?
findfile _get_diopts.ado
!cp `r(fn)' `r(fn)'.bak
Could you add
disp "`level'"
to GetDiopts, before and after the first syntax
statement (for me about line 81). Also after the last opts_exclusive
(for me around line 96) as well as before and after the second syntax
statement (for me around line 157, right before the level
error.
Then re-run the latest snippet. Thank you for all your help debugging.
from stata-gtools.
log: test.log
from stata-gtools.
Ok, I think this at least gives a clue, since level
is getting assigned to its default value and that turns out to be "95,0" for some bizarre reason... Although you said you you get the same error when you pass level. Ok, could you run:
clear all
sysuse auto
regress price weight, level(90)
gegen test = mean(price)
creturn list
disp "`c(level)', `c(clevel)'"
set trace on
_get_diopts diopts options, level(89)
(keep the modifications to _get_diopts.ado
).
from stata-gtools.
log: test.log
from stata-gtools.
Oh, wow, so it's not setting the default. It's actually appending ",0" to whatever you pass. That is exceedingly bizarre. Can you try:
capture program drop test
program test
syntax, [level(cilevel)]
disp "`level'"
end
clear all
set obs 10
gen x = 1
test
gegen y = sum(x)
test
from stata-gtools.
Yeah, bizarre is probably the right description. Here the output:
. set obs 10
number of observations (_N) was 0, now 10
. gen x = 1
. test
95
. gegen y = sum(x)
. test
95,0
from stata-gtools.
Can you try that with the develop branch? net uninstall gtools
; then from a fresh Stata session can you run:
local github "https://raw.githubusercontent.com"
net install gtools, from(`github'/mcaceresb/stata-gtools/develop/build/)
clear all
capture program drop test
program test
disp "`0'"
syntax, [foo(cilevel)]
disp "`foo'"
end
set obs 10
gen x = 1
test
gegen y = sum(x)
test
Can you also try
clear all
set obs 10
gen x = 1
syntax, [foo(cilevel)]
disp "`foo'"
gegen y = sum(x)
syntax, [foo(cilevel)]
disp "`foo'"
from stata-gtools.
same result after installing dev branch:
. gen x = 1
. test
95
. gegen y = sum(x)
. test
95,0
and for the second program
. gen x = 1
. syntax, [foo(cilevel)]
. disp "`foo'"
95
. gegen y = sum(x)
. syntax, [foo(cilevel)]
. disp "`foo'"
95,0
from stata-gtools.
By the way, I was unsure if some local configurations caused the error. Therefore, I installed Stata MP 15.1 + gtools in a fresh Ubuntu 18.04 virtualbox environment. Too bad, I was able to reproduce the same error.
from stata-gtools.
Tried fresh 18.04 in virtualbox as well and no error for me. I have no clue what's up! I'm really sorry. Un_n
- Ubuntu 18.04
- Virtualbox version 5.2.12 r122571
- Stata 15.1 SE
. version
version 15.1
. !uname -a
Linux mauricio-vb 4.15.0-20-generic #21-Ubuntu SMP Tue Apr 24 06:16:15 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
. clear all
. set obs 10
number of observations (_N) was 0, now 10
. gen x = 1
. syntax, [foo(cilevel)]
. disp "`foo'"
95
. gegen y = sum(x)
. syntax, [foo(cilevel)]
. disp "`foo'"
95
from stata-gtools.
Can you try again? Gtools 1.0 just hit the develop branch. While it doesn't address this specifically, the plugin has changed enough that I'm hoping whatever internals were messing it up are not there anymore (on Stata 15 it should use a newer version of the plugin interface, for example; make sure you have version 1.0.0
by running which gtools
).
from stata-gtools.
same same ...
. which gegen
/home/pp/ado/plus/g/gegen.ado
*! version 1.0.0 21Jul2018 Mauricio Caceres Bravo, [email protected]
*! implementation -egen- using C for faster processing
. clear all
. set obs 10
number of observations (_N) was 0, now 10
. gen x = 1
. syntax, [foo(cilevel)]
. disp "`foo'"
95
. gegen y = sum(x)
. syntax, [foo(cilevel)]
. disp "`foo'"
95,0
from stata-gtools.
Btw is there a workaround or is this issue persistent? For example,
clear all
set obs 10
gen x = 1
syntax, [foo(cilevel)]
disp "`foo'"
gegen y = sum(x)
syntax, [foo(cilevel)]
syntax, [foo(cilevel)]
disp "`foo'"
or
clear all
program gtools_check, plugin using("gtools_unix_v3.plugin")
set obs 10
gen x = 1
syntax, [foo(cilevel)]
disp "`foo'"
gegen y = sum(x)
plugin call gtools_check, check
syntax, [foo(cilevel)]
disp "`foo'"
or
clear all
set obs 10
gen x = 1
syntax, [foo(cilevel)]
disp "`foo'"
gegen y = sum(x)
global GTOOLS_CALLER ghash
_gtools_internal
syntax, [foo(cilevel)]
disp "`foo'"
from stata-gtools.
sry, none of the above examples work. The second disp
always shows 95,0
.
Maybe I should cross-prost the issue at statalist.org?
from stata-gtools.
Well, it would make sense to crosspost if this wasn't specific to gtools, but regular egen, etc. calls don't have this issue, right? Can you download test1.plugin, test2.plugin, and spookyhash.dll from here. Then try to run:
clear all
program test1, plugin using(test1.plugin)
plugin call test1
syntax, [foo(cilevel)]
disp "`foo'"
program test2, plugin using(test2.plugin)
plugin call test2
syntax, [foo(cilevel)]
disp "`foo'"
global GTOOLS_CALLER ghash
_gtools_internal
syntax, [foo(cilevel)]
disp "`foo'"
Does the "95,0" show in after all 3 or just the last one?
from stata-gtools.
I have tried running the test plugins, but get the following error:
. which gtools
/home/pp/ado/plus/g/gtools.ado
*! version 1.0.2 08Aug2018 Mauricio Caceres Bravo, [email protected]
*! Program for managing the gtools package installation
. !ls -l
insgesamt 296
-rw-r--r-- 1 pp users 382 14. Aug 07:10 Makefile
-rw-r--r-- 1 pp users 7176 14. Aug 07:10 spookyhash_api.h
-rw-r--r-- 1 pp users 18944 14. Aug 07:10 spookyhash.dll
-rw-r--r-- 1 pp users 199 14. Aug 07:10 stplugin.c
-rw-r--r-- 1 pp users 5701 14. Aug 07:10 stplugin.h
-rw-r--r-- 1 pp users 433 14. Aug 07:10 test1.c
-rw-r--r-- 1 pp users 119132 14. Aug 07:10 test1.plugin
-rw-r--r-- 1 pp users 765 14. Aug 07:10 test2.c
-rw-r--r-- 1 pp users 120343 14. Aug 07:10 test2.plugin
-rw-r--r-- 1 pp users 123 14. Aug 07:10 test.do
. do "/tmp/SD10587.000000"
. clear all
. program test1, plugin using(test1.plugin)
./test1.plugin: Ungültiger ELF-Header
Could not load plugin: ./test1.plugin
r(9999);
end of do-file
r(9999);
from stata-gtools.
Ok, that one is on me; really sorry. I had compiled that for Windows and I forgot you're also on Arch. I re-posted the files as a ZIP here. Can you try again? You can just do do test.do
for the test script on that folder.
from stata-gtools.
this is the output:
. do test.do
. clear all
. program test1, plugin using(test1.plugin)
. plugin call test1
Hello World
. syntax, [foo(cilevel)]
. disp "`foo'"
95
.
. program test2, plugin using(test2.plugin)
. plugin call test2
foo: 0.00
hash: 12485269540145512313, 5018404763475985066
. syntax, [foo(cilevel)]
. disp "`foo'"
95
.
. set obs 1
number of observations (_N) was 0, now 1
. global GTOOLS_CALLER ghash
. _gtools_internal
. syntax, [foo(cilevel)]
. disp "`foo'"
95,0
from stata-gtools.
I was really hoping it was the plugin interface in general, but it seems to be something in gtools.
Can you add
syntax, [foo(cilevel)]
disp "`foo'"
right before and right after line 2157 of _gtools_internal.ado (it would be in your adopath, for me ~/ado/plus/_/_gtools_internal.ado
) and re-run only the third chunk of the snippet with set trace on
then post the log?
The idea is to know whether it's something in the Sata code or the C code.
from stata-gtools.
here is the log file: log.log
from stata-gtools.
I've come across a similar (the same?) issue while using ppmlhdfe and gegen in the same do-file: using gegen before ppmlhdfe causes ppmlhdfe to fail with "invalid syntax" while using egen causes no such problem. This issue only occurs when using Stata on our linux cluster; the code with gegen worked without issues on my MacOS X laptop. (This causes no problems for me as I can simply use egen, but I thought it might be helpful to know that the issue might be Linux-specific. Although of course many other variables also differ between the cluster and my laptop...)
from stata-gtools.
If you run ppmlhdfe with the verbose(1) option, or if you run "set trace" beforehand, where in the program is it reporting an error?
Ppmlhdfe doesn't use C plugins, so on principle it should be like any other programs
from stata-gtools.
Using verbose(1) does not change the output (it still just says "invalid syntax"). The output from set trace on is attached; I believe it fails in _get_diopts. (I don't think this is an issue with ppmlhdfe; rather I think ppmlhdfe in my example fails in the same way as reg fails above. The error is presumably in gegen, or even more low-level)
from stata-gtools.
For whatever reason it seems that there are corner cases where gegen causes the default cilevel to be set to 95,0
instead of just 95
, leading to syntax errors.
It is very puzzling to me because I never set ci level in any gtools programs and I cannot reproduce the error (be it in my Linux laptop or the Linux servers I use).
from stata-gtools.
@ppoppitz @lmusolff If you still get this bug, can you try the develop branch? (gtools version 1.3.2; gtools, upgrade branch(develop)
)
I just got a report that I suspect is related (#53). It occurs to me that maybe if I save the state of level before running the plugin and restore it after then maybe I can bypass the problem.
from stata-gtools.
sry, I still get the same errors with version 1.3.2 14Feb2019
from stata-gtools.
@ppoppitz Can you try gtools version 1.3.3? gtools, upgrade branch(develop)
; I removed a dependency that might have been causing the problem. In #53 it looks like gegen
no longer sets level
to 95,0
; hope it's the same for you.
from stata-gtools.
I verified the code from #53 gives me the same error as there with gtools 1.1.2, but when I try to upgrade I get the following error:
. gtools, upgrade branch(develop)
checking gtools consistency and verifying not already installed...
could not copy https://raw.githubusercontent.com/mcaceresb/stata-gtools/develop/build/fasterxtile.sthlp
(no action taken)
--Break--
r(1);
from stata-gtools.
Try net uninstall gtools
and then
local github "https://raw.githubusercontent.com"
net install gtools, from(`github'/mcaceresb/stata-gtools/develop/build/)
If you get the same error, what happens if you run
do https://raw.githubusercontent.com/mcaceresb/stata-gtools/develop/build/fasterxtile.sthl
You should see
. .h gquantiles
invalid syntax
r(198);
Let me know if you don't.
from stata-gtools.
Here's the output from those commands:
. net uninstall gtools
package not found
r(111);
. local github "https://raw.githubusercontent.com"
. net install gtools, from(`github'/mcaceresb/stata-gtools/develop/build/)
checking gtools consistency and verifying not already installed...
could not copy https://raw.githubusercontent.com/mcaceresb/stata-gtools/develop/build/fasterxtile.sthlp
(no action taken)
--Break--
r(1);
. do https://raw.githubusercontent.com/mcaceresb/stata-gtools/develop/build/fasterxtile.sthl
file https://raw.githubusercontent.com/mcaceresb/stata-gtools/develop/build/fasterxtile.sthl not found
r(601);
from stata-gtools.
That was a typo on my part, I meant do https://raw.githubusercontent.com/mcaceresb/stata-gtools/develop/build/fasterxtile.sthlp
In any case, maybe try this:
copy https://raw.githubusercontent.com/mcaceresb/stata-gtools/develop/src/github-issues/40/gtools-1.3.3.zip gtools.zip
unzipfile gtools.zip
net install gtools, from(`c(pwd)'/gtools/) replace
from stata-gtools.
The former gives me r(1), but the second successfully upgraded to "version 1.3.3 15Feb2019". And now I can run the code from #53 without an error :) (I'll try my own at some point but I doubt it would differ, and it is part of a big job that I can't really isolate so need to schedule a run for)
Thanks for looking into this!
from stata-gtools.
I think I'll ask Stata why that file can't be copied on some systems. You're not the first person to have this issue. Can you run
copy https://raw.githubusercontent.com/mcaceresb/stata-gtools/develop/build/fasterxtile.sthlp fasterxtile.sthlp
and let me know the error code, if any?
from stata-gtools.
Yes, I vaguely remember having trouble installing it the normal way back when I did because I think of the same error. From that command I get:
. copy https://raw.githubusercontent.com/mcaceresb/stata-gtools/develop/build/fasterxtile.sthlp fasterxtile.sthlp
--Break--
r(1);
EDIT: I just ran the same command on my MacOsX Laptop and it gives me the same r(1). But maybe this should be a different issue; let me know if you'd like me to open one.
from stata-gtools.
Can you re-try? I want to know if this had anything to do with the fact it's a tiny file (14 bytes). I just modified the file to be a copy of gquantiles.sthlp
(what's your Stata version, by the way?)
from stata-gtools.
Oh wow, apparently that did it: the copy command executes now without problem (both on my laptop and the server). My Stata version is 13.1.
from stata-gtools.
Ok, great. I'm just going to have my build file copy gquantiles.sthlp
into fasterxtile.sthlp
and bypass this altogether. Thanks for testing it out!
from stata-gtools.
Of course, no worries! Thanks for writing & maintaining this awesome package!
from stata-gtools.
Awesome, it's working here too. Thanks again for your efforts!
from stata-gtools.
Related Issues (20)
- gegen total vs. egen total HOT 6
- Could not load gtools_macosx_v3.plugin, error 9999 HOT 9
- gegen normalize does not realize that a new variable shall be created HOT 1
- gunique missing scalars when there are no observations
- gtools version of merge HOT 4
- gtools not installing on macos Stata 16 HOT 3
- Problem with -if- condition in several commands HOT 1
- Please update the benchmark using Stata 17. HOT 5
- gtools 1.8.1 not working *at all* with Stata MP 16.1 on MacOS 11.6 HOT 7
- Plugin download error when using "ssc install gtools" HOT 2
- the option cw in gcollapse is invalid. HOT 2
- Error r(111) in Stata MP 16.1 and SE 17.0, macOS Monterey HOT 4
- OSX plugin fails; move OSX Compilation to github
- gegen max does not properly evaluate string expressions HOT 2
- Error trying to copy gtop.sthlp in Stata 14 HOT 3
- Could not load gtools_macosx_v3.plugin, error 9999 HOT 17
- Some commands appear to ignore [w=weights] HOT 3
- Export results to word or excel HOT 4
- Wrong number of groups HOT 1
- Will greshape support strL variabes in the future? HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from stata-gtools.