Comments (7)
Getting the error: The JPMML-SkLearn conversion application has failed. The Java process should have printed more information about the failure into its standard output and/or error streams' while trying to convert a regression pipeline into a PMML format.
The above is a "wrapper" Python exception. What was the original Java exception that was thrown?
[([column], [CategoricalDomain(),LabelEncoder(),LabelBinarizer()]) for column in ['MARITAL_STATUS_CD_MAL']]
The LabelEncoder
transformation is not necessary here. Simply do [CategoricalColumn(), LabelBinarizer()]
.
from sklearn2pmml.
Hi thanks for your help ! So I am running the LabelEncoder() first because the categorical inputs are non numeric so to convert them to numeric I am running the LableEncoder first as the Label Binarizer does not take non numeric categorical inputs.
Also below is the complete error that I am getting:
RuntimeError Traceback (most recent call last)
<ipython-input-5-745dd10d7243> in <module>()
125
126 from sklearn2pmml import sklearn2pmml
--> 127 sklearn2pmml(Elastic_net_1, out_PMML + 'Elastic_Net_1_xx.pmml', with_repr = True)
/opt/wakari/anaconda/lib/python2.7/site-packages/sklearn2pmml/__init__.pyc in sklearn2pmml(pipeline, pmml, user_classpath, with_repr, debug)
140 subprocess.check_call(cmd)
141 except CalledProcessError:
--> 142 raise RuntimeError("The JPMML-SkLearn conversion application has failed. The Java process should have printed more information about the failure into its standard output and/or error streams")
143 finally:
144 if(debug):
RuntimeError: The JPMML-SkLearn conversion application has failed. The Java process should have printed more information about the failure into its standard output and/or error streams
Any help/suggestions on this would be greatly appreciated. Also the error goes away if I just run LabelBinarizer() ( after transforming the text categorical variables as numbers using pandas)
from sklearn2pmml.
I am running the LableEncoder first as the LabelBinarizer does not take non numeric categorical inputs.
The LabelBinarizer
transformation accepts non-numeric inputs. For example, it can be directly applied to string columns.
Also below is the complete error that I am getting:
This is a front-end Python exception. I need to see the back-end Java exception.
You appear to be working in the IPython/Jupyter Notebook environment. Please check its log file for this Java exception.
Alternatively, please provide a reproducible example using some publicly available dataset. For example, here's are my integration test cases (based on the Audit dataset), where the LabelBinarizer
transformation is directly applied to string columns: https://github.com/jpmml/jpmml-sklearn/blob/master/src/test/resources/main.py#L121-L152
from sklearn2pmml.
Hi !
So the reason for running the Label Encoder and Binarizer both is that I am using a elastic net regression model which needs numeric inputs to run the regression so just a Binarizer wont do although you are absolutely right that a binarizer can take non numeric inputs.
Further below is the complete error that I am getting in Java. Also its interesting to see that I am getting this error in the sklearn2pmml(.....) step and not in the Elastic_net_1.fit() step ie the mapper function that i created is compatible with the sklearn library functions but not working correctly in the sklearn2pmml library.
Also I checked the output after running label encoder and label binarizer on the categorical variable that I am using and I am getting an array (with no columns = no unique values in the variable)
array([[0, 1, 0, 0, 0, 0],
...,
[0, 0, 1, 0, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0]])
which I suppose means that both label binarizer and label encoder functions are running just fine.
Error -
org.jpmml.sklearn.Main run
SEVERE: Failed to convert
java.lang.IllegalArgumentException: expected one element but was: <class java.lang.Double, class java.lang.String>
at com.google.common.collect.Iterators.getOnlyElement(Iterators.java:322)
at com.google.common.collect.Iterables.getOnlyElement(Iterables.java:294)
at sklearn.TypeUtil.getDataType(TypeUtil.java:49)
at sklearn.preprocessing.LabelEncoder.getDataType(LabelEncoder.java:59)
at sklearn_pandas.DataFrameMapper.encodeFeatures(DataFrameMapper.java:76)
at sklearn2pmml.PMMLPipeline.encodePMML(PMMLPipeline.java:110)
at org.jpmml.sklearn.Main.run(Main.java:146)
at org.jpmml.sklearn.Main.main(Main.java:93)
Exception in thread "main" java.lang.IllegalArgumentException: expected one element but was: <class java.lang.Double, class java.lang.String>
at com.google.common.collect.Iterators.getOnlyElement(Iterators.java:322)
at com.google.common.collect.Iterables.getOnlyElement(Iterables.java:294)
at sklearn.TypeUtil.getDataType(TypeUtil.java:49)
at sklearn.preprocessing.LabelEncoder.getDataType(LabelEncoder.java:59)
at sklearn_pandas.DataFrameMapper.encodeFeatures(DataFrameMapper.java:76)
at sklearn2pmml.PMMLPipeline.encodePMML(PMMLPipeline.java:110)
at org.jpmml.sklearn.Main.run(Main.java:146)
at org.jpmml.sklearn.Main.main(Main.java:93)
Let me know your thoughts and thanks again for all your help :)
from sklearn2pmml.
java.lang.IllegalArgumentException: expected one element but was: <class java.lang.Double, class java.lang.String>
That's the Java exception that I was waiting for!
The converter assumes that all the values in a column are of the same data type. However, there is a column in your dataset that contains strings mixed with float64s (aka doubles).
You can solve this issue by explicitly casting all values to the same data type. For example:
df["x1"] = df["x1"].astype(float64)
Sure, the converter could be a little bit smarter, and assume that if the column contains mixed data type values, one of which is string, then the other data type should "win".
Reopening this issue to implement this idea.
from sklearn2pmml.
Hey !
Firstly thanks for looking into this and sorry for the slow response was tied up somewhere else. So I looked into what you are saying and went through the various levels of the variable that was giving the error to see if there were any multiple data types in it. I found the following types
array(['MA', 'DI', 'SE', 'NM', 'WI', nan], dtype=object)
I suppose its not treating the nan as string while its treating the others as string. So as you suggested I forced converted the variable to string by using:
df["x1"] = df["x1"].astype(string)
After this I got the following distinct levels in the variable (The 'nan' got converted to string)
array(['MA', 'DI', 'SE', 'NM', 'WI', 'nan'], dtype=object)
After this I tried running the sklearn2pmml function but I am still getting the same error. During this run there are 7 variables/columns in total out of which 6 are float 64 (which are not giving a error) and 1 categorical (the string variable ) variable I described above which is still giving the same error when i run LabelEncoder().
Would appreciate your response !
Thanks
Dhruv
from sklearn2pmml.
After this I tried running the sklearn2pmml function but I am still getting the same error.
Please double-check, there must be some other mixed-data type column in your dataset.
Also, are you using the latest sklearn2pmml
package version? This should print 0.20.3
:
import sklearn2pmml
print(sklearn2pmml.__version__)
Regarding "the same error" - are you still using [LabelEncoder(), LabelBinarizer()]
, or did you switch to [LabelBinarizer()]
?
As a last resort, you could save your problematic pipeline/model object in Pickle data format, and attach it here/send to my e-mail for deeper analysis:
from sklearn.externals import joblib
joblib.dump(model, "model.pkl.z", compress = 9)
from sklearn2pmml.
Related Issues (20)
- Storing LinearSVC as coefficients instead of support vectors HOT 1
- Reshape transformation results from 2-D to 1-D (column vectors) HOT 1
- sklearn2pmml does not work with sklearn version 1.3.0 and newer HOT 6
- sklearn2pmml does not work with xgboost >= 2.0.0 HOT 9
- LookupTransformer TypeError when default value is not exactly the type from the mapping HOT 2
- xgb 生成的pkl 转pmml 失败 HOT 1
- Can sklearn2pmml just for variable transformations be used? HOT 1
- PyPMML is making systematically off predictions with XGBoost PMML documents? HOT 16
- ImportError: cannot import name 'LabelBinarizer' from 'sklearn2pmml.preprocessing' HOT 2
- Compatibility with Scikit-Learn 1.4.0 HOT 8
- Convenience API for embedding model evaluation metrics as `ModelExplanation` element HOT 13
- Ability to "refine" (categorical-) valid value spaces along the transformer pipeline HOT 17
- NullPointerException when using OneHotEncoder with Integer Categorical Variables in sklearn2pmml HOT 10
- Create .pmml File for Calibrated xgboost Model HOT 8
- Pandas 2.2 FutureWarning for time series offset aliases HOT 2
- Pandas not installed as dependency HOT 2
- CategoricalDomain decorator doesn't work with nullable pandas Int64 column HOT 3
- RegEx expressions are un-evaluatable due to missing `(pc)re` imports HOT 8
- ExplainableBoostingRegressor not supporting other link functions HOT 1
- How to properly re-use raw features HOT 23
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sklearn2pmml.