Comments (5)
I guess you should set the instances header.
just debug the code you will find the null part of your code.
dataset is null, try to set it.
from moa.
The instance header
for every instance d
is null, but not the data ... I did not mention that in my question because I did not think of it as the cause of the problem. I used the same data generator, and the same code for StreamKM, and there wasn't any problem with that.
This code works:
import com.yahoo.labs.samoa.instances.DenseInstance;
import moa.cluster.Clustering;
import moa.clusterers.streamkm.StreamKM;
public class TestingStreamKM {
static DenseInstance randomInstance(int size) {
DenseInstance instance = new DenseInstance(size);
for (int idx = 0; idx < size; idx++) {
instance.setValue(idx, Math.random());
}
return instance;
}
public static void main(String[] args) {
StreamKM streamKM = new StreamKM();
streamKM.numClustersOption.setValue(5); // default setting
streamKM. resetLearningImpl();
for (int i = 0; i < 1000; i++) {
DenseInstance d = randomInstance(2);
streamKM.trainOnInstanceImpl(d);
}
Clustering result = streamKM.getClusteringResult();
}
}
Now, if the null instance header is the problem, where should I set it? It must the same for the whole dataset ...
Thanks for answering so fast!
from moa.
I have updated the code.
It is working as I mentioned, you have to assign header to your instance.
here is stackoverflow link https://stackoverflow.com/questions/58869442/java-lang-nullpointerexception-when-trying-moa-stream-clustering-algorithm-denst/58910104#58910104
here is the updated code:
static DenseInstance randomInstance(int size) {
// generates the name of the features which is called as InstanceHeader
ArrayList<Attribute> attributes = new ArrayList<Attribute>();
for (int i = 0; i < size; i++) {
attributes.add(new Attribute("feature_" + i));
}
// create instance header with generated feature name
InstancesHeader streamHeader = new InstancesHeader(
new Instances("Mustafa Çelik Instance",attributes, size));
// generates random data
double[] data = new double[2];
Random random = new Random();
for (int i = 0; i < 2; i++) {
data[i] = random.nextDouble();
}
// creates an instance and assigns the data
DenseInstance inst = new DenseInstance(1.0, data);
// assigns the instanceHeader(feature name)
inst.setDataset(streamHeader);
return inst;
}
public static void main(String[] args) {
WithDBSCAN withDBSCAN = new WithDBSCAN();
withDBSCAN.resetLearningImpl();
withDBSCAN.initialDBScan();
for (int i = 0; i < 1500; i++) {
DenseInstance d = randomInstance(5);
withDBSCAN.trainOnInstanceImpl(d);
}
Clustering clusteringResult = withDBSCAN.getClusteringResult();
Clustering microClusteringResult = withDBSCAN.getMicroClusteringResult();
System.out.println(clusteringResult);
}
here is the screenshot of debug process, as you see the clustering result is:
from moa.
The
instance header
for every instanced
is null, but not the data ... I did not mention that in my question because I did not think of it as the cause of the problem. I used the same data generator, and the same code for StreamKM, and there wasn't any problem with that.This code works:
import com.yahoo.labs.samoa.instances.DenseInstance; import moa.cluster.Clustering; import moa.clusterers.streamkm.StreamKM; public class TestingStreamKM { static DenseInstance randomInstance(int size) { DenseInstance instance = new DenseInstance(size); for (int idx = 0; idx < size; idx++) { instance.setValue(idx, Math.random()); } return instance; } public static void main(String[] args) { StreamKM streamKM = new StreamKM(); streamKM.numClustersOption.setValue(5); // default setting streamKM. resetLearningImpl(); for (int i = 0; i < 1000; i++) { DenseInstance d = randomInstance(2); streamKM.trainOnInstanceImpl(d); } Clustering result = streamKM.getClusteringResult(); } }Now, if the null instance header is the problem, where should I set it? It must the same for the whole dataset ...
Thanks for answering so fast!
Algorithms have different abilities, and differs from some points. Streamkm algorithm can work without assigning header. WithDBSCAN needs the headers, you must assign them. The have different data structures. They may inherit from same classes, but works differently.
Debug your code and try to fill the null parameters. It is a good way to find the gaps.
from moa.
Hey, thanks! You really helped me out here :)
I have a few questions left to ask:
- why is the macro clusters
weight
field = 0 in the debugger?
If you open the nested micro clusters for a given macro cluster, you will see they have a weight defined.
- Furthermore, micro clusters have a null center and radius (which is weird because the
MicroCluster
class extendsCFCluster
which extendsSphereCluster
). I am able to get those values using the getter methods, but that called my attention.
- Check also the N value for a macro cluster is not the sum of the micro clusters it has inside N values ...
It seems something strange is happening ...
Thanks again for the support.
from moa.
Related Issues (20)
- How do you use Den-Stream in your project and replace the data source with Kafka? HOT 1
- classIndex() takes long time with AdaptiveRandomForest HOT 12
- Time measurement for AdaptiveRandomForest in parallel mode HOT 2
- build fails on AArch64, Fedora 3 HOT 1
- Issues about Multi label Hoeffding Tree
- Cannot find specific algorithms which are on Github but not on application HOT 2
- Date attribute error
- IllegalAccessError when trying to browse for an ARFF file in ArffFileStream
- MOA running error
- moa.classifiers.meta.imbalanced Tests are failing HOT 2
- data with noise class
- Data with noise class
- MOAObject measureByteSize() could return negative value if the size is > 2^31-1 HOT 1
- measureByteSize() gets called twice in EvaluatePrequential for every stats collection cycle
- AdaHoeffdingOptionTreeTest is failing on latest master (6eacf9b) HOT 1
- MOA devel docker image fails to build with Java 8 HOT 1
- How can I obtain the final clustering results (the labels) using Clustream algorihm?
- When I run massive online analysis, I get an error "Failed to load implementation from: com.github.fommil.netlib.NativeSystemARPACK".
- Method removeBadSplitNodes in FIMTDDNumericAttributeClassObserver cannot remove nodes
- Some MOA tasks fail when using filters because they (tasks) use InstanceStream instead of ExampleStream.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from moa.