GithubHelp home page GithubHelp logo

apache / accumulo-examples Goto Github PK

View Code? Open in Web Editor NEW
34.0 34.0 40.0 551 KB

Apache Accumulo Examples

Home Page: https://accumulo.apache.org

License: Apache License 2.0

Shell 1.60% Java 98.40%
accumulo big-data hacktoberfest

accumulo-examples's People

Contributors

ctubbsii avatar dependabot[bot] avatar dlmarion avatar domgarguilo avatar elinaawise avatar jmark99 avatar jzgithub1 avatar keith-turner avatar kevinrr888 avatar lbschanno avatar manno15 avatar mikewalch avatar milleruntime avatar mjwall avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

mikewalch keith-turner milleruntime idea99 lhfei yafengzhao scially dalavancloud elinaawise plainolneesh jzeiberg charygao pbr2019b eisber mmmika colincody726 jmark99 p1g17 neotim domgarguilo albertwhitlock manno15 sjyang18 foster33 christine-314 chaseknowlden isabella232 ctubbsii dlmarion edcoleman qpc-github kevinrr888 arbaazkhan1 rsingh433 ddanielr

accumulo-examples's Issues

Add a delete key/value pair example

While going through accumulo-examples, I wanted to understand what happens when a key/value pair has been deleted. The steps in this example walk through seeing the delete flag, compaction, flush, and viewing the compacted file to ensure the deleted pair was removed.

Remove examples.conf configuration

Examples are being transitioned to use conf/accumulo-client.properties. After all examples are transitioned, the examples.conf file should be removed and any docs referencing it should also be removed.

Expand instructions for use of UniqueColumns example.

The uniqueColumns example contains very little guidance in the documentation on how to run the example and what is required for the example to run successfully. Create detailed instructions that clarify how to use the example.

Update Table Property Examples

There are a few examples that take similar actions and then set a specific table property. These examples could be combined or at least use the same Java code (which uses API instead of shell commands) then set different table properties.

  • bloom
  • compactionStrategy
  • maxmutation

Add debug flag to runex

Would be nice to add a flag to the ./bin/runex script that will use the -X maven option instead of -q

SLF4J warning blocks output on all examples

When trying to run any of the ./bin/runex * examples, the following warning appears:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
SLF4J: Failed to load class "org.slf4j.impl.StaticMDCBinder".
SLF4J: Defaulting to no-operation MDCAdapter implementation.
SLF4J: See http://www.slf4j.org/codes.html#no_static_mdc_binder for further details.

I think this also blocks any output from the examples.

reservations example fails

reserve room06 20140101 alice bob eve mallory trent

2019-04-18 10:23:09,006 [examples.reservations.ARS] WARN : Could not write to the ConsoleReader.
java.lang.IllegalStateException: This client was closed.
	at org.apache.accumulo.core.clientImpl.ClientContext.ensureOpen(ClientContext.java:114)
	at org.apache.accumulo.core.clientImpl.ClientContext.createConditionalWriter(ClientContext.java:539)
	at org.apache.accumulo.examples.reservations.ARS.reserve(ARS.java:97)
	at org.apache.accumulo.examples.reservations.ARS$1.run(ARS.java:276)
	at java.lang.Thread.run(Thread.java:748)

ClassNotFoundException on several examples

When running the dirlist, filedata and possibly other examples, the following error occurs in the monitor.

java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.accumulo.examples.filedata.ChunkCombiner
	at org.apache.accumulo.core.conf.IterConfigUtil.loadIterators(IterConfigUtil.java:226)
	at org.apache.accumulo.core.conf.IterConfigUtil.convertItersAndLoad(IterConfigUtil.java:188)
	at org.apache.accumulo.server.compaction.FileCompactor.compactLocalityGroup(FileCompactor.java:362)
	at org.apache.accumulo.server.compaction.FileCompactor.call(FileCompactor.java:230)
	at org.apache.accumulo.tserver.tablet.MinorCompactor.call(MinorCompactor.java:95)
	at org.apache.accumulo.tserver.tablet.Tablet.minorCompact(Tablet.java:813)
	at org.apache.accumulo.tserver.tablet.MinorCompactionTask.run(MinorCompactionTask.java:110)
	at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.ClassNotFoundException: org.apache.accumulo.examples.filedata.ChunkCombiner
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at org.apache.accumulo.start.classloader.AccumuloClassLoader$1.loadClass(AccumuloClassLoader.java:213)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at org.apache.accumulo.core.classloader.ClassLoaderUtil.loadClass(ClassLoaderUtil.java:85)
	at org.apache.accumulo.core.conf.IterConfigUtil.loadClass(IterConfigUtil.java:236)
	at org.apache.accumulo.core.conf.IterConfigUtil.loadIterators(IterConfigUtil.java:211)
	... 11 more

There is the following note in the filedata example:

Note that in order to scan the 
examples.dataTable the class, org.apache.accumulo.examples.filedata.ChunkCombiner, must be in 
your classpath, or the accumulo-examples-shaded.jar should be moved to the accumulo lib directory.

This error occurs even with the shaded jar moved to the lib directory.

To reproduce: follow the steps on either the dirlist or filedata examples and check the monitor for errors.

sealing violation: package org.apache.accumulo.core.client is sealed

spark2-submit --class org.apache.accumulo.spark.CopyPlus5K /home/hduser/examples/accumulo-examples/spark/target/accumulo-spark-2.0.0-SNAPSHOT-jar-with-dependencies.jar bulk /opt/accumulo-1.9.2/conf/accumulo-client.properties

It is giving following error

19/08/05 06:59:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.SecurityException: sealing violation: package org.apache.accumulo.core.client is sealed
at java.net.URLClassLoader.getAndVerifyPackage(URLClassLoader.java:400)
at java.net.URLClassLoader.definePackageInternal(URLClassLoader.java:420)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:452)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.accumulo.spark.CopyPlus5K.main(CopyPlus5K.java:105)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
19/08/05 06:59:47 INFO util.ShutdownHookManager: Shutdown hook called
19/08/05 06:59:47 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-bd60a9c7-31d6-40d8-ad89-a7432e920fbf

Determine why examples aren't working properly with Accumulo 2.1

When attempting to run the examples with Accumulo 2.x I ran into issues where accumulo could not connect to zookeeper. There appear to be several classpath issues with the shaded jar that are causing issues. This ticket's purpose is to correct issues that would prevent the examples from running with the 2.x versions and to update the instructions as needed along the way.

compactionStrategy example fails

The SequentialBatchWriter command produces the error below. Lines 47-53 in the ClientOpts class, only gives --auths and --conf as param options.

./bin/runex client.SequentialBatchWriter -t test1 --start 0 --num 10000 --size 50 --batchMemory 20M --batchLatency 500 --batchThreads 20

Usage: org.apache.accumulo.examples.client.SequentialBatchWriter [options]
  Options:
    -auths, --auths
      the authorizations to use when reading or writing
      Default: <empty string>
    -c, --conf
      Path to accumulo-client.properties.If not set, defaults to path set by 
      env variable ACCUMULO_CLIENT_PROPS.
    -h, -?, --help, -help


Was passed main parameter '-t' but no main parameter was defined in your arg class

Invalid Root Table Metadata JSON version 2

I am unable to run any of the example code due to this exception.

I am running accumulo 3.1.0-SNAPSHOT via fluo-uno. I am able to manually interact with the instance via the shell however it seems that every attempt to make mutations from the example code results in this exception being thrown.

❯ ./bin/runex sample.SampleExample                                                                                                                                                                                                                          ⏎
2023-12-01T16:06:23,712 [examples.Common] INFO : Namespace already exists. User can ignore this message and continue. Namespace: examples
2023-12-01T16:06:23,917 [examples.Common] WARN : Table already exists. User may wish to delete existing table and re-run example. Table name: examples.sampex
2023-12-01T16:06:24,016 [clientImpl.TabletServerBatchWriter] ERROR: Error processing mutation set
java.lang.IllegalArgumentException: Invalid Root Table Metadata JSON version 2
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:191) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.metadata.schema.RootTabletMetadata.<init>(RootTabletMetadata.java:112) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.metadata.schema.RootTabletMetadata.<init>(RootTabletMetadata.java:107) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.metadata.schema.TabletsMetadata.getRootMetadata(TabletsMetadata.java:610) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.metadata.schema.TabletsMetadata.getRootMetadata(TabletsMetadata.java:590) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.metadata.schema.TabletsMetadata$Builder.build(TabletsMetadata.java:133) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.metadata.schema.AmpleImpl.readTablet(AmpleImpl.java:47) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.RootTabletLocator.getRootTabletLocation(RootTabletLocator.java:112) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.RootTabletLocator.locateTablet(RootTabletLocator.java:136) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.TabletLocatorImpl.lookupTabletLocation(TabletLocatorImpl.java:534) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.TabletLocatorImpl.lookupTabletLocationAndCheckLock(TabletLocatorImpl.java:717) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.TabletLocatorImpl._locateTablet(TabletLocatorImpl.java:702) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.TabletLocatorImpl.locateTablet(TabletLocatorImpl.java:506) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.TabletLocatorImpl.lookupTabletLocation(TabletLocatorImpl.java:534) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.TabletLocatorImpl.lookupTabletLocationAndCheckLock(TabletLocatorImpl.java:717) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.TabletLocatorImpl._locateTablet(TabletLocatorImpl.java:707) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.TabletLocatorImpl.binMutations(TabletLocatorImpl.java:223) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.SyncingTabletLocator.binMutations(SyncingTabletLocator.java:76) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.TimeoutTabletLocator.binMutations(TimeoutTabletLocator.java:86) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.TabletServerBatchWriter$MutationWriter.binMutations(TabletServerBatchWriter.java:705) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.TabletServerBatchWriter$MutationWriter.addMutations(TabletServerBatchWriter.java:756) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.clientImpl.TabletServerBatchWriter$MutationWriter.lambda$queueMutations$0(TabletServerBatchWriter.java:744) ~[accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52) [accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52) [accumulo-examples-shaded.jar:3.0.0-SNAPSHOT]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Exception in thread "main" org.apache.accumulo.core.client.MutationsRejectedException: # constraint violations : 0  security codes: {}  # server errors 0 # exceptions 1
	at org.apache.accumulo.core.clientImpl.TabletServerBatchWriter.checkForFailures(TabletServerBatchWriter.java:578)
	at org.apache.accumulo.core.clientImpl.TabletServerBatchWriter.flush(TabletServerBatchWriter.java:350)
	at org.apache.accumulo.core.clientImpl.BatchWriterImpl.flush(BatchWriterImpl.java:63)
	at org.apache.accumulo.examples.sample.SampleExample.main(SampleExample.java:82)
	Suppressed: org.apache.accumulo.core.client.MutationsRejectedException: # constraint violations : 0  security codes: {}  # server errors 0 # exceptions 1
		at org.apache.accumulo.core.clientImpl.TabletServerBatchWriter.checkForFailures(TabletServerBatchWriter.java:578)
		at org.apache.accumulo.core.clientImpl.TabletServerBatchWriter.close(TabletServerBatchWriter.java:376)
		at org.apache.accumulo.core.clientImpl.BatchWriterImpl.close(BatchWriterImpl.java:58)
		at org.apache.accumulo.examples.sample.SampleExample.main(SampleExample.java:73)
	Caused by: java.lang.IllegalArgumentException: Invalid Root Table Metadata JSON version 2
		at com.google.common.base.Preconditions.checkArgument(Preconditions.java:191)
		at org.apache.accumulo.core.metadata.schema.RootTabletMetadata.<init>(RootTabletMetadata.java:112)
		at org.apache.accumulo.core.metadata.schema.RootTabletMetadata.<init>(RootTabletMetadata.java:107)
		at org.apache.accumulo.core.metadata.schema.TabletsMetadata.getRootMetadata(TabletsMetadata.java:610)
		at org.apache.accumulo.core.metadata.schema.TabletsMetadata.getRootMetadata(TabletsMetadata.java:590)
		at org.apache.accumulo.core.metadata.schema.TabletsMetadata$Builder.build(TabletsMetadata.java:133)
		at org.apache.accumulo.core.metadata.schema.AmpleImpl.readTablet(AmpleImpl.java:47)
		at org.apache.accumulo.core.clientImpl.RootTabletLocator.getRootTabletLocation(RootTabletLocator.java:112)
		at org.apache.accumulo.core.clientImpl.RootTabletLocator.locateTablet(RootTabletLocator.java:136)
		at org.apache.accumulo.core.clientImpl.TabletLocatorImpl.lookupTabletLocation(TabletLocatorImpl.java:534)
		at org.apache.accumulo.core.clientImpl.TabletLocatorImpl.lookupTabletLocationAndCheckLock(TabletLocatorImpl.java:717)
		at org.apache.accumulo.core.clientImpl.TabletLocatorImpl._locateTablet(TabletLocatorImpl.java:702)
		at org.apache.accumulo.core.clientImpl.TabletLocatorImpl.locateTablet(TabletLocatorImpl.java:506)
		at org.apache.accumulo.core.clientImpl.TabletLocatorImpl.lookupTabletLocation(TabletLocatorImpl.java:534)
		at org.apache.accumulo.core.clientImpl.TabletLocatorImpl.lookupTabletLocationAndCheckLock(TabletLocatorImpl.java:717)
		at org.apache.accumulo.core.clientImpl.TabletLocatorImpl._locateTablet(TabletLocatorImpl.java:707)
		at org.apache.accumulo.core.clientImpl.TabletLocatorImpl.binMutations(TabletLocatorImpl.java:223)
		at org.apache.accumulo.core.clientImpl.SyncingTabletLocator.binMutations(SyncingTabletLocator.java:76)
		at org.apache.accumulo.core.clientImpl.TimeoutTabletLocator.binMutations(TimeoutTabletLocator.java:86)
		at org.apache.accumulo.core.clientImpl.TabletServerBatchWriter$MutationWriter.binMutations(TabletServerBatchWriter.java:705)
		at org.apache.accumulo.core.clientImpl.TabletServerBatchWriter$MutationWriter.addMutations(TabletServerBatchWriter.java:756)
		at org.apache.accumulo.core.clientImpl.TabletServerBatchWriter$MutationWriter.lambda$queueMutations$0(TabletServerBatchWriter.java:744)
		at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52)
		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
		at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52)
		at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: [CIRCULAR REFERENCE: java.lang.IllegalArgumentException: Invalid Root Table Metadata JSON version 2]

Update constraints example

Update constraints example to use API instead of CLI and consolidate maxmutation into constraints example.

Update filedata example to no longer use deprecated methods

Between Accumulo 2.0.x and 2.1.x, several methods have been deprecated. The fileData example currently relies on these methods. It would be good to update the example to use the latest methods and remove dependency on deprecated methods.

This issue specifically addresses the 2.1 line of accumulo examples running against the 2.1.x line of Accumulo development. Issue #109 address the need for the example to be updated for the 3.x line of Accumulo.

Update 2.0 branch to use examples namespace when creating tables.

The accumulo-examples repo was updated so that table names created during eecution would be less likely to collide with any existing table names that could exist on a users system. This was done by making use of an examples namespace when creating tables.

This update has only been applied to the main branch. The same updates should be applied to the example code and documentation for the 2.0 branch.

Missing ZK class

Running the ExamplesIT I am seeing this in the logs:

java.lang.NoClassDefFoundError: org/apache/zookeeper/txn/TxnDigest
        at org.apache.zookeeper.server.PrepRequestProcessor.setTxnDigest(PrepRequestProcessor.java:1122) ~[zookeeper-3.6.2.jar:3.6.2]
        at org.apache.zookeeper.server.PrepRequestProcessor.pRequest2Txn(PrepRequestProcessor.java:645) ~[zookeeper-3.6.2.jar:3.6.2]
        at org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:881) ~[zookeeper-3.6.2.jar:3.6.2]
        at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:155) ~[zookeeper-3.6.2.jar:3.6.2]
Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.txn.TxnDigest
        at jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) ~[?:?]
        at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) ~[?:?]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[?:?]
        ... 4 more

Connector.builder().usingProperties("conf/accumulo-client.properties").build()

if I create the package then this issue pop up:
ERROR] /home/hadoop/fluo-uno/accumulo-examples/src/main/java/org/apache/accumulo/examples/bloom/BloomFilters.java:[35,36] cannot find symbol
symbol: method builder()
location: class org.apache.accumulo.core.client.Connector
This issue is on all java file how have the fix usingProperties. Connector.builder does occurs in Accumulo-connector.

Update 2.0 branch to use examples namespace when creating tables.

The accumulo-examples repo was updated so that table names created during eecution would be less likely to collide with any existing table names that could exist on a users system. This was done by making use of an examples namespace when creating tables.

This update has only been applied to the main branch. The same updates should be applied to the example code and documentation for the 2.0 branch.

Remove or update tracing example

Upstream Accumulo replaced HTrace with opentelemetry. As such, the current HTrace-based example no longer makes sense as-is. It should either be changed to an example with opentelemetry, or it should be removed.

filedata example fails

This command produces the following error:

./bin/runmr filedata.CharacterHistogram -t dataTable --auths exampleVis --vis exampleVis
Exception in thread "main" java.lang.IllegalStateException: Bad configuration: the store method was not called.
	at org.apache.accumulo.hadoopImpl.mapreduce.lib.ConfiguratorBase.checkJobStored(ConfiguratorBase.java:212)
	at org.apache.accumulo.hadoop.mapreduce.AccumuloOutputFormat.checkOutputSpecs(AccumuloOutputFormat.java:61)
	at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:277)
	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:143)
	at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
	at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588)
	at org.apache.accumulo.examples.filedata.CharacterHistogram.main(CharacterHistogram.java:100)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:236)

Fix Tracing

In 3f3586b, tracing code was commented out due to upstream changes to tracing code in Accumulo core. This code should be removed or changed to reflect the current tracing best practices.

Update filedata example for Accumulo 3.x development line

The filedata example currently references code that no longer exists in the 3.x line of Accumulo. Update the example to work with the current line of development.

Three files have been temporarily commented out in order to allow the examples project to build successfully. These should be updated to allow the example to once again work properly.

Compaction Strategy Example is no longer accurate

The compaction strategy example should be updated to reflect the recent changes to compaction configuration in accumulo. Including the description of the strategies and the example configuration given.
Example below of what could be added to replace the current example (this is found I'm accumulo-testing)

tserver.compaction.major.service.cs1.planner=org.apache.accumulo.core.spi.compaction.DefaultCompactionPlanner \
tserver.compaction.major.service.cs1.planner.opts.executors=\
[{"name":"small","type":"internal","maxSize":"16M","numThreads":8},\
{"name":"medium","type":"internal","maxSize":"128M","numThreads":4},\
{"name":"large","type":"internal","numThreads":2

The blog post on the accumulo website also gives further details on how to configure the new compactions.
https://accumulo.apache.org/blog/2021/07/08/external-compactions.html

Dirlist example throws exceptions during execution

When running the Dirlist example and executing

./bin/runex dirlist.Viewer....

exceptions are thrown in the terminal window. The Viewer GUI appears to work properly but the exceptions should be examined more closely and resolved if possible.

All `./bin/runmr` commands fail to run

Many examples contain sections which use ./bin/runmr. In all cases that I have tested, these commands fail with the following error:

2022-03-28 11:12:15,167 INFO util.ApplicationClassLoader: classpath: [file:/tmp/hadoop-unjar5857181900655303064/, file:/home/dgarguilo/github/accumulo-examples/target/accumulo-examples-shaded.jar]
2022-03-28 11:12:15,167 INFO util.ApplicationClassLoader: system classes: [java., javax.accessibility., javax.activation., javax.activity., javax.annotation., javax.annotation.processing., javax.crypto., javax.imageio., javax.jws., javax.lang.model., -javax.management.j2ee., javax.management., javax.naming., javax.net., javax.print., javax.rmi., javax.script., -javax.security.auth.message., javax.security.auth., javax.security.cert., javax.security.sasl., javax.sound., javax.sql., javax.swing., javax.tools., javax.transaction., -javax.xml.registry., -javax.xml.rpc., javax.xml., org.w3c.dom., org.xml.sax., org.apache.commons.logging., org.apache.log4j., -org.apache.hadoop.hbase., org.apache.hadoop., core-default.xml, hdfs-default.xml, mapred-default.xml, yarn-default.xml]
Exception in thread "main" java.lang.LinkageError: loader constraint violation: when resolving method 'org.slf4j.ILoggerFactory org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()' the class loader org.apache.hadoop.util.ApplicationClassLoader @220a042e of the current class, org/slf4j/LoggerFactory, and the class loader 'app' for the method's defining class, org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type org/slf4j/ILoggerFactory used in the signature (org.slf4j.LoggerFactory is in unnamed module of loader org.apache.hadoop.util.ApplicationClassLoader @220a042e, parent loader 'app'; org.slf4j.impl.StaticLoggerBinder is in unnamed module of loader 'app')
	at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:423)
	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
	at org.apache.accumulo.core.singletons.SingletonManager.<clinit>(SingletonManager.java:48)
	at org.apache.accumulo.core.clientImpl.ClientContext$ClientBuilderImpl.buildClient(ClientContext.java:830)
	at org.apache.accumulo.core.clientImpl.ClientContext$ClientBuilderImpl.build(ClientContext.java:826)
	at org.apache.accumulo.examples.cli.ClientOpts.createAccumuloClient(ClientOpts.java:58)
	at org.apache.accumulo.examples.mapreduce.bulk.BulkIngestExample.ingestTestData(BulkIngestExample.java:141)
	at org.apache.accumulo.examples.mapreduce.bulk.BulkIngestExample.main(BulkIngestExample.java:112)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:236)

This is blocking the examples that use this from being run.

The example output above was produced by running the bulk ingest example but any example using ./bin/runmr should result in the same failure.

Sample example java program fails with Accumulo-2.1.x

When run against the current Accumulo-2.1.0-SNAPSHOT, the java sample.SampleExample class fails with the following exception:

Exception in thread "main" org.apache.accumulo.core.client.AccumuloException: TabletServer could not load CompactionStrategy class org.apache.accumulo.tserver.compaction.strategies.ConfigurableCompactionStrategy at org.apache.accumulo.core.clientImpl.TableOperationsImpl.ensureStrategyCanLoad(TableOperationsImpl.java:900) at org.apache.accumulo.core.clientImpl.TableOperationsImpl.compact(TableOperationsImpl.java:850) at org.apache.accumulo.examples.sample.SampleExample.main(SampleExample.java:105)

The sample example works as expected if run step by step per directions. This error occurs only when executing the automated java program via the command:

./bin/runex sample.SampleExample

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.