arangodb / oskar

ArangoDB

ArangoDB is a scalable graph database system to drive value from connected data, faster. Native graphs, an integrated search engine, and JSON support, via a single query language. ArangoDB runs on-prem, in the cloud – anywhere.

ArangoDB Cloud Service

The ArangoGraph Insights Platform is the simplest way to run ArangoDB. You can create deployments on all major cloud providers in many regions with ease.

Getting Started

• ArangoDB University
• Free Udemy Course
• Training Center
• Documentation

For the impatient:

• Test ArangoDB in the cloud with ArangoGraph for free.

• Alternatively, download and install ArangoDB. Start the server arangod if the installer did not do it for you.

  Or start ArangoDB in a Docker container:

  docker run -e ARANGO_ROOT_PASSWORD=test123 -p 8529:8529 -d arangodb
  

  Then point your browser to http://127.0.0.1:8529/.

Key Features of ArangoDB

Native Graph - Store both data and relationships, for faster queries even with multiple levels of joins and deeper insights that simply aren't possible with traditional relational and document database systems.

Document Store - Every node in your graph is a JSON document: flexible, extensible, and easily imported from your existing document database.

ArangoSearch - Natively integrated cross-platform indexing, text-search and ranking engine for information retrieval, optimized for speed and memory.

ArangoDB is available in a free and open-source Community Edition, as well as a commercial Enterprise Edition with additional features.

Community Edition features

• Horizontal scalability: Seamlessly shard your data across multiple machines.
• High availability and resilience: Replicate data to multiple cluster nodes, with automatic failover.
• Flexible data modeling: Model your data as combination of key-value pairs, documents, and graphs as you see fit for your application.
• Work schema-free or use schema validation for data consistency. Store any type of data - date/time, geo-spatial, text, nested.
• Powerful query language (AQL) to retrieve and modify data - from simple CRUD operations, over complex filters and aggregations, all the way to joins, graphs, and ranked full-text search.
• Transactions: Run queries on multiple documents or collections with optional transactional consistency and isolation.
• Data-centric microservices: Unify your data storage logic, reduce network overhead, and secure sensitive data with the ArangoDB Foxx JavaScript framework.
• Fast access to your data: Fine-tune your queries with a variety of index types for optimal performance. ArangoDB is written in C++ and can handle even very large datasets efficiently.
• Easy to use web interface and command-line tools for interaction with the server.

Enterprise Edition features

Focus on solving enterprise-scale problems for mission critical workloads using secure graph data. The Enterprise Edition has all the features of the Community Edition and offers additional features for performance, compliance, and security, as well as further query capabilities.

• Smartly shard and replicate graphs and datasets with features like EnterpriseGraphs, SmartGraphs, and SmartJoins for lightning fast query execution.
• Combine the performance of a single server with the resilience of a cluster setup using OneShard deployments.
• Increase fault tolerance with Datacenter-to-Datacenter Replication and and create incremental Hot Backups without downtime.
• Enable highly secure work with Encryption 360, LDAP integration, enhanced Data Masking, and detailed Auditing.
• Perform parallel graph traversals and graph analytics with Pregel for cluster deployments.
• Use ArangoSearch search highlighting and nested search for advanced information retrieval.

Latest Release

Packages for all supported platforms can be downloaded from https://www.arangodb.com/download/.

For what's new in ArangoDB, see the Release Notes in the Documentation.

Stay in Contact

• Please use GitHub for feature requests and bug reports: https://github.com/arangodb/arangodb/issues

• Ask questions about AQL, usage scenarios, etc. on StackOverflow: https://stackoverflow.com/questions/tagged/arangodb

• Chat with the community and the developers on Slack: https://arangodb-community.slack.com/

• Learn more about ArangoDB with our YouTube channel: https://www.youtube.com/@ArangoDB

• Follow us on Twitter to stay up to date: https://twitter.com/arangodb

• Find out more about our community: https://www.arangodb.com/community

oskar's Issues

ArangoDB in makeRelease for Windows is built with -DMAINTAINER_MODE=On

See https://github.com/arangodb/release-3.4/issues/57.

Enabling/disabling catch tests

Catch tests don't have a separate config option. While debugMode and releaseMode enable/disable them, the default is effectively releaseMode plus catch tests. This is not distinguishable with showConfig, and its impossible to go back to the default.

coredump analysis of tarballs

Clarify how to obtain debug symbols for the tarballs, and how to make them work.

error: numactl-libs-2.0.12-9.el8.x86_64.rpm: not an rpm package (or package manifest)

Hi,
It seems numactl-libs-2.0.12-9.el8.x86_64.rpm is replaced by numactl-libs-2.0.12-11.el8.x86_64.rpm
on http://mirror.centos.org/centos/8/BaseOS/x86_64/os/Packages/

Any plan to update that?

Thank you

log id check is incomplete in Oskar

The function that verifies the log ids in helper.fish is incomplete:

   set -l ids (find lib arangod arangosh enterprise -name "*.cpp" -o -name "*.h" \
     | xargs grep -h 'LOG_\(TOPIC\|TRX\|TOPIC_IF\)("[a-f0-9]*"' \
     | sed -e 's:^.*LOG_[^(]*("\([a-f0-9]*\)".*:\1:')
 
   set -l duplicate (echo $ids | tr " " "\n" | sort | uniq -d)
 
   set -l s 0
 
   if test "$duplicate" != ""
     echo "Duplicates: $duplicate"
     set s 1
   else 
     echo "Duplicates: NONE"
   end  

It only tests for duplicate log ids, but not for log ids that do not match the required pattern ^[0-9a-f]{5}$. If any log id is used in the code that doesn't match the 5 digit hex pattern, the function will currently not report it as invalid, but it should.

buildStaticArangoDB Error

While Trying to compile ArangoDb, I get the below error. Any Help Will be highly appreciated.

Using MacOS 12.6, 16" intel processor.

~ gcc -v
Apple clang version 14.0.0 (clang-1400.0.29.202)
Target: x86_64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

[ 57%] Building CXX object arangod/RocksDBEngine/CMakeFiles/arango_rocksdb.dir/RocksDBRestWalHandler.cpp.o
[ 57%] Building CXX object arangod/RocksDBEngine/CMakeFiles/arango_rocksdb.dir/RocksDBSavePoint.cpp.o
In file included from /Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/RocksDBEngine/RocksDBIndexFactory.cpp:29:
In file included from /Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/IResearch/IResearchRocksDBInvertedIndex.h:26:
In file included from /Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/IResearch/IResearchInvertedIndex.h:31:
In file included from /Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/IResearch/IResearchDataStore.h:27:
/Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/IResearch/Containers.h:162:14: error: 'notify_all' is unavailable: introduced in macOS 11.0
      _count.notify_all();
             ^
/Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/IResearch/Containers.h:88:16: note: in instantiation of member function 'arangodb::iresearch::AsyncValue<arangodb::iresearch::IResearchDataStore>::destroy' requested here
        _self->destroy();
               ^
/Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/IResearch/IResearchDataStore.h:70:3: note: in instantiation of member function 'arangodb::iresearch::AsyncValue<arangodb::iresearch::IResearchDataStore>::Value::~Value' requested here
  IResearchTrxState(LinkLock&& linkLock, irs::index_writer& writer) noexcept
  ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk/usr/include/c++/v1/atomic:1701:62: note: 'notify_all' has been explicitly marked unavailable here
    _LIBCPP_AVAILABILITY_SYNC _LIBCPP_INLINE_VISIBILITY void notify_all() _NOEXCEPT
                                                             ^
In file included from /Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/RocksDBEngine/RocksDBIndexFactory.cpp:29:
In file included from /Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/IResearch/IResearchRocksDBInvertedIndex.h:26:
In file included from /Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/IResearch/IResearchInvertedIndex.h:31:
In file included from /Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/IResearch/IResearchDataStore.h:27:
/Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/IResearch/Containers.h:146:14: error: 'wait' is unavailable: introduced in macOS 11.0
      _count.wait(count, std::memory_order_relaxed);
             ^
/Users/bernardgaitho/@arangodb/oskar310/work/ArangoDB/arangod/IResearch/IResearchInvertedIndex.h:155:37: note: in instantiation of member function 'arangodb::iresearch::AsyncValue<arangodb::iresearch::IResearchDataStore>::reset' requested here
  void unload() final { _asyncSelf->reset(); }
                                    ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk/usr/include/c++/v1/atomic:1693:62: note: 'wait' has been explicitly marked unavailable here
    _LIBCPP_AVAILABILITY_SYNC _LIBCPP_INLINE_VISIBILITY void wait(_Tp __v, memory_order __m = memory_order_seq_cst) const _NOEXCEPT
                                                             ^
2 errors generated.
make[2]: *** [arangod/RocksDBEngine/CMakeFiles/arango_rocksdb.dir/RocksDBIndexFactory.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [arangod/RocksDBEngine/CMakeFiles/arango_rocksdb.dir/all] Error 2
make: *** [all] Error 2
Build error!

JavaScript exception in new ARM Build

2023-02-22T09:19:15Z [8149] WARNING [0458b] {general} 🥑 This is a maintainer version intended for debugging. DO NOT USE IN PRODUCTION! 🔥
2023-02-22T09:19:15Z [8149] WARNING [bd666] {general} ===================================================================================
2023-02-22T09:19:15Z [8149] INFO [75ddc] {general} detected operating system: macos
2023-02-22T09:19:15Z [8149] INFO [a1c60] {syscall} file-descriptors (nofiles) hard limit is unlimited, soft limit is 10240
2023-02-22T09:19:15Z [8149] INFO [3bb7d] {cluster} Starting up with role SINGLE
2023-02-22T09:19:16Z [8149] INFO [fe333] {engines} RocksDB recovery starting, scanning WAL starting from sequence number 0, latest sequence number: 3, files in archive: 0
2023-02-22T09:19:16Z [8149] INFO [a4ec8] {engines} RocksDB recovery finished, WAL entries scanned: 4, recovery start sequence number: 0, latest WAL sequence number: 3, max tick value found in WAL: 1, last HLC value found in WAL: 0
2023-02-22T09:19:16Z [8149] ERROR [8a210] {general} JavaScript exception in file '/Users/bernardgaitho/@arangodb/oskar3.10.2/work/ArangoDB/./js/common/bootstrap/modules.js' at 561,9: ArangoError 3103: failed to invoke module\nFile: /Users/bernardgaitho/@arangodb/oskar3.10.2/work/ArangoDB/js/server/modules/@arangodb/index.js\nReason: ArangoError: cannot locate module\nFile: semver\n at Function.Module._resolveFilename (/Users/bernardgaitho/@arangodb/oskar3.10.2/work/ArangoDB/./js/common/bootstrap/modules.js:530:13)\n at Function.Module._load (/Users/bernardgaitho/@arangodb/oskar3.10.2/work/ArangoDB/./js/common/bootstrap/modules.js:449:27)\n at require (/Users/bernardgaitho/@arangodb/oskar3.10.2/work/ArangoDB/./js/common/bootstrap/modules.js:98:21)\n at Object. (/Users/bernardgaitho/@arangodb/oskar3.10.2/work/ArangoDB/js/server/modules/@arangodb/index.js:29:16)\n at Module._compile (/Users/bernardgaitho/@arangodb/oskar3.10.2/work/ArangoDB/./js/common/bootstrap/modules.js:604:8)\n at Object.Module._extensions..js (/Users/bernardgaitho/@arangodb/oskar3.10.2/work/ArangoDB/./js/common/bootstrap/modules.js:622:12)\n at Module.load (/Users/bernardgaitho/@arangodb/oskar3.10.2/work/ArangoDB/./js/common/bootstrap/modules.js:549:36)\n at Function.Module._load (/Users/bernardgaitho/@arangodb/oskar3.10.2/work/ArangoDB/./js/common/bootstrap/modules.js:494:16)\n at global.require (/Users/bernardgaitho/@arangodb/oskar3.10.2/work/ArangoDB/./js/common/bootstrap/modules.js:643:19)\n at server/initialize.js:65:19
2023-02-22T09:19:16Z [8149] ERROR [409ee] {general} ! throw Object.assign(
2023-02-22T09:19:16Z [8149] ERROR [cb0bd] {general} ! ^
2023-02-22T09:19:16Z [8149] FATAL [69ac3] {v8} error during execution of JavaScript file 'server/initialize.js'

Check metrics API output against promtool

We recently merged PR arangodb/arangodb#13722 into 3.8/devel.
This PR added a test to check the output of ArangoDB's metrics API (HTTP GET /_admin/metrics/v2) against promtool, which is an executable shipped with Promotheus.
promtool can validate metrics data, and we would like to leverage that, so that whenever we introduced issues with our metrics, promtool can tell us about it.

The test in the PR will only run on Linux and do nothing elsewhere. on Linux, it requires the PROMTOOL_PATH environment variable to set. If it is not set, the test will do nothing as well. This is where we are now.

So what we would need to do is to install promtool on our Linux Jenkins machines, and set the PROMTOOL_PATH environment variable to the directory containing it.

The promtool executable is part of a Prometheus distribution, and can be found in each Prometheus release distributable (e.g. in prometheus/[email protected] (release)).

We also require the sh command and the true command to be working on Linux. I hope this is a given, otherwise we will need to install these too. We require sh for running a command with input/output redirection, and we use sh -c true to test the shell functionality.

The test in question can be executed as follows:

PROMOTOOL_PATH=/home/jsteemann/Downloads/prometheus/ scripts/unittest shell_client --test shell-promtool.js --cluster true