GithubHelp home page GithubHelp logo

aerospike / aerospike-server Goto Github PK

View Code? Open in Web Editor NEW
973.0 83.0 163.0 5.63 MB

Aerospike Database Server – flash-optimized, in-memory, nosql database

Home Page: http://aerospike.com/

License: Other

Makefile 0.74% C 97.51% C++ 0.80% HTML 0.06% CSS 0.46% Shell 0.31% DTrace 0.03% Python 0.09%

aerospike-server's Introduction

Aerospike Database Server

Welcome to the Aerospike Database Server source code tree!

Aerospike is a distributed, scalable NoSQL database. It is architected with three key objectives:

  • To create a high-performance, scalable platform that would meet the needs of today's web-scale applications
  • To provide the robustness and reliability (i.e., ACID) expected from traditional databases.
  • To provide operational efficiency (minimal manual involvement)

For more information on Aerospike, please visit: http://aerospike.com

Telemetry Anonymized Data Collection

The Aerospike Community Edition collects anonymized server performance statistics. Please see the Aerospike Telemetery web page for more information. The full Telemetry data collection agent source code may be found in the "telemetry" submodule.

Build Prerequisites

The Aerospike Database Server can be built and deployed on various current 64-bit GNU/Linux platform versions, such as Red Hat Enterprise Linux 8/9, Amazon Linux 2023, Debian 11 or later, and Ubuntu 20.04 or later.

Dependencies

The majority of the Aerospike source code is written in the C programming language, conforming to the ANSI C99 standard.

To install dependencies for a development environment run ./bin/install-dependencies.sh in the aerospike-server repo.

In particular, the following tools and libraries are needed:

C Compiler Toolchain

Building Aerospike requires the GCC 4.1 or later C compiler toolchain, with the standard GNU/Linux development tools and libraries installed in the build environment, including:

  • autoconf

  • automake

  • cmake

  • libtool

  • make

C++

The C++ compiler is required for the Aerospike geospatial indexing feature and its dependency, Google's S2 Geometry Library (both written in C++.)

  • The required Red Hat Enterprise Linux 8/9 package to install is: gcc-c++.

  • The required Debian 11/12 and Ubuntu 20/22 package to install is: g++.

OpenSSL

OpenSSL 0.9.8b or later is required for cryptographic hash functions (RIPEMD-160 & SHA-1) and pseudo-random number generation.

  • The Red Hat Enterprise Linux 8/9 OpenSSL package to install: openssl-devel

  • The Debian 11/12 and Ubuntu 20/22 OpenSSL packages to install: libssl-dev.

Zlib

  • The Red Hat Enterprise Linux 8/9 requires zlib-devel

  • The Debian 11/12 and Ubuntu 20/22 requiresi zlib1g-dev.

Python 3 or 2

Running the Telemetry Agent requires either Python 3+ or Python 2.6+, at least one of which is generally available by default on most platforms. On some distros, such as Ubuntu 20+, it may be necessary to install the package python, while on other distros, such as Red Hat Enterprise Linux 8+, the package name includes the major (and/or minor) version number, e.g., python3 or python2.

Submodules

The Aerospike Database Server build depends upon the following submodules:

Submodule Description
abseil-cpp Support for the S2 Spherical Geometry Library
common The Aerospike Common Library
jansson C library for encoding, decoding and manipulating JSON data
jemalloc The JEMalloc Memory Allocator
libbacktrace A C library that may be linked into a C/C++ program to produce symbolic backtraces
lua The Lua runtime
mod-lua The Aerospike Lua Interface
s2geometry The S2 Spherical Geometry Library
telemetry The Aerospike Telemetry Agent (Community Edition only)

After the initial cloning of the aerospike-server repo., the submodules must be fetched for the first time using the following command:

$ git submodule update --init

Note: As this project uses submodules, the source archive downloadable via GitHub's Download ZIP button will not build unless the correct revision of each submodule is first manually installed in the appropriate modules subdirectory.

Building Aerospike

Default Build

$ make          -- Perform the default build (no packaging.)

Note: You can use the -j option with make to speed up the build on multiple CPU cores. For example, to run four parallel jobs:

$ make -j4

Build Options

$ make deb      -- Build the Debian (Ubuntu) package.

$ make rpm      -- Build the Red Hat Package Manager (RPM) package.

$ make source   -- Package the source code as a compressed "tar" archive.

$ make clean    -- Delete any existing build products, excluding built packages.

$ make cleanpkg -- Delete built packages.

$ make cleanall -- Delete all existing build products, including built packages.

$ make cleangit -- Delete all files untracked by Git.  (Use with caution!)

$ make strip    -- Build a "strip(1)"ed version of the server executable.

Overriding Default Build Options

$ make {<Target>}* {<VARIABLE>=<VALUE>}*  -- Build <Target>(s) with optional variable overrides.

Example:

$ make  -- Default build.

Configuring Aerospike

Sample Aerospike configuration files are provided in as/etc. The developer configuration file, aerospike_dev.conf, contains basic settings that should work out-of-the-box on most systems. The package example configuration files, aerospike.conf, and the Solid State Drive (SSD) version, aerospike_ssd.conf, are suitable for running Aerospike as a system daemon.

These sample files may be modified for specific use cases (e.g., setting network addresses, defining namespaces, and setting storage engine properties) and tuned for for maximum performance on a particular system. Also, system resource limits may need to be increased to allow, e.g., a greater number of concurrent connections to the database. See "man limits.conf" for how to change the system's limit on a process' number of open file descriptors ("nofile".)

Running Aerospike

There are several options for running the Aerospike database. Which option to use depends upon whether the primary purpose is production deployment or software development.

The preferred method for running Aerospike in a production environment is to build and install the Aerospike package appropriate for the target Linux distribution (i.e., an ".rpm", ".deb", or ".tgz" file), and then to control the state of the Aerospike daemon via systemctl on e.g., systemctl start aerospike.

Please refer to the full documentation on the Aerospike web site, https://docs.aerospike.com/, for more detailed information about configuring and running the Aerospike Database Server, as well as about the Aerospike client API packages for popular programming languages.

aerospike-server's People

Contributors

achakrab avatar ashishshinde avatar briannichols avatar bwhisler avatar gooding470 avatar jpollack-aerospike avatar kportertx avatar laurynas-biveinis avatar marknaero avatar mcoberly2 avatar mtendjou avatar pvinh-spike avatar rbotzer avatar srivarun91 avatar sunilvirus avatar tlo-as avatar uncle-betty avatar xorphox avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aerospike-server's Issues

ASan reports stack-buffer-underflow

OS: Ubuntu 18.04 LTS

Aerospike version: 2a709d7

Client version: c-5.2.0

=================================================================
==20195==ERROR: AddressSanitizer: stack-buffer-underflow on address 0x7ffc00dd503f at pc 0x000000a83a3a bp 0x7ffc00dd5010 sp 0x7ffc00dd5008
WRITE of size 1 at 0x7ffc00dd503f thread T0
    #0 0xa83a39  (/home/zyh/aerospike-server/target/Linux-x86_64/bin/asd+0xa83a39) cf/src/hardware.c:481
    #1 0xa81f31  (/home/zyh/aerospike-server/target/Linux-x86_64/bin/asd+0xa81f31) cf/src/hardware.c:501
    #2 0xa782d3  (/home/zyh/aerospike-server/target/Linux-x86_64/bin/asd+0xa782d3) cf/src/hardware.c:742
    #3 0xa764f4  (/home/zyh/aerospike-server/target/Linux-x86_64/bin/asd+0xa764f4) cf/src/hardware.c:1911
    #4 0x506555  (/home/zyh/aerospike-server/target/Linux-x86_64/bin/asd+0x506555) as/src/base/as.c:273
    #5 0x86f0b4  (/home/zyh/aerospike-server/target/Linux-x86_64/bin/asd+0x86f0b4) as/src/base/main.c:37
    #6 0x7fdfb3449bf6  (/lib/x86_64-linux-gnu/libc.so.6+0x21bf6)
    #7 0x459229  (/home/zyh/aerospike-server/target/Linux-x86_64/bin/asd+0x459229)

Address 0x7ffc00dd503f is located in stack of thread T0 at offset 31 in frame
    #0 0xa8369f  (/home/zyh/aerospike-server/target/Linux-x86_64/bin/asd+0xa8369f)

This frame has 3 object(s):
    [32, 132) 'buff' (line 473) <== Memory access at offset 31 underflows this variable
    [176, 184) 'limit' (line 474)
    [208, 216) 'end' (line 485)
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
    (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-underflow (/home/zyh/aerospike-server/target/Linux-x86_64/bin/asd+0xa83a39)
Shadow bytes around the buggy address:
0x1000001b29b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1000001b29c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1000001b29d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1000001b29e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1000001b29f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x1000001b2a00: 00 00 00 00 f1 f1 f1[f1]00 00 00 00 00 00 00 00
0x1000001b2a10: 00 00 00 00 04 f2 f2 f2 f2 f2 00 f2 f2 f2 f8 f3
0x1000001b2a20: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
0x1000001b2a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x1000001b2a40: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 f3 f3 f3
0x1000001b2a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable:           00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone:       fa
Freed heap region:       fd
Stack left redzone:      f1
Stack mid redzone:       f2
Stack right redzone:     f3
Stack after return:      f5
Stack use after scope:   f8
Global redzone:          f9
Global init order:       f6
Poisoned by user:        f7
Container overflow:      fc
Array cookie:            ac
Intra object redzone:    bb
ASan internal:           fe
Left alloca redzone:     ca
Right alloca redzone:    cb
Shadow gap:              cc
==20195==ABORTING

Though I'm using the old version, the issue seems still exists in the latest version.

ssize_t len = read(fd, (uint8_t *)buf + total, *limit - total);
CF_NEVER_FAILS(len);
if (len == 0) {
cf_detail(CF_OS, "EOF");
break;
}

Maybe we should add some error handling code if read returns error.

build fails on AArch64, Fedora 33

[jw@cn06 aerospike-server]$ make VERBOSE=1 -j$(nproc)
mkdir -p ./target/Linux-aarch64/gen ./target/Linux-aarch64/lib ./target/Linux-aarch64/bin
mkdir -p ./target/Linux-aarch64/obj/base ./target/Linux-aarch64/obj/fabric ./target/Linux-aarch64/obj/storage ./target/Linux-aarch64/obj/geospatial ./target/Linux-aarch64/obj/transaction ./target/Linux-aarch64/obj/xdr
build/gen_version community fc33 > ./target/Linux-aarch64/gen/version.c
cc -o target/Linux-aarch64/obj/version.o -c target/Linux-aarch64/gen/version.c
make -C /home/jw/aerospike-server/modules/luajit Q= TARGET_SONAME=libluajit.so CCDEBUG=-g
make[1]: Entering directory '/home/jw/aerospike-server/modules/luajit'
==== Building LuaJIT 2.0.3 ====
make -C src
make[2]: Entering directory '/home/jw/aerospike-server/modules/luajit/src'
lj_arch.h:55:2: error: #error "No support for this architecture (yet)"
55 | #error "No support for this architecture (yet)"
| ^~~~~
lj_arch.h:279:2: error: #error "No target architecture defined"
279 | #error "No target architecture defined"
| ^~~~~
lj_arch.h:55:2: error: #error "No support for this architecture (yet)"
55 | #error "No support for this architecture (yet)"
| ^~~~~
lj_arch.h:279:2: error: #error "No target architecture defined"
279 | #error "No target architecture defined"
| ^~~~~
lj_arch.h:55:2: error: #error "No support for this architecture (yet)"
55 | #error "No support for this architecture (yet)"
| ^~~~~
lj_arch.h:279:2: error: #error "No target architecture defined"
279 | #error "No target architecture defined"
| ^~~~~
lj_arch.h:55:2: error: #error "No support for this architecture (yet)"
55 | #error "No support for this architecture (yet)"
| ^~~~~
lj_arch.h:279:2: error: #error "No target architecture defined"
279 | #error "No target architecture defined"
| ^~~~~
lj_arch.h:55:2: error: #error "No support for this architecture (yet)"
55 | #error "No support for this architecture (yet)"
| ^~~~~
lj_arch.h:279:2: error: #error "No target architecture defined"
279 | #error "No target architecture defined"
| ^~~~~
lj_arch.h:55:2: error: #error "No support for this architecture (yet)"
55 | #error "No support for this architecture (yet)"
| ^~~~~
lj_arch.h:279:2: error: #error "No target architecture defined"
279 | #error "No target architecture defined"
| ^~~~~
Makefile:231: *** Unsupported target architecture. Stop.
make[2]: Leaving directory '/home/jw/aerospike-server/modules/luajit/src'
make[1]: *** [Makefile:104: default] Error 2
make[1]: Leaving directory '/home/jw/aerospike-server/modules/luajit'
make: *** [Makefile:54: aslibs] Error 2
[jw@cn06 aerospike-server]$

Note, luajit-devel is installed.

link wg0 state unknown

The issues submitted here should be Aerospike Server code related:

  • bugs (not behaving as expected/documented)

OS: "Debian 12"

Aerospike version: Aerospike Community Edition build 7.0.0.7


I want to use wireguard to protect the traffic, when set NETLINK to wg0 server give this error:

link  wg0 state unknown

Is there a option to skip the link state check ? or fix the wireguard problem ?

epoll_event isn't packed on all architectures

ubuntu-18.04 ppc64le - git master @ c302133

epoll_event isn't packed on all architectures so replicating the structure isn't portable and indeed causes runtime ppc64le problems.

The structure is redefined here:
https://github.com/aerospike/aerospike-server/blob/master/cf/include/socket.h#L156..L160

For portability the data.ptr union element could be used where the existing code rather than recreating the structure data structure element.

The structure is defined in the standard:
https://linux.die.net/man/2/epoll_wait

Aerospike pods consume a lot of RAM during migration, which is not released after migration

Platform: GKE
Aerospike container version: aerospike/aerospike-server:5.5.0.7


Aerospike pods consume a lot of RAM during migration, which is not released after migration: (graph below is container memory usage / container memory limit, which is 5Gi)
image

we run a 3 Aerospike community edition pods cluster:
pods resources:

resources:
  limits:
    cpu: 2
    memory: 5Gi
  requests:
    cpu: 500m
    memory: 4Gi

some of the aerospike helm values:

aerospikeNamespaceMemoryGB: "3"
aerospikeReplicationFactor: "2"
aerospikeConfFile: |
  # Aerospike database configuration file.
  # This stanza must come first.
  service { ... dedacted ... }
  logging { ... dedacted ... }
  network { ... dedacted ... }
  namespace ${NAMESPACE} {
    replication-factor ${REPL_FACTOR}
    memory-size ${MEM_GB}G
    default-ttl ${DEFAULT_TTL}
    nsup-period 1d
    storage-engine device {
      file /opt/aerospike/data/${MY_POD_NAME}-${NAMESPACE}.dat
      filesize 35G
      data-in-memory false # Store data in memory in addition to file.
      write-block-size 512K
    }
  }

at the start of the migration, running kubectl top -n aerospike pod aerospike-v2-aerospike-0 shows

NAME                         CPU(cores)   MEMORY(bytes)   
aerospike-v2-aerospike-0   2098m        1442Mi

and at the end of the migration it shows:

NAME                         CPU(cores)   MEMORY(bytes)   
aerospike-v2-aerospike-0   27m          4336Mi

output of asadm -e "info" after migration:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Network Information (2022-08-25 12:34:56 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
               Cluster|                                                                              Node|         Node ID|              IP|    Build|Migrations|~~~~~~~~~~~~~~~~~~Cluster~~~~~~~~~~~~~~~~~~|Client|  Uptime
                      |                                                                                  |                |                |         |          |Size|         Key|Integrity|      Principal| Conns|        
aerospike-v2-aerospike|aerospike-v2-aerospike-0.aerospike-v2-aerospike.aerospike.svc.cluster.local:3000| BB91617E999F27A|10.48.18.33:3000|C-5.5.0.7|   0.000  |   3|601ECA002FD7|True     |BB9ED2135EE683E|    19|00:21:47
aerospike-v2-aerospike|aerospike-v2-aerospike-1.aerospike-v2-aerospike.aerospike.svc.cluster.local:3000| BB913F72779F4F6|10.48.9.124:3000|C-5.5.0.7|   0.000  |   3|601ECA002FD7|True     |BB9ED2135EE683E|    17|00:21:20
aerospike-v2-aerospike|aerospike-v2-aerospike-2.aerospike-v2-aerospike.aerospike.svc.cluster.local:3000|*BB9ED2135EE683E|10.48.30.31:3000|C-5.5.0.7|   0.000  |   3|601ECA002FD7|True     |BB9ED2135EE683E|    18|00:21:52
Number of rows: 3

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Namespace Usage Information (2022-08-25 12:34:56 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Namespace|                                                                              Node|   Total|Expirations|Evictions|  Stop|~~~~~~~~~~~Disk~~~~~~~~~~~|~~~~~~~~~~~Memory~~~~~~~~~~|~Primary~
         |                                                                                  | Records|           |         |Writes|    Used|Used%|HWM%|Avail%|      Used|Used%|HWM%|Stop%|~~Index~~
         |                                                                                  |        |           |         |      |        |     |    |      |          |     |    |     |     Type
nyris    |aerospike-v2-aerospike-0.aerospike-v2-aerospike.aerospike.svc.cluster.local:3000| 7.511 M|    0.000  |  0.000  |False |2.669 GB|    8|   0|    91|458.418 MB|   15|   0|   90|undefined
nyris    |aerospike-v2-aerospike-1.aerospike-v2-aerospike.aerospike.svc.cluster.local:3000| 7.680 M|    0.000  |  0.000  |False |2.729 GB|    8|   0|    91|468.728 MB|   16|   0|   90|undefined
nyris    |aerospike-v2-aerospike-2.aerospike-v2-aerospike.aerospike.svc.cluster.local:3000| 7.536 M|    0.000  |  0.000  |False |2.678 GB|    8|   0|    91|459.940 MB|   15|   0|   90|undefined
nyris    |                                                                                  |22.726 M|    0.000  |  0.000  |      |8.075 GB|     |    |      |  1.355 GB|     |    |     |         
Number of rows: 3

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Namespace Object Information (2022-08-25 12:34:56 UTC)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Namespace|                                                                              Node|Rack|  Repl|   Total|~~~~~~~~~~~Objects~~~~~~~~~~~|~~~~~~~~~Tombstones~~~~~~~~|~~~~Pending~~~~
         |                                                                                  |  ID|Factor| Records|  Master|   Prole|Non-Replica| Master|  Prole|Non-Replica|~~~~Migrates~~~
         |                                                                                  |    |      |        |        |        |           |       |       |           |     Tx|     Rx
nyris    |aerospike-v2-aerospike-0.aerospike-v2-aerospike.aerospike.svc.cluster.local:3000|   0|     2| 7.511 M| 3.684 M| 3.827 M|    0.000  |0.000  |0.000  |    0.000  |0.000  |0.000  
nyris    |aerospike-v2-aerospike-1.aerospike-v2-aerospike.aerospike.svc.cluster.local:3000|   0|     2| 7.680 M| 3.930 M| 3.749 M|    0.000  |0.000  |0.000  |    0.000  |0.000  |0.000  
nyris    |aerospike-v2-aerospike-2.aerospike-v2-aerospike.aerospike.svc.cluster.local:3000|   0|     2| 7.536 M| 3.749 M| 3.787 M|    0.000  |0.000  |0.000  |    0.000  |0.000  |0.000  
nyris    |                                                                                  |    |      |22.726 M|11.363 M|11.363 M|    0.000  |0.000  |0.000  |    0.000  |0.000  |0.000  
Number of rows: 3

I have seen this post which explains why does it consumes this much memory, but i don't see why this memory is not released after migration is done.

Problem with gcc-9.2.0-r3(gentoo) -Werror compiler flag. -Werror=address-of-packed-member

When we are use simple make (assume -Wall compiler flag)
$ make
<-----------snip-------------->
server/ai/src»
cc -gdwarf-4 -g3 -O3 -fno-common -fno-strict-aliasing -Wall -D_FILE_OFFSET_BITS=64 -std=gnu99 -D_REENTRANT -D_GNU_SOURCE -DENHANCED_ALLOC -march=nocona -MMD -Werror -DMARCH_x86_64 -D__FILENAME__="ai_btree.c" -o ../../target/Linux-x86_64/obj/ai_btree.o -c -I/home/user_name/aero/server/modules/jemalloc/include -I../include -I../../target/Linux-x86_64/gen -I/home/user_name/aero/server/cf/include -I/home/user_name/aero/server/as/include -I/home/user_name/aero/server/as/include -I/home/user_name/aero/server/modules/common/target/Linux-x86_64/include -I/home/user_name/aero/server/modules/mod-lua/target/Linux-x86_64/include ai_btree.c
In file included from /home/user_name/aero/server/as/include/base/secondary_index.h:30,
from ../include/ai_types.h:37,
from ../include/ai_obj.h:30,
from ai_btree.c:29:
/home/user_name/aero/server/as/include/base/datamodel.h: In function «as_bin_set_id_from_name_w_len»:
/home/user_name/aero/server/as/include/base/datamodel.h:1293:67: ошибка: taking address of packed member of «struct as_bin_s» may result in an unaligned pointer value [-Werror=address-of-packed-member]
1293 | return as_bin_get_or_assign_id_w_len(ns, (const char *)buf, len, &b->id);
| ^~~~~~
cc1: all warnings interpreted as error
make[2]: *** [../../make_in/Makefile.targets:13: ../../target/Linux-x86_64/obj/ai_btree.o] Err 1
make[2]: leave «/home/user_name/aero/server/ai/src»
make[1]: *** [Makefile:9: all] Err 2
make[1]: leaving «/home/user_name/aero/server/ai»
make: *** [Makefile:53: all] Err 2

Porting PPC64LE

Hello,

I am trying to port aerospike to ppc64le, with that currently I can see there was an existing attempt already made #11 for the same.
Just wanted to understand, if there is any specific reason that the request is still open and without any reviews?
Before starting to put efforts, want to understand beforehand if there are any limitations in this?

Thank you.

How to horizontally add cluster nodes and modify configuration files to restart each service in the cluster without being affected

How to restart cluster services without being affected

  1. Configuration before horizontal expansion:

heartbeat {
mode mesh
address 192.168.127.128
port 3002
#all cluster
mesh-seed-address-port 192.168.127.128 3002
mesh-seed-address-port 192.168.127.131 3002

    # aerospike_mesh.conf for alternative.
    interval 150
    timeout 10      
    }
  1. Configuration after horizontal expansion:

heartbeat {
mode mesh
address 192.168.127.132
port 3002
#all cluster
mesh-seed-address-port 192.168.127.132 3002
mesh-seed-address-port 192.168.127.128 3002
mesh-seed-address-port 192.168.127.131 3002

    # aerospike_mesh.conf for alternative.
    interval 150
    timeout 10      
    }

[ARM support] Unable to compile for arm64.

I find a compilation issue, as follows:
The platform I use is arm64, but error when compiling module luajit:
version:47e253a363983209b814e16bea986df222ce9e77

mkdir -p ./target/Linux-aarch64/gen ./target/Linux-aarch64/lib ./target/Linux-aarch64/bin
mkdir -p ./target/Linux-aarch64/obj/base ./target/Linux-aarch64/obj/fabric ./target/Linux-aarch64/obj/storage ./target/Linux-aarch64/obj/geospatial ./target/Linux-aarch64/obj/transaction ./target/Linux-aarch64/obj/xdr
build/gen_version community ubuntu16.04 > ./target/Linux-aarch64/gen/version.c
cc -o target/Linux-aarch64/obj/version.o -c target/Linux-aarch64/gen/version.c
make -C /home/aerospike/aerospike-server/modules/luajit Q= TARGET_SONAME=libluajit.so CCDEBUG=-g
make[1]: Entering directory '/home/aerospike/aerospike-server/modules/luajit'
==== Building LuaJIT 2.0.3 ====
make -C src
make[2]: Entering directory '/home/aerospike/aerospike-server/modules/luajit/src'
lj_arch.h:55:2: error: #error "No support for this architecture (yet)"
#error "No support for this architecture (yet)"
^
lj_arch.h:279:2: error: #error "No target architecture defined"
#error "No target architecture defined"
^
lj_arch.h:55:2: error: #error "No support for this architecture (yet)"
#error "No support for this architecture (yet)"
^
lj_arch.h:279:2: error: #error "No target architecture defined"
#error "No target architecture defined"
^
lj_arch.h:55:2: error: #error "No support for this architecture (yet)"
#error "No support for this architecture (yet)"
^
lj_arch.h:279:2: error: #error "No target architecture defined"
#error "No target architecture defined"
^
lj_arch.h:55:2: error: #error "No support for this architecture (yet)"
#error "No support for this architecture (yet)"
^
lj_arch.h:279:2: error: #error "No target architecture defined"
#error "No target architecture defined"
^
lj_arch.h:55:2: error: #error "No support for this architecture (yet)"
#error "No support for this architecture (yet)"
^
lj_arch.h:279:2: error: #error "No target architecture defined"
#error "No target architecture defined"
^
lj_arch.h:55:2: error: #error "No support for this architecture (yet)"
#error "No support for this architecture (yet)"
^
lj_arch.h:279:2: error: #error "No target architecture defined"
#error "No target architecture defined"
^
Makefile:231: *** Unsupported target architecture. Stop.
make[2]: Leaving directory '/home/aerospike/aerospike-server/modules/luajit/src'
Makefile:103: recipe for target 'default' failed
make[1]: *** [default] Error 2
make[1]: Leaving directory '/home/aerospike/aerospike-server/modules/luajit'
Makefile:54: recipe for target 'aslibs' failed
make: *** [aslibs] Error 2

I found that it can compile normally with v2.1 of luajit(but it is no stable version), can I replace the original

Lost data written before all nodes initialized or data not synced?

OS: Ubuntu 18.04 LTS

Aerospike version: 2a709d7

Client version: c-5.2.0

It seems we could lose data written before all nodes initialized or data not synced?
See logs below:

start nodes....                                                                                                                                                                                                                                                                         node 0 start command is : /home/zyh/aerospike-server/target/Linux-x86_64/bin/asd --config-file aerospike_conf0
node 1 start command is : /home/zyh/aerospike-server/target/Linux-x86_64/bin/asd --config-file aerospike_conf1
node 2 start command is : /home/zyh/aerospike-server/target/Linux-x86_64/bin/asd --config-file aerospike_conf2
node 3 start command is : /home/zyh/aerospike-server/target/Linux-x86_64/bin/asd --config-file aerospike_conf3
node 4 start command is : /home/zyh/aerospike-server/target/Linux-x86_64/bin/asd --config-file aerospike_conf4

running AerospikeWrite
Failed to connect to seed 127.0.1.1 2000. AEROSPIKE_ERR_CLIENT Node FA0040011AC4202 127.0.1.1:2000 is not yet fully initialized
Failed to connect to seed 127.0.1.1 2001. AEROSPIKE_ERR_CLIENT Node FA1040011AC4202 127.0.1.1:2001 is not yet fully initialized
Failed to connect to seed 127.0.1.1 2002. AEROSPIKE_ERR_CLIENT Node FA2040011AC4202 127.0.1.1:2002 is not yet fully initialized
Failed to connect to seed 127.0.1.1 2003. AEROSPIKE_ERR_CLIENT Node FA3040011AC4202 127.0.1.1:2003 is not yet fully initialized
Node FA4040011AC4202 peers generation changed: 0
Node FA4040011AC4202 partition generation changed: 0
Add node FA4040011AC4202 127.0.1.1:2004
Node FA4040011AC4202 partition generation changed: 0
connect success!
write value is 0  <-- we write 0 here

... some more successful read/write

Node FA4040011AC4202 peers generation changed: 8                                                                                                                                                                                                                                        Node FA4040011AC4202 partition generation changed: 1                                                                                                                                                                                                                                    Add node FA0040011AC4202 127.0.1.1:2000                                                                                                                                                                                                                                                 Add node FA1040011AC4202 127.0.1.1:2001                                                                                                                                                                                                                                                 Add node FA3040011AC4202 127.0.1.1:2003                                                                                                                                                                                                                                                 Add node FA2040011AC4202 127.0.1.1:2002                                                                                                                                                                                                                                                 Node FA4040011AC4202 peers generation changed: 8
Node FA4040011AC4202 partition generation changed: 1
Add node FA0040011AC4202 127.0.1.1:2000
Add node FA1040011AC4202 127.0.1.1:2001
Add node FA3040011AC4202 127.0.1.1:2003
Add node FA2040011AC4202 127.0.1.1:2002
Node FA4040011AC4202 peers generation changed: 8
Node FA4040011AC4202 partition generation changed: 1
Add node FA0040011AC4202 127.0.1.1:2000
Add node FA1040011AC4202 127.0.1.1:2001
Add node FA3040011AC4202 127.0.1.1:2003
Add node FA2040011AC4202 127.0.1.1:2002
Node FA4040011AC4202 partition generation changed: 96
Node FA0040011AC4202 peers generation changed: 8
Node FA0040011AC4202 partition generation changed: 175
Node FA1040011AC4202 peers generation changed: 8
Node FA1040011AC4202 partition generation changed: 158
Node FA3040011AC4202 peers generation changed: 8
Node FA3040011AC4202 partition generation changed: 178
Node FA2040011AC4202 peers generation changed: 8
Node FA2040011AC4202 partition generation changed: 178
Node FA4040011AC4202 partition generation changed: 97
Node FA0040011AC4202 peers generation changed: 8
Node FA0040011AC4202 partition generation changed: 177
Node FA1040011AC4202 peers generation changed: 8
Node FA1040011AC4202 partition generation changed: 159
Node FA3040011AC4202 peers generation changed: 8
Node FA3040011AC4202 partition generation changed: 178
Node FA2040011AC4202 peers generation changed: 8
Node FA2040011AC4202 partition generation changed: 178
Node FA4040011AC4202 partition generation changed: 98
Node FA0040011AC4202 peers generation changed: 8
Node FA0040011AC4202 partition generation changed: 178
Node FA1040011AC4202 peers generation changed: 8
Node FA1040011AC4202 partition generation changed: 159
Node FA3040011AC4202 peers generation changed: 8
Node FA3040011AC4202 partition generation changed: 178
Node FA2040011AC4202 peers generation changed: 8
Node FA2040011AC4202 partition generation changed: 178
Node FA4040011AC4202 partition generation changed: 201
Node FA0040011AC4202 partition generation changed: 345
Node FA1040011AC4202 partition generation changed: 326
Node FA3040011AC4202 partition generation changed: 338
Node FA2040011AC4202 partition generation changed: 369
Node FA4040011AC4202 partition generation changed: 201
Node FA0040011AC4202 partition generation changed: 346
Node FA1040011AC4202 partition generation changed: 327
Node FA3040011AC4202 partition generation changed: 339
Node FA2040011AC4202 partition generation changed: 372

running operator AerospikeRead
connect success!
aerospike_key_get() returned 2 - 127.0.1.1:2002 AEROSPIKE_ERR_RECORD_NOT_FOUND    <-- we cannot get data here

In the logs, we can see we successfully write 0 to node (port 2004) before all nodes initialized, but could not read it after the initialization.

client write failed but still can read the data?

I got an issue that client write failed but still read the failed data, is it possible for my configuration?

# Aerospike database configuration file for deployments using mesh heartbeats.

service {
	user zyh
	group zyh
	paxos-single-replica-limit 0 # Number of nodes where the replica count is automatically reduced to 1.
	pidfile /home/zyh/distributed-system-test/aerospike_test/bin/run/asd3.pid
	proto-fd-max 15000
	work-directory run/work3
	run-as-daemon false
}

mod-lua {
	user-path run/work3/usr/udf/lua
}

logging {
	# Log file must be an absolute path.
	file /home/zyh/distributed-system-test/aerospike_test/bin/run/aerospike3.log {
		context any info
	}
}

network {
	service {
		address 127.0.0.1
		port 2002
	}

	heartbeat {
		mode mesh
		port 5004 # Heartbeat port for this node.

		# List one or more other nodes, one ip-address & port per line:
		# mesh-seed-address-port 127.0.0.1 3000
		mesh-seed-address-port 127.0.0.1 5005
		mesh-seed-address-port 127.0.0.1 5006

		interval 250
		timeout 10
	}

	fabric {
		port 4002
	}

	info {
		port 3002
	}
}

namespace test {
	replication-factor 2
	memory-size 4G

	storage-engine memory
}

Environment:
Ubuntu 18.04.5
Version: Aerospike Community Edition build 5.6.0.4
Client version: aerospike-client-c-5.2.0.ubuntu18.04

aerospike_telemetry.service is marked executable

OS: CentOS 7

Aerospike version: 5.4.0.1 (code shows problem still exists in 5.7.0.10)


journalctl shows the following message about 6 times/day:
systemd[1]: Configuration file /usr/lib/systemd/system/aerospike_telemetry.service is marked executable. Please remove executable permission bits. Proceeding anyway.

line 80 of pkg/rpm/Makefile shows the problem:
install -pm 755 $(DEPTH)/as/etc/aerospike_telemetry.service $(BUILD_ROOT)/usr/lib/systemd/system/aerospike_telemetry.service

(pkg/rpm/Makefile in 5.4.0.1 is identical to 5.7.0.10)

Build errors with g++ 7.3

Building the server with debian buster and g++ 7.3.0 gives multiple errors.


OS: Debian Testing (Buster)

Aerospike version: 4.0.0.1-0-g997e3d0


base/thr_info.c: In function ‘info_get_services_x’:
base/thr_info.c:5371:35: error: ‘%d’ directive output may be truncated writing between 1 and 10 bytes into a region of size 6 [-Werror=format-truncation=]
  snprintf(strip, sizeof(strip), ":%d", best_port);
                                   ^~
base/thr_info.c:5371:33: note: directive argument in the range [0, 2147483646]
  snprintf(strip, sizeof(strip), ":%d", best_port);
                                 ^~~~~
base/thr_info.c:5371:2: note: ‘snprintf’ output between 3 and 12 bytes into a destination of size 7
  snprintf(strip, sizeof(strip), ":%d", best_port);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In file included from ../include/base/datamodel.h:52:0,
                 from geospatial/geospatial.cc:33:
../include/base/proto.h:167:16: error: flexible array member ‘as_proto_s::data’ not at end of ‘struct as_comp_proto_s’
  uint8_t  data[];
                ^
../include/base/proto.h:178:14: note: next member ‘uint64_t as_comp_proto_s::org_sz’ declared here
  uint64_t    org_sz;    // Original size of compressed data hold in 'data'
              ^~~~~~
../include/base/proto.h:176:16: note: in the definition of ‘struct as_comp_proto_s’
 typedef struct as_comp_proto_s {
                ^~~~~~~~~~~~~~~
../include/base/proto.h:167:16: error: flexible array member ‘as_proto_s::data’ not at end of ‘struct cl_msg_s’
  uint8_t  data[];
                ^
../include/base/proto.h:330:10: note: next member ‘as_msg cl_msg_s::msg’ declared here
  as_msg  msg;
          ^~~
../include/base/proto.h:328:16: note: in the definition of ‘struct cl_msg_s’
 typedef struct cl_msg_s {
                ^~~~~~~~
In file included from geospatial/geospatial.cc:33:0:
../include/base/datamodel.h:161:16: error: flexible array member ‘as_particle_s::data’ not at end of ‘struct as_bin_s’
  uint8_t  data[];
                ^
In file included from geospatial/geospatial.cc:33:0:
../include/base/datamodel.h:261:15: note: next member ‘as_particle* as_bin_s::particle’ declared here
  as_particle *particle; // for embedded particle this is value, not pointer
               ^~~~~~~~
../include/base/datamodel.h:259:8: note: in the definition of ‘struct as_bin_s’
 struct as_bin_s {
        ^~~~~~~~
../../make_in/Makefile.targets:16: recipe for target '../../target/Linux-x86_64/obj/geospatial/geospatial.o' failed

Failed build on GCC 12

OS: Arch linux

Aerospike version: tags/6.2.0.3-0-g46a1008

Client version: -


Failed build on GCC 12

In file included from cf_thread.c:27:
../include/cf_thread.h: In function ‘cf_thread_yield’:
../include/cf_thread.h:81:9: error: ‘pthread_yield’ is deprecated: pthread_yield is deprecated, use sched_yield instead [-Werror=deprecated-declarations]
   81 |         pthread_yield();
      |         ^~~~~~~~~~~~~
In file included from /usr/include/features.h:490,
                 from /usr/include/pthread.h:21,
                 from ../include/cf_thread.h:29:
/usr/include/pthread.h:479:12: note: declared here
  479 | extern int __REDIRECT_NTH (pthread_yield, (void), sched_yield)
      |            ^~~~~~~~~~~~~~
In file included from alloc.c:47:
../include/cf_thread.h: In function ‘cf_thread_yield’:
../include/cf_thread.h:81:9: error: ‘pthread_yield’ is deprecated: pthread_yield is deprecated, use sched_yield instead [-Werror=deprecated-declarations]
   81 |         pthread_yield();
      |         ^~~~~~~~~~~~~
In file included from /usr/include/features.h:490,
                 from /usr/include/malloc.h:23,
                 from ../include/enhanced_alloc.h:29,
                 from alloc.c:26:
/usr/include/pthread.h:479:12: note: declared here
  479 | extern int __REDIRECT_NTH (pthread_yield, (void), sched_yield)
      |            ^~~~~~~~~~~~~~
In file included from dns.c:29:
../include/cf_thread.h: In function ‘cf_thread_yield’:
../include/cf_thread.h:81:9: error: ‘pthread_yield’ is deprecated: pthread_yield is deprecated, use sched_yield instead [-Werror=deprecated-declarations]
   81 |         pthread_yield();
      |         ^~~~~~~~~~~~~
In file included from /usr/include/features.h:490,
                 from /usr/include/bits/libc-header-start.h:33,
                 from /usr/include/limits.h:26,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include-fixed/limits.h:203,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include-fixed/syslimits.h:7,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include-fixed/limits.h:34,
                 from ../include/dns.h:25,
                 from dns.c:23:
/usr/include/pthread.h:479:12: note: declared here
  479 | extern int __REDIRECT_NTH (pthread_yield, (void), sched_yield)
      |            ^~~~~~~~~~~~~~
cc1: all warnings being treated as errors
In function ‘hmh_compatible_template’,
    inlined from ‘hll_modify_op_union’ at base/particle_hll.c:1274:2:
base/particle_hll.c:1595:34: error: ‘*(const hll_t **)hmhs’ may be used uninitialized [-Werror=maybe-uninitialized]
 1595 |         uint8_t index_bits = hmhs[0]->n_index_bits;
      |                              ~~~~^~~

Server fails to start on Rocky Linux 8 ARM64 because asd requires that the kernel page size is 4096.

OS: Rocky Linux 8

Aerospike version: 6.4.0.11

Client version: N/A - ASD not starting succesfully


Explain your Aerospike Server issue in detail here and (if applicable) provide logs snippets, configuration, and/or reproduction instructions.

Apr 30 17:40:51 aerospike-test1-usw2.dev.brogers.aerospike.cloud asd[20133]: Apr 30 2024 17:40:51 GMT: CRITICAL (alloc): (alloc.c:489) bad kernel page size 65536, expected 4096

Segmentation fault when selecting a record

OS: Linux version 4.4.0-112-generic (buildd@lgw01-amd64-010) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5) )

Aerospike version: 3.13.0.3

Client version: Nodejs 3.3.0


We are getting a segmentation fault error, this happens when selecting a specific record. The thing is, it happens using the client as well as aql. Also is only one bin, we think it may be because is too large or corrupted but we are not sure, we can't see it.

This is the stacktrace from the nodejs client:

PID 1535 received SIGSEGV for address: 0x5f1cab5a
/home/jobs/node_modules/segfault-handler/build/Release/segfault-handler.node(+0x2cae)[0x7f741a72acae]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f741cd0c390]
/lib/x86_64-linux-gnu/libc.so.6(+0x14e168)[0x7f741ca7f168]
/home/jobs/node_modules/aerospike/build/Release/aerospike.node(+0xb03bd)[0x7f741a4e83bd]
/home/jobs/node_modules/aerospike/build/Release/aerospike.node(as_unpack_val+0x2ba)[0x7f741a4ed1da]
/home/jobs/node_modules/aerospike/build/Release/aerospike.node(+0xb55b4)[0x7f741a4ed5b4]
/home/jobs/node_modules/aerospike/build/Release/aerospike.node(as_unpack_val+0x4d4)[0x7f741a4ed3f4]
/home/jobs/node_modules/aerospike/build/Release/aerospike.node(+0xb55b4)[0x7f741a4ed5b4]
/home/jobs/node_modules/aerospike/build/Release/aerospike.node(as_unpack_val+0x4d4)[0x7f741a4ed3f4]
/home/jobs/node_modules/aerospike/build/Release/aerospike.node(+0x8955e)[0x7f741a4c155e]
/home/jobs/node_modules/aerospike/build/Release/aerospike.node(as_command_parse_bins+0x21b)[0x7f741a4a4031]
/home/jobs/node_modules/aerospike/build/Release/aerospike.node(+0x5c5f1)[0x7f741a4945f1]
/home/jobs/node_modules/aerospike/build/Release/aerospike.node(+0x739e1)[0x7f741a4ab9e1]
node[0x1449cbf]
node[0x144a320]
node[0x144ffa8]
node(uv_run+0x156)[0x143edd6]
node(_ZN4node5StartEP9uv_loop_siPKPKciS5_+0xc8d)[0x122c1bd]
node(_ZN4node5StartEiPPc+0x163)[0x1224d03]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f741c951830]
node[0x8aee41]
Segmentation fault

The namespace config:

namespace foo {
        replication-factor 2
        memory-size 31G
        default-ttl 60d
        high-water-memory-pct 65
        high-water-disk-pct 50
        stop-writes-pct 90
        conflict-resolution-policy last-update-time
        migrate-sleep 500

        storage-engine device {
                device /dev/sdb
                device /dev/sdc
                device /dev/sdd

                scheduler-mode noop
                write-block-size 128K
       }
}

The service config

service {
        ...
        service-threads 23
        transaction-queues 23
        transaction-threads-per-queue 4
        proto-fd-max 70000
        migrate-max-num-incoming 1
        ...
}

The aerospike logs are empty

What is the difference between the enterprise version of the source code?

Please direct the following to our community forum:

  • general questions
  • help requests
  • feature requests
  • non Aerospike Server issues

The issues submitted here should be Aerospike Server code related. Examples include:

  • crashes (please provide stack trace from logs)
  • bugs (not behaving as expected/documented)
  • code quality

OS: Put your operating system here. For example: "Ubuntu 16.10", "CentOS 7", "Debian 8" etc.

Aerospike version: Put your Aerospike release version or git describe --long --all output here. For example: "3.15.0.1, heads/master-0-g450aee1"

Client version: *Put which client and the version of the client you are using (if applicable) here. For example: "Java 4.0.8, C 4.2.0"


Explain your Aerospike Server issue in detail here and (if applicable) provide logs snippets, configuration, and/or reproduction instructions.

error creating fabric published endpoint list

I got a CRITICAL log several times with same configuration while most of runs are correct.

Jun 07 2021 16:48:02 GMT: WARNING (fabric): (fabric.c:747) no IPv4 addresses configured for fabric
Jun 07 2021 16:48:02 GMT: CRITICAL (fabric): (fabric.c:373) error creating fabric published endpoint list
Jun 07 2021 16:48:02 GMT: WARNING (as): (signal.c:218) SIGUSR1 received, aborting Aerospike Community Edition build 5.6.0.4 os ubuntu18.04
Jun 07 2021 16:48:02 GMT: WARNING (as): (log.c:630) stacktrace: registers: rax 0000000000000000 rbx 000000000000000a rcx 00007f81356a4817 rdx 0000000000000000 rsi 00007ffd1fe467a0 rdi 0000000000000002 rbp 0000000000000003 rsp 00007ffd1fe467a0 r8 0000000000000000 r9 00007ffd1fe467a0 r10 0000000000000008 r11 0000000000000246 r12 0000000000000004 r13 0000000000000004 r14 0000000000000a00 r15 000055af36b8c9e8 rip 00007f81356a4817
Jun 07 2021 16:48:02 GMT: WARNING (as): (log.c:643) stacktrace: found 10 frames: 0x18cd64 0xd1f1b 0x7f81356a4980 0x7f81356a4817 0x18c76f 0x11d517 0x55275 0x53cf3 0x7f813435abf7 0x5445a offset 0x55af365ec000
Jun 07 2021 16:48:02 GMT: WARNING (as): (log.c:653) stacktrace: frame 0: /home/zyh/aerospike-server/target/Linux-x86_64/bin/asd(cf_log_stack_trace+0x115) [0x55af36778d64]
Jun 07 2021 16:48:02 GMT: WARNING (as): (log.c:653) stacktrace: frame 1: /home/zyh/aerospike-server/target/Linux-x86_64/bin/asd(as_sig_handle_usr1+0x72) [0x55af366bdf1b]
Jun 07 2021 16:48:02 GMT: WARNING (as): (log.c:653) stacktrace: frame 2: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7f81356a4980]
Jun 07 2021 16:48:02 GMT: WARNING (as): (log.c:653) stacktrace: frame 3: /lib/x86_64-linux-gnu/libpthread.so.0(raise+0xc7) [0x7f81356a4817]
Jun 07 2021 16:48:02 GMT: WARNING (as): (log.c:653) stacktrace: frame 4: /home/zyh/aerospike-server/target/Linux-x86_64/bin/asd(cf_log_write_no_return+0x93) [0x55af3677876f]
Jun 07 2021 16:48:02 GMT: WARNING (as): (log.c:653) stacktrace: frame 5: /home/zyh/aerospike-server/target/Linux-x86_64/bin/asd(as_fabric_init+0x223) [0x55af36709517]
Jun 07 2021 16:48:02 GMT: WARNING (as): (log.c:653) stacktrace: frame 6: /home/zyh/aerospike-server/target/Linux-x86_64/bin/asd(as_run+0x444) [0x55af36641275]
Jun 07 2021 16:48:02 GMT: WARNING (as): (log.c:653) stacktrace: frame 7: /home/zyh/aerospike-server/target/Linux-x86_64/bin/asd(main+0x9) [0x55af3663fcf3]
Jun 07 2021 16:48:02 GMT: WARNING (as): (log.c:653) stacktrace: frame 8: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f813435abf7]
Jun 07 2021 16:48:02 GMT: WARNING (as): (log.c:653) stacktrace: frame 9: /home/zyh/aerospike-server/target/Linux-x86_64/bin/asd(_start+0x2a) [0x55af3664045a]

Configuration is

# Aerospike database configuration file for deployments using mesh heartbeats.

service {
	user zyh
	group zyh
	paxos-single-replica-limit 0 # Number of nodes where the replica count is automatically reduced to 1.
	pidfile /home/zyh/distributed-system-test/aerospike_test/bin/run/asd3.pid
	proto-fd-max 15000
	work-directory run/work3
	run-as-daemon false
}

mod-lua {
	user-path run/work3/usr/udf/lua
}

logging {
	# Log file must be an absolute path.
	file /home/zyh/distributed-system-test/aerospike_test/bin/run/aerospike3.log {
		context any info
	}
}

network {
	service {
		address 127.0.0.1
		port 2002
	}

	heartbeat {
		mode mesh
		port 5004 # Heartbeat port for this node.

		# List one or more other nodes, one ip-address & port per line:
		# mesh-seed-address-port 127.0.0.1 3000
		mesh-seed-address-port 127.0.0.1 5005
		mesh-seed-address-port 127.0.0.1 5006

		interval 250
		timeout 10
	}

	fabric {
		port 4002
	}

	info {
		port 3002
	}
}

namespace test {
	replication-factor 2
	memory-size 4G

	storage-engine memory
}

Environment:

Ubuntu 18.04.5
Version: Aerospike Community Edition build 5.6.0.4

Vault integration breaks with 5.6.0.3


Aerospike version: Put your Aerospike release version or git describe --long --all output here. For example: "3.15.0.1, heads/master-0-g450aee1"

Docker 5.6.0.3

Client version: *Put which client and the version of the client you are using (if applicable) here. For example: "Java 4.0.8, C 4.2.0"

$ grep aerospike go.mod
        github.com/aerospike/aerospike-client-go v3.1.1+incompatible

Explain your Aerospike Server issue in detail here and (if applicable) provide logs snippets, configuration, and/or reproduction instructions.

The Vault tests for validating Aerospike as a storage backend started failing today. These tests use the "latest" docker image tag, and they fail in the same way when I tried changing the tag to be 5.6.0.3. Changing the tag to 5.5.0.10 makes the tests pass again, as I did in hashicorp/vault#11677.

I'm not familiar with the integration internals, so I'm afraid I can't give much more detail than this. An example failure is here: https://app.circleci.com/pipelines/github/hashicorp/vault/17659/workflows/d4f404a5-b4b5-4aac-996f-d77007fe2b64/jobs/208122/tests#failed-test-0

which contains:

=== RUN   TestAerospikeBackend
    aerospike_test.go:31: initial list failed: Node BB9040012AC4202 172.18.0.4:3000: Unsupported Server Feature
--- FAIL: TestAerospikeBackend (8.71s)

The integration is here: https://github.com/hashicorp/vault/blob/master/physical/aerospike/aerospike.go
And the test failure is here: https://github.com/hashicorp/vault/blob/master/sdk/physical/testing.go#L15

dirty read if the connection is not stable?

I see some logs about reading dirty data, and some logs about connection error.

Does the default read operation of client allow dirty read?

OS: Ubuntu 18.04.5
Version: Aerospike Community Edition build 5.6.0.5
Client: aerospike-client-c-5.2.0.ubuntu18.04.x86_64

Repo missing old branches and/or tags

It appears the aerospike-server github repo has recently been reset to some sort of blank slate. All the old branches and/or tags for previous versions are missing. The commit history is gone as well. This is problematic for those of us running older versions of Aerospike before version 4. Right now I can't upgrade past version 3.13 because I have to make code changes in order to run Aerospike server on my hardware set up. See:
https://discuss.aerospike.com/t/fatal-error-too-many-interfaces/3581

Normally for open source projects, source tar ball is distributed with each binary release. However, I can't seem to find such a thing. For example, on the download page for 3.13.0.3:
https://www.aerospike.com/download/server/3.13.0.3/

There is a link to github to get the source. However, the source for that version is not available via this repository currently.

Document supported hardware architectures

Hi!

I am reading the documentation and cannot find the information about supported combinations of OS-Hardware Architcture. Is there any? I only see a set of prebuilt binaries per release. But for the users will be much easier to get the information about the support status (and different support levels for each) from the documentation.

Maybe some kind of table or something like that will be a good start.

Thanks in advance!

Missing cf_queue_priority.h while make aerospike-server-master in CentOS7.5

Please direct the following to our community forum:

  • help requests

The issues submitted here should be Aerospike Server code related. Examples include:

  • crashes (please provide stack trace from logs)
  • bugs (not behaving as expected/documented)
  • code quality

OS: "CentOS 7.5"

Aerospike version: * " 5.2.0.32"*


Hello,
An error came out when I exec make in Aerospike-server-master,../include/sindex/job_manager.h:30:42: fatal error: citrusleaf/cf_queue_priority.h: No such file or directory.
I download Aerospike-server-masterand all modules from GitHub ,when I Run make command,one error came out:
make -C as make[1]: Entering directory/opt/aerospike-server-master/as'
make -C src all
make[2]: Entering directory /opt/aerospike-server-master/as/src' cc -gdwarf-4 -g3 -O3 -fno-common -fno-strict-aliasing -Wall -D_FILE_OFFSET_BITS=64 -std=gnu99 -D_REENTRANT -D_GNU_SOURCE -DENHANCED_ALLOC -march=nocona -MMD -Werror -DMARCH_x86_64 -D__FILENAME__="job_manager.c" -o ../../target/Linux-x86_64/obj/sindex/job_manager.o -c -I../include -I../../target/Linux-x86_64/gen -I/opt/aerospike-server-master/cf/include -I/opt/aerospike-server-master/ai/include -I/opt/aerospike-server-master/modules/common/target/Linux-x86_64/include -I/opt/aerospike-server-master/modules/jansson/src -I/opt/aerospike-server-master/modules/mod-lua/target/Linux-x86_64/include -I/opt/aerospike-server-master/modules/s2-geometry-library/geometry -I/opt/aerospike-server-master/modules/luajit/src sindex/job_manager.c In file included from sindex/job_manager.c:27:0: ../include/sindex/job_manager.h:30:42: fatal error: citrusleaf/cf_queue_priority.h: No such file or directory #include "citrusleaf/cf_queue_priority.h" ^ compilation terminated. make[2]: *** [../../target/Linux-x86_64/obj/sindex/job_manager.o] Error 1 make[2]: Leaving directory /opt/aerospike-server-master/as/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory /opt/aerospike-server-master/as' make: *** [all] Error 2
Is there someone can help me??? :D

Clarification about truncation with specific LUT

OS: Put your operating system here. For example: "Ubuntu 18.04 LTS", "CentOS 7", "Debian 9" etc.

Aerospike version: Put your Aerospike release version or git describe --long --all output here. For example: "3.15.0.1, heads/master-0-g450aee1"

Aerospike CE Docker version 5.6.0.3

Client version: *Put which client and the version of the client you are using (if applicable) here. For example: "Java 4.0.8, C 4.2.0"

Go v4.5+


Explain your Aerospike Server issue in detail here and (if applicable) provide logs snippets, configuration, and/or reproduction instructions.

A bit confused with truncate using specific LUT

I tried to specify LUT in Go i.e.

time.Now().Add(3 * time.Second)

It doesn't truncate anything.

From log it was written like this

command lut 1680540851534065369 (Apr 03 2023 16:54:11 GMT) is in the future

Checked in truncate.c, apparently LUT cannot > now ?

https://github.com/aerospike/aerospike-server/blob/master/as/src/base/truncate.c#L182

Too many addresses for interface kube-ipvs0

OS: CentOS 7

Aerospike version: Enterprise 4.8.0.5

Client version: go v2.8.2


I'm running Aerospike Enterprise on k8s with hostNetworking enabled. When TPS (50000/s) got really high, Aerospike server keeps crash with Too many addresses for interface kube-ipvs0

I searched the codebase, and find this

cf_crash(CF_SOCKET, "Too many addresses for interface %s", entry->name);

Is there any way to skip the check or build Enterprise version by myself?

Thanks.

Mar 10 2020 17:02:09 GMT: FAILED ASSERTION (socket): (socket.c:2247) Too many addresses for interface kube-ipvs0
Mar 10 2020 17:02:09 GMT: WARNING (as): (signal.c:213) SIGUSR1 received, aborting Aerospike Enterprise Edition build 4.8.0.5 os debian9
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1178) stacktrace: registers: rax 0000000000000000 rbx 000000000000000a rcx ffffffffffffffff rdx 0000000000000000 rsi 00007ffe9a489640 rdi 0000000000000002 rbp 00007ffe9a489c70 rsp 00007ffe9a4896b8 r8 0000000000000000 r9 00007ffe9a489640 r10 0000000000000008 r11 0000000000000246 r12 0000000000000006 r13 0000000000000000 r14 0000000000000070 r15 000055a75b42d1c0 rip 00007f5f46fabfbf
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1192) stacktrace: found 17 frames: 0x3391bb 0x10fd8a 0x7f5f46fac0e0 0x7f5f46fabfbf 0x3371f7 0x34c04e 0x34a1d3 0x34b337 0x34ffa5 0x35043b 0x18e2cc 0x18e4a4 0x18f601 0x13cf50 0x81a3e 0x7f5f45d592e1 0x823da offset 0x55a7546d3000
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 0: asd(cf_fault_print_signal_context+0x10e) [0x55a754a0c1bb]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 1: asd(as_sig_handle_usr1+0x5c) [0x55a7547e2d8a]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 2: /lib/x86_64-linux-gnu/libpthread.so.0(+0x110e0) [0x7f5f46fac0e0]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 3: /lib/x86_64-linux-gnu/libpthread.so.0(raise+0xcf) [0x7f5f46fabfbf]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 4: asd(cf_fault_event+0x231) [0x55a754a0a1f7]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 5: asd(+0x34c04e) [0x55a754a1f04e]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 6: asd(+0x34a1d3) [0x55a754a1d1d3]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 7: asd(+0x34b337) [0x55a754a1e337]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 8: asd(cf_inter_is_inter_name+0x2a) [0x55a754a22fa5]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 9: asd(cf_ip_addr_is_dns_name+0x9) [0x55a754a2343b]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 10: asd(+0x18e2cc) [0x55a7548612cc]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 11: asd(+0x18e4a4) [0x55a7548614a4]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 12: asd(as_service_list_init+0x9b) [0x55a754862601]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 13: asd(as_info_init+0x21c9) [0x55a75480ff50]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 14: asd(main+0x38e) [0x55a754754a3e]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 15: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f5f45d592e1]
Mar 10 2020 17:02:09 GMT: WARNING (as): (fault.c:1200) stacktrace: frame 16: asd(_start+0x2a) [0x55a7547553da]
Mar 10 2020 17:02:09 GMT: FAILED ASSERTION (socket): (socket.c:2247) Too many addresses for interface kube-ipvs0

Server repeatedly crashes during "reading device" after restart

OS: Ubuntu 18.04.6 LTS 4.15.0-144-generic

Aerospike version: 4.6.0.21

I stopped the server for a planned maintenance cleanly:

Sep 27 2022 20:41:19 GMT: INFO (as): (signal.c:198) SIGTERM received, shutting down Aerospike Community Edition build 4.6.0.21 os ubuntu18.04
Sep 27 2022 20:41:19 GMT: INFO (as): (as.c:446) initiating clean shutdown ...
Stopping Aerospike Server...
Sep 27 2022 20:41:19 GMT: INFO (storage): (storage.c:826) {bapi} partitions shut down
Sep 27 2022 20:41:19 GMT: INFO (storage): (storage.c:831) {bapi} storage devices flushed
Sep 27 2022 20:41:19 GMT: INFO (storage): (storage.c:837) {test} storage-engine memory - nothing to do
Sep 27 2022 20:41:19 GMT: INFO (as): (as.c:451) finished clean shutdown - exiting
Stopped Aerospike Server.

Later I started the server and got this:

...
Sep 27 2022 23:10:27 GMT: INFO (drv_ssd): (drv_ssd.c:2820) device /dev/disk/by-id/nvme-someserial2-part15: reading device to load index
Sep 27 2022 23:10:27 GMT: INFO (drv_ssd): (drv_ssd.c:2820) device /dev/disk/by-id/nvme-someserial2-part16: reading device to load index
Sep 27 2022 23:10:32 GMT: INFO (drv_ssd): (drv_ssd.c:3623) {bapi} loaded 4299960 objects, /dev/disk/by-id/nvme-someserial1-part1 0%, /dev/disk/by-id/nvme-someserial1-part2 0%, /dev/disk/by-id/nvme-someserial1-part3 0%, /dev/disk/by-id/nvme-someserial1-part4 0%, /dev/disk/by-id/nvme-someserial1-part5 0%, /dev/disk/by-id/nvme-someserial1-part6 0%, /dev/disk/by-id/nvme-someserial1-part7 0%, /dev/disk/by-id/nvme-someserial1-part8 0%, /dev/disk/by-id/nvme-someserial1-part9 0%, /dev/disk/by-id/nvme-someserial1-part10 0%, /dev/disk/by-id/nvme-someserial1-part11 0%, /dev/disk/by-id/nvme-someserial1-part12 0%, /dev/disk/by-id/nvme-someserial1-part13 0%, /dev/disk/by-id/nvme-someserial1-part14 0%, /dev/disk/by-id/
*** stack smashing detected ***: <unknown> terminated
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:108) SIGABRT received, aborting Aerospike Community Edition build 4.6.0.21 os ubuntu18.04
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: registers: rax 0000000000000000 rbx 00007ffd7ef4fe80 rcx 00007f5c3a7ebfb7 rdx 0000000000000000 rsi 00007ffd7ef4fbe0 rdi 0000000000000002 rbp 00007ffd7ef50010 rsp 00007ffd7ef4fbe0 r8 0000000000000000 r9 00007ffd7ef4fbe0 r10 0000000000000008 r11 0000000000000246 r12 00007ffd7ef4fe80 r13 0000000000001000 r14 0000000000000000 r15 0000000000000030 rip 00007f5c3a7ebfb7
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: found 12 frames: 0x55fff5a0033c 0x7f5c3bb18980 0x7f5c3a7ebfb7 0x7f5c3a7ed921 0x7f5c3a836967 0x7f5c3a8e1b61 0x7f5c3a8e1b22 0x55fff5a88698 0x55fff5a8a841 0x55fff5993c1f 0x7f5c3a7cebf7 0x55fff599460a offset 0x55fff5944000
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: frame 0: /usr/bin/asd(as_sig_handle_abort+0x145) [0x55fff5a0033c]
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: frame 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7f5c3bb18980]
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: frame 2: /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7) [0x7f5c3a7ebfb7]
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: frame 3: /lib/x86_64-linux-gnu/libc.so.6(abort+0x141) [0x7f5c3a7ed921]
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: frame 4: /lib/x86_64-linux-gnu/libc.so.6(+0x89967) [0x7f5c3a836967]
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: frame 5: /lib/x86_64-linux-gnu/libc.so.6(+0x134b61) [0x7f5c3a8e1b61]
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: frame 6: /lib/x86_64-linux-gnu/libc.so.6(+0x134b22) [0x7f5c3a8e1b22]
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: frame 7: /usr/bin/asd(as_storage_namespace_destroy_ssd+0) [0x55fff5a88698]
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: frame 8: /usr/bin/asd(as_storage_load+0x9f) [0x55fff5a8a841]
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: frame 9: /usr/bin/asd(main+0x32f) [0x55fff5993c1f]
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: frame 10: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f5c3a7cebf7]
Sep 27 2022 23:10:32 GMT: WARNING (as): (signal.c:110) stacktrace: frame 11: /usr/bin/asd(_start+0x2a) [0x55fff599460a]
aerospike.service: Main process exited, code=dumped, status=6/ABRT
aerospike.service: Failed with result 'core-dump'.

Now it repeats in a loop, with slightly varying "loaded N objects".
I recall the last time it happened for me on 4.5.0.24 version (previous version that I used) - I had several nodes, and all of them got corrupted this way after restarts - probably the corruption got replicated somehow?.. (or it just happened on all of them in a same way) I lost all the data that way and had a heavy downtime, that was very painful.
What should I do now? I can try to zero out the disks and try to resync the replica from scratch - but I fear that if other replicas got corrupted again somehow, they theoretically can crash during the scan so I will get majorly effed up again.
Is there some way to safely check the data integrity on this and other machines without taking down the servers? Is there something useful I can extract from coredump? How to do it?

single-bin in udf can't get bin data

bugs


OS: Docker Image

Aerospike version: 5.2.0.6"

Client version: *Python 4.0.0"


  • namespace-config
single-bin: true
data-in-index: true
  • define udf
function get(rec)
  local m = map()
  names = record.bin_names(rec)
  for i, name in ipairs(names) do
    m[name] = rec[name]
  end
  return m
end

function getval(rec)
  return rec['']
end
  • invoke udf
as_client.put(key, {"": 1})
as_client.apply(key, "xapi", "get", [])
as_client.apply(key, "xapi", "getval", [])

return

{}
''

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.