biokoda / actordb Goto Github PK
View Code? Open in Web Editor NEWActorDB distributed SQL database
License: Mozilla Public License 2.0
ActorDB distributed SQL database
License: Mozilla Public License 2.0
Any ideas? Here's the log.
actordb console
config is OK
-config /etc/actordb/app.config -args_file /etc/actordb/vm.args -vm_args /etc/actordb/vm.args
!!!!
!!!! WARNING: ulimit -n is 1024; 32768 is the recommended minimum.
!!!!
Exec: /usr/lib/actordb/erts-7.2/bin/erlexec -boot /usr/lib/actordb/releases/0.10.21/actordb -config /etc/actordb/app.config -args_file /etc/actordb/vm.args -vm_args /etc/actordb/vm.args -pa /usr/lib/actordb/lib/actordb-patches -- console
Root: /usr/lib/actordb
Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:3] [hipe] [kernel-poll:true]
[os_mon] memory supervisor port (memsup): Erlang has closed
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed
{"Kernel pid terminated",application_controller,"{application_start_failure,actordb_core,{bad_return,{{actordb_core,start,[normal,[]]},{'EXIT',{undef,[{actordb_conf,paths,[],[]},{actordb_core,prestart1,1,[{file,\"src/actordb_core.erl\"},{line,206}]},{actordb_core,start,2,[{file,\"src/actordb_core.erl\"},{line,307}]},{application_master,start_supervisor,3,[{file,\"application_master.erl\"},{line,327}]},{application_master,start_the_app,5,[{file,\"application_master.erl\"},{line,309}]},{application_master,start_it_new,7,[{file,\"application_master.erl\"},{line,295}]}]}}}}}"}
Crash dump is being written to: erl_crash.dump...done
Kernel pid terminated (application_controller) ({application_start_failure,actordb_core,{bad_return,{{actordb_core,start,[normal,[]]},{'EXIT',{undef,[{actordb_conf,paths,[],[]},{actordb_core,prestar
I was just in the test.I run 6 nodes ,5 in a group for write data. when I stop one node, most of the time it still work ,but the response time is very long,even take 10 seconds. Then I try to stop another node, this cluster works but the response time still very long.Is this a bug? Or shouldn't I take one node just for connection but not join in the group. This is my node.yaml:
nodes:
hello -
is there a forum i should be asking questions? it does not feel right to ask questions in the github "issues" area
does actorDB support the mariadb(mysql) JSON datatype ?
thank you very much
On building I get the error
./priv/mkconsole.escript
escript: exception error: no match of right hand side value error
I made a Dockerfile the reproduces the error. https://github.com/atomi/dockerfiles/blob/master/alpine-x86_64/actordb/Dockerfile
It appears that distributed actor transactions use a transaction manager and two-phase commit protocol. This is a very nice feature of actordb.
However it is well known that 2-phase commit can suffer from deadlock.
Does actordb provide deadlock detection and recovery?
Hi all,
I seem to have trouble getting my Go code working with ActorDB. It returns {error,no_actor_defined}
when requesting a connection.
Go code
package main
import (
"fmt"
"database/sql"
_ "github.com/go-sql-driver/mysql"
)
func main() {
DB, e := sql.Open("mysql", ":@(127.0.0.1:33307)/")
if e != nil {
panic(e)
}
if e := DB.Ping(); e != nil {
panic(e) // Line 14 where it crashes
}
if _, e := DB.Exec("ACTOR type1(payments) CREATE"); e != nil {
panic(e)
}
fmt.Println("Success!")
}
Crash report
panic: Error 1: Error-Id: "0d47f0b2-48fa-26b3-2092-397287fe620e"
Exception: {error,no_actor_defined}
goroutine 1 [running]:
main.main()
.../main.go:14 +0xe7
goroutine 5 [chan receive]:
database/sql.(*DB).connectionOpener(0xc208054780)
/usr/local/go/src/database/sql/sql.go:589 +0x4c
created by database/sql.Open
/usr/local/go/src/database/sql/sql.go:452 +0x31c
I can image you guy's don't write in Go everyday so I was hoping you could hint me how I can see where it goes wrong from the actordb side of things?
Folks,
I wanted to give it a try to actordb from Golang, I'm using the following snippet to connect to the database:
package main
import "database/sql"
import _ "github.com/go-sql-driver/mysql"
import "fmt"
func main() {
db, err := sql.Open("mysql", "myuser:mypass@tcp(127.0.0.1:33307)/actordb")
if err != nil {
fmt.Println("Failed to open mysql connection with error", err.Error())
}
defer db.Close()
stmtOut, err := db.Prepare("ACTOR story(clientA) CREATE; SELECT * FROM entry; c")
if err != nil {
panic(err.Error())
}
defer stmtOut.Close()
}
I keep getting error no_actor_defined
:
panic: Error 1: Error-Id: "0d47f0b2-48fa-26b3-2092-397287fe620e"
Exception: {error,no_actor_defined}
exit status 2
shell returned 1
I was following this example that shows how to connect to a regular mysql database, I suspect the problem might be in sql.Open("mysql", "myuser:mypass@tcp(127.0.0.1:33307)/actordb")
, I don`t know how to pass the actor type and actor name in the URL:
Any ideas?
My guess is that 0.9-1 is the newer version and not 0.9pre8-1?
[root@server~]# rpm -Uvh https://s3-eu-west-1.amazonaws.com/biokoda/actordb-0.9-1.el6.x86_64.rpm
Retrieving https://s3-eu-west-1.amazonaws.com/biokoda/actordb-0.9-1.el6.x86_64.rpm
Preparing... ########################################### [100%]
package actordb-0.9pre8-1.el6.x86_64 (which is newer than actordb-0.9-1.el6.x86_64) is already installed
[root@server ~]#
hello - from what little I have seen (which is practically nothing), it appears that actorDB and https://deepstream.io just might play very nicely together.
so far we have successfully combined deepstream with both mariaDB and cockroachDB, however I an thinking that actorDB might support KV as well as binary data better than the other two we have done.
my long-term goal (or delusional dream) is to have some sort of framework using deepstream and a back-end data storage that is KV SQL based.
any thoughts before i approach this?
I read over the actorDB docs and have a few questions:
Hex.pm is the package manager for the Erlang ecosystem. Uploading to Hex.pm would allow people to more easily discover this project.
https://hex.pm/packages?search=actordb
Use Rebar3 to publish. See publishing guide on http://www.rebar3.org/docs/publishing-packages
Ubuntu 14.04 x64
this is an output in log files
crash.log
2016-08-01 21:45:46 =CRASH REPORT====
crasher:
initial call: thrift_socket_server:acceptor_loop/1
pid: <0.9142.0>
registered_name: []
exception exit: {{error,closed},[{thrift_socket_server,acceptor_loop,1,[{file,"src/thrift_socket_server.erl"},{line,203}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}
ancestors: [<0.149.0>,<0.101.0>]
messages: []
links: []
dictionary: []
trap_exit: false
status: running
heap_size: 376
stack_size: 27
reductions: 120
neighbours:
2016-08-01 21:45:51 =ERROR REPORT====
Thrift error: func=undefined, reason=no_binary_protocol_version
error.log
2016-08-01 21:45:46.501 [error] <0.9142.0> Undefined Undefined CRASH REPORT Process <0.9142.0> with 0 neighbours exited with reason: {error,closed} in thrift_socket_server:acceptor_loop/1 line 203
2016-08-01 21:45:51.386 [error] <0.161.0> Undefined Undefined Thrift error: func=undefined, reason=no_binary_protocol_version
2016-08-01 21:46:20.154 [error] <0.203.0> Undefined Undefined Thrift error: func=undefined, reason=no_binary_protocol_version
This looks like a really interesting project, but I'd like to see a tutorial that covers transitioning through from 1 node to several, along with a transaction across actors. My use case would be having at least two actors that have (user, balance), and the ability to transfer amounts between users on different actors.
I'd like to know whether this can be done without downtime.
I think a tutorial like this would make things much clearer to me.
Hey!
I want to give actordb a shot on OS X. I'm using
I've build the thing like so:
$ git clone ...
$ cd actordb
$ make
And run it like this:
$ startdev.sh
Then I want to init with
$ ./actordbctrl init
And I get this...
ActorDB not running
So at this point I created lovely logs that I've attached to this issue report.
$ ./startdev.sh &> start_log.txt
$ cat start_log.txt
Eshell V5.10.3 (abort with ^G)
([email protected])1> WARNING: Lager's console backend is incompatible with the 'old' shell, not enabling it
([email protected])1>
=INFO REPORT==== 24-Jan-2014::17:14:52 ===
alarm_handler: {set,
{lager_console_backend,
"WARNING: Lager's console backend is incompatible with the 'old' shell, not enabling it"}}
([email protected])1> Starting bkdcore
([email protected])1> Application params [{rpcport,4380},
{statepath,"var"},
{cfgfiles,[{"schema.yaml",
[{autoload,true},
{mod,actordb_schema},
{preload,{actordb_util,parse_cfg_schema,[]}},
{onload,{actordb,schema_changed,[]}}]}]},
{name,<<"testnd">>},
{autocompile,["bkdcore","actordb_core"]},
{etc,none},
{{rpc,<<"testnd">>},
<<87,51,13,132,228,183,86,20,188,32,116,254,41,212,254,
163,84,210,149,147,93,55,108,44,107,140,200,162,173,
248,216,134,108,63,218,64,247,87,117,187>>},
{included_applications,[]}]
([email protected])1> Adding path "/Users/otobrglez/Projects/actordb/deps/actordb_core/ebin"
([email protected])1> Adding path "/Users/otobrglez/Projects/actordb/deps/bkdcore/ebin"
([email protected])1> Adding path "/Users/otobrglez/Projects/actordb/deps/distreg/ebin"
([email protected])1> Adding path "/Users/otobrglez/Projects/actordb/deps/esqlite/ebin"
([email protected])1> Adding path "/Users/otobrglez/Projects/actordb/deps/goldrush/ebin"
([email protected])1> Adding path "/Users/otobrglez/Projects/actordb/deps/lager/ebin"
([email protected])1> Adding path "/Users/otobrglez/Projects/actordb/deps/lager_syslog/ebin"
([email protected])1> Adding path "/Users/otobrglez/Projects/actordb/deps/ranch/ebin"
([email protected])1> Adding path "/Users/otobrglez/Projects/actordb/deps/syslog/ebin"
([email protected])1> Adding path "/Users/otobrglez/Projects/actordb/deps/yamerl/ebin"
([email protected])1> init changecheck
([email protected])1>
BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
(v)ersion (k)ill (D)b-tables (d)istribution
Can u give me a little help here? :)
Thanks and have a nice day!
I am interested in the key-value functionality, but I don't understand what actordb provides from reading the docs. Particularly I'm curious about how sharding and the key-value functionality interact.
For a kv actor, are keys kept in sorted order, so range queries are supported?
What happens if my key-value actor needs to expand beyond a single shard?
How does sharding and rebalancing happen in a key-value actor -- is balancing automatic? If not, could you give an example of how it should be done.
I see example create table commands for kv actor, but a fuller worked example with selects and updates and inserts would be helpful.
I don't understand the sub-table needing a foreign key to the main kv table -- perhaps an example would help here too. I didn't understand the purpose of splitting a key into master and sub-table.
Thank you.
I'm learning all about thrift and actordb and I'm trying to get a basic INSERT going. Here is the golang code I have
package main
import (
"atomi/go-actordb/actordb" // gen-go
"fmt"
"net"
"os"
"git.apache.org/thrift.git/lib/go/thrift"
)
// Transport: Not-framed TCP
// Protocol: Binary
func main() {
var protocolFactory thrift.TProtocolFactory
protocolFactory = thrift.NewTBinaryProtocolFactoryDefault()
var transportFactory thrift.TTransportFactory
transportFactory = thrift.NewTTransportFactory()
var transport thrift.TTransport
var err error
transport, err = thrift.NewTSocket(net.JoinHostPort("127.0.0.1", "33306"))
if err != nil {
fmt.Fprintf(os.Stderr, "NewTSocket: %s\n", err)
os.Exit(1)
}
transport = transportFactory.GetTransport(transport)
if err = transport.Open(); err != nil {
fmt.Fprintf(os.Stderr, "transport.Open(): %s\n", err)
os.Exit(1)
}
var clientFactory *actordb.ActordbClient
clientFactory = actordb.NewActordbClientFactory(transport, protocolFactory)
temp, err := clientFactory.Login("myuser", "mypass")
if err != nil {
fmt.Fprintf(os.Stderr, "clientFactor.Login: %s\n", err)
os.Exit(1)
}
defer transport.Close()
r1, e1 := clientFactory.ExecSql("INSERT INTO asdf (txt) Values ('rwrinf')")
fmt.Println("clientFactory.ExecSql():", r1, e1)
fmt.Printf("%v\n", temp.GetSuccess())
}
I can login and connect to a single running actordb but it fails on clientFactory.ExecSql <nil> InvalidRequestException({Code:Error Info:})
I'm hoping to test multiple connections and multiple instances I couldn't find any documentation on thrift/actordb specific api (I've just been going off the generated code) any advice would be appreciated. I can help with a blog post or two about my experience for others interested.
Thanks.
Is your SQLite build enabled for loading custom extension library (via load_extension call)? If so, what would be the right way to include custom library (maybe as a part of data schema)?
How do you deal with node failure? How do you retire a node, if you want to remove it from the cluster?
Are there guidelines for choosing actor types when users share data?
In the following scenarios, the actor breakdowns I come up with all require selecting data from multiple actors and manually joining it in my app, rather than relying on SQL operations inside individual actors. That sounds like it violates the intent of ActorDB. Alternatively, using extensive denormalization, which is error-prone and possibly storage-heavy.
How could these be elegantly solved in ActorDB?
When using show schema
either through actordb_console or through the MySQL interface, the returned sql
fields are prepended with the $
character.
For example:
actordb> s
**********************************************************************
sql type |
----------------------------------------------------------------------
$CREATE TABLE tab (id INTEGER PRIMARY KEY, txt TEXT); type1 |
----------------------------------------------------------------------
Is this intentional?
Hi,
can I use ActorDB with Phoenix 1.2.1 , Ecto 2.1 and Phoenix_Ecto 3.1 and do you have an example?
The example I found is for Ecto 0.9 and I couldn't get ActorDB it working on a new Phoenix installation.
I came across ActorDB and it seems like a very useful alternative to the existing options. I'm planning on playing a bit with it and testing it out.
I've gone through the documentation and one thing that's not clear is what kind of effort will be required to manage and administer an installation of ActorDB that has, let's say, 5 clusters with 5 nodes each.
It would be great if these questions could be answered from the perspective of a very small team (< 5) of developers.
I've been reading the docs and the other issues here and here's my understanding so far:
Another way to express my understanding is:
Is this understanding accurate or have I misunderstood something?
Through the Docker start actordb found that once the engine closed, restart a new Container and map the data file found that actordb can not be loaded into the data file, I have modified the db data host, but he still can not.
I've downloaded the OS X build and when trying to start actordb, I get this message:
!!!!
!!!! WARNING: ulimit -n is 7168; 32768 is the recommended minimum.
!!!!
actordb failed to start within 15 seconds,
see the output of 'actordb console' for more information.
If you want to wait longer, set the environment variable
WAIT_FOR_ERLANG to the number of seconds to wait.
actordb console
output is this:
config is OK
-config /Users/selvek/Downloads/actordb-0.10.20/bin/../etc/app.config -args_file /Users/selvek/Downloads/actordb-0.10.20/bin/../etc/vm.args -vm_args /Users/selvek/Downloads/actordb-0.10.20/bin/../etc/vm.args
!!!!
!!!! WARNING: ulimit -n is 7168; 32768 is the recommended minimum.
!!!!
Exec: /Users/selvek/Downloads/actordb-0.10.20/bin/../erts-7.3/bin/erlexec -boot /Users/selvek/Downloads/actordb-0.10.20/bin/../releases/0.10.20/actordb -config /Users/selvek/Downloads/actordb-0.10.20/bin/../etc/app.config -args_file /Users/selvek/Downloads/actordb-0.10.20/bin/../etc/vm.args -vm_args /Users/selvek/Downloads/actordb-0.10.20/bin/../etc/vm.args -pa /Users/selvek/Downloads/actordb-0.10.20/bin/../lib/actordb-patches -- console
Root: /Users/selvek/Downloads/actordb-0.10.20/bin/..
Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:3] [hipe] [kernel-poll:true]
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed
{"Kernel pid terminated",application_controller,"{application_start_failure,bkdcore,{bad_return,{{bkdcore_app,start,[normal,[]]},{'EXIT',{undef,[{crypto,hmac,[sha256,<<1,34,42,54,243,4,35,3,123,5,234,5,0,0,0,1>>,<<\"node1actordb\">>],[]},{butil,cipher,6,[{file,\"src/butil.erl\"},{line,3022}]},{butil,hash,4,[{file,\"src/butil.erl\"},{line,3018}]},{bkdcore,rpccookie,1,[{file,\"src/bkdcore.erl\"},{line,168}]},{bkdcore_app,start,2,[{file,\"src/bkdcore_app.erl\"},{line,44}]},{application_master,start_supervisor,3,[{file,\"application_master.erl\"},{line,327}]},{application_master,start_the_app,5,[{file,\"application_master.erl\"},{line,309}]},{application_master,start_it_new,7,[{file,\"application_master.erl\"},{line,295}]}]}}}}}"}
Crash dump is being written to: erl_crash.dump...done
Kernel pid terminated (application_controller) ({application_start_failure,bkdcore,{bad_return,{{bkdcore_app,start,[normal,[]]},{'EXIT',{undef,[{crypto,hmac,[sha256,<<1,34,42,54,243,4,35,3,123,5,234
erl_crash.dump is here.
I'm running OS X 10.11.4.
hi, when i try actordb, i found some difficulties, could you please provide some help?
1, i follow the document to run, the default username and password was in etc/init.example.sql, why not tell user firiendly?
2, i use navicat to connect to 127.0.0.1:33307, but can not list any table, just connected, nothing displayed, i also use intellij idea to connect actordb, but could not connected(intellij idea use java mysql client).
so is there a offical gui admin tool?
3, as problem 2, i'm not sure, can mysql client really work? if you provide some mysql client sample, it will be much better, php, java, ruby, python, golang, erlang etc.
there is no comminity of actordb, if the problems above can be solved, i think that will be much better.
thank you for your great hard work!
Hi,
I hope you can help me explain how to do the following.
I have created a new actorType in the schema.yaml and then created a new actor with this type and a id. I have done some testing and now I want to remove this actor with this id. How can I do this?
mysql> ACTOR accounting(test) CREATE;
Query OK, 0 rows affected (0.00 sec)
mysql> show tables;
+----------------+
| name |
+----------------+
| __transactions |
| __adb |
| accounting |
+----------------+
3 rows in set (0.01 sec)
mysql> select * FROM accounting;
Empty set (0.00 sec)
This is working fine. But how do I remove "test"? When I try the following:
mysql> ACTOR accounting(test); PRAGMA delete;
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
mysql> show tables;
+----------------+
| name |
+----------------+
| __transactions |
| __adb |
| accounting |
+----------------+
3 rows in set (0.00 sec)
There is still a accounting table for "test".
Hi, I had some problems with using actordb. When one of my cluster machine goes down,and i remove it in my nodes.yaml,and updatenodes ,but it still not work,I get the error message "unknown_query, invalidnode". what would be the cause. I use actordb-0.9-1.el6.x86_64.rpm on centos 6.5.Thanks for your help.
I watched @SergejJurecko 's excellent talk on ActorDB from February 2016 here --
vide: https://www.percona.com/resources/videos/actordb-alternative-view-distributed-database
slides: https://www.percona.com/live/data-performance-conference-2016/sites/default/files/slides/ActorDB.pdf
In the future plans slide, there are a number of exciting features discussed:
Geo replication
Kafka like pubsub
WAL for LMDB
Map-reduce (luajit)
Could you comment on the state of these? In particular, I would find (2) and (4) useful, and the kafka-like pubsub very useful.
Depending on the state or progress towards implementation, I could be interested in contributing; though these days I mostly write Go. I've worked with a couple of pub-sub systems in the past, namely mangos (https://github.com/go-mangos/mangos) and NATS (nats.io); I've written my own job distributed job scheduler as well, which is half the work of map-reduce (github.com/glycerine/goq).
Given the usual caveats about benchmarking, can anyone point to some ballpark performance expectations when using ActorDB as a distributed KV store?
A comparison relative to Riak KV would be very useful!
Impressive project...
Is this 100% fully compatible with MySQL drivers?
I am trying to follow setup steps (http://www.actordb.com/docs-getstarted.html#download_run) and on attempt to execute ./bin/actordb_console -f etc/init.example.sql, I am getting: Connect/login error: {error,
{login_failed,
<<"Username and/or password incorrect.">>}}
This project looks very interesting, indeed. What is its current status? I see that github sources have been updated, but documentation is stale and was not updated for long time. Is actordb used anywhere in production?
Is it possible to deploy ActorDB in a one-file executable?
What config and architecture you consider rigth to gather good performance and load-balancing on low-end VPS using Docker and some orchestration framework(Rancher/CoreOS/K8s)?
P.S. Assume that startup wants to use microservices with least possible cost. Also the app will use max 15-20 tables in relational way.
I downloaded the latest release for Mac OSX (https://dzbscw1ubdtyw.cloudfront.net/actordb-0.10.25-OSX-x86_64.tar.gz) and am getting this error trying to start actordb:
> ./bin/actordb start
!!!!
!!!! WARNING: ulimit -n is 256; 32768 is the recommended minimum.
!!!!
actordb failed to start within 15 seconds,
see the output of 'actordb console' for more information.
If you want to wait longer, set the environment variable
WAIT_FOR_ERLANG to the number of seconds to wait.
> ./bin/actordb console
dyld: lazy symbol binding failed: Symbol not found: _clock_gettime
Referenced from: /Users/bploetz/Downloads/actordb-0.10.25/erts-8.2/bin/beam.smp (which was built for Mac OS X 10.12)
Expected in: /usr/lib/libSystem.B.dylib
dyld: Symbol not found: _clock_gettime
Referenced from: /Users/bploetz/Downloads/actordb-0.10.25/erts-8.2/bin/beam.smp (which was built for Mac OS X 10.12)
Expected in: /usr/lib/libSystem.B.dylib
Error reading -config /Users/bploetz/Downloads/actordb-0.10.25/bin/../etc/app.config -args_file /Users/bploetz/Downloads/actordb-0.10.25/bin/../etc/vm.args -vm_args /Users/bploetz/Downloads/actordb-0.10.25/bin/../etc/vm.args
Any ideas?
Thanks for making actordb available -- I'm impressed by the strong foundation it provides.
Question:
If I have an actor representing a user's saved configuration, so that external users can add data to it, how can I establish a quota. Rationale: I want to prevent DOS attacks. I'd like the user to not be able to, either mistakenly or maliciously, do a denial of service attack by writing too much data to their configuration actor.
I see that sqlite has a quota operation at the C-level that refers to groups of files [1], however
a) I doubt that API will work now that storage is inside lmdb; and
b) I don't see anywhere where that quota setting is exposed via sql.
[1] http://www.sqlite.org/src/doc/trunk/src/test_quota.c
Alternative to quotas:
If there is a way to query how much space is currently being consumed by an actor, I could add application level checks and refuse to write more.
Is any of the mysql, thrift client-to-node, or inter-node traffic in actordb secured with TLS/certs?
If not, how could I add such protection?
(shared secret encryption would be okay as well; if certs are too complex).
Joins work fine from withing an actor, however, if you make a non syntactical error in your query the error is not very descriptive:
actordb> actor story(clientA); select e.id, e.title from entry e where e.id NOT IN(select THIS_ALIAS_DOES_NOT_EXISTS.entry_id from read THIS_IS_THE_REAL_ALIAS__OOPS);
actordb (1)> c
Error: {error,{sql_error,<<>>}}
actordb>
Error message should say that THIS_ALIAS_DOES_NOT_EXISTS
is not a known alias inside the given scope.
hello -
i am curious about the future ongoing development of actorDB. It appears there has not been any activity in several months.
generally, when i look at an ambitious project of this scale, i would think there would be new versions frequently.
after what happened with rethinkDB, i am a bit concerned.
thank you very much.
Is there a way to query an actor for table names?
Is this a proper way to list tables under filesystem actor? :
actor filesystem(*);
SELECT name FROM sqlite_master WHERE type='table';
I get no results from this query, however, in init.sql i have a rows:
actor filesystem kv
CREATE TABLE actors (id TEXT PRIMARY KEY, hash INTEGER, size INTEGER) WITHOUT ROWID
CREATE TABLE users (id INTEGER PRIMARY KEY AUTOINCREMENT, fileid TEXT, uid INTEGER, FOREIGN KEY (fileid) REFERENCES actors(id) ON DELETE CASCADE)
And i expect to see those tables somehow.
How to use something like PRAGMA table_info([tablename]); if this works in actordb?
While exploring your impressive project, I dared to think about a potential geo-replication scenario in ActorDB.
Basic idea is to have all data available at all locations:
I thought such geo-scenario might work for ActorDB, since a particular actor (i.e. a mobile user) typically operates from one location at the time. However groups of mobile users might be active at different locations simultaneously.
I am more a system level programmer and I do not have a lot of knowledge regarding databases, so please excuse me if you find this post irrelevant.
Hello,
Your project is a very clever piece of engineering ! I am trying to take a grasp of the pros & cons and the type of projects that fit well within actordb's replication model.
If I understand well, every actor instance is a kind of "monolith" in the sense that this instance should fully fit inside one node (except maybe for KV). So I guess that an actor instance should not grow too big otherwise it will create an unsolvable problem (?)
Can you tell me if FTS features (https://www.sqlite.org/fts3.html) or other sqlite plugins can be expected to work inside actordb ?
Right now the best solution to run in docker would be to use supervisord but if we could start the actordb daemon from the actordb_console containerization would be easier to work with.
I'm using Thrift that I've built using the latest netcore
bits to generate a client. The methods accessing actordb seem to work, as in, I can login with the usual login(username,password
method just fine.
However, I'm trying to implement login
using the salted 411 password and failing miserably. I've taken the algorithm for 411 scrambling directly from MySqlConnector for .NET as a starting point. Once I have the final hash byte array, every thing I've attempted to return it to a string that will be recognized by the thrift client has failed. I only see a single login
method in Thrift so I assume I can pass the stringified byte array into that.
I've tried byte[]->base64string, byte[]->hexstring with and without leading 0x
, and raw UTF8 conversion. I am missing the encoding or something similar that ActorDB's Thrift interface is expecting the hash's string representation to be.
public async Task<bool> LoginSecureAsync(string username, string password)
{
byte[] seed = await _thrift.saltAsync(_cancel.Token);
byte[] hash = Get411Password(password, seed);
string pass = ?? // Convert.ToBase64String(hash); //Encoding.UTF8.GetString(hash); // $"0x{hash.Hex()}";
var result = await _thrift.loginAsync(username, pass, _cancel.Token);
return result.Success;
}
public static byte[] Get411Password(string password, byte[] seed)
{
if (password.Length == 0) return new byte[1];
SHA1 sha = new SHA1CryptoServiceProvider();
byte[] firstHash = sha.ComputeHash(Encoding.Default.GetBytes(password));
byte[] secondHash = sha.ComputeHash(firstHash);
byte[] input = new byte[seed.Length + secondHash.Length];
Array.Copy(seed, 0, input, 0, seed.Length);
Array.Copy(secondHash, 0, input, seed.Length, secondHash.Length);
byte[] thirdHash = sha.ComputeHash(input);
byte[] finalHash = new byte[thirdHash.Length + 1];
finalHash[0] = 0x14;
Array.Copy(thirdHash, 0, finalHash, 1, thirdHash.Length);
for (int i = 1; i < finalHash.Length; i++)
finalHash[i] = (byte)(finalHash[i] ^ firstHash[i - 1]);
return finalHash;
}
}
Is there a way to tag certain actors and match them when updating just like I do with *
?
What I want to be able to modify data in many actors at the same time without having to name them all.
Hi,
I've built a Docker image that seems to work pretty well, except that it always exits with status 135 on its very first run.
I've raised an issue for my image's repo which has a few more details.
ianmjones/actordb-for-docker#1
Do you have any idea what exit status 135 might be related to?
It's strange that as long as the data files are available in /var/lib/actordb (created on first run) then the crash does not happen.
does actordb use the same protocol as postgres? if so, can we use any postgres GUI interface with actordb?
i know practically nothing about postgres, except that its protocol seems to be commonly used. previously i have used mariadb with mysql-gui-workbench, and will be looking at a drop-in replacement for workbench, preferably something that will work with actordb.
how is the new release coming along? i am eager to try it.
Hello,
Working with the Thrift client has a few areas of confusion. For example:
How can I list all users? 'SELECT User FROM mysql.user;' doesn't work but I can't see an alternative method
exec_config
seems to fail (InvalidRequestException
) on the documentation's init.sql
queries, that I can execute readily in actordb_console
, so I'm confused why I would use this over exec_sql
, which also applies to any other convenience method where I can approximate them all simply passing sql, but I can't find documentation describing its limitations/intended use -- I assumed it was a simple alias for use config
but that hasn't proved true. Every attempt to create a user via exec_config
or exec_sql
has failed.
I can't find the documentation for list<string> flags
that are available on several query methods
Example:
Calling exec_sql
or exec_config
and passing in the following SQL, I'm getting a Reply of "Execute exception."
Each SQL statement is on a newline. I can run this line for line in console successfully. I am logged in as root user (also created via the console).
use config
CREATE USER 'myuser' IDENTIFIED BY 'mypass'
GRANT read,write ON * to 'myuser'
commit
Hi,
you have an error in your 'Get Started ' documentation
Section 2.2 command:
"actordb_console -f etc/init.example.sql"
should be
"actordb_console -f /etc/actordb/init.example.sql"
It is a small but annoying error!
Hi,
Great work guys, very impressive !
I have a few questions regarding the project that I hope you could elaborate:
So using version 0.10.11
I can finally connect and perform ad-hoc queries(although, prepared statement don't work) using the Golang mysql driver, however, I'm still unable to insert with parametized values:
db, err := sql.Open("mysql", "myuser:mypass@tcp(127.0.0.1:33307)/__")
if err != nil {
fmt.Println("Failed to open mysql connection with error", err.Error())
}
defer db.Close()
ix := 0
for {
ix++
name := fmt.Sprintf("Name %d", ix)
if _, err := db.Exec("ACTOR type1(music) CREATE; INSERT INTO tab(i, txt) VALUES(?, ?);", ix, name); err != nil {
log.Fatal(err)
}
if ix > 10000 {
break
}
}
Error:
[MySQL] 2015/11/28 17:46:21 packets.go:32: unexpected EOF
[MySQL] 2015/11/28 17:46:21 statement.go:27: Invalid Connection
[MySQL] 2015/11/28 17:46:21 packets.go:32: unexpected EOF
[MySQL] 2015/11/28 17:46:21 statement.go:27: Invalid Connection
[MySQL] 2015/11/28 17:46:21 packets.go:32: unexpected EOF
[MySQL] 2015/11/28 17:46:21 statement.go:27: Invalid Connection
2015/11/28 17:46:21 driver: bad connection
This works though(no params):
if _, err := db.Exec(fmt.Sprintf("ACTOR type1(music) CREATE; INSERT INTO tab(i, txt) VALUES(%d, '%s');", ix, name)); err != nil {
Any chance to support this in the next version?
Btw, I can see the mysql interface is really buggy and for my needs unusable, where can I read some instructions on using the thrift interface?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.