kiwanami / emacs-edbi Goto Github PK

Database Interface for Emacs Lisp

Emacs Lisp 95.04% Perl 4.81% Shell 0.15%

emacs-edbi's Introduction

Emacs DBI

This program connects the database server through Perl's DBI, and provides DB-accessing API and the simple management UI.

Installation:

Package installation

If you use package.el with Marmalade or MELPA, you just select the package 'edbi' and install it.

Next, you install the perl module `RPC::EPC::Service', DBI and Database drivers with CPAN.

Example:

$ cpan RPC::EPC::Service DBI DBD::SQLite DBD::Pg DBD::mysql

Here, SQLite, Postgresql and MySQL drivers will be installed, in addition to the EPC module. Of course, you can choose the drivers for your environment.

Manual installation

This program depends on following programs:

deferred.el, concurrent.el / https://github.com/kiwanami/emacs-deferred
epc.el / https://github.com/kiwanami/emacs-epc
ctable.el / https://github.com/kiwanami/emacs-ctable
Perl/CPAN
RPC::EPC::Service (and some dependent modules)
DBI and drivers, DBD::Sqlite, DBD::Pg, DBD::mysql

Place this program (edbi.el and edbi-bridge.pl) in your load path and add following code.

(require 'edbi)

Helper application

One can install some helper applications:

edbi-django -- This program connects django project database quickly. -- (You can find more edbi projects at the author's repo, https://github.com/proofit404/edbi-sqlite3, https://github.com/proofit404/edbi-database-url)

Usage:

M-x `edbi:open-db-viewer' opens a dialog for DB connection.

Data Source : URI string for DBI::connect (Ex. dbi:SQLite:dbname=/path/db.sqlite )
User Name, Auth : user name and password for DBI::connect
History button : you can choose a data source from your connection history.
OK button : connect DB and open the database view

Database view

This buffer enumerates tables and views.

Check the key-bind `edbi:dbview-keymap'.

j,k, n,p : navigation for rows
c : switch to query editor buffer
RET : show table data
SPC : show table definition
q : quit and disconnect

Table definition view

This buffer shows the table definition information.

Check the key-bind `edbi:dbview-table-keymap'.

j,k, n,p : navigation for rows
c : switch to query editor buffer
V : show table data
q : kill buffer

Query editor

You can edit SQL in this buffer, which supports SQL syntax highlight and auto completion by auto-complete.el.

Check the key-bind `edbi:sql-mode-map'.

C-c C-c : Execute SQL
C-c q : kill buffer
M-p : SQL history back
M-n : SQL history forward

Query result viewer

You can browser the results for executed SQL.

Check the key-bind `edbi:dbview-query-result-keymap'.

j,k, n,p : navigation for rows
SPC : display the whole data at the current cell. (hitting the SPC again, it clears the popup.)
q : kill buffer

Navigation summary

E2WM perspective

Here is an example setup to open EDBI perspective with Super-d.

(autoload 'e2wm:dp-edbi "e2wm-edbi" nil t)
(global-set-key (kbd "s-d") 'e2wm:dp-edbi)

Emacs DBI API

TODO...

emacs-edbi's People

Contributors

Stargazers

Watchers

emacs-edbi's Issues

edit sql files and evaluating chunks of code

Hi,

Suppose I want to have a myprogram.sql file opened. How do I incorporate that into edbi's workflow?

C-c C-c currently evaluates the Query Editor buffer. Is there a way to evaluate a chunk of code? Mark region, C-c C-r to evaluate region? I'm thinking of the case where users are developing sql code and evaluating chunks to test the program. Then save the file at the end of the session.

Evil Mode, no history/popup menu in edbi:open-db-viewer

When using emacs evil mode clicking on the history button doesn't yield any popup menu.

For the time being there is two easy workarounds:

Temporarily switch out of evil mode back into emacs mode
Use the open a db with a function instructions as posted here

Although this is a minor issue with easy workarounds Let me know if i can provide any additional information, and thanks for a great module.

Merging firebird patch

I received, but have not merged.

add two tips

1、connect to a database contains chinese

uri: dbi:mysql:dbname=db:mysql_enable_utf8=1

2、connect to a remote database with the ssh tunnel
first, open a tunnel

ssh -fNL 3306:localhost:3306 myname@myhost

then, connect to the local port like tip 1

[Feature] Jump to column

Hi,

In the "Query Result" buffer, it would be nice if one could press some keystroke (maybe 'j' for 'jump', but 'j' is bound to ctbl:navi-move-down at the moment) to trigger ido-completing-read to let the user select a column name and jump to that column.

This would be especially useful for tables that contain a lot columns.

Regards,
Wenshan

Is it possible to store the connection details?

Thanks for sharing this awesome package! When I open edbi:open-db-viewer, I need to fill my login details. I can choose my previous login details from the history. But then I type my password.

However, I use only one datasource. So giving my credentials every time when I open edbi:open-db-viewer seems somewhat cumbersome to me.

Would it be possible to store them, and let edbi connect automatically without filling the connection details and credentials?

Multiple queries possible?

Is it possible to use EDBI query editor with multiple queries like:

INSERT INTO table_name (column1,column2,column3,...)
VALUES (foo1,foo2,foo3,...);
INSERT INTO table_name (column1,column2,column3,...)
VALUES (bar1,bar2,bar3,...);
INSERT INTO table_name (column1,column2,column3,...)
VALUES (foo4,foo5,foo6,...);
INSERT INTO table_name (column1,column2,column3,...)
VALUES (bar4,bar5,bar6,...);

Instead one query to evaluate per once in EDBI query editor?

can you remove auto-complete dependency?

Some people (like me) don't use auto-complete, we use company-mode instead.
I hope you can refactor edbi to use completion-at-point-functions (which is used by Emacs default, and supported by company-mode). it is better solution.

Query execute does not return?

Just found this for the day - using with emacs 29. I can get it to talk to me sqlite database and describe the tables fine. However when I try to query the database via C-c C-c I can see it getting as far as edbi:dbview-query-editor-execute-command but then the asynchronous function below is entered, but never seems to return.

Can anyone confirm if this should still work - I notice the package is quite old, but I can see people commenting on Reddit just a few years a go that it was working fine. It's exactly what I need to happy to patch the function if needs be!

(defun edbi:dbview-query-execute (conn sql result-buf)
  "[internal] Execute SQL and rendering results."
  (lexical-let ((conn conn) (time-begin (float-time))
                (sql sql) (result-buf result-buf))
    ;; THE CODE GETS THIS FAR FOR SURE - BUT THEN NO MESSAGE RETURNED
    (cc:semaphore-with edbi:dbview-query-execute-semaphore
      (lambda (x)
        (message "EDBI: Waiting for the query...")
        (deferred:$
          (edbi:seq
           (edbi:prepare-d conn sql)
           (edbi:execute-d conn nil)
           (lambda (exec-result)
             (message "Result Code: %S  /  %.3f seconds"
                      exec-result (- (float-time) time-begin))
             (cond
              ;; SELECT
              ((or (equal "0E0" exec-result)
                   (and exec-result (numberp exec-result))) ; some DBD returns rows number
               (edbi:dbview-query-editor-history-add sql)
               (lexical-let (rows header (exec-result exec-result))
                 (edbi:seq
                  (header <- (edbi:fetch-columns-d conn))
                  (rows <- (edbi:fetch-d conn edbi:dbview-query-result-fetch-num))
                  (lambda (x)
                    (cond
                     ((or rows (equal "0E0" exec-result)) ; select results
                      (if rows
                          (edbi:dbview-query-result-open conn result-buf header rows)
                        (progn
                          (message "No rows returned.")
                          (kill-buffer result-buf))))
                     (t    ; update results?
                      (edbi:dbview-query-result-text conn result-buf exec-result)))))))
              ;; ERROR
              ((null exec-result)
               (edbi:dbview-query-result-error conn result-buf))
              ;; UPDATE etc
              (t
               (edbi:dbview-query-editor-history-add sql)
               (edbi:dbview-query-result-text conn result-buf exec-result)))))
          (deferred:error it
            (lambda (err) (message "ERROR : %S" err))))))))

A way to connect to other not SQL dbs?

I love edbi, it simplifies a lot to simply view the database. I wonder is there a way to make such viewing for other db types - specifically mongoDB or redis? I know nothing about perl unfortunately, so I simply googled for DBI's for mongodb and wasn't able to find package on cpan for it. Isn't it possible and if is - please suggest cpan package install command and example for data source string. Thanks, and sorry if question is stupid)

Unable to obtain table definition for postgresql databases.

I connect to postgres database, press SPC on table. *edbi-dbviewer-table* buffer don't show any table information. Only database connection info available. Possibly it depends from https://github.com/kiwanami/emacs-edbi/blob/master/edbi.el#L543 typo.

make emacs-edbi aware of local::lib installations of RPC::EPC::Service and DBI modules

Would prefer to not install system wide, its appears by default edbi:open-db-viewer does't check local::lib installs based on the following error

Connection Error : (error Server may raise an error. Use "M-x epc:pop-to-last-server-process-buffer RET" to see full traceback:
Can't locate RPC/EPC/Service.pm in @INC (you may need to install the RPC::EPC::Service module) (@INC contains: /Library/Perl/5.18/darwin-thread-multi-2level /Library/Perl/5.18 /Network/Library/Perl/5.18/darwin-thread-multi-2level /Network/Library/Perl/5.18 /Library/Perl/Updates/5.18.2/darwin-thread-multi-2level /Library/Perl/Updates/5.18.2 /System/Library/Perl/5.18/darwin-thread-multi-2level /System/Library/Perl/5.18 /System/Library/Perl/Extras/5.18/darwin-thread-multi-2level /System/Library/Perl/Extras/5.18 .) at /Users/vedwin/.emacs.d/elpa/edbi-20160224.1741/edbi-bridge.pl line 4.
BEGIN failed--compilation aborted at /Users/vedwin/.emacs.d/elpa/edbi-20160224.1741/edbi-bridge.pl line 4.

Process epc:server:5 exited abnormally with code 2
)

is there any way we can add this functionality?

error connecting to mysql

Hi,

I'm trying out emacs-edbi to access all my sql databases using emacs. Never used Perl's DBI before, so I'm sure it must be me failing to set things up properly.

In the connection buffer, I entered:

data source: dbi:mysql:path.to.my.server
username: my.username
Auth: my.password

I get the following error:

Connection Error : (error "Can't call method "get_info" on an undefined value at /Users/Vnguyen/.emacs.d/emacs-edbi/edbi-bridge.pl line 20.

Can you let me know what I'm doing wrong? Do I have to create a list of DSN's like ODBC's ~/.odbc.ini? Please point me to some resources or let me know what I should search for. Thanks.

Request: Launch Database View from function

Hi there,

I love this emacs plugin. Instead of typing in the connection string each time when executing M-x edbi:open-db-view, Is there away to launch the open-db-view with a function and the connections string in it? So, something like this:

    (edbi:open-db-view("dbi:Pg:dbname=test_app_db" "username" "password"))

Then it will launch the Database View for that database automatically?

I can't connect mysql database

Connection Error : (error Server may raise an error. Use "M-x epc:pop-to-last-server-process-buffer RET" to see full traceback:
Can't locate RPC/EPC/Service.pm in @INC (you may need to install the RPC::EPC::Service module) (@INC contains: /Library/Perl/5.18/darwin-thread-multi-2level /Library/Perl/5.18 /Network/Library/Perl/5.18/darwin-thread-multi-2level /Network/Library/Perl/5.18 /Library/Perl/Updates/5.18.2 /System/Library/Perl/5.18/darwin-thread-multi-2level /System/Library/Perl/5.18 /System/Library/Perl/Extras/5.18/darwin-thread-multi-2level /System/Library/Perl/Extras/5.18 .) at /Users/myhome/.emacs.d/elpa/edbi-20160224.1741/edbi-bridge.pl line 4.
BEGIN failed--compilation aborted at /Users/myhome/.emacs.d/elpa/edbi-20160224.1741/edbi-bridge.pl line 4.

Process epc:server:3 exited abnormally with code 2
)

but I already install "RPC::EPC::Service module"

Export the query results

It would be good to be able to export the query results to e.g. JSON or CSV. Just copy/pasting the results buffer and then manipulating it is not enough, since long text values are truncated in the results view.

Can't install from melpa-stable

Hello

I'm interested in try this program, but can't install from repository melpa-stable, since it requires epc-0.1.1 and only the epc-0.1.0 is available.

Thanks!

Cannot open a sqlite database that has been created using WAL mode

Hi. Thanks in advance for the nice tool. I'm using it with sqlite, but find I cannot open a sqlite file in wal mode (see the attached T07_endToEnd_NestedDirectories.db file with associated -wal and -shm files). The same schema, when exported and re-imported to a new db is able to be read (attached file newdb.sqlite).

The error I'm getting (running on OSX, 10.12) upon trying to open T07_endToEnd_NestedDirectories.db is:

[connecting...]
Connection Error : (error "\"Can't call method \\\"fetchall_arrayref\\\" on unblessed reference at /Users/stebro/.emacs.d/elpa/edbi-20160224.1741/edbi-bridge.pl line 135.
\"")
Check your setting...

Archive.zip

PostgreSQL connection

Hi,

Thanks for your work!

I'm trying to connect with a PostgreSQL database but Emacs gives the following error messages:

Connection Error : (error (epc:manager-title mngr) is not a valid place expression)
invalid ((dbi:Pg:dbname=t0612 openerp openerp))

I don't quite understand what it is complaining about, any idea?

Regards,

Quoting tables

I have a table names 'foo-bar' and edbi fails to display it with

Error: near "-": syntax error

From the commandline, a

$ sqlite3 database.db "SELECT * FROM social-account;"

produces the same error, while

$ sqlite3 database.db "SELECT * FROM 'social-account';"

works.

Something wrong with edbi not quoting?

Does your library have functions for reading columns directly from the sql results into elisp?

emacs-sqlite3-api has the sqlite3-column-* functions for retrieving the results of a row.

Does your package have functions which can be used in this manner?

Change the window-title edbi:query-result without number

Situation: I use a monitor and a laptop screen as my multimonitor setup.

Unfortunately the monitor and the laptop differs in resolution.
So I use 2 frames, one frame on the monitor and one frame on the laptop screen.

Both frames are connected to one Emacs instance.

MONITOR:
I will call the frame that's opened on monitor as frame 1.
In the frame 1 on the monitor, I have a buffer opened with the EDBI query window.

LAPTOP:
On the laptop screen, there is the query result window being displayed in the Emacs frame 2.

SITUATION:
In the frame 1, in the buffer with EDBI query window, I do C-c C-c edbi:dbview-query-editor-execute-command to send a SQL-statement to the SQL-process.

I see the result on the laptop screen in frame 2. But in the meanwhile, EDBI propably have detected there is no window in the Emacs frame 1, so EDBI opens a new window with the same buffer of the ctbl-table-mode.

As the result, there are two windows on two screens, viewing the same buffer (the ctbl-table-mode).

I solved it with trying to reuse windows from another frame only for EDBI process buffer, with adding an entry to display-buffer-alist:

(push '("<*edbi:query-result 1>" nil . ((reusable-frames . t))) display-buffer-alist)

But the problem is that the number of *edbi:query-result 1 changes sometimes. The documentation of display-buffer-alist and display-buffer (via C-h v and C-h f) explains them in great detail. You can read related pages from the Elisp manual to learn more.

I would prevent that the EDBI uses number to label the query result window. So there is no need anymore to change the window name and pushing it to display-buffer-alist everytime. How could I 'tell' EDBI to use edbi:query-result without any number?

Unable to save query editor file content to file.

It is really handy use find-file and save-buffer functionally with
query editor buffers. Currently I use insert-file and manually
copy-paste edited content back to the opened file buffer.

How configure edbi to stay in the query window?

I would like to 'tell' EDBI to stay in the same query window, without switching to the window with the query result.

I tried it with binding C-c C-c in EDBI query window to another function:

 (defun edbi-evaluate ()
   (interactive)
   (edbi:dbview-query-editor-execute-command)
   (other-window 1)
  )

But it's not working, which I cannot understand. Any other suggestion?

参照できるオブジェクトが多いと反応が無いように見える

参照できるテーブルが多いと、かなりの時間 edbi:ac-editor-word-candidate-update あたりが semaphore を占有して edit buffer で C-c C-c しても反応がないように見えます。

多分Oracleの場合に顕著で、自分の環境だと dbviewer で取ってくる一覧の件数が3万件とかいう結構ひどい数字になってます(ほとんどが public synonym を管理してるスキーマ)。

Selected row colors in Emacs dark modes?

Are there any options in emacs-edbi to allow changing the colors of the selected row?

In the attached screenshot, the current row is unreadable. I've tried a few different dark modes, and the result is the same.

Process epc:server:2 bus error (core dumped)

I've been trying to get emacs-edbi working for a couple of hours. I know next to nothing about perl.
I installed the package via melpa, and installed epc with 'cpan RPC::EPC::Service'

After filling in the database details on edbi:open-db-viewer and hit OK, it repsonds with
Process epc:server:2 bus error (core dumped).
I've poked around trying to figure out what might cause that, but no luck so far. Any ideas?

I'm running ubuntu and using mysql.

(I realize this might not be the correct place to raise an issue like this, so if you feel it would be better answered elsewhere please direct me away)

Eager macro-expansion failure: (error "(ctbl:param-fixed-header param) is not a valid place expression")

Eager macro-expansion failure: (error "(ctbl:param-fixed-header param) is not a valid place expression")

I get this error with Emacs 24.3.1 when I (require 'edbi)
Apparently there were changes to the cl library and now setf is implemented somewhere else.
When I do M-x edbi:open-db-viewer I get the following but I suspect this has something to do with the ctbl:param struct.

let*: End of file during parsing

No rows shown in result for SQL Server query with ntext column

Stub bug report until I can reproduce from scratch and add more details...

Using

SELECT TOP 10 *  FROM some_table;

where some_table has no ntext column shows rows as expected. However if it has an ntext column this is shown instead:

OK. -1 rows are affected.

No rows are shown. That message is the only thing in the query result buffer.

Unclear to me whether this is a bug in emacs-edbi or DBD::ODBC. Apologies for sparse report, will try to investigate further later.

Database is on SQL Server 2008 R2.

column names in wide tables

I have a wide table (20+ variables). When I move to the right of the buffer to view additional data, the column names (header row) doesn't seem to update with the data. It remains static that the data does not match up to the column names.

Unable to open query editor if there is no any table in database.

If I open fresh (sqlite3) database dbviewer simply displays No Table. I can't use c or C keybinding to open query editor. It needs some ctable data. But since there is no tables there is no widget :) Seems like a usability deadlock.

Database w/o tables freezes Emacs

Hello! I was so excited to try this extension, but it doesn't appear to work for me :(

My datasource is dbi:Pg:dbname=vagrant;host=192.168.33.10. No authentication, so I just fill in the username. However, Emacs just hangs at the [connecting...] message until I C-g a few times. The following Perl program executes fine:

use DBI;

$dbh = DBI->connect("dbi:Pg:dbname=vagrant;host=192.168.33.10", "vagrant", "");

Edit: see below comment.

saving dsn information

Is there a way to save the dsn information so I don't have to re-enter each time? Thanks.

Add connection bookmarks/names

The edbi:open-db-viewer history feature is great. However, there are several databases that I connect to regularly that have very similar connection strings, so it takes me a moment to figure out which is which (and if my Emacs window is narrow or I'm on a small monitor they all show up the same, since the difference is more to the end of the connection string). Also, I often need to connect via an SSH tunnel, so I have a bunch of connections that connect to localhost with different user/password combinations (or sometimes with the same user and a different password), which makes it even more confusing - and it confuses edbi (it only saves one of the connections).

I've also found that the password is not saved to the history file (and if I manually edit it and add the passwords, they get removed). Am I doing something wrong?

It would be great to have a list of named connections, so that I could select by name instead of by connection string, that would also save the password. It would be even better if I could easily edit the connections file to copy/paste connection strings and credentials without worrying about edbi overwriting it (as it does with the history file).

Specify the port number to be used

Hi,

I SSH-forwarded a remote PostgreSQL database to localhost (SSH -L9999:localhost:5432 username@remote-server-domain).

In EDBI, how could I specify the port number (in my case, 9999) to be used?

Thanks,

RET on table doesn't display rows

Using a Postgres database with the schema found in edbi-demo.el, hitting RET on the test creates an empty, unfocused result buffer.

Afterwards, calling edbi:dbview-query-execute-semaphore-clear then edbi:dbview-query-editor-execute-command freezes Emacs, but does issue the query; C-g and manual selection of the results buffer contains rows.

Oracleの場合にepcがエラーを起こす

Oracleの場合、以下の理由でエラーが起きるようです。

DBD::Oracleは文字列にutf-8 flagを付けて返す
AnyEventはutf-8 flagが付いてるとエラーを起こす

以下のコードでOracle関係なしに再現できます。

サーバ側::

use RPC::EPC::Service;                                                                                                                                

my $server = RPC::EPC::Service->new(8888, {
  'identity' => sub {
    my ($args) = @_; return $args;                                                                                                                    
  }});                                                                                                                                                
$server->start;

クライアント側::

use RPC::EPC::Service;                                                                                                                                
use Data::Dumper;                                                                                                                                     
use Encode;                                                                                                                                           

my $client = RPC::EPC::Service->new(8888, {});                                                                                                        
$client->client_start;                                                                                                                                
foreach my $text ('abcd', '日本語', 'efgh') {
  print Dumper $client->call_method('identity', [Encode.decode_utf8($text)])->recv;                                                                   
}
$client->stop;

Query returns nil when value is 0

Hi there,

thank you for your awesome work, it helps me a lot.

When I send a query to the mysql database, when the column value is zero, I get a 'nil' value in the query results. But the value stored in the database is zero.

Is there a way to configure emacs-edbi so it shows the actual value instead of 'nil' or is it a bug?

In the attached file you can see the different results comming from edbi and sql-mysql modes using the same query.

Thanks

edit data in db

Just wondering, could this be used to easily edit data in the database?

For example the cell data view could support an edit action that would allow content to be edited and then an automatically generated sql would update the db with the new data

Postgres Issue: No tables found

Hello, it seems that edbi can connect with my local Postgres database fine, but edbi says "No tables found" after connection.

DB: dbi:Pg:dbname=test_app_dev

No tables.

However, there are several tables in the database it's connected to. Here is the connection string I'm using.

dbi:Pg:dbname=test_app_dev

I've verified the database exists with tables and I've connected to it with a short Perl script.