br0xen / boltbrowser

A CLI Browser for BoltDB Files

boltbrowser

A CLI Browser for BoltDB Files

Installing

Install in the standard way:

go get github.com/br0xen/boltbrowser

Then you'll have boltbrowser in your path.

Pre-built Binaries

Pre-build binaries are available on the Releases Page.

Usage

Just provide a BoltDB filename to be opened as the first argument on the command line:

boltbrowser <filename>

To see all options that are available, run:

boltbrowser --help

Troubleshooting

If you're having trouble with garbled characters being displayed on your screen, you may try a different value for TERM.
People tend to have the best luck with xterm-256color or something like that. Play around with it and see if it fixes your problems.

boltbrowser's Issues

Pre-built binaries link is broken

All pre-built binaries show Not found
Example: Linux 64-bit
https://git.bullercodeworks.com/attachments/29367198-79f9-4fb3-9a66-f71a0e605006

It was working a couple of months ago. This is the preferred method for using it on machines without go installed.

[Crash] boltbrowser crashes if the db has values stored in root

it's probably not a good idea to store values in the root bucket, but shouldn't boltbrowser account for that ?

the following code generates a boltdb boltbrowser can't open

func main() {
	db, err := bolt.Open("crash.db", 0600, nil)
	if err != nil {
		panic(err)
	}
	defer db.Close()

	db.Update(func(tx *bolt.Tx) error {
		return tx.Cursor().Bucket().Put([]byte("boat"), []byte("I'm on a root!"))
	})

	db.View(func(tx *bolt.Tx) error {
		fmt.Printf("%s\n", tx.Cursor().Bucket().Get([]byte("boat")))
		return nil
	})
}

Error when key begins with "/"

I happened to have an entry with the key="/e/210". When I browse that row, it doesn't allow me to rename/edit/delete it. It says "Not sure what to do here".

Value Formatting

I'd like to have the right pane do some syntax highlighting/formatting mostly for json values.

how do i build this?

i tried this

$ go build boltbrowser.go 
# command-line-arguments
./boltbrowser.go:18: undefined: BoltDB
./boltbrowser.go:44: undefined: defaultStyle
./boltbrowser.go:47: undefined: mainLoop

Numeric keys decoded as unsigned

The example itob function on github.com/etcd-io/bbolt for encoding integers accepts signed values. The browser does not take this into consideration on display and you get the below overflow:

1061401654: {"settings":{"missed":false,"success":true}}
1068588820: {"settings":{"missed":false,"success":false}}
18446743072381089533: {"settings":{"missed":true,"success":true}}

The 1844... number should be -1001328462083 when decoded. This is a display issue only as keys fetched from db within our app and decoded retain their correct value.

Read-Only mode?

I have a trivial testing app:

package main

import (
	"fmt"
	"strconv"

	"github.com/boltdb/bolt"
)

func main() {
	db, err := bolt.Open("my.db", 0600, nil)
	if err != nil {
		panic(err)
	}
	defer db.Close()

	err = db.Update(func(tx *bolt.Tx) (err error) {
		bucket, err := tx.CreateBucketIfNotExists([]byte("Test"))
		if err != nil {
			return err
		}
		raw := bucket.Get([]byte("num"))
		num := 0
		if len(raw) != 0 {
			num, err = strconv.Atoi(string(raw))
			if err != nil {
				return err
			}
		}
		num++
		fmt.Println(num)
		bucket.Put([]byte("num"), []byte(strconv.Itoa(num)))
		return nil
	})
	if err != nil {
		panic(err)
	}
}

Every time you run it, it reads a number, increments it, and puts it back.

If I have boltbrowser connected to my.db and I run my app then my app sits and waits until boltbrowser releases it's connection. I know bolt only allows 1 write transaction at a time. It'd be nice if boltbrowser had a read only flag so viewing a DB that's currently under use won't halt all interactions with the database.

I imagine if another process is changing the DB then there's no way to know for sure if a value in a bucket didn't change immediately after being accessed and shown on the screen. Personally I'm ok with that.

It looks like all you'd have to do is prevent p/P, b/B, D, e, and r key presses if an "-ro" flag is present. Maybe just show a popup saying "Cannot do that in Read Only mode."

Right Pane Scrolling

When the value of a key is long, allow scrolling of the right pane. Hooks are already in place listening for 'J' and 'K' to scroll the right pane.

Timeout if DB remains locked

The screen should probably show a message if the db it's trying to open is locked.

I'm not sure what a good amount of time to wait before giving up would be, but that can be tweaked.

Edit value in external editor

Probably this looks like:

• Saving the value to a temporary file
• executing $EDITOR on that file
• waiting for $EDITOR to be done
• reading the value back in
• deleting temporary file

Does not work

This does not work, after installing in mac. I am using Sonama 14.

zsh: command not found: boltbrowser

Bucket names as byte-encoded integers produces unreadable UI

The code below creates a series of buckets with integer-encoded names. See screenshot of what this results in. The bucket names are not decoded correctly on display. Integer-encoded keys, however, show up correctly.

package main

import (
	"encoding/binary"


	log "github.com/sirupsen/logrus"
	bolt "github.com/etcd-io/bbolt"
)

func main() {

	db, err := bolt.Open("rewards.db", 0600, nil)
	if err != nil {
		log.Fatal("Failed to init db:", err)
	}
	
	db.Update(func(tx *bolt.Tx) error {

		rBucket, err := tx.CreateBucketIfNotExists([]byte("rewards"))
		if err != nil {
			log.Fatal("Cannot create rewards bucket:", err)
		}
		
		for i := 1; i < 20; i++ {
			buk, err := rBucket.CreateBucketIfNotExists(inttob(i));
			if err != nil {
				log.Fatal("Cannot make bucket", i)
			}
			
			buk.Put(inttob(i*3), []byte("Hello"))
		}
		
		return nil
	})
	
	log.Print("Done")
}

func inttob(v int) []byte {
	return itob(int64(v))
}

// itob returns an 8-byte big endian representation of v.
func itob(v int64) []byte {
	b := make([]byte, 8)
	binary.BigEndian.PutUint64(b, uint64(v))
	return b
}

func btoi(b []byte) int64 {
	return int64(binary.BigEndian.Uint64(b))
}

Search option

Hey, I am using boltbrowser trying to fix a problem I have with the lnd software, which uses a bolt DB.

I need to delete some records in a 120k long list. Scrolling takes a bit, I estimate it will take 30+ hours of scrolling to get to the record.

Being able to jump to a record starting with XX, or searching, would be the absolute best option.

Please consider, it would help a lot. Thank you!

lightninglabs/chantools#22 (comment)

Error Installing

Not a Go person (rarely have to use it) so this is mostl likely something real simple, but I don't know how to proceed:

When I run:

go get github.com/br0xen/boltbrowser

I just get:

package github.com/rivo/uniseg: found packages uniseg (doc.go) and main (gen_breaktest.go) in /home/kacey/go/src/github.com/rivo/uniseg

and it stops. What can I do from here?

(Running Ubuntu)

thanks!

Version outdated in source

https://github.com/br0xen/boltbrowser/releases/tag/2.1

boltbrowser/main.go

Line 15 in 59a4e49

var VersionNum = 2.0

Need to be 2.1 or the version of next release.

Refresh option

When I make changes in a file, I need to close the browser and reopen it. I lose all my open buckets and it's uncomfortable. It may be convenient to have a 'refresh' option so there will be no need to close and reopen the browser.

scrolling/highlighting for buckets with lots of long values

If you are browsing a bucket with lots of keys, it seems that the terminal does not scroll or highlight properly. You can see the correct key in the right panel, but the left panel is always at the top, it does not scroll. Not sure whether this can be implemented or it is a limitation of the term library.

Help screen lists 'd' for delete but it should be 'D' instead

At least on a Mac, it doesn't look like 'd' works to delete items, but instead 'D' works.

The help screen should be updated.

Thank you.

Starting boltbrowser with large DBs hangs

When I am using a large DB, boltbrowser hangs and fails to load.

Exported files not closed correctly

When using the 'X' hotkey to export, you are prompted for a filename and the data is saved correctly.

However the file isn't closed (it cannot be opened by another process until you exit boltbrowser)

The next time a export is performed, the entered filename isn't used, and data appears to be appended to the original file (which is still open).

OS: Windows 10
Version: Most recent from readme.md link

Cannot edit keys with long values

Keys with long values delete from the end of the line instead of where the cursor is.

Git Repository for prebuilt binaries has expired certificate

While attempting to grab boltbrowser for linux (x64) I discovered that the letsencrypt cert for this URL has expired:

wget -O boltbrowser https://git.bullercodeworks.com/attachments/29367198-79f9-4fb3-9a66-f71a0e605006 --2022-01-14 12:28:48-- https://git.bullercodeworks.com/attachments/29367198-79f9-4fb3-9a66-f71a0e605006 Resolving git.bullercodeworks.com (git.bullercodeworks.com)... 149.28.127.101 Connecting to git.bullercodeworks.com (git.bullercodeworks.com)|149.28.127.101|:443... connected. ERROR: cannot verify git.bullercodeworks.com's certificate, issued by ‘/C=US/O=Let's Encrypt/CN=R3’: Issued certificate has expired. To connect to git.bullercodeworks.com insecurely, use --no-check-certificate'.
`

Is it possible to just release the binaries as part of a standard GitHub Release?

Rework the filter (/) function

Support --version

This not only helps with issue reports, but also with automated testing of builds.

boltbrowser --version
Invalid optionUsage: boltbrowser [OPTIONS] <filename(s)>
Options:
  -timeout=duration
        DB file open timeout (default 1s)
  -ro, -readonly   
        Open the DB in read-only mode

Run value through external program

It could be nice in some cases to be able to run a value through an external program and view the returned value in the right pane...
I'm thinking of instances where, perhaps, hex/binary data is stored, but it should be readable in ascii if converted through xxd or something.

Update readme file with go install command from go get

Starting in Go 1.17, installing executables with go get is deprecated. go install may be used instead.

Convert to a package

@br0xen I am using boltdb in one of my projects and I like boltbrowser. I would like to embed boltbrowser into my application binary. To do this, I need to be able to import it as a package. Current package name (main) does not allow me to that. I propose to move all code files except boltbrowser.go into a new package under github.com/br0xen/boltbrowser/boltbrowser. This will not break backwards compatibility and will allow other packages to import boltbrowser as a library. I can send you a PR if it's okay. Are you happy with this change?

can't edit byte'd integers

The value of 59629 is saved/encoded using the following:

tx.Bucket([]byte("config")).Put([]byte("lastBlockLevel"), inttob(blockLevel))

// itob returns an 8-byte big endian representation of v.
func itob(v int64) []byte {
	b := make([]byte, 8)
	binary.BigEndian.PutUint64(b, uint64(v))
	return b
}

Displays correctly when viewing bucket/kv, but attempting to edit doesn't work.

Am I storing integers correctly or is there a better way?

App crashes when multiple files are passed and one of them cannot be open

I got this panic while working on #18

main.(*BoltDB).buildVisiblePathSlice(0x0, 0x0, 0xc4200e0000, 0x3, 0x80, 0x0)
	$GOPATH/src/github.com/br0xen/boltbrowser/bolt_model.go:130 +0x37
main.(*BoltDB).getNextVisiblePath(0x0, 0x0, 0x0, 0x0, 0xc42003dc50, 0x10e3a35, 0x11db480)
	$GOPATH/src/github.com/br0xen/boltbrowser/bolt_model.go:167 +0x2f
main.(*BrowserScreen).moveCursorDown(0xc4200e8000, 0xc42003dcd0)
	$GOPATH/src/github.com/br0xen/boltbrowser/screen_browser.go:474 +0x4d
main.(*BrowserScreen).handleBrowseKeyEvent(0xc4200e8000, 0xffec0000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3, 0x2)
	$GOPATH/src/github.com/br0xen/boltbrowser/screen_browser.go:123 +0x9d
main.(*BrowserScreen).handleKeyEvent(0xc4200e8000, 0xffec0000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3, 0x0)
	$GOPATH/src/github.com/br0xen/boltbrowser/screen_browser.go:78 +0x19c
main.mainLoop(0x0, 0x3000100080001, 0x30001)
	$GOPATH/src/github.com/br0xen/boltbrowser/mainloop.go:25 +0x299
main.main()
	$GOPATH/src/github.com/br0xen/boltbrowser/boltbrowser.go:56 +0x21b

To reproduce it you need to invoke boltbrowser with more than one file and make sure that one of them produces an error while being opened (use non-db file for example). Once you get to the screen for this file, press any arrow button.

Scrolling right pane doesn't work

J/K act the same as arrow up/down. They move the left pane cursor. None of the keys seem to scroll the right pane..

macos version difficult to use

Very difficult to use/see. Lots of garbled characters on screen. Using latest iTerm2/UTF8/xterm.

Locked databases

Hi,

First of all, thanks for creating boltbrowser, it's the shit. 👍

When creating a database with algernon, with the following command:

algernon -t --boltdb test.db . :7777

And then closing the web server by typing exit or pressing ctrl-d, accessing the database with boltbrowser works fine:

boltbrowser test.db

Adding keys and values (on the Algernon Lua prompt) and then examining them in boltbrowser (after exiting the webserver) also works great:

kv = KeyValue("pageviews")
kv:set("counter", 9999)

However, if accessing the database while the web server is running, there is no error message and no timeout and the console interface just goes into black and hangs (can't be closed with esc, q or ctrl-c), until the web server has quit and released the lock on the database.

This is a feature request for adding a timeout and/or a way to exit boltbrowser when the database is locked. Ignoring the database lock and entering some sort of read-only mode would also be interesting, if possible.

Cheers,
Alexander F Rødseth

Update bbolt version to go.etcd.io/bbolt

panic when browse containerd 1.17.1‘s metadata

panic: runtime error: index out of range [18446744073709547520] with length 281474976710655

goroutine 1 [running]:
github.com/boltdb/bolt.(*DB).page(...)
	/data00/code/pkg/mod/github.com/boltdb/[email protected]/db.go:796
github.com/boltdb/bolt.Open({0x7ffd572e0544, 0x3a}, 0x48690?, 0xc00008de60)
	/data00/code/pkg/mod/github.com/boltdb/[email protected]/db.go:237 +0x545
main.main()
	/data00/code/pkg/mod/github.com/br0xen/[email protected]/main.go:117 +0x1bc
root@n37-006-180:~# boltbrowser /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db
panic: runtime error: index out of range [18446744073709547520] with length 281474976710655

goroutine 1 [running]:
github.com/boltdb/bolt.(*DB).page(...)
	/data00/code/pkg/mod/github.com/boltdb/[email protected]/db.go:796
github.com/boltdb/bolt.Open({0x7ffc1e7f2544, 0x3a}, 0x48690?, 0xc000059e60)
	/data00/code/pkg/mod/github.com/boltdb/[email protected]/db.go:237 +0x545
main.main()
	/data00/code/pkg/mod/github.com/br0xen/[email protected]/main.go:117 +0x1bc

Maybe we should update bolt version

Due to:

• Since https://github.com/boltdb/bolt has been archived from 2019.3.9，and boltdb/bolt is no longer maintained.
• https://github.com/etcd-io/bbolt is still under maintenance, which is a fork for boltdb/[email protected]

And etcd bbolt is a fork of Ben Johnson's Bolt key/value store. The purpose of this fork is to provide the Go community with an active maintenance and development target for Bolt; the goal is improved reliability and stability. bbolt includes bug fixes, performance enhancements, and features not found in Bolt while preserving backwards compatibility with the Bolt API.
see https://github.com/etcd-io/bbolt#bbolt

panic in jump down (ctrl+f)

here is the panic

panic: runtime error: index out of range [1] with length 1

goroutine 1 [running]:
main.(*BrowserScreen).jumpCursorDown(0xc000244000, 0x1, 0x1)
        /home/builder/boltbrowser/screen_browser.go:467 +0x35d
main.(*BrowserScreen).handleBrowseKeyEvent(0xc000244000, 0x60000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0xc00003dca0)
        /home/builder/boltbrowser/screen_browser.go:121 +0x73d
main.(*BrowserScreen).handleKeyEvent(0xc000244000, 0x60000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x0)
        /home/builder/boltbrowser/screen_browser.go:84 +0x1fa
main.mainLoop(0xc00000c060, 0x3000100080001, 0x30001)
        /home/builder/boltbrowser/mainloop.go:25 +0x21e
main.main()
        /home/builder/boltbrowser/main.go:141 +0x333

reproduced (with same database) over ssh and locally with both kitty and xfce4-terminal

linux, TERM=xterm-256color
latest master branch (commit 7f10a81cece0f6754d1902b69c03892f267f65ee)

i will try and come up with a patch, but using / in the meantime

Large bolt database cant delete bucket

Hi.

I have a database with a bucket of 816 large pairs but when i try to delete it boltbrowser shuts down with the resulting error:

panic: page 3278 already freed

goroutine 1 [running]:
github.com/boltdb/bolt.(*freelist).free(0xc0000ea090, 0xb999, 0x7f5b16891000)
/home/admin/bolt/src/github.com/boltdb/bolt/freelist.go:121 +0x2a6
github.com/boltdb/bolt.(*Bucket).free.func1(0x7f5b16891000, 0x0, 0x2)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:684 +0x51
github.com/boltdb/bolt.(*Bucket)._forEachPageNode(0xc00004c340, 0xcce, 0x2, 0xc00006d9e0)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:506 +0x87
github.com/boltdb/bolt.(*Bucket)._forEachPageNode(0xc00004c340, 0x5f, 0x1, 0xc00006d9e0)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:513 +0xe6
github.com/boltdb/bolt.(*Bucket)._forEachPageNode(0xc00004c340, 0x15b, 0x0, 0xc00006d9e0)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:513 +0xe6
github.com/boltdb/bolt.(*Bucket).forEachPageNode(0xc00004c340, 0xc00006d9e0)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:499 +0x72
github.com/boltdb/bolt.(*Bucket).free(0xc00004c340)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:682 +0x67
github.com/boltdb/bolt.(*Bucket).DeleteBucket(0xc0000102b8, 0xc00006dbb0, 0x12, 0x20, 0x12, 0x20)
/home/admin/bolt/src/github.com/boltdb/bolt/bucket.go:255 +0x27e
github.com/boltdb/bolt.(*Tx).DeleteBucket(...)
/home/admin/bolt/src/github.com/boltdb/bolt/tx.go:121
main.deleteKey.func1(0xc0000102a0, 0x540a68, 0xc0000102a0)
/home/admin/bolt/src/github.com/br0xen/boltbrowser/bolt_model.go:375 +0x444
github.com/boltdb/bolt.(*DB).Update(0xc0000ec000, 0xc00006dc60, 0x0, 0x0)
/home/admin/bolt/src/github.com/boltdb/bolt/db.go:598 +0x90
main.deleteKey(0xc005882050, 0x1, 0x1, 0x1, 0xc00587e2d0)
/home/admin/bolt/src/github.com/br0xen/boltbrowser/bolt_model.go:370 +0xd5
main.(*BrowserScreen).handleDeleteKeyEvent(0xc00585e000, 0x7900000000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x0)
/home/admin/bolt/src/github.com/br0xen/boltbrowser/screen_browser.go:263 +0x1d2
main.(*BrowserScreen).handleKeyEvent(0xc00585e000, 0x7900000000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x0)
/home/admin/bolt/src/github.com/br0xen/boltbrowser/screen_browser.go:84 +0x119
main.mainLoop(0xc00013e080, 0x3000100080001, 0xc000030001)
/home/admin/bolt/src/github.com/br0xen/boltbrowser/mainloop.go:25 +0x22d
main.main()
/home/admin/bolt/src/github.com/br0xen/boltbrowser/main.go:133 +0x30e

Update boltdb import path

boltbrowser/boltbrowser.go

Line 10 in 7d7e0bf

"github.com/boltdb/bolt"

The original project is not maintained anymore. Author recommends to use the fork at https://github.com/coreos/bbolt.

https://github.com/boltdb/bolt#a-message-from-the-author

Import JSON into bucket

It would be useful to import previously exported json data into an existing bucket

Values preview.

Hi, here is what happen:

Values preview is in pure bytearray but editing it the text is showed!
Any suggestions?
Thanks.

Add Support for other Data Types

Currently boltbrowser will only save ascii values back into the DB.
It'd be nice to have it check:

1. if the value is currently in the ascii range of bytes, if so, let things work as they are.
2. If not, prompt the user for a data type.

My current thought is that other supported data types would just be the basic go types:

• bool
• int, int8, int16, int32, int64
• uint, uint8, uint16, uint32, uint64, uintptr (doubt if we actually need that one)
• byte
• float32, float64
• complex64, complex128

Eventually, I would like to add support for custom data types, probably through the use of go plugins:
https://golang.org/pkg/plugin/

I believe that doing this should mostly resolve pull request #31 and alleviate issue #37