brentp / xopen

open files for buffered reading and writing in #golang

xopen

-- import "github.com/brentp/xopen"

xopen makes it easy to get buffered (possibly gzipped) readers and writers. and close all of the associated files. Ropen opens a file for reading. Wopen opens a file for writing. Both will use gzip when appropriate and will use buffered IO.

Usage

Here's how to get a buffered reader:

// gzipped
rdr, err := xopen.Ropen("some.gz")
// normal
rdr, err := xopen.Ropen("some.txt")
// stdin (possibly gzipped)
rdr, err := xopen.Ropen("-")
// https://
rdr, err := xopen.Ropen("http://example.com/some-file.txt")
// Cmd
rdr, err := xopen.Ropen("|ls -lh somefile.gz")
// User directory:
rdr, err := xopen.Ropen("~/brentp/somefile")

Get a buffered writer with xopen.Wopen. Get a temp file with xopen.Wopen("tmp:prefix")

func CheckBytes

func CheckBytes(b *bufio.Reader, buf []byte) (bool, error)

CheckBytes peeks at a buffered stream and checks if the first read bytes match.

func IsGzip

func IsGzip(b *bufio.Reader) (bool, error)

IsGzip returns true buffered Reader has the gzip magic.

func IsStdin

func IsStdin() bool

IsStdin checks if we are getting data from stdin.

func XReader

func XReader(f string) (io.Reader, error)

XReader returns a reader from a url string or a file.

type Reader

type Reader struct {
	*bufio.Reader
}

Reader is returned by Ropen

func Buf

func Buf(r io.Reader) *Reader

Return a buffered reader from an io.Reader If f == "-", then it will attempt to read from os.Stdin. If the file is gzipped, it will be read as such.

func Ropen

func Ropen(f string) (*Reader, error)

Ropen opens a buffered reader.

func (*Reader) Close

func (r *Reader) Close() error

Close the associated files.

type Writer

type Writer struct {
	*bufio.Writer
}

Writer is returned by Wopen

func Wopen

func Wopen(f string) (*Writer, error)

Wopen opens a buffered writer. If f == "-", then stdout will be used. If f endswith ".gz", then the output will be gzipped. If f startswith "tmp:" then a tempfile will be created with a prefix of the string following ":"

func (*Writer) Name

func (w *Writer) Name() string

The path to the underlying file handle.

func (*Writer) Close

func (w *Writer) Close() error

Close the associated files.

func (*Writer) Flush

func (w *Writer) Flush()

Flush the writer.

xopen's Issues

fastGzReader is not used

Hi, Brent,

I notice that your last commit had added the fastGzReader, but you did not use it. It's commented, see https://github.com/brentp/xopen/blob/master/xopen.go#L140. Did you forget or intend to do so?

Wei

not able to read from `stdin` when opening file handle of `stdout`

Hi Brent,

I used xopen frequently, but today, I found that it was not able to read from stdin when opening file handle of stdout.

Here's the code:

package main

import (
    "fmt"
    "os"

    "github.com/brentp/xopen"
)

func main() {
    outfh, err := xopen.Wopen("-")
    if err != nil {
        fmt.Println(err)
        os.Exit(-1)
    }
    defer outfh.Close()

    outfh.WriteString("write to stdout\n")
    fmt.Printf("are we reading from stdin: %v\n", xopen.IsStdin())
}

output:

are we reading from stdin: false
write to stdout

---

More detail

Error when calling stat, err := os.Stdin.Stat():

stat /dev/stdin: bad file descriptor

Would you help to solve this?

Wei