GithubHelp home page GithubHelp logo

freetaxii / server Goto Github PK

View Code? Open in Web Editor NEW
30.0 7.0 5.0 204 KB

A cyber threat intelligence server based on TAXII 2 and written in Golang

License: Apache License 2.0

Go 88.53% HTML 8.56% Makefile 2.91%
golang stix taxii taxii-server go

server's People

Contributors

jordan2175 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

server's Issues

Does not compile

The issue is that server is not compatible with the current release of libstix2-0.6.1 (or the latest source in the repo), but it also isnt' compatible with the previous release libstix2-0.5.1 either.

Here is the output compiling server when using libstix2-0.6.1
$ env GOOS=linux GOARCH=amd64 go build -v freetaxii.go
../../../libstix2/datastore/sqlite3/s_baseobject.go:16:2: cannot find package "github.com/freetaxii/libstix2/objects/baseobject" in any of:
/usr/local/Cellar/go/1.15.2/libexec/src/github.com/freetaxii/libstix2/objects/baseobject (from $GOROOT)
/Users/don/go/src/github.com/freetaxii/libstix2/objects/baseobject (from $GOPATH)
../../internal/config/serverConfig.go:13:2: cannot find package "github.com/freetaxii/libstix2/resources/apiroot" in any of:
/usr/local/Cellar/go/1.15.2/libexec/src/github.com/freetaxii/libstix2/resources/apiroot (from $GOROOT)
/Users/don/go/src/github.com/freetaxii/libstix2/resources/apiroot (from $GOPATH)
../../../libstix2/datastore/datastore.go:8:8: cannot find package "github.com/freetaxii/libstix2/resources/collections" in any of:
/usr/local/Cellar/go/1.15.2/libexec/src/github.com/freetaxii/libstix2/resources/collections (from $GOROOT)
/Users/don/go/src/github.com/freetaxii/libstix2/resources/collections (from $GOPATH)
../../internal/config/serverConfig.go:15:2: cannot find package "github.com/freetaxii/libstix2/resources/discovery" in any of:
/usr/local/Cellar/go/1.15.2/libexec/src/github.com/freetaxii/libstix2/resources/discovery (from $GOROOT)
/Users/don/go/src/github.com/freetaxii/libstix2/resources/discovery (from $GOPATH)
../../../libstix2/datastore/sqlite3/t_collectiondata.go:18:2: cannot find package "github.com/freetaxii/libstix2/resources/envelope" in any of:
/usr/local/Cellar/go/1.15.2/libexec/src/github.com/freetaxii/libstix2/resources/envelope (from $GOROOT)
/Users/don/go/src/github.com/freetaxii/libstix2/resources/envelope (from $GOPATH)
../../../libstix2/datastore/sqlite3/t_collectiondataManifest.go:16:2: cannot find package "github.com/freetaxii/libstix2/resources/manifest" in any of:
/usr/local/Cellar/go/1.15.2/libexec/src/github.com/freetaxii/libstix2/resources/manifest (from $GOROOT)
/Users/don/go/src/github.com/freetaxii/libstix2/resources/manifest (from $GOPATH)
../../internal/handlers/contentHandler.go:18:2: cannot find package "github.com/freetaxii/libstix2/resources/status" in any of:
/usr/local/Cellar/go/1.15.2/libexec/src/github.com/freetaxii/libstix2/resources/status (from $GOROOT)
/Users/don/go/src/github.com/freetaxii/libstix2/resources/status (from $GOPATH)
../../internal/handlers/errors.go:13:2: cannot find package "github.com/freetaxii/libstix2/resources/taxiierror" in any of:
/usr/local/Cellar/go/1.15.2/libexec/src/github.com/freetaxii/libstix2/resources/taxiierror (from $GOROOT)
/Users/don/go/src/github.com/freetaxii/libstix2/resources/taxiierror (from $GOPATH)
../../../libstix2/datastore/sqlite3/t_collectiondata.go:19:2: cannot find package "github.com/freetaxii/libstix2/resources/versions" in any of:
/usr/local/Cellar/go/1.15.2/libexec/src/github.com/freetaxii/libstix2/resources/versions (from $GOROOT)
/Users/don/go/src/github.com/freetaxii/libstix2/resources/versions (from $GOPATH)

Here is the output compiling server when using libstix2-0.5.1
$ env GOOS=linux GOARCH=amd64 go build -v freetaxii.go
internal/unsafeheader
runtime/internal/sys
runtime/internal/atomic
internal/race
internal/cpu
runtime/internal/math
sync/atomic
unicode
unicode/utf8
internal/bytealg
container/list
crypto/internal/subtle
math/bits
crypto/subtle
internal/testlog
unicode/utf16
math
vendor/golang.org/x/crypto/cryptobyte/asn1
internal/nettrace
vendor/golang.org/x/crypto/internal/subtle
encoding
github.com/freetaxii/libstix2/resources/versions
github.com/freetaxii/libstix2/defs
github.com/freetaxii/libstix2/vocabs
runtime
internal/reflectlite
sync
internal/singleflight
math/rand
errors
sort
internal/oserror
io
strconv
vendor/golang.org/x/net/dns/dnsmessage
syscall
bytes
hash
reflect
crypto
strings
crypto/internal/randutil
crypto/hmac
bufio
crypto/rc4
vendor/golang.org/x/crypto/hkdf
regexp/syntax
hash/crc32
time
internal/syscall/unix
internal/syscall/execenv
vendor/golang.org/x/text/transform
path
regexp
context
internal/poll
html
github.com/freetaxii/libstix2/stixid
os
encoding/binary
internal/fmtsort
path/filepath
crypto/cipher
fmt
net
crypto/sha512
crypto/aes
crypto/des
math/big
crypto/ed25519/internal/edwards25519
crypto/md5
crypto/sha1
crypto/sha256
encoding/hex
encoding/base64
io/ioutil
encoding/pem
net/url
vendor/golang.org/x/crypto/chacha20
vendor/golang.org/x/crypto/poly1305
vendor/golang.org/x/sys/cpu
vendor/golang.org/x/crypto/curve25519
vendor/golang.org/x/crypto/chacha20poly1305
encoding/json
database/sql/driver
crypto/elliptic
crypto/rand
crypto/dsa
encoding/asn1
github.com/freetaxii/libstix2/resources/envelope
crypto/ed25519
crypto/rsa
vendor/golang.org/x/crypto/cryptobyte
github.com/google/uuid
crypto/x509/pkix
database/sql
github.com/freetaxii/libstix2/timestamp
github.com/pborman/uuid
github.com/freetaxii/libstix2/objects/properties
github.com/freetaxii/libstix2/resources/properties
github.com/freetaxii/libstix2/objects/baseobject
crypto/ecdsa
github.com/freetaxii/libstix2/resources/manifest
github.com/freetaxii/libstix2/resources/collections
crypto/x509
github.com/freetaxii/libstix2/objects/indicator
github.com/gologme/log
github.com/freetaxii/libstix2/datastore
github.com/freetaxii/libstix2/resources/apiroot
github.com/freetaxii/libstix2/resources/discovery
github.com/mattn/go-sqlite3
log
compress/flate
vendor/golang.org/x/text/unicode/bidi
github.com/freetaxii/libstix2/datastore/sqlite3
compress/gzip
crypto/tls
vendor/golang.org/x/text/unicode/norm
vendor/golang.org/x/text/secure/bidirule
net/textproto
vendor/golang.org/x/net/http2/hpack
mime
mime/quotedprintable
net/http/internal
mime/multipart
vendor/golang.org/x/net/idna
github.com/freetaxii/libstix2/objects
github.com/freetaxii/libstix2/resources/status
github.com/freetaxii/libstix2/resources/taxiierror
text/template/parse
github.com/pborman/getopt
vendor/golang.org/x/net/http/httpguts
vendor/golang.org/x/net/http/httpproxy
text/template
net/http/httptrace
html/template
net/http
github.com/freetaxii/server/internal/headers
github.com/gorilla/mux
github.com/freetaxii/server/internal/config
github.com/freetaxii/server/internal/handlers

github.com/freetaxii/server/internal/handlers

../../internal/handlers/contentHandler.go:332:10: assignment mismatch: 2 variables but objects.Decode returns 3 values

Not able to build

root@root:~/go/src/github.com/freetaxii/server/cmd/freetaxii# go build freetaxii.go
../../../libstix2/datastore/sqlite3/s_baseobject.go:16:2: cannot find package "github.com/freetaxii/libstix2/objects/baseobject" in any of:
/usr/lib/go-1.14/src/github.com/freetaxii/libstix2/objects/baseobject (from $GOROOT)
/root/go/src/github.com/freetaxii/libstix2/objects/baseobject (from $GOPATH)
../../internal/config/serverConfig.go:13:2: cannot find package "github.com/freetaxii/libstix2/resources/apiroot" in any of:
/usr/lib/go-1.14/src/github.com/freetaxii/libstix2/resources/apiroot (from $GOROOT)
/root/go/src/github.com/freetaxii/libstix2/resources/apiroot (from $GOPATH)
../../../libstix2/datastore/datastore.go:8:8: cannot find package "github.com/freetaxii/libstix2/resources/collections" in any of:
/usr/lib/go-1.14/src/github.com/freetaxii/libstix2/resources/collections (from $GOROOT)
/root/go/src/github.com/freetaxii/libstix2/resources/collections (from $GOPATH)
../../internal/config/serverConfig.go:15:2: cannot find package "github.com/freetaxii/libstix2/resources/discovery" in any of:
/usr/lib/go-1.14/src/github.com/freetaxii/libstix2/resources/discovery (from $GOROOT)
/root/go/src/github.com/freetaxii/libstix2/resources/discovery (from $GOPATH)
../../../libstix2/datastore/sqlite3/t_collectiondata.go:18:2: cannot find package "github.com/freetaxii/libstix2/resources/envelope" in any of:
/usr/lib/go-1.14/src/github.com/freetaxii/libstix2/resources/envelope (from $GOROOT)
/root/go/src/github.com/freetaxii/libstix2/resources/envelope (from $GOPATH)
../../../libstix2/datastore/sqlite3/t_collectiondataManifest.go:16:2: cannot find package "github.com/freetaxii/libstix2/resources/manifest" in any of:
/usr/lib/go-1.14/src/github.com/freetaxii/libstix2/resources/manifest (from $GOROOT)
/root/go/src/github.com/freetaxii/libstix2/resources/manifest (from $GOPATH)
../../internal/handlers/contentHandler.go:18:2: cannot find package "github.com/freetaxii/libstix2/resources/status" in any of:
/usr/lib/go-1.14/src/github.com/freetaxii/libstix2/resources/status (from $GOROOT)
/root/go/src/github.com/freetaxii/libstix2/resources/status (from $GOPATH)
../../internal/handlers/errors.go:13:2: cannot find package "github.com/freetaxii/libstix2/resources/taxiierror" in any of:
/usr/lib/go-1.14/src/github.com/freetaxii/libstix2/resources/taxiierror (from $GOROOT)
/root/go/src/github.com/freetaxii/libstix2/resources/taxiierror (from $GOPATH)
../../../libstix2/datastore/sqlite3/t_collectiondata.go:19:2: cannot find package "github.com/freetaxii/libstix2/resources/versions" in any of:
/usr/lib/go-1.14/src/github.com/freetaxii/libstix2/resources/versions (from $GOROOT)
/root/go/src/github.com/freetaxii/libstix2/resources/versions (from $GOPATH)

FreeTaxii Server not starting - incompatibility with libstix2 > 0.5.1

Installing on a fresh Ubuntu Bionic docker image the freetaxii server fails to build.

root@4de5dbc2492a:/# go get -u -v github.com/freetaxii/server/cmd/freetaxii
github.com/freetaxii/server (download)
created GOPATH=/root/go; see 'go help gopath'
github.com/freetaxii/libstix2 (download)
github.com/pborman/uuid (download)
github.com/google/uuid (download)
github.com/gologme/log (download)
github.com/mattn/go-sqlite3 (download)
github.com/gorilla/mux (download)
github.com/pborman/getopt (download)
github.com/freetaxii/libstix2/resources/versions
github.com/freetaxii/libstix2/defs
github.com/freetaxii/libstix2/vocabs
github.com/freetaxii/libstix2/resources/envelope
github.com/google/uuid
github.com/gologme/log
github.com/freetaxii/libstix2/timestamp
github.com/freetaxii/libstix2/stixid
github.com/freetaxii/libstix2/objects/properties
github.com/gorilla/mux
github.com/freetaxii/server/internal/headers
github.com/mattn/go-sqlite3
github.com/pborman/uuid
github.com/pborman/getopt
github.com/freetaxii/libstix2/resources/properties
github.com/freetaxii/libstix2/objects/baseobject
github.com/freetaxii/libstix2/resources/manifest
github.com/freetaxii/libstix2/resources/collections
github.com/freetaxii/libstix2/datastore
github.com/freetaxii/libstix2/resources/apiroot
github.com/freetaxii/libstix2/objects/indicator
github.com/freetaxii/libstix2/resources/discovery
github.com/freetaxii/server/internal/config
github.com/freetaxii/libstix2/objects/attackpattern
github.com/freetaxii/libstix2/objects/campaign
github.com/freetaxii/libstix2/objects/courseofaction
github.com/freetaxii/libstix2/objects/identity
github.com/freetaxii/libstix2/objects/infrastructure
github.com/freetaxii/libstix2/objects/intrusionset
github.com/freetaxii/libstix2/objects/malware
github.com/freetaxii/libstix2/objects/observeddata
github.com/freetaxii/libstix2/objects/relationship
github.com/freetaxii/libstix2/objects/report
github.com/freetaxii/libstix2/objects/sighting
github.com/freetaxii/libstix2/objects/threatactor
github.com/freetaxii/libstix2/objects/tool
github.com/freetaxii/libstix2/objects/vulnerability
github.com/freetaxii/libstix2/resources/status
github.com/freetaxii/libstix2/resources/taxiierror
github.com/freetaxii/libstix2/objects
github.com/freetaxii/server/internal/handlers
# github.com/freetaxii/server/internal/handlers
root/go/src/github.com/freetaxii/server/internal/handlers/contentHandler.go:332:14: assignment mismatch: 3 variables but 2 values
github.com/freetaxii/libstix2/datastore/sqlite3
root@4de5dbc2492a:/# cd ~/go/src/github.com/freetaxii/server/cmd/freetaxii
root@4de5dbc2492a:~/go/src/github.com/freetaxii/server/cmd/freetaxii# go build freetaxii.go
# github.com/freetaxii/server/internal/handlers
../../internal/handlers/contentHandler.go:332:14: assignment mismatch: 3 variables but 2 values


The offending line in contentHandler.go seems to be:

                o, id, err := objects.Decode(v)

It looks like the libstix2 library was updated in Jan to use a different format, one which returns a different number of arguments

https://github.com/freetaxii/libstix2/commit/76c13db6df7c8f2740b2f654fd739b630f440447#diff-61a3fa27948243ef49db59d7d59b6b46

image

The following command which download the previous release of libstix2 fixes the problem.

cd ~/go/src/github.com/freetaxii
rm -rf libstix2
wget https://github.com/freetaxii/libstix2/archive/v0.5.1.tar.gz
tar -zxvf v0.5.1.tar.gz
ln -s libstix2-0.5.1 libstix2
root@4de5dbc2492a:~/go/src/github.com/freetaxii# cd ~/go/src/github.com/freetaxii/server/cmd/freetaxii
root@4de5dbc2492a:~/go/src/github.com/freetaxii/server/cmd/freetaxii# go build freetaxii.go
root@4de5dbc2492a:~/go/src/github.com/freetaxii/server/cmd/freetaxii#

Error when starting up freetaxii

Hi,

Thanks for working on an implementation of taxii2 server. I'm trying to install and get freetaxii to run but I've been getting errors one after another. I've sort of managed to work through most of it but now stuck at another error.

This is my folder structure
/Users/user/workspace/go/src/github.com/freetaxii/server/cmd/freetaxii

This is a tree view
screenshot 2018-12-16 at 10 54 17 pm

And this is the error I'm getting now if I execute the freetaxii binary.
2018/12/16 22:53:10 database execution error getting base index: %!(EXTRA sqlite3.Error=no such table: s_base_object)

Any advise on how to solve this?

No Access-Control-Allow-Origin

Hi Bret,
thanks for sharing this work on a TAXII 2.0 server and especially the test server.

I'm starting a TAXII 2.0 javascript client library at: https://github.com/workingDog/taxii2lib

I'm having some problems when trying to connect to your test server. I get:

Failed to load https://test.freetaxii.com:8000/taxii/: Response to preflight request
doesn't pass access control check: No 'Access-Control-Allow-Origin'
header is present on the requested resource. Origin 'http://localhost:8000'
is therefore not allowed access. The response had HTTP status code 405.
If an opaque response serves your needs, set the request's mode to 'no-cors'
to fetch the resource with CORS disabled.

I've tried many different headers and modes for my requests and also
browser (Chrome) settings on my part but to no avail. I just cannot get pass this.
Do you have any advice on how to overcome this?

Ringo

Can't build freetaxii.go

Hi all,

Probably my poor knowledge with go, but I can't succeed to build freetaxii.
Here is the error I have (extract only one)
package github.com/freetaxii/libstix2/resources/collections: cannot find package "github.com/freetaxii/libstix2/resources/collections" in any of:
/usr/local/go/src/github.com/freetaxii/libstix2/resources/collections (from $GOROOT)
/root/myGOProjects/src/github.com/freetaxii/libstix2/resources/collections (from $GOPATH)

If I go to the folder /root/myGOProjects/src/github.com/freetaxii/libstix2/resources then I have only these files/folder :
doc.go helpers resources.go

Thanks in advance for your kind help

--Kripskroll

Can't get object by id

It seems that new STIX objects are created every time the objects endpoint is queried. Every time I request it, the id/created/modified properties of all the objects are different. This makes it very hard to get an object by its id.

Objects endpoint

When testing the /<api-root>/collections/<id>/objects/ endpoint, it seems that the server returns JSON if the Accept header is set to application/vnd.oasis.taxii+json but returns HTML if it is set to application/vnd.oasis.stix+json. Looking at Section 5.3 in the spec it looks like it should accept the STIX media type.

can_write not coming through

I'm testing retrieving the collection with this:

`

<title>TAXII2</title>
<script> function loadTaxii() { var xhttp = new XMLHttpRequest(); xhttp.onreadystatechange = function() { if (this.readyState == 4 && this.status == 200) { document.getElementById("demo").innerHTML = this.responseText; } }; xhttp.open("GET", "https://test.freetaxii.com:8000/api1/collections/9cfa669c-ee94-4ece-afd2-f8edac37d8fd/", true); xhttp.setRequestHeader('Accept', 'application/vnd.oasis.taxii+json'); xhttp.setRequestHeader('Content-type', 'application/vnd.oasis.taxii+json'); xhttp.send(); }; loadTaxii(); </script> `

I get:

{"id":"9cfa669c-ee94-4ece-afd2-f8edac37d8fd","title":"Emerging Threats - Block Rules - Compromised IPs","description":"A collection of blocking rule indicators from Emerging Threats. Pulled live from: https://rules.emergingthreats.net/blockrules/emerging-botcc.excluded","can_read":true,"media_types":["application/vnd.oasis.stix+json"]}

which does not contain the required can_write attribute.

Server should return STIX 2.0 content when STIX 2.0 content is requested

I'm getting errors trying to use python-stix2 to request data from freetaxii:

collection = Collection("https://test.freetaxii.com:8000/osint/collections/2b37444f-3659-4dba-ad8e-960b4e61a546/")

# supply the TAXII2 collection to TAXIICollection
tc_source = TAXIICollectionSource(collection)

# build your filter -- kinda seems broken now
# f1 = Filter("type","=", "indicator")

#retrieve the STIX objects
results = tc_source.query(allow_custom=True)

Error is:

TAXIIServiceException: Unexpected Response Content-Type: application/vnd.oasis.stix+json; version=2.1; charset=utf-8

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.