GithubHelp home page GithubHelp logo

lensesio / schema-registry-ui Goto Github PK

View Code? Open in Web Editor NEW
415.0 36.0 112.0 10.46 MB

Web tool for Avro Schema Registry |

Home Page: http://lenses.io/product/features

JavaScript 60.78% HTML 26.16% CSS 9.46% Shell 2.68% Dockerfile 0.92%
schema-registry kafka avro

schema-registry-ui's Introduction

schema-registry-ui

release docker Join the chat at https://gitter.im/Landoop/support

This is a web tool for the confluentinc/schema-registry in order to create / view / search / evolve / view history & configure Avro schemas of your Kafka cluster.

Live Demo

schema-registry-ui.demo.lenses.io

Prerequisites

You will need schema-registry installed with CORS enabled.

In order to enable CORS, add in /opt/confluent-3.x.x/etc/schema-registry/schema-registry.properties

access.control.allow.methods=GET,POST,PUT,OPTIONS
access.control.allow.origin=*

And then restart the [schema-registry] service

Get the set up locally

We also provide the schema-registry and schema-registry-ui as part of the fast-data-dev docker image for local development setup that also gives all the relevant backends. Just run:

docker run -d --name=fast-data-dev -p 8081:8081 landoop/fast-data-dev

Checkout more about fast-data-dev docker container here

Running it via Docker

To run it via the provided docker image:

docker pull landoop/schema-registry-ui
docker run --rm -p 8000:8000 \
           -e "SCHEMAREGISTRY_URL=http://confluent-schema-registry-host:port" \
           landoop/schema-registry-ui

Please see the docker readme for more information and how to enable various features or avoid CORS issues via the proxy flag.

Build from source

    git clone https://github.com/Landoop/schema-registry-ui.git
    cd schema-registry-ui
    npm install
    npm start

Web UI will be available at http://localhost:8080

Nginx config

If you use nginx to serve this ui, let angular manage routing with

    location / {
        try_files $uri $uri/ /index.html =404;
        root /folder-with-schema-registry-ui/;
    }

Setup Schema Registry clusters

Use multiple schema registry clusters in env.js :

var clusters = [
   {
       NAME:"prod",
       // Schema Registry service URL (i.e. http://localhost:8081)
       SCHEMA_REGISTRY: "http://localhost:8081", // https://schema-registry.demo.landoop.com
       COLOR: "#141414", // optional
       readonlyMode: true // optional
     },
     {
       NAME:"dev",
       SCHEMA_REGISTRY: "http://localhost:8383",
       COLOR: "red", // optional
       allowGlobalConfigChanges: true, // optional
       //allowTransitiveCompatibilities: true        // if using a Confluent Platform release >= 3.1.1 uncomment this line
     }
  ];

  • Use COLOR to set different header colors for each set up cluster.
  • Use allowGlobalConfigChanges to enable configuring Global Compatibility Level from the UI.
  • Use allowTransitiveCompatibilities to enable transitive compatibility levels. This is supported in SR >= 3.1.1
  • Use allowSchemaDeletion to enable schema deletion from the UI. This is supported in SR >= 3.3.0
  • Use readonlyMode to prevent any configuration or schema changes from the UI. It overwrites the previous parameters (allowGlobalConfigChanges, allowSchemaDeletion).

Changelog

Here

License

The project is licensed under the BSL license.

Relevant Projects

  • kafka-topics-ui, UI to browse Kafka data and work with Kafka Topics
  • kafka-connect-ui, Set up and manage connectors for multiple connect clusters
  • fast-data-dev, Docker for Kafka developers (schema-registry,kafka-rest,zoo,brokers,landoop)
  • Landoop-On-Cloudera, Install and manage your kafka streaming-platform on you Cloudera CDH cluster

www.landoop.com

schema-registry-ui's People

Contributors

andmarios avatar antwnis avatar chdask avatar dhet avatar dsebastian avatar jglambed avatar nnordrum avatar sderosiaux avatar wallin avatar xialingxiao avatar xxx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

schema-registry-ui's Issues

Proxy environment variable doesn't seem to work

I tried launching the container with the optional "proxy" environment variable being set to true but it doesn't seem to proxy the requests.

The environment in which the container is being launched in has access to the registry. The environment from which i'm accessing the UI doesn't have access to the registry. This is why i'm trying to use the proxy variable.

The proxy variable works in other landoop tools such as the kafka-topics-ui but it doesn't seem to be working with the schema registry ui.

The only environment variables i'm passing in are SCHEMAREGISTRY_URL and PROXY. I'm setting the proxy value to 'true' and the shemaregistry_url to my schema registry url.

When loading the schema registry UI, i can see the requests for subjects and config at my schema registry URL in firebug. Those request are timing out because i need it to be proxied via the schema registry ui.

What am I missing?

add the possibility to set transitive compatibility

Since schema-registry 3.1.0, there are new option to set compatibility of a topic.
full_transitive, forward_transitive, backward_transitive

It will be nice to add this possibility to schema-registry-ui.

Can't create new Schema

When I try to create a new schema I get an alert with the message: "Selected Case Number is 1". The new schema does not seem to be registered after this happens.

The generated curl command works well though.

schema registry https support?

I have enabled https connections on my schema registry but it's within a private domain.
How do I specify the truststore or pem certificates so that the schema-registry-ui can connect to https://my-schema-registry.example.com/ ?

allow to pick an existing topic

currently an entry can be added and if the topic doesn't exist then it will be created

the opposite should be possible. picking an existing topic.

Support multiple schema-registry endpoints

In a realistic cluster - multiple micro-service instances of the schema-registry could be available

The UI should allow the user to define multiple locations (i.e. comma separated list of available schema registries ) and the API should fall-back to the next available service in case of a failure

0.9.2 only runs if installed at the root of the webserver

You can't install Schema Registry UI at a sub-directory anymore.
E.g you can't serve it at http://localhost:8080/schema-registry-ui, only at http://localhost:8080.

When build, stylesheets and scripts refer to absolute paths instead of relative. Eg.:

...
<link href="/assets/css/vendor.f29d9d10ae13a4307d6958d87a0f1e11.css" rel="stylesheet">
<link href="/assets/css/app.bccb4097051d2a32f78f78b945db9e7a.css" rel="stylesheet">
...
<script type="text/javascript" src="/js/manifest.5adf1765a1379cd1e61f.bundle.js"></script>
<script type="text/javascript" src="/js/vendor.5adf1765a1379cd1e61f.bundle.js"></script>
<script type="text/javascript" src="/js/app.5adf1765a1379cd1e61f.bundle.js"></script>
...

Build Failure on v0.9.0

Can't seem to install v0.9.0:

(.venv) โžœ  schema-registry-ui git:(f15128c) npm install        

> [email protected] postinstall /home/jramapuram/Dropbox/projects/schema-registry-ui
> bower install

(node:17514) DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
bower angular-ui-ace#bower  not-cached https://github.com/angular-ui/ui-ace.git#bower
bower angular-ui-ace#bower     resolve https://github.com/angular-ui/ui-ace.git#bower
bower ace-diff#*            not-cached https://github.com/gad2103/ace-diff.git#*
bower ace-diff#*               resolve https://github.com/gad2103/ace-diff.git#*
bower Stuk/jszip-utils=Stuk/jszip-utils#*       not-cached https://github.com/Stuk/jszip-utils.git#*
bower Stuk/jszip-utils=Stuk/jszip-utils#*          resolve https://github.com/Stuk/jszip-utils.git#*
bower Stuk/jszip=Stuk/jszip#*                   not-cached https://github.com/Stuk/jszip.git#*
bower Stuk/jszip=Stuk/jszip#*                      resolve https://github.com/Stuk/jszip.git#*
bower angular#1.5.9                                 cached https://github.com/angular/bower-angular.git#1.5.9
bower angular#1.5.9                               validate 1.5.9 against https://github.com/angular/bower-angular.git#1.5.9
bower angular-route#1.5.9                           cached https://github.com/angular/bower-angular-route.git#1.5.9
bower angular-route#1.5.9                         validate 1.5.9 against https://github.com/angular/bower-angular-route.git#1.5.9
bower angular-aria#1.5.9                            cached https://github.com/angular/bower-angular-aria.git#1.5.9
bower angular-aria#1.5.9                          validate 1.5.9 against https://github.com/angular/bower-angular-aria.git#1.5.9
bower angular-animate#1.5.9                         cached https://github.com/angular/bower-angular-animate.git#1.5.9
bower angular-animate#1.5.9                       validate 1.5.9 against https://github.com/angular/bower-angular-animate.git#1.5.9
bower bootstrap#^3.3.6                              cached https://github.com/twbs/bootstrap.git#3.3.7
bower bootstrap#^3.3.6                            validate 3.3.7 against https://github.com/twbs/bootstrap.git#^3.3.6
bower angular-material#^1.0.9                       cached https://github.com/angular/bower-material.git#1.1.3
bower angular-material#^1.0.9                     validate 1.1.3 against https://github.com/angular/bower-material.git#^1.0.9
bower angular-spinner#^0.8.1                        cached https://github.com/Attrash-Islam/bower-angular-spinner.git#0.8.1
bower angular-spinner#^0.8.1                      validate 0.8.1 against https://github.com/Attrash-Islam/bower-angular-spinner.git#^0.8.1
bower angular-sanitize#1.5.9                        cached https://github.com/angular/bower-angular-sanitize.git#1.5.9
bower angular-sanitize#1.5.9                      validate 1.5.9 against https://github.com/angular/bower-angular-sanitize.git#1.5.9
bower font-awesome#^4.6.3                           cached https://github.com/FortAwesome/Font-Awesome.git#4.7.0
bower font-awesome#^4.6.3                         validate 4.7.0 against https://github.com/FortAwesome/Font-Awesome.git#^4.6.3
bower angularUtils-pagination#^0.11.1               cached https://github.com/michaelbromley/angularUtils-pagination.git#0.11.1
bower angularUtils-pagination#^0.11.1             validate 0.11.1 against https://github.com/michaelbromley/angularUtils-pagination.git#^0.11.1
bower angular-material-data-table#^0.10.9           cached https://github.com/daniel-nagy/md-data-table.git#0.10.10
bower angular-material-data-table#^0.10.9         validate 0.10.10 against https://github.com/daniel-nagy/md-data-table.git#^0.10.9
bower angular-diff-match-patch#^0.2.4               cached https://github.com/amweiss/angular-diff-match-patch.git#0.2.4
bower angular-diff-match-patch#^0.2.4             validate 0.2.4 against https://github.com/amweiss/angular-diff-match-patch.git#^0.2.4
bower angular-json-tree#^1.0.1                      cached https://github.com/awendland/angular-json-tree.git#1.0.1
bower angular-json-tree#^1.0.1                    validate 1.0.1 against https://github.com/awendland/angular-json-tree.git#^1.0.1
bower ace-diff#*                                  checkout master
bower angular-ui-ace#bower                        checkout bower
bower angular-ui-ace#bower                    invalid-meta angular-ui-ace is missing "ignore" entry in bower.json
bower angular-ui-ace#bower                        resolved https://github.com/angular-ui/ui-ace.git#36844ff7c0
bower ace-diff#*                                  resolved https://github.com/gad2103/ace-diff.git#917afdc876
bower Stuk/jszip-utils=Stuk/jszip-utils#*           ENOENT ENOENT: no such file or directory, mkdir '/tmp/jramapuram/bower/Stuk/jszip-utils=Stuk/jszip-utils-17514-2hNk2k'

Stack trace:
Error: ENOENT: no such file or directory, mkdir '/tmp/jramapuram/bower/Stuk/jszip-utils=Stuk/jszip-utils-17514-2hNk2k'

Console trace:
Trace
    at StandardRenderer.error (/home/jramapuram/Dropbox/projects/schema-registry-ui/node_modules/bower/lib/renderers/StandardRenderer.js:82:17)
    at Logger.<anonymous> (/home/jramapuram/Dropbox/projects/schema-registry-ui/node_modules/bower/bin/bower:110:22)
    at emitOne (events.js:96:13)
    at Logger.emit (events.js:191:7)
    at Logger.emit (/home/jramapuram/Dropbox/projects/schema-registry-ui/node_modules/bower-logger/lib/Logger.js:29:39)
    at /home/jramapuram/Dropbox/projects/schema-registry-ui/node_modules/bower/lib/commands/index.js:40:20
    at _rejected (/home/jramapuram/Dropbox/projects/schema-registry-ui/node_modules/q/q.js:797:24)
    at /home/jramapuram/Dropbox/projects/schema-registry-ui/node_modules/q/q.js:823:30
    at Promise.when (/home/jramapuram/Dropbox/projects/schema-registry-ui/node_modules/q/q.js:1035:31)
    at Promise.promise.promiseDispatch (/home/jramapuram/Dropbox/projects/schema-registry-ui/node_modules/q/q.js:741:41)

System info:
Bower version: 1.3.12
Node version: 7.7.1
OS: Linux 4.9.13-1-MANJARO x64

npm ERR! Linux 4.9.13-1-MANJARO
npm ERR! argv "/usr/bin/node" "/usr/bin/npm" "install"
npm ERR! node v7.7.1
npm ERR! npm  v4.3.0
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] postinstall: `bower install`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] postinstall script 'bower install'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the schema-registry-ui package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     bower install
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs schema-registry-ui
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls schema-registry-ui
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/jramapuram/.npm/_logs/2017-03-13T12_30_36_944Z-debug.lo
(.venv) โžœ  schema-registry-ui git:(f15128c) npm --version
4.3.0
(.venv) โžœ  schema-registry-ui git:(f15128c) node --version
v7.7.1

I even updated bower systemwide, but it doesn't seem to get used?

(.venv) โžœ  schema-registry-ui git:(f15128c) sudo npm install -g bower
/usr/bin/bower -> /usr/lib/node_modules/bower/bin/bower
/usr/lib
โ””โ”€โ”€ [email protected] 

(.venv) โžœ  schema-registry-ui git:(f15128c) bower

Usage:

    bower <command> [<args>] [<options>]
Commands:

    cache                   Manage bower cache
    help                    Display help information about Bower
    home                    Opens a package homepage into your favorite browser
    info                    Info of a particular package
    init                    Interactively create a bower.json file
    install                 Install a package locally
    link                    Symlink a package folder
    list                    List local packages - and possible updates
    login                   Authenticate with GitHub and store credentials
    lookup                  Look up a single package URL by name
    prune                   Removes local extraneous packages
    register                Register a package
    search                  Search for packages by name
    update                  Update a local package
    uninstall               Remove a local package
    unregister              Remove a package from the registry
    version                 Bump a package version
Options:

    -f, --force             Makes various commands more forceful
    -j, --json              Output consumable JSON
    -l, --loglevel          What level of logs to report
    -o, --offline           Do not hit the network
    -q, --quiet             Only output important information
    -s, --silent            Do not output anything, besides errors
    -V, --verbose           Makes output more verbose
    --allow-root            Allows running commands as root
    -v, --version           Output Bower version
    --no-color              Disable colors
See 'bower help <command>' for more information on a specific command.
(.venv) โžœ  schema-registry-ui git:(f15128c) bower --version
1.8.0

I even tried setting up a nodeenv and that resulted in the same situation.

Getting Connectivity error (Docker compose)

Hi,

Testing this UI with the docker-compose.yml below but when accessing Schema Registry UI (http://localhost:8000) I get Connectivity Error.
Testing Schema Registry on http://localhost:8084/config in a browser I get
{"compatibilityLevel":"BACKWARD"}

docker-compose.yml

version: '2.1'

services:
  zoo1:
    image: zookeeper:3.4.9
    container_name: zoo1
    hostname: zoo1
    ports:
      - "2181:2181"
    environment:
        ZOO_MY_ID: 1
        ZOO_PORT: 2181
        ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
    volumes:
      - ./zk-multiple-kafka-multiple/zoo1/data:/data
      - ./zk-multiple-kafka-multiple/zoo1/datalog:/datalog

  zoo2:
    image: zookeeper:3.4.9
    container_name: zoo2
    hostname: zoo2
    ports:
      - "2182:2182"
    environment:
        ZOO_MY_ID: 2
        ZOO_PORT: 2182
        ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
    volumes:
      - ./zk-multiple-kafka-multiple/zoo2/data:/data
      - ./zk-multiple-kafka-multiple/zoo2/datalog:/datalog

  zoo3:
    image: zookeeper:3.4.9
    container_name: zoo3
    hostname: zoo3
    ports:
      - "2183:2183"
    environment:
        ZOO_MY_ID: 3
        ZOO_PORT: 2183
        ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
    volumes:
      - ./zk-multiple-kafka-multiple/zoo3/data:/data
      - ./zk-multiple-kafka-multiple/zoo3/datalog:/datalog


  kafka1:
    image: confluentinc/cp-kafka:5.0.1
    container_name: kafka1
    hostname: kafka1
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka1:19092,LISTENER_DOCKER_EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_INTERNAL
      KAFKA_ZOOKEEPER_CONNECT: "zoo1:2181,zoo2:2182,zoo3:2183"
      KAFKA_BROKER_ID: 1
      KAFKA_LOG4J_LOGGERS: "kafka.controller=INFO,kafka.producer.async.DefaultEventHandler=INFO,state.change.logger=INFO"
    volumes:
      - ./zk-multiple-kafka-multiple/kafka1/data:/var/lib/kafka/data
    depends_on:
      - zoo1
      - zoo2
      - zoo3

  kafka2:
    image: confluentinc/cp-kafka:5.0.1
    container_name: kafka2
    hostname: kafka2
    ports:
      - "9093:9093"
    environment:
      KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka2:19093,LISTENER_DOCKER_EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_INTERNAL
      KAFKA_ZOOKEEPER_CONNECT: "zoo1:2181,zoo2:2182,zoo3:2183"
      KAFKA_BROKER_ID: 2
      KAFKA_LOG4J_LOGGERS: "kafka.controller=INFO,kafka.producer.async.DefaultEventHandler=INFO,state.change.logger=INFO"
    volumes:
      - ./zk-multiple-kafka-multiple/kafka2/data:/var/lib/kafka/data
    depends_on:
      - zoo1
      - zoo2
      - zoo3

  kafka3:
    image: confluentinc/cp-kafka:5.0.1
    container_name: kafka3
    hostname: kafka3
    ports:
      - "9094:9094"
    environment:
      KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka3:19094,LISTENER_DOCKER_EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9094
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_INTERNAL
      KAFKA_ZOOKEEPER_CONNECT: "zoo1:2181,zoo2:2182,zoo3:2183"
      KAFKA_BROKER_ID: 3
      KAFKA_LOG4J_LOGGERS: "kafka.controller=INFO,kafka.producer.async.DefaultEventHandler=INFO,state.change.logger=INFO"
    volumes:
      - ./zk-multiple-kafka-multiple/kafka3/data:/var/lib/kafka/data
    depends_on:
      - zoo1
      - zoo2
      - zoo3

      
  # https://github.com/yahoo/kafka-manager/  
  kafka_manager:
    image: hlebalbau/kafka-manager:stable
    container_name: kafka-manager
    hostname: kafka-manager
    ports:
      - "9000:9000"
    environment:
      ZK_HOSTS: "zoo1:2181,zoo2:2182,zoo3:2183"
      APPLICATION_SECRET: "random-secret"
    command: -Dpidfile.path=/dev/null

    

  kafka-schema-registry:
    image: confluentinc/cp-schema-registry:5.0.1
    container_name: kafka-schema-registry
    hostname: kafka-schema-registry
    ports:
      - "8084:8081"
    environment:
      SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: PLAINTEXT://kafka1:19092,PLAINTEXT://kafka2:19093,PLAINTEXT://kafka3:19094
      SCHEMA_REGISTRY_HOST_NAME: kafka-schema-registry
      SCHEMA_REGISTRY_LISTENERS: http://0.0.0.0:8081
    depends_on:
      - zoo1
      - zoo2
      - zoo3
      - kafka1
      - kafka2
      - kafka3

  # https://hub.docker.com/r/landoop/schema-registry-ui
  schema-registry-ui:
    image: landoop/schema-registry-ui:latest
    container_name: kafka-schema-registry-ui
    hostname: kafka-schema-registry-ui
    ports:
      - "8000:8000"
    environment:
      SCHEMAREGISTRY_URL: http://localhost:8084/
      #PROXY: "true"
      ALLOW_GLOBAL: "true"
      ALLOW_TRANSITIVE: "true"
      ALLOW_DELETION: "true"
    depends_on:
      - kafka-schema-registry


Schema Registry in https

Hello,

What are the variables needed in the docker container in order to communicate with a Schema Registry configured in https ?

Thanks.

Issue creating and reusing custom types

Hi,

Using the Schema UI - is it possible to create a custom type which I can reuse?

So say I have the following:

{
    "type": "record",
    "name": "TestSchema",
    "namespace": "com.test",
    "fields": [
        {
            "name": "title",
            "type": {
                "type": "record",
                "name": "CustomType",
                "fields": [
                    {
                        "name": "value",
                        "type": "string"
                    },
                    {
                        "name": "secondLine",
                        "type": "string"
                    }
                ]
            }
        },
        {
            "name": "subtitle",
            "type": "CustomType"
        }
    ]
}

On validating the SchemaUI will complain that it does not know CustomType.

Even if I create a new schema and then try to reference the type using namespace.name doesn't work.

Is it possible to do this using Schema UI - or would i have to duplicate each time I want to use that specific type.

Thanks.

Can't evolve schema after setting per-topic compatibility

Evolving schemas with the Schema Registry UI usually is a fairly simple and convenient process.

However, using version 0.9.4 together with Confluent Platform 4.0, the following steps lead to a situation where I can't remove fields from a previously created schema.

  1. Create a new schema
  2. Set compatibility level to NONE (my global level is BACKWARD)
  3. Edit schema, remove one of the fields, click "Validate", the tooltip "You can now evolve your schema" displays as expected
  4. Click "Evolve schema"

The UI seems to accept the schema evolution and doesn't display an error, but it still shows the previous schema version with the exact schema as before. Nothing changed.

I am able to evolve the schema correctly and get a new version of my schema if I just add a field.

The output of `kafka-configs --zookeeper {myzookeeper} --entity-type topics --entity-name {mytopic} --describe is empty, although the compatibility level was changed according to the UI.

Any ideas?

Support configurable port via ENV variable

Hello,

Could you please add a possibility to customize server port when running UI in docker via ENV variable.
E.g.

$ docker run -e SCHEMA_REGISTRY_UI_PORT=9999 

This scenario is quite useful when one doesn't want to expose separate port for instance in Docker Swarm cluster, but use default proxy setting where services are routed to the same port.

Special Characters In Schema Are Not Escaped

When saving a schema with a escaped characters in a field value (e.g. \n, \t, \") the server validation fails with 500 error.

Example schema (note \n at the beginning of "doc" field):
{ "type": "record", "name": "evolution", "doc": "\nThis is a sample Avro schema to get you started. Please edit", "namespace": "com.landoop", "fields": [ { "name": "name", "type": "string" } ] }

The error happens because these characters are not escaped in the HTTP request:
image
The request should contain \\n instead of \n.

Collapse -key and -value in the list

Left hand side displays all the entries added.
Would be good to just display the topic and then it allows to chose the key or value.
it will make things a lot tidier

only latest docker image handles the PORT environment variable

We are currently using the PORT environment variable for the docker image, that makes our life easier for the topology that we are currently using in our Kubernetes clusters.
But the PORT environment variable is only available in the latest docker image. I would like to lock the docker version down to a specific version instead of using latest.
Maybe you can publish a new docker image version including the PORT environment variable on docker hub with a specific version, such as 0.9.5 if you are going to do a next release?
What do you think?

Thank you

Basic auth in Docker causes CONNECTIVITY ERROR

When using the following command to run your official docker image everything works fine:
docker run --rm -p 8080:8080 -e "SCHEMAREGISTRY_URL=https://user:xxx@url:port" -e "ALLOW_GLOBAL=1" -e "ALLOW_TRANSITIVE=1" -e "ALLOW_DELETION=0" -e "PROXY=1" -e "PORT=8080" landoop/schema-registry-ui:0.9.5

When I add basic auth like this:
docker run --rm -p 8080:8080 -e "SCHEMAREGISTRY_URL=https://user:xxx@url:port" -e "ALLOW_GLOBAL=1" -e "ALLOW_TRANSITIVE=1" -e "ALLOW_DELETION=0" -e "PROXY=1" -e "PORT=8080" -e "CADDY_OPTIONS=basicauth / admin admin" landoop/schema-registry-ui:0.9.5

Suddenly I get Url : /api/schema-registry CONNECTIVITY ERROR after I log in with admin/admin.
The container logs the following:

172.17.0.1 - - [01/Oct/2018:09:38:29 +0000] "GET /api/schema-registry/config HTTP/1.1" 502 16
172.17.0.1 - - [01/Oct/2018:09:38:30 +0000] "GET /api/schema-registry/subjects/ HTTP/1.1" 502 16

Also, I have noticed the requests actually never hit our Kafka cluster.

After removing the basic auth env var from the docker start command I have to clear my browser cache to make the schema-registry-ui work again.

Support for Delete Schema from UI

Delete functionality is recently added back to Schema Registry (Schema deletion #532). Is there any plan to add delete option/button in Schema Registry UI? is there any development going on for this?

Avro JSON not correctly displayed in UI - however is in the terminal

Hi,

I have topic that is created from a Kafka stream - a JSON serialized into an Avro object.

When I view the topic in the terminal i.e. using kafka-avro-console-consumer I correctly see it as:

{"id":"H7fed0430"}

However when I view in the UI - it doesn't show the key id neither does it show the correct data type as Avro.

screen shot 2018-04-23 at 16 16 58

Any ideas?

Export schemas

Export schemas in a bash script that can easily be customised (i.e. point it to a new schema registry) to restore them

Get global config rejection : 502 Bad Gateway

I got stuck with this and wanted to share my resolution in case anyone else hits the same thing. I'm running the schema-registry-ui as a Docker container with the following command:

docker run --rm -p 8000:8000 \
           -e "SCHEMAREGISTRY_URL=http://127.0.0.1:32797" \
           -e "PROXY=true" \
           landoop/schema-registry-ui

As it would happen, Schema Registry, Kafka, and Zookeeper are also all running in a separate Docker container. Schema Registry is exposed on port 8081 in the container, which maps to port 32797 on the host.

When I connect to http://localhost:8000 to view the schema-registry-ui, and view the console w/ Chrome Web Dev Tools, I see this error:

Failure with : "Get global config rejection : 502 Bad Gateway\n 502"

Indeed, when I visit http://localhost:8000/api/schema-registry/config/, I see 502 Bad Gateway.

After reading the docs, I realized that schema-registry-ui is attempting to connect to 127.0.0.1 from within the schema-registry-ui container, which of course resolves to something other than my host machine. The solution was to find my Mac's IP on the local network and use that instead:

$ ifconfig 
en0: ...
	inet 10.0.1.88 netmask ...
	...

Then the updated Docker command worked:

docker run --rm -p 8000:8000 \
           -e "SCHEMAREGISTRY_URL=http://10.0.1.88:32797" \
           -e "PROXY=true" \
           landoop/schema-registry-ui

Problem with subjects list while fetching subjects

Hello!

First of all, I wanna say that schema-registry-ui looks really nice and have nice feature like diff. At the moment we are thinking how to visualize schemas stored in schema-registry.

I have tried to run it, but I have problem with subject lists which contains only empty entries. I think, this is because of:

https://github.com/Landoop/schema-registry-ui/blob/master/src/factories/schema-registry-factory.js#L454

Where according to Confluent API:

http://docs.confluent.io/3.1.1/schema-registry/docs/api.html#get--subjects-(string-%20subject)-versions-(versionId-%20version)

answer does not contain any JSON field named "subject".

What version of schema-registry was checked with schema-registry-ui?
If that is the case, maybe subject could be somehow read from URL or in other way instead from the response.

What do you think?
Do you have any suggestion?

Kind regards,
Rafal Kluszczynski

Handle schema definition arrays

If you want to share some sub-components between schemas the cleanest way to do it (i.e. readability) is to put them into an array like a union e.g.

[
  {
    "type": "record",
    "name": "KafkaEventHeader",
    "namespace": "com.example.kafka.core",
    "doc" : "Our standard kafka header",
    "fields": [
        {
            "name": "correlationId",
            "type": "string",
            "doc" : "Unique identifier e.g. GUID that is passed to other events created from this one so that we can track across processes"
        },
        {
            "name": "processPath",
            "doc" : "A breadcrumbs structure to help analyse the provenance of an event",
            "type": [
                "null",
                {
                    "type": "array",
                    "items": {
                        "type": "record",
                        "namespace": "com.example.kafka.core",
                        "name": "ProcessPathElement",
                        "semVer" : "1.0.0",
                        "docURL" : "https://avro.example.com/kafka/core/ProcessPathElement/1.0.0",
                        "doc" : "Struct that contains info about each process step",
                        "fields": [
                            {
                                "name": "source",
                                "type": "string",
                                "doc" : "Name of the processor that generated this event"
                            },
                            {
                                "name": "timestamp",
                                "type": "string"
                            }
                        ]
                    }
                }
            ]
        }
    ]
  },
  {
    "type": "record",
    "name": "CustomerValue",
    "namespace": "com.example.customer",
    "doc" : "A person with a favourite colour",
    "fields": [
        {
            "name": "header",
            "type": "com.example.kafka.core.KafkaEventHeader"
        },
        {
            "name": "userId",
            "type": "string"
        },
        {
            "name": "favouriteColour",
            "type": "string"
        }
    ]
  }
]

In this example our actual record is created with the CustomerValue schema, which includes the KafkaEventHeader etc.

This is valid Avro, it works fine with the Kafka SerDes, and it works fine with the Schema Registry, although the Schema Registry merges the KafkaEventHeader into the CustomerValue so that the record definitions are inline. As far as I can tell it actually creates a GenericRecord (or something), then extracts the schema from that, so it's actually Avro tooling that merges the definitions.

However when using this approach the Info view in the schema-registry-ui is just blank.

Would it be possible to update the Info view to iterate through all of the record/schema definitions in the schema file?

Display a schema in a table view alongside the json editor

This would be a nice to have. while is great to have the flexibility with editing raw json, it could be nice to have a grid displaying the rows for each field flagging if the field is nullable and presenting the type (or logical type) and allowing to quickly search for a field.

Schema definition UI can't validate schema nested values within arrays

This is related to #43, which was closed, but not resolved. The problem arises when a schema element refers to one defined above, as in the example shown below. This fails to validate in the UI, but will validate when posting the UI-generated CURL command from the command line.

[{ "namespace": "com.example", "name": "SearchRecordAttr", "type": "record", "fields":[ { "name": "productUrl", "type": "string" } ] }, { "namespace": "com.example", "name": "SearchRecord", "type": "record", "fields":[ { "name": "attributes", "default":null, "type": ["null", "SearchRecordAttr"]} ] }]

Module not found: Error: Can't resolve 'babel-runtime/helpers/typeof'

npm start throws exception

ERROR in ./src/schema-registry/new/new.controller.js
Module not found: Error: Can't resolve 'babel-runtime/helpers/typeof' in 'D:\lab\schema-registry-ui\src\schema-registry\new'
 @ ./src/schema-registry/new/new.controller.js 2:0-51
 @ ./src/schema-registry/index.js
 @ ./src/app.js
 @ multi (webpack)-dev-server/client?http://localhost:8080 webpack/hot/dev-server ./src/app.js

solved by :

npm install -save babel-runtime

Docker image broken by change to support multiple environments

The run.sh script in the docker folder longer updates the JS correctly to provide the SCHEMAREGISTRY_URL.

From docker/run.js

else
    echo "Setting Schema Registry URL to $SCHEMAREGISTRY_URL."
    sed -e 's|^\s*var SCHEMA_REGISTRY =.*|  var SCHEMA_REGISTRY = "'"$SCHEMAREGISTRY_URL"'";|' \
        -i /schema-registry-ui/combined.js
fi

Is "Evolve Schema" intended to create a new version of a schema?

I've been using the kafka-topics-ui and schema-registry-ui over recent weeks and find them very useful.

As per below, I tried editing the field "private" so that it became a 'string' from an 'int'.
On hittiing 'evolve schema' ... is that intended to be the equivalent of a curl -X POST http://schema-registry:8081/subjects/my-connector/versions -d '{ ... }' ... in order to create a new version of the schema?

image

I couldn't get it to work ... although, admittedly ... when I try to generate a new schema using the schema-registry API ... it also fails with a 409 conflict. I even tried to delete the topic .. in case this conflict was being caused by existing entries. Do you think the change I'm making to the 'private' field qualifies as a legitimate schema change?

curl -i -X POST -H "Content-Type: application/json" http://localhost:8081/subjects/ckan-meta-package-value/versions --data '{"schema": "{\"type\":\"record\",\"name\":\"package\",\"fields\":[{\"name\":\"id\",\"type\":\"string\"},{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"title\",\"type\":[\"null\",\"string\"]},{\"name\":\"version\",\"type\":[\"null\",\"string\"]},{\"name\":\"url\",\"type\":[\"null\",\"string\"]},{\"name\":\"notes\",\"type\":[\"null\",\"string\"]},{\"name\":\"license_id\",\"type\":[\"null\",\"string\"]},{\"name\":\"revision_id\",\"type\":[\"null\",\"string\"]},{\"name\":\"author\",\"type\":[\"null\",\"string\"]},{\"name\":\"author_email\",\"type\":[\"null\",\"string\"]},{\"name\":\"maintainer\",\"type\":[\"null\",\"string\"]},{\"name\":\"maintainer_email\",\"type\":[\"null\",\"string\"]},{\"name\":\"state\",\"type\":[\"null\",\"string\"]},{\"name\":\"type\",\"type\":[\"null\",\"string\"]},{\"name\":\"owner_org\",\"type\":[\"null\",\"string\"]},{\"name\":\"private\",\"type\":[\"null\",\"string\"]},{\"name\":\"metadata_modified\",\"type\":[\"null\",{\"type\":\"long\",\"connect.version\":1,\"connect.name\":\"org.apache.kafka.connect.data.Timestamp\"}]},{\"name\":\"creator_user_id\",\"type\":[\"null\",\"string\"]}],\"connect.name\":\"package\"}"}'
HTTP/1.1 100 Continue

HTTP/1.1 409 Conflict
Date: Mon, 03 Oct 2016 10:29:33 GMT
Content-Type: application/vnd.schemaregistry.v1+json
Content-Length: 93
Server: Jetty(9.2.12.v20150709)

{"error_code":409,"message":"Schema being registered is incompatible with the latest schema"}

It seems the 'schema-registry API' doesn't allow for deleting schemas?

Also, does the output from the schema window (below) qualify as an '*.avsc' file-type ... and is a simple copy and paste?

image

Thanks.

Validating schema containing a simple enum fails

When adding a new schema containing an enum, the validating step fails and consequently, the schema can not be added.

Example schema:

{
   "type": "record",
   "name": "Test",
   "fields": [
      {
        "name": "category",
        "type": {
          "type": "enum",
          "name": "TestEnum",
          "symbols": [
            "SYMBOL1",
            "SYMBOL2",
            "SYMBOL3",
            "SYMBOL4",
            "SYMBOL5"
          ]
        }
     }
  ]
}

The error message says "SYMBOL5 is not valid". It doesn't matter what name the symbols have, the error message will always complain about the last symbol in the enum.

Backwards compatilibity with 2.* ?

Just tried this on a server running on a machine running confluent-platform-2.11.5 and it fails with a connectivity error. Any chances of making this backwards compatible?

Proxy multiple env clusters?

I've modified the env.js and made use of it, but I lost the ability to use PROXY=true from the run.sh script. How do I setup proxy for multiple environments? I see that the caddy file is what sets up the proxy but not sure how to apply multiple SCHEMAREGISTRY_URL.

Connectivity error when used with salsify/avro-schema-registry

We want to deploy a schema registry just for documentation purposes atm, we have a postgres set up already and we do not have access to any kafka cluster, so instead of using the default confluent schema registry we found the following one https://github.com/salsify/avro-schema-registry

They claim to support all api endpoints of the confluent one, however, when i set both the schema registry and the ui up, the ui gives a CONNECTIVITY error.

If I use the curl command provided by the ui everything connects and works great. Any idea why? is the ui using some confluent schema registry specific features?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.