zolyfarkas / avro Goto Github PK

This project forked from pimco/avro

Mirror of Apache Avro

License: Apache License 2.0

Shell 4.21% CMake 0.31% C++ 8.24% M4 0.39% LLVM 0.10% Yacc 0.07% Python 6.97% Batchfile 0.02% C 15.47% Makefile 1.42% C# 10.51% Java 44.85% HTML 1.00% CSS 0.05% JavaScript 0.84% Thrift 0.02% Perl 1.34% PHP 2.67% Ruby 1.52% Dockerfile 0.03%

avro's People

Contributors

Stargazers

Watchers

Forkers

lukasherman yogeshsangvikar mdaum tspannhw joelplourde3 dunzoit tomerpe senthilsivanath bagaeveugene psawmora intempt

avro's Issues

Unauthorized or forbidden for install

I was using this dependecy on my project and now I''m receiving the error below:

Authorization failed for https://dl.bintray.com/zolyfarkas/core/org/apache/avro/avro/1.8.1.48p/avro-1.8.1.48p.pom 403 Forbidden

I tried upgrade to new link, using github link but now I'm receiving an another error 401 - unauthorized.

Authentication failed for https://maven.pkg.github.com/zolyfarkas/*/org/apache/avro/avro/1.10.0.1p/avro-1.10.0.1p.pom 401 Unauthorized

Can help me?

Improve field javadoc for java generated classes for unions with 2 or more non-null types

currently reading the doc it is not clear what types are accepted.

Bintray is going away, any plans to publish the binary somewhere else?

Based on this article, Bintray is going away on May.1st.2021 and will no longer serve the binary for this project.
Do you have plans for moving the binary somewhere else, if yes, what's your timeframe.
Any recommendations of how to proceed with this after May 1st for projects that depends on the project?

https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/

Unable to build project

I checked out the code from release tag avro-toplevel-1.10.0.9p, and was trying the build the project under lang/java/avro. However it fails with below exception
[ERROR] Failed to execute goal on project avro: Could not resolve dependencies for project org.apache.avro:avro:bundle:1.10.0.9p: The following artifacts could not be resolved: org.spf4j.avro:core-schema:jar:0.19, org.apache.avro:avro:jar:1.10.0.2p: Could not find artifact org.spf4j.avro:core-schema:jar:0.19 in github (https://api.github.com/users/zolyfarkas/repos/*) -> [Help 1]
[ERROR]

core-schema:jar:0.19, seems unavailable in maven repository as well.

Support for Avro 1.11

Hi, do you have any plans for supporting Avro 1.11?

java.lang.ClassCastException: class org.apache.avro.LogicalTypes$Date cannot be cast to class org.apache.avro.Date

When loading a json that in its schema uses logical type Date, the schema below

{
  "type": "int",
  "logicalType": "date"
}

I get the error

Caused by: java.lang.ClassCastException: class org.apache.avro.LogicalTypes$Date cannot be cast to class org.apache.avro.Date (org.apache.avro.LogicalTypes$Date and org.apache.avro.Date are in unnamed module of loader 'app')

lowered I'll put the complete traceback

org.apache.avro.AvroRuntimeException: Cannot convert 19073:Integer: expected generic type
        at org.apache.avro.Conversions.convertToLogicalType(Conversions.java:259)
        at org.apache.avro.generic.GenericDatumReader.convert(GenericDatumReader.java:195)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:134)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:152)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:138)
        at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:223)
        at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:213)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:144)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:138)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:115)
        at com.grupozap.validator.validation.JsonToAvroDeserializer.validate(JsonToAvroDeserializer.java:27)
        at com.grupozap.validator.validation.JsonToAvroDeserializer.validateRecord(JsonToAvroDeserializer.java:61)
        at com.grupozap.validator.service.ValidatorService.valid(ValidatorService.java:23)
        at com.grupozap.validator.controller.ValidatorController.post(ValidatorController.java:22)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1594)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.ClassCastException: class org.apache.avro.LogicalTypes$Date cannot be cast to class org.apache.avro.Date (org.apache.avro.LogicalTypes$Date and org.apache.avro.Date are in unnamed module of loader 'app')
        at org.apache.avro.data.DateConverter.fromInt(DateConverter.java:91)
        at org.apache.avro.data.DateConverter.fromInt(DateConverter.java:17)
        at org.apache.avro.Conversions.convertToLogicalType(Conversions.java:245)

Support for higher avro versions

Hi @zolyfarkas ,
Is there any plan to support higher avro versions?

We have integrated custom avro with confluent schema registry(Confluent-5.5.0), but during the upgrade found below issue and blocked on upgrade:
From 6.2.0, a method overload was deleted between Avro version 1.9.1 and 1.10.1. Avro version that is earlier than 1.10.1 might result in below error:
Caused by: java.lang.NoSuchMethodError: org.apache.avro.Schema.toString(Ljava/util/Collection;Z)Ljava/lang/String;

Any update or suggestion will be helpful. Thanks in advance.

Deleted methods with no replacement?

While upgrading from version 1.9.0.11p to 1.10.0.8p, the org.apache.avro.LogicalType class appears to have methods deleted, with no mention of how to replace them. For example, #getLogicalTypeName() has been removed. However, in that case, I see in the code that the method was simply a wrapper for #getName(), so no issue there, although I would like some sort of deprecation message before it simply breaks.

There is another method that has been removed from LogicalType called #serialize() which I don't know how to resolve. We use this method and it has been removed with no deprecation. I checked the CHANGES.md document and the commit message as well, with no luck.

The change happened here. This is our usage:

    private static void writeLogicalType(final Schema schema, final JsonGenerator writer, final Object value)
            throws IOException {
        // We are - do we have a logical type defined?
        LogicalType logicalType = schema.getLogicalType();
        if (logicalType != null) {
            String typeName = logicalType.getLogicalTypeName(); // method no longer exists, will switch to "getName()"
            // Convert the logical type and write - we assume these are implicitly primitives
            if (DecimalEncoder.OPTIMIZED_JSON_DECIMAL_WRITE && "decimal".equals(typeName)) {
                writer.writeNumber((java.math.BigDecimal) value);
            } else if (DecimalEncoder.OPTIMIZED_JSON_DECIMAL_WRITE && "bigint".equals(typeName)) {
                writer.writeNumber((java.math.BigInteger) value);
            } else {
                Object serialized = logicalType.serialize(value); // method no longer exists, replacement?
                writer.writeObject(serialized);
            }
        } else {
            // Write the best value we can find
            writeJsonSimpleDataValue(value, writer);
        }
    }

ReflectData to recognize Iterable<Map.Entry<String, Type>> and a Map.

Read Long from String (datetime)

Hello, I had one case in a project where we have some timestamp fields printed as string (we cannot fix that) so when we try to use the Json decoder it fails. Do you think this feature could be added to the readLong function? I mean, to try to parse to one (or more, configurable) datetime formats? I have done some work on this and could open a MR in case you think it is relevant.

Tks

Add any type support

"any" logical type can be applied to a record type with the fields: string avsc and bytes content.

where avsc will contain the schema, and content will contain the serialized content acording to this schema.

ExtendedJson(De|En)Coder will allow for a more natural serialization in JSON.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

Jobs

Jooble