davidmoten / flatbuffers

Maven artifacts containing compiled flatbuffers binaries and flatbuffers-java runtime library

flatbuffers

Maven artifacts for use with flatbuffers.

For years the Google flatbuffers project team did not publish artifacts of any sort for flatbuffers to repositories like Maven Central. Users were expected to build from source. In May 2021 Google started publishing the built artifacts as downloads from their releases page but did not wrap them as Maven artifacts. This project shortcuts these actions for you and allows you to do all using Maven artifacts from Maven Central.

• Supports flatbuffers 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.10, 1.12, 2.0.3
• Supports Java 1.6+ (Java 8 for 1.10+)

Status: released to Maven Central

Versioning

The artifacts carry versions like 1.6.0.3 which correspond to a 0.3 release of the flatbuffers 1.6.0 release (from google).

Current versions:

flatbuffers-compiler, flatbuffers-java	Supports
1.3.0.1	1.3
1.4.0.1	1.4
1.5.0.3	1.5
1.6.0.3	1.6
1.7.0.1	1.7
1.8.0.1	1.8
1.10.0.2	1.10
1.12.0.1	1.12
2.0.3.2	2.0.3
2.0.8	2.0.8

flatbuffers-compiler

Contains compiled binaries to generate java classes (and other) from flatbuffers schema files:

• linux (x86 64-bit) (tar.gz artifact)
• osx (tar.gz artifact) (from 1.6, 1.7 will be supported in 1.7.0.2+)
• windows (zip artifact)

This artifact is used with maven-dependency-plugin to unpack, maven-exec-plugin to execute and build-helper-maven-plugin to add the generated source to the build path. See example project for details.

flatbuffers-java

Runtime library artifact for use with flatbuffers generated java classes.

<dependency>
    <groupId>com.google.flatbuffers</groupId>
    <artifactId>flatbuffers-java</artifactId>
    <version>2.0.3</version>
</dependency>

Example usage

See example project which is also used to unit test the artifacts.

Essentially you add this block of xml to the build/plugins section of your pom.xml to generate flatbuffers classes (java):

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <version>2.10</version>
    <executions>
        <execution>
            <id>unpack</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>unpack</goal>
            </goals>
            <configuration>
                <artifactItems>
                    <artifactItem>
                        <groupId>com.github.davidmoten</groupId>
                        <artifactId>flatbuffers-compiler</artifactId>
                        <version>2.0.3.1</version>
                        <type>tar.gz</type>
                        <classifier>distribution-linux</classifier>
                        <overWrite>true</overWrite>
                        <outputDirectory>${project.build.directory}</outputDirectory>
                    </artifactItem>
                </artifactItems>
            </configuration>
        </execution>
    </executions>
</plugin>
<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>1.6.0</version>
    <executions>
        <execution>
            <goals>
                <goal>exec</goal>
            </goals>
            <phase>generate-sources</phase>
        </execution>
    </executions>
    <configuration>
        <executable>${project.build.directory}/bin/flatc</executable>
        <workingDirectory>${fbs.sources}</workingDirectory>
        <arguments>
            <argument>--java</argument>
            <argument>--gen-mutable</argument>
            <argument>-o</argument>
            <argument>${fbs.generated.sources}</argument>
            <argument>monster.fbs</argument>
        </arguments>
    </configuration>
</plugin>
<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <version>3.1.0</version>
    <executions>
        <execution>
            <id>add-source</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>add-source</goal>
            </goals>
            <configuration>
                <sources>
                    <source>${fbs.generated.sources}</source>
                </sources>
            </configuration>
        </execution>
    </executions>
</plugin>

If you run the build on different operating systems, you can use kr.motd.maven:os-maven-plugin Maven extension to make Maven download binaries specifically for your operating system.

<buiild>
    <extensions>
        <extension>
            <groupId>kr.motd.maven</groupId>
            <artifactId>os-maven-plugin</artifactId>
            <version>1.7.1</version>
        </extension>
    </extensions>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
            <version>2.10</version>
            <executions>
                <execution>
                    <id>unpack</id>
                    <phase>generate-sources</phase>
                    <goals>
                        <goal>unpack</goal>
                    </goals>
                    <configuration>
                        <artifactItems>
                            <artifactItem>
                                <groupId>com.github.davidmoten</groupId>
                                <artifactId>flatbuffers-compiler</artifactId>
                                <version>2.0.3.1</version>
                                <type>tar.gz</type>
                                <classifier>distribution-${os.detected.name}</classifier>
                                <overWrite>true</overWrite>
                                <outputDirectory>${project.build.directory}</outputDirectory>
                            </artifactItem>
                        </artifactItems>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</buiild>

There are a couple of properties mentioned in the xml block above. I set them to these values for my projects:

<properties>
    <fbs.sources>${basedir}/src/main/fbs</fbs.sources>
    <fbs.generated.sources>${project.build.directory}/generated-sources/java</fbs.generated.sources>
</properties>

To use the generated classes you'll need the runtime dependency flatbuffers-java. Add the following to the dependencies section in pom.xml:

<dependency>
    <groupId>com.github.davidmoten</groupId>
    <artifactId>flatbuffers-java</artifactId>
    <version>1.12.0.1</version>
</dependency>

How to build the binaries that are placed in this artifact

You don't need to do this (it's why this artifact exists!):

Note that google now makes available the binaries for each platform at [https://github.com/google/flatbuffers/releases]. Just download them from there and replace the zip files in flatbuffers-compiler/bin.

sudo apt-get install cmake
git clone https://github.com/google/flatbuffers.git
git checkout vN.N.N

## for flatbuffers < 1.9.0
cd flatbuffers
mkdir build
cd build
cmake ..

## for flatbuffers >= 1.9.0 on xenial
cmake .
make
mkdir bin
cp flat* bin
cp libflat* bin

The flatbuffers/build/bin directory now contains the binaries that are placed in the bin directory of this artifact.

In addition, the java source classes need to be copied from flatbuffers/java(google repo) into flatbuffers/flatbuffers-java/src/main/java.

To generate java sources manually here's an example:

cd src/fbs
../../bin/flatc --java --gen-mutable -o ../../target/generated-sources/java monster.fbs

For project maintainers

To update this project with executables for a new version of flatbuffers:

• copy the files in flatbuffers/build/bin above to flatbuffers-compiler/bin/linux/
• download flatc_windows_exe.zip from releases, unpack it and copy flatc.exe to flatbuffers-compiler/bin/windows/
• update the google flatbuffers-java dependency in example/pom.xml to latest
• build and release!

flatbuffers's Issues

Generate Java and Python sources

How would I configure the pom.xml to generate both Python and Java sources?

Having multiple exec-maven-plugin blocks, one with:

--java

and the other with:

--python

Simply overwrites the fbs.generated.sources even when the -o is pointed at distinct library's.

Attached is my current pom that describes the above.

TIA,
-Matt
pom.xml.txt

Cooperate on a Gradle plugin

I wish you could cooperate with gregwhitaker/gradle-flatbuffers-plugin#2 on a Gradle plugin which automatically did the Right Thing. We have a diverse development team, and the cost of rolling out a custom build of flatbuffers is prohibitive right now.

The version of the 'flatc.exe' is incorrect.

On the windows platform, I use the flatbuffers-compiler plugin to generate java code, it always uses the 1.12.0 version of flatc.exe, how do I configure it to use the 2.0.0 flatc.exe.Because the flatc.zip in the image above will not be automatically decompressed, so always use the 1.12.0 version of flatc.exe

1.7 binary for osx

@frew Can you contribute osx binary for the 1.7 release please?

flatbuffers-java in version 1.12.0.1 is not availabile at mvn repository

flatbuffers-java in version 1.12.0.1 is not availabile at mvn repository
https://mvnrepository.com/artifact/com.github.davidmoten/flatbuffers-java

maven-os-plugin: Improve readme

        <extensions>
            <extension>
                <groupId>kr.motd.maven</groupId>
                <artifactId>os-maven-plugin</artifactId>
                <version>1.5.0.Final</version>
            </extension>
        </extensions>

Will give you the os name you can then use like

<classifier>${os.detected.name}</classifier>

Sample configuration for multiple operating systems.

Here's a configuration that works for linux, windows, and osx. Mabe you want to include it in the readme.

I've seen issue #5 but it won't handle the classifier and the distribution correctly.

<properties>
    <fbs.sources>${project.basedir}/src/main/resources/fbs</fbs.sources>
    <fbs.generated.sources>${project.build.directory}/generated-sources/java</fbs.generated.sources>
    <fbs.executable>${project.build.directory}/bin/flatc</fbs.executable>
    <fbs.executable.package.type>tar.gz</fbs.executable.package.type>
    <fbs.executable.package.distribution>distribution-linux</fbs.executable.package.distribution>
  </properties>

  <profiles>
    <profile>
      <id>platform-windows</id>
      <activation>
        <os>
          <family>windows</family>
        </os>
      </activation>
      <properties>
        <fbs.executable>${project.build.directory}/bin/flatc.exe</fbs.executable>
        <fbs.executable.package.type>zip</fbs.executable.package.type>
        <fbs.executable.package.distribution>distribution-windows</fbs.executable.package.distribution>
      </properties>
    </profile>
    <profile>
      <id>platform-osx</id>
      <activation>
        <os>
          <family>mac</family>
        </os>
      </activation>
      <properties>
        <fbs.executable.package.distribution>distribution-osx</fbs.executable.package.distribution>
      </properties>
    </profile>
  </profiles>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <configuration>
          <archive>
            <manifestEntries>
              <Automatic-Module-Name>geogig.flatbuffers</Automatic-Module-Name>
            </manifestEntries>
          </archive>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <executions>
          <execution>
            <id>unpack</id>
            <phase>generate-sources</phase>
            <goals>
              <goal>unpack</goal>
            </goals>
            <configuration>
              <artifactItems>
                <artifactItem>
                  <groupId>com.github.davidmoten</groupId>
                  <artifactId>flatbuffers-compiler</artifactId>
                  <version>${flatbuffers-compiler.version}</version>
                  <type>${fbs.executable.package.type}</type>
                  <classifier>${fbs.executable.package.distribution}</classifier>
                  <overWrite>true</overWrite>
                  <outputDirectory>${project.build.directory}</outputDirectory>
                </artifactItem>
              </artifactItems>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <!-- See https://github.com/davidmoten/flatbuffers -->
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>exec</goal>
            </goals>
            <phase>generate-sources</phase>
          </execution>
        </executions>
        <configuration>
          <executable>${fbs.executable}</executable>
          <workingDirectory>${fbs.sources}</workingDirectory>
          <arguments>
            <argument>--java</argument>
            <argument>--gen-mutable</argument>
            <argument>-o</argument>
            <argument>${fbs.generated.sources}</argument>
            <argument>Value.fbs</argument>
            <argument>RevisionObject.fbs</argument>
          </arguments>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <version>3.1.0</version>
        <executions>
          <execution>
            <id>add-source</id>
            <phase>generate-sources</phase>
            <goals>
              <goal>add-source</goal>
            </goals>
            <configuration>
              <sources>
                <source>${fbs.generated.sources}</source>
              </sources>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

Wrong flatc binary in maven repository for osx version 2.0.3.1

Hey

Not sure if this is the right place but it seems that the Mac version of the 2.0.3.1 flatc compiler .tar.gz
https://repo.maven.apache.org/maven2/com/github/davidmoten/flatbuffers-compiler/2.0.3.1/flatbuffers-compiler-2.0.3.1-distribution-osx.tar.gz contains the wrong binary, namely version 1.12.0

Cheers.

Version 1.6.0.1: Java library "flatbuffers-java" seems to be outdated. flatc generated Java sources can not be compiled against java-lib.

Version 1.6.0.1:
Java library "flatbuffers-java" seems to be outdated. flatc generated Java sources can not be compiled against java-lib.

For
`
table HMap {
key: string(key);
val: string;
}

table Message {
a: string;
b: string;
ttl: ulong;
head: [HMap];
payload: [byte];
}
`
I get problems in the generated HMap class that methods "protected static int __offset(int vtable_offset, int offset, ByteBuffer bb)" and "protected static int __indirect(int offset, ByteBuffer bb)" from Table class are missing.

In google original code
https://github.com/google/flatbuffers/blob/master/java/com/google/flatbuffers/Table.java
both methods are there.

Further a referenced "Table.UTF8_CHARSET.get()" in the generated HMap is not found.

I'm not sure if this is a problem in your library. Maybe it is also there in googles variant. At least the last problem seems to be a general one.

Can you update this for 1.5?

Any chance of updating the jar to flatbuffers 1.5?
https://github.com/google/flatbuffers/releases/tag/v1.5.0

Version 1.9.0

Could you update to this version? Thanks