GithubHelp home page GithubHelp logo

bblfsh / scala-client Goto Github PK

View Code? Open in Web Editor NEW
5.0 5.0 8.0 729 KB

Babelfish Scala client

Home Page: https://doc.bblf.sh/using-babelfish/clients.html

License: Apache License 2.0

Scala 41.00% Shell 22.04% C++ 36.96%
babelfish client scala

scala-client's Introduction

Babelfish Scala client Build Status

This is a Scala/JNI implementation of the Babelfish client. It uses ScalaPB for Protobuf/gRPC code generation and libuast for XPath queries.

Status

The latest scala-client v2.x supports the UASTv2 protocol.

Installation

Requirements

Java SDK 8

You need to install the Java SDK and its header files. The command for Debian and derived distributions would be:

sudo apt install openjdk-8 openjdk-8-jdk-headless
Scala 2.11

Currently we support Scala 2.11. We recommend using scalaVersion := "2.11.11" or above in your build.sbt file. This library is not compatible with Scala 2.12 or 2.13.

Building from sources

git clone https://github.com/bblfsh/scala-client.git
cd scala-client
./build.sh --clean --get-dependencies --all

This command would fetch dependencies for the package, compile native code (JNI) and the Scala code. gRPC/protobuf files are re-generate from src/main/proto on every ./sbt compile and are stored under ./target/src_managed/.

The jar file and the native module are generated in the build/ directory. If you move the jar file to some other path, the native (.so or .dylib) library must be in the same path.

If the build fails because it can't find the jni.h header file, run it setting JAVA_HOME (appropriately setting it to the right location for your system):

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
./build.sh --clean --get-dependencies --all

If the build fails with: build.sh: Permission denied, add execution permissions to the script:

chmod +x ./build.sh

build.sh script has been tested in Unix (Linux and macOS) and with Git Bash in Windows.

For more developer documentation please check our CONTRIBUTING guideline.

Apache Maven

The bblfsh-client package is available thorugh Maven central, so it can be easily added as a dependency in various package management systems. Examples of how to handle it for most common systems are included below; for other systems just look at Maven central's dependency information.

<dependency>
    <groupId>org.bblfsh</groupId>
    <artifactId>bblfsh-client</artifactId>
    <version>${version}</version>
</dependency>

Scala sbt

libraryDependencies += "org.bblfsh" % "bblfsh-client" % version

Usage

If you don't have a bblfsh server running you can execute it using the following command:

docker run --privileged --rm -it -p 9432:9432 --name bblfsh bblfsh/bblfshd

Please, read the getting started guide to learn more about how to use and deploy a bblfsh server, install language drivers, etc.

API

import scala.io.Source
import org.bblfsh.client.v2.BblfshClient, BblfshClient._
import gopkg.in.bblfsh.sdk.v2.protocol.driver.Mode

val client = BblfshClient("localhost", 9432)

val filename = "/path/to/file.py" // client responsible for encoding it to utf-8
val fileContent = Source.fromFile(filename).getLines.mkString("\n")
val resp = client.parse(filename, fileContent, Mode.SEMANTIC)

// Full response
println(resp.get)

// Filtered response
val it = client.filter(resp.get, "//uast:Identifier")
it.foreach(println)

Please read the Babelfish clients guide section to learn more about babelfish clients and their query language.

License

Apache 2.0

scala-client's People

Contributors

abeaumont avatar bzz avatar creachadair avatar erizocosmico avatar juanjux avatar ncordon avatar smacker avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

scala-client's Issues

fatal error during parsing

The easiest way to reproduce bug is to replace current java file with my go file (see links below) in BblfshClientParseTest. And then run ./sbt test. It produces an error:

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x000000010f71c270, pid=30822, tid=0x000000000000420f
#
# JRE version: Java(TM) SE Runtime Environment (8.0_152-b16) (build 1.8.0_152-b16)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.152-b16 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# V  [libjvm.dylib+0x31c270]  jni_GetObjectField+0x6e
#
# Core dump written. Default location: /cores/core or core.30822
#
# An error report file with more information is saved as:
# /Users/smacker/Work/client-scala/hs_err_pid30822.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp

Go file: https://gist.github.com/smacker/60e16f66aa4198eee533b229b11986a4
Log: https://gist.github.com/smacker/817eeb635e2ff896657cebb2aac5b3a9

I also tried to recompile client with libuast v1.9.5 but it didn't help.

Encoding UAST back does not produce the same bytes

A new test case when a decode UAST serialized though a new, different from original one, context does not result in the same bytes:

    // given
    val fileContent = Source.fromFile(fileName).getLines.mkString("\n")
    val resp = client.parse(fileName, fileContent)
    val node = resp.get

    // when
    var encodedBytes = node.toByteBuffer

    // then
    encodedBytes.capacity should be(resp.uast.asReadOnlyByteBuffer.capacity)
    encodedBytes shouldEqual resp.uast.asReadOnlyByteBuffer

In particular, this seems to be the case for JUint nodes.

v2: add a cache to JNI part

On the JNI side, we can cache a lot of things as global references instead of calling JVM every time:

  • classes, instead of env->FindClass
  • methodIds, instead of env->GetMethodID for things in hot checkJvmException

This can be done in JNI_OnLoad and all global reference must be deleted e.g in JNI_OnUnLoad.

Incorrect bytes in decode kill sbt

This issue is analogous to python-client#189:

import org.bblfsh.client.v2._, BblfshClient._
import java.nio.ByteBuffer

val bytes = ByteBuffer.allocateDirect(3)
bytes.put('a'.toByte)
bytes.put('b'.toByte)
bytes.put('c'.toByte)

decode(bytes)

crashes sbt with an uncaught std::runtime_error

Crash loading libscalauast in linux

It crashes with this error:

Loading native libscalauast
576function symbol table not sorted by program counter: 0x7fa90da9acc0 _cgo_topofstack > 0x7fa901ace6a0 runtime.goexit
577	0x7fa901a78530 sync/atomic.(*Value).Load
578	0x7fa901a78570 sync/atomic.(*Value).Store
579	0x7fa901a786b0 sync/atomic.CompareAndSwapUintptr
580	0x7fa901a786c0 sync/atomic.StoreUint32
581	0x7fa901a786d0 sync/atomic.StoreUintptr
582	0x7fa901a786e0 internal/cpu.Initialize
583	0x7fa901a78730 internal/cpu.processOptions
584	0x7fa901a78dd0 internal/cpu.indexByte
585	0x7fa901a78e10 internal/cpu.doinit
586	0x7fa901a79220 internal/cpu.cpuid
587	0x7fa901a79240 internal/cpu.xgetbv
588	0x7fa901a79260 type..hash.internal/cpu.CacheLinePad
589	0x7fa901a79270 type..eq.internal/cpu.CacheLinePad
590	0x7fa901a79280 type..hash.internal/cpu.arm64
591	0x7fa901a792e0 type..eq.internal/cpu.arm64
592	0x7fa901a79350 type..hash.internal/cpu.option
593	0x7fa901a793d0 type..eq.internal/cpu.option
594	0x7fa901a79480 type..hash.internal/cpu.x86
595	0x7fa901a794e0 type..eq.internal/cpu.x86
596	0x7fa901a79540 type..hash.[15]internal/cpu.option
597	0x7fa901a795b0 type..eq.[15]internal/cpu.option
598	0x7fa901a79640 runtime/internal/atomic.Cas64
599	0x7fa901a79660 runtime/internal/atomic.Casuintptr
600	0x7fa901a79670 runtime/internal/atomic.Storeuintptr
601	0x7fa901a79680 runtime/internal/atomic.Store
602	0x7fa901a79690 runtime/internal/atomic.Store64
603	0x7fa901a796a0 internal/bytealg.countGeneric
604	0x7fa901a796e0 internal/bytealg.countGenericString
605	0x7fa901a79720 internal/bytealg.init.0
606	0x7fa901a79750 internal/bytealg.init
607	0x7fa901a797b0 internal/bytealg.Compare
608	0x7fa901a797d0 runtime.cmpstring
609	0x7fa901a797f0 cmpbody
610	0x7fa901a79a30 internal/bytealg.Count
957	0x7fa901a79a60 internal/bytealg.CountString
958	0x7fa901a79a90 countbody
959	0x7fa901a79c10 internal/bytealg.Equal
960	0x7fa901a79c50 runtime.memequal
961	0x7fa901a79c80 runtime.memequal_varlen
962	0x7fa901a79cb0 memeqbody
963	0x7fa901a79df0 internal/bytealg.Index
964	0x7fa901a79e20 internal/bytealg.IndexString
965	0x7fa901a79e50 indexbody
966	0x7fa901a7a140 internal/bytealg.IndexByte
967	0x7fa901a7a160 internal/bytealg.IndexByteString
968	0x7fa901a7a180 indexbytebody
969	0x7fa901a7a2a0 runtime.memhash0
970	0x7fa901a7a2b0 runtime.memhash8
971	0x7fa901a7a310 runtime.memhash16
972	0x7fa901a7a370 runtime.memhash128
973	0x7fa901a7a3d0 runtime.memhash_varlen
974	0x7fa901a7a420 runtime.strhash
975	0x7fa901a7a480 runtime.f32hash
976	0x7fa901a7a5a0 runtime.f64hash
977	0x7fa901a7a6c0 runtime.c64hash
978	0x7fa901a7a730 runtime.c128hash
979	0x7fa901a7a7a0 runtime.interhash
980	0x7fa901a7a910 runtime.nilinterhash
981	0x7fa901a7aa80 runtime.memequal0
982	0x7fa901a7aa90 runtime.memequal8
983	0x7fa901a7aab0 runtime.memequal16
984	0x7fa901a7aad0 runtime.memequal32
985	0x7fa901a7aaf0 runtime.memequal64
986	0x7fa901a7ab10 runtime.memequal128
987	0x7fa901a7ab40 runtime.f32equal
988	0x7fa901a7ab70 runtime.f64equal
989	0x7fa901a7aba0 runtime.c64equal
990	0x7fa901a7abe0 runtime.c128equal
991	0x7fa901a7ac20 runtime.strequal
992	0x7fa901a7ac90 runtime.interequal
993	0x7fa901a7ad00 runtime.nilinterequal
994	0x7fa901a7ad70 runtime.efaceeq
995	0x7fa901a7aea0 runtime.ifaceeq
996	0x7fa901a7afd0 runtime.alginit
997	0x7fa901a7b080 runtime.initAlgAES
998	0x7fa901a7b150 runtime.atomicwb
999	0x7fa901a7b1d0 runtime.atomicstorep
1000	0x7fa901a7b220 sync/atomic.StorePointer
1001	0x7fa901a7b280 sync/atomic.CompareAndSwapPointer
1002	0x7fa901a7b300 runtime.cgoUse
1489	0x7fa901a7b350 runtime.mmap
1490	0x7fa901a7b4d0 runtime.munmap
1491	0x7fa901a7b580 runtime.sigaction
1492	0x7fa901a7b6e0 runtime.cgocall
1493	0x7fa901a7b7d0 runtime.cgocallbackg
1494	0x7fa901a7b940 runtime.cgocallbackg1
1495	0x7fa901a7bbb0 runtime.unwindm
1496	0x7fa901a7bc70 runtime.cgoCheckPointer
1497	0x7fa901a7be30 runtime.cgoCheckArg
1498	0x7fa901a7c580 runtime.cgoCheckUnknownPointer
1499	0x7fa901a7c8d0 runtime.cgoIsGoPointer
1500	0x7fa901a7c9c0 runtime.cgoCheckResult
1501	0x7fa901a7ca50 runtime._cgo_panic_internal
1502	0x7fa901a7cad0 runtime.cgoCheckWriteBarrier
1503	0x7fa901a7cbd0 runtime.cgoCheckMemmove
1504	0x7fa901a7cc70 runtime.cgoCheckSliceCopy
1505	0x7fa901a7cd50 runtime.cgoCheckTypedBlock
1506	0x7fa901a7d0e0 runtime.cgoCheckBits
1507	0x7fa901a7d1d0 runtime.cgoCheckUsingType
1508	0x7fa901a7d430 runtime.makechan
1509	0x7fa901a7d670 runtime.chansend1
1510	0x7fa901a7d6b0 runtime.chansend
1511	0x7fa901a7dcf0 runtime.send
1512	0x7fa901a7de10 runtime.sendDirect
1513	0x7fa901a7dea0 runtime.recvDirect
1514	0x7fa901a7df30 runtime.closechan
1515	0x7fa901a7e1b0 runtime.chanrecv1
1516	0x7fa901a7e1f0 runtime.chanrecv
1517	0x7fa901a7e8b0 runtime.recv
1518	0x7fa901a7ea70 reflect.chansend
1519	0x7fa901a7eae0 reflect.chanrecv
1520	0x7fa901a7eb50 reflect.chanlen
1521	0x7fa901a7eb70 reflect.chancap
1522	0x7fa901a7eb90 reflect.chanclose
1523	0x7fa901a7ebd0 runtime.(*waitq).dequeue
1524	0x7fa901a7ecc0 runtime.init.0
1525	0x7fa901a7ed20 runtime.(*cpuProfile).add
1526	0x7fa901a7ee40 runtime.(*cpuProfile).addNonGo
1527	0x7fa901a7ef40 runtime.(*cpuProfile).addExtra
1528	0x7fa901a7f150 runtime.GOMAXPROCS
1529	0x7fa901a7f210 runtime.debugCallCheck
1530	0x7fa901a7f2f0 runtime.debugCallWrap
1531	0x7fa901a7f3a0 runtime.gogetenv
1532	0x7fa901a7f4f0 runtime.(*TypeAssertionError).RuntimeError
826	0x7fa901a7f500 runtime.(*TypeAssertionError).Error
827	0x7fa901a7f950 runtime.errorString.RuntimeError
828	0x7fa901a7f960 runtime.errorString.Error
829	0x7fa901a7f9e0 runtime.plainError.RuntimeError
830	0x7fa901a7f9f0 runtime.plainError.Error
831	0x7fa901a7fa10 runtime.printany
832	0x7fa901a80210 runtime.panicwrap
833	0x7fa901a80610 runtime.Caller
834	0x7fa901a80800 runtime.Callers
835	0x7fa901a80880 runtime.GOROOT
836	0x7fa901a80910 runtime.float64frombits
837	0x7fa901a80920 runtime.memhash
838	0x7fa901a80c30 runtime.memhash32
839	0x7fa901a80ca0 runtime.memhash64
840	0x7fa901a80d10 runtime.getitab
841	0x7fa901a810a0 runtime.(*itabTableType).find
842	0x7fa901a81100 runtime.itabAdd
843	0x7fa901a81260 runtime.(*itabTableType).add
844	0x7fa901a812c0 runtime.(*itab).init
845	0x7fa901a81730 runtime.itabsinit
846	0x7fa901a81820 runtime.panicdottypeE
847	0x7fa901a81910 runtime.panicdottypeI
848	0x7fa901a81a20 runtime.convT2E
849	0x7fa901a81ab0 runtime.convT32
850	0x7fa901a81b20 runtime.convT64
851	0x7fa901a81ba0 runtime.convTstring
852	0x7fa901a81c40 runtime.convTslice
853	0x7fa901a81ce0 runtime.convT2Enoptr
854	0x7fa901a81d70 runtime.convT2I
855	0x7fa901a81e10 runtime.convI2I
856	0x7fa901a81ec0 runtime.assertI2I
857	0x7fa901a81fd0 runtime.assertI2I2
858	0x7fa901a82080 runtime.assertE2I
859	0x7fa901a82160 runtime.assertE2I2
860	0x7fa901a82210 reflect.ifaceE2I
861	0x7fa901a822a0 runtime.iterate_itabs
862	0x7fa901a82320 runtime.(*lfstack).push
863	0x7fa901a82480 runtime.(*lfstack).pop
864	0x7fa901a824c0 runtime.lfnodeValidate
865	0x7fa901a82580 runtime.lock
866	0x7fa901a82730 runtime.unlock
867	0x7fa901a82800 runtime.notewakeup
868	0x7fa901a828d0 runtime.notesleep
869	0x7fa901a829d0 runtime.notetsleep_internal
783	0x7fa901a82b90 runtime.notetsleep
784	0x7fa901a82c20 runtime.notetsleepg
785	0x7fa901a82cc0 runtime.mallocinit
786	0x7fa901a82f20 runtime.(*mheap).sysAlloc
787	0x7fa901a83730 runtime.sysReserveAligned
788	0x7fa901a838a0 runtime.(*mcache).nextFree
789	0x7fa901a83ae0 runtime.mallocgc
790	0x7fa901a84490 runtime.largeAlloc
791	0x7fa901a84630 runtime.newobject
792	0x7fa901a84690 reflect.unsafe_New
793	0x7fa901a846f0 runtime.newarray
794	0x7fa901a847c0 reflect.unsafe_NewArray
795	0x7fa901a84820 runtime.profilealloc
796	0x7fa901a84890 runtime.fastexprand
797	0x7fa901a84a10 runtime.persistentalloc
798	0x7fa901a84ac0 runtime.persistentalloc1
799	0x7fa901a84de0 runtime.inPersistentAlloc
800	0x7fa901a84e20 runtime.(*linearAlloc).alloc
801	0x7fa901a84f00 runtime.(*hmap).incrnoverflow
802	0x7fa901a84f90 runtime.(*hmap).newoverflow
803	0x7fa901a85270 runtime.makemap_small
804	0x7fa901a85310 runtime.makemap
805	0x7fa901a854f0 runtime.makeBucketArray
806	0x7fa901a85730 runtime.mapaccess1
807	0x7fa901a859c0 runtime.mapaccess2
808	0x7fa901a85c60 runtime.mapaccessK
809	0x7fa901a85eb0 runtime.mapassign
810	0x7fa901a864c0 runtime.mapdelete
811	0x7fa901a86920 runtime.mapiterinit
812	0x7fa901a86be0 runtime.mapiternext
813	0x7fa901a87190 runtime.hashGrow
814	0x7fa901a873c0 runtime.growWork
815	0x7fa901a87470 runtime.evacuate
816	0x7fa901a87ab0 runtime.advanceEvacuationMark
817	0x7fa901a87ba0 reflect.makemap
818	0x7fa901a87e70 reflect.mapaccess
819	0x7fa901a87ee0 reflect.mapassign
820	0x7fa901a87f60 reflect.mapdelete
821	0x7fa901a87fc0 reflect.mapiterinit
822	0x7fa901a88040 reflect.mapiternext
823	0x7fa901a88080 reflect.mapiterkey
824	0x7fa901a88090 reflect.mapitervalue
825	0x7fa901a880a0 reflect.maplen
537	0x7fa901a880c0 runtime.mapaccess1_fast32
538	0x7fa901a88280 runtime.mapaccess2_fast32
539	0x7fa901a88450 runtime.mapassign_fast32
540	0x7fa901a887e0 runtime.growWork_fast32
541	0x7fa901a88890 runtime.evacuate_fast32
542	0x7fa901a88cc0 runtime.mapaccess1_fast64
543	0x7fa901a88e90 runtime.mapaccess2_fast64
544	0x7fa901a89060 runtime.mapassign_fast64
545	0x7fa901a893f0 runtime.mapassign_fast64ptr
546	0x7fa901a897a0 runtime.mapdelete_fast64
547	0x7fa901a89ae0 runtime.growWork_fast64
548	0x7fa901a89b90 runtime.evacuate_fast64
549	0x7fa901a8a040 runtime.mapaccess1_faststr
550	0x7fa901a8a4c0 runtime.mapaccess2_faststr
551	0x7fa901a8a980 runtime.mapassign_faststr
552	0x7fa901a8ade0 runtime.growWork_faststr
553	0x7fa901a8ae90 runtime.evacuate_faststr
554	0x7fa901a8b310 runtime.typedmemmove
555	0x7fa901a8b3e0 reflect.typedmemmove
556	0x7fa901a8b440 reflect.typedmemmovepartial
557	0x7fa901a8b580 runtime.reflectcallmove
558	0x7fa901a8b620 runtime.typedslicecopy
559	0x7fa901a8b760 reflect.typedslicecopy
560	0x7fa901a8b870 runtime.typedmemclr
561	0x7fa901a8b8e0 reflect.typedmemclr
562	0x7fa901a8b930 reflect.typedmemclrpartial
563	0x7fa901a8b9b0 runtime.memclrHasPointers
564	0x7fa901a8ba10 runtime.(*mspan).refillAllocCache
565	0x7fa901a8ba30 runtime.(*mspan).nextFreeIndex
566	0x7fa901a8bbc0 runtime.markBitsForAddr
567	0x7fa901a8bcb0 runtime.findObject
568	0x7fa901a8c080 runtime.heapBits.nextArena
569	0x7fa901a8c0f0 runtime.heapBits.forward
570	0x7fa901a8c1b0 runtime.heapBits.forwardOrBoundary
571	0x7fa901a8c270 runtime.bulkBarrierPreWrite
572	0x7fa901a8c730 runtime.bulkBarrierPreWriteSrcOnly
573	0x7fa901a8c930 runtime.bulkBarrierBitmap
574	0x7fa901a8cad0 runtime.typeBitsBulkBarrier
575	0x7fa901a8cd90 runtime.heapBits.initSpan
1047	0x7fa901a8cfa0 runtime.heapBits.initCheckmarkSpan
1048	0x7fa901a8d0e0 runtime.heapBits.clearCheckmarkSpan
1049	0x7fa901a8d190 runtime.(*mspan).countAlloc
1050	0x7fa901a8d210 runtime.heapBitsSetType
1051	0x7fa901a8dc70 runtime.heapBitsSetTypeGCProg
1052	0x7fa901a8dfe0 runtime.progToPointerMask
1053	0x7fa901a8e110 runtime.runGCProg
1054	0x7fa901a8e770 runtime.materializeGCProg
1055	0x7fa901a8e820 runtime.allocmcache
1056	0x7fa901a8e8e0 runtime.freemcache
1057	0x7fa901a8e940 runtime.(*mcache).refill
1058	0x7fa901a8ea80 runtime.(*mcache).releaseAll
1059	0x7fa901a8eb20 runtime.(*mcache).prepareForSweep
1060	0x7fa901a8ec30 runtime.(*mcentral).cacheSpan
1061	0x7fa901a8f090 runtime.(*mcentral).uncacheSpan
1062	0x7fa901a8f1f0 runtime.(*mcentral).freeSpan
1063	0x7fa901a8f390 runtime.(*mcentral).grow
1064	0x7fa901a8f510 runtime.sysAlloc
1065	0x7fa901a8f620 runtime.sysUnused
1066	0x7fa901a8f810 runtime.sysUsed
1067	0x7fa901a8f890 runtime.sysFree
1068	0x7fa901a8f8e0 runtime.sysMap
1069	0x7fa901a8f9c0 runtime.queuefinalizer
1070	0x7fa901a8fc10 runtime.wakefing
1071	0x7fa901a8fca0 runtime.createfing
1072	0x7fa901a8fd10 runtime.runfinq
1073	0x7fa901a90140 runtime.SetFinalizer
1074	0x7fa901a90960 runtime.(*fixalloc).alloc
1075	0x7fa901a90ac0 runtime.gcinit
1076	0x7fa901a90ba0 runtime.readgogc
1077	0x7fa901a90c70 runtime.gcenable
1078	0x7fa901a90d00 runtime/debug.setGCPercent
1079	0x7fa901a90dd0 runtime.(*gcControllerState).startCycle
1080	0x7fa901a91170 runtime.(*gcControllerState).revise
1081	0x7fa901a91280 runtime.(*gcControllerState).endCycle
1082	0x7fa901a91740 runtime.(*gcControllerState).enlistWorker
1083	0x7fa901a918d0 runtime.(*gcControllerState).findRunnableGCWorker
1084	0x7fa901a91b30 runtime.pollFractionalWorkerExit
1085	0x7fa901a91bf0 runtime.gcSetTriggerRatio
1198	0x7fa901a91ff0 runtime.GC
1199	0x7fa901a92180 runtime.gcWaitOnMark
1200	0x7fa901a92250 runtime.gcStart
1201	0x7fa901a92910 runtime.gcMarkDone
1202	0x7fa901a92bd0 runtime.gcMarkTermination
1203	0x7fa901a93700 runtime.gcBgMarkStartWorkers
1204	0x7fa901a937d0 runtime.gcBgMarkWorker
1205	0x7fa901a93ca0 runtime.gcMark
1206	0x7fa901a94220 runtime.gcSweep
1207	0x7fa901a94410 runtime.gcResetMarkState
1208	0x7fa901a94560 sync.runtime_registerPoolCleanup
1209	0x7fa901a945c0 runtime.clearpools
1210	0x7fa901a94730 runtime.itoaDiv
1211	0x7fa901a94810 runtime.fmtNSAsMS
1212	0x7fa901a949a0 runtime.(*treapNode).pred
1213	0x7fa901a94ab0 runtime.(*treapNode).succ
1214	0x7fa901a94bc0 runtime.treapIter.prev
1215	0x7fa901a94c10 runtime.(*mTreap).start
1216	0x7fa901a94c40 runtime.(*mTreap).end
1217	0x7fa901a94c70 runtime.(*mTreap).insert
1218	0x7fa901a94f40 runtime.(*mTreap).removeNode
1219	0x7fa901a95060 runtime.(*mTreap).find
1220	0x7fa901a95110 runtime.(*mTreap).removeSpan
1221	0x7fa901a951a0 runtime.(*mTreap).erase
1222	0x7fa901a951f0 runtime.(*mTreap).rotateLeft
1223	0x7fa901a952b0 runtime.(*mTreap).rotateRight
1224	0x7fa901a95370 runtime.gcMarkRootPrepare
1225	0x7fa901a954c0 runtime.gcMarkRootCheck
1226	0x7fa901a95710 runtime.markroot
1227	0x7fa901a95a80 runtime.markrootBlock
1228	0x7fa901a95b40 runtime.markrootFreeGStacks
1229	0x7fa901a95c70 runtime.markrootSpans
1230	0x7fa901a95ea0 runtime.gcAssistAlloc
1231	0x7fa901a96170 runtime.gcAssistAlloc1
1232	0x7fa901a964c0 runtime.gcWakeAllAssists
1233	0x7fa901a96540 runtime.gcParkAssist
1234	0x7fa901a96690 runtime.gcFlushBgCredit
1235	0x7fa901a96820 runtime.scanstack
1236	0x7fa901a96ef0 runtime.scanframeworker
1237	0x7fa901a970d0 runtime.gcDrain
1238	0x7fa901a97510 runtime.gcDrainN
1327	0x7fa901a97750 runtime.scanblock
1328	0x7fa901a978d0 runtime.scanobject
1329	0x7fa901a97c80 runtime.shade
1330	0x7fa901a97d30 runtime.greyobject
1331	0x7fa901a982f0 runtime.gcDumpObject
1332	0x7fa901a98790 runtime.gcmarknewobject
1333	0x7fa901a98820 runtime.gcMarkTinyAllocs
1334	0x7fa901a98910 runtime.initCheckmarks
1335	0x7fa901a98a50 runtime.clearCheckmarks
1336	0x7fa901a98b90 runtime.init.1
1337	0x7fa901a98ba0 runtime.(*stackScanState).putPtr
1338	0x7fa901a98cc0 runtime.(*stackScanState).getPtr
1339	0x7fa901a98dd0 runtime.(*stackScanState).addObject
1340	0x7fa901a98f40 runtime.binarySearchTree
1341	0x7fa901a990a0 runtime.(*stackScanState).findObject
1342	0x7fa901a990f0 runtime.finishsweep_m
1343	0x7fa901a99140 runtime.bgsweep
1344	0x7fa901a992b0 runtime.sweepone
1345	0x7fa901a996a0 runtime.(*mspan).ensureSwept
1346	0x7fa901a997a0 runtime.(*mspan).sweep
1347	0x7fa901a9a150 runtime.deductSweepCredit
1348	0x7fa901a9a2b0 runtime.clobberfree
1349	0x7fa901a9a2d0 runtime.(*gcSweepBuf).push
1350	0x7fa901a9a490 runtime.(*gcSweepBuf).block
1351	0x7fa901a9a580 runtime.init.2
1352	0x7fa901a9a590 runtime.(*gcWork).init
1353	0x7fa901a9a600 runtime.(*gcWork).put
1354	0x7fa901a9a710 runtime.(*gcWork).putBatch
1355	0x7fa901a9a8e0 runtime.(*gcWork).tryGet
1356	0x7fa901a9a9d0 runtime.(*gcWork).dispose
1357	0x7fa901a9aac0 runtime.(*gcWork).balance
1358	0x7fa901a9abc0 runtime.(*workbuf).checknonempty
1359	0x7fa901a9ac20 runtime.(*workbuf).checkempty
1360	0x7fa901a9ac80 runtime.getempty
1361	0x7fa901a9aeb0 runtime.putempty
1362	0x7fa901a9af10 runtime.putfull
1363	0x7fa901a9af70 runtime.trygetfull
1364	0x7fa901a9aff0 runtime.handoff
1365	0x7fa901a9b0b0 runtime.prepareFreeWorkbufs
1366	0x7fa901a9b160 runtime.freeSomeWbufs
1367	0x7fa901a9b240 runtime.(*mheap).coalesce
1086	0x7fa901a9b4b0 runtime.(*mspan).scavenge
1087	0x7fa901a9b570 runtime.recordspan
1088	0x7fa901a9b730 runtime.inHeapOrStack
1089	0x7fa901a9b7c0 runtime.spanOfHeap
1090	0x7fa901a9b840 runtime.(*mheap).init
1091	0x7fa901a9bca0 runtime.(*mheap).reclaim
1092	0x7fa901a9bf00 runtime.(*mheap).reclaimChunk
1093	0x7fa901a9c250 runtime.(*mheap).alloc_m
1094	0x7fa901a9c560 runtime.(*mheap).alloc
1095	0x7fa901a9c650 runtime.(*mheap).allocManual
1096	0x7fa901a9c730 runtime.(*mheap).setSpans
1097	0x7fa901a9c800 runtime.(*mheap).pickFreeSpan
1098	0x7fa901a9c940 runtime.(*mheap).allocSpanLocked
1099	0x7fa901a9cdc0 runtime.(*mheap).grow
1100	0x7fa901a9cfd0 runtime.(*mheap).freeSpan
1101	0x7fa901a9d050 runtime.(*mheap).freeManual
1102	0x7fa901a9d100 runtime.(*mheap).freeSpanLocked
1103	0x7fa901a9d3f0 runtime.(*mheap).scavengeLargest
1104	0x7fa901a9d580 runtime.(*mheap).scavengeAll
1105	0x7fa901a9d6d0 runtime.(*mheap).scavenge
1106	0x7fa901a9d970 runtime.(*mSpanList).remove
1107	0x7fa901a9db00 runtime.(*mSpanList).insert
1108	0x7fa901a9dc20 runtime.(*mSpanList).insertBack
1109	0x7fa901a9dd40 runtime.(*mSpanList).takeAll
1110	0x7fa901a9dda0 runtime.addspecial
1111	0x7fa901a9df70 runtime.removespecial
1112	0x7fa901a9e120 runtime.addfinalizer
1113	0x7fa901a9e390 runtime.removefinalizer
1114	0x7fa901a9e440 runtime.setprofilebucket
1115	0x7fa901a9e500 runtime.freespecial
1116	0x7fa901a9e650 runtime.newMarkBits
1117	0x7fa901a9e990 runtime.nextMarkBitArenaEpoch
1118	0x7fa901a9ea50 runtime.newArenaMayUnlock
1119	0x7fa901a9eb40 runtime.newBucket
1120	0x7fa901a9ec10 runtime.(*bucket).mp
1121	0x7fa901a9ec80 runtime.(*bucket).bp
1122	0x7fa901a9ed00 runtime.stkbucket
1123	0x7fa901a9f050 runtime.eqslice
1124	0x7fa901a9f0a0 runtime.mProf_NextCycle
1125	0x7fa901a9f120 runtime.mProf_Flush
611	0x7fa901a9f190 runtime.mProf_FlushLocked
612	0x7fa901a9f270 runtime.mProf_PostSweep
613	0x7fa901a9f370 runtime.mProf_Malloc
614	0x7fa901a9f540 runtime.mProf_Free
615	0x7fa901a9f600 runtime.blockevent
616	0x7fa901a9f680 runtime.blocksampled
617	0x7fa901a9f710 runtime.saveblockevent
618	0x7fa901a9f8d0 sync.event
619	0x7fa901a9f9b0 runtime.tracealloc
620	0x7fa901a9fc30 runtime.tracefree
621	0x7fa901a9fd90 runtime.tracegc
622	0x7fa901a9fe80 runtime.init.3
623	0x7fa901a9ff10 runtime.ReadMemStats
624	0x7fa901a9ff90 runtime.readmemstats_m
625	0x7fa901aa0000 runtime.updatememstats
626	0x7fa901aa0250 runtime.cachestats
627	0x7fa901aa02d0 runtime.flushmcache
628	0x7fa901aa0350 runtime.flushallmcaches
629	0x7fa901aa03b0 runtime.purgecachedstats
630	0x7fa901aa0450 runtime.mSysStatInc
631	0x7fa901aa0510 runtime.mSysStatDec
632	0x7fa901aa05d0 runtime.(*wbBuf).reset
633	0x7fa901aa0660 runtime.wbBufFlush
634	0x7fa901aa0760 runtime.wbBufFlush1
635	0x7fa901aa09b0 internal/poll.runtime_pollServerInit
636	0x7fa901aa09f0 internal/poll.runtime_pollOpen
637	0x7fa901aa0b30 internal/poll.runtime_pollClose
638	0x7fa901aa0c50 runtime.(*pollCache).free
639	0x7fa901aa0cb0 internal/poll.runtime_pollReset
640	0x7fa901aa0d30 internal/poll.runtime_pollWait
641	0x7fa901aa0e50 internal/poll.runtime_pollSetDeadline
642	0x7fa901aa13d0 internal/poll.runtime_pollUnblock
643	0x7fa901aa15c0 runtime.netpollready
644	0x7fa901aa16b0 runtime.netpollblockcommit
645	0x7fa901aa1700 runtime.netpollblock
646	0x7fa901aa1860 runtime.netpollunblock
647	0x7fa901aa1900 runtime.netpolldeadlineimpl
648	0x7fa901aa1b10 runtime.netpollDeadline
649	0x7fa901aa1b90 runtime.netpollReadDeadline
650	0x7fa901aa1c10 runtime.netpollWriteDeadline
911	0x7fa901aa1c90 runtime.(*pollCache).alloc
912	0x7fa901aa1d60 runtime.netpollinit
913	0x7fa901aa1e40 runtime.netpollopen
914	0x7fa901aa1ed0 runtime.netpoll
915	0x7fa901aa20f0 runtime.futexsleep
916	0x7fa901aa21c0 runtime.futexwakeup
917	0x7fa901aa2260 runtime.getproccount
918	0x7fa901aa2370 runtime.newosproc
919	0x7fa901aa2540 runtime.newosproc0
920	0x7fa901aa2640 runtime.sysargs
921	0x7fa901aa28c0 runtime.sysauxv
922	0x7fa901aa29d0 runtime.osinit
923	0x7fa901aa2a10 runtime.getRandomData
924	0x7fa901aa2b70 runtime.libpreinit
925	0x7fa901aa2ba0 runtime.minit
926	0x7fa901aa2bf0 runtime.setsig
927	0x7fa901aa2cb0 runtime.setsigstack
928	0x7fa901aa2d40 runtime.sysSigaction
929	0x7fa901aa2db0 runtime.panicCheckMalloc
930	0x7fa901aa2e60 runtime.panicindex
931	0x7fa901aa2f90 runtime.panicslice
932	0x7fa901aa30c0 runtime.panicdivide
933	0x7fa901aa3130 runtime.throwinit
934	0x7fa901aa3180 runtime.deferproc
935	0x7fa901aa32b0 runtime.testdefersizes
936	0x7fa901aa3500 runtime.init.4
937	0x7fa901aa3580 runtime.newdefer
938	0x7fa901aa37a0 runtime.freedefer
939	0x7fa901aa39b0 runtime.freedeferpanic
940	0x7fa901aa3a00 runtime.freedeferfn
941	0x7fa901aa3a50 runtime.deferreturn
942	0x7fa901aa3b70 runtime.preprintpanics
943	0x7fa901aa3d40 runtime.printpanics
944	0x7fa901aa3e40 runtime.gopanic
945	0x7fa901aa43b0 runtime.getargp
946	0x7fa901aa43c0 runtime.gorecover
947	0x7fa901aa4410 sync.throw
948	0x7fa901aa4450 runtime.throw
949	0x7fa901aa44e0 runtime.recovery
950	0x7fa901aa4640 runtime.fatalthrow
951	0x7fa901aa46b0 runtime.fatalpanic
952	0x7fa901aa4760 runtime.startpanic_m
953	0x7fa901aa4910 runtime.dopanic_m
954	0x7fa901aa4ca0 runtime.shouldPushSigpanic
955	0x7fa901aa4d60 runtime.recordForPanic
956	0x7fa901aa4eb0 runtime.printlock
651	0x7fa901aa4f30 runtime.printunlock
652	0x7fa901aa4fa0 runtime.gwrite
653	0x7fa901aa50f0 runtime.printsp
654	0x7fa901aa5140 runtime.printnl
655	0x7fa901aa5190 runtime.printbool
656	0x7fa901aa5200 runtime.printfloat
657	0x7fa901aa54a0 runtime.printcomplex
658	0x7fa901aa5530 runtime.printuint
659	0x7fa901aa5630 runtime.printint
660	0x7fa901aa56a0 runtime.printhex
661	0x7fa901aa57b0 runtime.printpointer
662	0x7fa901aa57f0 runtime.printstring
663	0x7fa901aa5880 runtime.printslice
664	0x7fa901aa5950 runtime.printeface
665	0x7fa901aa5a00 runtime.hexdumpWords
666	0x7fa901aa5c50 runtime.main
667	0x7fa901aa5fe0 os.runtime_beforeExit
668	0x7fa901aa5ff0 runtime.init.5
669	0x7fa901aa6040 runtime.forcegchelper
670	0x7fa901aa6170 runtime.gopark
671	0x7fa901aa62c0 runtime.goready
672	0x7fa901aa6330 runtime.acquireSudog
673	0x7fa901aa66d0 runtime.releaseSudog
674	0x7fa901aa6a70 runtime.funcPC
675	0x7fa901aa6a80 runtime.badmcall
676	0x7fa901aa6ad0 runtime.badmcall2
677	0x7fa901aa6b20 runtime.badreflectcall
678	0x7fa901aa6b70 runtime.badmorestackg0
679	0x7fa901aa6bb0 runtime.badmorestackgsignal
680	0x7fa901aa6bf0 runtime.badctxt
681	0x7fa901aa6c20 runtime.allgadd
682	0x7fa901aa6d70 runtime.cpuinit
683	0x7fa901aa6f00 runtime.schedinit
684	0x7fa901aa7160 runtime.checkmcount
685	0x7fa901aa7220 runtime.mcommoninit
686	0x7fa901aa7450 runtime.ready
687	0x7fa901aa7710 runtime.freezetheworld
688	0x7fa901aa77c0 runtime.casfrom_Gscanstatus
689	0x7fa901aa7c20 runtime.castogscanstatus
690	0x7fa901aa7d00 runtime.casgstatus
691	0x7fa901aa7f00 runtime.scang
692	0x7fa901aa8290 runtime.restartg
693	0x7fa901aa8470 runtime.stopTheWorld
694	0x7fa901aa8520 runtime.startTheWorld
695	0x7fa901aa85c0 runtime.stopTheWorldWithSema
696	0x7fa901aa8880 runtime.startTheWorldWithSema
1126	0x7fa901aa8b00 runtime.mstart
1127	0x7fa901aa8b90 runtime.mstart1
1128	0x7fa901aa8ca0 runtime.mstartm0
1129	0x7fa901aa8d00 runtime.mexit
1130	0x7fa901aa8f70 runtime.forEachP
1131	0x7fa901aa9300 runtime.runSafePointFn
1132	0x7fa901aa93d0 runtime.allocm
1133	0x7fa901aa9680 runtime.needm
1134	0x7fa901aa9810 runtime.newextram
1135	0x7fa901aa98a0 runtime.oneNewExtraM
1136	0x7fa901aa9a50 runtime.dropm
1137	0x7fa901aa9b90 runtime.lockextra
1138	0x7fa901aa9c50 runtime.newm
1139	0x7fa901aa9da0 runtime.newm1
1140	0x7fa901aa9ec0 runtime.templateThread
1141	0x7fa901aa9fe0 runtime.stopm
1142	0x7fa901aaa150 runtime.mspinning
1143	0x7fa901aaa170 runtime.startm
1144	0x7fa901aaa390 runtime.handoffp
1145	0x7fa901aaa6a0 runtime.stoplockedm
1146	0x7fa901aaa960 runtime.startlockedm
1147	0x7fa901aaaa40 runtime.gcstopm
1148	0x7fa901aaab60 runtime.execute
1149	0x7fa901aaacd0 runtime.findrunnable
1150	0x7fa901aab8d0 runtime.pollWork
1151	0x7fa901aab9c0 runtime.resetspinning
1152	0x7fa901aabaa0 runtime.injectglist
1153	0x7fa901aabc30 runtime.schedule
1154	0x7fa901aac080 runtime.parkunlock_c
1155	0x7fa901aac0d0 runtime.park_m
1156	0x7fa901aac2a0 runtime.goschedImpl
1157	0x7fa901aac510 runtime.gosched_m
1158	0x7fa901aac5a0 runtime.gopreempt_m
1159	0x7fa901aac630 runtime.goexit1
1160	0x7fa901aac6a0 runtime.goexit0
1161	0x7fa901aac970 runtime.save
1162	0x7fa901aac9c0 runtime.reentersyscall
1163	0x7fa901aacc00 runtime.entersyscall
1164	0x7fa901aacc30 runtime.entersyscall_sysmon
1165	0x7fa901aaccb0 runtime.entersyscall_gcwait
1166	0x7fa901aacdb0 runtime.entersyscallblock
1167	0x7fa901aacf60 runtime.entersyscallblock_handoff
1168	0x7fa901aacff0 runtime.exitsyscall
1169	0x7fa901aad280 runtime.exitsyscallfast
1170	0x7fa901aad380 runtime.exitsyscallfast_reacquired
697	0x7fa901aad410 runtime.exitsyscallfast_pidle
698	0x7fa901aad500 runtime.exitsyscall0
699	0x7fa901aad6f0 runtime.malg
700	0x7fa901aad7c0 runtime.newproc
701	0x7fa901aad840 runtime.newproc1
702	0x7fa901aadd50 runtime.saveAncestors
703	0x7fa901aae0a0 runtime.gfput
704	0x7fa901aae210 runtime.gfget
705	0x7fa901aae3a0 runtime.gfpurge
706	0x7fa901aae470 runtime.unlockOSThread
707	0x7fa901aae500 runtime.badunlockosthread
708	0x7fa901aae550 runtime._System
709	0x7fa901aae590 runtime._ExternalCode
710	0x7fa901aae5d0 runtime._LostExternalCode
711	0x7fa901aae610 runtime._GC
712	0x7fa901aae650 runtime._VDSO
713	0x7fa901aae690 runtime.sigprof
714	0x7fa901aaeb90 runtime.sigprofNonGo
715	0x7fa901aaec20 runtime.sigprofNonGoPC
716	0x7fa901aaeca0 runtime.setsSP
717	0x7fa901aaed30 runtime.procresize
718	0x7fa901aaf7b0 runtime.acquirep
719	0x7fa901aaf810 runtime.wirep
720	0x7fa901aaf980 runtime.releasep
721	0x7fa901aafbb0 runtime.incidlelocked
722	0x7fa901aafc20 runtime.checkdead
723	0x7fa901ab00a0 runtime.sysmon
724	0x7fa901ab05c0 runtime.retake
725	0x7fa901ab0800 runtime.preemptall
726	0x7fa901ab0880 runtime.schedtrace
727	0x7fa901ab1360 runtime.schedEnableUser
728	0x7fa901ab14b0 runtime.schedEnabled
729	0x7fa901ab1520 runtime.globrunqget
730	0x7fa901ab1670 runtime.pidleput
731	0x7fa901ab1700 runtime.runqempty
732	0x7fa901ab1740 runtime.runqput
733	0x7fa901ab1840 runtime.runqputslow
734	0x7fa901ab1a70 runtime.runqget
735	0x7fa901ab1b20 runtime.runqgrab
736	0x7fa901ab1c80 runtime.runqsteal
737	0x7fa901ab1d80 sync.runtime_procPin
738	0x7fa901ab1db0 sync.runtime_procUnpin
739	0x7fa901ab1dd0 sync/atomic.runtime_procPin
740	0x7fa901ab1e00 sync/atomic.runtime_procUnpin
741	0x7fa901ab1e20 sync.runtime_canSpin
742	0x7fa901ab1eb0 sync.runtime_doSpin
743	0x7fa901ab1ee0 runtime.(*randomOrder).reset
744	0x7fa901ab2000 runtime.gcd
745	0x7fa901ab2020 runtime.(*profBuf).takeOverflow
746	0x7fa901ab2090 runtime.(*profBuf).incrementOverflow
747	0x7fa901ab20e0 runtime.(*profBuf).canWriteRecord
748	0x7fa901ab2190 runtime.(*profBuf).canWriteTwoRecords
749	0x7fa901ab2290 runtime.(*profBuf).write
750	0x7fa901ab2720 runtime.(*profBuf).wakeupExtra
751	0x7fa901ab27a0 syscall.runtime_envs
752	0x7fa901ab28a0 os.runtime_args
753	0x7fa901ab29a0 syscall.Exit
754	0x7fa901ab29d0 runtime.args
755	0x7fa901ab2a40 runtime.goargs
756	0x7fa901ab2b90 runtime.goenvs_unix
757	0x7fa901ab2ce0 runtime.testAtomic64
758	0x7fa901ab2f40 runtime.check
759	0x7fa901ab32e0 runtime.parsedebugvars
760	0x7fa901ab3650 runtime/debug.SetTraceback
761	0x7fa901ab3800 runtime.timediv
762	0x7fa901ab3860 reflect.typelinks
763	0x7fa901ab3c30 reflect.resolveNameOff
764	0x7fa901ab3c80 reflect.resolveTypeOff
765	0x7fa901ab3ce0 reflect.resolveTextOff
766	0x7fa901ab3d30 reflect.addReflectOff
767	0x7fa901ab3ec0 runtime.extendRandom
768	0x7fa901ab3fb0 runtime.waitReason.String
769	0x7fa901ab4020 runtime.(*rwmutex).rlock
770	0x7fa901ab40b0 runtime.(*rwmutex).runlock
771	0x7fa901ab41b0 sync.runtime_Semacquire
772	0x7fa901ab4200 internal/poll.runtime_Semacquire
773	0x7fa901ab4250 sync.runtime_Semrelease
774	0x7fa901ab42a0 sync.runtime_SemacquireMutex
775	0x7fa901ab42f0 internal/poll.runtime_Semrelease
776	0x7fa901ab4340 runtime.readyWithTime
777	0x7fa901ab43b0 runtime.semacquire1
778	0x7fa901ab4680 runtime.semrelease1
779	0x7fa901ab4850 runtime.cansemacquire
780	0x7fa901ab4880 runtime.(*semaRoot).queue
781	0x7fa901ab4cd0 runtime.(*semaRoot).dequeue
782	0x7fa901ab5060 runtime.(*semaRoot).rotateLeft
1445	0x7fa901ab5270 runtime.(*semaRoot).rotateRight
1446	0x7fa901ab5460 sync.runtime_notifyListCheck
1447	0x7fa901ab5520 sync.runtime_nanotime
1448	0x7fa901ab5560 runtime.dumpregs
1449	0x7fa901ab5c50 runtime.(*sigctxt).preparePanic
1450	0x7fa901ab5d20 runtime.sighandler
1451	0x7fa901ab6480 os.sigpipe
1452	0x7fa901ab64c0 runtime.init.6
1453	0x7fa901ab64d0 runtime.initsig
1454	0x7fa901ab66d0 runtime.setThreadCPUProfiler
1455	0x7fa901ab67b0 runtime.sigpipe
1456	0x7fa901ab6810 runtime.sigtrampgo
1457	0x7fa901ab6cb0 runtime.sigpanic
1458	0x7fa901ab7170 runtime.dieFromSignal
1459	0x7fa901ab7210 runtime.raisebadsignal
1460	0x7fa901ab7330 runtime.crash
1461	0x7fa901ab7360 runtime.noSignalStack
1462	0x7fa901ab73f0 runtime.sigNotOnStack
1463	0x7fa901ab7480 runtime.signalDuringFork
1464	0x7fa901ab7510 runtime.badsignal
1465	0x7fa901ab7570 runtime.sigfwdgo
1466	0x7fa901ab7730 runtime.unblocksig
1467	0x7fa901ab77c0 runtime.minitSignals
1468	0x7fa901ab7800 runtime.minitSignalStack
1469	0x7fa901ab7960 runtime.minitSignalMask
1470	0x7fa901ab7a70 runtime.unminitSignals
1471	0x7fa901ab7b10 runtime.signalstack
1472	0x7fa901ab7b70 runtime.sigsend
1473	0x7fa901ab7d00 runtime.makeslice
1474	0x7fa901ab7e00 runtime.growslice
1475	0x7fa901ab8440 runtime.stackinit
1476	0x7fa901ab84a0 runtime.stacklog2
1477	0x7fa901ab84c0 runtime.stackpoolalloc
1478	0x7fa901ab86a0 runtime.stackpoolfree
1479	0x7fa901ab8830 runtime.stackcacherefill
1480	0x7fa901ab8920 runtime.stackcacherelease
1481	0x7fa901ab8a40 runtime.stackcache_clear
1482	0x7fa901ab8b20 runtime.stackalloc
1483	0x7fa901ab8e40 runtime.stackfree
1484	0x7fa901ab9190 runtime.adjustpointers
1485	0x7fa901ab93d0 runtime.adjustframe
1486	0x7fa901ab9830 runtime.adjustctxt
1487	0x7fa901ab9880 runtime.adjustdefers
1488	0x7fa901ab9950 runtime.adjustsudogs
1003	0x7fa901ab9990 runtime.findsghi
1004	0x7fa901ab99e0 runtime.syncadjustsudogs
1005	0x7fa901ab9b30 runtime.copystack
1006	0x7fa901ab9e60 runtime.round2
1007	0x7fa901ab9e90 runtime.newstack
1008	0x7fa901ababc0 runtime.shrinkstack
1009	0x7fa901abad40 runtime.freeStackSpans
1010	0x7fa901abaee0 runtime.getStackMap
1011	0x7fa901abb790 runtime.morestackc
1012	0x7fa901abb7c0 runtime.concatstrings
1013	0x7fa901abbaa0 runtime.concatstring2
1014	0x7fa901abbb10 runtime.concatstring3
1015	0x7fa901abbb90 runtime.concatstring4
1016	0x7fa901abbc10 runtime.concatstring5
1017	0x7fa901abbc90 runtime.slicebytetostring
1018	0x7fa901abbd80 runtime.rawstringtmp
1019	0x7fa901abbe30 runtime.stringtoslicebyte
1020	0x7fa901abbf10 runtime.stringtoslicerune
1021	0x7fa901abc0b0 runtime.slicerunetostring
1022	0x7fa901abc2b0 runtime.intstring
1023	0x7fa901abc3e0 runtime.rawstring
1024	0x7fa901abc470 runtime.rawbyteslice
1025	0x7fa901abc5f0 runtime.rawruneslice
1026	0x7fa901abc7b0 runtime.gostring
1027	0x7fa901abc880 runtime.index
1028	0x7fa901abc980 runtime.atoi
1029	0x7fa901abcab0 runtime.findnull
1030	0x7fa901abcb70 runtime.badsystemstack
1031	0x7fa901abcbb0 reflect.memmove
1032	0x7fa901abcc10 runtime.fastrand
1033	0x7fa901abcc60 runtime.(*Frames).Next
1034	0x7fa901abd420 runtime.expandCgoFrames
1035	0x7fa901abd740 runtime.modulesinit
1036	0x7fa901abd9c0 runtime.moduledataverify
1037	0x7fa901abda20 runtime.moduledataverify1
1038	0x7fa901abe0b0 runtime.FuncForPC
1039	0x7fa901abe2e0 runtime.(*Func).Name
1040	0x7fa901abe390 runtime.(*Func).Entry
1041	0x7fa901abe3b0 runtime.(*Func).FileLine
1042	0x7fa901abe460 runtime.findmoduledatap
1043	0x7fa901abe4a0 runtime.findfunc
1044	0x7fa901abe620 runtime.pcvalue
1045	0x7fa901abebe0 runtime.funcname
1046	0x7fa901abec90 runtime.funcnameFromNameoff
1239	0x7fa901abed40 runtime.funcline1
1240	0x7fa901abef10 runtime.funcline
1241	0x7fa901abef90 runtime.funcspdelta
1242	0x7fa901abf100 runtime.pcdatavalue
1243	0x7fa901abf1b0 runtime.pcdatavalue1
1244	0x7fa901abf260 runtime.funcdata
1245	0x7fa901abf340 runtime.step
1246	0x7fa901abf4f0 runtime.readvarint
1247	0x7fa901abf550 runtime.addtimer
1248	0x7fa901abf630 runtime.(*timersBucket).addtimerLocked
1249	0x7fa901abf810 runtime.deltimer
1250	0x7fa901abf8e0 runtime.(*timersBucket).deltimerLocked
1251	0x7fa901abfa80 runtime.modtimer
1252	0x7fa901abfbc0 runtime.timerproc
1253	0x7fa901abff10 runtime.timejump
1254	0x7fa901abffe0 runtime.timejumpLocked
1255	0x7fa901ac0080 runtime.siftupTimer
1256	0x7fa901ac01a0 runtime.siftdownTimer
1257	0x7fa901ac0370 time.now
1258	0x7fa901ac03e0 runtime.traceReader
1259	0x7fa901ac04e0 runtime.traceProcFree
1260	0x7fa901ac05a0 runtime.traceEvent
1261	0x7fa901ac06a0 runtime.traceEventLocked
1262	0x7fa901ac0950 runtime.traceStackID
1263	0x7fa901ac0aa0 runtime.traceAcquireBuffer
1264	0x7fa901ac0b60 runtime.traceReleaseBuffer
1265	0x7fa901ac0be0 runtime.traceFlush
1266	0x7fa901ac0dd0 runtime.(*traceBuf).varint
1267	0x7fa901ac0e50 runtime.(*traceStackTable).put
1268	0x7fa901ac1050 runtime.(*traceStackTable).find
1269	0x7fa901ac1100 runtime.(*traceStackTable).newStack
1270	0x7fa901ac1160 runtime.(*traceAlloc).alloc
1271	0x7fa901ac1280 runtime.traceProcStart
1272	0x7fa901ac1310 runtime.traceProcStop
1273	0x7fa901ac13f0 runtime.traceGCSweepStart
1274	0x7fa901ac1470 runtime.traceGCSweepSpan
1275	0x7fa901ac1510 runtime.traceGCSweepDone
1276	0x7fa901ac15f0 runtime.traceGoCreate
1277	0x7fa901ac16f0 runtime.traceGoStart
1278	0x7fa901ac1890 runtime.traceGoPark
1279	0x7fa901ac1920 runtime.traceGoUnpark
1280	0x7fa901ac1a20 runtime.traceGoSysCall
870	0x7fa901ac1a80 runtime.traceGoSysExit
871	0x7fa901ac1b60 runtime.traceGoSysBlock
872	0x7fa901ac1c40 runtime.traceNextGC
873	0x7fa901ac1d00 runtime.tracebackdefers
874	0x7fa901ac1f30 runtime.gentraceback
875	0x7fa901ac3c70 runtime.getArgInfo
876	0x7fa901ac3e40 runtime.tracebackCgoContext
877	0x7fa901ac40b0 runtime.printcreatedby
878	0x7fa901ac4180 runtime.printcreatedby1
879	0x7fa901ac4340 runtime.traceback
880	0x7fa901ac43b0 runtime.tracebacktrap
881	0x7fa901ac4470 runtime.traceback1
882	0x7fa901ac4790 runtime.printAncestorTraceback
883	0x7fa901ac4990 runtime.printAncestorTracebackFuncInfo
884	0x7fa901ac4cd0 runtime.callers
885	0x7fa901ac4db0 runtime.gcallers
886	0x7fa901ac4e60 runtime.showframe
887	0x7fa901ac4f30 runtime.showfuncinfo
888	0x7fa901ac5140 runtime.goroutineheader
889	0x7fa901ac5420 runtime.tracebackothers
890	0x7fa901ac56d0 runtime.tracebackHexdump
891	0x7fa901ac58c0 runtime.isSystemGoroutine
892	0x7fa901ac59d0 runtime.printCgoTraceback
893	0x7fa901ac5b10 runtime.printOneCgoTraceback
894	0x7fa901ac5d50 runtime.callCgoSymbolizer
895	0x7fa901ac5dd0 runtime.cgoContextPCs
896	0x7fa901ac5eb0 runtime.(*_type).string
897	0x7fa901ac5f70 runtime.(*_type).uncommon
898	0x7fa901ac6020 runtime.(*_type).pkgpath
899	0x7fa901ac6130 runtime.resolveNameOff
900	0x7fa901ac6430 runtime.resolveTypeOff
901	0x7fa901ac6780 runtime.(*_type).textOff
902	0x7fa901ac6ac0 runtime.name.tagLen
903	0x7fa901ac6b20 runtime.name.name
904	0x7fa901ac6b70 runtime.name.tag
905	0x7fa901ac6c00 runtime.name.pkgPath
906	0x7fa901ac6d00 runtime.typelinksinit
907	0x7fa901ac73a0 runtime.typesEqual
908	0x7fa901ac8300 runtime.decoderune
909	0x7fa901ac8540 runtime.encoderune
910	0x7fa901ac86a0 runtime.vdsoInitFromSysinfoEhdr
1368	0x7fa901ac8b00 runtime.vdsoFindVersion
1369	0x7fa901ac8c40 runtime.vdsoParseSymbols
1370	0x7fa901ac8fd0 runtime.vdsoauxv
1371	0x7fa901ac90b0 runtime.inVDSOPage
1372	0x7fa901ac9120 runtime.mmap.func1
1373	0x7fa901ac91a0 runtime.munmap.func1
1374	0x7fa901ac91f0 runtime.sigaction.func1
1375	0x7fa901ac9260 runtime.cgocallbackg1.func1
1376	0x7fa901ac9270 runtime.cgoCheckWriteBarrier.func1
1377	0x7fa901ac9330 runtime.cgoCheckTypedBlock.func1
1378	0x7fa901ac9390 runtime.chansend.func1
1379	0x7fa901ac93e0 runtime.chanrecv.func1
1380	0x7fa901ac9430 runtime.debugCallCheck.func1
1381	0x7fa901ac9630 runtime.debugCallWrap.func1
1382	0x7fa901ac96a0 runtime.mallocgc.func1
1383	0x7fa901ac9710 runtime.persistentalloc.func1
1384	0x7fa901ac9780 runtime.freemcache.func1
1385	0x7fa901ac9840 runtime.SetFinalizer.func1
1386	0x7fa901ac9890 runtime.SetFinalizer.func2
1387	0x7fa901ac9920 runtime.gcStart.func1
1388	0x7fa901ac9960 runtime.gcStart.func2
1389	0x7fa901ac99d0 runtime.gcMarkDone.func1.1
1390	0x7fa901ac9a50 runtime.gcMarkDone.func1
1391	0x7fa901ac9af0 runtime.gcMarkDone.func2
1392	0x7fa901ac9bd0 runtime.gcMarkDone.func3
1393	0x7fa901ac9c20 runtime.gcMarkTermination.func1
1394	0x7fa901ac9c60 runtime.gcMarkTermination.func2
1395	0x7fa901ac9d80 runtime.gcMarkTermination.func3
1396	0x7fa901ac9dc0 runtime.gcMarkTermination.func4.1
1397	0x7fa901ac9e10 runtime.gcMarkTermination.func4
1398	0x7fa901ac9e50 runtime.gcBgMarkWorker.func1
1399	0x7fa901ac9ed0 runtime.gcBgMarkWorker.func2
1400	0x7fa901aca080 runtime.markroot.func1
1401	0x7fa901aca170 runtime.gcAssistAlloc.func1
1402	0x7fa901aca1c0 runtime.scanstack.func1
1403	0x7fa901aca220 runtime.getempty.func1
1404	0x7fa901aca290 runtime.freeSomeWbufs.func1
1405	0x7fa901aca380 runtime.(*mheap).coalesce.func1
1406	0x7fa901aca5a0 runtime.(*mheap).coalesce.func2
1407	0x7fa901aca780 runtime.(*mheap).alloc.func1
1408	0x7fa901aca7f0 runtime.(*mheap).freeSpan.func1
1409	0x7fa901aca920 runtime.mProf_Malloc.func1
1410	0x7fa901aca970 runtime.tracealloc.func1
1411	0x7fa901aca9d0 runtime.tracefree.func1
1412	0x7fa901acaa30 runtime.ReadMemStats.func1
1413	0x7fa901acaa70 runtime.wbBufFlush.func1
1414	0x7fa901acaac0 runtime.futexwakeup.func1
1415	0x7fa901acab70 runtime.sysSigaction.func1
1416	0x7fa901acabc0 runtime.newdefer.func1
1417	0x7fa901acad90 runtime.newdefer.func2
1418	0x7fa901acaee0 runtime.freedefer.func1
1419	0x7fa901acb070 runtime.preprintpanics.func1
1420	0x7fa901acb0e0 runtime.throw.func1
1421	0x7fa901acb170 runtime.fatalthrow.func1
1422	0x7fa901acb200 runtime.fatalpanic.func1
1423	0x7fa901acb2d0 runtime.fatalpanic.func2
1424	0x7fa901acb310 runtime.hexdumpWords.func1
1425	0x7fa901acb3b0 runtime.main.func1
1426	0x7fa901acb400 runtime.main.func2
1427	0x7fa901acb440 runtime.goready.func1
1428	0x7fa901acb490 runtime.casgstatus.func1
1429	0x7fa901acb550 runtime.casgstatus.func2
1430	0x7fa901acb650 runtime.startTheWorld.func1
1431	0x7fa901acb690 runtime.reentersyscall.func1
1432	0x7fa901acb790 runtime.entersyscallblock.func1
1433	0x7fa901acb8d0 runtime.entersyscallblock.func2
1434	0x7fa901acba10 runtime.exitsyscallfast.func1
1435	0x7fa901acbac0 runtime.exitsyscallfast_reacquired.func1
1436	0x7fa901acbb20 runtime.malg.func1
1437	0x7fa901acbb80 runtime.newproc.func1
1438	0x7fa901acbbf0 runtime.gfget.func1
1439	0x7fa901acbc50 runtime.(*rwmutex).rlock.func1
1440	0x7fa901acbd10 runtime.callers.func1
1441	0x7fa901acbdc0 runtime.tracebackHexdump.func1
1442	0x7fa901acbe00 runtime.vdsoParseSymbols.func1
1443	0x7fa901acbfa0 runtime.init.ializers
1444	0x7fa901acc120 runtime.init
1281	0x7fa901acc1b0 runtime.skipPleaseUseCallersFrames
1282	0x7fa901acc2b0 _rt0_amd64_lib
1283	0x7fa901acc360 _rt0_amd64_lib_go
1284	0x7fa901acc380 runtime.rt0_go
1285	0x7fa901acc4d0 runtime.asminit
1286	0x7fa901acc4e0 runtime.gogo
1287	0x7fa901acc540 runtime.mcall
1288	0x7fa901acc5a0 runtime.systemstack_switch
1289	0x7fa901acc5b0 runtime.systemstack
1290	0x7fa901acc650 runtime.morestack
1291	0x7fa901acc6e0 runtime.morestack_noctxt
1292	0x7fa901acc6f0 runtime.reflectcall
1293	0x7fa901acc8d0 callRet
1294	0x7fa901acc900 runtime.call32
1295	0x7fa901acc980 runtime.call64
1296	0x7fa901acca00 runtime.call128
1297	0x7fa901accab0 runtime.call256
1298	0x7fa901accb70 runtime.call512
1299	0x7fa901accc30 runtime.call1024
1300	0x7fa901acccf0 runtime.call2048
1301	0x7fa901accdb0 runtime.call4096
1302	0x7fa901acce80 runtime.call8192
1303	0x7fa901accf50 runtime.call16384
1304	0x7fa901acd020 runtime.call32768
1305	0x7fa901acd0f0 runtime.call65536
1306	0x7fa901acd1c0 runtime.call131072
1307	0x7fa901acd290 runtime.call262144
1308	0x7fa901acd360 runtime.call524288
1309	0x7fa901acd430 runtime.call1048576
1310	0x7fa901acd500 runtime.call2097152
1311	0x7fa901acd5d0 runtime.call4194304
1312	0x7fa901acd6a0 runtime.call8388608
1313	0x7fa901acd770 runtime.call16777216
1314	0x7fa901acd840 runtime.call33554432
1315	0x7fa901acd910 runtime.call67108864
1316	0x7fa901acd9e0 runtime.call134217728
1317	0x7fa901acdab0 runtime.call268435456
1318	0x7fa901acdb80 runtime.call536870912
1319	0x7fa901acdc50 runtime.call1073741824
1320	0x7fa901acdd20 runtime.procyield
1321	0x7fa901acdd30 runtime.publicationBarrier
1322	0x7fa901acdd40 runtime.jmpdefer
1323	0x7fa901acdd60 gosave
1324	0x7fa901acdda0 runtime.asmcgocall
1325	0x7fa901acde60 runtime.cgocallback
1326	0x7fa901acdeb0 runtime.cgocallback_gofunc
1171	0x7fa901acdfb0 runtime.setg
1172	0x7fa901acdfd0 setg_gcc
1173	0x7fa901acdfe0 runtime.abort
1174	0x7fa901acdff0 runtime.stackcheck
1175	0x7fa901ace020 runtime.cputicks
1176	0x7fa901ace040 runtime.aeshash
1177	0x7fa901ace060 runtime.aeshashstr
1178	0x7fa901ace080 runtime.aeshashbody
1179	0x7fa901ace5d0 runtime.aeshash32
1180	0x7fa901ace610 runtime.aeshash64
1181	0x7fa901ace650 runtime.checkASM
1182	0x7fa901ace670 runtime.return0
1183	0x7fa90da9acc0 _cgo_topofstack
1184fatal error: invalid runtime symbol table
1185runtime: panic before malloc heap initialized
1186
1187runtime stack:
1188runtime.throw(0x7fa901cb5043, 0x1c)
1189	/home/travis/.gimme/versions/go1.12.4.linux.amd64/src/runtime/panic.go:617 +0x74 fp=0x7fa8ff1c1d70 sp=0x7fa8ff1c1d40 pc=0x7fa901aa44c4
1190runtime.moduledataverify1(0x7fa90219c000)
1191	/home/travis/.gimme/versions/go1.12.4.linux.amd64/src/runtime/symtab.go:448 +0x5a6 fp=0x7fa8ff1c1e78 sp=0x7fa8ff1c1d70 pc=0x7fa901abdfc6
1192runtime.moduledataverify()
1193	/home/travis/.gimme/versions/go1.12.4.linux.amd64/src/runtime/symtab.go:416 +0x36 fp=0x7fa8ff1c1e98 sp=0x7fa8ff1c1e78 pc=0x7fa901abd9f6
1194runtime.schedinit()
1195	/home/travis/.gimme/versions/go1.12.4.linux.amd64/src/runtime/proc.go:537 +0x5c fp=0x7fa8ff1c1ef0 sp=0x7fa8ff1c1e98 pc=0x7fa901aa6f5c
1196runtime.rt0_go(0x7ffc71e686f8, 0xf, 0x7ffc71e686f8, 0x7fa8ff1c2700, 0x7fa97b6f0184, 0x0, 0x7fa8ff1c2700, 0x7fa8ff1c2700, 0x1051c9ce30de5c54, 0x0, ...)
1197	/home/travis/.gimme/versions/go1.12.4.linux.amd64/src/runtime/asm_amd64.s:195 +0x11e fp=0x7fa8ff1c1ef8 sp=0x7fa8ff1c1ef0 pc=0x7fa901acc49e

It does not happen in macOS.

Publish latest release on Maven

Right now latest release v2.0.1 has failed on publishing to Maven (https://github.com/src-d/infrastructure/issues/1125).

The idea is, in order to unblock https://github.com/src-d/gitbase-spark-connector-enterprise/pull/93 to release a v2.0.2-SNAPSHOT.

The last attempt of releasing from master though has failed on macOS cross-compilation

TODOs:

  • add new CI profile for macOS cross-compilation to .travis.yml
    In order to reproduce a failure from v2.0.2 release
     - <<: *release
       stage: test
       script: ./sbt assembly test
    
  • fix macOS build that seems to be failing on STDLIB usage
    Build logs
    
    In file included from src/main/native/org_bblfsh_client_v2_libuast_Libuast.cc:14:
    In file included from src/main/resources/libuast/libuast.hpp:6:
    In file included from /home/travis/osxcross/bin/../SDK/MacOSX10.11.sdk/usr/include/c++/v1/unordered_map:350:
    /home/travis/osxcross/bin/../SDK/MacOSX10.11.sdk/usr/include/c++/v1/__hash_table:1163:43: error: exception specification in declaration does not match previous declaration
    __hash_table<_Tp, _Hash, _Equal, _Alloc>::__hash_table()
                                              ^
    /home/travis/osxcross/bin/../SDK/MacOSX10.11.sdk/usr/include/c++/v1/__hash_table:850:5: note: previous declaration is here
        __hash_table()
        ^
    /home/travis/osxcross/bin/../SDK/MacOSX10.11.sdk/usr/include/c++/v1/__hash_table:1231:43: error: exception specification in declaration does not match previous declaration
    __hash_table<_Tp, _Hash, _Equal, _Alloc>::__hash_table(__hash_table&& __u)
                                              ^
    /home/travis/osxcross/bin/../SDK/MacOSX10.11.sdk/usr/include/c++/v1/__hash_table:864:5: note: previous declaration is here
        __hash_table(__hash_table&& __u)
        ^
    2 errors generated.
    
  • release a v2.0.2-SNAPSHOT and make sure it's published

Sporadic error: java.lang.NoClassDefFoundError: gopkg/in/bblfsh/sdk/v1/uast/generated/Node

Exception in thread "Executor task launch worker for task 3815" java.lang.NoClassDefFoundError: gopkg/in/bblfsh/sdk/v1/uast/generated/Node
at org.bblfsh.client.libuast.Libuast.filter(Native Method)
at org.bblfsh.client.BblfshClient.filter(BblfshClient.scala:34)
at tech.sourced.api.udf.QueryXPathUDF$$anonfun$queryXPath$2.apply(QueryXPathUDF.scala:46)
at tech.sourced.api.udf.QueryXPathUDF$$anonfun$queryXPath$2.apply(QueryXPathUDF.scala:44)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at tech.sourced.api.udf.QueryXPathUDF$.queryXPath(QueryXPathUDF.scala:44)
at tech.sourced.api.udf.QueryXPathUDF$$anonfun$apply$1.apply(QueryXPathUDF.scala:25)
at tech.sourced.api.udf.QueryXPathUDF$$anonfun$apply$1.apply(QueryXPathUDF.scala:24)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source)
at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:395)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:234)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:228)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:108)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Add support to query supported languages (with installed drivers)

In some applications, it is very useful to know which languages are supported (with installed drivers) beforehand. This helps to:

  • Display lists of supported languages in UX elements.
  • Reduce number of requests to bblfshd by avoiding requests for unsupported languages.

It would be nice to have such functionality implemented in the Scala client.

Set timeout for parse request

I'm running analyzis on many files some of them are big.
I see quite many

rpc error: code = DeadlineExceeded desc = context deadline exceeded

errors.

I would prefer to process slower but avoid these errors.

Timeout future similar to client-go could help here.

Context.root not implemented

Context.root. Even if we implemented it as in the python client it would not work, since Context tends to be a free context used only to copy nodes back and forth native memory.

I mean, right now we can call Context.root but it will say it does not find the native method. Probably we lost the method on the migration to v2.

We should provide a way of querying the root of a Context

Improve the CI

Potential action items:

  • make release job more modular #84 (comment)
  • do not install all native dependencies (presumably, some are only necessary for cross-compilation)
  • do not install all recommended drivers for bblfshd
  • check if building in separate job on OS X vm is faster then cross-compile and add it to normal CI flow

Deploy to Maven Central

  • Create a project at oss.sonatype.org
  • Add support to automatically deploy artifacts there.

Migrate scala-client to SDK.v3 + libuast.v3

An umbrella issue for client migration to new SDK.

Takes similar approach to bblfsh/client-python. but aims to avoid having all the changes in a single PR, so will be broken down.

  • SDK.v2 - parse API
    • parse() impl, that uses new protocol.v2
    • decode() impl, so the binary wire protocol can be read (on Go side)
    • root() impl, so the decoded (on Go side) UAST can be exposed to JVM
    • load() impl, so the Go-backed UAST can be transferred to host JVM memory #91
    • encode() impl. Reverse: JVM memory objects transferred to Go
    • expose mode argument for parse (native, annotated, semantic) #111
  • SDK.v2 - filter API
    • iterator() impl, so UAST can be iterated with and without loading it to JVM #106
    • filter() impl, so XPath filtering works #109
      take early feedback on API into account
    • Context impl (JVM-side factory), so multiple concurrent filter() calls from the client are supported
  • SDK.v2 - list languages API
    Support for languages/version request (as soon as bblfsh/sdk#400 is released)
  • doc: add JNI debugging instructions #115
  • doc: update API examples in README and CLI tool
  • investigate JNI warnings moved to #112
  • enable timeouts at v2 API again moved to #100
  • CI: refactor to multiple modular jobs (tests, releases per-platform, etc) moved to #87

StackOverflow error on filter method

In some cases, a StackOverflowError happens on bblfsh.filter method:

Exception in thread "Executor task launch worker for task 2273" java.lang.StackOverflowError
at org.bblfsh.client.libuast.Libuast.filter(Native Method)
at org.bblfsh.client.BblfshClient.filter(BblfshClient.scala:34)
at tech.sourced.api.udf.QueryXPathUDF$$anonfun$queryXPath$2.apply(QueryXPathUDF.scala:46)
at tech.sourced.api.udf.QueryXPathUDF$$anonfun$queryXPath$2.apply(QueryXPathUDF.scala:44)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at tech.sourced.api.udf.QueryXPathUDF$.queryXPath(QueryXPathUDF.scala:44)
at tech.sourced.api.udf.QueryXPathUDF$$anonfun$apply$1.apply(QueryXPathUDF.scala:25)
at tech.sourced.api.udf.QueryXPathUDF$$anonfun$apply$1.apply(QueryXPathUDF.scala:24)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source)
at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:395)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:234)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:228)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:827)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:108)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Restructure build

Right now, build consists of:

  • native part
  • Scala part

all being handled by sbt. As noted it would be a good idea to have a Makefile that CI uses, that takes care of native part and calls into the ./sbt.

Windows platform support

It looks like the current scala-client doesn't properly handle Windows. The same code works on Ubuntu 18.04 but causes an NPE at https://github.com/bblfsh/scala-client/blob/master/src/main/scala/org/bblfsh/client/v2/libuast/Libuast.scala#L62 on Windows.

It seems this is due to the scala-client not embedding the libuast.dll and loading this instead of libscalauast.so. Is this correct? If so, shouldn't the effort to get scala-client working with Windows be fairly low? I'm only seeing small changes to Libuast.scala and build.sbt being necessary. Am I missing something?

v2: investigate all warnings from -Xcheck:jni

When running test for JNI using -Xcheck:jni:

export JVM_OPTS='-Xcheck:jni'
./sbt test

JVM reports number of warnings that potentially can be memory leaks or a symptoms of another stability issue like:

WARNING in native method: JNI call made without checking exceptions when required to from CallStaticObjectMethod
WARNING: JNI local refs: 33, exceeds capacity: 32

This issue is about documenting, investigating and getting rid of as much of them as possible.

Make Node.filter() work

It would be more natural and Scala'ish.

Probably can be implemented using a Scala's implicit object or method taking Nodes.

Why JUInts?

Is there any strong reason to use JUints when Scala does not support the Unsigned Int type and python-client does not support them either?

Well, in fact in the python-client we have ways of casting to uint64 in the native part, but they are never going to get called, since the type of node, as linked before, is not going to ever return NODE_UINT

Connections left open when used from engine

When classifying languages and extracting UASTs it crashes as file descriptors are starved.

  • Spark shell (siva files are the ones that come with engine):
$ spark-shell --jars target/engine-uber.jar
scala> import tech.sourced.engine._
scala> val engine = Engine(spark, "./examples/siva-files")
scala> engine.getRepositories.getReferences.getHEAD.getFiles.classifyLanguages.extractUASTs.collect
[...]
[Stage 1:========>                                               (29 + 4) / 200]17/11/08 09:38:18 ERROR Executor: Exception in task 29.0 in stage 1.0 (TID 30)
org.apache.spark.SparkException: Failed to execute user defined function(anonfun$functionWithLang$1: (string, binary, string) => array<binary>)
[...]
Caused by: io.shadednetty.channel.ChannelException: Unable to create Channel from class class io.shadednetty.channel.socket.nio.NioSocketChannel
[...]
Caused by: io.shadednetty.channel.ChannelException: Failed to open a socket.
[...]
Caused by: java.net.SocketException: Too many open files
  • The number of files is 24559

  • lsof output after crash:

$ lsof | grep -F | grep -F -- '->z2:9432'
[...]
java      17657 18294 jfontan 4090u     IPv6            8054552       0t0        TCP localhost:43018->z2:9432 (ESTABLISHED)
java      17657 18294 jfontan 4091u     IPv6            8055029       0t0        TCP localhost:43022->z2:9432 (ESTABLISHED)
java      17657 18294 jfontan 4092u     IPv6            8055034       0t0        TCP localhost:43026->z2:9432 (ESTABLISHED)
java      17657 18294 jfontan 4093u     IPv6            8054562       0t0        TCP localhost:43030->z2:9432 (ESTABLISHED)
java      17657 18294 jfontan 4094u     IPv6            8056936       0t0        TCP localhost:43034->z2:9432 (ESTABLISHED)
java      17657 18294 jfontan 4095u     IPv6            8054568       0t0        TCP localhost:43038->z2:9432 (ESTABLISHED)
java      17657 18297 jfontan  356u     IPv6            8054691       0t0        TCP localhost:43050->z2:9432 (ESTABLISHED)
java      17657 18297 jfontan  357u     IPv6            8054693       0t0        TCP localhost:43054->z2:9432 (ESTABLISHED)
java      17657 18297 jfontan  381u     IPv6            7337140       0t0        TCP localhost:56368->z2:9432 (ESTABLISHED)
java      17657 18297 jfontan  382u     IPv6            7336274       0t0        TCP localhost:56364->z2:9432 (ESTABLISHED)
java      17657 18297 jfontan  383u     IPv6            7330339       0t0        TCP localhost:56356->z2:9432 (ESTABLISHED)
java      17657 18297 jfontan  384u     IPv6            7336275       0t0        TCP localhost:56354->z2:9432 (ESTABLISHED)
[...]
$ lsof | grep -F | grep -F -- '->z2:9432' | wc -l
330813
  • Excerpt from babelfish container log:
time="2017-11-08T08:37:42Z" level=warning msg="request processed content 773 bytes, status Error" elapsed=9.856827ms language=python
time="2017-11-08T08:37:42Z" level=error msg="error selecting pool: unexpected error: runtime failure: missing driver for language """
time="2017-11-08T08:37:42Z" level=error msg="request processed content 1493 bytes, status Fatal" elapsed=508.294µs language=
time="2017-11-08T08:37:42Z" level=warning msg="request processed content 773 bytes, status Error" elapsed=7.443573ms language=python
time="2017-11-08T08:37:42Z" level=error msg="request processed content 779 bytes, status Fatal" elapsed=29.212264ms language=python
time="2017-11-08T08:37:42Z" level=error msg="request processed content 1791 bytes, status Fatal" elapsed=57.275873ms language=python
time="2017-11-08T08:37:42Z" level=error msg="request processed content 779 bytes, status Fatal" elapsed=24.649807ms language=python

v2: expose filtering for values

Make it possible to query for individual value node types again and document it in README:

// Filtered using XPath functions returning value types
client.filterBool(resp.get, "boolean(//*[@strtOffset or @endOffset])")
client.filterString(resp.get, "name(//*[1])")
client.filterNumber(resp.get, "count(//*)")

Implement JNode encoding

After migration to SDK.3 and protocol.v2 both Python and Scala clients only support encoding of the trees that already have a Context (e.g created by decoding it first).

This issue is about exposing of encoding for an arbitrary JNode created on JVM side.

Expose method for gRPC connection termination

Right now BblfshClient manages the connection creation, but it relies on GC to shut the connection down.

It would be nice to expose something gRCP's channel .shutdownNow() method to allow clients to for more fine-grained resource control and to avoid this message on a process termination (i.e in tests):

ERROR 00:53:24,580 io.grpc.internal.ManagedChannelImpl$ManagedChannelReference (ManagedChannelImpl.java:1236) - *~*~*~ Channel io.grpc.internal.ManagedChannelImpl-1 for target 127.0.0.1:9432 was not shutdown properly!!! ~*~*~*
    Make sure to call shutdown()/shutdownNow() and awaitTermination().
java.lang.RuntimeException: ManagedChannel allocation site
	at io.grpc.internal.ManagedChannelImpl$ManagedChannelReference.<init>(ManagedChannelImpl.java:1185)
	at io.grpc.internal.ManagedChannelImpl.<init>(ManagedChannelImpl.java:504)
	at io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:369)
	at org.bblfsh.client.BblfshClient.<init>(BblfshClient.scala:19)
	at org.bblfsh.client.BblfshClient$.apply(BblfshClient.scala:81)

Will be happy to provide a patch, in case there is a consensus of usefulness of such method.

Fail to run latest release 1.3.4

Follow the steps from https://github.com/bblfsh/client-scala#usage

step to reproduce

  • wget http://central.maven.org/maven2/org/bblfsh/bblfsh-client/1.3.4/bblfsh-client-1.3.4.jar
  • java -jar ./bblfsh-client-1.3.4.jar --host 0.0.0.0 -f src/test/resources/python_file.py

Expected output
printed uast according to https://github.com/bblfsh/client-scala#usage

Actual output

Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: scala/Function0
	at java.lang.Class.getDeclaredMethods0(Native Method)
	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
	at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
	at java.lang.Class.getMethod0(Class.java:3018)
	at java.lang.Class.getMethod(Class.java:1784)
	at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: scala.Function0
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 7 more

Add MacOS support

The native libuast shared library must be built (or downloaded) for MacOS.

Node::ValueAt may be leaking memory

After a glimpse to the methods in Node (C++ side), I have the (untested of course) hypothesis that ValueAt may be leaking memory.

  • ValueAt creates new global ref for and object obj and calls lookupOrCreate(obj)
  • lookupOrCreate tries to find a Node for obj (if it previously existed, we created a new reference to it, and we are only deallocating one). If no Node existed, calls the constructor of Node, which creates a new reference to obj, but we do not release the original one at any point D:

Proposed solution is end ValueAt method with:

return lookupOrCreate(val);

instead of

return lookupOrCreate(env->NewGlobalRef(val));

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.