tgrez / sejm-ngram Goto Github PK
View Code? Open in Web Editor NEWvisualization of trends in the use of words and phrases in the parliament
Home Page: http://www.sejmotrendy.pl
visualization of trends in the use of words and phrases in the parliament
Home Page: http://www.sejmotrendy.pl
Na różnych poziomach (przynajmniej error i debug). Tak aby można było przynajmniej ogólnie wiedzieć co się dzieje w aplikacji, bez debugowania.
dodać Dropwizardowy healthcheck.
Można pomyśleć nad konfiguracją, aby informowało nas, gdy przestanie działać.
w momencie kiedy padnie serwer i nginx zwraca 502 bad gateway, to zamiast tego powinna wyświetlić się strona maintenance'owa.
In the data from PSC, even in /psc-data/partiaId.json file you can find many ids for "Sojusz Lewicy Demokratycznej" party (also for the others, sometimes the parties' names are similiar, but obviously they refer to the same entity).
currently it's based on a single day
na razie chyba mysqlową lub postgresową
a w przyszłości także podłączyć do travisa, by to wywoływał
Hej,
Trafiłem na to repozytorium. Jesteście super. Będzie świetnie.
Co to jest?
Z wyrazami szacunku,
Podobnie jak na pierwszym screenie
Things left:
Powinno wyglądać to tak:
https://www.dropbox.com/s/6i58adygm42hqge/ST_main_how-it-works.png?dl=0
Ikonki są dostępne na Dropboxie
Failed to load resource: the server responded with a status of 404 (Not Found) http://sejmotrendy.pl/javascripts/smoothScroll.js
Powinno to wyglądać mniej więcej tak:
https://www.dropbox.com/s/ch3cvd5r8c7nif6/ST_results_oct_styles.png?dl=0
Ja rozumiem, że po wejściu na stronę widoczny jest aktualny screen, a pod nim byłaby sekcja "Jak to działa". Po wpisaniu jakiegoś ngramu rozsuwa się area z grafem (tak jak to jest teraz).
with not too much data
When we had one ngram, it was simple: data series that might be visualized represent date for various parties, you can turn them on / of (show / hide). Each party + ALL parties have thei'r own color.
However in the laters design portion, we have a possibility to specify few ngrams (seprated by comma ','). What happens with legend and colors then? What's the interaction there?
to avoid additional config fiels
do sprawdzenia
Because Jedis is not thread-safe.
z użyciem travis ci na serwer testowy (aws lub e24cloud)
Jeśli wyjątki nie mogą być w sensowny sposób obsłużone należy rzucać WebApplicationException.
Utility class może zostać stoworzone, aby uniknąć powtarzania kodu.
Powinno to wyglądać tak, jak na screenie z pierwszego linka:
https://www.dropbox.com/s/ch3cvd5r8c7nif6/ST_results_oct_styles.png?dl=0
Maćku, kiedyś już były takie ikonki na stornie, z naszymi zdjęciami, co się z nimi stało? Ira, czy masz gdzieś te ikonki, mogłabyś je podesłać?
after some time db connection on AWS server gets dropped and request to REST API gets HTTP 500 as a response.
Stacktrace:
ERROR [2014-04-24 17:35:28,512] com.example.helloworld.resources.DemoNgramResource: aborcja
ERROR [2014-04-24 17:35:28,517] com.yammer.dropwizard.jersey.LoggingExceptionMapper: Error handling a request: 89a537dca3b3f137
! org.jooq.exception.DataAccessException: SQL [select sejmngram
.ngrams
.id
, sejmngram
.ngrams
.datefrom
, sejmngram
.ngrams
.dateto
, sejmngram
.ngrams
.ngram
, sejmngram
.ngrams
.nrOccurences
, sejmngram
.ngrams
.content
from sejmngram
.ngrams
where sejmngram
.ngrams
.ngram
= ?]; No operations allowed after connection closed.
! at org.jooq.impl.Utils.translate(Utils.java:1288) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.impl.DefaultExecuteContext.sqlException(DefaultExecuteContext.java:495) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:327) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.impl.AbstractResultQuery.fetch(AbstractResultQuery.java:330) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.impl.SelectImpl.fetch(SelectImpl.java:2256) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.sejmngram.database.fetcher.connection.MySqlDbConnector.queryDatabase(MySqlDbConnector.java:63) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.sejmngram.database.fetcher.connection.MySqlDbConnector.retrieve(MySqlDbConnector.java:56) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.example.helloworld.factory.NgramProvider.generateNgramResponse(NgramProvider.java:49) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.example.helloworld.resources.DemoNgramResource.sayHello(DemoNgramResource.java:48) ~[rest-server.jar:1.0-SNAPSHOT]
! at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) ~[na:na]
! at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_51]
! at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_51]
! at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.yammer.metrics.jersey.InstrumentedResourceMethodDispatchProvider$TimedRequestDispatcher.dispatch(InstrumentedResourceMethodDispatchProvider.java:32) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.yammer.dropwizard.jersey.OptionalResourceMethodDispatchAdapter$OptionalRequestDispatcher.dispatch(OptionalResourceMethodDispatchAdapter.java:37) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.server.impl.uri.rules.ResourceObjectRule.accept(ResourceObjectRule.java:100) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511) [rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442) [rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391) [rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381) [rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) [rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538) [rest-server.jar:1.0-SNAPSHOT]
! at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716) [rest-server.jar:1.0-SNAPSHOT]
! at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:669) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1448) [rest-server.jar:1.0-SNAPSHOT]
! at com.yammer.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:29) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) [rest-server.jar:1.0-SNAPSHOT]
! at com.yammer.metrics.jetty.InstrumentedHandler.handle(InstrumentedHandler.java:200) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:264) [rest-server.jar:1.0-SNAPSHOT]
! at com.yammer.dropwizard.jetty.BiDiGzipHandler.handle(BiDiGzipHandler.java:123) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.Server.handle(Server.java:368) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.server.nio.BlockingChannelConnector$BlockingChannelEndPoint.run(BlockingChannelConnector.java:298) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [rest-server.jar:1.0-SNAPSHOT]
! at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [rest-server.jar:1.0-SNAPSHOT]
! at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
Caused by: ! com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.
! at sun.reflect.GeneratedConstructorAccessor16.newInstance(Unknown Source) ~[na:na]
! at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.7.0_51]
! at java.lang.reflect.Constructor.newInstance(Constructor.java:526) ~[na:1.7.0_51]
! at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.Util.getInstance(Util.java:386) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1015) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.ConnectionImpl.throwConnectionClosedException(ConnectionImpl.java:1304) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.ConnectionImpl.checkClosed(ConnectionImpl.java:1296) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4514) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.ConnectionImpl.prepareStatement(ConnectionImpl.java:4479) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.impl.ProviderEnabledConnection.prepareStatement(ProviderEnabledConnection.java:112) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.impl.SettingsEnabledConnection.prepareStatement(SettingsEnabledConnection.java:76) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.impl.AbstractResultQuery.prepare(AbstractResultQuery.java:230) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:297) ~[rest-server.jar:1.0-SNAPSHOT]
!... 54 common frames omitted
Caused by: ! com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 65,028,684 milliseconds ago. The last packet sent successfully to the server was 65,028,686 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
! at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.7.0_51]
! at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) ~[na:1.7.0_51]
! at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.7.0_51]
! at java.lang.reflect.Constructor.newInstance(Constructor.java:526) ~[na:1.7.0_51]
! at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3941) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2551) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2731) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2818) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2157) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1379) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.tools.jdbc.DefaultPreparedStatement.execute(DefaultPreparedStatement.java:194) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.impl.AbstractResultQuery.execute(AbstractResultQuery.java:261) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:323) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.impl.AbstractResultQuery.fetch(AbstractResultQuery.java:330) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.jooq.impl.SelectImpl.fetch(SelectImpl.java:2256) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.sejmngram.database.fetcher.connection.MySqlDbConnector.queryDatabase(MySqlDbConnector.java:63) ~[rest-server.jar:1.0-SNAPSHOT]
! at org.sejmngram.database.fetcher.connection.MySqlDbConnector.retrieve(MySqlDbConnector.java:56) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.example.helloworld.factory.NgramProvider.generateNgramResponse(NgramProvider.java:49) ~[rest-server.jar:1.0-SNAPSHOT]
! at com.example.helloworld.resources.DemoNgramResource.sayHello(DemoNgramResource.java:48) ~[rest-server.jar:1.0-SNAPSHOT]
! at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_51]
! at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_51]
!... 47 common frames omitted
Caused by: ! java.net.SocketException: Broken pipe
! at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.7.0_51]
! at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113) ~[na:1.7.0_51]
! at java.net.SocketOutputStream.write(SocketOutputStream.java:159) ~[na:1.7.0_51]
! at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.7.0_51]
! at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[na:1.7.0_51]
! at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3922) ~[rest-server.jar:1.0-SNAPSHOT]
!... 63 common frames omitted
127.0.0.1 - - [24/Apr/2014:17:35:28 +0000] "GET /service/api2/ngram/aborcja HTTP/1.0" 500 1498 8 8
Po wyszukaniu hasła powinien się zmienić adres URL strony, aby można było udostępnić URL w internecie.
Przykład.
When i Type for instance "Tusk" it shows 0 results. When I go for "tusk" - here are the results. I guess simple downcasing on frontend side would help that.
w bazie są integery, więc db-fetcher powinien zwracać "wzbogacone dane"
może np. memcached albo redis?
Zgodnie z sugestią Iry, dla wielu ngramów checkboxy z nazwami partii powinny się zmienieć na radiobuttony. W taki sposób będzie możliwe wybranie nazwy tylko jednej partii i nasz wykres umożliwi porównanie kilku ngramów dla jednej tylko partii.
pliki:
nowe_daty.txt
poselId.txt
partieId.txt
powinny być wciągane poprzez plik hello-world.yml i HelloWorldConfiguration (nazwy do zmiany).
Dla pojedynczego ekranu ściagamy z API (póki co) od razu wystąpienia dla wszystkic partii. W związku z tym możemy stworzyć legendę zawierająca odpowiednie kolory i checkboxy dla tych partii właśnie.
Powinno to wygląda tak, jak tu:
https://www.dropbox.com/s/wmdlovfuy8rya96/ST_results_oct_preview.jpg?dl=0
W bieżacej wersji fajne jest to, że że area grafu jest b. szeroka. Niestety, chyba trzeba będzie ją zwęzić, żeby zmieściłą się legenda (tak jak na screenshocie).
Pozbierać SQLki budujące bazę (tabelę wystapienia), tworzące użytkowników, tworzące procedurę GetWystapiania oraz funkcję term_count
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.