mminella / scaling-demos Goto Github PK

View Code? Open in Web Editor NEW

86.0 9.0 61.0 24.45 MB

Demos of the various methods of scaling Spring Batch

Shell 30.47% Batchfile 23.53% Java 46.00%

scaling-demos's People

Contributors

Stargazers

Watchers

scaling-demos's Issues

https://stackoverflow.com/questions/64575208/how-to-use-classifiercompositeitemprocessor-in-spring-batch-and-write-data-into ?

Hi Michael,

Would you mind sharing the solution for this problem - https://stackoverflow.com/questions/64575208/how-to-use-classifiercompositeitemprocessor-in-spring-batch-and-write-data-into ?

org.springframework.batch.item.file.FlatFileParseException: Parsing error - Spring Batch fixedwidth file

I am looking to read the fixed-width flatfile using Spring Batch generated from Mainframe system. This file does not have any delimiter and one complete records has 1-2000 characters or columns and 2001 to 4000 as characters length. The main issue is for few records, data spread across two lines or three lines and there I am facing issue while reading the code.

Could you please guide me here: https://stackoverflow.com/questions/63674370/caused-by-org-springframework-batch-item-file-transform-incorrectlinelengthexce

test

00200621 SUNDAY   0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         000000000000000000                                                                                                                                                                     
560000000000411999999992052300000000D 0000        0000000000010000000100000040000000000000  00000000                    NYNNVX      N N 0      N004 000100000001000100000001000100000001000100000001000100000001000100000001000100000001                                                YNYNYYNNNNNYNNNN0004000000070000000300010000000000000000000000020000000000000000NN1N                         N00NNNND                                                                                                                                                       001NNN              00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000O840000000000000000000AN0201000000NNNC840                N  N00N A NN00400000000NNNNNUSAN       NNNN00000000000000NN141900INNNNNN   N                 N000000                 NN   �        200//0055//20000YNN MO    �¶200528000000       !!B3K555800000001A****00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             0005230000000000000000                                                                                                                                                                 
560000000000711111000614000300000000D   00        0000000002010000389762888580000000000000  00000000                    NNNN        N N 6      N    000000000000000000000000000000000000000000000000000000000000000000000000000000000000                                                YYYYYYYYNNNYYYNN0002000300040005000100060009000700000000000000080010001100000000NN N                         N00NNNND                                                                                                                                                       001NNF00000000007   00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000B840000000000238976288NY0201000000NNNC840                N  N00Y N NN84000000000NNNNNUSAN       APPN00000000000000NN161912NBD NNN044N                 N000000                 YY   �        200//0011//00000YYU AL    �¶200109000000       !�)+M555800000001A****00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             0000030000000000000000                                                                                                                                                                 
560000000004011072000326003304550456C 0121                  010000000000008400000400040011F 00090000        �#Eg‰�#E    NNNN       NNNYN0      N    000000000000000000000000000000000000000000000000000000000000000000000000000000000000                5C7E571E8C176C54                YYYNYYNNNNNNNNNN0002000300040000000100050000000000000000000000000000000000000000NN N                         N00                                                                                                                                                               NNN           MSI00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000011110001111100011111000111111001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000O840000000000000000000NY0102000000NNN                    NYNN00N0NNNN84000007504NNNNNUSAN   0MPFNNNN00130001000000NN160000NNN NNN   N                 N000000                 NN1600        155//1122//10009YYU     00  111111000000       ��Ô��025500000255C****00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             0000330004550004560000                                                                                                                                                                 
560000000010111111000614000300000000D 0300        0000000002010000000000008580000000000000  00000000                    NNNN        N N 6      N840 000000000000000000000000000000000000000000000000000000000000000000000000000000000000                                                YYYYYYYYNNNYYYNN0002000300040005000100060009000700000000000000080010001100000000NN N                         N00NNNND                                                                                                                                                       001NNF00000000101   00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000B840000000000200000000NY0202000000NNNC840                N  N00Y N NN85800000000NNNNNUSAN       APPN00000000000000NN161912NBDNNNN044N                 N000000                 YY   �        200//0011//10000YYU AL    �¶200117000000       !�GÛI555800000001A****00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             0000030000000000000000                                                                                                                                                                 
560000000011111111000614000300000000D   00        0000000002010000389762888580000000000000  00000000                    NNNN        N N 6      N    000000000000000000000000000000000000000000000000000000000000000000000000000000000000                                                YYYYYYYYNNNYYYNN0002000300040005000100060009000700000000000000080010001100000000NN N                         N00NNNND                                                                                                                                                       001NNF00000000111   00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000B840000000000238976288NY0202000000NNNC840                N  N00Y N NN84000000000NNNNNUSAN       APPN00000000000000NN161912NBD NNN044N                 N000000                 YY   �        200//0011//10000YYU AL    �¶200110000000       !��J555800000001A****00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             0000030000000000000000                                                                                                                                                                 
560000000045711072000326003300330033C   00                  010000000000008400000000000000  00000000                    NNNN       NNNYN0      N    000000000000000000000000000000000000000000000000000000000000000000000000000000000000                                                NNNNNYNNNNNNNNNN0000000000000000000000010000000000000000000000000000000000000000NN N3ABCD                    N00NNNN                                                                                                                                                           NNN              00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000011101000000020002220000033303000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000O840000000000000000000NY0102000000NNN                    NYNN00N0NNNN84000000000NNNNNUSAN   0MPFNNNN00000000000000NN160000NNN NNN   N                 N000000                 NN �CP        199//0077//00009YNU     00'
080604000000        ²�›p999700082768C****00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             0000330000330000330000                                                                                                                                                                 
560000000456 10072000326045604560456D 0100                  010000000020008401000000000000  00000000                    NYYYVXVXVX NYNYY5      Y840 0010000002000005000001000005000001000005000001000005000001000005000001000005000001005D60811AB3DB1E73                                YNNNYNNNNNNNNN  0002000000000000000100000000000000000000000000000000000000000000NN N                         N00                                                                                                                                                               NNN           MSI00022200000000000000003330000300003000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000112222303111121112200122223001333220033333300330000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000O840000000000000000000NY0102000000NNN                    NYNN00N0NNNN84077777701YNNNNUSAN   0   NNNN00000000000000NN160000NBB NNN   N                 N000000                 NN1600  000000144//0022//00009YYU MI  0016050507000000 USA   �¹œ…¸025500000255C****00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             00             0004560004560004560000

how-to-override-spring-batch-compositeitemwriter-manage-transaction-for-delegate

Dear Michael,

Could you please guide us here : https://stackoverflow.com/questions/63296065/how-to-override-spring-batch-compositeitemwriter-manage-transaction-for-delegate ?

We're looking to achieve this scenario somehow as its business need.

HikariPool-1 - Connection org.postgresql.jdbc.PgConnection@7d18a7dc marked as broken because of SQLSTATE(08006), ErrorCode(0)

Can you please guide me here: https://stackoverflow.com/questions/60377396/postgresql-connection-marked-as-broken-by-hikari?

Remote Partitioning - No StepExecution could be located for this request

Followed all the steps shared and executed Master and slaves separately. When master is executed messages are sent to JMS but facing below issue in slave JVM's. Can you please help and suggest where am going wrong. Below is error log.

Caused by: org.springframework.batch.core.step.NoSuchStepException: No StepExecution could be located for this request: StepExecutionRequest: [jobExecu
tionId=0, stepExecutionId=4, stepName=workerStep]
at org.springframework.batch.integration.partition.StepExecutionRequestHandler.handle(StepExecutionRequestHandler.java:54) ~[spring-batch-integ
ration-4.3.0.jar!/:4.3.0]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:564) ~[na:na]
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:171) ~[spring-messaging-5.3.1.j
ar!/:5.3.1]
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120) ~[spring-messaging-5.3.1.jar
!/:5.3.1]
at org.springframework.integration.handler.support.MessagingMethodInvokerHelper$HandlerMethod.invoke(MessagingMethodInvokerHelper.java:1105) ~[
spring-integration-core-5.4.1.jar!/:5.4.1]
at org.springframework.integration.handler.support.MessagingMethodInvokerHelper.invokeHandlerMethod(MessagingMethodInvokerHelper.java:583) ~[sp
ring-integration-core-5.4.1.jar!/:5.4.1]
at org.springframework.integration.handler.support.MessagingMethodInvokerHelper.processInternal(MessagingMethodInvokerHelper.java:478) ~[spring
-integration-core-5.4.1.jar!/:5.4.1]
at org.springframework.integration.handler.support.MessagingMethodInvokerHelper.process(MessagingMethodInvokerHelper.java:356) ~[spring-integra
tion-core-5.4.1.jar!/:5.4.1]
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:108) ~[spring-inte
gration-core-5.4.1.jar!/:5.4.1]
... 42 common frames omitted

2021-12-06 15:06:17.228 WARN 384784 --- [erContainer#0-1] o.s.j.l.DefaultMessageListenerContainer : Execution of JMS message listener failed, and no
ErrorHandler has been set.

Can you create few tutorials around Spring Cloud Data Flow?

Need code

Hey,

Thanks for the wonderful demo and it's a great asset to start the development. However I want to get the code you shown in the video for MasterConfiguration and WorkerConfiguration?

Could you please also guide on below ?

How can we execute B batch job after the success of A batch by making configurations in SCDF UI ?
How can we dynamically pass the Job Param through SCDF UI ?
Would you mind sharing the latest SCDF tutorials to work with Spring Boot v2.2.6.RELEASE + ?

Caused by: java.lang.NullPointerException: Cannot invoke "com.thoughtworks.xstream.XStream.fromXML(String)" because "this.xstream" is null

Can you please guide here - https://stackoverflow.com/questions/70533792/caused-by-java-lang-nullpointerexception-cannot-invoke-com-thoughtworks-xstre?noredirect=1#comment124725851_70533792?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

Jobs

Jooble