azure / azure-batch-sdk-for-java

Azure Batch Libraries for Java http://azure.com/java

This repository is deprecated and the current Azure Batch Java SDK is located here.

Azure Batch Libraries for Java

This README is based on the latest released version Azure Batch SDK (5.0.1). If you are looking for other releases, see the More Information section below.

The Azure Batch Libraries for Java is a higher-level, object-oriented API for interacting with the Azure Batch service.

5.0.1 is a release that supports all features of Azure Batch service with API version "2018-12-01.8.1". We will be adding support for more new features and tweaking the API associated with Azure Batch service newer release.

Azure Batch Authentication

You need to create a Batch account through the Azure portal or Azure cli.

• Use the account name, key, and URL to create a BatchSharedKeyCredentials instance for authentication with the Azure Batch service. The BatchClient class is the simplest entry point for creating and interacting with Azure Batch objects.

BatchSharedKeyCredentials cred = new BatchSharedKeyCredentials(batchUri, batchAccount, batchKey);
BatchClient client = BatchClient.open(cred);

• The other way is using AAD (Azure Active Directory) authentication to create the client. See this document for detail.

BatchApplicationTokenCredentials cred = new BatchApplicationTokenCredentials(batchEndpoint, clientId, applicationSecret, applicationDomain, null, null);
BatchClient client = BatchClient.open(cred);

Create a pool using an Azure Marketplace image

You can create a pool of Azure virtual machines which can be used to execute tasks.

System.out.println("Created a pool using an Azure Marketplace image.");

VirtualMachineConfiguration configuration = new VirtualMachineConfiguration();
configuration.withNodeAgentSKUId(skuId).withImageReference(imageRef);
client.poolOperations().createPool(poolId, poolVMSize, configuration, poolVMCount);

System.out.println("Created a Pool: " + poolId);

Create a Job

You can create a job by using the recently created pool.

PoolInformation poolInfo = new PoolInformation();
poolInfo.withPoolId(poolId);
client.jobOperations().createJob(jobId, poolInfo);

Sample Code

You can find sample code that illustrates Batch usage scenarios in https://github.com/azure/azure-batch-samples

Download

5.0.1

If you are using released builds from 5.0.1, add the following to your POM file:

<dependency>
    <groupId>com.microsoft.azure</groupId>
    <artifactId>azure-batch</artifactId>
    <version>5.0.1</version>
</dependency>
<dependency>
    <groupId>com.microsoft.azure</groupId>
    <artifactId>azure-client-runtime</artifactId>
    <version>1.6.4</version>
</dependency>

Pre-requisites

• A Java Developer Kit (JDK), v 1.7 or later
• Maven
• Azure Service Principal - see how to create authentication info.

Help

If you encounter any bugs with these libraries, please file issues via Issues or checkout StackOverflow for Azure Java SDK.

Contribute Code

If you would like to become an active contributor to this project please follow the instructions provided in Microsoft Azure Projects Contribution Guidelines.

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request

Build Code

To build the code open a console, navigate to the git repository, and run

maven build

Test Code

To run tests, set the following environment variables:

• AZURE_BATCH_ENDPOINT
• CLIENT_ID
• APPLICATION_SECRET
• AZURE_BATCH_ACCOUNT
• AZURE_BATCH_ACCESS_KEY
• STORAGE_ACCOUNT_NAME
• STORAGE_ACCOUNT_KEY Then run any test in src/test/java directory.

More Information

• Javadoc
• http://azure.com/java
• If you don't have a Microsoft Azure subscription you can get a FREE trial account here

Previous Releases and Corresponding Repo Branches

Version	SHA1	Remarks
5.0.1	5.0.1	Tagged release for 5.0.1 version of Azure Batch libraries
5.0.0	5.0.0	Tagged release for 5.0.0 version of Azure Batch libraries
4.0.1	4.0.1	Tagged release for 4.0.1 version of Azure Batch libraries
4.0.0	4.0.0	Tagged release for 4.0.0 version of Azure Batch libraries
3.2.0	3.2.0	Tagged release for 3.2.0 version of Azure Batch libraries
3.1.0	3.1.0	Tagged release for 3.1.0 version of Azure Batch libraries
3.0.0	3.0.0	Tagged release for 3.0.0 version of Azure Batch libraries
2.1.0	2.1.0	Tagged release for 2.1.0 version of Azure Batch libraries
2.0.0	2.0.0	Tagged release for 2.0.0 version of Azure Batch libraries
1.0.0	1.0.0	Tagged release for 1.0.0 version of Azure Batch libraries
1.0.0-beta2	1.0.0-beta2	Tagged release for 1.0.0-beta2 version of Azure Batch libraries

---

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

azure-batch-sdk-for-java's Issues

Add Async Methods at Convience Layer

Auto-rest exposes a Java 1.7 pattern for Futures at the Protocol layer. Extend this to the convenience layer

DetailLevel on Job ExecutionInfo/PoolId does not work

NOT AN ISSUE ANYMORE, It was an error on my side (wrong batch account). How do I close this issue ?

I try to get all the jobs for a given pool id using the following construct:
DetailLevel detailLevel = new DetailLevel.Builder().withFilterClause(String.format("executionInfo/poolId eq '%s'", myPoolId)) .build();
this.batchClient.jobOperations().listJobs(detailLevel);
myPoolId is a string with a valid pool name with jobs running on it.

The returned list is always empty

Using Batch Lib SDK 2.1.0

Changes to CreateTasksTerminatedException

Some issues and changes to consider for the CreateTasksTerminatedException class:

• The pendingList parameter is not used in the constructor.
• The constructor should probably take readonly copies of the list
• Consider renaming the class to CreateTasksErrorException or something similar?
• Consider renaming the properties and parameters to either failedTasks and pendingTasks or failureList and pendingList. Currently, failureTasks and pendingList aren't aligned with each other.

Bulk add task uses first instance of BatchClientParallelOptions

The bulk add task operation uses the first instance of the BatchClientParallelOptions in the behaviors list:

https://github.com/Azure/azure-batch-sdk-for-java/blob/master/src/main/java/com/microsoft/azure/batch/TaskOperations.java#L199

    for (BatchClientBehavior op : bhMgr.getMasterListOfBehaviors()) {
        if (op instanceof BatchClientParallelOptions) {
            threadNumber = ((BatchClientParallelOptions)op).maxDegreeOfParallelism();
            break;
        }
    }

Typically, the behaviors are processed in order, and the last interceptor to modify a property is the winner. This is advertised in our docs. Using the first instance of the parallel options is inconsistent.

TaskOperations.createTasks does not release lock in some scenarios

Hi,

When we continuously executing batch tasks by calling createTasks, lock is not released in a call and then block the thread.

Stack like this:

    at com.microsoft.azure.batch.TaskOperations.createTasks(TaskOperations.java:256)
    - locked <0x00000006d68b5200> (a java.lang.Object)
    at com.microsoft.azure.batch.TaskOperations.createTasks(TaskOperations.java:93)

Snippet in TaskOperations:

            if (taskList.size() > 0) {
                submit_chunk(taskList);
            }

            synchronized (lock) {
                // Notify main thread that sub thread finished
                lock.notify();
            }

If some exception throws from submit_chunk and not be caught, will not notify to release lock.

That is a problem, maybe lock.notify() should be surround in finally block. Please check, thanks!

Need unit tests + CI job to run them.

There's a few integration tests that hit a live endpoint, but we need some unit tests that we can add to a CI job.

Trim the ‘/’ in URL

Minor bug in java SDK if not trimmed by user.