Comments (7)
Im with the same issue!
from aztk.
FYI, after creating resources manually, aztk works.
from aztk.
I have the same issue.
from aztk.
FYI, after creating resources manually, aztk works.
Tks for sharing, i will use the Azure AKS for the same propose, for those who are interested
https://docs.microsoft.com/en-us/azure/aks/spark-job
from aztk.
FYI, after creating resources manually, aztk works.
Im with the same issue!
FYI, after creating resources manually, aztk works.
can u explain how?
from aztk.
can u explain how?
I assume you have already an Azure subscription and that you have run aztk spark init.
You have to:
- Open Azure AD and get the tenant ID -> copy that into .aztk/secrets.yaml in service_principal.tenant_id
- Create an app registration in Azure AD and get the application ID -> copy into client_id
- Create a new client secret -> copy into credential
- Create a new Batch Account (if you dont'have one already) and give the App you created before the "Contributor" role on that. Then take the Batch Account resource Id from the Properties pane and paste it on the secrets.yaml, service_principal, batch_account_resource_id
- Create a new Storage Account (if you dont'have one already) and give the App you created before the "Contributor" role on that. Then take the Storage Account resource Id from the Properties pane and paste it on the secrets.yaml, service_principal, storage_account_resource_id
- (Optional) If you want to use Mixed Mode (spot instances), you have to create a Vnet, to give the app the Contributor role on that, to create a subnet into the Vnet, and to put the subnet Resource ID into cluster.yaml
from aztk.
Please follow the manual steps here: https://aztk.readthedocs.io/en/v0.10.2/00-getting-started.html
from aztk.
Related Issues (20)
- Requested number of low priority VMs are not provisioned. HOT 1
- AZTK container has 338 CVEs HOT 2
- SparklyR Setup HOT 1
- FileNotFoundError while ssh HOT 1
- Contain the pools within a VNET completely. HOT 2
- Adding user not working HOT 1
- Azure Dalalake Gen2 Support
- issue while creating cluster HOT 1
- User login prompts for password I never provided HOT 1
- setup_host.sh has packages with unstable transitive dependencies HOT 1
- aztk cluster spinup failure HOT 2
- start task failed HOT 4
- Error when installing on Azure CLI HOT 5
- Error when installing in new virtualenv
- Error when setting up account HOT 1
- Getting started scripts link in step 4, 404s
- Batch account Nodes in unusable state for clusters in VNET
- Master and Worker Node on different Python Versions HOT 1
- Support for higher version of spark
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aztk.