- A MongoDB database or API compatible (Azure Cosmos DB)
- Azure service bus
- Kubernetes
git clone https://github.com/webhintio/online-service.git
cd online-service
npm install
npm run buildYou need to configure at least the following env variables in order to run the service:
- database: The connection string for the database
- queue: The connection string for Azure service bus
- port: The port where the job manager will be listening
Run the Job manager
npm run online-service -- --microservice job-managerRun the Config manager
npm run online-service -- --microservice config-manager --name new-config-name --file path/to/you/config-file.json --cache 120 --run 120NOTE: use npm run online-service -- --help to get more information about
what each argument means
Run the Worker service
npm run online-service -- --microservice workerRun the Sync service
npm run online-service -- --microservice syncRun everything at the same time (except the config manager)
npm run online-service -- --microservice allNOTE: You need to set up the following environment
variables before continue: NODE_ENV=production,
database=YourConnectionStringToTheDatabase and
queue=YourConnectionStringToServiceBus
If you want to run the online-service in you local machine,
you just need to run:
kubectl apply -f kubernetes.ymlNOTE:
Kubernetes needs to be installed in your computer.
We are assuming that you are in the folder compose before running
kubectl.
If you are in another folder, replace the file with your path to
the file kubernetes.yml.
Remember you need to replace the enviroment variables values in
compose/kubernetes.yml with your own values before run kubectl.
First of all we need to deploy kubernetes services in Azure.
To do so, first you need to go to the folder deploy inside
the folder scripts.
cd scripts/deployMake the file deploy.sh executable.
chmod +x deploy.shRun deploy.sh. If you don't want to pass the parameters,
the script will ask you later for the information needed.
./deploy.sh -i <subscriptionName> -g <resourceGroupName> -l <resourceGroupLocation> -k <sshPublicKey> -r <containerRegistryName>"The deployment script will create all the necessary to run the online scanner:
- An Azure kubernetes service.
- An Azure Service bus.
- A Virtual machine with linux to install nginx.
- If needed, a replicated mongodb.
- The peering between the linux VM and the database (the new one or an old one).
NOTE:
sshPublicKey is the path to the file containing the public ssh key.
To build all the images at the same time you need
to run the script build-images.js.
To do so, first you need to go to the folder scripts.
cd scriptsRun build-images.js with the name of your repository and the
image version as parameters.
node build-images.js --repository webhint --version 1To upload all the images at the same time you need
to run the script update-images.js.
To do so, first you need to go to the folder scripts.
cd scriptsRun upload-images.js with the name of your repository and the
image version as parameters.
node upload-images.js --repository webhint --version 1NOTE:
You need to be logged in into you Container Registry
az acr login --name ContainerRegistryName before upload the images.
Before deploy, the configuration file needs to point to the right
repository and to the current version of the images. You can do that
manually or you can run the script update-config-file.js.
To do so, first you need to go to the folder scripts.
cd scriptsRun update-config-file.js with the name of your repository, the
image version and the path to the configuration file as parameters.
node update-config-file.js --repository webhint --version 1 --kubernetes ../compose/kubernetes-azure.ymlNOTE:
The file path is optional, by default the value is
../compose/kubernetes-azure.yml
You also need the config from your cluster
az aks get-credentials --resource-group ResourceGroupName --name ClusterName
To deploy the online scanner in kubernetes you need to run the
script deploy-kubernetes.js
To do so, first you need to go to the folder scripts.
cd scriptsRun deploy-kubernetes.js with the path to the configuration file as
a parameter.
node deploy-kubernetes.js --kubernetes ../compose/kubernetes-azure.ymlNOTE:
The file path is optional, by default the value is ../compose/kubernetes-azure.yml
You can run all the previous steps with just one script using build-and-deploy.js
To do so, first you need to go to the folder scripts.
cd scriptsRun build-and-deploy.js with the name of your repository and
the path to the configuration file as parameters.
node build-and-deploy.js --repository webhint --kubernetes ../compose/kubernetes-azure.ymlNOTE:
The version for the images will be auto calculated using your current images you have in your computer.
To deploy NGINX, you will find a few files in scripts/deploy/nginx:
install-nginx.shconfigure-nginx.shnginx-step1.confnginx-step2.conf
Before start deploying NGINX, you need to copy these files to your NGINX machine:
Copy files into the machine where we want to deploy:
-
Copy files:
scp *.sh *.conf [email protected]:~
-
Go into the machine:
ssh -p your.port [email protected]
-
Install NGINX and Certbot:
sudo ./install-nginx.sh
NOTE:
If you get an error GPG error, look for the instructions
in install-nginx.sh.
-
Configure NGINX
sudo ./configure-nginx.sh -s <serverName> -j <jobsIpAndPort> -c <configIpAndPort>
NOTE:
jobsIPAndPort is the IP and port where the job-manager is deployed.
configIpAndPort is the IP and port where the config-manager is deployed.
After everything is deployed, the next step is to configure the scanner.
-
Authorize an user to have access to the config panel
Access the database the way you prefer and then create a
usercollection if it doesn't exist already and create an item with the propertynameset to the user (github user) you want to give access, manually.{ "name": "githubusername" } -
Create the OAuth App in github.
Go to github => Settings => Developer settings
The
Authorization callback URLwill be something like:https://yourip/admin/auth/github/callback -
Add a configuration
Once you access to the configuration panel (https://youip/admin) the next step is to add the configuration we want to use in the scanner.
The scanner configuration is an array of
hintconfiguration.To add the scanner configuration go to the admin panel => Configuration.
This project adheres to the JS Foundation's code of conduct.
By participating in this project you agree to abide by its terms.
The code is available under the Apache 2.0 license.
![azure-pipelines[bot] avatar](https://avatars.githubusercontent.com/in/9426?v=4 "azure-pipelines[bot]")
![dependabot-preview[bot] avatar](https://avatars.githubusercontent.com/in/2141?v=4 "dependabot-preview[bot]")
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
![greenkeeper[bot] avatar](https://avatars.githubusercontent.com/in/505?v=4 "greenkeeper[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent