GithubHelp home page GithubHelp logo

chenjianjx / srb4j Goto Github PK

View Code? Open in Web Editor NEW
47.0 16.0 23.0 4.9 MB

A Java RESTFul backend framework, with user/password/access-token support.

License: Apache License 2.0

Shell 0.02% Java 96.00% FreeMarker 0.20% CSS 2.36% JavaScript 0.41% TSQL 1.01%

srb4j's Introduction

Srb4j = Simple RESTFul Backend for Java

Srb4j (pronounced "/srəb/ for J") is a Java RESTFul backend code skeleton, with common response data structures, user/password/access-token support, social login and API document generation.

It can collaborate with html clients, mobile clients and other types of clients such as desktop applications at the same time.

With Srb4j you can launch a restful backend in several minutes.

Table of Contents

Summary of Features

  1. Registration/login based on standard OAuth2 password flow (access tokens, refresh tokens, etc.)
  2. Social account login support (Google,Facebook...)
  3. Forget password flow and random code login
  4. Swagger-based API document generation and client stub generation
  5. Built as an uber jar, instead of a war file
  6. Managed SQL migration which can be done automatically during system startup, or run manually with Maven
  7. PaaS friendly (e.g. AWS Beanstalk)
  8. Robust J2EE Stack: JAX-RS + Spring + MyBatis + MySQL
  9. An out-of-box back office portal

Prerequisites

  1. JDK 8+
  2. Maven 3.1+
  3. MySQL Server 5.7+

Quick Start in Dev Env

Generate a Java project

cd /path/to/your/workspace

mvn -X org.apache.maven.plugins:maven-archetype-plugin:3.0.0:generate  \
-DarchetypeGroupId=com.github.chenjianjx -DarchetypeArtifactId=srb4j -DarchetypeVersion=3.0.0 \
-DgroupId=your.groupId  \
-DartifactId=yourArtifactId \
-Dpackage=your.pkg.name \
-Dversion=1.0-SNAPSHOT 

Setup database and configure db credentials

Create a db and a user

mysql> create database yourdb default character set utf8;	 ## Has to be utf8
mysql> create user 'your_user'@'localhost' identified by 'your_password';
mysql> grant all privileges on yourdb.* to 'your_user'@'localhost' with grant option;	

Update db credentials for the system

vi yourArtifactId/webapp/src/main/resources/config/app.override.dev.properties 
#You can hard code the credentials for now. A safer way will be told later. 

Create tables

cd yourArtifactId
mvn clean install -DskipTests
cd data-migration
mvn initialize flyway:migrate 

Build the Java project

cd /path/to/your/workspace/yourArtifactId

mvn clean install -DskipTests

java -jar webapp/target/yourArtifactId-webapp-1.0-SNAPSHOT-shaded.jar

Verify the installation

Open

Run the backend in QA/PROD/Other Environments

  • Decide your environment name, such as "qa".
  • Set the environment name as env prop. The key is "yourArtifactId_environment"
export yourArtifactId_environment="qa"

If you don't do this, the default environment is "dev"

  • Create "app.override.qa.properties" under "webapp/src/main/resources/config", and edit the env-specific properties according to "app.properties" ** They system will read "app.properties" first, and then "app.override.qa.properties" to override. So you don't have to create a duplicate entry in "app.override.qa.properties" if you are not going to override that property ** A value can be a hardcoded one, or the value of an environment variable. The suggested strategy is to put as many properties in the properties file as possible, and only define sensitive information as environment variables. ** By default, the db's username/password is got from env variables, as you can see on "app.properties":
dbUsername=${env:yourArtifactId_dbUsername}
dbPassword=${env:yourArtifactId_dbPassword}

So what you need to do is

export yourArtifactId_dbUsername="your_user"
export yourArtifactId_dbPassword="your_password"
  • Let SQL Migration be Done Automatically during System Startup

Go to "app.override.qa.properties" and add

dataMigrationOnStartup=true
  • Build the artifact
mvn clean install -DskipTests

And you will get a uber jar like "webapp/target/yourArtifactId-webapp-1.0-SNAPSHOT-shaded.jar".

  • Deploy the artifact Upload the built uber jar to your target machine and run
java -jar yourArtifactId-webapp-1.0-SNAPSHOT-shaded.jar

Quick Start for Client-Side Developers

Refer to the API doc

The API doc has been generated on your backend at http://your-backend/fo-rest-doc

Sample Code For HTML and Javascript Developers

//login
$.ajax({
	async: false,
	url: "http://localhost:8080/fo/rest/token/new/local",
	type: "POST",
	contentType: 'application/x-www-form-urlencoded',				
	data: "grant_type=password&[email protected]&password=abc123",
	success: function(data, statusText, xhr){					
		console.log(data.access_token); //You can save this token to cookie or LocalStorage
		console.log(data.refresh_token);
		console.log(data.expires_in);
		console.log(data.user_principal); // the full user name			
		 
		
	},
	error: function(xhr,statusText, e){
		console.log(xhr.status)										
		var response = $.parseJSON(xhr.responseText);
		console.log(response.error); // "the error code"
		console.log(response.error_description); // "the error description for developers"
		console.log(response.error_description_for_user); // "user-friendly error desc for users"
		// "the server side developer can use this id to do troubleshooting"
		console.log(response.exception_id); 
	}				
});

//call a business web service
$.ajax({
	async: false,
	url: "http://localhost:8080/fo/rest/bbs/posts/new",
	type: "POST",
	contentType: 'application/json',
	headers: {					 
		'Authorization': "Bearer " + accessToken
	},				  
	data: JSON.stringify({content:"my-first-post"}),
	success: function(data, statusText, xhr){					
		console.log(data); 
	},
	error: function(xhr,statusText, e){
		console.log(xhr.status);
		
		if(xhr.status == 400 || xhr.status == 401 || xhr.status == 403){// "token error"
		    // "See https://tools.ietf.org/html/rfc6750#page-7"						 
			var authHeader = xhr.getResponseHeader("WWW-Authenticate");
			console.log(authHeader); 
			//in this case, you can redirect the user to login 
		}
		else if (xhr.status == 460) { // "biz error"
			var response = $.parseJSON(xhr.responseText);
			console.log(response.error); // "the error code"
			console.log(response.error_description); // "the error description for developers"
			console.log(response.error_description_for_user); // "user-friendly error desc for users"
			// "the server side developer can use this id to do troubleshooting"
			console.log(response.exception_id); 
		}else{
			console.log(xhr.responseText);
		}
			
	}
});

//logout
$.ajax({
	async: false,
	url: "http://localhost:8080/fo/rest/token/delete",
	type: "POST",
	contentType: 'application/json',
	headers: {
		'Authorization': "Bearer " + accessToken
	}
});
			

Check out more code here .

Sample Code For Desktop and Mobile Developers

import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.JsonNode;
import com.mashape.unirest.http.Unirest;


// do login
HttpResponse<JsonNode> loginResponse = Unirest
		.post("http://localhost:8080/fo/rest/token/new/local")
		.header("Content-Type", "application/x-www-form-urlencoded")
		.field("grant_type", "password")
		.field("username", "[email protected]")
		.field("password", "abc123").asJson();

if (loginResponse.getStatus() == 200) {
	JSONObject token = loginResponse.getBody().getObject();
	System.out.println(token.get("access_token")); //You can save the token for later use
	System.out.println(token.get("refresh_token"));
	System.out.println(token.get("expires_in"));
	System.out.println(token.get("user_principal")); // "the full user name"			
} else {
	System.out.println(loginResponse.getStatus());
	System.out.println(loginResponse.getStatusText());
	JSONObject error = loginResponse.getBody().getObject();
	System.out.println(error.get("error")); // "the error code"
	// "the error description for developers"
	System.out.println(error.get("error_description")); 
	// "user-friendly error desc for users"
	System.out.println(error.get("error_description_for_user")); 
	// "the server side developer can use this id to do troubleshooting"
	System.out.println(error.get("exception_id")); 
}

// call a business web service
NewPostRequest bizRequest = new NewPostRequest();
bizRequest.setContent("my-first-post");
HttpResponse<String> bizResponse = Unirest
		.post("http://localhost:8080/fo/rest/bbs/posts/new")
		.header("Content-Type", "application/json")
		.header("Authorization", "Bearer " + accessToken)
		.body(toJson(bizRequest)).asString();

if (bizResponse.getStatus() == 200) {
	Post post = fromJson(bizResponse.getBody(), Post.class);
	System.out.println(post);
}

else if (Arrays.asList(400, 401, 403).contains(bizResponse.getStatus())) { // "token error"
	String authHeader = bizResponse.getHeaders()
			// "See https://tools.ietf.org/html/rfc6750#page-7"
			.get("WWW-Authenticate").get(0);
	System.out.println(bizResponse.getStatus());
	
	//You can also further parse auth header if needed. 
	//Search "decodeOAuthHeader" in this repository.
	System.out.println(authHeader);  
	
	//You should then redirect the user to login UI
}

else if (bizResponse.getStatus() == 460) { // "biz error"
	JSONObject error = new JSONObject(bizResponse.getBody());
	System.out.println(error.get("error")); // "the error code"
	System.out.println(error.get("error_description")); // "the error description for developers"
	// "user-friendly error desc for users"
	System.out.println(error.get("error_description_for_user")); 
	// "the server side developer can use this id to do troubleshooting"
	System.out.println(error.get("exception_id")); 
} else {
	System.out.println(bizResponse.getStatus());
	System.out.println(bizResponse.getBody());
}


// logout
Unirest.post("http://localhost:8080/fo/rest/bbs/posts/delete")
		.header("Content-Type", "application/json")
		.header("Authorization", "Bearer " + accessToken).asJson();

Check out more code here or here .

DDL/DML migration

Let the system take care of SQL migraiton for you. Srb4j uses flyway to do this. Flyway won't re-run SQLs that have been run before. So don't worry.

  • Go to "data-migration/src/main/resources/data-migration-flyway-sql", and add a new SQL file
  • In "dev" environment, "dataMigrationOnStartup" is by default false. So you should run the sql manually.
cd data-migration
mvn clean package flyway:migrate
  • In other environments, if "dataMigrationOnStartup" is set as true, the system will automatically run the SQL when the system starts up.

PaaS Cloud Integration

Here we use AWS Beanstalk as an example.

  • Make sure "dataMigrationOnStartup=true" in your "app.override.xxx.properties", unless you prefer to run the sql manually.
  • Configure environment variables in Beanstalk's web console, such as environment name, RDS db username/password and smtp credentials. To make your life easier, you can set the following db credentials in your "app.override.xxx.properties"
dbHost=${env:RDS_HOSTNAME}
dbPort=${env:RDS_PORT}
dbSchema=${env:RDS_DB_NAME}
dbUsername=${env:RDS_USERNAME}
dbPassword=${env:RDS_PASSWORD}
  • (If you want) On the load balancer configuration page, you can add "https://your-backend/health" as the health check endpoint (The handler is HealthCheckServlet.java)
  • Build your artifact, upload to Beanstalk and trigger a deployment.

Development of Backend

User Model

  1. Every user has a "source" property to indicate where this user is from. "local" means the user registers here, "facebook" means the user is created when he logged into the backend with a facebook account.
  2. source + email make up of a username, the business key.

The code organization

  • The layers:

layering

  • Notes:

    • Front End: Encapsulate use case-specific logic from business services. The users are common users such customers.
    • Back Office: Encapsulate use case-specific logic from business services. The users are administrators, staff and so on.
  • And you get these maven projects:

code org

  • Notes
    • "webapp" has "runtime" dependency on "impl"
    • "intf.bo" depends on "intf.fo" since back office users also need common-user perspectives.
    • Check full explanation here

Social Login Integration

Basic Flow

  1. The client obtains an auth code or an access token/id token from the social website after the user has loggged into a social website
  2. The client then exchanges this code or token with the backend for srb4j's access token
  3. The backend will verify the code or token against the social website's server, before it sends an access token to the client

Integrate with this or that social site

API Documentation and Client Stub Generation and Online Testing

Thanks to swagger, you can have a WSDL-like API document of your restful web services with swagger-ui, generate client stubs with swagger-codegen and test the services with a web-ui.

Srb4j has embedded swagger support. The document endpoint is http://your-backend/fo/rest/swagger.json . If you know swagger well, you know what to do.

If you just want to see a download-able API doc, check http://your-backend/fo-rest-doc , which is generated by swagger2html.

The Back Office Portal

You can log into the back office to manage some data such as a list of users.

  • To enable the protal, go to "app.override.xxx.properties" and set
enableBackOfficePortal=true
  • To let someone login, you must generate a StaffUser record for him or her:
cd data-migration
mvn clean package exec:java -Dexec.mainClass="your.pkg.name.datagen.StaffUserPasswordGenerator"

With the username/password they can then go to http://your-backend/bo/portal/login . They'll have to change the password after first logging in, so the password generator won't be able to log in with the original password.

  • Development
  • The portal is based on Jersey MVC + JSP + Twitter Bootstrap
  • Users of the portal are called "StaffUser", which has nothing to do with front-end "User"s. Different entity class, different table.
  • No Role/Permission infrastructure. You must do it yourself if necessary.

srb4j's People

Contributors

chenjianjx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

srb4j's Issues

Run the ddl with flyway

I should be able to do sql migration with a single maven command

  1. during development of my artifact, I can run it manually
  2. during deployment of my artifact, it is possible to run the migration as part of system start up -- will be done in #3

Deploy the system as a single executable jar

  1. Give up war packaging. Let it always be a single executable uber-jar
  2. Developers should be able to run the program from IDE as a Main application
  3. Put the configuration inside the jar.
  4. During development, the flyway can only be run manually; but in testing/production envs, the flyway should be run automatically when the system starts up.

email verification - verification flow

Email verification status

As a user,

  1. If I register with email/password, by default my email is not verified
  2. If I register with social account, my email is verified

Email verification

As a user, I can verify my email if I registered with email/password.

Typical flow:

I can ask for a verification link to be sent to my email. After clicking it, I will be directed to a web page which tells me that the verification has been done.

Alternative flows:

  1. If I registered with social account, request for a verification link will fail
  2. If my email has been verified, request for a verification link will fail
  3. If my email has been verified, clicking the verification link will "appear" successful
  4. If my email has not been verified, and I have asked a verification link before, when I ask for this link again, the former link will be invalid.

401, 403 and WWW-Authenticate header in oauth2 responses

Missing token/ token invalid / token expired: should use 401 + WWW-Authenticate header , not 400 . And in this case the frontend should do a login.

Insufficient scope: should use 403 + WWW-Authenticate header. In this case the frontend should not do a login

A good discussion can be found here: bshaffer/oauth2-server-php#143

Things that should be changed

  • Backend code that write response
  • Documentation about frontend code in readme.md

email verification - apply the verification status to login flow and other flows

As a developer,

I can choose between two schemes concerning verification status

  1. VERIFICATION_REQUIRED: Without email verified, a user can't even login. They may not be able to something else.
  2. VERIFICATION_NOT_REQUIRED: Without email verification, the user can still login

To be more specific,

Feature scheme = VERIFICATION_REQUIRED scheme = VERIFICATION_NOT_REQUIRED More
Register with email/password An verification email will be sent. The user will NOT be automatically logged in An verification email will be sent. The user will be automatically logged in
Login with email/password An verification email will be sent. Login unsuccessful Login successful
Register/Login with social account Login successful Login successful In this case the email verification status is alway "verified"
Request for random login code An verification email will be sent. "Please verify your email first" Allowed
Login using random login code N/A - The user won't have any Login successful

Make a neat BO site

User Stories

As a staff user

  • I will know the username/password from offline
  • I can log into the BO site with the combination. But for the first time of log in, I have to change my password
  • The BO site is a responsive html5 website, so that I can use it from a mobile device
  • I can then view all the FO users in list, with pagination
  • I can also view other records if the developer provide them
  • I can change my password

As a DevOps

  • I can use a maven command to generate a staff user's password and its encrypted version that is going to be stored in DB
  • I can then create a staff user with this password by manually running some sql

As a developer

  • By default the back office website is disabled.
  • I can enable it by changing some configuration
  • If I want to completely remove the website, it should be done easily, which means all the code shouldn't scatter

Design and Implementation

Features

  • Log In
  • Log out
  • Change password
  • See the list of FO users (password hidden)
  • Disable/Enable BO site

The web page framework

  • Jersey + some template solution which can do template inclusion
  • Jsersey Filter to be used as session filter
  • Html5 boilerplate
  • No Javascript allowed
  • All the nav links are in a common file, where CSS is inlined in this page file
  • All the BO jersey resource endpoints are declared in a single Java class
  • All the template files are put together in a single dir

Password generation

  • A main class can be used for this
  • Then a mvn exec plugin can be used to hook it with maven

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.