GithubHelp home page GithubHelp logo

adessose / jekyll2cms Goto Github PK

View Code? Open in Web Editor NEW
3.0 4.0 3.0 543 KB

Tool for exporting github repo to a CMS

Home Page: https://www.adesso.de

License: MIT License

Java 97.52% Dockerfile 2.48%
jekyll ruby java spring conversion cms docker adesso

jekyll2cms's Introduction

jekyll2cms converts markdown files into xml in order to export Jekyll blogs to content management systems. It is delivered as a docker image, which you can easily use in your blog.

Usage

jekyll2cms requires a Jekyll repository on GitHub. This repository will be used to convert its markdown files into xml files using the xml-template files in this repository. Repositories must adhere to structural requirements. Make sure to check out the adesso SE devblog as an example.

  • The markdown files to be converted must be in the _posts folder.
  • The generated xml files will be located in assets/first-spirit-xml.
  • The format of the generated xml files will be determined by _layouts/post-xml.xml .

How to execute

Running jekyll2cms requires some environment variables that must be passed to the docker run command or be defined as GitHub Secrets:

Name Description
REPOSITORY_LOCAL_USER_NAME The GitHub username of the user who will commit the generated xml files to the repository. This user needs to have at least write access to the repository. If the branches are protected, this user needs admin access.
REPOSITORY_LOCAL_USER_PASSWORD The password of this user. It is needed to provide the credentials to allow the commit.
REPOSITORY_LOCAL_USER_MAIL The email of this user. This is needed to provide further commit information.
REPOSITORY_REMOTE_URL The URL of the repository which contains the markdown files. This is also the location where the generated xml files are pushed to.

Executing localy

Run jekyll2cms on your machine using the following command:

docker run -e REPOSITORY_REMOTE_URL=<repository_url> -e REPOSITORY_LOCAL_USER_NAME=<github_username> -e REPOSITORY_LOCAL_USER_MAIL=<github_email> -e REPOSITORY_LOCAL_USER_PASSWORD=<github_password> jekyll2cms/jekyll2cms:<tag>

Automatic execution on GitHub

It is also possible to use jekyll2cms as a GitHub Action - Create a workflow and copy the following steps. We recommend using GitHub Secrets to provide the configuration.

 - name: Pull Docker image
   run: docker pull <your_docker_hub_user>/<your_docker_hub_image>:<tag>

 - name: Run Docker image
   run: docker run -e REPOSITORY_REMOTE_URL='${{ secrets.REPOSITORY_REMOTE_URL }}' -e REPOSITORY_LOCAL_USER_NAME='${{ secrets.REPOSITORY_LOCAL_USER_NAME }}' -e REPOSITORY_LOCAL_USER_MAIL='${{ secrets.REPOSITORY_LOCAL_USER_MAIL }}' -e REPOSITORY_LOCAL_USER_PASSWORD='${{ secrets.REPOSITORY_LOCAL_USER_PASSWORD }}' <your_docker_hub_user>/<your_docker_hub_image>:<tag>

Program sequence

jekyll2cms is a Java Spring Boot application. The execution is separated into four steps.

  • Check configuration
  • Clone repository
  • Execute jekyll build and copy generated xml files to specified folder
  • Commit and push generated files to remote repository

jekyll2cms will first check if the configuration and the associated environment variables are passed correctly. The target repository will then be cloned. jekyll build is executed and generates the xml files in the _site directory. The generated xml files are moved from _site to assets/first-spirit-xml. Images used in the markdown files are copied to assets/images.

jekyll2cms then checks if any files have changed. This is done using git status in JGit. Any changes are committed and pushed to the remote repository using a commit message that specifies the changes. The process is finished without commits if no changes were detected. jekyll2cms then exits.

Program exit

jekyll2cms will exit successfuly if

  • the generated xml files were pushed successfuly
  • or if a commit doesn't containt new changes.

jekyll2cms will exit with code 0 in both cases.

Error codes

jekyll2cms can fail during execution. Reasons might include misconfiguration or errors in Jekyll. We have defined error codes to make debugging easier.

Exit code Description
01 An error ocurred that is not handled by jekyll2cms.
10 Environment variable REPOSITORY_REMOTE_URL not found.
11 Environment variable REPOSITORY_LOCAL_USER_NAME not found.
12 Environment variable REPOSITORY_LOCAL_USER_MAIL not found.
13 Environment variable REPOSITORY_LOCAL_USER_PASSWORD not found.
20 Error while cloning remote repository.
30 Execution of jekyll build returned a non zero exit code.
31 Couldn't generate jekyll build. Execution directory not found.
32 Error calling generated jekyll build command. This is not an error with jekyll.
33 Error assembling files to copy.
34 Error copying genereated files.
35 Error copying images.
40 Error pushing files to remote repository.

Developing

Feel free to fork and adapt jekyll2cms to your needs. We've prepared a short guide to help you get started.

Prerequisites

Make sure that the following components are installed correctly on your system:

Running jekyll2cms on your local machine

We highly recommend to always execute jekyll2cms using docker. This prevents errors due to missing environment variables or different path separators. However you can also use the jar to execute it.

Run with Docker

To run your local changes using docker follow these steps:

  1. Rebuild your app with gradlew build
  2. Run docker build -t jekyll2cms to build a new image
  3. Use the docker run command as specified in Executing localy

Releases

On every commit on the master branch a new release for jekyll2cms is built using GitHub actions. The version number is defined in the docker-build-and-push.yml workflow file. If it is not changed the current image on docker hub is overwritten.

Questions?

We do not provide any official support for this software. You can however create an issue in this repository if you have any questions, feedback or technical difficulties. You can also reach us at devblog[replace with the at-sign]adesso.de.

jekyll2cms's People

Contributors

acetous avatar calle-lo avatar ceverke avatar daklassen avatar dohack-bot avatar florianluediger avatar jo2 avatar s-gbz avatar shsanayei avatar silasmahler avatar thombergs avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

jekyll2cms's Issues

Add XML conversion.

  • According to the given XML-Template, convert all correspondig data of a blog post (metadata from markdown files, jekyll html output) to a XML file.

Add `bundle install` to getting started docs

When starting jekyll2cms, I got an error that some ruby modules were not installed. A bundle install fixed that. Add this to the prerequisite part of the getting started documentation.

Remove Travis

We have a travis ci implemented which is not used anymore. We should remove it.

Prevent build fail in forked repos

In forks of adessoAG/devblog the actions are also triggered. We should stop the triggering or at least make it pass without errors for forks, because we're not interested in these forks and their run results. Right now, we're getting the email notifications though.

Establish a connection to a database

In the final result we want to save all posts into a extern database. For the moment its good enough to save posts into a H2 database.

Task: Establish a connection to a database create the necessary methods and classes to save posts into this database.

Split `RepoService` in multiple classes

RepoService should be split up in multiple classes that are better testable. For example:

  • GitRepoCloner
  • GitRepoPuller
  • GitRepoDiffer (checks for changes since the last version)
  • MarkdownTransformer (runs Jekyll to transform Markdown to XML, reads XML files from the source folder and copies them to the target folder)
  • GitRepoPusher
    *...

each of those classes can then be rather easily covered with unit tests

Find a way to detect new posts or changes

The application should only process new files, by adding them to the extern database.

Task: Find a way to detect new posts or changes and to only process these changes.

How should we output information to the console?

My solution propsal:

  • Deactivate spring logger on INFO level
  • Use spring logger for technical error
  • Print any info on System.out
  • Use System.err for "logical" errors (e.g. "repo already cloned")

Date validation: How should dates be saved and validated?

Questions:

  • How should dates of new posts and modified posts be saved and validated ?
  • What should happen when dates and their formats are invalid?
  • What is the best type for saving a date withing Java application and database?

=> implement the solution!

Extract meta information from post

Task: Create a controller that provides necessary method for extracting meta information from blog post.

Given: A markdown file with following header:

---
id: 42
layout: article
title: Github's Fork & Pull Workflow for Git Beginners
categories: [hacks]
modified: 2017-01-02
author: tom
tags: [git, github, pull request, branch, fork, howto, guide, pull]
comments: true
ads: false
image:
  feature: 
  teaser: teaser/github-mark.png
  thumb:
---

Wanted: A controller class that extracts information from markdown file and stores it into a hashmap.

Create some documentation

It would be nice to have some documentation about this project in the github wiki of this repository.

Task: Create some useful documentation.

Create javadoc

Add javadoc for all public fields and methods and also for private ones, if necessary.

RepoService shouldn't be stateful

Currently, the RepoService has a field oldHead which stores the old Head of the local repository. A Service shouldn't hold state like this. If the application restarts, the state is lost, what happens then?

Instead, the application should store the commit id or commit date of the latest commit in a file and push that file into the repository. Then, when the repo is updated, it could access the old state by reading this file.

Create a concept for useful CLI commands

Goal: In production we want to be able to do all necessary task only with the help of commands (with initial configuration of paths).

Task: Create a rough concept on which commands are necessary to achive the described goal.

XML Optimization

XML-Generation needs following optimizations:

  • Clarify what needs to be inside 'contents'-field
  • do not escape < sign
  • put empty string for null values

Exit Code should be "1" on error instead of "0"

The exit code in the following code snippet should be "1" and not "0", since "0" means everything is good, which it isn't :).

@Override
public void run(ApplicationArguments args) throws Exception {
	if (!initService.init()) {
		System.exit(0);
	}
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.