ethanmoffat / etheos

[Eth]an's [E]ndless [O]nline [S]erver - fork of eoserv project

ETHEOS

Table of Contents

• Getting Started on Windows
• Getting Started on Linux
• Docker Image
• Running
• Development
• Integration Tests
• Sample servers

Getting Started on Windows

Visual Studio 2017 or 2019 is required for the compiler toolchain in order to build on Windows. You will need to select the "Desktop Development with C++" workload when installing. Windows 10 SDK is required for the ODBC (SQL server) driver libraries (can be installed as part of Visual Studio).

⚠️ If you have previously attempted to build eoserv using MinGW on Windows, please uninstall MinGW as it conflicts with the standard libraries provided by Visual Studio.

Getting the source

⚠️ You must use git to clone the repository. Downloading the zip and trying to build the source from there is not supported due to the fact that there is an ICO file that is stored in git-lfs, which is an invalid format when downloaded via the zip file.

You may download git for Windows from: https://git-scm.com/downloads. Most users will want to use the installation defaults. If you don't know what a setting means during installation, do not change it.

The following commands must be run on a new system

1. git lfs install

   a. This command only needs to be run once per machine.

2. git clone https://github.com/ethanmoffat/etheos.git

3. cd etheos

The build and dependency install scripts may now be run from the etheos directory.

Dependencies

The dependencies for building ETHEOS on Windows are:

• CMake (>= 2.8.2)
• SQLite
• MariaDB
• vswhere
• git (for getting bcrypt/googletest components)

Automatic Dependency Installation

A convenience script has been provided which installs Chocolatey (package manager) and each of the required dependencies. To automatically install the dependencies, run .\scripts\install-deps.ps1 as administrator or from an elevated powershell terminal. Examine the script contents to view the options for running this script.

Build and Install

Run .\build-windows.ps1 in a new powershell terminal to build the source with support for all available database engines (default: SQL server) and install the project into a local directory (default: install) under the repository root. To compile with support for all database engines, run .\build-windows.ps1 -SqlServer ON -MariaDB ON -Sqlite ON. -Debug can be added to the command-line arguments to build in debug mode, otherwise, release mode is used.

Getting Started on Linux

This process has been tested on Ubuntu 18.04 and 20.04 (including WSL/Windows Subsystem for Linux).

Dependencies

The dependencies for building ETHEOS on Linux are:

• g++
• CMake (>= 2.6)
• SQLite
• MariaDB
• git (for getting bcrypt/googletest components)
• ODBC (SQL server) [optional]

Automatic Dependency Installation

A convenience script has been provided which installs each of the required dependencies. To automatically install the dependencies, run sudo ./scripts/install-deps.sh. Examine the script contents to view the options for running this script.

Build and Install

Run ./build-linux.sh -i to build the source with support for all available database engines (default: SQL server) and install the project into a local directory (default: install) under the repository root.

Docker Image

A docker image is available on docker hub at darthchungis/etheos. Environment variables in the form of ETHEOS_### (all uppercase, case-sensitive) may be used to set configuration options. These options will override what is set by default in the configuration files.

For example:

docker pull darthchungis/etheos
docker run -it --rm -e 'ETHEOS_PORT=8079' -p 8079:8079 -v .\install\data:/etheos/data -v .\install\config_local:/etheos/config_local darthchungis/etheos:latest

Running

⚠️ Configuration and data files need to be placed before executing.

From the local install directory, run the ETHEOS executable (.\etheos.exe on Windows or ./etheos on Linux).

Development

Development within Visual Studio Code (vscode) is supported. The following vscode extensions are recommended:

• C/C++
• CMake

Integration Tests

Integration tests have been added under src/test/integration. These are integration test scripts that are run via EOBot.

These tests are run as part of the release pipeline that deploys the sample server to etheos.moffat.io:8078 (see below section). However, they may also be run locally to validate integration scenarios or add more integration test coverage.

Running these integration tests is only supported on Linux. Ubuntu 22.04 is the suggested platform. WSL 2 is supported.

First, download/install the following dependencies (installation instructions are left as an exercise for the user):

• python3
• docker cli
  • Please ensure you add your local user to the docker group via sudo usermod -aG docker <username> to avoid running the ci-test script with sudo
• EOBot from the latest EndlessClient release
  • You may also clone EndlessClient, build the EndlessClient.Linux.sln file, and reference the EOBot output directory

Then, run the script as follows (where path/to/eobot is the path to the directory container the EOBot executable file):

cd deploy
./ci-test.sh --self-contained --use-local --botdir path/to/eobot

This script will use a self-contained docker environment based on your local repository, build the source + docker image locally (omit --use-local to use the latest public etheos image), and set up an etheos container with a SQLite database backend on port 8078. Integration tests defined in the src/test/integration directory and listed in the exec_tests() function in ci-test.sh will be executed against the local docker container.

For more information on authoring test scripts, see the output of EOBot --help.

Sample Servers

Sample servers are hosted in different environments. These servers use SQL Server as a database backend. Servers use default assets from EO v28.

1. moffat.io:8078

   • ⚠️ This server is not guaranteed to be running the latest version

2. etheos.moffat.io:8078

   • This server is deployed on completion of successful CI builds

etheos's Issues

Add EditorConfig

Add an EditorConfig (.editorconfig) file to explicitly state expected file formatting parameters and to improve rendering of existing files.

Add Azure CI pipelines

Add Windows and Linux Azure CI pipelines.

Release Plan

I thought it might be worthwhile to start a discussion about a tentative release plan. Here are some points I think might be worth discussing:

1. The last release (0.7.0) was created over 3 years ago (April 2016). Do we have a list of changes that occurred in this time period so that we can provide a release note with the next release? If not, then I can start compiling this list.
2. Do we have a list of current features and bugs? I think the easiest way to allow for tracking of bugs and features is through the use of GitHub issues. The use of GitHub issues would greatly simplify the generation of release notes for future releases. I'm wondering if anyone else has an opinion on this.
3. While committing directly to the repository provides a quick-to-update workflow, it makes code review/oversight and tracking difficult. The use of branches and pull requests can be helpful for associating a collection of changes with the bug or feature they were intended to address. Again, I'm wondering if anyone else has any opinion on this.
4. Lastly, do we have any idea of when our next release will be and if it will be a major or minor version? If not, maybe we should have a discussion about our roadmap to the next release so that the community has something to look forward to and so that we have something to work towards.

Build system unification

Right now we have disparate scripts for building on Linux and Windows. It would be nice to unify these to a single script so we only have one thing to maintain.

Ideas for this (in order of preference):

• Powershell core
• Bash on Windows somehow (Cygwin?)
• Cake build with similar ps1/sh bootstrap scripts

PUB files

I can't seem to find pub files in order to load the server. Any ideas?

SQLite support is broken

Background

• EOSERV does all of its database operations in one big transaction on the main thread.
• SQLite only supports one simultaneous write transaction.

Problem

• Since the addition of asynchronous database operations on background threads for the LoginManager, SQLite support has been broken.

Reproduction case

• Configure EOSERV to use an SQLite database
• Create an account
• Account creation will fail, and a Database_QueryFailed exception will be raised.
• The error message from SQLite is "database is locked"

See

• World::BeginDB
• SQLite: Transaction

Board MaxUserRecentPosts is not defined in the default config files

Boards use the client default of '2', with 1 minute between posts, because the config file has a string mismatch with the value read in code.

Character::ShowBoard line 1421

Board state is not saved in the world dump when the server shuts down

Board state needs to be added to the crash dump JSON file.

Welcome/Players Agree and Refresh Reply use constant AddChar(6) when it should be the player's class ID

See map.cpp line 761 - builder.AddChar(6); // ?. This unknown value is actually supposed to be class ID.

ethanmoffat/EndlessClient#156 (comment)

Update IgnorePacketFamilies to support more advanced filters

It would be nice if the IgnorePacketFamilies feature supported more advanced filtering options. Currently, you specify packet families to exclude from logging. It would be helpful to have negative filters, like !Login, * would only show Login family of packets.

MapInfo requests are not implemented

See: ethanmoffat/EndlessClient#154

Need to implement the Autorefresh packets and rename Appear/Autorefresh to their appropriate "MapInfo" equivalents.

Mirror with history

@ethanmoffat here is what I’ve run to create and maintain a mirror of the eoserv source with a full commit history:

#!/usr/bin/env bash

function main()
{
  local svn_url="svn://svn.eoserv.net/eoserv"
  grep -r svn-remote .git/config &> /dev/null \
    || git svn init -s "${svn_url}"

  git svn fetch \
    && git svn rebase
}

main "$@"