open-duelyst / duelyst Goto Github PK
View Code? Open in Web Editor NEWDuelyst is a digital collectible card game and turn-based strategy hybrid, developed by Counterplay Games.
License: Creative Commons Zero v1.0 Universal
Duelyst is a digital collectible card game and turn-based strategy hybrid, developed by Counterplay Games.
License: Creative Commons Zero v1.0 Universal
There is a minor vulnerability in our version of moment
: https://security.snyk.io/vuln/SNYK-JS-MOMENT-2440688)
Upgrading to v2.29.2 will resolve this, but we have several date strings which don't satisfy RFC2822 or ISO formats (which raise errors in the new version).
We should update these strings and then upgrade moment
.
See http://momentjs.com/guides/#/warnings/js-date/ for more info.
We currently have a terraform/iamPolicy.json
file with an example IAM policy which can be used to run Terraform. This is out of date now that I've added a ton of new resources, so I need to update this once we have a final set of permissions required to manage a deployment.
Spot Instances are 70% cheaper than on-demand instances, and they're perfect for any workload which can be shut down within 2 minutes of a warning signal. This includes API and Worker, which are largely stateless (but not the Game/SP WebSocket servers, since those should remain available for the full duration of games).
Adding Spot Instance support in ECS and using it for API+Worker will allow us to get more CPU and Memory for user traffic while still saving money in both staging and production deployments.
We need to configure various pipelines:
We currently have yarn cdn:upload:staging
, which uploads all of dist/src
including 450MB+ of resources to S3.
We should add an alternative script which uploads only the rebuilt files (~13MB):
We may also want to automate CloudFront cache invalidations (first 1,000 validations per month are free).
AWS is predicting 62,000 GET requests to S3 in October, which is only $0.20 or so, but this is much higher than it should be. Let's enable the CloudFront cache policy for resources with a minimum TTL of one day.
Also, see if we can better target tests based on the contents of a PR (e.g. only linting TF code in TF PRs).
I don't have a significant amount of frontend expertise, but I know that Webpack is one of the more popular module bundlers for frontend assets.
Would this be beneficial for the project? We should investigate this and see if it's worth switching over.
There are likely some improvements we can make to how we cache node_modules
and Yarn data in Docker. We should determine the current best practices for these and make sure we're following them in the repo.
This might save quite a bit of time in local development iteration.
We currently have around 1.2GB of images, sounds, and fonts in the app/resources
and app/original_resources
directories, which results in slower git operations and longer workflow durations in CI.
We should find a way to isolate these binary files from the core repo to keep things lean and fast. One solution here could be simply moving the resources to another repo (e.g. open-duelyst/resources
), configuring .zip
and .tar.gz
releases for that repo, and then having the core repo pull those in at build time.
Duelyst uses the Firebase Realtime Database product for communication between the game client and game backends.
We are currently using version 2.0.3 of the firebase
package from NPM, which was released in early 2015. We should upgrade to the latest version here: https://www.npmjs.com/package/firebase
Much has changed in 7 years: following Google's acquisition of Firebase in 2014, the service has been incorporated into its Google Cloud offerings. The largest change is the deprecation of Firebase "legacy tokens" in favor of Google Cloud "service accounts". Duelyst currently relies on legacy tokens in the game client, but legacy tokens may no longer be created in new Firebase projects.
One complication for this work is the usage of Browserify in the Gulp build process, so any client changes need to maintain compatibility with this flow.
The Firebase dependency has already been updated for the game backend, which is now using service accounts.
ECS deployments currently have a few rough spots:
We should iron these out to make it easier for us to deploy the staging environment (or in the future, the production environment).
Describe the bug
Migration code is unable to resolve relative paths to app/
:
duelyst-migrate-1 | Error: Cannot find module 'app/sdk/quests/questBeginnerWinPracticeGames'
duelyst-migrate-1 | Require stack:
duelyst-migrate-1 | - /app/app/sdk/progression/newPlayerProgressionHelper.coffee
duelyst-migrate-1 | - /app/server/migrations/20160314130406_migrate_ftue_data_for_all_users.js
To Reproduce
Run docker compose up migrate
As of #47, we are running integration tests in Github Actions. However, we are only running a subset of the available tests. We should repair the following tests and enable them in .github/workflows/integration_tests.yaml
:
yarn test:integration:data_access
: All of these tests are failing due to misuse of Knex in the Users module:
14) users module
"before all" hook in "users module":
TypeError: knex(...).where(...).first(...).bind is not a function
at Function.UsersModule.createNewUser (server/lib/data_access/users.coffee:138:4)
yarn test:integration:firebase
: In order to run Firebase tests, we need to create a Firebase Realtime Database which can be used by Github Actions.
There are also tests which are commented out and need to be reworked, either because they were broken or because they were too slow:
% git grep -l 'Test disabled'
test/integration/achievements/collector_supreme.js
test/integration/achievements/home_turf.js
test/integration/achievements/seven_sisters.js
test/integration/data_access/cosmetic_chests.js
test/integration/data_access/inventory.js
test/integration/data_access/migrations.js
test/integration/data_access/rift.js
test/integration/data_access/users.js
Even after a full yarn build
, I see errors in the game client (browser console) regarding missing fonts:
GET http://localhost:3000/fonts/fontawesome-webfont.woff?v=4.2.0 net::ERR_ABORTED 404 (Not Found)
GET http://localhost:3000/fonts/fontawesome-webfont.ttf?v=4.2.0 net::ERR_ABORTED 404 (Not Found)
Not sure exactly what is happening here, but login seems to hang until I refresh the game client, at which point I get to the main menu as usual.
If others can repro this, we should track it down and fix it.
In focusing on the web deployment, there have been some breaking changes to the desktop build. We should:
loading.gif
referenceon: workflow_dispatch
for manual triggers and add on: release
later)I didn't add this when I added JS linting, since I thought we'd decaffeinate right away.
Let's go ahead and add CoffeeScript linting in CI until that's done.
When loading the browser client in the development
environment, all assets are served over Express in the API process.
This results in 75MB of additional data transfer to users: 47MB for duelyst.js
, vendor.js
, and duelyst.css
, and 28MB for additional resources (music, images, and more).
We already have code for serving these from S3 for the production
environment, so we should make sure this works and make use of it.
This will reduce both cloud provider costs and CPU utilization on the API process.
We have several files with overrides set in .eslintrc.json
:
{
"files": ["app/**/*.js"],
"rules": {"camelcase": ["warn"]}
},
{
"files": ["test/**/*.js"],
"rules": {"no-unused-expressions": ["off"]}
}
We should remove these overrides and resolve the underlying linting issues.
An old hbs
dependency is causing the following warning logs during yarn install
:
warning [email protected]: The engine "ender" appears to be invalid.
warning [email protected]: The engine "ender" appears to be invalid.
These are dependencies of walk
, which is in turn a dependency of hbs
. Upgrading hbs
should resolve these.
Following the changes over the last few weeks, the game appears to function in Firefox on Android, but the viewport is not correctly sized.
It may not be too much work to get the window scaling down so the game can be fully played on mobile.
Since packaging and releasing Android & iOS apps is a pain, let's focus on mobile web initially.
To Do:
The single-player server (server/single_player.coffee
) starts and runs, but playing local games is not yet a bug-free experience.
Users should be able to clone this repo and run docker compose up
, then open http://localhost:3000
in their browser to play all single player content.
We have an ALL_CARDS_AVAILABLE
config setting, but its usage is currently commented out in the app: https://github.com/open-duelyst/duelyst/blob/d861bca/app/ui/views2/collection/cards_collection.js#L308L325
We should re-enable this so we can grant full collections in local development and staging.
The AWS Free Tier for ALB, RDS, and ElastiCache is limited to 12 months. After this time, they cost the following amounts:
We can find cheaper alternatives for the staging environment.
Note: Our usage of ECS, EC2, S3, CloudFront, KMS, and SSM Parameter Store are not subject to Free Tier expiration. They're either always free, or cost mitigation work has already been done. These will continue to total $6/mo.
This will largely be done in Terraform, with reusable modules we can use to complete #3.
Done in #72:
t4g.micro
instances for running Node.js apps (API, Game, SP, Worker)db.t4g.micro
RDS instance for Postgrescache.t3.micro
Elasticache instance for RedisSince so much of the above qualifies for the AWS Free Tier, we should be able to provision an always-on staging environment very cheaply. The ECS instances will cost $0.0084 per hour, or $0.0025 if we can leverage Spot Instance pricing (we get a 2 minute warning before Spot Instances are reclaimed; might be suitable for API and Worker). The S3 bucket will have some minor costs (maybe a few dollars per month; depends on CDN configuration and user traffic). Everything else will be completely free.
Done in #75:
Done in #79:
NODE_ENV=staging
Several unit tests have been disabled until we have time to go back and update them. search for Test disabled
in the repo to find them:
% git grep -l 'Test disabled'
test/unit/sdk/ai/ai.js
test/unit/sdk/ai/bosses.js
test/unit/sdk/cards/bloodstorm/faction6.js
test/unit/sdk/cards/bloodstorm/neutrals.js
test/unit/sdk/cards/core/faction2/artifacts.js
test/unit/sdk/cards/core/faction3/minions.js
test/unit/sdk/cards/core/faction5/spells.js
test/unit/sdk/cards/core/neutrals/epics.js
test/unit/sdk/cards/coreshatter/faction1.js
test/unit/sdk/cards/misc/bloodborn_spells.js
test/unit/sdk/cards/monthlies/month7.js
test/unit/sdk/cards/monthlies/month9.js
test/unit/sdk/cards/monthlies/seven_sisters.js
test/unit/sdk/cards/shimzar/battle_pets.js
test/unit/sdk/cards/shimzar/faction1.js
test/unit/sdk/cards/shimzar/faction2.js
test/unit/sdk/cards/shimzar/faction4.js
test/unit/sdk/cards/shimzar/faction5.js
test/unit/sdk/cards/unity/faction5.js
test/unit/sdk/cards/unity/faction6.js
test/unit/sdk/cards/wartech/faction2.js
test/unit/sdk/cards/wartech/faction3.js
test/unit/sdk/cards/wartech/faction4.js
test/unit/sdk/cards/wartech/faction5.js
test/unit/sdk/game/basic.js
We should revisit these and turn them back on once we get them working.
We're currently using [email protected]
, which has poor support for connection timeouts.
Replacing this with [email protected]
fixes the above, but requires some changes. For example, the zrevrankAsync
function does not exist, so we need to use zrevrank
instead and remove the .then(result)
chain.
Alternatively, upgrading node-redis
may help with these issues.
See https://github.com/luin/ioredis/wiki/Migrating-from-node_redis
Running Terraform in CI would solve a few things:
Something like https://www.runatlantis.io/ would be good here, to avoid building something similar ourselves.
ES6 and / or TypeScript mostly makes CoffeeScript no longer necessary. Let's try running codebase through decaffeinate
.
We currently have a small note here pointing to open issues: https://github.com/open-duelyst/duelyst#contributing
We should add a larger contributor onboarding guide with more information around processes, development environments, editor configuration, etc.
Noticed this error + stack trace while playing a practice game locally:
Uncaught TypeError: WebGLRenderingContext.bindBuffer: Argument 2 is not an object.
postStep vendor.js:52124
postStep vendor.js:51396
update BaseParticleSystem.js:1337
resetSystem BaseParticleSystem.js:1163
updateSourceToTarget BaseParticleSystem.js:584
setSourceScreenPosition BaseParticleSystem.js:250
onResize BattleMap.js:303
triggerEvents eventbus.js:156
trigger eventbus.js:89
resize Scene.js:270
onResize application.coffee:3489
later vendor.js:16021
setTimeout handler*later vendor.js:16017
setTimeout handler*later vendor.js:16017
setTimeout handler*_.debounce/< vendor.js:16033
dispatch vendor.js:4676
handle vendor.js:4360
vendor.js:52124:8
I'm not sure what the impact is, yet. It seems specific to BattleMap.js
.
[x] The frontend app
[ ] The game servers
[ ] Game builds or automation scripts
[ ] Tests or Github Actions
[ ] Infrastructure (Terraform)
When loading the app, these errors appear in the console:
GET https://duelyst.freetls.fastly.net/loading.gif 500 (Internal Server Error)
GET http://localhost:3000/favicon.ico 404 (Not Found)
We should add these files to the web assets. I don't see these files in git, though.
Documentation regarding how cards work and how to add, remove, and modify them would be very helpful
Pulling this work out of #69 and #3 to reduce their scope.
We should automate the process of deploying OpenDuelyst when a new release is published on Github.
This can be done by creating an Actions workflow for the release event, and having that action publish container to ECR. If ECS is using the latest
version of the container, I believe it will be able to pull it down automatically. If not, it will still be doable with a small amount of work.
Knex.js, our SQL query builder, has a few minor vulnerabilities in the current 0.19.5 version:
โ Prototype Pollution [High Severity][https://security.snyk.io/vuln/SNYK-JS-UNSETVALUE-2400660] in [email protected]
introduced by:
[email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected]
[email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected]
[email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected]
[email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected]
[email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected]
We should upgrade to 0.95.0 by following this guide: https://github.com/knex/knex/blob/master/UPGRADING.md#upgrading-to-version-0950
This may require changes to code in the cli
, scripts
, server
, test
, and worker
directories.
We should document our plans and priorities for the project.
Nearly every function is server/game.coffee
is duplicated in server/single_player.coffee
:
dnsHealthCheck = () ->
server = http.createServer (req, res) ->
server.listen config.get('game_port'), () ->
savePlayerCount = (playerCount) ->
saveGameCount = (gameCount) ->
getConnectedSpectatorsDataForGamePlayer = (gameId,playerId)->
onGamePlayerJoin = (requestData) ->
onGameSpectatorJoin = (requestData) ->
onGameLeave = (requestData) ->
onGameEvent = (eventData) ->
onGameDisconnect = () ->
playerLeaveGameIfNeeded = (socket, silent=false) ->
spectatorLeaveGameIfNeeded = (socket) ->
destroyGameSessionIfNoConnectionsLeft = (gameId,persist=false)->
tearDownSpectateSystemsIfNoSpectatorsLeft = (gameId)->
clearDisconnectedPlayerTimeout = (gameId) ->
startDisconnectedPlayerTimeout = (gameId,playerId) ->
onDisconnectedPlayerTimeout = (gameId,playerId) ->
restartTurnTimer = (gameId) ->
stopTurnTimer = (gameId) ->
onGameTimeTick = (gameId) ->
restartSpectatorDelayedGameInterval = (gameId) ->
stopSpectatorDelayedGameInterval = (gameId) ->
onSpectatorDelayedGameTick = (gameId) ->
flushSpectatorNetworkEventBuffer = (gameId) ->
_logSpectatorTickInfo = _.debounce((gameId)->
emitGameEvent = (fromSocket,gameId,eventData)->
initGameSession = (gameId,onComplete) ->
initSpectatorGameSession = (gameId)->
onBeforeRollbackToSnapshot = (event) ->
onStep = (event) ->
onInvalidAction = (event) ->
subscribeToGameSessionEvents = (gameId)->
unsubscribeFromGameSessionEvents = (gameId)->
afterGameOver = (gameId, gameSession, mouseAndUIEvents) ->
shutdown = () ->
We should deduplicate as much of this as possible to make maintenance easier.
In order to avoid needing to deal with sender reputation, user data privacy, and legal compliance, let's remove email functionality from the user signup flow.
[Note from Will: This issue is tracking a stable "production" environment. See #69 for staging.]
Many dependencies are outdated, have security issues, or are no longer maintained. We'd like to update dependencies while balancing need to work around API compatibility changes.
There are client errors preventing us from reaching the main menu. Based on checking out one commit at a time and rebuilding, I believe this started happening in 63cc664 (Part of linting changes in #24 .
The old description below was resolved in #30
In c4d3352, just before #15 was merged, the app will start as expected when accessing http://localhost:3000
after running docker compose up
. This is because despite process.env
being empty, the following code succeeds:
# app/application.coffee
Firebase = window.Firebase = require 'firebase'
minBrowserVersionRef = new Firebase(process.env.FIREBASE_URL).child("system-status").child('browsers')
After #15, this results in an error when loading the client:
Unhandled rejection Error: FIREBASE FATAL ERROR: Cannot parse Firebase url. Please use https://<YOUR FIREBASE>.firebaseio.com
Bower is a package manager for web assets like HTML, CSS, JS, fonts, and images. Bower is still maintained, but its developers recommend instead using Yarn and Webpack.
Some of our existing Gulp tasks may also be a better fit for Webpack.
Noise images (such as noise.png
) seem to be intended to make subtle visual effects. However, they're currently appearing without transparency when starting or finishing games.
We should determine why this is happening and fix it. It may be related to the pixelFormat
values in the app.
[x] The frontend app
[ ] The game servers
[ ] Game builds or automation scripts
[ ] Tests or Github Actions
[ ] Infrastructure (Terraform)
noise.png
on the loading screen.Duelyst previously used Mandrill to send outgoing emails: https://github.com/open-duelyst/duelyst/blob/main/server/mailer.coffee#L29L34
This has since been disabled, but we should re-enable this by using AWS SES: https://aws.amazon.com/ses/
This service is free for the first 2,000 emails sent per month, and $0.10 per 1,000 emails beyond that.
Emails were previously sent for the following purposes:
Of these, the first priority is email verification + password resets. User reports can be considered later, and error emails are not needed (since we route error logs to CloudWatch now).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.