fi0x / edct

EDCT is a tool for large quantity trades in Elite: Dangerous. It is very useful for fleet carrier owners

⚠️ WARNING - Development of this tool is discontinued ⚠️

The tool uses outdated data-collection-mechanics that currently won't work! There are some issues during its initialization phase, which can cause the inara-website to get blocked for a while.

Elite: Dangerous Carrier Trader

EDCT is a tool for large quantity trades in Elite: Dangerous. It is very useful for fleet carrier owners to find profitable trades.

Features

• GUI
• Search for the best commodity prices of all commodities
• Sorted result display
• Display profit/t
• Display distance between systems
• Display station supply / demand
• Display general station information
• Display carrier information
  • How high buy / sell prices need to be
  • How high the profit for traders is
• Filters
  • Stations that have a minimum supply / demand (useful for fleet carriers)
  • Stations with large landing pads
  • Orbital stations
  • Fleet carriers
  • Odyssey stations

Data

EDCT uses trade information from the inara and EDDB websites and the EDDN. It also uses the EDSM API to get system coordinates.

Since inara does not provide any API endpoints for trade-data, EDCT gathers the required information directly from the HTML files that are provided to every website user. Therefor the trade information is always as accurate as the information on inara itself.

To reduce traffic on inara, the HTTP requests are delayed, which results in a few minutes of waiting time at the first launch of the program. This is only required on the first launch. Every other startup will provide results much faster, since downloaded trade data is stored in a local database.

The program also uses the EDDN to keep the local database updated.

It also gathers information from EDDB to get the galactic average of each commodity.

To provide information about the distance between two systems, the tool gathers coordinates from each system via the EDSM API.

For more information about how this tool operates, visit the wiki

If you find any bugs or would like a feature to be implemented, please create a new issue here.

edct's Issues

Use EDDN in addition to INARA

Is your feature request related to a problem? Please describe.
Inara only provides limited commodity information

Describe the solution you'd like
Use EDDN to receive new trade data instead of Inara

Additional context

• Links
  • https://github.com/EDCD/EDDN
  • https://eddn.edcd.io/schemas/commodity/3
• EDDN only supplies currently uploaded information
• Inara is still required for the initial setup of the local db

Improve documentation

Is your feature request related to a problem? Please describe.
There is no detailed information about what the program does and how it operates

Describe the solution you'd like

• Update the wiki home-page to explain briefly what the program does
• Add a wiki page that explains the program in detail
• Add a wiki page for the filters
• Add a wiki page to describe the buttons (reload button)
• Update the README.md

Add Mixpanel support

Is your feature request related to a problem? Please describe.

It is currently not possible to accurately measure how many users use this tool

Describe the solution you'd like

Add mixpanel support with several events

• Initialization
• Shutdown
• Error
• Warning
• Settings change
• Filters change
• Button presses
  • Trade reloader
  • Settings
  • Download new version
  • Error information
  • Report a bug

Describe alternatives you've considered

Only send mixpanel events for initialization, shutdown, errors and warnings

Fix github actions

Describe the bug
javafx is not working on github actions

Additional context
Find out which downloads / tasks are missing in the yml file

• Update to java 11+
• Use javafx as dependency

Stop all threads when application is closed

Describe the bug
When closing the application window there is still at least one thread running in the background.

To Reproduce
Steps to reproduce the behavior:

1. Start the application
2. Close the window

Expected behavior
The IDE should display that the process stopped, but doesn't

Add min-galactic average filter

Is your feature request related to a problem? Please describe.

Commodities with a low galactic average give no high profit to the carrier

Describe the solution you'd like

• Add a filter that allows the user to select a minimum galactic average
• Only display commodities that have a higher galactic average price

Describe alternatives you've considered

Make it an option in the settings

Add .exe compile function to maven

Is your feature request related to a problem? Please describe.
The project is currently only compiled as jar

Describe the solution you'd like
Extract a .exe version with maven

Remove hidden entries completely

Describe the bug

When switching between detailed and simple view, the GUI only hides details and does not remove the space they occupied

To Reproduce

Version: 1.1.3.3
Steps to reproduce the behavior:

1. Go to the settings
2. Switch to simple view instead of details
3. Check the GUI

Expected behavior

The hidden details should be removed so that the GUI gets smaller

Additional context

Maybe remove the labels from the V-Box / Container and add them at the correct index again when feature gets enabled

Copy system name to clipboard

Describe the solution you'd like

• Copy the system name to the clipboard when user clicks on it

Describe alternatives you've considered

Add a new button next to the systemname instead

Better explanation of the background phases

Is your feature request related to a problem? Please describe.

A user was curious what the "Initializing: x/230" means.

Describe the solution you'd like

Explain in the tooltip what each phase means

Describe alternatives you've considered

Add a link to the wiki

Additional context

Update the tooltip from the background logic to explain current state

Improve GUI-structure

Is your feature request related to a problem? Please describe.
The whole gui is in a single fxml file which is not very maintainable

Describe the solution you'd like
Add multiple fxml files and combine them to build up the gui

• Filters
• Request button
• Results with commodity name and buttons
  • Sell data with buttons
  • Buy data with buttons

Additional context

• This will also simplify the use of css files
• Maybe change the position for the profit to be at the top of the results
• Add a status indicator at the top to inform the user about the age of the data and if there is currently a data-update running in the background (#10 )

Update local database structure

Is your feature request related to a problem? Please describe.

The local database is not very efficient

Describe the solution you'd like

Change the entire structure of the database to use different tables

• Stations
  • StationName
  • SystemName
  • PadSize
  • StationType
• Trade
  • StationName
  • SystemName
  • InaraID
  • Age
  • Supply
  • Demand
  • BuyPrice
  • SellPrice
• Systems
  • SystemName
• Commodity
  • CommodityName
  • InaraID
  • GalacticAverage
  • LastUpdated
• Distances
  • SystemName1
  • SystemName2
  • Distance

Additional context

This will allow faster queries

Clean up code

Is your feature request related to a problem? Please describe.

The general structure of the code is not very good

Describe the solution you'd like

Re-structure the whole code

Additional context

• Create additional classes to clean up current ones
• Move methods to correct classes
• Move classes to correct directory
• Check if wiki links still work

Improve Number format

Is your feature request related to a problem? Please describe.
Big numbers are hard to read

Describe the solution you'd like
Add dots or spaces in numbers

Additional context
Might be easiest to add a number converter that gets used in the controllers

Replace FXTaskbarProgressBar with custom version

Is your feature request related to a problem? Please describe.
The .exe has grown a lot by adding the FXTaskbarProgressBar dependency

Describe the solution you'd like
Write a custom version of the FXTaskbarProgressBar that does not increase the .exe size that much

Additional context

• https://stackoverflow.com/questions/39192528/how-can-you-send-information-to-the-windows-task-bar-from-java-o-javafx

Periodically update trade data instead of waiting for retrieval

Is your feature request related to a problem? Please describe.
If trade data needs to be downloaded from inara, the gui only updates after all data is collected

Describe the solution you'd like
The GUI should either update periodically with the currently available data, or the InaraCalls should send a notification when new data is available.

Additional context
There might be need of a button to let the user decide weather to update now or later

Clean up output

Is your feature request related to a problem? Please describe.
Some outputs are not necessary, other are missing

Describe the solution you'd like

• Remove all outputs that are neither a warning nor an error
• Make sure every error / warning produces an output

Additional context
Wait for #19

Add a filter or label to display the distance between stations

Describe the solution you'd like

• Add a filter option that filters the distance (in Ly) between the buy and sell station
• Add a label that displays the distance (in Ly) between the buy and sell station

Describe alternatives you've considered

• Use the distance to Sol as a fixed point instead of the distance between the stations
• Only display the distance without adding a filter option

Additional context

• Check what information INARA provides

Use a local database instead of single files

Is your feature request related to a problem? Please describe.
The current data-system is not easily adjustable / adaptable for other websites

Describe the solution you'd like
Implement a database instead of single files. This may also help storing older data that is not any more provided by the websites

• Store downloaded data in db
• Update db regularly with updater that runs in the background
• Use db information to display results
• Provide methods for easy access of commodity data

Describe alternatives you've considered
Improve local data-structure to be more adjustable and support older data

Additional context
SQLite with JDBC driver might be a good solution

Access-restrictions

Describe the bug

Some Mixpanel-events indicate problems when writing to the local database without access permissions.

To Reproduce

Version: 1.2.6.4, 1.2.6.5
Steps to reproduce the behavior:
Mixpanel reports show Warnings, unknown how to reproduce the behavior

Expected behavior

Log-files, settings-file and database-file should be accessible for the program

Additional context

• Check for admin / user permissions
• Find out if Roaming-folder is restricted to some users
• Maybe file gets locked when DBHandler accesses database and second instance gets blocked

Slow DB-requests

Describe the bug

Reloading trades is very slow (~10s)

To Reproduce

Version: 1.2.6.4
Steps to reproduce the behavior:

1. Start the program
2. Wait for the trades to be loaded
3. Click the reload trades button

Expected behavior

The reload trades button should re-appear quicker

Additional context

• Change filters to be included in sql-queries
  • Remove filters from code
  • Add filters to getTrades query
• Sort trades in query
  • Remove sorting algorithms from code
  • Add order command to sql queries

Change the way commodities are downloaded and updated

Is your feature request related to a problem? Please describe.
The user has to click on buttons to update local files or display the trade options

Describe the solution you'd like

• The local files should be updated in the background
• Remove the button to force download files
• Update trade list as soon as local files are updated
  • Fire an event when local files get updated
  • Update trade data in GUI directly from db when event gets fired
  • Remove Replace "Find best trades" button with "Reload" button
  • Display the Reload button if there are new best trades
• Load best trades as soon as program starts

Trade-reloading status indicator

Is your feature request related to a problem? Please describe.

When the trade reloader button is pressed, there is no way of knowing if it's still running

Describe the solution you'd like

• Add a label that tells the user that the trades are updating
• Display the label where the trade-reloader button is when the button is not visible

Station update button

Describe the solution you'd like

Add a button to update a stations stats

Additional context

• Add the button
• Gather market information for current station from inara
• Update local database
• Update display

Add min-profit option

Is your feature request related to a problem? Please describe.
It takes very long to request all commodities

Describe the solution you'd like
Filter out commodities from the initial request when their max-profit is below a threshold

Additional context
Read max-profit from start-up request

Provide more trade information

Describe the solution you'd like

• Add a settings option for enhanced details
  • Make a hidden part of the settings visible that contains more options
    • Ship cargo space (0-999)
    • Loading Profit (Min profit/t when loading)
    • Unloading Profit (Min profit/t when unloading)
    • Carrier profit/ton (The carrier difference between buy and sell price)
• Add a new section to the trade information that is only visible with this setting active
  • Display Galactic average
  • Display loading profit
    • Price span depending on loading prices
    • Profit/ton
    • Profit/x-tons
      • Profit for selling as many tons as were defined in the settings
      • Also show how many tons were selected in the settings
  • Display unloading profit
    • Price span depending on unloading prices
    • Profit/ton
    • Profit/x-tons
      • Profit for selling as many tons as were defined in the settings
      • Also show how many tons were selected in the settings
  • Display ccarrier stats
    • Profit/t
    • Total profit
    • Required buy price
      • Respect galactic average
      • Depends on loading profit for traders
    • Required sell price
      • Respect galactic average
      • Depends on unloading profit for traders

Additional context

• Display additional information in panel between buy and sell stations
• Remove panel completely if disabled, not only hide it

Automatically download new versions

Is your feature request related to a problem? Please describe it

User has to manually download and install new versions.

Describe the solution you'd like

• Don't open the github page on update button press
• Download correct asset file from release instead
• Remove outdated versions

Add better exception-handling

Describe the bug
Some GUI-elements produce exceptions when clicked

Expected behavior

• Avoid creating errors
  • Add null-pointer checks
  • Check for correct index
• Catch remaining exceptions

Save information in registry

Is your feature request related to a problem? Please describe.

• There is no way of knowing weather or not inara has blocked access due to too many requests.

Describe the solution you'd like

• Store information about timeouts on inara in registry to avoid too many requests
• Store filter options in registry settings file

Describe Alternatives you've considered

• Store timeout information in the local-storage directory in the error file
  • Use OUT class to write errors and warnings in log file
• If registry can't be accessed, don't save filter options

Additional context
Find other information that could be stored in registry

Add a settings and bug-report button

Describe the solution you'd like

• Add a button that links to the github issues page
• Add a button that opens a settings window with some options
  • Select which errors to display
  • Reset settings / filters
  • Set low / high profit border (For red / green profit text)
  • Select which information to display
  • Select time-interval for inara updates
  • Clear logs
  • Clear DB
  • Set max age for DB entries

Describe alternatives you've considered
Instead of opening a settings window, display the settings all the time next to the filters

Getting trades that do not exist

Describe the bug
I get trades that do not really exist. (Selling water for 61k at West Gateway in Khernidjal)

To Reproduce
Version: 1.0.2.2+
Steps to reproduce the behavior:

1. Start program with development version
2. Wait a while to gather trades
3. Maybe restart once
4. Allow small pads as filter option
5. Deactivate small pad filter option
6. Go through the commodities to see if there are weird trades

Expected behavior
The program should only display trades that exist

Screenshots

Additional context
Possible error sources:

• EDDN update
  • A station update always uses the same commodity and doesn't change while it iterates through the for-loop
• INARA update
• Number conversion

Possible solutions:

• Fix EDDN updater
• Add a "delete from db" button to the stations to remove wrong trades and make it work

Retrieve distance between stations

Is your feature request related to a problem? Please describe.
It is not possible to know how far the stations are apart from each other

Describe the solution you'd like

• Get distance between stars from http://elitedangerous.hozbase.co.uk/calc/distances when trade gets displayed
• Store distance in trade information

Describe alternatives you've considered
Store the distance to SOL for each station in the database

Create new logs for each session

Is your feature request related to a problem? Please describe.

Every error and warning gets stored in the same log file

Describe the solution you'd like

• Create a folder for all log files
• Create a new log file for each session

Store commodity-information locally

Is your feature request related to a problem? Please describe.
It takes very long to request over 200 commodity pages

Describe the solution you'd like
Store each html string in a file on disk and only update it every x-minutes.
This would reduce the amount of requests that inara receives and increases the speed at which the ui could respond to trade-data requests, since the trade-data-update could run on a different thread in the background

Outdated trades are not removed from db

Describe the bug
Old database entries are not removed and displayed as good trades

To Reproduce
Steps to reproduce the behavior:

1. Start the program
2. Start it again a few days later

Expected behavior
The program should only display trades that exist

Additional context
Remove data that is older than x days

Remove windows warnings on executing the program

Is your feature request related to a problem? Please describe.
When starting the .exe, you get prompted with a warning from windows about an unknown publisher

Describe the solution you'd like
The .exe should start without any security questions

Additional context
Adding a companyName in the launch4j version information might be sufficient

[bug] Explorer's Anchorage is a valid sell station

Describe the bug

When calculating trades, Explorer's Anchorage is considered to be a valid destination. This being in spite of the fact that Explorer's Anchorage is several thousand light years outside of the bubble and shouldn't be a profitable destination. I would suggest also similarly filtering Colonia, but generally speaking the Colonia markets are already collapsed and overfed to begin with so they shouldn't be a problem.

To Reproduce

Version: 1.1.3.3
Steps to reproduce the behavior:

1. Open the game.
2. Get a trade.
3. Trade Destination is Explorers Anchorage.

Expected behavior

Trade Destinations should not be Explorer's Anchorage

Screenshots

Clean up GUI

Is your feature request related to a problem? Please describe.

• The dropdown menus in the settings don't fit to the design
• The gui itself has no icons

Describe the solution you'd like

• Fix design issues with dropdown-menus
• Add icons for station types
• Add animated loading-icon whenever something gets loaded
• Make "Updating" Text more descriptive
• Add text for the amount of DB entries that are stored locally
• Open station page on inara when station name is clicked
• Re-style the popup windows / Alerts

200 response has no content

Problem

When requesting prices for a specific commodity, the response has a 200 status code, but contains no content.

Possible solutions

• The url parameters might be malformed
  • Find out what the parameters exactly are
  • Try adding a ? at the start of the parameter string
  • Try adding / removing the / at the end of the url
• The url parameters might not be added to the request correctly.
  • Remove URLConnection parameter
  • Add parameters as part of the url

Fix Sorting order

Describe the bug

Commodities and Stations are not sorted

To Reproduce

Version: 1.2.5.4 beta
Steps to reproduce the behavior:

1. Go through the commodity list or stations
2. check commodity profit / station price
3. Prices are not sorted from high to low / low to high

Expected behavior

• Commodity profit should be sorted descending
• Buy stations should be sorted after price ascending
• Sell stations should be sorted after price descending

New GUI elements

Add missing GUI elements

• Min-profit text-field #6
  • Should be in filters.fxml
• Label for age of local data #7
  • Should be in datastorage.fxml
• Button to switch between minimal and maximal information
  • Should be in datastorage.fxml
• Labels for additional station information
  • Should be in station.fxml
  • Possible information
    • Landing Pad
    • Station Type
    • System
    • Ls to main star
  • Add content to new labels
• Label for distance between both stations / systems
  • Should be in stations.fxml
  • This information needs to be gathered first from some website (EDSM?)
• Explaining labels everywhere
  • Min t Label
  • Min profit Label
  • Label for background calculation status (error, downloading from inara)
  • Station-Name Label
  • Sell / Buy Price Label
  • Supply / Demand Label

Add an option to disable the system distance or run it in the background

Is your feature request related to a problem? Please describe.
The GUI waits each time a distance is acquired via HTTP request.

Describe the solution you'd like

• Run the request in the background and update the GUI as soon as a result is available

Describe alternatives you've considered

• Add an option in the settings to disable this feature or only request it on a button-press (Place a button where the label is)

Improve release-check

Is your feature request related to a problem? Please describe.

Drafts and Pre-Releases get also recognized as new versions at the version-check

Describe the solution you'd like

• Add a setting to allow notifications for new releases that are pre-releases, hotfixes or drafts
• Check for draft and pre-release tag on github-release-check
• Check if only last version number increased -> hotfix
• Update "New version" button when draft, hotfix or pre-release settings get changed

Make the GUI look nice

• Add css to the GUI
• Fix margins
• Fix min-sizes
• Add tooltips
• Fix alignment
• Add some borders / lines to seperate sections
• Fix data-age for stations

Add new table to store distances

Describe the solution you'd like
Add a new table that stores the distance between 2 star systems

Additional context

• Requires System1, System2, Distance
• Save as soon as information is collected from website

Fix FX-thread exception

Describe the bug
When requesting trade data, the program throws an exception.

Additional context

• Try to use binding and listeners to change the text-field properties in the gui from a different thread
• Try using Platform.runLater()

Fix DB update-errors

Describe the bug

DBHandler.setTradeData() creates a new row each time it gets called instead of updating existing ones

To Reproduce

Version: 1.2.5.4 beta
Steps to reproduce the behavior:

1. Clear your database
2. Wait for initialization to finish
3. Wait for a few EDDN updates
4. Check the Trades table for duplicate station entries for the same commodity

Expected behavior

Each station should only appear once in the trades table for each commodity

Additional context

• Fix the setTrades() method in the DBHandler
  • Create new rows for new entries
  • Update existing rows if they exist with the same station and commodity

Maybe helpful:

• INSERT INTO
• ON CONFLICT
• UPDATE
• https://stackoverflow.com/questions/23622504/on-duplicate-key-not-working-in-sqlite/23622827

Add errors to error file

Is your feature request related to a problem? Please describe.
Errors and warnings are only printed in the console

Describe the solution you'd like

• Write every error and warning to the error-file

Additional context
Wait for #20 to be completed

Add a version updater button

Is your feature request related to a problem? Please describe.
Users don't get notified if there is a new version available.

Describe the solution you'd like
Add a button with the following features

• Becomes visible when there is a newer version on Github (Check on startup)
• Opens the github release page with the default browser

Fix java runtime environment error

Describe the bug
The .exe could not be executed on a pc with java 1.8

Expected behavior
The .exe should always be executable, even if there is no jre at all

Possible solution
Use inno-setup to create an installer that adds the jre

Additional context
Check the launch4j config example: http://launch4j.sourceforge.net/docs.html
These links might help

• TheBoegl/gradle-launch4j#86
• https://github.com/vZome/vzome/blob/master/desktop/platform/windows/build.gradle
• https://howtodoinjava.com/maven/create-windows-exe-file-for-java-application/