jamithnimantha / eia_oil

Hi there, I'm Jamith! 👋

[]

eia_oil's Issues

Scrape data from CSV File instead of XLS file

<style> </style>

EIA Oil Modification

1.     Purpose; Scrape data from CSV File instead of XLS file

 

2.     Sources of Data;

2.1. https://ir.eia.gov/wpsr/table1.csv

2.2. ….Data\investing_output.csv

 

3.     Source Details

3.1. CSV Table from Web Page (https://ir.eia.gov/wpsr/table1.csv)

Right before the time of release, the website may provide an Error page (as in case of .xls file, but it does not automatically download anything).

In case of an error, script will keep trying every 50 milliseconds for not more than 5 minutes.

Copy & Paste of the Error Page and HTML for the web page is provided in CSV Empty File.docx

In case, there is no error, webpage will show CSV file data as shown in EIA Oil CSV File.txt

We are only interested in Stub 1 data. Stub 1 data has been saved in EIA Oil CSV File Part Interested.txt.

This content has been saved in EIA Oil CSV File Part Interested.xlsx. In order to highlight the data we are interested in.

We are interested in 3 Columns

·       Difference (Column D)

·       Percent Change (Column E)

·       Percent Change (Column H)

We are interested in 6 Rows

·       Crude Oil [2]

·       Commercial (Excluding SPR) [3]

·       Total Motor Gasoline [5]

·       Distillate Fuel Oil [11]

·       Total Stocks (Including SPR) [19]

·       Total Stocks (Excluding SPR) [20]

Note; Row numbers are 1 less than the row numbers in the .xlsx file. This is because, a row at the top has been added in order to establish correspondence with Final Output

3.2. CSV File in Data sub-folder (investing_output.csv)

CSV File has been saved as .xlsx file (investing_output.xlsx)  in order to highlight the cells we are interested in.3 Columns

·       Event

·       Actual

·       forecast

We are only interested in 4 Rows with following texts in the Event Column. Columns Names of interest for each rows is provided alongside

·       API Weekly Crude Oil Stock; Actual

·       Crude Oil Inventories; forecast

·       EIA Weekly Distillates Stocks; forecast

·       Gasoline Inventories; forecast

4.     Output Details

The desired Output file (EIA Oil Report.htm) will be saved in PDF sub-folder. Contents are shown below;

It would need to be formatted exactly as below;

Zebra pattern rows

Bold Column Headings

Bold Column A Values for Rows 2,3,5 & 6

Bold Values in % vs Forecast Column & % vs Last Wk for rows 2 & 3 & % vs API for row 5.

 

Column Values for Difference, % vs Last Week & % vs Last Yr will all be derived from https://ir.eia.gov/wpsr/table1.csv.

Column Values for Forecast, API will all be derived from investing_output.csv

 

Column Values for Difference, Forecast & API will be formatted as “###,##0” & Column Values for all % columns will be formatted as Percent.

 

% vs Forecast Column Values will be calculated as follows (Difference – Forecast)/ Asbsolute Value for forecast

% vs API Column Values will be calculated as follows (Difference – API)/ Asbsolute Value for API

 

All the positive values will have red fill and all the negative values will have green fill.

Stub	Difference	Freocast	API	% vs Forecast	% vs Last Wk	% vs API	% vs Last Yr
Crude Oil Ex-SPR	2,346	-418		661.24%	0.60%		-14.60%
Gasoline	805	-279		388.53%	1.50%		-0.70%
Crude Oil Total	1,427				0.10%		-8.50%
Total Stocks Ex-SPR	805		951		0.10%	-15.35%	-13.10%
Total Stocks	-114				0.00%		-10.20%
EIA Weekly Distillates Stocks	-396	-1,007		60.68%	-0.30%		-24.70%

 

5.     Email

 

HTML Table will also be put in the Email body and an email will sent as per the Specs provided in Control.csv

Subject would contain; “:EIA_Oil:”