apearson / southern-company-api Goto Github PK

Southern Company changed how their login system works. Need to pull ScWebToken from https://webauth.southernco.com/api/login. Inside the body there is data. html which contains the form that contains the ScWebToken as the value.

main.js:354

The bill series(data.series[2]) does not exist for accounts that are not on budget billing. This data needs to be wrapped in a check to make sure it exists for the account.

UnhandledPromiseRejectionWarning: FetchError: invalid json response body at https://customerservice2api.southerncompany.com/api/MyPowerUsage/MonthlyGraph reason: Unexpected end of JSON input

(node:1564) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'match' of null
at SouthernCompanyAPI. (C:\scapi\node_modules\southern-company-api\dist\main.js:123:47)
at Generator.next ()
at fulfilled (C:\scapi\node_modules\southern-company-api\dist\main.js:5:58)
at
at process._tickCallback (internal/process/next_tick.js:188:7)
(node:1564) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)

Line 123 is
const matches = resData.data.html.match(regex);

Added some debug lines and the payload is null at [html]? Everything is null actually. Not sure what it should have looked like before.

{ statusCode: 200,
  message: null,
  data:
   { result: 0,
     token: '6%2bp97GxHMEtrFSuDj_redacted_long_token]',
     errorMessage: 'Login Succeeded.',
     messages: [],
     username: 'redacted_username',
     rememberUsername: null,
     staySignedIn: null,
     recaptchaResponse: null,
     targetPage: 0,
     params:
      { appBannerUrl: null,
        appTheme: null,
        appType: null,
        appID: null,
        returnUrl: 'null',
        cancelUrl: null,
        logonID: null,
        loginUrl: null,
        originalReturnUrl: null,
        origin: null,
        scWebToken: null,
        userID: null,
        addProfileLink: null,
        editProfileLink: null,
        forgotPasswordLink: null,
        forgotInfoLink: null,
        southerncoApplication: null,
        newUserHeaderText: null,
        newUserText: null,
        updateProfileText: null,
        forgotInfoText: null,
        emailValidationTicket: null,
        ticket: null,
        emailAddress: null,
        postTarget: null,
        compactDisplay: null,
        errorDisplayType: null,
        extendedIconVisible: null,
        extendedPopupEnabled: null,
        showLogin: null,
        noTokenReturnUrl: null,
        returnMethod: null,
        tokenEncoding: null,
        mfaConfigInternal: null,
        mfaConfigUnknown: null,
        mfaExcludeInternal: null,
        mfaExcludeUnknown: null,
        firstName: null,
        lastName: null,
        email: null,
        phoneNumber: null },
     html: null,
     redirect: null,
     origin: null },
  isSuccess: true,
  modelErrors: null }

Need to make a list of what all needs to be completed to push to stable. Right now it looks like the builds are successful every day.

Hello,
It looks like SouthernCo, or in my case, Georgia Power, just introduced hourly usage. At least on the graph it looks to be delayed by 2 days, but still much more useful than daily data for tracking individual appliances.

This is less a bug and more a feature request for daily usage. I'm trying to find a way to pull this data into Home Assistant.

you asked for issue tickets if someone was willing to help test a southern company API. I'm hopeful GA power is on the list to be able to test against.

Made it working within Home Assistant a week ago, but suddenly it started throwing these errors from yesterday:

/node_modules/southern-company-api/dist/main.js:166
                    throw new Error(`Failed to get secondary ScWebToken: Could not find any token matches in headers`);

I think something has changed within auth model: either JWT acquiring mechanism or the way that cookies are used for it

Hello, thank you for this repo!
I am not receiving all cookies during the last authentication step and only receiving dtCookie. Have you seen this before? any tips?

API is returning back null for the last day recorded if two days are requested.

Hello,

I was interested in using this API and ran into trouble. I saw an earlier issue talking about testing and followed the instructions in that post. I failed the test attempting to get the monthly data and have attached the log file below.

Let me know if I can be of any assistance in fixing the issue.

2022-08-05T14_42_54_188Z-debug.log

Hi! I recently upgraded my Southern Company account to premium, and now the hourly API no longer works for me. After investigating more, I see that the normal graph display for hourly on the Southern Company's My Power Usage page (which I believe is what your API pulls data from) no longer displays data for days since I upgraded. The graph that looked like this up until October 3rd (when I upgraded my account):

now just says "no data available" for every day after October 3rd, which explains why your hourly api can't get the data for those days. Instead, my hourly power usage is displayed in a line graph on energydirect.com that looks like this:

This is a welcome change since this graph has much more precise data down to every 30 minutes (while the old hourly data was always a multiple of 40 kwh for me for some reason, not very precise). However, I was really hoping to use your api or build something similar on top of your api for the more precise hourly data. Do you think this is a realistic goal, and do you have any tips?

Right now I'm wondering how you went about getting the URLs you needed to get the data from the Southern Company. If I could find the api URL used to fetch the data displayed in my second attached graph, then I imagine I could extract the data much like you guys do in your current methods. I'm also really hoping that the login/verification process for EnergyDirect is the same as it is for the main Southern Company website. Please let me know if you can help in any way. If we can figure this out, I'd be happy to try to add the premium hourly data fetching to this repo (assuming you would want that). Just let me know! Thanks!

So I ran the example in the readme:

/* Importing Library */
var SouthernCompanyAPI = require("southern-company-api").SouthernCompanyAPI;

/* Instantiating API */
const SouthernCompany = new SouthernCompanyAPI({
  username: "REDACTED",
  password: "REDACTED",
  accounts: ["REDACTED"],
});

/* Listening for login success */
SouthernCompany.on("connected", () => {
  console.info("Connected...");

  async function fetchMonthly() {
    /* Getting Monthly Data */
    const monthlyData = await SouthernCompany.getMonthlyData();

    /* Printing Monthly Data */
    console.info("Monthly Data", JSON.stringify(monthlyData));
  }
  fetchMonthly();

  async function fetchDaily() {
    /* Getting Daily Data */
    const startDate = new Date(2020, 2, 1);
    const endDate = new Date();
    const dailyData = await SouthernCompany.getDailyData(startDate, endDate);

    /* Printing daily data */
    console.info("Daily Data", JSON.stringify(dailyData));
  }
  fetchDaily();
});

/* Listening for any errors */
SouthernCompany.on("error", console.error);

And I got the following exception:

TypeError: Cannot read properties of undefined (reading '1')
    at SouthernCompanyAPI.<anonymous> (REDACTED/node_modules/southern-company-api/dist/main.js:356:48)
    at Generator.next (<anonymous>)
    at fulfilled (REDACTED/node_modules/southern-company-api/dist/main.js:5:58)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

I popped into main.js and did some good 'ol console.log debugging.

I added:

console.log("rawUsageData",  date_fns_1.differenceInCalendarDays(endDate, startDate), rawUsageData)

right before the offending line and got the following output:

rawUsageData 709 [ [], [] ]

I then took a look at what graphData had in it. Looked like:

graphData [ { labels: [ [Object] ] }, { labels: [ [Object] ] } ]

Here's what the raw graphset data looks like:

{"graphset":[{"labels":[{"text":"No Bill History Available","visible":true,"font-family":"Open Sans","y":0,"x":0,"text-align":"center","font-size":"18","font-color":" #DA312A","font-weight":"300"}]}],"gui":{"behaviors":[{"id":"ShowAll","enabled":"none"},{"id":"ViewSource","enabled":"none"},{"id":"3D","enabled":"none"},{"id":"BugReport","enabled":"none"},{"id":"About","enabled":"none"},{"id":"FullScreen","enabled":"none"},{"id":"Print","enabled":"none"},{"id":"Reload","enabled":"none"},{"id":"LogScale","enabled":"none"},{"id":"SaveAsImage","enabled":"none"},{"id":"DownloadPDF","enabled":"none"},{"id":"DownloadSVG","enabled":"none"}],"context-menu[mobile]":{"button":{"visible":false,"background-color":"transparent","alpha":0,"border-radius":0},"gear":{"visible":false,"alpha":0}}}}

No series property as the code expects. The code checks for this with a if data.series != null however series is undefined here, not null. Additionally, the code is expecting 709 data points.

I was just going to fix this with a simple if series is truthy but that's not enough. What do we want to about the 708 missing data points? Do we want to throw an error if we see No Bill History Available?

Note: If I change the example code to be something slightly more reasonable it works as expected:

    const endDate = new Date();
    const startDate = new Date();
    startDate.setDate(startDate.getDate() - 7); // 7 days
    const dailyData = await SouthernCompany.getDailyData(startDate, endDate);

My python implementation of the api failed, and after some testing, it seems that usage no longer takes AccountNumber as a param. I'd double check me, but removing that fixed some issues.

Delayed reading of data needs to be accounted for. The delay is determined by the rules inside the series.

Will most likely need to check for the existent of rules on both requests, if either of them have a ruleset. Then parse the rules array and pull out all bad data indexes and mark them as undefined or some other indicator.

In this example the index 1 is the delayed reading and needs to be undefined.

{"graphset":[{"background-color":"transparent","plotarea":{"margin":"10 50 50 50"},"chart":{},"scale-x":{"zooming":true,"zoom-to":[1,7],"line-color":"#CFCFCF","values":[0,1],"labels":["26","27"],"font-angle":0,"guide":{"alpha":0,"visible":true},"offset-end":"3px","offset-start":"3px","tick":{"alpha":0,"visible":false},"alpha":1,"items-overlap":true,"decimals":0,"font-color":"#646464","label":{"font-weight":"300","font-family":"Open Sans","font-color":"#646464","alpha":1}},"scroll-x":{"handle":{"background-color":"#215b9e","border-width":"5px","alpha":1},"bar":{"background-color":"transparent","alpha":"0.4","height":"27px"},"zoom-to":0},"scale-y":{"min-value":"0","format":"%v","zooming":false,"line-color":"#CFCFCF","font-angle":270,"guide":{"alpha":1,"line-color":"#CFCFCF","line-style":"solid","visible":true},"offset-end":"5px","tick":{"alpha":0,"visible":false},"alpha":1,"items-overlap":false,"decimals":0,"font-color":"#646464","label":{"font-family":"Open
Sans","font-color":"#646464","alpha":1,"font-angle":270,"text":"kWh"}},"scale-y-2":{"format":"%v°","zooming":false,"values":"35:80:5","font-angle":90,"guide":{"alpha":0,"visible":false},"offset-end":"55px","tick":{"alpha":0,"visible":false},"alpha":0,"items-overlap":false,"decimals":0,"font-color":"#646464","label":{"font-family":"Open Sans","font-color":"#646464","alpha":1,"font-angle":90,"text":"Outside Temperature"}},"type":"mixed","font-family":"Open Sans","series":[{"type":"area","values":[[-1.0,8.0],[0.0,8.0],[2.0,8.0]],"font-family":"Open Sans","background-color":"transparent","marker":{"type":"none","shadow":false,"background-repeat":false},"legend-item":{"visible":false},"line-width":"0px","stacked":"false","data-dateTime":[],"tooltip":{"visible":false,"decimals":0},"legend-marker":{"visible":false}},{"type":"bar","scales":"scale-x,scale-y","values":[[0,8.0],[1,8.0]],"text":"Regular Usage","font-family":"Open Sans","background-color":"#94c955","legend-item":{"visible":false},"rules":[{"rule":"%i == 1","background-color":"#FFFFFF","border-color":"#C8C8C8","border-width":"2px","tooltip":{"visible":true,"decimals":0},"tooltip-text":"Delayed Reading","font-color":"#646464"}],"tooltip-text":"Weekdays","data-dateTime":[],"tooltip":{"visible":true,"decimals":0},"legend-marker":{"visible":false}},{"type":"area","scales":"scale-x,scale-y-2","values":[[0,54.0],[1,69.0]],"text":"High Temp","font-family":"Open Sans","gradient-colors":"#ef881f transparent transparent","gradient-stops":"0.1 0.3 0.9","line-color":"#ef881f","border-color":"#D52B1E","marker":{"type":"none","shadow":false,"background-repeat":false},"legend-item":{"visible":true},"line-width":"2px","stacked":"false","tooltip-text":"High Temp","data-dateTime":[],"aspect":"spline","tooltip":{"visible":true,"decimals":0},"legend-marker":{"visible":true}},{"type":"area","scales":"scale-x,scale-y-2","values":[[0,49.0],[1,48.0]],"text":"Low Temp","font-family":"Open Sans","gradient-colors":"#01a3e4 transparent transparent","gradient-stops":"0.2 0.3 0.9","line-color":"#01a3e4","border-color":"#4060AF","marker":{"type":"none","shadow":false,"background-repeat":false},"legend-item":{"visible":true},"line-width":"2px","stacked":"false","tooltip-text":"Low Temp","data-dateTime":[],"aspect":"spline","tooltip":{"visible":true,"decimals":0},"legend-marker":{"visible":true}}],"labels":[],"arrows":[],"plot":{"stacked":true,"hover-state":{"visible":false}},"images":[]}],"gui":{"behaviors":[{"id":"ShowAll","enabled":"none"},{"id":"ViewSource","enabled":"none"},{"id":"3D","enabled":"none"},{"id":"BugReport","enabled":"none"},{"id":"About","enabled":"none"},{"id":"FullScreen","enabled":"none"},{"id":"Print","enabled":"none"},{"id":"Reload","enabled":"none"},{"id":"LogScale","enabled":"none"},{"id":"SaveAsImage","enabled":"none"},{"id":"DownloadPDF","enabled":"none"},{"id":"DownloadSVG","enabled":"none"}],"context-menu[mobile]":{"button":{"visible":false,"background-color":"transparent","alpha":0,"border-radius":0},"gear":{"visible":false,"alpha":0}}}}

Is this still a valid method to get data from Southern Co? Been playing around with / debugging it a bit and mostly i'm just getting internal server errors.

Need to read JWT for the expiration time and then redo the login or renew the token on expiration.

How should the config object passed to the object's constructor.

Current ideas:

1. Have account and accounts options separate
2. Have accounts and have it always be a list
3. Have accounts and have it accept an array and a string

Option 2 looks like the best

Monthly data and daily data seem to be returning no data sometimes during tests