CPU / Energy usage about uebersicht

[felixhageloh]

My average energy impact for Übersicht at the moment is 5 and don't really see it spiking above that, except when updating background blur effects. I am only using the top cpu and top mem widgets at the moment though, but they do update every 1 and 5 seconds respectively (and use background blur).

So what you are describing sounds very high and far from ideal. Do you see anything suspicious in the logs and can I ask what widgets you are using?

[skonesam]

Hm… so it's definitely my widgets! I pull them (and add things like battery monitor and cpu monitor) and energy/CPU is nice and low.

I don't think my widgets are doing anything too complex – and I thought I'd be saved with the low refresh times in any event – but clearly that's not the case. Both call an external script and simply render the output. Some details:

Weather: is a php script I wrote for GeekTool and adapted for Übersicht. The widget just invokes the script every ten minutes, which pulls the JSON from Forecast.io. The styling it in the widget. There is no update call; the entire widget is refreshed each time. This is by far the biggest cause of the energy usage; with it alone usage jumps up to a 30+ average energy impact. The icon is an .svg with a css-defined opacity of .35; the table is from a php library (pChart) that outputs a transparent .png (Please forgive the background; didn't want to show a picture of my kid.)

Automatic: is just a summary table for my Automatic car-tracker dealie; it calls a swift script every hour. The styling, again, is in the widget; also again, the entire output is refreshed each time. Adding it alone still causes usage to jump, but it's only to ~7 or 8.

Only thing odd-looking in the system logs is this, periodically (upon refresh): "Übersicht[19692]: ERROR: unable to get the receiver data from the DB!"

So… not sure which of my assumptions are most wrong. ;-) Is it "calling a script infrequently has little to do with overhead", or "re-generating the entirety of output each time is the simplest way to display widgets"?

(Minor updates. Had blindly/stupidly just brought the entire pChart folder in with my script to the widget. Removed some unused parts of the library; this brought it down to 40s instead of 50s. Moving other scripts out of widget folder helped a little, but it's still in the high 30s. Also, removing the transparent icon didn't have a noticeable impact on energy consumption.)

[krid78]

Thanks for the question. It is my most important drawback for Übersicht as well. For me the impact is 13-15 and therefore a factor of 3-4 about the next process (usually Safari). Therefore, I turn of Übersicht when on battery. Localnode is around 15-20% CPU.

My setup is a little more complex, thought. I may give more details, if wanted.

[felixhageloh]

Thanks for sharing your setup. So yeah, that's the drawback of using a Webview for rendering widgets: On one hand it gives you a lot of power, on the other hand it can be very resource intensive if you are not careful. From your description I would guess that there are two things going on:

• The php script that generates the chart seems to be quite heavy. This should have been the case for GeekTool as well though, so if not then there is something else going on.
• Re-rendering a complex DOM structure every time is resource intensive. Ideally you'd just render stuff once and use update to only update parts that changed. However, if it is something simple like an image and a bit of text I wouldn't expect such a big impact. It is hard to guess without seeing the code, but from the top of my head I could think of maybe image scaling: if the png the php library spits out is quite large and you are scaling it down in the widget, that could explain at least some of the high energy usage.

And finally about the error message: I see it pop up sometimes as well, but usually for debug builds. You can delete the cache folder called ~/Library/Caches/tracesOf.Uebersicht and it should go away. I doubt it has anything to do with the energy/cpu usage though

[felixhageloh]

@krid78 go ahead and share your setup as well since I doubt there is a silver bullet kind of answer here.

[krid78]

Concerning your previous answer, I'll tackle it down further to help understanding which of my widgets are critical and wich are not. This may take a day or two. At least I've already had some optimizations in mind, starting with on-demand drawing of some images.

[skonesam]

I have no doubt my php script is, um… "less than optimal." But – and clearly I'm misunderstanding something here – why does it matter if it's only being run every 10 minutes? (Same thing with the Swift script – it's less complex and runs even less frequently at once per hour.)

The output of the script isn't anything too fancy. (I've commented-out the background icon.)

<h1>54°</h1>
<h2>Partly Cloudy</h2>
<h3>50</h3>
<div><h4>Home: Partly cloudy for the hour.</h4>
<h5>6:49am | 7:17pm</h5></div>
<table>
<tr><td class ='date'>Monday</td>
<td class ='maxTemp'>73</td><td class ='minTemp'>51</td>
<td class ='summary'>Partly cloudy throughout the day.</td></tr>
<tr><td class ='date'>Tuesday</td>
<td class ='maxTemp'>73</td><td class ='minTemp'>61</td>
<td class ='summary'>Partly cloudy in the morning.</td></tr>
<tr><td class ='date'>Wednesday</td>
<td class ='maxTemp'>71</td><td class ='minTemp'>53</td>
<td class ='summary'>Partly cloudy overnight.</td></tr>
<tr><td class ='date'>Thursday</td>
<td class ='maxTemp'>71</td><td class ='minTemp'>55</td>
<td class ='summary'>Partly cloudy in the morning.</td></tr>
<tr><td class ='date'>Friday</td>
<td class ='maxTemp'>71</td><td class ='minTemp'>53</td>
<td class ='summary'>Partly cloudy until evening.</td></tr>
</table>
<img class ='lines' src='Weather/temperatureLine.png'>

So I guess the question is what is the most efficient way to implement these widgets? Would it be radically more efficient if I re-coded it in CoffeeScript?

[felixhageloh]

Well, it does matter for the overall energy impact (i'm not sure the energy impact numbers you reported where averages or current numbers). Running something very CPU intensive every 10min will drain more of your battery than running something very light every 10min, if you know what I mean.

However, if you are seeing a constantly high CPU usage instead of spikes every 10min, then something else is going on here.

[skonesam]

It's consistently high. With the two widgets running, CPU averages ~35%. When I pull those widgets (but keep the load/CPU widgets running) it drops all the way to 0.7%.

I'm well past my knowledge level on this stuff. Does it possibly have anything to do with my not calling the '''update''' function? Snippet below:

command: "'/Users/mcconps1/Dropbox/Scripts/Übersicht/Weather/weather.php'"

refreshFrequency: 600000 #ten minutes

render: (output) -> """
    #{output}
"""
style: """
    top: 20px
    left: 40px
    color: #fff
    font-family: Futura
    font-weight: 100
    etcetera…
"""

[felixhageloh]

Oh I see!
So the widget looks fine and not having a update method shouldn't matter. Do you mind sharing your php script (you could use a gist if it is a single file or a zip on a public dropbox folder if it is several).
It looks like it is the only thing that could cause the high cpu usage

[skonesam]

https://dl.dropboxusercontent.com/u/598466/Weather.zip

(Unusual command call is because I have the widgets folder symlinked to my dropbox)

Another data point that implies something weird with the php (or with my php) script is that when I run the CPU/load indicator + Swift script (1 hr refresh) CPU holds ~8% – still higher than it seems it should be, but nowhere near the spike caused by the Weather/php script.

[felixhageloh]

cool, thanks! I'll try to get it to run tomorrow

[felixhageloh]

Alright, managed to get your script to run and I think I know what the issue is. Not 100% yet but what I have so far:

The good news is, it is not your script - at least not directly. The problem is the way Übersicht watches for file changes in the widget dir. I am using a npm module that - if I remember correctly - uses the more reliable but also more resource intensive approach. So two things could be happing:

• because you've added lots of files (your php script dir) the file watcher blows up
• the script generates many temporary files, which makes the file watcher blow up

I'll need to dig deeper to confirm this and see if I can handle things better in Übersicht. In the meantime, if you find a way to put your script outside the widget dir and only symlink/genrate the image files into the widget dir, I am sure CPU usage will go back to normal :)

[skonesam]

Hm… well, I had everything (including images) in a different, external-to-widget folder originally and resource usage was high. I moved it back just to make things clearer and easier to test.

On Sep 16, 2014, at 1:44 PM, Felix [email protected] wrote:

Alright, managed to get your script to run and I think I know what the issue is. Not 100% yet but what I have so far:

The good news is, it is not your script - at least not directly. The problem is the way Übersicht watches for file changes in the widget dir. I am using a npm module that - if I remember correctly - uses the more reliable but also more resource intensive approach. So two things could be happing:

because you've added lots of files (your php script dir) the file watcher blows up
the script generates many temporary files, which makes the file watcher blow up
I'll need to dig deeper to confirm this and see if I can handle things better in Übersicht. In the meantime, if you find a way to put your script outside the widget dir and only symlink/genrrate the image files into the widget dir, I am sure CPU usage will go back to normal :)

—
Reply to this email directly or view it on GitHub.

[rudders]

I had a hunch it was the file monitoring that did it - I get higher cpu usage with more widgets - no matter what they are - especially if I add depth to the hierarchy. Good luck finding a solution - or maybe a preference to turn this feature off for production use rather than development.

[felixhageloh]

@rudders thanks, good info! I'll first see if I can get the monitoring more efficient, otherwise an option might be a good idea.

@skonesam you mentioned something about a symlink tho?

[skonesam]

Yeah, sorry - the symlink is just the widget folder itself, which only contained .coffee and .png files. Script/library was external.

On Sep 16, 2014, at 3:51 PM, Felix [email protected] wrote:

@rudders thanks, good info! I'll first see if I can get the monitoring more efficient, otherwise an option might be a good idea.

@skonesam you mentioned something about a symlink tho?

—
Reply to this email directly or view it on GitHub.

[felixhageloh]

To give a quick update on this one: I have build that uses a non-polling file watcher and it improves performance a lot. I can drop your entire php folder in the widget dir without any real impact. Unfortunately - and I guess this is why it used polling by default - it is a bit quirky, at least on OS X 10.10 for now. If I work around those quirks then it breaks on 10.9 so I need to dig a bit deeper.

[skonesam]

You're awesome - thanks! I've been enjoying Übersicht so much I just let it go at my desk… and then I go through withdrawals when I turn it off to save battery.

[rudders]

Felix, how about a build with no file-watching - we're all geeks and a restart of Übersicht while we're in set-up mode would be perfectly acceptable if we could leave it running the rest of the time I believe.

[felixhageloh]

@rudders I hear ya, but I am still hoping that we can have the best of both worlds!

@skonesam still haven't released this one since things are a bit shaky. It seems like it is bad timing for file watchers on OS X since apple is changing things, so many file watchers out there are having issues. Nonetheless, I have a build here that you can try if you like:

https://www.dropbox.com/s/dpbxoh5m1urfuus/%C3%9Cbersicht-edge.app.zip?dl=0

[skonesam]

Whoa – dropped my average energy usage from 20-22 to about 1! Thanks – this is great!

[krid78]

What a great improvement! Energy usage is down by a factor of 10.

[felixhageloh]

awesome! Do you notice any glitches with changes being pushed when you modify a widget?

[krid78]

Not here. I tried it several times but reloading widgets works immediately after saving (1-2 seconds to wait).

[skonesam]

I haven't seen anything adverse, either. Been completely stable, too. Nice work!

On Oct 6, 2014, at 2:51 AM, Daniel Kriesten [email protected] wrote:

Not here. I tried it several times but reloading widgets works immediately after saving (1-2 seconds to wait).

—
Reply to this email directly or view it on GitHub.

[felixhageloh]

thanks guys, good to know!

[rudders]

Very nice work mate. Opens up a new world of opportunities with this little processor impact. Well done.

[felixhageloh]

New version is out, but this is not included yet - still waiting for a proper fix so my hacks around this issue aren't needed anymore.

I've updated the Dropbox link so you can have the newest version plus this fix.

@rudders thanks, all credits go to the makers of chokidar :)

[felixhageloh]

btw, here is a link to the bug report in case anybody is interested.

[rudders]

@felixhageloh - will give this new one a try too mate...

[felixhageloh]

the newest release includes a fix now!

[skonesam]

Great - thanks you!!

On Oct 26, 2014, at 8:44 AM, Felix [email protected] wrote:

the newest release includes a fix now!

—
Reply to this email directly or view it on GitHub.