kaiede / rpilight Goto Github PK

Ran across this situation last night with my tanks:

Lunar period runs overnight. But the schedule puts out the lights at around 10:30pm. At 10:30pm, instead of switching segments and waiting, it seems to have actually decided to do nothing. No new segment is reached, the timer isn't updated. It just does nothing.

Many lights turn off at a duty cycle higher than 0%. The Twinstar starts having problems at about 0.2% duty cycle, with half the lights off, and completely shuts off just under 0.1% luminosity.

It would be useful to configure a per-channel minimum luminosity setting in the hardware configuration to get better on/off ramps that turn on or off when the user's schedule says.

There is still a possible race condition here when SetBehavior is not being called from the Light Controller’s run loop. This mostly happens in test.py and preview.py

Changes meant to mitigate this are checked in, but it is still possible that this isn’t completely fixed.

Right now, if someone wants to run the daemon, they have to do it themselves. Rhere should be a mechanism to launch it on boot of the device like a standard service.

Investigate launching pigpiod on launch as well.

As currently implemented, the JSON for the schedule makes the assumption that all events are in order in the JSON. I think we either need to validate that the ordering is correct when reading the JSON, or we need to sort it on load so that the Schedule object we create has the events in sorted order.

• Raspbian Buster shipped pulling packages from the 'testing' Debian Buster repo. It needs an 'apt-get update' before Swift 5's dependencies can get installed.
• Need to make sure Swift is in the PATH before calling the build script while bootstrapping. I've got a patch ready, just need to test it.

It’s very possible this line of code has a bug in it. It really just should set the start date to the day of ‘now’ to make it reliable.

Saw something unusual in the logs this morning. Segment switches are somehow getting delayed. Not entirely sure how that is happening or why. Some sort of drift? Delay? Not sure. Need to collect better data.

Aug 25 10:58:28 bigtank systemd[1]: Started RPiLight.
Aug 25 10:58:28 bigtank RPiLight[2888]: [info][10:58:28.577] PWM0-IO18 Minimum Intensity: 0.0
Aug 25 10:58:28 bigtank RPiLight[2888]: [info][10:58:28.600] Startup User: root
Aug 25 10:58:28 bigtank RPiLight[2888]: [info][10:58:28.604] Final User: pi
Aug 25 10:58:28 bigtank RPiLight[2888]: [info][10:58:28.605] Configured Board: Raspberry Pi / Zero (ARMv6)
Aug 25 10:58:28 bigtank RPiLight[2888]: [info][10:58:28.605] Configured PWM Module: Hardware GPIO
Aug 25 10:58:28 bigtank RPiLight[2888]: [info][10:58:28.606] Configured PWM Frequency: 10000 Hz
Aug 25 10:58:28 bigtank RPiLight[2888]: [info][10:58:28.607] Configured Gamma: 1.8
Aug 25 10:58:28 bigtank RPiLight[2888]: [info][10:58:28.637] Initial Segment Guess: [Schedule] (0.278255940220712, ∆0.0) 08:30:00.000 -> 13:30:00.000
Aug 25 10:58:28 bigtank RPiLight[2888]: DEPRECATED USE in libdispatch client: Changing target queue hierarchy after object has started executing
Aug 25 10:58:28 bigtank RPiLight[2888]: [info][10:58:28.678] Next Event: lunar (25 Aug 21:15:00.000)
Aug 25 13:30:00 bigtank RPiLight[2888]: [info][13:30:00.019] Switched to Segment: [Schedule] (0.278255940220712, ∆0.402139059866476) 13:30:00.000 -> 14:00:00.000
Aug 25 14:00:00 bigtank RPiLight[2888]: [info][14:00:00.727] Switched to Segment: [Schedule] (0.680395000087188, ∆0.0) 14:00:00.000 -> 19:00:00.000
Aug 25 19:00:00 bigtank RPiLight[2888]: [info][19:00:00.019] Switched to Segment: [Schedule] (0.680395000087188, ∆-0.491071049616456) 19:00:00.000 -> 20:00:00.000
Aug 25 20:00:01 bigtank RPiLight[2888]: [info][20:00:01.276] Switched to Segment: [Schedule] (0.189323950470732, ∆0.0) 20:00:00.000 -> 21:00:00.000
Aug 25 21:00:00 bigtank RPiLight[2888]: [info][21:00:00.019] Switched to Segment: [Schedule] (0.189323950470732, ∆-0.136643436626199) 21:00:00.000 -> 21:15:00.000
Aug 25 21:15:00 bigtank RPiLight[2888]: [info][21:15:00.000] Firing Event: lunar
Aug 25 21:15:00 bigtank RPiLight[2888]: [info][21:15:00.006] Starting Lunar Night: 0.999039316878303 -> 0.999849414409972
Aug 25 21:15:00 bigtank RPiLight[2888]: [info][21:15:00.008] Lunar Night Period: 21:15:00.000 -> 08:00:00.000
Aug 25 21:15:00 bigtank RPiLight[2888]: [info][21:15:00.031] Initial Segment Guess: [Lunar] (1.0, ∆0.0) 08:00:00.000 -> 21:15:00.000
Aug 25 21:15:00 bigtank RPiLight[2888]: [info][21:15:00.040] Next Event: lunar (26 Aug 21:15:00.000)
Aug 25 21:15:00 bigtank RPiLight[2888]: [info][21:15:00.088] Switched to Segment: [Schedule] (0.0526805138445332, ∆0.0) 21:15:00.000 -> 22:25:00.000
Aug 25 21:15:00 bigtank RPiLight[2888]: [info][21:15:00.102] Switched to Segment: [Lunar] (1.0, ∆-0.000533826837881946) 21:15:00.000 -> 21:20:00.000
Aug 25 21:21:51 bigtank RPiLight[2888]: [info][21:21:51.614] Switched to Segment: [Lunar] (0.999466173162118, ∆0.000450165377043565) 21:20:00.000 -> 07:55:00.000
Aug 25 22:30:55 bigtank RPiLight[2888]: [info][22:30:55.634] Switched to Segment: [Schedule] (0.0, ∆0.0) 22:30:00.000 -> 06:30:00.000
Aug 26 07:16:34 bigtank RPiLight[2888]: [info][07:16:34.156] Switched to Segment: [Schedule] (0.0526805138445332, ∆0.0) 06:35:00.000 -> 08:00:00.000
Aug 26 08:48:45 bigtank RPiLight[2888]: [info][08:48:45.978] Switched to Segment: [Schedule] (0.278255940220712, ∆0.0) 08:30:00.000 -> 13:30:00.000
Aug 26 08:48:45 bigtank RPiLight[2888]: [info][08:48:45.991] Switched to Segment: [Lunar] (1.0, ∆0.0) 08:00:00.000 -> 21:15:00.000

A schedule like this is much more readable and compact compared the current examples:

{
    "hardware" :{
        "board": "PiZero",
        "pwmMode": "hardware",
        "freq": 960,
        "channels": 2
    },

    "schedule" : {
        "PWM0-IO18" : [
            { "time": "08:00:00", "brightness": 0.0 },
            { "time": "08:30:00", "brightness": 0.25 },
            { "time": "12:00:00", "brightness": 0.25 },
            { "time": "14:00:00", "brightness": 0.50 },
            { "time": "18:00:00", "brightness": 0.50 },
            { "time": "20:00:00", "brightness": 0.10 },
            { "time": "22:30:00", "brightness": 0.10 },
            { "time": "23:00:00", "brightness": 0.0 },
        ],
        "PWM1-IO19" : [
            { "time": "08:00:00", "brightness": 0.0 },
            { "time": "08:30:00", "brightness": 0.30 },
            { "time": "12:00:00", "brightness": 0.30 },
            { "time": "14:00:00", "brightness": 0.30 },
            { "time": "18:00:00", "brightness": 0.30 },
            { "time": "20:00:00", "brightness": 0.15 },
            { "time": "22:30:00", "brightness": 0.15 },
            { "time": "23:00:00", "brightness": 0.0 },
        ]
    }
}

Depends on #39 in order to work, because it changes around ownership of lighting segments.

It would be nice if the schedule included the ability to insert thunderstorms.

Example:
User decides that starting at 4:30pm every day, there should be a 20% chance of a thunderstorm lasting 30 minutes should occur. This is added as a special event into their schedule JSON.

The easiest way to implement this is to make a Behavior that handles the thunderstorm. It should have a higher priority than the basic ramp. It can fire at fairly short intervals to mimic lightning. It may also be worth making it possible for an active behavior to set the next run time for the behavior job to lessen some of the load on the CPU, letting it idle between lighting strikes. Since it will want to do updates to the PWM at rates on the order of a millisecond during a lightning strike, but go dormant for multiple seconds afterwards.

The trick here I think is to handle the down/up ramp of the lighting to simulate clouds when entering the thunderstorm and leaving it. Ramping down to the thunderstorm is easy, but ramping back up afterwards can be tricky. I'm thinking of situations where the ramp back up crosses an event point in the schedule. We may want to think about leveraging something like the "Behavior Plugin" during the transitions, and a Behavior for the thunderstorm itself.

The built-in support for serialization does weird things starting with Swift 4.

For example, “0.0” is seen as an Int, not a Double like in Swift 3.

Swift 4 introduces Codable which would be a good approach, but only works with Swift 4. In the short term we need something that works for 3 and 4, sadly. SwiftyJSON at least aligns with Kitura.

LightBehavior calculates the interval that should be used in a rather naive manner. It fires fairly often in very shallow ramps with little delta, despite not having a whole lot to do during most of the ramp.

By improving this, we can reduce power consumption a little in many real world cases. It just isn't a huge priority when without this improvement, it uses <1% CPU on a Pi 3B.

Wound up creating a new swift file for extensions when one already existed. Clean this up.

There's some value in refactoring the LightController class to be broken down a little more.

What I'm envisioning is a hierarchy like this:

• Light Controller
  Root controller. Handles dispatching updates to individual controllers, owner of the dispatch queue.

• Channel Controller
  One of these exists per channel, and controls that channel specifically. Each channel has its own list of events, and set of "layers" that calculate light levels for that channel.

• Layer
  These are what behaviors are currently called. They return a value between 0.0 and 1.0. They can be added in an array. These contain segments chained together to create certain effects (previews, or lunar effects), or are built to cover the whole 24-hour cycle.

• LayerSegment
  A single segment of a layer. By definition, a segment starts and end at specific times of day, and represents a specific segment of time where a particular change (or non-change) occurs.

As always, what happens when the preview mode is used? That tends to break a lot when designing the controller.

A neat feature for the scheduler would be support for Lunar Cycles to apply a % multiplier to the user's ramp based on the approximate phase of the moon.

Example:
User decides that between 9pm and 9am, any light levels they pick should be subject to the lunar cycle. They run a night light level until about 11pm, and a morning light level starting at 7am. The lunar cycle would act as an override behavior, using the existing ramp, but applying the mask to it. So if the moon is half full, the light should be 50% of what the user set.

One trick here is to determine if it should affect brightness or luminosity. Luminosity is probably more correct, but requires that the math be done on gamma-corrected values, while so far we only use brightness outside the PWM module. The PWM module owns the conversion between brightness and luminosity by applying the gamma correction to it.

Thoughts around that are that we expose the ability to set either brightness or luminosity on the PWM channel. It is convenient, but doesn't really save a conversion back to brightness to be stored on the PWM channel object. The behavior could convert to luminosity, do the math, and convert back to brightness before setting the information on the PWM channel. Not sure it matters much which way we go here.

The second trick is how to implement it. If implemented as an overriding Behavior object, then it needs to lookup the existing LightBehavior, and mimic it's interval and results. It also needs to handle dealing with multiple objects at the same time. Instead, it's probably better to add support for a "Behavior Plugin" type which is passed to the LightBehavior and is called by it to modify the channel's value during the normal update interval.

Also, keep in mind any sort of blending/ramping that needs to occur at the edges. It's possible that some override behaviors are still needed to go between the masked and non-masked lighting brightness/luminosity.

Timing of updates to the PWM duty cycle is tricky. In part because of how the eye is sensitive to changes.

If the change is too big, then the eye catches it. One aspect is the delta brightness change. The other is the absolute brightness.

RPiLight accounts for the delta brightness, and will update more frequently if the ramp is faster/bigger between two points in time. This is what allows previews to be smooth, or to have a 30 second ramp that doesn't look terrible in the morning if that's what you want.

It doesn't account for the absolute brightness though. So if it decides it only needs to update every 8 seconds when first turning on, it will do that. But in the very low brightness range, you can see the steps happening much more easily, even though it only represents one-tenth of a percent brightness change.

So the algorithm for picking an interval should also account for how close to 0 brightness the channels currently are.

One idea is to use a logistic function to push the interval towards minimum when the brightness is very low, but to these very large multi-second pauses when the brightness is very high.

Since QEMU means we can do ARM builds via Github workflows, we should look at what it would take to be able to automatically package things for users when we tag a release.

Right now the big blocker is that 32-bit Debian Buster (i.e. Raspbian Buster) has some issues with certain things like certificates when run from inside QEMU when the host is 64-bit. Since we want to support Raspbian Buster as that is the current latest stable version, that makes this a no-go for now.

When RPiLight is installed by the package, it causes systemd to think that ‘systemctl daemon-reload’ needs to be run. This step should be put into a postinst script so that users can simply start/restart the daemon without systemd yelling at them.

Something is happening that causes parts of the scheduler to get skipped.

Notes:

• Configuration of the job did happen at the expected time
• The job is still executing as expected, but state has gone wrong

I'll probably need to run the schedule with more logging from apscheduler enabled to catch errors. Odds are there's an exception being thrown, but apscheduler is swallowing it. Fix whatever the exception is and this should go away.

Have a priority queue so multiple behaviors can be active at the same time, but allowing for one behavior with higher priority to override one with a lower priority.

Instead of having multiple behavior jobs, have a single interval job that calls into the current behavior.

This has the effect of quashing some bugs around scheduling because the code was quickly written and doesn't have any good way of resolving conflicts between old a new behaviors.

I caught the following in the logs, which suggests we’ve got some ordering issues here.

I do know what is very likely going on. It’s a queue issue:

• Event fires, queues setBehavior
• Behavior Handler fires, queues clearCurrentBehavior
• setBehavior async
• clearCurrentBehavior async

It can happen this way because when the two timers fire, they add work to the queue. So if they fire simultaneously, they will both be in the queue, one waiting for the other to finish, as expected. But because setBehavor is async, the behavior handler operates on the old behavior, not the new one.

So this leads me to the lesson I should learn here: if letting work get delayed is the plan, ensure I keep a copy of the input state (which clearCurrentBehavior doesn’t do) so it can check it when executed later in the queue.

That or make it so that I don’t have to async setBehavior and clearCurrentBehavior if I’m already in the controller queue.

I think the former option is cleaner.

Aug 05 11:50:55 raspberrypi systemd[1]: Started RPiLight.
Aug 05 11:50:55 raspberrypi RPiLight[6755]: DEPRECATED USE in libdispatch client: Changing target queue hierarchy after object has started executing
Aug 05 11:50:55 raspberrypi RPiLight[6755]: [info][11:50:55.865] LightLevelChangedEvent: { 08:30:00 -0700 -> 12:00:00 -0700 }
Aug 05 11:50:55 raspberrypi RPiLight[6755]: [info][11:50:55.895] Scheduling Next Event @ 12:00:00 -0700
Aug 05 11:50:55 raspberrypi RPiLight[6755]: [info][11:50:55.900] Starting New Behavior
Aug 05 12:00:00 raspberrypi RPiLight[6755]: DEPRECATED USE in libdispatch client: Changing target queue hierarchy after object has started executing
Aug 05 12:00:00 raspberrypi RPiLight[6755]: [info][12:00:00.004] LightLevelChangedEvent: { 12:00:00 -0700 -> 14:00:00 -0700 }
Aug 05 12:00:00 raspberrypi RPiLight[6755]: [info][12:00:00.015] Scheduling Next Event @ 14:00:00 -0700
Aug 05 12:00:00 raspberrypi RPiLight[6755]: [info][12:00:00.022] Starting New Behavior
Aug 05 12:00:00 raspberrypi RPiLight[6755]: [info][12:00:00.024] Ending Current Behavior
Aug 05 12:37:02 raspberrypi systemd[1]: Stopping RPiLight...

Meanwell drivers don't like to take frequencies above 1KHz or so. But I was able to test the MOSFET trigger switches at 2.8KHz, which gave really good results when it came to PWM flicker using the Pi's built-in PWM hardware.

I can personally still see the flicker at 480Hz in the bubbles made by an airstone. Even at 960Hz and 1.92KHz they were visible, but improving. At 2.8KHz I can't tell the difference.

I think there should be a config property in the config.json file that would let the user pick a frequency from a few valid settings.

This is calculated out in issue #48 . The formula used to fix that bug can still contain error of up to about 4 seconds. This means that starting a light change can be delayed by up to about 4 seconds, in practice.

It should be possible to get the error down a bit further. Somehow. Not entirely sure at the moment.

On the Pi Zero, it's possible to hit 100% CPU usage during faster ramps.

The culprit here is two-fold:

1. apscheduler adds a lot of overhead to the work being done. This becomes very noticeable on the Pi Zero, when compared to the Pi 3.
2. The update frequency caps out at 100 times a second. This is a bit excessive, and not really needed.

These combine to create a pretty poor situation for the Pi Zero.

Title says it all.

Cycle: 9pm to 7am
Start Service at 9:30pm

Expected to see the lunar cycle light levels applied, but they aren't.

Now that Moderator natively builds Swift 3 and 4 again, RPiLight should be able to depend on it directly.

Does depend on the maintainer tagging a new release, though.

Right now, the errors hit when the JSON isn't quite right are pretty unhelpful. It would be good to have better errors.

Apple now has Swift argument parsing, so we probably want to use that going forward.

Looks like there's a bug here where this test fails every so often where the behavior gets an opportunity to get called. But not always. Ordering/Timing issues with the dispatch queue, unfortunately.

These two functions do exactly the same thing, but don't do it in the same way. They are different in subtle ways that can lead to bugs.

Made an issue to take the changes made to SwiftyGPIO back to the main repo. Hasn't been taken yet, but assuming uraimo is still around, we want to switch back to using the main repo once things settle down.

It would be handy to be able to create a new schedule, test it, and then run cp testSchedule.json schedule.json to cause the service to reload the schedule.

This would make it possible to run more than 2 channels using the built-in GPIOs. It requires two things:

1. An implementation of PWM-backed DMA in SwiftyGPIO (this can be done in my fork)
2. A driver that uses this implementation in the PWM module of RPiLight (the easy part)

As a proof of concept, let's aim for this using GPIO 22, 23, 24 & 25. This gives us a respectable 4 channels to start with.

Hurdles

The main issue here is that the dimming resolution is limited. With it being about 1 microsecond, we get about 960Hz with 10 bits of dimming, roughly. Usable, but with he Adafruit add on board doing better, mucking with DMA is somewhat low priority.

SwiftyGPIO requires that you tell it where the hardware registers are, by telling it what board you have. RPiLight needs to be able to detect the board it is running on and pass the correct value onto SwiftyGPIO.

Right now the minimum intensity is used to shape the 0-1 scale of intensity on the channel. This isn’t really accurate. Instead it should be used to shape the ramp for segments of the schedule that start or end with a value less than the minimum intensity. The channel itself should simply turn itself off if asked to set the intensity lower than its minimum.