Timecode Transforms - offset, etc. about timecodekit

[orchetect]

Good to know. These calculations are fairly lightweight, I would not be concerned about generating new instances of Timecode at a fairly high frequency, even on iOS devices. As a struct value-type there is not much overhead.

[ryanfrancesconi]

yes totally - just wanted to clarify what I'm using it for. thanks!

[orchetect]

(Also side note: I think realTime should be renamed to realTimeValue to be consistent.)

[ryanfrancesconi]

i like that! i also like keeping the timecode struct as simple as possible. no real need for a class like above, i think i thought it was going to be more complicated but it ended up working a bit easier than i expected.

my vote is for sure whatever runs the fastest. i'm having to blast these things to the screen so often that optimizing this is very preferable. I'm stoked to see what you decide to do, or if you don't decide to include it, then i think what i have there works ok for now.

It's way better than the way I was doing calculating this before, so very grateful for your nice framework to work with now.

[ryanfrancesconi]

Oh yeah for sure. I have been dealing with it as well the last few years, so like I started out by saying - this is a great project!

I like the transform idea. Sounds very CoreAnimationy

[orchetect]

I'm exploring the implementation of a new struct called Timecode.Transformer which can contain a transformer definition enum (such as .offset(by: Timecode.Delta) and simple have an input Timecode → transform → output Timecode function.

(Timecode.Delta is also a new struct that describes an abstract delta/distance in timecode, with a positive/negative sign and relevant methods to allow flattening down to a concrete Timecode object with various options. Refer to #15 for progress.)

This makes transform functionality modular, scalable (with additional transform types possible in future), and keeps Timecode lightweight when transforms are not needed.

[orchetect]

Yes, the idea of an offset has come up before.

I think what you're doing is fine.

One reason I didn't implement it was that an offset timebase could be anything, depending on how the dev wants to deal with it. It could be seconds, it could be an actual timecode value offset, etc. Therefore deciding what value type the offset property should be stored wasn't immediately obvious. It depends on the time resolution desired. Ostensibly, the most finite resolution for an offset should be 1 subframe. With that in mind, it could be implemented as a TCC() offset.

I'm not sure if it would be much more performant, but you can use instance methods on a single recycled instance of Timecode. So you could store a Timecode instance at a certain frame rate in a variable, then just use .setTimecode(TimeValue(seconds: seconds + offsetSeconds)) on that instance. Then only create a new Timecode instance when you need to use a different frame rate.

[ryanfrancesconi]

Oh that sounds smart. I'll try that and report any difference. Thanks!

[ryanfrancesconi]

No difference in speed between the two methods measured with a simple test case:

    public func timecodeStringFrom(seconds: TimeInterval) -> String {
        let value = TimeValue(seconds: seconds + offsetSeconds)
        masterTimecode.setTimecode(from: value)
        return masterTimecode.stringValue
    }

// VS:

    public func timecodeStringFrom2(seconds: TimeInterval) -> String {
        let timecode = timecodeFrom2(seconds: seconds)
        return timecode?.stringValue ?? ""
    }

    public func timecodeFrom2(seconds: TimeInterval) -> Timecode? {
        let value = TimeValue(seconds: seconds + offsetSeconds)
        return value.toTimecode(at: frameRate)
    }

[ryanfrancesconi]

One reason I didn't implement it was that an offset timebase could be anything, depending on how the dev wants to deal with it. It could be seconds, it could be an actual timecode value offset, etc. Therefore deciding what value type the offset property should be stored wasn't immediately obvious. It depends on the time resolution desired. Ostensibly, the most finite resolution for an offset should be 1 subframe. With that in mind, it could be implemented as a TCC() offset.

I'm using your actual Timecode object for the offset so that when the frameRate changes, the seconds offset can be also recalculated based on the current frame rate. It seems like this could be done and kept up to date internally to the result Timecode if there was an offset object kept in a Timecode object. This means that for me, I'm ultimately storing this offset in preferences as the stringValue version so that it can be reinterpreted. Not really making any statement here other than letting you know how it ended up working for me. I'm not sure if the stringValue is the most portable / reparseable format or not, but that works for this.

[orchetect]

A basic performance measurement, on my system with a Release build using default -Os optimization, creating a Timecode instance from real time takes 0.24 microseconds on my system (averaged from repeating the code 100,000 times in a test harness). Which equates to 4,166,666 instances per second. The setTimecode method is only a hair faster, at 0.22 microseconds.

[orchetect]

I'm using your actual Timecode object for the offset so that when the frameRate changes, the seconds offset can be also recalculated based on the current frame rate. It seems like this could be done and kept up to date internally to the result Timecode if there was an offset object kept in a Timecode object.

My initial idea was to use a Timecode instance for the offset property, but suggested using TCC() since it is more lightweight and if we are assuming the offset will always be at the same frame rate as its parent Timecode object, it can save a few CPU cycles internally. If it were stored as Timecode you could have an offset expressed in a different timecode but I can't imagine this would ever be useful.

struct Timecode {
    var offset: Timecode.Components? = nil
}

Being optional defaulting to nil can also save some CPU cycles internally when there is no offset value.

You could still employ using a Timecode object externally to derive the value, if it made sense.

var tc = "01:00:00:00".toTimecode(at: ._24)

tc?.offset = "00:00:01:00".toTimecode(at: ._24)?.components // default to nil if this fails (no offset value)

But it would usually be easier just to assign the TCC directly.

var tc = "01:00:00:00".toTimecode(at: ._24)

tc?.offset = TCC(s: 1)

I would also add a defaulted offset argument to all the Timecode initializers so you could pass an offset value at time of creating the object.

[orchetect]

This means that for me, I'm ultimately storing this offset in preferences as the stringValue version so that it can be reinterpreted. Not really making any statement here other than letting you know how it ended up working for me. I'm not sure if the stringValue is the most portable / reparseable format or not, but that works for this.

Storing as a stringValue is not a bad idea, as long as you store the frame rate associated with it of course.

As far as storage in a file, if you need to store Timecode then stringValue and frameRate (and optionally subFrameDivisor if you care about subframes) should be reliable. Or if it's not too difficult, instead of stringValue you can store the Int values of .components but would not offer much advantage.

I would be a bit hesitant to work out a codable/decodable implementation that converts it to a binary/base64 format then back again because it may become increasingly difficult to maintain as the library evolves.

[orchetect]

One consideration is that the offset should allow for positive or negative offsets. Technically, timecode wraps around the clock (whether traditional 24-hour mode, or allowing for days). Not sure how that would best be described in an offset if TCC() is used as the value type.

Idea 1: separate properties

struct Timecode {
    var offset: Timecode.Components? = nil
    var offsetPositive: Bool = true
}

Idea 2: tuple

struct Timecode {
    var offset: (offset: Timecode.Components, positive: Bool)? = nil
}

Idea 3: enum cases

extension Timecode {
    enum OffsetValue {
        case positive(Timecode.Components)
        case negative(Timecode.Components)
    }
}

struct Timecode {
    var offset: OffsetValue? = nil
}

Idea 4: allow for negative timecode components

tc?.offset = TCC(s: -1)

Having two separate properties (idea 1) is a more traditional method but may lead to some confusion on how to effect a negative offset.

The advantage of tuple (idea 2) or enum case (idea 3) is that the polarity of the offset is tightly coupled to the offset value itself, and thus becomes more self-evident to the developer.

I like negative TCC values (idea 4). It seems fairly simple and intuitive. But adds some complexity if the dev needs to convert positive timecode units into negative ones to translate a timecode offset into a negative offset.

[ryanfrancesconi]

in terms of a stringValue i guess it could be shown as:
-01:00:00:00

should there just be an optional Bool sign in the Components for positive or negative that applies to the whole object?

it seems like it would be sort of confusing if each component could be positive or negative?

[orchetect]

in terms of a stringValue i guess it could be shown as:
-01:00:00:00

I'd like to refrain from introducing more complexity in this way. A stringValue is just one representation of time/timecode duration for purposes of an offset. This would have to also translate to TCC, real time, samples, etc. which I think would be more confusing. So I'm leaning toward a boolean flag attached to the offset value, but not actually a part of the offset value itself.

should there just be an optional Bool sign in the Components for positive or negative that applies to the whole object?

Interesting idea but again this may introduce unneeded complexity or confusion, as TCC is used for many applications throughout TimecodeKit. You are able to represent negative values in TCC already, as described below:

it seems like it would be sort of confusing if each component could be positive or negative?

Not if it's representing values for the purposes of an operation, such as offset. Technically this is already possible with the library:

These produce the same outcome. Add -1 second, or subtract +1 second

"01:00:00:00"
    .toTimecode(at: ._24)?
    .adding(TCC(s: -1))?
    .stringValue // == "00:59:59:00"

"01:00:00:00"
    .toTimecode(at: ._24)?
    .subtracting(TCC(s: 1))?
    .stringValue // == "00:59:59:00"

In this manner, these methods are actually already doing the work of performing an offset. We're just talking about making an actual offset property which makes this sort of thing automatic.

[ryanfrancesconi]

On a practical note, I'm unsure how negative offset would be used in the way that I'm using it. For me, sometimes videos have an embedded timecode start, and that's used to make a relative timecode which basically just renames what zero means. Just wanted to clarify what I am using this offset for. Probably not new information in anyway, but here are a few examples:

In Logic it looks like:

And Pro Tools:

Premiere calls it Start Time:

Final Cut:

[orchetect]

Yes, but there can be many other scenarios where the possibility of a negative offset for a timecode may be useful. I'm not talking about your use case specifically. I just want to ascertain the best way to implement the offset in the event that there is a negative offset desired, in the best interest of offering the broadest generalized use applications.

[orchetect]

Here is another consideration:

If an offset is added, would it affect all representations emitted by the Timecode instance transparently? Or just some of them? And how would that be done intuitively and not confusing?

ie:

The Timecode instance would still be initialized or modified via .set... instance methods using the non-offset timecode values as usual, regardless of what value format is used (string, real time, samples, etc.).

But when an offset value exists, the return (get) value of all value types would be rendered offset?

• .stringValue
• .realTime
• .samples
• .components
• the Timecode instance properties (.days, .hours, .minutes, .seconds, .frames, .subframes)

When setting any of those properties, you would probably still set them with original timecode. It would be a bit strange if you had to set them with offset timecode values. So then you end up with non-symmetry between get and set on those values. Which I feel would just be confusing.

You would also not be able to poll the Timecode object for any of its original time values in case you needed to, unless you set .offset back to nil so that getting values would be reset back to original values.

Possible solution 1:

Implement as described above. When an offset is present, all initializers or setters would set original timecode values as usual, but all getters would produce offset values.

Possible solution 2:

All computed properties could become methods instead, and then add an argument to the get... methods so you have implicit control over whether the returned values are offset or not. This would require a bit of surgery on the existing API but wouldn't be too much trouble. Not my favorite, because then each property would be broken up into a set and get method instead of being get or set on a single property, which is cleaner.

So this API (currently):

extension Timecode {
    var components: Timecode.Components {
        get { ... }
        set { ... }
    }
    var stringValue: String {
        get { ... }
        set { ... }
    }
    var realTimeValue: TimeValue {
        get { ... }
        set { ... }
    }
    var samplesValue: Double {
        get { ... }
        set { ... }
    }
}

becomes this:

extension Timecode {
    func getComponents(offset: Bool = true) -> Timecode.Components { ... }
    func getStringValue(offset: Bool = true) -> String { ... }
    func getRealTimeValue(offset: Bool = true) -> TimeValue { ... }
    func getSamplesValue(offset: Bool = true) -> Double { ... }
    
    func setComponents( ... ) { ... }
    func setStringValue( ... ) { ... }
    func setRealTimeValue( ... ) { ... }
    func setSamplesValue( ... ) { ... }
}

It may mean that the Timecode instance properties (.days, .hours, .minutes, .seconds, .frames, .subframes) remain as original values regardless of get or set.

[ryanfrancesconi]

Perhaps it warrants a new object that encapsulates two Timecode structs? The main timecode, and an optional offset one.

I think the sets would interact only on the main timecode, but there could be new getters or properties that provide the offset result? I think you could argue pretty well that this is more of an application level request as for my application, I did it that way:

// a thin wrapper to store a frameRate for calculations
// and stores an offset to be applied to all incoming values
public class TransportTimecode {
    public static var defaultFrameRate: Timecode.FrameRate = ._25

    public private(set) var masterTimecode = Timecode(at: defaultFrameRate)

    public var frameRate: Timecode.FrameRate = TransportTimecode.defaultFrameRate {
        didSet {
            Log.debug("⏰ frameRate to", frameRate.stringValue)
            frameRateFloatValue = frameRate.floatValue

            masterTimecode = Timecode(at: frameRate)

            if let timecodeOffset = timecodeOffset {
                var newOffset = Timecode(at: frameRate)
                if newOffset.setTimecode(exactly: timecodeOffset.stringValue) {
                    self.timecodeOffset = newOffset
                }
            }
        }
    }

    // don't calculate this more than necessary
    public private(set) var frameRateFloatValue: Float = TransportTimecode.defaultFrameRate.floatValue

    private var _timecodeOffset: Timecode?
    public var timecodeOffset: Timecode? {
        get {
            _timecodeOffset
        }
        set {
            _timecodeOffset = newValue
            offsetSeconds = newValue?.realTime.seconds ?? 0

            Log.debug("⏰ timecodeOffset", _timecodeOffset, "in seconds:", offsetSeconds)
        }
    }

    private var offsetSeconds: TimeInterval = 0

    public func timecodeStringFrom(seconds: TimeInterval) -> String {
        let value = TimeValue(seconds: seconds + offsetSeconds)
        masterTimecode.setTimecode(from: value)
        return masterTimecode.stringValue
    }
}

[ryanfrancesconi]

but of course, the above is really just used to generate a running clock that might not start on 0. That is the only time I need to use this offset, in a timeline that has a virtual start time.

Underneath that the time is still in fractions of a second starting at 0.

[orchetect]

Clever solution and something to chew on. I appreciate your input on this.

I'd be tempted to make it a struct still, channeling those progressive Swifty design patterns 😁 Trying to get away from concurrent mutability, and being thread-safe as much as possible.

It was worth woodshedding the details on an internal implementation of an offset in Timecode but atomically it may be best to keep Timecode as lightweight/performant as possible. Any ancillary functionality could be app-side extensions or encapsulations as you've presented here so the dev can elect to add abstraction only when needed.

Wouldn't be out of the question to have such an abstraction included in the library. Or a suite of "Transforms" that could be wrappers for Timecode, one of which could be a simple offset.

[orchetect]

Thanks! After so much confusing information and outright misinformation (usually just naïve) out on the web about fine details of various frame rates and how to perform calculations with them, I decided to combine everything and make a library that brought everything to one place and was easy to use.

I'll keep this thread open and might bounce some ideas around regarding ideas surrounding transform wrappers at some point.

[orchetect]

Introduced Timecode.Transformer in release 1.1.0: https://github.com/orchetect/TimecodeKit/releases/tag/1.1.0