NewStartConnectionTransferImpl Prefix about lifecycle-rebalance-revisited

[algernon-A]

Yes (assuming 'transpiler'), it's planned (will increase priority). Will do it with Harmony 2 migration as the transpiler bugfixes over 1.2.0.1 are highly desirable.

Could possibly add a quick workaround in the meantime, possibly with prefix chaining - is there a specific case you want to implement now?

[MacSergey]

I need to redefine each call Singleton<VehicleManager>.instance.GetRandomVehicleInfo(...) in my mod

https://steamcommunity.com/sharedfiles/filedetails/?id=2069057130
https://github.com/MacSergey/NoBigTruck

I make it with transpiler, but now, if your mod enable, after calling your prefix, the original method is not executed and my changes are ignored

[algernon-A]

Okay, I see what you're trying to do. I don't know how long it'll take me to get the transpiler together - as you can see, my (well, technically mostly WG's) changes aren't trivial.

As an interim, could you expose an interface in your mod that I could call from mine? That way all I need to do is check to see if your mod is installed and then call either the original or your version - a quick fix that should do until I can get the transpiler working.

[algernon-A]

I believe it's also technically possible for you to target your transpiler against my Prefix - that might be worth a look.

[MacSergey]

I thought that my transpiler would correct your prefix, but I did not know and did not check if this would work. I did not understand in detail how the harmony works, whether it will depend on the order in which the game loads the libraries: first mine or yours first

[algernon-A]

It shouldn't really matter, as all mod static methods would be loaded via OnEnabled. My method is static (well, it has to be, being a Prefix), and so will definitely be there. If you apply your transpiler on OnLevelLoaded, that's an extra level of safety since my Prefix is applied on OnCreated().

[MacSergey]

I looked in more detail at what changes you are making and realized that they all go in one block. Therefore, it seems to me that the transpiler will not be difficult to do. just put your change code in a separate method and call it instead of executing unchanged code. I could try to do it if you don't mind

[algernon-A]

Oh, yes, that's a good point - I hadn't thought of that. I'm a bit flat out for the next day or so, so if you could take a look, that'd be great!

[algernon-A]

Note that it will need to take values off the stack at the start and leave them on at the end, as it's a truely inline statement that interacts with variables in the method before and after the segment that's patched.

[MacSergey]

It seems like it happened. Here is the patch code. I didn’t replace it in your code myself, I’ll leave it for you

[HarmonyPatch]
public static class StartConnectionTransferImplPatch
    {
        const int educationVarIndex = 2;
        const int numVarIndex = 3;
        const int iVarIndex = 16;
        const int flag4VarIndex = 22;

        public static MethodBase TargetMethod() => AccessTools.Method(typeof(OutsideConnectionAI), "StartConnectionTransferImpl");

        public static IEnumerable<CodeInstruction> Transpiler(MethodBase original, IEnumerable<CodeInstruction> instructions, ILGenerator generator)
        {
            Debug.Log($"LifecycleRebalance Transpiler start");

            var ageArray = generator.DeclareLocal(typeof(int[]));
            var childrenAgeMax = generator.DeclareLocal(typeof(int));
            var childrenAgeMin = generator.DeclareLocal(typeof(int));
            var minAdultAge = generator.DeclareLocal(typeof(int));

            var instructionsEnumerator = instructions.GetEnumerator();
            var instruction = (CodeInstruction)null;
            var forStartFounded = false;
            var left = 1;

            //find for start
            while (instructionsEnumerator.MoveNext() && left > 0)
            {
                instruction = instructionsEnumerator.Current;
                Debug.Log(instruction.ToString());

                yield return instruction;

                if (forStartFounded)
                    left -= 1;
                else if (instruction.opcode == OpCodes.Stloc_S && instruction.operand is LocalBuilder builder && builder.LocalIndex == iVarIndex)
                {
                    forStartFounded = true;

                    //set additional local variable value
                    yield return new CodeInstruction(OpCodes.Ldc_I4_0);
                    yield return new CodeInstruction(OpCodes.Stloc_S, childrenAgeMax.LocalIndex);

                    yield return new CodeInstruction(OpCodes.Ldc_I4_0);
                    yield return new CodeInstruction(OpCodes.Stloc_S, childrenAgeMin.LocalIndex);

                    yield return new CodeInstruction(OpCodes.Ldc_I4_0);
                    yield return new CodeInstruction(OpCodes.Stloc_S, minAdultAge.LocalIndex);

                    yield return new CodeInstruction(OpCodes.Ldloc_S, numVarIndex);
                    yield return new CodeInstruction(OpCodes.Call, AccessTools.Method(typeof(StartConnectionTransferImplPatch), nameof(StartConnectionTransferImplPatch.GetAgeArray)));
                    yield return new CodeInstruction(OpCodes.Stloc_S, ageArray.LocalIndex);
                }

            }

            //save first for operation lable
            var startForLabels = instructionsEnumerator.Current.labels;

            //skip
            do
            {
                instruction = instructionsEnumerator.Current;
                Debug.Log($"SKIP {instruction}");
            }
            while ((instruction.opcode != OpCodes.Stloc_S || !(instruction.operand is LocalBuilder builder && builder.LocalIndex == flag4VarIndex)) && instructionsEnumerator.MoveNext());

            //call changes method block
            yield return new CodeInstruction(OpCodes.Ldloc_S, iVarIndex) { labels = startForLabels };
            yield return new CodeInstruction(OpCodes.Ldloc_S, educationVarIndex);
            yield return new CodeInstruction(OpCodes.Ldloc_S, ageArray.LocalIndex);
            yield return new CodeInstruction(OpCodes.Ldloca_S, childrenAgeMax.LocalIndex);
            yield return new CodeInstruction(OpCodes.Ldloca_S, childrenAgeMin.LocalIndex);
            yield return new CodeInstruction(OpCodes.Ldloca_S, minAdultAge.LocalIndex);
            yield return new CodeInstruction(OpCodes.Call, AccessTools.Method(typeof(StartConnectionTransferImplPatch), nameof(StartConnectionTransferImplPatch.Changes)));

            while(instructionsEnumerator.MoveNext())
            {
                instruction = instructionsEnumerator.Current;
                Debug.Log(instruction.ToString());
                yield return instruction;
            }

            Debug.Log($"LifecycleRebalance Transpiler complite");
        }

        public static void Changes(int i, Citizen.Education education, int[] ageArray, ref int childrenAgeMax, ref int childrenAgeMin, ref int minAdultAge)
        {
            int min = ageArray[0];
            int max = ageArray[1];

            if (Debugging.UseImmigrationLog)
            {
                Debugging.WriteToLog(Debugging.ImmigrationLogName, $"{nameof(i)}={i};{nameof(childrenAgeMin)}={childrenAgeMin};{nameof(childrenAgeMax)}={childrenAgeMax};{nameof(minAdultAge)}={minAdultAge};{nameof(min)}={min};{nameof(max)}={max};");
            }

            if (i == 1)
            {
                // Age of second adult - shouldn't be too far from the first. Just because.
                min = Math.Max(minAdultAge - 20, DataStore.incomingAdultAge[0]);
                max = Math.Min(minAdultAge + 20, DataStore.incomingAdultAge[1]);
            }
            else if (i >= 2)
            {
                // Children.
                min = childrenAgeMin;
                max = childrenAgeMax;
            }

            // Calculate actual age randomly between minumum and maxiumum.
            int age = Singleton<SimulationManager>.instance.m_randomizer.Int32(min, max);

            // Adust age brackets for subsequent family members.
            if (i == 0)
            {
                minAdultAge = age;
            }
            else if (i == 1)
            {
                // Restrict to adult age. Young adult is 18 according to National Institutes of Health... even if the young adult section in a library isn't that range.
                minAdultAge = Math.Min(age, minAdultAge);

                // Children should be between 80 and 180 younger than the youngest adult.
                childrenAgeMax = Math.Max(minAdultAge - 80, 0);  // Allow people 10 ticks from 'adulthood' to have kids
                childrenAgeMin = Math.Max(minAdultAge - 178, 0); // Accounting gestation, which isn't simulated yet (2 ticks)
            }

            if (i < 2)
            {
                // Adults.
                // 24% different education levels
                int eduModifier = Singleton<SimulationManager>.instance.m_randomizer.Int32(-12, 12) / 10;
                education += eduModifier;
                if (education < Citizen.Education.Uneducated)
                {
                    education = Citizen.Education.Uneducated;
                }
                else if (education > Citizen.Education.ThreeSchools)
                {
                    education = Citizen.Education.ThreeSchools;
                }
            }
            else
            {
                // Children.
                switch (Citizen.GetAgeGroup(age))
                {
                    case Citizen.AgeGroup.Child:
                        education = Citizen.Education.Uneducated;
                        break;
                    case Citizen.AgeGroup.Teen:
                        education = Citizen.Education.OneSchool;
                        break;
                    default:
                        // Make it that 80% graduate from high school
                        education = (Singleton<SimulationManager>.instance.m_randomizer.Int32(0, 100) < 80) ? Citizen.Education.TwoSchools : education = Citizen.Education.OneSchool;
                        break;
                }
            }

            if (Debugging.UseImmigrationLog)
            {
                Debugging.WriteToLog(Debugging.ImmigrationLogName, "Family member " + i + " immigrating with age " + age + " (" + (int)(age / 3.5) + " years old) and education level " + education + ".");
            }
        }

        public static int[] GetAgeArray(int num) => num == 1 ? DataStore.incomingSingleAge : DataStore.incomingAdultAge;
    }

[algernon-A]

Excellent, thanks! I'll test this out over the weekend.

[MacSergey]

Are there any test results?

[algernon-A]

Cautiously optimistic, but not conclusive (but I'm pretty sure that there was a confounding factor). I've just restarted testing after upgrading to Harmony 2; we'll see how this goes.

[MacSergey]

I really did and test with Harmony 2

[algernon-A]

Issue was mine, not yours :-) Running now and looking good. Calculations match and about to run output through regressions while I do demographic analysis.

[algernon-A]

Committing to 1.4 BETA branch.

[algernon-A]

Calculated and logged education levels are not transferring to actual immigrants.

[algernon-A]

Calculated and logged ages are not transferring to actual immigrants.

[algernon-A]

Replaced with original patch and confirmed issues unique to transpiler. Correct values aren't being properly left on stack.

[MacSergey]

what variables you mean?

[MacSergey]

You mean public static void Changes calls and log inside it correct, but after return the values are not correct?

[algernon-A]

Yes, at least for education2 and age.

[algernon-A]

Specifically, they seem to be set to zero, so all immigrants end up as newborn babies with no education, regardless of what was calculated.

[algernon-A]

It's late for me here - I'll have another look at it tomorrow and see if I can figure out exactly what's going on.

[MacSergey]

I see an inaccuracy, now I’ll fix it

[MacSergey]

try it. I did not see the values of education2 and age variables need return from change method

[HarmonyPatch]
public static class StartConnectionTransferImplPatch
{
    const int educationVarIndex = 2;
    const int education2VarIndex = 21;
    const int numVarIndex = 3;
    const int iVarIndex = 16;
    const int flag4VarIndex = 22;
    const int ageVarIndex = 20;

    public static MethodBase TargetMethod() => AccessTools.Method(typeof(OutsideConnectionAI), "StartConnectionTransferImpl");

    public static IEnumerable<CodeInstruction> Transpiler(MethodBase original, IEnumerable<CodeInstruction> instructions, ILGenerator generator)
    {
        //Debug.Log($"LifecycleRebalance Transpiler start");

        var ageArray = generator.DeclareLocal(typeof(int[]));
        var childrenAgeMax = generator.DeclareLocal(typeof(int));
        var childrenAgeMin = generator.DeclareLocal(typeof(int));
        var minAdultAge = generator.DeclareLocal(typeof(int));

        var instructionsEnumerator = instructions.GetEnumerator();
        var instruction = (CodeInstruction)null;
        var forStartFounded = false;
        var left = 1;

        //find for start
        while (instructionsEnumerator.MoveNext() && left > 0)
        {
            instruction = instructionsEnumerator.Current;
            //Debug.Log(instruction.ToString());

            yield return instruction;

            if (forStartFounded)
                left -= 1;
            else if (instruction.opcode == OpCodes.Stloc_S && instruction.operand is LocalBuilder builder && builder.LocalIndex == iVarIndex)
            {
                forStartFounded = true;

                //set additional local variable value
                yield return new CodeInstruction(OpCodes.Ldc_I4_0);
                yield return new CodeInstruction(OpCodes.Stloc_S, childrenAgeMax.LocalIndex);

                yield return new CodeInstruction(OpCodes.Ldc_I4_0);
                yield return new CodeInstruction(OpCodes.Stloc_S, childrenAgeMin.LocalIndex);

                yield return new CodeInstruction(OpCodes.Ldc_I4_0);
                yield return new CodeInstruction(OpCodes.Stloc_S, minAdultAge.LocalIndex);

                yield return new CodeInstruction(OpCodes.Ldloc_S, numVarIndex);
                yield return new CodeInstruction(OpCodes.Call, AccessTools.Method(typeof(StartConnectionTransferImplPatch), nameof(StartConnectionTransferImplPatch.GetAgeArray)));
                yield return new CodeInstruction(OpCodes.Stloc_S, ageArray.LocalIndex);
            }

        }

        //save first for operation lable
        var startForLabels = instructionsEnumerator.Current.labels;

        //skip
        do
        {
            instruction = instructionsEnumerator.Current;
            //Debug.Log($"SKIP {instruction}");
        }
        while ((instruction.opcode != OpCodes.Stloc_S || !(instruction.operand is LocalBuilder builder && builder.LocalIndex == flag4VarIndex)) && instructionsEnumerator.MoveNext());

        //call changes method block
        yield return new CodeInstruction(OpCodes.Ldloc_S, iVarIndex) { labels = startForLabels };
        yield return new CodeInstruction(OpCodes.Ldloc_S, educationVarIndex);
        yield return new CodeInstruction(OpCodes.Ldloc_S, ageArray.LocalIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, childrenAgeMax.LocalIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, childrenAgeMin.LocalIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, minAdultAge.LocalIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, education2VarIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, ageVarIndex);
        yield return new CodeInstruction(OpCodes.Call, AccessTools.Method(typeof(StartConnectionTransferImplPatch), nameof(StartConnectionTransferImplPatch.Changes)));

        while (instructionsEnumerator.MoveNext())
        {
            instruction = instructionsEnumerator.Current;
            //Debug.Log(instruction.ToString());
            yield return instruction;
        }

        //Debug.Log($"LifecycleRebalance Transpiler complite");
    }

    public static void Changes(int i, Citizen.Education education, int[] ageArray, ref int childrenAgeMax, ref int childrenAgeMin, ref int minAdultAge, out Citizen.Education resultEducation, out int resultAge)
    {
        int min = ageArray[0];
        int max = ageArray[1];

        if (Debugging.UseImmigrationLog)
        {
            Debugging.WriteToLog(Debugging.ImmigrationLogName, $"{nameof(i)}={i};{nameof(childrenAgeMin)}={childrenAgeMin};{nameof(childrenAgeMax)}={childrenAgeMax};{nameof(minAdultAge)}={minAdultAge};{nameof(min)}={min};{nameof(max)}={max};");
        }

        if (i == 1)
        {
            // Age of second adult - shouldn't be too far from the first. Just because.
            min = Math.Max(minAdultAge - 20, DataStore.incomingAdultAge[0]);
            max = Math.Min(minAdultAge + 20, DataStore.incomingAdultAge[1]);
        }
        else if (i >= 2)
        {
            // Children.
            min = childrenAgeMin;
            max = childrenAgeMax;
        }

        // Calculate actual age randomly between minumum and maxiumum.
        resultAge = Singleton<SimulationManager>.instance.m_randomizer.Int32(min, max);

        // Adust age brackets for subsequent family members.
        if (i == 0)
        {
            minAdultAge = resultAge;
        }
        else if (i == 1)
        {
            // Restrict to adult age. Young adult is 18 according to National Institutes of Health... even if the young adult section in a library isn't that range.
            minAdultAge = Math.Min(resultAge, minAdultAge);

            // Children should be between 80 and 180 younger than the youngest adult.
            childrenAgeMax = Math.Max(minAdultAge - 80, 0);  // Allow people 10 ticks from 'adulthood' to have kids
            childrenAgeMin = Math.Max(minAdultAge - 178, 0); // Accounting gestation, which isn't simulated yet (2 ticks)
        }

        resultEducation = education;
        if (i < 2)
        {
            // Adults.
            // 24% different education levels
            int eduModifier = Singleton<SimulationManager>.instance.m_randomizer.Int32(-12, 12) / 10;
            resultEducation += eduModifier;
            if (resultEducation < Citizen.Education.Uneducated)
            {
                resultEducation = Citizen.Education.Uneducated;
            }
            else if (resultEducation > Citizen.Education.ThreeSchools)
            {
                resultEducation = Citizen.Education.ThreeSchools;
            }
        }
        else
        {
            // Children.
            switch (Citizen.GetAgeGroup(resultAge))
            {
                case Citizen.AgeGroup.Child:
                    resultEducation = Citizen.Education.Uneducated;
                    break;
                case Citizen.AgeGroup.Teen:
                    resultEducation = Citizen.Education.OneSchool;
                    break;
                default:
                    // Make it that 80% graduate from high school
                    resultEducation = (Singleton<SimulationManager>.instance.m_randomizer.Int32(0, 100) < 80) ? Citizen.Education.TwoSchools : Citizen.Education.OneSchool;
                    break;
            }
        }

        if (Debugging.UseImmigrationLog)
        {
            Debugging.WriteToLog(Debugging.ImmigrationLogName, "Family member " + i + " immigrating with age " + resultAge + " (" + (int)(resultAge / 3.5) + " years old) and education level " + education + ".");
        }
    }

    public static int[] GetAgeArray(int num) => num == 1 ? DataStore.incomingSingleAge : DataStore.incomingAdultAge;
}

[algernon-A]

Made a couple more tweaks and seems to have solved that issue - starting proper regression testing now.

[algernon-A]

Do you have a copy of the stack analysis?

[MacSergey]

I don’t understand what you mean. List of IL instructions?

[algernon-A]

Stack index calculations, to confirm which variables are where on the stack to start with and what should be left on the stack to finish with.

[MacSergey]

everything that I put on the stack, I pick up the next one of the following instructions. Therefore, after the changes, the stack remains exactly the same as before the changes

                //set additional local variable value
                yield return new CodeInstruction(OpCodes.Ldc_I4_0); //put 0 to stack
                yield return new CodeInstruction(OpCodes.Stloc_S, childrenAgeMax.LocalIndex); //take 0 from the stack

                yield return new CodeInstruction(OpCodes.Ldc_I4_0); //put 0 to stack
                yield return new CodeInstruction(OpCodes.Stloc_S, childrenAgeMin.LocalIndex);//take 0 from the stack

                yield return new CodeInstruction(OpCodes.Ldc_I4_0); //put 0 to stack
                yield return new CodeInstruction(OpCodes.Stloc_S, minAdultAge.LocalIndex);//take 0 from the stack

                yield return new CodeInstruction(OpCodes.Ldloc_S, numVarIndex); //put num value to stack
                yield return new CodeInstruction(OpCodes.Call, AccessTools.Method(typeof(StartConnectionTransferImplPatch), nameof(StartConnectionTransferImplPatch.GetAgeArray)));//take num value from stack and put method result to stack
                yield return new CodeInstruction(OpCodes.Stloc_S, ageArray.LocalIndex);//take method result from stack and write to ageArray variable

//Put 8 values to stack
yield return new CodeInstruction(OpCodes.Ldloc_S, iVarIndex) { labels = startForLabels };
        yield return new CodeInstruction(OpCodes.Ldloc_S, educationVarIndex);
        yield return new CodeInstruction(OpCodes.Ldloc_S, ageArray.LocalIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, childrenAgeMax.LocalIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, childrenAgeMin.LocalIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, minAdultAge.LocalIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, education2VarIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, ageVarIndex);
//Take 8 values from stack and nothink put to stack
        yield return new CodeInstruction(OpCodes.Call, AccessTools.Method(typeof(StartConnectionTransferImplPatch), nameof(StartConnectionTransferImplPatch.Changes)));

[algernon-A]

Let me put it another way: I'm after the calculations for these, to make sure I've got it right:

const int educationVarIndex = 2;
const int education2VarIndex = 21;
const int numVarIndex = 3;
const int iVarIndex = 16;
const int flag4VarIndex = 22;
const int ageVarIndex = 20;

[MacSergey]

This follows from the decompiled code. These are not indices on the stack, they are indices of local variables. They are static and immutable.

	// int num = 0;
	IL_000a: ldc.i4.0
	IL_000b: stloc.3 // 3 - num variable index

	// for (int i = 0; i < num; i++)
	IL_076a: ldc.i4.0
	IL_076b: stloc.s 16 // 16 - i variable index

[algernon-A]

Ahh, gotcha - I was barking up the wrong tree. That's what I needed, thanks.

[MacSergey]

When I add new local variables, they are placed at the end of the list of local variables

        var ageArray = generator.DeclareLocal(typeof(int[]));
        var childrenAgeMax = generator.DeclareLocal(typeof(int));
        var childrenAgeMin = generator.DeclareLocal(typeof(int));
        var minAdultAge = generator.DeclareLocal(typeof(int));

After that, I can get their indices in the list of local variables through .LocalIndex

        yield return new CodeInstruction(OpCodes.Ldloc_S, ageArray.LocalIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, childrenAgeMax.LocalIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, childrenAgeMin.LocalIndex);
        yield return new CodeInstruction(OpCodes.Ldloca_S, minAdultAge.LocalIndex);

[MacSergey]

I looked at version 1.4. You forgot to remove Prefix. It also continues to be called.

[MacSergey]

I was mistaken, I did not unsubscribe from 1.3.7

[algernon-A]

So far, it's looking good from my end. The logs are filling up with [NoBigTruck] VehicleSelected: 1686317262.VW Crafter 35 LWB Cargo_Data (VehicleInfo) etc.

[algernon-A]

Confirmed fixed by 713645c.