Klipper preprocessor makes saving/uploading from Cura extremely slow on larger files about klipper-preprocessor

[pedrolamas]

HI @milosh-dev, thank you for reporting this, however I have just tried to follow exactly the same steps as you mentioned (downloaded ap1.stl from the link you provided), sliced and saved it in Cura 5.3 in Windows, and I got the file saved in less than 5 seconds!

[milosh-dev]

Thank you!

I was able to track down the issue - it was a combination of the preprocessor script and the script that I was running after the preprocessor (DisplayRemainingTimeOnLCD).

The later script used line indexing (search of the within all the lines) as a reference mechanism to inject M73 commands. For some reason this turned out to be very slow with the gcode file from klipper_estimator, while it was much faster with the original internal data file. However, the script was still functional (smaller files were processed). Therefore it was not immediately obvious that the source of the problem was elsewhere. As a result I wrote my own M73 injection script, which is more capable to work with klipper preprocessor. :)

Anyway, that was not preprocessor issue and I am closing with this comment.

[pedrolamas]

Thanks for the update @milosh-dev, I do wonder if changing the order of the scripts would have helped too (so DisplayRemainingTimeOnLCD before Klipper Preprocessor)

Updated to remove that comment above that didn't make any sense given it has to go AFTER klipper_estimator has done its work!

[milosh-dev]

Yes, running it before klipper_estimator would not make any sense.

In principle I solved it like this (some of this code is custom to my printer) - not utterly elegant, as I open and close the temporary gcode file several times :

`
def execute(self, data):
# 1. Update mesh calculations for delta printer
if self.getSettingValueByKey("local_mesh_enabled"):
_DATA_START_GCODE = 1
_DATA_LAYER_0 = 2

        # Calculate bounds of first layer
        bounds = self.findBounds(data[_DATA_LAYER_0])

        # Fill in bounds in start GCODE
        data[_DATA_START_GCODE] = self.fillBounds(data[_DATA_START_GCODE], bounds)

    ## 2. klipper_estimator
    with TemporaryDirectory() as work_dir:
        filename = os.path.join(work_dir, "work.gcode")
        with open(filename, 'w+') as work_file:
            for line in data:
                work_file.write(line + "\n")

            work_file.close()

            # External funktsioon to run klipper_estimator
            self.execute_klipper_estimator(filename, work_dir)

    ## 3. Process the klipper_estimator result and write it as a separate file
        with open(filename, 'r') as work_file:
            mylines = work_file.readlines()

            # Start from -1
            total_time = -1
            #previous_layer_end_time = 0
            line_index = -1

            tulemus = []

            for line in mylines:
                # Skip empty lines
                if line == '\n':
                    continue
                line_index = line_index + 1
                tulemus.append(line)

                # Layer info
                if self.getSettingValueByKey("add_set_print_stats_info"):
                    if line.startswith(';LAYER:'):
                        tulemus.append("SET_PRINT_STATS_INFO CURRENT_LAYER=%i\n" % (int(line[7:]) + 1,))
                        line_index = line_index + 1
                    elif line.startswith(';LAYER_COUNT:'):
                        tulemus.append("SET_PRINT_STATS_INFO TOTAL_LAYER=%s\n" % (line[13:],))
                        line_index = line_index + 1

                # Time info
                if self.getSettingValueByKey("add_print_time_info"):
                    if (line.startswith(";TIME:") or line.startswith(";PRINT.TIME:")) and total_time == -1:
                        total_time = self.getTimeValue(line)

                        # Percentage
                        tulemus.append("M73 P0\n")

                        # Remaining time
                        total_time = max(total_time, 0)
                        m, s = divmod(total_time, 60)
                        h, m = divmod(m, 60)
                        # Create the string
                        mins = int(60 * h + m + s / 30)
                        tulemus.append("M73 R{}".format(mins)+"\n")

                        line_index = line_index + 2

                    elif line.startswith(";TIME_ELAPSED:"):
                        current_time = self.getTimeValue(line)

                        # Here we calculate remaining time
                        time_left = total_time - current_time
                        time_left = max(time_left, 0)
                        m, s = divmod(time_left, 60)
                        h, m = divmod(m, 60)
                        # Create the string
                        mins = int(60 * h + m + s / 30)
                        tulemus.append("M73 R{}".format(mins)+"\n")
                        line_index = line_index + 1

                        layer_end_percentage = int((current_time / total_time) * 100)
                        output = min(layer_end_percentage, 100)
                                # Now insert the sanitized percentage into the GCODE
                        tulemus.append("M73 P{}".format(output)+"\n")

    # and write everything back
        with open(filename, 'w') as result_file:
            result_file.writelines(tulemus)

        with open(filename) as result_file:
            return result_file.readlines()

`