Installation fails on Ubuntu 22.04. about kiri

[leoheck]

Yes, I could figure out exactly what you are saying, in summary, this is happening.

1. Kicad does not create folders. They have to exist otherwise when you save the project, it is going to throw an error about permissions and not about the existing path. This may not be a bug, but I consider it a bug. Especially because the Error is not that good. Now, this can be easily overcome by creating the folder modules manually before of saving the project.

2. The new page is created in the place where the parent page is. And this definitely sucks. Easy to overcome, but it sucks anyway. It looks like an amateur decision if not a bug.

maybe some explanation is in order where a module inside a...

Agree, but you don't have to explain this to me. I understand that the user should use it the way he likes most. I just stated before that this is not what most people do. But in general, they just do what others do, and they are not trying new things as you are.

[T0jan]

yes I saw this while testing. it really helps to keep on overview on the modules. espicially in the way I normally name my files.

In the last days I did not found any more issues so I will close this for now. If I find something more, I will surely come back to you.

[leoheck]

Thanks for reporting this.

I run Kiri on my Ubuntu 22.04 by default and I have no problems while installing dependencies. But since you are reporting this, I am going to check. I will setup a new installation with the VirtualBox it is going to take some time to prepare and have an idea of this issue.

Meanwhile, how are you running this? Do you have any other info to add?

[leoheck]

I was able to install dependencies on a clean Ubuntu 22.04 as you can see the final part of the process.

Opam/Ocaml related packages are a pain, indeed.
Maybe you have an previous installations that is not kind of working.
I would recommend you to remove opam and plotgitsh/kicadsh first.

You can start fresh by doing this

# Remove opam itself
sudo apt purge opam

# Remove opam-related stuff from your user, manualy
rm -rf $HOME/.opam

Then you can run the installation script again.
KiRi script should install opam version 2.1.2

Now, I could find an issue with the wxWidgets but it is not important for KiRI, only for Kicad-Diff if you want to run it manually with GUI. Otherwise, it will work just fine on the command line.

But sure, I have to check how to fix this. But again, this is not a big deal for you, since even failing to install wxWidgest the install_depencencies script finishes.

[T0jan]

hmm. I first thought starting from a clean install works as the dependency install runs through without any problems now. So I definitely had a problem with my configuration before. but still the kiri installation itself fails because it searches for plotkicadsch but can't find it. looking on the installation list of opam it also seems way to short compared to your list above. any ideas?

for the wxWidgets error you got: I had also a problem there. it seems the gtk+3.0 packages needed for compiling wxWidgets are not installed per default on ubuntu.

sudo apt-get install build-essential libgtk-3-dev

did the trick for me there.

log for the kiri install now:

[email protected]:~$ bash -c "$(curl -fsSL https://raw.githubusercontent.com/leoheck/kiri/main/install_dependencies.sh)"
Hit:1 http://de.archive.ubuntu.com/ubuntu jammy InRelease
Hit:2 http://de.archive.ubuntu.com/ubuntu jammy-updates InRelease              
Hit:3 http://de.archive.ubuntu.com/ubuntu jammy-backports InRelease            
Hit:4 https://ppa.launchpadcontent.net/kicad/kicad-6.0-releases/ubuntu jammy InRelease
Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
7 packages can be upgraded. Run 'apt list --upgradable' to see them.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
git is already the newest version (1:2.34.1-1ubuntu1.2).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
libgmp-dev is already the newest version (2:6.2.1+dfsg-3ubuntu1).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
pkg-config is already the newest version (0.29.2-1ubuntu3).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
opam is already the newest version (2.1.2-1).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
python-is-python3 is already the newest version (3.9.2-2).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
python3-pip is already the newest version (22.0.2+dfsg-1).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
kicad is already the newest version (6.0.5-0-202205030842+a6ca702e91~116~ubuntu22.04.1).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
dos2unix is already the newest version (7.4.2-2).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
coreutils is already the newest version (8.32-4.1ubuntu1).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
zenity is already the newest version (3.42.1-0ubuntu1).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
dune is already the newest version (2.9.1-1build1).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
scour is already the newest version (0.38.2-2).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
librsvg2-bin is already the newest version (2.52.5+dfsg-3).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
imagemagick is already the newest version (8:6.9.11.60+dfsg-1.3build2).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
xdotool is already the newest version (1:3.20160805.1-4).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pillow>8.2.0 in ./.local/lib/python3.10/site-packages (9.1.1)
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: six>=1.15.0 in /usr/lib/python3/dist-packages (1.16.0)
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: dateutils>=0.6.12 in ./.local/lib/python3.10/site-packages (0.6.12)
Requirement already satisfied: python-dateutil in ./.local/lib/python3.10/site-packages (from dateutils>=0.6.12) (2.8.2)
Requirement already satisfied: pytz in /usr/lib/python3/dist-packages (from dateutils>=0.6.12) (2022.1)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil->dateutils>=0.6.12) (1.16.0)
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: python_dateutil>=2.8.1 in ./.local/lib/python3.10/site-packages (2.8.2)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python_dateutil>=2.8.1) (1.16.0)
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pytz>=2021.1 in /usr/lib/python3/dist-packages (2022.1)
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pathlib>=1.0.1 in ./.local/lib/python3.10/site-packages (1.0.1)
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: wxpython>=4.0.7 in ./.local/lib/python3.10/site-packages (4.1.1)
Requirement already satisfied: pillow in ./.local/lib/python3.10/site-packages (from wxpython>=4.0.7) (9.1.1)
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from wxpython>=4.0.7) (1.16.0)
Requirement already satisfied: numpy in /usr/lib/python3/dist-packages (from wxpython>=4.0.7) (1.21.5)

<><> Updating package repositories ><><><><><><><><><><><><><><><><><><><><><><>
[default] no changes from https://opam.ocaml.org
[NOTE] Package digestif is already installed (current version is 1.1.2).
[NOTE] Package lwt is already installed (current version is 5.5.0).
[NOTE] Package lwt_ppx is already installed (current version is 2.0.3).
[NOTE] Package cmdliner is already installed (current version is 1.1.1).
[NOTE] Package base64 is already installed (current version is 3.5.0).
[NOTE] Package sha is already installed (current version is 1.15.2).
[NOTE] Package tyxml is already installed (current version is 4.5.0).
[NOTE] Package git-unix is already installed (current version is 3.9.1).
[email protected]:~$ bash -c "$(curl -fsSL https://raw.githubusercontent.com/leoheck/kiri/main/install_kiri.sh)"

Installing KiRI

Installation path ~/.local/share
Change it using KIRI_INSTALL_PATH environment variable

Hit ENTER to continue or Ctrl+C to leave.


cp: cannot stat '../kiri': No such file or directory
environment: line 44: cd: ~/.local/share/kiri/submodules/plotkicadsch: No such file or directory

If you need anymore info on my system let me know.

[leoheck]

Thanks for letting me know about the missing dependencies.

[leoheck]

Alright, it looks like it was not that hard to fix this.
The install_kiri script was fixed and also the README.md has a fix in the command used to call this script

This is the fixed command that you have to use.

bash -c "INSTALL_KIRI_REMOTELLY=1; $(curl -fsSL https://raw.githubusercontent.com/leoheck/kiri/main/install_kiri.sh)"

If the problem is fixed, please, close the issue here. Otherwise, let me know. Cheers!

[T0jan]

Thanks for dealing with this so quickly. The installation works now completely and I can run kiri as before. The display of the changes in the PCB seems to work most of the time.

However the changes in the schematics seems not to work for me. After the web interface launches I can see the diff of the top sheet of the two last commits. None of the subsheets are available to select in the pages list. After I change the selection of commits also the top sheet is not displayed anymore. If I hover above the only available item int the pages list I see the following:

This looks not really correct for me. I checked the files in the .kiri folder and it looks like all images of subsheets are created correctly but are simply not displayed. The bug occurs in firefox and chromium.

During creation of the files via kiri-command I also get the errors, that kicad can't open the subsheets as often as they are present in commits.

Do you want to track this problem in this issue or should I create a new one for this?

[leoheck]

We can continue here. Is it possible to share your project so I can check what is the issue?

Also, run it with this command to try find the reason it is failing

kiri project.kicad_pro -D -r

It will remove previous runs and also show extra messages.

If it fails on a command you can go inside the folder it is running (indicated mostly in top of the command) and the run the command there manually to check what is going on.

Edit: Command fixed, it should be uppercase d: -D

[leoheck]

Also, checking the image, there may be some extra whitespaces somewhere. Your project also has schematic files inside a subfolder which is not a common thing then Kiri may have been failing on this too.

[T0jan]

Is it possible to share your project so I can check what is the issue?

Sure, I created the project just for this purpose as I can't share my other designs where the same problem persists:

test_project.zip

The zip also contains the kiri files the last run has created if you wan't to check them.

Also, run it with this command to try find the reason it is failing

you probably meant the -D option, right?

log.zip

I think it runs through without any problems but if you need me to call a command somewhere, feel free to tell me.

Also, checking the image, there may be some extra whitespaces somewhere. Your project also has schematic files inside a subfolder which is not a common thing then Kiri may have been failing on this too.

I don't think it is a problem with whitespaces, as the problem exists in all of my projects. I think it has way more to do with the structure. If it was not programmed to work with sheets in a subfolder then this is probably the big problem here. I don't know if it is uncommon but I have to say everyone I worked with puts their subsheets in extra folders either for module persistence across different projects or simply due to the fact that the root folder gets otherwise quite crowded if you have more then a handfull sheets in your design.

Also interesting sidenote: the successful running of kiri changes my keyboard layout to the english layout. This is probably unintended?

[leoheck]

Yes, sorry, the command is uppercase D.

Alright, the project you are sending is not a valid project.
It has to be a git repo, with at least 2 commits to compare, if I am not mistaken. Or maybe with a single commit, considering that you have local changes to be added.

So, transforming the repo as mentioned above, I could run it, take a look:

Focusing on changes in the schematics

Nothing to say about the PCB diff, it is working as you mentioned. Just sharing here for completion.

But I could reproduce your issue.

1. The tooltip issue on the schematics page is still there, as you showed me.
2. Using arrow keys makes the images disappear (at least on the schematic page), which may be related to the wrong tooltip name.

Ok, I could find an issue on Kiri. It was not a problem with whitespaces. My test project does not have modules, then I may have forgotten this. I've pushed a fix. It may work better now.

Also interesting sidenote: the successful running of kiri changes my keyboard layout to the english layout. This is probably unintended?

Intereseting. haha, maybe this command is doing this setxkbmap us

[T0jan]

Definitely some progress as the diff for the top sheet now works as expected and also the sheets can be selected individually. However I still get errors during the creation of the files when kiri opens kicad and does the exports. Can you check if you get also errors when you run kiri on my project (now with full git history and a log of my last kiri run included)?

git-tests-with-kicad.zip

For every subsheet there is nothing plotted, they are all just empty sheets in the html view.

[leoheck]

The sub sheet was added in the last commit but there is no modules/subsheet.kicad_sch folder and file neither in the zip file or in the .git versioning.

If you open your project with Kicad you will see this:

Here is the subsheet details modules/subsheet.kicad_sch

The .zip does not have this subsheet. It is missing from the .git versioning too. See, this is all I got.

But I found a bug where the Page Name (and Filename in the tooltip) were having the spaces removed. It is fixed now.

[leoheck]

If I add a submodule like you want, then I am having another issue with whitespace... I am thinking how I can fix this one, still.

[leoheck]

Fixed, hope it works for you too.

[leoheck]

let me know if you see other issues, please

[T0jan]

ah man sorry for the missing module, I forgot git add there. still weird that it wasn't in the zip anyway, it was on my side. It works now with the subsheets in a subfolder also on my side 👍

But I discovered an additional problem as I was running the latest version on my projects: There is an issue if modules are used inside other modules. Looking into the kiri files it seems it does not pull these modules to the commit history created by kiri and so kicad can|t open them. I updated the test-project to reflect this problem, I hope I managed to catch everything this time 😅

git-tests-with-kicad.zip

[leoheck]

No problem, I am discovering bugs with this, which is cool.

Now, modules inside modules, this is the first time I am seeing this. I am pro for organization, but I have never done or seen someone doing that.

Well, let me but let me see your zip.

[leoheck]

Solving this will be hard, but lets see if it is doable.

[leoheck]

Well, this is getting hard since I believe Kicad may have a bug. Also, your .kicad_sch files all have the same UUID. It is not important for me, but it looks like they were not created by Kicad but replicated manually. This is not important for me, but I am having a hard time trying to detect which file belongs to the project and which one does not. Also, the way Kicad is storing files looks like a bug or it is crazy bad. Since the first page shows something like modules/my_1st_module.kicad_sch and the second page, which is already inside the modules folder, shows the module like my_2nd_module.kicad_sch. It should show the module as modules/my_2nd_module.kicad_sch because this second module is inside the modules folder too. In other words, it should show all the modules relative to the root of the project, and not to the folder they currently are. This behavior is crazy. I am trying to work around this, but it is getting pretty hard, haha.

Also, there is something wrong with this hierarchical navigator. It shows pages 1, 2, 3 and then it jumps to 32, 33, 34... I am not sure if this is because your project was kind of manually created or if this is another bug in Kicad. See:

[T0jan]

Yes I had to work with copy and paste and modifying the subsheets via editor as kicad was really buggy in reassigning the modules and crashed way to often. So sorry for the messed up file structure there. I assure you the problem persists also in projects where you create/link the modules normally inside kicad. I hope 6.0.6 will get a bit more stable...

regarding the problem with the modules/ in the sheet link: It is like this since at least kicad4 (where I started using it). I think kicad itself always uses the path of the currently opened sheet instead of the root sheet. I think this is a bit weird as well but if you know it, you can work around it.

maybe some explanation is in order where a module inside a module can be quite useful: I worked on designs where a lot of UART to rs422 interfaces where needed but the MCU only had three UARTs available or so. So I used a multiplexer to split up the UARTs and connected them to the RS422 drivers right in the multiplexer module. ofc I also could have put them outside the multiplexer sheet on the top sheet but if you have 4 multiplexer circuits with 4 RS422 drivers each your top sheet gets crowded very quickly.

[leoheck]

Hey @T0jan I am going to give it a try to implement this. Could you share a working project with nested modules so I can use it as a second example? It does not need to be a git repo. And it has to have valid pages, if possible. You could use a copy of an existing project, and you can also remove most parts of the circuits in the schematics... just leave something on each sheet (as you've done before) so I can visualize and make changes to test it with Kiri.

[T0jan]

@leoheck sure, I actually remembered that my old bachelor thesis PCB can be used here:

Helmholtz-Ansteuerung.zip

If something is missing, please let me know. the symbols are not really up to standards but this should not matter for this purpose here.

[T0jan]

btw, not sure if this is really a useful idea but maybe it could be an option to generate a diff for each schematic kiri finds in the project (sub-)folders on its own instead of triggering it from the main schematic view?

[leoheck]

Thanks for sharing your project.

btw, not sure if this is really a useful idea but maybe it could be an option to generate a diff for each schematic...

Is this a new idea or, it is related to this one, it looks like you are talking about a new thing or maybe i did not understand yet the whole idea? If this is a new idea, could you open a new issue? so we can discuss there better.

This issue here is pretty complex already.
Kicad is a nightmare for parson sheets and sheets instances.

[T0jan]

Is this a new idea or, it is related to this one, it looks like you are talking about a new thing or maybe i did not understand yet the whole idea? If this is a new idea, could you open a new issue? so we can discuss there better.

no it was an idea how to deal with the whole nested file structure problem. in my understanding you now create the export of the whole schematic (so main sheet + eventual sub-sheets) from the main sheet and as the main sheet does not carry the correct information for sub-sheets in folders the problems start here. so my idea now was: create instead an export of each sheet in the project folder and sub-folder on its own and continue from there.

[leoheck]

Yeah, this is exactly what I will do, and I am kind of doing this already.

Now, the problem is the lack of organization in the Kicad project, so this information is stored split in many different places.

Also, when I export things using plotgitsch, the exported .svg carries the name of the exported page. Which is what I want. But, now, when I use xdotool/cliclick, plotting graphically with Kicad 6, the generated .svg files are named with a mix of hierarchical place and the instance names. This is a problem because Kiri's javascript that displays images, is presenting unique files, and not instances to avoid duplication of the same sheet. Now, I have to track where the instance name is, with the page filename and its path that I have to reconstruct myself all using Kicad project since I don't think it is good to trust on what I can find on git, since users may have made mistakes.

[leoheck]

@T0jan could you share your Helmholtz-Ansteuerung.zip the project with the history, or maybe share the URL, if it is in github.

[leoheck]

I could push some code to be able to use Kiri with multiple nested projects that use Kicad 6.
Could you please, test if it works for you?
https://github.com/leoheck/kiri/tree/nested-hierrarchical-sheets

It works with my projects, and also my nested multipage test project.

However, for some reason it did not work with your git-tests-with-kicad.zip and I don't know why yet but I believe that project is a bit broken, unfortunately, I don't know what is happening.

[leoheck]

Ah, if you are inside the project, you can use "kiri -r" it does not need to pass the filename anymore.

[T0jan]

hmm sadly it does not seem to work for me. for each commit I get at least one error, always in the form:

python3: can't open file '/home/{$USERNAME}/.local/share/kiri/bin/{$USERNAME}.py': [Errno 2] No such file or directory

log.txt

In the browser view there is now again only one sheet, the main sheet, displayed and selectable.

Maybe I did somethin wrong in switching to the branch version? I just switched the branch in the main kiri install folder, is there something else I have to do?

Concerning the test project: best thing is to leave it alone, it seems I have broken something there in the way I created it via editor.

Concerning the Helmholtz-Cage: I wont be able to provide you with a history there for the next few days as I have no access to the git server due to covid...

[leoheck]

the error looks strange.. especially the variables

{$USERNAME} --> they should be ${USERNAME}

but I have no clue where this is... do you?
I don't think it is in my files, well I will check but maybe you have to check your configs too..

To run the new thing, It will need a bit more work than changing branches.

You also need to run, from the root path of Kiri's repo

git submodule update --init --recursive

There is a new submodule and this submodule has also a submodule...

[leoheck]

I also made 2 new repos to test these things.
If it does not work for you, you could test these repos too.
So we can talk to check how to improve your repo.
https://github.com/leoheck/multisheet-kicad-5
https://github.com/leoheck/multisheet-kicad-6

[leoheck]

Hi, I saw one path I was using was hardcoded. Please, update your branch before trying it again.

[T0jan]

hi,

{$USERNAME} --> they should be ${USERNAME}

sorry, my fault, I used search and replace there to make the log better readable and switched the bracket and the dollar sign there.

Hi, I saw one path I was using was hardcoded. Please, update your branch before trying it again.

hi, I did a pull on the branch and also the submodule update but the error persists with my own projects as well as the the projects you shared above.

It seems like it's searching for a python file with my username in the kiri folder that can not be right or is it?

[leoheck]

Hm, in this case it looks like a setup issue in your side, still.

It runs with the project you gave me, except that it does not show extra pages since they were not well generated. But that is fine.

Also, It is working fine here on my Linux box (I could also tested it on Windows and on macOS).

You could ...

# Remove the current kiri repo
rm -rf kiri

# Clone it again
gh repo clone leoheck/kiri

# Change branches
cd kiri
git checkout nested-hierrarchical-sheets

# Make sure submodules are in place
git submodule update --init --recursive

# Update Kiri home environement variable 
export KIRI_HOME=$(pwd)

And this should work,

[T0jan]

no changes after I did step by step what you wanted me to do. The error stays the same. I still think:

python3: can't open file '/home/${USERNAME}/.local/share/kiri/bin/${USERNAME}.py': [Errno 2] No such file or directory

is the error here. In the steps I see nothing which should create a python file with my username as filename in the git repo so this is probably the problem here?

[leoheck]

If you grep recursivelly all files inside the Kiri folder looking for this string, like this:

cd kiri
grep USERNAME -R

... you are not going to find any file with the USERNAME string since most of the time the correct variable is ${USER} and not ${USERNAME}. Then I still think it is a configuration issue on your side. Maybe check the setup in the ~/.bashrc or ~/.zshrc

Check if the configuration on those files is something like this, please

# Kiri environment setup
eval $(opam env)
KIRI_HOME=/home/lheck/Documents/kiri
export PATH=${KIRI_HOME}/submodules/KiCad-Diff/bin:${PATH}
export PATH=${KIRI_HOME}/bin:${PATH}

[T0jan]

... you are not going to find any file with the USERNAME string

yes I know that I won't find it there. the point is, your code is searching for this file...

Check if the configuration on those files is something like this, please

yes, my ~/.bashrc looks like this (adjusted to my kiri path ofc)

[leoheck]

I could reproduce this issue on a macOS. I could fix the issue there, and also maybe for you too.
You could update everything by running these 2 lines again.

bash -c "$(curl -fsSL https://raw.githubusercontent.com/leoheck/kiri/main/install_dependencies.sh)"
bash -c "INSTALL_KIRI_REMOTELLY=1; $(curl -fsSL https://raw.githubusercontent.com/leoheck/kiri/main/install_kiri.sh)"

[T0jan]

YES! great work, we definitely moved a step ahead now!

But there is still an issue remaining, I think it is related to a timeout set somewhere:

Plotting a92eef2, ${PRJDIR}/.kiri/a92eef2/
   timeout 15 plot_kicad_sch project.kicad_sch 

In the html-view all modules are listed now but if you scroll through them not all are actually displayed. some of them only got an empty screen. I assume they are not getting plotted due to the timeout striking in? Looking in the debug output I at least saw no other warning or error which could explain why some pages are not displayed.

[leoheck]

Are you running with kiri -r -D ?
It a bad previous run was done before, it wont generate artifacts in a new run. You have to use -r to start it clean.

[T0jan]

yeah kiri -r -D gave me the warning above. kiri -r runs without any warning (obviously), running kiri afterwards does not change anything on the output (as expected) and I also tried kiri -r -D -t 10 to save some time but the result was the same there as well.

I just tried with the same project on a other PC (ubuntu 20.04) and got the same result there.

[T0jan]

I also checked the kiri files just now: the sch_sheets which is located directly in the .kiri folder lists all sheets in the project. but for each commit the sch_sheets only lists a handful (always the same) or none at all. what is a bit curios for me is, that these handful sheets it seems to detect are not the first five sheets of the list in the .kiri folder.

[leoheck]

kiri -r -D -t 10 to save some time...

Yeah, that is good, I use -t 3 sometimes when debugging...

This ./kiri/sch_sheets is my try to have something quick working.. but I should do something on-the-fly with javascript, to update sheets and layers, considering only projects on the selected commits. It does not do that yet and I also have to investigate how to do that.

What is missing for you? Schematics, Layouts, or both?
Your repos are Kicad 6 or 5?

Ah, maybe it is missing the submodules..

Try to go inside kiri's repo and then update submodules with

git submodule update --init --recursive

You can also use the master branch now since I've merged the other one.

[T0jan]

What is missing for you? Schematics, Layouts, or both?

I miss the schematic plots for most of my modules, layout seems to be fine. if I run kiri -r -D -t 10 it actually also does not list the missing modules in the page list but only the handful from before. but the detection of subsheets seems to work as two of the displayed schematics are submodules.

Your repos are Kicad 6 or 5?

A mix. They started with k5 but are now converted to k6. with kiri -r -D -t 10 I only catch k6 commits.

Ah, maybe it is missing the submodules..

I don't think so as I did a clean fresh install on the ubuntu 20.04 PC. with missing submodules none of the plotting would have worked, right?

I also tried to reproduce the problem with the the test project but was not successful until now. maybe its the mix of k5 and k6 commits which leads to problems here?

[T0jan]

ok I can say it is not the mix of k5 and k6. I just created a new repo with just k6 commits of the project and I still get the output for only 5 sheets. I will see if and how I can reproduce the error with a shareable test project.

[leoheck]

Which files/images are missing?
Images of schematics, layout, or what?

When the commit has Kicad 6 project, Kiri uses xdotool on Linux to plot schematics, which is a graphical tool... this may be the issue, sometimes.

Now, if the commit has Kicad 5 files, Kiri plots schematics using plotkicadsh/plotgitsch. If it is installed, it mostly works, but it
requires repo having cache-lib file which sometimes is missing since people use to forget to commit it.

Now, for layout, on both Kicad 6 and 5 versions, it uses Kicad-Diff, which may work better than plotting schematics.

[T0jan]

The images of most of the schematic sheets for kicad6. as far as I can tell it is dependent from the project because for the test project every schematic is plotted.

I still suspect this timeout above as part of the problem, is there a way to increase it?

[T0jan]

Ok I found the reason for the sheets not being plotted. it is not the timeout but you probably won't like it either:

sheets with spaces (and probably other non standard signs, I did not test this) in the sheet-name do not get plotted. see the F with spaces sheet in the test project:

multisheet-kicad-6.zip

[leoheck]

I see, Kiri should work with these things (maybe not out of the box). I did the test with folders having spaces, but not with the project files having spaces. I will check and fix it if there is something wrong.

[T0jan]

it's not the file naming! it's how the subsheets are named inside kicad, the sheet name in kicad terminology.

I can say it already worked with the spaces in the names before you added the support for the second+ level sheets.

[leoheck]

You mean, the sheet instance.

This is the script that I have to fix

find_kicad6_sch_hier multipage-kicad.kicad_sch

You can run it in your computer to see the results...

[leoheck]

It calls this one, that looks it is working already, see.

➜  multisheet-kicad-6 git:(main) kicad6_sheet_instances multipage-kicad.kicad_sch
0999d842-9bc9-4370-bae4-7c28aeba0392|A|sch/a.kicad_sch
19432d09-e376-42a6-8b10-a54a895963d1|F with spaces|sch/f.kicad_sch
6483d8c3-df4d-40b4-b616-c993cba432d4|A1|sch/a.kicad_sch

[leoheck]

I think that adding this IFS=$'\n' on line 50 of this find_kicad6_sch_hier fixes this issue. At least it worked here. Could you try it?

Current

		sch_instances=($(kicad6_sheet_instances "${current_sch}"))

Fix

		IFS=$'\n'
		sch_instances=($(kicad6_sheet_instances "${current_sch}"))

[leoheck]

Result

[leoheck]

Or try this branch here where I change layers and sheets dynamically for every commit. This may be better for the consistency of the commits view. I put the fix of schematics instances with spaces there.

dynamic-sheets-and-layers

[T0jan]

YES! IT'S WORKING! 🥳

thank you so much for fixing it for all my unusual side cases and for keeping patience with me for the last few weeks. is there any way I can spend you a coffee or so via paypal for all the time you invested?

interestingly KiCad now forbids the usage of / in the instance name so all my designs using it are now broken, but that's nothing you have to worry about 😅

[leoheck]

Nice! It will definitely have some bugs still and I appreciate your efforts to share the issues you were/are having.

Considering the tip jar, I have this one here. I would love to drink a coffee or a beer bought by you. Cheers!
https://www.paypal.com/donate/?hosted_button_id=HT74DFNLH2TLQ

[leoheck]

Ah, please, close this issue if there is nothing else related. Otherwise if you find any other issue you can open a new one so we can start fresh. haha

[leoheck]

The branch dynamic-sheets-and-layers also improves the navigation of multiple sheets. you may like those changes too.

• it tries to keep the current page selected when commits change, whenever possible.
• it also sorts sheets alphabetically, for each hierarchical level

[leoheck]

By the way, thank you for the big coffee. That was nice and kind.

[leoheck]

Hey, @T0jan do you mind testing the latest code here: https://github.com/leoheck/kiri/tree/dynamic-sheets-and-layers?

• The svg-pan-zoom was refactored to work better.
• I improved the multi-sheet/layers to avoid changing these lists when commits have the same pages/layers
• It was added new shortcuts to pan with the keyboard (alt+arrow keys)
• It was improved zoom with keyboard shortcuts too (+/-/0/f)

interestingly KiCad now forbids the usage of / in the instance name so all my designs using it are now broken, but that's nothing you have to worry about sweat_smile

Could you talk more about this issue here that you have described before? I did not understand what you want to say. I mean, I don't understand how this can be an issue.

If you think this is a limitation with kicad, then you can run a command to rewrite git history fixing your instance names and replacing this character with something else.. I could help you with that if you are having problems with this.

[T0jan]

Hey, @T0jan do you mind testing the latest code here: https://github.com/leoheck/kiri/tree/dynamic-sheets-and-layers?

sure, no problem. I checked with two of my projects and nothing seems to got broken by the changes which is a big plus :D

the new hotkeys work fine and I am actually very happy about them, so thanks for this!

I just found two small issues. one is with the documentation: it seems Kicad-Diff changed its file structure. the path export now needs to be:

export PATH=${KIRI_HOME}/submodules/KiCad-Diff/bin:${PATH}

for the system to find the kidiff-binary.

the other one is with the displaying of the results but here I am not sure f it is a problem with the hardware I run kiri on today: when changing between schematic and layout view, the screen does not get updated. so if I switch from schematic to layout for example I still see the latest schematic view and vice versa. I have to change the selected commits or switch layers/pages first for the right page to be shown. the problem does not occur when I switch between layers/sheets or commits.

I also have a feature request: it would be nice if one could hide the commits and/or layers list in the inspector or if one can get the current viewed page in full screen.

Could you talk more about this issue here that you have described before? I did not understand what you want to say. I mean, I don't understand how this can be an issue.

If you think this is a limitation with kicad, then you can run a command to rewrite git history fixing your instance names and replacing this character with something else.. I could help you with that if you are having problems with this.

Well its not really an issue its more like kicad changed things between v5 and v6 and now some old things do not work anymore. I think this has to be expected between major versions.

in this case I named my instances something like Mux/Demux before but as kicad6 now doesn't allow / in instance names anymore it causes some weird side effects in exporting etc which also affect what kiri can display.

My point was that you don't need to fix anything on kiris side to fix this, as this is an heritage issue and border case which should not occur in the future.

If you have an command sequence to change the instance names in the git history it would be nice as I would then not start to search for it once I really need it :D

[leoheck]

export PATH=${KIRI_HOME}/submodules/KiCad-Diff/bin:${PATH}

Got it, it is already fixed, thanks.

====

the other one is with the displaying of the results but here I am not sure f it is a problem with the hardware I run kiri on today: when changing between schematic and layout view, the screen does not get updated. so if I switch from schematic to layout for example I still see the latest schematic view and vice versa. I have to change the selected commits or switch layers/pages first for the right page to be shown. the problem does not occur when I switch between layers/sheets or commits.

This issue was introduced in the last commits. I had to rewrite the way images were being displayed. I have not seen this happening on my Google-Chrome, but I definitely need to investigate this deeply. Nice you found a way to work around this for now. Does it happen always, or just with the first 2 selected commits?

====

I also have a feature request: it would be nice if one could hide the commits and/or layers list in the inspector or if one can get the current viewed page in full screen.

The fullscreen is an interesting feature, since the whole interface can be used with keyboard shortcuts.
I will check if I can make it work.

====

Regarding the instance names. I will try to find a solution for your instance names. Let me see if I can do something.

[leoheck]

Hey, good news, it was easy to implement this fullscreen view. It is good that is true fullscreen. It lacks some things like displaying important info so we know which commits and sheet/layers are being displayed. But this is secondary and can be added later..

I also have to create this info window that teaches the shortcuts that are increasing in number...

You can try it yourself, upgrading the branch you are already testing.
The shortcut to toggle it is z

[leoheck]

Actually, it is not z anymore, the shortcut i is working now, to show a window with shortcuts. you are going to find that the new shortcut for fullscreen is f

[leoheck]

Hey @T0jan

I think I could make a command for you to fix your repos.

Download this script sed_instance_names, and put it in the PATH, so it will be available to your command line.
It finds all .kicad_sch and .sch files and then executes sed to update the instance names.

The syntax is

sed_instance_names REGEXP_TO_MATCH REPLACEMENT

You can run it inside your current branch to check if it is working,

For example, this appends an underscore in front of each sheet instance name

sed_instance_names '^' '_'

Then this removes the previously added underscore

sed_instance_names '^_' ''

================

Now, make a copy of your repo to run it, just in case you have unsaved stuff.
Go inside this repo...

Then run the following command

# Just to make sure the next command won't complain about unsaved work.
git stash 

# The following command executes sed_instance_names on each commit, it may take some time... 

# Example of how to remove the `/` in front of each instance
git filter-branch --tree-filter "sed_instance_names '^/' '' " -f

# Example of how to replace all the instances of `/` that appear in the instance name with `-`
git filter-branch --tree-filter "sed_instance_names '/' '-' " -f

================

How do you check if it worked?

1. You are going to see the output of the command in the command line.

2. You can run kiri -r and click the green link on the commits to launch Kicad on each commit, to check visually there in the Eeschema

3. You can find and cat all sch_sheets that kiri has created find -name sch_sheets | xargs cat

================

If you are fine with the result then you just have to push force rewriting the history in upstream...

git push -f

Hope it helps you to fix your repos.

[T0jan]

Actually, it is not z anymore, the shortcut i is working now, to show a window with shortcuts. you are going to find that the new shortcut for fullscreen is f

Very nice, the fullscreen is working beautiful and all shortcuts except (i) also work like expected. for (i) it opens the information in the background of the fullscreen and only shows it after one goes back to the window-view. don't know if there is a fix for this but I would consider this a minor problem.

There is however a shortcut clash in firefox: alt + left / alt + right are going back / forward in firefox.

I have not seen this happening on my Google-Chrome, but I definitely need to investigate this deeply. Nice you found a way to work around this for now. Does it happen always, or just with the first 2 selected commits?

I happens for all possible commit combinations but only if you switch between schematic and layout with the buttons. if you use the shortcut (s) the page is immediately correctly displayed. I checked it for firefox and chromium now on two different machines and it always happens.

I also just noticed that the same problem occurs if one wants to change the displayed schematic sheet via mouse and not with the shortcuts. so clicking on another sheet then the one currently displayed will not update the page. interestingly it is not the same for the layout. here the clicking only does not work until other commits are selected or the layer was changed by shortcut. after this you can select a layer via mouse and it is correctly displayed.

thx for the script in advance, I will check it a bit later.