boost::filesystem::path diverged from std::filesystem::path about filesystem

[Trafo]

From my perspective the boost::filesystem::path is broken in many ways. I wanted to fix the whole thing, but there was no interest in my fixes. (the fixes are quite easy and I guess that are bugs because they look like classical bugs) The only answer I received was, that boost is not std and that somebody is working on changes anyways. And I did not want to work for something, what is build only for the glory. At the end, I wrote my own wrapper to fix all kinds of these bugs. And now I try to use std::filesystem because clang and gcc seem to support it finally as well. And std::filesystem::path works very well.

[Lastique]

In general, "foo/bar/" URI path, with the trailing slash, indeed refers to the location inside the "bar" directory, so the interpretation "foo/bar/." seems correct to me. Also, an empty filename doesn't look valid to me. The URI path "foo/bar", without the trailing slash, identifies the "bar" file. I'm not very familiar with the standard rules or the rationale behind them, so I can't comment on whether the behavior for std::filesystem described in the OP is correct and required by the standard.

[pdimov]

When Boost.Filesystem was written, there were no C++ "standard rules" yet, so it can't have violated them.

For foo/bar/, I suspect the reason it normalizes to foo/bar/. and not to foo/bar is that the two are not the same when bar is a symbolic link. foo/bar refers to the link, and foo/bar/. refers to the directory to which bar points.

This behavior was probably changed as part of the standardization, and Boost.Filesystem has not been updated to match.

[Lastique]

Boost.Filesystem predates std::filesystem by many years, obviously some design decisions were different when Boost.Filesystem was initially written. You can find some insight on design decisions in the library documentation, although this may not necessarily answer your questions.

I think, @Beman intended to update Boost.Filesystem for better compatibility with the standard library but didn't have the time to do it. In the long run, I think, better compatibility is desired, but not an absolute requirement if there are reasons for the difference.

[Trafo]

When Boost.Filesystem was written, there were no C++ "standard rules" yet, so it can't have violated them.

I know that. And sorry if i choose to offensive words. What i mean is, that i find the standard rules much better and i was wondering why it is different to boost. You have a point with the symbolic link. But i really see no advantages with that, only disadvantages. (look to #76) And what is with this example : fs::path("/foo/./bar/baz").lexically_normal() == "/foo/bar/baz". Here we could have the same arguments. If foo is a symlink to "a/b/c", this could points to a/b/c/bar/baz. But like the method name suggest, the normalization is lexically. And because of that symlinks should not matter.

But their still a lot of things inconsistent. For example fs::path("/").filename() == "/". Because "/" is not a valid filename.

@Lastique
So it makes no sense, if i open a pull request, if somebody else is already working with another vision then me?

In general, "foo/bar/" URI path, with the trailing slash, indeed refers to the location inside the "bar" directory, so the interpretation "foo/bar/." seems correct to me.

I disagree here. A Normalization function should return the shortest path to a object inside of the filesystem. For this purpose Normalization functions should exist. And "foo/bar/." reference to bar what is equal to "foo/bar". But second path is shorter. Even in case of a symlink, "foo/bar" is referencing to the same place like "foo/bar/."

Also, an empty filename doesn't look valid to me.

Okay, but it is much better then '/'. And if boost::filesystem::path would be consistent, it should return '.'. Or did i miss something here?

[pdimov]

Yes, the standard does mandate that filename() is "" for foo/bar/: http://eel.is/c++draft/fs.path.decompose#7

[Trafo]

Another question. Is it right, that you can not calculate the distance between boost::filesystem::path::iterator (directly because std::distance(end(), begin()) will fail)? Or that no operator< is defined even we have the m_pos value? But the m_pos value is not accessible from the outside. Or did i miss something and boost have a special iterator treatment?

[Trafo]

Their are so much more wrong states from my point of view. So my question again, make it sense, if i rewrite path class? Or is it a waste of time for me, because other people already work on it. Because i found other scary things, or other components are used to the wrong output? It should not be to complicated. Because writing tests for the expected behaviour and then go for it.

After normalization -> C:\..\..\foo\bar could happens. Or boost::fs::path(".hidden").extension() == ".hidden" what is wrong from my point of view. And i found a lot of other things, i not remember anymore.

[EDIT]
Try to collect other issues here:
boost::fs::path("C:\").parent() = "C:" (not valid path)
boost::fs::path("/").parent() = "" (making a absolute path to an empty relative path)
boost::fs::path("/").filename() = "/"
boost::fs::path("//").filename() = "//"
boost::fs::path("///").filename() = "/"
boost::fs::path("foo/bar//").filename() = "/" (i already found the issue. It is in filename_pos)

I found the issue inside of the filename_pos method. For me it seems again, that the previous coder did that on purpose. But why? I try at the weekend to open a pull request. It will my first big contribution. Did you have some recommendations for me?

[gri1n]

I just thought I'd chime in and say that I really hope boost::filesystem::path never adopts an implicit conversion to string operator like the c++ standard has.

[jothepro]

I considered using boost::filesystem as a polyfill for std::filesystem, but the implementation looks broken to my untrained eyes:

// boost 1.72.0
auto testfolder = boost::filesystem::path("//testfolder").lexically_normal(); // testfolder = "//testfolder"
auto relativePath = testfolder.lexically_relative("/"); // relativePath = ""

I'm not super deep into filepath-parsing but this seems wrong to me in every possible way.

In std::filesystem the results make a lot more sense:

// C++17
auto testfolder = std::filesystem::path("//testfolder").lexically_normal(); // testfolder = "/testfolder"
auto relativePath = testfolder.lexically_relative("/"); // relativePath = "testfolder"

I'd also be super interested if this behaviour is there for a reason or if it's a bug that needs fixing. If it is intended then it should be documented, from what the documentation currently says I don't expect this behaviour.
@Trafo what happened to your attemts to change the implementation?

[Lastique]

I believe, the // prefix is interpreted as a UNC path prefix.

[jothepro]

Yeah but it obviously isn't a UNC prefix. But I'm not here to fight about the details, I already figured out that boost::filesystem does not work as polyfill for std::filesystem.

If anybody else reads this because he needs an alternative to c++17 std::filesystem, I found this compatible implementation for older c++-versions: https://github.com/gulrak/filesystem

I've not tested it yet but it sounds pretty promising!

[Trafo]

The UNC misinterpretation is not the only misinterpretation of boost. My favorite is still boost::fs::path(".hidden").extension() == ".hidden". (that is only one example) Even windows, where the starting point does not have any impact, .hidden is not a file extension. And under Unix and macOS it is completely wrong. All these kinds of misinterpretations lead to follow up errors. In this case, it leads only to errors in the case of using replace_extension(...). But for example, in case of finding not the right root_path/root_name, it causes a lot of other follow up failures in many other functions.

But anyway I am over it. :) And the posted repo looks really promising.

[Lastique]

The .hidden filename will be interpreted as a stem (and not an extension) starting with Boost.Filesystem 1.77.

The leading double slashes (forward or backward) are interpreted as a UNC prefix, and this will not change. Triple slashes are interpreted as redundant leading slash (not a UNC prefix followed by a slash). Hence the difference in behavior.

The C: path is a valid path, which refers to the current directory on drive C. Similarly, C:foo is also a valid path that refers to file foo in the current directory for drive C.

In the boost::fs::path("/").filename() = "/" case, filename returns root directory. I'm not sure how valid that is, or what was the rationale behind it, but it seems intentional. Probably worth changing.

The only other issue that I can see mentioned here was the trailing . element as the implicit filename. I think, the rationale for this is that iterating over /foo/bar and /foo/bar/ as must not produce equal sequences of path elements. This is documented here. Also, the semantic difference that I mentioned earlier, that /foo/bar refers to bar and /foo/bar/. refers to a file inside bar, is also valid. One consequence of that is in the latter case bar must be a directory, while there is no such requirement implied by the former path.

However, I'm not extremely tied to the current behavior. I can see the value in better compatibility with std::filesystem, so I'm not opposed to changing the current behavior. I'll create a separate issue for this.

If I missed some issues mentioned in the discussion, please create separate bugs for those - one bug per issue, with a repro. I'm going to close this issue.

Thanks everyone.