read_csv issues a misleading warning when using non-utf8 encoding and glob pattern about polars

[cmdlineluser]

Just some notes from attempting to debug this:

It appears that there are 2 places where this can happen depending on whether fsspec is installed or not.

polars/py-polars/polars/io/_utils.py

Lines 174 to 176 in 24c0f47

	# decode first
	with Path(file).open(encoding=encoding_str) as f:
	return _check_empty(

polars/py-polars/polars/io/_utils.py

Lines 199 to 207 in 24c0f47

	if isinstance(file, str):
	file = normalize_filepath(file, check_not_directory=check_not_dir)
	if not has_utf8_utf8_lossy_encoding:
	with Path(file).open(encoding=encoding_str) as f:
	return _check_empty(
	BytesIO(f.read().encode("utf8")),
	context=f"{file!r}",
	raise_if_empty=raise_if_empty,
	)

And it's being passed directly to pathlib.Path() - hence the lack of glob expansion.

[christian-hnz]

The _utils module has a _is_glob_pattern checker function, which is also used in the DataFrame._read_csv method (called by the read_csv function) to capture an unsupported combination of arguments ... a similar move would at least get rid of the misleading error.

Considering the alternative, my first impression is that using, e.g., the stdlib's glob.glob internally is quite a bit of effort for such an edge case .. which can be done by something like pl.concat(pl.read_csv(f, encoding=...) for f in glob.glob("test_*.py")).