Songs mistaken for Albums when .search() with language is not "en" about ytmusicapi

[shibotto]

I'm exploring this issue a bit because I have time. Before 614999c in parse_search_result() if an element had video_type it was assumed to be either a song or a video, else get_search_result_type() would be called to try and determine what we are dealing with.
Now this behavior has been (somewhat) reversed. The problem is that get_search_result_type() is unable to identify songs/videos and evaluates everything it doesn't recognize as an album, and since it's now an album it never gets a chance to become a song or a video further down the code.

I thought about reversing that part back, but I also noticed that now parse_search_result() is always called with some dirty value (category[:-1]) for result_type:

due to changes in here:

ytmusicapi/ytmusicapi/mixins/search.py

Lines 220 to 225 in ae966bb

	elif "musicShelfRenderer" in res:
	shelf_contents = res["musicShelfRenderer"]["contents"]
	category = nav(res, MUSIC_SHELF + TITLE_TEXT, True)
	type_filter = filter or category
	type = type_filter[:-1].lower() if type_filter else None

[sigma67]

Please check again if the results are good now. There might be some browseId edge cases I've missed.

[shibotto]

I tried trowing a bunch of different queries at it, observing each element returned and yep, everything matches perfectly now! If I was simply lucky enough to miss any obscure browseId I'll get back to you.
Thank you very much! 😀

[sigma67]

Thanks!

Sorry the code is so obscure, it's just hard to handle all the different cases. I've been trying to add more tests and make it more readable. I think I got a better grasp on some of the edge cases now, please check #583

[shibotto]

I'm sorry to inform you that now English is broken as well :D

The original behavior, as I understood it, was to [:-1].lower() the category as a first step, which works well for English (Songs -> song, Albums -> album, etc). Then if nothing matched, which in most cases meant a language different than English, try to guess from the available data. Now this doesn't happen as category is never taken into account before calling parse_search_results(), and honestly I think it's a good thing as at least everything fails equally.

The fundamental problem however remains, maybe the last part I added later overshadowed it a little: get_search_result_type() is the first one to be fed, but get_item_text() feeds it the wrong data. As an example:

And as a consequence the next step, result_type = 'album'. I took just a quick look, but nowhere inside the data structure of a song element I saw something that could actually be used by parse_search_results().
We have MUSIC_VIDEO_TYPE_ATV though, which as far as I can observe always identifies songs.

Oh, don't worry about the code, I once tried to do something similar for generic Youtube. After few months I trashed everything because the many edge cases and my poor skill made the thing incomprehensible. Kudos for hanging on for so many years.

[sigma67]

Thanks for your diligence.

The heuristic of using the first sub-run as an indicator is inherently broken because it's so inconsistent server-side. Depending on where a Song appears the Song indicator is sometimes there and sometimes not - it's just guesswork that's bound to change at their whim.

The problem was that that bad heuristic was actually covering up the better one that came after.

So I removed that part entirely and am only using the browseId (and MUSIC_VIDEO_TYPE_ATV) now, which seems to work pretty well. It uses the mapping in your screenshot (from the FAQ and my experience basically).