Curl_hash_clean destroys hash table about curl

[jay]

Hi Stefan, does this have anything to do with the changes you made a while back or is it unrelated?

[bagder]

@jay, the point is actually more global than so but @tvbuehler figured that out when I edited his recent patch set. Here's the communication on that: 81ee1e6#commitcomment-10732175

[aavina]

I've been looking through the code regarding this issue, along with writing a unit test to verify it, and you can see the usage for Curl_hash_clean within Curl_hash_destroy. It's used as a preliminary before freeing the hash table struct.

The unit test I wrote verifies the issue (modelled by @tvbuehler 's comment) which shows itself when you call Curl_hash_clean and then attempt to Curl_hash_add another element. If you use the Curl_hash_str as the hash fn (which is what hostip.c does), it will attempt to mod the h variable with hash->slots which was reset to 0, thus crashing.

gist of unit test (please comment, if any!) https://gist.github.com/aavina/aea2a9afe5bcdda6b9c4

Maybe we can create the Curl_hash_reset function and use that in hostip.c and wherever else Curl_hash_clean is being used improperly?

[bagder]

I'm fine with the functions doing slightly different things as long as we document them so - and assuming that the code using them is actually working fine. Can you provide your suggested unit test as a pull request please, @aavina ?

[aavina]

I've submitted the pull request: #262
I can work on creating the new function Curl_hash_reset and have the current callers to Curl_hash_clean use it if they're more suited to. Is the only documentation needed within the implementation file, above the functions themselves, in hash.c?

[aavina]

I've ran into an issue when implementing the fixes for this. The current Curl_hash_clean function is used in hostip.c with the intention of possibly re-initializing the hash later. If it does end up re-initializing the hash via Curl_global_host_cache_init, it will leak the memory that the hash points to prior to the re-initializing. See usage of host_cache_initialized symbol in hostip.c for info.

As a fix, I propose that instead of using Curl_hash_clean, hostip.c can instead use Curl_hash_destroy to avoid the memory leak and that we change Curl_hash_clean (which will be turned to Curl_hash_reset) into a static function so that only hash.c can use it as an auxiliary function. Thoughts?

[tvbuehler]

The current implementation of Curl_hash_clean is the counterpart for Curl_hash_init, while Curl_hash_destroy matches Curl_hash_alloc. So as long as Curl_hash_init is intended to be usable Curl_hash_clean is also required.

The Curl_hash_clean call in Curl_global_host_cache_dtor is supposed to be the counterpart for Curl_hash_init in Curl_global_host_cache_init, so after the rename it should call Curl_hash_reset instead.

The Curl_hash_clean call in Curl_hostcache_clean is currently a bug, and should be the "new" Curl_hash_clean instead, which only removes all entries (similar to Curl_hash_clean_with_criterium). This is the only place I know of where Curl_hash_clean should not get renamed to Curl_hash_reset.

That said I don't see the memory leak you are speaking of; if the current Curl_hash_clean would "leak" any memory apart from struct curl_hash, which is not allocated by Curl_hash_init, Curl_hash_destroy would leak too.

[aavina]

You're correct, sorry. I think I was confused with the usage because in my unit test case I use the Curl_hash_destroy-Curl_hash_alloc pair with a dynamically allocated Curl_hash. I see now that hostip.c uses a statically allocated Curl_hash so it shouldn't use Curl_hash_destroy on it. I'll withhold those proposed fixes from my previous comment. Thanks for the feedback!

[bagder]

I'm going to work on some further fixes to reduce or perhaps even completely remove the use of Curl_hash_alloc() since it turns out I think all our current uses could just as well be changed into Curl_hash_init() of an existing struct and have us avoid extra dynamic allocations.

[aavina]

Sounds good to me. Until then, I'll be exercising both dynamic and static allocations in unit1602. If your changes are in before mine I'll remove the dynamic allocation testing from the unit test.

[bagder]

There, the hash_alloc / hash_destroy functions are now history as of commit b419e7a!

[tvbuehler]

@bagder I think unsharing of DNS in b419e7a doesn't clean the cache anymore, which it previously did - are you sure you want this?

[bagder]

Yes. I noticed, and I actually think that clearing the cache on the unshare action might be a bit surprising, so I think the current way is fine. Although I also think that most users will kill the share object after unsharing anyway.

[bagder]

Why is (still) it a problem that hash_clean() frees ->table ?

[bagder]

ah, because the dns cache is cleaned in for example Curl_hostcache_clean()

[bagder]

fixed now!