Status visible, old, and deleted fics about fimfarchive

[JockeTF]

All of the metadata (except for archive) come directly from Fimfiction. So, when something goes missing it's essentially frozen in time. It's not really by design, but I just haven't done all that much when it comes to cleaning things up.

There is a way for you to filter out stories that are no longer available though! You can do this by looking in to the (somewhat poorly named) timestamps in archive. Note that some of these are null since I haven't retroactively added things such as creation dates.

• date_checked: When did we last try to update the story?
• date_created: When was the story added to the archive?
• date_fetched: When was the last update of the metadata?
• date_updated: When was the last update of the content?

The other timestamps will exactly match date_checked if a change happened for that version of the archive. So, to check if a story was publicly available on Fimfiction you could compare date_checked to date_fetched.

>>> from fimfarchive.fetchers import FimfarchiveFetcher
>>> 
>>> def was_available(story):
...     archive = story.meta['archive']
...     date_checked = archive['date_checked']
...     date_fetched = archive['date_fetched']
... 
...     return date_checked == date_fetched
... 
>>> 
>>> fetcher = FimfarchiveFetcher('fimfarchive.zip')
>>> sum(1 for story in fetcher if was_available(story))
131322

Hope that helps!

[JockeTF]

Also, I'll be looking into story 31718.

It seems you might have found a rather significant bug, so thank you!

[htnyquist]

Thanks, date_fetched and date_checked sound like exactly what I need!
I am curious about the case of 31718, but ultimately if it's limited to the series tag that's something I can deal with by just assuming any fic without one is automatically 'MLP-FiM' (I assume that's what Fimfic did retroactively).

[JockeTF]

→ @htnyquist

I kept track of 31718 during the last archive update, and it seems to have updated fine without me doing anything. I believe what happened was that the story had been unpublished for a while. It probably got published again some time after the previous release.

[JockeTF]

There seem to be comments hinting at that as well!