crouvpony47 / furdown

A mass download tool for FurAffinity written in C#

furdown

Yet another mass downloader for FurAffinity.net.

Note: only works with the new (Modern) theme.

Download stable win32 builds.

Changelog (v.0.5.2.0)

• added support for downloading content from the submissions inbox (submissions from the watched artists);
• %SCRAPS% value is set correctly for all submissions regardless of the URL used to produce the submissions list (previously this did not work properly for favorites);
• fixed update mode related issues;
• minor quality of life improvements, including: a distinct app icon, better Win7+ taskbar progress indication, autocorrecting common issues with the URLs (leaving page number in the URL, no https://, etc.), support for legacy download URLs parsing
• v.0.5.3.0 implements a fix for a bug preventing submission inbox from being downloaded fully when the default order is set to be "Oldest first"

A note about CF's "I'm Under Attack" mode

• If you are already logged in but are shown the login form anyway, simply navigate to the FA main page.

System requirements

• Windows Vista SP2 or newer, might not work on server editions
• .NET 4.5
• IE11 (for systems where IE11 is not available, compile Furdown from source after adjusting src/Program.cs:WebBrowserEmulationSet() accordingly)

IE11 requirement can be bypassed if you implement an alternative cookie provider, see "Advanced options" section below.

Portable mode

To leave no traces in ApplicationData directory you can create an empty file called furdown-portable.conf in the working directory, it will then be used as a settings storage file instead of the default %AppData%\furdown\furdown.conf.

Batch/CLI mode

v.0.3.6 has a basic support for being run within a batch script, or from a terminal.

For that -b must be passed as the first argument, followed by a combination of one or more arguments:

• -g URL

Download a gallery / scraps / folder located at URL. If URL is prefixed with $, it will be treated like a file to read the url list from.

• -d

Tells furdown you also want submission descriptions saved as HTML.

• -u, -upd, -update (since 0.4.9.9b)

Check for content updates. This means comparing the file IDs with the ones stored in furdown's internal cache, and downloading the changed files. (details)

• -f FILE

Download all galleries listed in FILE. Same as -g $FILE

• -o DIR

Override downloads directory with DIR.

• -dbignore, -dbi

Prohibits re-downloading files already marked as downloaded in furdown's internal database.

• -dbforce, -dbf

Allows re-downloading files already marked as downloaded in furdown's internal database.

• -st TMPLT

Override submissions naming template with TMPLT. *

• -dt TMPLT

Override descriptions naming template with TMPLT. *

* As templates use the syntax much like that used for environment variables, you may get unexpected results if, say, %FILEPART% is an existing variable.

Other notes:

• You must login in GUI mode at least once before using batch/CLI mode.
• For non-overridden parameters the value from a current configuration file (the same file used in GUI mode) is used.

Examples:

• furdown.exe -b -g https://www.furaffinity.net/gallery/flipstick
• furdown.exe -b -d -dbi -f "Z:\Saved Galleries\list.txt" -o "Z:\Saved Galleries\"
• furdown.exe -b -d -g https://www.furaffinity.net/scraps/flipstick -dt %ARTIST%.s\%SUBMID%.%FILEPART%.dsc.htm -st %ARTIST%.s\%SUBMID%.%FILEPART%

Note that % might need to be escaped as %% in batch scripts.

Advanced options

The builtin authentication mechanism based on the embedded Internet Explorer can be bypassed by setting FURDOWN_COOKIES and FURDOWN_USERAGENT environment variables to the appropriate values. If only FURDOWN_USERAGENT is set, furdown and its embedded IE will use the User-Agent value provided, and if only FURDOWN_COOKIES is set, it is your responsibility to match the User-Agent of the cookies source and the one used by furdown.

Note that the FURDOWN_COOKIES is expected to contain the entire cookie header value (something like b=XXX; __gads=XXX; a=XXX; s=XXX; __qca=XXX; sz=XXX; cc=XXX; __cfduid=XXX where XXXs are some values)

furdown's Issues

Feature Request - Overwrite Save Location For Each Entry When Using Download List

Just stumbled across this program recently, and it works really well. This may already be possible, but I wished to ask to confirm as I wasn't too sure.

Using a download list text file, is it possible to overwrite the save folder for each entry?

Say for example if there are 2 entries in the list, are there any methods of assigning a specific save location for each of those two entries rather than using the default save path? Currently I believe you can do this overall via command line switches, but it would be helpful if you could set individual download directories.

Click <login to fur affinity>, no response

Click , no response.There is no jump to any interface

"broken filename detected, replacing with "unknown.jpg""

When I tried to download сrystal-for-ever's gallery, some of the files had "unknown" after the numbers, while the program writes "broken filename detected, replacing with "unknown.jpg""

Can't change Download location

All downloads still goes to %AppData%\Roaming\furdown\downloads even after changing the path in the settings.

Complete crash when hitting certain FA posts.

I get this when downloading pages like id 28749873

Microsoft Defender SmartFilter: "This app might harm your device"

`Microsoft Defender SmartScreen reported that this app is not commonly downloaded or is not signed by its publisher.

Name: furdown.exe
Publisher: Unknown`

Downloaded files don't move into their folders

So the folder gets made but the files/images just gets placed in the main folder instate of placing them in the folder it made.

Warning :: cannot parse date :: String was not recognized as a valid DateTime.

Auth check completed successfully.
Building submissions list for https://www.furaffinity.net/gallery/holivi
Getting page: https://www.furaffinity.net/gallery/holivi/1
Getting page: https://www.furaffinity.net/gallery/holivi/2
Getting page: https://www.furaffinity.net/gallery/holivi/3
Reached an empty page. Total elements found: 134
Saving submissions list...
Submissions list saved to "latest_subs.log"
Processing submissions list...
> Processing submission #29924294
Getting page: https://www.furaffinity.net/view/29924294
Title: Rarity
Warning :: cannot parse date :: String was not recognized as a valid DateTime.
target filename: furaffinity\2020_01_01_21_19_Rarity.jpg
Downloading: https://d.facdn.net/download/art/holivi/1546254701/1546254701.holivi_???_???????.jpg
Done: #29924294

I don't know if that download link supposed to have many question mark is a bug, but it downloaded the images perfectly

I do not get the captcha

Hi, I tried solution #22 but even if I try it I don't get the captcha, is there any solution?

Starting two days ago, when starting Furdown, I do not get the furdown front page.

Click on Furdown and get the following response:
furdown 0.4.3.0
<github.com/crouvpony47> <crouvpony47.itch.io>
Checking authorization...
Found some cookies.
Does not seem to be authorized, showing login page

It then shows me the login page for FurAffinity. I never get the front page for furdown with all the windows - such as section to download, where to download, etc.

Feature Request - Linux build?

Would it be possible to get a Linux build that could run on my Ubuntu 20.04 installation?

How difficult would that be?

i cant download

[

](url)

Verification process failed, please try again.

Every time I try and log in i get Verification process failed, please try again. Their is no captin loading.

Save .htm descriptions Startindex error

Whenever I try to download something I with the "also save .htm descriptions" option ticked, I get the following error:

Startindex cannot be less than zero.
Parameter name: startindex

This happens with any user gallery, e.g. https://www.furaffinity.net/gallery/dragoneer/.

Issue can be deleted

Feature Request - Download results of a search

Is this at all possible? It would be super awesome to have!

Downloading folder options

Is there a way to change how the files are organized? I would like them to all go into a single folder without being put in individual artist folder.

Failed to make sure target directories do exist

login is now successful, but now I get "Failed to make sure target directories do exist" error. Is there any way to give information for you to debug more better?
I try to download Cenegan024's gallary.

Permition issue

I'm having problems with openning the file due to permition lack as mentioned on the message below.

This problem only happens with the most updated version (v.0.5.2.0)

[Warning] got page, but it doesn't contain any download links.

starting earlier today, whenever I run furdown 0.5.0.1, it doesn't download anything... it lists the submissions, but says this:
Getting page: https://www.furaffinity.net/view/<submission ID#>
[Warning] got page, but it doesn't contain any download links.

Did FurAffinity change something on their end? or cloudflare?

11/10 Program

Hi, I saw you updated your program to now work with favorites extremely fast, and I'm glad that you are paying that close attention to this project. I had a question though since there is no way to normally comment on your project, unfortunately, it is being listed as a "issue". Is there a present way to have the files downloaded not put into Artist Folders. I.E. just one folder full of all the images instead of filtered to each artist. If there is please comment back :). If not, then that's fine, I hope you have a great day. And thank you for the speedy update! <3

Can't login

After login the authentication window goes to the furaffinity homepage and just sits there

request filename template

Hai is it possible to save images filenames based on the date of its submission (ex. %DATE% or %YEAR%.%MONTH%.%DAY%.%HOUR%.%MINUTE%) and the title of the submission (Spit Roasting Celestia) instead of URL filename (img_20180818_0001_new_new)?

Settings don't work

For some reasons, I changed the settings before downloading submissions, but it's keeping the default ones, what's the matter?

[feature request] checkbox for blocking reminders from being downloaded (or blacklist)

A feature that allows to exclude submissions that have certain keywords (either as tags, or as part of the submission-title) would be nice to have.

I don't know how feasible it is to implement, since FA does not force people to use tags, nor has a separate field to mark submission as a reminder (those are the most annoying, if I don't remove them by hand before downloading, I will end up with several pictures that are exactly the same, but most of them are reminders, so they're just take up space) but even the simple "blacklist" from just the submission name would be something.

Maybe a variable with flag, that would be checked around AppCore.cs line 436, where the title, date etc. are processed?

Looking at FA submission page, implementing tag blacklist over that should be (relatively) simple, as they're wrapped in <section class="tags-row"> with each tag being in its own <span class="tags">

Unable to download watchlist from second account

Pretty much the title. I have two accounts, and I can't seem to find a way to download from the other once I've logged in to one. I can't find a way to log out or change the account in the app either.

Feature Request - Download submission inbox

Would be super handy if it's possible! Tried doing it just now but it seems to list pages forever.

Favorites

Does not work with favorites galleries.

Locks up upon reaching "authorization"

I did notice i had to authorize my connection to FA this morning when booting the site up, maybe thats stopping the bot?

Errors from log:

"Error: Response status code does not indicate success: 403 (Forbidden)."
"If it is caused by Clouflare validation, pass it and navigate to the FA main page."
"Does not seem to be authorized (or need to pass CF validation) . . ."

keep getting error Unsuccessfull login atempt

After login to FA, furdown says Not ahthorized! and get Unsuccessfull login atempt error.
Is there any solution for this? or maybe how to give you information for debug?

StartIndex must not be less than zero error

On download, a message box with that error will be shown. Not able to download anything.

[v.0.5.2.0 Bug] Downloading recent submissions only possible one page at a time

When downloading submissions from the watched artists furdown when submissions are sorted from oldest to newest only downloads the first page (with file number depending on user settings: 24, 48 or 72 submissions). If submissions number is greater than page number, rest of the submissions is not downloaded.

Steps to reproduce:
0. Set Site Settings -> Submission Notifications Order -> Oldest first

1. Submissions inbox have more than 1 page
2. Download submissions by entering Furdown -> Tasks -> link: https://www.furaffinity.net/msg/submissions
3. Click "Go!" button

Result: Only the first page with the number of submissions specified in FA Account settings is downloaded.

Additional info: the same happens if checkbox "updates check mode" is ticked - still only the first page is downloaded.

This is probably due to AppCore.cs expecting only "new@" string being part of the page address - it does not take "old@" into account (line 358 of the code, method CreateSubmissionsListFromGallery).

Feature Request - Option to grab URLs of images without downloading.

A mode to grab the URL of the images from FurAffinity database[d.furaffinity.net].

[New feature] Update mode

This issue is used for tracking progress on the update mode implementation. This is also a place where implementation details are to be described.

The problem

FurAffinity allows users to update their published content, but doesn't make it apparent to the users which content got updated, and when. This feature allows for easy detection of changes to keep your local backup up to date, even if something got sneakily updated.

Solution

Let's examine the two semi-random URLs:
https://d.facdn.net/art/voxell/1427061816/1427061575.voxell_voxell_01_small.jpg
https://d.facdn.net/art/voxell/1441545741/1441545741.voxell_voxell_pixel_stretch_01.jpg
One of those has been updated in its lifetime, and the other one hasn't.
We can see that the first numeric id in the URL (let's call it Current File ID, %CURFILEID%) matches the second one (File ID) in cases when the content has not been altered. This also applies to the thumbnails, and even to the story thumbs, which is good, as it makes it possible to detect changed content without a single extra request, assuming we've saved Current File IDs the previous time.

Implementation details

When furdown detects changed content, the new version is saved under the name determined by the usual filename template, %ARTIST%\%SUBMID%.%FILEPART% by default, except in cases when this file already exists, then the template is extended by appending [%CURFILEID%].%EXT% to it.

Things get a bit messier when the local archive that needs to be checked for updates has been produced by an older version of furdown (below 0.4.9.9). On the first such update, the current IDs cache needs to be built from scratch, and this involves requesting all the relevant submissions pages. If those appear to have never been updated, IDs are just saved and no further actions are performed. Otherwise, the actual file (picture) is downloaded and compared against the local file (if one exists). If those match, the newly downloaded file gets discarded, otherwise it is kept.

Note that running an app version <0.4.9.9 after >=0.4.9.9 will delete the file IDs cache. If you need to use the two alongside each other, use at least one in the portable mode with System and temp files path setting pointing to two different places.

Progress

• research
• basic implementation
• documenting, extensive testing
• release