Update of my apps 🙂
Changelog:
- use sqllite for tracking all downloaded images.
- add new options in config.ini
- uselist = if True, it will import the list.txt into the database. If False, won’t import the list.txt
- processfromdb = if True, it will use the list in the database for downloading. If False, use list.txt
- If both set to False, it will ask for the member_id
You can get it in here, and for the source code in here. Use this one (source)
As usual, use at your own risk. No warranty given :).
UPDATE:
- As mentioned by Anonymous, I forgot to add the page limiter, Fixed in the 20100627 version.
- Requested by Moko, I’ve add the support for tags and member token for the filename format, use %tags% and %member_token%. If you use %tags%, please be warned if the filename maybe longer than 250++ character long and may cause error (windows filename limit).
Thanks for the fix!
As for the overwrite issue, my overwrite flag is always set to false. The downloader ignores the flag rarely, perhaps maybe once in every forty or fifty artist galleries.
When the downloader erroneously ignores the overwrite flag, even though it’s set to false, it deletes the artist directory, recreates it, and begins redownloading images. If this happens when number_of_pages is set to something other than 0, then only a portion of the artist’s gallery will be redownloaded, so you could end up losing images. And if you aren’t paying attention to the console window, then you won’t know which artist this happened to. I hope that made sense. >.>
Also, would it be possible to add a logging feature? Just something that dumps the console output to a txt file.
>> flag reset/random set:
maybe I will add a line to reload the config.ini after processing 1 member_id
>> logging:
will do
I had to go back to the previous version. I don’t know if I was doing something wrong, but the downloader insisted on checking, or downloading if the file didn’t already exist, every image in an artist’s gallery, regardless of what number_of_page was set to in the config.ini.
Also in the previous version, and this may carry over to the current one, the downloader will occasionally not skip over images that it has already downloaded in the past, but will instead delete the artist directory, recreate it, and start redownloading every image up to the set page maximum.
Anyway, I just figured I’d point out a couple of bugs that I’ve noticed.
Thanks for working on this program.
thanks for the report, it is because I rewrite and refactor the apps 🙂 and (maybe) forget to add the functionality 🙂
EDIT: I forgot to add 1 line to check the page limit,
…as for the overwrite/folder deletion, maybe you can check the config.ini for the filename_format. In the version, the apps will check the database first whether the image has already been downloaded or not, after that it will check whether there is a file with duplicated filename, as long the overwrite flag is not set, the apps won’t download and overwrite the image.
Looking good. I hope to see you continue to maintain this.
What features are you considering adding in the future and do you mind being sent code modifications?
I’ve helped anekos, who develops Ank Pixiv Tool, to debug cross-platform problems and suggested a few features he’s added. One of the things he introduced early on is adding many tokens with which you can build your file name. Although some may be overkill, we agreed that they were easy to add and someone may find them usefull. There is one token he added which I consider essential, however…
Your code refers to the Member Name or %artist%, which is useful, but the problem is that it’s not constant. And although the Member ID is available, it’s numeric and not memorable. Can I suggest you add as a token the Pixiv ID (the name used for the directory in which a user’s images are stored)? It’s constant, and is used by related sites such as pixiv.cc and drawr.net
Also, less crucial, but I’d love if support was added for adding tags to the filename as well. 😀
Looking forward to your progress, and, hope everything went well with school.
will try, but won’t promise 🙂
Just got a chance to try again with your revisions. Coming along nicely!
I you can indulge me, I have a few more requests. ^_^;
Firstly, I love that you implemented the %member_token%, and I don’t mean to nitpick but since it is a unique ID might it not be appropriate to name it something like %pixiv_id%, as a token is like a placeholder or variable, which can change. It may be misleading. Sorry ^____^;
Would it be possible to have the base download directory configurable rather than downloading to the directory the program resides in?
Could you have a separate renaming option for the handling of pixiv’s image sequences so we can have more control? I’d like to be able to place all images in a sequence in a subdirectory of an artist folder unique to that sequence. This may require introducing another token like %seq_num% or something so, sor naming of images within the sequence.
I see that you are separating tags (thanks!) using commas. Can we have control over how the tags are separated? Since tags aren’t supposed to contain spaces, a space is an adequate delimiter for me.
Keep up the great work!