29 thoughts on “pixiv downloader 20111202”

  1. Thank you very much for this great program.

    I have, however, noticed an issue when using the ‘Download by tags’ option once it gets to the last page it just continuously loops, displaying all the already downloaded IDs then incrementing the page by one. So for example if page 5 is the last (s_mode=s_tag&p=5) then it will increment the s_mode=s_tag&p=6, display all the image IDs saying already downloaded then increment to s_mode=s_tag&p=7, and so forth. Maybe it is something I am doing wrong.

    …
    Image #12345
    Image id: 12345
    Processing Image Id: 12345
    Already downloaded: 12345
    Looping… for http://www.pixiv.net/search.php?s_mode=s_tag&p=7
    …
    etc

    Also I was wondering if you would be able to implement a couple of additional features, both should hopefully be minor modifications based on the ‘Download by tags’ option.

    The first is to download by the exact tag rather than the wild-card search that is used in the ‘Download by tags’ option currently. Hopefully this would be a small change of the URL to http://www.pixiv.net/tags.php?tag='+tags'&p='+str(i) .

    The second is being able to download based on Title/Caption searching, which again should hopefully be a URL change to http://www.pixiv.net/search.php?s_mode=s_tc&p='+str(i)+'&word='+tags .

      1. Just as an update on the looping issue. I have been taking a further look at exactly what it is doing. It seems to be only when you are downloading to a folder that already includes an image that would be downloaded in the batch but has not previously been downloaded with the pixiv downloader. In other words it was downloaded manually from the site, etc.

        At the start of each page loop it first attempts to download the images it has skipped from “File exist! (Identical Size)” before beginning on the images from that page. Once it gets to the last page it will continue to attempt to do this looping only the image IDs of those none batched images.

        For example, if I have manually downloaded image “1” manually without this software the following would happen:

        1) First it would attempt to DL the image, but finds it:
        Image #1
        Image id: 1
        Processing Image Id: 1
        image_id OK
        …
        Start downloading… File exist! (Identical Size)

        2) Then it would continue downloading the other 19 images on the page OK as these don’t exist
        3) Loops to the next page ( Looping… for http://www.pixiv.net/search.php?s_mode=s_tag&p=2&word=tag )
        4) Attempts to download the skipped image “1” again, however this time it is in the database:
        Image id: 1
        Processing Image Id: 1
        Already downloaded: 1

        5) It then processes through the page as normal
        6) On the next page it would again attempt image 1

        7) Once it gets to the end of the pages it loops to the next page, however it doesn’t stop as it still attempts to download the image it skipped:

        Looping… for http://www.pixiv.net/search.php?s_mode=s_tag&p=n&word=tag
        Image id: 1
        Processing Image Id: 1
        Already downloaded: 1
        Looping… for http://www.pixiv.net/search.php?s_mode=s_tag&p=n+1&word=tag
        Image id: 1
        Processing Image Id: 1
        Already downloaded: 1

        And so forth

        Hope that makes sense.

  2. Hey, I noticed it gives an error when I try ti use a list that has more than 18,000 entries

      1. When I exported the memberlist using d>3, something in there made it not work when using it as the list.
        It gave me the following error:

        PixivDownloader2 version 20111202
        http://bit.ly/pixivDownloader
        Reading config file… done.
        Creating database… done.
        Traceback (most recent call last):
        File “PixivUtil2.py”, line 1032, in main
        File “PixivModel.pyc”, line 274, in parseList
        ValueError: invalid literal for int() with base 10: ‘xefxbbxbf11’
        press enter to exit.

        Next time I want to export my memberlist, is there a way to export just the memberids and not the #comments?

        1. Did you edit the exported list? can you upload the list into paste bin? I tried to export my list using d->3 and use it as list.txt and it work OK.

          Anyway, xefxbbxbf is UTF-8 BOM, so I assume you edit the exported files in notepad and save it as UTF-8. I will add handler for this case, for now better save the list files using different text editor 🙂

        1. No problem with the 18k long list, it can be imported correctly. The problem is on the UTF-8 BOM inserted by Notepad. For now, just save your list as ascii in the notepad. I will add the BOM handler in the next version.

  3. thank u very much!

    but i wouild rather use other download sites.. (media fire or etc..)

    i have to wait 60secs every time…

  4. I love your downloder, but I have a small request. Could you make the taskbar button flash when it’s finished with w/e you tell it to do? 🙂

    I haven’t bothered making a list yet, mainly because I’m lazy, but I was thinking that it’d be less bothersome if the program auto created/saved to specific folders on it’s own (inside the directory where the program is located), rather than me having to tell it where to save (as it does when you go after individual artists). This way I would only have to list the id’s. Perhaps a second list? (I’ve only glanced over the readme, so if it can already do this, sorry =s)

    1. >> flashing windows
      It is possible, but require additional library and not cross platform, so not recommended. I can make exit automatically after it is finished, tough.

      >> save folder
      there are option for it, see filenameformat and rootdirectory option in config.ini, check the readme for details.

  5. This version (20111202) bookmarks are imported good.
    Sorry I was testing with version 20111028. (For a synchronization bug in the VM)
    I have reviewed the code and the structure is well defined. (I see good experience and programming level)
    As I finish the current project, I get to work on this. (I tell you when I joined)

  6. Thanks for your reply.
    Sorry my bad explanation.

    When you run the software and choose the option: “5 Download from bookmark”
    Just import the last 50 bookmark.
    Would be interesting if the user could define the number of bookmark.

    I downloaded the source code, I’ll read it slowly.
    I will focus on debugging and add some additional options.
    I will also work in the GUI, but is an optional element.
    The graphical interface is for the software and to query the database.
    Then update the documentation and choice of language.
    Some other utilities I have in mind.

    Thank you for your hard work.

  7. Great app, saved me months of clicking 😀 Thank you

    One little thing, I wonder if you could include a “end page” aswell on the “download new illust from bookmark” function in the next update. I would be forever thankful. That said, truly epic app!

  8. You guessed NHO.
    And I’ll get it eventually.
    From what I’ve seen there is a limit of 50 markers (specifically last 50)
    What I do is add 50, run the explanation, I add another 50 more and run the application again and again and again.
    You have thought about putting multiple connections simultaneously. Timelines much would be gained to download.
    On the other hand, it is possible to put a GUI?
    On the other hand I’m thinking of a system of mirrors so as not to saturate, or you upload packages to a direct download server.
    In the future, I will join the project, I am programmer and I like this application. I hope the principal of the year.
    As have most of the pictures and I will notify you share it.
    The most important part is to debug some errors, for example the “ul” and when you cut the communication with the server.
    Thanks for your time.
    Much encouragement and Keep up the great work.
    Happy Holidays.

    1. > From what I’ve seen there is a limit of 50 markers (specifically last 50)
      you mean tags? haven’t try that, can you explain more?

      > multiple connection
      I don’t want to handle the Threading for now.

      > it is possible to put a GUI?
      I don’t really use the gui 🙂

      > I will join the project, I am programmer and I like this application.
      Thanks :), you can get the latest source code in github here: https://github.com/Nandaka/PixivUtil2

  9. Thank you very much for this! 😀

    Just one question though: Is it possible to have a feature that allows the program to update the downloaded artists’ folders in the download directory stated in the config without using list.txt?

    1. nope, because then you need to parse the folder name, which might be different for every one, and try to find the member_id. Better use list.txt.

      1. I think someone want to use your downloader to download Pixiv. All of it. Impressive ambition.
        AjaxProxDev, you is crazy and slightly awesome.

  11. Thank you God / Goddess.
    As I can download for range image ID Example ID: 1-500000
    Thank you again.

Comments are closed.