Pixiv Downloader 20101024

UPDATED here

Long time no update, kinda busy looking for a job here, just recently graduated from my uni 🙂

Changelog:

  • 20101024:
    – by Nandaka:
    – Fix error when dumping html.
    – Change exit behaviour, no need to press enter twice if you select exit.
  • 20101005:
    – by Yavos:
    – added commandline-option: <-n number> to temporarily overwrite config-settings for <numberofpage>
    – added new options in config.ini (for details see readme.txt)
    (<createDownloadLists>, <downloadListDirectory>, <startIrfanView>, <startIrfanSlide>, <IrfanViewPath>)
    – added commandline-option <-i> to automatically start IrfanView with list of downloaded Images
    (this option only takes effect when <startIrfanView> and <startIrfanSlide> from config.ini are not set True)
    – images will get the extension <.pixiv> until they are complete (prevent incomplete image-downloads)
    – fix: all downloaded images will be added in DB (until now it was limited to images downloaded by member_id/list of member_ids)
    (note: you can reset it again by setting <Yavos = False> in sourcecode line 27)
    – added feature: when (directly or indirectly) set in config.ini pixivUtil will create lists of your downloaded images in <Downloaded_on_YYYY_MM_DD.txt> (using date when pixivUtil is launched)
    – added feature: start IrfanView when exiting pixivUtil (Slideshow will be started before normal IrfanView-Window)
    – added feature: typing <-all> in menu will temporarily activate/deactivate pagelimit set in config.ini (it will set commandline-option <-n 0>)

Download link here (compiled windows binary) , source code here.

25 thoughts on “Pixiv Downloader 20101024”

  1. Hi~
    When previous download fails, in case it leaves an incomplete file, I added the following lines in downloadImage() section:

    if not overwrite and os.path.exists(filename) and os.path.isfile(filename) :
    # print “localfilesize = “, localfilesize
    # print “remotefilesize = “, filesize
    if int(filesize) == os.path.getsize(filename) :
    print “tFile exist!”
    return 0 #Yavos: added 0 -> updateImage() will be executed
    else :
    os.remove(filename)
    try:
    save = file(filename + ‘.pixiv’, ‘wb+’, 4096)
    except IOError:
    msg = ‘Error at downloadImage(): Cannot save ‘ + url +’ to ‘ + filename + ‘ ‘ + str(sys.exc_info())
    safePrint(msg)
    __log__.error(unicode(msg))
    save = file(os.path.split(url)[1], ‘wb+’, 4096)

      3. Thanks, please check this:
        def downloadImage(url, filename, referer, overwrite, retry):
        try:
        print ‘Start downloading…’,

        if not overwrite and os.path.exists(filename) and os.path.isfile(filename) :
        # print “localfilesize = “, localfilesize
        # print “remotefilesize = “, filesize
        if int(filesize) == os.path.getsize(filename) :
        print “tFile exist!”
        return 0 #Yavos: added 0 -> updateImage() will be executed
        else :
        os.remove(filename)
        try:
        save = file(filename + ‘.pixiv’, ‘wb+’, 4096)
        except IOError:
        msg = ‘Error at downloadImage(): Cannot save ‘ + url +’ to ‘ + filename + ‘ ‘ + str(sys.exc_info())
        safePrint(msg)
        __log__.error(unicode(msg))
        save = file(os.path.split(url)[1], ‘wb+’, 4096)
        print “tFile exist!”
        return 0 #Yavos: added 0 -> updateImage() will be executed

        All Right?

      4. @Ivan
        Those lines must follow these lines in downloadImage():
        try:
        filesize = res.info()[‘Content-Length’]
        except KeyError:
        filesize = 0
        res.info()[‘Content-Length’] retrieves filesize of image from serverside.
        os.path.getsize(filename) gets filesize of the image already downloaded.

        Any way, I modified the PixivUtil2.py quite a bit, the code becomes very dirty looking… It might not work if you just copy + paste “this” block of code to the preferred place. So…, better wait for nandaka’s version.

  2. Help please, how you change 澪秋山 to % E7% A7 % 8B% E5% B1% B1% E6% BE% AA
    i don’t think it
    sorry for my bad english, because i’m russian 🙂

  3. Whenever I set createdownloadlists = True uselist = True, the moment I run the program, it automatically sets both back to False.
    What am I doing wrong?

  4. You App save my day, works great 😀
    Only have one problem (under linux), the app doesn’t create correctly the directories. Instead of create the image into a directoy, a file is created with the name of the directory plus image name.

    That’s my filenameformat:

    filenameformat = %artist% (%member_id%)(%image_id%) %title%

    Btw, the app works great under windows and create the directories correctly, the problem is only under linux (ubuntu 10.10)

    Any fix for that?

    1. try to replace “” to “/”, linux has different directory separator from windows.

      filenameformat = %artist% (%member_id%)/(%image_id%) %title%

      1. Different person here.
        I went through and cleaned up the windows separators, but I still seem to have a problem.
        The “/” is being sanitized preventing artist directories from being made.
        Any advice?

        1. try to modify the folder sanitizer in the source code, you can remove “/” from the list.

          modify this line
          __badchars__ = re.compile(r'^.|.$|^ | $|^$|?|:||/|||*|"') #|'')
          into
          __badchars__ = re.compile(r'^.|.$|^ | $|^$|?|:|||*|"') #|'')

          1. I noodled around with a code a bit more and got it working comfortably.
            Thanks for all the work you’ve put into this — it’s really useful!

  5. I love the new changes but I seem to have a problem trying to turn overwrite to false. It just goes no matter what.

    False overrides without checking if the file exists and True yields the same error windows does when you try to rename a file to an existing one via explorer.

  6. Well now it manages to notice the ones I’ve downloaded just now. it still doesn’t acknowledge the existence of the things i downloaded previous to now.

    Would there be any way I can have it see the other things I’ve downloaded so I can avoid downloading the artists again?

    1. As long you save it to the same folder and doesn’t change the filename format (so it will generate identical filename with the previous one), then it will try to check if the file has been downloaded or not from the filename.

  7. I don’t know whether or not I’m working it right but the DB manager doesn’t seem to function.

      1. going to the DB manager function, i try to choose say: “Show all member” and it prints “member_id name save_folder created_date last_update_date”. If i choose to show images by member ID and type in what should be a valid ID of a member I’ve downloaded images for, it just reprints the menu with nothing in between.

        The log doesn’t show that I’ve done anything aside from log in and exit:
        pixivutil.log (piece)
        2010-10-24 04:24:08,019 – PixivUtil20101024 – INFO – Using Username: [username]
        2010-10-24 04:24:08,019 – PixivUtil20101024 – INFO – logging in
        2010-10-24 04:24:11,368 – PixivUtil20101024 – INFO – Logged in
        2010-10-24 04:27:12,838 – PixivUtil20101024 – INFO – EXIT

Comments are closed.