pixiv downloader 20120317b

Change log:

  • Fix useTagsAsDir (20120317b)
  • Fix start/end page for Download by Tag list (20120317b)
  • Fix start/end page parameter parsing as reported by repu.
  • Add option to use the query tags as the save folder (appended to root directory) as requested by Ruca.
  • Update tags list parser to handle text files with BOM.
  • Add blacklist and suppress tags list.
    • useblacklisttags = True/False
      Skip image if containing blacklisted tags.
      The list is taken from blacklist_tags.txt, each tags is separated by new line.
    • usesuppresstags = True/False
      Remove the suppressed tags from %tags% meta for filename.
      The list is taken from suppress_tags.txt, each tags is separated by new line.
  • Add tagsLimit to be used for %tags% meta in the filename as requested by Anonymous User.
    • Use -1 to use all tags.

Download link here updated, source code here updated or at my GitHub.

18 thoughts on “pixiv downloader 20120317b”

  1. Hi,while the picture is “My pixiv only “,and I’m in マイピク,it happens like:

    Member Id: 39046
    2012-03-30 10:43:37,694 – PixivUtil20120317b – ERROR – Error at processImage(): (, TypeError(“cannot concatenate ‘str’ and ‘int’ objects”,), )
    2012-03-30 10:43:37,700 – PixivUtil20120317b – ERROR – Dumping html to: image_12467674.html
    2012-03-30 10:43:41,841 – PixivUtil20120317b – ERROR – Error at processImage(): (, TypeError(“cannot concatenate ‘str’ and ‘int’ objects”,), )
    2012-03-30 10:43:41,848 – PixivUtil20120317b – ERROR – Dumping html to: image_12467674.html
    2012-03-30 10:43:46,112 – PixivUtil20120317b – ERROR – Error at processImage(): (, TypeError(“cannot concatenate ‘str’ and ‘int’ objects”,), )
    2012-03-30 10:43:46,118 – PixivUtil20120317b – ERROR – Dumping html to: image_12467674.html
    2012-03-30 10:43:50,302 – PixivUtil20120317b – ERROR – Error at processImage(): (, TypeError(“cannot concatenate ‘str’ and ‘int’ objects”,), )
    2012-03-30 10:43:50,309 – PixivUtil20120317b – ERROR – Dumping html to: image_12467674.html
    2012-03-30 10:43:54,559 – PixivUtil20120317b – ERROR – Error at processImage(): (, TypeError(“cannot concatenate ‘str’ and ‘int’ objects”,), )
    2012-03-30 10:43:54,566 – PixivUtil20120317b – ERROR – Dumping html to: image_12467674.html
    2012-03-30 10:43:54,571 – PixivUtil20120317b – ERROR – Giving up image_id: 12467674
    2012-03-30 10:43:56,476 – PixivUtil20120317b – INFO – EXIT
    =====

    Thank you.

        1. Found the problem, I forgot to convert the datatype to str, and also they pad the image url if it is listed as private so the regex cannot detect the url.

          I will fix it in the next release.

  2. Thanks for releasing the source code to your tools, having the source for the pixiv downloader was part of the reason i started learning python and i found it helpful as a reference.

  3. I’m under linux, and I managed to modify your code to suite my system before (like path.replace and filename parse, and change some command not compatible with Python 2.6.6).
    And it’s been quite sometime since last time I used pixiv downloader, so when I try to modify your newest version of pixiv downloader, I met some problem…
    When I execute PixivUtil2.py, it gives me one line of error msg:
    ‘: [Errno 2] No such file or directory- coding: UTF-8 -*-

    The rootDirectory in PixivConfig.py seems ok, I’ve got no idea why this pop out…
    Just trying to look for help, thanks…

    1. And my current working directory is /dev/shm/pixivutil20120317.source , should be ok..

      1. ok now… it seems that the first line “#!/usr/bin/python……” caused the problem

    2. In PixivUtil2.py near line 247,
      req = customRequest(PixivConstant.PIXIV_URL+PixivConstant.PIXIV_LOGIN_URL)
      and, in PixivConstant.py
      PIXIV_URL = ‘http://www.pixiv.net/’
      PIXIV_LOGIN_URL = ‘/login.php’

      the resulting request url is http://www.pixiv.net//login.php,(have two ‘/’) which caused error in my system.
      change PIXIV_LOGIN_URL to login.php fixed it.

    3. # -*- coding: UTF-8 -*- << this one to notify Python that the script is containing non-ascii character

      Can you update the python version? This is the required version for the release as stated in readme.txt:
      – Python 2.7.2++
      – mechanize 0.2.5
      – BeautifulSoup 3.2.

  4. For some reason, I can’t seem to download using japanese tags, while using the tagsdirectory function.

    I can, for example, download the tag “Fate/Prototype”, but I can’t download the tag “キュアサニー”.

    I get this error:

    2012-03-17 13:50:45,334 – PixivUtil20120317 – ERROR – Error at processTags(): (, UnicodeDecodeError(‘ascii’, ‘x83Xx83}x83Cx83x8bx83vx83x8ax83Lx83x85x83A’, 0, 1, ‘ordinal not in range(128)’), )
    2012-03-17 13:50:45,339 – PixivUtil20120317 – ERROR – Unknown Error: ‘ascii’ codec can’t decode byte 0x83 in position 0: ordinal not in range(128)

    My locale is set to Japanese, and I am also running it as an administrator. I was able to download japanese tags on the previous Pixiv downloaders so I have no idea why this is happening.

    1. can you use the updated version? This one should be ok. Please note the list file should be in utf-8 if containing non-ascii character.

      1. I just observed a few things with the old version.

        The old one before this update just simply couldn’t read japanese Tags(I used the downloadbyTags function, not Tagslist), same for Tagslist.

        This new one still has the same problem for downloadbyTags, but the tagslist function works perfectly. Thank you for the hard work 🙂

      2. Oh, by the way, by “old” I mean the 20120317 version, and by new I mean the 20120317b version. ^^;

      3. Ah, by the way, this one is a common error that I’ve had on nearly all the Pixiv downloaders.

        2012-03-19 03:50:56,197 – PixivUtil20120317b – ERROR – Error at processImage(): (, timeout(‘timed out’,), )
        2012-03-19 03:50:56,242 – PixivUtil20120317b – ERROR – Cannot dump page for image_id: 25389181
        2012-03-19 03:50:56,243 – PixivUtil20120317b – ERROR – Error at processTags(): (, UnboundLocalError(“local variable ‘mediumPage’ referenced before assignment”,), )
        2012-03-19 03:50:56,250 – PixivUtil20120317b – ERROR – Error at processTagsList(): (, UnboundLocalError(“local variable ‘mediumPage’ referenced before assignment”,), )
        2012-03-19 03:50:56,252 – PixivUtil20120317b – ERROR – Unknown Error: local variable ‘mediumPage’ referenced before assignment

        It happens pretty randomly, and generally after ~100 pictures, although that’s just an estimate. This time it happened after downloading pic #553 from the tag “キュアハッピ”.

Comments are closed.