Updated: pixiv downloader 20120929

Change Log:

  • Update medium image page parser for tags and unit test.
  • Fix PixivConfig doesn’t allow %page_number% in settings.
  • Update medium image parser to handle no tags (20120929).

Download link for pixiv downloader 20120928 pixiv downloader 20120929, source code on GitHub. Donation Link on the left ๐Ÿ™‚

On side note, I’m also doing this android project, go to Baka-Tsuki development thread or the app name suggestion thread for discussion ๐Ÿ˜€

20 thoughts on “Updated: pixiv downloader 20120929”

  2. I sometimes get this error at random:

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 589, in processImage
    File “mechanize_mechanize.pyc”, line 569, in follow_link
    File “mechanize_mechanize.pyc”, line 550, in click_link
    File “mechanize_mechanize.pyc”, line 443, in viewing_html
    BrowserStateError: not viewing any document
    Error at processImage(): (, Brow
    serStateError(‘not viewing any document’,), )
    Dumping html to: Error Medium Page for image 16128712.html
    Stuff happened, trying again after 2 second ( 1 )
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 471, in processMember
    UnboundLocalError: local variable ‘ex’ referenced before assignment
    Error at processMember(): (, UnboundLocalEr
    ror(“local variable ‘ex’ referenced before assignment”,), )
    Dumping html to: Error page for member 294656.html
    Error at processBookmark(): (, UnboundLocal
    Error(“local variable ‘ex’ referenced before assignment”,), )
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 1409, in main
    File “PixivUtil2.py”, line 1160, in menuDownloadFromOnlineUserBookmark
    File “PixivUtil2.py”, line 885, in processBookmark
    File “PixivUtil2.py”, line 471, in processMember
    UnboundLocalError: local variable ‘ex’ referenced before assignment
    press enter to exit.

      1. it starts as a 10060 error or something, for some reason wordpress hates it when i try to paste it here.

  3. I’m getting this error when I try downloading by tag:

    2012-10-04 10:37:57,655 – PixivUtil20120929 – INFO – Starting…
    2012-10-04 10:37:57,655 – PixivUtil20120929 – INFO – Only process member where day last updated >= 7
    2012-10-04 10:37:57,655 – PixivUtil20120929 – INFO – Using Username: 1idd0kun
    2012-10-04 10:37:57,655 – PixivUtil20120929 – INFO – Log in using form.
    2012-10-04 10:38:04,233 – PixivUtil20120929 – INFO – Logged in
    2012-10-04 10:38:14,546 – PixivUtil20120929 – INFO – tags mode.
    2012-10-04 10:39:12,265 – PixivUtil20120929 – INFO – Searching for: (%E3%82%A2%E3%83%AA%E3%83%A2%E3%83%AB) %E3%82%A2%E3%83%AA%E3%83%A2%E3%83%AB
    2012-10-04 10:39:12,265 – PixivUtil20120929 – INFO – Looping… for http://www.pixiv.net/search.php?s_mode=s_tag&p=1&word=%E3%82%A2%E3%83%AA%E3%83%A2%E3%83%AB
    2012-10-04 10:39:12,265 – PixivUtil20120929 – ERROR – Error at processTags(): (, KeyError(u’xe3′,), )
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 748, in processTags
    File “mechanize_mechanize.pyc”, line 203, in open
    File “mechanize_mechanize.pyc”, line 230, in _mech_open
    File “mechanize_opener.pyc”, line 188, in open
    File “mechanize_http.pyc”, line 320, in http_request
    File “robotparser.pyc”, line 139, in can_fetch
    File “urllib.pyc”, line 1250, in quote
    KeyError: u’xe3′
    2012-10-04 10:39:12,265 – PixivUtil20120929 – ERROR – Unknown Error: u’xe3′
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 1404, in main
    File “PixivUtil2.py”, line 1107, in menuDownloadByTags
    File “PixivUtil2.py”, line 748, in processTags
    File “mechanize_mechanize.pyc”, line 203, in open
    File “mechanize_mechanize.pyc”, line 230, in _mech_open
    File “mechanize_opener.pyc”, line 188, in open
    File “mechanize_http.pyc”, line 320, in http_request
    File “robotparser.pyc”, line 139, in can_fetch
    File “urllib.pyc”, line 1250, in quote
    KeyError: u’xe3′
    2012-10-04 10:39:16,921 – PixivUtil20120929 – INFO – EXIT
    2012-10-04 10:39:16,921 – PixivUtil20120929 – INFO –

    It happened when I used tags that were encoded from Japanese, like these ones:

    %E5%9C%92%E6%99%BA%E6%81%B5%E7%90%86

    %E3%82%A2%E3%83%AA%E3%83%A2%E3%83%AB

    But when I used a tag in Latin characters (eg: “GOSICK”), it worked just fine.

    1. Try to set userobots = False in config.ini. I have tested in my PC (Windows 7 64 bit) and it is working.

      The url is encoded correctly ใ‚ขใƒชใƒขใƒซ => %E3%82%A2%E3%83%AA%E3%83%A2%E3%83%AB, only somehow mechanize/urllib cannot quote the url.
      Can you directly use the encoded value (%E3%82%A2%E3%83%AA%E3%83%A2%E3%83%AB) to search? You can use http://meyerweb.com/eric/tools/dencoder/ to encode the search string.

  4. Hi, thanks for the wonderful tool!

    I was just wandering if you could add an option to tag non-r18 pictures. I’d like to organize my pictures by something like this:

    pic/[searched tag or whatever]/r-18/artist/title
    pic/[searched tag or whatever]/general/artist/title

    Or like that. Is it possible to have r-18 pics in one folder (%r-18% tag) and all other pictures in some other folder? %r-18% only generates r-18 folder if picture is set as R18 in pixiv, but dumps all other pictures / folders at that root.

    I hope I’m making sense. If this feature could be added into config (set non-r18 folder name or something) and a tag for that could be created, this utility would be near perfect! (I was tempting to say perfect but there’s always room for improvement, no?)

    Thanks in advance.

    1. Oh and here is the error itself;

      2012-09-28 20:23:27,108 – PixivUtil20120928 – ERROR – Error at processImage(): (, AttributeError(“‘NoneType’ object has no attribute ‘findAll'”,), )
      2012-09-28 20:23:27,108 – PixivUtil20120928 – ERROR – Error at processImage(): 9175987
      Traceback (most recent call last):
      File “PixivUtil2.py”, line 537, in processImage
      File “PixivModel.pyc”, line 189, in __init__
      File “PixivModel.pyc”, line 260, in ParseTags
      AttributeError: ‘NoneType’ object has no attribute ‘findAll’
      2012-09-28 20:23:27,108 – PixivUtil20120928 – ERROR – Dumping html to: Error Medium Page for image 9175987.html
      2012-09-28 20:23:27,108 – PixivUtil20120928 – ERROR – Unknown Error: ‘NoneType’ object has no attribute ‘findAll’
      Traceback (most recent call last):
      File “PixivUtil2.py”, line 1402, in main
      File “PixivUtil2.py”, line 1079, in menuDownloadByImageId
      File “PixivUtil2.py”, line 537, in processImage
      File “PixivModel.pyc”, line 189, in __init__
      File “PixivModel.pyc”, line 260, in ParseTags
      AttributeError: ‘NoneType’ object has no attribute ‘findAll’

  8. Been getting this error for a while now:

    Image #1
    Image Id: 30436541
    Bookmark Count: -1
    Processing Image Id: 30436541
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 537, in processImage
    File “PixivModel.pyc”, line 185, in __init__
    File “PixivModel.pyc”, line 40, in __init__
    File “PixivModel.pyc”, line 56, in ParseInfo
    AttributeError: ‘NoneType’ object has no attribute ‘find’
    Error at processImage(): (, AttributeError(“‘N
    oneType’ object has no attribute ‘find'”,), )
    Dumping html to: Error Medium Page for image 30436541.html
    Error at processTags(): (, AttributeError(“‘No
    neType’ object has no attribute ‘find'”,), )
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 1404, in main
    File “PixivUtil2.py”, line 1107, in menuDownloadByTags
    File “PixivUtil2.py”, line 772, in processTags
    File “PixivUtil2.py”, line 537, in processImage
    File “PixivModel.pyc”, line 185, in __init__
    File “PixivModel.pyc”, line 40, in __init__
    File “PixivModel.pyc”, line 56, in ParseInfo
    AttributeError: ‘NoneType’ object has no attribute ‘find’
    press enter to exit.

    Any idea what might it be?

        1. The page you give is only happen if you haven’t login to Pixiv. Check you login info in config.ini on [Authentication] section, try to delete the cookie and set usessl = True if still failed.

      2. Oh yeah, gotta add, been having this weird issue where IE and Chrome don’t want to save my sessions (strangely, firefox does), I log into pixiv and as soon I go elsewhere in there I’m logged out, maybe I should try to fix that and see if it’s affecting the download app somehow.

      3. Upon further research I found out some cookies won’t work if computer has a wrong time set.
        So, issue fixed, I apologize for the inconvenience!

Comments are closed.