pixiv downloader 20131122

Change Log:

  • Fix Unicode error for Download by Tag.
  • Add feature to replace the root path of image in database.

Download link for pixiv downloader 20131122, source code in GitHub.

Mediafire folder for all the releases. Donation link on the sidebar :D .

EDIT: For those who encounter httperror_seek_wrapper: HTTP Error 403: request disallowed by robots.txt
Set userobots = False in config.ini

27 thoughts on “pixiv downloader 20131122”

  1. And you did it with python. As a fellow python user, I am impressed. Usually my rip-scripts are shabby at best, but this is a functional program.

    To anyone having problems, first you must set the mode by entering the number (1 then enter to set member id) then you can enter the member id or tags to download.

    Can you have this program just write the image links to a txt file, then they can be dumped into another downloader.

  2. using my list,txt and I get this error:

    Member Url: http://www.pixiv.net/member_illust.php?id=1889&p=1
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 411, in process_member
    File “mechanize_mechanize.pyc”, line 203, in open
    File “mechanize_mechanize.pyc”, line 255, in _mech_open
    httperror_seek_wrapper: HTTP Error 404: Not Found
    Error at processing Artist Info: (, <httperror_seek_wrapper (urllib2.HTTPError instance) at 0x36bdb58 whose
    wrapped object = <closeable_response at 0x5181af8 whose fp = <response_seek_wrap
    per at 0x517faf8 whose wrapped object = <closeable_response at 0x517f7b0 whose f
    p = >>>>, )
    1 2 3 4 5 6 7 8 9 10 11 12 13 14

    the artist is gone, but the program doesn’t just skip and move onto the next like normal

      1. They change the http code for missing/deleted member id.

        Previously was 200 (OK), now changed to 404 (Not Found). I’ll update it.

        1. While you at it: Ctrl-C stops current process and restarts the downloader. Problem is when you started downloader with console commands, like “python PixivUtil2.py -s 4 -x”, after Ctrl-C it restarts and continues to do what arguments tell it to do. Is there a way to tell it from console “Stop and do not restart” beyond sending kill signals?

  4. it wont let me download from list.txt

    before this i used pixivutil20130804, works fine…
    now pixiv wont let me use pixivutil20130804… so i DL the newest one, copas the config and list.txt

    the log says

    2013-12-11 19:37:12,266 – PixivUtil20131122 – INFO – ###############################################################
    2013-12-11 19:37:12,266 – PixivUtil20131122 – INFO – Starting…
    2013-12-11 19:37:12,275 – PixivUtil20131122 – INFO – Setting log level to: DEBUG
    2013-12-11 19:37:12,276 – PixivUtil20131122 – INFO – No default cookie jar available, creating…
    2013-12-11 19:37:12,282 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadNON-H
    2013-12-11 19:37:12,283 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadNON-H
    2013-12-11 19:37:12,283 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,285 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,285 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadNON-H
    2013-12-11 19:37:12,285 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,286 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadNON-H
    2013-12-11 19:37:12,286 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadNON-H
    2013-12-11 19:37:12,286 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,286 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,286 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,286 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadNON-H
    2013-12-11 19:37:12,288 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,288 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,288 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,289 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,289 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,289 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,289 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,289 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,289 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18G
    2013-12-11 19:37:12,290 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,290 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18G
    2013-12-11 19:37:12,292 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,292 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadNON-H
    2013-12-11 19:37:12,292 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,292 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,293 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadNON-H
    2013-12-11 19:37:12,293 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,293 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,295 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18
    2013-12-11 19:37:12,295 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadNON-H
    2013-12-11 19:37:12,295 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadNON-H
    2013-12-11 19:37:12,296 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18G
    2013-12-11 19:37:12,296 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18G
    2013-12-11 19:37:12,296 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18G
    2013-12-11 19:37:12,296 – PixivUtil20131122 – DEBUG – Sanitized Filename: [Renzan TEMP]Pixiv Batch DownloadR-18G
    2013-12-11 19:37:12,299 – PixivUtil20131122 – ERROR – Unknown Error: ‘utf8’ codec can’t decode byte 0x8f in position 0: invalid start byte
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 1680, in main
    File “PixivModel.pyc”, line 516, in parseList
    File “codecs.pyc”, line 684, in next
    File “codecs.pyc”, line 615, in next
    File “codecs.pyc”, line 530, in readline
    File “codecs.pyc”, line 477, in read
    UnicodeDecodeError: ‘utf8’ codec can’t decode byte 0x8f in position 0: invalid start byte
    2013-12-11 19:37:15,487 – PixivUtil20131122 – INFO – EXIT
    2013-12-11 19:37:15,487 – PixivUtil20131122 – INFO – ###############################################################

    thank you

    1. what is the problem id? can you upload the list.txt? Looks like your list.txt got corrupted, cos I didn’t touch the logic for parseList between those version.

        1. Looks like you save the list.txt in ANSI codec, try to save it to UTF-8 (in notepad, click save-as, choose UTF-8 encoding).

  5. Hello, I have an error which makes me confused.

    2013-12-09 04:52:46,894 – PixivUtil20131122 – ERROR – Error at process_tags(): (, ValueError(“invalid literal for int() with base 10: ””,), )
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 888, in process_tags
    File “PixivModel.pyc”, line 696, in parseTags
    ValueError: invalid literal for int() with base 10: ”
    2013-12-09 04:52:46,894 – PixivUtil20131122 – ERROR – Unknown Error: invalid literal for int() with base 10: ”
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 1756, in main
    File “PixivUtil2.py”, line 1386, in menu_download_by_tags
    File “PixivUtil2.py”, line 888, in process_tags
    File “PixivModel.pyc”, line 696, in parseTags
    ValueError: invalid literal for int() with base 10: ”

    I have read bug section of readme file but I can’t figure out what is the best solution for this problem.
    I hope you can help me about this error.

      1. Well.. an hour ago, I came back home and tried same tag and it worked.
        Maybe it was Pixiv’s problem, not the program because sometimes Pixiv doesn’t reflect user’s illust deletion immediately, so a blank remains deleted illust’s position.
        Anyway the tag is ‘魔法少女まどか☆マギカ(Mahou Shoujo Madoka Magica=Puella Magi Madoka Magica)’

  6. Anyone have an idea what is causing this?

    Error at pixiv_login(): (, <
    httperror_seek_wrapper (mechanize._http.RobotExclusionError instance) at 0x2a295
    00 whose wrapped object = <closeable_response at 0x2a314b8 whose fp = >>, )
    failed
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 1736, in main
    File “PixivUtil2.py”, line 258, in pixiv_login
    File “mechanize_mechanize.pyc”, line 203, in open
    File “mechanize_mechanize.pyc”, line 255, in _mech_open
    httperror_seek_wrapper: HTTP Error 403: request disallowed by robots.txt
    press enter to exit.

  8. I have to use “userobots = False” , But…
    I keep getting this error not sure what the problem is, I even did a fresh install of the program.

    new cookie value: 182239_2f006f052a95f88e2abe36ad11e82274
    Writing config file… Backing up old config to config.ini.bak
    Error at pixiv_login(): (, IOError(13, ‘Permission de
    nied’), )
    failed
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 1736, in main
    File “PixivUtil2.py”, line 267, in pixiv_login
    File “PixivUtil2.py”, line 289, in pixiv_process_login
    File “PixivConfig.pyc”, line 442, in writeConfig
    File “shutil.pyc”, line 298, in move
    File “shutil.pyc”, line 127, in copy2
    File “shutil.pyc”, line 82, in copyfile
    IOError: [Errno 13] Permission denied: ‘config.ini.bak’
    press enter to exit.

  10. i keep getting this error not sure what the problem is, i even did a fresh install of the program.
    just thought i’d let you known.

    2013-11-22 20:17:16,821 – PixivUtil20131122 – ERROR – Error at pixiv_login(): (, <httperror_seek_wrapper (mechanize._http.RobotExclusionError instance) at 0x2c05c00 whose wrapped object = <closeable_response at 0x2c21558 whose fp = >>, )
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 258, in pixiv_login
    File “mechanize_mechanize.pyc”, line 203, in open
    File “mechanize_mechanize.pyc”, line 255, in _mech_open
    httperror_seek_wrapper: HTTP Error 403: request disallowed by robots.txt
    2013-11-22 20:17:16,822 – PixivUtil20131122 – ERROR – Unknown Error: HTTP Error 403: request disallowed by robots.txt
    Traceback (most recent call last):
    File “PixivUtil2.py”, line 1736, in main
    File “PixivUtil2.py”, line 258, in pixiv_login
    File “mechanize_mechanize.pyc”, line 203, in open
    File “mechanize_mechanize.pyc”, line 255, in _mech_open
    httperror_seek_wrapper: HTTP Error 403: request disallowed by robots.txt

  12. Amazing feature I would be really glad to have last year. Well, next year, with planned PC update, it would be useful, too.

    Will donate later.

Comments are closed.