Change log:
- Update makeFilename
- Clean up double space
- Update filenameFormat
- %urlFilename% ==> the actual filename stored in server without the file extensions
- Add filenameMangaFormat
- %page_index% ==> for manga mode, add big in the filename.
- %page_number% ==> for manga mode, add page number with 0-index.
- %page_big% ==> for manga mode, add page number with 1-index.
Please note that the createmangadir option needs to have %urlFilename% in the filenameMangaFormat to create the directory, or you can update the format from %image_id% to %image_id%%page_index%.
For issues or new feature request, you can put it in the comment or through GitHub tracker.
Download link for PixivUtil20120724, and source code.
Got some errors just now, seems like I can no longer download pictures. Log file gave:
2012-08-03 03:00:15,756 – PixivUtil20120724 – ERROR – Error at processImage(): (, AttributeError(“‘NoneType’ object has no attribute ‘string'”,), )
2012-08-03 03:00:15,773 – PixivUtil20120724 – ERROR – Dumping html to: Error Medium Page for image 19010647.html
2012-08-03 03:00:19,506 – PixivUtil20120724 – ERROR – Error at processImage(): (, AttributeError(“‘NoneType’ object has no attribute ‘string'”,), )
2012-08-03 03:00:19,523 – PixivUtil20120724 – ERROR – Dumping html to: Error Medium Page for image 19010647.html
2012-08-03 03:00:23,240 – PixivUtil20120724 – ERROR – Error at processImage(): (, AttributeError(“‘NoneType’ object has no attribute ‘string'”,), )
2012-08-03 03:00:23,256 – PixivUtil20120724 – ERROR – Dumping html to: Error Medium Page for image 19010647.html
2012-08-03 03:00:26,927 – PixivUtil20120724 – ERROR – Error at processImage(): (, AttributeError(“‘NoneType’ object has no attribute ‘string'”,), )
2012-08-03 03:00:26,944 – PixivUtil20120724 – ERROR – Dumping html to: Error Medium Page for image 19010647.html
2012-08-03 03:00:30,887 – PixivUtil20120724 – ERROR – Error at processImage(): (, AttributeError(“‘NoneType’ object has no attribute ‘string'”,), )
2012-08-03 03:00:30,911 – PixivUtil20120724 – ERROR – Dumping html to: Error Medium Page for image 19010647.html
2012-08-03 03:00:30,913 – PixivUtil20120724 – ERROR – Giving up image_id: 19010647
2012-08-03 03:00:30,913 – PixivUtil20120724 – INFO – Processing Member Id: 557399
2012-08-03 03:00:34,029 – PixivUtil20120724 – ERROR – Error at processImage(): (, AttributeError(“‘NoneType’ object has no attribute ‘string'”,), )
2012-08-03 03:00:34,046 – PixivUtil20120724 – ERROR – Dumping html to: Error Medium Page for image 28020592.html
2012-08-03 03:00:37,709 – PixivUtil20120724 – ERROR – Error at processImage(): (, AttributeError(“‘NoneType’ object has no attribute ‘string'”,), )
2012-08-03 03:00:37,726 – PixivUtil20120724 – ERROR – Dumping html to: Error Medium Page for image 28020592.html
and so on. Haven’t been able to download anything since 3:00 AM for some reason
Likewise. Maybe some recent changes in Pixiv? I was trying to download from my bookmarks and I got this in the logfile.
2012-08-03 10:38:53,171 – PixivUtil20120724 – INFO – Starting…
2012-08-03 10:38:53,342 – PixivUtil20120724 – INFO – Only process member where day last updated >= 7
2012-08-03 10:39:06,140 – PixivUtil20120724 – INFO – Starting…
2012-08-03 10:39:06,171 – PixivUtil20120724 – INFO – Only process member where day last updated >= 7
2012-08-03 10:39:06,171 – PixivUtil20120724 – INFO – Using Username: cpgendo
2012-08-03 10:39:06,171 – PixivUtil20120724 – INFO – logging in with saved cookie
2012-08-03 10:39:06,171 – PixivUtil20120724 – INFO – Trying to log with saved cookie
2012-08-03 10:39:25,265 – PixivUtil20120724 – INFO – Logged in using cookie
2012-08-03 10:39:30,608 – PixivUtil20120724 – INFO – New Illust from Bookmark mode.
2012-08-03 10:39:53,265 – PixivUtil20120724 – ERROR – Error at processImage(): (, AttributeError(“‘NoneType’ object has no attribute ‘string'”,), )
2012-08-03 10:39:53,265 – PixivUtil20120724 – ERROR – Dumping html to: Error Medium Page for image 29089153.html
2012-08-03 10:39:53,265 – PixivUtil20120724 – ERROR – Error at processNewIllustFromBookmark(): (, AttributeError(“‘NoneType’ object has no attribute ‘string'”,), )
2012-08-03 10:39:53,265 – PixivUtil20120724 – ERROR – Unknown Error: ‘NoneType’ object has no attribute ‘string’
Traceback (most recent call last):
File “PixivUtil2.py”, line 1423, in main
File “PixivUtil2.py”, line 1225, in menuDownloadNewIllustFromBookmark
File “PixivUtil2.py”, line 939, in processNewIllustFromBookmark
File “PixivUtil2.py”, line 554, in processImage
File “PixivModel.pyc”, line 169, in __init__
File “PixivModel.pyc”, line 217, in ParseInfo
AttributeError: ‘NoneType’ object has no attribute ‘string’
2012-08-03 10:40:10,312 – PixivUtil20120724 – INFO – EXIT
2012-08-03 10:40:10,312 – PixivUtil20120724 – INFO –
they changes the page again… I will update the parser
I get this error when trying to download from a specific user:
PixivDownloader2 version 20120724
https://nandaka.wordpress.com/tag/pixiv-downloader/
Reading config file… Error at loadConfig(): (, No option ‘filenamemangaformat’ in section: ‘Settings’, )
Failed to read configuration.
Writing config file… done.
done.
Creating database… done.
Importing list… Found 2822 items done.
Updated 2822 items.
Using Username: stevej018
logging in with saved cookie
Trying to log with saved cookie
done.
PixivDownloader2 version 20120724
https://nandaka.wordpress.com/tag/pixiv-downloader/
1. Download by member_id
2. Download by image_id
3. Download by tags
4. Download from list
5. Download from online user bookmark
6. Download from online image bookmark
7. Download from tags list
8. Download new illust from bookmark
9. Download by Title/Caption
10. Download by Tag and Member Id
————————
d. Manage database
e. Export online bookmark
x. Exit
Input: 1
Member id: 298409
Start Page (default=1):
End Page (default=0, 0 for no limit):
Processing Member Id: 298409
Reading config file… done.
Page 1
Member Name : dai
Member Avatar: http://i1.pixiv.net/img17/profile/04061102/4389316.png
Member Token : 04061102
HTTP Error 504: Gateway Time-out
504
1 2 3 4
HTTP Error 504: Gateway Time-out
504
1 2 3 4
HTTP Error 504: Gateway Time-out
504
1 2 3 4
HTTP Error 504: Gateway Time-out
504
Traceback (most recent call last):
File “PixivUtil2.py”, line 450, in processMember
File “PixivUtil2.py”, line 221, in downloadImage
File “PixivUtil2.py”, line 221, in downloadImage
File “PixivUtil2.py”, line 221, in downloadImage
File “PixivUtil2.py”, line 120, in downloadImage
File “mechanize_mechanize.pyc”, line 203, in open
File “mechanize_mechanize.pyc”, line 255, in _mech_open
httperror_seek_wrapper: HTTP Error 504: Gateway Time-out
Error at processMember(): (,
<httperror_seek_wrapper (urllib2.HTTPError instance) at 0x28ae730 whose wrapped
object = <closeable_response at 0x28b9508 whose fp = <response_seek_wrapper at
0x28b5dc8 whose wrapped object = <closeable_response at 0x28b5da0 whose fp = >>>>, )
Dumping html to: Error page for member 298409.html
Traceback (most recent call last):
File “PixivUtil2.py”, line 1408, in main
File “PixivUtil2.py”, line 1076, in menuDownloadByMemberId
File “PixivUtil2.py”, line 450, in processMember
File “PixivUtil2.py”, line 221, in downloadImage
File “PixivUtil2.py”, line 221, in downloadImage
File “PixivUtil2.py”, line 221, in downloadImage
File “PixivUtil2.py”, line 120, in downloadImage
File “mechanize_mechanize.pyc”, line 203, in open
File “mechanize_mechanize.pyc”, line 255, in _mech_open
httperror_seek_wrapper: HTTP Error 504: Gateway Time-out
press enter to exit.
HTTP Error 504: Gateway Time-out << got problem with your internet connection, try again?
I have a question regarding to manga.
When using option number 1, is there a way to:
– Exclude manga and download pictures only?
– Download each manga in separate sub folders?
If possible, I want to download manga and picture separately or download pictures only excluding manga.
Once again, thank you for the help.
> – Exclude manga and download pictures only?
This one is not supported.
> – Download each manga in separate sub folders?
Either you set
createmangadir = True
, or use different filenameMangaFormat from filenameFormat.For example: if you set
filenameFormat = %member_token% (%member_id%)%urlFilename%
, then you can set thefilenameMangaFormat = manga%member_token% (%member_id%)%urlFilename%
or%member_token% (%member_id%)Manga%urlFilename%
Yep, it worked. I’ve set the createmanga to true and filenamemanga. The manga images saves in separate sub folders.
One more thing, is there way to not download “folder” image? you know, the artists profile pics.
Anyway thank you very much.
downloadavatar = False, should be in the readme.txt
Hi,nandaka,I’m here again to report something caused by the internet connection.
Downloaded image file is broken while you’re using a not stable link (I’m in China,GFW),but the downloader said it’s already downloaded.
After update to 20120724 from 20120704,I can’t log in,back to old version and that’s okay.
Thank you!
2012-07-26 21:11:36,605 – PixivUtil20120724 – INFO – Starting…
2012-07-26 21:11:36,615 – PixivUtil20120724 – INFO – Only process member where day last updated >= 7
2012-07-26 21:11:36,615 – PixivUtil20120724 – INFO – Using Username: *************
2012-07-26 21:11:36,625 – PixivUtil20120724 – INFO – logging in with saved cookie
2012-07-26 21:11:36,630 – PixivUtil20120724 – INFO – Trying to log with saved cookie
2012-07-26 21:11:56,072 – PixivUtil20120724 – INFO – Cookie already expired/invalid.
2012-07-26 21:11:56,078 – PixivUtil20120724 – INFO – Log in using form.
2012-07-26 21:13:55,092 – PixivUtil20120724 – ERROR – Error at pixivLogin(): (, timeout(‘timed out’,), )
2012-07-26 21:13:55,128 – PixivUtil20120724 – ERROR – Unknown Error: timed out
Traceback (most recent call last):
File “PixivUtil2.py”, line 1392, in main
File “PixivUtil2.py”, line 287, in pixivLogin
File “mechanize_mechanize.pyc”, line 541, in submit
File “mechanize_mechanize.pyc”, line 203, in open
File “mechanize_mechanize.pyc”, line 230, in _mech_open
File “mechanize_opener.pyc”, line 204, in open
File “mechanize_urllib2_fork.pyc”, line 457, in http_response
File “mechanize_opener.pyc”, line 221, in error
File “mechanize_urllib2_fork.pyc”, line 332, in _call_chain
File “mechanize_urllib2_fork.pyc”, line 571, in http_error_302
File “mechanize_mechanize.pyc”, line 203, in open
File “mechanize_mechanize.pyc”, line 230, in _mech_open
File “mechanize_opener.pyc”, line 204, in open
File “mechanize_http.pyc”, line 201, in http_response
File “mechanize_http.pyc”, line 169, in parse_head
File “mechanize_response.pyc”, line 195, in read
File “socket.pyc”, line 380, in read
File “httplib.pyc”, line 541, in read
File “httplib.pyc”, line 592, in _read_chunked
File “httplib.pyc”, line 647, in _safe_read
File “socket.pyc”, line 380, in read
timeout: timed out
2012-07-26 21:21:08,960 – PixivUtil20120724 – INFO – EXIT
weird, there is no changes for the logging in. Try to set usessl to true?
using ssl,problem sloved.
But is there a way to fix downloaded image broken?
Thanks
nope, need to be re-downloaded.
opps,but the file info has been already written into the db.sqlite,it told you the file has been downloaded,so I can’t download new illustrations from bookmarks which image file is broken…
Either you set overwrite = True in config.ini, or you delete the corrupted download and do db cleaning (d. Manage database => c. Clean Up Database) to remove the entried. Please note the db cleaning will try to detect if the file is exists/not moved/not renamed in the hdd.
Got it,Thank you!
Hi, thanks for the update, but i have one questions. I’m trying to download the images from Pixiv by a tag name, and whenever i choose #3 option, which is download by a tag, it asks me bookmark counts. What do i have to put in the bookmark counts ? Also, when I successfully download the images by a tag, images somehow always separated by their own to different folders with the artists name on it. As a result, I always get a hundreds of different folders with a few pictures that are related to one tag. is there any way to put every images downloaded by tags to one folder ?
put 0 to ignore the bookmark count, it is the little blue star if you go to the search page at pixiv.
For the filename, please check the filename format, by default it will split based on artist/member_id (check for directory separator or ”), just remove it or change to your taste. Please refer to readme.txt for more information.
Nice update with the manga mode! However, ever thought of implementing a numbering system that starts from “00” or “01” rather than 0 and 1? I mean…
123456_big_01 rather than pic123456_big_1
123456_big_02 rather than pic123456_big_2
123456_big_03 rather than pic123456_big_3
and so on.
This would be useful with mangas that are more than 10 pages long, since programs tend to list them wrongly (first 1, then 10-19, then 2, then 20-29…)
which program do you use? The file sorting is showed correctly in explorer. The downloading is not affected by the numbering.
Thanks so much for this update. I look forward to trying this out. I know you didn’t want to add support for leading zeros, but there are all sorts of apps that have poor support for non-leading-zero file name sequences, so a sequence with 20 file would show up as:
1
10
11
12
13
14
15
16
17
18
19
2
20
3
4
5
6
7
8
9
…which is why I feel adding leading zeros is a much better default. It always works.