Change Log:
- Fix Issue #60: Update image parser to handle old and new layout.
- Fix html dump logic.
Download link for pixiv downloader 20141212 pixiv downloader 20141212b, source code in GitHub.
Please Donate so I can buy T2 Art Girls “Seiya no Okurimono” Noel Chat Noir 1/6 Complete Figure
EDIT: updated!! For whatever reason, some pages still uses the old layout. Go donate 😛
EDIT2: Please redownload the 20141212b, I forgot to add the layout parser logic :{
Yeah, I’m currently seeing the same problem. Donated, hope you have a great holiday!
Image #7
Image Id: 47572400
Bookmark Count: 198
Processing Image Id: 47572400
Title: かよちん
Tags : ラブライブ!, 小泉花陽, ラブライブ!100users入り, おへそ
Date : 2014-12-15 15:59:00
Mode : bigNew
Image URL :
same problem with archet and zoram
same problem
can’t download big type image
only can download manga files
any solution? can you fix this??
No rest from Pixiv… apparently now only multi-image entries can be downloaded, with single-image entries it tells the mode is “bignew” but nothing is actually downloaded. Seems a fairly new problem (last I used the software was 3 days ago and didn’t notice something like this). I also tested that some single images published on the 12th can still be downloaded, so a switch must have happened in the meantime.
P.S.: made a donation 😉
Seems to have happened just a few hours ago. I’ve got a grab that’s been running since last night, and have single images from it up to 09:15 GMT. Everything from that point on is either a multi-image submission or an animation. The single images either return a blank URL, or… register what appears to be the URL of the thumbnail for the next image in the upper navigation bar, oddly enough. Either way, nothing is downloaded.
#89
Processing Image Id: 18029570
Title: ??????????????????????????
Tags : ????????????? R-18, ???????????????? ????? ???
Date : 2011-04-09 14:14:00
Mode : bigNew
Image URL : http://i2.pixiv.net/img-inf/img/2011/04/18/01/06/16/18232765_64x64.jpg
#90
Processing Image Id: 17753367
Title: ? ??????
Tags : ??????? ??
Date : 2011-03-30 02:43:00
Mode : bigNew
Image URL :
The problem is that the code is searching for a class=”big” attribute in an tag, and there isn’t one to be found in the page’s source. Instead there’s a unique class=”original-image” attribute in the tag that references the image you want to download.
I added:
temp = page.find(‘img’, attrs={‘class’: ‘original-image’})
if temp is not None:
return str(temp[‘data-src’])
to ParseBigImages in PixivModel.py and am running a batch job on a member ID right now. It seems to be working so far, but there might be other changes to PixIV or HTML attributes that need to be captured (since I see Nadaka just added the code to find the class=”big” attribute in the last update).
And thanks for the batch downloader, Nadaka. I’m new to PixIV but I wouldn’t even try to use it without a tool like this!
The problem is that the code is searching for a class=”big” attribute in an tag, and there isn’t one to be found in the page’s source. Instead there’s a unique class=”original-image” attribute in the tag that references the image you want to download.
I added:
temp = page.find(‘img’, attrs={‘class’: ‘original-image’})
if temp is not None:
return str(temp[‘data-src’])
to ParseBigImages in PixivModel.py and am running a batch job on a member ID right now. It seems to be working so far, but there might be other changes to PixIV or HTML attributes that need to be captured (since I see Nadaka just added the code to find the class=”big” attribute in the last update).
This problem also highlights a bug, since if you have “writeimageinfo = True” in the config.ini, the call to .WriteInfo() in process_image() tries to write the default, unchanged filename “N/A” which causes an exception to be thrown.
And thanks for the batch downloader, Nadaka. I’m new to PixIV but I wouldn’t even try to use it without a tool like this!
PixivModel.py?
it’s located on library.zip??
The problem is that the code is searching for a class=”big” attribute in an tag, and there isn’t one to be found in the page’s source. Instead there’s a unique class=”original-image” attribute in the tag that references the image you want to download.
I added:
temp = page.find(‘img’, attrs={‘class’: ‘original-image’})
if temp is not None:
return str(temp[‘data-src’])
to ParseBigImages in PixivModel.py and I’ve run the code with a few “by member ID” batch jobs and it appears to work now. There might be other changes to PixIV or other HTML attributes that need to be captured (since I see Nadaka just added the code to find the class=”big” attribute in the last update), however.
The problem also highlighted a bug, which is that if you have “writeimageinfo = True” set in the config.ini, the call to .WriteInfo() in process_image() tries to write the default, unchanged filename “N/A”, which causes an exception to be thrown.
In any case, thanks for the batch downloader, Nadaka. I’m new PixIV user but I wouldn’t even try to use it without a tool like this!
ty, it’s working
the image from pixiv cannot be downloaded ==> http://i.want.tf/to/that/fMVyg.png
thanks for your work, i donated 🙂
Thanks 😀
member=1540899
Most of the images are not downloaded.
ERROR – Image: 8064225 ==> No images found for: 8064225
redownload the new archive, I forgot to add the check for the new layout.
Actually.. the error I got was slightly different…
2014-12-11 22:10:28,384 – PixivUtil20141212 – ERROR – Dumping html to: Error medium page for image 47514381.html
2014-12-11 22:10:28,384 – PixivUtil20141212 – ERROR – Error at process_tags(): (, TypeError(“‘NoneType’ object is not subscriptable”,), )
Traceback (most recent call last):
File “PixivUtil2.py”, line 771, in process_tags
File “PixivUtil2.py”, line 496, in process_image
File “PixivModel.pyc”, line 378, in ParseImages
File “PixivModel.pyc”, line 388, in ParseBigImages
TypeError: ‘NoneType’ object is not subscriptable
And knowing somebody that works there helps in situations like this.. gives me somebody to yell at “what did you change this time!”
Should be fixed with the 20141212b version 😛
Can Confirm that it does work now, thanks!
please redownload the 20141212b again :{
Hello 🙂
I’ve got a problem here, downloading from recent images (8):
Image #205
Processing Image Id: 47508610
Traceback (most recent call last):
File “PixivUtil2.py”, line 496, in process_image
File “PixivModel.pyc”, line 378, in ParseImages
File “PixivModel.pyc”, line 388, in ParseBigImages
TypeError: ‘NoneType’ object is not subscriptable
Error at process_image(): (, TypeError(“‘NoneType’ object is
not subscriptable”,), )
Dumping html to: Error medium page for image 47508610.html
Error at process_new_illust_from_bookmark(): (, TypeError(“‘N
oneType’ object is not subscriptable”,), )
Traceback (most recent call last):
File “PixivUtil2.py”, line 1687, in main
File “PixivUtil2.py”, line 1497, in main_loop
File “PixivUtil2.py”, line 1381, in menu_download_new_illust_from_bookmark
File “PixivUtil2.py”, line 967, in process_new_illust_from_bookmark
File “PixivUtil2.py”, line 496, in process_image
File “PixivModel.pyc”, line 378, in ParseImages
File “PixivModel.pyc”, line 388, in ParseBigImages
TypeError: ‘NoneType’ object is not subscriptable
press enter to exit.
I am getting the same error as well.. maybe they changed more than you thought…
they *partially* change the layout, some pages using new one, other using old one, da fuq!
Wow, that’s all sorts of wacky…
Also, from somebody I know that works there : “ust read img.big’s data-src attribute instead of img.medium”
u means works@pixiv? Ask him to provide official API doc *wink*
Anyway, the 47508610 still uses old layout, maybe because this page is still cached on their forward proxy. I’ll update the logic to handle both new and old layout.