Change Log:
- Refactor out PixivException and PixivBrowserFactory.
- Update error codes.
- Update artist token parser to handle all img-inf in image list.
- Update filename sanitizer to remove all Unicode Control Character (Cc).
Download Link forΒ pixiv downloader 20130511, source code in GitHub.
Hey nandaka,
I think pixiv updated something, but all the thumbnails for manga submissions are being downloaded along with the full-size images. Quite annoying…
I have a problem: I was using an old version, saw that only small manga pages were downloading, deleted the folder, downloaded this version and now only avatar image downloads, while the rest says ‘already downloaded’. This behaviour continue even if I delete both the downloaded files and the log, thus leaving no trace of downloaded images in the system. On a more minor issue, avatar downloads despite downloadavatar being set to False. What can I do to resolve this?
Go into “manage database” (option d) then chose option c: clean up database.
yap yap π
if u want to delete the whole db, just delete the db.sqlite file.
Yep, that helped. Is this the intended behaviour?
Also, avatars still download. π
Yap, it will always check the avatar for the 1st page.
Set downloadavatar = False to disable avatar download.
downloadavatar _is_ set to false. It was working correctly in the old version, but downloading the new one (with old config) broke it.
Pixiv’s devs have altered their Manga mode pages again. The changes seem to block the Utility from downloading the ‘big’ images, returning a 404 when it tries to grab them. It can still download the small images on the page, however.
Also, would it be possible to have an option where, when the utility finds a file with the same name as an existing file but a different size, it renames the new version of the file instead of overwriting the old version? I’ve been running into that while updating existing grabs, and I’d kinda like to see what the differences between the image’s versions are.
I must correct myself. It *is* downloading the ‘big’ images normally, but it is also trying to grab the ‘big’ images for the *mobile* site. It’s 404ing on those, and grabbing the small 128×128 images as well as the normal ‘big’ versions.
Checking, u better create an issue on github π
EDIT: Updated in GitHub.
Thankfully these images have 128×128 in their filename…
This is brilliant. Thank you.
Almost forgot. Does r18mode incluges R18G tag in addition to just R18? If not, could it be added? Like False, True, R18G ?
Oh, and update copyright? it is two years out of date.
it was written in 2011. I don’t think I need to update the year if there is no changes since then.
It include both. You can exclude R-18G through pixiv settings in the website.
Thank you. And I sorry to inconvenience.
Greetings! I finally got tired of manually batch downloading (yes I am that lazy) so I wrote a command line program to pass parameters from the URL of my browser to the PixivUtil2.exe so I could download images and batch artists right from the browser…. however it’s downloading all the images into the browser’s directory:
http://oi40.tinypic.com/2pr5zyq.jpg
Here is my config.ini settings:
[Settings]
proxyaddress =
useproxy = False
useragent = Mozilla/5.0 (X11; U; Unix i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1
debughttp = False
userobots = True
filenameformat = %member_id%%image_id% – %artist% (%member_id%) – %title%
filenamemangaformat = %member_id%%image_id% – %artist% (%member_id%) – %title% %page_index%
timeout = 60
uselist = False
processfromdb = True
overwrite = False
tagsseparator = ,
daylastupdated = 7
rootdirectory = .
retry = 3
retrywait = 5
createdownloadlists = False
downloadlistdirectory = .
irfanviewpath = C:Program Files (x86)IrfanView
startirfanview = False
startirfanslide = False
alwayscheckfilesize = False
checkupdatedlimit = 0
downloadavatar = True
createmangadir = False
usetagsasdir = False
useblacklisttags = False
usesuppresstags = False
tagslimit = -1
writeimageinfo = False
datediff = 0
Set your
rootdirectory
in config.ini to the target directory.Hello, I’m sorry to be a bother. I adore your pixiv batch downloader but I have a question. Is it possible to download a member’s bookmarks by tag? Thank you a lot in advance. I tried looking it in the readme file but couldn’t find it. If I’m misreading something please let me know. Thank you again for everything!
Nope, currently only can get all images from the member’s Bookmark.
Ahh, ok that clears that up! Thanks for the reply and cheers for the good work!
Sometimes theres a folder.pixiv that gets left behind when I close the program suddenly.
This error occurs when the program encounters it:
2013-06-15 01:01:59,858 – PixivUtil20130511 – ERROR – Error at downloadImage(): (, WindowsError(183, ‘Cannot create a file when that file already exists’), )(http://i1.pixiv.net/img13/profile/nmtk/2064478.jpg)
Traceback (most recent call last):
File “PixivUtil2.py”, line 175, in downloadImage
WindowsError: [Error 183] Cannot create a file when that file already exists
2013-06-15 01:02:14,890 – PixivUtil20130511 – ERROR – Error at downloadImage(): (, WindowsError(183, ‘Cannot create a file when that file already exists’), )(http://i1.pixiv.net/img13/profile/nmtk/2064478.jpg)
Traceback (most recent call last):
Can you get it to attempt to remove the offending .pixiv?
Log an issue in https://github.com/Nandaka/PixivUtil2/issues
I’ll update it later on.
So. Pixiv got new look in beta. Would you update downloader preemptively?
Nope, most likely there is still a lot of changes. I’ll wait until they release the new look to all user.
excuse me, somehow I’ve got this when I’m input the member ID, how do I fix this? http://s22.postimg.org/ybei2r6tt/error.png
Weird, it is working in my side. Can you open the url on browser? What is the configured language? This application only support English and Japanese.
I can open it in the browser just fine, and it’s not just that memberID, every memberID is like that too,
and I’m already changed the language into japanese tho.. http://s10.postimg.org/hgdvpn9nd/error2.png
Hmm, that’s weird. The error is because it cannot find the elements from the page for the image list.
Can you clear the cookie in Authentication section in config.ini and upload the saved pages from your browser?
I’ll update the application to do auto dump the page if encounter AttributeError.
something like this? http://www.mediafire.com/download/ucsslww797w9iks/config.ini
yap. Upload the saved page from browser if you still encounter the problem.
here is my saved config,ini http://www.mediafire.com/download/ucsslww797w9iks
Is pixiv just really slow lately for everyone or is it just me? I recently moved and so I have a new ISP and i’m getting download speeds low as 10kbps and can’t get above 100kbps. Before I moved I was ranging from 150-400kbps which was still nowhere near my actual speed but was way better. The downloads also seem to stall a lot more now too.
Just wondering if it’s an issue with Pixiv Downloader, my ISP or everyone.
It’s just you, I got 250KB/s in my place.
Maybe your ISP doing some throttling, try to use proxy?
I had this problem for a week. I discovered that pixiv downloader went through privoxy (a local filtering proxy) for whatever reason.
Disabling the proxy (via ie settings) and everything went normal again (100KB/s to 400KB/s).
The chance it helps you are minimal but who knows…
When I try to log in, I get the message: “To prove you’re human, type in the words below.” I guess pixiv has some kind of CAPTCHA verification now? Is there anything that can be done about this?
Oh, but it worked when using HTTPS!
Good to know. For worst case scenario, you can copy the cookie data (PHPSESSID) to config.ini.
I moved my folder recently, and after a db cleaning it seems the path is stocked as absolute and not relative (even if my config is relative, i.e. rootdirectory = ..Pixiv).
As my 90Mo db.sqlite vanished in the process, is it possible to change that, or is it a limitation of sqlite?
Recovering the db? Not possible.
Storing the file path using relative path into the db? Need to check, cos currently the filename will be resolved to absolute path when it is being sanitized to avoid error.
If it is not possible, maybe an option to change manually the path would do.
Excuse me, I know it’s a bit stupid, but…
What should I type after the “Input” text?
->>http://postimg.org/image/8jbzucp15/
The number for the task you want to do.
Hi nandaka,
May I ask how to input Japanese word tags?
Thank you
See readme.txt on faq section.
Thank you very much nandaka!
i think file path config is some changed.
H:%artist% (%member_id%)%urlFilename% – %title%
former config is working alright to download ‘Dirvenumber H’ but now,
pixivutil20130511..H_%artist% (%member_id%)%urlFilename% – %title%
download path is like this ‘H_ Folder under pixivutil path’ not a ‘H Drive’
how can i use to drivepath Evironment Variables ?
You need to set it on rootdirectory portion, e.g.: rootdirectory = H:
Thx for fast Reply π
:c hmm i set the parameter
rootdirectory = H:
but still not working. same path either.
Remove H: from filenameformat/filenamemangaformat. If not working, upload your config.ini to pastebin.
oh, it’s working.
i didn’t notice rootdirectory parameter is already there. ;s
(cause i input same two parameters)
thx for keep watching it.
somehow i get this when logging in, how do i fix this? http://puu.sh/303Gt.png
Looks like your internet connection got problem. Try it again.
i believe there’s nothing wrong my internet connection… it’s works just fine in firefox and skype
Weird, as I can use it in my pc. From the error message, it said somehow the connection is interrupted from your pc. Maybe you AntiVirus/firewall closes the connection?
Can you try to delete the config.ini (backup first)? Or try it on another pc?
Oh i tried to clear the cookies and now it manages to do login. π
still downloading real slow though but at least now it could download as it’s supposed to. I believe my internet is pretty decent, still wondering why it couldn’t be faster. Btw thanks for the fast replies… π
Hi there, I’ve gotta ask you something; There’s any method or option inside the pixiv utility to download only the pics with errors? I’m talking about these http://prntscr.com/15kgaa
Try to open the error page and check what is the content there. Usually you can do it by downloading it manually using option 2.
i can’t login =/
error at pixivLogin
Please give more information! I’ve tested in my pc and it is working correctly.
Ensure you can login to the pixiv website itself. If it still fail, try to set
usessl=True
in config.ini and/or try to change your password to simpler form for testing purpose.If it still fail, upload your config.ini & pixivutil.log to pastebin/mediafire.
Looks good, thanks for all these updates. π