UPDATED here
Long time no update, kinda busy looking for a job here, just recently graduated from my uni 🙂
Changelog:
- 20101024:
– by Nandaka:
– Fix error when dumping html.
– Change exit behaviour, no need to press enter twice if you select exit. - 20101005:
– by Yavos:
– added commandline-option: <-n number> to temporarily overwrite config-settings for <numberofpage>
– added new options in config.ini (for details see readme.txt)
(<createDownloadLists>, <downloadListDirectory>, <startIrfanView>, <startIrfanSlide>, <IrfanViewPath>)
– added commandline-option <-i> to automatically start IrfanView with list of downloaded Images
(this option only takes effect when <startIrfanView> and <startIrfanSlide> from config.ini are not set True)
– images will get the extension <.pixiv> until they are complete (prevent incomplete image-downloads)
– fix: all downloaded images will be added in DB (until now it was limited to images downloaded by member_id/list of member_ids)
(note: you can reset it again by setting <Yavos = False> in sourcecode line 27)
– added feature: when (directly or indirectly) set in config.ini pixivUtil will create lists of your downloaded images in <Downloaded_on_YYYY_MM_DD.txt> (using date when pixivUtil is launched)
– added feature: start IrfanView when exiting pixivUtil (Slideshow will be started before normal IrfanView-Window)
– added feature: typing <-all> in menu will temporarily activate/deactivate pagelimit set in config.ini (it will set commandline-option <-n 0>)
Download link here (compiled windows binary) , source code here.
Hi~
When previous download fails, in case it leaves an incomplete file, I added the following lines in downloadImage() section:
if not overwrite and os.path.exists(filename) and os.path.isfile(filename) :
# print “localfilesize = “, localfilesize
# print “remotefilesize = “, filesize
if int(filesize) == os.path.getsize(filename) :
print “tFile exist!”
return 0 #Yavos: added 0 -> updateImage() will be executed
else :
os.remove(filename)
try:
save = file(filename + ‘.pixiv’, ‘wb+’, 4096)
except IOError:
msg = ‘Error at downloadImage(): Cannot save ‘ + url +’ to ‘ + filename + ‘ ‘ + str(sys.exc_info())
safePrint(msg)
__log__.error(unicode(msg))
save = file(os.path.split(url)[1], ‘wb+’, 4096)
sorry about the indentation, wordpress eats theme… 🙁
where is downloadImage() section, and where to enter this code?
@Ivan
In PixivUtils2.py, python source code.
Thanks, please check this:
def downloadImage(url, filename, referer, overwrite, retry):
try:
print ‘Start downloading…’,
if not overwrite and os.path.exists(filename) and os.path.isfile(filename) :
# print “localfilesize = “, localfilesize
# print “remotefilesize = “, filesize
if int(filesize) == os.path.getsize(filename) :
print “tFile exist!”
return 0 #Yavos: added 0 -> updateImage() will be executed
else :
os.remove(filename)
try:
save = file(filename + ‘.pixiv’, ‘wb+’, 4096)
except IOError:
msg = ‘Error at downloadImage(): Cannot save ‘ + url +’ to ‘ + filename + ‘ ‘ + str(sys.exc_info())
safePrint(msg)
__log__.error(unicode(msg))
save = file(os.path.split(url)[1], ‘wb+’, 4096)
print “tFile exist!”
return 0 #Yavos: added 0 -> updateImage() will be executed
All Right?
@Ivan
Those lines must follow these lines in downloadImage():
try:
filesize = res.info()[‘Content-Length’]
except KeyError:
filesize = 0
res.info()[‘Content-Length’] retrieves filesize of image from serverside.
os.path.getsize(filename) gets filesize of the image already downloaded.
Any way, I modified the PixivUtil2.py quite a bit, the code becomes very dirty looking… It might not work if you just copy + paste “this” block of code to the preferred place. So…, better wait for nandaka’s version.
updated: http://nandaka.wordpress.com/2010/12/02/pixiv-downloader-20101202/
shuffling the order so it can get the remote filesize first then compare it with the local one, please notify if there is any bug :0
Help please, how you change 澪秋山 to % E7% A7 % 8B% E5% B1% B1% E6% BE% AA
i don’t think it
sorry for my bad english, because i’m russian 🙂
If I’m not read it wrong, you want to change from 澪秋山 into html encoded format (%E7%A7%8B%E5%B1%B1%E6%BE%AA)?
You can try http://meyerweb.com/eric/tools/dencoder/ , just paste the word and click encode.
thanks
this is work 🙂
So what about introducing a new program functions as a download picture from “bookmark” jumps to the artist, for example http://www.pixiv.net/bookmark.php?id=59968
I think it would be nice to introduce this =)
And again, sorry for my bad English)
Whenever I set createdownloadlists = True uselist = True, the moment I run the program, it automatically sets both back to False.
What am I doing wrong?
try to open the config.ini using wordpad.
You App save my day, works great 😀
Only have one problem (under linux), the app doesn’t create correctly the directories. Instead of create the image into a directoy, a file is created with the name of the directory plus image name.
That’s my filenameformat:
filenameformat = %artist% (%member_id%)(%image_id%) %title%
Btw, the app works great under windows and create the directories correctly, the problem is only under linux (ubuntu 10.10)
Any fix for that?
try to replace “” to “/”, linux has different directory separator from windows.
filenameformat = %artist% (%member_id%)/(%image_id%) %title%
Different person here.
I went through and cleaned up the windows separators, but I still seem to have a problem.
The “/” is being sanitized preventing artist directories from being made.
Any advice?
try to modify the folder sanitizer in the source code, you can remove “/” from the list.
modify this line
__badchars__ = re.compile(r'^.|.$|^ | $|^$|?|:||/|||*|"') #|'')
into
__badchars__ = re.compile(r'^.|.$|^ | $|^$|?|:|||*|"') #|'')
I noodled around with a code a bit more and got it working comfortably.
Thanks for all the work you’ve put into this — it’s really useful!
I love the new changes but I seem to have a problem trying to turn overwrite to false. It just goes no matter what.
False overrides without checking if the file exists and True yields the same error windows does when you try to rename a file to an existing one via explorer.
can you paste the error?
Well now it manages to notice the ones I’ve downloaded just now. it still doesn’t acknowledge the existence of the things i downloaded previous to now.
Would there be any way I can have it see the other things I’ve downloaded so I can avoid downloading the artists again?
As long you save it to the same folder and doesn’t change the filename format (so it will generate identical filename with the previous one), then it will try to check if the file has been downloaded or not from the filename.
I don’t know whether or not I’m working it right but the DB manager doesn’t seem to function.
More details please, can you post the log?
going to the DB manager function, i try to choose say: “Show all member” and it prints “member_id name save_folder created_date last_update_date”. If i choose to show images by member ID and type in what should be a valid ID of a member I’ve downloaded images for, it just reprints the menu with nothing in between.
The log doesn’t show that I’ve done anything aside from log in and exit:
pixivutil.log (piece)
2010-10-24 04:24:08,019 – PixivUtil20101024 – INFO – Using Username: [username]
2010-10-24 04:24:08,019 – PixivUtil20101024 – INFO – logging in
2010-10-24 04:24:11,368 – PixivUtil20101024 – INFO – Logged in
2010-10-24 04:27:12,838 – PixivUtil20101024 – INFO – EXIT
can you try to download some images and try to do it again?