Saturday, April 15, 2017

A bit of script for downloading images from a website

So I wanted to download only images from this one website for archival usage. After looking at all ready available page, I couldn't find anything suiting for my purpose. So I decided to throw in a little shell-script.
It went on as follows:
The pages are incremental in index. So I can use a while loop to fetch all pages one by one. Done.
Need to fetch the page. Wget is fine. Done.
Then I need to look for the image URL in retrieved HTML. Hmm. Bit of grep with cut does that. Done.
Next get the actual image. Again prepare the image URL and wget. Done.

Error handling? Gaah. Nothing since this is not that trivial.

After testing for few pages it was golden.

So I put it up with counter loop of hundred images at one time. That's because this script is kind of slow since I don't know nor care to put in multithreading in shellscript. Gaah.
Plus as I later found out having 100's counter is good since wget hung out when the net went down for a moment, and I had to restart the script from that page count.

And somehow https proxy is set on my laptop. Why, god only knows. Below is the script.

###########################
#!/bin/bash

export https_proxy=""
count=11000

while [ $count -gt 10800 ]
do

echo "Downloading page $count..."

content=$(wget https://www.mygallery.com/photo/$count -q -O -)
#echo $content
echo "Page downloaded!"

line=$(echo $content | grep -b -o "class=\"main-photo\"")
offset=$(echo $line| cut -d : -f1)
((offset1=offset+50))
img="https://www.mygallery.com/resized/"$(echo $content | cut -c$offset-$offset1 | tail -c18)

echo "Getting Image: "$img"..."
wget $img -q
((count=count-1))
#echo "count: $count"

done
exit 0
###########################


First big problem with our bike Hero Duet

Yesterday we went out for an outing on our bike. It was about 90kms round trip. We left in morning and planned to be back by noon. Going was nice, with normal weather. After reaching the place and sightseeing for about couple of hours we left by 11:30AM.
We could feel the heat. It was not scorching yet, but we felt like we should have left an hour early, since it was easily an hour and half's ride back with all the traffic. So we were only 10kms away when the scooter lost power. We tried electric start first, but it won't start. Then we tried kick start, and still nothing. So we searched the net and found a mechanic and had him sent someone over.
Then after 45 minutes couple of young mechanics reached us. After fiddling with the scooter they managed to kick start it. Initial prognosis was dead air filter. So we rode to their garage, and got it replaced in about half-an-hour. In Hero Duet the filter comes as a block of striped foam. In bikes it's often a round hose-like design. It was pretty gunky from what we saw. The replacement cost us about 250Rs, that plus the mechanic visit of 150Rs, totalled to 400Rs.
At least now the scooter runs better.

Tuesday, April 11, 2017

LG phones bootloop/ restarts/ slowness issues

Been reading about the troubles with LG. To be frank, LG actually has crappy phones. Their mid range is non-existent and their high end is in search of identity. Add in these QA issues. I mean come on.... I have seen phones getting literally cooked at about 45-50 degrees when gaming and they did not fail - e.g. my Nubia Z9 mini and few more older phones. If LG cannot make a phone motherboard, they need to be put out of phone making business.
Latest news is they have a Class Action lawsuit in US. Hmmm let everybody know LG hardware is crap.
For comparisons we have not heard such things from almost all other manufacturers. And please leave aside chinese knock-offs. They can't even align their plastic edges. Haha.
Cheers.

Monday, April 10, 2017

The death of Canonical Unity (and Mir)?

It was a surprise announcement. But then again maybe Mark Shuttleworth saw the writing on the wall, and thought about stopping the bleeding. And so the culling began...
I was kind of shocked when I read Jono Bacon's post. Why? Because I'm not much online on G+. And Jono's was the latest post on Planet Ubuntu.
Anyway I did try Unity 8, and it did look slick. Funny think is probably KDE has similar graphics and animations for ages, and they never appealed to me. Frankly I just hate the way KDE looks. The odd highlights, the cramped fonts, the weird effects I immediately disable... gaah!!! What I saw in Unity 8 though, I liked it. Anyway again Canonical has few resources and they are better spent on improving Canonical's longevity. Because It's Canonical that took Linux to the masses and not Redhat or SuSe or anybody else.
For that..... Salute!!!

Thursday, April 6, 2017

Boot Ubuntu Fast

Recently I noticed that my Ubuntu installation has become slow to boot. It has seen couple of dist upgrades and a lot of application install/remove's.
So I decided to dig in and see if there's anything I can do about it. First thing is how to measure current boot time. After a quick google search I ended up with following commands.
systemd-analyze
systemd-analyze blame
systemd-analyze critical-path
systemd-analyze plot > plot.svg

systemd-analyze prints something like:
Startup finished in 9.002s (kernel) + 22.322s (userspace) = 31.325s
Which means system is taking 31.325 Secs to boot, out of which kernel part is 9Sec and remaining is user part.

systemd-analyze blame prints list of applications/ services according to time taken to load.
critical-path option shows a critical path graph and highlights applications/services which are taking more than average time.
plot option plots a nice graph of the booting process.

There are other ways too. E.g. if you want to see more in what kernel is doing for 9 secs, you can look at output of dmesg command.
For more details on userspace operations there's journalctl.

Anyway to start with I noticed my machine taking almost a minute to boot. After going through above data I noticed following:
Networkmanager wait online service taking long - not needed - masked it.
modem-manager service not needed - mask it.
VirtualBoxDriver taking more time in dmesg logs - don't need VirtualBox anymore - purge it.
Some more minute tuneups - and now my system boots in 30-40 secs, which is huge improvement from a full minute.

Wednesday, April 5, 2017

Using Mate and WTF is wrong with Nautilus!!!

Well, I have been using Ubuntu 16.04, then 16.10 and now about to upgrade to 17.04 whenever it gets released. Problem is with Unity is we get updated Nautilus. And over the time I have seen the changes that have made me say WTF!!!
More on this later. But this caused me to switch to Mate. I didn't want to replace my existing installation because there are about a dozen apps and half-dozen drivers that I have installed for office work. And I don't want to go through re-configuring everything again. So I just installed Mate desktop and been using it.
Frankly, Mate is killer. It's lightweight and so fast. Very responsive. I didn't have much problems with Unity's search the app metaphor since I pin half a dozen apps which I use regularly, and that's it. But Unity has been pretty sluggish and I used it only because the indicators and stuff keeps running without breaking over updates.(Have used XFCE and LXDE, both exhibited this issue.)
Anyway, so Mate it is for me.
Lets get back to Nautilus. Mate's using fork of Nautilus from Gnome 2 which they have named Caja. Now, the Gnome dev's have really messed up Nautilus. First they trimmed the top bar. That means the various options and view types are gone.
Then the split view is gone.
Then they replaced the copy/move dialogs with silly popups. I mean really??? When I have two windows side by side and when I copy a big file from one to another these two shitty popups show up both the places. WTF??? And they don't even go away when the copy is complete!!! Gods!!!
Add to the insult, previews stopped a month back and whatever I try, no more previews. Damn! Got fed up!!!
Anyway's am going to install Mate on my home laptop too since I cannot take this shit anymore. F U Gnome 3 devs. Take back your free shit!!!