Wednesday, October 30, 2013

[LINK] Making Software Successful

Original Link:

Making software successful

What makes good software into excellent usable software?  In my opinion software that fails to be excellent software generally lacks some key features:


A README file in the source is not sufficient for the end-user.  Good quality documentation must explain how to use all the key features in a coherent manner.  For example, a command line tool should be at least shipped with a man page, and this must cover all the options in the tool and explain clearly how each option works.

Documentation must be kept up to date with the features in the software.  If a feature is added to a program and the documentation has not been updated then the developer has been careless and this is a bug. Out of date documentation that provides wrong and useless information make a user lose faith in the tool.

Try to include worked examples whenever possible to provide clarity. A user may look at the man page, see a gazillion options and get lost in how to use a tool for the simple use-case.  So provide an example on how to get a user up and running with the simple use-case and also expand on this for the more complex use cases.  Don't leave the user guessing.

If there are known bugs, document them. At least be honest with the user and let them know that there are issues that are being worked on rather than let them discover it by chance.

Expect quality when developing

Don't ship code that is fragile and easily breaks. That puts users off and they will never come back to it.  There really is no excuse for shipping poor quality code when there are so many useful tools that can be used to improve quality with little or no extra cost.

For example, develop new applications with pedantic options enabled, such as gcc's -Wall -Wextra options.   And check the code for memory leaks with tools such as valgrind and gcc's mudflap build flags.

Use static code analysis with tools such as smatch or Coverity Scan.  Ensure code is reviewed frequently and pedantically.  Humans make mistakes, so the more eyes that can review code the better.   Good reviewers impart their wisdom and make the process a valuable learning process.

Be vigilant with all buffer allocations and inputs.  Use gcc's -fstack-protector to check for buffer overflows and tools such as ElectricFence.  Just set the bar high and don't cut corners.

Write tests.  Test early and often. Fix bugs before adding more functionality.  Fixing bugs can be harder work than adding new shiny features which is why one needs to resist this temptation and focus on putting fixes first over features.

Ensure any input data can't break the application.  In a previous role I analysed an MPEG2 decoder and manually examined every possible input pattern in the bitstream parsing to see if I could break the parser with bad data.  It took time, it required considerable effort, but it ended up being a solid piece of code.  Unsanitised input can break code, so be careful and meticulous.

Sane Defaults

Programs have many options, and the default modus operandi of any tool should be sane and reasonable.   If the user is having to specify lots of extra command line options to make it do the simplest of tasks then they will get frustrated and give up. 

I've used tools where one has to set a whole bunch of obtuse environment variables just to make it do the simplest thing.  Avoid this. If one has to do this, then please document it early on and not buried at the end of the documentation.

Don't break backward compatibility

Adding new features is a great way to continually improve a tool but don't break backward compatibility.  A command line option that changes meaning or disappears between releases is really not helpful.     Sometimes code can no longer be supported and a re-write is required.  However, avoid rolling out replacement code that drops useful and well used features that the user expects.

The list could go on, but I believe that if one strives for quality one will achieve it.  Let's strive to make software not just great, but excellent!

Tuesday, October 29, 2013

Separation, MVC, Threads and signal-slots

In Qt you can't display anything from other than the Gui thread.

I didn't know this.

In Qt you need to set the GUI class parent to the thread class to access the Main GUI Thread.

I didn't know this.

In fact I didn't know much about the current thing that I was working on. So 1. I tried to use threads to directly modify the GUI. Failed with aborts! 2. Wrote a wrapper in main that would modify the gui and called it by setting main class as parent. Still there are aborts. Then spent some time thinking about alternative solutions. And found some stuff about inter-thread communication. So stumbled upon solution of using Signal-Slots for inter-thread communication. Got a lot of weird errors involving MOC and stuff.
Further googling and furious search led to the pristine knowledge that we need a Q_OBJECT macro when we have a signal in a class. OK. Put in the macro; still more errors! Ok some more googling, and came to know that we need to qmake it all. OK, so after a super clean build, I had it all working and the UI was updated.

So now it all looks like a big hell of a mess and I need to clean it all up. So some more googling and I came to know about MVC thingie. Took me couple of days to completely understand it. ( Actually I still don't know if I have groked it all, but whatever I have consumed, works!!!) So after this, there were few days of equally furious and confused refactoring, some scope jumbles and lot of commenting around to keep track of everything. And finally, I got it all working.

I'll still need to clear it up into a separate project which I can present as a module, but this was a fortnight of intensive learning. Working on new stuff is always so interesting!

Thursday, October 24, 2013

Replace String in multiple files using Perl

1. Create a list of files that you want to replace the word in

e.g find . | grep Makefile | grep -v Makefile. > list.txt

This will list all Makefile's and exclude etc

2. create a script


while read line


perl -pi -e "s/user1/user2/g" $line


3. give executable permissions to the script

chmod +x

4. run replace script as

./ < list.txt

Wednesday, October 16, 2013

[LINK] Want to Evade NSA Spying? Don't Connect to the Internet

Original article:

Want to Evade NSA Spying? Don't Connect to the Internet

Photo: Ariel Zambelich / WIRED; Illustration: Ross Patton / WIRED

Since I started working with Snowden's documents, I have been using a number of tools to try to stay secure from the NSA. The advice I shared included using Tor, preferring certain cryptography over others, and using public-domain encryption wherever possible.

I also recommended using an air gap, which physically isolates a computer or local network of computers from the internet. (The name comes from the literal gap of air between the computer and the internet; the word predates wireless networks.)

But this is more complicated than it sounds, and requires explanation.

Since we know that computers connected to the internet are vulnerable to outside hacking, an air gap should protect against those attacks. There are a lot of systems that use — or should use — air gaps: classified military networks, nuclear power plant controls, medical equipment, avionics, and so on.

Osama Bin Laden used one. I hope human rights organizations in repressive countries are doing the same.

Air gaps might be conceptually simple, but they're hard to maintain in practice. The truth is that nobody wants a computer that never receives files from the internet and never sends files out into the internet. What they want is a computer that's not directly connected to the internet, albeit with some secure way of moving files on and off.

But every time a file moves back or forth, there's the potential for attack.

And air gaps have been breached. Stuxnet was a U.S. and Israeli military-grade piece of malware that attacked the Natanz nuclear plant in Iran. It successfully jumped the air gap and penetrated the Natanz network. Another piece of malware named agent.btz, probably Chinese in origin, successfully jumped the air gap protecting U.S. military networks.

These attacks work by exploiting security vulnerabilities in the removable media used to transfer files on and off the air gapped computers.

Since working with Snowden's NSA files, I have tried to maintain a single air-gapped computer. It turned out to be harder than I expected, and I have ten rules for anyone trying to do the same:

1. When you set up your computer, connect it to the internet as little as possible. It's impossible to completely avoid connecting the computer to the internet, but try to configure it all at once and as anonymously as possible. I purchased my computer off-the-shelf in a big box store, then went to a friend's network and downloaded everything I needed in a single session. (The ultra-paranoid way to do this is to buy two identical computers, configure one using the above method, upload the results to a cloud-based anti-virus checker, and transfer the results of that to the air gap machine using a one-way process.)

2. Install the minimum software set you need to do your job, and disable all operating system services that you won't need. The less software you install, the less an attacker has available to exploit. I downloaded and installed OpenOffice, a PDF reader, a text editor, TrueCrypt, and BleachBit. That's all. (No, I don't have any inside knowledge about TrueCrypt, and there's a lot about it that makes me suspicious. But for Windows full-disk encryption it's that, Microsoft's BitLocker, or Symantec's PGPDisk — and I am more worried about large U.S. corporations being pressured by the NSA than I am about TrueCrypt.)

3. Once you have your computer configured, never directly connect it to the internet again. Consider physically disabling the wireless capability, so it doesn't get turned on by accident.

4. If you need to install new software, download it anonymously from a random network, put it on some removable media, and then manually transfer it to the air gapped computer. This is by no means perfect, but it's an attempt to make it harder for the attacker to target your computer.

5. Turn off all auto-run features. This should be standard practice for all the computers you own, but it's especially important for an air-gapped computer. Agent.btz used autorun to infect U.S. military computers.

6. Minimize the amount of executable code you move onto the air-gapped computer. Text files are best. Microsoft Office files and PDFs are more dangerous, since they might have embedded macros. Turn off all macro capabilities you can on the air-gapped computer. Don't worry too much about patching your system; in general, the risk of the executable code is worse than the risk of not having your patches up to date. You're not on the internet, after all.

7. Only use trusted media to move files on and off air-gapped computers. A USB stick you purchase from a store is safer than one given to you by someone you don't know — or one you find in a parking lot.

8. For file transfer, a writable optical disk (CD or DVD) is safer than a USB stick. Malware can silently write data to a USB stick, but it can't spin the CD-R up to 1000 rpm without your noticing. This means that the malware can only write to the disk when you write to the disk. You can also verify how much data has been written to the CD by physically checking the back of it. If you've only written one file, but it looks like three-quarters of the CD was burned, you have a problem. Note: the first company to market a USB stick with a light that indicates a write operation — not read or write; I've got one of those — wins a prize.

9. When moving files on and off your air-gapped computer, use the absolute smallest storage device you can. And fill up the entire device with random files. If an air-gapped computer is compromised, the malware is going to try to sneak data off it using that media. While malware can easily hide stolen files from you, it can't break the laws of physics. So if you use a tiny transfer device, it can only steal a very small amount of data at a time. If you use a large device, it can take that much more. Business-card-sized mini-CDs can have capacity as low as 30 MB. I still see 1-GB USB sticks for sale.

10. Consider encrypting everything you move on and off the air-gapped computer. Sometimes you'll be moving public files and it won't matter, but sometimes you won't be, and it will. And if you're using optical media, those disks will be impossible to erase. Strong encryption solves these problems. And don't forget to encrypt the computer as well; whole-disk encryption is the best.

One thing I didn't do, although it's worth considering, is use a stateless operating system like Tails. You can configure Tails with a persistent volume to save your data, but no operating system changes are ever saved. Booting Tails from a read-only DVD — you can keep your data on an encrypted USB stick — is even more secure. Of course, this is not foolproof, but it greatly reduces the potential avenues for attack.

Yes, all this is advice for the paranoid. And it's probably impossible to enforce for any network more complicated than a single computer with a single user. But if you're thinking about setting up an air-gapped computer, you already believe that some very powerful attackers are after you personally. If you're going to use an air gap, use it properly.

Of course you can take things further. I have met people who have physically removed the camera, microphone, and wireless capability altogether. But that's too much paranoia for me right now.

Thursday, October 10, 2013

Parenting in Qt

Qt has the concept of parenting the widgets. The reason is  to arrange the widgets in a hierarchical manner.
This way you can access / find almost all child widgets using a pointer / object of the parent.

The reason I'm highlighting it here is that when you are trying to generate the UI at runtime, this ability of able to find almost any widget comes in very handy.

Recently I had to work on a small utility which basically used an external configuration file to generate the contents of a form dynamically. The external file would contain the field definition and the constraints and I would create a corresponding widget and assign it the properties.

Now to access these, I didn't even have any pointers since everything was getting generated dynamically, I used the object names.

I assigned them object names that would correspond to the data that was contained in them. Then I would use findChildren method to get the matching objects.

But you didn't hear anything about parenting here. So why parenting is so important. See, even I didn't know it. But when I ran the binary through valgrind, I saw that my good enough code is leaking like a shipwreck.

So after going through a lot of internet posts and fair amount of qt documentation, I learned that When you delete a parent widget, all its Childs are automatically deleted'.  This translates do no more memory leaks. So after spending some more time getting my concepts right, I dug in. After a stressful evening of lot of cussing and keyboard banging, I got it right. All Child widgets properly coded to have appropriate parents and proper validity and existence checks along with some clever coding made the binary leak free!!!

Isn't it awesome when such a simple thing like parenting the widgets helps in such a big way!

Friday, October 4, 2013

Moving from CentOS 5 to CentOS 6

Recently we ended up upgrading the base OS at my organization from CentOS 5 to CentOS 6. Our product is an application built in Qt with dependencies on PostgreSQL and Qwt.

Now with the old solution being based off CentOS 5, Qt 4.5 and Qwt 5.1, there were a number of stability and performance issues that were observed. So finally we decided to bite the bullet and decided to shift to CentOS 6. The reason for not proceeding earlier was there are a number of hardware peripherals and interfaces that our system uses which doesn't have upgraded drivers etc. So the task fell on me to go through all the pieces and get a good estimation of efforts required for this jump and what problems we would face.

It was decided that we would also upgrade to Qt 5 and Qwt 6 alongwith jump to CentOS 6. After getting a fresh installation ready we - me and one of my colleagues, started going through the requirements one by one. PostgreSQL was easy. Add the repository and fetch the required version. Qt was toughest. It took a lot of tweaking and installing lots of development libs etc to get it compiled properly and even then we ended up recompiling it again since we ended up with screwed up prefix setting.

Anyway after a fortnight we had the system up with Qt and Qwt compiled and the application refactored to work with Qt 5. Refactoring the application itself took about a week for two of us, and even then I don't think it's a good job. Mostly just good enough to get the UI up and get started. So after performing a quick prayer we started the App through it's startup script. Thankfully it started without any issues and we let it run for couple of days. After a steady run we are now nitpicking through the performance corners, compatibility corners and life is going to be interesting.

It was a good experience for me to see such a generation-big upgradation of a system and have a first-hand. We faced a number of problems and some of them readily solved while some other took a while to fix. At the same time it was good to know that CentOS has as good a community as any and lots of helpful people.

[External] Fingerprints are Usernames, not Passwords

Good Read !
Original Link:

As one of the maintainers of eCryptfs, and a long time Thinkpad owner, I have been asked many times to add support to eCryptfs for Thinkpad's fingerprint readers.
I actually captured this as a wishlist bug in Launchpad in August 2008, but upon thinking about it a bit more, I later closed the bug "won't fix" in February 2009, and discussed in a blog post, saying:
Hi, thanks so much for the bug report.I've been thinking about this quite a bit lately. I'm going to have to mark this "won't fix" for now. The prevailing opinion from security professionals is that fingerprints are perhaps a good replacement for usernames. However, they're really not a good replacement for passwords. Consider your laptop... How many fingerprints of yours are there on your laptop right now? As such, it's about as secret as your username. You don't leave your password on your spacebar, or on your beer bottle :-) This wikipedia entry (although it's about Microsoft Fingerprint Readers) is pretty accurate: *, I'm sorry, but I don't think we'll be fixing this for now.
I'm bringing this up again to highlight the work released last week by The Chaos Computer Club, which has demonstrated how truly insecure Apple's TouchID is.
There may be civil liberties at issue as well.  While this piece is satire, and Apple says that it is not sharing your fingerprints with the government, we've been kept in the dark about such things before.  I'll leave you to draw your own conclusions on that one.
But let's just say you're okay with Apple sharing your fingerprints with the NSA, as I've already told you, they're not private at all.  You leave them on everything you touch.  And let's say you're insistent on using fingerprint (biometric) technology because you can.  In that case, your fingerprints might identify you, much as a your email address or username identifies you, perhaps from a list.

I could see some value, perhaps, in a tablet that I share with my wife, where each of us have our own accounts, with independent configurations, apps, and settings.  We could each conveniently identify ourselves by our fingerprint.  But biometrics cannot, and absolutely must not, be used to authenticate an identity.  For authentication, you need a password or passphrase.  Something that can be independently chosen, changed, and rotated.  I will continue to advocate this within the Ubuntu development community, as I have since 2009.

Once your fingerprint is compromised (and, yes, it almost certainly already is, if you've crossed an international border or registered for a driver's license in most US states and many countries), how do you change it?  Are you starting to see why this is a really bad idea?
There are plenty of inventions that exist, but turned out to be bad ideas.  And I think fingerprint readers are another one of those.

This isn't a knock on Apple, as Thinkpad have embedded fingerprint readers for nearly a decade.  My intention is to help stop and think about the place of biometrics in security.  Biometrics can be use used as a lightweight, convenient mechanism to establish identity, but they cannot authenticate a person or a thing alone.
So please, if you have any  respect for the privacy your data, or your contacts' information, please don't use fingerprints (or biometrics, in general) for authentication.