Don’t Hate on C

I am sure that many of us have had this discussion with someone, at some point, somwhere. Why is C still around? Straight off the bat, I’m going to show my colors: C rocks. And so do other languages. It’s true, other languages which happen to be very readable, quite forgiving and incorporation features which aid the developer (garbage collection, boundary checking, implicit references, dictionaries,…), and in many cases perform very well. As a C guy, I’m the first who used to resort to performance as an argument to dismiss other languages as inferior. But the bottomline is that isn’t necessarily true. Don’t get me wrong: good C code – yay, we can also embed assembler, a good compiler, etc. 9/10 times will yield superior results – so the performance argument is still often valid. However that doesn’t equate to all other languages being slow. Furthermore, not all applications require fine-graining the code for supereme performance. One language that has come a long way since its earlier days is Java, particularly in what refers to performance. Java used to be slow. Very slow. It always had it’s good things though: C style syntax, object oriented, garbage collection, supposedly portable to any platform with a JVM… But that JVM the code ran on, and JIT’d on. Oh, it was slow. But it has come a long way, and now in certain operations such as arithmetic ops, it’s pretty much just as fast as unoptimized C. I mean in many cases that argument is not valid anymore. Same goes for python. Man, some people dismiss python arguing it’s an interpreted language and therefore slow. First of all, implementations are compiled or interpreted. The language is a language. Secondly, and most important: not strictly true. Python can generate bytecode and run on the python VM (just read more on pypy or cpython). Other languages, functional languages for instance, aid in defining very elegant solutions in certain contexts. I’ve heard people say Haskell’s more widespread implementation, ghc, isn’t particularly fast, but I’ve also read otherwise…. My point is there’s very stiff competition, but C hasn’t gone anywhere. C is still around, and will continue to be around. Lets try to understand why.
(more…)

Read More

GDB, LD_PRELOAD and libc

OK, this is meant to be a short post, just a couple of tips regarding debugging of code you would typically run as such:
LD_PRELOAD=./somedlib.so ./yourbin
If you’re reading this your’re probably well aware about what LD_PRELOAD is for, but just in case you don´t let me explain real quick. LD_PRELOAD is an environment variable that allows you to specify the path to a certain shared object (shared library), that library will then be loaded before any other library in the LD_LIBRARY_PATH. That includes libc.so. Doing so allows you, amongst other things, to “intercept” calls to the libc. For example, I’m currently building a memory profiler – a la valgrind, although far less features, but with the goal of minimizing runtime performance impact for the target binary. To build this profiler, I’ve had to intercept malloc, calloc, realloc, free, etc, because I must account for all memory allocations. So I created my own malloc() (and others) which wraps the call to libc’s malloc and performs accounting in the meantime. The way to achieve this is by using LD_PRELOAD. By loading the shared object which contains the code of my custom malloc, I can make sure that when the target binary calls malloc(), it’s my malloc that gets executed, and not libc’s.
(more…)

Read More

Scheduling, “soft” realtime, and CPU affinity.

As many of you will already know your stock linux scheduler allows to flag a process to be scheduled with some “soft” realtime characteristics. This may be excellent in some contexts, but perhaps not a very good idea if not used wisely. More on that later…

Lets go over a few simple ideas and some history regarding process scheduling. A multiprocess OS needs to manage the processes in execution such that they can share the limited hardware resources (particularly the CPU) to execute them in a seemingly parallel manner. Back in the days of single-core monoprocessor systems (it seems like a long time ago, but most desktop computers up to 2005 or so were such) nothing was really running concurrently on your CPU, but you got that impression nevertheless. Fortunately, processors run at a frequency high enough that we can intertwine the execution of these processes into small slices or chunks of time during which that particular process will occupy the CPU. These slices need to be big enough to allow the CPU to get some real work done for the process and amortize the context switch, but not so lengthy that we cannot execute a bunch of processes in a seemingly parallel form. Just so that you get an idea, imagine 8 concurrent processes and time slices of a quarter of a second. A process will run for 0.25 secs, and stall for another 1.75 secs. If you hate your computer now, imagine what that would be!? A typical time-slice period or quantum has been 10ms (100Hz), which would allow to run 100 processes per second. That sounds better doesn’t it. It becomes obvious very quickly how important the scheduling algorithms become, deciding who should run next is definitely not random, and round-robin is simply not good enough.
(more…)

Read More

Saving RIM

Alright, please forgive me for digressing a little bit from the usual technical posts. The current tech and mobile ecosystems are experimenting an interesting transition which has made me reflect a little bit on the business side of things. I absolutely do not consider myself to be particularly business savvy, this is simply an opinion prompted by some recent announcements and articles I’ve read. Let me get started.

Unfortunately for RIM, their market share (and value) has plummeted in the past few years. It all slowly began when Apple introduced the game changing iPhone. The Blackberry until then, had been a great mobile phone with superior messaging capabilities thanks to RIM’s BB Messenger service and corporate email. But the moment the iPhone changed the landscape of mobililty for good, the blackberry became immediately obsolete. OK, maybe not obsolete, but just not up to par in anything that wasn’t messaging. RIM was in trouble, but they stil felt had a cheaper handheld device with plenty to offer. Unfortunately for RIM, the scenario was about to get a lot more dire for them. Google steps up to the plate and delivers with Android. Now the two biggest and best innovators in technology are neck deep and full-streams ahead in the mobility sector. Apple, the powerhouse and leading brand for product engineering, and Google, the most technically gifted company on the planet and undisputed kings of software, now have about 70% of US market share between the two (earned in about 5 years). Could it get worse? It did.
(more…)

Read More

Code with style (that is, googlestyle)

Just a very quick post with a great link I came across with respect to style, in particularly relevant for C/C++, Python and Objective-C (for web and browser developers, there’s also some HTML/CSS and Javascript guidelines). Named the google-styleguide it provides some great guidelines with respect to good coding practices and style, at least those practiced at google. The truth is that although style can be a little subjective as to what things are acceptable, and what not, after going through these I personally agree a lot with google’s sense of style (that’s right, in italics). People from different backgrounds may differ, but for linux coders who like the kernel conventions (talking about C/C++ here) most of these will look great to them. They even provide an emacs settings file for compliance (too bad I use vim), and a tool for cpp code.

If you’re going to write code, at least do so with style (because the code might suck anyways ;-))

Read More

Hooking Malloc (libc’s facilities vs LD_PRELOAD)

Quick post, this could develop into a series of posts (not on hooking malloc, but on memory profiling), but for now I just wanted to make a quick one. Hooking malloc is fairly common practice, in fact so common that the GNU C library allows you to define these hooks explicitly. If you define the available variables (function pointers) for hooking __malloc_hook, __realloc_hook, __free_hook, __memalign_hook, __malloc_initialize_hook, you will be able to have your hooks called everytime malloc(), free(), etc… are called. There is another method to achieve the exact same goal, which is based upon the fact that we can preload a dynamic library at runtime with LD_PRELOAD. If that library defines malloc(), free(), etc…, those will take precedence over the ones defined in libc; and you can then call the original implementation from there or write your own. This LD_PRELOAD technique is very versatile, so versatile in fact that it is very common in userspace rootkits (you should probably note that if LD_PRELOAD is set as an environment variable all programs/commands will load those libraries before running, effectively hooking malloc() or whatever, for all of them), etc. Either possibility offers a wide array of opportunities, just to name a few:

  • custom malloc implementations
  • Accounting, Statistics
  • Improved performance memory profiling

(more…)

Read More

LCOV/GCOV

Long time, I know. I’m not fulfilling my blogging duties lately, and although I’ve been busy it still not a good enough excuse. One post every three months or something like that… really not acceptable. Anyway, hopefully I can pick up the slack in the near future and make up for the lost time.

Anyway, cutting the chase. Recently I was asked at work to increase the code coverage for a bunch of regression and unit tests. The framework used is based around python’s unit test framework, and the actual code to run the test’s on is C/C++. Although some additional work had to go to create the python wrappers+bindings using boost::python, this allowed to write fairly convoluted and advanced blackbox modular tests for C/C++ code with the ease and comfort python provides. I’m personally quite comfortable and happy with the CppUnit framework, but I have to admit python allows to code things very quickly.

Prior to this little task I was familiar with gcc’s gcov and what a helpful tool it was to evaluate code coverage for unit tests:

  • % (fraction) of lines executed.
  • % (fraction) of functions/methods executed.
  • % (fraction) of classes used.
  • branching data.

What I wasn’t quite aware was of the amount of data that was actually stored into gcov’s data files (with .gcda extension). This is where lcov enters the game to generate some outstanding and impressive HTML reports for the corresponding code. I will get right back to this. Gcov is real similar to gprof and both work together very well to deliver some very detailed profiling information. They both require their corresponding flags activated at compile-time, -ftest-coverage and -pg respectively, to instrument the code appropriately and generate files necessary to reconstruct all runtime information. In the case of gcov these are the .gcno files which contain information to reconstruct the basic block graphs and assign source line numbers to blocks.

Once you run your tests, the .gcda files are created with the mentioned runtime info. Lcov will the parse everything to generate reports like the following (I am unable to post reports for our reports, but here are some publicy available examples):

(more…)

Read More

KONY 2012

This will be probably my first and last post dealing with social issues, and the like, because this is probably not the place or time for them. But given the immense success and the huge amount of opinion and criticism it has generated, I felt compelled to give my take on the KONY 2012 video. It somehow stays relevant to the blog (barely) because it has become one of the most successful stories in terms of viral information delivery of all time. Social media has changed the game, and with it has come the possibility to create a huge impact stemming from a comparably minute source. This kind of media has the intrinsic characteristic of being potentially unreliable, manipulative, etc. I don’t think this is the case with the KONY 2012 Invisible Children video, and here’s why.

Uhmmm, where do I start? There’s been a shitload of buzz generated over this KONY 2012 video, which I fully support, and I’ve also read a lot of criticism. Maybe I should stay mute cause what I think or feel isn’t worth much, but you know me, IJDGAF.

I’ve read arguments about the long term futility of the viral video, or how this is too narrowly focused given the never ending list of perils that desolate the African people. I’ve read criticism towards the American society and how “they” (we) depict this whole complex issue into a simplified Hollywoodian plot. I’ve read opinions claiming this video derails ongoing positive negotiations, and how it shifts attention from more serious problems in Africa like drought, famine, malaria or AIDS. There’s a lot that has been said about this effort by pundits and strategists, some with a very somber perspective….

It is not my intention to judge these people. They are, possibly, way more clever and prepared than I am to make any sort of evaluation, however I do think that the only important point here is morality. Obviously KONY 2012 is an act of propaganda. They are harnessing the power of social media to flood our news feeds, our twitter updates, and ultimately our brains with a cry for help on a severe issue. They’re doing so with a simplified message. They’re using powerful imagery, etc. It has all the symptoms of propaganda, because it is. But that’s cool because the cause is just. As simple as that. This guy is scum and should be brought to justice. He is not the only guy that should be brought to justice in Africa or the world, but that is beyond the point. He belongs, at the very least, in a cell for the rest of his existence, and lobbying for this to happen, and doing so with this force should never be destructively criticized. Perhaps things may be done better, perhaps there are other ways. Whichever way, it is our choice as individuals to decide what to do with the information/propaganda provided. Some will argue that propaganda is always polarized. That doesn’t matter when the issue at hand is so blatantly morally clear there is not much room for manipulation anyway.

I’m pretty much a cynic most of my days, but I’m also a firm believer that when a message is as clearly and morally sound as this one then pushing for it cannot be detrimental to any other cause. Of course we need to solve the problem of drought, famine, illnesses, education, increasing technological gap, etc in Africa. And we all know that Joseph Kony is not the first or last of Africa’s problems. That doesn’t matter, it is a problem nonetheless and must be confronted. It’s not just about Kony, it’s also about Vincent Otti, or Okhot Odiambo, or any war lord for that matter. Problems need to be addressed, and they need not be handled one at a time. I’m not saying the video, or Invisible Children is perfect. I don’t even know how they handle donations, etc. That doesn’t matter. All that matters is that we now know, and we can raise our voices for justice. And if we don’t, our conscience knows. And if this doesn’t bring a sense of urgency about helping Africa (and don’t just stop there), then what are we worth?

Sadly, 99% of the senate or congress (of any country) had no idea who Joseph Kony was before this video. They know now, and that can’t be a bad thing.

Sorry for this rant.

Read More

DDClient Gentoo init script

Hello people, I know I’ve been absent for a while and I’m sorry about that. Then again, its not like I have legions following me, so my guess is no one has really noticed :)
Anyway, I have a nice few posts coming up. Particularly one regarding a small “fork” or mod of intel’s igb NIC driver that allows parallel traffic capture from several processes from userspace with zero-copy and avoiding skbuff overhead. But I don’t want to get ahead of myself, that will be coming up, together with the github(most likely) repo and explanation.

This post is really nothing special, I just hope it can help some gentoo users like myself. My ISP recently changed my service from fixed ip to dynamic ip (yup, I somehow managed to keep a fixed IP address for something like 8 years). So my ddclient and inadyn, and as you can see in their links, they provide some decent literature and even config file generators, not bad. I went for ddclient, a perl based client. Ddclient comes ready with init script for debian, fedora, ubuntu,… users. But as you may know, gentoo users are not as common. Furthermore, init scripts are a little different in gentoo (I think in this case different is better). I want the ddclient daemon to start automatically at startup, so I needed/wanted the corresponding init.d gentoo script. I quickly hacked one up. Enjoy.

#!/sbin/runscript
# This file is part of ddclient for gentoo.
#
# Author: Jaime Fullaondo
# 

CONF=/etc/ddclient/ddclient.conf
DDCLIENT="/usr/sbin/ddclient"

PID=`ps -aef | grep "$program - sleep" | grep -v grep | awk '{print $2}'`

depend() {
	need net
	need logger
}

start() {
	ebegin "Starting ddclient"
	if [ ! -d /var/run/ddclient ]; then
		mkdir /var/run/ddclient || return 1
	fi

	DELAY=`grep -v '^s*#' $CONF | grep -i -m 1 "daemon" | awk -F '=' '{print $2}'`
	if [ -z "$DELAY" ] ; then
        	DELAY="-daemon 300"
	else
		DELAY=''
	fi
	${DDCLIENT} $DELAY
	eend $? "Failed to start ddclient"
}

stop() {
	ebegin "Stopping ddclient"
	if [ -n "$PID" ] ; then
		kill $PID
	fi
	eend $? "Failed to stop ddclient"
}

restart() {
	if ! service_stopped "${SVCNAME}" ; then
		svc_stop || return "$?"
		sleep 1
	fi
	svc_start
}

Read More