Open Source

Getting the troops mobilized

Saturday, February 25th, 2006

I gave a talk last Tuesday called “Open Source Development: A Rapid Introduction.”

Here’s the blurb I sent out when I advertised the talk:

Have you wanted to work on open source projects, but just don’t know how to get started? This talk will provide the basics you need to start working on open source software the next time you sit down at your computer.

In particular, this talk will cover:

(1) A brief overview of open source development in the industry and press.

(2) The UNIX development platform. A brief and whimsical overview of the UNIX shell, its surrounding tools, and the power of shell scripting. (Useful to anyone wanting to learn more about UNIX tools.) Learn how to do in a few lines of shell script what you only thought was possible with a big, extravagant hundred- or thousand- line program, and learn why so many of the world’s best hackers hack on a *nix system.

(3) The basics you need in order to hack on open source project: how mailing lists, wikis, bugzillas, source code revision systems all come together to form an organic code management process, and how to get started using those tools and others to learn about a project and what parts need development work done. This will include a brief introduction to CVS.

(4) The last part of the talk will involve actually watching open source development in action. In particular, the speaker will checkout some code from a source repository, make a change to it, create a patchfile from that change, then track down the mailing list or bugzilla related to the project and submit the patch to the maintainer. You will actually get to see open source “in action,” and will want to do it right when you get home!

If you’re a Windows or Mac OS developer who has always wanted to learn more about *nix systems, or if you’re a developer who wants to either take his own project open source or work on existing open source projects, this talk is for you.

Of course, if you’re someone who is just interested in the concept of open source, this talk will give you an inside look at “how things get done” in this community.

This talk was at least partially inspired by Nat Friedman’s blog post, check it out here:

“How to be a Hacker”

The talk was a general success, I think. About 12 people attended. You can see the talk in PDF or ODP formats, and you can also download the patches I wrote specifically for the talk to illustrate “open source development in action.” The patches are pretty stupid, but do illustrate the point, at least. Plus, each of the three patches served one of my own goals (hacking my CPU frequency scaler, fixing a gnome-terminal bug, and hacking galeon “for fun”), so that’s that. I think it’d be cool to give this talk again (maybe a little refined to include less basic UNIX tools and more hacking stuff) at a later date. We’ll see.

Instilling Doubt

Monday, November 28th, 2005

A pretty insightful comment coming from an anonymous poster (in the comments section),

I always wonder about these Linux puff articles — what motivates the writers to pimp OSs that any experienced person knows will be nosebleeds for 90%+ of the novices trying to do a real, complete migration from XP?

Linux never has been, nor will it ever be, an effortless turn-key OS for all-purpose desktop (let alone laptop!). Ubuntu is an improvement, but it’s still Linux — which, if you consider the costs of replacing unsupported hardware and your time struggling with setup at about two cents an hour, is the most lavishly expensive OS ever devised.

I’ve been screwing around with Ubuntu 5.04 for about five months, all but living on the official support forum, which provides — at best! — solutions for only about a quarter of the hardware and software problems I’ve encountered, and there are tons of them.

When 5.10 was released, the forum exploded with bug reports from those who had upgraded from 5.04 and found they had major-to-fatal problems.

I’ve tried to install 5.10 on my new laptop, but the VIA video is not supported, nor is the RaLink wireless, nor WPA authentication…etc., etc., etc.

If you want to spend the rest of your life talking with geeks (and in the process learn that OS advocacy is best understood as a psychiatric symptom above all else) and trying out an infinite number of kludges, you can get a lot of these problems worked out (more or less), but what you normally wind up doing is succumbing to what I call “Linux disfunction drift,” eliminating tasks that you used to do in Windows because your Linux distro won’t do them, or support the hardware you needed to do them.

Slashdot comes through for once: on the viability of open source “business”

Monday, November 21st, 2005

There’s also a larger problem with this approach – it sucks for small companies trying to become bigger.

If you are only able to profit off of service contracts, you can’t ‘write once, reach many’ like you can with COTS software. Moreover, companies like IBM and Novell which have large established sales and service teams will win all the larger contracts.

If you write a great peice of software, and then have to sell, educate the customer AND hire/train all the workforce, how much time are you going to have to devote to Rev. 2 of your world beating product?

Whenever folks talk about OSS in the context of markets, I think it should be with a jaundiced eye towards our “helpmates” at IBM, Novell, SAP/MySQL and Sun.

Ultimately, IBM et al are about making money for shareholders, if they didn’t see that as the likley outcome, they would not be out there pimping OSS.

I think a world where software is only ‘sold’ in the context of a service contract is bad for the next great idea. OSS is great in its place, but to preclude software for sale isn’t the answer.

The truth hurts for Free Software zealots, but it’s the truth.

Free Software isn’t about eliminating proprietary software, at least it shouldn’t be. It should be about developing a free system for development, learning, and sharing, because we can.

The MDI Plague and Window Management Woes

Saturday, November 19th, 2005

There is a nice article on Wikipedia that discusses the multiple document interface, a horrible hack that took hold in the Windows world to deal with the fact that Microsoft’s default window manager was inadequate to handle multiple windows existing under the same application.

I think Mac OS/OS X handles the MDI plague best by simply grouping all application windows under a single application class, with a single menubar. But usability experts have debated whether that makes the most sense, since the menubar can change, for example, depending on what window is in focus. I think users get used to that, and it also allows the menubar to be as long as necessary while the window can remain as small as necessary. That’s a nice win.

However, given our current model on *nix/Windows of menubars for every reasonably complex window, and given the lack of the MDI hack in GTK+, we do have a mess for applications that need more than one window to operate properly, i.e. Glade and The Gimp.

What I’ve been doing is giving these programs their own workspace as a workaround. That seems quite greedy of them, and indeed it is. What’s more, however, is that it’s unusable. Even when I switch to my Glade workspace, I see 4 windows in my taskbar, each with the same icon and with the following names: “Glade: h2h”, “Properties: image95”, “Widget Tree”, “h2h”. The first three are actually part of the Glade window class, and if I enable Metacity’s taskbar grouping, I see them as part of the same group. The last one, however, is just my actual window, and so is separate.

This is good–but I only get some form of usability when I actually enable window grouping. The thing is, in Metacity you can’t enable window grouping on a per-application basis. It’s all, sometimes, or nothing. Sometimes means metacity only groups windows when I’m running out of space. Otherwise it’s either on or off.

The thing is, grouping isn’t just about space saving. It’s about being able to perform window manager operations on a group of windows, i.e minimize all and maximize all.

This doesn’t even solve all problems: alt+tab still shows me all 4 glade windows, which can be quite confusing since only one comes into focus at a time. But that’s a separate issue, separate debate.

Wouldn’t it be nice if instead of Metacity just “figuring out” when to group my windows together, it let me just press a hotkey “Group all X windows on this workspace”, where X is the application I’m currently in?

I’ve decided this feature is so valuable, I may just hack metacity to add it. It will at least provide a path for solving the MDI nonsense.

Update: check out these screenies of a “different” approach to MDI written in GTK. It’s called GTK ADI.

Met Runar, Discussed Software

Saturday, November 12th, 2005

I met with Runar (he’ll have a blog soon, I swear) today, and we discussed open source, Python, and all related goodness over coffee and vegetarian lunch free-riding on the ‘sNice wireless network.

We spent about 3 hours there, just talking about Runar’s project, “sqlstring”, my ideas about inferred typing and static source code analysis in Python, Python’s niceness in general, user interface toolkits, AJAX being a big, nasty hack, and web application frameworks in Java and Python. Our discussion really degrenerated into praise of vim once we discovered that we were both happy users. Text editors really bring people together.

Runar kind of convinced me that trying to infer all the types of objects is very “unpythonic,” which I guess is true since it discourages the crazy stuff you can do with Python. Maybe the best thing to do is judiciously eval code, as was my original impulse for getting nice completion out of Python? Not sure.

Or maybe I should just give up the idea and accept the fact that vim plus ipython is just about as good as it gets. That seems like a cop-out, though.

Regardless, Runar seemed somewhat willing (only half-willing) perhaps to give a small talk for Free Coders on Python, I’ll see if I can convince him that it’ll be fun. I suppose I could give the talk myself, but I already do all the talkin’.

Found out how I got hacked originally

Sunday, November 6th, 2005

I run a tool on my server which creates charts based on basic server vitals, like free disk space and CPU load averages.

It’s called cacti, and it’s great.

Except, apparently this security hole allowed the hacker who originally broke into my server to get in. He was able to execute arbitrary commands via the good old URL string hack. (He did the same thing as is described in there: wget’ed his own script which added a new user for himself and added him to sudoers, and then connected via ssh).

In my latest upgrades, I saw that this cacti bug’s been fixed.

Scary stuff. Computer security, these days. How does a php script have code which can run an arbitrary command? My Java Servlets never have a way to run command line apps by way of specific arguments in the URL string. Sigh. In *nix we may have [basically] all-or-nothing security (that is, if you discount ACL support)–but knowing this, please prefer “nothing” to “all”, for crying out loud!

Got bored with Descartes and Spinoza, wrote a patch to powernowd

Saturday, October 15th, 2005

While I was studying, I noticed that I wasn’t exactly happy with the cooling/cpu frequency scaling on my laptop. I use a nice program called powernowd which scales my CPU speed up and down depending on various factors related to system load. But I didn’t like how my setup was kind of “all or nothing.” When I am plugged into AC, I switch to “performance” mode which just runs me at 100% CPU frequency all the time (making my laptop hot, my fan noisy, but my machine fast), whereas when I’m unplugged I switch to “userspace” mode, which lets powernowd kick in, and he jumps about from 400mhz to the full 1.6Ghz based on load, keeping the machine cool but also making it feel a bit sluggish since if I’m overloading my CPU at 400mhz it’s already “too late” to pump it up, it will have already felt slow for at least an instant.

So I have this conflict: hot and responsive, or cool and sluggish. I thought, well, I must be able to come to a compromise.

I decided to take a look at powernowd’s code, and it turns out it’s written quite straightforwardly. Within 30 minutes of tinkering, I had a patch that did what I wanted. With another 30 minutes, I polished it and made it quite commitable.

Basically, I added a new mode called “COOLING” to powernowd, which runs your CPU a few notches below your full frequency (which I call your “cool_spot”), based on the following approach:

  • if you have two frequencies available, you normally run with the lowest.
  • if you have three frequencies available, you normally run with the second from highest.
  • if you have four frequencies available, you normally run with the third from highest.
  • if you have five or more frequencies available, you normally run with the fourth from highest.
  • if your load goes above your specified trigger (“highwater” in the code), you jump to highest frequency. When it lowers (“lowwater”), you go down to your cool_spot, but not below it.

On my machine, I have 5 frequencies (1.6Ghz, 1.5Ghz, 1.4Ghz, 1.2Ghz, and 400Mhz), and so I normally am running at 1.2Ghz. This new COOLING mode runs while I’m plugged in, and keeps my machine nice and cool but still lets it immediately respond when I want to do something, like a workspace switch.

I then hacked the init.d script to have a BATTERY and AC mode, and switch between AGGRESSIVE and COOLING modes accordingly. Now, when I’m unplugged, I get the best battery life and pretty good performance, and when I’m plugged in I get a cool notebook with good performance.

I’ll probably post the patch after my midterms…

Development under Windows: why so painful?

Friday, October 14th, 2005

It’s really weird. Lately, I’ve been doing so much development in a *nix environment, that doing the development in Windows is really painful for me. I don’t have any of my good old UNIX tools, I don’t have hotkey-optimized user interfaces, I don’t have speed and control. But more than anything else, I don’t feel like I know what’s going on under the hood.

Today, to take a break from reading Philosophy, I decided to work a bit on this little Java Servlet project I’ve been hacking on. (Will be “released” later.) At some point this past summer, I decided to remove Linux from my main desktop machine and just consolidate all my Linux data onto one machine–this made my life easier so I didn’t have three total (one Windows, two Linuxes) places where my shit could be. But the sacrifice is that my laptop screen is small, so sometimes I want to develop with a big screen and thus want to use my desktop.

Web development, especially, makes sense for me under Windows, since I’m comfortable with the major graphic and web design tools (Photoshop, Dreamweaver, Illustrator) and don’t think the Linux “equivalents” (GIMP, Bluefish, Inkscape) are good enough.

But I decided–may as well have the code open on Windows too, since it’s not C hacking I’m doing, but Java. So I installed Eclipse, and the J2EE, and got cracking.

But under Windows, there are all sorts of gotchas. When my UNIX tool craving gets really bad, I need to drop into cygwin, which isn’t so bad. But without good workspace switching (I have VirtuaWin, but it kinda sucks), and without a customizable window manager, I am really much slower. But here’s the other weird thing I ran into. After awhile of coding, I realized that Eclipse wasn’t reading my JavaDoc information for the JDK (no cool descriptions in my autocomplete tooltips). So I go snooping around the preferences file and can’t find anything, I enable a billion options but no luck. But then, eventually, I realize that it’s very possible Eclipse is using a different JDK. In fact, I look in the dialog, and Eclipse is using some J2SE environment that some other application installed, not the J2EE I installed right before Eclipse. And that J2SE is missing the Java API source code and JavaDoc comments.

The reason this seemed so non-obvious to me is because I’m not used to systems which are completely fucking disorganized. Say what you will about Linux not being user friendly, but, by God, you won’t find it likely to find two different JDKs installed on my machine, and even if you do, only one will be getting used (thanks to Debian’s “alternatives” system). Every application on Windows statically compiles, includes its own libraries, and spews its shit all over the file system and registry. No database tracks it, so your system is a fucking nightmare.

I couldn’t even do a reasonable search to find the JDK I needed, either. It turns out it was in C:\Program Files\Sun\j2sdk1.4_02, which may not sound so bad, but considering on Linux I just think, “Where are libraries stored? /usr/lib” and then in there I think, “What is what I’m looking for called? j2sdk” I quickly find any Java environments in /usr/lib/j2sdk1.x-sun.

On my Linux system, which has not that much installed, I just ran a du -hs /usr/lib/ and got 1.7GB. That means on my relatively lightly-loaded Linux system, 1.7GB of raw 0s and 1s are sitting there waiting to be used as SHARED libraries. Meanwhile, on Windows, there could be any amount of duplication of the equivalent libraries, floating around in various Program Files directories.

I can’t believe there are acutally some Linux critics that believe we should be going in this direction, eliminating things like emerge, apt, and rpm and instead just have statically-compiled binaries that come with their own binary libraries and have users duplicating this stuff across their system. Not only is it insane from the point of view of giving control to the user, but it’s also just plain wasteful.

I can’t wait till Ubuntu breezy

Wednesday, October 5th, 2005

If for nothing else, for Mac OS 9-style file browsing in Nautilus, a la this screenshot:

clicky clicky

Just For Fun: The Story of Linus Torvalds

Thursday, September 22nd, 2005

For the last couple of weeks, my bedside reading has been this half-biography, half-autobiography on Linus Torvalds. I have to say, however, that the book is like two books mixed into one. Chapters alternate between Linus talking about his life and about big moments in Linux’s history to David Diamond describing modern-day Linus with a kind of forced wonder. Truthfully, Diamond comes off as a sycophant who could care less about Linus’s flaws and positive characteristics, and cares more about molding some kind of “image” of Linus as containing a humility and genius simultaneously. Near the end, I started only skimming the chapters not written by Linus. Diamond’s really not a good writer, either. (Sorry Dave.)

Truthfully, the book kind of pops the lid off Linux and makes you understand it as much less glamorous than say Wired Magazine described it to the public. Linus really just talks about not having a social life, sitting in his room with curtains covering his window, coding all day. Not exactly the ideal role model, I think. Don’t get me wrong, I love the Linux kernel (as much as one can love imperfect software), and Linus made a great contribution toward keeping the UNIX world and UNIX principles alive, but it’s just that I like to think of open source developers as something other than the stereotypical, introverted geek. In fact, much of Linus’s chapters is devoted to his apprehension about giving a public talk about Linux. When I think about the fact that I’ve given three or four of them to date, and enjoy it more every time, I see how different I am from this kind of stereotypical geek.

It also kind of made me dislike Linus. When I saw Revolution OS (a DVD on the rise of open source), the movie kind of endeared me to Linus’s practical nature as opposed to Richard Stallman’s religious idealism. I like idealism, but Stallman is really religious about it. And he’s bitter. Linus, on the other hand, has that great Northern European, “I’m just gonna go with the flow” attitude.

But this book made me realize that Linus is religious is his own sort of way. Included in the book is Linus’s flame war with Andy Tanenbaum on monolithic versus microkernel designs. Truthfully, I’ve studied operating systems and I’m not even sure which design is best, and Linus makes a decent argument of why microkernels end up being just as complex, or more complex than monolithic ones. But what I didn’t like is that in the flamefest, Tanenbaum said that deficiencies in MINIX were due to it being a hobby, and that he had duties as a professor. Linus responded, “Re 2: your job is being a professor and researcher: That’s one hell of a good excuse for some of the brain-damages of minix. I can only hope (and assume) that Amoeba [Tanenbaum’s future OS project] doesn’t suck like minix does.”

This just shows me that Linus really is an asshole sometimes. He states this outright in his book. So now, truthfully, I may like the open source movement, but I think I “at least dislike” two of its most major players (Torvalds and Stallman).

Finally, I think a clip from Tanenbaum’s website points out a nice principal in OS design:

Also, Linus and I are not “enemies” or anything like that. I met him once and he seemed like a nice friendly, smart guy. My only regret is that he didn’t develop Linux based on the microkernel technology of MINIX. With all the security problems Windows has now, it is increasingly obvious to everyone that tiny microkernels, like that of MINIX, are a better base for operating systems than huge monolithic systems. Linux has been the victim of fewer attacks than Windows because (1) it actually is more secure, but also (2) most attackers think hitting Windows offers a bigger bang for the buck so Windows simply gets attacked more. As I did 20 years ago, I still fervently believe that the only way to make software secure, reliable, and fast is to make it small. Fight Features.

I agree. But does a microkernel design actually reduce the overall size of the operating system, or does it just reduce the size of whatever you consider to be the “microkernel”? That is, just because a file system is implemented as a file system daemon talking to a driver subsystem through message passing doesn’t necessarily mean the file system, or driver subsystem, are secure. Insecurity could exist even at the boundaries, no? Not to mention instability.

I think Linus and Tanenbaum have to agree that this debate isn’t an open and shut case. The best kernel is probably one that mixes modularity, a strong kernel/userspace boundary, and some of the fancier features of a microkernel approach, while not sacrificing elegance of design or performance.