Here’s the report: ftp://ftp.software.ibm.com/linux/pdfs/GCG_IBM_and_Linux-9_years_later.pdf

The opening teaser:

In 1999, IBM issued a series of announcements fully committing the company to supporting Linux. IBM vowed to Linux-enable all of their hardware platforms, including their non-x86 based mainframe, mini, and RISC-based systems. They also promised to release Linux versions of their software products and develop
Linux-centric service practices. Moreover, they pledged significant resources to the Linux community with the goal of advancing Linux and open source technology.

So, nine years later, did IBM deliver on these promises? Was their commitment to Linux genuine or just lip service? This report examines IBM’s current Linux products, services, and community support in light of the promises they made in 1999…

While I think it’s obvious IBM has been a huge investor in the Linux community, one thing that I noticed reading the report is just how much IBM is actually different from other community members. There are some noticeable differences in the investments and approach to supporting the Linux platform and community. I often forget to just take in all the Linux technologies IBM has been heavily involved in from Xen, KVM and libvirt to filesystems, to systemtap, kprobes and then there’s RAS, scalability and performance enhancements.

Another interesting thought to reflect on is just how important it has been that there are multiple investors in this field. If this report captures just what IBM did, think of the industry combined. IBM couldn’t have done anything this big with Linux if it weren’t for co-creating with a community of enthusiasts, researchers, governments, Intel, AMD, Google, Nokia, Motorola, Oracle and thousands more. What would the report look like if you compiled all the investments and work the entire community leveraged across the industry. Linux is “bigger than huge” when you stop to think about it. This is also why I’ve said for a couple years now when you extend the investment model 3 to 5 years into the future, Sun and its anti-Linux,  Solaris push against the tide of the industry loses in the end. I think we’re starting to witness that now. Sure, OpenSolaris is a great idea… it’s just 9 years late and it’s too late to matter now.

I’m interested in outside perspectives too – where do you think IBM stands? Has the community development and investment model worked? Where will this lead in the future and what will be the next evolution of the model? Red Hat seems to think the model will evolve to include increased customer co-creation – I tend to agree. Why? Because the incentive model to invest aligns very well – and when you have alignment, it almost naturally will happen.

http://www.flickr.com/photos/28532182@N00/2316388926

UPDATE: Frank sent in a comment pointing out a ton of swag available on Cafepress

http://www.cafepress.com/systemtap/ 

http://zombieprocess.wordpress.com/2008/01/03/sample-real-world-use-of-systemtap/

SystemTap has been sometimes plagued with “solution in search of a problem” complaints. It was interesting to run across an example of SystemTap being used to solve a real-world problem. A developer discovered an OOM (Out Of Memory) condition in the upstream kernel. In the quest to obtain additional information regarding the issue, the kernel refused to boot with additional debug printk()’s added to quicklist.c. The developer, being familiar with SystemTap, used the following script:

http://eugeneteo.livejournal.com/7484.html

Interesting article over at LinuxWorld.com:

A graph of all the developers involved in the upcoming 2.6.22 release, and the relationships of who reviewed whose patches, extends to a 40-foot-long printout with names in tiny type. The graph is on display at the Ottawa event.

The new “mess” results in innovative new features getting integrated into Linux distributions much more quickly, says Jonathan Corbet, author of the camera driver for One Laptop Per Child and another co-author of Linux Device Drivers. Previously, when developers maintained both “stable” and “development” kernels, it could have been two to three years before a feature made it from development to mainstream users.

Also I hadn’t noticed Sven Wegener had taken over maintaining systemtap for Gentoo. Nice to see the mainstreaming support in Gentoo! Quite a few updates since he took over.

To steal an adjective from the guys over at Redmonk, the “enterprisey” types will find a January, 2007 paper by Forrester’s Brad Day quite interesting. It’s called “Why Choose Linux on a Mainframe” – a very compelling question. It’s common to be asked here, “why would anyone want to do Linux on a mainframe?” I was intrigued and read the paper.

I’ve discovered at IBM, mainframes are not a dying technology, in fact the longer I’m in IBM watching what customers are doing, the more I’m realizing that mainframes are actually a window into the future of where other platforms (x86/RISC) are heading. Sun, HP, Intel/AMD, and even the other IBM server lines watch what’s going on with mainframes and try to build those technologies/features into their products. It’s an odd thing having personally grown into the x86 culture thinking that the whole point was to get those outdated UNIX/Mainframes to x86 b/c of price points…

Anyway, this is a shameless plug for IBM-wares and I apologize, but I’m finding the customers using mainframes are doing some of the most interesting work and proof points around having 1 OS that runs and is supported everywhere. Most of these customers end up deciding Linux should be their strategic OS and once you have moved to Linux – workloads are ‘floating’ – they are not tied to any one type of hardware (within reason). Nationwide.com migrating their entire mission critical web infrastructure off a mix of 700 x86 and Unix servers was just the tip of the iceberg.

And if you think this is just a “couple guys out there” doing zLinux, Brad Day cites some of the metrics from LAST YEAR that still show how many customers are doing zLinux:

According to IBM, the Q2 2006 Linux/System z business closed out the quarter with more than 1,000 Linux customers on System z, 19% of System z revenues coming from Linux, and being deployed with 28% of System z customers and more than 800 System z Linux projects in production.

Unfortunately I can’t find a link to this paper other than one on Forrester’s website… If you’re ‘enterprisey’ then I’m sure the $279 won’t set you too far back (or you may have a subscription already).

On another note, for those of you concerned about observability, SystemTAP for Linux now has S/390 support for mainframe Linux (RHEL and SLES) ;-) Observers have to remember stap has enabled features built into the architecture to assist different types of users. Recently it seems the question on some minds is “why have a guru option at all” and “doesn’t that imply the architecture is unsafe”? I don’t necessarily speak for the developers (you can ask them though, they’re very open and engaging on the mailing list) but I’ll try to give my view.

In normal, user mode, stap is no different than other observability tools and it’s restricted like other observability tools to those things that could never crash a system. I’m told quite clearly, safety is not a goal; it’s a must. But, you have to understand the community users to see where a guru mode comes in.

SystemTAP guru users are the types living on the edge not caring what they do to a system and these users are going to try one set of things that you wouldn’t want to do on production systems. They are not the ordinary user – some are testing bleeding edge code and inevitably find bugs. Others are using stap and finding bugs in other code. The switch to manually/explicitly allow stap to execute a script with embedded C code is -g. I wouldn’t expect to see many, if any -g type users in production environments… it just wouldn’t make sense – and many can’t use -g even if they tried on a production system b/c compiling the embedded C code on a production machine requires gcc (which production deployments don’t – at least shouldn’t – have). Without gcc, the script would simply exit.

To use -g on a production system, you’d have to compile your script on another system using the new stap cross-compile feature and then run it on your production system… that’s a whole different class of user who 1) knows where in the kernel they want to look, 2) is dangerous enough to know they can write embedded C code to find that probe point, 3) knows how to cross-compile a script on one environment for another (and setup the test environment correctly), and 4) has the access to run it on a production system… not to mention that if they get to point 3), they’re going to test it on that system and work out any bugs there before running it on a production system.

The main point of having a guru option is that this one technology has the capability to do whatever someone can imagine (and I know of a few customers with great imaginations, just like those who imagined ways to save millions by running Linux on a mainframe). The way I see it is this: the Linux kernel is open source and even in the kernel developer crowd – who’s going to tell them what parts of their kernel they can/cannot have access to?

SystemTAP is architected for different goals than other tools b/c it’s architected in an open community. Just as Linux is modular by design to be open and adaptable when some others felt monolithic/prescriptive was a better path. When development happens in a closed environment, you get what someone else wants you to get.

Ironically, observability features first appeared way back when on the mainframe. It’s great to see that today as features evolve in an open community, an observability tool like SystemTAP can be used across all of the major system architectures: x86/64, RISC/POWER, and now S/390 mainframe. It’s still relatively early on for SystemTAP, but the project is definitely making strides and I’m impressed by the number and diversity of users I’ve seen so far. Just think about this: one observability tool can be used across your entire enterprise environment on an OS that runs on all hardware architectures… I wonder where that could lead?