Technology

There is a new release of OpenOffice with version 3.1. You can download it at the usual location. The version adds features like font aliasing and further grammar checker integration. You find a complete list of new features on the website of Openoffice.org.

Techreport published an article about the upcoming 6 core Opteron "Istanbul". AMD seems to have working silicon. This is an especially interesting news for our X4600 as this system contains a feature to reduce cache coherency traffic on the Hypertransport. The more procs and cores you have in your system, the more this starts to be a challenge and. An system with reduced cache coherency traffic would leave more bandwith for other traffic. The X4600 - a really fast implementation of an 8-socket Opteron system - would be even faster with such reduced traffic. AMD solves this with a directory of memory pages cached in the processor:

Part of the answer, it seems, may be a feature new to Istanbul that AMD calls HT assist (presumably for HyperTransport assist). This feature is what the company calls a probe filter (and may more commonly be called a snoop filter) that functions to reduce traffic on socket-to-socket HyperTransport links by storing an index of all caches and preventing unnecessary coherency synchronization requests

And you have to keep in mind: A X4600M2 with Istanbul would be an 48 core system on 4 rack units. As i wrote before: Interesting technology.

In the last few days, i thought a little bit about cloud computing. As you may know as a frequent reader of this blog, i have a strong dislike to any hype topic. Virtualisation is such a topic and cloud computing is something overhyped, too.

At first: I don´t think, that cloud computing is a technology. It´s a business process. It´s the business process to deploy servers with a standardized components and bill them. Do you remember the N1 vision of Sun? Well, it was one of this visions of Sun, that came to early for the market. It was the vision to centrally control the deployment of system for other users. In my opinion, cloud computing is nothing else than N1SPS in conjunction with accounting in conjunciton with resource management in conjunction with Solaris Containers in conjunction with a system collects and processes usage data records and feed it to a credit card transaction system. Besides of the credit card system everything is already there for years available for Sun. Some years ago, many people told us : Nah ... nobody needs it. Today: We want cloud! We want cloud! A really strange development.

At second: Cloud Computing is interesting as it has two medals. It´s the ultimate commoditisation and it´s the ultimate decommoditisation of technology. From the view of the customer, IT is reduced to the provided interfaces. It´s like power, as long it´s 220V at 50 hertz it doesn´t matter where it came from (and many tree huggers in southern germany would wonder that there 100% organic electricity generated by wind mills in northern germany, is indeed generated by french nuclear plants). As long as the cloud-provided J2EE container complies to the standard.

At the other end it´s the end of commoditization. Providers of clouds have something many of the small users of IT doesn´t: Load ... a lot of load. They have a need for the most efficient system with the most efficient operating system with the most efficient applications. If for example an operating system is just 5 percent more cost efficent, this matters. If the webserver is 5 percent efficent, this matters. Good enough isn´t good enough in such situations. Or memory. When it´s just your own mailserver you may tolerate memory or hard disks from the hardware shop at the next corner. If there are 500 mailservers on it ... well ... the equation looks different. You can´t use commodity components ... at least when you want to deliver the same or a better availability as a user is used to from his or her own IT..

I see in cloud computing a chance, that some rational thinking about quality will take place in IT again. On the other side, it´s just a overhyped collective term for some really old technologies in a new mix. At the end it isn´t even a technology, it´s a business modell ... and this explains, why a book shop is one of the important players in this area.

I finally found a good usecase for Hadoop at home .... speeding up grep-like operations on a desktop with 4 cores ...

Techcrunch reports about the availability of an offline version of Google Mail. Nice ... really ... but what happened to IMAP and the IMAP client of personal choice? I thought, "offline mail service" was a problem solved in in the last century ...

Yesterday i´ve got a new mobile. It´s one of this superduperallinclusive smartphone. There is even a GPS receiver in the device. Additionally the phone includes all this hip three to six letter acronym technologies. But the best feature of the mobile doesn´t need all this arcane technological wizardry.

Before tell you about this feature, i have to describe one of my habits: I use my mobile as a alarm clock at home as the ringing of a telephone is more efficient than the ringing of a alarm clock as a high piority sleep interrupt. I assume ringing of the telephone is hardwired to "an important message" whereas the sound of an alarm clock is hardwired to "waking up and leaving the warm bed" in my brain.

Okay ... the absolute best feature of the E71 is: When you press the middle of the cursor button of the locked mobile , it displays the time white on black (so it doesn´t dazzle you) with a font size you can even read with the small eyes at 05:00 o´clock when you have to reach the red-eye train ...

I´m exploring Apache Hadoop at the moment. This looks like a really interesting technology. What´s Hadoop? Hmm ... to explain it in a really simplified manner: It´s a distributed, highly available datastore. Okay ... yawn ... no big deal so far.

But there is an interesting twist in Hadoop. Let´s assume you have vast amounts of log files. A pile of data in the size of multiple Terabytes. You want to know the URLs of the Top-10 pages of your website.

The standard old-school approach to this problem is: Starting an analyser on a big server which gathers all data via block or file based protocols to this analysing server. Of course this approach has several bottlenecks: The size of the network pipes, the amount of computing power in a single box, the amount of IOPS in a single server, the amount of IOPS of a single storage attached to the server, the amount of memory in a single server.

But now think in HPC terms about this problem: You could divide this task in several ones. Let´s assume 64 MB shards. You could compute the result for each of the shards on a seperate node. To stay in our example: This step print outs the pageviews of any URL in it´s shard. This fragments of the final result are collected and reduced to the final result: For example by adding the pageviews of a URL from every shard. By using the concepts you you seperate the task of analysing the log files on thousands of nodes in parallel. You get your answer in minutes, not hours or days. The advantage of doing so is to bring the data intensive parts of computation to the data instead of bringing the data to the computation.

Such algorithms are called MapReduce. This concept was introduced by Google and the core competency of Google is to analyse big piles of data, thus such an mechanism is quite handy. I have several usecases in mind for such a solution: Commercial data warehousing, billing of large heaps of Call Data Records, mass converting jobs ... and so on ...

What has all this stuff to do with Hadoop. Hadoop is an open-source implementation of this concepts. The Hadoop Wiki writes:

Hadoop is a framework for running applications on large clusters built of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a computational paradigm named Map/Reduce, where the application is divided into many small fragments of work, each of which may be executed or reexecuted on any node in the cluster. In addition, it provides a distributed file system (HDFS) that stores data on the compute nodes, providing very high aggregate bandwidth across the cluster. Both Map/Reduce and the distributed file system are designed so that node failures are automatically handled by the framework.

It does the housekeeping, the seperation of data in shards, the distribution of the analysing tasks on the server. You can view at it as an API/command line controlled grid engine for data distribution and data processing.

It consists out of the Hadoop Core, the Hadoop Distributed File System (it´s not a POSIXfilesystem integrated to the VFS, you can think of it like FTP, you need a client or you use an API to use it) , there is even an scripting language helping you to write the analysing jobs. This language is called Pig. Additional there is an effort to implement a database for structured data on top of Hadoop with HBase.

At the moment there are some gotchas in this technology. Fore example you can´t work with compressed files in it, as gzip files shards aren´t decompressible seperately (okay, it´s a problem of gzip, but it prevents you to work with it) (Update: I´m not entirely sure about this, it seems that you can work with block based compression like bz2, and gzip is indevelopment, at least according to the Pig documents). But here ZFS on-the-fly compression can be very helpful. I think i will create a Hadoop testbed with multiple zones on one of my systems this weekend.

Next year in February i have to buy or lease a new car. I think, i just found my new car. A Volvo. To be exact: The Volvo XC60. Not because it´s a Volvo or because of the fact that it´s a really cool car. No, because of a little function integrated into the car. It´s called City Assist. The car has the capability to automatically brake the car in such situations like stop-and-go at a traffic light. As accidents in such situations hit me once or twice in the time i own a car, this would really reduce the costs for repairs and this feature would let my car insurance finally drop to a decent level. Sometimes technology is really a heaven-send...

I assume this demo reel of this project in computerized imaging shows the drain, where the next two or three bigger speed bumps and core multiplications will find their way into consumption. The projects website "Using Photographs to Enhance Videos of a Static Scene" describes:

We present a framework for automatically enhancing videos of a static scene using a few photographs of the same scene. For example, our system can transfer photographic qualities such as high resolution, high dynamic range and better lighting from the photographs to the video.

At least you should look at the embedded video presenting the capabilities of this technology.

Nice presentation about the most southern installation of SamFS: Ein Robot auf hoher See - SAM-FS bei 70 Grad Süd (sorry, only in german language, but nice photos). It´s about a SamFS installation on the Polarstern, a german research vessel. I found this presentation held by Dr. Hans Pfeiffenberger from the Alfred Wegener Institute for Polar and Marine Research after a hint from a colleague. This is storage under extreme conditions.

It´s a common practice between us admins to look after the load averages. Interestingly the correct math behind this numbers is not really simple. Roman wrote about it at How many kernel engineers does it take to solve one differential equation?. Interesting read.

When somebody told me in 1995 that i will hear music on a mobile telephon, which updates my play list on a website while walking around in my room via wireless lan while reading mail ... in thirteen years from that day, i would had told him or her to take less of the pills he or she took . Well ... today i´ve installed MobileScrobbler on my iPhone and this now my mobile does exactly the stuff described in the beginning. How time flies.

Looking TV educates. This morning i viewed a documentation i´ve recorded a few days ago on the german TV channel Phoenix. It was a documentation about a reconstruction of the motorglider of Karl Jatho. I didn´t know that Jatho made his first motorized flight a few months before the Wright Brothers.

zpool.org found a really interesting computer case for a Thumper@home: Chenbro ES34069. This case is a small ITX case with four hot swapable drive bays and room for a single boot disk (would populate with a flash based solid state disk)

A fully automatic Triple-A goes berzerk and kills 9 soldier in south africa.

A robot cannon began wildly and autonomously firing its huge gun in South Africa last Friday, killing 9 soldiers and wounding 14. The Oerlikon GDF-005 antiaircraft gun suddenly began uncontrollably shooting as it swung back and forth, spraying hundreds of high-explosive 35mm cannon shells all over the place

This is a good example why software should never ever control the actual firing of bullets. In wartimes it may be a good tradeoff to design a trigger-happy robot for antimortar/antishell artillery, as it´s propable that more people are killed by mortar rounds then by a berzerk robot. But at peace time without incoming rounds? At least it wasn´t a wise choise to put live ammunition into this robot.