Stefano's World

Shared by tumbleweed
Interesting idea, I'd quite like to try it (if I had a thinkpad)

A long while ago I wrote a plugin for gaim called gaim-thinklight that blinks ones ThinkPad ThinkLight when a new message arrives. By now it is called pidgin-blinklight and supports some other hardware as well, but has not changed since over a year. Today, I implemented a new feature, and I’m curious if it will actually work:

Until now, the blink pattern was hardcoded: ON, wait 150ms, OFF, wait 125ms, ON, wait 150ms, OFF. Since version 0.11, pidgin-blinklight will calculate these three delay times based on the contacts login name. So different contacts will have very slightly different blinking patterns. The idea is that, after a while, you start to recognize your frequent buddies already by the blinking. The wait times are from the range from 50ms to 250ms, I hope that range works well.

Users of Debian unstable will get the new version automatically. If you want to compile pidgin-blinklight from source, you will have to grab it from the debian ftp server. The source is in the pidgin-blinklight Darcs repository.

Interesting idea, I'd quite like to try it (if I had a thinkpad)

@mdubakov wrote:
Good article about paradigm shift in IT -> http://bit.ly/8NJCIY #agile #lean

In 1979 a group of western dignitaries visited Japan to learn more about the manufacturing models that had been applied to great success. Konosuke Matsushita, the president of Matsushita Corporation (Panasonic, Legend, Technics etc.), opened his talk with the famous statement….

“We are going to win and the industrial west is going to lose: there’s nothing much you can do about it, because the reasons for your failure are within yourselves. Your firms are built on the Taylor model; even worse, so are your heads. For you, the essence of management is getting the ideas out of the heads of the bosses and into the hands of labour. We are beyond the Taylor model.”.

This leads to two questions: What did Matsushita-san mean by this bold statement?... and why did his visitors deserve such a warm welcome?!

To understand Matsushita-san’s point we need to take a brief look at the history of management science. Prior to the Industrial Revolution in the 1830’s, businesses were small-scale, intimate affairs, consisting of a limited number of individuals. To look at the management of large numbers of people prior to this point requires the study of governments and armies.

During the early 1800’s technological advances led to the rise of larger industrial enterprises. Factories emerged to produce goods at a larger scale and at lower costs than traditional cottage industries.

Cue the entrance of Frederick Winslow Taylor to our story. Taylor was a mechanical engineer who was fascinated with industrial efficiency. He is regarded as being one of the founding fathers of management science and wrote a book on ‘Scientific Management’1. Taylor’s industrial models separated ‘working’ from ‘doing’; he believed that it was the role of management to determine the ‘one best way’ to perform the work…

“It is only through enforced standardisation of methods, enforced adoption of the best implements and working conditions and enforced cooperation that this faster work can be assured. And the duty of enforcing the adoption of standards and enforcing this cooperation rests with management alone.”.

When Henry Ford set about his mission to revolutionise mass transportation in the early 1900’s, he turned to the latest management thinking to make his dream a reality. Ford created a business of such scale and effectiveness that it must have seemed to his competitors and peers analogous to turning up to the proverbial knife fight with an F-14. He progressed from building a handful of vehicles in 1909 to over 500,000 units just six years later, inventing all of the technology he needed to do it along the way.

The success of Henry Ford, and later Alfred Sloan at General Motors, led to a cascade in management thinking. The Taylor model that Ford and Sloan had applied to such great success became the archetypal model for the western corporation in the 20th century.

Cut to 1947… The Second World War has ended and Japanese industry has been decimated by two atomic bombs: one at Hiroshima, the other at Nagasaki. The allies, keen to support the redevelopment efforts in Japan, send over a party of management consultants to help with the work required to rebuild industry. Amongst them is William Edwards Deming, a statistician and management theorist whose philosophies had been largely ignored at home. Deming, in contrast to Taylor, believed that ‘thinking’ and ‘doing’ should not be separated, and further, employees should be encouraged and empowered to make decisions on how work should be performed.

 

“All anyone asks for is a chance to work with pride.”.

 

While Deming had been largely ignored in the US, the Japanese got religion. In particular organisations like Toyota and Matsushita built organisational philosophies around empowerment, teamwork and collaboration. They went from some of the worst performing businesses in the country to the strongest. By the late 1970’s world governments were looking to these emerging giants to understand what made them so successful, and in particular, so resilient. It was at this point in 1979 that Matsushita-san delivered his famous speech to western industrial leaders keen to learn the secrets of Japan’s success.

It was not until 1991 that the world began to understand the power of the management systems that had been developed in Japan. A book2 was published following a study by MIT’s automotive industry research programme. The book studied the history of the automotive industry and the rise and rise of the mighty Toyota Motor Corporation; the term used to describe Toyota’s secret sauce? Lean Thinking.

What Konosuke Matsushita, Toyota’s Taichi Ohno and their contemporaries understood was that the key to the success of their businesses didn’t lie in their tools, techniques, or processes, but was the result of the management philosophies that underpinned their corporations. They thought about their businesses as socio-technical systems and because of this created organisations that encouraged the right behaviours throughout.

So, what does this have to do with IT? Firstly we have to recognise that IT is failing. Standish Group estimate that $85B to $145B is spent every year on failed and cancelled IT projects, and that 60% to 70% of all projects either fail outright or are considered troubled (time, cost, scope issues)3. This is a repeated pattern; we can change the country, the industry, the people and the business; the data shows a similar pattern – the problem is systemic.

To solve this kind of systemic problem we need to investigate the system more closely and understand the ‘games’ that are being played out within our IT divisions. But what are the components of our IT ecosystem? When we lift the hood, there are a few areas of focus for us to investigate: people, structure, process, culture and technology.

The first thing that we may notice about our corporate IT divisions is how little they differ to the Taylorised models Henry Ford and Alfred Sloan built their businesses around 100 years ago. They are structured around functional silo’s, management philosophies are command and control and empowerment is just a word that appears on corporate mouse mats. They are structured for an industrial paradigm in an information age.

To make matters worse, most IT managers aspire to create self-managed teams, high levels of collaboration, innovation and continuous improvement. Many have little appreciation that the management models that they enforce, often the only ones that they know, are the very things that are preventing them from achieving the results that they dream of.

So how do we change things? Firstly, it’s important to understand the differences in the two management philosophies, as the contrast is stark. For example, where Taylor’s ‘scientific management’ teaches us that managers should manage people, systems management theory teaches us that managers should manage processes. ‘Scientific management’ advocates for maximising the utilisation of our people and machinery, ‘systems management theory’ teaches us to ruthlessly eliminate waste.

Although the transition is anything but easy, the results, at least so far, appear impressive. At a recent conference, Jeff Smith, CIO of Suncorp’s 2000 person IT division, estimated that they had increased throughput by over 40% whilst at the same time reducing net operating costs by over 20%4. Similarly, the BBC’s David Joyce announced in a recent article that they had reduced the time taken to engineer a software feature by over 50%5.

These organisations are re-engineering many of the components of the IT taxonomy. By taking Agile software principles and introducing statistical control and scheduling techniques from Lean Thinking, teams are radically improving the efficiency and throughput of software delivery processes. They aren’t stopping here though. Product development is being refined to ensure the teams aren’t ‘building the wrong things at speeds previously thought impossible’. Planning and governance processes are being simplified to support responsiveness and adaptability in the business. Even the organisational structure itself isn’t sacred, with some of the more progressive IT divisions moving away from a top-down, hierarchical design, towards a systems based, bottom-up model, removing organisational silo’s to increase collaboration and introduce a stronger customer focus.

The real change for these organisations isn’t in the structure, processes or tools of course, but in something much more subtle and complex: the way they think. Changing 100 years of western management thinking is not a simple task but industrial models just don’t cut it in an information age. Deming taught us that the processes and structures we create as leaders always produce exactly the results they are designed to produce; the system always works perfectly. In IT we have created approaches that fail (or have difficulties) 60% to 70% of the time. It’s our responsibility as leaders to change the system.

This leads to the title of the article. The most common excuse I hear for avoiding change and improvement in IT leadership is that we’re not Japanese and we don’t build cars. I hear this excuse every day and it misses the point. Lean Thinking, and the management paradigm that underpins it, Systems Management Theory, focus on changing the role of leadership; it knows no national or industrial bounds, and this has been proven time again over the last 30 years, from manufacturing to healthcare. IT leadership is once again lagging behind the management curve.

To re-enforce the point even further, a recent article in the Harvard Business Review6 asked some of the worlds leading academic and industrial business thinkers for the big ticket changes required in western management thinking over the next 10 years. Retraining managerial minds in systems thinking appeared in it’s top 25. Also in there was reducing the pull of the past, eliminating the pathologies of formal hierarchy and reconstructing management’s philosophical foundations. This is nothing new – management science has pointed towards collaboration, teamwork and trust for over 30 years. But mouthing the words is easy; Systems Management Theory gives us the tools to go execute.

Introducing this paradigm shift, although not an easy journey, has certainly been proven achievable. Agile software development methods, based on the Toyota Production System, help us to quickly introduce Lean concepts to our software development operations. Statistical control techniques can help us improve and refine them. Recently, Lean concepts have helped scale these working-level techniques to the enterprise by borrowing philosophies, tools and techniques to solve historic problems with structure, budgeting and governance in top-down, command and control organisations. And middle management are proving willing to change, and even lead the charge, given the right leadership, support and opportunities.

Driving change is hard. I often compare being a CIO to the job of a grounds keeper in a cemetery; there are a lot of people underneath you but no-one is listening. Of course, we don’t have to strive to improve the problem; I’ll leave the last word to Deming himself…

“It’s not necessary to change. Survival is not mandatory.”.

 

 

1. The Principles of Scientific Management: Frederick Winslow Taylor.

2. The Machine That Changed The World: Womack, Jones, Roos.

3. Standish Chaos Reports 2000 to 2009.

4. Agile Australia (http://www.agileaustralia.com/video.html)

5. David Joyce (http://leanandkanban.wordpress.com)

6. Harvard Business Review, February 2009: Moonshots for Managers

 

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

  Hide links from @mdubakov
  Filter out hashtags: #agile
  Let us know if this story is not showing up correctly

Shared by tumbleweed
http://lwn.net/SubscriberLink/370157/b6f1648ad53a6e6e/ Josh Berkus is well known as a PostgreSQL hacker, but, as it happens, he also picked up some valuable experience during his stint at "The Laboratory for the Destruction of Communities," otherwise known as Sun Microsystems. That experience has been distilled into a "patented ten-step method" on how to free a project of unwelcome community involvement. Josh's energetic linux.conf.au presentation on this topic was the first talk in the "business of open source" miniconf; it was well received by an enthusiastic crowd.

http://lwn.net/SubscriberLink/370157/b6f1648ad53a6e6e/

Shared by tumbleweed
must play with that

After some delay, uzbl is now available in Debian (testing and unstable). Uzbl is a webkit based browser which aims to adhere to the Unix philosophy of having programs that do one thing and do it well; as such, it comes with a main binary (uzbl-core), acting as the real web-renderer, which is controllable by third-parties via standard fifo’s and sockets. The package also bundle some external helper scripts for daily web-surfing (ie. handling history, bookmarks, cookies, etc.) and a ready-to-user wrapper (uzbl-browser) to help setting up the environment, using the default config.
As a standalone browser, it may look quite similar to vimperator (default key-bindings and interface are strongly vim-like), but overall it offers a more powerful and completely reconfigurable user-interface, can be fully scripted and hooked with self-made helpers and is even embeddable in other applications (eg. Emacs). Nonetheless, it brings all the power of a fully fledged webkit renderer.

You may find it particularly useful when working within a tiling window manager (like awesome or xmonad), or used as a full-screen remote monitor, easily controllable via socket (you should really try socat with it!) or plain-text fifo (for a bunch of stats, as easy as a `echo "uri italiaora.org" > /tmp/uzbl_fifo_your-instance`)

must play with that

I recently came across a useful tool I hadn't heard of before: dstat, by Dag Wieers (of DAG RPM-building fame). He describes it as "a versatile replacement for vmstat, iostat, netstat, nfsstat and ifstat."

The most immediate benefit I found is the collation of system resource monitoring output at each point in time, removing the need to look at output from multiple monitors. The coloring helps readability too:

% dstat                                                                         
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--         
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw          
  4   1  92   3   0   0|  56k   84k|   0     0 |  94B  188B|1264  1369          
  3   7  43  44   1   1| 368k   11M| 151B  222B|   0   260k|1453  1565          
  3   2  46  48   1   0| 432k 5784k|   0     0 |   0     0 |1421  1584          
  2   2  47  49   0   0| 592k    0 |   0     0 |   0     0 |1513  1763          
  6   2  44  49   1   0| 448k  248k|   0     0 |   0     0 |1398  1640          
  8   4  41  45   3   0| 456k    0 | 135B  222B|   0     0 |1530  2102          
 18   4  38  41   0   0| 408k  128k|   0    47B|   0     0 |1261  1977          
 10   4  44  43   0   0| 728k  208k|   0     0 |   0     0 |1445  2203          
  6   3  39  51   0   0| 648k  256k|3607B 4124B|   0     0 |1496  2180          
  7   7  34  53   0   0|1088k    0 |1234B  582B|   0     0 |1465  2057          
 14   8  28  49   0   0|2856k  104k|   0     0 |   0    52k|1610  2995          
  6   6  43  45   0   0|1992k    0 |5964B 4836B|   0     0 |1493  2391          
  9  14  34  44   0   0|2432k  112k|7854B  726B|   0     0 |1527  2190          
  9  11  40  41   1   0|2680k    0 |1382B  972B|   0     0 |1550  2298          
  5   4  68  22   0   0| 576k 1096k|  12k 4628B|   0     0 |1522  1731 ^C       

(Textual screenshot by script of util-linux and Perl module HTML::FromANSI.)

Its default one-line-per-timeslice output makes it good for collecting data samples over time, as opposed to full-screen top-like utilities such as atop, which give much more detailed information at each snapshot, but don't show history.

Since dstat is a standard package available in RHEL/CentOS and Debian/Ubuntu, it is a reasonably easy add-on to get on various systems.

dstat also allows plugins, and just in the most recent release last month were added new plugins "for showing NTP time, power usage, fan speed, remaining battery time, memcache hits and misses, process count, top process total and average latency, top process total and average CPU timeslice, and per disk utilization rates."

It sounds like it'll grow even more useful over time and is worth keeping an eye on.

Shared by tumbleweed
whoa, interesting idea. Not for the feint of heart Lately I have been thinking about methods to provide a stripped down, secured environment for running untrusted code on GNU/Linux. With this post I would like to present you with the first results of my research.

ujail - brief introduction

I have chosen ujail as the name for the technique I am proposing. ujail stands for micro jail in userspace and, in itself, describes the concept briefly. The main idea is to have a userspace process monitor system calls of one of its childs and emulate some calls, if needed. This is done using ptrace and namely both PTRACE_SYSEMU and PTRACE_SYSCALL.
The ujail process should not be able to monitor syscalls, like strace does, but also intercept and emulate them.

This sounds a lot like user mode linux (uml), but the method is different. Whilst uml comes with a complete kernel, emulates all system calls and this way provides a virtualized system, ujail is intended to only emulate some systemcalls, without emulating the kernel.

Revisiting PTRACE_SYSCALL & PTRACE_SYSEMU

To better explain how the ujail technique works I would like to have a quick look at PTRACE_SYSCALL and PTRACE_SYSEMU again.

PTRACE_SYSCALL allows a userspace process to be notified whenever a traced process enters or leaves a system call. This means that two notifications are normally sent: one before system call entry and one afterwards. Even though one is able to change the parameters of system calls this method does not allow system calls to be fully emulated (think virtual filesystem here).

PTRACE_SYSEMU on the other hand provides one notification on syscall entry and expects the receiver of the notification to emulate the syscall. This method alone sounds great, but this also means that memory allocation needs to be emulated too, which is quite complex in userspace.

A hybrid of PTRACE_SYSCALL & PTRACE_SYSEMU

Now on to the concept behind ujail. The method I am describing works by calling PTRACE_SYSEMU for a specific process and this way taking over emulation of all system calls. However, some system calls are complex to emulate in userspace, and so a hybrid of both PTRACE_SYSEMU and PTRACE_SYSCALL is needed. In short this works by checking whether the syscall needs to be emulated when the PTRACE_SYSEMU event is received.
Now one way is emulating the syscall, filling the processes' registers and resuming execution of the process. This is simple and straight-forward.

The second way is forwarding the system call to the kernel. The problem here is that calling the syscall in the monitoring process will make the new resources available to that very process, and not the process to be jailed. This is where the hybrid method kicks in.

The proof of concept code creates a backup of the next instruction to be executed along with a copy of the instruction pointer at this point and patches it with the opcodes for "int $0x80", causing the syscall to be made again. After that it resumes execution with PTRACE_SYSCALL and waits again. The first event to be received now is the program leaving the emulated system call, which can be ignored. Resuming yet again will give use two PTRACE_SYSCALL events, one for syscall entry and one for syscall exit.

The first event is not really interesting, but at the second event the opcode backup is restored and the eip set from the saved value. Now the kernel has handled the syscall and the result is ready for the child process. A final call of PTRACE_SYSEMU resumes execution of the child and waits for the next syscall.

Proof of concept

The proof of concept code can be downloaded from its bazaar branch at launchpad.net. It is intended to be used on i386 systems only and works with simple programs, but is known not to work with anything using fork, vfork and most likely will not work for binaries using threading.

Finally, I would like to thank Pradeep Padala for his "Playing with ptrace" articles [0][1], which were fun to read and worked as a great introduction of ptrace for me.

Now there is only one thing left to say: if you are interested in this method, see loopholes or problems or want to contribute, please go ahead and contact me:

debian at sp dot or dot at whoa, interesting idea. Not for the feint of heart